A wave runs to data_at + declared_size, and there is seek magic exactly there
whose little-endian packet count at +12 times 2048 equals the declared size --
7620/7620 disc-wide. That is the decoder-independent boundary and the thing that
proves the declared sizes honest, so it should not be able to regress silently.
The boundary routinely lies outside the entry's own comp_size window, so reading
it needs the flat segment stream rather than the entry slice; PakArchive gains a
small data_at(offset, len) accessor for that.
Test walks a bounded slice to stay fast -- 792 banks in this run, all holding.
8 disc tests pass.
I read the seek chunk's packet count big-endian; it is little-endian at seek+12,
with size == 8 + 4*count. And a seek sits immediately AFTER its own data, so an
entry's first seek usually belongs to the PREVIOUS bank (implied start -25232
for D_452, -145988 for TCAF_608). I was comparing an entry's first seek against
its first data -- different waves by construction, which is why no reading lined
up.
With that fixed, the declared sizes are honest: every RIFF-bearing entry on the
disc has seek magic at exactly data_at + declared_size with count*2048 ==
declared. 7620/7620, zero failures.
VOICE_TCAF_608 is not truncated. Its Channels is 2 and I decoded it as mono;
read as stereo it gives 6520176 bytes = 33.96 s, agreeing with both length
signals in the bank (33.88 s from cumulative samples, 33.97 s from
PsuedoBytesPerSec). 170 of 8021 banks (2.12%) are stereo -- exactly the rate of
my 1-in-60 outlier.
This is the mono/stereo trap already documented on this very page, met from the
other direction: I had written 'at two channels every bank yields one frame' and
then spent several passes blaming missing data for a one-frame decode.
Code fix: to_xma_riffs built the leading segment with a hard-wired mono fmt. It
now reads Channels from the bank's first RIFF. 7 disc tests pass.
The ties needed a different signal, not a longer scan. Banks carry one: a seek
chunk sitting on a packet boundary, so seek_pos % 2048 IS the data offset. On
the 6033 labelled banks with a seek before their first RIFF, 6031 agree
(99.97%) -- better than the packet scan and structural rather than statistical,
so scan_data_offset now tries it first.
On the scan's 28 ties it resolves 26 correctly and 0 wrongly (2 have no usable
seek). Combined rule scores 7354/7358 = 99.95%, up from 99.62%. 762 of the 1495
RIFF-less banks carry a seek, so the signal exists where it is needed.
Also ruled out, since a wrong offset was this page's whole subject: the header
is not audio being discarded. Adding 0 to the candidate set, it wins 6 of 7358.
7 disc tests pass.
Two comments claimed it is 'honest per sub-wave'. Measured: 5296 of 7586 banks
declare more than the entry holds and none declares exactly what it holds, so
the existing .min(slb.len()) clamp is load-bearing rather than defensive.
Comment-only change; 7 disc tests still pass.
1495 banks carry no RIFF and take a separate path that hardcoded both the
offset and stereo. Across a random 48-bank sample there was NOT ONE where the
old stereo-at-1392 pair beat the best mono offset; median gain 184x, individual
banks going from 0-4816 decoded bytes to 180000-380000. Stereo shows the same
stop-after-one-frame signature already recorded for the leading segment.
With no RIFF the offset cannot be derived, so scan_data_offset picks among the
four disc offsets by XMA1 packet-header plausibility. Validated on the LABELLED
set -- all 7358 banks that do have a RIFF, where the answer is forced: 7330
correct (99.62%), and all 28 misses are ties on the top score, never a wrong
unique winner. Ties fall back to 1392.
The winning offsets also reproduce, by directory, the distribution measured
independently from the RIFF-bearing banks. jpn\etc splits 1468/1600, so path
alone is not sufficient -- which is why this is a scan and not a lookup table.
7 disc tests pass (build-reborn test -p sylpheed-formats --test
slb_leading_segment_disc, SYLPHEED_DISC wired up).
HEADERLESS_DATA_OFFSET is the value the offset takes in <lang>\etc\, not a
property of the format. The leading stream is a whole number of 2048-byte XMA1
packets ending at the first RIFF, so its start is first_riff % XMA1_PACKET.
Disc-wide that takes four values -- 1392, 1468, 1600, 1728 -- varying by
language and subdirectory.
Verified by decoding, not by arithmetic: on a random 140-bank sample with a
non-empty leading region, the derived offset yields more audio in 85, identical
in 54 (the eng\etc controls, where it must and does reproduce the old
behaviour) and less in 1. Median gain among the improved is 70x --
eng\Voice\VOICE_TCAF_592 goes 1506 -> 97152 bytes, jpn 2910 -> 127178.
This withdraws my own claim from earlier today that the Japanese banks were a
different undecoded layout. They are the same format with a different offset;
I had treated a constant derived from one subdirectory as a property of the
format. The same error was hiding the identical defect in 1873 eng\Voice banks.
ixud.rs now has an IdxdObject-shaped reader, IxudObject, and build_caption_text
reads captions as FIELDS instead of pairing them with whatever token follows in
the pool.
build_demo_text token adjacency 134 ids 537 lines
build_caption_text token adjacency 3721 8074
build_caption_text record fields 4085 8800 = all of them
Verified over the whole disc by tests/ixud_records_disc.rs: 1104/1104 objects
parse, 1476/1476 records and 628165/628165 named fields reproduce their
ixud_hash, 48 positional, zero failures. The header word at 0x08 is record 0's
hash, asserted per object -- there is no schema field, exactly as for IDXD. The
module doc described a 12-byte record directory and a "schema/type hash"; both
were wrong and are corrected.
I also have to correct my own number from the previous commit. "1.3% of the
game's text" counted OCCURRENCES: each family lives in 24-45 IXUD blocks and
the same key repeats across them. Distinct text-bearing MSG_* keys number 8800,
not 44579, and every one has the <id>_<page>_<line> shape. So the real coverage
was 537/8800 = 6.1%, and I overstated the gap about fivefold. Direction right,
magnitude wrong.
The DEMO control is the sharpest evidence for the change: token adjacency finds
537 lines there, the field reader 541. It was dropping lines even in the one
family it was written for -- which is why the test now asserts "must not lose
lines" rather than "must be identical".
Same lesson twice in one session: pool adjacency is a consequence of how
records are written, not a rule of the format.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
build_caption_text generalises the key parser from MSG_DEMO_* to all eight
families. The shapes are uniform and each family is 100% consistent with its
own: seven use MSG_<FAM>_<id>_<page>_<line>, and VOICE alone inserts a family
letter before the id.
ids lines
build_demo_text 134 537
build_caption_text 3721 8074
The DEMO family comes out identical through both readers -- 537 lines either
way -- which is the control that generalising changed nothing that already
worked. Pinned by tests/caption_families_disc.rs, along with VOICE ids keeping
their family letter.
But this does NOT close the gap, and the write-up says so: 8074 against the
44579 text-bearing fields the record-level scan counts is about 18%.
The reason is the same lesson this session already learned once.
build_caption_text pairs a value with the key that happens to follow it in the
raw UTF-16 token stream -- the adjacency heuristic that was wrong for IDXD and
is wrong here for the same reason. ixud.rs has no record/field reader at all.
The IXUD record table IS decoded and verified disc-wide (1104/1104 objects,
628165/628165 fields reproducing their key) and was simply never wired into
the crate.
Next step recorded: give ixud.rs an IdxdObject-shaped reader and read captions
as fields rather than adjacent tokens. The decode exists; only the plumbing is
missing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Measured first, over GP_MAIN_GAME_E.pak, comparing every named-field read the
six struct loaders performed against the record table: 4435 reads, 2872 agreed,
**966 returned None for a field that has a value**, **596 flattened a field that
several records carry**, 1 was wrong (a weapon whose TargetType is the empty
string read back as the neighbouring token "Skip"). The prior report of
4453/2887/974/591/1 is the same picture; the small differences are definitional
(I count a read as flattened only when the records disagree).
Every read now goes through IdxdObject::record, and the types say where a value
comes from:
* Weapon = the `Weapon` record (launcher) + the `Shell` record (projectile).
Both carry an ID and a Name and — with `ShellWake` — an `Interval`, which the
flat reader merged; they are separate fields now. Power/Velocity/ranges/
LifeTime are Shell fields, which is why 427 weapon reads used to miss.
* CraftUnit/Vessel = `Generic` (hull) + `Maneuver` (flight model) +
`StructureCount` (counts) + `Shield`, plus a new `hardpoints: Vec<Hardpoint>`
— one entry per Turret_/Bridge_/Thruster_/Hatch_/ShieldGenerator_ record, each
with its own HP. A flat HP could only ever be one of them.
* PlayerConfig = `Player`, plus `phases: Vec<PlayerPhase>` (SpaceSize/SupplyRange
are per Phase_N) and `score: ByDifficulty<ScoreRules>` (MainMissionBonus is per
Score_<difficulty>; the flat answer was the Easy one).
* Character faces come from the `Faces` record's field names (identical output to
the old token scrape, 0 of 68 objects differ — now by construction).
* Stage = `StageResource` + `phases: Vec<StagePhase>`, and the packages it names.
* The `fields: BTreeMap` on every struct became `records: RecordSet`, which keeps
the record boundary; `RecordSet::everywhere(field)` answers "which record".
The token-scraping loaders move too, and this is where the old reader was worst:
* Arsenal: options are the positional fields of the STANDARD_<slot> records. The
scrape returned 16 nose options of which 8 were field keys and pilot names, and
47 for arm3 of which 38 were junk, while missing Mine_B2A and No_Equipment.
Now 8/12/9/9, all real weapons.
* Squadron: one record per squadron, members are Count*4 positional slots
(unit, message set, n, pilot) — 1160 squadrons with ids and 2295 member tuples,
against 28 idless squadrons and 47 members before. Agrees exactly with the
independent Python decode in docs/re/structures/unit-group-table.md.
* DemoMessage: 11775 lines against 10263, every one with a speaker, a portrait,
a delivery mode and a voice token, from fixed positional slots.
* PilotRoster: assignments are the records the `UNITS` record names, so each one
now carries its unit id, its loadout and the player marker.
* UnitRoster: the roster is the field *names* of the single `EnumUnit` record.
* load_weapons selects on the records (Weapon + Shell) rather than on token[0],
whose first byte is often a stray pool byte ("#Weapon", "%Weapon"). Same 131
objects, no heuristic. GP_HANGAR_ARSENAL.pak holds none of them — the module
doc's claim that player weapons live there was wrong.
schema:: constants keep their names and values but are documented for what they
are: record 0's name hash (PLAYER = Difficulty_Easy, UNIT = Maneuver, VESSEL =
Bridge_000, MESSAGE = Message_000), not a schema id.
Two things the migration exposes and does not fix, flagged in the docs instead:
load_units' bucket is 43 Type=Craft + 46 Type=Vessel objects (new `unit_type`
field lets a caller separate them), and StructureCount.TurretCount is not the
number of Turret_* records (the player's craft says 4 and has 63).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
to_xma_riffs now emits the leading headerless segment when it sits at a whole
number of XMA1 packets and carries a non-zero byte. VOICE_D_453 goes from a
0.14 s trailing fragment to a 45116-byte leading sub-wave that dominates it.
I withdrew this exact change earlier for two reasons. Both are now answered
rather than argued away:
* "It recovers no audio" -- it used the STEREO format. At two channels every
bank yields exactly 1792 bytes, one frame, whatever its size. Mono yields up
to 113x more.
* "It matches 1524 of 8021 RIFF-bearing entries" -- the byte-level reach is
still 1524, but the audible reach is not. Across the 84 movie-bound banks
the segment adds >1 s to exactly 7, the hokyu_*_H tankers on D_453/D_454 --
precisely the broken ones -- and <=0.25 s to 66 of the rest. The largest
non-resupply addition is S04A at +0.66 s on a 256 s movie.
The safety oracle is recorded with its limits: 8 of the 84 banks ALREADY
exceed their movie's duration before the change, by hundredths of a second,
so it cannot resolve differences at that scale. It establishes scoping, not
correctness. Callers clamp to the movie length regardless.
VOICE_D_451's all-zero leading region is skipped by the non-zero guard, so
the rule cannot prepend silence to a bank that does not need it. Pinned, as
is the packet arithmetic (n = 8, 1, 7, 22, 29) which has no tunable.
slb_disc, movie_subtitle_disc and movie_manifest_disc all still pass.
NOT verified by ear -- that needs a human.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The structure is exact. In all five resupply banks the first RIFF sits at
HEADERLESS_DATA_OFFSET + n*2048, where 1392 is a constant this crate already
had and 2048 is the XMA1 packet size: n = 8, 1, 7, 22, 29. No free parameter
to tune, and the raw bytes agree -- high entropy from offset 0, then a zero
run immediately before the RIFF. VOICE_D_451 is the control, its single
packet being all zeros.
So I made the obvious fix, emitting that region as a sub-wave, and then
withdrew it on two measurements:
* It does not recover audio. Coverage went 5.4% -> 89.9% for VOICE_D_453, but
the emitted stream decodes through FFmpeg to 1792 PCM bytes -- silence --
while the RIFF sub-waves from the same banks decode to 150-270 KB. Byte
coverage was the wrong success metric and it looked like progress.
* It is not narrow. The rule matches 1524 of the 8021 RIFF-bearing entries in
sound.pak, including RT* movie banks that decode correctly today. Landing
it would have risked a wide regression in order to not-fix five banks.
to_xma_riffs is back to its previous behaviour, verified by re-measuring:
coverage is 5.4% / 9.7% again. The refuted attempt is recorded in the code
beside the branch it would have changed, so the next person does not
re-derive the arithmetic and re-make the change.
XMA1_PACKET is kept as a named constant because the blast-radius scan uses
it. Artifacts: examples/voice_bank_shape.rs (structure), voice_bank_dump.rs
(sub-waves for decoding), slb_hybrid_scan.rs (the 1524 count).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
movie_manifest::parse now reads BASE_INFO's positional field keys (the game's
own cutscene ids, stage*100 + slot) and follows each to its record, instead of
scraping the string pool. The pool stores each distinct string once, so a
REPEAT reference produced no token and read as "no binding".
That single cause explains every wrong cell: 13 later references to
VOICE_D_450..454, two to SUBTITLE_hokyu_LS_s11A.tbl, and MS01A's share of
pwterop_s01a.prt. All 18 hokyu movies are bound, not five.
Counts, verified independently by me against the disc before recording:
104 cutscene SLOTS binding 101 distinct MOVIES; 99 slots / 96 movies with a
voice track, 99 / 96 with a subtitle, 22 / 22 with a telop. The docs' old
94 / 83 / 21 are exactly the counts of DISTINCT POOL STRINGS -- not wrong
measurements, measurements of the wrong thing. Three denominators were being
conflated; the new test pins all three.
Two assertions in movie_manifest_disc.rs were false and are corrected:
hokyu_DS_s13A binds VOICE_D_452 and resolves to eng\etc\VOICE_D_452.slb. The
in-game verdict that rejected that value tested an INFERENCE from a shared
demo id, on a decoder that discards 85-87% of banks in this class -- see
voice-bank-leading-region.md, committed earlier today.
The ~104 script ids are no longer open: they are literal positional keys,
each naming its record, and all 104 resolve. The old "counts differ by three,
positional pairing does not work" has a concrete cause -- three resupply
movies are bound by TWO slots each.
Also corrected: the naming convention has 3 subtitle exceptions (s24A/s27A
borrow s11A's track) and 18 voice exceptions, not one and five.
The legacy scraper is kept as a fallback for blobs with no record table, so
the synthetic unit fixtures still exercise it.
Artifacts: examples/movie_map_csv.rs regenerates the CSV, now slot-keyed
(104 rows; the movie-keyed version silently dropped one slot of each
duplicate). Disc tests green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The binary region in front of the string pool was the parser's oldest open
note ("Not yet decoded"). It is a uniform 16-byte record array sorted by
name hash, a field count, a 12-byte field array sorted by key, a pool size,
and the pool. The trailing `pool_size == file_len - pool_base` identity makes
the layout self-checking, which is what caught the first wrong version.
Verified over the WHOLE disc with zero failures: 7750/7750 IDXD objects,
190782/190782 records reproducing their stored tag_hash, 1271462/1271462
named fields reproducing their key. IXUD is the same container with
ixud_hash, UTF-16BE and every offset in chars — 1104/1104 objects,
628165/628165 fields, checked with an independent parser.
Field names are stored on disc, so no preimage search is needed: a field's
middle word points at its own name. Only 504 fields disc-wide are hash-keyed
with no name; the other 1485073 nameless fields are positional, keyed by a
literal integer (line slots, movie ids).
Two long-held beliefs are WITHDRAWN:
* The word at 0x08 is not a schema hash. It is record 0's name_hash — the
format has no type field at all, and an object's kind is known only from
the caller that loads it. It survived as "schema" because tables of one
kind share their lowest-hashed record name. Caught by a test asserting
every movie id names a real record: 1005 -> STAGE10_PHASE01 failed because
tag_hash("STAGE10_PHASE01") IS 0x067025B9, that table's supposed schema id.
* The field's middle word is not an always-0xFFFFFFFF flags word. It is
0xFFFFFFFF for 54% of fields, enough to look constant in a small sample;
the tell was that it is constant per key ACROSS records, which a per-record
flag cannot be but a per-name pointer must.
`schema_hash` keeps its name rather than churn 33 call sites, with corrected
docs. The first sweep globbed dat/** and missed hidden/DefTables.pak (1425
objects); the test now walks the whole disc root.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Found the routines in the disassembly DB rather than guessing from data:
sub_82447DF0 IDXD tag hash (lbz+extsb, modulus 0x00FFFFDF, magic 0x2101)
sub_82447E70 IXUD tag hash (lhz, 64-bit, modulus 0xFFFFFF67 then 0x00FFFFDF)
Both transcribed instruction-for-instruction into Python and Rust.
IXUD SOLVED. It defeated every single-modulus search because it chains TWO
exact moduli -- the loop reduces mod 2^32-153 in 64-bit arithmetic and only the
result is folded mod 2^24-33. A polynomial mod M1 folded through M2 is not a
polynomial mod anything, which is exactly why the gcd test returned 1. Verified
independently: 86/86 record keys and 108,261/108,261 field tags in
GP_MAIN_GAME_E.pak, and NoRecord -> 0x1c6d9c96.
CORRECTION 1: tag_hash must SIGN-EXTEND each byte (extsb). My reconstruction
used unsigned bytes and matched all 1.27M disc names -- every one is ASCII --
while disagreeing on ~90% of random inputs with a byte >= 0x80 (verified:
18096/20000). The disc could never have caught this; only the disassembly did.
CORRECTION 2: name_hash's reduction is EXACT, not lossy. The module doc claimed
the missing conditional subtract made it something other than %. rlwinm r6,r6,
9,23,31 is just hi>>23, and with RECIP = floor(2^55/M)+1 that is Granlund-
Montgomery magic division -- 0 wrong at every quotient boundary across the full
32-bit domain. Retracted.
cargo test -p sylpheed-formats --lib hash: 10/10.
Closes the 4-byte record key. tag_hash is name_hash's shape -- byte-sum
checksum in the top byte over a 24-bit modular polynomial -- with two different
constants: modulus 0x00FFFFDF (2^24-33, prime) instead of 0x00FFF9D7, and no
lowercasing, so tags are case-sensitive. name_hash explains 0 of 8643.
Recovered from the tables rather than the executable: every inline field name
is a known (name -> tag) pair, and comparing names differing in one character
gives the per-position weights 1, 0x100, 0x10000, 0x21, 0x2100, ... -- a byte
leaving bit 24 re-enters as 33, i.e. reduction mod 2^24-33. Holds where it is
easy to get wrong (distance 8 and 9 carry correctly).
A record's key is the tag of its own name: FormationSet rosters 362/362,
UnitGroup rosters 281/281, S02 squadron names 111/111 -- so records can be
addressed by name without reading the roster first.
Implemented in Python (unitgroup.tag_hash) and Rust
(sylpheed_formats::hash::tag_hash) with 3 new unit tests carrying disc-derived
vectors; cargo test -p sylpheed-formats --lib hash is 8/8 green.
Not settled: the guest routine is unlocated, so this uses exact modular
arithmetic where the game may use a Barrett step without final fixup.
The three orders the derived rule was built from all live in GP_TITLE.pak, so
they cannot confirm it - the rule was fitted to them. These two are from
GP_SAVE_LOAD.pak, read off the running game now that the Canary threading fix
makes the main menu dependable.
The 9-element slot-list header composites EXACTLY as the sort predicts, on all 6
instances of it, and nothing about this screen was fed into the rule:
measured 7 8 0 1 2 3 4 5 6
derived 7 8 0 1 2 3 4 5 6
including TWO tied groups (0xb102 x2 and 0xb210 x5) that both come out in
declaration order, and the unkeyed pfeff00.prm fade quad last.
The 13-element save/load frame differs in exactly the two open questions and no
new ones: two unkeyed pfbase.tbm backgrounds paint FIRST where the sort puts the
keyless last - the splash's palogo_eff0.prm behaviour in a different file type,
so implied_layer_key now covers it - and the 0xb100 group of four paints
10,11,8,12 where declaration order is 8,10,11,12.
That second point is a SECOND screen with a mis-ordered tie, which is what the
question needed, and it immediately kills a candidate: 10 and 11 are kind=0x2002
while 8 and 12 are 0x0000, so "descending kind then declaration index"
reproduces 10,11,8,12 exactly - and then fails both title groups, where every
element of 0x8083 is kind 0 and where 0x80a0 would predict 2,3,4,5,0,1,7 against
a measured 0,2,4,7,1,3,5. Seven candidates refuted now.
16 disc tests green.
Two places checked for the key a .prm element sorts by, both empty:
* the declaration entry's four unread words are constant across every element
of all three measured screens (+28=0, +36=0xffffffff, +56=0, and +44 is a
button ordinal 1-5, 0xffffffff elsewhere);
* the bundle carries NO data for a primitive at all - the menu build declares
pteff00.prm, pteff02.prm and pteff05.t32 and has zero RATC children for any
of them, its 34 children being 21 T8aD sprites and 13 .rat records.
So the layer comes from the game code. But it is consistent, which is what makes
a per-name table a measurement rather than a fudge. Bracketing each unkeyed
element between the keys of its measured neighbours: pteff02.prm falls in
(0x8010, 0x8040) on BOTH screens it appears on, pteff00.prm is past the maximum
on both, and palogo_eff0.prm is below the minimum on the splash.
implied_layer_key records exactly those and nothing else; an unlisted primitive
keeps u32::MAX and still sorts last. With it, derived_paint_order produces the
same layer-key sequence as the order read off the running game on all three
measured screens - primitives included - and matches element-for-element on four
of the five bundle instances. The fifth is the title, differing only inside its
tied groups, which is a separate open question.
This does not make include_primitives safe by default: the 36 builds that come
out one colour are wiped by pzeff00.prm and pceff00.prm, never measured, hence
not in the table.
14 disc tests green.
Read off the running game with screen_children.py and identified by pivot
signature as GP_TITLE.pak ratc-index 8, the NEW GAME / LOAD GAME / TUTORIAL /
OPTIONS / EXTRAS screen:
paint order: 1 3 4 2 5 8 9 6 7 15 10 11 12 13 14 0
It is the first measured screen carrying TWO primitives, and they land in
different places, which is the point. pteff02.prm (the 25% dim) paints 4th,
beneath the whole UI; pteff00.prm (the transition fade, resting transparent)
paints last. Both match their positions on the title screen exactly. So a
primitive's place is per-element and stable by role - backdrop first, dim at
slot 4, fade last - and there are now three permutations to test a derivation
against rather than two.
Wired into measured_paint_order, keyed by element names so both language builds
get it. The English build composited with --primitives edge-correlates at 0.9591
at shift (0,0) against a framebuffer capture taken in the same session - a third
screen confirming paint order, resting pose, fade alpha and primitives at once,
against a capture this project had not seen before.
13 disc tests green.
fill_quad composites an untextured primitive as a solid rectangle of the
keyframes fade colour, pivot x 2 in size, placed and scaled exactly as a sprite
is. Behind ComposeOptions::include_primitives and screen render --primitives.
On the title screen, whose paint order is ground truth, it is measurably right:
mean luminance 76.30 -> 63.72 against the captures 64.58, i.e. from +18% to
-1.3%, and mean absolute difference 16.07 -> 13.08. The background was ~40% too
bright; pteff02.prm, a 25% black dim, was what was missing. The wordmark is not
dimmed by it because the measured order paints that quad at slot 4, beneath the
logo. Edge correlation moves 0.9538 -> 0.9480, which is not informative here: a
uniform dim scales gradients uniformly so a normalised edge score barely sees it.
OFF BY DEFAULT, and that is the finding. A primitive has no T8aD header, so no
layer key, and derived_paint_order sorts the keyless to the end. GP_DIALOGs
pzeff00.prm is a single keyframe of opaque black at full screen; painted last it
wipes the build. Of the 125 builds that draw a visible primitive, 36 come out
>99% one colour with the derived order.
No constant default works, because the two screens read off the running game
disagree: the splash paints its primitive FIRST (the black backdrop) while the
title paints one at slot 4 and another LAST (the fade-out). Declaration order
fails the title too. A disc test measures the damage rather than asserting the
feature works, so the number moves when the ordering is solved.
Also records a false alarm worth keeping: a first pass reported 36 GP_DIALOG
builds at "100% black", which was a crude near-black pixel threshold and not a
black screen - those dialogs are dimmed 50% and perfectly legible. The genuinely
wiped builds are a different set.
blit modulated by tint only, which is 0xffffffff on essentially every keyframe,
so the fade word was decoded, stored and then discarded. Applying it as an ARGB
modulate takes the title composite's edge correlation against the framebuffer
capture of the running game from 0.4597 to 0.9538, both at zero shift. The white
wordmark with its blue outline, the trademark, the copyright and the orange
exploding planet all appear -- the planet because a full-screen blue effect that
rests at alpha 0 had been painting over it at full opacity.
ARGB is measured: across a fade-in the high byte walks 0x00 -> 0x80 -> 0xc0 ->
0xe0 -> 0xff while the low three stay ffffff, and the low 24 bits are 0xffffff on
5276 of the disc's 5453 resting keyframes.
A modulate can only remove pixels, so the risk is a blank screen. Measured: no-op
on 4060 of 5200 sprite elements, partial on 453, hides 687 -- transient HUD
indicators that should not be lit at rest -- and NO build is left with nothing
visible. A disc test asserts that, and that the no-op share stays high, which
also guards the resting rule against drifting onto ramp frames.
Both changes are in one commit because the second is invisible without the first.
Applying fade erased the word PAUSE, which the running-game capture plainly
shows: pgptitle.rat has three runs of two identical keyframes -- pre-roll, hold,
exit -- and the 'later run wins' tie-break grabbed the exit. A group carries the
screen's entry animation AND its exit, so a run ending on the last keyframe is
now excluded unless it is the only one. The title correlation is unchanged.
Element::rest() picked the keyframe with the largest gap to the next keyframe's
time. That reads a keyframe as a value held until the next one; it is the start
of a ramp toward it. A long gap after keyframe k means the screen spends that
time arriving at k+1, so the settled pose is at the far end of the gap.
The title wordmark zooms in over five frames and holds at (184,193) at 100% from
t=251 to t=264. The old rule picked the frame before the long gap: (179,186) at
101%, still mid-zoom.
Measured against the framebuffer capture of the running title screen, which is a
1:1 crop so frame coordinates map directly (confirmed: the copyright line lands
on row 669 in the capture and in both composites). Edge-correlated over the
wordmark box:
plateau (landed) best 0.4597 at shift (0,0)
longest dwell (old) best 0.1511 at shift (+3,+8), 0.1268 at (0,0)
The old composite scores 3x lower and only peaks after being moved, by about the
(-5,-7) that picking kf4 instead of kf5 predicts.
It also fixes six title elements the old rule rested at alpha 0x00 where the
capture plainly shows them, and pteff00.prm - the full-screen fade quad painted
last - which rested at opaque black. That was the blocker on .prm compositing.
Adds tools/re-capture/align_to_capture.py, which is how this was scored, and
turns the .prm test that deliberately asserted the old defect into a guard on
the fix.
Not settled and now the next item: compose ignores the keyframe fade alpha
entirely (blit modulates by tint only), which is why choosing the wrong keyframe
was invisible until now.
The developer-logo splash declares its sprites directly and has no .rat layout
child, so is_build rejected it and no screen command could render it — despite
it being one of only two screens whose paint order has been read off the running
game, and the one where the layer key explains the whole permutation.
Adds ui_layout::is_composable (a declaration table plus at least one element
resolving to a T8aD the bundle carries) and an opt-in --all on screen
list/info/render. Measured on the disc: 2859 RATC bundles, 965 pass is_build,
2751 pass is_composable, and 0 pass is_build without passing it — a strict
superset. It is opt-in because the 1786 extra bundles are mostly two-element
fragments (a button and its glow), and because widening the default would
renumber --build for every pak, invalidating the build indices the corpus's
notes cite by number.
The splash now renders 6/7 elements, painting its glows first in the order
measured off the game; a disc test pins that order.
`compose` skips focused-state records by default, so whatever that flag matches
vanishes from every composite. It matched a trailing `f` in the name and nothing
else, which is not the convention — it is a letter.
Measured across the disc: 2458 elements match the suffix and only 54 of them
have the base element they would be the focused version of; all 54 are
`pgmenu_btnNNf.t32`. The other 2404, spread over 864 bundles, are `_eff` glow
layers whose names end in the same letter — `pb_name_eff.t32` alone accounts for
1122 of them, and `palogo_gamearts_eff.t32` is one the draw capture shows the
game painting, before its logo, with its own layer key.
Requiring the pair recovers 587 glows in the composable builds. GP_OPTIONS is
the plainest case: without `pbmwindow_eff.t32` the panel had no window at all,
just two floating brackets (captures/ui-layout/options-with{out,}-its-glow-layer.png).
Two other candidate rules were tried and refuted first. The `opt` link is not a
focus link: 221 elements are opt-targets, only 2 of them suffix-match, and the
targets include `pjnet_bg.rat` and `pv_loading_loop1.rat`. And the suffix alone,
as shown above, is unrelated to anything.
compose now sorts elements by the word at +0x08 of their sprite's T8aD header
instead of painting in declaration order, for every build except the two whose
measured order is hard-coded. That word is non-decreasing in the order the game
actually paints both measured screens, so every screen nobody has captured now
gets its layering from the file rather than from the declaration table, which is
provably not the paint order.
Verified with artifacts and both ways, not by a green build: the disc test
asserts the measured orders never invert the key and that the composite's key
sequence is sorted, and reading the word from +0x0c instead makes it fail; the
title composites identically; and GP_MISSION_SELECT — uncaptured — now composites
cleanly, committed as a capture.
Two things recorded rather than smoothed over: ties keep declaration order
because the game breaks them some other way that is not known, and the
developer-logo splash has no .rat child, so is_build rejects it and the
compositor never sees that bundle at all — its measured order is inert in
practice and screen render cannot draw it.
Checking the blast radius of the previous commit's `kind = 0x4` skip found 174
elements on the disc that are 0x4 with no non-0x4 element of the same sprite —
GP_READY_ROOM pak entry 75 is 56 elements and every one is 0x4, a list of real
icons a blanket skip would erase. So the rule is narrowed: skip an instance only
when its template is present, which is the case the title capture covers. The
title render is byte-identical before and after, so the narrowing changes nothing
the evidence covers.
And the claim that this "caught a regression" is WITHDRAWN, because measuring it
refuted it: none of those 174 elements is in a bundle `is_build` accepts, so none
of them ever reaches the compositor. The blanket skip would have been harmless in
practice. The narrow rule is a precaution, not a fix, and the comment says so.
The test that was going to assert the regression could not find such a build —
correctly, since none is composable. It is replaced by one that pins the fact
which makes the narrow rule safe: no composable build on the disc has a 0x4
element without its template, checked across 500+ builds. If that stops being
true it fails, instead of a screen quietly going empty.
The compositor painted in declaration order, which the draw capture proved wrong
— the title's background is declared ninth and paints first. The order the game
uses is its runtime child list, and no decoded field reproduces it, so rather
than leave the screen composited wrongly this lands the orders that HAVE been
read off the running game and falls back to declaration order everywhere else.
Two builds are covered: GP_TITLE's title build and the GAME ARTS / SETA / studio
anima splash. Keyed by element names, which identify a build across paks and
language variants.
Rendering it exposed a second defect, and the same capture settles it: the
kind = 0x4 elements are motion-trail ghosts, not resting content. The bundle
declares three instances of each wordmark; the capture shows exactly ONE quad at
each wordmark's position. Drawing them at their resting keyframe put three
oversized PROJECT SYLPHEED copies across the composite. They are now skipped.
Verified with an artifact, not a green build: the composite is committed
(captures/title-composited-measured-order.png) and now reads as the title screen
— background, planet, ship, wordmark, TM, copyright, correctly layered.
The test is disc-gated and was checked BOTH ways: it passes as landed, and
disabling the order table makes it fail. It reads one pak rather than every
build on the disc — the first version used the all-builds helper and got the
test process OOM-killed running alongside the other three.
`static_assembly_matches_runtime_capture` walks the capture's parts and looks
each up in ours, so an extra static placement was originally invisible to it —
that is how `e303_wep_01` sat here decoded 100x too large. `64d372c` closed that
with a set-of-names check; BACKLOG.md still says otherwise, and it is wrong.
One direction stayed open, though: a set cannot fail on a resource placed
*twice* when the capture lists it once. That is not hypothetical — a duplicated
instance is what a bad node walk emits, and the two legitimate duplicates here
(`e106_eng_01`, `e303_wep_01`) are why the test had to special-case counts at
all. So pin the whole multiset — 9 resources, 11 placements, against the
capture's 8 dedup'd parts — which subsumes both hand-written counts and fails on
an extra, a missing, or a duplicated placement.
Refuted before believing: perturbing the expected table (declaring
`e106_bdy_01` twice) fails with the real multiset on the left, so the assertion
runs against the disc rather than skipping for want of SYLPHEED_ISO.
The remaining `for want in &cap.parts` loop guards the table itself: editing it
cannot quietly drop a part the runtime actually draws.
The title screen needs an order that puts element 13 (`ptbase2.t32`, the
full-screen background) behind elements 0-5 (the wordmarks) — the capture shows
the wordmarks on top, so the declaration table is not it. Two other orderings
the bundle carries were the cheap candidates, and both are now dead:
* the **placement region** stores a keyframe group per element with an explicit
element index, so it could be a second ordering. It is not — it equals the
declaration order on every build on the disc. `UiBuild::placement_order`
exposes it and `placement_region_order_is_never_a_second_ordering` pins it, so
the refutation stays checkable instead of remembered.
* the **RATC child order** is the declaration order with the `.prm` elements
absent — strictly less information, and no place to put the background other
than where the table already puts it.
`screen info --geometry` prints both, plus each element's decoded sprite size
beside `pivot*2` and every keyframe's scale/position/time — the numbers a
placement hypothesis has to be tested against, and how the pivot/scale rule in
the previous commit was found.
`title_background_is_full_screen` pins that rule against the disc rather than a
synthetic sprite. `scaled_elements_are_a_small_and_mostly_undiscriminating_minority`
reports the scope honestly: 865 of 5 130 resting placements are scaled at all,
and only 213 of those could tell "about the pivot" from "about the sprite
centre" — which the capture did *not* settle, because `ptbase2`'s pivot is its
centre. It also counts how far `pivot*2` is from the decoded size disc-wide
(2 521 agree, 1 884 are off by more than 16 px), which demotes the "pivot is
exactly half the texture" result to a property of the tutorial bundle.
The compositor read a keyframe as `top-left = (X,Y)`, `size = decoded · scale`
and ignored the declared pivot. That is right at 100 %, which is every element
the format was ever checked against — the pause menu, the ARSENAL chip ruler —
and wrong for every element that is scaled.
Measured against a framebuffer capture of Canary on the title screen.
`GP_TITLE.pak` build 7 element 13 is `ptbase2.t32`: 640x360, pivot (320,180),
one keyframe at (320,180) with scale 200 %. From the corner that is a 1280x720
rect at 320..1600 x 180..900 — a quarter-screen slab with the top-left quadrant
bare. Anchored at the pivot it is (0,0)..(1280,720), and the capture shows the
background art reaching all four edges. Normalised cross-correlation of the
composite against the capture, searched over +-40 px, peaks at (0,0): 0.90 on
the planet limb, 0.72 on the lower-left ship.
`ptcopyright.t32` calibrates the other half: unscaled, 694x20 at (293,655), and
the capture's glyph run is x 295..986 / y 700..718 once the 45 px of window
chrome is taken off. So the keyframe really is the top-left at 1:1.
Disc-wide this moves 865 of 5 130 resting placements. The pause menu's own
`pgpeff01` glow stops hanging off the menu frame to the bottom-right and
centres on it.
`ComposeOptions::backdrop` comes with it: the default dim slate stands in for
the PRMD dim-quad behind an in-mission screen, but comparing against a
framebuffer needs the black the game actually composites over, so
`screen render --black` can ask for it.
Two independent lines landed a `.rat` reading and neither was the whole
picture, so this merges them into one module and fixes what the merge exposed.
ui_layout — the screen is the BUNDLE, not the set of .rat records
------------------------------------------------------------------
`feat/ui-layout-preview` parsed `.rat` records; the autopilot stack documented
the RATC header and probed it in `examples/screen_layout.rs` but never landed a
library module. The `.rat`-only reading structurally cannot see an element that
has no record -- the `eff*` frame corners, the `deli*` dividers, `msg` -- which
is exactly what the committed real-vs-rebuilt capture shows missing. Rebuilt
around the header:
* element declaration table at 0x20 (60-byte entries: name, parent index at
+32, kind flags, pivot) = the back-to-front draw list;
* the placement region after it = per-element keyframe groups.
Verified against the disc, each against a fact the docs state independently:
`pgpeff02a` -> parent 3 = `pgpeff02`; `pgp_ttrl_btn10` rests at (546,288); the
pause buttons sit at 268/337/407/478, the documented 70 px pitch; the Arsenal
carries X = -516. The tutorial PAUSE menu now composites 11/11 elements and
matches the real screen more closely than the earlier rebuild did.
Three defects found while validating, none of which any test would have caught:
* the keyframe block is 40 bytes with X/Y/time at +28/+32/+36 and an
alpha-ramping ARGB at +0 -- the fade, previously unread;
* a group's data stops 4 bytes short of its last block's time slot, so that
word is the NEXT group's element index. Reading it produced times like
1869640736 and silently corrupted the max-dwell pick. Last-frame time is
now `None`;
* the `.rat` sprite-name field is not 16 bytes. Capping it there truncated
`pgp_ttrl_title.t32` to `pgp_ttrl_title.t`, which resolved against nothing
and dropped 4 of 11 tutorial elements from the composite.
Max-dwell also needed a tie-break: on equal gaps take the LATER frame, or
`pgpmsg` reports the y=645 fly-through instead of the y=605 it settles at.
savegame -- a Rust port of tools/re-capture/savegame.py
------------------------------------------------------
GDHA container, zlib payload, chunk stream (GDAA / phase / GHAD 122 B / 16x20 B
SHAB / trailer). Every GHAD word carries its own confidence rather than the
block being presented as solved: 6 named, 2 recorded as REFUTED (+36, +56 were
tested as difficulty and as stage and are neither), 7 still unknown.
Tested against the three real saves committed under docs/re/captures -- no disc
and no emulator needed. The load-bearing assertion is the byte-identical
round-trip; the develop differential is asserted as a property (spending 4000 P
moves +24 and not its twin +28, steps the clear ratio, and moves exactly two
blob entries), and the header summary is checked to agree with the payload it
mirrors -- the trap that makes the Details panel a bad oracle.
CLI: `screen list|info|render` and `save info`, so both are checkable headlessly
in the same spirit as `mesh render`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
With per-sub-mesh declarations enabled, n201_01 decoded as a 2-part fragment 4 bytes
off. Both starts validate for the pivot — 0x32BA718 at pad 2 (earlier in file order,
so first-match took it) and the capture-proven 0x32BA71C at pad 0 — so the pivot
alone cannot separate them; at the early one two of four sub-meshes fall out as
out-of-range.
anchor_grouped_meshes now builds each accepted candidate and keeps the one that
explains the most of the declared pool: it returns immediately when a candidate
explains all n sub-meshes, else keeps the best partial, so it can never decode less
than first-match did. n201_01 lands on all four capture-proven offsets
(0x32BA71C / 0x32BEFF4 / 0x32C416C / 0x32C536C) and its two sibling copies take their
own pools, so the twin collapse is gone.
XBG7_SUBMESH_DECLS is therefore on by default (=0 reverts):
resources that never decode 85 -> 47
resources decoding in no container 63 -> 30
degenerate index runs 1 -> 1 (unchanged)
cross-container minority decodes 96 -> 96 (unchanged)
captured index runs, stage-02 93/93 (unchanged)
captured index runs, stage-05 124/128 -> 128/128
The last line is the point: the buffers the capture could not name are the n201
family, and they now decode and match the GPU's indices byte for byte. Suite green
including twin_pairs_do_not_share_a_buffer, apart from the pre-existing
known-failing cross-container consistency test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
desc_dump shows each index marker is followed by its OWN element triples: n201_01
declares strides 24, 24, 24 and 28 (the last sub-mesh has a fourth element), which
matches the runtime capture's stride=28 on that draw and the four distinct vertex
shaders. parse_vertex_decl read the first declaration for the whole pool.
all_vertex_decls reads one per marker; anchor_grouped_meshes uses each sub-mesh's
own stride for the pool walk, the pivot validation and the read. debug_grouped_report
follows the same setting so the diagnostic cannot blame the wrong gate — at n201_01's
capture-proven pool start it now reports "pad 0: ACCEPTED" instead of a NaN position.
With XBG7_SUBMESH_DECLS=1: resources that never decode 85 -> 47, resources decoding
in no container 63 -> 30, capture oracles unchanged (93/93 index runs, 42/42 index
counts), consistency unchanged at 96.
Off by default because selection has not caught up: the three n201_0x copies then
settle on one pool (twin_pairs_do_not_share_a_buffer fails), production still picks
a start 4 bytes before sub-mesh #1 rather than the proven one even though the proven
start validates and is unclaimed, and four newly decoded ptc_pack .dat composites
carry degenerate triangles. Format truth is settled; choosing among candidates is
the remaining work.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
Two follow-ups to the pad-scoring fix, both driven by the same invariant (a
correctly located index run has no degenerate triangles):
- anchor_grouped_meshes picked its pad by first-match too; scoring the pivot run
the same way cleared every remaining ptc_pack composite (f102/f104/e107).
- anchor_pool_mesh now prefers a degenerate-free candidate over an earlier dirty
one. examples/better_home.rs showed the last two resources each had exactly one
degenerate-free, pool-covering block, sitting later in file order than the
lookalike we took. First-match order is kept for every clean hit, and a dirty
block is still used if nothing clean exists, so coverage cannot regress.
degenerate index runs, disc-wide: 582 -> 11 -> 1
captured index runs identical: 93/93 (unchanged)
resources decoded / misses: 6 209 / 85 (unchanged)
index runs changed / anchors moved: 590 / 10 (_rou_f402_dead x8, e201_bdy_03_m x2)
Cross-container minority decodes 89 -> 96, and that is progress: all seven new
rows are _rou_f402_dead, which now has a majority (32x25x8) for the first time, so
its seven wrong copies are named instead of hidden behind "no majority".
The last dirty run (_rou_f402_dead in Stage_S09) is blocked by distinct assignment
— its clean block is claimed by e_rou_f003_Near, both 24-vertex bounding boxes. A
winding-floor escalation for that case was written, measured to fire for nothing,
and reverted; the reasoning is kept as a comment.
Regression threshold tightened to 1. Suite green with --include-ignored apart from
the pre-existing known-failing cross-container consistency test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
Extending the capture comparison from index COUNTS to index VALUES
(`examples/capture_index_bytes.rs`, using the batch offsets the new ib logging
gives) showed 76 of 93 Stage_S02 index runs identical to the GPU's and 17
differing — every difference a shift by exactly one element, on buffers whose
index data sits at pad 2.
`anchor_pool_mesh` returned the FIRST pad that validated, and pad 0 is tried
first with the looser winding gate (0.70 vs 0.85). Read at pad 0, a pad-2 block
yields [true[1], true[2], …, garbage]: every index in range, the pool covered,
the positions right, the winding often just above 0.70 — so it validated, and
every triangle was mis-wired. Nothing count-based could see it.
The signature is decidable without the capture: a shifted run wires arbitrary
vertices, so triangles come out degenerate. 282 of 283 correctly anchored
Stage_S02 blocks have zero degenerate triangles, while the shifted readings carry
1–2 156. So score every validating pad by (degenerate triangles, then winding)
and keep the best. `XBG7_PAD_FIRST_MATCH=1` restores the old behaviour.
captured index runs identical: 76/93 -> 93/93 (2 025 elements)
decoded runs with a degenerate triangle: 579 -> 16 (disc-wide)
sub-meshes whose index run changed: 575 of 8 850
resources decoded / vertex anchors / consistency: unchanged (6 209 / same vb / 89)
Locked in by tests/mesh_disc.rs::decoded_index_runs_have_almost_no_degenerate_triangles.
Suite green with --include-ignored apart from the pre-existing known-failing
cross-container consistency test (the 24-vertex bounding-box class).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
The decoder's central unstated assumption — a block's index buffer sits
immediately before its vertex buffer (`vb - idx_count*2 - pad`, pad <= 3) — was
also the prime suspect for the residual anchor misses, since a capture-proven
`e106_eng_02_l` block was rejected outright. Measured it instead of assuming:
- extended the F10 ship capture to log each draw's index buffer (base, count,
min/max index) and to key its de-dup on the index range, so every draw batch
is recorded rather than only the first;
- `examples/capture_ib_truth.rs` places each drawn buffer in the container by its
dumped positions and scores the capture against our decode.
Stage_S02, 42 drawn buffers placed: our idx_count == the sum of the draw's index
batches for 42/42, the batch union covers the vertex pool exactly for 42/42, and
all 30 single-block cases sit at pad <= 3 (20 at pad 0, 10 at pad 2). The other
12 are grouped pools, where one index pool serves the whole group. So the layout
holds, the decoded index count is exact, and eng_02_l died on the connectivity
gate (since replaced by the winding gate) — not on index location. The shipped
exact-coverage rule is independently confirmed.
The recorded "capture indices=21 vs our 246" disagreement was an artefact of the
old de-dup key: 21 was the first of two batches, 21 + 225 = 246. Any conclusion
from a pre-2026-08-13 capture's `indices=` or `vbase - ibase` is about one batch,
not about the block.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
The offset->field map read out of sub_82341A20 is now usable rather than just
documented: data/unit_definition_layout.txt carries the 159 fields with their
provenance and the two conventions (angles are degrees on disc and radians in the
object; a defaulted field keeps the accessor's 0.0 miss value), and
sylpheed_formats::unit_layout exposes fields()/field_at()/offset_of() so a memory
snapshot can be read by name.
tests/unit_layout_disc.rs replays the verification against the checked-in live
dump -- every mapped float of all 11 identified objects must equal its disc value,
angles compared in radians -- asserting 0 disagreements and >=400 agreements. It
needs no emulator. Full suite green: 11 binaries.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Filtering after the assignment made every subset query a full decode (~15s on a
50MB container). full_decode_cached memoises it per container -- fingerprint is
length plus three sampled 4KB windows, keyed with min_consistency, last four kept.
Decoding five ships from Stage_S02 in turn: 10.5s for the first, then 48us-1.4ms.
A stage now costs one decode rather than one per ship. Ten suites green, viewer
builds, and a spot-checked resource still lands on the same offset.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
models_named pruned to the wanted set BEFORE distinct assignment, so collision
resolution saw a different resource population and returned different offsets:
27 of 356 resources in Stage_S02 decoded differently when asked for alone,
including real geometry (f001_bdy_30, f106_sld_02_l/m/d, f101_wep_01_l). Both the
viewer and assemble_ship decode subsets, so both could disagree with the
container's own answer. This was a regression from distinct assignment itself.
Fixed by filtering the OUTPUT: the assignment always runs over the whole
container. One-name, three-name and full decodes now agree exactly. Cost: a
single-resource query on a 50MB container goes from near-instant to ~15s;
per-container caching is the follow-up. Ten suites green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
XBG7_MONOTONE=1 enforces descriptor order per (stride, vtx, idx) signature: 89
minority decodes, unchanged. Monotonicity constrains order WITHIN a container
while the disagreement is BETWEEN containers, which hold different numbers of
these boxes in different arrangements. The remaining 89 are bounding-box identity
ambiguities needing descriptor-level information, not another anchoring
heuristic. Knob kept, default off, measurement recorded; suites green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rendering assembled e106 from Stage_S02 showed the old slab: e106_bdy_03 spanning
600x1600x998 where three other containers give 276x236x941. Its anchor had slack
3, and the coverage gate tolerated up to three unreferenced tail vertices --
tolerance that was hiding a mis-anchor, since real blocks reach their last vertex
exactly (8580 of 8629). Requiring exact coverage moves it to the block the other
containers agree on and the slab disappears.
Costs 3 resources (6212 -> 6209), inconsistency 39 -> 38, capture oracle
unchanged at 46/46, suite green. Evidence: captures/e106-cover-slack-before-after.png
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sweeping XBG7_GROUPED_CONSISTENCY 0.85 -> 0.80 -> 0.75 decodes no additional
resources and leaves inconsistency at 39, while changing which geometry some
grouped models get -- strictly worse, so 0.85 stays. With the degeneracy/extent
fix already refuted, the remaining 82 misses are not a threshold away; they need
a structural answer (marker lists that do not map onto stored blocks), which
means a capture, not more tuning. Records the state of the four shipped changes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Under the winding gate, coverage is 6212/6294 (98.7%) with 82 misses left,
attributed 42 degenerate/extent, 31 winding, 9 coverage, 0 connectivity. The
biggest bucket turns out NOT to be the blocker: replacing the absolute area test
with a scale-free collinearity test decodes no more resources and raises
inconsistency 39 -> 44, and dropping the extent floor to 0.05 adds two. Both stay
as opt-in knobs (XBG7_REL_DEGEN, XBG7_MIN_EXTENT) rather than defaults. Also
fixed another stale default label in edge_cap_sweep.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Gate attribution put connectivity behind 153 of 225 misses -- the largest
blocker, and the gate already known to reject a capture-proven block. Testing it
against winding consistency (an objective topology signal: ~1.0 or ~0.0 for a
real mesh, ~0.5 for a mis-carve) shows a floor of 0.70 with connectivity inert
dominates the previous default on both axes: 6212 resources decoded (+143) with
39 shared inconsistencies (-17), capture oracle unchanged at 46/46, twin
invariant clean, coverage invariant intact. Suite green.
The 0.80 cliff (5770 resources, 0 inconsistent) is documented, not taken --
consistency is the weaker witness, and both points are one env var apart.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The last two blocks that under-covered their vertex pool were f102_break.dat and
f104_break.dat in ptc_pack.xpr, each reading a neighbouring block's index buffer
against the wrong declaration (414 verts indexed to 404; 160 indexed to 79).
Their marker lists do not map onto the stored blocks -- only 2 of 9 and 4 of 10
sub-meshes decoded at all. Requiring coverage (max_idx + 4 >= vtx_count) for
pre-pivot sub-meshes drops exactly the mismatched pieces.
Every decoded sub-mesh disc-wide now covers its pool: 8580 at slack 0, 49 within
tolerance, none beyond, none negative. Coverage 6069/6294, inconsistency 56,
truth table 46/46 -- all unchanged. Suite green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
coverage_audit measures index coverage per decoded sub-mesh. 8586 of them
reference their last vertex exactly, so the 'buffer not covered' gate is well
founded -- but 18 had NEGATIVE slack: indices up to 364 vertices past the end of
their own buffer, emitted because anchor_grouped_meshes reads pre-pivot
sub-meshes unconditionally. Quality gates stay relaxed there (a tiny flat lead
part is legitimately poor) but index range is now required.
Coverage 6069/6294 unchanged, inconsistency 56 unchanged, truth table still 46/46
claimed, suite green; the vertex total drops by exactly the 1546 garbage verts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
undecoded.rs supplies the denominator the coverage numbers never had; the disc
holds 6294 XBG7 resources, 6069 decode, 225 are searched and missed, 0 lack a
descriptor. gate_histogram.rs attributes each miss to the furthest gate its best
candidate reached: 120 connectivity, 74 grouped-pool (different path), 15
degenerate/extent, 9 winding, 7 buffer-not-covered. Recorded as a work-list, not
a verdict -- a wrong candidate can pass more gates than the true block.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
debug_grouped_report (why_rejected) shows n206_02's alternative pool is ACCEPTED
at pad 0 under production gates and is in the candidate list -- and the decoder
does take it: n206_02 now anchors at 0x342d984. The 'still collapsed' reading
came from the audit comparing decoded geometry, and that offset holds a direct
(unmirrored) copy, so a separated pair still looked identical. The audit now
requires a SHARED BUFFER to call it a collapse: disc-wide 18 exact mirrors, 16
related, 0 collapses, 0 unrelated -- and the regression test drops its exception.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
anchor_grouped_meshes takes the same taken set; a colliding grouped model is
re-placed whole past everything claimed, or keeps what it had. Runtime oracle
unchanged (45/46 exact, 0 unclaimed), coverage unchanged (6069), cross-container
inconsistency 62 -> 56, suite green.
It does not clear the n206 twin collapse: no alternative pool validates for the
loser, so that pair is a validator case (like eng_02_l before the cap move), not
a selection one. Also fixes a stale '0.28 (default)' label in edge_cap_sweep.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
