Spent a session getting to flight. Each obstacle presents as "the emulator
died" and none of them is.
* --audio prevents boot. run-canary's header already says the flag is not a
cvar in this tree and that an unknown argument blocks in a message box
before logging starts. Measured anyway, because the corpus also holds runs
that passed it and booted: 3 trials each in BOTH orders, 67 565 bytes of log
without the flag and 209 with -- and 209 is run-canary's own banner, not one
line from xenia. Order was reversed on purpose; this corpus has a standing
lesson that an A/B from run order is noise. Eight scripts on branch
auto/idxd-unnamed-keys still pass it; main and this branch are clean, which
reconciles August's successes with today's failures.
* launch_mission.sh's skip_intro deadlocks. It calls the attract loop a
"movie" and refuses to tap, and waited out 600 s of unbroken movie verdicts
before timing out. nav_to_flight.sh, against the same running emulator,
reached the main menu in 12 s and flight in 2 min 20 s by tapping A at the
title. The "wait it out" premise is wrong: the loop does not end.
* "EMULATOR GONE at ~40 s" is this project's own Stop hook killing xenia when
a Claude turn ends. That is recorded further down this same file and I
rediscovered it over three boots because I did not look. Sequential tool
calls within one turn are fine; ending the turn is what kills it.
The world unit is still unmeasured. Flight was reached and the screen
classifier agrees, but entities2.py finds 0 unit definitions -- its committed VA
window does not match this run, the same run-dependent-address problem this file
documents for the OB counter. Next attempt must hunt the range.
An audit of BACKLOG.md turned up a class of error with a single root cause: the
README defines only the CONFIRMED/PROBABLE/HYPOTHESIS confidence scale, while
the pages actually use a second vocabulary -- and 🔴 appears 98 times without
ever being defined. It gets used for two different things, "refuted" and
"blocked", and three entries slid from one into the other.
README now defines ✅/🟡/❔/🔴/❌/🚧 and states the rule the corpus was missing:
🔴 never means "we have not run it yet". That is ❔ or 🚧. Its blocked sense is
only for a real limit of the box -- no push credentials, no hardware Vulkan, a
decision only the user can make -- and since the box can run the emulator,
script input, screenshot and read guest memory, "needs a run" is never blocked.
I made exactly this mistake on the world-unit item earlier today, which is what
prompted looking for others.
Fixed in BACKLOG.md:
* the elimination test, marked 🔴 UNRUN and in fact run and refuted nine
lines further down;
* the frozen capture, marked 🔴 STILL UNRUN and in fact taken eleven lines
down -- 🔴 wrong twice, since "the freeze did not happen this run" is a
scheduling outcome and not a refutation;
* a 🚧 STILL UNRUN item whose stated blocker (the boot-nav bug) is fixed;
* the objective-counter heading, which asserts 0xbdb59668 as the answer while
its own first body line refutes that address -- retitled to say what is
actually solved, the method;
* the paint-order "third measured permutation" question, answered inside its
own entry by a third, fourth and fifth screen;
* the UTF-16 endianness question -- resolved, and it is not a stale comment:
localization.rs both documents LE and decodes with u16::from_le_bytes, so
it is a code bug worth filing.
Also fixes the corpus's only dangling link (INDEX.md pointed at
structures/idxd-unnamed-keys.md, never written).
I wrote "blocked on the oracle" for the unit-to-metre conversion. That was a
mislabel: red is for what the container cannot do, and run-canary works here.
What the km-name sweep actually established is narrower -- no STATIC test can
settle it, because the disc has exactly one size-bearing asset name.
The run is well-supported by tooling that already exists: findplayer.py
recovers the player position triple from motion, the HUD prints the distance to
the selected target in the game own units, and the same separation read both
ways is the conversion. Recorded as amber with the experiment written out.
The census filtered pak entries whose own first four bytes are T8aD. A sprite
is usually a child of a RATC bundle, and a bundle entry's magic is RATC, so a
top-level magic filter cannot see one:
top-level T8aD entries (counted) 4 525 sprites, 45 keys
T8aD inside RATC bundles (missed) 16 659 sprites, 204 keys
both 21 184 sprites, 216 keys
171 of the 216 keys exist only inside bundles. The sharpest statement of the
error: that census never saw GP_TITLE.pak at all -- the pak holding both of the
screens this page's entire evidence comes from.
Retracted: "45 values", "the keys are pak-local", "each auxiliary pak occupies
its own narrow high-byte band". On the full population 68/216 keys (31%, not
9%) cross a pak family and the per-pak ranges overlap heavily -- GP_BUNK
0x8000-0xa110, GP_TITLE 0x8000-0xc150, GP_LEADERBOARD 0x8000-0xf100. The tidy
banding was an artifact of seeing one or two keys per pak. So the key looks
like a shared vocabulary, which is the opposite of what I published.
Survives, now on the full population: the field is a u16 at +0x0A (upper half
zero 21 184/21 184), and it is an enumeration (216 values for 21 184 sprites).
Three wrong numbers on this page now, all the same shape -- a statistic computed
over a population I had not checked was the population in question. Stated once
at the end of the section rather than three times: check the sampling frame
before the statistic.
The census filters pak entries whose own first four bytes are T8aD. The
sprites this page measures paint order on are children of a RATC bundle --
ui_layout.rs reaches them via ratc::parse, and a bundle entry's magic is RATC,
so a child T8aD never matches a top-level magic filter.
So the 45 keys may describe a population that only partly overlaps, or does not
overlap at all with, the one the page's two measured screens come from. I do
not yet know which; the comparison is running. Marking the section rather than
leaving the counts to be read as covering the screens' sprites.
Same failure shape as the 37/45 language-duplication note lower down the page:
a number computed over a population I had not checked was the population in
question. Recording it as such.
The page rested on twelve values from two screens. This walks all 4525 sprites
on the disc.
* The field is a u16 at +0x0A. The upper half of the 32-bit word the page
reads is zero in 4525/4525. Nothing above changes -- 0x00008100 sorts the
same as 0x8100 -- but a future value with the high half set would mean
something had been misread rather than that the layer got deeper.
* It is an enumeration: 45 values for 4525 sprites, one of which (0x8100)
covers 1188 of them.
* The reading worth trying -- a global layer vocabulary shared across the UI
-- is refuted. Only 4 of 45 keys cross a pak family and 33 of 45 live only
in GP_MAIN_GAME_2D; every other pak owns a narrow high-byte band (0x90-0x94
for the in-game overlays, 0xa4 mission log, 0xb1-0xb2 save/load). A screen
that owns one or two keys is not ordering itself with them.
That supports "group id in the high bits, order in the low bits", which is what
the page already suspected, but it does NOT test it: paint order has been
measured on two screens and both are inside GP_MAIN_GAME_2D, so there is no
ground truth to check the split against. Left amber.
The first number I got was 37/45 shared, which would have supported precisely
the wrong conclusion. It came from counting paks instead of pak families: the
six GP_MAIN_GAME_*2D paks are the same screens in six languages and their key
sets are byte-for-byte identical. Recorded on the page, because the shape
recurs -- a corpus with near-duplicate members manufactures agreement.
Two follow-ups on yesterday's^Wthis morning's CollisionSet write-up.
1. The _cmesh <-> render-model link, which I recorded as UNTESTED because
matching stems against .xbg object names covered 4 of 158. The disc keeps
only one build manifest, so that corpus was never going to answer it. The
right corpus is the GameResourceID field of the DefTables / GP_MAIN_GAME
records -- 480 distinct values. Against those, with a control that shuffles
the characters of each stem:
ship/mob stems prefixed by a real resource id 108/112 = 96.4%
same stems, characters shuffled (control) 0/112 = 0.0%
asteroid stems prefixed (expected none) 0/46
So a CollisionSet entry is <GameResourceID>[_<part>]_cmesh. The 0/46 on
asteroids matters as much as the 108/112: a test that fired on everything
would be the bound-check hazard again.
2. The world unit. Sweeping every pak for a name carrying a kilometre figure
returns mapmesh_box_500km.col/.rgn and nothing else -- 162 references, all to
that one pair. The reading rests on a single filename with no corroborating
instance anywhere in the data, so no static test can settle it; marking it
blocked on the oracle rather than leaving it as an open static question.
My objection's premise did survive: rou_e010 is a real GameResourceID and
e010_ADAN_Attacker_S is in the stage tables, so the 133-unit mesh does belong
to a craft the game calls an attacker. Whether the trailing _S means "small"
is a further guess (there are _EX4 / _HF / _HF_Wayne variants), so it stays
suggestive rather than evidence.
All 18 blobs are byte-identical: the per-stage naming is nominal, and every
stage points at one shared 1675148-byte library stored eighteen times. That
identical size was the reason to open the item, and it turned out to be the
answer to it.
Record layout: {u32 size, u32 name_len, char name[name_len], u32 nv, u32 nt,
f32[3] x nv, u32[3] x nt}, next record at off + 8 + size. The indices are u32
here where MCOL uses u16 -- two different serialisers in one archive.
What makes this a decode rather than a plausible reading: the walk consumes the
file to the byte over 158 variable-length records, with the size word predicted
from the two counts 158/158. A wrong field would desynchronise within a few
records and could not land exactly on the end. All indices in range 158/158;
98.24% of edges shared by exactly two triangles; 147/158 fully manifold.
158 meshes, 90 836 triangles: per-part ship proxies (_bdy/_brg/_eng/_wep/_sld,
the XBG7 sub-part vocabulary) plus 46 stage asteroid meshes whose prefixes are
exactly the stages that have an _AsteroidVolume_wp MCOL.
Two things this file makes me walk back:
* The "1 unit = 1 metre" reading from mapmesh_box_500km is downgraded to
amber. The 500000 arithmetic stands, but it implies that a craft the game's
own tables call "small" is 133 m and that rob_f002 is 447 km -- 89% of the
arena width. The format check survives; the interpretation has no
independent support.
* The _cmesh <-> render-model name link is recorded as UNTESTED, not
confirmed: only one .xbg build manifest survives on the disc, so matching
stems against object names covers 4 of 158, which is no coverage at all.
The names live outside MiscBin: they are the MapPath / MapMesh /
CollisionMeshes field values of the per-stage StageResource object (IDXD schema
3c9ae32e, in every GP_MAIN_GAME_<lang>.pak), and each hashes with the ordinary
pak name_hash straight to a TOC entry. 40/40 resolve, no collisions -- the 11
REGN as <stem>.rgn, the 11 MCOL as <stem>.col, and the 18 remaining blobs as
CollisionSet_S01..S16 / _Tutorial / _test.bin. The .pe string table at 651540
was the way in: MapMesh and MapPath sit adjacent there.
This upgrades the pairing claim. The first section of mcol-collision.md could
only say REGN and MCOL had matching *distributions* of bbox and cell size, and
flagged that as not an object-to-object link. A phase record names one .rgn and
one .col, and all 11/11 pairs share a stem and agree exactly on both.
The names also check the format work from outside it: mapmesh_box_500km.col is
the object decoded here as 8 vertices and 12 triangles spanning exactly
+-250000, and its name says that cube is 500 km across -- so one world unit is
one metre, and a wrong stride could not have produced a box that measures what
its own filename claims. 70 of the 87 phases use it: most stages' only
collision is the arena wall, and _AsteroidVolume_ names the rest.
Still open: the 18 CollisionSet_*.bin are named but not decoded (all exactly
1675148 bytes), and CMapColliderBridge in the RTTI names the runtime consumer
without following it into the code.
The 0x50 header word, which the first section of this page had dismissed as "a
large value", is two u16 counts: vertices and triangles. They give the two
remaining blocks their stride, and every derived length is exact in 11/11 --
len(0x54) == align16(12*nv), len(0x58) == align16(6*nt), and nt equals the
bounding-sphere count decoded last iteration.
Checks that cannot pass by accident:
* sphere i is the TIGHT bounding sphere of triangle i, 4768/4768, with
max|v-c|/r median 0.99990 (a fixed 1.0001 epsilon), against a 1.32%
random-triangle control;
* the mesh is watertight -- every edge shared by exactly two triangles,
7152/7152, zero degenerate triangles, zero unreferenced vertices;
* the two smallest objects are 8 vertices and 12 triangles whose positions
are the eight +-250000 corners of the map bbox: a bare bounding cube.
The cell lists are a correct broad phase: with an exact triangle/box SAT test
only 3 overlapping triangles in 18 577 entries are absent, so a query walking
one cell's list cannot miss a hit. The 730 conservative extras bracket the
builder's own test between exact-SAT and AABB, which retires the 18 unexplained
"sphere misses" from the previous commit as that same margin.
mcol_probe.py gains `mesh` and `obj`; `verify` now runs all three checks and its
output is recorded in docs/re/data/mcol-verify.txt.
The unexplained ~0.75 ratio left at the end of the last iteration was my own
stride. I had read the block as 12-byte points because REGN's vertex section
is 12 bytes, and never checked it: len(0x5C) is not a multiple of 12 in 5 of
the 11 objects, so that stride was never arithmetically possible.
At stride 16 the relation is exact in 11/11 -- max u16 == len(0x5C)/16 - 1 --
and the record reads as {centre f32[3], radius f32}. Powered test, since a
u16 is reached through a specific grid cell: the sphere it names reaches that
cell in 18 559/18 577 = 99.90%, against a 12.02% random-sphere control. Both
fields carry signal (centre alone 26.75%, radius shuffled 70.19%).
The converse -- is the list *exactly* the intersecting set? -- is 0.38%, which
is the expected direction: a bounding sphere is conservative, so membership
implies overlap but not the reverse. The tighter geometry is in 0x54/0x58,
still undecoded. 18 entries (0.10%) go the wrong way and are recorded as open.
tools/re-capture/regn_decode.py is copied unchanged from auto/regn-reader so
the probe's POF0 reader is the known-good one rather than a second copy.
Two tests. Counts modulo 3 are spread across all three residues (639/2063/1786),
so the u16 array is not a triangle list. And a B record is reached through a
specific cell, so a point it references should lie in that cell -- referenced
points score 0.79% against a 0.48% random-point control. Chance.
The contrast is the point. One section earlier the same u16 entries scored
18379/18379 (100%) on 'are these valid point indices'. I flagged that at the
time as the weak bound-check and recorded it as consistent rather than as a
finding. The caution was right: the powered version of the same question now
returns chance, and had the 100% been written up as the decode this page would
carry a confident false statement about MCOL's geometry.
Fifth appearance of the pattern across REGN and MCOL and the first time both
halves have been run on the same field, so the page now states it plainly: a
bound-check asks whether something could be an index, and the answer is set by
the size of the target collection rather than by the field's meaning.
Datum for the next attempt: the maximum u16 is consistently about 0.75x the
point count (923/1232, 1019/1360, 1163/1552, 59/80), too consistent to be
coincidence and not explained.
Two corrections got there. MCOL has 9020 relocated words and only 4488 are
A-record pointers; I assumed the rest sat at B+8 mirroring A, refuted 0/4488.
Measuring their offset from the nearest preceding B record gives B+4 for 4488
(99.0%) and 44 before the first B -- exactly the four header pointers times 11
objects. Nothing unaccounted for.
So B is {u32 count, pointer at +4}. The pointers advance by exactly twice the
count: 4477/4477 (100.00%) over all 11 objects. That is a packed u16 array with
no padding, and it is the load-bearing evidence -- an exact arithmetic identity
over 4477 consecutive pairs. The companion check that those u16s are valid point
indices is the same weak bound-check flagged earlier and is recorded as
consistent rather than as evidence.
Chain: position -> cell -> A {cell index, count 1, ->B, sphere} -> B {count n,
->u16[n]} -> n indices into the point block. Same shape as REGN's cell -> item
-> refs -> geometry.
Open: the 0x54 and 0x58 blocks, which this chain never reaches, and what the
indexed points form.
Filtering array A by the cell-index criteria and following each pointer: every A
record points at a distinct B record, 11/11, and every A count field is exactly
1, 11/11. Per object the A-record count is the number of occupied cells (110 to
575) and the counts sum to it exactly.
That is the same design REGN uses -- the corpus already records 'every occupied
cell has count exactly 1' there. Two sibling formats, one convention, and a
further independent confirmation of the A reading since the filter and the
cardinality are unrelated criteria.
Array B resisted, and both attempts failed in ways worth recording. The record
boundary was off by 8 again, producing records that start with the tail of the
previous structure -- the same mistake as the 0x74 check two iterations ago. And
the u16-index test had no power: the 0x5C block holds ~1232 points, so 'is this
u16 below the point count' passes for almost any small value, and duly reported
100% at seven offsets. Fourth time in these two formats that a bound-check
against a large collection produced a meaningless 100%.
Recorded what would have power instead: B records are 1:1 with occupied cells
and A carries that cell's bounding sphere, so a B field can be tested for
spatial consistency with that specific cell.
Separating the two interleaved arrays by address and re-running the same three
criteria: array A (2509 records) gives byte3==1 at 100.00%, a valid cell index
at 100.00%, and the sphere reaching that cell at 99.92%. Array B (6467) gives
30.65% and 30.60% -- a different record type, and the control showing A's 100%
is not what any 32-byte block would score.
So array A is the per-cell record: cell index (x,y,z), count, pointer into array
B, bounding sphere -- the same role REGN's section 3 plays. That confirms the
earlier 50% was the interleaving artifact and not a half-working reading.
The split was crude, first-half-by-address giving 2509 vs 6467 rather than an
even cut, and A still came out at 100%. A rough partition isolating a perfect
population is stronger than a careful one isolating a good-ish population.
Array B's layout is still unread.
Read as 8 big-endian words, word 0 as four bytes is (x,y,z,1) -- a 3-D cell
index, matching the object's 5x5x5 grid. Words 4-6 are a position and word 7 a
positive scalar (a bounding sphere); word 1 is a count, word 2 the relocated
pointer. Every record pointer lands in the same region, 8976/8976, each a fixed
distance on with the same stride -- so there are two parallel arrays, A and B.
The 50% is the tell. Three independent criteria -- byte 3 == 1, valid cell
index, sphere reaches that cell -- all land on 50.0%, which says half the
records are not this type rather than that the reading half-works. The POF0 slot
list interleaves both arrays and I was testing B's records against A's layout.
So array A is a per-cell record, the same role REGN's section 3 plays. Array B
is unread, and the criteria have not yet been re-run on A alone -- if the
reading is right they should go to 100%.
Used the known-good decoder rather than my own broken one. My version had three
errors: the delta stream starts at table+16 not +8, the tag bits are
0x40/0x80/0xC0 rather than 0/1/2, and slots carry the +16 fixup base. Sanity
check passes -- on REGN the tool returns header slots 0x70-0x84 exactly.
On MCOL, over all 11: the header-region relocated slots are exactly 0x54, 0x58,
0x5C and 0x74 (11/11); 0x5C resolves to 0x80, the first byte after the header
(11/11); and 0x74 resolves to eight bytes before the first array pointer (10/11).
So MCOL has four top-level pointers where REGN has six.
92.7% of gaps between consecutive relocated words are 32 bytes, in 7-127
contiguous runs per object, which with the 0x74 offset reads as arrays of
32-byte records each carrying a pointer at +8.
The four targets are two float blocks, a block of small ints / u16 pairs, and
counts followed by the record array -- the shape of a mesh, stated as a reading
of the shape since none of the blocks is decoded.
Open: the record layout, what the index block indexes, the one object where 0x74
does not land 8 before the array, and whether the 7.3% non-32 gaps are just run
boundaries.
MCOL sits beside REGN in hidden/MiscBin.pak, 11 of each, never decoded. Over all
11: POF0 at data_size+16 11/11, bbox pad words 1.0/1.0/0.0 11/11, and
extent == max-min 11/11. So the header prefix is the same shape as REGN's and
the POF0 mechanism applies, which means the chunk+0x10 base and the loader's own
pointer list -- the two things that cracked REGN -- are available here too.
The map parameters are not merely similar but identical in distribution: bboxes
2/6/3 at 250k/50k/25k and the 0x40 triple 2 at 50000 and 9 at 10000, matching
REGN exactly. Eleven maps, each with an MCOL and a REGN over the same volume at
the same cell size. Noted that this matches distributions, not a demonstrated
object-to-object pairing.
Everything past 0x40 diverges from REGN and is open. Also recorded that my own
POF0 delta decoder is wrong here -- eight leading zero deltas -- and that the
working one is regn_decode.py on auto/regn-reader, which should be used rather
than re-derived. Re-deriving it is the mistake I made.
The original reason for investigating REGN was that a mission's enemy count
rises and falls, so a scheduler with parameters must exist somewhere, and a
per-map uniform grid is what such a thing would be indexed by.
Now that it is decoded that reasoning is answered: REGN is a tetrahedral
navigation mesh -- vertices, faces carrying plane equations and adjacency,
tetrahedra with portal costs between face pairs, and a grid indexing which tets
fall in each cell. Every section is accounted for by that structure, and there is
no time field, no unit reference and no trigger anywhere in it.
So the wave-scheduler search should treat REGN as excluded rather than unread.
The page's original hedge was right to keep the reading provisional, but the
reasoning it hedged was a guess from shape, and the shape belonged to
pathfinding -- which is what pointed the whole investigation here.
The arrival timetable in Route_S<NN>.tbl, keyframed per squadron per phase with
t in seconds, remains the only located part of the mechanism.
Re-derived the other branch's central check with my own code, my own reading of
the record and my own control, rather than accepting the number.
At base chunk+0x10, with the plane at intra-record +0 and three u16 vertex
indices at +32: every one of 133573 faces has all three named vertices
satisfying its own plane equation. Random-vertex control 2782/400719 = 0.69%.
100% against 0.69% is not a fit.
That also settles the record boundary from my side: the u16s describing a plane
sit after it in the same 48-byte record, so my earlier 'four zeros at the start
of each record' was those integers seen 16 bytes out of position, one record
late.
Two implementations, two independent guesses at the intra-record layout, the
same 100%.
I recorded the static coupling search as exhausted and needing PE code. The PE
work was done on auto/regn-reader and it solved the whole thing: REGN is a
tetrahedral navigation mesh, reached via the POF0 fixup table -- the loader's own
list of which words are pointers, so nothing needed guessing. Six sections;
position -> cell -> 32-byte item -> tet refs -> tetrahedron, with section 2 a
face carrying a plane plus its 3 vertices and the two tets either side.
Controlled checks: face through 3 of 4 tet vertices 253722/253722 against a
0.07-2.2% random control; portal cost equals face-centroid distance
380460/380460.
Also records against myself that the base is chunk+0x10 and my offsets here were
16 bytes early, that the plane arithmetic survives only because those fields
landed on the same bytes, and that my points-in-bbox count was never evidence --
a shift inside a homogeneous f32 array yields other floats from the same array.
Leaving the wrong conclusion in the backlog would have told the next reader the
avenue was closed when it was the one that worked.
The other branch supplied concrete offsets: for 3506e972 its face record 0 and
its plane normal both begin at 0x1c700, and my chunk + offset_at_0x78 + 16 gives
0x1c700. Same bytes, different bookkeeping -- so the n.p+d result stands
unchanged and was never in dispute.
The base is chunk+0x10, on evidence with power: the loader does addi r3,r31,16;
at +0x10 the six POF0-relocated slots land exactly on 0x70-0x84, the six section
pointers, whereas at +0 they would relocate the u16 counts and leave two section
pointers unrelocated, which is non-functional; and section-0 record 0 reads as a
bbox corner at +0x10 and garbage at +0.
So my '13467/13467 points inside the bbox' was vacuous. Only 11 of 13467 read as
denormal at the wrong base -- the rest were still plausible coordinates, because
a 16-byte shift inside a packed array of f32 triples yields other floats from
the same array. Recorded the general form: a containment test cannot detect a
shift inside a homogeneous array, because the shifted values come from the same
distribution. For that class of error it is not a weak check, it is no check.
'Section 0 is a point list' happens to be right; the evidence I gave for it was
not evidence.
That branch decodes REGN as a tetrahedral navmesh with strong checks and claims
the POF0 fixup base is chunk+0x10, so every offset on this page was read 16
bytes early.
I could not reproduce that on the one independently checkable thing here: the
plane list gives 133573/133573 unit normals at the unshifted base and 0/133573
at +16, and n.p+d = 0 holds to float round-off unshifted. A 16-byte shift
destroys it, so the blanket statement does not hold for this record.
Likely reconciliation is bookkeeping: a 48-byte face whose plane fields sit at a
different intra-record offset addresses the same bytes from a different origin.
That is a guess and I am not adopting either wording until checked.
Also confirmed: my own section-0 point test passes at 100% at BOTH bases, so it
never had power to distinguish them and should not be cited as validating the
offsets.
Both preceding sections split the file at 0x88 and called everything after it
the slot table. Wrong: this page already documents the container as GDHA + a
146-byte header + a zlib stream, and savegame.rs implements it. 0x88 is where
the Z1/zlib payload begins (5a31 = 'Z1', 78da = zlib), so the bytes I treated as
slot fields are deflate output.
Invalidated: the '6 pointer-shaped words in the slot region' are not words and
not pointers, so the false-positive rate I derived from them measured nothing.
Survives with different reasoning: 'the slot region is byte-identical' is true
because the two COMPRESSED streams are identical, which does imply identical
payloads -- and inflating both confirms it, 545 bytes byte-for-byte equal.
Stands and is better founded: all 12 differing words are in the header, which is
the only uncompressed region and therefore the only place a word-wise diff means
anything.
Done correctly, the result is a confirmation rather than a discovery: inflating
all three payloads reproduces the documented clear-ratio field stepping 5 -> 6
on the developed save.
The lesson: I found Z1/78da by inspection and nearly wrote it up as new. It was
already decoded four sections above where I was appending. Reading the whole
page first would have saved two wrong commits, and the wrongness was not
cosmetic -- a pointer census ran over deflate output and produced a
plausible-looking table.
Splitting the same comparison at the documented slot-table boundary (save+136):
the header holds 34 words and ALL 12 differing words, while the slot table holds
35 words and ZERO differences. The slot region is byte-identical between the two
same-state saves, so the documented slot fields are untouched by the churn and a
slot-region diff is meaningful where a whole-file diff is not.
Also corrects my own framing from the previous commit. The slot region contains
6 words in the pointer value range that do NOT change across runs -- a pointer
captured from a moving heap would not sit still, so those are data whose values
merely land in 0x70-0x8F / 0xB0-0xBF. The value-range test alone does not
identify a pointer; a third of what it flags here is not one. What identifies a
pointer is being pointer-shaped AND varying, ideally with a shared constant
delta. So '26% of the file is heap addresses' overstates it and the defensible
figure is the 8 header words that are both.
Left the earlier section in place with the correction after it: a value-range
classifier on 32-bit words always has a false-positive rate, and quoting its raw
count as a fact about the format is the error.
Comparing the two same-state saves (both 276 bytes), only 12 of 69 u32 words
differ, and nine of those carry guest addresses -- 0xBC/0xBD/0x70 prefixes, the
same regions the runtime work uses. Three differ by exactly 0x101080 and two by
exactly 0x300000: a shared constant offset is what a relocated heap does to a
pointer and not what data does.
The split is sharp: pointer-shaped words are 26% of the file and 44% of them
differ; non-pointer words are 8%. So a quarter of this structure is captured
heap addresses that change run to run regardless of play. That is the concrete
form of the page's existing 'much of the rest is uninitialised memory', now
demonstrated by constant deltas rather than inferred from odd-looking values.
Also records that my first pass was wrong: a byte-wise three-way diff reported
135 of 276 bytes differing, but game03 is 280 bytes with a ZERO-byte common
suffix, so offsets do not correspond past 0x00A. The valid equal-length pair
gives 30 bytes in 12 words. The inflated figure looked plausible -- half a save
changing is what dense state would do -- which is why it is written down.
The previous test only tried section-1 targets. Closing that gap: the payload's
three index-shaped u32s, followed into the point list and the plane list and
checked for the target lying inside the referencing cell, all sit at the 0.203%
random control.
Two cells read 0.81%, 4x the baseline. I am not treating that as a lead: across
this and the previous iteration roughly twenty such tests have been run, and at
that count a single 4x enrichment on ~8000 trials is what noise looks like.
Calling it a signal would be the multiple-comparisons error a long hypothesis
sweep invites.
So REGN's header, grid, points, planes and cell index are decoded, section 1's
slot regions are censused, and the link between the grid and the geometry is not
reachable by any static test I can construct. The honest next step is the PE
code that reads a REGN object -- the same kind of work that cracked the .slb
packing phase -- rather than a twenty-first correlation.
The natural coupling in a file with a uniform grid and a list of small volumes
is that the grid indexes the volumes. Tested by spatial agreement, it does not.
Test 1: every u16 in a cell's 32-byte payload, tried as a section-1 index and
checked for its centre lying inside the referencing cell. Every field sits at
the 0.138% random-control rate.
Test 2: every float triple in the payload, checked for lying inside its own
cell. 0.15-0.81%, also chance.
Recorded a worthless number from the same run rather than dropping it: those
triples lie inside the object's BOUNDING BOX in 100.00% at five different
offsets. The bbox spans the whole 500 km map so any mid-range triple passes, and
overlapping windows at +0 and +4 both scoring 100% is the tell -- a real field
would not survive a four-byte shift. Third time in this investigation that a
containment test against something large has produced a meaningless 100%.
Incidental and real: u32 slots at +0, +8 and +12 are below 0x10000 in 100% of
payload records while +4/+16/+20/+24 are in 11% and +28 never, so the record has
three index-shaped fields and four wide ones.
Last iteration I offered 'position + scalar + integer links is the shape of a
BVH node' as a reading of the shape. Tested properly it fails.
Following every u16 half of every integer slot and checking child-sphere-inside-
parent-sphere gives 0.00% for every candidate -- but the informative number is
the RANDOM control, also 0.00%. No node's sphere contains any other node's
sphere anywhere in the file, so there is no nesting for an index to point at and
the hypothesis dies before the indices matter.
The reason is scale: slot 7 has a median of 3139 against a median inter-node
distance of 45457, 14x smaller, and a random other centre falls within it 0.40%
of the time. It is also smaller than the smallest grid cell on any map.
So slot 7 is a LOCAL scale, not a hierarchy radius. 63410 scattered centres each
with a sub-cell extent is the shape of many small independent volumes, which
would fit per-object collision hulls for asteroids and debris -- a reading, not
a measurement.
What this removes is a wrong frame: the file is not a tree, so tree-shaped tests
will keep returning nothing.
96 bytes is 24 slots. Over all 63410 records: slots 4-6 hold values in the
header bbox range (a position), slot 7 is always positive 519..107600 (a radius
or extent), slots 8-11 are DENORMAL as floats -- 1.4e-45 upward -- so they are
integers a float reader would turn into near-zero garbage, and slots 12-23 are
six pairs with distinct even/odd distributions. Slots 2 and 3 are ~always zero.
Position + positive scalar + integer links is the shape of a BVH node, which
would fit a file carrying a point list and a plane list. That is a reading of
the shape and nothing more.
Recorded a failed test and why it failed: splitting the integer slots into u16
halves and checking them against each section's record count accepts ALL THREE
sections at ~100% for slots 8 and 9. A test that accepts every hypothesis
rejects none -- section 2 has tens of thousands of records, so the check
measures the section's size rather than the field's meaning. Slot 11's halves
are consecutive in 54%, which is suggestive and not a rule.
What would settle it is a test with power: follow a candidate index and check
the target is spatially consistent with the record's own position and radius.
The three sections recorded as undecoded are fixed-stride arrays and counts[0..2]
are their record counts: 12, 96 and 48 bytes. Section 1's remainder is exactly 0
in 11/11 objects and section 2's exactly 96 in 11/11, which is what makes these
strides rather than a coincidence of division.
Section 0 is a point list: 13467 of 13467 records lie inside their object's own
header bounding box.
Section 2 is a plane list, 12 f32: four zeros, a unit normal (|n|=1 in
133573/133573), a signed distance, a point inside the bbox (133573/133573), and
a trailing 1.0 (133573/133573). The decisive check is algebraic -- n.p + d must
vanish for a real plane, and over all 133573 records the relative residual has a
median of 2.29e-08 and a maximum of 2.15e-07. That is float round-off, not a fit.
So a REGN object carries a point list and a plane list beside its uniform grid,
which fits collision or region-boundary geometry and sits next to MCOL.
Still open: section 1 (96 B, 60631 records), what queries the planes, the zeros
at [0..3], and the constant 96-byte tail.
The u16 at +0x10 is 0 in all 2985 bundles; the content is a 16-bit flag word at
+0x12 with 83 distinct values. Reading it as a u32 inflates the field and hides
that the header is built from u16 pairs -- the same shape +0x0c turned out to
have. All 16 bits are used, from 1.4% to 91.5%.
Cross-tabulated every bit against four properties measurable from the bundle:
multi-element, animated, window-starts-at-zero, 30fps. No bit is close to a
clean predicate. The strongest is bit 10 against window-at-zero, 0.79 vs 0.21 --
a real association but not a rule, and exactly the kind of moderate split that
invites over-reading.
Bit meanings stay open, but four candidate readings are excluded rather than
untried and the field is correctly sized. Every property visible in the file has
now been tried, so assigning meanings likely needs the game observed with
individual bundles loaded.
Tested the alternative I recorded last iteration. Parsing every bundle's
keyframe times (2985/2985 parse), the derived-summary reading -- (high,low) ==
(min,max) keyframe time -- holds in 6 of 2985 (0.2%).
The apparent 34.2% match on 'high == min' is a coincidence of zeros: the minimum
keyframe time is 0 in 96% of bundles and high is 0 in 34.9%, so the 1022
'matches' are exactly the both-zero cases. Worth noting that last iteration I
declined to treat the high==0 share as support -- it turned out to be the
confound rather than the signal.
The interval is also narrow: (low-high)/(max-min) has a median of 0.019, about
2% of the keyframe span. It lies inside the keyframe range in 88.6%, entirely
after in 174 and entirely before in 68.
A short authored window is not the shape of a playback range or a whole-animation
loop region, so those readings weaken too. What it is stays open.
Read as a u32 it looks meaningless -- 179 distinct values up to 248581842. The
raw values give it away (0x0007000F, 0x000F001A, 0x003C0064): two big-endian
u16s. Over all 2985 bundles, high < low in 2985/2985 with no equal and no
inverted cases, and both are bounded by the animation length at +0x08. Span
runs 1-1200, clustering on 1/10/30/8/20; low equals the animation length in 4%.
A strict ordering holding 2985 times rules out flags or a packed count. Which
interval it is stays yellow -- playback range, loop region and active window all
fit equally.
Recorded the alternative I did NOT test: that (high, low) is simply the min and
max keyframe time, making it a derived summary rather than an authored range.
First step written down. The 34.9% of bundles with high == 0 leans against it
but is not evidence on its own.
The page is 724 lines of chronological record across 16 sections, several of
them superseded readings kept for their reasoning. A reader arriving fresh had
to read all of it to know what is currently true.
Adds a summary table at the top covering the twelve questions the page answers,
each with its confidence, plus the wave-enumeration recipe and an explicit note
that everything below is history. Also names the four mistakes recorded below so
a reader knows they are deliberate rather than stale.
Last iteration I noticed BR09_04's block id names BR10_03, itself the next
exception entry, and flagged it as suggestive but untested. Tested: sorting all
282 by offset, entry[i].field equals entry[i+1]'s own id in 8 of 281 (2.8%),
indistinguishable from chance -- and BR09_04 was one of the eight.
Recorded rather than dropped because it is exactly the kind of pattern that
reads as a discovery when spotted in a sample of one.
The offset from the entry's own id is broadly spread, 7 to 14 and beyond with a
peak at 9-11, so the field is near the entry's id without standing in any fixed
relation to it.
Not noise -- a small structured record in three runs at about +1790, +3840 and
+7940. Across all 282: the leading 16-bit value names a real SOUNDS cue
282/282, the block carries two IEEE 1.0 floats in 281, and an 01/02/02/0x64 tail
in 281. Two unit floats and a 100 are the shape of volume, pitch and priority,
so this reads as an XACT cue record -- marked yellow, since every value is a
default and nothing varies enough to prove it.
The id is NOT this entry's own cue: it is consistently a different, higher one
(8501 -> 8504 = BR02_01; 5027 -> 5036 = VOICE_A_036), with a varying offset so
not a fixed stride. BR09_04's field names BR10_03, which is itself the next such
entry -- suggestive of a chain, but one observation and untested.
The practical point, which is settled: the block is a populated metadata record
naming a real cue, not padding.
Both, 282/282 exact: w0 == table[-2], w1 == table[-1]. The trailer's first two
words are a verbatim copy of the tail of the wave's own seek table.
That closes the question and kills the 'usable length or loop end' reading of
w0 -- it is just the previous seek point. It also explains why w1-w0 is always a
multiple of 512 without needing a separate fact: consecutive seek points advance
by whole 512-sample frames, so adjacent entries always differ by a whole number
of them. What I had recorded as a meaningful constraint was a property of the
table the values were copied from.
Neither word is the wave's true length -- w1 lands within 512 of the
PsuedoBytesPerSec length in only 9 of 282, consistent with the correction above.
Still unidentified: the ~40 scattered non-zero bytes elsewhere in the block.
Chasing w0 I decoded leading waves and the output matched neither trailer word.
Following that into RIFF waves, where the extent is certain, the decoded sample
count exceeds the seek table's last cumulative sample by a median 9.7%.
The tiebreak is the bank's own PsuedoBytesPerSec: over 14 banks it agrees with
an actual FFmpeg decode to a mean of 0.007 s and with the seek-table duration
only to 0.287 s. Two independent quantities agree with each other and both
disagree with the seek total, so that last entry is the last SEEK POINT, not the
sample count.
Both artifacts regenerated with data_bytes / PsuedoBytesPerSec. Total audio is
408.3 minutes, not the 390.9 I published -- 4.3% in aggregate. Dialogue timings
all rise, e.g. 'They got Leader!' from 1.78/2.46 s to 2.25/2.93 s.
What made this hard to catch is worth recording: the seek total gave BGM of 2.4
minutes, chatter of 2.8 seconds and cutscenes of 11 minutes, and I cited that as
the chain validating itself. It validated the structure, not the scale -- a
uniform 10% error preserves every ratio I checked.
The trailer opens with two little-endian u32 words. w1 is the leading wave's
total sample count -- 282/282 exact, the same number as the last entry of that
wave's own seek table. And w0 < w1 with w1-w0 always a whole multiple of 512
(282/282), i.e. a whole number of XMA1 frames, 9 to 91 with a median of 14.
What w0 means is left at yellow: trailing the total by a whole number of frames
is the shape of a usable-length or loop-end field, but that is a reading of the
shape rather than a measurement, and nothing here separates the candidates.
Also corrects my own description: I called the region after the seek chunk 'zero
padding'. It is not padding -- about 48 non-zero bytes are scattered thinly
across the 12 KiB, roughly 17 per KiB in blocks 1, 3 and 7. That is a sparse
table, still unidentified, and the distinction matters to anyone skipping it.
Dumped it. In all 282 exceptions the region from the earlier wave's seek to the
first RIFF is exactly 12288 bytes: the seek chunk (240-260 bytes), then 47-57
non-zero trailer bytes, then zero padding out to the block size.
So an entry may hold a leading wave, a 12288-byte padded block, then its RIFF
wave -- and the assignment rule completes: first-or-second wave at/after the
entry offset is 7620/7620 = 100.00%, with ZERO unexplained.
This overturns my own refutation from one iteration ago. I proposed these were
leading segments, tested it as 'the seek should sit AT the first RIFF', got 0 of
282, and recorded the hypothesis refuted. It was right; my predicate was wrong
by exactly the padded block, a constant I had not yet found. A negative result is
only as good as the predicate it tests.
Still unidentified: what the 47-57 trailer bytes hold.
Scanning the flat stream for seek chunks satisfying size == 8+4*count with a
non-negative implied start yields 9661 waves and rejects NOTHING -- no false
positives in a gigabyte of audio. Each wave's extent is
[seek_pos - count*2048, seek_pos).
Assignment rule 'an entry names the first wave starting at or after its offset'
is 96.30% correct (7338/7620) against entries whose answer is independently
known from their RIFF.
The 282 exceptions are one class: the gap between the earlier wave's seek and
the first RIFF is EXACTLY 12288 bytes in all 282. I guessed they were leading
segments, which would put that seek AT the RIFF; refuted, 0 of 282.
Recorded at yellow. Correcting by 12288 would reach 100% on this set but is
curve-fitting until the region is identified, and the rule cannot yet unblock
headerless durations -- there is nothing to check it against there.
343 English headerless banks carry a valid seek chunk, which would lift dialogue
duration coverage from 62% to ~75%, with a median 3.25 s that looks exactly like
in-mission voice.
It is wrong. A seek sits immediately after its OWN data, so the implied wave
start is seek_pos - packets*2048 -- and for 277 of the 343 that start lies
BEFORE the entry begins, meaning the seek describes the previous bank's wave.
Applying it would attach a neighbour's duration to 81% of the recovered rows.
The 66 that start inside the entry are not proven to be that entry's wave
either; 'starts inside' is necessary, not sufficient.
Recorded as refused rather than applied with a caveat, because a plausible
median is exactly what has misled this work twice today. Recovering these needs
the bank-to-wave assignment settled first.
Joins the ISL script's built-in 64 call sites, the caption table and the sound
bank seek tables. Duration is the last cumulative sample over the sample rate --
read, never decoded. 1663 of 2683 call sites (62.0%) have an English duration,
2198 (81.9%) a Japanese one; the gap is the headerless banks, and which language
lacks one differs per line, so the columns are complementary.
Two columns deliberately: FILES names the jpn banks and the game substitutes the
language directory, so a line's English and Japanese recordings are different
files of different lengths -- 'They got Leader!' is 1.78 s in English and 2.46 s
in Japanese. The captions are English, so eng is the matching column. The first
version of this artifact had one column and would have attached Japanese timings
to English text; that is why it was regenerated.
With the wave boundary exact and Channels read rather than assumed, every field
describing a bank can be read: path, cue, sound id, channels, rate, data bytes,
packets, samples, seconds. Duration comes from the last cumulative sample in the
seek table over the sample rate -- arithmetic, no decoding.
Nothing was fitted to an expectation, yet every category lands where its content
says it should: BGM mean 146 s, Movie mean 79 s with an 11-minute maximum,
Briefing 10.8 s, in-mission Voice 2.79 s never exceeding 7.3. If the boundary
rule, the channel field or the seek table were misread, the numbers would not
sort into those five shapes.
Incidental: 4 banks run at 44100 Hz against 48000 everywhere else, and the BGM
tracks are the stereo ones. 1021 of the 5135 FILES paths have no RIFF and are
excluded as headerless.
The declared sizes are honest (seek magic at data_at + declared_size, 7620/7620)
and VOICE_TCAF_608 was stereo decoded as mono, not truncated. The four offsets
are a segment-packing phase, not a per-directory header size. Neither was closed
by finding something new; both were closed by correcting a mistake of mine.
I read the seek chunk's packet count big-endian; it is little-endian at seek+12,
with size == 8 + 4*count. And a seek sits immediately AFTER its own data, so an
entry's first seek usually belongs to the PREVIOUS bank (implied start -25232
for D_452, -145988 for TCAF_608). I was comparing an entry's first seek against
its first data -- different waves by construction, which is why no reading lined
up.
With that fixed, the declared sizes are honest: every RIFF-bearing entry on the
disc has seek magic at exactly data_at + declared_size with count*2048 ==
declared. 7620/7620, zero failures.
VOICE_TCAF_608 is not truncated. Its Channels is 2 and I decoded it as mono;
read as stereo it gives 6520176 bytes = 33.96 s, agreeing with both length
signals in the bank (33.88 s from cumulative samples, 33.97 s from
PsuedoBytesPerSec). 170 of 8021 banks (2.12%) are stereo -- exactly the rate of
my 1-in-60 outlier.
This is the mono/stereo trap already documented on this very page, met from the
other direction: I had written 'at two channels every bank yields one frame' and
then spent several passes blaming missing data for a one-frame decode.
Code fix: to_xma_riffs built the leading segment with a hard-wired mono fmt. It
now reads Channels from the bank's first RIFF. 7 disc tests pass.
The '3 or 4 wrapped lines' sentence is in isl-message-dialogue-link.md itself,
not in cutscene-message-table.md as my correction claimed. Corrected the
sentence in place -- a page has four line SLOTS, of which an English page fills
1 to 4 -- and fixed the misattribution in the correction note rather than
quietly repointing it.
The five European packs have byte-for-byte identical MSG key sets: 18813 keys,
4091 ids, zero differences in any direction. Japanese has 14295 keys, which
looks like 4521 missing captions and is not: its id set matches to within 5, its
page count is the same (4765 vs 4758), and its mean lines per page is 1.52
against English's 1.85 with no 4-line pages at all. It carries the same dialogue
and wraps into fewer lines, which is what a language without inter-word spaces
does in a fixed-width box.
Also corrects my own wording in cutscene-message-table.md, which said a page
'holds 3 or 4 wrapped lines'. That counted key SLOTS, which MSG_DEMO records
allocate four at a time. Counting lines with text, an English page holds 1 to 4
(1574/2439/632/113). And it reconciles the 8800 figure used throughout: that is
distinct English keys with non-empty text, out of 18813 keys total.
