The page rested on twelve values from two screens. This walks all 4525 sprites
on the disc.
* The field is a u16 at +0x0A. The upper half of the 32-bit word the page
reads is zero in 4525/4525. Nothing above changes -- 0x00008100 sorts the
same as 0x8100 -- but a future value with the high half set would mean
something had been misread rather than that the layer got deeper.
* It is an enumeration: 45 values for 4525 sprites, one of which (0x8100)
covers 1188 of them.
* The reading worth trying -- a global layer vocabulary shared across the UI
-- is refuted. Only 4 of 45 keys cross a pak family and 33 of 45 live only
in GP_MAIN_GAME_2D; every other pak owns a narrow high-byte band (0x90-0x94
for the in-game overlays, 0xa4 mission log, 0xb1-0xb2 save/load). A screen
that owns one or two keys is not ordering itself with them.
That supports "group id in the high bits, order in the low bits", which is what
the page already suspected, but it does NOT test it: paint order has been
measured on two screens and both are inside GP_MAIN_GAME_2D, so there is no
ground truth to check the split against. Left amber.
The first number I got was 37/45 shared, which would have supported precisely
the wrong conclusion. It came from counting paks instead of pak families: the
six GP_MAIN_GAME_*2D paks are the same screens in six languages and their key
sets are byte-for-byte identical. Recorded on the page, because the shape
recurs -- a corpus with near-duplicate members manufactures agreement.
Two follow-ups on yesterday's^Wthis morning's CollisionSet write-up.
1. The _cmesh <-> render-model link, which I recorded as UNTESTED because
matching stems against .xbg object names covered 4 of 158. The disc keeps
only one build manifest, so that corpus was never going to answer it. The
right corpus is the GameResourceID field of the DefTables / GP_MAIN_GAME
records -- 480 distinct values. Against those, with a control that shuffles
the characters of each stem:
ship/mob stems prefixed by a real resource id 108/112 = 96.4%
same stems, characters shuffled (control) 0/112 = 0.0%
asteroid stems prefixed (expected none) 0/46
So a CollisionSet entry is <GameResourceID>[_<part>]_cmesh. The 0/46 on
asteroids matters as much as the 108/112: a test that fired on everything
would be the bound-check hazard again.
2. The world unit. Sweeping every pak for a name carrying a kilometre figure
returns mapmesh_box_500km.col/.rgn and nothing else -- 162 references, all to
that one pair. The reading rests on a single filename with no corroborating
instance anywhere in the data, so no static test can settle it; marking it
blocked on the oracle rather than leaving it as an open static question.
My objection's premise did survive: rou_e010 is a real GameResourceID and
e010_ADAN_Attacker_S is in the stage tables, so the 133-unit mesh does belong
to a craft the game calls an attacker. Whether the trailing _S means "small"
is a further guess (there are _EX4 / _HF / _HF_Wayne variants), so it stays
suggestive rather than evidence.
All 18 blobs are byte-identical: the per-stage naming is nominal, and every
stage points at one shared 1675148-byte library stored eighteen times. That
identical size was the reason to open the item, and it turned out to be the
answer to it.
Record layout: {u32 size, u32 name_len, char name[name_len], u32 nv, u32 nt,
f32[3] x nv, u32[3] x nt}, next record at off + 8 + size. The indices are u32
here where MCOL uses u16 -- two different serialisers in one archive.
What makes this a decode rather than a plausible reading: the walk consumes the
file to the byte over 158 variable-length records, with the size word predicted
from the two counts 158/158. A wrong field would desynchronise within a few
records and could not land exactly on the end. All indices in range 158/158;
98.24% of edges shared by exactly two triangles; 147/158 fully manifold.
158 meshes, 90 836 triangles: per-part ship proxies (_bdy/_brg/_eng/_wep/_sld,
the XBG7 sub-part vocabulary) plus 46 stage asteroid meshes whose prefixes are
exactly the stages that have an _AsteroidVolume_wp MCOL.
Two things this file makes me walk back:
* The "1 unit = 1 metre" reading from mapmesh_box_500km is downgraded to
amber. The 500000 arithmetic stands, but it implies that a craft the game's
own tables call "small" is 133 m and that rob_f002 is 447 km -- 89% of the
arena width. The format check survives; the interpretation has no
independent support.
* The _cmesh <-> render-model name link is recorded as UNTESTED, not
confirmed: only one .xbg build manifest survives on the disc, so matching
stems against object names covers 4 of 158, which is no coverage at all.
The 0x50 header word, which the first section of this page had dismissed as "a
large value", is two u16 counts: vertices and triangles. They give the two
remaining blocks their stride, and every derived length is exact in 11/11 --
len(0x54) == align16(12*nv), len(0x58) == align16(6*nt), and nt equals the
bounding-sphere count decoded last iteration.
Checks that cannot pass by accident:
* sphere i is the TIGHT bounding sphere of triangle i, 4768/4768, with
max|v-c|/r median 0.99990 (a fixed 1.0001 epsilon), against a 1.32%
random-triangle control;
* the mesh is watertight -- every edge shared by exactly two triangles,
7152/7152, zero degenerate triangles, zero unreferenced vertices;
* the two smallest objects are 8 vertices and 12 triangles whose positions
are the eight +-250000 corners of the map bbox: a bare bounding cube.
The cell lists are a correct broad phase: with an exact triangle/box SAT test
only 3 overlapping triangles in 18 577 entries are absent, so a query walking
one cell's list cannot miss a hit. The 730 conservative extras bracket the
builder's own test between exact-SAT and AABB, which retires the 18 unexplained
"sphere misses" from the previous commit as that same margin.
mcol_probe.py gains `mesh` and `obj`; `verify` now runs all three checks and its
output is recorded in docs/re/data/mcol-verify.txt.
The unexplained ~0.75 ratio left at the end of the last iteration was my own
stride. I had read the block as 12-byte points because REGN's vertex section
is 12 bytes, and never checked it: len(0x5C) is not a multiple of 12 in 5 of
the 11 objects, so that stride was never arithmetically possible.
At stride 16 the relation is exact in 11/11 -- max u16 == len(0x5C)/16 - 1 --
and the record reads as {centre f32[3], radius f32}. Powered test, since a
u16 is reached through a specific grid cell: the sphere it names reaches that
cell in 18 559/18 577 = 99.90%, against a 12.02% random-sphere control. Both
fields carry signal (centre alone 26.75%, radius shuffled 70.19%).
The converse -- is the list *exactly* the intersecting set? -- is 0.38%, which
is the expected direction: a bounding sphere is conservative, so membership
implies overlap but not the reverse. The tighter geometry is in 0x54/0x58,
still undecoded. 18 entries (0.10%) go the wrong way and are recorded as open.
tools/re-capture/regn_decode.py is copied unchanged from auto/regn-reader so
the probe's POF0 reader is the known-good one rather than a second copy.
The original reason for investigating REGN was that a mission's enemy count
rises and falls, so a scheduler with parameters must exist somewhere, and a
per-map uniform grid is what such a thing would be indexed by.
Now that it is decoded that reasoning is answered: REGN is a tetrahedral
navigation mesh -- vertices, faces carrying plane equations and adjacency,
tetrahedra with portal costs between face pairs, and a grid indexing which tets
fall in each cell. Every section is accounted for by that structure, and there is
no time field, no unit reference and no trigger anywhere in it.
So the wave-scheduler search should treat REGN as excluded rather than unread.
The page's original hedge was right to keep the reading provisional, but the
reasoning it hedged was a guess from shape, and the shape belonged to
pathfinding -- which is what pointed the whole investigation here.
The arrival timetable in Route_S<NN>.tbl, keyframed per squadron per phase with
t in seconds, remains the only located part of the mechanism.
I recorded the static coupling search as exhausted and needing PE code. The PE
work was done on auto/regn-reader and it solved the whole thing: REGN is a
tetrahedral navigation mesh, reached via the POF0 fixup table -- the loader's own
list of which words are pointers, so nothing needed guessing. Six sections;
position -> cell -> 32-byte item -> tet refs -> tetrahedron, with section 2 a
face carrying a plane plus its 3 vertices and the two tets either side.
Controlled checks: face through 3 of 4 tet vertices 253722/253722 against a
0.07-2.2% random control; portal cost equals face-centroid distance
380460/380460.
Also records against myself that the base is chunk+0x10 and my offsets here were
16 bytes early, that the plane arithmetic survives only because those fields
landed on the same bytes, and that my points-in-bbox count was never evidence --
a shift inside a homogeneous f32 array yields other floats from the same array.
Leaving the wrong conclusion in the backlog would have told the next reader the
avenue was closed when it was the one that worked.
The previous test only tried section-1 targets. Closing that gap: the payload's
three index-shaped u32s, followed into the point list and the plane list and
checked for the target lying inside the referencing cell, all sit at the 0.203%
random control.
Two cells read 0.81%, 4x the baseline. I am not treating that as a lead: across
this and the previous iteration roughly twenty such tests have been run, and at
that count a single 4x enrichment on ~8000 trials is what noise looks like.
Calling it a signal would be the multiple-comparisons error a long hypothesis
sweep invites.
So REGN's header, grid, points, planes and cell index are decoded, section 1's
slot regions are censused, and the link between the grid and the geometry is not
reachable by any static test I can construct. The honest next step is the PE
code that reads a REGN object -- the same kind of work that cracked the .slb
packing phase -- rather than a twenty-first correlation.
The three sections recorded as undecoded are fixed-stride arrays and counts[0..2]
are their record counts: 12, 96 and 48 bytes. Section 1's remainder is exactly 0
in 11/11 objects and section 2's exactly 96 in 11/11, which is what makes these
strides rather than a coincidence of division.
Section 0 is a point list: 13467 of 13467 records lie inside their object's own
header bounding box.
Section 2 is a plane list, 12 f32: four zeros, a unit normal (|n|=1 in
133573/133573), a signed distance, a point inside the bbox (133573/133573), and
a trailing 1.0 (133573/133573). The decisive check is algebraic -- n.p + d must
vanish for a real plane, and over all 133573 records the relative residual has a
median of 2.29e-08 and a maximum of 2.15e-07. That is float round-off, not a fit.
So a REGN object carries a point list and a plane list beside its uniform grid,
which fits collision or region-boundary geometry and sits next to MCOL.
Still open: section 1 (96 B, 60631 records), what queries the planes, the zeros
at [0..3], and the constant 96-byte tail.
The u16 at +0x10 is 0 in all 2985 bundles; the content is a 16-bit flag word at
+0x12 with 83 distinct values. Reading it as a u32 inflates the field and hides
that the header is built from u16 pairs -- the same shape +0x0c turned out to
have. All 16 bits are used, from 1.4% to 91.5%.
Cross-tabulated every bit against four properties measurable from the bundle:
multi-element, animated, window-starts-at-zero, 30fps. No bit is close to a
clean predicate. The strongest is bit 10 against window-at-zero, 0.79 vs 0.21 --
a real association but not a rule, and exactly the kind of moderate split that
invites over-reading.
Bit meanings stay open, but four candidate readings are excluded rather than
untried and the field is correctly sized. Every property visible in the file has
now been tried, so assigning meanings likely needs the game observed with
individual bundles loaded.
Tested the alternative I recorded last iteration. Parsing every bundle's
keyframe times (2985/2985 parse), the derived-summary reading -- (high,low) ==
(min,max) keyframe time -- holds in 6 of 2985 (0.2%).
The apparent 34.2% match on 'high == min' is a coincidence of zeros: the minimum
keyframe time is 0 in 96% of bundles and high is 0 in 34.9%, so the 1022
'matches' are exactly the both-zero cases. Worth noting that last iteration I
declined to treat the high==0 share as support -- it turned out to be the
confound rather than the signal.
The interval is also narrow: (low-high)/(max-min) has a median of 0.019, about
2% of the keyframe span. It lies inside the keyframe range in 88.6%, entirely
after in 174 and entirely before in 68.
A short authored window is not the shape of a playback range or a whole-animation
loop region, so those readings weaken too. What it is stays open.
Read as a u32 it looks meaningless -- 179 distinct values up to 248581842. The
raw values give it away (0x0007000F, 0x000F001A, 0x003C0064): two big-endian
u16s. Over all 2985 bundles, high < low in 2985/2985 with no equal and no
inverted cases, and both are bounded by the animation length at +0x08. Span
runs 1-1200, clustering on 1/10/30/8/20; low equals the animation length in 4%.
A strict ordering holding 2985 times rules out flags or a packed count. Which
interval it is stays yellow -- playback range, loop region and active window all
fit equally.
Recorded the alternative I did NOT test: that (high, low) is simply the min and
max keyframe time, making it a derived summary rather than an authored range.
First step written down. The 34.9% of bundles with high == 0 leans against it
but is not evidence on its own.
The declared sizes are honest (seek magic at data_at + declared_size, 7620/7620)
and VOICE_TCAF_608 was stereo decoded as mono, not truncated. The four offsets
are a segment-packing phase, not a per-directory header size. Neither was closed
by finding something new; both were closed by correcting a mistake of mine.
Built-in 12 activate_unit returns 0 when the live object is NULL: it registers an
object that already exists and cannot create one. So a wave arrival is a craft
reaching a point on its route, not a new record, and a count of entity records
was never going to move.
Stage 02's UnitGroup totals Count = 116 across 111 squadrons, matching the live
probe's flat 116. The natural objection is that 116 might match by accident --
it does not: across all 28 stages the roster totals run 2..116, and 116 is the
MAXIMUM and unique to Stage 02, with nothing else within 5.
Three independent things agree: the handler cannot spawn, the roster totals 116,
and the live count sits at 116 while 22 of 113 routed squadrons have a route
starting after t=0.
Still n=1 for the per-member identity (only a Stage 02 save exists), and this
cannot settle timetable-vs-event -- that needs a position probe, not a count.
Route keyframes are (time, quat x4, pos x3), so each consecutive pair gives an
implied speed. The flight-speed law was measured live against a real-time clock
(cruise ~420, max ~1530 world units/s), which makes it an independent yardstick
in known units.
Across all 28 stage tables, 331 Route_* records and 1104 keyframe pairs: under
SECONDS not one pair exceeds the 1530 ceiling (median 147); under frames at 30
or 60 fps, 89% and 94% of the game's routes would demand impossible speeds.
Two corrections recorded. A first pass silently skipped non-numeric fields while
flattening records, shifting every later value -- it reported a median implied
speed of 65534, suspiciously 2^16, which is what exposed it. And the remaining
outliers were Frame_S<NN>_Asteroid records, not routes; the final measurement
filters by record kind rather than dropping whole stages.
The ties needed a different signal, not a longer scan. Banks carry one: a seek
chunk sitting on a packet boundary, so seek_pos % 2048 IS the data offset. On
the 6033 labelled banks with a seek before their first RIFF, 6031 agree
(99.97%) -- better than the packet scan and structural rather than statistical,
so scan_data_offset now tries it first.
On the scan's 28 ties it resolves 26 correctly and 0 wrongly (2 have no usable
seek). Combined rule scores 7354/7358 = 99.95%, up from 99.62%. 762 of the 1495
RIFF-less banks carry a seek, so the signal exists where it is needed.
Also ruled out, since a wrong offset was this page's whole subject: the header
is not audio being discarded. Adding 0 to the candidate set, it wins 6 of 7358.
7 disc tests pass.
16 of the 17 pages with an empty slot 4 belong to MSG_DEMO_600..604, whose
caption text matches, word for word, the five resupply lines documented from the
movie side as VOICE_D_450..454. Slot 4 is empty on purpose: that audio binds
through the movie path, not through a DEMO_nnn cue.
That also disposes of the '5 ids with more than one record' item -- each appears
once per stage that uses it, and the repeat counts equal the movie-slot counts
recorded independently for the matching banks: 3/3/4/4/2 both ways, 5 of 5. The
11 extra records over 138 ids are these repeats.
Worth more than closing two items: the cutscene message table and the movie
subtitle chain were decoded from different containers by different routes and
agree on the same five lines with the same multiplicities.
MSG_DEMO_500 remains: one page, KATANA, 2s, no text and no cue.
Of the 7586 banks with a RIFF and a data chunk after it, 5296 declare a data
size larger than the pak entry holds; 2290 declare less (the ordinary
multi-sub-wave case); NONE declare exactly what they hold. This contradicts the
decoder comment claiming the declared size 'is honest per sub-wave'. The code
clamps, so it is a documentation defect, not a crash.
It also closes the loose end from the offset work: eng\Voice\VOICE_TCAF_608,
the single bank where neither offset decoded, is 99% short -- there is nothing
there to decode.
Method note recorded: my first pass searched for 'data' from offset 0, which can
match by chance inside the leading audio region. Anchoring the search after the
first RIFF moved the count 5038 -> 5296.
Separately, the 55 'early RIFF' English banks are not an anomaly: all 55 sit at
exactly 1392 behind a zero-filled header -- a zero-length leading region, which
both the old and new code already handle correctly.
static.slb (8970240 bytes, the shared SE bank) and Pj_Silph.xgs (533 bytes, XACT
global settings) both hash into the TOC. Their names come from the BANK_SE and
SETTINGS records of the very IDXD object this page documents -- I had printed
them at the top of the write-up and then reported the entries as unidentified.
9519 of 9519 now: 5100 jpn + 4382 eng + 35 root + these 2.
My sound-cue write-up said the table 'does not explain' the VOICE_D_452
rejection and implied it was still open. It is not: voice-bank-leading-region.md
settled it earlier -- the bank holds one generic line reused across four stages,
so the binding is correct and the expectation was wrong. The narrower true
statement, which is what the cue table actually adds, is that the reuse is a
movie-slot -> bank relation and not a cue-level one.
to_xma_riffs now emits the leading headerless segment when it sits at a whole
number of XMA1 packets and carries a non-zero byte. VOICE_D_453 goes from a
0.14 s trailing fragment to a 45116-byte leading sub-wave that dominates it.
I withdrew this exact change earlier for two reasons. Both are now answered
rather than argued away:
* "It recovers no audio" -- it used the STEREO format. At two channels every
bank yields exactly 1792 bytes, one frame, whatever its size. Mono yields up
to 113x more.
* "It matches 1524 of 8021 RIFF-bearing entries" -- the byte-level reach is
still 1524, but the audible reach is not. Across the 84 movie-bound banks
the segment adds >1 s to exactly 7, the hokyu_*_H tankers on D_453/D_454 --
precisely the broken ones -- and <=0.25 s to 66 of the rest. The largest
non-resupply addition is S04A at +0.66 s on a 256 s movie.
The safety oracle is recorded with its limits: 8 of the 84 banks ALREADY
exceed their movie's duration before the change, by hundredths of a second,
so it cannot resolve differences at that scale. It establishes scoping, not
correctness. Callers clamp to the movie length regardless.
VOICE_D_451's all-zero leading region is skipped by the non-zero guard, so
the rule cannot prepend silence to a bank that does not need it. Pinned, as
is the packet arithmetic (n = 8, 1, 7, 22, 29) which has no tunable.
slb_disc, movie_subtitle_disc and movie_manifest_disc all still pass.
NOT verified by ear -- that needs a human.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The last open question was whether the leading XMA1-mono region duplicates the
RIFF sub-wave, which would make the earlier totals double-count. It does not.
Decoding both parts of every bank to mono PCM and measuring energy:
bank leading secs / RMS riff secs / RMS
VOICE_D_450 0.49 / 158 2.82 / 9898
VOICE_D_451 0.01 / 0 1.58 / 9128
VOICE_D_452 0.31 / 301 2.18 / 9061
VOICE_D_453 2.12 / 9770 0.14 / 14462
VOICE_D_454 3.07 / 10428 0.43 / 11639
Two shapes, and no bank holds the same content twice. In 450/451/452 the
leading region is silence or near-silence (RMS 0-301 against ~9000 for
speech) and the RIFF holds the line. In 453/454 the leading region holds the
line and the RIFF is a short loud tail fragment. Sequential segments of one
clip, so the totals stand and with them the 48 kHz fit.
This also closes the mystery that started the whole thread. The corpus
recorded 450 = 2.8 s, 451 = 1.6 s, 452 = 2.2 s as plausible but 453 = 0.14 s
and 454 = 0.43 s as "far too short". The decoder skips everything before the
first RIFF: for the first three that discards only silence, so they looked
fine; for the last two it discards the line itself and leaves the trailing
fragment. One rule, two outcomes, depending on which segment holds the
speech.
The fix is now well-posed in a way the withdrawn attempt was not: emit the
leading region only when it carries signal. That also avoids the 1524-bank
blast radius that sank the earlier version.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The load-bearing error of the whole voice-bank thread, and it is mine. It
stood for three iterations across two write-ups that each called the result
proven.
I treated a subtitle cue as a timestamp that must fall INSIDE the voice clip,
and concluded a 0.07 s clip could not host a cue at 4.70 s. A cue is when the
line STARTS. The voice plays from the cue, so the clip only has to fit the
window between the cue and the end of the movie. Under that reading every
bank fits at plain 48 kHz:
bank samples @48kHz cue window
D_450 158967 3.31 4.00 5.30
D_451 76084 1.59 3.70 5.60
D_452 119562 2.49 0.00 8.34
D_453 108608 2.26 4.70 4.60
D_454 167828 3.50 0.00 9.50
2-3.5 s is also the right length for the lines. Nothing is missing, and the
17091-20563 Hz window from the previous commit is void with it -- its lower
bound came from the same misreading.
What survives, because it was measured rather than interpreted: the leading
region is XMA1 mono, the decode runs to the final frame, and cue values are
seconds.
Separately settled, and it is what exposed the error: each shared bank holds
ONE generic line. The 3-5 movies bound to a bank have IDENTICAL subtitle
text, 5 banks out of 5 -- "Rhino 3 has landed. Commencing resupply.",
"Resupply complete. You are cleared for take-off!", and so on.
That also explains the historical in-game rejection of hokyu_DS_s13A ->
VOICE_D_452 that started this whole thread. The line is generic, identical
for s02A/s07A/s08A/s13A. Someone expecting a stage-13-specific line would
hear the generic one and call it wrong -- while the binding is exactly right.
The disc said so; the subtitle text now says so independently.
Still open: whether the leading mono region is additional audio or an
alternate take, since the totals above add it to the RIFF sub-waves.
Artifact: examples/shared_bank_takes.rs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Two things settled, one of them a correction of my own claim from the last
iteration.
The cue unit is verified rather than assumed. parse_timing computes mm*60+ss,
but only if the token really is mm:ss.cc, so I checked against an independent
oracle: the movies are on the disc and a cue must land inside its own movie.
66 English movies with subtitle tracks, 0 cues land after the movie ends.
Centiseconds would have overflowed essentially all 66. The seconds reading
stands and the verdicts built on it survive.
"The sample rate does not converge" does not. I reported implied rates of
39742 / 20563 / 23108 Hz as irreconcilable. They are not estimates of the
same quantity -- each is a ONE-SIDED BOUND. The audio must be at least as
long as the last cue, so samples/cue is an UPPER bound; it cannot outlast its
movie, so samples/movie is a LOWER bound. Intersecting:
bank samples cue movie lower Hz upper Hz
VOICE_D_450 158967 4.00 9.30 17091 39742
VOICE_D_451 76084 3.70 9.30 8180 20563
VOICE_D_453 108608 4.70 9.30 11677 23108
=> 17091-20563 Hz, non-empty. A single rate IS consistent.
I had been comparing them as competing point estimates, which is why they
looked contradictory.
What is still open, and stated as such: that window contains no standard XMA
rate. The lower bound assumes a whole bank plays inside one movie, and each of
these banks is bound to 3-5 movie slots -- so if a bank holds several takes
the lower bound is void, leaving rate <= 20563, which 22050 nearly meets.
Next step recorded: establish whether a shared bank is one line or several.
Artifact: examples/cue_unit_check.rs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
I had been discarding FFmpeg's stderr with Stdio::null(). It says exactly
what happens: an unimplemented "Reserved bit", then a negative bit-skip --
and the failing frame index is always the last one.
bank packets frames fr/packet fails at
VOICE_D_450 8 45.7 5.71 44
VOICE_D_452 7 29.4 4.21 28
VOICE_D_453 22 198.9 9.04 198
VOICE_D_454 29 287.5 9.92 287
So the previous entry's "the decodes are visibly partial" is wrong, and it
was mine. I read "samples per input byte ranges 2.10-4.96" as truncation; it
is ordinary XMA1 variable bitrate. Only the final frame of each stream is
lost. The decode is essentially complete.
The sample rate still does not converge. I tried the obvious repair --
counting the whole bank, leading region plus RIFF sub-waves, since the two
split the audio very differently per bank. Two banks then agreed at a tidy
~2.1x ratio pointing near 22 kHz, and the third refuted it: implied rates are
39742, 20563 and 23108 Hz.
So the container is identified, the decode is essentially complete, and the
duration still does not reconcile -- which moves suspicion to the other side
of the comparison. movie_subtitle::track_voice_cues returns (u32, f32) and I
have been reading that f32 as SECONDS on the strength of the format notes
describing mm:ss.cc cue text. If it is centiseconds, a frame index or a
per-page offset, every "audio missing" verdict inherits the error. Recorded
as the next thing to check, and to be checked BEFORE any more audio work.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Retried the format probe with the fmt chunk built to synth_xma1_fmt's exact
byte layout, and with the bank's own RIFF sub-wave decoded through the same
pipe as a CONTROL so a broken harness cannot masquerade as a result.
The channel count is the whole story:
bank lead B channels=2 channels=1
VOICE_D_450 16384 1792 46756
VOICE_D_451 2048 1792 896 (all-zero region: control)
VOICE_D_452 14336 1792 30154
VOICE_D_453 45056 1792 203648
VOICE_D_454 59392 1792 294440
channels=2 yields EXACTLY 1792 bytes for every bank regardless of size -- one
frame, then it stops. That constant is the tell. At channels=1 the same data
yields up to 113x more, and the control sub-wave decodes to 13568, so the
pipe works.
Why the previous probe got 0 bytes everywhere is now named: I read
synth_xma1_fmt(2, 2, 48000)'s second argument as a STREAM COUNT when it is a
CHANNEL MASK, and built the WAVEFORMATEX around that misreading.
Also recorded as a refutation, because it was tempting: solving for the
sample rate as decoded-samples / last-subtitle-cue does NOT converge. D_453
implies 21665 Hz -- close enough to 22050 that I nearly wrote it down -- but
D_450 implies 5844 Hz. No single rate explains both, and the decodes are
visibly partial (samples per input byte ranges 2.10-4.96 where a clean decode
would be near-constant).
So the container is identified and the duration is not. Next step recorded:
find why FFmpeg stops early, likely the hardcoded packet/subframe fields.
Artifact: examples/slb_fmt_probe.rs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The corpus said 0.14 s is "far too short for the spoken line". That is a
judgement about audio, and audio judgements cannot be made in this container.
The subtitle tracks settle it without listening: each carries cue START
times, and a subtitle that appears at t seconds cannot sit inside a clip
shorter than t.
FFmpeg-measured (not estimated from a compression ratio -- the first version
of this example used an 8:1 guess, which is not good enough to hang a
conclusion on):
hokyu_LS_s02A D_450 cue 4.00 s audio 1.41 s MISSING
hokyu_LS_s09A D_451 cue 3.70 s audio 1.81 s MISSING
hokyu_LS_s02H D_453 cue 4.70 s audio 0.07 s MISSING
hokyu_DS_s13A D_452 cue 0.00 s audio 1.21 s no signal
hokyu_DS_s07H D_454 cue 0.00 s audio 0.21 s no signal
Three of five are decisive; the other two have their only cue at 0.0 s and
say nothing either way. So something is genuinely missing from these banks --
established independently of the leading-region work, and measured rather
than felt.
The fmt-variation probe I recorded as the next step is INCONCLUSIVE and is
written up as such: 36 combinations over VOICE_D_453's 22-packet leading
region all produced 0 PCM bytes, including ones that should be equivalent to
the crate's own synth_xma1_fmt, which does parse. So the probe tested my
hand-built fmt chunk, not the hypothesis, and it is NOT evidence that the
region is non-XMA. The retry should use the crate's helper.
Artifact: examples/voice_len_vs_subs.rs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The structure is exact. In all five resupply banks the first RIFF sits at
HEADERLESS_DATA_OFFSET + n*2048, where 1392 is a constant this crate already
had and 2048 is the XMA1 packet size: n = 8, 1, 7, 22, 29. No free parameter
to tune, and the raw bytes agree -- high entropy from offset 0, then a zero
run immediately before the RIFF. VOICE_D_451 is the control, its single
packet being all zeros.
So I made the obvious fix, emitting that region as a sub-wave, and then
withdrew it on two measurements:
* It does not recover audio. Coverage went 5.4% -> 89.9% for VOICE_D_453, but
the emitted stream decodes through FFmpeg to 1792 PCM bytes -- silence --
while the RIFF sub-waves from the same banks decode to 150-270 KB. Byte
coverage was the wrong success metric and it looked like progress.
* It is not narrow. The rule matches 1524 of the 8021 RIFF-bearing entries in
sound.pak, including RT* movie banks that decode correctly today. Landing
it would have risked a wide regression in order to not-fix five banks.
to_xma_riffs is back to its previous behaviour, verified by re-measuring:
coverage is 5.4% / 9.7% again. The refuted attempt is recorded in the code
beside the branch it would have changed, so the next person does not
re-derive the arithmetic and re-make the change.
XMA1_PACKET is kept as a named constant because the blast-radius scan uses
it. Artifacts: examples/voice_bank_shape.rs (structure), voice_bank_dump.rs
(sub-waves for decoding), slb_hybrid_scan.rs (the 1524 count).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The record table gives a DIRECT binding hokyu_DS_s13A -> VOICE_D_452, where
the corpus records the movie as unbound and movie_manifest_disc.rs asserts
None, citing an in-game verdict that this exact value was "the wrong
recording". That is the only place on the disc where a runtime observation
disagrees with the record table, so it was worth settling.
First, shape: these banks are SHARED. Five slots bind VOICE_D_452, five bind
451, four 450, four 453, three 454 -- 21 hokyu slots over five banks, and the
movies repeat too. Generic resupply cutscenes, not per-stage recordings.
The recorded explanation for 453 decoding to 0.14 s and 454 to 0.43 s was
that the banks are "likely multi-subwave / not cleanly sliced". Refuted: the
count of RIFF magics EQUALS the number of sub-waves recovered in all five
banks, and the last data chunk ends exactly at EOF in four of them. Nothing
between or after sub-waves is being missed.
The real defect: slb::to_xma_riffs finds audio by searching for the RIFF
magic, and a large region PRECEDES it. 87% of VOICE_D_453 and 85% of
VOICE_D_454 sit in front of the first RIFF -- 21-27% zero over 256 distinct
byte values, i.e. content, not padding. VOICE_D_451 is the control: its
leading region is 100% zero, 1 distinct value, real padding.
So the in-game verdict listened to a decode that had discarded most of the
bank, for exactly this bank class. It is evidence about the decoder, not
about the mapping. Note also that what was rejected was a value INFERRED
from a shared demo id; the record table supplies the same value as a stored
field, and only the inference was ever tested.
This does NOT establish the binding is right -- it removes the only recorded
evidence against it. What the leading region actually holds is undecoded, and
confirming the binding needs a human listening.
Artifact: examples/voice_bank_shape.rs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Last iteration I said launch_mission died because skip_intro only tests for
the title on a static frame, gated at rmse <= 1500, and that run logged 1503
and 1549 just above the cut. I also said the fix was NOT to nudge 1500 but
to measure both signals through a boot first. Measured, and the diagnosis
does not survive.
boot_trace.sh logs the two signals skip_intro decides on -- frame-to-frame
RMSE and the is_title.py green-glyph count -- through a clean boot with no
presses at all. One run, 29 samples over 484 s:
8 samples had rmse <= 1500, so the gate OPENED eight times
0 samples had glyph > 0, so the title was never seen
At t=145 s the RMSE was 1205, comfortably inside the threshold, and the
glyph test was called and answered zero. A frame can be perfectly static
without being the title -- the intro movie has long quiet stretches, three
reading RMSE exactly 0. So 1503/1549 were almost certainly movie frames too,
and raising the constant would have admitted two more of them.
What is left is narrower and honest: the interactive title never appeared,
rather than appearing and being missed by a threshold.
The limitation is recorded rather than buried: the tracer intended 1 s
sampling and achieved 16.9 s, because each iteration forks two screenshots,
ImageMagick compare and a fresh Python. So this does NOT prove the title
never appeared -- only that it was absent from 29 samples. A window shorter
than ~17 s falls between them. The recorded next step is to make the tracer
sample at the rate it claims before concluding anything stronger.
Artifact: docs/re/captures/boot-signal-trace.tsv.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Third independent line for yesterday's built-in 100 rename, from the callee
this time. sub_8226E3B8 was labelled "push", which is what made built-in 100
look like push_trigger. It reads the element count, returns immediately when
the container is EMPTY, and otherwise walks the node list splicing nodes out
until it is empty. A push links one node; this unlinks all of them. It is
clear(). The append is sub_8226E160, reached from built-ins 19 and 25.
So the rename now rests on the handler, the usage (all 12 Stage 02 sites sit
in the phase terminator next to timer_stop / clear_flag(-1) /
MARK_LAST_PHASE), and the callee.
The dynamic half did NOT run, and the write-up says so. phase_watch.py now
samples [phase+272+20] (triggers queued) and [phase+216+8] (coroutines
alive) so a phase terminator's effect on the VM is visible in one line —
written here, never yet exercised against a live guest.
Boot-nav could not reach the title in 381 s. Diagnosed rather than retried:
skip_intro.sh only runs the title test on a static frame, gated at
rmse <= 1500, and this run measured 1503 at 104 s and 1549 at 139 s — just
above the cut — so is_title.py was never called and the one allowed press
was never spent. Recorded in BACKLOG with the explicit instruction NOT to
raise the constant: the first step is to log rmse and the glyph count
through a whole boot and look at the two distributions, because tuning a
threshold to make one run pass is fitting to a single sample.
Also reaped a stale lock: a gdb orphaned 2h14m earlier was holding
/tmp/xenia-canary.lock with an already-defunct emulator child.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
With the record table decoded there is finally a ground truth to check the
old string-pool reader against. It infers `key -> value` from pool adjacency,
which is a consequence of how records are written, not a rule of the format.
Verified by hand against the disc, with an independent parser:
* `FCSRange` = 500000.0 — the module docs' own canonical example of a field
"left at its default" that "omits the value string".
* `ShieldRatio` = 1.0, where `tests/pak_idxd_disc.rs` asserted None with the
comment "a defaulted/omitted field must be None". That test encoded the
false belief; it now keeps the None as a deliberate characterisation of the
legacy reader, with the true value asserted beside it.
* `get_raw("Model")` on GP_HANGAR_ARSENAL returns the first record's model for
every record — silent corruption, not an absent value. New test pins four
records that disagree with it.
The cause is the flat API having no way to name a record: only 548 of 6325
objects have one. `HP` on the DeltaSaber answers 1000.0, the hull, while 63
Turret_* records each carry their own 100.0 (measured — a first draft said 34,
taken from a report rather than from the disc).
Disc-wide rates are recorded as single-source and labelled as such: get_raw
52% wrong, typed getters 38% miss, but 100% correct on single-record objects.
Also records a negative result: the 504 unnamed field keys were NOT recovered.
A 572464-string dictionary and 73191 variants gave 0/42. The key deltas do
prove the preimage ends with the two decimal digits.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The binary region in front of the string pool was the parser's oldest open
note ("Not yet decoded"). It is a uniform 16-byte record array sorted by
name hash, a field count, a 12-byte field array sorted by key, a pool size,
and the pool. The trailing `pool_size == file_len - pool_base` identity makes
the layout self-checking, which is what caught the first wrong version.
Verified over the WHOLE disc with zero failures: 7750/7750 IDXD objects,
190782/190782 records reproducing their stored tag_hash, 1271462/1271462
named fields reproducing their key. IXUD is the same container with
ixud_hash, UTF-16BE and every offset in chars — 1104/1104 objects,
628165/628165 fields, checked with an independent parser.
Field names are stored on disc, so no preimage search is needed: a field's
middle word points at its own name. Only 504 fields disc-wide are hash-keyed
with no name; the other 1485073 nameless fields are positional, keyed by a
literal integer (line slots, movie ids).
Two long-held beliefs are WITHDRAWN:
* The word at 0x08 is not a schema hash. It is record 0's name_hash — the
format has no type field at all, and an object's kind is known only from
the caller that loads it. It survived as "schema" because tables of one
kind share their lowest-hashed record name. Caught by a test asserting
every movie id names a real record: 1005 -> STAGE10_PHASE01 failed because
tag_hash("STAGE10_PHASE01") IS 0x067025B9, that table's supposed schema id.
* The field's middle word is not an always-0xFFFFFFFF flags word. It is
0xFFFFFFFF for 54% of fields, enough to look constant in a small sample;
the tell was that it is constant per key ACROSS records, which a per-record
flag cannot be but a per-name pointer must.
`schema_hash` keeps its name rather than churn 33 call sites, with corrected
docs. The first sweep globbed dat/** and missed hidden/DefTables.pak (1425
objects); the test now walks the whole disc root.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Re-extracting the same gdb capture per thread rather than by grep qualifies the
previous entry. Of eighteen threads whose frame 3 is XObject::Wait, eight have
[rbx] equal to the XEvent vtable plus sixteen, two equal to the XSemaphore
vtable plus sixteen, and eight hold a pointer into the mmap region that is not a
vtable at all. So the waits are on two distinct kernel types, XEvent and
XSemaphore, and the earlier claim that the object is an XEvent was right for the
majority but not the whole picture.
The eight non-vtable readings are the method checking itself rather than a
failure. A polymorphic object's first word is always a vtable pointer, so those
values are simply not this: rbx could not be restored for those frames and the
unwind returned whatever the register held. A reading counts only if [rbx] lands
in the binary's vtable range and resolves to a "vtable for" symbol; anything
else is discarded rather than interpreted. Ten of eighteen resolve and the rest
are honestly unknown.
That also settles the previous entry's worry that the 0x7ffc and 0x7ffd
addresses looked like stack. They are the shared mmap region, which holds thread
stacks and large allocations alike, so the vtable check rather than the address
range is what separates an object from a stack slot.
The follow-up run that would have added /proc/<pid>/maps classification and a
wider object dump never booted -- EMULATOR GONE at 0s, skip_intro exit 4 --
most likely a stale emulator or lockfile from the preceding gdb session, whose
process tree is parented differently and escaped the usual cleanup. So the map
classification, the multi-word object dump and the frozen-state capture are all
still unrun.
Executed the wait-object read on a live run under gdb. For every thread whose
frame 3 is XObject::Wait, rbx gives a pointer whose first quadword is
0x5555562db8f0, which is the PIE base plus 0xd878f0, and the symbol table has
vtable for xe::kernel::XEvent at 0xd878e0. The stored pointer is the vtable
symbol plus sixteen, since offset-to-top and RTTI come first, so it matches
exactly. The same vtable appears on every sampled waiting thread while the this
pointers differ, meaning many threads waiting on different XEvent instances. The
whole chain needs no DWARF and no rebuild, as the static groundwork predicted.
Two caveats, both recorded rather than smoothed over.
This is a healthy-play snapshot, not the freeze. The capture landed at 195 s of
flight with screen_id reporting flight and a non-black mean, because boot under
gdb costs about 300 s and the entire experiment has to fit inside one call --
a timeout kills the process group and takes the emulator with it, which lost an
earlier attempt outright. So this describes what threads wait on during normal
play, which is the control the frozen capture never had, but it is not the
frozen case.
And the this addresses look like host stack rather than heap. Either xenia
places these objects somewhere unusual, or rbx at frame 3 is not Wait's this
after the unwind and the vtable match is coincidence. An exact plus-sixteen
match on a known symbol is hard to get by accident, but the address range is not
what was expected. The settling check is to dump a few words at rbx and see
whether they look like an XEvent -- vtable, KernelState pointer, handle and type
fields -- or like saved registers.
