Closes the open question left by the take-2 audio capture, where the probe
logged five streams on one boot when only ADV s three were accounted for.
Both unexplained sizes are whole packet counts, 562 and 620. Searching every
inter-descriptor span of the voice stream and every sound.pak entry large enough
finds zero hits in the voice stream and ONE entry carrying both -- hash
9799c546, which candidate enumeration recovers as BGM_102.slb, two streams of
1150976 and 1269760 B. One entry holding both sizes is the two-stem shape rather
than two coincidental matches. So the boot s five streams were ADV s three voice
streams plus one music bank s two stems, and nothing is unaccounted for.
What it does not establish is which screen it belongs to. The window ran from
launch to t=253 s with the title arriving at 262, so BGM_102 was decoded
somewhere inside a launch-to-just-before-title window -- but the probe fires on
first decode and its lines carry a thread id rather than a timestamp, so a title
BGM decoded moments before the title appears is equally consistent with the
evidence. Cue 1103 is already the main menu, which makes 1102 as the title at
least suggestive. The settling experiment is written down and not done.
Refutation attempt on HANDOFF s BGM_103 wave sizes: exact match on both
(3876864 / 3930112). The claim survives unchanged.
Also a third route to two-stems-of-identical-duration, from the XMA1 header
alone now that PsuedoBytesPerSec is read correctly: BGM_102 37.487/37.487,
BGM_103 87.750/87.749, BGM_001 173.821/173.821. The one apparent disagreement
resolves in the corpus s favour -- BGM_001 reads 173.821 here against the port s
decoded 167.663, a gap of 6.158 s, and HANDOFF already records 6.15 s of trailing
silence after its fade-out. Declared duration covers the encoded stream including
that silence.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QsEPXWVaEpyfudtR6re1Pd
parse_riff_wave read every fmt chunk as a WAVEFORMATEX. XMA1 (tag 0x0165) is
not one, so audio info reported the disc s movie voices as 16 channels,
4310 Hz, 2-bit: 16 is wBitsPerSample read as a channel count and 4310 is
wEncodeOptions (0x10d6) read as a sample rate. This misled me earlier in the
session and I recorded it as a limitation before finding the cause.
XMA1 carries XMAWAVEFORMAT followed by one XMASTREAMFORMAT per stream. The
reader now branches on the tag and reads bits at +2, PsuedoBytesPerSec at +12,
SampleRate at +16 and Channels at +29. The same three files now report 2
channels, 48000 Hz, 16-bit.
The consequence worth having: this crate has no XMA decoder, and
data_bytes / PsuedoBytesPerSec is the only route to a duration. Checked against
durations decoded independently by the port:
ADV presentation 1 137.34 s declared 137.324 s decoded +0.012 percent
ADV presentation 2 137.33 s declared 137.324 s decoded +0.004 percent
S00A presentation 1 93.71 s declared 93.694 s decoded +0.017 percent
So the corpus can now get XMA1 durations off the disc without a decoder, which
is a capability I had written down as absent. It is a declared rate rather than
a measurement of the samples, and the CLI labels it as such.
Regression test pins the real on-disc header bytes and asserts the duration
against the independently decoded 137.324 s. 115 lib tests and 3 media disc
tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QsEPXWVaEpyfudtR6re1Pd
The port chose a voice presentation on the argument that ADV chunk 1 is
mono-in-stereo and chunk 2 is dual-mono, so chunk 2 s extra bytes encode a
duplicated channel rather than fidelity -- which would explain its higher
declared PsuedoBytesPerSec without appealing to encode quality.
Their ADV channel measurement stands. The generalisation does not. If stream 3
were systematically the same take with its channel duplicated, its size ratio to
stream 2 would be tight across the 28 three-stream cues. Measured:
min 0.0778 (S00A, the silent one)
median 1.2565
max 2.9163 (S06A)
sd 0.5057
within 15 percent of 1.0: 12 of 28
A 37x spread is not a duplicated channel, and the declared rates scatter with
them -- S06A is 5661 against 16513 B/s. Whatever distinguishes the three streams
varies per cue rather than being a fixed channel-configuration triple.
This does not touch the port s decision, which is to take the loudest
presentation: that is a per-asset content measurement, not a structural rule, so
a scattering ratio cannot undermine it. It touches the explanation, which should
not harden into a fact about the format.
Two curiosities recorded: S12B s three streams are byte-size identical at 14396
each, and BIRD_224 is 3-stream while being a non-movie cue, so the shape is not
exclusive to cutscenes.
Also narrows the settle-time page s own generalisation. The port measured its
boot the way this corpus measured the game and found the sequencer NOT late --
its 0.6 s discrepancy was arrival-to-arrival timestamps compared against visible
spans, the plate-delay trap in a second place. So what is supported is that
rest.t is the wrong landmark for the TITLE, not that everything paced off it is
late. And the offered re-take of the one-run menu figures is recorded as
declined, with the reason, rather than left looking unfinished.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QsEPXWVaEpyfudtR6re1Pd
Closes the last open question on the voice regions: why one cue s byte span
decodes to ~2.6x the movie s length.
The port measured, with controls including a cross-movie negative, that a
region s leading chunk is the TAIL of the full-length chunk that follows it --
r = 0.998 at a lag that puts it flush against that chunk s end, residual 16.7 dB
down over 84.5 s. They withdrew their own earlier 0.768, which came from a
search that scored best on the boundary of its own lag range.
Checked it here by an independent route that needs no decoder. If the leading
chunk is the tail of a full-length first stream, the whole leading stream should
be one complete take of chunk 1 s duration. For ADV: 504464 + 808304 = 1312768 B
at chunk 0 s byte rate of 9559.7 B/s is 137.323 s, against chunk 1 s measured
137.324 s. One millisecond over 137 seconds, from byte rates rather than from
envelope correlation.
And the byte structure settles the shape disc-wide. Counting stream starts inside
every inter-descriptor span: 258 hold exactly 1 stream, 28 hold exactly 3, and
nothing holds 2 or any other number. All 20 spans over 1.5 MB are 3-stream. The
95 movie regions decompose 70 + 8 + 17, and the 8 are independently the same 8
the first census found as bank-header-with-3-chunks.
So 359 s = 84.55 + 137.32 + 137.32: three presentations of one take, the first
clipped by resolve_movie_voice_region s own 1.5 MB guard.
Consequences recorded for the port: dropping the leading chunk is removing a
duplicate rather than truncating, so the hedge is lifted; but summing chunk 1
and chunk 2 is wrong, because they are the same take at different gain, not two
stems. Take one stream.
Also flags a coincidence I nearly built on: the 504464 B constant is structural,
not proportional -- ADV s proportional prediction lands within 8 bytes of it and
S00A s is 4305 B out.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QsEPXWVaEpyfudtR6re1Pd
My own leading hypothesis -- that the leading chunk is an in-mission VOICE_D_*
line -- is refuted, on the route the port suggested: widen the enumeration past
the 95 manifest-bound movies and the byte-span test settles it without anyone
listening.
Scanning the stream for every trailer descriptor (the (id, 0x11) pair whose id
repeats at +0x800) gives the complete cue partition, mission lines included:
287 descriptors in a 116.2 MB window, all 287 carrying an id the 4280-name
registry names. Every one of the 17 leading spans is bracketed by
desc(N-1)..desc(N) where desc(N) is that movie s OWN cue id. Zero mission lines.
The mechanism is a guard in our own resolver. resolve_movie_voice_region takes
the predecessor trailer as the region start, guards it with
end - start < 1_500_000, and falls back to the .slb TOC anchor when that fails.
Cues with a true span over the guard: 17, of which 17 are stream-opening. Cues
under it: 78, of which 0. Perfect discrimination both ways. The anchor sits a
constant 504464 B after the true predecessor trailer on all 17, which is
unexplained.
Not established, and stated as such: this does NOT mean the export truncates N
seconds. The port s decode already has ADV s region at 359 s against a 137 s
movie, so it over-covers and the byte-to-time mapping is not linear. No XMA1
decoder in this container to check.
Also withdraws a claim this page had adopted from the port -- that chunks 1 and
2 are two stems of one performance. The port refuted its own claim by decoding:
S00A chunk 2 is digital silence, ADV chunk 2 is 0.60x chunk 1 with the residual
26.8 dB down. Equal duration was a shape match and Q10 s music census should not
have been carried across to voice on it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QsEPXWVaEpyfudtR6re1Pd
The port hit a 2+1 chunk signature on a resolved movie-voice region and asked
whether the bank-header explanation that closed HANDOFF Q10 also covers it,
rather than assuming it. It does not, and the discriminator is mechanical.
Disc-wide over the 95 English movie-voice regions the manifest binds:
78 open with a bank header -- bank_header_len fires, 10240 B = 5 packets
exactly, every time. That is the BGM case.
17 open with a leading headerless stream -- bank_header_len is None, and all
17 have length congruent to 1392 mod 2048, the disc s own derived data
offset. No other residue occurs.
0 begin at a RIFF.
Counting chunks does not discriminate: 8 bank-header regions also yield three
chunks. slb.rs already predicted this in its own doc comment -- the header
signature has "zero false positives on the 7993 mid-bank windows, where the
leading region IS real" -- and a voice region is a mid-bank window by
construction.
Also tested the obvious defence of dropping the leading chunk, that it is the
predecessor cue s audio: 0 of 17 leading spans lie inside any other resolved
region, 0.0 percent on every one. The test finds overlaps where they exist (16
overlapping pairs among the regions, 60 exactly-adjacent boundaries, 73 of 78
bank-header regions starting where another ends), so the zero is not the
instrument.
Left open, with reach: the census covers movie-voice regions only, and the same
stream carries the in-mission VOICE_D_* cues, which are not enumerated -- the
leading bytes plausibly belong to one of those. Could not be settled by
listening: no XMA1 decoder in this container, and sylpheed-cli audio info
reports these chunks as 16 channels / 4310 Hz / 2-bit, which is visibly wrong.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QsEPXWVaEpyfudtR6re1Pd
The census bounded WHERE a wrong tie-break could show and said outright that
nobody had measured how many of those pairs change a pixel. Measured.
`compose_with_order` renders a bundle in a caller-supplied order; the new
example renders each screen twice, once derived and once with one tied pair
swapped, and diffs. Same-key elements are contiguous under a stable sort on
(key, i), so a swap paints nothing else in between.
Controlled per entry: swapping an OVERLAPPING pair with DIFFERENT keys moves
36 305 to 771 479 px (max delta 254). Where no such pair is drawn the output
says so rather than reporting an uninterpretable zero.
* EXTRAS (entries 6/9) and the main menu (5/8): 0 px. The tied ptframe pairs
ink ~3 600 px each and share NONE of them -- the 102x132 rect overlap was an
artefact of approximating an element as pivot x 2. Blend-independent.
* Across all 31 drawable overlapping tied pairs in GP_TITLE, the largest change
any of them makes to any channel is 3/255.
* Withdrawn: "a wrong tie-break can be wrong by a whole layer". That rested on
ptlogo_back2eff5 geometrically containing two other glows. Rendered, the swap
moves 6 390 px by max delta 2. Containment is not occlusion when the container
is a near-transparent glow, and nobody had rendered it before asserting it.
Reach: this measures our compositor's sensitivity to order, not the game's. The
zero-shared-ink results hold under any per-pixel blend; the delta<=3 figures
assume ours.
13 disc-gated ui_paint_order_disc tests and 114 lib tests pass unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QsEPXWVaEpyfudtR6re1Pd
The Cutscenes window printed the voice token as text and offered no way to play
it, which left the most confusing thing on the disc invisible.
The movie voices are one continuous XMA stream chunked into VOICE_*.slb entries
whose boundaries do NOT match the cutscene cues, so the bank named after a movie
need not hold that movie's audio. Measured, on the retail disc:
ADV region 433930240..437044592 inside VOICE_ADV.slb name honest
S00A region 452798464..455499120 inside VOICE_S00A.slb name honest
RT01A region 437044592..437345648 inside VOICE_ADV.slb NAME LIES
RT01A's voice sits in bytes belonging to the entry named after the intro movie.
A viewer that played the name-matched bank would be confidently wrong for
exactly the cutscenes where it matters, and would look right on the two that are
easiest to check.
So the window now shows BOTH locations -- the named bank with its byte range,
and the resolved region -- and states plainly whether the name is honest,
highlighting it when it is not. Play routes through the movie form of
RequestAudio, which resolves the region rather than reading the bank.
Static data only: sound.pak and tables.pak, both on the disc.
The placement region is `frames` records of `{u32 time; 36-byte pose}` after an
8-byte header, so the time word PRECEDES the pose it belongs to. Our parser's
40-byte window opened at the pose, four bytes into the record, and then read the
word at its `+36` as that pose's time -- which is the NEXT pose's. Every pose
field was right; only the time association slipped by one.
Two things the corpus has carried for weeks are that off-by-one and nothing
else: "a group's data stops 4 bytes short of its final block's time slot", and
"the last keyframe carries no time". The group is not short (8 + frames*40 is
exact) and no time is missing -- the first pose's time is the lead-in word at
`header + 8` that `parse_placements` skipped without asking what it was.
Disc-wide, 33 archives, 13 991 groups, each test with a control:
A lead-in prepended to the shifted times is non-decreasing 13991/13991
B a non-zero lead-in is strictly below the next time 5058/5058
control (another group's lead-in, same bundle) 70.9%
C multi-segment alpha ramp at a constant rate, corrected 857/1540
the same, under the old reading 0/1042
C is the one that cannot be argued with: interpolation between keyframes is
linear, and under the old reading not one multi-keyframe ramp on the disc comes
out at a constant rate.
Adoption is free on every static composite, which is what the corpus previously
declined it over. `SYLPHEED_KF_TIME_SHIFT=1` moved GP_TITLE build 7 by 13.1% of
its pixels because it left pose 0 untimed; with the lead-in restored, all 12
GP_TITLE builds render byte-identically, and across 217 builds in six archives
only two elements pick a different rest pose -- both times between two poses
that are equally invisible.
`SYLPHEED_KF_TIME_SHIFT` is gone; `SYLPHEED_KF_TIME_LEGACY=1` restores the old
reading for A/B work.
ui_header_time_disc needed one line: 546 bundles whose every group is a single
static pose now report max_time = 0 where they previously reported no time at
all. Excluding them, the result it guards strengthened -- the bound holds over
2 859 bundles instead of 2 313, still with zero violations.
Not established: the executable's own parser. Reach is written down.
docs/re/ui-keyframe-record-layout.md
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nsxw1A9JseUw99Yw1ZRQzY
The port hit `sound_bank_riffs("BGM_103.slb")` returning three against a census
that says two, and refused to guess which to drop. It was our reader.
`to_xma_riffs`'s hybrid branch derives a leading packet stream's start as
`first_riff % XMA1_PACKET`. That is right only when the bank header is smaller
than one 2048-byte packet -- true of the voice banks the branch was written for
(1392/1468/1600/1728), false of a music bank, whose header is exactly five
packets. The modulus returned 0 and the whole 10 240-byte header was emitted as
sub-wave 0.
The header states its own length, so the guard needs no threshold: BE u32 0x800
at +0x18 with the bank id repeated at +0x00 and +0x20, header length in blocks at
+0x24. Disc-wide over sound.pak's 9 519 entries, 28 match at offset 0 -- every
music bank, ids 1001-1023 and 1101-1105 -- and on 28/28 the declared header ends
EXACTLY at the first RIFF. Zero have a gap, so a header and a leading packet
stream never coexist here; zero false positives among the other 9 491.
Controlled rather than argued: decoding the emitted region through the same
chain, on the same bank, in the same run gives 0.009 s of PCM where the bank's
real wave 0 gives 87.744 s against a declared 87.75. The region is also 99.1%
zero bytes. And the oracle had already said two -- the XMA probe at the main menu
saw exactly two streams, at BGM_103's two declared wave sizes.
BGM_106-109 are deliberately NOT in the 28: their entries start mid-bank, so they
have no header at offset 0 and their leading region is real audio. The
VOICE_D_453 recovery is untouched and its tests still pass, 10/10 green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014voBspJ6kFncNErZJuZcLw
Merges the Godot port into the reverse-engineering repository, preserving both
histories -- 1019 commits of corpus plus the port's 31, brought in by subtree
merge and then moved into place so git can follow each file across the rename.
The reason is not tidiness. The two-repo split forced the exporter to depend on
the decoders by pinned revision, and that created a whole class of failure that
now disappears: a sha reachable only from a topic branch, orphaned by a
squash-merge, breaking a fresh checkout silently at build time. It also forced a
live read-only mount of one agent's working tree into another's container, which
is why a contract file could move mid-iteration. With a path dependency, a
decoder change and the exporter change it requires land in the same commit or
not at all.
Canary stays separate: it is a fork tracking upstream.
New structure for the long term:
docs/game/ how the game is NAVIGATED -- menus, modals, prompts, alerts,
and in-game flight. Written so nobody rediscovers it. Mostly
open questions on purpose; the in-game tutorials are the
resource for the flight half.
docs/port/MODDING.md
modding as a constraint on the exporter TODAY, not a later
feature: one logical asset in one file (the disc splits nearly
everything, and resolving that is the exporter's job), names a
person recognises, PNG/OGG/OGV/JSON only, base-and-overrides so
re-exporting is always safe, provenance in every file.
data/base + data/mods
generated tree and drop-in overrides, both gitignored
exchange/ transient inter-agent files, deliberately outside history
docs/agents/ the team protocol
Both the README and the navigation doc lead with the correction that cost the
most: the oracle is the real game under Xenia Canary. Reborn's renderer is a
hypothesis under test, it has been wrong, and treating it as ground truth
propagated into three documents and both agents before a human caught it.
Scripted modding stays possible without being built: no screen name is hardcoded
in GDScript and there is no native code in port/, which is what Godot Mod Loader
needs to be able to substitute behaviour later.
Three things, all from parsing ptbtn0Nf.rat as a build.
1. THE RING SPINS. Its two keyframes differ in exactly one field: rotation_deg
ramps 0 -> 360 with position, scale, alpha and tint all constant. A spin in
place, the same shape as the GP_BUNK example already recorded.
2. THE ORACLE CONFIRMS THE GAME RENDERS IT. In the OPTIONS-focused capture the
ring's bright head sits in a completely different angular position from the
sprite's own -- caught mid-spin. This is a SECOND independent confirmation that
rotation_deg is drawn, now on a different screen and a different element from the
ptloop sweeps, and it raises rotation's priority: it is not a title-only concern
that sits off-screen at rest, it is the main menu's focus marker.
NO ANGLE IS QUOTED. A brightest-region centroid says ~250 deg, but the control
refuses that precision -- rotating the sprite by a known 30/90/180/270 and
re-measuring gives errors up to 19.8 deg. What survives the error bar is that a
<=20 deg error cannot manufacture a ~250 deg displacement.
3. WHICH PLACEMENT WINS -- correcting this page's own earlier caveat, which said
to use the leaf only for elements the parent does not declare. Right for a BASE
record, wrong for an f record: the parent declares NO element for ptbtn0Nf.rat at
all (zero of build 5's 16), so the f record's placement comes from its leaf for
BOTH elements, label included. The label's (-7,-7) is load-bearing -- the f
sprite is 13px larger per axis and -7 keeps them concentric (535+96/2 = 583 vs
542+83/2 = 583.5). Corroborated against the oracle: the focused-minus-unfocused
region is x 505..703, and the leaf predicts a right edge near 707 where the
parent reading predicts 714.
Also exposes UiBuild::records (name -> (offset, size) of a nested .rat leaf).
Nested records were parsed into a PRIVATE map, so a consumer holding a UiBuild
could not locate a leaf's bytes at all -- which is exactly what blocked the port
from reaching the ring.
The port is forbidden from reimplementing media assembly and Static.slb is
exactly that case: no RIFF, no seek chunk, no XACT container, just a packed run
of whole 2048-byte XMA1 packets, so a wave is defined only by (offset, packet
count) and the header has to be synthesized. That step now happens once, in the
crate that owns the format, instead of in each consumer.
`slb::xma1_wave_riff` wraps raw packets; `media::se_wave_riff` looks the bank up
and reads just the packets asked for. Both reuse the existing synth_xma1_fmt /
build_riff, which are already byte-identical to what tools/re-capture/
slb_extract_wave.py writes -- so this is exposure, not a second implementation.
It reads a TARGETED range rather than the whole bank, and that is load-bearing:
Static.slb is the ONE entry of sound.pak's 9 519 whose declared extent runs past
the end of the extracted segments -- by exactly 616 768 B -- so reading it whole
fails outright on this extraction. Every cue we need is in the first few hundred
KB. Recorded rather than worked around silently.
Verified as an artifact, not a compile: all three cues decode through ffmpeg to
mono 48 kHz PCM at 0.533 / 0.344 / 1.016 s, non-silent (rms 2085 / 2985 / 4327,
peaks 29813 / 16973 / 32767). The refusal path is exercised in the same run --
an impossible packet count is rejected rather than returning a short stream,
because a truncated XMA decodes to plausible-sounding garbage.
Also adds docs/re/captures/ORACLE-CAPTURES.md: an index of the nine canary
framebuffer captures already in this repo, and a plain statement that THEY are
the reference and `screen render` is not.
The port needed ptbtneff01.t32's placement and was about to author it from an
eyeballed PNG measurement. It does not have to: a `.rat` leaf needs no new
reader. Its first 32 bytes have a bundle header's shape -- "RATC", 0x3c
declaration-entry size at +4, element count at +20, design 1280x720 at +24/+28 --
so ui_layout::parse_build reads it unchanged.
The control is the base record, whose position is known independently: the parent
screen reports ptbtn01.rat resting at (542,162), and parsing the leaf alone
returns ptbtn01.t32 at (542,162). It reproduces all five buttons.
Positions are absolute design-space top-left. The ring rests at (500, 156/236/
316/396/476) for buttons 1-5 -- a uniform (-42,-6) from each button's own rest,
identical in the Japanese bundle. The bright label is a uniform (-7,-7).
Two things recorded rather than smoothed over: a leaf's placement DUPLICATES the
parent's rather than being relative to it, and the two copies are not always
byte-equal (ptbtn04's parent says y=401, its leaf says 402) -- the parent is what
compose honours, so the leaf is the source only for elements the parent does not
declare, which is exactly the ring. And `screen render --focus` is blind to the
ring for the same reason the port's exporter was: el.focused is name-based on
top-level elements and neither walks into the leaf.
Closes the reach caveat the `opt ` name fix left behind: 60 of 18 002 RATC
children carry no `opt ` block, and it was not established whether they lack one
or sit past our 128-byte window.
Neither. They are not children. `examples/ratc_optless_children.rs` re-runs
`ratc::parse`'s own guards over the disc and reports which one fired: all 60 are
"tag beyond the window", none is rejected by length, gap or charset, none is
child #0, and all 60 live in six bundles of one archive. Within a bundle the
distances back to the nearest tag are an exact arithmetic progression, step
60 600 -- ten different records finding the SAME tag, because there is only one.
Reading a bundle directly: children 1..10 are equal-size T8aD blocks under a
single `opt ` name, `pb_f15_eg_anm.tan`. `.tan` is a FRAME SEQUENCE. One block
declares the resource; its payload is a run of T8aD frames.
Disc-wide, over all 18 718 `opt ` names in all 33 paks: a RATC bundle names
exactly six kinds of resource -- `.t32` 14 756, `.rat` 3 311, `.prm` 367,
`.tbm` 224, `.sbo` 54, `.tan` 6. Six `.tan`, ten frames each = 60, the entire
population with nothing left over. The negative is closed, not narrowed.
Consequence recorded but deliberately not fixed: `ratc::parse` over-reports
there, listing a `.tan`'s frames as anonymous children. Nothing in the menu
milestone reads a `.tan` -- it occurs only in GP_READY_ROOM, which S1 ruled
out -- so no screen the port draws changes.
Also a METHOD entry for this container OOM-killing `slb_leading_segment_disc`
under default test parallelism (SIGKILL, no assertion; 8/8 pass with
--test-threads=1).
`ratc::parse` named each child by scanning backwards for the last printable run
of bytes before its magic. The format states the name explicitly instead, in an
`opt ` block: `"opt " | BE32 len | name | NUL | 3 bytes | magic` -- the same
block `ui_layout::opt_link` already read for a button's focus link.
The scan agrees with it 17 918 times out of 17 942 and is wrong 24 times, every
one the same failure: the 3 trailing payload bytes are themselves printable and
beat the real name. For `pteff05.t32` those bytes are `38 41 58` = `8AX`, so the
full-resolution background of all five menu screens registered under a name no
element declares, resolved to no sprite, and `compose` dropped it through an
early `continue` that -- unlike the two arms above it -- records nothing. The
screen lost its background and `screen render` still reported "all resolved".
`8AX` was never a name. Docs that treated it as one are corrected here.
Disc-wide, and the control is the 17 918 the scan already got right: the `opt `
reading reproduces every one of them. Effect on the five screens is the
signature of the same art at twice the resolution -- mean brightness unmoved,
high-frequency detail x1.15..x1.30 -- which is what the separately-measured
`ui-8ax-fullres-background` result said the game draws.
Also closes a long-standing dangling reference: `pmbase.t32`, recorded as "on
the disc nowhere", is the `GP_STAGE_CLEAR` child the scan called `8AX`. RATC
sibling references now resolve 10 148 of 10 148.
Verified: 114/114 sylpheed-formats unit tests (including two new ones pinning
the `8AX` case byte for byte and the no-block fallback), and every disc-gated
integration suite in sylpheed-formats/sylpheed-cli.
The model's sharpest prediction, tested with its control. The draw log
says that on the developer splash the _eff glows are drawn on frames
94-115 and the logos on 116-211, so at the moment the reference capture
was taken EVERY glow is already finished -- including the two that have
plateaus and which rest_plateau therefore renders visible. Suppressing
them should help on the splashes and hurt where a screen genuinely
settles.
publisher splash +0.9604 -> +0.9982 +0.0377
developer splash +0.9659 -> +0.9980 +0.0321
title (control) +0.9500 -> +0.9480 -0.0020
main menu(control) +0.9460 -> +0.8544 -0.0916
EXTRAS (control) +0.9440 -> +0.8370 -0.1070
Both splashes jump to about 0.998; all three persistent screens get
worse. The control is what makes this a finding rather than a
coincidence: the same edit helps exactly where the model says it should
and hurts exactly where it says it should not.
So rest_plateau is not over-drawing in general -- it over-draws on
TRANSIENT screens. A plateau mid-animation means the element is held at
that point in the timeline, not that it is on screen once the screen has
settled. Where a screen settles, the held pose IS the settled pose and
the rule is measurably right.
And that answers the question left open several iterations ago -- what
"rest" means for a transient element. It does not mean anything: the
splashes never rest. A static composite of them can match a chosen frame,
and about 0.998 is what these captures' frame is worth, but the format
does not answer a question the screen never poses.
For the port: play the timeline for the two splashes, which the settled
keyframe timing now supports, and composite statically for title, main
menu and EXTRAS.
METHOD: an edit that improves one set of cases is only interesting once
you have shown it damages the cases where it should.
The shifted keyframe-time reading looked like it implied something
simple: the final pose is reached at a definite time and nothing follows,
so rest should just be the last keyframe and the plateau heuristic could
go. Tested by applying it to EVERY element:
title +0.9500 -> +0.6819 -0.2681
main menu +0.9460 -> +0.6416 -0.3044
EXTRAS +0.9440 -> +0.5745 -0.3695
publisher splash +0.9600 -> blank (zero variance, corr undefined)
developer splash +0.9643 -> blank
Refuted, and the failure supplies the model. A group is entry -> hold ->
exit, and the exit is the screen's DISMISSAL. While a screen is displayed
it has not reached its last keyframe; it is sitting at the hold. So
rest_plateau is the correct primary rule, and the last keyframe is the
post-exit state -- correct only once the screen is gone, which is why
applying it everywhere blanks the splashes.
This does not contradict the shifted reading. That reading says when each
pose is reached; it says nothing about the group being played to
completion while the screen is still up. The step between them was mine.
And it explains why last wins for the two plateau-less elements: an
element with no hold is a transient, it flashes and is over, and at any
settled moment it is gone -- which is its last keyframe. The draw capture
says the same independently: on the developer splash the _eff glows draw
on frames 94-115 and the logos on 116-211, so the glows are already
finished when the logos are up.
Three independent observables -- animation timing, static composites, and
the per-frame draw log -- now agree on one rule: plateau where there is
one, last keyframe where there is not.
METHOD: a blank render is a NaN correlation, not a low score, and that
NaN was the strongest form of the result; and when a model predicts
something the measurement refuses, suspect the step you supplied between
them.
Scored candidate rest-pose rules by rendering and correlating instead of
arguing, and both results correct something I had published.
First, the exposure. The guessing fallback is reached only by an element
that is plateau-less AND multi-keyframe -- a single-keyframe element
short-circuits at `match len { 1 => first }`. Per screen:
title (4) 24 elements 2 plateau-less 0 reach the fallback
main menu (5) 16 5 0
EXTRAS (6) 18 5 0
publisher splash (10) 3 2 1
developer splash (11) 7 2 1
So on the three screens the port cares most about, rest() never guesses.
That is why three different rules render builds 4/5/6 to identical
correlations -- the code is unreachable there, which I nearly read as
"the choice does not matter".
Second, where it does fire, the last keyframe is markedly better:
publisher splash dwell +0.9600 last +0.9982 maxalpha +0.9600
developer splash dwell +0.9643 last +0.9758 maxalpha +0.9643
That refutes my own earlier refutation. I had killed the last-keyframe
rule by arguing it makes palogo_anima_eff invisible while its two
siblings stay lit, which looked like an artefact. The capture says
otherwise: making it invisible is what improves the match. The sibling
symmetry was my expectation, not evidence.
Caveat kept in front: both captures are single frames of a transient
animation, so this fixes which pose matches THOSE frames, not which is
canonically at rest. Default unchanged -- better on both screens where it
fires and identical on the other three, but it would move 2 305 elements
disc-wide on two measurements. Reachable via SYLPHEED_REST_RULE=last.
Also confirmed: all 195 zero-scale rest poses are inside the corrected
2 305 ambiguous population; none is a single-keyframe element.
METHOD: score a rule where it can differ, or you measure nothing; and an
argument from symmetry is a prediction, not a refutation.
Two corrections from one experiment.
A keyframe group is entry -> hold -> exit, and the exit ends invisible:
on the five port screens the final keyframe is invisible for 21/24
(title), 8/16 (main menu), 12/18 (EXTRAS), 2/3 and 6/7 (splashes). So the
screen as seen is the HOLD, which is why rest_plateau is the right
primary rule and why "rest = last keyframe" would empty every screen.
That suggested a fix: an element with no hold has no representative pose,
so draw nothing rather than guess an endpoint. Tested through compose's
visible mask and correlated against the live captures:
title +0.9500 -> +0.6839 -0.2661
main menu +0.9460 -> +0.9037 -0.0423
EXTRAS +0.9440 -> +0.9094 -0.0346
Refuted on all three, and the reason invalidates a number I published. An
element with a SINGLE keyframe has no adjacent pair, so the plateau test
marks it plateau-less -- but its one pose is unambiguously its rest.
Suppressing those removes backgrounds and full-screen layers, which is
the title's -0.27.
no plateau (as published) 3 807 (24.57 %)
... single-keyframe 1 502 trivially at rest, not a guess
genuinely ambiguous 2 305 (14.88 %)
So rest() guesses for 2 305 elements, not 3 807 -- the figure I gave the
port overstated the defect by 65%. Corrected in HANDOFF and the page.
METHOD: a predicate over adjacent PAIRS silently misclassifies a
one-element list; and acting on a claim is a better test of it than
re-reading it -- this flaw survived a census, a write-up and a handoff
row, and died the moment the rule was used to change a rendering.
A resolve-check on HANDOFF's own rows. Q8 read "SE audio is undecodable
from the disc -- no XACT container exists anywhere". menu-audio-cues.md
retracted exactly that ("### Retracting 'cannot be extracted'") and
locates three cues in Static.slb that decode to PCM: d-pad move 0x1ec0
(4 packets), (B) back 0x0ec0 (2), (A) confirm 0x5d6c0 (6), all mono
48 kHz. The retraction landed in docs/re/ and the page the port reads
kept the superseded text -- the fourth time in this corpus.
Writing the rule down has not worked, so there is a tool now.
handoff_lint.py flags every HANDOFF line making a strong negative claim
that links a doc containing retraction language. First run: found the Q8
row, plus one benign false positive (Q3 links a doc whose retraction is
about a sprite count, not about the tie-break -- checked, and HANDOFF
repeats none of the retracted figures). The lint also caught its own bug
first: it reported existing docs as missing because it joined a guessed
repo root, so it now resolves links relative to the file as markdown does.
Separately, EXTRAS's paint-order risk narrows twice more. Of its 15 tied
pairs only 2 overlap, and of those, ptloop01 x ptloop02 are loop*
animations compose skips by default -- so exactly ONE tie can be drawn:
ptframe3 x ptframe4, overlapping 102x132 px. Against live-extras.png that
contested region correlates +0.9622, better than the whole frame (+0.9440)
and inside the range of regions where order cannot matter (+0.8502 /
+0.9903). Consistent with our order, not proof: correlation cannot see a
swap between locally similar art.
15 -> 2 -> 1 -> consistent is now the whole paint-order risk on the five
screens, and HANDOFF says so.
`compose` claimed the derived paint order "reproduces both measured
orders up to ties". That sentence was never measured and was stale by
one: there are three measured orders, not two. examples/paint_order_audit.rs
checks it.
main menu (entries 5, 8) derived == measured 0 inverted pairs
developer splash (11, 14) derived == measured 0 inverted pairs
title (entry 4) DIFFERS 8, all same-key ties
So the claim holds and the exception is entirely ties -- but two of those
ties are total occlusions, not near-misses. The tied family is the five
ptlogo_back2eff glows (key 32899); back2eff5 is 1133x280 and FULLY
CONTAINS back2eff3 (82,824 px^2 = 100% of the smaller) and back2eff4
(152,047 px^2 = 100%). Derived paints it on top of two glows it entirely
covers; the game paints it underneath. A tie-break by declaration index
can therefore be wrong by a whole layer. The title itself is unaffected --
it has a measured order.
The port's actual exposure, per screen: title, main menu and developer
splash all use MEASURED orders; the publisher splash is derived but has
ZERO ties, so it is fully determined; EXTRAS is derived with 15 tied
pairs of which only 2 OVERLAP. Two element pairs on one screen is the
whole risk, and that is what HANDOFF now says -- not the raw 15, which
would have overstated it 7x.
Reach stated: this compares the derived order against orders measured
from the game, not an independent derivation, so where no measured order
exists only the tie exposure can be checked. Overlap uses pivot*2 as the
element size at its resting placement.
Stale comment in compose corrected. METHOD: a hedge in a code comment is
an unmeasured claim; and count the cases that can bite, not the ones that
match the pattern.
blit() and fill_quad() both opened with
let sx_pct = if kf.scale_x == 0 { 100 } else { kf.scale_x };
so an element whose pose is collapsed to nothing rendered at FULL SIZE.
I first described this as "a 1-pixel sliver", reading the .max(1) in the
size arithmetic two lines below -- the guard above it meant .max(1) never
saw a zero. Read the whole function, not the lines you went looking for.
Control run before the change: if 0 meant "unset", something on the disc
would use it throughout. Nothing does.
elements with a keyframe group 15 493
at least one keyframe at scale 0 2 166
EVERY keyframe at scale 0 0
zero is a transient the element grows out of 1 762
ptlogo_eff3.t32 runs 0% -> 200%. An "unset" marker no element ever uses
throughout is not a marker.
Both functions now return without drawing when either scale is 0.
Reach: 24 of 24 renders byte-identical across GP_TITLE (all 16 builds),
GP_PAUSE_MENU and GP_OPTIONS -- additive on every screen the port needs.
195 elements have a guessed rest pose at scale 0 and 126 of those have a
non-zero alpha, so the old code painted them; all 126 are in
GP_READY_ROOM.pak, which S1 already declared a no-go. They are
tactical-map sprites at sx=0/sy=100, mid-horizontal-wipe, that the
coercion drew full-width.
So: a real correctness fix with no visible effect on the menu port.
Suite green, 122 passed / 0 failed across 3 suites at commit time.
Follow-up on last iteration's unadopted candidate (+36 holds the NEXT
pose's time, not its own). Two new results, pointing opposite ways, and
both are reported.
FOR, and calibration-free: the observed full-alpha hold : fade-out ratio
on palogo_gamearts is 83 : 13 frames = 6.38. The shifted reading predicts
8.00. The current reading predicts 0.25 -- off by 26x. With the glow's 2
units/frame fixed and nothing else free, the current reading says the
logo holds full alpha for 2.0 frames; the capture holds it for 83. This
is no longer the shape argument the candidate rested on.
Also for: rest()'s plain max-dwell fallback picks a=0 -- a transparent
pose, for a publisher logo -- under the current reading, and the visible
a=255 hold under the shift. Only the rest_plateau special case rescues
the render today, and that is the case the port agent reported a bug in.
AGAINST: rendering every build of six UI paks under both readings, 10 of
11 compared are byte-identical and one changes -- GP_TITLE build 7, the
Japanese twin of build 4, by 13.1% of pixels. Build 4, the one verified
against a live capture, is unchanged either way, so the single build the
shift moves is the one with no capture to adjudicate it. The proxy goes
against the shift: language twins are the same artwork, and build 7 reads
70.94 mean luminance as decoded against build 4's 71.41, but 76.32
shifted. Correlation does not separate them (0.6206 vs 0.6201).
These constrain different things -- timing versus pose selection -- and
rest() is a heuristic layered on the times, so moving the times moves its
tie-breaks. Adopting the shift means revisiting that heuristic in the
same change, with no build-7 capture to verify against.
Default UNCHANGED. Experiment reachable via SYLPHEED_KF_TIME_SHIFT=1.
Shifted-mode suite: 122 passed, 0 failed across 3 suites at commit time.
The rotated quads on the title screen come from the keyframe block after
all. The earlier negative -- "every GP_TITLE build 4 element has all three
angle words at zero" -- read the right bytes over too small a region: it
walked the top-level declaration table, and the rotated elements are the
nested leaf records ptloop01.rat / ptloop02.rat.
Confirmed against the framebuffer rather than against our own renderer.
The two records declare +12 = 30 and -45; the GPU capture submits their
quads at +30.26 and -45.28 degrees -- magnitude and sign, two different
values. Corroborated by shape in GP_BUNK 117ca14f, where +12 ramps
0 -> 360 with position, scale and alpha constant: a spin in place.
Identifying which draw it was needed edge lengths, not bounding boxes:
400x1076 and 400x1444 against pteff03/pteff03a 399x180 at the elements'
two different declared scales, 600% (1080) and 800% (1440). The same test
names three known-positives in the capture (ptlogo1, ptcopyright,
ptbtn00), so it passes its own control.
Keyframe gains rotation_deg plus unknown_4/unknown_8, carried rather than
dropped. NOT rendered -- ui_layout::blit is axis-aligned only, so the
reference renderer and the port will both draw these upright until a
rotating blit exists.
The census tool ships with the trap that broke its first version: nested
RATC blobs are not 4-byte aligned, so an aligned scan found 0/3 of its
own control blocks and missed 16 341 blocks. Disc-wide +12 is non-zero in
14.50 % of 83 862 blocks.
sylpheed-formats tests, SYLPHEED_DISC set: 131 passed, 0 failed across the
6 suites finished at commit time; the run had not yet completed.
title's rotation
Looking for where the rotated quads come from, the obvious candidates
were the three keyframe words at +4, +8 and +12 that ui_layout.rs
documents as zero.
They are not zero. Across 72287 keyframe blocks disc-wide they are
non-zero in 4.81, 4.56 and 15.82 percent of blocks, and read as signed
values clustering on 180, -180, 90, -90, 120 and 22 -- degrees. Three of
them, so plausibly rotation about three axes. I have marked that amber
because it is the shape of the numbers and nothing more; no observed
rotation has been tied to a value. The doc comment is corrected
regardless: "0 on every frame seen" was an artefact of the sample.
And they do not explain the screen I was chasing. Every element of
GP_TITLE build 4 has all three at zero, element by element, while the
game demonstrably submits rotated parallelograms there. So the title's
rotation comes from outside the keyframe data and is still unidentified.
One correction to my own last write-up, flagged rather than left: I
stated that the skewed draw IS the swoosh. It is the only skewed geometry
in the capture and the swoosh is the only diagonal element on the screen,
so the inference is reasonable -- but I never confirmed it by matching
the draw's texture or screen position to that element, and I should have
said so the first time.
residual is smaller than I said
Testing the candidate I raised last iteration rather than carrying it.
Blending bit-0x02 sprites additively moves every measure the wrong way --
whole-frame mean diff +0.55 to +1.04, swoosh-band mean +1.83 to +3.98,
band edge-correlation 0.6971 down to 0.5578. So the bit is real and
independent but does not select an additive blend. I reverted the
experiment and kept the word as T8adImage::flags, documented and not
acted on; the render is byte-identical to before.
Second refutation: the swoosh is not displaced. Shifting the band over
plus or minus 80 by 8 pixels peaks sharply at zero, 0.7342, falling to
0.22 at 24 px. So the pivot story is dead twice over -- inert at scale
100, and no displacement to explain anyway.
And I have restated the residual, because earlier sections overstated it.
The +16 to +34 band tiles I quoted were measured WITHOUT --primitives.
With the dim drawn the band's average is nearly right at +1.83; what is
wrong is its structure, tiles running -38.6 then +33.8 and cancelling.
Six candidates eliminated now and none confirmed.
One caveat I owe the port agent about the capture I gave them: it is at
t=4.0s, roughly 174 keyframe units into a screen whose elements have
keyframes out to t=600. I judged "settled" from mean luminance, which
cannot see a thin sprite still moving. It is settled for the bulk of the
screen and not proven settled for every element -- which is a live
alternative explanation for a structural difference in exactly the band
the sweeps cross.
METHOD: cargo build passing does not mean cargo test compiles. Adding the
field built the library in 1.48s and broke two test-only struct literals;
cargo test failed with exit 101.
VISIBLE, not when its pose repeats
The port agent's report was right about the defect and about which
elements it hits, and its proposed condition does not survive the case
the exclusion was written for.
Their test -- the final untimed keyframe has the same pose as the last
timed one -- is also true of pgptitle.rat, whose trailing run is two
identical transparent frames. Adopting it would erase the word PAUSE
again, which a committed capture disproves. What separates the two is
alpha: an exit fades the element out so its last keyframe is transparent,
while an element with no exit ends on the pose you can see. So
rest_plateau now accepts a trailing run exactly when it is visible.
Verified against a capture rather than against another renderer, which is
the point their own report made: ptframe1's rest moves from (620,108)
t=16 to (440,108) t=62; the changed pixels are 10082 in a bounding box of
x 440-839, y 108-577, which is exactly the 400x470 at (440,108) they
predicted; and correlation against main-menu-oracle.png over that region
improves from 0.9596 to 0.9748. The PAUSE wordmark is unchanged across
all three pause builds.
This also closes a question ui-paint-order-key.md has carried for a
while, that ptframe1 and ptframe2 rest at alpha 0 while the capture shows
the menu frame plainly.
One trap cost most of this iteration and goes in METHOD: CARGO_TARGET_DIR
is redirected in this container, so ./target/debug/sylpheed-cli is hours
stale and every render I made against it was old code. Byte-identical
before and after reads as "no effect" when it actually means "you ran the
old binary".
The full disc-gated test suite was still executing when this was
committed; the verification above is artifact-based.
The trickiest reading on the disc lived in the Bevy viewer: resolving a
cutscene's voice to a continuous byte REGION of the sound stream, because the
movie voices are one XMA stream chunked into VOICE_*.slb entries whose
boundaries do not match the cues -- a cue routinely spans two chunks, so a .slb
need not hold the track its name claims.
That put the logic most likely to be re-derived incorrectly in the crate least
likely to be reused. The Godot port's exporter needs the same answers, and there
must be one implementation of them.
New `sylpheed_formats::media` owns every case where the bytes of one playable
thing are not one archive entry: segment-spanning reads, multi-sub-wave banks,
and the voice-region resolution. Callers supply bytes through a `DiscSource`
trait, so the viewer keeps its ISO/directory abstraction and a headless consumer
gets `DirectorySource` for free.
The seam is deliberate: this module returns XMA RIFFs, not PCM. Decoding means
shelling out to FFmpeg, which is native-only and a policy decision for the
consumer -- everything up to "here are the bytes that belong together" is disc
knowledge, everything after it is a codec choice.
The four moved functions were previously untested; `tests/media_disc.rs` now
pins them, including the negative the corpus paid for -- an unbound movie must
stay unvoiced rather than borrow a neighbour's clip, which was tried and played
the WRONG recording.
The algorithm is unchanged, moved verbatim (same window sizes, same fallbacks).
The new disc tests pass; the broader audio suite was not re-run in this pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`movie_manifest` has been parsed since the movie-voice work and rendered
nowhere: it resolved a voice bank and that was all. So the only way to find a
cutscene was to hunt `.wmv` files in the ISO tree, where nothing tells you which
mission a file belongs to, whether it has subtitles, or what is said in it.
View ▸ Cutscenes lists all 104 manifest slots with mission/phase, kind, movie,
subtitle track, voice token and telop, and -- the part that needed no new
parsing, only a route -- resolves the captions to a readable TRANSCRIPT with a
language selector. Subtitles were previously burned into the video during
playback and reachable no other way.
Three negatives are shown rather than smoothed over:
* 5 manifest-bound movies have no `.wmv` (logo1-4 and an encoder test clip).
They are marked and get no Play button instead of one that would fail.
* 9 of 101 movies resolve no English transcript.
* the `.prt` telop overlay is named by the manifest and we have no parser, so
the reference is shown labelled "not decoded" rather than omitted.
`cutscene_catalog_binds_movies_and_transcripts` pins all of it against the disc
-- 104/101/99/99/22, the exact absent-movie list, 92 transcripts -- because a
browser that quietly dropped these would look complete and be wrong. The counts
independently reproduce docs/re/movie-subtitle-link.md.
Play routes through the normal FileSelected path, so the existing video player
handles it exactly as it would from the tree.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The library enumerator kept only names containing VOICE or \Briefing\, and read
eng\sounds.tbl unconditionally. So the Explorer could reach 4382 of the 9519
banks in sound.pak: no music, no jingles, no sound effects, and no Japanese
voice at all -- roughly half the disc's audio had no route to the UI.
`slb::list_audio_entries` now returns every named bank with the category its
path implies (Music / Jingles / Sound effects / Radio / Dialogue / Movie voice /
Briefing). `list_voice_clips` is that, restricted to the spoken categories, so
its existing test still guards the old behaviour. The 36 root banks carry no
language component and appear whichever table is read; the window gets an
English/Japanese switch that re-reads the other sounds.tbl, since the table name
IS the selector.
Two defects the decode found, both recorded in
docs/re/structures/sound-pak-contents.md:
* `Static.slb` -- the SFX bank -- declares 616768 bytes more than sound.p04
holds. Not our extraction: p04 matches the ISO's own directory record, and a
sweep of every pak on the disc finds this one entry over-running and no other.
It is the highest-offset entry, so its comp_size is an allocation size. A
short read is now allowed for the tail entry ONLY; any other overrun stays an
error, because clamping it would hide real damage behind a half-decoded asset.
The bank went from unreadable to 514 s of audio.
* the left-channel downmix was applied to everything. Right for voice (mono
content however stored), wrong for music (a real stereo mix, half of it
discarded). The caller now decides from the category.
35 of the 36 shared banks decode; JNGL_001 does not, and says so in the player
instead of the panel silently closing. Its payload is not a whole number of XMA1
packets from any known data offset, so it is likely not a plain headerless
stream -- written up rather than papered over.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The UI Screens window enumerated with `is_build` while the PAK browser's
inline preview composes anything `parse_build` accepts. So the browser drew
screens this window flatly refused to list -- most visibly `palogo`, the
publisher splash, which declares its sprites directly and has no `.rat`
layout child.
`compose_screen` now gates on `is_composable`, the documented superset (every
`is_build` bundle passes it), so anything the list offers is drawable and the
two presenters share one rule. A "Fragments" toggle widens the enumeration to
it as well, off by default: the extra ~1786 bundles are mostly two-element
fragments (a button beside its glow) that would bury the real screens. It
re-scans, which renumbers the display ordinal -- harmless, because the pak
ENTRY index is the locator and the stale catalog is cleared.
The toggle sets a `rescan` flag rather than self-sending RequestScreenCatalog:
a system that both reads and writes one event type is a B0002 panic at
startup, which is how the Save browser broke.
Also plumbs the last two ComposeOptions the CLI had and the UI did not --
`black backdrop` (what a framebuffer capture must be compared against) and
`primitives` (decoded, but paint order unsolved; hover text says so).
Verified: `screen render --all --build 13 GP_TITLE.pak` draws the SQUARE ENIX
splash, 2/3 elements, the third being the .prm quad. Workspace builds, viewer
reaches steady state, disc-gated suite 20/20 green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three compounding causes, found by tracing every place a `loading` flag is set
against every place it is cleared.
THE COST. `compose_screen` inflated EVERY entry in the pak and held every build
in memory at once, purely to index into the result -- then did it again on every
checkbox click, re-opening the ISO from scratch each time. But the catalog
already records each build's pak ENTRY INDEX, so the locator was there all
along: compose now reads exactly one entry. `RequestScreenCompose` carries
`entry` (the locator) beside `build` (the display ordinal).
`build_screen_catalog` had no budget at all, while the pak browser has capped
exactly this work since it was written. It now skips oversized entries and stops
at a ceiling.
That ceiling is 384 MB, not the pak browser's 64 MB, and the difference is the
point: GP_HANGAR_ARSENAL inflates past 160 MB and holds ~390 builds, so a 64 MB
cap would have quietly hidden most of them -- trading a hang for a wrong answer.
When the ceiling IS hit the pak is marked `truncated` and the UI says so.
THE LATCH. `poll_loader_channel` treated a disconnected channel exactly like an
empty one, so if a worker died every in-flight spinner stayed up for the life of
the process -- and the `if loading { return }` guard at the top of each handler
then refused every retry. Disconnect now clears the flags and reports it.
Verified: the workspace builds, the viewer reaches steady state, and one
composite still draws 11/11 elements of the tutorial pause menu.
A wave runs to data_at + declared_size, and there is seek magic exactly there
whose little-endian packet count at +12 times 2048 equals the declared size --
7620/7620 disc-wide. That is the decoder-independent boundary and the thing that
proves the declared sizes honest, so it should not be able to regress silently.
The boundary routinely lies outside the entry's own comp_size window, so reading
it needs the flat segment stream rather than the entry slice; PakArchive gains a
small data_at(offset, len) accessor for that.
Test walks a bounded slice to stay fast -- 792 banks in this run, all holding.
8 disc tests pass.
I read the seek chunk's packet count big-endian; it is little-endian at seek+12,
with size == 8 + 4*count. And a seek sits immediately AFTER its own data, so an
entry's first seek usually belongs to the PREVIOUS bank (implied start -25232
for D_452, -145988 for TCAF_608). I was comparing an entry's first seek against
its first data -- different waves by construction, which is why no reading lined
up.
With that fixed, the declared sizes are honest: every RIFF-bearing entry on the
disc has seek magic at exactly data_at + declared_size with count*2048 ==
declared. 7620/7620, zero failures.
VOICE_TCAF_608 is not truncated. Its Channels is 2 and I decoded it as mono;
read as stereo it gives 6520176 bytes = 33.96 s, agreeing with both length
signals in the bank (33.88 s from cumulative samples, 33.97 s from
PsuedoBytesPerSec). 170 of 8021 banks (2.12%) are stereo -- exactly the rate of
my 1-in-60 outlier.
This is the mono/stereo trap already documented on this very page, met from the
other direction: I had written 'at two channels every bank yields one frame' and
then spent several passes blaming missing data for a one-frame decode.
Code fix: to_xma_riffs built the leading segment with a hard-wired mono fmt. It
now reads Channels from the bank's first RIFF. 7 disc tests pass.
The ties needed a different signal, not a longer scan. Banks carry one: a seek
chunk sitting on a packet boundary, so seek_pos % 2048 IS the data offset. On
the 6033 labelled banks with a seek before their first RIFF, 6031 agree
(99.97%) -- better than the packet scan and structural rather than statistical,
so scan_data_offset now tries it first.
On the scan's 28 ties it resolves 26 correctly and 0 wrongly (2 have no usable
seek). Combined rule scores 7354/7358 = 99.95%, up from 99.62%. 762 of the 1495
RIFF-less banks carry a seek, so the signal exists where it is needed.
Also ruled out, since a wrong offset was this page's whole subject: the header
is not audio being discarded. Adding 0 to the candidate set, it wins 6 of 7358.
7 disc tests pass.
Two comments claimed it is 'honest per sub-wave'. Measured: 5296 of 7586 banks
declare more than the entry holds and none declares exactly what it holds, so
the existing .min(slb.len()) clamp is load-bearing rather than defensive.
Comment-only change; 7 disc tests still pass.
1495 banks carry no RIFF and take a separate path that hardcoded both the
offset and stereo. Across a random 48-bank sample there was NOT ONE where the
old stereo-at-1392 pair beat the best mono offset; median gain 184x, individual
banks going from 0-4816 decoded bytes to 180000-380000. Stereo shows the same
stop-after-one-frame signature already recorded for the leading segment.
With no RIFF the offset cannot be derived, so scan_data_offset picks among the
four disc offsets by XMA1 packet-header plausibility. Validated on the LABELLED
set -- all 7358 banks that do have a RIFF, where the answer is forced: 7330
correct (99.62%), and all 28 misses are ties on the top score, never a wrong
unique winner. Ties fall back to 1392.
The winning offsets also reproduce, by directory, the distribution measured
independently from the RIFF-bearing banks. jpn\etc splits 1468/1600, so path
alone is not sufficient -- which is why this is a scan and not a lookup table.
7 disc tests pass (build-reborn test -p sylpheed-formats --test
slb_leading_segment_disc, SYLPHEED_DISC wired up).
HEADERLESS_DATA_OFFSET is the value the offset takes in <lang>\etc\, not a
property of the format. The leading stream is a whole number of 2048-byte XMA1
packets ending at the first RIFF, so its start is first_riff % XMA1_PACKET.
Disc-wide that takes four values -- 1392, 1468, 1600, 1728 -- varying by
language and subdirectory.
Verified by decoding, not by arithmetic: on a random 140-bank sample with a
non-empty leading region, the derived offset yields more audio in 85, identical
in 54 (the eng\etc controls, where it must and does reproduce the old
behaviour) and less in 1. Median gain among the improved is 70x --
eng\Voice\VOICE_TCAF_592 goes 1506 -> 97152 bytes, jpn 2910 -> 127178.
This withdraws my own claim from earlier today that the Japanese banks were a
different undecoded layout. They are the same format with a different offset;
I had treated a constant derived from one subdirectory as a property of the
format. The same error was hiding the identical defect in 1873 eng\Voice banks.
ixud.rs now has an IdxdObject-shaped reader, IxudObject, and build_caption_text
reads captions as FIELDS instead of pairing them with whatever token follows in
the pool.
build_demo_text token adjacency 134 ids 537 lines
build_caption_text token adjacency 3721 8074
build_caption_text record fields 4085 8800 = all of them
Verified over the whole disc by tests/ixud_records_disc.rs: 1104/1104 objects
parse, 1476/1476 records and 628165/628165 named fields reproduce their
ixud_hash, 48 positional, zero failures. The header word at 0x08 is record 0's
hash, asserted per object -- there is no schema field, exactly as for IDXD. The
module doc described a 12-byte record directory and a "schema/type hash"; both
were wrong and are corrected.
I also have to correct my own number from the previous commit. "1.3% of the
game's text" counted OCCURRENCES: each family lives in 24-45 IXUD blocks and
the same key repeats across them. Distinct text-bearing MSG_* keys number 8800,
not 44579, and every one has the <id>_<page>_<line> shape. So the real coverage
was 537/8800 = 6.1%, and I overstated the gap about fivefold. Direction right,
magnitude wrong.
The DEMO control is the sharpest evidence for the change: token adjacency finds
537 lines there, the field reader 541. It was dropping lines even in the one
family it was written for -- which is why the test now asserts "must not lose
lines" rather than "must be identical".
Same lesson twice in one session: pool adjacency is a consequence of how
records are written, not a rule of the format.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
build_caption_text generalises the key parser from MSG_DEMO_* to all eight
families. The shapes are uniform and each family is 100% consistent with its
own: seven use MSG_<FAM>_<id>_<page>_<line>, and VOICE alone inserts a family
letter before the id.
ids lines
build_demo_text 134 537
build_caption_text 3721 8074
The DEMO family comes out identical through both readers -- 537 lines either
way -- which is the control that generalising changed nothing that already
worked. Pinned by tests/caption_families_disc.rs, along with VOICE ids keeping
their family letter.
But this does NOT close the gap, and the write-up says so: 8074 against the
44579 text-bearing fields the record-level scan counts is about 18%.
The reason is the same lesson this session already learned once.
build_caption_text pairs a value with the key that happens to follow it in the
raw UTF-16 token stream -- the adjacency heuristic that was wrong for IDXD and
is wrong here for the same reason. ixud.rs has no record/field reader at all.
The IXUD record table IS decoded and verified disc-wide (1104/1104 objects,
628165/628165 fields reproducing their key) and was simply never wired into
the crate.
Next step recorded: give ixud.rs an IdxdObject-shaped reader and read captions
as fields rather than adjacent tokens. The decode exists; only the plumbing is
missing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
A disc-gated test that fails if the loaders regress to one flat answer per
object. It checks the three structural claims that came with the migration
request, and one of them is overstated:
* ✅ MainMissionBonus IS per difficulty — 18 of the 24 missions pay three
different values, in a strict 1:2:4 easy:normal:hard ratio (all 24); the other
6 pay 0 at every difficulty. The flat reader returned the Easy value.
* ✅ RankScore_S/A/B/C/D repeat identically across the three Score_* records in
24 of 24 missions, so the single flat answer happened to be right. The
difficulty scaling lives in the earnings (CraftScore_Adjustment,
KillBonus_Maximum, …), not in the thresholds.
* 🟡 SpaceSize is *stored* per phase (three Phase_N records), but only **1 of the
24** missions varies it — the one that reads 250000/100000/100000. The other 23
repeat a single value (500000 ×1, 100000 ×18, 50000 ×4). "Phase_1 = 250000,
Phase_2/3 = 100000" is one mission's numbers, not a rule.
Also pinned: the Delta Saber's 63 Turret_NNN mounts at HP 100 each against a
TurretCount of 4; the SD-Battleship's per-component HP (bridge 10000, thruster
20000, shield generator 5000 at PowerRatio 0.25, hatch 100 launching
Squadron_Test2, anti-ship gun 1500) against a hull of 100000, and its 37 records
that define an HP; 418 hardpoints across the 23 capital ships, none missing HP;
131 weapons whose shell id always differs from the launcher id, 40 with a
ShellWake, and Weapon_NULL as the only one without a Shell.Power; and
fcs_range/shield_ratio/cruising_velocity/maximum_velocity/acceleration/
deceleration now set on 89 of 89 units, all of which the pool reader could miss.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Measured first, over GP_MAIN_GAME_E.pak, comparing every named-field read the
six struct loaders performed against the record table: 4435 reads, 2872 agreed,
**966 returned None for a field that has a value**, **596 flattened a field that
several records carry**, 1 was wrong (a weapon whose TargetType is the empty
string read back as the neighbouring token "Skip"). The prior report of
4453/2887/974/591/1 is the same picture; the small differences are definitional
(I count a read as flattened only when the records disagree).
Every read now goes through IdxdObject::record, and the types say where a value
comes from:
* Weapon = the `Weapon` record (launcher) + the `Shell` record (projectile).
Both carry an ID and a Name and — with `ShellWake` — an `Interval`, which the
flat reader merged; they are separate fields now. Power/Velocity/ranges/
LifeTime are Shell fields, which is why 427 weapon reads used to miss.
* CraftUnit/Vessel = `Generic` (hull) + `Maneuver` (flight model) +
`StructureCount` (counts) + `Shield`, plus a new `hardpoints: Vec<Hardpoint>`
— one entry per Turret_/Bridge_/Thruster_/Hatch_/ShieldGenerator_ record, each
with its own HP. A flat HP could only ever be one of them.
* PlayerConfig = `Player`, plus `phases: Vec<PlayerPhase>` (SpaceSize/SupplyRange
are per Phase_N) and `score: ByDifficulty<ScoreRules>` (MainMissionBonus is per
Score_<difficulty>; the flat answer was the Easy one).
* Character faces come from the `Faces` record's field names (identical output to
the old token scrape, 0 of 68 objects differ — now by construction).
* Stage = `StageResource` + `phases: Vec<StagePhase>`, and the packages it names.
* The `fields: BTreeMap` on every struct became `records: RecordSet`, which keeps
the record boundary; `RecordSet::everywhere(field)` answers "which record".
The token-scraping loaders move too, and this is where the old reader was worst:
* Arsenal: options are the positional fields of the STANDARD_<slot> records. The
scrape returned 16 nose options of which 8 were field keys and pilot names, and
47 for arm3 of which 38 were junk, while missing Mine_B2A and No_Equipment.
Now 8/12/9/9, all real weapons.
* Squadron: one record per squadron, members are Count*4 positional slots
(unit, message set, n, pilot) — 1160 squadrons with ids and 2295 member tuples,
against 28 idless squadrons and 47 members before. Agrees exactly with the
independent Python decode in docs/re/structures/unit-group-table.md.
* DemoMessage: 11775 lines against 10263, every one with a speaker, a portrait,
a delivery mode and a voice token, from fixed positional slots.
* PilotRoster: assignments are the records the `UNITS` record names, so each one
now carries its unit id, its loadout and the player marker.
* UnitRoster: the roster is the field *names* of the single `EnumUnit` record.
* load_weapons selects on the records (Weapon + Shell) rather than on token[0],
whose first byte is often a stray pool byte ("#Weapon", "%Weapon"). Same 131
objects, no heuristic. GP_HANGAR_ARSENAL.pak holds none of them — the module
doc's claim that player weapons live there was wrong.
schema:: constants keep their names and values but are documented for what they
are: record 0's name hash (PLAYER = Difficulty_Easy, UNIT = Maneuver, VESSEL =
Bridge_000, MESSAGE = Message_000), not a schema id.
Two things the migration exposes and does not fix, flagged in the docs instead:
load_units' bucket is 43 Type=Craft + 46 Type=Vessel objects (new `unit_type`
field lets a caller separate them), and StructureCount.TurretCount is not the
number of Turret_* records (the player's craft says 4 and has 63).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
It is a throwaway measurement harness I wrote in this session ("TEMPORARY
measurement scratch — not for commit") that a concurrent commit picked up
along with its own files. It asserts nothing and duplicates work that is now
covered by tests/game_data_disc.rs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
to_xma_riffs now emits the leading headerless segment when it sits at a whole
number of XMA1 packets and carries a non-zero byte. VOICE_D_453 goes from a
0.14 s trailing fragment to a 45116-byte leading sub-wave that dominates it.
I withdrew this exact change earlier for two reasons. Both are now answered
rather than argued away:
* "It recovers no audio" -- it used the STEREO format. At two channels every
bank yields exactly 1792 bytes, one frame, whatever its size. Mono yields up
to 113x more.
* "It matches 1524 of 8021 RIFF-bearing entries" -- the byte-level reach is
still 1524, but the audible reach is not. Across the 84 movie-bound banks
the segment adds >1 s to exactly 7, the hokyu_*_H tankers on D_453/D_454 --
precisely the broken ones -- and <=0.25 s to 66 of the rest. The largest
non-resupply addition is S04A at +0.66 s on a 256 s movie.
The safety oracle is recorded with its limits: 8 of the 84 banks ALREADY
exceed their movie's duration before the change, by hundredths of a second,
so it cannot resolve differences at that scale. It establishes scoping, not
correctness. Callers clamp to the movie length regardless.
VOICE_D_451's all-zero leading region is skipped by the non-zero guard, so
the rule cannot prepend silence to a bank that does not need it. Pinned, as
is the packet arithmetic (n = 8, 1, 7, 22, 29) which has no tunable.
slb_disc, movie_subtitle_disc and movie_manifest_disc all still pass.
NOT verified by ear -- that needs a human.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The load-bearing error of the whole voice-bank thread, and it is mine. It
stood for three iterations across two write-ups that each called the result
proven.
I treated a subtitle cue as a timestamp that must fall INSIDE the voice clip,
and concluded a 0.07 s clip could not host a cue at 4.70 s. A cue is when the
line STARTS. The voice plays from the cue, so the clip only has to fit the
window between the cue and the end of the movie. Under that reading every
bank fits at plain 48 kHz:
bank samples @48kHz cue window
D_450 158967 3.31 4.00 5.30
D_451 76084 1.59 3.70 5.60
D_452 119562 2.49 0.00 8.34
D_453 108608 2.26 4.70 4.60
D_454 167828 3.50 0.00 9.50
2-3.5 s is also the right length for the lines. Nothing is missing, and the
17091-20563 Hz window from the previous commit is void with it -- its lower
bound came from the same misreading.
What survives, because it was measured rather than interpreted: the leading
region is XMA1 mono, the decode runs to the final frame, and cue values are
seconds.
Separately settled, and it is what exposed the error: each shared bank holds
ONE generic line. The 3-5 movies bound to a bank have IDENTICAL subtitle
text, 5 banks out of 5 -- "Rhino 3 has landed. Commencing resupply.",
"Resupply complete. You are cleared for take-off!", and so on.
That also explains the historical in-game rejection of hokyu_DS_s13A ->
VOICE_D_452 that started this whole thread. The line is generic, identical
for s02A/s07A/s08A/s13A. Someone expecting a stage-13-specific line would
hear the generic one and call it wrong -- while the binding is exactly right.
The disc said so; the subtitle text now says so independently.
Still open: whether the leading mono region is additional audio or an
alternate take, since the totals above add it to the RIFF sub-waves.
Artifact: examples/shared_bank_takes.rs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Two things settled, one of them a correction of my own claim from the last
iteration.
The cue unit is verified rather than assumed. parse_timing computes mm*60+ss,
but only if the token really is mm:ss.cc, so I checked against an independent
oracle: the movies are on the disc and a cue must land inside its own movie.
66 English movies with subtitle tracks, 0 cues land after the movie ends.
Centiseconds would have overflowed essentially all 66. The seconds reading
stands and the verdicts built on it survive.
"The sample rate does not converge" does not. I reported implied rates of
39742 / 20563 / 23108 Hz as irreconcilable. They are not estimates of the
same quantity -- each is a ONE-SIDED BOUND. The audio must be at least as
long as the last cue, so samples/cue is an UPPER bound; it cannot outlast its
movie, so samples/movie is a LOWER bound. Intersecting:
bank samples cue movie lower Hz upper Hz
VOICE_D_450 158967 4.00 9.30 17091 39742
VOICE_D_451 76084 3.70 9.30 8180 20563
VOICE_D_453 108608 4.70 9.30 11677 23108
=> 17091-20563 Hz, non-empty. A single rate IS consistent.
I had been comparing them as competing point estimates, which is why they
looked contradictory.
What is still open, and stated as such: that window contains no standard XMA
rate. The lower bound assumes a whole bank plays inside one movie, and each of
these banks is bound to 3-5 movie slots -- so if a bank holds several takes
the lower bound is void, leaving rate <= 20563, which 22050 nearly meets.
Next step recorded: establish whether a shared bank is one line or several.
Artifact: examples/cue_unit_check.rs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Retried the format probe with the fmt chunk built to synth_xma1_fmt's exact
byte layout, and with the bank's own RIFF sub-wave decoded through the same
pipe as a CONTROL so a broken harness cannot masquerade as a result.
The channel count is the whole story:
bank lead B channels=2 channels=1
VOICE_D_450 16384 1792 46756
VOICE_D_451 2048 1792 896 (all-zero region: control)
VOICE_D_452 14336 1792 30154
VOICE_D_453 45056 1792 203648
VOICE_D_454 59392 1792 294440
channels=2 yields EXACTLY 1792 bytes for every bank regardless of size -- one
frame, then it stops. That constant is the tell. At channels=1 the same data
yields up to 113x more, and the control sub-wave decodes to 13568, so the
pipe works.
Why the previous probe got 0 bytes everywhere is now named: I read
synth_xma1_fmt(2, 2, 48000)'s second argument as a STREAM COUNT when it is a
CHANNEL MASK, and built the WAVEFORMATEX around that misreading.
Also recorded as a refutation, because it was tempting: solving for the
sample rate as decoded-samples / last-subtitle-cue does NOT converge. D_453
implies 21665 Hz -- close enough to 22050 that I nearly wrote it down -- but
D_450 implies 5844 Hz. No single rate explains both, and the decodes are
visibly partial (samples per input byte ranges 2.10-4.96 where a clean decode
would be near-constant).
So the container is identified and the duration is not. Next step recorded:
find why FFmpeg stops early, likely the hardcoded packet/subframe fields.
Artifact: examples/slb_fmt_probe.rs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The corpus said 0.14 s is "far too short for the spoken line". That is a
judgement about audio, and audio judgements cannot be made in this container.
The subtitle tracks settle it without listening: each carries cue START
times, and a subtitle that appears at t seconds cannot sit inside a clip
shorter than t.
FFmpeg-measured (not estimated from a compression ratio -- the first version
of this example used an 8:1 guess, which is not good enough to hang a
conclusion on):
hokyu_LS_s02A D_450 cue 4.00 s audio 1.41 s MISSING
hokyu_LS_s09A D_451 cue 3.70 s audio 1.81 s MISSING
hokyu_LS_s02H D_453 cue 4.70 s audio 0.07 s MISSING
hokyu_DS_s13A D_452 cue 0.00 s audio 1.21 s no signal
hokyu_DS_s07H D_454 cue 0.00 s audio 0.21 s no signal
Three of five are decisive; the other two have their only cue at 0.0 s and
say nothing either way. So something is genuinely missing from these banks --
established independently of the leading-region work, and measured rather
than felt.
The fmt-variation probe I recorded as the next step is INCONCLUSIVE and is
written up as such: 36 combinations over VOICE_D_453's 22-packet leading
region all produced 0 PCM bytes, including ones that should be equivalent to
the crate's own synth_xma1_fmt, which does parse. So the probe tested my
hand-built fmt chunk, not the hypothesis, and it is NOT evidence that the
region is non-XMA. The retry should use the crate's helper.
Artifact: examples/voice_len_vs_subs.rs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
