Closes the reach caveat the `opt ` name fix left behind: 60 of 18 002 RATC
children carry no `opt ` block, and it was not established whether they lack one
or sit past our 128-byte window.
Neither. They are not children. `examples/ratc_optless_children.rs` re-runs
`ratc::parse`'s own guards over the disc and reports which one fired: all 60 are
"tag beyond the window", none is rejected by length, gap or charset, none is
child #0, and all 60 live in six bundles of one archive. Within a bundle the
distances back to the nearest tag are an exact arithmetic progression, step
60 600 -- ten different records finding the SAME tag, because there is only one.
Reading a bundle directly: children 1..10 are equal-size T8aD blocks under a
single `opt ` name, `pb_f15_eg_anm.tan`. `.tan` is a FRAME SEQUENCE. One block
declares the resource; its payload is a run of T8aD frames.
Disc-wide, over all 18 718 `opt ` names in all 33 paks: a RATC bundle names
exactly six kinds of resource -- `.t32` 14 756, `.rat` 3 311, `.prm` 367,
`.tbm` 224, `.sbo` 54, `.tan` 6. Six `.tan`, ten frames each = 60, the entire
population with nothing left over. The negative is closed, not narrowed.
Consequence recorded but deliberately not fixed: `ratc::parse` over-reports
there, listing a `.tan`'s frames as anonymous children. Nothing in the menu
milestone reads a `.tan` -- it occurs only in GP_READY_ROOM, which S1 ruled
out -- so no screen the port draws changes.
Also a METHOD entry for this container OOM-killing `slb_leading_segment_disc`
under default test parallelism (SIGKILL, no assertion; 8/8 pass with
--test-threads=1).
`ratc::parse` named each child by scanning backwards for the last printable run
of bytes before its magic. The format states the name explicitly instead, in an
`opt ` block: `"opt " | BE32 len | name | NUL | 3 bytes | magic` -- the same
block `ui_layout::opt_link` already read for a button's focus link.
The scan agrees with it 17 918 times out of 17 942 and is wrong 24 times, every
one the same failure: the 3 trailing payload bytes are themselves printable and
beat the real name. For `pteff05.t32` those bytes are `38 41 58` = `8AX`, so the
full-resolution background of all five menu screens registered under a name no
element declares, resolved to no sprite, and `compose` dropped it through an
early `continue` that -- unlike the two arms above it -- records nothing. The
screen lost its background and `screen render` still reported "all resolved".
`8AX` was never a name. Docs that treated it as one are corrected here.
Disc-wide, and the control is the 17 918 the scan already got right: the `opt `
reading reproduces every one of them. Effect on the five screens is the
signature of the same art at twice the resolution -- mean brightness unmoved,
high-frequency detail x1.15..x1.30 -- which is what the separately-measured
`ui-8ax-fullres-background` result said the game draws.
Also closes a long-standing dangling reference: `pmbase.t32`, recorded as "on
the disc nowhere", is the `GP_STAGE_CLEAR` child the scan called `8AX`. RATC
sibling references now resolve 10 148 of 10 148.
Verified: 114/114 sylpheed-formats unit tests (including two new ones pinning
the `8AX` case byte for byte and the no-block fallback), and every disc-gated
integration suite in sylpheed-formats/sylpheed-cli.
The shifted keyframe-time reading looked like it implied something
simple: the final pose is reached at a definite time and nothing follows,
so rest should just be the last keyframe and the plateau heuristic could
go. Tested by applying it to EVERY element:
title +0.9500 -> +0.6819 -0.2681
main menu +0.9460 -> +0.6416 -0.3044
EXTRAS +0.9440 -> +0.5745 -0.3695
publisher splash +0.9600 -> blank (zero variance, corr undefined)
developer splash +0.9643 -> blank
Refuted, and the failure supplies the model. A group is entry -> hold ->
exit, and the exit is the screen's DISMISSAL. While a screen is displayed
it has not reached its last keyframe; it is sitting at the hold. So
rest_plateau is the correct primary rule, and the last keyframe is the
post-exit state -- correct only once the screen is gone, which is why
applying it everywhere blanks the splashes.
This does not contradict the shifted reading. That reading says when each
pose is reached; it says nothing about the group being played to
completion while the screen is still up. The step between them was mine.
And it explains why last wins for the two plateau-less elements: an
element with no hold is a transient, it flashes and is over, and at any
settled moment it is gone -- which is its last keyframe. The draw capture
says the same independently: on the developer splash the _eff glows draw
on frames 94-115 and the logos on 116-211, so the glows are already
finished when the logos are up.
Three independent observables -- animation timing, static composites, and
the per-frame draw log -- now agree on one rule: plateau where there is
one, last keyframe where there is not.
METHOD: a blank render is a NaN correlation, not a low score, and that
NaN was the strongest form of the result; and when a model predicts
something the measurement refuses, suspect the step you supplied between
them.
Scored candidate rest-pose rules by rendering and correlating instead of
arguing, and both results correct something I had published.
First, the exposure. The guessing fallback is reached only by an element
that is plateau-less AND multi-keyframe -- a single-keyframe element
short-circuits at `match len { 1 => first }`. Per screen:
title (4) 24 elements 2 plateau-less 0 reach the fallback
main menu (5) 16 5 0
EXTRAS (6) 18 5 0
publisher splash (10) 3 2 1
developer splash (11) 7 2 1
So on the three screens the port cares most about, rest() never guesses.
That is why three different rules render builds 4/5/6 to identical
correlations -- the code is unreachable there, which I nearly read as
"the choice does not matter".
Second, where it does fire, the last keyframe is markedly better:
publisher splash dwell +0.9600 last +0.9982 maxalpha +0.9600
developer splash dwell +0.9643 last +0.9758 maxalpha +0.9643
That refutes my own earlier refutation. I had killed the last-keyframe
rule by arguing it makes palogo_anima_eff invisible while its two
siblings stay lit, which looked like an artefact. The capture says
otherwise: making it invisible is what improves the match. The sibling
symmetry was my expectation, not evidence.
Caveat kept in front: both captures are single frames of a transient
animation, so this fixes which pose matches THOSE frames, not which is
canonically at rest. Default unchanged -- better on both screens where it
fires and identical on the other three, but it would move 2 305 elements
disc-wide on two measurements. Reachable via SYLPHEED_REST_RULE=last.
Also confirmed: all 195 zero-scale rest poses are inside the corrected
2 305 ambiguous population; none is a single-keyframe element.
METHOD: score a rule where it can differ, or you measure nothing; and an
argument from symmetry is a prediction, not a refutation.
`compose` claimed the derived paint order "reproduces both measured
orders up to ties". That sentence was never measured and was stale by
one: there are three measured orders, not two. examples/paint_order_audit.rs
checks it.
main menu (entries 5, 8) derived == measured 0 inverted pairs
developer splash (11, 14) derived == measured 0 inverted pairs
title (entry 4) DIFFERS 8, all same-key ties
So the claim holds and the exception is entirely ties -- but two of those
ties are total occlusions, not near-misses. The tied family is the five
ptlogo_back2eff glows (key 32899); back2eff5 is 1133x280 and FULLY
CONTAINS back2eff3 (82,824 px^2 = 100% of the smaller) and back2eff4
(152,047 px^2 = 100%). Derived paints it on top of two glows it entirely
covers; the game paints it underneath. A tie-break by declaration index
can therefore be wrong by a whole layer. The title itself is unaffected --
it has a measured order.
The port's actual exposure, per screen: title, main menu and developer
splash all use MEASURED orders; the publisher splash is derived but has
ZERO ties, so it is fully determined; EXTRAS is derived with 15 tied
pairs of which only 2 OVERLAP. Two element pairs on one screen is the
whole risk, and that is what HANDOFF now says -- not the raw 15, which
would have overstated it 7x.
Reach stated: this compares the derived order against orders measured
from the game, not an independent derivation, so where no measured order
exists only the tie exposure can be checked. Overlap uses pivot*2 as the
element size at its resting placement.
Stale comment in compose corrected. METHOD: a hedge in a code comment is
an unmeasured claim; and count the cases that can bite, not the ones that
match the pattern.
blit() and fill_quad() both opened with
let sx_pct = if kf.scale_x == 0 { 100 } else { kf.scale_x };
so an element whose pose is collapsed to nothing rendered at FULL SIZE.
I first described this as "a 1-pixel sliver", reading the .max(1) in the
size arithmetic two lines below -- the guard above it meant .max(1) never
saw a zero. Read the whole function, not the lines you went looking for.
Control run before the change: if 0 meant "unset", something on the disc
would use it throughout. Nothing does.
elements with a keyframe group 15 493
at least one keyframe at scale 0 2 166
EVERY keyframe at scale 0 0
zero is a transient the element grows out of 1 762
ptlogo_eff3.t32 runs 0% -> 200%. An "unset" marker no element ever uses
throughout is not a marker.
Both functions now return without drawing when either scale is 0.
Reach: 24 of 24 renders byte-identical across GP_TITLE (all 16 builds),
GP_PAUSE_MENU and GP_OPTIONS -- additive on every screen the port needs.
195 elements have a guessed rest pose at scale 0 and 126 of those have a
non-zero alpha, so the old code painted them; all 126 are in
GP_READY_ROOM.pak, which S1 already declared a no-go. They are
tactical-map sprites at sx=0/sy=100, mid-horizontal-wipe, that the
coercion drew full-width.
So: a real correctness fix with no visible effect on the menu port.
Suite green, 122 passed / 0 failed across 3 suites at commit time.
Follow-up on last iteration's unadopted candidate (+36 holds the NEXT
pose's time, not its own). Two new results, pointing opposite ways, and
both are reported.
FOR, and calibration-free: the observed full-alpha hold : fade-out ratio
on palogo_gamearts is 83 : 13 frames = 6.38. The shifted reading predicts
8.00. The current reading predicts 0.25 -- off by 26x. With the glow's 2
units/frame fixed and nothing else free, the current reading says the
logo holds full alpha for 2.0 frames; the capture holds it for 83. This
is no longer the shape argument the candidate rested on.
Also for: rest()'s plain max-dwell fallback picks a=0 -- a transparent
pose, for a publisher logo -- under the current reading, and the visible
a=255 hold under the shift. Only the rest_plateau special case rescues
the render today, and that is the case the port agent reported a bug in.
AGAINST: rendering every build of six UI paks under both readings, 10 of
11 compared are byte-identical and one changes -- GP_TITLE build 7, the
Japanese twin of build 4, by 13.1% of pixels. Build 4, the one verified
against a live capture, is unchanged either way, so the single build the
shift moves is the one with no capture to adjudicate it. The proxy goes
against the shift: language twins are the same artwork, and build 7 reads
70.94 mean luminance as decoded against build 4's 71.41, but 76.32
shifted. Correlation does not separate them (0.6206 vs 0.6201).
These constrain different things -- timing versus pose selection -- and
rest() is a heuristic layered on the times, so moving the times moves its
tie-breaks. Adopting the shift means revisiting that heuristic in the
same change, with no build-7 capture to verify against.
Default UNCHANGED. Experiment reachable via SYLPHEED_KF_TIME_SHIFT=1.
Shifted-mode suite: 122 passed, 0 failed across 3 suites at commit time.
The rotated quads on the title screen come from the keyframe block after
all. The earlier negative -- "every GP_TITLE build 4 element has all three
angle words at zero" -- read the right bytes over too small a region: it
walked the top-level declaration table, and the rotated elements are the
nested leaf records ptloop01.rat / ptloop02.rat.
Confirmed against the framebuffer rather than against our own renderer.
The two records declare +12 = 30 and -45; the GPU capture submits their
quads at +30.26 and -45.28 degrees -- magnitude and sign, two different
values. Corroborated by shape in GP_BUNK 117ca14f, where +12 ramps
0 -> 360 with position, scale and alpha constant: a spin in place.
Identifying which draw it was needed edge lengths, not bounding boxes:
400x1076 and 400x1444 against pteff03/pteff03a 399x180 at the elements'
two different declared scales, 600% (1080) and 800% (1440). The same test
names three known-positives in the capture (ptlogo1, ptcopyright,
ptbtn00), so it passes its own control.
Keyframe gains rotation_deg plus unknown_4/unknown_8, carried rather than
dropped. NOT rendered -- ui_layout::blit is axis-aligned only, so the
reference renderer and the port will both draw these upright until a
rotating blit exists.
The census tool ships with the trap that broke its first version: nested
RATC blobs are not 4-byte aligned, so an aligned scan found 0/3 of its
own control blocks and missed 16 341 blocks. Disc-wide +12 is non-zero in
14.50 % of 83 862 blocks.
sylpheed-formats tests, SYLPHEED_DISC set: 131 passed, 0 failed across the
6 suites finished at commit time; the run had not yet completed.
title's rotation
Looking for where the rotated quads come from, the obvious candidates
were the three keyframe words at +4, +8 and +12 that ui_layout.rs
documents as zero.
They are not zero. Across 72287 keyframe blocks disc-wide they are
non-zero in 4.81, 4.56 and 15.82 percent of blocks, and read as signed
values clustering on 180, -180, 90, -90, 120 and 22 -- degrees. Three of
them, so plausibly rotation about three axes. I have marked that amber
because it is the shape of the numbers and nothing more; no observed
rotation has been tied to a value. The doc comment is corrected
regardless: "0 on every frame seen" was an artefact of the sample.
And they do not explain the screen I was chasing. Every element of
GP_TITLE build 4 has all three at zero, element by element, while the
game demonstrably submits rotated parallelograms there. So the title's
rotation comes from outside the keyframe data and is still unidentified.
One correction to my own last write-up, flagged rather than left: I
stated that the skewed draw IS the swoosh. It is the only skewed geometry
in the capture and the swoosh is the only diagonal element on the screen,
so the inference is reasonable -- but I never confirmed it by matching
the draw's texture or screen position to that element, and I should have
said so the first time.
residual is smaller than I said
Testing the candidate I raised last iteration rather than carrying it.
Blending bit-0x02 sprites additively moves every measure the wrong way --
whole-frame mean diff +0.55 to +1.04, swoosh-band mean +1.83 to +3.98,
band edge-correlation 0.6971 down to 0.5578. So the bit is real and
independent but does not select an additive blend. I reverted the
experiment and kept the word as T8adImage::flags, documented and not
acted on; the render is byte-identical to before.
Second refutation: the swoosh is not displaced. Shifting the band over
plus or minus 80 by 8 pixels peaks sharply at zero, 0.7342, falling to
0.22 at 24 px. So the pivot story is dead twice over -- inert at scale
100, and no displacement to explain anyway.
And I have restated the residual, because earlier sections overstated it.
The +16 to +34 band tiles I quoted were measured WITHOUT --primitives.
With the dim drawn the band's average is nearly right at +1.83; what is
wrong is its structure, tiles running -38.6 then +33.8 and cancelling.
Six candidates eliminated now and none confirmed.
One caveat I owe the port agent about the capture I gave them: it is at
t=4.0s, roughly 174 keyframe units into a screen whose elements have
keyframes out to t=600. I judged "settled" from mean luminance, which
cannot see a thin sprite still moving. It is settled for the bulk of the
screen and not proven settled for every element -- which is a live
alternative explanation for a structural difference in exactly the band
the sweeps cross.
METHOD: cargo build passing does not mean cargo test compiles. Adding the
field built the library in 1.48s and broke two test-only struct literals;
cargo test failed with exit 101.
VISIBLE, not when its pose repeats
The port agent's report was right about the defect and about which
elements it hits, and its proposed condition does not survive the case
the exclusion was written for.
Their test -- the final untimed keyframe has the same pose as the last
timed one -- is also true of pgptitle.rat, whose trailing run is two
identical transparent frames. Adopting it would erase the word PAUSE
again, which a committed capture disproves. What separates the two is
alpha: an exit fades the element out so its last keyframe is transparent,
while an element with no exit ends on the pose you can see. So
rest_plateau now accepts a trailing run exactly when it is visible.
Verified against a capture rather than against another renderer, which is
the point their own report made: ptframe1's rest moves from (620,108)
t=16 to (440,108) t=62; the changed pixels are 10082 in a bounding box of
x 440-839, y 108-577, which is exactly the 400x470 at (440,108) they
predicted; and correlation against main-menu-oracle.png over that region
improves from 0.9596 to 0.9748. The PAUSE wordmark is unchanged across
all three pause builds.
This also closes a question ui-paint-order-key.md has carried for a
while, that ptframe1 and ptframe2 rest at alpha 0 while the capture shows
the menu frame plainly.
One trap cost most of this iteration and goes in METHOD: CARGO_TARGET_DIR
is redirected in this container, so ./target/debug/sylpheed-cli is hours
stale and every render I made against it was old code. Byte-identical
before and after reads as "no effect" when it actually means "you ran the
old binary".
The full disc-gated test suite was still executing when this was
committed; the verification above is artifact-based.
The trickiest reading on the disc lived in the Bevy viewer: resolving a
cutscene's voice to a continuous byte REGION of the sound stream, because the
movie voices are one XMA stream chunked into VOICE_*.slb entries whose
boundaries do not match the cues -- a cue routinely spans two chunks, so a .slb
need not hold the track its name claims.
That put the logic most likely to be re-derived incorrectly in the crate least
likely to be reused. The Godot port's exporter needs the same answers, and there
must be one implementation of them.
New `sylpheed_formats::media` owns every case where the bytes of one playable
thing are not one archive entry: segment-spanning reads, multi-sub-wave banks,
and the voice-region resolution. Callers supply bytes through a `DiscSource`
trait, so the viewer keeps its ISO/directory abstraction and a headless consumer
gets `DirectorySource` for free.
The seam is deliberate: this module returns XMA RIFFs, not PCM. Decoding means
shelling out to FFmpeg, which is native-only and a policy decision for the
consumer -- everything up to "here are the bytes that belong together" is disc
knowledge, everything after it is a codec choice.
The four moved functions were previously untested; `tests/media_disc.rs` now
pins them, including the negative the corpus paid for -- an unbound movie must
stay unvoiced rather than borrow a neighbour's clip, which was tried and played
the WRONG recording.
The algorithm is unchanged, moved verbatim (same window sizes, same fallbacks).
The new disc tests pass; the broader audio suite was not re-run in this pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`movie_manifest` has been parsed since the movie-voice work and rendered
nowhere: it resolved a voice bank and that was all. So the only way to find a
cutscene was to hunt `.wmv` files in the ISO tree, where nothing tells you which
mission a file belongs to, whether it has subtitles, or what is said in it.
View ▸ Cutscenes lists all 104 manifest slots with mission/phase, kind, movie,
subtitle track, voice token and telop, and -- the part that needed no new
parsing, only a route -- resolves the captions to a readable TRANSCRIPT with a
language selector. Subtitles were previously burned into the video during
playback and reachable no other way.
Three negatives are shown rather than smoothed over:
* 5 manifest-bound movies have no `.wmv` (logo1-4 and an encoder test clip).
They are marked and get no Play button instead of one that would fail.
* 9 of 101 movies resolve no English transcript.
* the `.prt` telop overlay is named by the manifest and we have no parser, so
the reference is shown labelled "not decoded" rather than omitted.
`cutscene_catalog_binds_movies_and_transcripts` pins all of it against the disc
-- 104/101/99/99/22, the exact absent-movie list, 92 transcripts -- because a
browser that quietly dropped these would look complete and be wrong. The counts
independently reproduce docs/re/movie-subtitle-link.md.
Play routes through the normal FileSelected path, so the existing video player
handles it exactly as it would from the tree.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The library enumerator kept only names containing VOICE or \Briefing\, and read
eng\sounds.tbl unconditionally. So the Explorer could reach 4382 of the 9519
banks in sound.pak: no music, no jingles, no sound effects, and no Japanese
voice at all -- roughly half the disc's audio had no route to the UI.
`slb::list_audio_entries` now returns every named bank with the category its
path implies (Music / Jingles / Sound effects / Radio / Dialogue / Movie voice /
Briefing). `list_voice_clips` is that, restricted to the spoken categories, so
its existing test still guards the old behaviour. The 36 root banks carry no
language component and appear whichever table is read; the window gets an
English/Japanese switch that re-reads the other sounds.tbl, since the table name
IS the selector.
Two defects the decode found, both recorded in
docs/re/structures/sound-pak-contents.md:
* `Static.slb` -- the SFX bank -- declares 616768 bytes more than sound.p04
holds. Not our extraction: p04 matches the ISO's own directory record, and a
sweep of every pak on the disc finds this one entry over-running and no other.
It is the highest-offset entry, so its comp_size is an allocation size. A
short read is now allowed for the tail entry ONLY; any other overrun stays an
error, because clamping it would hide real damage behind a half-decoded asset.
The bank went from unreadable to 514 s of audio.
* the left-channel downmix was applied to everything. Right for voice (mono
content however stored), wrong for music (a real stereo mix, half of it
discarded). The caller now decides from the category.
35 of the 36 shared banks decode; JNGL_001 does not, and says so in the player
instead of the panel silently closing. Its payload is not a whole number of XMA1
packets from any known data offset, so it is likely not a plain headerless
stream -- written up rather than papered over.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A wave runs to data_at + declared_size, and there is seek magic exactly there
whose little-endian packet count at +12 times 2048 equals the declared size --
7620/7620 disc-wide. That is the decoder-independent boundary and the thing that
proves the declared sizes honest, so it should not be able to regress silently.
The boundary routinely lies outside the entry's own comp_size window, so reading
it needs the flat segment stream rather than the entry slice; PakArchive gains a
small data_at(offset, len) accessor for that.
Test walks a bounded slice to stay fast -- 792 banks in this run, all holding.
8 disc tests pass.
I read the seek chunk's packet count big-endian; it is little-endian at seek+12,
with size == 8 + 4*count. And a seek sits immediately AFTER its own data, so an
entry's first seek usually belongs to the PREVIOUS bank (implied start -25232
for D_452, -145988 for TCAF_608). I was comparing an entry's first seek against
its first data -- different waves by construction, which is why no reading lined
up.
With that fixed, the declared sizes are honest: every RIFF-bearing entry on the
disc has seek magic at exactly data_at + declared_size with count*2048 ==
declared. 7620/7620, zero failures.
VOICE_TCAF_608 is not truncated. Its Channels is 2 and I decoded it as mono;
read as stereo it gives 6520176 bytes = 33.96 s, agreeing with both length
signals in the bank (33.88 s from cumulative samples, 33.97 s from
PsuedoBytesPerSec). 170 of 8021 banks (2.12%) are stereo -- exactly the rate of
my 1-in-60 outlier.
This is the mono/stereo trap already documented on this very page, met from the
other direction: I had written 'at two channels every bank yields one frame' and
then spent several passes blaming missing data for a one-frame decode.
Code fix: to_xma_riffs built the leading segment with a hard-wired mono fmt. It
now reads Channels from the bank's first RIFF. 7 disc tests pass.
The ties needed a different signal, not a longer scan. Banks carry one: a seek
chunk sitting on a packet boundary, so seek_pos % 2048 IS the data offset. On
the 6033 labelled banks with a seek before their first RIFF, 6031 agree
(99.97%) -- better than the packet scan and structural rather than statistical,
so scan_data_offset now tries it first.
On the scan's 28 ties it resolves 26 correctly and 0 wrongly (2 have no usable
seek). Combined rule scores 7354/7358 = 99.95%, up from 99.62%. 762 of the 1495
RIFF-less banks carry a seek, so the signal exists where it is needed.
Also ruled out, since a wrong offset was this page's whole subject: the header
is not audio being discarded. Adding 0 to the candidate set, it wins 6 of 7358.
7 disc tests pass.
Two comments claimed it is 'honest per sub-wave'. Measured: 5296 of 7586 banks
declare more than the entry holds and none declares exactly what it holds, so
the existing .min(slb.len()) clamp is load-bearing rather than defensive.
Comment-only change; 7 disc tests still pass.
1495 banks carry no RIFF and take a separate path that hardcoded both the
offset and stereo. Across a random 48-bank sample there was NOT ONE where the
old stereo-at-1392 pair beat the best mono offset; median gain 184x, individual
banks going from 0-4816 decoded bytes to 180000-380000. Stereo shows the same
stop-after-one-frame signature already recorded for the leading segment.
With no RIFF the offset cannot be derived, so scan_data_offset picks among the
four disc offsets by XMA1 packet-header plausibility. Validated on the LABELLED
set -- all 7358 banks that do have a RIFF, where the answer is forced: 7330
correct (99.62%), and all 28 misses are ties on the top score, never a wrong
unique winner. Ties fall back to 1392.
The winning offsets also reproduce, by directory, the distribution measured
independently from the RIFF-bearing banks. jpn\etc splits 1468/1600, so path
alone is not sufficient -- which is why this is a scan and not a lookup table.
7 disc tests pass (build-reborn test -p sylpheed-formats --test
slb_leading_segment_disc, SYLPHEED_DISC wired up).
HEADERLESS_DATA_OFFSET is the value the offset takes in <lang>\etc\, not a
property of the format. The leading stream is a whole number of 2048-byte XMA1
packets ending at the first RIFF, so its start is first_riff % XMA1_PACKET.
Disc-wide that takes four values -- 1392, 1468, 1600, 1728 -- varying by
language and subdirectory.
Verified by decoding, not by arithmetic: on a random 140-bank sample with a
non-empty leading region, the derived offset yields more audio in 85, identical
in 54 (the eng\etc controls, where it must and does reproduce the old
behaviour) and less in 1. Median gain among the improved is 70x --
eng\Voice\VOICE_TCAF_592 goes 1506 -> 97152 bytes, jpn 2910 -> 127178.
This withdraws my own claim from earlier today that the Japanese banks were a
different undecoded layout. They are the same format with a different offset;
I had treated a constant derived from one subdirectory as a property of the
format. The same error was hiding the identical defect in 1873 eng\Voice banks.
ixud.rs now has an IdxdObject-shaped reader, IxudObject, and build_caption_text
reads captions as FIELDS instead of pairing them with whatever token follows in
the pool.
build_demo_text token adjacency 134 ids 537 lines
build_caption_text token adjacency 3721 8074
build_caption_text record fields 4085 8800 = all of them
Verified over the whole disc by tests/ixud_records_disc.rs: 1104/1104 objects
parse, 1476/1476 records and 628165/628165 named fields reproduce their
ixud_hash, 48 positional, zero failures. The header word at 0x08 is record 0's
hash, asserted per object -- there is no schema field, exactly as for IDXD. The
module doc described a 12-byte record directory and a "schema/type hash"; both
were wrong and are corrected.
I also have to correct my own number from the previous commit. "1.3% of the
game's text" counted OCCURRENCES: each family lives in 24-45 IXUD blocks and
the same key repeats across them. Distinct text-bearing MSG_* keys number 8800,
not 44579, and every one has the <id>_<page>_<line> shape. So the real coverage
was 537/8800 = 6.1%, and I overstated the gap about fivefold. Direction right,
magnitude wrong.
The DEMO control is the sharpest evidence for the change: token adjacency finds
537 lines there, the field reader 541. It was dropping lines even in the one
family it was written for -- which is why the test now asserts "must not lose
lines" rather than "must be identical".
Same lesson twice in one session: pool adjacency is a consequence of how
records are written, not a rule of the format.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
build_caption_text generalises the key parser from MSG_DEMO_* to all eight
families. The shapes are uniform and each family is 100% consistent with its
own: seven use MSG_<FAM>_<id>_<page>_<line>, and VOICE alone inserts a family
letter before the id.
ids lines
build_demo_text 134 537
build_caption_text 3721 8074
The DEMO family comes out identical through both readers -- 537 lines either
way -- which is the control that generalising changed nothing that already
worked. Pinned by tests/caption_families_disc.rs, along with VOICE ids keeping
their family letter.
But this does NOT close the gap, and the write-up says so: 8074 against the
44579 text-bearing fields the record-level scan counts is about 18%.
The reason is the same lesson this session already learned once.
build_caption_text pairs a value with the key that happens to follow it in the
raw UTF-16 token stream -- the adjacency heuristic that was wrong for IDXD and
is wrong here for the same reason. ixud.rs has no record/field reader at all.
The IXUD record table IS decoded and verified disc-wide (1104/1104 objects,
628165/628165 fields reproducing their key) and was simply never wired into
the crate.
Next step recorded: give ixud.rs an IdxdObject-shaped reader and read captions
as fields rather than adjacent tokens. The decode exists; only the plumbing is
missing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Measured first, over GP_MAIN_GAME_E.pak, comparing every named-field read the
six struct loaders performed against the record table: 4435 reads, 2872 agreed,
**966 returned None for a field that has a value**, **596 flattened a field that
several records carry**, 1 was wrong (a weapon whose TargetType is the empty
string read back as the neighbouring token "Skip"). The prior report of
4453/2887/974/591/1 is the same picture; the small differences are definitional
(I count a read as flattened only when the records disagree).
Every read now goes through IdxdObject::record, and the types say where a value
comes from:
* Weapon = the `Weapon` record (launcher) + the `Shell` record (projectile).
Both carry an ID and a Name and — with `ShellWake` — an `Interval`, which the
flat reader merged; they are separate fields now. Power/Velocity/ranges/
LifeTime are Shell fields, which is why 427 weapon reads used to miss.
* CraftUnit/Vessel = `Generic` (hull) + `Maneuver` (flight model) +
`StructureCount` (counts) + `Shield`, plus a new `hardpoints: Vec<Hardpoint>`
— one entry per Turret_/Bridge_/Thruster_/Hatch_/ShieldGenerator_ record, each
with its own HP. A flat HP could only ever be one of them.
* PlayerConfig = `Player`, plus `phases: Vec<PlayerPhase>` (SpaceSize/SupplyRange
are per Phase_N) and `score: ByDifficulty<ScoreRules>` (MainMissionBonus is per
Score_<difficulty>; the flat answer was the Easy one).
* Character faces come from the `Faces` record's field names (identical output to
the old token scrape, 0 of 68 objects differ — now by construction).
* Stage = `StageResource` + `phases: Vec<StagePhase>`, and the packages it names.
* The `fields: BTreeMap` on every struct became `records: RecordSet`, which keeps
the record boundary; `RecordSet::everywhere(field)` answers "which record".
The token-scraping loaders move too, and this is where the old reader was worst:
* Arsenal: options are the positional fields of the STANDARD_<slot> records. The
scrape returned 16 nose options of which 8 were field keys and pilot names, and
47 for arm3 of which 38 were junk, while missing Mine_B2A and No_Equipment.
Now 8/12/9/9, all real weapons.
* Squadron: one record per squadron, members are Count*4 positional slots
(unit, message set, n, pilot) — 1160 squadrons with ids and 2295 member tuples,
against 28 idless squadrons and 47 members before. Agrees exactly with the
independent Python decode in docs/re/structures/unit-group-table.md.
* DemoMessage: 11775 lines against 10263, every one with a speaker, a portrait,
a delivery mode and a voice token, from fixed positional slots.
* PilotRoster: assignments are the records the `UNITS` record names, so each one
now carries its unit id, its loadout and the player marker.
* UnitRoster: the roster is the field *names* of the single `EnumUnit` record.
* load_weapons selects on the records (Weapon + Shell) rather than on token[0],
whose first byte is often a stray pool byte ("#Weapon", "%Weapon"). Same 131
objects, no heuristic. GP_HANGAR_ARSENAL.pak holds none of them — the module
doc's claim that player weapons live there was wrong.
schema:: constants keep their names and values but are documented for what they
are: record 0's name hash (PLAYER = Difficulty_Easy, UNIT = Maneuver, VESSEL =
Bridge_000, MESSAGE = Message_000), not a schema id.
Two things the migration exposes and does not fix, flagged in the docs instead:
load_units' bucket is 43 Type=Craft + 46 Type=Vessel objects (new `unit_type`
field lets a caller separate them), and StructureCount.TurretCount is not the
number of Turret_* records (the player's craft says 4 and has 63).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
to_xma_riffs now emits the leading headerless segment when it sits at a whole
number of XMA1 packets and carries a non-zero byte. VOICE_D_453 goes from a
0.14 s trailing fragment to a 45116-byte leading sub-wave that dominates it.
I withdrew this exact change earlier for two reasons. Both are now answered
rather than argued away:
* "It recovers no audio" -- it used the STEREO format. At two channels every
bank yields exactly 1792 bytes, one frame, whatever its size. Mono yields up
to 113x more.
* "It matches 1524 of 8021 RIFF-bearing entries" -- the byte-level reach is
still 1524, but the audible reach is not. Across the 84 movie-bound banks
the segment adds >1 s to exactly 7, the hokyu_*_H tankers on D_453/D_454 --
precisely the broken ones -- and <=0.25 s to 66 of the rest. The largest
non-resupply addition is S04A at +0.66 s on a 256 s movie.
The safety oracle is recorded with its limits: 8 of the 84 banks ALREADY
exceed their movie's duration before the change, by hundredths of a second,
so it cannot resolve differences at that scale. It establishes scoping, not
correctness. Callers clamp to the movie length regardless.
VOICE_D_451's all-zero leading region is skipped by the non-zero guard, so
the rule cannot prepend silence to a bank that does not need it. Pinned, as
is the packet arithmetic (n = 8, 1, 7, 22, 29) which has no tunable.
slb_disc, movie_subtitle_disc and movie_manifest_disc all still pass.
NOT verified by ear -- that needs a human.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The structure is exact. In all five resupply banks the first RIFF sits at
HEADERLESS_DATA_OFFSET + n*2048, where 1392 is a constant this crate already
had and 2048 is the XMA1 packet size: n = 8, 1, 7, 22, 29. No free parameter
to tune, and the raw bytes agree -- high entropy from offset 0, then a zero
run immediately before the RIFF. VOICE_D_451 is the control, its single
packet being all zeros.
So I made the obvious fix, emitting that region as a sub-wave, and then
withdrew it on two measurements:
* It does not recover audio. Coverage went 5.4% -> 89.9% for VOICE_D_453, but
the emitted stream decodes through FFmpeg to 1792 PCM bytes -- silence --
while the RIFF sub-waves from the same banks decode to 150-270 KB. Byte
coverage was the wrong success metric and it looked like progress.
* It is not narrow. The rule matches 1524 of the 8021 RIFF-bearing entries in
sound.pak, including RT* movie banks that decode correctly today. Landing
it would have risked a wide regression in order to not-fix five banks.
to_xma_riffs is back to its previous behaviour, verified by re-measuring:
coverage is 5.4% / 9.7% again. The refuted attempt is recorded in the code
beside the branch it would have changed, so the next person does not
re-derive the arithmetic and re-make the change.
XMA1_PACKET is kept as a named constant because the blast-radius scan uses
it. Artifacts: examples/voice_bank_shape.rs (structure), voice_bank_dump.rs
(sub-waves for decoding), slb_hybrid_scan.rs (the 1524 count).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
movie_manifest::parse now reads BASE_INFO's positional field keys (the game's
own cutscene ids, stage*100 + slot) and follows each to its record, instead of
scraping the string pool. The pool stores each distinct string once, so a
REPEAT reference produced no token and read as "no binding".
That single cause explains every wrong cell: 13 later references to
VOICE_D_450..454, two to SUBTITLE_hokyu_LS_s11A.tbl, and MS01A's share of
pwterop_s01a.prt. All 18 hokyu movies are bound, not five.
Counts, verified independently by me against the disc before recording:
104 cutscene SLOTS binding 101 distinct MOVIES; 99 slots / 96 movies with a
voice track, 99 / 96 with a subtitle, 22 / 22 with a telop. The docs' old
94 / 83 / 21 are exactly the counts of DISTINCT POOL STRINGS -- not wrong
measurements, measurements of the wrong thing. Three denominators were being
conflated; the new test pins all three.
Two assertions in movie_manifest_disc.rs were false and are corrected:
hokyu_DS_s13A binds VOICE_D_452 and resolves to eng\etc\VOICE_D_452.slb. The
in-game verdict that rejected that value tested an INFERENCE from a shared
demo id, on a decoder that discards 85-87% of banks in this class -- see
voice-bank-leading-region.md, committed earlier today.
The ~104 script ids are no longer open: they are literal positional keys,
each naming its record, and all 104 resolve. The old "counts differ by three,
positional pairing does not work" has a concrete cause -- three resupply
movies are bound by TWO slots each.
Also corrected: the naming convention has 3 subtitle exceptions (s24A/s27A
borrow s11A's track) and 18 voice exceptions, not one and five.
The legacy scraper is kept as a fallback for blobs with no record table, so
the synthetic unit fixtures still exercise it.
Artifacts: examples/movie_map_csv.rs regenerates the CSV, now slot-keyed
(104 rows; the movie-keyed version silently dropped one slot of each
duplicate). Disc tests green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The binary region in front of the string pool was the parser's oldest open
note ("Not yet decoded"). It is a uniform 16-byte record array sorted by
name hash, a field count, a 12-byte field array sorted by key, a pool size,
and the pool. The trailing `pool_size == file_len - pool_base` identity makes
the layout self-checking, which is what caught the first wrong version.
Verified over the WHOLE disc with zero failures: 7750/7750 IDXD objects,
190782/190782 records reproducing their stored tag_hash, 1271462/1271462
named fields reproducing their key. IXUD is the same container with
ixud_hash, UTF-16BE and every offset in chars — 1104/1104 objects,
628165/628165 fields, checked with an independent parser.
Field names are stored on disc, so no preimage search is needed: a field's
middle word points at its own name. Only 504 fields disc-wide are hash-keyed
with no name; the other 1485073 nameless fields are positional, keyed by a
literal integer (line slots, movie ids).
Two long-held beliefs are WITHDRAWN:
* The word at 0x08 is not a schema hash. It is record 0's name_hash — the
format has no type field at all, and an object's kind is known only from
the caller that loads it. It survived as "schema" because tables of one
kind share their lowest-hashed record name. Caught by a test asserting
every movie id names a real record: 1005 -> STAGE10_PHASE01 failed because
tag_hash("STAGE10_PHASE01") IS 0x067025B9, that table's supposed schema id.
* The field's middle word is not an always-0xFFFFFFFF flags word. It is
0xFFFFFFFF for 54% of fields, enough to look constant in a small sample;
the tell was that it is constant per key ACROSS records, which a per-record
flag cannot be but a per-name pointer must.
`schema_hash` keeps its name rather than churn 33 call sites, with corrected
docs. The first sweep globbed dat/** and missed hidden/DefTables.pak (1425
objects); the test now walks the whole disc root.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Found the routines in the disassembly DB rather than guessing from data:
sub_82447DF0 IDXD tag hash (lbz+extsb, modulus 0x00FFFFDF, magic 0x2101)
sub_82447E70 IXUD tag hash (lhz, 64-bit, modulus 0xFFFFFF67 then 0x00FFFFDF)
Both transcribed instruction-for-instruction into Python and Rust.
IXUD SOLVED. It defeated every single-modulus search because it chains TWO
exact moduli -- the loop reduces mod 2^32-153 in 64-bit arithmetic and only the
result is folded mod 2^24-33. A polynomial mod M1 folded through M2 is not a
polynomial mod anything, which is exactly why the gcd test returned 1. Verified
independently: 86/86 record keys and 108,261/108,261 field tags in
GP_MAIN_GAME_E.pak, and NoRecord -> 0x1c6d9c96.
CORRECTION 1: tag_hash must SIGN-EXTEND each byte (extsb). My reconstruction
used unsigned bytes and matched all 1.27M disc names -- every one is ASCII --
while disagreeing on ~90% of random inputs with a byte >= 0x80 (verified:
18096/20000). The disc could never have caught this; only the disassembly did.
CORRECTION 2: name_hash's reduction is EXACT, not lossy. The module doc claimed
the missing conditional subtract made it something other than %. rlwinm r6,r6,
9,23,31 is just hi>>23, and with RECIP = floor(2^55/M)+1 that is Granlund-
Montgomery magic division -- 0 wrong at every quotient boundary across the full
32-bit domain. Retracted.
cargo test -p sylpheed-formats --lib hash: 10/10.
Closes the 4-byte record key. tag_hash is name_hash's shape -- byte-sum
checksum in the top byte over a 24-bit modular polynomial -- with two different
constants: modulus 0x00FFFFDF (2^24-33, prime) instead of 0x00FFF9D7, and no
lowercasing, so tags are case-sensitive. name_hash explains 0 of 8643.
Recovered from the tables rather than the executable: every inline field name
is a known (name -> tag) pair, and comparing names differing in one character
gives the per-position weights 1, 0x100, 0x10000, 0x21, 0x2100, ... -- a byte
leaving bit 24 re-enters as 33, i.e. reduction mod 2^24-33. Holds where it is
easy to get wrong (distance 8 and 9 carry correctly).
A record's key is the tag of its own name: FormationSet rosters 362/362,
UnitGroup rosters 281/281, S02 squadron names 111/111 -- so records can be
addressed by name without reading the roster first.
Implemented in Python (unitgroup.tag_hash) and Rust
(sylpheed_formats::hash::tag_hash) with 3 new unit tests carrying disc-derived
vectors; cargo test -p sylpheed-formats --lib hash is 8/8 green.
Not settled: the guest routine is unlocated, so this uses exact modular
arithmetic where the game may use a Barrett step without final fixup.
The three orders the derived rule was built from all live in GP_TITLE.pak, so
they cannot confirm it - the rule was fitted to them. These two are from
GP_SAVE_LOAD.pak, read off the running game now that the Canary threading fix
makes the main menu dependable.
The 9-element slot-list header composites EXACTLY as the sort predicts, on all 6
instances of it, and nothing about this screen was fed into the rule:
measured 7 8 0 1 2 3 4 5 6
derived 7 8 0 1 2 3 4 5 6
including TWO tied groups (0xb102 x2 and 0xb210 x5) that both come out in
declaration order, and the unkeyed pfeff00.prm fade quad last.
The 13-element save/load frame differs in exactly the two open questions and no
new ones: two unkeyed pfbase.tbm backgrounds paint FIRST where the sort puts the
keyless last - the splash's palogo_eff0.prm behaviour in a different file type,
so implied_layer_key now covers it - and the 0xb100 group of four paints
10,11,8,12 where declaration order is 8,10,11,12.
That second point is a SECOND screen with a mis-ordered tie, which is what the
question needed, and it immediately kills a candidate: 10 and 11 are kind=0x2002
while 8 and 12 are 0x0000, so "descending kind then declaration index"
reproduces 10,11,8,12 exactly - and then fails both title groups, where every
element of 0x8083 is kind 0 and where 0x80a0 would predict 2,3,4,5,0,1,7 against
a measured 0,2,4,7,1,3,5. Seven candidates refuted now.
16 disc tests green.
Two places checked for the key a .prm element sorts by, both empty:
* the declaration entry's four unread words are constant across every element
of all three measured screens (+28=0, +36=0xffffffff, +56=0, and +44 is a
button ordinal 1-5, 0xffffffff elsewhere);
* the bundle carries NO data for a primitive at all - the menu build declares
pteff00.prm, pteff02.prm and pteff05.t32 and has zero RATC children for any
of them, its 34 children being 21 T8aD sprites and 13 .rat records.
So the layer comes from the game code. But it is consistent, which is what makes
a per-name table a measurement rather than a fudge. Bracketing each unkeyed
element between the keys of its measured neighbours: pteff02.prm falls in
(0x8010, 0x8040) on BOTH screens it appears on, pteff00.prm is past the maximum
on both, and palogo_eff0.prm is below the minimum on the splash.
implied_layer_key records exactly those and nothing else; an unlisted primitive
keeps u32::MAX and still sorts last. With it, derived_paint_order produces the
same layer-key sequence as the order read off the running game on all three
measured screens - primitives included - and matches element-for-element on four
of the five bundle instances. The fifth is the title, differing only inside its
tied groups, which is a separate open question.
This does not make include_primitives safe by default: the 36 builds that come
out one colour are wiped by pzeff00.prm and pceff00.prm, never measured, hence
not in the table.
14 disc tests green.
Read off the running game with screen_children.py and identified by pivot
signature as GP_TITLE.pak ratc-index 8, the NEW GAME / LOAD GAME / TUTORIAL /
OPTIONS / EXTRAS screen:
paint order: 1 3 4 2 5 8 9 6 7 15 10 11 12 13 14 0
It is the first measured screen carrying TWO primitives, and they land in
different places, which is the point. pteff02.prm (the 25% dim) paints 4th,
beneath the whole UI; pteff00.prm (the transition fade, resting transparent)
paints last. Both match their positions on the title screen exactly. So a
primitive's place is per-element and stable by role - backdrop first, dim at
slot 4, fade last - and there are now three permutations to test a derivation
against rather than two.
Wired into measured_paint_order, keyed by element names so both language builds
get it. The English build composited with --primitives edge-correlates at 0.9591
at shift (0,0) against a framebuffer capture taken in the same session - a third
screen confirming paint order, resting pose, fade alpha and primitives at once,
against a capture this project had not seen before.
13 disc tests green.
fill_quad composites an untextured primitive as a solid rectangle of the
keyframes fade colour, pivot x 2 in size, placed and scaled exactly as a sprite
is. Behind ComposeOptions::include_primitives and screen render --primitives.
On the title screen, whose paint order is ground truth, it is measurably right:
mean luminance 76.30 -> 63.72 against the captures 64.58, i.e. from +18% to
-1.3%, and mean absolute difference 16.07 -> 13.08. The background was ~40% too
bright; pteff02.prm, a 25% black dim, was what was missing. The wordmark is not
dimmed by it because the measured order paints that quad at slot 4, beneath the
logo. Edge correlation moves 0.9538 -> 0.9480, which is not informative here: a
uniform dim scales gradients uniformly so a normalised edge score barely sees it.
OFF BY DEFAULT, and that is the finding. A primitive has no T8aD header, so no
layer key, and derived_paint_order sorts the keyless to the end. GP_DIALOGs
pzeff00.prm is a single keyframe of opaque black at full screen; painted last it
wipes the build. Of the 125 builds that draw a visible primitive, 36 come out
>99% one colour with the derived order.
No constant default works, because the two screens read off the running game
disagree: the splash paints its primitive FIRST (the black backdrop) while the
title paints one at slot 4 and another LAST (the fade-out). Declaration order
fails the title too. A disc test measures the damage rather than asserting the
feature works, so the number moves when the ordering is solved.
Also records a false alarm worth keeping: a first pass reported 36 GP_DIALOG
builds at "100% black", which was a crude near-black pixel threshold and not a
black screen - those dialogs are dimmed 50% and perfectly legible. The genuinely
wiped builds are a different set.
blit modulated by tint only, which is 0xffffffff on essentially every keyframe,
so the fade word was decoded, stored and then discarded. Applying it as an ARGB
modulate takes the title composite's edge correlation against the framebuffer
capture of the running game from 0.4597 to 0.9538, both at zero shift. The white
wordmark with its blue outline, the trademark, the copyright and the orange
exploding planet all appear -- the planet because a full-screen blue effect that
rests at alpha 0 had been painting over it at full opacity.
ARGB is measured: across a fade-in the high byte walks 0x00 -> 0x80 -> 0xc0 ->
0xe0 -> 0xff while the low three stay ffffff, and the low 24 bits are 0xffffff on
5276 of the disc's 5453 resting keyframes.
A modulate can only remove pixels, so the risk is a blank screen. Measured: no-op
on 4060 of 5200 sprite elements, partial on 453, hides 687 -- transient HUD
indicators that should not be lit at rest -- and NO build is left with nothing
visible. A disc test asserts that, and that the no-op share stays high, which
also guards the resting rule against drifting onto ramp frames.
Both changes are in one commit because the second is invisible without the first.
Applying fade erased the word PAUSE, which the running-game capture plainly
shows: pgptitle.rat has three runs of two identical keyframes -- pre-roll, hold,
exit -- and the 'later run wins' tie-break grabbed the exit. A group carries the
screen's entry animation AND its exit, so a run ending on the last keyframe is
now excluded unless it is the only one. The title correlation is unchanged.
Element::rest() picked the keyframe with the largest gap to the next keyframe's
time. That reads a keyframe as a value held until the next one; it is the start
of a ramp toward it. A long gap after keyframe k means the screen spends that
time arriving at k+1, so the settled pose is at the far end of the gap.
The title wordmark zooms in over five frames and holds at (184,193) at 100% from
t=251 to t=264. The old rule picked the frame before the long gap: (179,186) at
101%, still mid-zoom.
Measured against the framebuffer capture of the running title screen, which is a
1:1 crop so frame coordinates map directly (confirmed: the copyright line lands
on row 669 in the capture and in both composites). Edge-correlated over the
wordmark box:
plateau (landed) best 0.4597 at shift (0,0)
longest dwell (old) best 0.1511 at shift (+3,+8), 0.1268 at (0,0)
The old composite scores 3x lower and only peaks after being moved, by about the
(-5,-7) that picking kf4 instead of kf5 predicts.
It also fixes six title elements the old rule rested at alpha 0x00 where the
capture plainly shows them, and pteff00.prm - the full-screen fade quad painted
last - which rested at opaque black. That was the blocker on .prm compositing.
Adds tools/re-capture/align_to_capture.py, which is how this was scored, and
turns the .prm test that deliberately asserted the old defect into a guard on
the fix.
Not settled and now the next item: compose ignores the keyframe fade alpha
entirely (blit modulates by tint only), which is why choosing the wrong keyframe
was invisible until now.
The developer-logo splash declares its sprites directly and has no .rat layout
child, so is_build rejected it and no screen command could render it — despite
it being one of only two screens whose paint order has been read off the running
game, and the one where the layer key explains the whole permutation.
Adds ui_layout::is_composable (a declaration table plus at least one element
resolving to a T8aD the bundle carries) and an opt-in --all on screen
list/info/render. Measured on the disc: 2859 RATC bundles, 965 pass is_build,
2751 pass is_composable, and 0 pass is_build without passing it — a strict
superset. It is opt-in because the 1786 extra bundles are mostly two-element
fragments (a button and its glow), and because widening the default would
renumber --build for every pak, invalidating the build indices the corpus's
notes cite by number.
The splash now renders 6/7 elements, painting its glows first in the order
measured off the game; a disc test pins that order.
`compose` skips focused-state records by default, so whatever that flag matches
vanishes from every composite. It matched a trailing `f` in the name and nothing
else, which is not the convention — it is a letter.
Measured across the disc: 2458 elements match the suffix and only 54 of them
have the base element they would be the focused version of; all 54 are
`pgmenu_btnNNf.t32`. The other 2404, spread over 864 bundles, are `_eff` glow
layers whose names end in the same letter — `pb_name_eff.t32` alone accounts for
1122 of them, and `palogo_gamearts_eff.t32` is one the draw capture shows the
game painting, before its logo, with its own layer key.
Requiring the pair recovers 587 glows in the composable builds. GP_OPTIONS is
the plainest case: without `pbmwindow_eff.t32` the panel had no window at all,
just two floating brackets (captures/ui-layout/options-with{out,}-its-glow-layer.png).
Two other candidate rules were tried and refuted first. The `opt` link is not a
focus link: 221 elements are opt-targets, only 2 of them suffix-match, and the
targets include `pjnet_bg.rat` and `pv_loading_loop1.rat`. And the suffix alone,
as shown above, is unrelated to anything.
compose now sorts elements by the word at +0x08 of their sprite's T8aD header
instead of painting in declaration order, for every build except the two whose
measured order is hard-coded. That word is non-decreasing in the order the game
actually paints both measured screens, so every screen nobody has captured now
gets its layering from the file rather than from the declaration table, which is
provably not the paint order.
Verified with artifacts and both ways, not by a green build: the disc test
asserts the measured orders never invert the key and that the composite's key
sequence is sorted, and reading the word from +0x0c instead makes it fail; the
title composites identically; and GP_MISSION_SELECT — uncaptured — now composites
cleanly, committed as a capture.
Two things recorded rather than smoothed over: ties keep declaration order
because the game breaks them some other way that is not known, and the
developer-logo splash has no .rat child, so is_build rejects it and the
compositor never sees that bundle at all — its measured order is inert in
practice and screen render cannot draw it.
Checking the blast radius of the previous commit's `kind = 0x4` skip found 174
elements on the disc that are 0x4 with no non-0x4 element of the same sprite —
GP_READY_ROOM pak entry 75 is 56 elements and every one is 0x4, a list of real
icons a blanket skip would erase. So the rule is narrowed: skip an instance only
when its template is present, which is the case the title capture covers. The
title render is byte-identical before and after, so the narrowing changes nothing
the evidence covers.
And the claim that this "caught a regression" is WITHDRAWN, because measuring it
refuted it: none of those 174 elements is in a bundle `is_build` accepts, so none
of them ever reaches the compositor. The blanket skip would have been harmless in
practice. The narrow rule is a precaution, not a fix, and the comment says so.
The test that was going to assert the regression could not find such a build —
correctly, since none is composable. It is replaced by one that pins the fact
which makes the narrow rule safe: no composable build on the disc has a 0x4
element without its template, checked across 500+ builds. If that stops being
true it fails, instead of a screen quietly going empty.
The compositor painted in declaration order, which the draw capture proved wrong
— the title's background is declared ninth and paints first. The order the game
uses is its runtime child list, and no decoded field reproduces it, so rather
than leave the screen composited wrongly this lands the orders that HAVE been
read off the running game and falls back to declaration order everywhere else.
Two builds are covered: GP_TITLE's title build and the GAME ARTS / SETA / studio
anima splash. Keyed by element names, which identify a build across paks and
language variants.
Rendering it exposed a second defect, and the same capture settles it: the
kind = 0x4 elements are motion-trail ghosts, not resting content. The bundle
declares three instances of each wordmark; the capture shows exactly ONE quad at
each wordmark's position. Drawing them at their resting keyframe put three
oversized PROJECT SYLPHEED copies across the composite. They are now skipped.
Verified with an artifact, not a green build: the composite is committed
(captures/title-composited-measured-order.png) and now reads as the title screen
— background, planet, ship, wordmark, TM, copyright, correctly layered.
The test is disc-gated and was checked BOTH ways: it passes as landed, and
disabling the order table makes it fail. It reads one pak rather than every
build on the disc — the first version used the all-builds helper and got the
test process OOM-killed running alongside the other three.
`static_assembly_matches_runtime_capture` walks the capture's parts and looks
each up in ours, so an extra static placement was originally invisible to it —
that is how `e303_wep_01` sat here decoded 100x too large. `64d372c` closed that
with a set-of-names check; BACKLOG.md still says otherwise, and it is wrong.
One direction stayed open, though: a set cannot fail on a resource placed
*twice* when the capture lists it once. That is not hypothetical — a duplicated
instance is what a bad node walk emits, and the two legitimate duplicates here
(`e106_eng_01`, `e303_wep_01`) are why the test had to special-case counts at
all. So pin the whole multiset — 9 resources, 11 placements, against the
capture's 8 dedup'd parts — which subsumes both hand-written counts and fails on
an extra, a missing, or a duplicated placement.
Refuted before believing: perturbing the expected table (declaring
`e106_bdy_01` twice) fails with the real multiset on the left, so the assertion
runs against the disc rather than skipping for want of SYLPHEED_ISO.
The remaining `for want in &cap.parts` loop guards the table itself: editing it
cannot quietly drop a part the runtime actually draws.
The title screen needs an order that puts element 13 (`ptbase2.t32`, the
full-screen background) behind elements 0-5 (the wordmarks) — the capture shows
the wordmarks on top, so the declaration table is not it. Two other orderings
the bundle carries were the cheap candidates, and both are now dead:
* the **placement region** stores a keyframe group per element with an explicit
element index, so it could be a second ordering. It is not — it equals the
declaration order on every build on the disc. `UiBuild::placement_order`
exposes it and `placement_region_order_is_never_a_second_ordering` pins it, so
the refutation stays checkable instead of remembered.
* the **RATC child order** is the declaration order with the `.prm` elements
absent — strictly less information, and no place to put the background other
than where the table already puts it.
`screen info --geometry` prints both, plus each element's decoded sprite size
beside `pivot*2` and every keyframe's scale/position/time — the numbers a
placement hypothesis has to be tested against, and how the pivot/scale rule in
the previous commit was found.
`title_background_is_full_screen` pins that rule against the disc rather than a
synthetic sprite. `scaled_elements_are_a_small_and_mostly_undiscriminating_minority`
reports the scope honestly: 865 of 5 130 resting placements are scaled at all,
and only 213 of those could tell "about the pivot" from "about the sprite
centre" — which the capture did *not* settle, because `ptbase2`'s pivot is its
centre. It also counts how far `pivot*2` is from the decoded size disc-wide
(2 521 agree, 1 884 are off by more than 16 px), which demotes the "pivot is
exactly half the texture" result to a property of the tutorial bundle.
The compositor read a keyframe as `top-left = (X,Y)`, `size = decoded · scale`
and ignored the declared pivot. That is right at 100 %, which is every element
the format was ever checked against — the pause menu, the ARSENAL chip ruler —
and wrong for every element that is scaled.
Measured against a framebuffer capture of Canary on the title screen.
`GP_TITLE.pak` build 7 element 13 is `ptbase2.t32`: 640x360, pivot (320,180),
one keyframe at (320,180) with scale 200 %. From the corner that is a 1280x720
rect at 320..1600 x 180..900 — a quarter-screen slab with the top-left quadrant
bare. Anchored at the pivot it is (0,0)..(1280,720), and the capture shows the
background art reaching all four edges. Normalised cross-correlation of the
composite against the capture, searched over +-40 px, peaks at (0,0): 0.90 on
the planet limb, 0.72 on the lower-left ship.
`ptcopyright.t32` calibrates the other half: unscaled, 694x20 at (293,655), and
the capture's glyph run is x 295..986 / y 700..718 once the 45 px of window
chrome is taken off. So the keyframe really is the top-left at 1:1.
Disc-wide this moves 865 of 5 130 resting placements. The pause menu's own
`pgpeff01` glow stops hanging off the menu frame to the bottom-right and
centres on it.
`ComposeOptions::backdrop` comes with it: the default dim slate stands in for
the PRMD dim-quad behind an in-mission screen, but comparing against a
framebuffer needs the black the game actually composites over, so
`screen render --black` can ask for it.
Two independent lines landed a `.rat` reading and neither was the whole
picture, so this merges them into one module and fixes what the merge exposed.
ui_layout — the screen is the BUNDLE, not the set of .rat records
------------------------------------------------------------------
`feat/ui-layout-preview` parsed `.rat` records; the autopilot stack documented
the RATC header and probed it in `examples/screen_layout.rs` but never landed a
library module. The `.rat`-only reading structurally cannot see an element that
has no record -- the `eff*` frame corners, the `deli*` dividers, `msg` -- which
is exactly what the committed real-vs-rebuilt capture shows missing. Rebuilt
around the header:
* element declaration table at 0x20 (60-byte entries: name, parent index at
+32, kind flags, pivot) = the back-to-front draw list;
* the placement region after it = per-element keyframe groups.
Verified against the disc, each against a fact the docs state independently:
`pgpeff02a` -> parent 3 = `pgpeff02`; `pgp_ttrl_btn10` rests at (546,288); the
pause buttons sit at 268/337/407/478, the documented 70 px pitch; the Arsenal
carries X = -516. The tutorial PAUSE menu now composites 11/11 elements and
matches the real screen more closely than the earlier rebuild did.
Three defects found while validating, none of which any test would have caught:
* the keyframe block is 40 bytes with X/Y/time at +28/+32/+36 and an
alpha-ramping ARGB at +0 -- the fade, previously unread;
* a group's data stops 4 bytes short of its last block's time slot, so that
word is the NEXT group's element index. Reading it produced times like
1869640736 and silently corrupted the max-dwell pick. Last-frame time is
now `None`;
* the `.rat` sprite-name field is not 16 bytes. Capping it there truncated
`pgp_ttrl_title.t32` to `pgp_ttrl_title.t`, which resolved against nothing
and dropped 4 of 11 tutorial elements from the composite.
Max-dwell also needed a tie-break: on equal gaps take the LATER frame, or
`pgpmsg` reports the y=645 fly-through instead of the y=605 it settles at.
savegame -- a Rust port of tools/re-capture/savegame.py
------------------------------------------------------
GDHA container, zlib payload, chunk stream (GDAA / phase / GHAD 122 B / 16x20 B
SHAB / trailer). Every GHAD word carries its own confidence rather than the
block being presented as solved: 6 named, 2 recorded as REFUTED (+36, +56 were
tested as difficulty and as stage and are neither), 7 still unknown.
Tested against the three real saves committed under docs/re/captures -- no disc
and no emulator needed. The load-bearing assertion is the byte-identical
round-trip; the develop differential is asserted as a property (spending 4000 P
moves +24 and not its twin +28, steps the clear ratio, and moves exactly two
blob entries), and the header summary is checked to agree with the payload it
mirrors -- the trap that makes the Details panel a bad oracle.
CLI: `screen list|info|render` and `save info`, so both are checkable headlessly
in the same spirit as `mesh render`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
With per-sub-mesh declarations enabled, n201_01 decoded as a 2-part fragment 4 bytes
off. Both starts validate for the pivot — 0x32BA718 at pad 2 (earlier in file order,
so first-match took it) and the capture-proven 0x32BA71C at pad 0 — so the pivot
alone cannot separate them; at the early one two of four sub-meshes fall out as
out-of-range.
anchor_grouped_meshes now builds each accepted candidate and keeps the one that
explains the most of the declared pool: it returns immediately when a candidate
explains all n sub-meshes, else keeps the best partial, so it can never decode less
than first-match did. n201_01 lands on all four capture-proven offsets
(0x32BA71C / 0x32BEFF4 / 0x32C416C / 0x32C536C) and its two sibling copies take their
own pools, so the twin collapse is gone.
XBG7_SUBMESH_DECLS is therefore on by default (=0 reverts):
resources that never decode 85 -> 47
resources decoding in no container 63 -> 30
degenerate index runs 1 -> 1 (unchanged)
cross-container minority decodes 96 -> 96 (unchanged)
captured index runs, stage-02 93/93 (unchanged)
captured index runs, stage-05 124/128 -> 128/128
The last line is the point: the buffers the capture could not name are the n201
family, and they now decode and match the GPU's indices byte for byte. Suite green
including twin_pairs_do_not_share_a_buffer, apart from the pre-existing
known-failing cross-container consistency test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
desc_dump shows each index marker is followed by its OWN element triples: n201_01
declares strides 24, 24, 24 and 28 (the last sub-mesh has a fourth element), which
matches the runtime capture's stride=28 on that draw and the four distinct vertex
shaders. parse_vertex_decl read the first declaration for the whole pool.
all_vertex_decls reads one per marker; anchor_grouped_meshes uses each sub-mesh's
own stride for the pool walk, the pivot validation and the read. debug_grouped_report
follows the same setting so the diagnostic cannot blame the wrong gate — at n201_01's
capture-proven pool start it now reports "pad 0: ACCEPTED" instead of a NaN position.
With XBG7_SUBMESH_DECLS=1: resources that never decode 85 -> 47, resources decoding
in no container 63 -> 30, capture oracles unchanged (93/93 index runs, 42/42 index
counts), consistency unchanged at 96.
Off by default because selection has not caught up: the three n201_0x copies then
settle on one pool (twin_pairs_do_not_share_a_buffer fails), production still picks
a start 4 bytes before sub-mesh #1 rather than the proven one even though the proven
start validates and is unclaimed, and four newly decoded ptc_pack .dat composites
carry degenerate triangles. Format truth is settled; choosing among candidates is
the remaining work.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
Two follow-ups to the pad-scoring fix, both driven by the same invariant (a
correctly located index run has no degenerate triangles):
- anchor_grouped_meshes picked its pad by first-match too; scoring the pivot run
the same way cleared every remaining ptc_pack composite (f102/f104/e107).
- anchor_pool_mesh now prefers a degenerate-free candidate over an earlier dirty
one. examples/better_home.rs showed the last two resources each had exactly one
degenerate-free, pool-covering block, sitting later in file order than the
lookalike we took. First-match order is kept for every clean hit, and a dirty
block is still used if nothing clean exists, so coverage cannot regress.
degenerate index runs, disc-wide: 582 -> 11 -> 1
captured index runs identical: 93/93 (unchanged)
resources decoded / misses: 6 209 / 85 (unchanged)
index runs changed / anchors moved: 590 / 10 (_rou_f402_dead x8, e201_bdy_03_m x2)
Cross-container minority decodes 89 -> 96, and that is progress: all seven new
rows are _rou_f402_dead, which now has a majority (32x25x8) for the first time, so
its seven wrong copies are named instead of hidden behind "no majority".
The last dirty run (_rou_f402_dead in Stage_S09) is blocked by distinct assignment
— its clean block is claimed by e_rou_f003_Near, both 24-vertex bounding boxes. A
winding-floor escalation for that case was written, measured to fire for nothing,
and reverted; the reasoning is kept as a comment.
Regression threshold tightened to 1. Suite green with --include-ignored apart from
the pre-existing known-failing cross-container consistency test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
Extending the capture comparison from index COUNTS to index VALUES
(`examples/capture_index_bytes.rs`, using the batch offsets the new ib logging
gives) showed 76 of 93 Stage_S02 index runs identical to the GPU's and 17
differing — every difference a shift by exactly one element, on buffers whose
index data sits at pad 2.
`anchor_pool_mesh` returned the FIRST pad that validated, and pad 0 is tried
first with the looser winding gate (0.70 vs 0.85). Read at pad 0, a pad-2 block
yields [true[1], true[2], …, garbage]: every index in range, the pool covered,
the positions right, the winding often just above 0.70 — so it validated, and
every triangle was mis-wired. Nothing count-based could see it.
The signature is decidable without the capture: a shifted run wires arbitrary
vertices, so triangles come out degenerate. 282 of 283 correctly anchored
Stage_S02 blocks have zero degenerate triangles, while the shifted readings carry
1–2 156. So score every validating pad by (degenerate triangles, then winding)
and keep the best. `XBG7_PAD_FIRST_MATCH=1` restores the old behaviour.
captured index runs identical: 76/93 -> 93/93 (2 025 elements)
decoded runs with a degenerate triangle: 579 -> 16 (disc-wide)
sub-meshes whose index run changed: 575 of 8 850
resources decoded / vertex anchors / consistency: unchanged (6 209 / same vb / 89)
Locked in by tests/mesh_disc.rs::decoded_index_runs_have_almost_no_degenerate_triangles.
Suite green with --include-ignored apart from the pre-existing known-failing
cross-container consistency test (the 24-vertex bounding-box class).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
