37bfdcabdb4e5e9b8e43ec88a8878f6863ff7df0
200 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
sylph-pi
|
37bfdcabdb |
fix(viewer,cli,export): clear the remaining 30 clippy lints
`cargo clippy --workspace -- -D warnings` now exits 0. `cargo test --workspace` still reports 207 passed, 0 failed, 14 ignored across 30 suites — identical to runs 203 and 204, so none of this changed behaviour. The workspace total was 73, not the 48 run 204 reported. `-D warnings` turns a lint into a hard compile error, so `sylpheed-formats` failing stopped its dependents from ever being built: `sylpheed-viewer` (14) and `sylpheed-cli` (11) had never been linted by anyone. Clearing formats in |
||
|
sylph-pi
|
4ea29ec9de |
fix(formats): clear all 43 clippy lints in sylpheed-formats
Run 204 gave this repository its first clippy measurement — 48 errors, 43 of them in `sylpheed-formats`. This clears that 43 to zero under the exact invocation CI runs, `cargo clippy -p sylpheed-formats -- -D warnings`. Why this crate first, and why it is safe to touch: Every one of the 43 sites was checked against the line ranges that `auto/frame-blend-draw-path` (495 commits) and `auto/port-p6-audio` (366) actually modify. None of them overlap. Eleven of the fifteen affected files are byte-identical on both branches, including `mesh.rs` and `texture.rs`, which carry 28 of the hits between them. The three sites in `audio.rs`, `ui_layout.rs` and `slb.rs` that live in files those branches do change fall outside every modified hunk. The collision argument that defers #12 does not transfer here; it was tested rather than assumed. It also unblocks a measurement. `-D warnings` turns a lint in this crate into a hard compile error, so its dependents never build — `sylpheed-cli` and `sylpheed-viewer` have never been linted at all, and viewer is the largest crate in the workspace. Both depend only on `sylpheed-formats` (`sylpheed-export` pins it from a git tag instead), so this commit is what makes their real counts knowable. 38 applied by `cargo clippy --fix` — chunks_exact_to_as_chunks, manual_div_ceil / is_multiple_of / range_contains, unnecessary_map_or, needless_borrow, let_and_return, dead_code, unused_mut/variables. Purely local expression rewrites: 38 insertions, 39 deletions. 2 by hand: a doc continuation that markdown was parsing as a list, and `d / frame` behind a `frame > 0` guard becoming `checked_div`. 3 `#[allow(clippy::too_many_arguments)]` with a stated reason. On those three allows: 8 parameters against a threshold of 7, in the mesh anchor path. The real fix is a shared params struct across `anchor_pool_mesh`, `validate_block` and `validate_block_report` — the latter two take the same eight arguments and one delegates to the other — which is a change to the decoder's signatures and belongs to whoever owns that path, not to a CI-lint pass. This is not the shape PROTOCOL.md forbids. `continue-on-error` suppresses everything, present and future, at the job level, and cannot tell "not yet" from "no longer". A site-local `#[allow]` with a reason is a decision recorded where it applies: one lint, one function, and any new violation anywhere else still fails the build. `sylpheed-export`'s remaining 5 are deliberately untouched — three of them sit inside hunks both long-lived branches modify, and that crate blocks nothing. Left for #13. Refs #13 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01McNbzUeq1KRBWs4G6X2YVj |
||
|
MechaCat02
|
c3758e3850 |
port: land the play-tested work, and only that
Takes the port branch up to |
||
|
MechaCat02
|
d1685d67c9 |
viewer: show where a cutscene's voice actually is, and let you hear it
Some checks failed
The Cutscenes window printed the voice token as text and offered no way to play it, which left the most confusing thing on the disc invisible. The movie voices are one continuous XMA stream chunked into VOICE_*.slb entries whose boundaries do NOT match the cutscene cues, so the bank named after a movie need not hold that movie's audio. Measured, on the retail disc: ADV region 433930240..437044592 inside VOICE_ADV.slb name honest S00A region 452798464..455499120 inside VOICE_S00A.slb name honest RT01A region 437044592..437345648 inside VOICE_ADV.slb NAME LIES RT01A's voice sits in bytes belonging to the entry named after the intro movie. A viewer that played the name-matched bank would be confidently wrong for exactly the cutscenes where it matters, and would look right on the two that are easiest to check. So the window now shows BOTH locations -- the named bank with its byte range, and the resolved region -- and states plainly whether the name is honest, highlighting it when it is not. Play routes through the movie form of RequestAudio, which resolves the region rather than reading the bank. Static data only: sound.pak and tables.pak, both on the disc. |
||
|
MechaCat02
|
65cefa74c3 |
monorepo: one repository for the decoders, the port and the corpus
Some checks failed
Merges the Godot port into the reverse-engineering repository, preserving both
histories -- 1019 commits of corpus plus the port's 31, brought in by subtree
merge and then moved into place so git can follow each file across the rename.
The reason is not tidiness. The two-repo split forced the exporter to depend on
the decoders by pinned revision, and that created a whole class of failure that
now disappears: a sha reachable only from a topic branch, orphaned by a
squash-merge, breaking a fresh checkout silently at build time. It also forced a
live read-only mount of one agent's working tree into another's container, which
is why a contract file could move mid-iteration. With a path dependency, a
decoder change and the exporter change it requires land in the same commit or
not at all.
Canary stays separate: it is a fork tracking upstream.
New structure for the long term:
docs/game/ how the game is NAVIGATED -- menus, modals, prompts, alerts,
and in-game flight. Written so nobody rediscovers it. Mostly
open questions on purpose; the in-game tutorials are the
resource for the flight half.
docs/port/MODDING.md
modding as a constraint on the exporter TODAY, not a later
feature: one logical asset in one file (the disc splits nearly
everything, and resolving that is the exporter's job), names a
person recognises, PNG/OGG/OGV/JSON only, base-and-overrides so
re-exporting is always safe, provenance in every file.
data/base + data/mods
generated tree and drop-in overrides, both gitignored
exchange/ transient inter-agent files, deliberately outside history
docs/agents/ the team protocol
Both the README and the navigation doc lead with the correction that cost the
most: the oracle is the real game under Xenia Canary. Reborn's renderer is a
hypothesis under test, it has been wrong, and treating it as ground truth
propagated into three documents and both agents before a human caught it.
Scripted modding stays possible without being built: no screen name is hardcoded
in GDScript and there is no native code in port/, which is what Godot Mod Loader
needs to be able to substitute behaviour later.
|
||
|
Sylpheed RE agent
|
7eeae3006a |
re(ui): the focus ring SPINS, the game draws it, and the leaf owns the f record
Three things, all from parsing ptbtn0Nf.rat as a build. 1. THE RING SPINS. Its two keyframes differ in exactly one field: rotation_deg ramps 0 -> 360 with position, scale, alpha and tint all constant. A spin in place, the same shape as the GP_BUNK example already recorded. 2. THE ORACLE CONFIRMS THE GAME RENDERS IT. In the OPTIONS-focused capture the ring's bright head sits in a completely different angular position from the sprite's own -- caught mid-spin. This is a SECOND independent confirmation that rotation_deg is drawn, now on a different screen and a different element from the ptloop sweeps, and it raises rotation's priority: it is not a title-only concern that sits off-screen at rest, it is the main menu's focus marker. NO ANGLE IS QUOTED. A brightest-region centroid says ~250 deg, but the control refuses that precision -- rotating the sprite by a known 30/90/180/270 and re-measuring gives errors up to 19.8 deg. What survives the error bar is that a <=20 deg error cannot manufacture a ~250 deg displacement. 3. WHICH PLACEMENT WINS -- correcting this page's own earlier caveat, which said to use the leaf only for elements the parent does not declare. Right for a BASE record, wrong for an f record: the parent declares NO element for ptbtn0Nf.rat at all (zero of build 5's 16), so the f record's placement comes from its leaf for BOTH elements, label included. The label's (-7,-7) is load-bearing -- the f sprite is 13px larger per axis and -7 keeps them concentric (535+96/2 = 583 vs 542+83/2 = 583.5). Corroborated against the oracle: the focused-minus-unfocused region is x 505..703, and the leaf predicts a right edge near 707 where the parent reading predicts 714. Also exposes UiBuild::records (name -> (offset, size) of a nested .rat leaf). Nested records were parsed into a PRIVATE map, so a consumer holding a UiBuild could not locate a leaf's bytes at all -- which is exactly what blocked the port from reaching the ring. |
||
|
Sylpheed RE agent
|
d110cf38c7 |
media: expose se_wave_riff -- the menu's SE cues, assembled where the format lives
The port is forbidden from reimplementing media assembly and Static.slb is exactly that case: no RIFF, no seek chunk, no XACT container, just a packed run of whole 2048-byte XMA1 packets, so a wave is defined only by (offset, packet count) and the header has to be synthesized. That step now happens once, in the crate that owns the format, instead of in each consumer. `slb::xma1_wave_riff` wraps raw packets; `media::se_wave_riff` looks the bank up and reads just the packets asked for. Both reuse the existing synth_xma1_fmt / build_riff, which are already byte-identical to what tools/re-capture/ slb_extract_wave.py writes -- so this is exposure, not a second implementation. It reads a TARGETED range rather than the whole bank, and that is load-bearing: Static.slb is the ONE entry of sound.pak's 9 519 whose declared extent runs past the end of the extracted segments -- by exactly 616 768 B -- so reading it whole fails outright on this extraction. Every cue we need is in the first few hundred KB. Recorded rather than worked around silently. Verified as an artifact, not a compile: all three cues decode through ffmpeg to mono 48 kHz PCM at 0.533 / 0.344 / 1.016 s, non-silent (rms 2085 / 2985 / 4327, peaks 29813 / 16973 / 32767). The refusal path is exercised in the same run -- an impossible packet count is rejected rather than returning a short stream, because a truncated XMA decodes to plausible-sounding garbage. Also adds docs/re/captures/ORACLE-CAPTURES.md: an index of the nine canary framebuffer captures already in this repo, and a plain statement that THEY are the reference and `screen render` is not. |
||
|
Sylpheed RE agent
|
b21c8e4118 |
re(ui): the focus ring's position is decoded -- a .rat leaf parses as a build
The port needed ptbtneff01.t32's placement and was about to author it from an eyeballed PNG measurement. It does not have to: a `.rat` leaf needs no new reader. Its first 32 bytes have a bundle header's shape -- "RATC", 0x3c declaration-entry size at +4, element count at +20, design 1280x720 at +24/+28 -- so ui_layout::parse_build reads it unchanged. The control is the base record, whose position is known independently: the parent screen reports ptbtn01.rat resting at (542,162), and parsing the leaf alone returns ptbtn01.t32 at (542,162). It reproduces all five buttons. Positions are absolute design-space top-left. The ring rests at (500, 156/236/ 316/396/476) for buttons 1-5 -- a uniform (-42,-6) from each button's own rest, identical in the Japanese bundle. The bright label is a uniform (-7,-7). Two things recorded rather than smoothed over: a leaf's placement DUPLICATES the parent's rather than being relative to it, and the two copies are not always byte-equal (ptbtn04's parent says y=401, its leaf says 402) -- the parent is what compose honours, so the leaf is the source only for elements the parent does not declare, which is exactly the ring. And `screen render --focus` is blind to the ring for the same reason the port's exporter was: el.focused is name-based on top-level elements and neither walks into the leaf. |
||
|
Sylpheed RE agent
|
f817dd5939 |
re(ui): the 60 nameless RATC children are frames, not children -- .tan decoded
Closes the reach caveat the `opt ` name fix left behind: 60 of 18 002 RATC children carry no `opt ` block, and it was not established whether they lack one or sit past our 128-byte window. Neither. They are not children. `examples/ratc_optless_children.rs` re-runs `ratc::parse`'s own guards over the disc and reports which one fired: all 60 are "tag beyond the window", none is rejected by length, gap or charset, none is child #0, and all 60 live in six bundles of one archive. Within a bundle the distances back to the nearest tag are an exact arithmetic progression, step 60 600 -- ten different records finding the SAME tag, because there is only one. Reading a bundle directly: children 1..10 are equal-size T8aD blocks under a single `opt ` name, `pb_f15_eg_anm.tan`. `.tan` is a FRAME SEQUENCE. One block declares the resource; its payload is a run of T8aD frames. Disc-wide, over all 18 718 `opt ` names in all 33 paks: a RATC bundle names exactly six kinds of resource -- `.t32` 14 756, `.rat` 3 311, `.prm` 367, `.tbm` 224, `.sbo` 54, `.tan` 6. Six `.tan`, ten frames each = 60, the entire population with nothing left over. The negative is closed, not narrowed. Consequence recorded but deliberately not fixed: `ratc::parse` over-reports there, listing a `.tan`'s frames as anonymous children. Nothing in the menu milestone reads a `.tan` -- it occurs only in GP_READY_ROOM, which S1 ruled out -- so no screen the port draws changes. Also a METHOD entry for this container OOM-killing `slb_leading_segment_disc` under default test parallelism (SIGKILL, no assertion; 8/8 pass with --test-threads=1). |
||
|
Sylpheed RE agent
|
56cc7acfc3 |
re(ui): a RATC child's name is stated, not inferred -- and it was hiding every menu background
`ratc::parse` named each child by scanning backwards for the last printable run of bytes before its magic. The format states the name explicitly instead, in an `opt ` block: `"opt " | BE32 len | name | NUL | 3 bytes | magic` -- the same block `ui_layout::opt_link` already read for a button's focus link. The scan agrees with it 17 918 times out of 17 942 and is wrong 24 times, every one the same failure: the 3 trailing payload bytes are themselves printable and beat the real name. For `pteff05.t32` those bytes are `38 41 58` = `8AX`, so the full-resolution background of all five menu screens registered under a name no element declares, resolved to no sprite, and `compose` dropped it through an early `continue` that -- unlike the two arms above it -- records nothing. The screen lost its background and `screen render` still reported "all resolved". `8AX` was never a name. Docs that treated it as one are corrected here. Disc-wide, and the control is the 17 918 the scan already got right: the `opt ` reading reproduces every one of them. Effect on the five screens is the signature of the same art at twice the resolution -- mean brightness unmoved, high-frequency detail x1.15..x1.30 -- which is what the separately-measured `ui-8ax-fullres-background` result said the game draws. Also closes a long-standing dangling reference: `pmbase.t32`, recorded as "on the disc nowhere", is the `GP_STAGE_CLEAR` child the scan called `8AX`. RATC sibling references now resolve 10 148 of 10 148. Verified: 114/114 sylpheed-formats unit tests (including two new ones pinning the `8AX` case byte for byte and the no-block fallback), and every disc-gated integration suite in sylpheed-formats/sylpheed-cli. |
||
|
Sylpheed RE agent
|
1f2b469f7e |
re(ui): a static composite is only meaningful for a screen that settles
The model's sharpest prediction, tested with its control. The draw log says that on the developer splash the _eff glows are drawn on frames 94-115 and the logos on 116-211, so at the moment the reference capture was taken EVERY glow is already finished -- including the two that have plateaus and which rest_plateau therefore renders visible. Suppressing them should help on the splashes and hurt where a screen genuinely settles. publisher splash +0.9604 -> +0.9982 +0.0377 developer splash +0.9659 -> +0.9980 +0.0321 title (control) +0.9500 -> +0.9480 -0.0020 main menu(control) +0.9460 -> +0.8544 -0.0916 EXTRAS (control) +0.9440 -> +0.8370 -0.1070 Both splashes jump to about 0.998; all three persistent screens get worse. The control is what makes this a finding rather than a coincidence: the same edit helps exactly where the model says it should and hurts exactly where it says it should not. So rest_plateau is not over-drawing in general -- it over-draws on TRANSIENT screens. A plateau mid-animation means the element is held at that point in the timeline, not that it is on screen once the screen has settled. Where a screen settles, the held pose IS the settled pose and the rule is measurably right. And that answers the question left open several iterations ago -- what "rest" means for a transient element. It does not mean anything: the splashes never rest. A static composite of them can match a chosen frame, and about 0.998 is what these captures' frame is worth, but the format does not answer a question the screen never poses. For the port: play the timeline for the two splashes, which the settled keyframe timing now supports, and composite statically for title, main menu and EXTRAS. METHOD: an edit that improves one set of cases is only interesting once you have shown it damages the cases where it should. |
||
|
Sylpheed RE agent
|
4012d5b555 |
re(ui): why rest_plateau is right -- and last is right only for a transient
The shifted keyframe-time reading looked like it implied something simple: the final pose is reached at a definite time and nothing follows, so rest should just be the last keyframe and the plateau heuristic could go. Tested by applying it to EVERY element: title +0.9500 -> +0.6819 -0.2681 main menu +0.9460 -> +0.6416 -0.3044 EXTRAS +0.9440 -> +0.5745 -0.3695 publisher splash +0.9600 -> blank (zero variance, corr undefined) developer splash +0.9643 -> blank Refuted, and the failure supplies the model. A group is entry -> hold -> exit, and the exit is the screen's DISMISSAL. While a screen is displayed it has not reached its last keyframe; it is sitting at the hold. So rest_plateau is the correct primary rule, and the last keyframe is the post-exit state -- correct only once the screen is gone, which is why applying it everywhere blanks the splashes. This does not contradict the shifted reading. That reading says when each pose is reached; it says nothing about the group being played to completion while the screen is still up. The step between them was mine. And it explains why last wins for the two plateau-less elements: an element with no hold is a transient, it flashes and is over, and at any settled moment it is gone -- which is its last keyframe. The draw capture says the same independently: on the developer splash the _eff glows draw on frames 94-115 and the logos on 116-211, so the glows are already finished when the logos are up. Three independent observables -- animation timing, static composites, and the per-frame draw log -- now agree on one rule: plateau where there is one, last keyframe where there is not. METHOD: a blank render is a NaN correlation, not a low score, and that NaN was the strongest form of the result; and when a model predicts something the measurement refuses, suspect the step you supplied between them. |
||
|
Sylpheed RE agent
|
93e9b185ea |
re(ui): the rest fallback fires on 2 elements, and "last keyframe" wins there
Scored candidate rest-pose rules by rendering and correlating instead of
arguing, and both results correct something I had published.
First, the exposure. The guessing fallback is reached only by an element
that is plateau-less AND multi-keyframe -- a single-keyframe element
short-circuits at `match len { 1 => first }`. Per screen:
title (4) 24 elements 2 plateau-less 0 reach the fallback
main menu (5) 16 5 0
EXTRAS (6) 18 5 0
publisher splash (10) 3 2 1
developer splash (11) 7 2 1
So on the three screens the port cares most about, rest() never guesses.
That is why three different rules render builds 4/5/6 to identical
correlations -- the code is unreachable there, which I nearly read as
"the choice does not matter".
Second, where it does fire, the last keyframe is markedly better:
publisher splash dwell +0.9600 last +0.9982 maxalpha +0.9600
developer splash dwell +0.9643 last +0.9758 maxalpha +0.9643
That refutes my own earlier refutation. I had killed the last-keyframe
rule by arguing it makes palogo_anima_eff invisible while its two
siblings stay lit, which looked like an artefact. The capture says
otherwise: making it invisible is what improves the match. The sibling
symmetry was my expectation, not evidence.
Caveat kept in front: both captures are single frames of a transient
animation, so this fixes which pose matches THOSE frames, not which is
canonically at rest. Default unchanged -- better on both screens where it
fires and identical on the other three, but it would move 2 305 elements
disc-wide on two measurements. Reachable via SYLPHEED_REST_RULE=last.
Also confirmed: all 195 zero-scale rest poses are inside the corrected
2 305 ambiguous population; none is a single-keyframe element.
METHOD: score a rule where it can differ, or you measure nothing; and an
argument from symmetry is a prediction, not a refutation.
|
||
|
Sylpheed RE agent
|
0265da31a1 |
re(ui): refute my own fix for rest(), and correct the defect rate by 65%
Two corrections from one experiment.
A keyframe group is entry -> hold -> exit, and the exit ends invisible:
on the five port screens the final keyframe is invisible for 21/24
(title), 8/16 (main menu), 12/18 (EXTRAS), 2/3 and 6/7 (splashes). So the
screen as seen is the HOLD, which is why rest_plateau is the right
primary rule and why "rest = last keyframe" would empty every screen.
That suggested a fix: an element with no hold has no representative pose,
so draw nothing rather than guess an endpoint. Tested through compose's
visible mask and correlated against the live captures:
title +0.9500 -> +0.6839 -0.2661
main menu +0.9460 -> +0.9037 -0.0423
EXTRAS +0.9440 -> +0.9094 -0.0346
Refuted on all three, and the reason invalidates a number I published. An
element with a SINGLE keyframe has no adjacent pair, so the plateau test
marks it plateau-less -- but its one pose is unambiguously its rest.
Suppressing those removes backgrounds and full-screen layers, which is
the title's -0.27.
no plateau (as published) 3 807 (24.57 %)
... single-keyframe 1 502 trivially at rest, not a guess
genuinely ambiguous 2 305 (14.88 %)
So rest() guesses for 2 305 elements, not 3 807 -- the figure I gave the
port overstated the defect by 65%. Corrected in HANDOFF and the page.
METHOD: a predicate over adjacent PAIRS silently misclassifies a
one-element list; and acting on a claim is a better test of it than
re-reading it -- this flaw survived a census, a write-up and a handoff
row, and died the moment the rule was used to change a rendering.
|
||
|
Sylpheed RE agent
|
7373035868 |
re: the port was still being told SE audio is undecodable -- it is not
A resolve-check on HANDOFF's own rows. Q8 read "SE audio is undecodable
from the disc -- no XACT container exists anywhere". menu-audio-cues.md
retracted exactly that ("### Retracting 'cannot be extracted'") and
locates three cues in Static.slb that decode to PCM: d-pad move 0x1ec0
(4 packets), (B) back 0x0ec0 (2), (A) confirm 0x5d6c0 (6), all mono
48 kHz. The retraction landed in docs/re/ and the page the port reads
kept the superseded text -- the fourth time in this corpus.
Writing the rule down has not worked, so there is a tool now.
handoff_lint.py flags every HANDOFF line making a strong negative claim
that links a doc containing retraction language. First run: found the Q8
row, plus one benign false positive (Q3 links a doc whose retraction is
about a sprite count, not about the tie-break -- checked, and HANDOFF
repeats none of the retracted figures). The lint also caught its own bug
first: it reported existing docs as missing because it joined a guessed
repo root, so it now resolves links relative to the file as markdown does.
Separately, EXTRAS's paint-order risk narrows twice more. Of its 15 tied
pairs only 2 overlap, and of those, ptloop01 x ptloop02 are loop*
animations compose skips by default -- so exactly ONE tie can be drawn:
ptframe3 x ptframe4, overlapping 102x132 px. Against live-extras.png that
contested region correlates +0.9622, better than the whole frame (+0.9440)
and inside the range of regions where order cannot matter (+0.8502 /
+0.9903). Consistent with our order, not proof: correlation cannot see a
swap between locally similar art.
15 -> 2 -> 1 -> consistent is now the whole paint-order risk on the five
screens, and HANDOFF says so.
|
||
|
Sylpheed RE agent
|
ba47bdebe8 |
re(ui): measure the paint-order hedge -- exact on 4 of 5, and bound the rest
`compose` claimed the derived paint order "reproduces both measured orders up to ties". That sentence was never measured and was stale by one: there are three measured orders, not two. examples/paint_order_audit.rs checks it. main menu (entries 5, 8) derived == measured 0 inverted pairs developer splash (11, 14) derived == measured 0 inverted pairs title (entry 4) DIFFERS 8, all same-key ties So the claim holds and the exception is entirely ties -- but two of those ties are total occlusions, not near-misses. The tied family is the five ptlogo_back2eff glows (key 32899); back2eff5 is 1133x280 and FULLY CONTAINS back2eff3 (82,824 px^2 = 100% of the smaller) and back2eff4 (152,047 px^2 = 100%). Derived paints it on top of two glows it entirely covers; the game paints it underneath. A tie-break by declaration index can therefore be wrong by a whole layer. The title itself is unaffected -- it has a measured order. The port's actual exposure, per screen: title, main menu and developer splash all use MEASURED orders; the publisher splash is derived but has ZERO ties, so it is fully determined; EXTRAS is derived with 15 tied pairs of which only 2 OVERLAP. Two element pairs on one screen is the whole risk, and that is what HANDOFF now says -- not the raw 15, which would have overstated it 7x. Reach stated: this compares the derived order against orders measured from the game, not an independent derivation, so where no measured order exists only the tie exposure can be checked. Overlap uses pivot*2 as the element size at its resting placement. Stale comment in compose corrected. METHOD: a hedge in a code comment is an unmeasured claim; and count the cases that can bite, not the ones that match the pattern. |
||
|
Sylpheed RE agent
|
b973d8cde3 |
re(ui): scale 0 means collapsed, not "unset" -- stop drawing it full size
blit() and fill_quad() both opened with
let sx_pct = if kf.scale_x == 0 { 100 } else { kf.scale_x };
so an element whose pose is collapsed to nothing rendered at FULL SIZE.
I first described this as "a 1-pixel sliver", reading the .max(1) in the
size arithmetic two lines below -- the guard above it meant .max(1) never
saw a zero. Read the whole function, not the lines you went looking for.
Control run before the change: if 0 meant "unset", something on the disc
would use it throughout. Nothing does.
elements with a keyframe group 15 493
at least one keyframe at scale 0 2 166
EVERY keyframe at scale 0 0
zero is a transient the element grows out of 1 762
ptlogo_eff3.t32 runs 0% -> 200%. An "unset" marker no element ever uses
throughout is not a marker.
Both functions now return without drawing when either scale is 0.
Reach: 24 of 24 renders byte-identical across GP_TITLE (all 16 builds),
GP_PAUSE_MENU and GP_OPTIONS -- additive on every screen the port needs.
195 elements have a guessed rest pose at scale 0 and 126 of those have a
non-zero alpha, so the old code painted them; all 126 are in
GP_READY_ROOM.pak, which S1 already declared a no-go. They are
tactical-map sprites at sx=0/sy=100, mid-horizontal-wipe, that the
coercion drew full-width.
So: a real correctness fix with no visible effect on the menu port.
Suite green, 122 passed / 0 failed across 3 suites at commit time.
|
||
|
Sylpheed RE agent
|
6f23f4d113 |
re(ui): the keyframe-time shift is favoured 26x by timing, rejected by a render
Follow-up on last iteration's unadopted candidate (+36 holds the NEXT pose's time, not its own). Two new results, pointing opposite ways, and both are reported. FOR, and calibration-free: the observed full-alpha hold : fade-out ratio on palogo_gamearts is 83 : 13 frames = 6.38. The shifted reading predicts 8.00. The current reading predicts 0.25 -- off by 26x. With the glow's 2 units/frame fixed and nothing else free, the current reading says the logo holds full alpha for 2.0 frames; the capture holds it for 83. This is no longer the shape argument the candidate rested on. Also for: rest()'s plain max-dwell fallback picks a=0 -- a transparent pose, for a publisher logo -- under the current reading, and the visible a=255 hold under the shift. Only the rest_plateau special case rescues the render today, and that is the case the port agent reported a bug in. AGAINST: rendering every build of six UI paks under both readings, 10 of 11 compared are byte-identical and one changes -- GP_TITLE build 7, the Japanese twin of build 4, by 13.1% of pixels. Build 4, the one verified against a live capture, is unchanged either way, so the single build the shift moves is the one with no capture to adjudicate it. The proxy goes against the shift: language twins are the same artwork, and build 7 reads 70.94 mean luminance as decoded against build 4's 71.41, but 76.32 shifted. Correlation does not separate them (0.6206 vs 0.6201). These constrain different things -- timing versus pose selection -- and rest() is a heuristic layered on the times, so moving the times moves its tie-breaks. Adopting the shift means revisiting that heuristic in the same change, with no build-7 capture to verify against. Default UNCHANGED. Experiment reachable via SYLPHEED_KF_TIME_SHIFT=1. Shifted-mode suite: 122 passed, 0 failed across 3 suites at commit time. |
||
|
Sylpheed RE agent
|
67fa1a1b0b |
re(ui): decode keyframe +12 as screen-plane rotation in degrees
The rotated quads on the title screen come from the keyframe block after all. The earlier negative -- "every GP_TITLE build 4 element has all three angle words at zero" -- read the right bytes over too small a region: it walked the top-level declaration table, and the rotated elements are the nested leaf records ptloop01.rat / ptloop02.rat. Confirmed against the framebuffer rather than against our own renderer. The two records declare +12 = 30 and -45; the GPU capture submits their quads at +30.26 and -45.28 degrees -- magnitude and sign, two different values. Corroborated by shape in GP_BUNK 117ca14f, where +12 ramps 0 -> 360 with position, scale and alpha constant: a spin in place. Identifying which draw it was needed edge lengths, not bounding boxes: 400x1076 and 400x1444 against pteff03/pteff03a 399x180 at the elements' two different declared scales, 600% (1080) and 800% (1440). The same test names three known-positives in the capture (ptlogo1, ptcopyright, ptbtn00), so it passes its own control. Keyframe gains rotation_deg plus unknown_4/unknown_8, carried rather than dropped. NOT rendered -- ui_layout::blit is axis-aligned only, so the reference renderer and the port will both draw these upright until a rotating blit exists. The census tool ships with the trap that broke its first version: nested RATC blobs are not 4-byte aligned, so an aligned scan found 0/3 of its own control blocks and missed 16 341 blocks. Disc-wide +12 is non-zero in 14.50 % of 83 862 blocks. sylpheed-formats tests, SYLPHEED_DISC set: 131 passed, 0 failed across the 6 suites finished at commit time; the run had not yet completed. |
||
|
Sylpheed RE agent
|
5115b4894a |
re: three angle fields found in the keyframe -- and they are not the
title's rotation Looking for where the rotated quads come from, the obvious candidates were the three keyframe words at +4, +8 and +12 that ui_layout.rs documents as zero. They are not zero. Across 72287 keyframe blocks disc-wide they are non-zero in 4.81, 4.56 and 15.82 percent of blocks, and read as signed values clustering on 180, -180, 90, -90, 120 and 22 -- degrees. Three of them, so plausibly rotation about three axes. I have marked that amber because it is the shape of the numbers and nothing more; no observed rotation has been tied to a value. The doc comment is corrected regardless: "0 on every frame seen" was an artefact of the sample. And they do not explain the screen I was chasing. Every element of GP_TITLE build 4 has all three at zero, element by element, while the game demonstrably submits rotated parallelograms there. So the title's rotation comes from outside the keyframe data and is still unidentified. One correction to my own last write-up, flagged rather than left: I stated that the skewed draw IS the swoosh. It is the only skewed geometry in the capture and the swoosh is the only diagonal element on the screen, so the inference is reasonable -- but I never confirmed it by matching the draw's texture or screen position to that element, and I should have said so the first time. |
||
|
Sylpheed RE agent
|
5d02d5cd60 |
re: additive blending refuted, the swoosh is not displaced, and the
residual is smaller than I said Testing the candidate I raised last iteration rather than carrying it. Blending bit-0x02 sprites additively moves every measure the wrong way -- whole-frame mean diff +0.55 to +1.04, swoosh-band mean +1.83 to +3.98, band edge-correlation 0.6971 down to 0.5578. So the bit is real and independent but does not select an additive blend. I reverted the experiment and kept the word as T8adImage::flags, documented and not acted on; the render is byte-identical to before. Second refutation: the swoosh is not displaced. Shifting the band over plus or minus 80 by 8 pixels peaks sharply at zero, 0.7342, falling to 0.22 at 24 px. So the pivot story is dead twice over -- inert at scale 100, and no displacement to explain anyway. And I have restated the residual, because earlier sections overstated it. The +16 to +34 band tiles I quoted were measured WITHOUT --primitives. With the dim drawn the band's average is nearly right at +1.83; what is wrong is its structure, tiles running -38.6 then +33.8 and cancelling. Six candidates eliminated now and none confirmed. One caveat I owe the port agent about the capture I gave them: it is at t=4.0s, roughly 174 keyframe units into a screen whose elements have keyframes out to t=600. I judged "settled" from mean luminance, which cannot see a thin sprite still moving. It is settled for the bulk of the screen and not proven settled for every element -- which is a live alternative explanation for a structural difference in exactly the band the sweeps cross. METHOD: cargo build passing does not mean cargo test compiles. Adding the field built the library in 1.48s and broke two test-only struct literals; cargo test failed with exit 101. |
||
|
Sylpheed RE agent
|
4bc970688f |
re: rest() fixed -- a trailing keyframe run is the hold when it is
VISIBLE, not when its pose repeats The port agent's report was right about the defect and about which elements it hits, and its proposed condition does not survive the case the exclusion was written for. Their test -- the final untimed keyframe has the same pose as the last timed one -- is also true of pgptitle.rat, whose trailing run is two identical transparent frames. Adopting it would erase the word PAUSE again, which a committed capture disproves. What separates the two is alpha: an exit fades the element out so its last keyframe is transparent, while an element with no exit ends on the pose you can see. So rest_plateau now accepts a trailing run exactly when it is visible. Verified against a capture rather than against another renderer, which is the point their own report made: ptframe1's rest moves from (620,108) t=16 to (440,108) t=62; the changed pixels are 10082 in a bounding box of x 440-839, y 108-577, which is exactly the 400x470 at (440,108) they predicted; and correlation against main-menu-oracle.png over that region improves from 0.9596 to 0.9748. The PAUSE wordmark is unchanged across all three pause builds. This also closes a question ui-paint-order-key.md has carried for a while, that ptframe1 and ptframe2 rest at alpha 0 while the capture shows the menu frame plainly. One trap cost most of this iteration and goes in METHOD: CARGO_TARGET_DIR is redirected in this container, so ./target/debug/sylpheed-cli is hours stale and every render I made against it was old code. Byte-identical before and after reads as "no effect" when it actually means "you ran the old binary". The full disc-gated test suite was still executing when this was committed; the verification above is artifact-based. |
||
|
Sylpheed RE agent
|
8b6dbcfead |
formats: move media assembly out of the viewer, where it could not be reused
The trickiest reading on the disc lived in the Bevy viewer: resolving a cutscene's voice to a continuous byte REGION of the sound stream, because the movie voices are one XMA stream chunked into VOICE_*.slb entries whose boundaries do not match the cues -- a cue routinely spans two chunks, so a .slb need not hold the track its name claims. That put the logic most likely to be re-derived incorrectly in the crate least likely to be reused. The Godot port's exporter needs the same answers, and there must be one implementation of them. New `sylpheed_formats::media` owns every case where the bytes of one playable thing are not one archive entry: segment-spanning reads, multi-sub-wave banks, and the voice-region resolution. Callers supply bytes through a `DiscSource` trait, so the viewer keeps its ISO/directory abstraction and a headless consumer gets `DirectorySource` for free. The seam is deliberate: this module returns XMA RIFFs, not PCM. Decoding means shelling out to FFmpeg, which is native-only and a policy decision for the consumer -- everything up to "here are the bytes that belong together" is disc knowledge, everything after it is a codec choice. The four moved functions were previously untested; `tests/media_disc.rs` now pins them, including the negative the corpus paid for -- an unbound movie must stay unvoiced rather than borrow a neighbour's clip, which was tried and played the WRONG recording. The algorithm is unchanged, moved verbatim (same window sizes, same fallbacks). The new disc tests pass; the broader audio suite was not re-run in this pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Sylpheed RE agent
|
77cd58202b |
viewer: a Cutscenes browser -- the manifest was invisible plumbing
`movie_manifest` has been parsed since the movie-voice work and rendered nowhere: it resolved a voice bank and that was all. So the only way to find a cutscene was to hunt `.wmv` files in the ISO tree, where nothing tells you which mission a file belongs to, whether it has subtitles, or what is said in it. View ▸ Cutscenes lists all 104 manifest slots with mission/phase, kind, movie, subtitle track, voice token and telop, and -- the part that needed no new parsing, only a route -- resolves the captions to a readable TRANSCRIPT with a language selector. Subtitles were previously burned into the video during playback and reachable no other way. Three negatives are shown rather than smoothed over: * 5 manifest-bound movies have no `.wmv` (logo1-4 and an encoder test clip). They are marked and get no Play button instead of one that would fail. * 9 of 101 movies resolve no English transcript. * the `.prt` telop overlay is named by the manifest and we have no parser, so the reference is shown labelled "not decoded" rather than omitted. `cutscene_catalog_binds_movies_and_transcripts` pins all of it against the disc -- 104/101/99/99/22, the exact absent-movie list, 92 transcripts -- because a browser that quietly dropped these would look complete and be wrong. The counts independently reproduce docs/re/movie-subtitle-link.md. Play routes through the normal FileSelected path, so the existing video player handles it exactly as it would from the tree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Sylpheed RE agent
|
306a8a5661 |
viewer: open the whole sound bank, not just the voice half
The library enumerator kept only names containing VOICE or \Briefing\, and read eng\sounds.tbl unconditionally. So the Explorer could reach 4382 of the 9519 banks in sound.pak: no music, no jingles, no sound effects, and no Japanese voice at all -- roughly half the disc's audio had no route to the UI. `slb::list_audio_entries` now returns every named bank with the category its path implies (Music / Jingles / Sound effects / Radio / Dialogue / Movie voice / Briefing). `list_voice_clips` is that, restricted to the spoken categories, so its existing test still guards the old behaviour. The 36 root banks carry no language component and appear whichever table is read; the window gets an English/Japanese switch that re-reads the other sounds.tbl, since the table name IS the selector. Two defects the decode found, both recorded in docs/re/structures/sound-pak-contents.md: * `Static.slb` -- the SFX bank -- declares 616768 bytes more than sound.p04 holds. Not our extraction: p04 matches the ISO's own directory record, and a sweep of every pak on the disc finds this one entry over-running and no other. It is the highest-offset entry, so its comp_size is an allocation size. A short read is now allowed for the tail entry ONLY; any other overrun stays an error, because clamping it would hide real damage behind a half-decoded asset. The bank went from unreadable to 514 s of audio. * the left-channel downmix was applied to everything. Right for voice (mono content however stored), wrong for music (a real stereo mix, half of it discarded). The caller now decides from the category. 35 of the 36 shared banks decode; JNGL_001 does not, and says so in the player instead of the panel silently closing. Its payload is not a whole number of XMA1 packets from any known data offset, so it is likely not a plain headerless stream -- written up rather than papered over. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Sylpheed RE agent
|
fb70511242 |
viewer: make the two screen presenters agree, and expose the compose options
The UI Screens window enumerated with `is_build` while the PAK browser's inline preview composes anything `parse_build` accepts. So the browser drew screens this window flatly refused to list -- most visibly `palogo`, the publisher splash, which declares its sprites directly and has no `.rat` layout child. `compose_screen` now gates on `is_composable`, the documented superset (every `is_build` bundle passes it), so anything the list offers is drawable and the two presenters share one rule. A "Fragments" toggle widens the enumeration to it as well, off by default: the extra ~1786 bundles are mostly two-element fragments (a button beside its glow) that would bury the real screens. It re-scans, which renumbers the display ordinal -- harmless, because the pak ENTRY index is the locator and the stale catalog is cleared. The toggle sets a `rescan` flag rather than self-sending RequestScreenCatalog: a system that both reads and writes one event type is a B0002 panic at startup, which is how the Save browser broke. Also plumbs the last two ComposeOptions the CLI had and the UI did not -- `black backdrop` (what a framebuffer capture must be compared against) and `primitives` (decoded, but paint order unsolved; hover text says so). Verified: `screen render --all --build 13 GP_TITLE.pak` draws the SQUARE ENIX splash, 2/3 elements, the third being the .prm quad. Workspace builds, viewer reaches steady state, disc-gated suite 20/20 green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Sylpheed RE agent
|
47e4e43310 |
viewer: stop the UI Screens browser loading forever
Three compounding causes, found by tracing every place a `loading` flag is set
against every place it is cleared.
THE COST. `compose_screen` inflated EVERY entry in the pak and held every build
in memory at once, purely to index into the result -- then did it again on every
checkbox click, re-opening the ISO from scratch each time. But the catalog
already records each build's pak ENTRY INDEX, so the locator was there all
along: compose now reads exactly one entry. `RequestScreenCompose` carries
`entry` (the locator) beside `build` (the display ordinal).
`build_screen_catalog` had no budget at all, while the pak browser has capped
exactly this work since it was written. It now skips oversized entries and stops
at a ceiling.
That ceiling is 384 MB, not the pak browser's 64 MB, and the difference is the
point: GP_HANGAR_ARSENAL inflates past 160 MB and holds ~390 builds, so a 64 MB
cap would have quietly hidden most of them -- trading a hang for a wrong answer.
When the ceiling IS hit the pak is marked `truncated` and the UI says so.
THE LATCH. `poll_loader_channel` treated a disconnected channel exactly like an
empty one, so if a worker died every in-flight spinner stayed up for the life of
the process -- and the `if loading { return }` guard at the top of each handler
then refused every retry. Disconnect now clears the flags and reports it.
Verified: the workspace builds, the viewer reaches steady state, and one
composite still draws 11/11 elements of the tutorial pause menu.
|
||
|
Sylpheed RE agent
|
0a0a0333dd |
slb: guard the wave-boundary identity with a test, and expose data_at
A wave runs to data_at + declared_size, and there is seek magic exactly there whose little-endian packet count at +12 times 2048 equals the declared size -- 7620/7620 disc-wide. That is the decoder-independent boundary and the thing that proves the declared sizes honest, so it should not be able to regress silently. The boundary routinely lies outside the entry's own comp_size window, so reading it needs the flat segment stream rather than the entry slice; PakArchive gains a small data_at(offset, len) accessor for that. Test walks a bounded slice to stay fast -- 792 banks in this run, all holding. 8 disc tests pass. |
||
|
Sylpheed RE agent
|
aa84aaf53f |
slb: read Channels instead of assuming mono -- and retract the TCAF_608 conclusion
I read the seek chunk's packet count big-endian; it is little-endian at seek+12, with size == 8 + 4*count. And a seek sits immediately AFTER its own data, so an entry's first seek usually belongs to the PREVIOUS bank (implied start -25232 for D_452, -145988 for TCAF_608). I was comparing an entry's first seek against its first data -- different waves by construction, which is why no reading lined up. With that fixed, the declared sizes are honest: every RIFF-bearing entry on the disc has seek magic at exactly data_at + declared_size with count*2048 == declared. 7620/7620, zero failures. VOICE_TCAF_608 is not truncated. Its Channels is 2 and I decoded it as mono; read as stereo it gives 6520176 bytes = 33.96 s, agreeing with both length signals in the bank (33.88 s from cumulative samples, 33.97 s from PsuedoBytesPerSec). 170 of 8021 banks (2.12%) are stereo -- exactly the rate of my 1-in-60 outlier. This is the mono/stereo trap already documented on this very page, met from the other direction: I had written 'at two channels every bank yields one frame' and then spent several passes blaming missing data for a one-frame decode. Code fix: to_xma_riffs built the leading segment with a hard-wired mono fmt. It now reads Channels from the bank's first RIFF. 7 disc tests pass. |
||
|
Sylpheed RE agent
|
6044ba49a0 |
slb: the seek chunk gives the data offset structurally, and breaks the 28 ties
The ties needed a different signal, not a longer scan. Banks carry one: a seek chunk sitting on a packet boundary, so seek_pos % 2048 IS the data offset. On the 6033 labelled banks with a seek before their first RIFF, 6031 agree (99.97%) -- better than the packet scan and structural rather than statistical, so scan_data_offset now tries it first. On the scan's 28 ties it resolves 26 correctly and 0 wrongly (2 have no usable seek). Combined rule scores 7354/7358 = 99.95%, up from 99.62%. 762 of the 1495 RIFF-less banks carry a seek, so the signal exists where it is needed. Also ruled out, since a wrong offset was this page's whole subject: the header is not audio being discarded. Adding 0 to the candidate set, it wins 6 of 7358. 7 disc tests pass. |
||
|
Sylpheed RE agent
|
bd5bd572c3 |
slb: the declared data size is an upper bound, not an exact one
Two comments claimed it is 'honest per sub-wave'. Measured: 5296 of 7586 banks declare more than the entry holds and none declares exactly what it holds, so the existing .min(slb.len()) clamp is load-bearing rather than defensive. Comment-only change; 7 disc tests still pass. |
||
|
Sylpheed RE agent
|
e5ce7e4ba3 |
slb: the headerless path was decoding stereo at a fixed offset; both are wrong
1495 banks carry no RIFF and take a separate path that hardcoded both the offset and stereo. Across a random 48-bank sample there was NOT ONE where the old stereo-at-1392 pair beat the best mono offset; median gain 184x, individual banks going from 0-4816 decoded bytes to 180000-380000. Stereo shows the same stop-after-one-frame signature already recorded for the leading segment. With no RIFF the offset cannot be derived, so scan_data_offset picks among the four disc offsets by XMA1 packet-header plausibility. Validated on the LABELLED set -- all 7358 banks that do have a RIFF, where the answer is forced: 7330 correct (99.62%), and all 28 misses are ties on the top score, never a wrong unique winner. Ties fall back to 1392. The winning offsets also reproduce, by directory, the distribution measured independently from the RIFF-bearing banks. jpn\etc splits 1468/1600, so path alone is not sufficient -- which is why this is a scan and not a lookup table. 7 disc tests pass (build-reborn test -p sylpheed-formats --test slb_leading_segment_disc, SYLPHEED_DISC wired up). |
||
|
Sylpheed RE agent
|
d15b3d8d85 |
slb: derive the leading-stream data offset instead of assuming 1392
HEADERLESS_DATA_OFFSET is the value the offset takes in <lang>\etc\, not a property of the format. The leading stream is a whole number of 2048-byte XMA1 packets ending at the first RIFF, so its start is first_riff % XMA1_PACKET. Disc-wide that takes four values -- 1392, 1468, 1600, 1728 -- varying by language and subdirectory. Verified by decoding, not by arithmetic: on a random 140-bank sample with a non-empty leading region, the derived offset yields more audio in 85, identical in 54 (the eng\etc controls, where it must and does reproduce the old behaviour) and less in 1. Median gain among the improved is 70x -- eng\Voice\VOICE_TCAF_592 goes 1506 -> 97152 bytes, jpn 2910 -> 127178. This withdraws my own claim from earlier today that the Japanese banks were a different undecoded layout. They are the same format with a different offset; I had treated a constant derived from one subdirectory as a property of the format. The same error was hiding the identical defect in 1873 eng\Voice banks. |
||
|
Sylpheed RE agent
|
80a45bfbd7 |
re: wire the IXUD record table into the crate — captions go 537 to 8800 of 8800
ixud.rs now has an IdxdObject-shaped reader, IxudObject, and build_caption_text reads captions as FIELDS instead of pairing them with whatever token follows in the pool. build_demo_text token adjacency 134 ids 537 lines build_caption_text token adjacency 3721 8074 build_caption_text record fields 4085 8800 = all of them Verified over the whole disc by tests/ixud_records_disc.rs: 1104/1104 objects parse, 1476/1476 records and 628165/628165 named fields reproduce their ixud_hash, 48 positional, zero failures. The header word at 0x08 is record 0's hash, asserted per object -- there is no schema field, exactly as for IDXD. The module doc described a 12-byte record directory and a "schema/type hash"; both were wrong and are corrected. I also have to correct my own number from the previous commit. "1.3% of the game's text" counted OCCURRENCES: each family lives in 24-45 IXUD blocks and the same key repeats across them. Distinct text-bearing MSG_* keys number 8800, not 44579, and every one has the <id>_<page>_<line> shape. So the real coverage was 537/8800 = 6.1%, and I overstated the gap about fivefold. Direction right, magnitude wrong. The DEMO control is the sharpest evidence for the change: token adjacency finds 537 lines there, the field reader 541. It was dropping lines even in the one family it was written for -- which is why the test now asserts "must not lose lines" rather than "must be identical". Same lesson twice in one session: pool adjacency is a consequence of how records are written, not a rule of the format. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE |
||
|
Sylpheed RE agent
|
1b295e3cd2 |
re: read all eight caption families — 15x more text, and the same lesson twice
build_caption_text generalises the key parser from MSG_DEMO_* to all eight
families. The shapes are uniform and each family is 100% consistent with its
own: seven use MSG_<FAM>_<id>_<page>_<line>, and VOICE alone inserts a family
letter before the id.
ids lines
build_demo_text 134 537
build_caption_text 3721 8074
The DEMO family comes out identical through both readers -- 537 lines either
way -- which is the control that generalising changed nothing that already
worked. Pinned by tests/caption_families_disc.rs, along with VOICE ids keeping
their family letter.
But this does NOT close the gap, and the write-up says so: 8074 against the
44579 text-bearing fields the record-level scan counts is about 18%.
The reason is the same lesson this session already learned once.
build_caption_text pairs a value with the key that happens to follow it in the
raw UTF-16 token stream -- the adjacency heuristic that was wrong for IDXD and
is wrong here for the same reason. ixud.rs has no record/field reader at all.
The IXUD record table IS decoded and verified disc-wide (1104/1104 objects,
628165/628165 fields reproducing their key) and was simply never wired into
the crate.
Next step recorded: give ixud.rs an IdxdObject-shaped reader and read captions
as fields rather than adjacent tokens. The decode exists; only the plumbing is
missing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
|
||
|
Sylpheed RE agent
|
8d1c9d1c77 |
re: pin the per-record game data — and correct the "SpaceSize is per phase" claim
A disc-gated test that fails if the loaders regress to one flat answer per object. It checks the three structural claims that came with the migration request, and one of them is overstated: * ✅ MainMissionBonus IS per difficulty — 18 of the 24 missions pay three different values, in a strict 1:2:4 easy:normal:hard ratio (all 24); the other 6 pay 0 at every difficulty. The flat reader returned the Easy value. * ✅ RankScore_S/A/B/C/D repeat identically across the three Score_* records in 24 of 24 missions, so the single flat answer happened to be right. The difficulty scaling lives in the earnings (CraftScore_Adjustment, KillBonus_Maximum, …), not in the thresholds. * 🟡 SpaceSize is *stored* per phase (three Phase_N records), but only **1 of the 24** missions varies it — the one that reads 250000/100000/100000. The other 23 repeat a single value (500000 ×1, 100000 ×18, 50000 ×4). "Phase_1 = 250000, Phase_2/3 = 100000" is one mission's numbers, not a rule. Also pinned: the Delta Saber's 63 Turret_NNN mounts at HP 100 each against a TurretCount of 4; the SD-Battleship's per-component HP (bridge 10000, thruster 20000, shield generator 5000 at PowerRatio 0.25, hatch 100 launching Squadron_Test2, anti-ship gun 1500) against a hull of 100000, and its 37 records that define an HP; 418 hardpoints across the 23 capital ships, none missing HP; 131 weapons whose shell id always differs from the launcher id, 40 with a ShellWake, and Weapon_NULL as the only one without a Shell.Power; and fcs_range/shield_ratio/cruising_velocity/maximum_velocity/acceleration/ deceleration now set on 89 of 89 units, all of which the pool reader could miss. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE |
||
|
Sylpheed RE agent
|
6ce1a87d49 |
re: rebuild game_data on the IDXD record table — 966 misses and 596 flattened reads
Measured first, over GP_MAIN_GAME_E.pak, comparing every named-field read the
six struct loaders performed against the record table: 4435 reads, 2872 agreed,
**966 returned None for a field that has a value**, **596 flattened a field that
several records carry**, 1 was wrong (a weapon whose TargetType is the empty
string read back as the neighbouring token "Skip"). The prior report of
4453/2887/974/591/1 is the same picture; the small differences are definitional
(I count a read as flattened only when the records disagree).
Every read now goes through IdxdObject::record, and the types say where a value
comes from:
* Weapon = the `Weapon` record (launcher) + the `Shell` record (projectile).
Both carry an ID and a Name and — with `ShellWake` — an `Interval`, which the
flat reader merged; they are separate fields now. Power/Velocity/ranges/
LifeTime are Shell fields, which is why 427 weapon reads used to miss.
* CraftUnit/Vessel = `Generic` (hull) + `Maneuver` (flight model) +
`StructureCount` (counts) + `Shield`, plus a new `hardpoints: Vec<Hardpoint>`
— one entry per Turret_/Bridge_/Thruster_/Hatch_/ShieldGenerator_ record, each
with its own HP. A flat HP could only ever be one of them.
* PlayerConfig = `Player`, plus `phases: Vec<PlayerPhase>` (SpaceSize/SupplyRange
are per Phase_N) and `score: ByDifficulty<ScoreRules>` (MainMissionBonus is per
Score_<difficulty>; the flat answer was the Easy one).
* Character faces come from the `Faces` record's field names (identical output to
the old token scrape, 0 of 68 objects differ — now by construction).
* Stage = `StageResource` + `phases: Vec<StagePhase>`, and the packages it names.
* The `fields: BTreeMap` on every struct became `records: RecordSet`, which keeps
the record boundary; `RecordSet::everywhere(field)` answers "which record".
The token-scraping loaders move too, and this is where the old reader was worst:
* Arsenal: options are the positional fields of the STANDARD_<slot> records. The
scrape returned 16 nose options of which 8 were field keys and pilot names, and
47 for arm3 of which 38 were junk, while missing Mine_B2A and No_Equipment.
Now 8/12/9/9, all real weapons.
* Squadron: one record per squadron, members are Count*4 positional slots
(unit, message set, n, pilot) — 1160 squadrons with ids and 2295 member tuples,
against 28 idless squadrons and 47 members before. Agrees exactly with the
independent Python decode in docs/re/structures/unit-group-table.md.
* DemoMessage: 11775 lines against 10263, every one with a speaker, a portrait,
a delivery mode and a voice token, from fixed positional slots.
* PilotRoster: assignments are the records the `UNITS` record names, so each one
now carries its unit id, its loadout and the player marker.
* UnitRoster: the roster is the field *names* of the single `EnumUnit` record.
* load_weapons selects on the records (Weapon + Shell) rather than on token[0],
whose first byte is often a stray pool byte ("#Weapon", "%Weapon"). Same 131
objects, no heuristic. GP_HANGAR_ARSENAL.pak holds none of them — the module
doc's claim that player weapons live there was wrong.
schema:: constants keep their names and values but are documented for what they
are: record 0's name hash (PLAYER = Difficulty_Easy, UNIT = Maneuver, VESSEL =
Bridge_000, MESSAGE = Message_000), not a schema id.
Two things the migration exposes and does not fix, flagged in the docs instead:
load_units' bucket is 43 Type=Craft + 46 Type=Vessel objects (new `unit_type`
field lets a caller separate them), and StructureCount.TurretCount is not the
number of Turret_* records (the player's craft says 4 and has 63).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
|
||
|
Sylpheed RE agent
|
62d8264cbb |
chore: drop zz_scratch_measure.rs, swept into 49a09a9 by accident
It is a throwaway measurement harness I wrote in this session ("TEMPORARY
measurement scratch — not for commit") that a concurrent commit picked up
along with its own files. It asserts nothing and duplicates work that is now
covered by tests/game_data_disc.rs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
|
||
|
Sylpheed RE agent
|
a32c00057e |
re: recover the .slb leading segment — mono, and scoped by measurement
to_xma_riffs now emits the leading headerless segment when it sits at a whole number of XMA1 packets and carries a non-zero byte. VOICE_D_453 goes from a 0.14 s trailing fragment to a 45116-byte leading sub-wave that dominates it. I withdrew this exact change earlier for two reasons. Both are now answered rather than argued away: * "It recovers no audio" -- it used the STEREO format. At two channels every bank yields exactly 1792 bytes, one frame, whatever its size. Mono yields up to 113x more. * "It matches 1524 of 8021 RIFF-bearing entries" -- the byte-level reach is still 1524, but the audible reach is not. Across the 84 movie-bound banks the segment adds >1 s to exactly 7, the hokyu_*_H tankers on D_453/D_454 -- precisely the broken ones -- and <=0.25 s to 66 of the rest. The largest non-resupply addition is S04A at +0.66 s on a 256 s movie. The safety oracle is recorded with its limits: 8 of the 84 banks ALREADY exceed their movie's duration before the change, by hundredths of a second, so it cannot resolve differences at that scale. It establishes scoping, not correctness. Callers clamp to the movie length regardless. VOICE_D_451's all-zero leading region is skipped by the non-zero guard, so the rule cannot prepend silence to a bank that does not need it. Pinned, as is the packet arithmetic (n = 8, 1, 7, 22, 29) which has no tunable. slb_disc, movie_subtitle_disc and movie_manifest_disc all still pass. NOT verified by ear -- that needs a human. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE |
||
|
Sylpheed RE agent
|
b32006d2a8 |
re: RETRACTED "audio is missing" — a subtitle cue is a START time
The load-bearing error of the whole voice-bank thread, and it is mine. It stood for three iterations across two write-ups that each called the result proven. I treated a subtitle cue as a timestamp that must fall INSIDE the voice clip, and concluded a 0.07 s clip could not host a cue at 4.70 s. A cue is when the line STARTS. The voice plays from the cue, so the clip only has to fit the window between the cue and the end of the movie. Under that reading every bank fits at plain 48 kHz: bank samples @48kHz cue window D_450 158967 3.31 4.00 5.30 D_451 76084 1.59 3.70 5.60 D_452 119562 2.49 0.00 8.34 D_453 108608 2.26 4.70 4.60 D_454 167828 3.50 0.00 9.50 2-3.5 s is also the right length for the lines. Nothing is missing, and the 17091-20563 Hz window from the previous commit is void with it -- its lower bound came from the same misreading. What survives, because it was measured rather than interpreted: the leading region is XMA1 mono, the decode runs to the final frame, and cue values are seconds. Separately settled, and it is what exposed the error: each shared bank holds ONE generic line. The 3-5 movies bound to a bank have IDENTICAL subtitle text, 5 banks out of 5 -- "Rhino 3 has landed. Commencing resupply.", "Resupply complete. You are cleared for take-off!", and so on. That also explains the historical in-game rejection of hokyu_DS_s13A -> VOICE_D_452 that started this whole thread. The line is generic, identical for s02A/s07A/s08A/s13A. Someone expecting a stage-13-specific line would hear the generic one and call it wrong -- while the binding is exactly right. The disc said so; the subtitle text now says so independently. Still open: whether the leading mono region is additional audio or an alternate take, since the totals above add it to the RIFF sub-waves. Artifact: examples/shared_bank_takes.rs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE |
||
|
Sylpheed RE agent
|
b324278ae9 |
re: cue times ARE seconds — and "the rate does not converge" was my error
Two things settled, one of them a correction of my own claim from the last iteration. The cue unit is verified rather than assumed. parse_timing computes mm*60+ss, but only if the token really is mm:ss.cc, so I checked against an independent oracle: the movies are on the disc and a cue must land inside its own movie. 66 English movies with subtitle tracks, 0 cues land after the movie ends. Centiseconds would have overflowed essentially all 66. The seconds reading stands and the verdicts built on it survive. "The sample rate does not converge" does not. I reported implied rates of 39742 / 20563 / 23108 Hz as irreconcilable. They are not estimates of the same quantity -- each is a ONE-SIDED BOUND. The audio must be at least as long as the last cue, so samples/cue is an UPPER bound; it cannot outlast its movie, so samples/movie is a LOWER bound. Intersecting: bank samples cue movie lower Hz upper Hz VOICE_D_450 158967 4.00 9.30 17091 39742 VOICE_D_451 76084 3.70 9.30 8180 20563 VOICE_D_453 108608 4.70 9.30 11677 23108 => 17091-20563 Hz, non-empty. A single rate IS consistent. I had been comparing them as competing point estimates, which is why they looked contradictory. What is still open, and stated as such: that window contains no standard XMA rate. The lower bound assumes a whole bank plays inside one movie, and each of these banks is bound to 3-5 movie slots -- so if a bank holds several takes the lower bound is void, leaving rate <= 20563, which 22050 nearly meets. Next step recorded: establish whether a shared bank is one line or several. Artifact: examples/cue_unit_check.rs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE |
||
|
Sylpheed RE agent
|
116fd7d2ff |
re: the .slb leading region is XMA1 MONO — 113x more audio than stereo
Retried the format probe with the fmt chunk built to synth_xma1_fmt's exact byte layout, and with the bank's own RIFF sub-wave decoded through the same pipe as a CONTROL so a broken harness cannot masquerade as a result. The channel count is the whole story: bank lead B channels=2 channels=1 VOICE_D_450 16384 1792 46756 VOICE_D_451 2048 1792 896 (all-zero region: control) VOICE_D_452 14336 1792 30154 VOICE_D_453 45056 1792 203648 VOICE_D_454 59392 1792 294440 channels=2 yields EXACTLY 1792 bytes for every bank regardless of size -- one frame, then it stops. That constant is the tell. At channels=1 the same data yields up to 113x more, and the control sub-wave decodes to 13568, so the pipe works. Why the previous probe got 0 bytes everywhere is now named: I read synth_xma1_fmt(2, 2, 48000)'s second argument as a STREAM COUNT when it is a CHANNEL MASK, and built the WAVEFORMATEX around that misreading. Also recorded as a refutation, because it was tempting: solving for the sample rate as decoded-samples / last-subtitle-cue does NOT converge. D_453 implies 21665 Hz -- close enough to 22050 that I nearly wrote it down -- but D_450 implies 5844 Hz. No single rate explains both, and the decodes are visibly partial (samples per input byte ranges 2.10-4.96 where a clean decode would be near-constant). So the container is identified and the duration is not. Next step recorded: find why FFmpeg stops early, likely the hardcoded packet/subframe fields. Artifact: examples/slb_fmt_probe.rs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE |
||
|
Sylpheed RE agent
|
af320466bb |
re: the resupply banks really are missing audio — the subtitles prove it
The corpus said 0.14 s is "far too short for the spoken line". That is a judgement about audio, and audio judgements cannot be made in this container. The subtitle tracks settle it without listening: each carries cue START times, and a subtitle that appears at t seconds cannot sit inside a clip shorter than t. FFmpeg-measured (not estimated from a compression ratio -- the first version of this example used an 8:1 guess, which is not good enough to hang a conclusion on): hokyu_LS_s02A D_450 cue 4.00 s audio 1.41 s MISSING hokyu_LS_s09A D_451 cue 3.70 s audio 1.81 s MISSING hokyu_LS_s02H D_453 cue 4.70 s audio 0.07 s MISSING hokyu_DS_s13A D_452 cue 0.00 s audio 1.21 s no signal hokyu_DS_s07H D_454 cue 0.00 s audio 0.21 s no signal Three of five are decisive; the other two have their only cue at 0.0 s and say nothing either way. So something is genuinely missing from these banks -- established independently of the leading-region work, and measured rather than felt. The fmt-variation probe I recorded as the next step is INCONCLUSIVE and is written up as such: 36 combinations over VOICE_D_453's 22-packet leading region all produced 0 PCM bytes, including ones that should be equivalent to the crate's own synth_xma1_fmt, which does parse. So the probe tested my hand-built fmt chunk, not the hypothesis, and it is NOT evidence that the region is non-XMA. The retry should use the crate's helper. Artifact: examples/voice_len_vs_subs.rs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE |
||
|
Sylpheed RE agent
|
49a09a9496 |
re: the "sibling default" rules are a dedup artefact — WITHDRAWN
The corpus recorded that some unit fields the disc leaves defaulted inherit from a sibling: Size_Y from Size_X, FCSRange from RadarRange, DefencePoint from AttackVesselPoint. Size_Y was marked the one to trust, on 9/9 support across 7 independent ships, and it is restated in INDEX.md. The premise is false. These fields are not defaulted -- they are on disc for 113-114 of 114 unit tables -- and Size_Y DIFFERS from Size_X in 90 of them. The mechanism, cross-tabulating "legacy reader missed it" against "equal on disc": pair seen+differ seen+equal miss+differ miss+equal Size_Y / Size_X 90 0 0 24 FCSRange / RadarRange 54 0 1 58 DefencePoint / AttackVesselPoint 51 0 1 61 seen+equal is 0 for all three: a value shared with a sibling is ALWAYS invisible to the string-pool reader, because the pool stores each distinct string once. And the reader almost never misses a value that differs. So "the missing value equals the sibling's" was true BY CONSTRUCTION -- the rule re-derived the very condition that made the field go missing. That is why the support looked perfect: it could not fail on the cases it was fitted to. The two miss+differ cells are its real wrong predictions, both named: UN_e104_ADAN_Carrier DefencePoint is 0.2 (rule says 0.003), and UN_e011_ADAN_Attacker_B_HF_Wayne FCSRange is 3000.0 (rule says 6000.0). Retracted in unit-struct-runtime.md (original reasoning kept below the correction), live-unit-definitions.md and INDEX.md. Pinned by a disc test that asserts the seen+equal cells stay zero, so the mechanism itself is guarded, not just the counts. Artifact: examples/sibling_rule_check.rs. This one was found by my own check after the subagent assigned to it stalled. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE |
||
|
Sylpheed RE agent
|
6fa6564be8 |
re: the .slb leading region is 1392+n*2048 — and my fix for it is withdrawn
The structure is exact. In all five resupply banks the first RIFF sits at HEADERLESS_DATA_OFFSET + n*2048, where 1392 is a constant this crate already had and 2048 is the XMA1 packet size: n = 8, 1, 7, 22, 29. No free parameter to tune, and the raw bytes agree -- high entropy from offset 0, then a zero run immediately before the RIFF. VOICE_D_451 is the control, its single packet being all zeros. So I made the obvious fix, emitting that region as a sub-wave, and then withdrew it on two measurements: * It does not recover audio. Coverage went 5.4% -> 89.9% for VOICE_D_453, but the emitted stream decodes through FFmpeg to 1792 PCM bytes -- silence -- while the RIFF sub-waves from the same banks decode to 150-270 KB. Byte coverage was the wrong success metric and it looked like progress. * It is not narrow. The rule matches 1524 of the 8021 RIFF-bearing entries in sound.pak, including RT* movie banks that decode correctly today. Landing it would have risked a wide regression in order to not-fix five banks. to_xma_riffs is back to its previous behaviour, verified by re-measuring: coverage is 5.4% / 9.7% again. The refuted attempt is recorded in the code beside the branch it would have changed, so the next person does not re-derive the arithmetic and re-make the change. XMA1_PACKET is kept as a named constant because the blast-radius scan uses it. Artifacts: examples/voice_bank_shape.rs (structure), voice_bank_dump.rs (sub-waves for decoding), slb_hybrid_scan.rs (the 1524 count). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE |
||
|
Sylpheed RE agent
|
fedb31a5f9 |
re: rebuild the movie manifest on the record table — the old counts measured pool strings
movie_manifest::parse now reads BASE_INFO's positional field keys (the game's own cutscene ids, stage*100 + slot) and follows each to its record, instead of scraping the string pool. The pool stores each distinct string once, so a REPEAT reference produced no token and read as "no binding". That single cause explains every wrong cell: 13 later references to VOICE_D_450..454, two to SUBTITLE_hokyu_LS_s11A.tbl, and MS01A's share of pwterop_s01a.prt. All 18 hokyu movies are bound, not five. Counts, verified independently by me against the disc before recording: 104 cutscene SLOTS binding 101 distinct MOVIES; 99 slots / 96 movies with a voice track, 99 / 96 with a subtitle, 22 / 22 with a telop. The docs' old 94 / 83 / 21 are exactly the counts of DISTINCT POOL STRINGS -- not wrong measurements, measurements of the wrong thing. Three denominators were being conflated; the new test pins all three. Two assertions in movie_manifest_disc.rs were false and are corrected: hokyu_DS_s13A binds VOICE_D_452 and resolves to eng\etc\VOICE_D_452.slb. The in-game verdict that rejected that value tested an INFERENCE from a shared demo id, on a decoder that discards 85-87% of banks in this class -- see voice-bank-leading-region.md, committed earlier today. The ~104 script ids are no longer open: they are literal positional keys, each naming its record, and all 104 resolve. The old "counts differ by three, positional pairing does not work" has a concrete cause -- three resupply movies are bound by TWO slots each. Also corrected: the naming convention has 3 subtitle exceptions (s24A/s27A borrow s11A's track) and 18 voice exceptions, not one and five. The legacy scraper is kept as a fallback for blobs with no record table, so the synthetic unit fixtures still exercise it. Artifacts: examples/movie_map_csv.rs regenerates the CSV, now slot-keyed (104 rows; the movie-keyed version silently dropped one slot of each duplicate). Disc tests green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE |
||
|
Sylpheed RE agent
|
49c00e0955 |
re: the voice decoder discards up to 87% of a bank — "multi-subwave" refuted
The record table gives a DIRECT binding hokyu_DS_s13A -> VOICE_D_452, where the corpus records the movie as unbound and movie_manifest_disc.rs asserts None, citing an in-game verdict that this exact value was "the wrong recording". That is the only place on the disc where a runtime observation disagrees with the record table, so it was worth settling. First, shape: these banks are SHARED. Five slots bind VOICE_D_452, five bind 451, four 450, four 453, three 454 -- 21 hokyu slots over five banks, and the movies repeat too. Generic resupply cutscenes, not per-stage recordings. The recorded explanation for 453 decoding to 0.14 s and 454 to 0.43 s was that the banks are "likely multi-subwave / not cleanly sliced". Refuted: the count of RIFF magics EQUALS the number of sub-waves recovered in all five banks, and the last data chunk ends exactly at EOF in four of them. Nothing between or after sub-waves is being missed. The real defect: slb::to_xma_riffs finds audio by searching for the RIFF magic, and a large region PRECEDES it. 87% of VOICE_D_453 and 85% of VOICE_D_454 sit in front of the first RIFF -- 21-27% zero over 256 distinct byte values, i.e. content, not padding. VOICE_D_451 is the control: its leading region is 100% zero, 1 distinct value, real padding. So the in-game verdict listened to a decode that had discarded most of the bank, for exactly this bank class. It is evidence about the decoder, not about the mapping. Note also that what was rejected was a value INFERRED from a shared demo id; the record table supplies the same value as a stored field, and only the inference was ever tested. This does NOT establish the binding is right -- it removes the only recorded evidence against it. What the leading region actually holds is undecoded, and confirming the binding needs a human listening. Artifact: examples/voice_bank_shape.rs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE |
||
|
Sylpheed RE agent
|
9141ea2b24 |
re: audit the legacy IDXD reader against the real field table — and fix a test that encoded its error
With the record table decoded there is finally a ground truth to check the
old string-pool reader against. It infers `key -> value` from pool adjacency,
which is a consequence of how records are written, not a rule of the format.
Verified by hand against the disc, with an independent parser:
* `FCSRange` = 500000.0 — the module docs' own canonical example of a field
"left at its default" that "omits the value string".
* `ShieldRatio` = 1.0, where `tests/pak_idxd_disc.rs` asserted None with the
comment "a defaulted/omitted field must be None". That test encoded the
false belief; it now keeps the None as a deliberate characterisation of the
legacy reader, with the true value asserted beside it.
* `get_raw("Model")` on GP_HANGAR_ARSENAL returns the first record's model for
every record — silent corruption, not an absent value. New test pins four
records that disagree with it.
The cause is the flat API having no way to name a record: only 548 of 6325
objects have one. `HP` on the DeltaSaber answers 1000.0, the hull, while 63
Turret_* records each carry their own 100.0 (measured — a first draft said 34,
taken from a report rather than from the disc).
Disc-wide rates are recorded as single-source and labelled as such: get_raw
52% wrong, typed getters 38% miss, but 100% correct on single-record objects.
Also records a negative result: the 504 unnamed field keys were NOT recovered.
A 572464-string dictionary and 73191 variants gave 0/42. The key deltas do
prove the preimage ends with the two decimal digits.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
|
||
|
Sylpheed RE agent
|
af32540190 |
re: decode the IDXD/IXUD record table — and there is no schema hash
The binary region in front of the string pool was the parser's oldest open
note ("Not yet decoded"). It is a uniform 16-byte record array sorted by
name hash, a field count, a 12-byte field array sorted by key, a pool size,
and the pool. The trailing `pool_size == file_len - pool_base` identity makes
the layout self-checking, which is what caught the first wrong version.
Verified over the WHOLE disc with zero failures: 7750/7750 IDXD objects,
190782/190782 records reproducing their stored tag_hash, 1271462/1271462
named fields reproducing their key. IXUD is the same container with
ixud_hash, UTF-16BE and every offset in chars — 1104/1104 objects,
628165/628165 fields, checked with an independent parser.
Field names are stored on disc, so no preimage search is needed: a field's
middle word points at its own name. Only 504 fields disc-wide are hash-keyed
with no name; the other 1485073 nameless fields are positional, keyed by a
literal integer (line slots, movie ids).
Two long-held beliefs are WITHDRAWN:
* The word at 0x08 is not a schema hash. It is record 0's name_hash — the
format has no type field at all, and an object's kind is known only from
the caller that loads it. It survived as "schema" because tables of one
kind share their lowest-hashed record name. Caught by a test asserting
every movie id names a real record: 1005 -> STAGE10_PHASE01 failed because
tag_hash("STAGE10_PHASE01") IS 0x067025B9, that table's supposed schema id.
* The field's middle word is not an always-0xFFFFFFFF flags word. It is
0xFFFFFFFF for 54% of fields, enough to look constant in a small sample;
the tell was that it is constant per key ACROSS records, which a per-record
flag cannot be but a per-name pointer must.
`schema_hash` keeps its name rather than churn 33 call sites, with corrected
docs. The first sweep globbed dat/** and missed hidden/DefTables.pak (1425
objects); the test now walks the whole disc root.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
|
||
|
Sylpheed RE agent
|
f6508dd8ac |
re: locate both guest hash routines; IXUD solved; two corrections
Found the routines in the disassembly DB rather than guessing from data: sub_82447DF0 IDXD tag hash (lbz+extsb, modulus 0x00FFFFDF, magic 0x2101) sub_82447E70 IXUD tag hash (lhz, 64-bit, modulus 0xFFFFFF67 then 0x00FFFFDF) Both transcribed instruction-for-instruction into Python and Rust. IXUD SOLVED. It defeated every single-modulus search because it chains TWO exact moduli -- the loop reduces mod 2^32-153 in 64-bit arithmetic and only the result is folded mod 2^24-33. A polynomial mod M1 folded through M2 is not a polynomial mod anything, which is exactly why the gcd test returned 1. Verified independently: 86/86 record keys and 108,261/108,261 field tags in GP_MAIN_GAME_E.pak, and NoRecord -> 0x1c6d9c96. CORRECTION 1: tag_hash must SIGN-EXTEND each byte (extsb). My reconstruction used unsigned bytes and matched all 1.27M disc names -- every one is ASCII -- while disagreeing on ~90% of random inputs with a byte >= 0x80 (verified: 18096/20000). The disc could never have caught this; only the disassembly did. CORRECTION 2: name_hash's reduction is EXACT, not lossy. The module doc claimed the missing conditional subtract made it something other than %. rlwinm r6,r6, 9,23,31 is just hi>>23, and with RECIP = floor(2^55/M)+1 that is Granlund- Montgomery magic division -- 0 wrong at every quotient boundary across the full 32-bit domain. Retracted. cargo test -p sylpheed-formats --lib hash: 10/10. |