926 Commits

Author SHA1 Message Date
Sylpheed RE agent
7eeae3006a re(ui): the focus ring SPINS, the game draws it, and the leaf owns the f record
Three things, all from parsing ptbtn0Nf.rat as a build.

1. THE RING SPINS. Its two keyframes differ in exactly one field: rotation_deg
ramps 0 -> 360 with position, scale, alpha and tint all constant. A spin in
place, the same shape as the GP_BUNK example already recorded.

2. THE ORACLE CONFIRMS THE GAME RENDERS IT. In the OPTIONS-focused capture the
ring's bright head sits in a completely different angular position from the
sprite's own -- caught mid-spin. This is a SECOND independent confirmation that
rotation_deg is drawn, now on a different screen and a different element from the
ptloop sweeps, and it raises rotation's priority: it is not a title-only concern
that sits off-screen at rest, it is the main menu's focus marker.
NO ANGLE IS QUOTED. A brightest-region centroid says ~250 deg, but the control
refuses that precision -- rotating the sprite by a known 30/90/180/270 and
re-measuring gives errors up to 19.8 deg. What survives the error bar is that a
<=20 deg error cannot manufacture a ~250 deg displacement.

3. WHICH PLACEMENT WINS -- correcting this page's own earlier caveat, which said
to use the leaf only for elements the parent does not declare. Right for a BASE
record, wrong for an f record: the parent declares NO element for ptbtn0Nf.rat at
all (zero of build 5's 16), so the f record's placement comes from its leaf for
BOTH elements, label included. The label's (-7,-7) is load-bearing -- the f
sprite is 13px larger per axis and -7 keeps them concentric (535+96/2 = 583 vs
542+83/2 = 583.5). Corroborated against the oracle: the focused-minus-unfocused
region is x 505..703, and the leaf predicts a right edge near 707 where the
parent reading predicts 714.

Also exposes UiBuild::records (name -> (offset, size) of a nested .rat leaf).
Nested records were parsed into a PRIVATE map, so a consumer holding a UiBuild
could not locate a leaf's bytes at all -- which is exactly what blocked the port
from reaching the ring.
2026-08-29 08:46:53 +00:00
Sylpheed RE agent
d110cf38c7 media: expose se_wave_riff -- the menu's SE cues, assembled where the format lives
The port is forbidden from reimplementing media assembly and Static.slb is
exactly that case: no RIFF, no seek chunk, no XACT container, just a packed run
of whole 2048-byte XMA1 packets, so a wave is defined only by (offset, packet
count) and the header has to be synthesized. That step now happens once, in the
crate that owns the format, instead of in each consumer.

`slb::xma1_wave_riff` wraps raw packets; `media::se_wave_riff` looks the bank up
and reads just the packets asked for. Both reuse the existing synth_xma1_fmt /
build_riff, which are already byte-identical to what tools/re-capture/
slb_extract_wave.py writes -- so this is exposure, not a second implementation.

It reads a TARGETED range rather than the whole bank, and that is load-bearing:
Static.slb is the ONE entry of sound.pak's 9 519 whose declared extent runs past
the end of the extracted segments -- by exactly 616 768 B -- so reading it whole
fails outright on this extraction. Every cue we need is in the first few hundred
KB. Recorded rather than worked around silently.

Verified as an artifact, not a compile: all three cues decode through ffmpeg to
mono 48 kHz PCM at 0.533 / 0.344 / 1.016 s, non-silent (rms 2085 / 2985 / 4327,
peaks 29813 / 16973 / 32767). The refusal path is exercised in the same run --
an impossible packet count is rejected rather than returning a short stream,
because a truncated XMA decodes to plausible-sounding garbage.

Also adds docs/re/captures/ORACLE-CAPTURES.md: an index of the nine canary
framebuffer captures already in this repo, and a plain statement that THEY are
the reference and `screen render` is not.
2026-08-29 08:41:42 +00:00
Sylpheed RE agent
b21c8e4118 re(ui): the focus ring's position is decoded -- a .rat leaf parses as a build
The port needed ptbtneff01.t32's placement and was about to author it from an
eyeballed PNG measurement. It does not have to: a `.rat` leaf needs no new
reader. Its first 32 bytes have a bundle header's shape -- "RATC", 0x3c
declaration-entry size at +4, element count at +20, design 1280x720 at +24/+28 --
so ui_layout::parse_build reads it unchanged.

The control is the base record, whose position is known independently: the parent
screen reports ptbtn01.rat resting at (542,162), and parsing the leaf alone
returns ptbtn01.t32 at (542,162). It reproduces all five buttons.

Positions are absolute design-space top-left. The ring rests at (500, 156/236/
316/396/476) for buttons 1-5 -- a uniform (-42,-6) from each button's own rest,
identical in the Japanese bundle. The bright label is a uniform (-7,-7).

Two things recorded rather than smoothed over: a leaf's placement DUPLICATES the
parent's rather than being relative to it, and the two copies are not always
byte-equal (ptbtn04's parent says y=401, its leaf says 402) -- the parent is what
compose honours, so the leaf is the source only for elements the parent does not
declare, which is exactly the ring. And `screen render --focus` is blind to the
ring for the same reason the port's exporter was: el.focused is name-based on
top-level elements and neither walks into the leaf.
2026-08-29 08:36:53 +00:00
Sylpheed RE agent
9ca1eb50fd re(ui): answer four of the port's five asks -- splash, fade-out, focus, gamma
1 SPLASH ADDRESSING (was blocking P3). No content predicate exists: design size
fails (every extra composable bundle sampled is 1280x720, like every screen) and
element count fails (fragments run 2..15, the splash halves are 3 and 7). But
GP_TITLE needs none -- `--all` adds exactly four bundles there and all four are
real screens, with the --all index equal to the pak entry index 1:1. And there
are TWO splash screens: 11/14 are the developer logos, 10/13 are the SQUARE ENIX
publisher wordmark, which the port did not have and which the boot shows first.

2 FADE-OUT (was blocking P3). It is (a), and it is bigger than the fade quad.
Every element ends on exactly ONE untimed keyframe, which rules out (b); that
block is where the screen plays out -- quad to a=255, buttons/labels/glows to
a=0, frames hold. (c) is refuted by a null test that discriminates: a black quad
alone holds the button/background brightness ratio constant, and through the
fade it falls 6.50 -> 1.94, 3.4x monotonic.

3 FOCUS (saves P5 rework). Over-vs-instead is unobservable -- the focused sprite
covers the base at 100.0% of base-visible pixels on three pairs once aligned
(true offset (7,7); the centre alignment reads a misleading 78-84%), and
compositing both ways differs by RMSE 1.1 inside the button rect. The real defect
is the focus record's SECOND element: ptbtn0Nf.rat declares ptbtneff01.t32 (a
42x46 glowing ring, focus only) plus the bright label, where the base declares
one sprite. That ring is the marker the port draws nowhere.

5 GAMMA. The capture is not neutral: capture ~ 255*(render/255)^g, g ~ 1.34-1.49,
and the chain says it is a ramp the GAME installed, not a capture artefact. So
RMSE against captures has a floor. Reach stated: the flat patches are all dark
(render ~0-60), so midtones and highlights are unconstrained.

4 ROTATION is a human's call and is recorded in MISSION, not acted on -- the port
rotating while the reference renderer does not would make verify-screen report a
large diff meaning "the port is right". The RE half is answered: rotation is
about the declared pivot, measured against a GPU capture.

The focus record's +20 element-count word is marked 🟡 not  -- read on GP_TITLE's
ten button records only; the disc-wide check is written and still running.
2026-08-29 08:19:23 +00:00
Sylpheed RE agent
9a0ca0d71f docs(method): time the disc mesh suite -- 22 minutes of silence is not a hang
Two runs were killed this session for looking stuck. Measured: 1 318 s serial,
no output while it runs. Also records that `build-reborn test` forces
--workspace and silently ignores a `-p`, so scoping needs the cargo
passthrough (`build-reborn t -p <crate>`).
2026-08-29 07:49:18 +00:00
Sylpheed RE agent
f817dd5939 re(ui): the 60 nameless RATC children are frames, not children -- .tan decoded
Closes the reach caveat the `opt ` name fix left behind: 60 of 18 002 RATC
children carry no `opt ` block, and it was not established whether they lack one
or sit past our 128-byte window.

Neither. They are not children. `examples/ratc_optless_children.rs` re-runs
`ratc::parse`'s own guards over the disc and reports which one fired: all 60 are
"tag beyond the window", none is rejected by length, gap or charset, none is
child #0, and all 60 live in six bundles of one archive. Within a bundle the
distances back to the nearest tag are an exact arithmetic progression, step
60 600 -- ten different records finding the SAME tag, because there is only one.

Reading a bundle directly: children 1..10 are equal-size T8aD blocks under a
single `opt ` name, `pb_f15_eg_anm.tan`. `.tan` is a FRAME SEQUENCE. One block
declares the resource; its payload is a run of T8aD frames.

Disc-wide, over all 18 718 `opt ` names in all 33 paks: a RATC bundle names
exactly six kinds of resource -- `.t32` 14 756, `.rat` 3 311, `.prm` 367,
`.tbm` 224, `.sbo` 54, `.tan` 6. Six `.tan`, ten frames each = 60, the entire
population with nothing left over. The negative is closed, not narrowed.

Consequence recorded but deliberately not fixed: `ratc::parse` over-reports
there, listing a `.tan`'s frames as anonymous children. Nothing in the menu
milestone reads a `.tan` -- it occurs only in GP_READY_ROOM, which S1 ruled
out -- so no screen the port draws changes.

Also a METHOD entry for this container OOM-killing `slb_leading_segment_disc`
under default test parallelism (SIGKILL, no assertion; 8/8 pass with
--test-threads=1).
2026-08-29 07:39:09 +00:00
Sylpheed RE agent
56cc7acfc3 re(ui): a RATC child's name is stated, not inferred -- and it was hiding every menu background
`ratc::parse` named each child by scanning backwards for the last printable run
of bytes before its magic. The format states the name explicitly instead, in an
`opt ` block: `"opt " | BE32 len | name | NUL | 3 bytes | magic` -- the same
block `ui_layout::opt_link` already read for a button's focus link.

The scan agrees with it 17 918 times out of 17 942 and is wrong 24 times, every
one the same failure: the 3 trailing payload bytes are themselves printable and
beat the real name. For `pteff05.t32` those bytes are `38 41 58` = `8AX`, so the
full-resolution background of all five menu screens registered under a name no
element declares, resolved to no sprite, and `compose` dropped it through an
early `continue` that -- unlike the two arms above it -- records nothing. The
screen lost its background and `screen render` still reported "all resolved".

`8AX` was never a name. Docs that treated it as one are corrected here.

Disc-wide, and the control is the 17 918 the scan already got right: the `opt `
reading reproduces every one of them. Effect on the five screens is the
signature of the same art at twice the resolution -- mean brightness unmoved,
high-frequency detail x1.15..x1.30 -- which is what the separately-measured
`ui-8ax-fullres-background` result said the game draws.

Also closes a long-standing dangling reference: `pmbase.t32`, recorded as "on
the disc nowhere", is the `GP_STAGE_CLEAR` child the scan called `8AX`. RATC
sibling references now resolve 10 148 of 10 148.

Verified: 114/114 sylpheed-formats unit tests (including two new ones pinning
the `8AX` case byte for byte and the no-block fallback), and every disc-gated
integration suite in sylpheed-formats/sylpheed-cli.
2026-08-29 07:26:56 +00:00
Sylpheed RE agent
49f109deb1 agent: the loop must never stop itself
A run ended with a clean exit 0 while the display title read "Loop interval
optimization", leaving four files uncommitted in the tree. Nothing crashed --
the loop was ended, and ending the loop ends the run: the container exits and
there is no next iteration.

The prompt said the agent did not NEED to arm a wakeup. It never said not to
stop one, and an agent that reads about a pacing control will reasonably try to
use it. Now explicit: do not call ScheduleWakeup at all, and if the cadence is
wrong, say so and leave it to a human -- the interval is set outside the prompt.

Recorded with the date and the symptom, because "the container exited cleanly"
looks like a finished job rather than a self-inflicted stop.
2026-08-29 08:12:42 +02:00
Sylpheed RE agent
7a4e4333f4 docs: withdraw yesterday's "paint order is a sequence" -- wrong source
Last iteration I claimed the splash's measured_paint_order [0,2,4,6,1,3,5]
records, between its glow and logo halves, the temporal order they were
seen in rather than depth -- because the halves never share a frame.

The no-overlap measurement is right (glows f94-115, logos f116-211). The
inference is wrong, on two independent grounds:

  * Wrong source. That vector is not a read of the draw capture. It is a
    read of the live screen object's CHILD ARRAY -- ui-screen-runtime.md
    records it literally as "paint order (child slots)". A child list has
    a definite order whether or not its children are ever drawn together,
    so co-occurrence does not bear on it. The capture was the CHECK.
  * The order is in the file anyway. paint_order_audit on GP_TITLE entry
    11: derived == measured, 0 inverted pairs, 0 same-layer-key ties. The
    glows and logos carry distinct T8aD keys (0xa100 < 0xa110), so the
    file orders the halves statically, no capture involved.

I asked the question that started this iteration -- do the title and menu
orders have the same problem -- and the answer is that none of the three
does, for the same reason.

What survives is narrower and now recorded with numbers: how much of each
order its capture actually cross-checks. The title capture is stable (8
draws / 12 quads / 5 textures, identical in all five captured frames
across two logs) and confirms 7 of 24 positions; the menu capture is not
(texture 0x11C30000 present in frame 0, gone by frame 3); the splash
capture cannot cross-check its middle at all.

A counting trap worth the tool: count QUADS, not draws. The menu's draw 9
is indices=24 -- six quads batched from one texture. Counting draws reads
9 where 16 are on screen, and an earlier pass of this analysis briefly
"found" three quads for six declarations that way and concluded elements
were missing. They were batched.

METHOD: check what a "measured" value was measured FROM before reasoning
about its limits. The co-occurrence rule is real, and it is specific to
orders read from draw captures.
2026-08-29 05:19:44 +00:00
Sylpheed RE agent
2532c056be docs: the splash .prm measured -- and its paint order is a sequence, not depth
ui-prm-primitives recorded that where a .prm paints on a screen without a
measured order is unsolved. For the developer splash it is now measured.

Every frame opens with two full-screen draws before any sprite. The
second is untextured in all 212 frames with a constant vertex colour of
FF000000 -- opaque black -- matching palogo_eff0.prm's declaration
exactly: kind 0x10, pivot (640,360) -> 1280x720, one keyframe, a = 255.
So the splash backdrop is an opaque black full-screen quad from the
bundle itself, painted behind every sprite, which is why a splash render
needs --black rather than the default backdrop.

Not a general rule, and said so: the measured main-menu order puts
pteff02.prm at position 4 and pteff00.prm LAST, the latter being the
screen-transition fade.

And a correction to an existing row. measured_paint_order returns
[0, 2, 4, 6, 1, 3, 5] for the splash, described as "the .prm, then all
three glows, then the three logos". But the glows and the logos never
appear in the same frame -- 0 overlapping frames in 235 -- and two
elements that never co-occur have no observable relative depth. Between
those halves the vector records the order they were SEEN IN, not a
front-to-back relationship.

That does not make the render wrong, and element 0 is a real depth
observation since the .prm co-occurs with everything. But the type of the
claim matters: reading the vector as depth invites compositing all seven
elements at once, which is exactly what does not reproduce the screen.

METHOD: two things that never co-occur have no observable relative order;
when recording an order, note which pairs actually appeared together.
2026-08-29 05:06:37 +00:00
Sylpheed RE agent
7988f52ce7 re(ui): deliver the measured splash sequence the port has to author
The activation decision is code, and MISSION already says the port
authors the sequence -- so the useful move is to hand over the sequence
measured rather than chase the code. From the 235-frame draw capture, at
1 frame = 1/30 s (2 units/frame, 1 unit = 1/60 s, both settled in Q1):

  publisher: SQUARE ENIX logo    f1-90     90 frames   3.00 s+   at 0.00
  (gap, nothing drawn)           f91-93     3 frames   0.10 s    at 3.00
  developer: both glows          f94-115   22 frames   0.73 s    at 3.10
  developer: both logos          f116-211  96 frames   3.20 s    at 3.83

Three limits, stated with the numbers rather than after them. The capture
opens with palogo_sqex already at a=255, so the publisher phase began
before the window and 3.00 s is a FLOOR -- every "starts at" is relative
to the capture, not to boot. palogo_anima and palogo_anima_eff get 0
draws in all 214 frames, so a third pair's phase is not in this
measurement. And it is one capture, one run: the glow->logo switch being
a single frame boundary with no overlap is a strong shape, but each
duration is one sample.

What is solid is the part that matters: the 0.73 s and 3.20 s phases are
each within 2% of their element's declared span, so the durations are the
bundle's own and only the ordering is authored. That is the difference
between a port transcribing timing and inventing it.
2026-08-29 05:00:54 +00:00
Sylpheed RE agent
7156591654 re(ui): re-establish selective activation by killing the alternative statically
Last iteration I withdrew "a bundle is a palette" because the evidence
did not choose between selective activation within one bundle and two
compositions shown in sequence. The alternative can be killed from the
disc, which I had not tried.

Hypothesis 2 needs a bundle declaring the GLOWS WITHOUT THE LOGOS. There
is none. Every GP_TITLE entry carrying palogo elements:

  10, 13 (publisher twins)  palogo_eff0.prm, palogo_sqex, palogo_sqex_eff
  11, 14 (developer twins)  palogo_eff0.prm + all three logos + all three glows

Four entries, and each developer entry declares the complete set of six.
So whichever bundle was active across frames 94-211 -- entry 11, entry
14, or both in turn -- it declared the logos and the glows, while the
game drew two sprites at a time in disjoint phases.

Therefore only a subset of a bundle's elements is drawn at any moment,
whatever the bundle-loading story is. The conclusion no longer depends on
resolving how many bundles are involved, which is why the texture-base
test's failure stopped mattering.

So the claim is reinstated -- this time by eliminating the alternative
rather than by assuming it away. What worked was not a better capture but
asking what the competing hypothesis would REQUIRE on the disc and
finding it absent.

METHOD: a hypothesis that predicts an artefact can be killed by looking
for the artefact, which is often far cheaper than measuring the behaviour.
2026-08-29 04:56:13 +00:00
Sylpheed RE agent
2a086051b3 re(ui): withdraw the mechanism -- "palette" was an explanation, not a finding
Last iteration I wrote that a bundle is a palette whose elements are
selectively activated. The disjoint glow/logo phases have two
explanations and I asserted one:

  1. one bundle, some elements run then others;
  2. two bundle-loads shown in sequence (entries 11 and 14 are twins
     declaring identical sprites).

The draw log's tex[base=...] looked like it would separate them. It does
not, and the control is in the same table:

  publisher splash  f1-90     0x11C30000, 0x10000000
  glows             f94-115   0x11C30000, 0x10000000
  logos             f116-211  0x11C30000, 0x10000000

The publisher splash is certainly a DIFFERENT bundle from the developer
splash, and it uses the same base. So 0x11C30000 is a reused upload slot,
not a bundle identity, and the test cannot choose between the two
hypotheses.

Survives: a bundle's declared elements are not what gets drawn.
palogo_anima and palogo_gamearts carry byte-identical keyframe times and
in the same run one is drawn 95 frames and the other none -- and
whichever twin was active declares both. The phases are strictly disjoint
(0 overlapping frames in 235).

Withdrawn: the mechanism. The practical consequence is unchanged --
compositing every element of a bundle does not reproduce what the game
shows over time -- but the why is not established and I stated it as
though it were.

What would separate them: a per-draw capture recording the bundle each
draw came from, or a file-IO log showing whether a second RATC entry is
read between frames 115 and 116.

METHOD: a shared resource address does not identify the resource's owner;
and state the mechanism as a separate claim from the observation, or the
weaker one inherits the stronger one's evidence.
2026-08-29 04:51:34 +00:00
Sylpheed RE agent
baed44a9ea re(ui): the sequencing survives refutation -- and a bundle is a palette
Two checks on last iteration's "sequential, not simultaneous" reading.

First, the phases really are disjoint. If glows and logos ever shared a
frame the claim would be wrong. Across all 235 captured frames the count
of frames containing both is ZERO, and the switch is a single clean
boundary -- f110-115 draw 1280x720 + 262x108 + 525x90, f116 onward
1280x720 + 243x86 + 499x72. Two sprites either side, no transition frame.

Second, and larger: a third of the bundle is never drawn. Entry 11
declares three logo/glow pairs and only two appear.

  palogo_gamearts / _eff    95 / 22 frames
  palogo_seta     / _eff    95 / 22 frames
  palogo_anima    / _eff    never

palogo_anima declares the SAME keyframe times as palogo_gamearts. Two
elements with byte-identical data, 95 frames and 0 frames in one run.
Reach: the capture covers frames 1-214, so this is "never in the window".

So a bundle is a palette, not a script. Its elements say what to draw and
for how long; which of them run, and when each starts, is decided outside
the placement data. That is the same conclusion the boot-order work
reached from the other end -- the driver is code, not data -- now with a
per-element measurement behind it.

For the port, concretely: compositing every element of a bundle does not
reproduce what the game shows over time. It is the right thing for a
static screen that settles, and it is not a timeline.

METHOD: two elements with identical data and different outcomes is the
strongest possible evidence that the decision is elsewhere.
2026-08-29 04:47:05 +00:00
Sylpheed RE agent
7fb6bdfad8 re(ui): a group's duration is in the data, its start time is not
Tested whether the splash timeline, played, reproduces the capture --
the last gap in the animation model. Half of it does.

Durations match. At 2 units/frame under the shifted reading, from the
235-frame draw capture of the developer splash:

  glows  drawn f94-115  (22 frames = 44 units)  declared ~0..45  = 45   97.8%
  logos  drawn f116-211 (96 frames = 192 units) declared 15..210 = 195  98.5%

Each element is on screen for its declared span to within 2%.

Starts do not. Every glow declares the same times 15,30,45 and every
logo the same 15,30,190,194,206,210, so on one clock they would overlap
almost entirely -- and they do not overlap at all. The glows run 94-115
and the logos 116-211, strictly sequential, the logos starting the frame
after the glows end. Fitting one origin needs f0 ~ 93.5 for gamearts_eff
and ~103 for gamearts, about 19 units apart, and aligning one throws the
other off by ~9 frames at both ends.

The obvious candidate is refuted. parse_placements reads each group
header as (element index, keyframe count) plus one undecoded LEAD-IN
WORD -- exactly where a per-group start offset would live. It is
0x00000000 for all seven elements, glows and logos alike.

Reach: not the keyframe times (identical within each family), not that
word (zero), not declaration order (which interleaves logos and glows
where the observed sequencing groups them), not the RATC child order.
What remains is that the sequencing is code-driven, which agrees with
what the boot-order work concluded independently.

For the port: a group says how long an element animates and what it does,
not when it starts relative to its neighbours. The observed order on the
developer splash -- both glows, then both logos -- is measured for one
screen, not a decoded rule, so the sequencing has to be authored.

METHOD: when a model reproduces durations but not positions, the missing
piece is an origin, not a rate.
2026-08-29 04:42:55 +00:00
Sylpheed RE agent
1f2b469f7e re(ui): a static composite is only meaningful for a screen that settles
The model's sharpest prediction, tested with its control. The draw log
says that on the developer splash the _eff glows are drawn on frames
94-115 and the logos on 116-211, so at the moment the reference capture
was taken EVERY glow is already finished -- including the two that have
plateaus and which rest_plateau therefore renders visible. Suppressing
them should help on the splashes and hurt where a screen genuinely
settles.

  publisher splash   +0.9604 -> +0.9982   +0.0377
  developer splash   +0.9659 -> +0.9980   +0.0321
  title    (control) +0.9500 -> +0.9480   -0.0020
  main menu(control) +0.9460 -> +0.8544   -0.0916
  EXTRAS   (control) +0.9440 -> +0.8370   -0.1070

Both splashes jump to about 0.998; all three persistent screens get
worse. The control is what makes this a finding rather than a
coincidence: the same edit helps exactly where the model says it should
and hurts exactly where it says it should not.

So rest_plateau is not over-drawing in general -- it over-draws on
TRANSIENT screens. A plateau mid-animation means the element is held at
that point in the timeline, not that it is on screen once the screen has
settled. Where a screen settles, the held pose IS the settled pose and
the rule is measurably right.

And that answers the question left open several iterations ago -- what
"rest" means for a transient element. It does not mean anything: the
splashes never rest. A static composite of them can match a chosen frame,
and about 0.998 is what these captures' frame is worth, but the format
does not answer a question the screen never poses.

For the port: play the timeline for the two splashes, which the settled
keyframe timing now supports, and composite statically for title, main
menu and EXTRAS.

METHOD: an edit that improves one set of cases is only interesting once
you have shown it damages the cases where it should.
2026-08-29 04:37:59 +00:00
Sylpheed RE agent
458585d136 re(ui): the structural case for last -- 2 293 of 2 305, checked disc-wide
The weakness in the rest-rule finding was that `last` had been SCORED on
only two elements. It cannot be scored on more -- only two ambiguous
elements sit on a screen with a live capture -- but the entry -> hold ->
exit model makes a prediction that can be checked on all 2 305: what does
each element's FINAL keyframe look like?

  final keyframe invisible (a = 0)          1 618   transient: gone at rest
  final keyframe visible, at max alpha        675   faded in and stopped
  final keyframe visible, BELOW max alpha      12   genuinely unclear

Of the 687 that end visible, 472 have monotonically non-decreasing alpha
-- a plain fade-in that stops, [0, 255] over two keyframes in the
commonest case (pjex_eff.rat, pghud_speed_cut.t32) -- and another 203 end
at their maximum after dipping.

So `last` is structurally defensible for 2 293 of 2 305 (99.5 %), against
a dwell rule that returns a mid-movement frame by construction. Observed
correct for 2, structural for 675, model-consistent for 1 618, unclear
for 12.

The assumption carrying the 1 618 is stated rather than buried: that a
plateau-less element's animation has finished by the time the screen is
settled. The draw log establishes exactly this for the two splash glows
(drawn frames 94-115, logos 116-211) and establishes nothing for the
rest.

Default still unchanged. The case is now observational, structural and
model-based rather than two data points, but it would move 1 896 elements
and the decision belongs with whoever owns the renderer.
2026-08-29 04:34:21 +00:00
Sylpheed RE agent
4012d5b555 re(ui): why rest_plateau is right -- and last is right only for a transient
The shifted keyframe-time reading looked like it implied something
simple: the final pose is reached at a definite time and nothing follows,
so rest should just be the last keyframe and the plateau heuristic could
go. Tested by applying it to EVERY element:

  title              +0.9500 -> +0.6819   -0.2681
  main menu          +0.9460 -> +0.6416   -0.3044
  EXTRAS             +0.9440 -> +0.5745   -0.3695
  publisher splash   +0.9600 -> blank (zero variance, corr undefined)
  developer splash   +0.9643 -> blank

Refuted, and the failure supplies the model. A group is entry -> hold ->
exit, and the exit is the screen's DISMISSAL. While a screen is displayed
it has not reached its last keyframe; it is sitting at the hold. So
rest_plateau is the correct primary rule, and the last keyframe is the
post-exit state -- correct only once the screen is gone, which is why
applying it everywhere blanks the splashes.

This does not contradict the shifted reading. That reading says when each
pose is reached; it says nothing about the group being played to
completion while the screen is still up. The step between them was mine.

And it explains why last wins for the two plateau-less elements: an
element with no hold is a transient, it flashes and is over, and at any
settled moment it is gone -- which is its last keyframe. The draw capture
says the same independently: on the developer splash the _eff glows draw
on frames 94-115 and the logos on 116-211, so the glows are already
finished when the logos are up.

Three independent observables -- animation timing, static composites, and
the per-frame draw log -- now agree on one rule: plateau where there is
one, last keyframe where there is not.

METHOD: a blank render is a NaN correlation, not a low score, and that
NaN was the strongest form of the result; and when a model predicts
something the measurement refuses, suspect the step you supplied between
them.
2026-08-29 04:28:19 +00:00
Sylpheed RE agent
515456c59a re(ui): quantify what changing the rest rule would do disc-wide
The open question was whether "last keyframe" holds beyond the two
elements I could score against a capture. It cannot be scored disc-wide --
only two ambiguous elements sit on a screen with a live capture -- but the
blast radius can be measured, and it argues the same way.

  genuinely ambiguous elements                       2 305
    the two rules AGREE on                             409  (17.7 %)
    they DIFFER on                                   1 896  (82.3 %)

  dwell (current): invisible pose 1 711 (74.2 %), zero-scale 195 (8.5 %)
  last           : invisible pose 1 618 (70.2 %), zero-scale  43 (1.9 %)

Two things follow. It is not a marginal choice: the rules disagree on 82%
of the affected elements, so "either is fine" is not available. And the
current rule produces 4.5x more degenerate poses -- a zero-scale pose is
collapsed to nothing, i.e. an element's PRE-ROLL before it has grown in,
which is definitionally not a rest. 195 elements currently rest at a frame
they are only passing through, against 43 under last.

That is an argument from the data's own structure rather than from the two
captures, and it points the same direction.

Kept honest: it is indirect. Fewer degenerate results is not the same as
more correct results, and last still returns an invisible pose 70% of the
time -- right for a transient element, wrong for a persistent one. The
default stays put; the numbers are in HANDOFF for whoever decides.
2026-08-29 04:24:40 +00:00
Sylpheed RE agent
93e9b185ea re(ui): the rest fallback fires on 2 elements, and "last keyframe" wins there
Scored candidate rest-pose rules by rendering and correlating instead of
arguing, and both results correct something I had published.

First, the exposure. The guessing fallback is reached only by an element
that is plateau-less AND multi-keyframe -- a single-keyframe element
short-circuits at `match len { 1 => first }`. Per screen:

  title (4)               24 elements   2 plateau-less   0 reach the fallback
  main menu (5)           16            5                0
  EXTRAS (6)              18            5                0
  publisher splash (10)    3            2                1
  developer splash (11)    7            2                1

So on the three screens the port cares most about, rest() never guesses.
That is why three different rules render builds 4/5/6 to identical
correlations -- the code is unreachable there, which I nearly read as
"the choice does not matter".

Second, where it does fire, the last keyframe is markedly better:

  publisher splash   dwell +0.9600   last +0.9982   maxalpha +0.9600
  developer splash   dwell +0.9643   last +0.9758   maxalpha +0.9643

That refutes my own earlier refutation. I had killed the last-keyframe
rule by arguing it makes palogo_anima_eff invisible while its two
siblings stay lit, which looked like an artefact. The capture says
otherwise: making it invisible is what improves the match. The sibling
symmetry was my expectation, not evidence.

Caveat kept in front: both captures are single frames of a transient
animation, so this fixes which pose matches THOSE frames, not which is
canonically at rest. Default unchanged -- better on both screens where it
fires and identical on the other three, but it would move 2 305 elements
disc-wide on two measurements. Reachable via SYLPHEED_REST_RULE=last.

Also confirmed: all 195 zero-scale rest poses are inside the corrected
2 305 ambiguous population; none is a single-keyframe element.

METHOD: score a rule where it can differ, or you measure nothing; and an
argument from symmetry is a prediction, not a refutation.
2026-08-29 04:20:27 +00:00
Sylpheed RE agent
0265da31a1 re(ui): refute my own fix for rest(), and correct the defect rate by 65%
Two corrections from one experiment.

A keyframe group is entry -> hold -> exit, and the exit ends invisible:
on the five port screens the final keyframe is invisible for 21/24
(title), 8/16 (main menu), 12/18 (EXTRAS), 2/3 and 6/7 (splashes). So the
screen as seen is the HOLD, which is why rest_plateau is the right
primary rule and why "rest = last keyframe" would empty every screen.

That suggested a fix: an element with no hold has no representative pose,
so draw nothing rather than guess an endpoint. Tested through compose's
visible mask and correlated against the live captures:

  title       +0.9500 -> +0.6839   -0.2661
  main menu   +0.9460 -> +0.9037   -0.0423
  EXTRAS      +0.9440 -> +0.9094   -0.0346

Refuted on all three, and the reason invalidates a number I published. An
element with a SINGLE keyframe has no adjacent pair, so the plateau test
marks it plateau-less -- but its one pose is unambiguously its rest.
Suppressing those removes backgrounds and full-screen layers, which is
the title's -0.27.

  no plateau (as published)      3 807  (24.57 %)
    ... single-keyframe          1 502  trivially at rest, not a guess
    genuinely ambiguous          2 305  (14.88 %)

So rest() guesses for 2 305 elements, not 3 807 -- the figure I gave the
port overstated the defect by 65%. Corrected in HANDOFF and the page.

METHOD: a predicate over adjacent PAIRS silently misclassifies a
one-element list; and acting on a claim is a better test of it than
re-reading it -- this flaw survived a census, a write-up and a handoff
row, and died the moment the rule was used to change a rendering.
2026-08-29 04:15:58 +00:00
Sylpheed RE agent
60285a6ead re(ui): replicate the keyframe-time shift -- three elements, two screens
The case for reading +36 as "the time the NEXT pose is reached" rested on
one element's fade-out shape, then on one element's hold duration. Both
splash halves supply more, and they agree.

  element            screen              observed hold  as decoded  shifted
  palogo_gamearts    developer splash        83 f          8 f       80 f
  palogo_seta        developer splash        83 f          6 f       80 f
  palogo_sqex        publisher splash       >=77 f *       6 f      102 f

  * the capture opens mid-hold at frame 1, so 77 is a floor.

The readings predict opposite structures. For palogo_gamearts, as decoded:
hold 8f, in 80f, hold 2f, out 6f, out 2f -- an eighty-frame FADE-IN and a
two-frame hold. Shifted: in 8f, hold 80f, out 2f, out 6f, out 2f. The
capture shows an 83-frame hold and no fade-in at all.

The elements that cannot discriminate are not contradicted:
palogo_gamearts_eff observed in 7f / hold 7f / out 8f, and both readings
give 8f phases -- with four blocks the shift only relabels which phase is
which. So the glows, which is where Q1's linear law was measured, say
nothing either way rather than arguing against.

The decoder's default is still unchanged, and the reason is now
articulated rather than assumed. The single thing opposing the shift is
rest() on ptlogo_eff3, where the shifted reading makes the longest-dwell
fallback return the bloom's 200% peak. That fallback is unsound whenever
it runs -- it returns an endpoint of a movement, neither of which is held
-- and checked: the shift does not fix it either. So the objection was
never evidence about the times. Timing had three discriminating
measurements; pose selection had a heuristic guessing.

For the port: animation timing should use the shift; static composites are
unaffected and the five screens' correlations stand. Classified measured,
not decoded -- three elements in one screen family, not a disc-wide check.
2026-08-29 04:09:36 +00:00
Sylpheed RE agent
fb6376a548 docs: one page saying how good the five screens actually are
The answers for the port's five screens were spread across a dozen
documents and none of them said how good the result IS. Measured:

  screen              build  drawn   corr vs capture  alignment
  title                 4    15/24      +0.9500       dy=0 dx=0
  main menu             5    11/16      +0.9460       dy=0 dx=0
  EXTRAS                6    13/18      +0.9440       dy=0 dx=0
  publisher splash     10     2/3       +0.9600       dy=0 dx=0
  developer splash     11     6/7       +0.9643       dy=0 dx=0

Every one aligns at exactly zero offset over a +/-2 px search in both
axes, so placement and scale are right and the residual is tone and
detail rather than geometry.

The drawn/total counts are not slack. Each undrawn element has a reason
already documented: kind & 0x4 ghost instances (4 on the title), .prm
primitives off by default (2 per menu, 1 per splash), loop* animations
off by default and off-screen at rest (2 per menu screen), and the 8AX
name mismatch (1 per menu screen) whose art reaches the screen anyway via
ptbase. 4+2+2+1 = 9, 2+2+1 = 5, 1. Nothing unexplained.

The residual is ranked for a consumer: tone first (gamma 1.34-1.49, the
game's own display ramp), then 8AX resolution, then one drawable
paint-order tie on EXTRAS alone, then rotation-decoded-but-not-rendered
which does not affect these five at rest.

Reach stated: these are static composites at the resting pose against
single frames, so nothing here speaks to animation, and a whole-frame
correlation is a sanity figure rather than a per-element check.
2026-08-29 04:04:15 +00:00
Sylpheed RE agent
0efd692b4c docs: the suite is heavy, not hung -- correcting my own "cannot terminate"
Last iteration I wrote that build-reborn test cannot finish in a working
session, from having watched it run 3h26m. That was the stronger claim
and I made it without measuring the work.

Timing `mesh info` on each of the 166 .xpr containers with a 25 s cap:

  files scanned                166
  exceeding 25 s                19   Hangar, 17 Stage_*, ptc_pack
  Stage_S02 to completion      144 s, rc = 0

Nothing hangs. Nineteen heavy containers at roughly two minutes each is
about 45-60 minutes for one pass, before the 147 fast ones. The 3h26m
observed was that hour of work running at a load average of 9-14 --
inflated by the two duplicate runs I had left going, which did not merely
coexist with the slowness but multiplied it.

The practical conclusion is unchanged and only the wording softens: an
hour-scale suite is not an iteration-scale gate, and every "green" I
reported from it this session was partial. But an hour-scale gate can be
run deliberately, whereas a hung one cannot be run at all, so the
distinction is worth having right.

File list committed as reference data so the cost is attributable without
re-scanning.

METHOD: a slow thing observed under contention looks like a stuck thing;
measure the work before choosing between "cannot finish" and "takes an
hour".
2026-08-29 04:02:34 +00:00
Sylpheed RE agent
145046f88f docs: the verification gate cannot terminate, and I left two runs going 4h
Two findings from checking whether last iteration's partial green had
finished. It had not, and the reason matters for anyone using the gate.

build-reborn test contains twin_pairs_do_not_share_a_buffer, which
decodes every .xpr in hidden/resource3d -- 166 files, 1.4 GB -- through
the full Xbg7Model anchoring path, and is NOT #[ignore]d. Its sibling in
the same file walks the same 166 files and IS ignored as known-failing,
which makes the binary's cost easy to underestimate. Measured: one
instance accumulated 3h26m of CPU at 89% without finishing.

So every "green run" reported in this corpus from a workspace or
sylpheed-formats test is necessarily PARTIAL unless it says the suite
terminated -- including the ones I reported this session. The honest form
is the suite count and elapsed state, not the word "green". Not proposing
to #[ignore] or subsample it: that changes what the suite asserts and is
the project's call, not an audit side-effect.

And the mess is mine. Two cargo test -p sylpheed-formats runs launched
detached in earlier iterations never exited, because they were sitting in
that test:

  pid 103375  4h12m elapsed  child mesh_consistency_disc  3h26m CPU  89.3%
  pid  99965  4h39m elapsed  child pak_idxd_disc          1h16m CPU  93.8%

Load average 14.18 on 12 cores. Killed, after checking the legitimately
running workspace suite and leaving it alone; load fell to 9.68.

What this does NOT explain, because it is tempting: the session's
emulator troubles. screenshot cost 0.49 s with both runaways live and the
emulator stopped, against 10.8 s measured earlier with the emulator
running. The 92x figure really was emulator contention; the runaways were
a background tax on top. The black surface and the unreachable title
stand as measured, with their own controls.

METHOD: a detached job you never check can outlive many iterations --
setsid was added so a timeout could not kill them, which also means
nothing does; and know whether your verification gate can terminate.
2026-08-29 03:45:14 +00:00
Sylpheed RE agent
6d9214fd07 docs: check that the docs' headline figures match their committed data
Nothing had ever verified that a number written in prose matches the
reference data file committed beside it. The figure is written once from
a run; the prose is edited around it afterwards and the data file is
regenerated independently, so drift is silent.

All 19 headline figures across four censuses -- the eff-bit census, the
plateau census, the top-level rotation census and the eff-bit alpha test
-- currently agree with their data files.

The checker had to be numeric, and the first attempt is the reason it is
a script rather than a grep: comparing strings reported almost every
figure as a mismatch, because the data files write 14709 where the docs
write "14 709" with a thin space, and the docs round 33.66 to 33.7. A
consistency check that fails on formatting trains you to ignore it, so
the tolerance is explicit: exact against the data, within 0.05 against
the doc to allow rounding.

Also ran the full disc-gated workspace suite (build-reborn test, which
wires SYLPHEED_DISC -- without it the disc tests self-skip and green
means almost nothing), covering this session's three decoder changes:
rotation_deg on Keyframe, the scale-0 fix in blit/fill_quad, and the
flags field on T8adImage. 122 passed / 0 failed across the four suites
that had completed; the long disc-gated integration tests
(records_roundtrip_disc, first_header_word_is_record0_hash) were still
running and are not counted here.
2026-08-29 03:42:13 +00:00
Sylpheed RE agent
714a26769d re(ui): premultiplied alpha refuted for bit 0x02; parking the field
A per-sprite premultiplied-vs-straight-alpha flag would matter a lot to a
port and has a sharp static signature: premultiplied means RGB <= A
everywhere. Over the 170 decoded GP_TITLE textures that pair to a flag
word:

  bit SET    n= 61   mean %(RGB>A) 55.52   median 52.52
  bit clear  n=109   mean %(RGB>A) 33.66   median 30.17

Premultiplied requires ~0% for the flagged group. Both groups are far
from it and the flagged group violates MORE -- the opposite of the
hypothesis. Refuted.

What remains is a weak association: flagged sprites carry more
bright-RGB/low-alpha pixels, which is what glow art looks like. But the
best single threshold classifies 76.5% against a 64.1% base rate -- a
12-point lift with badly overlapping distributions. A tendency, not a
rule, and reported with its base rate so it cannot read as more.

Noted for whoever returns: "0x02 selects an additive blend" was refuted
by blending those sprites additively and finding every measure worse
against the capture -- but that ran through a title render since fixed
twice (rest_plateau, and the 8AX background the composer drops). The
refutation may well stand; it was measured through a renderer with known
other errors, so it is worth one re-run if blit ever gains additive
blending.

Parking the field. Four candidate meanings are dead -- additive blend,
eff name in both directions, transient element, premultiplied alpha --
none produced a positive account, and the bit blocks nothing: the port's
screens composite at 0.947 correlation against a capture without it. The
negative space and the sound attribution method (child order, not size)
are written down so a later attempt starts here.

METHOD: report a classifier's lift over its base rate; and park a field
after N failed hypotheses, saying what was eliminated.
2026-08-29 03:23:40 +00:00
Sylpheed RE agent
489ea12759 re(ui): the eff-name implication for bit 0x02 is refuted disc-wide
Last iteration I killed the biconditional and reported that the one-way
reading survived: all 10 bit-set sprites on GP_TITLE build 4 are eff
names, so "bit set => eff name". Checked over the disc, that is false.

  sprites with a resolvable preceding name   14 709
    bit SET   & name has 'eff'                2 338
    bit SET   & name lacks 'eff'              2 657   <-- counterexamples
    bit clear & name has 'eff'                1 399
    bit clear & name lacks 'eff'              8 315

  P(eff | set)   = 0.468
  P(eff | clear) = 0.144

The implication fails more often than it holds. What survives is an
association -- 3.3x enrichment -- and build 4's 10/10 was a local naming
habit in an 18-element bundle, not a format rule.

The counterexamples are the useful part: pv_loading_ring0,
pv_loading_light0-3, pv_loading_line, px_bunk_line, px_top_extra. Rings,
glows, lights, thin lines -- effect-like artwork that does not carry the
eff naming convention. Consistent with the bit marking effect sprites by
authoring intent rather than by name, which is a description and not a
decode, and is labelled as such.

Names here come from the string immediately preceding each T8aD,
validated 17/18 on build 4 against the RATC child order; the single
mismatch is the known pteff04.t32 -> registered as 8AX case, so this is
the element (opt) name rather than the sprite's registered name. That
mismatch is itself an independent confirmation of the 8AX finding,
reached from the opposite direction.

METHOD: a pattern perfect on one screen can be near-chance on the disc;
and when an association survives a refuted implication, the
counterexamples are the finding.
2026-08-29 03:20:24 +00:00
Sylpheed RE agent
5f701db588 re(ui): kill two candidate meanings for the T8aD 0x02 bit, and fix attribution
The bit at +0x04 was recorded as a real field with its meaning "not
diagnosed", noting ptlogo_back2eff is 0x8830 "despite its name". That note
rested on a size match -- and its size is ambiguous, which is the trap
this corpus already records.

First, a sound attribution. T8aD headers appear in the bundle in RATC
CHILD ORDER, verified on GP_TITLE build 4 against an independent property
-- each header's decoded dimensions versus the dimensions the named child
should have: 18 of 18 match, 0 mismatches. Two of those eighteen share a
size (ptlogo_back2eff and ptlogo_back2eff5, both 1133x280), so a size-keyed
lookup cannot separate them; ordering can. Index 12 is back2eff5 (0x8832,
bit set), index 14 is back2eff (0x8830, bit clear). The documented
counterexample is real and correctly attributed -- now on evidence.

Two candidate meanings tested and refuted:

  bit <=> name contains "eff"        REFUTED: ptlogo_back2eff is an eff
                                     name with the bit clear. All 10
                                     bit-set sprites are eff names, so
                                     the implication holds one way only.
  bit <=> the element is transient   REFUTED: pteff03/pteff03a carry the
                                     bit and run to t=250, ramping to
                                     a=255 and holding.

Up close, the exception pair differs in two header words: +0x04
0x8832/0x8830 and +0x08 0x8083/0x8081 -- layer keys 32899 and 32897. They
are NOT duplicates: their alpha summaries agree to one decimal (4.5%
opaque, 86.7% clear, mean 19.3) but a pixel compare gives max abs diff 21.
Two renditions of one image at one size, which is why the summaries were
not trusted.

Still not diagnosed, and said so -- but the search space is two smaller
and the attribution beneath it is now sound.

METHOD: T8aD headers sit in child order, use that not the size; and
identical summary statistics are not identical data.
2026-08-29 03:14:40 +00:00
Sylpheed RE agent
c463f98164 re: the GPU trace is compiled out of the release build -- both my guesses wrong
Last iteration left two candidates for why trace_gpu_stream produced no
file: the CLI flag not reaching the cvar, or BeginTracing failing
silently. Neither. Following the code instead of guessing:

BeginTracing only sets trace_state_ = kStreaming ("Streaming starts on
the next primary buffer execute"). The file is opened later, in
ExecutePrimaryBuffer, inside

  #if XE_ENABLE_TRACE_WRITER_INSTRUMENTATION == 1

and trace_writer.h defines that as 0 under NDEBUG, 1 otherwise -- the
trace writer exists only in debug builds.

Confirmed against the binaries, with a control. The format string
"{:08X}_stream.xtr" lives only inside that guard:

  build/bin/Linux/Release/xenia_canary          0 occurrences
  build/bin/Linux/Debug/xenia_canary            1 occurrence
  /sylph-home/re/canary-build/.../Release/...   0   <- what run-canary uses

The debug binary is the control: it proves the test finds the string when
it is present, so the release zero means something.

So trace_gpu_stream is a no-op in this container's emulator -- the cvar
parses, BeginTracing runs, and nothing can open a file. The kill -9 was
not the cause either, though it would have destroyed a trace had one
existed.

The route exists but is not cheap: a debug build with the writer compiled
in sits at build/bin/Linux/Debug/xenia_canary, 253 MB against Release's
18 MB, so a much slower boot plus a trace of every GPU packet on a disk
at 95%. Recorded as available rather than attempted -- what it would
confirm, the DC_LUT write, is already a well-supported inference, and the
cost is out of proportion to the gain.

METHOD: a cvar existing does not mean the feature is compiled in; and
test a compile-time gate against the binary, with a control.
2026-08-29 03:08:57 +00:00
Sylpheed RE agent
921a3cfc44 re: GPU trace attempt produced nothing -- and the config dump is not the flags
Tried to turn the gamma-ramp inference into a direct observation.
canary's trace_gpu_stream records gamma ramps as their own command type
(kGammaRamp, index 11 in TraceCommandType), so a boot trace should show
the write. Two bounded runs produced NO trace file at all -- nothing under
the prefix, no .xtr anywhere, no scratch/gpu/.

Bounded deliberately: the disk is at 95% (50 GiB free) and a trace of all
GPU packets during boot includes video decode, so the runner carried its
own watchdog that killed the emulator the moment output passed a 2 GiB
cap. It never fired -- there was nothing to cap -- and disk stayed at 95%
throughout. Bounding from inside cost nothing and removed any need to
gamble on how coarsely I could poll.

What the attempt did establish. BeginTracing() runs at GPU init when the
cvar is set (graphics_system.cc:237), but EndTracing() runs only from
GraphicsSystem::Shutdown() -- so the kill -9 this session has used
routinely can never finalise a trace. The second run was stopped with
SIGTERM and exited cleanly; still no file, so that is not the whole
story. Two candidates remain unseparated: the CLI flag not reaching the
cvar, or BeginTracing failing silently. The next run removes the
ambiguity by setting trace_gpu_stream in the config FILE instead.

And a trap I nearly fell into. The startup config dump showed
trace_gpu_stream = false after I passed --trace_gpu_stream=true, which
reads as "flag ignored". It is not evidence either way: the gamma run
passed --log_mask=12 --log_level=3, its dump printed log_mask = 0 and
log_level = 2, and Kernel Debug logging was demonstrably ON -- that run
is where VdGetCurrentDisplayGamma was captured. The dump reflects the
config file and can neither confirm nor refute a command-line override.
(It does not undo the earlier user_language conclusion: absence of a NAME
from the dump still shows a cvar is unregistered.)

The gamma-ramp write therefore remains an inference, unchanged.
2026-08-29 03:05:03 +00:00
Sylpheed RE agent
421c61a517 re(ui): close the gamma chain from canary's defaults -- the game writes a ramp
Continues the previous iteration, where the game was measured calling
VdGetCurrentDisplayGamma at video init. The remaining link -- does it then
WRITE the ramp -- is a GPU register operation (XE_GPU_REG_DC_LUT_RW_INDEX
in CommandProcessor::WriteRegister), unlogged and invisible to kernel
logging. Two facts from the source close it without instrumenting.

1. The swap-path gamma stage is a PURE LUT. apply_gamma_table.xesli is
   the whole transform: index by input*255, fetch from a 256-entry ramp
   buffer, output. No sRGB encode, no second transfer function.

2. The table DEFAULTS TO IDENTITY. CommandProcessor::Initialize fills it
   with value = i * 0x3FF / 0xFF, and its own comment says the linear
   default is "what games set when starting with the sRGB (return value 1)
   VdGetCurrentDisplayGamma". An unwritten ramp is a no-op.

So the only transform is a LUT, the LUT is identity unless written, the
game queries the display gamma at init, and the capture differs from our
composite by gamma 1.34-1.49 -- which identity cannot produce. The guest
wrote a non-identity ramp.

Labelled an inference, with its weak joint named: it assumes our
composite reproduces the PRE-RAMP framebuffer, which it does not exactly.
What carries it is the shape -- a systematic ~1.4 fitted on flat patches
across three screens is not a compositor bug. The obvious alternative, a
fixed sRGB stage in the presenter, fits neither direction: an encode
(^0.45) brightens and we measured darkening; a decode (^2.2) darkens far
more than 1.4.

Direct observation remains available and cheap, and needs the emulator
only to boot: a GPU trace records gamma ramps as their own command type,
or one log line at the DC_LUT register write would settle it outright.
Not done.

METHOD: a default value is evidence; and name the weak joint of an
inference in the same breath as the conclusion.
2026-08-29 02:53:45 +00:00
Sylpheed RE agent
c6ce000002 re(ui): the game does query the display gamma -- measured, with its control
Last iteration's corrected experiment, run. Boot with --log_mask=12
--log_level=3 (Kernel logging on, Cpu/Gpu off), which changes nothing
about the output and so cannot perturb the capture harness the way the
gamma-cvar experiment would have.

VdGetCurrentDisplayGamma is called once, at video init:

  d> VdGetSystemCommandBuffer(701CF830, 701CF804)
  d> VdGetCurrentDisplayGamma(701CE1F8(00000000), 701CE1F0(0))
  d> VdSetDisplayMode(40000000)
  d> VdGetCurrentDisplayInformation(701CF110)

The control is in the same log: 359 VdRetrainEDRAM and 358
VdGetSystemCommandBuffer lines, so an absent call would have been
visible. Per the export's own comment the returned type is "used in D3D
SetGammaRamp/SetPWLGamma" -- the game asks the question a ramp-builder
asks, at the moment one would ask it.

Still open, and stated: whether it then WRITES the ramp, and whether the
measured gamma 1.34-1.49 is that ramp. The write is a GPU register
operation (DC_LUT), invisible to kernel logging; a GPU trace records
gamma ramps as a command type (TraceWriter::WriteGammaRamp), which is
where to look next.

Worth its own METHOD line: this had been parked behind "needs the
emulator to reach a menu" for several iterations, and it needed the
emulator only to BOOT -- video init happens in the first seconds. A
blocker that stops one experiment does not stop every experiment in the
same area.
2026-08-29 02:51:32 +00:00
Sylpheed RE agent
93b0a6b20f re(ui): the gamma confound is refuted from source -- canary applies none of its own
I had parked the tone-curve finding behind "this may be the emulator, not
the game: canary applies kernel_display_gamma_type = 2 (BT.709) on
output", with a planned run setting it to 0 and re-fitting. Reading the
source kills both the confound and the experiment.

VdGetCurrentDisplayGamma_entry is a kStub GETTER the guest calls
(xboxkrnl_video.cc). Its own comment: "Used in D3D SetGammaRamp/
SetPWLGamma to adjust the ramp for the display." The cvar is a value
REPORTED TO THE GAME, which then builds its own ramp. Canary's role is
downstream: the guest writes DC_LUT, command_processor.cc reads it into
gamma_ramp_256_entry_table_, and the swap path applies it via
swap_apply_gamma_pipeline_layout with apply_gamma_table.ps /
apply_gamma_pwl.ps compiled in.

So there is no emulator-side BT.709 post-process to subtract, and any
gamma in a captured frame is a ramp the game installed.

What is NOT established, and the reach is stated: that this game installs
a ramp at all, or that the measured 1.34-1.49 is it. The run logs cannot
say -- kernel exports log at Debug and this harness masks Kernel logging
(log_mask = 13, per boot_menu.sh's own comment), so their silence is
guaranteed regardless of what the game did.

The planned experiment was wrong in design: changing the cvar changes
what the GUEST is told and therefore which ramp the GAME builds, so it
could never isolate a stage that does not exist -- and it perturbs the
capture harness, since skip_intro classifies movie-vs-static on an
absolute rmse threshold that a brighter frame biases. The right run
changes nothing about the output: LOG_MASK=12 LOG_LEVEL=3 and look for
the call and the DC_LUT writes.

Also for the port: the ramp depends on the display type the game is told,
and canary hard-codes TV/BT.709 where hardware uses a console setting. So
this is a display profile, not a fixed property of the game.

METHOD: read what a cvar does before building an experiment around it;
and an absence in a log is only evidence if the log would have shown it.
2026-08-29 02:44:30 +00:00
Sylpheed RE agent
3490fba9e3 re(ui): settle 8AX vs ptbase statically -- the game draws the full-res one
I had parked this as "needs a per-draw capture recording texture base
addresses". It did not. 8AX (1280x720) and ptbase (640x360 at 200%) are
the SAME artwork at two resolutions, which is exactly why comparing
either against a capture is inconclusive -- and why comparing their
DIFFERENCE is not. Compute 8AX - upscale(ptbase), the detail only 8AX
has, and ask whether the capture contains it.

Both candidates are first mapped into the capture's tone domain with the
measured gamma; without that the residual is dominated by the tone
difference and the test is blind.

  main menu  corr +0.0475   controls +0.0032 shift, -0.0075 flip   68% of ceiling
  title      corr +0.0634   controls +0.0095 shift, +0.0086 flip   68% of ceiling

Two independent screens, both at 68% of the theoretical ceiling
(sd of the 8AX-only detail over sd of the capture residual), 7-15x their
matched controls. The controls preserve spatial correlation and destroy
only alignment, so they are what "no signal" looks like.

So the recommendation changes: resolve the name and draw 8AX at 1:1.
Upscaling ptbase 2x is wrong, not merely softer. Still do not draw both --
an opaque layer over an identical one costs fill and hides later changes,
and ptbase's element is the one carrying the keyframes, so a consumer
needs its timing with 8AX's pixels.

Also recorded and withdrawn: a cruder pixel-pair test gave 0.00-0.72 for
upscales, 0.98 native and 1.01 for the capture -- apparently decisive.
Additive noise raises both terms of that ratio equally and drives any
value toward 1; fitting a noise term, both "native + noise" and "bilinear
+ noise" reproduce the observed numbers. The conclusion is right, that
test does not establish it, and it is in REFUTED because the number looks
conclusive and is not.

Not shown: whether ptbase is also drawn underneath. 8AX is ~86% opaque
and carries the same art, so it would hide it either way.
2026-08-29 02:40:53 +00:00
Sylpheed RE agent
aaaa08b164 docs: the UI decode's own evidence images were unreachable -- 11 links repaired
The brief's rule is to commit reference data beside the finding so the
port can be built without a disc. Nothing had ever checked that the docs'
cited artifacts actually exist. doc_link_check.py walks every markdown
file under docs/, resolves each relative link, and reports targets that
are missing -- and separately targets that resolve to a ZERO-BYTE file,
which looks fine in any listing.

  links resolving   1038 -> 1049
  missing targets     16 -> 5
  empty targets        0 -> 0

+11 resolving and -11 missing against 11 edits: the counts pair, which is
the confirmation the pass did what it claimed and touched nothing else.

Two of the sixteen were the evidence for the UI layout decode itself.
structures/ui-rat-layout.md is what the port is built on, and its two
figures -- backing "the tutorial PAUSE menu rebuilds pixel-accurately
from its sprites" and "the same method reproduces the main menu" -- were
written as captures/ui-layout/... from a file in structures/, one
directory too shallow. The headline evidence for the decode could not be
opened from its own document.

Eleven links had the wrong relative depth with the target present. Each
was rewritten only where exactly one candidate path resolved, so nothing
was guessed; the first pass left three alone because equivalent spellings
(captures/../captures/x) failed to collapse, and a second pass normalised
them.

Five remain genuinely absent and are left rather than invented: two point
at MEMORY.md outside the repo, one at a header in the separate
xenia-canary-native tree, and two name documents that were never written
(weapon-datasheet-runtime.md, canary-build-verified-env-confound.md).
None is port-relevant. A missing document is a different problem from a
bad path and is not something a link fix should paper over.
2026-08-29 02:34:16 +00:00
Sylpheed RE agent
7373035868 re: the port was still being told SE audio is undecodable -- it is not
A resolve-check on HANDOFF's own rows. Q8 read "SE audio is undecodable
from the disc -- no XACT container exists anywhere". menu-audio-cues.md
retracted exactly that ("### Retracting 'cannot be extracted'") and
locates three cues in Static.slb that decode to PCM: d-pad move 0x1ec0
(4 packets), (B) back 0x0ec0 (2), (A) confirm 0x5d6c0 (6), all mono
48 kHz. The retraction landed in docs/re/ and the page the port reads
kept the superseded text -- the fourth time in this corpus.

Writing the rule down has not worked, so there is a tool now.
handoff_lint.py flags every HANDOFF line making a strong negative claim
that links a doc containing retraction language. First run: found the Q8
row, plus one benign false positive (Q3 links a doc whose retraction is
about a sprite count, not about the tie-break -- checked, and HANDOFF
repeats none of the retracted figures). The lint also caught its own bug
first: it reported existing docs as missing because it joined a guessed
repo root, so it now resolves links relative to the file as markdown does.

Separately, EXTRAS's paint-order risk narrows twice more. Of its 15 tied
pairs only 2 overlap, and of those, ptloop01 x ptloop02 are loop*
animations compose skips by default -- so exactly ONE tie can be drawn:
ptframe3 x ptframe4, overlapping 102x132 px. Against live-extras.png that
contested region correlates +0.9622, better than the whole frame (+0.9440)
and inside the range of regions where order cannot matter (+0.8502 /
+0.9903). Consistent with our order, not proof: correlation cannot see a
swap between locally similar art.

15 -> 2 -> 1 -> consistent is now the whole paint-order risk on the five
screens, and HANDOFF says so.
2026-08-29 02:31:37 +00:00
Sylpheed RE agent
f7f9b555f6 re(ui): the paint-order tie-break is undecodable from the bundle
Q3 was delivered as "decoded: a u16 layer key at +0x0A". The audit last
iteration showed the key does not fully order a screen -- elements
sharing a key are tied, and on the title that tie-break decides two total
occlusions. This searches for what breaks the tie, and closes it as a
negative with reach.

The game paints the five tied ptlogo_back2eff glows in the order
eff1, eff2, eff5, eff3, eff4. Three static structures were searched:

1. The declaration table. Entries 14-18 are byte-identical apart from the
   pivot, which is only half the sprite's own size.

2. The T8aD headers. All five carry identical +0x04 (0x8832) and
   identical +0x08/+0x0A (32899, the key itself), differing only in
   position and tile count. Searched exhaustively -- every offset
   0x00-0x7f, u8/u16/u32, ascending and descending:

     fields sorting to the MEASURED order:              0
     fields sorting to the DECLARATION order (control): 64

   The control is the point: 64 fields can be found that reproduce a
   known ordering, so the scan finds ordering fields when they exist. It
   finds none for the order the game uses.

3. The RATC child order -- a genuinely different permutation on other
   screens -- gives eff1..eff5 here, declaration order again.

All three static orderings give eff1..eff5; the game gives eff1,2,5,3,4.
That agrees with ui-screen-runtime's conclusion from the other direction:
the game builds a reordered child list at load time and paints that.

Q3 now reads honestly: the layer key is decoded and orders 4 of the 5
measured bundles exactly; the tie-break within a key is undecodable, and
a consumer must use a measured order or accept declaration index as an
arbitrary stand-in. The port's exposure remains 2 overlapping tied pairs
on EXTRAS.

METHOD: an exhaustive field search needs a positive control, or "found
nothing" is worthless.
2026-08-29 02:26:19 +00:00
Sylpheed RE agent
ba47bdebe8 re(ui): measure the paint-order hedge -- exact on 4 of 5, and bound the rest
`compose` claimed the derived paint order "reproduces both measured
orders up to ties". That sentence was never measured and was stale by
one: there are three measured orders, not two. examples/paint_order_audit.rs
checks it.

  main menu (entries 5, 8)      derived == measured   0 inverted pairs
  developer splash (11, 14)     derived == measured   0 inverted pairs
  title (entry 4)               DIFFERS               8, all same-key ties

So the claim holds and the exception is entirely ties -- but two of those
ties are total occlusions, not near-misses. The tied family is the five
ptlogo_back2eff glows (key 32899); back2eff5 is 1133x280 and FULLY
CONTAINS back2eff3 (82,824 px^2 = 100% of the smaller) and back2eff4
(152,047 px^2 = 100%). Derived paints it on top of two glows it entirely
covers; the game paints it underneath. A tie-break by declaration index
can therefore be wrong by a whole layer. The title itself is unaffected --
it has a measured order.

The port's actual exposure, per screen: title, main menu and developer
splash all use MEASURED orders; the publisher splash is derived but has
ZERO ties, so it is fully determined; EXTRAS is derived with 15 tied
pairs of which only 2 OVERLAP. Two element pairs on one screen is the
whole risk, and that is what HANDOFF now says -- not the raw 15, which
would have overstated it 7x.

Reach stated: this compares the derived order against orders measured
from the game, not an independent derivation, so where no measured order
exists only the tie exposure can be checked. Overlap uses pivot*2 as the
element size at its resting placement.

Stale comment in compose corrected. METHOD: a hedge in a code comment is
an unmeasured claim; and count the cases that can bite, not the ones that
match the pattern.
2026-08-29 02:22:01 +00:00
Sylpheed RE agent
c3cf3c2e81 re: a title negative that survives its own cross-check
Three earlier "the title never appears" claims came from instruments
later found broken -- a stale pixel oracle, a 41 s sampling interval, a
freezing stream. This one carries its own evidence.

title_probe_xchecked.py restarts its capture stream every 30 s AND prints
its reading beside an independent `import` grab every 60 s:

  1851 frames in 560.2 s = 3.30 fps
  cross-checks 9, disagreements 1
  max glyph 0

  t= 62s stream   6.05 | import   0.07  disagree (a fade, logos mid-transition)
  t=123s stream   7.40 | import   7.49  agree
  t=183s stream   8.18 | import   8.29  agree
  t=243s stream   0.23 | import   0.10  agree
  t=311s stream  89.68 | import  89.51  agree
  t=371s stream  80.97 | import  81.58  agree
  t=426s stream 117.43 | import 117.72  agree
  t=487s stream  77.71 | import  76.25  agree
  t=546s stream  70.43 | import  70.55  agree

Eight of nine agree within 2%, fps held at 3.30 with no collapse to 1.60,
and the surface moved through dark and bright phases. So the frames were
live: over 560 continuous seconds from launch, sampled 3.3 times a
second, the interactive title's green (A) plate never appears while the
game renders throughout. The final frame correlates 0.0145 / -0.0047 /
0.0102 with our title / main menu / EXTRAS renders -- attract-movie
content, not a UI screen.

Why remains unknown. live-title-press-a.png with its 753 glyph pixels
proves the title was reachable from this container on 2026-08-28, and
clearing the shader cache fixed the black surface but not this.

The two emulator-side questions (gamma control, 8AX vs ptbase) are
therefore blocked on a characterised failure rather than a suspicion.
Neither blocks the five menu screens, so I am returning to static work;
the probe is committed for whoever picks it up.

METHOD: a probe that cross-checks itself turns "no result" into a result.
2026-08-29 02:14:21 +00:00
Sylpheed RE agent
46006406a4 re: the fast probe stalls -- its own dense negatives are withdrawn
Cross-checked the instrument built last iteration against an independent
grabber while both watched the same screen, and it fails.

A single long-lived ffmpeg x11grab stream degrades and then freezes:

  862 frames in 540.1 s = 1.60 fps        (it starts at 3.98)
  t=450/480/510/540 s: surface mean 5.21, identical every time

At that same moment `import` read surface mean 125.65, and a freshly
started ffmpeg stream read 122.43 -- agreeing with import to 3%. So the
acquisition was broken, not the analysis: the stream replayed a stale
frame while the screen was 24x brighter.

That withdraws last iteration's headline. "2391 frames over 600 s from
t=0, max glyph 0" cannot distinguish "the title never appeared" from "the
stream froze early and repeated one frame 2391 times". Its 3.98 fps was
measured over the first 20 s, before the degradation. Sample count is not
coverage unless the samples are known independent.

Fixed: the stream is now torn down and restarted every 30 s. Startup is
~0.3 s, cheap against the title's window, and it guarantees live frames.

Separately, the cache hypothesis was tested and is SUPPORTED. cache,
cache0, cache1, cache_host moved aside (to /tmp/xenia-cache-aside, not
deleted) and the surface renders again: import reads mean 54.8 and 68.6
with 100% non-black warm content, against 0.07 and 0.08% non-black in the
black run; 773 of 862 probe frames had >2% non-black. One run each side
and many kill -9s before the black one, so it is supported, not proven --
the old caches are kept for reproduction.

Still no title, but that number now comes from a stalling probe and
establishes nothing either way.

METHOD: validating a probe on static images tests its analysis, not its
acquisition -- cross-check against an independent grabber during a run.
2026-08-29 02:01:47 +00:00
Sylpheed RE agent
a3946e5005 re: the game surface is rendering BLACK -- check that before explaining absences
The named experiment was to attach the fast probe at t=0 so the boot
title could not be missed. Done, default config, English:

  2391 frames in 600.4 s = 3.98 fps; max glyph 0; hits 0

Ten minutes sampled four times a second FROM LAUNCH, no green-(A) glyph.
So "the plate only shows in an early boot window I keep missing" is mine
and refuted -- the third explanation refuted in three iterations.

Then the check that should have come first. Splitting the raw root grab
into bands:

  y   0- 44 (GTK menu bar)   100.00% non-black   mean 210.50
  y  45-719 (game surface)     0.08% non-black   mean   0.07

The game is rendering black, reproducibly across back-to-back samples,
while the guest is alive and polling input (XamInputGetKeystrokeEx past
1201 calls) and MEM-WATCH keeps reporting. The crop and every pixel
oracle were correct; there was nothing on the surface to detect.

What this does NOT do is retroactively explain the earlier failures, and
claiming so would be the fourth over-reach in a row. Those runs had
content: run 2 sampled mean 33.1, run 3's classifier measured real
frame-to-frame rmse, the gamma_type=0 run measured mean 122.8. The
failure mode CHANGED over the session; black is the newest and worst.

Hypothesis for the regression, untested: canary's shader/pipeline cache
is 47 MB and was last written 23:49 on Aug 28, during the failed runs,
and this session has kill -9'd the emulator repeatedly. The test is to
move cache* aside and boot again -- one line and one run, not done.

Two METHOD lines: ask whether the screen is drawing anything before
explaining why a feature of it is missing; and a newly found fault does
not retroactively explain older failures.
2026-08-29 01:49:42 +00:00
Sylpheed RE agent
e9924ff9e8 re: build the fast probe -- and it refutes the diagnosis that motivated it
Last iteration I blamed four failed runs on the probe sampling every
~41 s, slower than the title screen lasts, and withdrew three earlier
conclusions on that basis. Building the fix tested the claim and killed
it.

The speedup is real and control-verified. One long-lived ffmpeg x11grab
stream, raw RGB, glyph counted in numpy -- no per-sample process startup,
no PNG encode, no convert -crop:

  wrapper `screenshot`            3.98 s per sample (emulator running)
  import -window root -> PPM      1.20 s
  long-lived x11grab stream       0.29 s          13.7x

The counter is byte-identical to is_title.py: 753 on the committed title
capture, 327 on the main menu.

Pointed at a running game it says the opposite of what I expected:

  332 frames in  85.3 s = 3.89 fps; max glyph 0
  1674 frames in 420.0 s = 3.99 fps; max glyph 0

1674 consecutive samples over seven unbroken minutes, four per second,
zero green-(A) pixels. Sampling rate was a real defect that happened not
to be the cause.

So "neither locale reaches the interactive title without a pad press" --
withdrawn last iteration for want of evidence -- is reinstated, now as a
dense measurement, with its reach stated: a MID-RUN window only, silent
about the boot title.

Leading hypothesis, unconfirmed: the PRESS (A) plate appears only in the
boot title window and the attract loop's title carries none, which is
exactly what title_states_capture.sh was written to test. The experiment
is to start the fast probe from t=0 rather than attach to a run already
in progress.

METHOD: fixing the instrument is how you test the explanation that blamed
it -- a plausible mechanism is a hypothesis, and the fix is its
experiment, not its proof.
2026-08-29 01:34:40 +00:00
Sylpheed RE agent
cc4e5e04a7 re: the boot harness was blinking slower than the title -- diagnosed
Four consecutive runs failed to reach the interactive title, across two
locales, two launch paths and two display-gamma settings. I attributed it
in turn to a stale oracle, to the locale, and to the attract loop. It was
none of those.

  one `screenshot` call, emulator running:  10.8 s
  one `screenshot` call, emulator killed:    0.117 s

92x, measured at a 1-minute load average of 1.80 -- so it is contention
with the emulator through the X server, not background load.
skip_intro.sh takes two grabs per iteration plus a numpy import, giving a
median sampling interval of 41 s in the last run (38/82/41/20/30/30/35/
47/46/45/44/43/21/19). The title lasts "a few seconds" before the attract
loop reclaims it -- wait_title.sh's own header says so. The harness was
sampling slower than the event it was waiting for. That also explains why
runs at 16:43-18:05 the same day succeeded.

Withdrawn as CAUSES, though the observations stand: "the JP run never
reaches the interactive title", "neither locale reaches it without a pad
press", and "the game sat in the attract loop for 604 s". The English
control did control for locale -- it just shared the same defect.

Also recorded: I set kernel_display_gamma_type = 0 for the gamma control
run, which brightens the frame (mid-attract mean 122.8 vs 52.5/82.8 at
type 2) -- and skip_intro classifies movie-vs-static on an ABSOLUTE rmse
threshold, so the gamma change biased the very classifier the run
depended on. Changing a display setting and a capture behaviour in one
run confounds both. Config restored to type 2.

The fix is not applied: make the probe cheap enough to outpace the title
window (small region, no convert round trip, one long-lived process).
Every remaining emulator-side question is waiting on that.
2026-08-29 01:18:05 +00:00
Sylpheed RE agent
ee73de50be re(ui): put a number on the render-vs-capture tone difference
Closes an observation I left dangling last iteration ("the capture is ~4x
darker than the render") and puts a figure on the  INDEX's texture row
already carried: exact gamma/sRGB fidelity untested because a hue
comparison cannot see it.

Geometry first: cross-correlating the main-menu capture against our
render over +/-6 px puts the best alignment at exactly dy=0 dx=0,
correlation 0.9466. Only the tone differs.

Two methods failed before one worked, and both failures are recorded.
Three dark patches gave "4x darker" -- the whole-frame best linear scale
is 0.914, so three patches from one region are not a transfer curve. A
pixel-wise fit over 854,685 pixels then produced a NON-MONOTONIC transfer
(render 96-127 mapping brighter than render 128-159) with mean abs error
10-14 for every candidate model. That is edge misalignment, not a tone
curve: at correlation 0.947 a bright pixel routinely lands on a dark one.

Flat patches fix it -- 16x16 blocks with std < 8 in BOTH images, a
threshold chosen from the counts (0/83/404/1055/1788 at std<3/5/8/12/20):

  main menu  404 patches  gamma 1.491  err 0.28   (best linear 0.276, 0.34)
  EXTRAS     382 patches  gamma 1.493  err 0.22   (best linear 0.273, 0.28)
  title      506 patches  gamma 1.338  err 1.08   (best linear 0.842, 9.02)

Reach, stated because it is narrow: those patches span only render values
~0-60, where gamma and a plain scale are nearly indistinguishable -- the
two menus decide nothing (0.28 vs 0.34, 0.22 vs 0.28) and only the title
separates them. Nothing constrains midtones or highlights.

The held-out control FAILED TO DISCRIMINATE and is reported as such: the
splash's 2918 flat patches are pure black (render 0-4), so every model
scores ~0.00. That is a test with no power, not corroboration.

Confound left open: this compares our composite to what canary DISPLAYS,
and canary applies kernel_display_gamma_type = 2 (BT.709). The exponent
may be its output stage. The discriminating run -- set it to 0, recapture,
refit -- needs one emulator session reaching the main menu and was not
done.

Classified measured, not decoded; HANDOFF says plainly that a port
applying it is authoring.
2026-08-29 01:05:25 +00:00
Sylpheed RE agent
2b4ec20f00 re(ui): a full-screen element is dropped on three port screens -- do not "fix" it
Audited what `screen render` silently omits on the port's five screens,
since an element the game draws but we skip is the one defect class the
port agent has actually hit. Everything is accounted for -- kind & 0x4
ghost instances, .prm primitives, loop* animations -- except pteff04.t32
on the title and pteff05.t32 on both menus. Those are kind 0x0, one
keyframe, rest a=255, pivot (640,360): full-screen and opaque.

Cause: the element declares pteff05.t32, but the T8aD behind its `opt `
link is registered under the name 8AX, so build.sprites.get() misses and
compose hits a silent continue. Bytes at 0x0e2035 of GP_TITLE entry 5:

  opt  ... 70 74 65 66 66 30 35 2e 74 33 32 00  38 41 58  54 38 61 44
           p  t  e  f  f  0  5  .  t  3  2  \0  8  A  X   T  8  a  D

8AX is 1280x720 and present in all six title-family bundles; it is a
sprite in builds 4/5/6 and never an element.

It does not currently show, and that is the useful half. ptbase.t32 is
640x360 drawn at 200% and carries THE SAME ARTWORK: its 2x upscale
differs from 8AX by mean abs diff 2.05 (max 80), and our rendered
background is pixel-identical to 8AX in every patch sampled. So resolving
the name and drawing it in addition would double-draw an opaque
full-screen layer -- invisible as a doubling, which is worse than a
visible bug. Written into HANDOFF as a do-not-do.

The free win, offered and not taken: use 8AX at 1:1 and drop ptbase
instead of upscaling a half-res copy. That is a rendering choice and
ptbase's element carries the keyframes, so it is the port's call.

Not established: which of the two the game actually draws. Both carry the
same art, so pixels cannot separate them; it needs a per-draw capture
recording texture base addresses, since the two differ in size.
2026-08-29 01:00:36 +00:00
Sylpheed RE agent
7d07eb945f re: the title capture is not reachable here either; stopping this line
Run 3 used the proven path rather than my own probe: launched exactly as
boot_menu.sh does (DISPLAY=:98, --apu=sdl, /dev/shm/xenia_* cleared, the
existing profile signed in) and drove skip_intro.sh, the detector that is
documented to work and that is stricter than mine (glyph >= 800 AND a
static frame).

It classified every one of 20 samples over 604 s as "movie", waited them
all out, and timed out. A direct check at 604 s says it was right: zero
green-glyph pixels, screen_id = other, warm mean (69,53,40), correlation
0.09 with build 7. The game really was playing attract movies for ten
minutes.

Three runs, two locales, two launch paths, ~35 minutes of emulator time,
no interactive title. Either the attract loop is far longer than the
600-780 s windows tried, or something regressed since
live-title-press-a.png was captured -- that one came from a pad-driven
boot (88b3ce9, "booting to the main menu and walking it").

Also settled a discrepancy I had flagged without checking: there is no
contradiction with title_states_capture.sh. It waits on screen_id.py, a
LOOSER oracle than is_title.py, and was written precisely to test whether
the interactive title draws ptbtn00 and the attract one does not. It
never claimed to reach the interactive title.

Stopping this line. The capture blocks none of the five menu screens and
has now cost five iterations. The locale mechanism, the fixed oracle and
the launch recipe are all committed, so what remains is patience with the
attract loop, not tooling. METHOD line on knowing when to stop paying for
a non-blocking answer.

Emulator stopped, lock cleared, locale English.
2026-08-29 00:49:49 +00:00
Sylpheed RE agent
8783adda14 re: the English control removes the locale from the title-capture problem
Two iterations framed the JP title capture as possibly locale-specific.
It is not. Same flags, same oracle, English locale: 75 samples over 734 s,
every one glyph = 0. The English boot does not present the interactive
title either. Canary was alive throughout, polling
XamInputGetKeystrokeEx (1801 calls); the frame at 734 s has content but
correlates 0.15 with build 4's render and 0.05 with the main menu -- an
attract-movie frame, not title art.

So neither locale reaches the interactive title in ~12 minutes without a
pad press. What is actually untested is the pad: pad.py and
nav_to_flight.sh exist, and title_states_capture.sh reaches title states
with a flag set this probe did not replicate (--log_ui_draws
--ui_draw_capture_frames, plus xdotool window focus). It is a capture
problem, not a locale one. MISSION.md updated.

Also resolved a caveat I had given the port agent without checking it:
"rotation is decoded but not rendered" does NOT affect their five screens
at rest. Title, main menu, EXTRAS and both splash halves have ZERO
top-level elements with a non-zero rotation. The only rotations on any of
them are the title's two nested ptloop records (r = 30 and -45), and at
rest those sit at x = 1521 and x = -839 -- a 399-wide sprite entirely off
both edges of a 1280 screen. The caveat now applies only to animating the
title build-in, where the sweeps cross the screen rotated.

Two METHOD lines: run the control before theorising about the difference,
and log every sample so a failure is a measurement rather than a silence.

Emulator stopped, lock cleared, locale English.
2026-08-29 00:35:49 +00:00
Sylpheed RE agent
f10edf1e79 re: fix wait_title.sh's stale oracle; the JP title still is not reached
Last iteration's "never reached the title in 787 s" was a broken tool
reporting on the world. wait_title.sh was still sampling the single pixel
(625,618) that is_title.py had already been written to replace -- its
docstring says why: a 1280x720 coordinate sampled against the 1279x675
game surface, so it always reads the copyright line. The replacement sat
in the same directory. wait_title.sh now delegates to it.

is_title.py passes its own controls before being trusted here: 753
green-glyph pixels on the committed English title capture, 327 on the
main menu, threshold 400.

Re-ran with the working oracle and the profile flag the English captures
use. The game STILL did not present the interactive title -- but that is
now a measurement rather than an artefact: not one frame showed a single
green-(A) glyph pixel, and content correlation against either build-7
render never exceeded 0.22. Canary was alive and polling
XamInputGetKeystrokeEx (601 calls), sitting in the attract movie.

So the open question narrowed again, and is written into MISSION.md:
whether the attract loop returns to the INTERACTIVE title without a pad
press. title_states_capture.sh claims it does on the English boot with no
pad input; if that holds, the difference is the locale.

Nothing decided about the keyframe-time association or the rest() rule.
Emulator stopped, lock cleared, locale restored to English.
2026-08-29 00:21:44 +00:00
Sylpheed RE agent
0f1c0f4e14 re: the JP-locale capture is not blocked -- I stopped one grep too early
Last iteration I wrote into MISSION.md that a Japanese-locale capture is
impossible here, because user_language is DECLARE_int32 at four call
sites with no DEFINE and no entry in xenia-canary.config.toml. That is
true, and it was not the question. The language is PERSISTED:
kernel_state.cc builds XConfig over <storage_root>/xconfig.settings,
SetDefaults() only supplies a value when the file has none, and the file
is writable. Checking where a setting is stored rather than where it is
configured turned "blocked, needs a human decision" into a two-line edit.
Withdrawn from MISSION.md; METHOD and REFUTED lines added.

The field is located from struct landmarks rather than a hard-coded
offset, and the check re-runs on every invocation so it fails loudly if
the layout moves:

  music_volume 0.7f at User+449  -> BE float at 2727 -> User base 0x8e6
  language          at User+44   -> reads 1 (kEnglish) at 0x912
  country           at User+64   -> reads 103 (US) at 0x926

XLanguage::kJapanese = 2 (xbox.h:307). set_console_language.py wraps it
with a backup and a --restore.

The capture itself is still NOT taken, for a smaller reason than I
claimed. A run with the locale set to Japanese booted fine but never
reached the title in 787 s: wait_title.sh's green-(A) oracle never fired
and burst-sampling found no frame correlating above 0.18 with either
build-7 render -- the run sat in the attract loop. So it needs a longer
or pad-driven run, not a rebuilt emulator. Emulator stopped, lock
cleared, locale restored to English.

Nothing is decided about the keyframe-time association or the rest()
rule; this only changes what standing between us and deciding them.
2026-08-29 00:05:58 +00:00
Sylpheed RE agent
6d246c7a97 re(ui): finish the top-level rotation census; record the JP-capture blocker
Two threads had converged on needing one capture this container cannot
take, so this iteration records that and finishes something reachable.

BLOCKED, written into MISSION.md rather than worked around: Q1's keyframe
time association and the rest() rule for plateau-less elements both now
hinge on a running capture of GP_TITLE build 7, the Japanese title. The
console language is not settable here -- user_language appears only as
DECLARE_int32 at four call sites with no DEFINE anywhere in the tree, and
it is absent from the registered cvars in xenia-canary.config.toml. There
is no flag to pass, and guessing one is specifically unsafe: run-canary's
own header records that xenia calls ShowSimpleMessageBox from
ParseLaunchArguments before logging starts, so a bad flag blocks forever
with an empty log. Rebuilding canary to add the cvar would be improvising
around the blocker; it needs a human decision. Neither question blocks
the five menu screens.

FINISHED: the disc-wide top-level rotation count, left running four
iterations ago as a shell loop over `screen info --geometry` that never
completed (it decodes every texture per build). Walking the placement
region directly takes seconds.

  top-level elements with a keyframe group   15 493
  carrying a non-zero rotation                2 152  (13.89 %)

Both controls pass: GP_TITLE build 4 reports 0 (its rotations are the
nested ptloop records) and GP_DIALOG build 0 reports the expected two.

The control earned its place -- the first version indexed the pak with a
`screen list` BUILD number and got 0 for a screen that has two, because
GP_DIALOG build 0 is entry 2. GP_TITLE maps 1:1, which is how the
assumption survived. METHOD line added.

Two free corroborations of the rotation decode. The rotated population is
dominated by tactical-map ship icons -- pbb_destroyer 444, pbr_destroyer
402, pbr_fighter 276 -- i.e. markers rotated to heading, the single
largest use of the field on the disc. And GP_TITLE entry 7's Japanese
wordmark pieces settle from ALTERNATING tilts:

  ptlogo3a  r = 0, -14, -4, -1, 0, ...
  ptlogo3b  r = 0, +14, +4, +1, 0, ...
  ptlogo3c  r = 0, -14, -4, -1, 0, ...

Same magnitudes, opposite signs, all decaying to upright. A misread field
does not produce that.
2026-08-28 23:46:54 +00:00
Sylpheed RE agent
b973d8cde3 re(ui): scale 0 means collapsed, not "unset" -- stop drawing it full size
blit() and fill_quad() both opened with

    let sx_pct = if kf.scale_x == 0 { 100 } else { kf.scale_x };

so an element whose pose is collapsed to nothing rendered at FULL SIZE.
I first described this as "a 1-pixel sliver", reading the .max(1) in the
size arithmetic two lines below -- the guard above it meant .max(1) never
saw a zero. Read the whole function, not the lines you went looking for.

Control run before the change: if 0 meant "unset", something on the disc
would use it throughout. Nothing does.

  elements with a keyframe group                    15 493
  at least one keyframe at scale 0                   2 166
  EVERY keyframe at scale 0                              0
  zero is a transient the element grows out of       1 762

ptlogo_eff3.t32 runs 0% -> 200%. An "unset" marker no element ever uses
throughout is not a marker.

Both functions now return without drawing when either scale is 0.

Reach: 24 of 24 renders byte-identical across GP_TITLE (all 16 builds),
GP_PAUSE_MENU and GP_OPTIONS -- additive on every screen the port needs.
195 elements have a guessed rest pose at scale 0 and 126 of those have a
non-zero alpha, so the old code painted them; all 126 are in
GP_READY_ROOM.pak, which S1 already declared a no-go. They are
tactical-map sprites at sx=0/sy=100, mid-horizontal-wipe, that the
coercion drew full-width.

So: a real correctness fix with no visible effect on the menu port.
Suite green, 122 passed / 0 failed across 3 suites at commit time.
2026-08-28 23:38:37 +00:00
Sylpheed RE agent
c2c69b02be re(ui): size the rest() guess disc-wide, and refute my own proposed fix
Follows up the defect found last iteration: rest()'s dwell fallback is
guessing whenever it runs. Two things were open -- how big it is, and
whether "rest = the last keyframe" is the fix. Both are now answered, and
the second is answered no.

plateau_census.py walks the placement regions directly instead of going
through `screen info --geometry`, which decodes every texture and cannot
do a disc-wide pass in reasonable time. Its control reproduces GP_TITLE
build 7's three fallback elements and names ptlogo_eff3.t32 among them
before counting anything.

  elements with a keyframe group      15 493
  no plateau -> rest pose is guessed   3 807  (24.57 %)
    ... current rule returns invisible 1 711  (44.9 %)
    ... current rule returns scale=0     195  ( 5.1 %)
  the two candidate rules agree        1 911  (50.2 %)

195 elements get a rest pose with scale 0%, which is not a pose. And
disc-wide the choice of rule is not cosmetic: the candidates agree half
the time.

But the port's exposure is one element. Across main menu, EXTRAS, title
and the developer splash, 14 elements are plateau-less and the two rules
agree on 13. The single disagreement is palogo_anima_eff.t32.

And "last keyframe" loses there, on a control that needed no new capture:
the splash carries three sibling glows with identical structure and
identical times --

  palogo_gamearts_eff  15:a=0 30:a=255 45:a=255 -:a=0  plateau -> visible
  palogo_seta_eff      15:a=0 30:a=255 45:a=255 -:a=0  plateau -> visible
  palogo_anima_eff     15:a=0 30:a=255 45:a=212 -:a=0  no plateau

-- differing in one byte. "Last keyframe" makes anima alone invisible
while its two siblings stay lit. The capture agrees weakly: box-mean
ratios capture/render are gamearts 0.717, seta 0.723, anima 0.772, and a
glow we drew that the game does not would put anima below its siblings,
not above.

So the defect is measured and the fix is still undecided. Nothing in the
decoder changed.
2026-08-28 23:23:55 +00:00
Sylpheed RE agent
b947fa8e8e re(ui): withdraw the render evidence -- rest()'s dwell fallback is unsound
Last iteration I reported a 13.1% render change in GP_TITLE build 7 as
evidence AGAINST the keyframe-time shift, arguing that language twins
should match in brightness. Withdrawn. Localising the diff to an element
shows it is not about the time association at all.

The element is ptlogo_eff3.t32, a transient bloom:

   46: (98,42)  100%,100% a=0
   61: (108,72)   0%,0%   a=0
  103: (108,72) 200%,200% a=255 r=80
    -: (108,72)   0%,0%   a=0   r=150

No two adjacent poses are equal, so there is no plateau, and rest() falls
through to its longest-dwell fallback. The longest gap is 61->103, during
which the sprite grows from nothing to 200% at full alpha and then
collapses. The rule returns whichever end of that movement the indexing
lands on: the invisible frame as decoded, the 200% peak shifted. An
896x389 sprite at 200% is larger than the screen, which accounts for the
entire 13.1% and the entire 4.9-unit luminance gap. I was comparing a
heuristic against itself.

The defect generalises, and structurally. A dwell gap is time spent
interpolating FROM pose k TO pose k+1; neither pose is held during it
unless the two are equal -- which is a plateau, and the plateau path has
already returned by then. So the fallback is guaranteed to be guessing
every time it is reached, under any reading of the times. Every element
with no two adjacent identical poses has a guessed rest pose, in our
renderer and in anything built from it.

Consequences: the case FOR the shift (26x on the hold:fade-out ratio) is
no longer opposed by render evidence -- 10 of 11 builds byte-identical,
the 11th differing only through this heuristic. It is still not adopted,
now because it flips this element to the visibly wrong answer, so the
shift and a decision about plateau-less elements must land together, and
neither half has a capture to verify against.

Default-mode suite green (122 passed, 0 failed across 3 suites so far).
2026-08-28 23:10:41 +00:00
Sylpheed RE agent
6f23f4d113 re(ui): the keyframe-time shift is favoured 26x by timing, rejected by a render
Follow-up on last iteration's unadopted candidate (+36 holds the NEXT
pose's time, not its own). Two new results, pointing opposite ways, and
both are reported.

FOR, and calibration-free: the observed full-alpha hold : fade-out ratio
on palogo_gamearts is 83 : 13 frames = 6.38. The shifted reading predicts
8.00. The current reading predicts 0.25 -- off by 26x. With the glow's 2
units/frame fixed and nothing else free, the current reading says the
logo holds full alpha for 2.0 frames; the capture holds it for 83. This
is no longer the shape argument the candidate rested on.

Also for: rest()'s plain max-dwell fallback picks a=0 -- a transparent
pose, for a publisher logo -- under the current reading, and the visible
a=255 hold under the shift. Only the rest_plateau special case rescues
the render today, and that is the case the port agent reported a bug in.

AGAINST: rendering every build of six UI paks under both readings, 10 of
11 compared are byte-identical and one changes -- GP_TITLE build 7, the
Japanese twin of build 4, by 13.1% of pixels. Build 4, the one verified
against a live capture, is unchanged either way, so the single build the
shift moves is the one with no capture to adjudicate it. The proxy goes
against the shift: language twins are the same artwork, and build 7 reads
70.94 mean luminance as decoded against build 4's 71.41, but 76.32
shifted. Correlation does not separate them (0.6206 vs 0.6201).

These constrain different things -- timing versus pose selection -- and
rest() is a heuristic layered on the times, so moving the times moves its
tie-breaks. Adopting the shift means revisiting that heuristic in the
same change, with no build-7 capture to verify against.

Default UNCHANGED. Experiment reachable via SYLPHEED_KF_TIME_SHIFT=1.
Shifted-mode suite: 122 passed, 0 failed across 3 suites at commit time.
2026-08-28 23:03:55 +00:00
Sylpheed RE agent
531840b417 re(ui): Q1 -- the interpolation law holds, the group timeline does not
Q1's gate asks whether the ramp is linear. It is, and that result stands:
it rests on the splash's _eff glows, which reproduce exactly. This adds
the part that does not.

The test is a calibration, not a fit. Fix the clock on
palogo_gamearts_eff -- declared 15-unit fade-in 0@15 -> 255@30 against
captured alphas 34,68,102,136,170,204,238, a constant step of 34, giving
t = 2f - 171 -- then check that against the glow's own next landmark: its
declared hold ends t=45, predicted frame 108.0, observed last full-alpha
frame 107. Then apply it to palogo_gamearts in the same bundle and the
same frames, with no free parameter left:

  declared a=232 at t=206 -> frame 188.5, observed alpha 255
  declared a= 32 at t=210 -> frame 190.5, observed alpha 255

The logo is still at full alpha nine frames after it should read 32; its
fade-out runs ~17 frames late; its declared 80-frame fade-in is never
drawn. Not culling -- the same element is submitted down to a=7 on the
way out. Calibration-free version: the declared fade-out spends 12 of 16
units dropping 23/255 of the alpha, and the capture has no such plateau.

Candidate, offered and NOT adopted: if +36 held the NEXT keyframe's time,
the fade-out shape fits (RMS 4.05 vs 12.13, two elements) and the
decoder's "last block's time is unreadable" special case disappears --
the last block would simply have no successor. Rejected for now because
it explains neither the missing fade-in nor the lateness, and because the
_eff elements cannot discriminate between the readings at all (with four
blocks the shift only relabels the phases). Decoder unchanged.

Also withdrawn, mine, within the iteration: "the _eff glows hold a
constant alpha 33". They ramp 34 -> 255 in steps of 34. I printed the
series minimum and read it as its range, with a "14 distinct colours"
column sitting next to it saying otherwise.
2026-08-28 22:56:17 +00:00
Sylpheed RE agent
f11fde51e5 re(ui): rotation is not nested-only, and the pivot-anchored scale is measured
Two corrections and one new confirmation, all from finishing the sweep I
left running last iteration.

Refuted, by my own sweep, within the hour: "rotation appears to live only
in nested .rat leaf records". That held for GP_TITLE, GP_BUNK and
GP_CHALLENGE -- the three archives the sweep had reached when I wrote it
-- and fails on GP_DIALOG and GP_DEBRIEFING_PILOTLOG, which rotate
top-level elements. Scoping the claim made it cheap to withdraw, but the
sentence should have waited for the sweep.

Those top-level cases are the best evidence on the disc, and they show up
in `screen info --geometry` without a hex dump. GP_DIALOG build 0:
pceff03/pceff04 ramp r = 90 -> 30 -> 10 -> 3 -> 0 while alpha ramps
0 -> 255 and they slide into place -- a swing-in settling upright. Build
6: pzeff02 ramps 43 -> 61 -> 75 -> 90 while scaling 112% -> 200% and
fading to 0 -- a spin-out. Rows committed as reference data.

New: the pivot-anchored scale term in blit, kf.x - pivot*(scale-100)/100,
was implemented and reasoned about but never measured, because every
element previously examined sits at 100% scale where the term is exactly
zero. The ptloop pair scale 600% and 800%, where it is worth 450 and 630
px. Formula predicts centre y = 360.0 for both; capture measures 359.1
and 360.0. Top-left anchoring predicts 810 and 990. Horizontally it makes
t-from-position agree with t-from-alpha to 0.33 / 0.65 units against ~8
without it.

That agreement does NOT prove linear interpolation -- both fields were
inverted through the same linear map, so a shared easing curve cancels.
It shows position and alpha ride one shared parameter. Recorded as such.

The disc-wide sweep is still running; the count is incomplete, the
existence is settled.
2026-08-28 22:46:44 +00:00
Sylpheed RE agent
67fa1a1b0b re(ui): decode keyframe +12 as screen-plane rotation in degrees
The rotated quads on the title screen come from the keyframe block after
all. The earlier negative -- "every GP_TITLE build 4 element has all three
angle words at zero" -- read the right bytes over too small a region: it
walked the top-level declaration table, and the rotated elements are the
nested leaf records ptloop01.rat / ptloop02.rat.

Confirmed against the framebuffer rather than against our own renderer.
The two records declare +12 = 30 and -45; the GPU capture submits their
quads at +30.26 and -45.28 degrees -- magnitude and sign, two different
values. Corroborated by shape in GP_BUNK 117ca14f, where +12 ramps
0 -> 360 with position, scale and alpha constant: a spin in place.

Identifying which draw it was needed edge lengths, not bounding boxes:
400x1076 and 400x1444 against pteff03/pteff03a 399x180 at the elements'
two different declared scales, 600% (1080) and 800% (1440). The same test
names three known-positives in the capture (ptlogo1, ptcopyright,
ptbtn00), so it passes its own control.

Keyframe gains rotation_deg plus unknown_4/unknown_8, carried rather than
dropped. NOT rendered -- ui_layout::blit is axis-aligned only, so the
reference renderer and the port will both draw these upright until a
rotating blit exists.

The census tool ships with the trap that broke its first version: nested
RATC blobs are not 4-byte aligned, so an aligned scan found 0/3 of its
own control blocks and missed 16 341 blocks. Disc-wide +12 is non-zero in
14.50 % of 83 862 blocks.

sylpheed-formats tests, SYLPHEED_DISC set: 131 passed, 0 failed across the
6 suites finished at commit time; the run had not yet completed.
2026-08-28 22:42:00 +00:00
Sylpheed RE agent
130ad8d1b3 re: the skewed draw is not the swoosh -- refuting the identification five
iterations of work were built on

I flagged last iteration that "draw 2 is the swoosh" was an inference I
had never checked. Checking it took one line, and it is wrong.

Converting the quads from NDC to screen space: quad A spans y -209 to
925, quad B spans y -292 to 1012. Both cover the full screen height and
run well off it. The swoosh, ptlogo_back2, rests at (71,126) and is a
234-pixel band at y 126 to 360. Draw 2 is not it.

A candidate offered as one: the two ptloop sweeps. pteff03a is 399x180
with element scale (100,800), so 399x1440, and pteff03 carries (100,600)
-- two long thin sprites, two quads, and the drawn vertex alphas 0xC3 and
0xB6 sit inside the 0x80 to 0xff ramp those records declare. Not
confirmed; no texture or position match was made.

If that is right it matters, and I have written it conditionally rather
than acting on it: our renderer parks those sweeps at their final
keyframe, both off screen, and draws nothing, while the game draws them
across the screen. That would mean the capture caught them mid-sweep
inside their t=150 to 600 window, and that the "groups hold" reading
needs re-examining for those two elements.

What this cost is worth stating plainly. Five iterations -- the additive
blend test, the pivot analysis, the vertex-colour capture -- were built
on an identification made by elimination on one screen and never checked
against the draw's own coordinates. The eliminations themselves stand,
because each was measured against the capture rather than against the
identification. The chain that pointed them at ptlogo_back2 did not.
METHOD takes the general form.
2026-08-28 22:21:42 +00:00
Sylpheed RE agent
5115b4894a re: three angle fields found in the keyframe -- and they are not the
title's rotation

Looking for where the rotated quads come from, the obvious candidates
were the three keyframe words at +4, +8 and +12 that ui_layout.rs
documents as zero.

They are not zero. Across 72287 keyframe blocks disc-wide they are
non-zero in 4.81, 4.56 and 15.82 percent of blocks, and read as signed
values clustering on 180, -180, 90, -90, 120 and 22 -- degrees. Three of
them, so plausibly rotation about three axes. I have marked that amber
because it is the shape of the numbers and nothing more; no observed
rotation has been tied to a value. The doc comment is corrected
regardless: "0 on every frame seen" was an artefact of the sample.

And they do not explain the screen I was chasing. Every element of
GP_TITLE build 4 has all three at zero, element by element, while the
game demonstrably submits rotated parallelograms there. So the title's
rotation comes from outside the keyframe data and is still unidentified.

One correction to my own last write-up, flagged rather than left: I
stated that the skewed draw IS the swoosh. It is the only skewed geometry
in the capture and the swoosh is the only diagonal element on the screen,
so the inference is reasonable -- but I never confirmed it by matching
the draw's texture or screen position to that element, and I should have
said so the first time.
2026-08-28 22:17:14 +00:00
Sylpheed RE agent
0ecb20bdbc re: the swoosh is solved -- the game draws it as rotated quads and our
blit cannot

Ran the route I corrected last iteration, and it answered the question I
had classified as undecodable.

First the ninth candidate died: every vertex colour in the whole capture
is <alpha>FFFFFF, white RGB with only alpha varying. The game passes no
colour, so that hypothesis is refuted from the running game rather than
from the file.

Then the geometry gave it away. Draw 2 submits TWO PARALLELOGRAMS,
neither axis-aligned -- edges (0.54,-0.56) and (0.44,0.79), roughly 45
and 61 degrees, both extending to y=+/-1.81 in NDC, well off screen.
That is the diagonal Z stroke. And ui_layout::blit walks rows and columns
of an axis-aligned rectangle; it has no rotation at all. So we blit the
sprite upright where the game draws it skewed, which is exactly the
signature I measured: right on average at +1.83, right in position with
the correlation peaking at zero shift, wrong in structure at 0.70.

Nine candidates, and the answer was never going to be a field, because
the difference is not in the sprite -- it is in how the quad is built.

Two things left open and written as such. The decoded keyframe carries
fade, scale, tint, x, y and time and NO rotation, so where the rotation
comes from is not decoded. And the pink-versus-white reading I have been
carrying was a visual comparison of two differently-shaped renderings; it
should be re-checked after geometry rather than treated as a separate
defect.
2026-08-28 22:13:50 +00:00
Sylpheed RE agent
6930d301c1 re: full suite green, and correcting the route I recommended last
iteration

Two loose ends, both on my own work.

The disc-gated suite finally completed: all six binaries, 131 passed,
0 failed, one pre-existing ignored. So the rest() fix and the T8adImage
flags field are fully verified rather than verified-on-three-binaries.
The reason earlier attempts produced empty logs was that the tool
timeout's SIGTERM reached the whole process group; setsid nohup survives
it, which is now a METHOD line.

And a correction that matters more. Last iteration I closed the swoosh as
undecodable from the disc and named a per-draw GPU capture as the next
route, "because it reads the actual blend state". It does not. Reading
command_processor.cc, each captured draw records primitive type, index
count, index-buffer address, VS and PS ucode hashes, the pixel shader's
texture bindings, and vertex attribute 0 of binding 0. There is no
RB_BLENDCONTROL dump.

So the route splits, and I have said so rather than leaving the wrong
version standing: the capture can test a per-draw VERTEX COLOUR today
with no code change, which would explain white-versus-pink directly, and
getting the blend mode itself needs a Canary change to dump the blend
registers. Either way it is instrumentation rather than another field.

The general lesson goes in METHOD too: validate a recommendation before
leaving it as advice. A named next step is a claim like any other, and I
made it without checking.
2026-08-28 22:09:07 +00:00
Sylpheed RE agent
e10a6d6f2c re: the capture is settled after all, and the swoosh is undecodable from
the disc

Testing my own caveat instead of leaving it hanging. I had worried the
plate-free title capture at t=4.0s was too early, since elements carry
keyframes out to t=600 and I had judged "settled" from mean luminance,
which cannot see a thin sprite still moving.

The plate sits at y 550-600 and the swoosh band at y 112-225, disjoint,
so a LATE capture works even with the plate present. The band correlates
0.7342 at t=4.0s and 0.7353 at t=21.5s -- identical to a thousandth over
17.5 seconds, with the band mean flat at 127. So the band is settled by
4 seconds, the capture I handed the port agent is sound, and my caveat is
withdrawn. It also corroborates the earlier finding that groups hold:
nothing crosses that band in 22 seconds.

That was the seventh candidate for the swoosh and the last one that was
not a format field. Pivot is inert at scale 100 and there is no
displacement anyway; fade is white-with-alpha; tint is white; the texture
is blue-leaning; additive via +0x04 bit 0x02 makes everything worse; the
capture is settled. The residual is stable and modest -- band mean +1.83,
edge-correlation 0.70 against 0.92 frame-wide.

So I have classified it rather than generating an eighth candidate:
undecodable from the disc, with the reach written out. And I have named
where a next attempt should start, which is not another field -- a
per-draw GPU capture of the running guest reads the actual blend state,
vertex colours and draw order for that band instead of inferring them
from the file. That is where static RE stops here.
2026-08-28 22:02:32 +00:00
Sylpheed RE agent
5d02d5cd60 re: additive blending refuted, the swoosh is not displaced, and the
residual is smaller than I said

Testing the candidate I raised last iteration rather than carrying it.
Blending bit-0x02 sprites additively moves every measure the wrong way --
whole-frame mean diff +0.55 to +1.04, swoosh-band mean +1.83 to +3.98,
band edge-correlation 0.6971 down to 0.5578. So the bit is real and
independent but does not select an additive blend. I reverted the
experiment and kept the word as T8adImage::flags, documented and not
acted on; the render is byte-identical to before.

Second refutation: the swoosh is not displaced. Shifting the band over
plus or minus 80 by 8 pixels peaks sharply at zero, 0.7342, falling to
0.22 at 24 px. So the pivot story is dead twice over -- inert at scale
100, and no displacement to explain anyway.

And I have restated the residual, because earlier sections overstated it.
The +16 to +34 band tiles I quoted were measured WITHOUT --primitives.
With the dim drawn the band's average is nearly right at +1.83; what is
wrong is its structure, tiles running -38.6 then +33.8 and cancelling.
Six candidates eliminated now and none confirmed.

One caveat I owe the port agent about the capture I gave them: it is at
t=4.0s, roughly 174 keyframe units into a screen whose elements have
keyframes out to t=600. I judged "settled" from mean luminance, which
cannot see a thin sprite still moving. It is settled for the bulk of the
screen and not proven settled for every element -- which is a live
alternative explanation for a structural difference in exactly the band
the sweeps cross.

METHOD: cargo build passing does not mean cargo test compiles. Adding the
field built the library in 1.48s and broke two test-only struct literals;
cargo test failed with exit 101.
2026-08-28 22:00:55 +00:00
Sylpheed RE agent
e0d02dce71 re: a candidate blend flag -- T8aD header +0x04, bit 0x02
Continuing the swoosh. Last iteration ended with "the next step is
finding where a blend mode would be encoded, and I do not know the format
carries one". It does carry a candidate.

The kind field is not it -- the swoosh sprites are kind 0x0, the same as
ordinary ones. But the T8aD header word at +0x04 splits the title's
sprites exactly along effect versus normal: pteff01, pteff03a,
ptlogo_back2eff1..5 and both ptlogoall_eff are 0x8832, while ptlogo1,
ptlogo2, ptlogo_tm, ptbase2, ptlogo_back2 and ptcopyright are 0x8830.
One bit, 0x02.

Disc-wide it behaves like a real flag rather than an artefact: 19216
sprites, 18 distinct values, bit 0x02 set in 27.1 percent, and it toggles
against otherwise identical words -- 0x8830 against 0x8832, 0x0830
against 0x0832, 0x0810 against 0x0812, 0x0030 against 0x0032.

Marked as correlation and not decode, because nothing here shows the bit
MEANS additive. The one thing that makes it more than a guess is
ptlogo_back2eff, which carries 0x8830 despite having eff in its name --
so the split is the field's and not my pattern-matching on names. The
test is to blend bit-0x02 sprites additively and re-correlate the title
against the plate-free capture.

METHOD gets the trap that cost the first attempt: searching for a
sprite's name and taking the next T8aD returns the SAME header for every
sprite, because the names all live together in the declaration table. It
failed its own control at once -- different sprites reporting identical
dimensions -- and the fix was to match on width and height instead of on
proximity.
2026-08-28 21:46:18 +00:00
Sylpheed RE agent
c10802536f re: the pivot mismatch cannot be the swoosh's cause -- refuting my own
candidate with the check I named

Last iteration I offered the pivot mismatch as a quantified candidate for
the swoosh rendering too thick, and said explicitly that whether it bites
depends on which pivot the compositor uses and that I had not checked. I
checked.

blit takes the drawn size from the TEXTURE and uses the pivot only as the
scale anchor, kf.x minus pivot times scale-minus-100 over 100. At scale
100 that term is zero. And every one of the seven swoosh elements --
ptlogo_back2, its glow and the five eff segments -- reports a single
scale of (100,100) across all its keyframes. So the mismatch, real as it
is in the data, cannot move or resize the swoosh in our render. Refuted,
under my own name.

It is not harmless everywhere, and I have said where it does bite:
ptlogo1 and ptlogo2 run 100, 101, 103, 112, 150 during the build-in, so
there the wrong pivot really does displace them -- during the animation
rather than at rest.

What that leaves is the blend, by elimination rather than by evidence.
Position and size are the texture's own and right, fade is
white-with-alpha, tint is white, the texture is blue-leaning rather than
pink. Seven overlapping sprites at 5 to 23 percent opacity stacked with
plain alpha-over saturate toward opaque white, which is what we draw and
would read as thicker beside the game's thin coloured stroke. Recorded as
the remaining candidate and explicitly not as a diagnosis -- no blend
mode has been identified in the data.
2026-08-28 21:40:46 +00:00
Sylpheed RE agent
2a0afc1405 port: land the pivot finding on the handoff (third assertion miss)
The scripted HANDOFF edit asserted on text an earlier edit had rewrapped,
so the pivot and colour findings landed in docs/re and not on the page
the port agent reads. Third time this session; METHOD now says the rule
was insufficient and names the mechanical fix.
2026-08-28 21:37:51 +00:00
Sylpheed RE agent
e249b2e2ff re: the swoosh's pivot belongs to the Japanese sprite, and the colour is
not in any field I can find

Chasing the swoosh defect. The colour half is a clean negative and the
geometry half now has a number.

Ruled out for the colour: every keyframe of every swoosh element carries
fade 0x??ffffff -- white RGB with only the alpha varying -- and no tint
is anything but white. The texture is not pink either; ptlogo_back2
decodes blue-leaning at (175,174,198) and its glow warm at (255,253,234).
So the magenta edge the game draws comes out of blending, and I have said
that rather than leaving "wrong colour" as an undifferentiated mystery.

The geometry half: the declared pivot does not match the element's own
texture. Build 4's ptlogo_back2 pivot is (500,117), which is exactly half
the JAPANESE sprite at 1000x234, not half its own English one at
1118x262 -- off by 59 px, the right order to make the swoosh too thick
and extend too far right. The layout record is authored once and shared
while the sprites are swapped per language, which ui-rat-layout.md warns
about in general; this is that warning measured on the screen where our
render disagrees with the capture. It cuts both ways -- the Japanese
build's ptlogo1 is off by 40.

Marked as a quantified candidate rather than a diagnosis, because whether
it bites depends on which pivot our compositor uses and I did not check.

One METHOD line, because the first version of that check was misleading:
a pivot-equality test conflates rounding with real breakage. Equality
flagged 74 percent of title elements. Split by magnitude it is 17 exact,
54 off by a pixel from odd dimensions, and 24 genuinely off -- and those
24 are the finding.
2026-08-28 21:37:24 +00:00
Sylpheed RE agent
cdb5a99def re: the title "slab" is the logo swoosh drawn white and thick, and it
lands on the known tie-break group

Chasing the residual I left open last iteration, and it turns out I had
attributed the port agent's slab to the wrong thing.

First the refutation. My hypothesis was that our dim is applied over the
whole frame rather than beneath the UI where its layer key puts it. If
that were true the logo would render too dark. It reads +2.36 against a
background of -0.74, so the compositor honours the paint order and the
hypothesis is dead.

What the residual actually is: with the dim in place the error is not
uniform but concentrated in one band, running -38.6 in one tile and +33.8
three tiles to its right. Cropping that band from capture and render
shows it immediately -- the game draws the logo's Z swoosh thin with a
pink edge, and we draw it thick and solid white. That, not the missing
dim, is the washed-out slab over the title logo the port agent reported.
The dim explains a uniform +13; this explains the slab.

The elements are ptlogo_back2, its glow, and the five ptlogo_back2eff1..5
segments -- which are also the group carrying this corpus's known
unsolved paint-order tie-break at key 0x8083, on the same screen. I have
recorded that as a lead and explicitly not as a diagnosis, because a
blend-order swap is a poor explanation for white instead of pink and I
would expect a second cause.
2026-08-28 21:31:52 +00:00
Sylpheed RE agent
d217c07db2 re: the title's arrived pose is rest, and the washed-out slab is a
MISSING dim quad

Both halves of what the port agent could not settle, answered against the
plate-free capture rather than against another renderer.

Rendering build 4 and edge-correlating to the capture of the real screen
gives 0.9163 at zero shift, 0.92/0.78/0.93 per band. The geometry of rest
is the title's arrived pose, so the timeline is not needed there.

The slab is the more useful half, and it is inverted from how it looked.
rest alone renders uniformly +13.14 too bright, R +12.35, G +13.58,
B +13.48. Drawing the .prm primitives collapses that to +0.55. The
element is pteff02.prm, the 25 percent dim at rest t=46 with fade 0x40,
and --primitives is OFF BY DEFAULT. So the washed-out cyan slab is not
something being drawn that should not be; it is a dim that should be
drawn and is not, and because the title art is blue-dominant the
shortfall reads as a cyan wash. Any consumer of screen render that omits
--primitives on this screen gets it.

Two residuals recorded rather than tidied. With the dim drawn the mean is
essentially exact but per-pixel agreement gets slightly WORSE --
edge-correlation 0.9163 down to 0.9066 and pixels off by more than 20
rising from 108051 to 162636 -- so the dim's average contribution is
right while its application is not exactly the game's. And separately the
per-tile map shows a +25 to +38 hot spot on the upper-right Z swoosh,
which a capture from the first iteration of this work already showed pink
in the game and white in ours. Same region, still undiagnosed.
2026-08-28 21:27:04 +00:00
Sylpheed RE agent
5414db34bf re: the rest() fix gets its disc-wide check, and one question back to the
port agent

Closing the two things I flagged as unconfirmed when I committed the fix.

The unit gate is green: cargo test -p sylpheed-formats with SYLPHEED_DISC
set gives 131 passed, 0 failed across six binaries including the
disc-gated ones. The earlier background run that produced an empty log
had died with SIGTERM, which is why it looked like nothing happened.

And the disc-wide check the classification requires. Reimplementing both
rules over every RATC bundle on the disc -- 2859 bundles, 13991 elements
with at least two keyframes -- the fix moves rest for 30 elements, which
is 0.21 percent. Four go invisible to visible. Zero go visible to
invisible, which is the safety property I wanted and did not have when I
committed. The four are ptframe1 and ptframe2 in GP_TITLE entries 5 and
8, the same pair once per language.

That surfaces a discrepancy worth handing back rather than smoothing
over. On the English main menu exactly TWO elements satisfy the port
agent's pose-equality condition, not six, so their six must span the
whole twelve-screen export. That fits their own observation that the
timeline and rest differ in exactly one region, the bounding box of
ptframe1 and ptframe2 and nothing else. But if any of their other four
have a TRANSPARENT trailing run, my alpha rule leaves them alone
deliberately -- that exclusion is what protects PAUSE -- so I have asked
which screens they are on and whether a capture shows any of them drawn.
If one is, the alpha rule is incomplete and needs a third discriminator.
2026-08-28 21:24:42 +00:00
Sylpheed RE agent
4bc970688f re: rest() fixed -- a trailing keyframe run is the hold when it is
VISIBLE, not when its pose repeats

The port agent's report was right about the defect and about which
elements it hits, and its proposed condition does not survive the case
the exclusion was written for.

Their test -- the final untimed keyframe has the same pose as the last
timed one -- is also true of pgptitle.rat, whose trailing run is two
identical transparent frames. Adopting it would erase the word PAUSE
again, which a committed capture disproves. What separates the two is
alpha: an exit fades the element out so its last keyframe is transparent,
while an element with no exit ends on the pose you can see. So
rest_plateau now accepts a trailing run exactly when it is visible.

Verified against a capture rather than against another renderer, which is
the point their own report made: ptframe1's rest moves from (620,108)
t=16 to (440,108) t=62; the changed pixels are 10082 in a bounding box of
x 440-839, y 108-577, which is exactly the 400x470 at (440,108) they
predicted; and correlation against main-menu-oracle.png over that region
improves from 0.9596 to 0.9748. The PAUSE wordmark is unchanged across
all three pause builds.

This also closes a question ui-paint-order-key.md has carried for a
while, that ptframe1 and ptframe2 rest at alpha 0 while the capture shows
the menu frame plainly.

One trap cost most of this iteration and goes in METHOD: CARGO_TARGET_DIR
is redirected in this container, so ./target/debug/sylpheed-cli is hours
stale and every render I made against it was old code. Byte-identical
before and after reads as "no effect" when it actually means "you ran the
old binary".

The full disc-gated test suite was still executing when this was
committed; the verification above is artifact-based.
2026-08-28 21:19:00 +00:00
Sylpheed RE agent
ed4e5c7b37 re: build 4 captured without the plate, and keyframe groups hold rather
than loop

The port agent ranked a plate-free capture of build 4 above any further
static RE, so that came first.

B from the main menu returns to the title and the plate fades in a beat
later, which opens a clean window. Recorded at 20 fps from the press: the
art appears at 1.10 s, builds in to 3.70 s, sits settled and unobstructed
until 5.00 s, and the plate arrives at 5.10 s -- the band jumps from 282
to 3755 bright pixels. Committed the frame at 4.0 s as the reference for
the cyan glow slab they report drawing and the game not having.

Their other sub-question -- whether a keyframe group loops or holds --
falls out of the decoded sweeps plus a measurement I already had, and the
two agree. ptloop01's final keyframe parks pteff03.t32 at x=1521 and
ptloop02's parks pteff03a.t32 at x=-839, both off-screen on a 1280-wide
design; and over 18 s of settled title the centre tiles sit at sd <= 0.01
when a looping group would recross the screen every 7.5 s. So groups HOLD
at the last keyframe. The loop*.rat name is misleading.

Also recorded, in the corpus rather than only in their report: the
rest_plateau bug, with their exact identifying condition -- the final
untimed keyframe has the same pose as the last timed one -- the six
elements it misses on main_menu, and the bracket it drops. That closes an
open question ui-paint-order-key.md has carried for a while about
ptframe1 and ptframe2 resting at alpha 0 while the capture shows the
frame plainly. Same two elements, same cause. Not fixed yet; the change
is in ui_layout's rest().

And a METHOD line I would not have written myself: two renderers agreeing
is not evidence the field is right. Their composite and screen render
matched to 3/255 on main_menu and both omitted two elements the game
draws, because both read one field through one decoder.
2026-08-28 21:05:35 +00:00
Sylpheed RE agent
1b95001feb re: the last keyframe time is the chunk terminator -- observed, not
assumed

Continuing the pulse's cycle length, which I had called a structural
limit of the format. It is, and now I can show it rather than assert it.

Dumping past what I thought was the final keyframe found an EIGHTH block:
fade 0x00ffffff at the same position, so the glow returns to fully
transparent and the pulse is a closed cycle rather than a one-shot ramp.
That is worth having on its own -- the port knows the animation ends
where it starts.

And the eighth block's time slot contains the four bytes "end ", the
record's ASCII terminator. So the corpus's rule that a group's last block
has no time of its own holds here in a second form: not the next group's
element index, but the chunk terminator. The value does not exist. That
goes into ui-rat-layout.md, because it generalises beyond this record.

The measured 2.3 s would need a final step of about 33 units. I have
written that number down explicitly labelled as fitted to the measurement
rather than read from the file, so nobody later re-derives it as a
decode.

One thing I had half-assumed and can now rule out: the word at +0x004 is
not a keyframe count. It reads 60 here with 8 keyframes and 30 in the
loop records with 3. Unknown, and marked so.
2026-08-28 21:02:14 +00:00
Sylpheed RE agent
c176fa53bc re: the 2.3 s pulse is the PRESS (A) plate, not the title
Closing the residual I left explicitly unidentified last iteration, and
correcting my own claim twice over in the process.

The cheap test first: is the oscillation global? No -- the bottom-right
corner is flat at sd 0.003 and uncorrelated with the whole frame. So a
per-tile amplitude map over an 8x6 grid, which localises it hard: sd 7.65
in the band x 318..954, y 560..672 at lag 2.3 s, against 0.06 on the
wordmark. That band is the PRESS (A) BUTTON plate's rest position.

Decoding ptbtn00f.rat, the plate's highlight variant, gives the pulse
itself: alpha 0x00 -> 0x06 -> 0x4a -> 0x50, held, then back down through
0x4a to 0x06 across t=6..105. A glow that fades in and out, which is
exactly what a press-start prompt does.

So "the title screen loops at 2.2 s" was wrong in both halves. The title
ART is near-static apart from the two decoded sweeps at 7.5 s and 9.5 s;
what pulses is the PLATE, and the plate is build 2, not build 4. REFUTED
carries it under my own name.

Still amber, and said so rather than rounded off: the cycle LENGTH is not
readable. The keyframe group's last block has no time -- that slot
belongs to the next group -- so the declared span is at least 105 units,
1.75 s, against a measured 2.3 s. Consistent with a final block
extending the tail. Not confirmed.
2026-08-28 20:58:49 +00:00
Sylpheed RE agent
83293c3f81 port: land the loop decode on the handoff, and note the edit trap
Second iteration running where a scripted HANDOFF edit failed its
assertion on rewrapped text, leaving the port's page stale while docs/re
had the finding. Patched by reading the file first, and METHOD now says
to check the commit's file count for a silently dropped edit.
2026-08-28 20:53:33 +00:00
Sylpheed RE agent
3717293fa9 re: the loop records decode -- and they are not what I measured
Two of my own claims from last iteration die here, and the decode is the
thing that kills them.

Searching for the SHAPE instead of an assumed count found it immediately:
fade words matching 0x??ffffff at alternating strides of 24 and 16, which
is a 40-byte block carrying fade at +0 and tint at +24 -- exactly the
build layout, starting at +0x68. So "a leaf record's keyframes are not in
the build's 40-byte layout" is withdrawn. They are. My scan failed
because I demanded 29 strictly-increasing times, having read the word at
+0x004 as a keyframe count; the records hold THREE.

Decoded, both are horizontal light sweeps. ptloop01 runs pteff03.t32
from x=-639 to x=1521 over t=150..600, which is 450 units or 7.5 s;
ptloop02 runs pteff03a.t32 the other way over 570 units or 9.5 s.

Which refutes the second thing. Last iteration I attributed the measured
2.2 s oscillation to these elements because build 4 declares them --
association, not evidence. A 7.5 s cycle would give about three peaks in
a 22 s capture and I counted eight. So the loops are not what I measured,
the 2.2 s stands as a measurement, and its source is now explicitly
unidentified rather than wrongly assigned.

METHOD gets the general form, because it is what cost the previous
iteration: do not hard-code an expected element count into a structure
scan. Scan for the shape and let the count fall out.
2026-08-28 20:53:08 +00:00
Sylpheed RE agent
00ce0f33d0 port: carry the loop-cycle negative into the handoff
The previous commit's handoff edit failed its own assertion -- the text
had been rewrapped since I wrote the pattern -- so the finding landed in
docs/re and REFUTED but not on the page the port agent actually reads.
An answer not reachable from HANDOFF.md has not been delivered.
2026-08-28 20:49:39 +00:00
Sylpheed RE agent
a048910497 re: the loop record is located, its cycle is still not readable
Continuing the one gap from last iteration -- turning the title's
measured 2.2 s loop into a decoded cycle. It did not turn.

The record itself is found and partly confirmed. ptloop01.rat is an
opt-linked leaf RATC at 0xbb5966 in build 4, placing pteff03.t32, with
0x1e -- thirty -- in the high half of the word at +0x004, which is the
~30 keyframes ui-rat-layout.md predicts for a loop record. Its pivot
fields read 200 and 90, matching exactly what screen info prints for
ptloop01, so this is the right blob and the header offsets hold.

The times are not there. A build's keyframes are 40-byte blocks with the
time at +36; scanning every 4-byte-aligned start across a 0xC0 window for
29 strictly increasing values at that stride finds nothing, in either
loop record. So a leaf record's keyframe layout is NOT the build
placement layout, which is now a line in REFUTED because it is the
obvious first assumption and it is wrong.

The 2.2 s stays measured and the port hardcodes it. What the next attempt
inherits is the record's address, a confirmed pivot, and one layout ruled
out -- which is the useful part of a negative.
2026-08-28 20:49:11 +00:00
Sylpheed RE agent
5f3d8f86a4 re: the title loops at about 2.2 s -- and the build-in test I intended
did not work

I set out to do for the title what worked for the splash: measure the
build-in and check it against the declared keyframes. That part failed,
and I am recording the failure rather than the number.

It failed for two reasons, both mine. The title was reached by skipping
the movie with A, which cuts to black and brings the title up on a path
that may not be the normal one. And the visible rise, about 2.7 s, is a
luminance curve -- which screen-transitions.md already establishes is not
the fade quad's ramp. So 2.7 s neither confirms nor contradicts build 4's
declared 16 -> 261, four point oh eight seconds. They are not measuring
the same thing, and I have said so instead of reporting a discrepancy.

What the recording did establish is worth having anyway: the title never
settles. Over 22 s the mean luminance oscillates continuously, peaks at
5.6, 7.8, 10.0, 12.7, 15.2, 17.1, 19.2 and 21.3 s, mean interval 2.24 s.
Marked as about 2.2 s plus or minus 0.4 rather than a precise period,
because peak-picking a low-amplitude signal is coarse.

The mechanism was already decoded and I did not re-derive it: build 4
declares ptloop01.rat and ptloop02.rat, and loop*.rat is a looping sprite
animation rather than a composition. What is new is the period, and that
it runs forever. For the port that matters -- drawing the resting pose
and stopping will look wrong.

Not confirmed, and flagged: that 2.2 s is the loop's OWN declared cycle.
screen info prints the element's placement keyframes but not the frame
count inside the .rat, and I did not open it.
2026-08-28 20:45:53 +00:00
Sylpheed RE agent
113c25aac1 re: the splash's timing is decoded, and the fade-out matches to 0.03 s
Closing the caveat I left last iteration -- the splash timing was +/-0.5s
because I sampled at 2 fps, and at that rate I could not see ramps at
all, only plateaus.

Re-recorded at 10 fps: the splash FADES, both in and out, rather than
cutting. And the bundles declare it. palogo_sqex.t32 carries keyframes
[15 30 235 239 251 255] and palogo_gamearts.t32 [15 30 190 194 206 210],
each with an _eff glow child on [15 30 45]. Under Q1's 1 unit = 1/60 s
that is a 0.25 s ramp in, a 3.42 s or 2.67 s hold, and a 0.33 s fade out
-- against measured holds of about 3.5 s and 2.4 s and fade-outs of about
0.3 s. The constant 0.35 s offset between declared and measured start is
just that my recording's t=0 is when the WINDOW appears, not when the
guest starts drawing.

So the first screen's animation moves from measured to decoded: the port
reads it off the disc instead of trusting my stopwatch.

It also explains something the capture showed and I had no account for.
The brightness overshoots on the way in -- peaks at 0.8 s, settles by
1.1 s -- which reads as a bloom. It is the _eff glow child, whose
keyframes run 15 -> 30 -> 45, ramping in after the logo and back down
while the logo holds. Mechanical, not a rendering artifact.
2026-08-28 20:39:17 +00:00
Sylpheed RE agent
9f39e8ee72 re: the splash confirmed against the running game, and timed
Closing the amber I raised last iteration: the splash renders had no
framebuffer capture to diff against. Recording the boot from the moment
the window appears, at 2 fps, catches it before the movie.

Both halves match their own render at zero shift, each with the OTHER
half as a negative control: publisher frame 0.9146 against entry 10 and
0.027 against entry 11; developer frame 0.9792 against entry 11 and
-0.033 against entry 10. The EN/JP twins score 0.876 and 0.966, so the
test tells a screen from a different screen but cannot tell a pair apart,
and I have said so rather than implying it discriminates.

The recording also times the front of the boot, which Q6 and Q7 only had
from the title onward: SQUARE ENIX from about 0.5 to 4.0 s, black,
developer logos from about 5.0 to 7.5 s, black, then ADV.wmv from about
9.5 s.

One trap avoided and written into METHOD. The frames around 9.5 to 12.5 s
show SQUARE ENIX again in cyan, and read from the capture alone that is a
third logo screen. It is not -- it is the intro movie's own opening,
which the milestone-2 notes describe as white SQUARE ENIX plus cyan glow
plus red diamonds. A logo appearing twice may be two different things.
2026-08-28 20:35:10 +00:00
Sylpheed RE agent
b302558d26 re: the developer splash renders -- screen 1 of 5 finally has a composite
Two pages of this corpus disagreed. MISSION says the splash "is the RATC
screen, which already renders"; ui-paint-order-key.md says it "cannot be
rendered by screen render at all". Running the tool both ways settles it.

screen list --all shows all 16 GP_TITLE entries instead of 12, and the
four the default listing drops are the splash, each half shipped twice:
entries 10 and 13 are the white SQUARE ENIX publisher logo, entries 11
and 14 the GAME ARTS / SETA / studio anima developer logos. Entry 11's
seven elements are the three logos, their three _eff glows and the
palogo_eff0.prm backdrop -- exactly the composition ui-paint-order-key.md
had measured for the splash without being able to draw it.

So the "cannot be rendered" line is wrong and is corrected in place. What
is true is narrower and worth keeping: the splash is invisible to the
DEFAULT listing because is_build wants a .rat child, so anyone who does
not pass --all concludes it is missing. That goes in METHOD -- a default
filter can hide a whole screen and the corpus will record it as absent.

The payoff is for the port rather than for the RE: the first of the five
screens now has a reference composite, which it did not have. Marked
amber on one point -- there is no framebuffer capture of the splash in
this repo, so the match to the running game is by description against the
milestone-1 notes, not by pixels.
2026-08-28 20:29:33 +00:00
Sylpheed RE agent
cc72f8b5a8 re: the phase transitions -- and withdrawing yesterday's refutation
because the test probed the wrong state

Nine stores to the phase field this+132, control-gated on the 136 query
returning at least the 18 known ones. Attributed to the handler each
lives in, they give the phase graph: entry -> 2, then 2 -> 0 (splash),
2 -> 3, 3 -> 4, and 4 -> 2 on event 0.

That last edge is the problem with what I did last iteration. The event-0
block sets TWO fields one instruction apart -- stw r28,136(r30) for state
0 and stw r11,132(r30) with r11 = 2 for phase 2. So B from the menu lands
in PHASE 2, the same phase as the boot title, and the test I ran
believing it probed phase 4 state 0 probed nothing of the kind.

So the refutation is withdrawn. Worse for me and better for the idea: A
working on the B-returned title is exactly what the hypothesis predicts,
since phase 2 is the phase that references BUTTON. The hypothesis is back
to untested, now consistent with two observations rather than one, and
what it still needs is the attract-returned title's phase -- which no
test so far has read.

The measurement from that iteration stands, because it does not depend on
the phase: the B-returned title accepts A, so only the ATTRACT-returned
title is inert. I have said so explicitly in the page rather than letting
the withdrawal take the good half down with it.

METHOD gets the lesson: before testing "state X behaves like this", check
your route actually reaches state X. When a transition writes more than
one field, read the whole block and not just the store you were looking
for.
2026-08-28 20:25:06 +00:00
Sylpheed RE agent
4830c63402 re: my phase hypothesis is dead, and the input puzzle is narrower than
this corpus thought

Last iteration I proposed that the boot title is phase 2 and the
attract-returned title is phase 4 state 0, and that only phase 2 wires up
A -- a candidate answer to the oldest open question in
canary-scripted-input-traps.md. I marked it untested and named a test
that needed reading this+132 at runtime.

There was a cheaper one. The decoded graph says B from the main menu is
the edge 4 -> 0, so the B-returned title IS phase 4 state 0. If that
state were the inert one, A would not work there.

A works. Control first -- A on the boot title opened the menu -- then A
on the B-returned title opened it too. So phase 4 state 0 handles A and
the hypothesis is refuted, one iteration after I raised it. Recorded in
REFUTED under my own name, and the conditional stated: the edge is
decoded, but which title it produces is my inference.

The test earned something better than it cost, though. That page frames
the problem as "the boot title accepts A, a later title does not", and
that is too broad -- the B-returned title accepts A perfectly well, a
case nobody had tried. Only the ATTRACT-returned title is inert. Whatever
causes it is tied to the attract cycle specifically rather than to the
title having been shown before, which is a much smaller haystack.

Also reproduced in passing: 4 s after B the plate is absent and fades in
after, the same near-miss this corpus has already been caught by once.
2026-08-28 20:21:23 +00:00
Sylpheed RE agent
4380305921 re: phase 0 is the splash, phase 2 is the title -- and a candidate answer
to this corpus's oldest input puzzle

Continuing Q6 on the phases left unread. Strings each phase handler
references, plus whether it carries its own jump table:

  phase 0  sub_821C5690   LOGO                                    no switch
  phase 2  sub_821C5818   BASE_INFO, BUTTON, TITLE_SCREEN         no switch
  phase 3  sub_821C5EC0   (none)                                  one switch
  phase 4  sub_821C6458   BASE_INFO, LOADING, TITLE_MENU, TITLE_SCREEN

Phase 0 referencing LOGO is a second independent confirmation that it is
the developer splash -- the iterate3E notes reached the same function
from the guest side and named the splash's LOGO items. Phase 2 draws the
title WITH the PRESS A plate, which the archive side had already
established is a build of its own.

Which produces something worth more than either: the title is installed
from TWO places, phase 2 and phase 4 state 0. Same screen, different
code. canary-scripted-input-traps.md has recorded for months, and never
explained, that the boot title accepts A while the attract-returned title
accepts nothing, with the giveaway that a draw capture in each is
identical. Two code paths installing one screen is exactly that shape. I
have written it into that page as a candidate with the cheap test named
-- read this+132 on each title -- and marked it untested, because it is.

One query in this iteration failed its own control and I threw its half
away: counting stw rX,136(r30) per phase returned 0 for phase 4, which
has 18, because the operand text has a space the pattern did not allow.
The bctr half passes its control and is reported. METHOD gets the
underlying trap: instructions.function is unpopulated for most rows, so a
query scoped on it silently returns nothing.
2026-08-28 20:15:02 +00:00
Sylpheed RE agent
86e52b01d0 re: the menu's music is BGM_103 -- found by accident while chasing Q6,
confirmed three ways

Chasing where the event code comes from, sub_821C5580 turned out to do
two things worth having.

For Q6 it arms the outer gate: li r11,3 ; stw r11,16(r28) is exactly the
value sub_821C7850 tests before dispatching, which answers the "what does
this+16 == 3 gate on" question I left open last iteration. It then passes
r5 = [r27+4], an event read out of a structure, so the vocabulary is
still not enumerable. Of the four callers of sub_821C7850, three pass
constants -- 0, 0 and 5 -- and this one does not.

The accident is a few instructions earlier: li r5, 1103 into a sound-play
call. 1103 is a BGM cue id, BGM_103. That closes a residual I had written
off as undecodable: which bank the menu plays. The cue table genuinely
cannot say, since its BGM entries are numeric -- but the code can, and it
checks out three independent ways. The census says BGM_103.slb is two
waves of 3876864 and 3930112 bytes; the XMA probe at the main menu saw
two stereo streams of 3876864 and 3930112 bytes. Byte for byte on both.

That also corrects me. bgm-two-stems.md said those observed sizes matched
no bank's declared waves and inferred the decoder gets a window rather
than a whole wave. Wrong on both counts -- I had checked only the BGM_0xx
rows of the census because that was the block on screen. METHOD gets it:
check a measurement against the whole reference set, not the part you
happened to be reading.
2026-08-28 20:10:41 +00:00
Sylpheed RE agent
a98e96b0f0 re: the ten states are ONE PHASE of five -- and phase 0 is the splash
Chasing where the event code comes from turned up the level above and
corrected my own framing three sections running.

sub_821C6458 has exactly one caller, sub_821C7850, and that caller is
itself a dispatcher -- on a SECOND, outer state field at this+132, five
phases, gated on this+16 == 3. Phase 4 is the title/menu machine I have
been decoding. Phase 0 is sub_821C5690, which the iterate3E notes had
already fingered as the splash mechanics from a completely different
direction. That agreement is the useful check here.

So GamePart_Title has two nested state fields: a phase at this+132
choosing which sub-machine runs, and the state at this+136 inside phase
4. Everything I decoded over the last three iterations -- ten states,
eighteen edges, the event dispatch -- is phase 4 ONLY. Phases 1, 2 and 3
are untouched. I have called that out in REFUTED against my own earlier
wording rather than quietly restating it.

And the event code is forwarded, not created: all five phase handlers are
called with the same (this, r29, r28), where r28 is sub_821C7850's own
third argument passed through untouched. So the event vocabulary is
defined at least one level further up, and finding what 3, 5, 8, 10 and
25 mean means going up again. Not done.
2026-08-28 20:05:27 +00:00
Sylpheed RE agent
52275ddd53 re: state 4's edges are selected by an event code -- 6 of 26 handled
Continuing Q6's last item: the condition on each edge.

sub_821C6458's third argument is an EVENT CODE, and state 4 -- reached
straight after the main menu is installed, so the input-waiting state --
dispatches on it through a second jump table at 0x821c6b7c, 26 cases.
Six are handled and twenty fall through changing nothing:

  event 0 -> state 0 (TITLE_SCREEN, back to the title)
  events 3, 5, 8, 25 -> state 8 (LOADING)
  event 10 -> state 5

So the edges I listed last iteration as "4 -> 0, 5, 8x4" are each now
attributed to a specific event, and the input state's graph is complete.

One thing I deliberately did NOT promote. The main menu has five items, B
returns to the title, and this table has one event to the title, four to
LOADING and one elsewhere. That is a tempting fit -- B plus four items
that load an external archive plus EXTRAS staying inside GP_TITLE -- and
it matches everything measured. It is also a count-match and nothing
more: the events are unnamed, nothing shows event 3 is a particular menu
row, and state 5 installs no named screen so the EXTRAS half has no
support at all. Written down as an observation with that said plainly,
because the same shape of reasoning already cost me DIFFICULTY and
EXTRA_MENU two iterations ago.

Still open: what the event numbers mean, the conditions on edges out of
the other states, and what states 1/3/5/6/7/9 do.
2026-08-28 20:00:49 +00:00
Sylpheed RE agent
6e92ae4597 re: the title's state machine is decoded -- ten states, eighteen edges
Q6's last open piece was which state leads to which, and it turned out to
be a plain switch.

state = this+136, compared against 9, dispatched through a jump table at
0x821C6498. The ten words at that address disassemble as lwz r16,N(r28)
instructions and are nothing of the kind -- they are the table. That is
the same "self-reference" I flagged two iterations ago as looking like a
jump table; it is one, and the disassembler was decoding its data.

Three states install a named screen: 0 is TITLE_SCREEN, 2 is TITLE_MENU,
8 is LOADING. Eighteen transitions, every one a literal li/stw pair into
the state field, giving the full graph.

It lines up with the behaviour measured weeks of iterations ago without
either side knowing about the other: boot reaches state 0 and A opens
state 2; B at the main menu returns to the title, and 4 -> 0 is the ONLY
edge back to state 0, reached from 2 -> 4; entering a submenu goes
through LOADING and comes back, which is 4 -> 8 at four separate sites
then 8 -> 2. I have marked that as corroboration and said plainly it is
me matching a graph to observations -- the conditions on the edges are
not decoded, so nothing here proves which input picks which branch.

Still open and written down as such: the condition on each edge, what
states 1/3/5/6/7/9 do, and the fact that state 3 is never a destination
in this function, so something outside sets it.
2026-08-28 19:56:22 +00:00
Sylpheed RE agent
c8c7677f6d re: the lookup argument is decoded at 46 of 48 sites -- and it refutes my
own corroboration from last iteration

Last iteration I listed a candidate name vocabulary from "upper-case
strings the callers reference", marked it amber because those are not
proven arguments, and named the check: confirm per call site which string
actually lands in r5. Ran the check. It fired.

The first extractor FAILED ITS OWN CONTROL -- a backward scan for
addi r5 recovered 7 of 48 sites and missed both title sites I had already
read by eye, because there the name is built in r27 and reaches r5 via a
later mr. Rewritten as a forward register simulation it reproduces all
three known sites and recovers 46 of 48, giving 28 distinct names.

Two of my own claims die. DIFFICULTY and EXTRA_MENU are NOT arguments at
any site -- they are strings that merely live in the same functions, so
the "independently corroborated by measurement" line I wrote was an
artifact of the loose filter. Only TUTORIAL_MENU survives. And
sub_821CC860 is not a screen factory at all: its real arguments include
BG, BLACK, FADE, FILE, KEY, PAD, SOUND and GAMMA_RGB. It is a generic
name-keyed lookup, mostly config, which the title part happens to call
with its three state names.

The title finding itself is untouched -- sub_821C6458 really does
transition by calling this with TITLE_SCREEN, TITLE_MENU and LOADING.
What is gone is the inference that the 28 names are a screen vocabulary.

Two METHOD lines, both earned here: strings a function references are not
arguments it passes, and a backward scan for a destination register
misses arguments staged through a scratch register.
2026-08-28 19:52:02 +00:00
Sylpheed RE agent
8b4965f0a5 re: a title-side transition is a screen lookup BY NAME, not by id
Continuing Q6's last residual -- the control flow I left unread.

All three state-name sites in sub_821C6458 compile to the same shape:
load the name, then bl 0x821CC860 with it in r5 and 0 in r6, then hand
the result to sub_82187B78. So a transition is lookup-by-string then
install: sub_821CC860(this+88, this+24, "TITLE_SCREEN", 0). Not a numeric
id and not a table index. That also explains something that had been
sitting unexplained -- GP_ADVERTISE_DEMO having zero xrefs -- because at
this level the screen graph is keyed by name rather than by GamePart id.

sub_821CC860 has 28 callers, and the upper-case identifiers in that
neighbourhood split cleanly into screen names and config keys. Three of
the screen names are corroborated by measurements I took before ever
opening this function: DIFFICULTY is what NEW GAME opens, EXTRA_MENU is
the EXTRAS submenu, TUTORIAL_MENU the lesson list. That is the static
side agreeing with the dynamic side on names neither knew about the
other.

Held at amber deliberately. The 35 strings are what those callers
REFERENCE, not proven arguments, and the list plainly mixes screen names
with things like TEXT_FONT and GAMMA_RGB. Confirming it means checking
per call site which string actually lands in r5, and I did not do that.

Still unread: which state leads to which. The three lookups sit in
different branches and TITLE_MENU's is guarded by a cmplwi/bne, but I did
not trace the branch structure, so the ORDER still comes from measurement
rather than from the code.
2026-08-28 19:46:43 +00:00
Sylpheed RE agent
2be5ba7934 re: the title part's states are named in the executable -- TITLE_SCREEN,
TITLE_MENU, LOADING

Taking the one residual I had ranked as worth doing: sub_821C6458, which
the previous note left explicitly unread.

It is the title part's screen-state function -- 4460 bytes, has EH,
called from exactly one place, sub_821C7850, which sits beside the
creator the registration site points at. Its string references are
TITLE_SCREEN, TITLE_MENU, LOADING and BASE_INFO. Those are precisely the
states measured off the running game, in the game's own words, and
BASE_INFO is this corpus's own marker for a screen-config lookup rather
than a table read.

That sharpens Q6's negative rather than overturning it. The title part
DOES ask for configuration keyed by TITLE_SCREEN and TITLE_MENU, and
config.ini -- the disc's only config -- has no such sections, only an
empty [SYSTEM] and [LANGUAGE]. So the game asks the question, the shipped
disc does not answer it, and the defaults live in code. The port can
transcribe the state names even though it must author their contents.

Marked amber, not green: this is characterisation from string references
and call counts, not a read of the control flow. Two self-references
inside the function look like jump tables, which is what a state machine
compiles to, but I did not confirm that and nothing here shows which
state leads to which.

One refutation recorded because it looked like a find for several
minutes: the {func, func, ptr} triples at 0x820a3b48 read convincingly as
a state/handler table and are nothing of the kind -- the bytes before
them are the tail of a RegisterToFactory diagnostic string and the data
column is zero-filled descriptors. Static-init records, not dispatch.
2026-08-28 19:42:39 +00:00
Sylpheed RE agent
2b0e66b6ca re: confirm, move and back -- every cue the five screens need is now
located, and two of them reproduce

Continuing Q8 rather than opening anything new. The gap that mattered was
the confirm cue: a menu needs a sound on A, and I had only cursor and
cancel.

The fix was where I was counting from. The first run started counting
streams at the main menu, so the confirm cue had already fired during
boot and showed up as "played before any input". Counting from the TITLE
instead attributes it cleanly: the A that advances title -> main menu
fires the 12288-byte wave at 0x5d6c0, together with the two stereo BGM
stems, which is the menu's music starting.

So move (0x1ec0, 0.533 s), confirm (0x5d6c0, 1.016 s) and back (0x0ec0,
0.344 s) are all located and decodable. Move and back came back with
IDENTICAL head bytes and sizes on a second independent boot, so the
dedup-keyed method is stable and those two are now n=2 rather than n=1.

Two honest limits recorded rather than smoothed over. The A press both
confirms and opens a screen, so its wave could be the cue the vocabulary
calls DECIDE or the one it calls SUB_WIN_OPN -- the port gets the sound
the game plays, not a name. And left/right fired no new stream, which
excludes a DISTINCT invalid cue but cannot exclude them quietly replaying
one of the three already heard, because the probe dedups on head bytes.
2026-08-28 19:38:09 +00:00
Sylpheed RE agent
b1b2576769 re: the UI cues decode -- 0.53 s, 0.34 s and 1.02 s of real audio
Finishing the step I left open last iteration rather than starting
something new. Offsets and packet counts were in hand; what was missing
was proof they are actually waves.

slb_extract_wave.py wraps a (bank, offset, packets, channels, rate) slice
in a synthesized XMA1 RIFF, following the layout the Rust decoder already
uses. The three located cues decode to 0.533 s, 0.344 s and 1.016 s of
mono 48 kHz audio, audible from sample 0, each with the percussive
attack-and-decay envelope of a UI blip. Bitrates come out at 12-15 kB/s,
about half the stereo BGM rate, which is what mono should be.

The control matters more than the results. The SAME wrapper applied to
BGM_001's first wave decodes to 173.808875 s -- identical to the duration
that bank's own on-disc RIFF header produced back when Q10 was answered.
So the header I synthesized is not approximately right, it reproduces a
known-good decode exactly, and the cue durations are trustworthy for the
same reason.

I did not commit the decoded audio. Three commands regenerate it from the
disc, and the corpus's job is measurements and tooling rather than
extracted game assets. The offsets, the packet counts and the tool are
the deliverable.
2026-08-28 19:32:16 +00:00
Sylpheed RE agent
0d7199bc22 re: the UI sound effects ARE extractable -- retracting "cannot be
extracted", and the tool was in the build all along

Two iterations ago I closed Q8 by declaring the SE audio undecodable:
Static.slb has no wave boundaries, there is no XACT container anywhere,
and I said the index "exists only at runtime" as though that put it out
of reach. The disc half of that stands. The conclusion did not.

This build of Canary carries a cvar called xma_param_probe, added by this
project, whose own comment says it logs each XMA stream's parameters and
head bytes so raw sound.pak entries can be matched to real decode params.
It has been sitting in the startup CONFIG DUMP of every log I have read
this session.

Run with it, driving the main menu: a d-pad move spawns a new mono 48 kHz
stream of 4 packets / 8192 bytes, and B spawns a different one of 2
packets / 4096 bytes. Searching their logged head bytes in Static.slb
finds each at exactly one offset -- 0x1ec0 and 0x0ec0 -- and the two are
contiguous, 0x0ec0 + 4096 = 0x1ec0. So the bank is a packed run of whole
2048-byte packets with no delimiters, which is precisely why the seek
scan found nothing: there is nothing to find. A wave is (offset, packet
count) and nothing else.

That splits Q8's binding cleanly. Event -> WAVE is now measured: the port
can have the audio. Event -> cue NAME is still a name match on the
authors' identifiers, and the page says so.

The same run settled something for Q10 too. Sitting on the main menu, TWO
stereo 48 kHz streams were decoding simultaneously. bgm-two-stems.md said
that observation was what it needed and that this container could not
make it; it can, and a music bank's two waves are now measured as
simultaneous rather than only inferred.

METHOD gets the general lesson, because it cost two iterations: check
what instrumentation the local build already has before declaring a
question blocked on tooling.
2026-08-28 19:30:03 +00:00
Sylpheed RE agent
aea529e7dc re: S00A.wmv confirmed off the running game -- the movie was never
ambiguous, the sampling was

Continuing last iteration's amber rather than starting something new. The
fix was already named there: stop using 5-second screenshots and record
the display.

Recorded with ffmpeg x11grab at 10 fps and matched every 0.5 s. Across
the 25 consecutive samples from capture 5.0 s to 17.5 s the S00A playhead
is strictly non-decreasing -- 1.0 through 11.0 s, advancing at
essentially real time -- with scores at or above 0.96 and four of them at
0.999 or 1.000 against a runner-up in the 0.78-0.83 range. S00A is the
top match on 23 of the 27 frames carrying signal. So MS00A -> S00A.wmv is
decoded from the manifest AND measured off the game, and the intro begins
about 4.5 s after A on the save slot.

The previous attempt's failure is worth its own METHOD line, because it
did not look like a sampling problem: it looked like weak evidence for
the wrong film. Sparse sampling produced contrast-23 frames, a playhead
that would not join up, and one frame preferring ADV. Sampling does not
weaken a signal gracefully; it turns it into noise shaped like a
different answer.

One aside recorded and not chased: in the S00A 5.5-10 s window, ADV also
scores 0.97-0.99 at its OWN monotone playhead of 33.5-37.5 s. Two films
matching strongly with two consistent playheads is not noise -- it reads
as the boot movie being a trailer cut from the story cutscenes, which
also explains why the sparse run kept flipping between them.
2026-08-28 19:23:57 +00:00
Sylpheed RE agent
dfcaac221e re: the new-game path completes, the intro is only weakly identified, and
this container kills long runs

Three results, and the middle one is deliberately not upgraded.

First: NEW GAME -> DIFFICULTY -> NORMAL -> SELECT DATA -> pick a slot ->
a movie plays, with no crash. The previous run's throw at 0x82307128 is
therefore not inherent to the menu path -- the same six presses got
through it -- which fits the incomplete-cache trigger the corpus already
names. n=1 either way, and the note says so. Worth recording because one
observation had already hardened once into "A on NEW GAME hangs", and it
would have hardened again into "the new-game path crashes".

Second: with the path open, the manifest's MS00A -> S00A.wmv finally got
a runtime test, and it did NOT reproduce the quality of the attract-movie
identification. The first four samples give a clean monotonic S00A
playhead of 2, 6, 9, 13 s and the strongest frame matches at 0.916, but
the run as a whole is not one advancing timeline, most frames sit at
contrast 23-37 where this page's own rule says discrimination is poor,
and one high-contrast frame prefers ADV. So it stays amber: consistent
with S00A, corroborating the manifest decode, not confirming it. A clean
answer needs the 30fps x11grab route rather than 5-second screenshots.

Third, and it bounds the rest: the emulator vanished at ~145 s with no
crash line in its own log, last entries routine MEM-WATCH at 1.15 GB RSS.
Second time this session -- the other was ~50 s into a boot. That is an
external kill on a box with ~1 GB free and swap exhausted, not a guest
fault. METHOD and the handoff's residue table both now say dynamic
experiments here have to fit in about two minutes of guest time, which is
why the DELTASABER plates went unseen again: they need a mission load and
the run never got there.
2026-08-28 19:19:14 +00:00
Sylpheed RE agent
bc6354dce0 re: the title presents at 28.5 fps idle -- the keyframe unit is settled at
1/60 s, and the 60 Hz reading is excluded

ui-keyframe-time-unit.md named its own re-test and nobody had run it: time
300 submitted frames on the IDLE title, where nothing is streaming from
the ISO. Reaching the title is cheap now that one A skips the intro, so
this cost one boot.

Two clean trials: 300 frames in 10.40 s and 10.60 s, i.e. 28.8 and 28.3
fps. That is the same rate as the 27.6 fps measured during the loading
splash, which is exactly the page's own criterion for "the game is 30 Hz
and the unit is 1/60 s". The 60 Hz alternative is now excluded rather
than disfavoured: it needs the emulator at 47% of real time while sitting
idle on a screen the capture says costs 1526 draws over 300 frames, about
five per frame. Nothing there halves an emulator.

So the durations stand as written -- title 4.2 s, main menu 1.1 s, EXTRAS
0.87 s -- and Q1 leaves the handoff's open-residue table. It stays
MEASURED, not decoded: no field on the disc says sixtieths of a second.

One trap, and it nearly cost the conclusion. My first attempt reported
39.5 fps, which sits between the two hypotheses and would have left the
question open. It was a harness bug: I polled the log for a "done" line
rather than for the COUNT of them increasing, so it matched a capture
that had already completed and timed 0.1 s of nothing. METHOD gets the
general form -- a log-polling probe must compare a count, not test for
presence.
2026-08-28 19:11:11 +00:00
Sylpheed RE agent
2c4005795b port: audit the handoff -- four stale statements, a broken link, and the
residue in one place

Every MISSION row is answered, so this iteration checked the deliverable
itself rather than opening new scope: an answer that is not reachable
from HANDOFF.md has not been delivered, and I have been editing that page
by surgical replacement for a dozen iterations.

Four things were wrong or stale:

- the Q5 row still said initial focus was 2x TUTORIAL, 1x NEW GAME while
  its own bullet said four boots -- I updated the body last time and not
  the row;
- the Q2 bullet still described the other four main-menu destinations as
  "likely by name -- an inference, not a measurement", which Q4 has since
  MEASURED, and it listed GP_MISSION_SELECT as a main-menu destination
  when MISSION SELECT lives under EXTRAS;
- the Q1 bullet claimed that if the game presents at 60 Hz "nothing else
  on this page changes". That is now false: Q7's declared fade-in lengths
  are keyframe-derived and would halve. The same bullet also missed that
  the transition capture is a SECOND independent measurement supporting
  1 unit = 1/60 s -- the quad is declared black for 12 units, 0.20 s
  under the conversion, and the capture measured the plateau at
  0.17-0.23 s. Recorded as corroboration, not proof, because the plateau
  spans two screens' fades and cannot separate them;
- bgm-two-stems.md linked ../port/MISSION.md from inside structures/,
  which resolves nowhere.

Also named the two videos actually in scope, since "3.3 GB of video, two
files are in scope" is not much use without the filenames.

And added a What Is Still Open section. With every row green the residue
was scattered across eight pages; it is now one table with why each item
is stuck. Two entries share a single unblocker -- an emulator whose audio
path can be observed -- and they are the only ones that would change a
port decision.
2026-08-28 19:02:49 +00:00
Sylpheed RE agent
f5e7426cfa re: there is no XACT container on this disc -- retracting my own lead and
closing Q8

Last iteration I named Pj_Silph.xgs as the route to the SE cue index, on
the reasoning that XACT keeps cue->wave there. That route is dead, and
the negative is properly bounded: its 533-byte entry is high entropy with
no XGSF magic, +/-8 KB around it has none either, all 1.08 GB of
sound.pak contains zero XGSF, SDBK and WBND, and the executable has no
XACT or .xgs string anywhere. Control run first -- the same scan finds
RIFF in BGM_001 exactly where the bank structure says it should be, so
the scan works and the magic genuinely is not there.

So the .xgs and .slb extensions came from the authoring tool and survived
into shipped data that is not XACT at all. That goes in METHOD as a
general trap: check for the magic before committing to a parser for the
format an extension advertises. Writing an XACT parser would have been
wasted work.

What that leaves for the SE audio is undecodable-with-reach rather than
pending: Static.slb has no wave boundaries, there is no project file, and
none of the sound table's five records carries an offset. The bank is
certainly the right one -- 8 353 472 bytes over 322 cues is 0.84 to 1.01
seconds each at the bitrates the disc uses elsewhere, exactly the shape
of a bank of short effects. Only the index into it is missing, and it
exists only at runtime.

Q8 closes: vocabulary and bank decoded, the event binding a name match on
the authors' own event names, and the audio explicitly something the port
must source or author rather than extract. With this every MISSION row is
answered.
2026-08-28 18:58:06 +00:00
Sylpheed RE agent
c760cdc509 re: the boot sequence is not data-driven -- closing Q6 with the negative
rather than leaving it amber

Q6's second half asked what the game READS to decide the boot order. The
answer is nothing, and the value here is the reach rather than a find.

Four places checked, the order in none of them: config.ini's [SYSTEM] is
empty and it is the disc's only config; the movie manifest carries the
boot-side assets but no transitions; the requested GamePart id was
already shown to exist only as a stack argument in flight, with no
persistent field and no literal store; and the string GP_ADVERTISE_DEMO
has zero xrefs of any kind, so nothing in the code reads the attract
entry of the id table. A transition is a call with an id argument.

Traced as far as it goes cheaply: the RegisterToFactory<0,
GamePart_Title> string is referenced from exactly one site,
sub_8280E148, which also takes the address of sub_821C7D98 -- where a
factory template puts its creator. Marked amber, because that is position
and convention rather than proof, and sub_821C7D98 has zero .rdata
references, which fits a new+ctor thunk and not a state machine. The
substantial function in that neighbourhood is sub_821C6458 and I did not
read it.

So Q6 closes as answered with the driver classified as code rather than
data, which means the port AUTHORS the sequence -- and that is fine,
because the sequence itself is measured end to end and the handoff now
carries it in one line.
2026-08-28 18:53:14 +00:00
Sylpheed RE agent
c8e8dc0427 re: what the game actually reads at boot -- config.ini, and which
GameParts exist at all

Q6's second half, advanced but not closed, and the negative is the point.

config.ini is the disc's ONLY config file -- one find over the whole
extract -- and its own Shift-JIS header calls it the "Application /
GamePart initial settings table". Its [SYSTEM] section, which that same
comment says holds what the game and every game part share, is EMPTY. So
the boot order is not in disc-side configuration at all, and that search
space is now closed rather than merely unexplored.

What the file DOES carry is the language: XC_LANGUAGE_* -> eng/jpn/deu/
fra/esp/ita, defaulting to eng. That is the mechanism behind the EN/JP
build pairs in GP_TITLE and the <lang>.pak families -- a question the
corpus had described but never traced to its input.

Then the registry. Pulling every RegisterToFactory diagnostic string
binds 24 of the 29 GamePart ids to a C++ class, and five ids have no
registration site: 1, 2, 16, 18, 28. Id 1 is GP_ADVERTISE_DEMO, which
agrees with what I measured two iterations ago -- the attract loop is the
TITLE replaying ADV.wmv, not a transition into an advertise part. Marked
amber, not green: it is an argument from an error message, not from code.

Two things fall out for Q4: ids 3 and 4 are the same class
(GamePart_SaveLoad, one part with two ids), and the menu buttons' ids now
match the executable's own class names rather than a list of table names.
Still a name match, one level closer to the code.

What is still missing is the transitions themselves, and I say so: the
manifest gives the boot-side assets, config.ini the language, the
registry which parts exist. What decides to advance is in
GamePart_Title's code and that dig has not been started.
2026-08-28 18:49:19 +00:00
Sylpheed RE agent
935f7ecab8 re: one A skips the intro, and NEW GAME was never a hang
Two corpus claims died in one boot, and both were costing us.

Q9's last row: a movie IS skippable with a single A. One tap ~45s into
the boot put the title up at ~57s, against a ~193s no-input baseline
measured over three boots, and Canary's own keystroke counter went 3->4
so exactly one press was delivered. The skipped-to title is fully
functional -- it draws the PRESS A plate and a second A opens the main
menu. What actually breaks the boot is hammering: the 88-press run in the
traps doc. The scripts' "tapping breaks the title" comment is too broad
and costs every scripted boot two and a half minutes.

Q4's last row: A on NEW GAME does not hang. It opens DIFFICULTY
(EASY/NORMAL/HARD/BACK, focus on NORMAL), then SELECT DATA, and only then
does the guest throw -- at PC 0x82307128, which is inside sub_823070B0,
the cache-manager STL erase this corpus already documents and which has
nothing to do with the menu path. The screen sat unchanged for 90s
because it was a menu waiting for input from a loop that never pressed
anything. That is now a METHOD line: a screen that never changes is not
necessarily hung, and the fix is to look at it and press something.

Also METHOD: never run ps -ef in this container -- all three long-lived
processes carry the entire loop prompt as argv.
2026-08-28 18:45:59 +00:00
Sylpheed RE agent
1f822f8b30 re: the menu's sound events are named on the disc; the binding to them is
not

Q8. The cheapest thing nobody had tried was to look at the SE cue names,
and they are semantic: SE_UI_CURSOR (2), SE_UI_DECIDE (3), SE_UI_CANSEL
(4), SE_UI_IMPOSI (5, the error buzz), SE_UI_SUB_WIN_OPN/CLS, and
SE_UI_SPLASH_IN/OUT. That is exactly the move/confirm/back/error
vocabulary the question asks for, named by the authors after the EVENT
rather than the sound. 322 SE cues committed as reference data.

They all live in one bank: BANK_SE is a single field reading Static.slb,
and the disc-wide check agrees -- 0 of the 322 has an entry in FILES, the
5135-path list that names every voice, briefing and BGM bank.

I am calling the binding a NAME MATCH, not a measurement, and the page
says so in the same breath as it gives the table. It is a stronger name
match than Q4's GamePart ids -- these are the authors' own event names --
but nobody has watched the game emit cue 2 on a d-pad press, and this
container is muted against a dummy device so there is no audio path to
watch either.

One new negative, with its reach. Static.slb's 8 353 472 readable bytes
contain zero RIFF, zero seek and zero WAVE, scanned over the whole
buffer. The boundary marker that works for all 7 620 other banks is
absent, so the cue is named, the bank is named, and the wave inside it is
not locatable. The UI sound effects cannot be exported yet, and that is a
gap in the assets rather than in the naming. The named next step is
Pj_Silph.xgs -- the XACT project SETTINGS points at, which IS in
sound.pak at TOC 9454, though its 533 bytes carry no XGSF magic and its
region's phase says they are probably the previous bank's tail.
2026-08-28 18:33:58 +00:00
Sylpheed RE agent
e81dcad632 re: the boot intro and the attract loop are the same video, and the
new-game intro has a name

Q9, answered twice over -- once from the disc and once from frames I had
already captured and not used.

The movie manifest's first eight slots are the whole boot-side flow, and
the slot key is the role: LOGO1-4 -> logo1-4.wmv (not on the disc, which
is why the splash is a screen), ADVERTISE_MOVIE -> ADV.wmv, STAFF_ROLL ->
the credits reel, MS00A -> S00A.wmv, MS01A -> S01A.wmv. So the new-game
intro is S00A.wmv, decoded -- which is the half I could not test at
runtime, because A on NEW GAME hangs the emulator.

And there is no separate boot-intro slot: ADV.wmv IS the advertise movie,
the boot just plays it first. Confirmed independently by matching 19
attract frames against five candidates -- 15 hit ADV with a playhead that
advances monotonically at the sampling rate and ends at 137 s, its full
length, with the title back on the next sample. So the attract movie
plays to its end; nothing cuts it short.

That corrects me. Two iterations ago I recorded the attract movie as
"~85 s, so probably not ADV.wmv" -- arithmetic on a start I never
observed, since sampling began 39 s in. REFUTED and METHOD both take it,
along with the matcher's real failure mode: a near-black frame has no
signature, and its runner-up is not evidence.

Skippability I did NOT settle, and the corpus contradicts itself: one
page says A skips a movie every time, while the boot harness deliberately
never taps during one because it breaks the title. Named the one-boot
test rather than picking a side.
2026-08-28 18:29:39 +00:00
Sylpheed RE agent
e64965b3f9 re: S1 -- the Ready Room probe is a no-go, and not for the reason the
question expected

One iteration, as the mission gates it, and no Ready Room work started.

The two criteria it named both pass. It is NOT 3D: the corpus already
holds a capture of the running screen and it is a flat six-item menu in
the same visual language as the title. And unrecoverable paths do not
mean unrenderable screens -- content enumeration finds 60 builds in the
1106 entries.

What kills it is something the question did not ask. GP_READY_ROOM.pak
does not appear to hold the Ready Room's own menu at all. Two signals
agree: the four largest builds composite to briefing panels -- a planet
readout, unit markers, direction arrows -- and, much harder, the 902
bundles declare 317 distinct element names of which NONE matches any of
the six labels on the screen (takeoff, brief, hangar, pilot, option,
mission, ready all return nothing). The pak is the briefing /
tactical-map content behind the BRIEFINGS item. If this is ever picked
up, the first job is finding where the menu lives -- GP_BUNK.pak is the
obvious place -- not naming this pak's keys.

The probe also reaches back into something already delivered: kind ==
0x3002 finds ZERO buttons across all 902 bundles here, while GP_TITLE has
16. 0x3002 is one member of a 0x3000 family with sub-bits, and this pak
uses 0x3000/0x3004/0x300c/0x3008. Nothing in the milestone changes --
every screen in scope is GP_TITLE -- but the handoff now says not to ship
that equality as a general button test.
2026-08-28 18:24:48 +00:00
Sylpheed RE agent
b3c8632006 re: a music bank is two stems that play together -- not intro+loop, and
not something to concatenate

Q10, and it starts by withdrawing the question's own premise. BGM_001 is
not three sub-waves of 10 KB / 4.47 MB / 4.67 MB: the 10 KB is the bank
header. A bank is exactly TWO waves, and across all 32 BGM banks on the
disc the two always have the SAME duration -- equal to 0.01 s over
lengths from 37 s to 277 s. That alone kills intro+loop and kills two
halves of one piece, both of which require unequal lengths.

Four banks appear to break the rule and do not: BGM_106-109 are the known
leading-region straddle, and the giveaway is that the entry named
BGM_107.slb contains BANK id=1108. The seek packet counts pin each join
exactly, so they realign to the same two-equal-waves shape rather than
being dropped as noise. That trap goes in METHOD.

Then the roles, by decoding both waves to PCM. They are
sample-synchronous: transient-envelope correlation searched over +/-5 s
peaks at lag +0.00 s, and both waves stop at the same millisecond,
167.663 s. Two stems of one performance, meant to sound at once. Wave 1
is quieter, has almost no bass and is far more L/R-decorrelated, which
reads as a surround-rear pair or a second intensity layer -- I cannot
separate those two from the file, and say so: ChannelMask is 0x0002 on
both, and this game's channel metadata is already documented as
meaningless.

Two things the port needs that are NOT on the disc, both marked as
authored: the track is not a seamless loop (BGM_001 fades out and is
followed by 6.15 s of silence, no loop-point field found), and nothing
names which bank the menu plays -- all 32 BGM cues are numeric.
2026-08-28 18:13:10 +00:00
Sylpheed RE agent
7fef19b3a0 re: the transition between screens is a fade through black, and most of
its timing is on the disc

Q7. Every title-side screen carries a full-screen black .prm quad that
paints last, and its keyframe group IS the transition: black at T0, clear
by T1, clear until T2, then back to black on exit. Read with the corpus's
start-of-a-ramp rule and Q1's time unit that gives 0.87s for EXTRAS,
0.97s for the main menu, 4.08s for the title -- from the file, not from a
stopwatch.

The disc-wide check is per-pak all-or-nothing rather than the 41% the
headline count suggests, and GP_TITLE's 6 of 12 is the useful row: the
six builds carrying a fade quad are exactly the six SCREENS, and the six
without are exactly the six overlays. GP_DIALOG is 0 of 133. That is
independent corroboration of the overlay finding from two iterations ago.

One piece is NOT on the disc and says so: the fade-OUT length. The fourth
keyframe has no time slot, because a group's last block stops four bytes
short. Measured instead, at 30fps, ~0.4s and the same both directions.

And a warning I earned: the luminance rise after a transition is NOT the
quad's ramp. The incoming screen's own elements animate in after the quad
has cleared -- 1.47s observed against a declared 0.97s. Time the fade
from where the frame is pure black.

Rig: screenshot samples at 0.5 Hz and cannot see a 0.4s fade at all,
which is why an earlier burst called this an instant cut. ffmpeg x11grab
at 30fps instead; both go in METHOD.
2026-08-28 18:05:19 +00:00
Sylpheed RE agent
4e745c8177 re: the title menu wraps -- Q5 measured, Q4 driven, and one of my own
method lines withdrawn

Q5, measured off two boots: up/down move one item and WRAP at both ends
on the 5-item main menu and the 3-item EXTRAS alike; left/right do
nothing; B goes up one level and restores focus to the item you came
from (4/4); B on the main menu returns to the title; B on the title does
nothing. The menu opens on TUTORIAL -- the middle item -- 2/2, though a
third recorded run implies NEW GAME, so that one is reproducible rather
than invariant and says so.

Q4 by driving: LOAD GAME opens the save-slot list, TUTORIAL the lesson
list, OPTIONS the settings menu, EXTRAS build 6, MISSION SELECT the stage
list. NEW GAME is not tested -- A on it hangs the emulator and this
iteration needed the session. The GamePart ID behind each is NOT
measured: it is the entry of the decoded id table whose name matches the
screen I saw, and the page says so rather than wearing the badge.

And the withdrawal. Last iteration I wrote that these menus drop d-pad
presses shorter than ~0.3s. They do not. Once wrap is measured, every
press count I had is exactly right -- four presses moved four steps
THROUGH the bottom, which lands one above where a non-wrapping menu
would. I invented hardware flakiness instead of testing the ends of the
list. METHOD keeps the withdrawal rather than deleting the line.

Also: label brightness is not a cursor oracle here -- the background art
outshines the highlight on some rows. menu_focus.py reads the focus ring
instead, 254 vs <82, no tuning.
2026-08-28 17:53:46 +00:00
Sylpheed RE agent
bebdb18a02 re: Q3 was already answered in docs/re -- deliver it, and test it on EXTRAS
The handoff had Q3 as "runtime-solved only". It is not: the layer key at
+0x0A of the T8aD sprite header, stable-sorted, is a file-derivable paint
order, already checked against five measured orders and already driving
the compositor for every build on the disc. That answer had simply never
reached the page the port agent reads.

The new evidence is EXTRAS -- the first screen composited from the rule
alone and scored against a framebuffer capture the rule had never seen.
0.9620 at zero shift, against a same-tooling control of 0.9657 on the
screen the rule was fitted to.

Written down with its limit rather than its headline: align_to_capture
correlates edges, and a paint-order change moves blends, not edges, by
=<45/255 on a few per cent of pixels. So the score is evidence for
placement and only CONSISTENT with the order. The sharp A/B needs a
rebuild and is named in the doc instead of being claimed.
2026-08-28 17:39:29 +00:00
Sylpheed RE agent
88b3ce9af5 re: which GP_TITLE build is which screen, measured against the game
Q2. The archive is eight screens shipped twice, English and Japanese --
not the "build 4 title, 5 main menu, 6/8/9 submenus" the handoff claimed.
Build 8 is the JAPANESE main menu; 6 and 9 are the EN and JP EXTRAS, and
EXTRAS is the only submenu GP_TITLE holds. The PRESS (A) BUTTON plate is
its own build (2/3), composited over the title art and faded in a beat
later, not a state of build 4.

Confirmed by booting to the main menu and walking it: title, PRESS (A),
main menu and EXTRAS each match their render element for element. Builds
0/1 and 10/11 -- a DELTASABER / SYLPHEED A.I. plate -- were looked for in
the whole boot filmstrip, every title-side screen and the attract loop,
and appear in none of them; the reach of that negative is written down
rather than filled in with a guess.

Two rig traps went into METHOD: the menus drop d-pad presses shorter than
~0.3 s, and a grab 2.5 s after a transition can catch a screen mid-fade
-- which nearly wrote "the returned title has no plate" into the corpus.
2026-08-28 16:55:33 +00:00
Sylpheed RE agent
27e8c51978 re: a keyframe time is worth half a rendered frame, and the ramp is linear
Q1 of the menu port, measured against the running game rather than reasoned
about. The developer-logo splash is the cheap target: it is the first thing the
guest draws and its bundle declares short, unambiguous ramps.

Two results, both frame-exact and both emulator-speed-independent (frame numbers
are VdSwap counts, the guest's own frames):

  * the ramp is LINEAR. A declared 15-unit fade lands on round(255*k/15) for all
    seven of its samples with zero error, k stepping 2,4,6,8,10,12,14. No ease
    can reproduce a constant step of 34 at both ends.
  * the animation clock advances 2.000 time units per submitted frame, over six
    consecutive intervals with no residual, with 1 unit as the quantum
    underneath (one frame in the fade-out advances by 1).

The conversion to seconds is one step further and is flagged as such: 300 frames
took 10.87 s = 27.6 present-frames/second, which reads as a 30 Hz title at 92 %
under the emulator and gives 1 unit = 1/60 s -- the title build 4.2 s, the main
menu build 1.1 s. That reading is not proven, because the rate was measured
while the guest was still streaming from the ISO; the page names the one test
that would settle it and says what changes if it goes the other way.

Committed beside it: the raw draw capture and the per-frame quad CSV, so the
numbers can be re-derived without a disc or an emulator.
2026-08-28 15:56:33 +00:00
Sylpheed RE agent
8b6dbcfead formats: move media assembly out of the viewer, where it could not be reused
Some checks failed
CI / Native — ubuntu-latest (push) Failing after 8m42s
CI / WASM — Web (push) Failing after 7m33s
CI / Formatting (push) Failing after 1m15s
CI / Native — windows-latest (push) Has been cancelled
CI / Native — macos-latest (push) Has been cancelled
The trickiest reading on the disc lived in the Bevy viewer: resolving a
cutscene's voice to a continuous byte REGION of the sound stream, because the
movie voices are one XMA stream chunked into VOICE_*.slb entries whose
boundaries do not match the cues -- a cue routinely spans two chunks, so a .slb
need not hold the track its name claims.

That put the logic most likely to be re-derived incorrectly in the crate least
likely to be reused. The Godot port's exporter needs the same answers, and there
must be one implementation of them.

New `sylpheed_formats::media` owns every case where the bytes of one playable
thing are not one archive entry: segment-spanning reads, multi-sub-wave banks,
and the voice-region resolution. Callers supply bytes through a `DiscSource`
trait, so the viewer keeps its ISO/directory abstraction and a headless consumer
gets `DirectorySource` for free.

The seam is deliberate: this module returns XMA RIFFs, not PCM. Decoding means
shelling out to FFmpeg, which is native-only and a policy decision for the
consumer -- everything up to "here are the bytes that belong together" is disc
knowledge, everything after it is a codec choice.

The four moved functions were previously untested; `tests/media_disc.rs` now
pins them, including the negative the corpus paid for -- an unbound movie must
stay unvoiced rather than borrow a neighbour's clip, which was tried and played
the WRONG recording.

The algorithm is unchanged, moved verbatim (same window sizes, same fallbacks).
The new disc tests pass; the broader audio suite was not re-run in this pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 17:49:33 +02:00
Sylpheed RE agent
69291a315f port: add Q10 -- what a music bank's sub-waves actually are
Some checks failed
CI / Native — windows-latest (push) Has been cancelled
CI / WASM — Web (push) Has been cancelled
CI / Formatting (push) Has been cancelled
CI / Native — macos-latest (push) Has been cancelled
CI / Native — ubuntu-latest (push) Has been cancelled
BGM_001.slb is three sub-waves (10 KB, 4.47 MB, 4.67 MB) and the decoder
concatenates them into one 347 s track. That is a default nobody chose, not a
decision: two near-equal halves could be intro + loop, two variations, or two
halves of one piece, and a menu that loops its music needs to know which.

Found while wiring the Audio Library up to the shared banks. Recorded in
HANDOFF.md as a trap too, so the port does not build looping on top of the
concatenated track before the question is answered.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 17:43:25 +02:00
Sylpheed RE agent
4734797e50 agent: stop the launcher claiming it cannot push when it can
Some checks failed
CI / Native — ubuntu-latest (push) Failing after 8m23s
CI / WASM — Web (push) Failing after 7m9s
CI / Formatting (push) Failing after 1m12s
CI / Native — macos-latest (push) Has been cancelled
CI / Native — windows-latest (push) Has been cancelled
The `loose` footer announced "cannot push -- no git credentials are mounted"
unconditionally. It was written before the credential mount existed and then
went stale, so with credentials correctly mounted it told the operator their
work would be lost -- the exact failure the mount was added to prevent.

It now reports what is actually true, and keeps the real warning for the case
where the file genuinely is missing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 17:22:02 +02:00
Sylpheed RE agent
ecbd70f69a port: the container agent answers the questions, it does not build the port
Some checks failed
CI / Native — macos-latest (push) Has been cancelled
CI / Native — windows-latest (push) Has been cancelled
CI / WASM — Web (push) Has been cancelled
CI / Formatting (push) Has been cancelled
CI / Native — ubuntu-latest (push) Has been cancelled
Corrects the split I got wrong in b5a1938. The container is for reverse
engineering, now focused on the menu port; a SEPARATE agent builds the port from
its investigation results. My previous version had the container agent writing
the exporter and the Godot project, which is not the intent.

This lands on the research/engineering line that was already in the estimate:
the container agent takes the research half, the port agent the engineering half.

docs/port/MISSION.md is now a list of open QUESTIONS (Q1-Q9) rather than build
milestones, ordered by what blocks the port earliest -- the keyframe time unit,
which build is which screen state, paint order for the six screens, button ->
GamePart, navigation semantics, the boot sequence driver, transitions, menu
audio bindings, and video binding. Each is done when a written result with
evidence exists, not when something compiles. S1, the Ready Room probe, stays
gated at one iteration and a go/no-go.

Most of these are BEHAVIOUR questions -- timing, transitions, what a d-pad press
does at the end of a list -- so the mission and the loop prompt both push hard
on measuring the oracle rather than reasoning from the file.

docs/port/FORMAT.md is deleted. The export schema is the port agent's design and
was not mine to specify. It is replaced by docs/port/HANDOFF.md, the single page
the port agent reads: a status table, what is already settled and can be relied
on today, and the facts that will trip the port up (the WMV3/WMA Pro intro, the
Static.slb size over-declaration, the voice-vs-music downmix, JNGL_001).

The derived/authored idea survives as the thing it always was -- a finding, not
a design. Every answer must be classified DECODED, MEASURED or UNDECODABLE-with-
reach, and never a fourth thing, because measured and undecodable both mean the
port agent is authoring that value and has to know it. Labelling a guess as a
decode would put it into the port wearing the badge of a measurement.

Reverts the Godot install from the RE container, its AGENT.md section, and the
export/ gitignore entry -- none of that belongs on this side of the wall.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 17:16:47 +02:00
Sylpheed RE agent
b5a193839c port: make the Godot menu shell the agent's primary objective
Sets a new mission: boot the real disc through developer splash -> intro video
-> title -> main menu -> submenus in Godot 4, interactively, with no gameplay,
no 3D and no emulator.

docs/port/MISSION.md defines it -- eight gated milestones, each finished by an
ARTIFACT rather than by compiling, plus the Ready Room as an explicitly gated
stretch goal with a one-iteration probe that decides go/no-go. GP_READY_ROOM is
1106 entries with 6 recoverable names and is ISL-scripted, so it is either a
week or a quarter, and the agent must not start it on its own authority.

Architecture, per the user's decision: the Godot project is INDEPENDENT of the
Rust viewer and never reads a disc format. An offline Rust exporter converts the
disc into open formats; Godot reads only those. No GDExtension, no Rust in the
Godot project, and sylpheed-viewer is off limits -- it stays the human's
verification tool with its static-data rule intact.

docs/port/FORMAT.md specifies the open format, versioned, because modding is the
port's second goal and that makes the layout a deliverable rather than a temp
directory: JSON over XML (Godot parses JSON natively; its XMLParser is SAX),
names never hashes, provenance in every generated file, and unknowns listed
rather than guessed.

The discipline the whole thing rests on is the derived/authored split. `export/`
is regenerated wholesale and never hand-edited; `authored/` is hand-written and
survives a re-export. Three things this milestone needs are NOT on the disc in
any decoded form -- which button does what, paint order, and menu sound cues --
so they live in `authored/` with a stated `why`. Deleting an authored entry
because the exporter can now emit it IS the measure of progress.

`export/` is gitignored: it is generated from the user's own disc and this stays
a clean-room repo.

Container: adds a pinned Godot 4 (windowed under Xvfb for screenshots, plus a
headless wrapper). ffmpeg already carries libtheora, which is the video target --
Godot 4 plays only Ogg Theora natively and the disc's ADV.wmv is WMV3/WMA Pro.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 17:11:35 +02:00
Sylpheed RE agent
5cdff5e515 docs/re: the disc atlas -- one map of how the assets reference each other
A reference page covering the four layers (disc media, IPFB archives, container
formats, domain chains) and the three hashes that join them: name_hash for
archive TOC keys (case-insensitive), tag_hash for IDXD records and fields
(case-sensitive), ixud_hash for localised text.

Counts were measured on the retail extract for this page rather than copied
forward from the corpus -- 41 archives, 26443 entries, 166 xpr, 97 wmv -- and
where they overlap with docs/re they reproduce it independently (104 cutscene
slots / 101 movies / 99 / 99 / 22).

It gives the dangling references their own section, because they are properties
of the shipped disc and a port has to survive them: pmbase.t32, SUBTITLE_S12B,
the four boot logos, GP_TEST, the six 2D paks at exactly 0% named, and the 42
provably unrecoverable ISL symbol keys.

Also published as an artifact for reading.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 16:51:45 +02:00
Sylpheed RE agent
77cd58202b viewer: a Cutscenes browser -- the manifest was invisible plumbing
`movie_manifest` has been parsed since the movie-voice work and rendered
nowhere: it resolved a voice bank and that was all. So the only way to find a
cutscene was to hunt `.wmv` files in the ISO tree, where nothing tells you which
mission a file belongs to, whether it has subtitles, or what is said in it.

View ▸ Cutscenes lists all 104 manifest slots with mission/phase, kind, movie,
subtitle track, voice token and telop, and -- the part that needed no new
parsing, only a route -- resolves the captions to a readable TRANSCRIPT with a
language selector. Subtitles were previously burned into the video during
playback and reachable no other way.

Three negatives are shown rather than smoothed over:

* 5 manifest-bound movies have no `.wmv` (logo1-4 and an encoder test clip).
  They are marked and get no Play button instead of one that would fail.
* 9 of 101 movies resolve no English transcript.
* the `.prt` telop overlay is named by the manifest and we have no parser, so
  the reference is shown labelled "not decoded" rather than omitted.

`cutscene_catalog_binds_movies_and_transcripts` pins all of it against the disc
-- 104/101/99/99/22, the exact absent-movie list, 92 transcripts -- because a
browser that quietly dropped these would look complete and be wrong. The counts
independently reproduce docs/re/movie-subtitle-link.md.

Play routes through the normal FileSelected path, so the existing video player
handles it exactly as it would from the tree.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 16:47:46 +02:00
Sylpheed RE agent
306a8a5661 viewer: open the whole sound bank, not just the voice half
The library enumerator kept only names containing VOICE or \Briefing\, and read
eng\sounds.tbl unconditionally. So the Explorer could reach 4382 of the 9519
banks in sound.pak: no music, no jingles, no sound effects, and no Japanese
voice at all -- roughly half the disc's audio had no route to the UI.

`slb::list_audio_entries` now returns every named bank with the category its
path implies (Music / Jingles / Sound effects / Radio / Dialogue / Movie voice /
Briefing). `list_voice_clips` is that, restricted to the spoken categories, so
its existing test still guards the old behaviour. The 36 root banks carry no
language component and appear whichever table is read; the window gets an
English/Japanese switch that re-reads the other sounds.tbl, since the table name
IS the selector.

Two defects the decode found, both recorded in
docs/re/structures/sound-pak-contents.md:

* `Static.slb` -- the SFX bank -- declares 616768 bytes more than sound.p04
  holds. Not our extraction: p04 matches the ISO's own directory record, and a
  sweep of every pak on the disc finds this one entry over-running and no other.
  It is the highest-offset entry, so its comp_size is an allocation size. A
  short read is now allowed for the tail entry ONLY; any other overrun stays an
  error, because clamping it would hide real damage behind a half-decoded asset.
  The bank went from unreadable to 514 s of audio.

* the left-channel downmix was applied to everything. Right for voice (mono
  content however stored), wrong for music (a real stereo mix, half of it
  discarded). The caller now decides from the category.

35 of the 36 shared banks decode; JNGL_001 does not, and says so in the player
instead of the panel silently closing. Its payload is not a whole number of XMA1
packets from any known data offset, so it is likely not a plain headerless
stream -- written up rather than papered over.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 16:28:20 +02:00
Sylpheed RE agent
fb70511242 viewer: make the two screen presenters agree, and expose the compose options
The UI Screens window enumerated with `is_build` while the PAK browser's
inline preview composes anything `parse_build` accepts. So the browser drew
screens this window flatly refused to list -- most visibly `palogo`, the
publisher splash, which declares its sprites directly and has no `.rat`
layout child.

`compose_screen` now gates on `is_composable`, the documented superset (every
`is_build` bundle passes it), so anything the list offers is drawable and the
two presenters share one rule. A "Fragments" toggle widens the enumeration to
it as well, off by default: the extra ~1786 bundles are mostly two-element
fragments (a button beside its glow) that would bury the real screens. It
re-scans, which renumbers the display ordinal -- harmless, because the pak
ENTRY index is the locator and the stale catalog is cleared.

The toggle sets a `rescan` flag rather than self-sending RequestScreenCatalog:
a system that both reads and writes one event type is a B0002 panic at
startup, which is how the Save browser broke.

Also plumbs the last two ComposeOptions the CLI had and the UI did not --
`black backdrop` (what a framebuffer capture must be compared against) and
`primitives` (decoded, but paint order unsolved; hover text says so).

Verified: `screen render --all --build 13 GP_TITLE.pak` draws the SQUARE ENIX
splash, 2/3 elements, the third being the .prm quad. Workspace builds, viewer
reaches steady state, disc-gated suite 20/20 green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 16:14:23 +02:00
Sylpheed RE agent
47e4e43310 viewer: stop the UI Screens browser loading forever
Three compounding causes, found by tracing every place a `loading` flag is set
against every place it is cleared.

THE COST. `compose_screen` inflated EVERY entry in the pak and held every build
in memory at once, purely to index into the result -- then did it again on every
checkbox click, re-opening the ISO from scratch each time. But the catalog
already records each build's pak ENTRY INDEX, so the locator was there all
along: compose now reads exactly one entry. `RequestScreenCompose` carries
`entry` (the locator) beside `build` (the display ordinal).

`build_screen_catalog` had no budget at all, while the pak browser has capped
exactly this work since it was written. It now skips oversized entries and stops
at a ceiling.

That ceiling is 384 MB, not the pak browser's 64 MB, and the difference is the
point: GP_HANGAR_ARSENAL inflates past 160 MB and holds ~390 builds, so a 64 MB
cap would have quietly hidden most of them -- trading a hang for a wrong answer.
When the ceiling IS hit the pak is marked `truncated` and the UI says so.

THE LATCH. `poll_loader_channel` treated a disconnected channel exactly like an
empty one, so if a worker died every in-flight spinner stayed up for the life of
the process -- and the `if loading { return }` guard at the top of each handler
then refused every retry. Disconnect now clears the flags and reports it.

Verified: the workspace builds, the viewer reaches steady state, and one
composite still draws 11/11 elements of the tutorial pause menu.
2026-08-28 15:59:00 +02:00
Sylpheed RE agent
1054d43676 docs/re: finish the INDEX merge I left unresolved
Commit 03ece95 ("Merge branch 'auto/idxd-unnamed-keys'") shipped INDEX.md with
live conflict markers. My check was worthless in a way worth naming: I ran

  git add docs/re/INDEX.md ; git status --short | grep -E '^(UU|AA)'

in that order. Staging a conflicted file is exactly what marks it resolved, so
the grep found nothing and reported "no unmerged" -- the control passed BECAUSE
of the action it was meant to gate. Both halves of the hunk then went into the
tree verbatim, leaving two competing IDXD rows.

Resolved as a union, which is what it always was:
  * HEAD's "IDXD object/table" row -- the 2026-08-25 record/index-region decode
    -- supersedes the branch's older one-line version of the same row;
  * the branch's "IDXD nameless field keys" row is new and is kept.
2026-08-28 15:55:18 +02:00
Sylpheed RE agent
e1dcc689bc agent: move the loop prompt's memory into the corpus, and let it push
The autonomous agent's loop prompt had grown to ~9 000 words of accumulated
findings, refutations and traps. That is a symptom, not a style: it was doing
the job the corpus should do, in the one place that dies with the container.
Three concrete failures followed from it, and each gets a structural fix rather
than a louder instruction.

1. IT REPEATED WORK IT HAD ALREADY DONE.
   The "do not revive" list and the method traps existed ONLY in the prompt, so
   nothing pointed a fresh iteration at them. Extracted verbatim into two
   tracked files:

     docs/re/REFUTED.md  105 claims tested and dead, grouped by subject so a
                         grep for your noun finds the neighbourhood
     docs/re/METHOD.md   the traps already paid for -- controls, inference,
                         searching, reading data, runtime

   Both are linked from INDEX.md, and the loop prompt now opens by requiring
   them to be read. This is the fix for "re-derived something already known":
   the knowledge is now where the next iteration looks, not in a context window.

2. IT FORGOT TO ARM THE NEXT WAKEUP.
   The prompt tried to solve this by shouting at itself in the first line. The
   real fix is to stop asking: `loose` now defaults to a FIXED interval (45m),
   so the harness owns the cadence and a forgotten ScheduleWakeup cannot end the
   run. SYLPH_LOOP_INTERVAL= (empty) restores self-pacing.

3. IT COULD NOT PUBLISH, SO THE WORK ONLY EXISTED IN THE CONTAINER.
   New `push-work`, plus a read-only credentials mount
   (SYLPH_GIT_CREDENTIALS, default ~/.sylph-git-credentials). It pushes the
   CURRENT branch only, refuses anything that is not auto/*, and never
   force-pushes -- so the consolidated line stays a human's decision and a
   confused iteration cannot rewrite history. The loop prompt now requires a
   push on every iteration that commits, rather than at the end of some longer
   arc, which is exactly when a container dies.

The prompt itself drops from ~9 000 words to 85 lines and, more importantly,
stops accumulating: findings go in the corpus, and the prompt points at it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 15:46:59 +02:00
Sylpheed RE agent
03ece95c06 Merge branch 'auto/idxd-unnamed-keys'
# Conflicts:
#	docs/re/INDEX.md
2026-08-28 15:27:51 +02:00
Sylpheed RE agent
1fce3f9c71 Merge branch 'auto/slb-loader'
# Conflicts:
#	docs/re/INDEX.md
#	docs/re/structures/slb-data-offset.md
2026-08-28 15:27:46 +02:00
Sylpheed RE agent
8124b85fc7 Merge branch 'auto/regn-reader'
# Conflicts:
#	docs/re/BACKLOG.md
#	docs/re/INDEX.md
#	docs/re/structures/regn-map-grid.md
#	tools/re-capture/regn_decode.py
2026-08-28 15:26:35 +02:00
Sylpheed RE agent
245b73243b Merge branch 'auto/isl-builtins-26-28-29'
# Conflicts:
#	docs/re/INDEX.md
#	tools/re-capture/isl.py
2026-08-28 15:24:55 +02:00
Sylpheed RE agent
35538e3b5b re: the Time/Points record is not on the debriefing path - and a ring drainer matches the message bus
Read 0x821A0AD0, the debriefing, end to end, resolving every call and string in
code order. The word-B bit-setter sub_8219F2E8 is its FIRST call, before anything
else; the function makes 44 calls and references exactly two strings, DEBRIEFING
and BASE_INFO.

No Time and no Points anywhere on this path - not in the debriefing, not in the
bit-setter. Section 5.3's expectation that clearing a challenge stage records its
metric alongside the bit is not supported by the code that clears it. The reach of
that negative: the debriefing function and its directly-called bit-setter, both
read whole; it does not rule out a store deeper inside one of the other 43
callees.

0x8219F670 turns out to be an online-state predicate: 88 bytes, no strings, reads
the singleton, tests bit 1 of [0]+60, and if set passes [0]+64 to 0x824A9C90 and
returns whether the result is 2. A connection or sign-in check, not a record
store.

The useful find is 0x8219F460, 368 bytes and self-recursive, which drains a ring
based at r29+320 with element base +4, capacity +8, head +12 and count +16. Those
are the corpus's documented message-bus offsets exactly - BACKLOG records
sub_82175C20(bus+4, &msg) as a ring-buffer append with capacity +8, head +12,
count +16 - and sub_82175C20 is the very notifier section 5.6 found the word-B
writer calling with bit + 64. So the open question "ring-buffer PUSH, no handler"
now has a candidate drainer with a byte-for-byte matching field layout, reached
from the debriefing.

What is NOT shown is that these are the same ring instance. The push targets
bus+4, this drain targets r29+320. Matching offsets prove a shared container
shape, not a shared object, and the identity needs the two base pointers tied
together.

All seventeen artefacts byte-identical.
2026-08-28 13:07:02 +00:00
Sylpheed RE agent
f7a0cf47b1 re: progress word B's writer is sub_8219F2E8 - it sets bit (stage - 24)
Section 5.5 narrowed the search to 21 functions but could not name one, because
the scan assumed the copy destination was an r1-relative stack local. It is not.
Reading the known word-A writer instead of guessing its shape shows a frame
register: bl singleton, addi r3,r31,96 (dest), addi r4,r11,80 (src = obj+80), bl
copier, stw r10,96(r31). That one wrong assumption made the first scan return 0
for all 21, the known-good included - the third filter this week killed by its own
control.

With the real shape - addi r3,rF,K before the copier, then stores at K+d on rF -
word B sits at d = 1956-80 = 1876, and exactly one of the 21 hits: sub_8219F2E8,
copy dest r31+560, storing 2436(r31), and 560+1876 = 2436. Control: the word-A
writer 0x821C1630 stores at relative +0, i.e. word A.

The proof is the bias. Word B's writer computes addi r11,r11,-24 then 1 <<
(index-24) before or-ing into the copy; word A's writer shifts by the raw stage
number with no bias. That reproduces section 5.1's read-side rule - n >= 24 means
bit (n-24) of word B - on the write side, so word B's bits 0-5 are challenge
stages 24-29, the same six that 5.4's MISSION_ID names. The two writers are exact
complements.

It has exactly one real caller, 0x821A0B1C inside 0x821A0AD0, whose strings are
BASE_INFO and DEBRIEFING - so clearing a challenge mission is recorded from the
debriefing screen.

A second bitfield falls out of the same function: it reads the copy at +1880
(2440(r31) = obj+1960, the word after B), re-reads obj+1960 after the setter, xors
old against new, and loops the 32 result bits calling 0x82175C20 with bit + 64. So
obj+1960 is a second 32-bit progress bitfield with ids 64-95, whose newly-set bits
are announced one at a time - an unlock-notification loop.

Against 5.3's own prediction: neither sub_8219F2E8 nor its caller references Time
or Points, 0 for both. 5.3 expected word B's writer to also store the stage's
Time/Points record. It does not - this is the bit-setter only, and the record
store is still unlocated.

All seventeen artefacts byte-identical.
2026-08-28 12:50:16 +00:00
Sylpheed RE agent
a41a6b9783 re: progress word B - search space closed at 21 functions, the field itself still unwritten
Section 5.3 left word B (+1956) with no known writer. Three routes ran this turn:
one produced a population, two were refuted, and the field is still unwritten.

The offset route has power here, unlike +184. +1956 is a large unusual offset with
9 stores and 28 loads image-wide, against +80's 5403 and 6923. Control: the known
reader 0x821898C4 sits in the gate's list-builder 0x82189870, exactly where 5.1
puts it. But none of the 9 stores turns out to be a progress write - filtering by
whether the storing function reaches the progress object gives 0 of 8 distinct
functions, and the 9th stores through r1, the stack pointer, so it is a local.
+1956 is simply an offset several unrelated structures also use.

The obvious filter is dead, and it fails its own control. Testing "does the
function reference the singleton global 0x828F48B0 or call 0x821707C0" against the
KNOWN word-A writer returns no contact, because that path reaches the progress
object through the copier and never through the global. A filter that rejects the
known-good answer proves nothing about the others.

What did work: progress is only ever changed copy-modify-store, so every writer
must call both the copier 0x82175110 and the setter 0x8216FF70. Measured, 21
functions call the setter, 21 call the copier, and 21 call both - the two caller
sets are the same set. That is the complete progress-write population and it is
small enough to read exhaustively. Control: the word-A writer is among them as
0x821C1630 (BASE_INFO + DIFFICULTY), and 5.2's store 0x821C1820 lies inside it -
the doc's 0x821C09D8-0x821C29F0 is the enclosing method, 0x821C1630 the function.

No member of the 21 stores to +1956 directly. Their only large store offsets are
into this - 1004, 1980, 2040, 2100, 2199, 2436 - with 1004 being 5.2's own
this+1004 & 0x20000 guard. So word B is written through the stack copy, at
localbase + 1876, which no offset scan can pick out.

One false friend worth recording: two of the 21 reference Points, which looks
exactly like the challenge record-storing path 5.3 predicts. It is not - their full
string sets are Dependency, MissionObjective, Points, WEAPON, WEAPONS, i.e. the
arsenal development economy, and neither references Time. There are two Points
vocabularies, development points and the leaderboard metric.

Next candidate, not yet checked: 0x8218EFE0 is the one progress writer whose
strings are BASE_EXTRA and DIFFICULTY - the only member of the 21 carrying the
EXTRA vocabulary.

All seventeen artefacts byte-identical.
2026-08-28 12:33:32 +00:00
Sylpheed RE agent
3b80779180 re: which challenge stage is which leaderboard mode - CLOSED from the record's own fields
challenge-mission-gate.md section 5.3 left a marked 🟡 on the exact pairing and named
the right route: read the record's binary index section rather than the string pool.
Running that with the corpus's own reader settles all six missions at once.

  TimeAttack    MISSION_ID 24  REQUIREMENT 16  RECORD_TYPE Time
  ScoreAttack   MISSION_ID 25  REQUIREMENT 24  RECORD_TYPE Points
  Extra01       MISSION_ID 26  REQUIREMENT 25  RECORD_TYPE Points
  Extra02       MISSION_ID 27  REQUIREMENT 26  RECORD_TYPE Time
  Extra03       MISSION_ID 28  REQUIREMENT 27  RECORD_TYPE Time
  Extra04       MISSION_ID 29  REQUIREMENT 28  RECORD_TYPE Points

Control: the table appears in 6 pak entries, all in tables.pak, and all six are the
same table byte for byte - one per language, i.e. one user under the per-pak copy
rule. Zero variants.

Three things fall out, none of them assumed. MISSION_ID is the stage number: the six
IDs are 24-29 in slot order, exactly the six challenge stages S24-S29, bijectively -
so the "four Extra0n modes against five _EXn stages" worry dissolves, because _EXn is
a unit-variant suffix that indexes nothing here. REQUIREMENT is a strict chain, 16,
24, 25, 26, 27, 28, each mission requiring the previous one's stage with TimeAttack
anchored on stage 16, the final story mission; 5.3 inferred a chain from the shape of
the string pool, and it is now exact and branchless. RECORD_TYPE is the leaderboard
metric and it is per-stage, not per-name: 3 Time (S24, S27, S28) and 3 Points (S25,
S26, S29), so the tempting reading that the Extra0n family shares one metric is
wrong - Extra01 and Extra04 are Points while Extra02 and Extra03 are Time.

Withdrawn from 5.3: the string-pool reading listed the numeric tokens as 16, 25, 26,
27, 29 with 24 and 28 deduped away. The real REQUIREMENT set is 16, 24, 25, 26, 27,
28 - 29 is a MISSION_ID, never a requirement. A value pulled from a deduped pool by
adjacency got one member wrong; the indexed read has no such failure mode.

Left as an association rather than a proof: RECORD_TYPE takes exactly the two values
Points and Time, and result-screens names two ranking screens POINT_RANKING and
CLEAR_TIME_RANKING, but no code path has been shown selecting one screen from that
field.

All seventeen artefacts byte-identical.
2026-08-28 12:15:48 +00:00
Sylpheed RE agent
0ace5db7c7 re: the 95 BASE_INFO screens partitioned - BASE_INFO discriminates screen-config from table-read
Item (a): mine the screen population found last iteration. Resolving every one's
key list via the string-xref join over its own body gives 298 distinct keys and a
three-way split: 95 = 9 corpus-known + 48 new with a readable key list + 38 bare,
where "bare" means BASE_INFO and nothing else. The bare 38 are the honest limit
of the instrument - a screen whose only catalogued string is BASE_INFO cannot be
named this way.

The control turned the finding into a law. Only 9 of the 19 loaders the corpus
documents appear among the 95, and which nine is not arbitrary. The nine that
reference BASE_INFO all read a SCREEN's own configuration: debriefing, career,
save, medals, evaluation list, gamma, POINT_RANKING, CLEAR_TIME_RANKING, hangar.
The ten that do not all read a DATA TABLE: unit definitions, weapon datasheet,
AIParams, stage settings, chatter, PlayerParams, ControlTweak, the PG* HUD
roster, material slots, and the leaderboard. 9/9 and 10/10, both off-diagonals
empty. The leaderboard falling on the table side is a check rather than an
exception - its own doc records that all 18 of its names are tables.pak names, so
it is a table reader that happens to paint a screen. That exclusion is what last
commit's "universal screen marker" claim was missing.

48 new screens are named by their most distinctive keys, including the title
screen (sub_821C4EB0: TITLE_SCREEN/TITLE_MENU/LOADING), the ready room
(sub_821D6D28: SCRIPT_ID/SCRIPT_PATH, confirmed independently by its own error
string silph::GamePart_ReadyRoom::Impl::PrepareScript), movie subtitles, the
three menu modes STANDARD/EXTRA/TUTORIAL, the speaker window, loadout and gallery
views, mission select, and an equip screen carrying the game's own typo
EQUIIP_LIMITATION.

One caution worth recording: a high key count is not a rich screen. The two
27-key entries sub_82297550 and sub_822A2F00 resolve to pure coordinate pairs
(1080,163 / 734,502 / 1164,403) - a layout table, not configuration. Read the
values before trusting the rank.

Still open: the 38 bare screens, and whether the EX_ in EX_MENU_DISABLE_SKIP and
EX_LETTERS is the same EXTRA mode word as +184.

All seventeen artefacts byte-identical.
2026-08-28 11:58:51 +00:00
Sylpheed RE agent
80406ce3e4 re: the RTTI route is dead - BASE_INFO enumerates 95 screens instead
Item (a): can the vtable scan name the corpus's anonymous classes?

The route I proposed last turn - "the RTTI behind those two anon classes" - does
not exist. Of all 1150 catalogued vtables, 1150 carry an ANON_ class name, 0 have
rtti_present set, and 0 have a non-empty base_classes_json. Not one named class,
not one RTTI flag, not one base-class edge in the whole table. Withdrawn as
unrunnable; the ANON_Class_XXXXXXXX labels are synthesised, not recovered.

The sibling methods cannot name a class either: slots 0, 2, 3 and 4 of both EX_
classes reference zero strings. Every naming datum sits in slot 1, the method the
corpus already had.

Re-verified at the true addresses (0x820A9760 and 0x820A9350) that sub_822814D8
and sub_8227A3A0 are each slot 1 uniquely, and sub_82286BC8 is in no vtable.

What the shape DID give: both classes are 5-slot with an exactly-88-byte slot 0.
83 vtables have 5 slots, 16 also have the 88-byte slot 0, 13 of those have a
catalogued slot 1, and 10 build a key list. Ranked by key count the two known
screens sit at the top and the rest name themselves - a medals screen
(MEDALS/ACHIEVED_MEDALS/NEW_MEDAL), an evaluation list (EVALUATION_LIST plus
eval_%02d_eval / eval_%02d_points), the gamma screen (GAMMA_RGB/GAMMA_WB),
POINT_RANKING, CLEAR_TIME_RANKING, and three bare ones.

POINT_RANKING and CLEAR_TIME_RANKING being two separate screens is a direct lead
for the open item "which challenge stage is which leaderboard mode": the
leaderboard is ranked two ways and each way has its own class.

But the obvious follow-on claim - that BASE_INFO marks this family - is false,
and the control says so. BASE_INFO is a single string at 0x820A20EC referenced by
95 functions: 10 inside the family, 85 outside, including sub_82286BC8 (the save
screen, in no vtable at all) and sub_822AE628 (the hangar loader). So the
direction inverts: BASE_INFO is the universal first key of every screen key list,
a 95-function population, while the 5-slot/88-byte shape is only a weak filter
that catches 10 of them plus 6 non-screens. Enumerate screens by BASE_INFO, not
by vtable shape.

All seventeen artefacts byte-identical.
2026-08-28 11:41:52 +00:00
Sylpheed RE agent
141a65cb4a re: the ind_call damage is bounded - no corpus claim was contaminated
The protective sweep after last commit's refutation. The question was which
existing claims the ind_call cross product voids. Answer: none of the ones the
corpus makes.

Every caller-count claim in the corpus verifies against call edges, with zero
ind_call contamination:

  title-crash-stl-tree      sub_82457780  "one caller"     call=1   ind_call=0
  isl-trigger-queue         sub_8226EAB8  "16 callers"     call=16  ind_call=0
  isl-coroutine-spawner     sub_822737C8  "seven callers"  call=7   ind_call=0
  challenge-mission-gate    0x82175110    "22 callers"     call=22  ind_call=0

4 of 4 exact, and not one of the four carries a single ind_call edge.

There is a structural reason for that rather than luck. Partitioning the two
target populations: 1710 targets have ind_call edges, 12170 have call edges, 103
have BOTH, 1607 are ind_call-only, 12067 are call-only. And all 1710 ind_call
targets are vtable entries - 1710 of 1710, zero partials - against a control
where only 273 of the 12170 call targets are. So ind_call fires only on virtual
methods, and 1607 of them have no direct caller at all, which is exactly why the
two EX_ screens came back empty on call and flooded on ind_call.

The residual risk is narrow and nameable: the 103 targets carrying both kinds,
where a caller count taken without a kind filter would be inflated by about 633.
Everywhere else the two populations are disjoint, so a call-based count is safe
and an ind_call-based one is meaningless.

Rule for future work: always filter kind='call'; and when a function has no call
edges, that is a fact about it - it is a virtual method - not a gap to be filled
from ind_call.

All seventeen artefacts byte-identical.
2026-08-28 11:24:14 +00:00
Sylpheed RE agent
749567e835 re: the +184 writer is blocked - and xrefs.ind_call is a CROSS PRODUCT, not a call graph
Chasing what supplies the EX_ mode word. Two routes, both measured to have no
power, plus one structural fact that did come out.

The offset route is dead. stw ..., 184(rN) occurs 301 times in the image, lwz
from +184 occurs 351 times, and 115 functions touch both +144 and +184. +184 is
an ordinary small offset shared by many unrelated classes - the same shape the
corpus already recorded as failing three times. Nothing narrows to a writer.

The bigger result is an instrument refutation with corpus-wide reach. Asking
xrefs for the callers of the two screens returns 633 sources for EACH, and the
two lists are IDENTICAL, which cannot be right. Measuring the relation itself:

  ind_call rows                       1827297
  distinct targets                       1710
  distinct sources                       6992
  targets with EXACTLY 633 sources        236

236 different functions sharing an identical source count is the signature of an
unresolved-indirect-call cross product, not of a call graph. Any reading that
treats an ind_call edge as "X calls Y" is void, here and anywhere else in the
corpus it may have been used.

The control shows the other kinds are sound: sub_82286BC8 has exactly one
caller, kind call. And both EX_ screens have ZERO non-ind_call edges - they are
reached only through function pointers, which is why the direct graph is empty
for them.

What did come out: scanning the entries of all 1150 catalogued vtables in the
flat .pe, both EX_ screens are slot 1 of their own class - sub_822814D8 in
ANON_Class_271D5F25 and sub_8227A3A0 in ANON_Class_CAA8AD62 - while sub_82286BC8
is in no catalogued vtable at all. So the 2/0/0/1 partition from the previous
commit reflects a structural difference rather than a coincidence: the two
screens that select on the mode word are vtable methods of their own classes and
the one that does not select is not a vtable method.

Still not settled: what writes +184. Both the offset sweep and the call graph are
exhausted for it. A route with actual power would be the RTTI behind those two
anonymous classes, or a runtime watch on the field - not another static offset
search.

All seventeen artefacts byte-identical.
2026-08-28 11:08:18 +00:00
Sylpheed RE agent
35a35f753f re: the EX_ selection is shown - a mode word at +184 tested against 3
result-screens.md left this explicitly open: "the selection has not been shown".
It is shown now. Both screens that reference EX_ names carry the same
five-instruction idiom immediately before the choice - sub_822814D8 at 0x822815b4
and sub_8227A3A0 at 0x8227a408:

  lwz    r11, 4(rX)              the owning object
  lwz    r11, 184(r11)           a mode word at +184
  addi   r11, r11, -3
  cntlzw r11, r11
  rlwinm r11, r11, 27, 31, 31    r11 = 1 iff [+184] == 3
  cmpli  cr6, 0, r11, 0x0
  bc     12, 4*cr6+eq, <plain>   not 3 -> STAGE_RESULT / OVERVIEW
                                 else  -> EX_STAGE_RESULT / EX_OVERVIEW

cntlzw of zero is 32, and rotating left 27 lifts bit 5 into bit 31, so the
sequence is a branchless equals-3 test. The equal-to-zero branch takes the plain
name, so EX_ is chosen exactly when the word is 3.

A partition over the three key-list loaders the corpus already names:

                        loads +184   does not
  references EX_ names       2           0
  does not                   0           1

sub_822814D8 and sub_8227A3A0 both load it; sub_82286BC8, which references no EX_
name, has no +184 access at all. Both off-diagonal cells are empty.

3 is the game's own EXTRA. challenge-mission-gate already established the
vocabulary for this constant at a three-way switch: == 3 is EXTRA, == 5 or 6 is
CHALLENGE, otherwise FILE. So the prefix reads literally - EX_ is the EXTRA
variant.

That REFINES the doc's tentative reading rather than confirming it. The section
proposed "EX_ = the challenge-mission debriefing" by analogy with _EXn on asset
names; the measured constant is EXTRA, which in the game's own three-way
vocabulary is a DIFFERENT kind from CHALLENGE (5/6).

Boundary of what was measured: the EXTRA = 3 constant is established at +144 of
the stage-loader class, while the word tested here is at +184 of [object + 4].
Same constant and the same EX/EXTRA naming, but two different offsets in what may
be two different objects - that the two fields are the same field is NOT shown,
only that both compare a mode word against 3.

All seventeen artefacts byte-identical.
2026-08-28 10:51:16 +00:00
Sylpheed RE agent
33671c5086 re: wep_85 is the ONLY declared-but-unshipped asset; the package gaps are unused numbers
The obvious follow-up to last commit's wep_85 finding: the weapon packages number
00 to 84 with 26 holes, so are those holes cut weapons too? Partitioning every
number 00-85 by package-present x named-anywhere-in-the-parsed-pak-data:

                     named in data   not named
   package present        59             0
   package absent          1            26

59 + 0 + 1 + 26 = 86, the numbers 00 through 85.

The 26 gaps are simply unused numbers, not unshipped weapons - nothing in the
data names them at all. The one occupied cell is the one already known: wep_85,
named 16 times, with no package. It is an isolated cut, not a pattern, which
refutes the natural expectation that finding one cut weapon means finding a
family of them.

The other zero is worth as much: 0 shipped packages are never named. Every
rou_f001_wep_NN.xpr on the disc is referenced by the data, so the package set and
the weapon vocabulary agree in both directions but for that single entry.

Running the identical test on the craft family gives a clean negative. Every
rou_f### number appearing inside an .xpr against every number named in the parsed
data: 16 in both, 0 named-but-unshipped, 0 shipped-but-unnamed. The 16 are
f001-f004, f101, f102, f104-f106, f201, f202, f301-f303, f401, f402 - the
hundreds digit groups them into five families with holes inside each (no f103, no
f203). No craft was cut.

So across both families the disc ships exactly what it declares, with the single
exception of that one weapon.

All seventeen artefacts byte-identical.
2026-08-28 10:33:41 +00:00
Sylpheed RE agent
7ba415fbae re: the last 9 effects are genuinely unshipped; wep_85 accounts for two
EF_IDX_ proved that prefixes exist, so the residual deserved the same treatment
across every package rather than one. Censusing prefixes over all 36
effect-carrying .xpr gives seven in use:

  EF_IDX_       223 names
  mdl_           45
  EF_IDX_mdl_    45
  VolumeLine_    10
  GN_            10
  GN__            6
  bare           the rest

Testing all nine unlocated names against all seven prefixes: 0 of 9 resolve.

That zero has force where the earlier disc-wide zero did not, and for the stated
reason - the control shows each prefix genuinely carries names the same search
reads (223, 45, 45, 10, 10, 6), so the instrument demonstrably works on the
population it is being asked about. The nine are bound by a datasheet field and
shipped in no package.

Two of them turn out to belong to one cut asset. eff_m010_wep_85 and
eff_m011_wep_85 name a weapon, and the weapon is real in the data:
Weapon_DSaber_P_wep_85_Beam with 24 uses, its Shell_, WeaponCannonName_ and
WeaponShellName_ siblings, and GameModel_eff_m010_wep_85 / _m011_ declaring the
two effects. But the weapon packages stop at 84 - hidden/resource3d/ holds 59
rou_f001_wep_NN.xpr files numbering 00 to 84 with gaps, and no wep_85. So wep_85
is a declared-but-unshipped weapon and its two effects go missing with it.

This does not contradict "every weapon is placed - 131 = 105+22+0+4". That
partition is declared x MOUNTED IN A LOADOUT, which is a different question from
whether a package ships.

Seven remain with no account: eff_e0044, eff_f0002, eff_f0002_barn, eff_h308,
eff_j002_e01, eff_j002_e02, eff_n0071. The .xpr route is now exhausted for them
under every prefix the disc uses; a different container or a runtime generator is
what is left.

All seventeen artefacts byte-identical.
2026-08-28 10:17:20 +00:00
Sylpheed RE agent
23a6cfd979 re: the EF_IDX_ prefix - ptc_pack has 727 names, and the map reaches 128 of 137
Censusing ptc_pack's own naming vocabulary turned up a third variant of the
prefix trap, and this one had been corrupting a number the corpus carried.

268 of ptc_pack's names do not start with eff_ at all. They start with EF_IDX_,
as in EF_IDX_eff_d001_f. A regex anchored at eff_ chops that prefix off and
merges distinct names, which is exactly where the earlier figure of 532 came
from. Enumerating maximal [A-Za-z0-9_] runs gives 727. The two earlier traps were
a STORED name being longer (rot_n001_break) and a BOUND name being a prefix
(eff_f0002 inside eff_f0002_barnhaze); this is the third - a prefix the pattern
cannot see at all, because its anchor sits in the middle of the real name.

Looking each bound name up bare AND under EF_IDX_ resolves 25 of the 34 that were
unlocated. The map is now 128 of 137, and the residual is 9, small enough to
print: eff_e0044, eff_f0002, eff_f0002_barn, eff_h308, eff_j002_e01,
eff_j002_e02, eff_m010_wep_85, eff_m011_wep_85, eff_n0071. All 17 eff_l### are
among the recovered.

This withdraws my own previous correction. I had recorded Base.xpr (53) as
holding more bound effects than ptc_pack (46), and struck out "ptc_pack is the
effect library". With the prefixed keys counted ptc_pack holds 71 - it IS the
larger library, and the 46 was an undercount from the same truncating pattern.
Two shared libraries remains right; which one is bigger does not.

The suffix vocabulary: 106 distinct tokens over the 727 names - IDX 223 (the
prefix above), _f 137, _e 119, _root 87, _col 54, _mdl 45, _break 43, _ring 38,
_ALL 17, _haze 14, _thunder 10. That census counts ALL tokens rather than
trailing ones, which is precisely how the EF_IDX_ PREFIX surfaced inside what I
had first labelled a suffix list - the mislabel found the bug.

Testing the structural candidates the way _hangar was tested, does the suffixed
name have a bare parent:

  _ALL      17 names    17 of 17
  _root     87          64 of 87
  _break    30          15 of 30
  _e        74           0 of 74
  _f        61           0 of 61

_root is strictly terminal - 87 of 87, and it never appears mid-name. The
compound shapes put it outermost: _e_root 19, _f_root 18, _break_root 13, bare
_root 30. So the order is <stem>_[<faction>|<break>]_root and _root reads as a
hierarchy marker rather than a variant - though 64 of 87 having a bare parent
means it is not simply the parent of an existing node, and _break at 15 of 30 is
likewise not a plain destroyed-twin-of-everything.

_e/_f never have a bare parent, 0 of 135. That is independent asset-side
confirmation of the faction law: an effect is authored per faction and there is
no faction-neutral original for either side to derive from.

effect-homes.txt changes 5/30 and every line pairs: five values changed (103->128,
34->9, ptc_pack 46->71 and its sort position, the residual header, 3-digit
80->105 of 110) plus 25 pure deletions, exactly the 25 recovered names. All are
3-digit, so the 4-digit line is unchanged at 23 of 27. The other sixteen
artefacts are byte-identical.
2026-08-28 10:01:53 +00:00
Sylpheed RE agent
aa478d9444 re: the faction law generalises - 564 of 564, four fields, all six paks
The previous commit measured _e/_f on one pak and only through the effect
binders. Widening the sweep to EVERY string field of every unit object in ALL
SIX GP_MAIN_GAME_* paks:

                value _e   value _f
  UN_e###          198          0
  UN_f###            0        366

564 of 564 agree and the mismatch residual is empty. The law is not confined to
one field either - it holds separately, at 100%, in each of four:

  LowerHPFxModel            252 of 252
  ShieldHitEffectName       210 of 210
  ShieldRecoverEffectName    84 of 84
  ExplosionFxModel           18 of 18

The two shield fields were not in the earlier measurement at all, so the law
reaches further than the *FxModel family that suggested it.

Scope stated exactly, because "general" would overclaim: this is a law about
EFFECTS, not about assets in general. The sweep covered every field, and every
_e/_f-suffixed value a unit binds turns out to live in those four effect fields.
No model, motion or SE value carries the suffix at all, so the faction pairing is
NOT shown for those kinds - there was simply nothing to test.

UN_n### (TTRL) binds no _e/_f value in any of the six paks: 12 objects, = 2
users, with nothing on either side. That confirms over the whole population what
was only a single-pak observation before.

Also corrects the ID grammar and reconciles a count. The earlier section reported
42 + 26 + 2 = 70 unit objects using a regex that required UN_<letter>###_<FACTION>_;
the looser UN_<letter>###_ finds 71. The extra one is UN_e910_core_ADAN_GeneratorCore,
which inserts a sub-kind token BEFORE the faction tag. So the grammar is
UN_<letter>###_[<subkind>_]<FACTION>_<name>, and both counts were right for their
own pattern.

All seventeen artefacts byte-identical.
2026-08-28 09:44:20 +00:00
Sylpheed RE agent
1265512880 re: _e/_f on an effect name is the binding unit's FACTION (94 of 94)
Chasing the 17 unlocated eff_l### turned up their shape first: they come in
_e/_f PAIRS - eff_l101_e + eff_l101_f, and the same for l102, l104, l105, l106,
l201, plus _e-only l010/l011/l107/l108 and _f-only l002.

Partitioning every eff_<letter><digits>_<e|f> binding by the ID letter of the
OWNING unit (one GP_MAIN_GAME_* pak = one user):

                effect _e   effect _f
  UN_e###           33          0
  UN_f###            0         61

94 of 94 agree and both off-diagonal cells are empty. The control reads the
factions straight off the IDs: UN_e### -> ADAN (42 objects), UN_f### -> TCAF
(26), UN_n### -> TTRL (2, tutorial, binding neither). So an effect ending _e
belongs to an ADAN ship and one ending _f to a TCAF ship - the same visual is
authored twice, once per faction, which is exactly why eff_l### arrives in pairs.

What the 34 unlocated ARE is now also clear, even though where they live is not.
They are one job, not a scatter: Generic binds 32 of the 34, Explosion 19,
Shell 9, Level_0 and Weapon 2 each. The binder fields rank LowerHPFxModel 252,
HitFxModel 144, then JetFxModel_00N and AfterBurnerFxModel_00N. They sit in the
six GP_MAIN_GAME_* paks at 130 bindings each plus 32 in DefTables.pak. Since
LowerHPFxModel is the damaged-ship effect, the residual is largely the
per-faction battle-damage and hit visuals. None of the 34 is a record name and
only one is a field name, so they are asset references.

Stated plainly: they remain unlocated AS ASSETS. Knowing the family and its
naming law does not say where the geometry lives - the .xpr route is exhausted
for them and the parsed pak payloads hold references, not meshes.

Also fixes a defect in the artefact shipped last commit. effect-homes.txt came
back with two equal-count lines swapped: Counter.most_common() breaks ties by
insertion order, so the package listing was not deterministic. Now sorted by
(-count, name) and verified to regenerate byte-identical twice running. This is
the corpus's own rule - any map built by iterating a set or Counter needs
sorted() - and the new tool had violated it.

The other sixteen artefacts are byte-identical; effect-homes.txt changes only in
the tie-break ordering of the five 1-count rows, with every line pairing.
2026-08-28 09:28:39 +00:00
Sylpheed RE agent
e72f0f14be re: the effect->package map, 103 of 137 - and eff_f0002 was a substring artefact
Enumerating eff_* names per .xpr across all 166 packages and matching the bound
names EXACTLY gives a real home for 103 of the 137, up from the 71 ptc_pack
alone accounted for. Only 36 packages carry an effect name at all, and two
dominate:

  Base.xpr                  53
  ptc_pack.xpr              46
  Stage_S28.xpr              2
  five rou_f001_wep_NN.xpr   1 each

So there are TWO shared effect libraries, not one - and ptc_pack.xpr is the only
*_pack bundle on the disc, so no third shared library is hiding. By digit-width:
3-digit 80 resolved of 110, 4-digit 23 of 27. The previous iteration's split
survives and sharpens - the four-digit series really does live outside ptc_pack
(that zero stands), and now we can say where: Base.xpr.

Correction to the previous commit. It reported eff_f0002 and eff_f0002_barn as
present in Base.xpr. Both were SUBSTRING artefacts: what the file actually holds
is eff_f0002_barnhaze, one longer resource name that grep -l eff_f0002 and
grep -l eff_f0002_barn each match inside. Neither bound name is there. This is
the corpus's own paid-for prefix lesson arriving from the other direction - last
time it was rot_n001 vs rot_n001_break with the stored name longer; here the
BOUND name was the prefix. The new map is exact-keyed and does not have this
failure mode, so the earlier positive is withdrawn.

34 names remain unlocated, dominated by a family the last pass did not single
out: eff_l### with 17 of the 34, then h 4, s 4, j 2, m 2, t 1, and four
four-digit names - eff_e0044, eff_f0002, eff_f0002_barn, eff_n0071.

Scope note worth keeping: the j 22 / t 14 clustering reported last time was the
residual against ptc_pack ALONE; against all packages those families are largely
accounted for and l is what is left. Both numbers are right for their own
population, which is exactly why a residual has to say what it was measured
against.

New artefact with its regenerator: tools/re-capture/effect_homes.py ->
docs/re/data/effect-homes.txt, which lists all 34 by name. All sixteen existing
artefacts byte-identical.
2026-08-28 09:12:15 +00:00
Sylpheed RE agent
7e9d1437c0 re: effects split into two families by digit-width; ptc_pack.xpr holds one
The open question was how an effect mesh is reached at all, after last
iteration's .xpr byte search was refuted by its own control. The corpus already
held the pointer: xbg7-mesh names ptc_pack.xpr, a 20 MB shared particle package
in hidden/resource3d/. It lists 532 distinct eff_* resources - 527 three-digit,
3 four-digit, 2 unnumbered.

Joining it against the 137 effect names the datasheets actually bind:

                    in ptc_pack   not
  3-digit                71        39
  4-digit                 0        27

Zero of the 27 four-digit names resolve in ptc_pack, and that series is a closed
three-letter set: e (10), f (12), n (5). Since eff_e0033 was found in Base.xpr,
the reading is two effect families - a shared three-digit particle library in
ptc_pack.xpr, and a four-digit series that lives in the per-model and base
packages instead.

The instrument passes its own control this time, which is the difference from
last iteration. The same kind of byte search demonstrably reads names out of
this file - 532 of them - so a zero WITHIN ptc_pack is meaningful in a way the
earlier disc-wide zero was not.

Among the 39 three-digit misses the letters cluster hard - j 22, t 14, m 2, h 1 -
and ptc_pack contains just one j name against 149 m and 81 s, so eff_j### is a
third grouping that is almost entirely elsewhere.

Not closed: 66 of the 137 bound effects still have no located home, eff_n0071
among them. But the route is now real and has a number on it, and the next step
is the eff_j### family and the four-digit series rather than another disc-wide
grep.

All sixteen artefacts byte-identical.
2026-08-28 08:55:41 +00:00
Sylpheed RE agent
dd542c23a4 re: the unit family is the only schema on the disc; an .xpr search cannot prove absence
Two exhaustive probes agree on the same 15 records. The six Shift-JIS type words
occur in the six GP_MAIN_GAME_* paks and nowhere else, and a disc-wide sweep of
every parsed record name for a wildcard shape (???, *, ###, NNN, <...>) returns
7 distinct names - exactly the seven already in the schema: Turret_???,
Hatch_???, Bridge_???, Thruster_???, ShieldGenerator_???, Versatile_???, NS_*,
each x6. So the weapon datasheet, the arsenal item and StageResource ship NO
schema; the unit datasheet is the only structure the disc describes to itself.
Wildcard field names are confined to the schema records too - NozzleSpec_???,
NozzleFrame_???, CannonFrame_???, MuzzleFrame_???.

No untyped gaps either. The full residual is 30 slots and every one holds a
sample value rather than a missing type: 28 booleans spelled Yes, plus
Generic.ID = Ship_ and Generic.Type = Vessel. The booleans follow one pattern -
the six destructible part types each carry the same four-boolean core
(IsDestructible, IsInvolved, IsRadarVisible, IsShielded), Turret_??? adds IsAuto,
Generic carries only IsDestructible, and NS_* has its own pair AttenuationAlpha /
AttenuationVolume. 28 + 2 = 30; 7 fully-typed records + 8 with examples = 15.

The third result is a refutation of my own instrument. Testing the four
genuinely-undeclared effects against the 166 .xpr packages put eff_f0002 and
eff_f0002_barn in Base.xpr and found nothing for eff_e0044 or eff_h308 - but the
control kills the negative: eff_n0071, which we measured LIVE as an Explosion
record's ExplosionFxModel, also returns nothing from the same search. A
known-live name the test cannot find means the test has no power here. Only the
positive half counts: eff_f0002/_barn do ship. Nothing follows about eff_e0044
or eff_h308, and the earlier "no mesh" remarks about eff_e0058/_e0059/_e0060 are
weaker than written - absent from a byte search over .xpr is not absent from the
disc. How an effect mesh is actually reached is now the open question, since
eff_n0071 is not a plain name string in any of the 166 packages.

All sixteen artefacts byte-identical.
2026-08-28 08:38:26 +00:00
Sylpheed RE agent
fa8a538c33 re: the disc ships the unit datasheet's own SCHEMA (and 文字列 was not a placeholder)
This corrects the previous commit. 文字列 is not a developer's leftover: it is
one member of a six-word Shift-JIS TYPE vocabulary, and the records carrying it
are a machine-readable schema for the unit datasheet.

The whole non-ASCII population on the disc is 6 distinct values out of 99328 -
0 of 3496 record names and 0 of 12173 field names - and all six are type words:

  文字列              string                366 uses
  NS_"文字列"         NS_ string             12
  整数 / 整数値        integer            66 / 6
  浮動小数値           floating-point value  504
  浮動小数値[0〜1]     float in [0,1]         36

990 type-valued fields. So the reader defect noted last time is real but bounded
to these six strings, and name_hash re-encodes Latin-1 byte-for-byte, so hashing
was never affected.

They sit in 15 records x 6 GP_MAIN_GAME_* paks = 90 instances, i.e. 15 records
with ONE user. The names are exactly the unit substructure family, and six carry
a literal wildcard: Turret_???, Hatch_???, Bridge_???, Thruster_???,
ShieldGenerator_???, Versatile_???, and NS_*. ??? is the numeric-suffix wildcard
at record AND field level - Turret_??? is the schema for Turret_000..00N, and
inside it CannonFrame_??? / MuzzleFrame_??? stand for the numbered slots.

Where a field's type is an enumeration the schema holds an EXAMPLE value instead
of a type name: Yes for the five booleans, Vessel for Generic.Type (the 43
Craft + 71 Vessel split), Ship_ for the ID prefix convention. Maneuver is the
one fully-typed record, 34 of 34. Every ResistanceTo* and every Color_* channel
is declared FLOAT[0..1] - normalised by declaration, matching the sampled values
in unit-datasheet-static. Generic.NozzleSpec_??? has its own type NS_"文字列"
and NS_* is a record, so the nozzle spec is a nested sub-schema.

Control separates schema from data cleanly: the _??? records and NS_* exist ONLY
as schema, 6 of 6 instances typed, while the eight real substructure names are
typed in 6 instances and untyped in the rest - Generic 6 of 3651, the others
6 of 684 each.

Turret_??? carries the game's own typo NomalModel beside DamagedModel.

This gives the port an authoritative field-type table: types the disc declares,
rather than types inferred from sampled values.

New artefact with its regenerator: tools/re-capture/datasheet_schema.py ->
docs/re/data/datasheet-schema.txt. All fifteen existing artefacts byte-identical.
2026-08-28 08:22:36 +00:00
Sylpheed RE agent
44bdbad2d6 re: every bound effect vs every declared effect; a Shift-JIS dev placeholder
The Explosion substructure's field list was already in the corpus, so the open
part was whether the effect names RESOLVE. Declared x used, sweeping 34 binder
field names (*FxModel*, *EffectName, Effect_*, ShellModel, CoverModel,
SilhouetteModel) against every name declared by a LOD_Effect_<n> or
GameModel_<n> field anywhere on the disc:

                 declared   not declared
  used              172          58
  not used          318           -

490 declared, 230 used. Top binders: Effect_Paralyze 2874, ShellModel 996,
HitFxModel 738, JetFxModel_000 408, MuzzleFlashFxModel_Loop 384.

The 58-cell is almost one field. 53 of the 58 are bound by SilhouetteModel
alone and are all rou_f###_wep* names - the arsenal item silhouettes already
documented in arsenal-item-weapon-chain. They are undeclared because they are
the wrong KIND: each resolves as its own standalone package, rou_f001_wep_01.xpr
and friends, never as a LOD-table entry. Nothing is missing; the sweep was
reading an asset-file name as though it were an effect name. That leaves four
genuinely undeclared effects - eff_e0044, eff_f0002, eff_f0002_barn, eff_h308 -
none of which resolves as a record or field name either.

One "effect name" in that cell is not a name at all. Bound by Effect_Explosion
and Effect_Flare, 72 occurrences = 12 users, its bytes are 95 B6 8E 9A 97 F1 -
Shift-JIS for the word "character string". A developer's placeholder. This
exposes a reader defect worth fixing before these strings reach a port:
unitgroup.py hands the value back as Latin-1 mojibake, so an IDXD string field
can carry Shift-JIS and our decode does not know it.

Following the silhouettes into the ISO tree turned up a convention that IS real:
59 of the 166 .xpr packages end _hangar.xpr, and 59 of 59 have a bare twin of
the same stem. The direct contrast to yesterday's refutation, where _all/_child
was 1 of 166 with a single stem. Suffix conventions in this corpus are worth
testing precisely because they are not all real.

Controls reproduced from the previous iteration: eff_n0071 is declared and used
6x (= one user under the per-pak-copy rule); eff_e0033 is declared and used 0x,
sitting in the 318-cell. That cell is expected rather than alarming - the
EnumLODSet/EnumGameModel family is overwhelmingly equipment, which no unit
datasheet binds.

All fifteen artefacts byte-identical.
2026-08-28 08:05:28 +00:00
Sylpheed RE agent
25b5f60ea1 re: _all/_child is not a convention (1 of 166); eff_n0071 is a live explosion
Refuted, and it was my own suggestion from last turn. Across all 166 .xpr
packages the _child suffix occurs in ONE file, Stage_S24.xpr, with exactly one
stem, stg24_04, 20 occurrences. Across the 105394 distinct parsed pak names -
record names, field names and string values - names ending _all or _child occur
ZERO times. So the pair is mesh-internal to a single model and never reaches the
data tables; the tables only ever name the bare stg24_04. n = 1 is not a
convention, and the multi-part mesh mechanism the corpus actually owns is
xbg7-mesh's grouped index/vertex pools, which is a different thing.

The orphan tables' five unique effects split three ways, chased through PARSED
names rather than .xpr bytes:

  eff_n0071                  LIVE - Generic.GameResourceID in DefTables.pak and
                             an Explosion record's ExplosionFxModel in all six
                             GP_MAIN_GAME_* archives (x6 = one user)
  eff_e0033                  a mesh in Base.xpr, but no data reference
  eff_e0058/_e0059/_e0060    neither a mesh nor any reference

So one "abandoned" entry is not abandoned: the orphan LOD tables list an effect
the shipped _test table omits while the game still binds it elsewhere.

Control recorded so the negative has force: effect names bind through a whole
family of fields - Effect_Paralyze (2874 values), HitFxModel (738), ShellModel
(516), JetFxModel_000/001/002, MuzzleFlashFxModel_Loop - so a value that
resolves through none of them really is unreferenced, not merely missed by a
narrow search.

All fifteen artefacts byte-identical.
2026-08-28 07:47:06 +00:00
Sylpheed RE agent
da41f95db1 re: the challenge LOD extras are meshes in that stage's own .xpr - 11 of 11
Perfect diagonal. Each of stg24_01..04, stg26_01..03, stg27_01..02 and
stg29_01..02 appears in Stage_S<NN>.xpr and in no other stage package. So a
challenge stage's bespoke LOD/model table exists to declare the handful of
scenery meshes packaged with that stage, consistent with xbg7-mesh's account of
a stage .xpr as a bag of XBG7 resources.

stg24_04 is a composite: 22 occurrences resolve as 1 bare + 1 _all + 20 _child.
Every other stg* name occurs exactly once.

rou_f004 is NOT in Stage_S28.xpr - it is in DeltaSaber_A.xpr. S28's LOD table
declares the craft but the mesh ships in the player-craft package, confirming
from the ISO side that _A is the f004 variant: the stage package carries
scenery, the craft comes from its own file. rot_n001_break resolves too, in
Tutorial.xpr and Stage_S28.xpr, matching exactly the tables that list it.

The orphan tables' unique effects only partly resolve: eff_e0033 is present in
Base.xpr, but eff_e0058 and eff_n0071 appear in NO .xpr on the disc. So the six
abandoned LOD tables name at least two effects that were never packaged - the
same "declared, never shipped" shape as S14's asteroid meshes, on a much smaller
scale.

Method caveats recorded in the doc: these are byte searches over .xpr files, not
parsed resource-table reads. They establish presence, not an enumeration, and
are acceptable only because the names are long and distinctive and the match was
expanded to the full trailing token. Also grep -c counts LINES, not occurrences;
the counts above come from grep -o | wc -l after that was caught.

All fifteen artefacts byte-identical.
2026-08-28 07:30:37 +00:00
Sylpheed RE agent
d08e464897 re: challenge LOD tables are _test + set dressing; orphan count corrected 8 -> 6
Diffed the field sets, the same move that cracked EnumWeapon_EX5.

The five bespoke challenge tables are EXACT SUPERSETS of _test - 10 of 10 (five
stages x the EnumLODSet and EnumGameModel families) with zero removals. Extras
run +2 to +9 and name the stage's own scenery: stg24_01..04, stg26_01..03,
stg27_01..02, stg29_01..02, plus rot_n001_break shared by S24, S26, S27, S28 and
Tutorial.

S28's nine extras independently confirm the player-craft result:
LOD_Frame_rou_f004, four rou_f004_mnv*/turn180 poses and three eff_j004_*
effects - the LOD table for the one stage that flies rou_f004. Two unrelated
tables agreeing on S28's oddity is a real cross-check.

This does NOT revive "rot_n001 is on the disc": that refutation concerned the
bare name, which still occurs 0 times exactly; only the _break form is real,
exactly as recorded.

Self-correction: 6 orphan tables, not 8. name_hash is case-INSENSITIVE, so _test
and _Test are the same pak entry - EnumLODSet hashes to 3485720498 and
EnumGameModel to 4020329537 for both spellings. The family therefore has 10
distinct keys, not 11 tags, and the unreferenced set is S01, S02, S16 x 2 = 6.

And the six are not stale copies of _test. They are far smaller (120, 129 and
178 EnumLODSet fields against _test's 676) yet each carries 4-5 entries _test
does NOT have: LOD_Effect_eff_e0033, _e0058, _e0059, _e0060, _n0071, and S16
adds five rou_e106_* boss parts matching stage16-boss. So the live table is not
a superset of the abandoned ones. Tutorial has the same shape: 66 fields, 3
unique.

All fifteen artefacts byte-identical.
2026-08-28 07:15:28 +00:00
Sylpheed RE agent
0536e287b5 re: EnumLODSet_test.tbl is the story campaign's live LOD table; S25 premise withdrawn
The question was why S25 is absent from the DefTables +-path families. The
premise is withdrawn: it was an artifact of listing DISTINCT values of
LodEnumTable. S25 is not missing anything - it shares EnumLODSet_test.tbl with
the sixteen story stages, and test is in the list. Counting distinct values
hides who uses them; join per user.

The per-stage join: EnumLODSet_test.tbl / EnumGameModel_test.tbl serve 17
stages - all of S01-S16 plus S25. The five challenge stages S24, S26, S27, S28,
S29 have bespoke tables. Hangar.xpr has no pair at all. 17 + 5 + 1 = 23 objects
carrying a StageResource.

So the test-named table is the live LOD and model table for the whole story
campaign - the strongest case yet of "a test_-named table can be live", serving
17 of the 22 missions.

Eight shipped tables that nothing references: the family covers only 11 tags
(S01, S02, S16, S24, S26-S29, Tutorial, test, Test) and is NOT per stage -
S03-S15, S17-S23 and S25 have none. Seven tags are referenced; S01, S02, S16 and
Test are present in DefTables.pak with zero references disc-wide, i.e. 8 orphan
tables. "Early per-stage LOD sets, abandoned when the campaign was pointed at
the shared test table" is a reading, not a measurement.

Scope note recorded so the family is not mistaken for a stage one:
EnumLODSet/EnumGameModel is overwhelmingly an EQUIPMENT family - 116 of the 130
referenced values are bare Equip_EnumGameModel_wep_NN.tbl names reached from
PlayerLOD, WingmanLOD, PlayerMotion and WingmanMotion (58 distinct each); only
the 14 stage-tagged values use the + path form.

All fifteen artefacts byte-identical.
2026-08-28 07:00:34 +00:00
Sylpheed RE agent
9be52cb2e9 re: the archive.pak+entry dictionary is 20 keys and misses the 2D blocker entirely
A path that names its own archive is a free name->key fact, since a pak TOC holds
only hashes. Swept them disc-wide and aimed the result at the 2D /
GP_READY_ROOM blocker.

Seven fields carry a game: value - BackGroundPackage 18, StageResourcePackage 23,
TBL_3D 6, LodEnumTable 7, MotionEnumTable 7, one PATH, one anonymous - giving 63
distinct paths in two forms: 42 file-form (41 present on the extracted ISO, the
single miss being the bare directory game:\hidden\Resource3D\, which is not a
file) and 21 archive+entry (20 of 20 resolve inside the archive they name; the
21st is sound.pak+ with an empty entry).

The blocker test is a measured zero. The six 2D paks and GP_READY_ROOM hold 1817
distinct keys between them and NONE is named by a + path. The dictionary's total
reach is 20 of 16630 keys, 0.12%, and every one lands in DefTables or
GP_HANGAR_ARSENAL - archives that are already partly named.

The route is exhausted rather than unlucky: these paths exist only in the seven
fields above, and none of those fields ever addresses a 2D or ready-room asset.
The blocker keeps its earlier structural reason (six identical 711-key TOCs, 0 of
711 shared with any other archive).

The full dictionary is recorded: 6 x <lang>\GP_HANGAR_ARSENAL_3D.tbl in
GP_HANGAR_ARSENAL.pak, and EnumGameModel_* plus EnumLODSet_* for S24, S26, S27,
S28, S29, Tutorial and test in DefTables.pak.

Left unexplained: both DefTables families are seven names with S25 absent and no
story stage present at all.

All fifteen artefacts byte-identical.
2026-08-28 06:43:52 +00:00
Sylpheed RE agent
af8292c940 re: the rest of StageResource - two more value kinds, one convention refuted
Swept all 19 StageResource fields by value kind.

Refuted: <X>ID + <X>Package is not a convention. The record has exactly one *ID
field (BackGroundID) and exactly one pairing stem. StageResourcePackage has no
StageResourceID; WingmanIconID_0..3 have no package. n = 1 is not a rule, so
last pass's generalisation is withdrawn.

Fourth kind - the game:\ external path. BackGroundPackage (18 values) and
StageResourcePackage (23) are neither records nor pak keys because they are ISO
file paths. Mapping game:\ to the extract root and \ to /, case-folded, 41 of 41
exist on the extracted disc (hidden/resource3d/BG_Acheron.xpr, Stage_S01.xpr,
...; 166 .xpr files in that directory).

Fifth form - game:\hidden\<archive>.pak+<entry>. LodEnumTable, MotionEnumTable
and TBL_3D name an archive AND an entry inside it, e.g.
game:\hidden\DefTables.pak+EnumLODSet_S24.tbl. Splitting on + and hashing the
right half against THAT archive's keys resolves 20 of 21; the one failure is
sound.pak+ with an empty entry name.

Some of those entry names carry an explicit <lang>\ prefix
(deu\GP_HANGAR_ARSENAL_3D.tbl, eng\, esp\), each present in 0 other archives, so
the language-prefix form is genuinely used by the engine. This does NOT unblock
the 2D naming problem: archive-naming refuted <lang>\ there structurally, since
the six 2D paks share one identical 711-key TOC and cannot carry per-language
names. Both facts stand.

The 4 never-shipped _Test tables - the residual of the 344-name census - are now
named: EnumCharacter_Test.tbl, EnumLocalString_Test.tbl, EnumSquadron_Test.tbl,
StageMessageSet_test.tbl (lower-case test on the last).

WingmanIconID_1..3 hold PGHUD_WING2/WING3/WING4, HUD element names from the PG*
roster; WingmanIconID_0 is never valued.

All fifteen artefacts byte-identical.
2026-08-28 06:28:14 +00:00
Sylpheed RE agent
3da617d04b re: BackGroundID is an identity, not a file name; its sibling names the asset
The one value class that resolved to neither a record nor a file resolves to
neither because it is neither. BackGroundID lives on StageResource (174 records
carry it) directly beside BackGroundPackage, and that sibling holds the real
asset: game:\hidden\Resource3D\BG_<something>.xpr.

My earlier suffix sweep had a real bug: it tried BG_ and .xpr as alternatives,
never as a pair, so BG_Acheron.xpr was never in the search space. A
one-transform-at-a-time sweep cannot find a two-transform name.

But the obvious follow-on hypothesis - "the package is BG_<id>.xpr" - is also
refuted: it holds in only 12 of the 24 StageResource records carrying both
fields. Hargenteen spans six packages (BG_Hargenteen{,_near,_far,_dead}.xpr plus
BG_Acheron.xpr and BG_Stg26.xpr), and BG_Acheron.xpr serves both the Acheron id
(S09) and the Hargenteen id (S25). The relation is many-to-many in both
directions, roughly 18 distinct packages across 10 ids.

So the id is a logical place - Acheron, Earth, Hargenteen, Lebendorf - and the
package is the mesh actually loaded, chosen per stage from that place's _near /
_far / _dead variants. Two details fall out: PD is the Prometheus Driver
(BG_P_Driver.xpr, on S16, the boss stage), and the challenge stages that
re-dress a story sky carry bespoke BG_Stg24/26/27.xpr packages while keeping the
story stage's id - the same borrow-and-re-dress pattern already recorded for
maps and objectives.

Not settled: Original carries no BackGroundPackage at all; whether that means no
sky or an engine default is unknown.

All fifteen artefacts byte-identical.
2026-08-28 06:11:41 +00:00
Sylpheed RE agent
aa60d14ae7 re: value classes resolve by KIND; the 344-name residual drops from 31 to 4
Generalised the "does the referent exist?" sweep that cracked S14. The value
classes resolve as three different kinds:

  AIID                   31 values   31 records    0 files    0 unresolved
  FormationID           126 values  126 records    0 files    0 unresolved
  CollisionMeshes        18 values    0 records   18 files    0 unresolved
  EnumerateSubobjective  17 values    0 records   17 files    0 unresolved
  NamePlate              52 values   27 records   24 files    1 (empty string)
  BackGroundID           10 values    0 records    0 files   10 unresolved

FormationID 126/126 re-derives unit-group-table's own "every FormationID
resolves" from a different direction, as the control.

The 27 NamePlate values that stage-definition-table called "non-file plate ids"
are record names - ACROPOLIS x114, NP_ArrowHead x84, NP_ASFrigate x78 - and the
split 24 files + 27 records + 1 empty = 52 matches the two-vocabularies count
already recorded there. So the census residual shrinks: 313 resolve as files
plus 27 as records = 340 of 344, leaving the 4 never-shipped _Test tables as the
entire residual. The old reading was a KIND error, not a missing-file error.

BackGroundID is a genuinely unresolved third kind: the ten sky names, used 6-36
times each, are neither record names nor pak keys under 10 prefixes x 9 suffix
forms.

Recorded as powerless so it is not re-run: the image holds 0 exact strings for
all ten backgrounds, but also 0 for Formation_4_Bird, AI_Structure,
NP_ArrowHead, ACROPOLIS and CollisionSet_S03.bin, every one of which does
resolve in data. The executable names no data value at all, so the negative
discriminates nothing.

All fifteen artefacts byte-identical.
2026-08-28 05:55:52 +00:00
Sylpheed RE agent
5ec24b8ef9 re: S14's 13 are dangling deployments - the unit chain is absent for them
Two measurements settle the last open cell of the unit partition.

The deployment side is identical. S14's asteroid-cmesh squadron records match a
declaring stage's field for field - same squadron ids (GNN901, GNN902, ...) and
the same Count 1 / SideID Neutral / AIID AI_Structure / FormationID
Formation_1_only / DisableInterval No. Nothing about how S14 deploys them
differs, and these are the same kind of record compared like for like.

The definition side differs completely. Per stage, deployed / declared /
Generic-with-HP / any Generic: S01 8/8/8/8, S04 10/10/10/10, S05 5/5/5/5,
S08 6/6/6/6, S13 4/4/4/4 - 33/33/33 for the five - and S14 p1 9/0/0/0,
S14 p2 4/0/0/0.

So the 13 are not a manifest omission: they have no unit definition anywhere on
the disc, not even an un-HP'd Generic row. UnitGroup_S14.tbl names entities that
do not exist - 13 dangling references. That rescues the law (nothing is
deployed-without-being-loaded in any working sense) and explains why the gap is
exactly the cmesh class and nothing else in S14.

What it predicts needs the emulator: either those collision meshes fail to
appear in Stage 14 - a shipped bug - or another route supplies them. Flying S14
and checking whether the large asteroids collide distinguishes the two; no
static route can. Recorded as NEEDS-HUMAN.

Also recorded so the two asteroid populations are not conflated: asteroid-fields'
AsteroidGroup_00N tables scatter mob_n0NN models, while these are
UN_<stage>_Asteroid_cmesh_* squadron entities. S28.Phase_1 borrows
S14_p2_asteroid.tbl but deploys no cmesh of its own, so the borrow does not
carry the dangling references.

All fifteen artefacts byte-identical.
2026-08-28 05:39:44 +00:00
Sylpheed RE agent
f0dfff4fcd re: EnumUnit_S<NN> is the stage's LOAD MANIFEST; all 9 never-deployed units named
Joining each stage's own EnumerateUnit table to its own EnumerateSquadron table
is far sharper than the global partition: declared == deployed EXACTLY in 20 of
22 stages. S16 declares 4 it never deploys; S14 deploys 13 it never declares.
Every per-stage deployed-but-undeclared count is 0 except S14's 13, and that 13
is the entire disc-wide total - so the exception is one stage, not a scattering.

S16 explains what the table is. It declares 6 and deploys 2, and two of the six
are UN_e901_ADAN_Boss and UN_e910_core_ADAN_GeneratorCore - exactly the units
stage16-boss shows are brought in by their own loader chain (sub_82315AE8),
not by the squadron roster. So EnumUnit_S<NN> is the stage's load manifest, not
a mirror of the roster: a unit that code spawns is declared but never deployed.

That also closes the 9 declared-but-never-deployed units by name: the S16
code-spawned pair above, Test_ADAN_PrometheusDriver_InsideP2_01, and six
Ship_Test* placeholders.

S14 remains the one unexplained cell, and it is not a missing table:
EnumUnit_S14.tbl exists and declares six names, one of which is
UN_mn040_Asteroid_Big. S14 therefore declares an asteroid but not the 13
Asteroid_cmesh entities it deploys, which refutes "asteroids are exempt from the
manifest". Whatever the cause, it is specific to the collision-mesh entities.

All fifteen artefacts byte-identical.
2026-08-28 05:24:09 +00:00
Sylpheed RE agent
0ae295e9eb re: the weapon law does not generalise to units; the whole exception is S14
Applied the declared x deployed partition to units. Three vocabularies had to
be separated first: declared = field names of the 31 EnumUnit records (118);
deployed = slot 0 of each UnitGroup member tuple (122); datasheet = Generic
records carrying HP (114, which re-derives unit-datasheet-static's own number
from a different direction, as a control). Declared and deployed share the UN_*
vocabulary and overlap on 109; the datasheet's IDs are a third naming, disjoint
from both.

                    deployed   not deployed
  declared            109           9
  not declared         13           1

The cell that is empty for weapons (0 mounted-but-undeclared of 131) holds 13
for units, so the general form of that law is refuted: a unit CAN be deployed
with no table declaring it.

But the 13 are one coherent family - every one is UN_S14_p1_Asteroid_cmesh_* or
UN_S14_p2_Asteroid_cmesh_*. Asking the other population settles it: S01 8/8,
S04 10/10, S05 5/5, S08 6/6, S13 4/4 declared - 33 of 33 - while S14 declares
0 of 13, and non-asteroid deployed-but-undeclared is 0. The law survives in a
narrower form: every deployed unit is declared, except S14's asteroid collision
meshes.

Why S14 differs is not settled. It is the stage whose asteroid table S28
borrows, but nothing measured connects the borrow to the missing declarations,
so that is recorded as a coincidence rather than a cause.

All fifteen artefacts byte-identical.
2026-08-28 05:07:35 +00:00
Sylpheed RE agent
074c0296af re: every weapon placed (declared x mounted, 131); a fourth undeclared table
Partitioned all 131 Weapon.ID records against the two data routes by which a
weapon is ever named - a field name in an EnumWeapon record, and a Turret_NNN
record's WeaponID value:

                    mounted   not mounted
  declared            105         22
  not declared          0          4

105 + 22 + 0 + 4 = 131, with no unexplained cell. Both marginals re-derive
unit-substructure-records' own numbers independently: mounted 105, and
22 + 4 = its "26 weapons no turret mounts".

The empty cell is the result: nothing is mounted without being declared, 0 of
131, so declaration is the superset of mounting. The four undeclared and
unmounted records are therefore reachable by neither route. Each appears
exactly once per pak copy (x6) and only as a Weapon record's ID value; the live
control Weapon_TCAF_DeltaSaber_Beam appears x18 as an EnumWeapon field name and
x228 as a Turret_NNN.WeaponID. That places Weapon_TCAF_DeltaSaber_Laser_Ttrl,
the weapon this pass set out to find, and unifies all four as orphan datasheet
rows.

Recorded as powerless rather than as evidence: the image holds 0 exact strings
for the four, but also 0 for the live control and 0 for Weapon_NULL. Only three
Weapon_-prefixed strings exist at all and 0 contain Ttrl - the executable names
no weapon, so the negative does not discriminate.

New: sweeping EnumWeapon RECORDS rather than the three declared tables gives 130
distinct field names, not 127. The extra three come from a fourth record, pak
key 305595319, resolved as EnumWeapon_TestS01.tbl, carrying Weapon_TestS01_Gun/
_Laser/_Missile. It dangles at both ends - no stage declares it, and its three
weapons are the only declared names with no Weapon.ID record. 130 - 3 = 127
restores the previous pass's union.

All fifteen artefacts byte-identical.
2026-08-28 04:50:35 +00:00
Sylpheed RE agent
a304df1e50 re: _EXn is the n-th challenge stage; the Extra0n join is refuted
Reading the docs first paid: unit-substructure-records already had
UN_e001_ADAN_Elan_EX4 and arsenal-item-weapon-chain already had _T_EX5_el, so
last pass's "the only two _EX5 names on the disc" was too narrow. Withdrawn and
replaced by a census.

Every name ENDING in _EX<digit> across all 41 archives, taken from parsed IDXD
record names, field names and string values (never a raw byte scan): 26 names,
two digits only - _EX4 (7 units, 6 ADAN + 1 TCAF) and _EX5 (4 TCAF units + 1
weapon), each with a UnitName_/WeaponCannonName_ twin.

The stage join is exact. _EX4 appears only in tables S27 declares
(EnumUnit_S27.tbl, UnitGroup_S27.tbl); _EX5 only in S28's (EnumUnit_S28.tbl,
UnitGroup_S28.tbl, EnumWeapon_EX5.tbl); the other 20 stages have neither. The
challenge missions are S24-S29 in order, so S27 is challenge #4 and S28 is
challenge #5: _EXn names the n-th challenge ("EXtra") mission's bespoke
variants. S28 declares both schemes at once - EnumUnit_S28.tbl by stage number,
EnumWeapon_EX5.tbl by challenge index, for the same mission.

Refuted: _EXn is not the Extra0n numbering. There are four Extra slots but _EX5
exists, and under that reading Extra04 would be S29, which has no _EX assets.

Makes available but does not settle result-screens' open "what does EX_ mean":
"the challenge-mission debriefing" fits its evidence that the EX_ twin drops
overview_rank/overview_medals. Recorded there as a reading, not adopted.

All fifteen artefacts byte-identical.
2026-08-28 04:34:11 +00:00
Sylpheed RE agent
ad03991fa1 re: EnumWeapon_EX5.tbl adds exactly one weapon; a tutorial expectation refuted
Diffed S28's bespoke weapon table against the common one. The disc has exactly
three weapon tables, and each is a single EnumWeapon record whose FIELD NAMES
are the weapons - the Enumerate-style declaration shape again:

  EnumWeapon_Test.tbl       126 names
  EnumWeapon_EX5.tbl        127 names = the same 126, same order, plus one
  EnumWeapon_Tutorial.tbl     9 names = a strict subset of Test
  union                     127

The whole of S28's bespoke table is one extra weapon,
Weapon_TCAF_Ship_AAGun_EX5. Zero removals, identical order on the shared 126.

That explains one of weapon-struct-runtime's five "never instantiated" disc
records: the AA-gun variant is declared by EnumWeapon_EX5.tbl only, and that
table belongs to S28 alone, which the captured save was not playing.

It also refutes that document's expectation that "running the tutorial should
instantiate the _Ttrl pair". It will not: EnumWeapon_Tutorial.tbl's nine names
are Weapon_NULL, Weapon_ADAN_Puppy_NoseGun, Weapon_DSaber_P_wep_{01_Beam,
02_Missile,58_Laser} and Weapon_TCAF_DeltaSaber_{Beam,Cannon,Missile,
ASMissile} - neither _Ttrl weapon is among them. Four of the five
uninstantiated records are declared by no weapon table on the disc at all.

EX is still not identified but narrowed hard: the entire content of S28's own
weapon table is a single _EX5 AA-gun variant, and the only other _EX5 name on
the disc is UN_f001_TCAF_DeltaSaber_T_EX5.

All fifteen artefacts byte-identical.
2026-08-28 04:17:15 +00:00
Sylpheed RE agent
13ac9ccc29 re: every challenge mission is flown on a story mission's map
Applied the per-stage join to the other stage-resource fields. Of 338 distinct
(field, value) pairs across the 22 stage objects, 31 are used by more than one
stage. The strongest signal is CollisionMeshes:

  S24  CollisionSet_S11.bin  SUBObjectiveSettings_S01  Anastasis         Test
  S25  CollisionSet_S07.bin  SUBObjectiveSettings_S07  Hargenteen        Test
  S26  CollisionSet_S03.bin  SUBObjectiveSettings_S06  Hargenteen        Test
  S27  CollisionSet_S03.bin  SUBObjectiveSettings_S03  Planet_Lebendorf  Test
  S28  CollisionSet_S14.bin  SUBObjectiveSettings_S01  Lebendorf         EX5
  S29  CollisionSet_S15.bin  SUBObjectiveSettings_S15  Earth             Test

6 of 6 challenge missions reuse a story stage's collision set. Control: S01-S16
are perfectly diagonal, CollisionSet_SNN <-> SUBObjectiveSettings_SNN, 16/16.

Map, objectives and sky are borrowed independently - S24 flies S11's map with
S01's objectives; S26 flies S03's map with S06's objectives under a Hargenteen
sky belonging to neither; S28 flies S14's map with S01's objectives under S01's
sky. Three of six mix, so "S24 is S11" would be wrong.

S28 is the only stage with its own weapon table: 21 stages use
EnumWeapon_Test.tbl (x132), the tutorials EnumWeapon_Tutorial.tbl (x36), and S28
alone EnumWeapon_EX5.tbl (x6). S28 was already the odd stage - the only rou_f004
flight - so this is a second hook on it.

A datapoint for result-screens' open "what does EX_ mean": EX5 is also a variant
suffix on real names - UN_f001_TCAF_DeltaSaber_T_EX5 and
Weapon_TCAF_Ship_AAGun_EX5, 24 each. Not enough to fix the meaning.

Method note: an unanchored EX[0-9] search over binary payloads returns EX0..EX9
at 100-180 hits each - pure noise from matching inside compressed data. Only
prefix-anchored names are real.

All fifteen artefacts byte-identical.
2026-08-28 04:00:52 +00:00
Sylpheed RE agent
168bebc700 re: S28 does have an asteroid definition - it borrows Stage 14's
Withdrawing my own repeated claim that S28_p1 has an asteroid volume with no
definition. It came from a set-difference over FILE NAMES, which cannot see
reuse. Joining the fields per PHASE settles it:

  S28.Phase_1   MapMesh = S28_p1_AsteroidVolume_wp.col
                AsteroidDefinition = S14_p2_asteroid.tbl

S28 scatters Stage 14 phase 2's rock list inside its own volume. Nine
definition tables and nine _AsteroidVolume_wp meshes, with one table shared by
two phases: S14_p2_asteroid.tbl -> S14.Phase_2 and S28.Phase_1.

The earlier census's own arithmetic already contained the answer: the four
values counted x12 (2 per language pack) were S01, S04, S05 - two phases of ONE
stage - and S14_p2, which is two phases of TWO different stages. The count was
right; reading x12 as "two phases of the same stage" was the error.

test_s8p1_asteroid.tbl is not test-only either - it is what S02.Phase_1 ships
with. A test_ name here is a leftover, not a dead table.

The true residual is two phases and it is the opposite pairing to what I
recorded: S01.Phase_3 has the volume and no definition (S01 keeps its volume
across all three phases but scatters rocks only in 1-2), and S02.Phase_1 has a
definition and no volume (MapMesh is the plain mapmesh_box_500km).

All fifteen artefacts byte-identical.
2026-08-28 03:44:14 +00:00
Sylpheed RE agent
524e032542 re: the 2D/GP_READY_ROOM blocker - retry already done, two new structural facts
Item: retry the blocker with the "wrong prefix" lesson from
StageMessageSet_S02.tbl. The retry was already run and the owning doc says so:
archive-naming's "13 name transformations" list already includes 2d\, eng\,
hud\ and GP_MAIN_GAME_2D\ prefixes, every one scoring 0 against both the E2D
pak and all 16630 entries. Reading the doc end to end first would have shown
that in a minute. Recorded so the idea is not proposed a third time.

What the re-open did add - two measurements the earlier passes did not make:

1. The six 2D archives share ONE key set: 711 keys, identical in all six.
   Intersection = union = 711; 4266 entries, 711 distinct keys. The entry names
   are language-independent - the language lives in the pak file name, not the
   entry path. That refutes any <lang>\ prefix hypothesis structurally, not
   merely by a zero score: if the names carried a language directory the six
   key sets would differ.

2. The 2D key space is disjoint from the rest of the disc - 0 of 711 appear in
   any non-2D archive, while disc-wide 2010 of 16630 keys are shared between
   archives. Sharing is common; this set never does it.

The six named GP_READY_ROOM entries are all fonts - deu\GOTHICB.TTF, eng\,
esp\, fra\, ita\ likewise, and jpn\HGRGE00.TTF. The doc recorded the count "6
named" without saying what they are: localisation fonts named from elsewhere on
the disc, not a sample of the archive's own artwork vocabulary, so they cannot
seed the dictionary attack the doc names as the last lever. Only 2 of 1106
GP_READY_ROOM keys appear in another archive.

Still blocked, now for a sharper reason: the 2D names are language-independent,
unshared, and absent from disc and executable alike.

All fifteen artefacts byte-identical.
2026-08-28 03:29:37 +00:00
Sylpheed RE agent
9bf1973e27 re: every declared table name censused; NamePlate holds two vocabularies
From last iteration's n=1 control (EnumSquadron_Test.tbl matched no pak key),
censused properly: every declared value disc-wide - the eleven Enumerate*
fields plus MessageSet, NamePlate, CollisionMeshes, MotionEnumTable,
LodEnumTable, AsteroidDefinition, MapMesh, MapPath - against the 16630 pak keys
under the sixteen path prefixes.

  distinct declared values                        344
     resolve to a pak object                      313
     do not                                        31
        NamePlate values that are not file names   27
        _Test tables that were never shipped        4

313 + 27 + 4 = 344. Every family is complete (EnumerateUnit 29/29,
EnumerateFormation 29/29, EnumerateNullFrame 29/29, EnumerateAIParams 23/23,
CollisionMeshes 18/18, AsteroidDefinition 9/9, MapMesh 11/11, MapPath 11/11,
LodEnumTable 7/7, MotionEnumTable 7/7, EnumerateSubobjective 17/17,
EnumerateWeapon 3/3) except the four below.

NamePlate carries two disjoint vocabularies, split by owning record.
StageResource.NamePlate (174 uses) holds file names - nameplate_S01.tbl etc,
24 distinct, 24/24 resolve. Generic.NamePlate (522 uses) holds bare plate ids -
NP_Cyclops, NP_Gargantua, ACROPOLIS, OTHERENEMY - 27 distinct, 0/27 resolve,
none with a dot. Zero crossover, so the 27 "missing" are a category error.

The four genuinely dangling names are all the template's own -
EnumCharacter_Test.tbl, EnumLocalString_Test.tbl, EnumSquadron_Test.tbl,
StageMessageSet_test.tbl - and in each of those four families the count is
n-1 / n with the _Test entry the sole miss. Every non-_Test declaration
resolves.

Closed a corpus question on the way past: stage-mission-tables recorded
StageMessageSet_S02.tbl as "not in GP_MAIN_GAME_E.pak under that name (?,
probably localised elsewhere)". It is there - name_hash("message\" +
"StageMessageSet_S02.tbl") = key 705319170, in all six packs. The stage family
uses stage\; the message tables use message\.

All fifteen artefacts byte-identical.
2026-08-28 03:13:21 +00:00
Sylpheed RE agent
e222016dfc re: the Enumeration self-index is not a naming route, and the residual is two _Test variants
The tempting move - 360 objects state their own contents, so use that to name
them - does not work, and it is now measured rather than assumed:

  objects carrying an Enumeration record                       360
     key already named by archive-naming routes 1 + 2          348
     still unnamed                                              12
     newly named by name_hash(prefix + own record name + ext)    0

Sixteen prefixes x seven extensions x every record name of every unnamed
object: zero gained. A self-index names an object's RECORDS; a TOC key is
name_hash(path + file name). The two vocabularies never meet.

The residual is exactly two objects x six packs, each byte-identical across the
packs and each an unnamed VARIANT of a named _Test table, not a duplicate:

  2390212806 (1493 B)  Placement_00..02 + RouteTest_00..01 - a trimmed
                       Stage\Null_Test.tbl (3121167452, 27545 B, same five
                       records plus four Frame_Test_Asteroid_*)
  2719765792 (14124 B) Formation_100, Formation_Fleet_01, Formation_Fleet_02 -
                       the same three record names as Formation_Test.tbl
                       (3463590559, 14025 B) but a different payload

That answers the previous iteration's question about the six-record
Placement/RouteTest sibling: it is Null_Test.tbl minus the asteroid frames.

Independent confirmation for stage-mission-tables: name_hash(prefix +
"Formation_Test.tbl") -> key 3463590559, whose records are exactly those three,
so the two frame-ratio exceptions do live in the file it names.

Dangling link recorded, not fixed: idxd-container.md references
idxd-unnamed-keys.md twice and that file does not exist on this branch - it
belongs to auto/idxd-unnamed-keys.

One control missed unexplained: EnumSquadron_Test.tbl matches no pak key under
any of the sixteen prefixes. n=1, not chased.

All fifteen artefacts byte-identical.
2026-08-28 02:57:20 +00:00
Sylpheed RE agent
027e6bb7a7 re: Placement_*/RouteTest_* are Null_Test.tbl, and Enumeration is a self-index
The template object is stage\Null_Test.tbl: its key hashes as
name_hash("stage\" + "Null_Test.tbl"), the file stage-definition-table names as
EnumerateNullFrame. Ten records - 4 Frame_Test_Asteroid_01..04, 3
Placement_00..02, 2 RouteTest_00..01, Enumeration - and the nine carrying a
FrameCount are exactly the "9/9 for Null_Test.tbl" stage-mission-tables already
counted. The corpus knew the file and the count, not the record names. A second
object (the same five plus Enumeration, without the asteroid frames) matched
none of ten candidate names across sixteen prefixes.

Enumeration is a record that indexes its own object. Disc-wide: 7750 IDXD
objects, 360 carry one, and in 360/360 its field names equal exactly that
object's other record names - zero subsets, zero strays. Field/record counts
run from 2 past 25 ((3,3)x36, (9,9)x24, (13,13)x24, (2,2)x24, (25,25)x18,
(16,16)x12), every pair on the diagonal.

Enumeration vs Enumerate is one letter and opposite scope: Enumerate is
archive-naming's route-2 seed, a single-record object whose fields name OTHER
objects; Enumeration names the records of its own object. Written up in
idxd-container, which had left "an object's kind is known only from the caller
that loads it" open.

Sharper version of the previous correction: I credited the stage\ prefix to
archive_naming.py, but stage-mission-tables states it in prose ("The prefix is
stage\ - name_hash is case-insensitive") and its stagetbl.py examples pass
'stage\AIParams_S02.tbl'. Noted in asteroid-fields.md.

All fifteen artefacts byte-identical.
2026-08-28 02:36:20 +00:00
Sylpheed RE agent
eac6759f25 re: the eight-value frame censused disc-wide, and my novelty claim withdrawn
Grepping FrameCount first would have saved the previous iteration an overclaim.

Withdrawn: the 8-value layout was not a discovery. stage-mission-tables already
documents Route_* records as (time, quat.x,y,z,w, pos.x,y,z) with
len(fields) == 8*FrameCount + 1 holding 1449 of 1449, and unit-group-table
already documents 1 + 8*FrameCount for formation slot lists. I re-derived the
shape from the asteroid tables and called slot 0 an "index" - it is the
corpus's time slot. Corrected in asteroid-fields.md. What the asteroid pass did
add is the norm test.

The census the corpus was missing: both prior statements were measured on one
table family each. Sweeping every IDXD record on the disc carrying a
FrameCount - 10986 records, 10974 with anon == 8 * FrameCount, and the entire
residual is 12 records: Formation_Fleet_01 (ratio 136) and Formation_Fleet_02
(ratio 4), six each, one per language pack. The two exceptions the doc already
names are the only two on the disc.

The quaternion is now measured, not asserted: over 78762 frames in five
families, |q| is within 1e-4 of 1.0 on 78762 / 78762.

Slot 0 is time only where something moves. It equals the running ordinal
0..N-1 in every formation (2196/2196), asteroid (384/384), placement (36/36)
and test-route (24/24) record, and in only 6228 of 8334 real routes - the one
family that stores true timestamps there.

Two record families new to the corpus: Placement_00..02 and RouteTest_00..01,
sharing a six-record template object whose sixth record is named Enumeration,
two such objects per language pack.

All fifteen artefacts byte-identical.
2026-08-28 02:20:15 +00:00
Sylpheed RE agent
1f0f90e304 re: the 8 values per asteroid are index, quaternion, position
Measured over every Frame_*Asteroid* record on the disc - 384 records, 37518
items (6253 per language pack x 6; the remaining 380 per pack belong to the
seven record-less Alpha frames, the same 57-vs-64 split as before).

  slot 0    equals the running index 0..N-1   37518 / 37518, max 587, and the
            largest FrameCount is 588, so the last index is N-1
  slots 1-4 a unit quaternion                 37518 / 37518 within 1e-4 of 1.0
            (min 0.999999, max 1.000001)
  slot 4    sign of the real part             min +0.001, never negative -
            canonical w-last, so the order is (x, y, z, w)
  slots 5-7 position in metres                -163858..+79572, -114308..+76422,
            -105224..+110263 - inside the +/-250 km half-extent of
            mapmesh_box_500km

len(anonymous fields) == 8 * FrameCount on 384 of 384, zero mismatches.

The unit-norm test is the control: an arbitrary 4-tuple has no reason to sit on
the unit sphere and every one of the 37518 does. Two further cross-checks fell
out - the maximum index is exactly max(FrameCount) - 1, and every position lies
inside the 500 km box the corpus already knows (world unit = 1 metre).

There is no scale slot: an asteroid's size comes from its group's
AsteroidModelName, which is why a group is one model plus one placement list.

Not settled: whether the quaternion is object->world or world->object and its
handedness - a norm test cannot separate those; and why S28_p1 has an asteroid
volume with no definition.

All fifteen artefacts byte-identical.
2026-08-28 02:03:23 +00:00
Sylpheed RE agent
863a88dbf3 re: EnumAsteroid counts the rocks, and the hash join was mine to fix
EnumAsteroid is the frame record's FrameCount - 57/57, zero mismatches. Each
AsteroidFrameName is itself an IDXD record name; those records carry one named
field, FrameCount, plus a run of anonymous ones, and the anonymous fields
number exactly 8 x FrameCount on all 384 occurrences (ratio histogram
{8.0: 384}). The frame record is the placement table, 8 values per asteroid,
and EnumAsteroid counts the rocks. The frame-name -> EnumAsteroid map is
single-valued for all 64 names.

mission-wave-arrivals already met these records - they were the outliers that
broke its route-speed measurement, filtered out by name. regn-map-grid already
guessed the link and said so honestly ("a reading, not a measurement; nothing
here counts objects"). Neither says the count is exact.

The abandoned table lost its placement data too: exactly 7 of 64 frame names
have no record, and they are exactly the seven groups of the unreferenced
tenth object (Frame_Alpha_S01_Asteroid_{01,02,04}, Frame_Alpha_S01_Haze_
{01,02,03}, Frame_TAlpha_S01_Asteroid_03). 57 + 7 = 64, zero partials.

Withdrawing the previous commit's red flag: the hash join works and my call
was wrong. AsteroidDefinition values resolve to their objects' pak entry keys
as name_hash("stage\" + name), 9 of 9, each landing on the object the
frame-prefix join independently predicted. I had hashed the bare file name;
archive_naming.py hashes prefix + name over a path-prefix list. Two
independent joins, 9/9 agreement, both leaving the same tenth object
unreferenced.

Not settled: what the 8 values per asteroid are; why S28_p1 has a volume and
no definition.

All fifteen artefacts byte-identical.
2026-08-28 01:48:52 +00:00
Sylpheed RE agent
c53da9ea2a re: the asteroid-field tables (AsteroidGroup_00N) decoded
New doc docs/re/structures/asteroid-fields.md, indexed.

Half the chain was already in the corpus: stage-definition-table and
stage-mission-tables record the per-phase AsteroidDefinition naming a .tbl,
and unit-datasheet-static partitions Generic and finds 10 per pack whose only
field is {EnumAsteroidGroup}, "an asteroid-group file". Neither says what is
inside them.

The table: 10 objects per pack x 6 packs = 60, 384 AsteroidGroup_* records,
4 or 7 groups per object. Every record carries exactly three fields, all 384,
no partials - AsteroidModelName (28 distinct; mob_n050..n056 x24 each,
mob_n040..n043 x18), AsteroidFrameName (64 distinct, Frame_S<NN>[_pN]_
Asteroid_NN, the same names regn-map-grid reports the Route tables showing),
and EnumAsteroid (37 distinct, 1..588, 6633 per pack summed).

Enum<Thing> is a COUNT; Enumerate<Thing> names a TABLE. EnumAsteroidGroup
equals its object's group count in 54 of 60 and EnumAsteroid is a plain
integer - the opposite of the Enumerate* join rule. The -erate is the
difference. Count-style fields still are not safe: the 6 failures are ONE
object repeated per language pack, declaring 4 while holding 7.

The tenth object is unreferenced and it is the one that miscounts. Joining by
frame prefix, nine objects match the nine AsteroidDefinition values one to
one; the tenth carries Frame_Alpha_S01_Asteroid, Frame_Alpha_S01_Haze and
Frame_TAlpha_S01_Asteroid and is named by nothing.

S28 has an asteroid VOLUME but no asteroid DEFINITION: mcol-collision lists
nine _AsteroidVolume_wp meshes (…, S28_p1); the nine AsteroidDefinition values
are those eight plus test_s8p1. Eight shared, one different each way.

Not settled: what EnumAsteroid counts of; why S28_p1 has a volume and no
definition. The name->object join did not reproduce by hash under name_hash,
mod 0x00FFF9D7, or a 24-bit mask - the join used is by frame prefix.

All fifteen artefacts byte-identical.
2026-08-28 01:32:52 +00:00
Sylpheed RE agent
9480f0aa8a re: the roster's two vocabularies, and a prefix match I mistook for a hit
Correction to the previous commit's own result. The raw byte sweep reported
rot_n001 "26 times in 7 archives" - a SUBSTRING count. The exact field value
occurs zero times, in every IDXD record and every non-IDXD payload. What the
disc carries is rot_n001_break x14, LOD_Frame_rot_n001_break x7,
GameModel_rot_n001_break x5. rot_n001 exact is not on this disc. A prefix
match is not an exact match, and a byte sweep is the tool that breaks it.

rou_e202 does exist, in the other vocabulary: a Model field of a Generic
record, six times, one per GP_MAIN_GAME_<lang> pack, with no GameResourceID -
which is why the GameResourceID census called it absent.

Set arithmetic over every IDXD record: roster 40, Generic.Model 46,
GameResourceID 480; roster n Model 34, roster n GID 38, roster n both 33.
38 + 34 - 33 = 39, so 39 of 40 are one or the other and rot_n001 is the sole
exception. The six that are GID but not Model: mob_n040..043 are the four
AsteroidGroup_00N.AsteroidModelName values (asteroid models, not units) and
rou_e004 is a Level_0.Model.

What selects the 40 is still open, but two answers are dead: not the
Generic.Model set (34 of 46), not the GameResourceID set (38 of 480). The 12
Generic.Model values outside the roster include rou_f002 and rou_f004 - the
player's own craft for S7-S16/S24-27,29 and for S28 - while rou_f001 is kept,
so "the playable models" is dead too.

All fifteen artefacts byte-identical.
2026-08-28 01:15:38 +00:00
Sylpheed RE agent
692a8045f0 re: the 60 duplicated functions are static-object constructors
Each of the 60 builds a 40-element array of 24-byte string objects: addis
0x820B + addi fetches the literal, addi r30,r11,12944 fixes the array
(0x828E3290 for the first), bl sub_8217FA08(dest, literal, -1) assigns, the
rest of the object is zeroed, addi r3,r30,24 steps on.

Why they have no callers: each appears exactly twice in the image - once in
the .pdata unwind table, where EVERY function appears as (address, prolog/
length word) and reading it as a registry is a trap - and once as a slot of a
null-terminated 182-entry function-pointer array at 0x82870018-0x828702EC,
170 of whose targets have zero direct xrefs. That is the static-initialiser
list. Zero callers here means "static ctor", not "dead code".

The teardown side confirms it: 39-43 functions sit between consecutive clones
(median 40), the dominant size in 0x8281xxxx-0x8284xxxx is 11 instructions
(2280 of them, about 60 x 40), and three consecutive ones call sub_823F3D68
on 0x828E3290, +24, +48 - one destructor thunk per element, same stride, same
global, all zero-xref.

rot_n001 and rou_e202 ARE on the disc: a raw byte sweep finds rot_n001 26
times in 7 archives (DefTables included) and rou_e202 6 in 6, exactly one per
GP_MAIN_GAME_<lang> pack. Controls: rou_e201 151/8, rou_e901 917/8, mob_n500
46/8. Narrows the earlier claim to "not GameResourceID values".

Not settled: what reads the arrays. An exact-address search returns 2347
functions outside the 60 - too broad to be a consumer list; the globals sit in
a dense data region and that route has no power.

All fifteen artefacts byte-identical.
2026-08-28 00:59:34 +00:00
Sylpheed RE agent
105ccab038 re: the base-solver's confidence axis was inverted
The remaining named false-positive mode - "107 rows solve to a 64K-boundary
base, a bare addis with no addi of its own, so any scatter of displacements
votes for it" - is refuted by its own measurement.

New positive test in the tool: simulate lis/addis rD,r0,HI + addi rD,rA,N +
or rD,rA,rA forward through each row's function and ask whether the solved
base lands in the solved register.

  64K-boundary bases ("low confidence") : 107 / 107 confirmed
  non-zero low half ("trustworthy")     :   8 / 154 confirmed

A round base is the case where the compiler needed no second instruction, so
`addis r11, r0, 0x820B` stands in the code in full. A miss on the other class
is silence (base built in the caller or loaded from memory), not refutation.

Control: every row the corpus independently validated against the disc has a
64K-boundary base - debriefing, career, save, leaderboard, the 205-name PG*
HUD roster, material slots, the S16 boss collision/frames/motions and its
loader. 13 rows over 10 functions. The dense-short-string false positives the
corpus did name (r31 = 0x8202xxxx) all sit in the "trustworthy" class.

The 0x820B0000 cluster is DUPLICATION, not error: 60 of its 82 rows are one
function emitted 60 times, exactly 491 instructions each, two instructions
differing (both global data pointers), identical 41-address string sequences.
40 resource names written into a per-copy global via sub_8217FA08 at 24-byte
strides. 38 of the 40 are disc GameResourceID values (480 distinct); rot_n001
and rou_e202 are not, and no disc GameResourceID uses the rot_ prefix.

Artefact diff 13/4, confined to the replaced section; the 261-row table and
the 64K histogram untouched; byte-identical on a second run. Fourteen other
artefacts byte-identical.
2026-08-28 00:42:46 +00:00
Sylpheed RE agent
bb6fb9024b re: r0 is not a base register - the last shortlisted row withdrawn
sub_825F2CF0 / sub_825F2F88 (r0, 30 names each, 97% disc names) are not
name-block readers at all: they are the compiler's VMX save/restore helper
pair. 145 instructions each, 72 differ, every difference stvx<->lvx over
v14..v127.

Root cause: in `addi rD, rA, N` the rA slot reads as literal zero when it
names r0, so `addi r11, r0, -N` is `li r11, -N` - vector spill offsets, not
displacements. The solver excluded r1 only; the bogus base landed inside the
stage-settings name block, dense enough to "resolve" 30 real field names.

name_block_bases.py now skips r0 as well. Rows 277 -> 261, functions 190 ->
176, non-64K bases 170 -> 154, data-table rows 53 -> 50; 154+107=261 and
277-16=261. Artefact diff -100/+3, every removed row line carries r0, control
sub_82341A20 r30 = 0x82088F94 217/226 untouched.

Nothing lost: all 30 names are a strict subset of sub_8230D1F8 r29 (129
names), the stage-settings loader already in the corpus. String-xref join
agrees - FinalPassBG, FogMinDistance, ScreenColorR, ExposureKey_BeginValue
each exist once with exactly one xref, from sub_8230D1F8.

Fourteen other artefacts byte-identical; name-block-bases.txt byte-identical
on a second run.
2026-08-28 00:21:26 +00:00
Claude (auto)
aeec0dab8f re: the leaderboard screen, and a Detail_* prefix trap
sub_8219E560 r11, censused against tables.pak exactly like the result screens.
444 instructions, one call site (sub_821A2A80), 100 % of its names are tables.pak
names.

The 18 names partition exactly: 13 record names + 5 field-only names.  The 13 are
the 12 Detail_* panel elements plus MISSIONS -- a 0-field record already owned by
isl-condition-builtins.md and challenge-mission-gate.md, so not new.  The five
field-only names are the screen's parameters: DETAIL_TITLE, one field in each of
six mode records (ScoreAttack, TimeAttack, Extra01..Extra04), and
Detail_Board_{Permanent,Monthly,Friend,Self}, four fields of CHIPS -- the board
scopes.

All 12 Detail_* record names are in the code block, zero residual -- the opposite
of the boss motions, where the block was a 25-of-55 subset.  The panel is Rank,
GamerTag, Main_{Time,Points}_{Self,Live}, Warships, Warplanes,
Weapon_{Nose,Main1,Main2,Main3}: your figure against the selected entry's, plus
that pilot's kills and four weapon slots.  _Live, GamerTag and the Friend/Monthly
boards make it the Xbox Live detail view.

The trap worth keeping: tables.pak has 12 Detail* RECORD names and 12 Detail*
FIELD names, and they are DISJOINT (0 overlap).  The fields are a different axis
-- 4 board scopes + 6 modes + Detail_Window_{Known,Unknown}.  A Detail_* name
means nothing until you say whether it is a record or a field.

Not settled: what Detail_Window_Known/_Unknown switch between, and whether
Extra01..04 map onto the four challenge missions.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 23:58:13 +00:00
Claude (auto)
24c3168768 re: the S16 boss's frames and motions; attackC is a cut attack
sub_823BDAA8, the next boss piece.  The subsystem is a linear chain, one call site
each: sub_82385D48 -> sub_823AE908 (collision pairs) -> sub_823BDAA8 (frames +
motions) -> sub_823C0260 (hull part list).

r11, 33 names = the attach-frame roster, partitioning exactly: 15 hull frames + 9
muzzles (GN_MainGun_01_MuzC and GN_MainGun_02_Muz01..08 -- one gun with a centre
muzzle, one with eight) + 1 shield + 3 gear + 5 model ids = 33.  The 15 hull
frames are the same 15 the collision table pairs, and rou_e901_body_01 is present
here -- the one whose _c mesh the collision list did not carry.

r10, 25 names = a SUBSET of the disc's motion set.  The disc object is DefTables
2ef95cdd: 58 records = 3 structural + 55 motions; all 25 code names are in it
(25/25); 30 disc motions are not in the code block.

A reading I nearly published, refuted by checking the other population: in the
code block Motion_guard_start stands alone while every other stem carries _b1/_b2,
which looks like the exception.  It is not -- the disc has guard_start_b1/_b2 plus
guard_keep and guard_end with their variants.  The asymmetry is in the code's
list, not the data.

attackC is a third boss attack the executable never names.  Most of the 30 omitted
motions do exist as strings elsewhere (Motion_guard_keep, Motion_dead,
Motion_break, Motion_damage01 -- one row each), but the whole
Motion_attackC_{start,keep,end,charge} x {,_b1,_b2} family, 12 names, has 0 string
rows, against two identically-shaped controls at 3/3 (attackA_charge,
attackB_charge).  The boss ships with an animated third attack pattern no code
path can select by name.

Not settled: whether attackC is reachable by index rather than name, and what
eff_s900_e_Charge attaches to.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 23:45:28 +00:00
Claude (auto)
e548db693e re: the Stage 16 boss closed end to end; five disc fields the image never names
New doc docs/re/structures/stage16-boss.md joining three iterations' pieces: the
data object (player-tuning-tables), the collision table (collisionset) and now the
loader.

sub_82315AE8 r11 is the boss loader, and it says so itself: 558 instructions, one
call site (sub_82308AB8), and its 21-entry block is 19 field names plus the
Japanese error "the boss's laser shell ID could not be obtained".

Zero partials: every one of the 19 is a Guardian or Core field -- 16 of Guardian's
19, 5 of Core's 10, DamageLevel1/2 shared.  The boss reads as a rotating,
vertically-moving platform with a guard cycle, three shell ids (AAGun, Laser,
HomingLaser -- the Shell_S16Boss_* set) and a Core that periodically drops
lock-on.

The residual is the finding: 8 disc fields the loader never names, and five of
them are not strings in the executable at all -- InitHight (the game's own typo
for Height) and the four DamageLevel{1,2}Bomb{,Rand}Time.  Controls: ChargeTime
(10 chars) and GuardTime (9) are the same shape and both ARE present, so it is not
an extraction artefact; HP (2 chars) is below the extractor's floor (shortest row
is 6); Radius exists but is generic.

So the disc carries fields no code names -- the mirror image of the usual trap, a
loader naming a field the disc never values.  Whether they are read positionally
or are dead data is not settled.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 23:32:09 +00:00
Claude (auto)
ced73c5488 re: all 277 base-solver rows classified objectively; 53 are data-table schemas
name_block_bases.py extended with a per-row data-table test; artefact +57/-0,
byte-identical across two runs (now ~2 min 12 s -- it adds a disc-wide pak scan).

The test is objective, not by eye: a row is a data-table schema if its names are
IDXD record/field names on the disc (13450 such names disc-wide).  53 of 277 rows
are >=50 % disc names with >=8 names; the other 224 are engine/XDK vocabulary,
compiled key lists, or noise.

The two axes are independent: against base confidence, solved bases split 34
table / 136 not, round bases 16 / 91.  "Round base" and "not a table" are
different questions.

The 53 contain every loader already known -- that is the control.  Five rows in
the 53 are unowned, each noun grepped and appearing in no docs/re/ file:
sub_823BDAA8 r11 (33) = the S16 boss's muzzle/attach frames (GN_MainGun_*_Muz*);
sub_823BDAA8 r10 (25) = motion names (Motion_stand, Motion_attackA_start), the
EnumMotions family DefTables declares; sub_82315AE8 r11 (20) = the Guardian
record's own fields, i.e. the S16 boss loader; sub_8219E560 r11 (18) = the
leaderboard screen keys; sub_825F2CF0 + sub_825F2F88 r0 (30 each, same base) =
post-processing (FinalPassBG, FogMin/MaxDistance).

Four rows that look new are not, and their disc-overlap says so -- 53-70 % rather
than ~100 %, because they mix arsenal fields the corpus owns (ConditionToDevelop,
WeaponDesc, SilhouetteModel) with literal screen coordinates as strings.

Not settled: none of the five was opened -- this iteration produced the shortlist,
not the findings.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 23:09:57 +00:00
Claude (auto)
dd6b13c751 re: the last base-solver blocks -- two findings, and a correction to how I read the index
CORRECTION FIRST: a row in name-block-bases.txt is a (function, REGISTER) pair,
not a function.  Every one-line label I carried for these blocks quoted one of
two rows, and the other row of the same function is an unrelated schema:
sub_822E3EC8 r11 = 29 shader constants + techniques, r10 = 15 material map slots;
sub_822AFA50 r11 = XDK shader-compiler tokens, r10 = 13 menu camera tags;
sub_823AE908 r11 = the 43-name S16 boss collision table, r31 = 33 shader tokens
(the dense-block false positive).  Quote the register.

The engine's material vocabulary (into xbg7-mesh.md, which owns the
_col/_spc/_gls/_lum suffixes): nine base map slots -- ColorMap, LuminosityMap,
SpecularityMap, GlossinessMap, ReflectionMap, TransparencyMap, NormalMap,
FresnelMap, EdgeTransparencyMap -- and seven *MapArray twins; 9 - 7 = 2, and the
two without a twin are FresnelMap and EdgeTransparencyMap.  Four of the nine are
exactly the four documented suffixes; the other five have no suffix in the
shipped meshes, so the engine supports more channels than the assets use.  Plus
17 g_* shader constants, three cube samplers, and DepthOnly / MotionVector /
Transparent techniques -- the ...Last matrices with the motion-vector technique
say the renderer keeps the previous frame's per-object transform.

The S16 boss's collision parts (into collisionset.md): sub_823AE908 r11 is an
interleaved (mesh, logical-part) table -- rou_e901_body_02_c ->
Boss16CollisionBody02, five wing pairs, both tails, barrier, rou_e910_core_cmesh
-> Boss16CollisionCore, and mob_n900{,_b01,_b02,_gear}_cmesh -> Room00/01/02 +
Gear00/01/02.  23 logical parts + 20 meshes = 43, differing by exactly three:
Body01 opens with no mesh and the one gear mesh carries three gear names.  Cross-
links the S16 Guardian/Core object found during the PlayerParams work.

Not settled: whether those meshes are CollisionSet entries, why Body01 has no
mesh, and no shader slot was traced to a shipped shader.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 22:51:53 +00:00
Claude (auto)
e7c89fdcd1 re: the debriefing and pilot-record screens; the save-screen block was already owned
Read-before-claiming caught one of the three.  savegame-format.md already
documents the shape AND this exact list -- the LOAD/SAVE screen's config key list
is compiled into the executable as a pointer array of key strings, run starting
0x820A0074.  sub_82286BC8's 18 names are that list (17/18 are a tables.pak field
or record).  Nothing new; residual PLAYER_AMMO_LESS_10, not a tables.pak name.

The other two are new screens of the same shape and they close exactly.  All
names resolve into tables.pak: sub_822814D8 24/24, sub_8227A3A0 21/21.

sub_822814D8 = the debriefing screen.  24 = 2 screen ids (STAGE_RESULT,
EX_STAGE_RESULT) + 21 stage_* fields + one sound cue (SE_BOSS_CORE_CHARGE); the
tables.pak STAGE_RESULT record has exactly 21 fields.

sub_8227A3A0 = the pilot record / career screen.  21 = 5 screen ids (LAST_RESULT,
EX_BASE, EX_MAIN, EX_OVERVIEW, OVERVIEW) + 7 ex_overview_* + 9 overview_*; the
records EX_OVERVIEW and OVERVIEW have exactly 7 and 9 fields.

2+21+1 = 24 and 5+7+9 = 21, with 21/7/9 measured independently off the pak.

The debriefing readout is nine num/points pairs plus three points-only lines:
kills by class, objectives, clear time, shoot-down ratio and both damage
categories carry a raw count and its score contribution, while friendly_fire,
shoot_down_others and weight have no counter column -- the same partition
mission-scoring.md measures on the settings side.  OVERVIEW is the seven
EX_OVERVIEW fields plus overview_rank and overview_medals.

Not settled: what EX_ means (its ids have no record of that name), and
PLAYER_AMMO_LESS_10.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 22:38:56 +00:00
Claude (auto)
418fbf30ed re: the executable's own PG* HUD roster; the disc side was already owned
Took the biggest of the five unowned base-solver blocks, sub_822215D0 (205/206).

Grep-and-READ first, and it paid.  PGHUD* pointed at mission-script-manifest.md,
which documents HudResource's 24 fields -- but hud-config.md owns the whole
family: all 16 records in the six GP_MAIN_GAME_*2D paks, 419 distinct asset
paths.  Stopping at the first grep hit would have "discovered" a documented table
for the second time in three iterations.

Control: counting PG* field names straight off the paks reproduces that
document's own numbers exactly -- Sight 35, Radar 29, Marker 27, Wing 23,
Manuva 15; HudResource 23 PG* + HUD_RES_FONT = its stated 24.

The increment: the executable carries its own roster.  sub_822215D0 reads a
205-name block at base 0x820A0000, 205/205 PG*-prefixed, zero partials -- PGHUD
77, PGTARGET 31, PGGAUGE 25, PGMANUVA 16, PGMARK 12, PGWARNING 10, PGMARKTGT 9.
The disc has 228 distinct PG* names; 200 are in both, 5 code-only, 28 disc-only.
200 + 5 = 205 and 200 + 28 = 228, closing both ways.

Four of the five code-only names are printf families whose stem is a disc field
(PGHUD_HIT_NUM_EFF%d, PGHUD_HIT_NUM_RED%d, PGREMAIN_NUM%d, PGTIMER_NUM%d) and all
four stems sit in the disc-only 28: the code builds indexed instance names from a
base name the table declares.  After pairing, the residual is one code-only name
(PGMANUVA_EFF0) and 24 disc-only ones, listed in the doc.

Not settled: what the block is for -- the function body was not read.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 22:25:40 +00:00
Claude (auto)
9e2524d028 re: every AIParams object mapped to its stage, zero residual
StageResource.EnumerateAIParams names the table, and every name resolves.

29 StageResource records; 28 carry EnumerateAIParams (the one without it is the
_Test template); 23 distinct table names declared; 23/23 hash to an AIParams
object key under the prefix "stage\"; 0 objects left unnamed; 0 declared names
with no object.

AIParams_S01..S16 (16) + AIParams_S24..S29 (6) + AIParams_Tutorial.tbl shared by
six (UnitGroup_S18..S23, the tutorials) = 23 tables over 16 + 6 + 6 = 28 records.
The arithmetic closes both ways and matches the 23-object count found last
iteration.

Same sharing shape as the settings family (stage-settings-table.md: 24 objects,
StageParameter_Tutorial shared by six tutorials).  Two independent families agree
on how the tutorials are handled -- n=2, a pattern rather than a rule.

aiparams_census.py extended with the stage mapping; artefact +7/-0, byte-identical
across two runs; the other fourteen verify unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 22:12:58 +00:00
Claude (auto)
2456d28f1a re: AIParams disc-wide, and a correction to my own last entry
CORRECTION FIRST.  Last iteration I recorded sub_8233C368 as unblocking a
NEEDS-HUMAN item -- "the corpus carries the AI tail of Maneuver as
NEEDS-HUMAN/runtime; it is statically reachable after all."  That was wrong.
stage-mission-tables.md already documents AIParams_S02.tbl as exact original
values obtained by static RE, directly portable, listing all 20 field names and
both shapes.  I grepped FiringLength and saw the file but did not read the
section.  Finding the owning doc is not reading it.  The only genuinely new part
was the loader's name.

What is new: the census generalises Stage 02 to the disc.  23 AIParams objects,
identical in all six GP_MAIN_GAME_* paks, sharing ONE declared-name set of 34
profiles; 782 profile records = 23 x 34; 0 declared names without a record in
their own object.  So "34 AI profiles" is not a Stage-02 fact -- every stage
carries the same 34 and only the values move.  The roster is declared by an
Enumerate_AIs record whose field names are the profile names, the same
declaration-table mechanism that closed DefTables.

Type predicts the field count with exactly two exceptions: Fleet -> 6 fields is
253/253 zero partials; Squad -> 20 fields is 483/529.  The 46-record residual in
full: AI_Test and AI_CraftSquadron_Test, both Type = Squad with only the six base
fields, in all 23 objects.  No profile's shape varies between objects.

New regenerator aiparams_census.py, 45-line artefact, byte-identical across two
runs; the other fourteen verify unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 21:57:24 +00:00
Claude (auto)
17b4a791eb re: mining the base-solver index -- a false-positive mode named, and the AI-table reader found
277 rows over 190 distinct functions (a function can read several blocks).
name_block_bases.py extended with a confidence split.

The tool's false-positive mode, measured and named: 107 of 277 rows solve to a
base on a 64K boundary -- a bare "addis rX, r0, 0xHHHH" with no addi, so any
scatter of displacements votes for it.  82 are 0x820B0000: about 60
near-identical functions in 0x8281xxxx-0x8284xxxx all "naming" the same rou_e0NN
list.  The 170 rows with a non-zero low half are the trustworthy set.  A round
base is not automatically wrong -- sub_822215D0 sits on 0x820A0000 and resolves
205/206 -- so read the ratio, not the base.

The index re-derives every loader we already knew (unit 217, stage settings 129,
PlayerParams 90, hangar 81, squadron orders, missile guidance, shell movement,
substructures, six camera/fog readers) -- that is the control.

The find: sub_8233C368 reads the AI behaviour table.  r28, base 0x8208583C, 20
names -- Enumerate_AIs, FiringLength, GuardLength, AutoGuardLength, CounterLength,
MusterLength.  stage-mission-tables.md owns those field names on the data side,
but Enumerate_AIs appears in no document and no reader was known; the corpus
carries the AI tail of Maneuver as NEEDS-HUMAN/runtime.  It is statically
reachable after all.  The same base also serves sub_82338EE0 (97 names, Weapon
TargetType SpecialWeaponType ReticleType IsCharging ...) -- the weapon datasheet
loader, also not previously named.

Five unowned blocks surfaced and NOT opened: PGHUD_*/PGREMAIN_NUM HUD part names
(205/206), STAGE_RESULT/stage_num_shoot_down_aircrafts/EX_OVERVIEW,
g_mWorldViewProjection/NormalMap/GlossinessMap engine material slots,
Boss16Collision* (cross-links the S16 Guardian object), and roh_n001_menu1_cam_pos
menu camera tags.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 21:41:03 +00:00
Claude (auto)
ae37e15d7d re: the base-solver -- 277 name-block loaders indexed, and the analog block is SOLVED
New reusable tool, tools/re-capture/name_block_bases.py -> docs/re/data/name-block-bases.txt
(2880 lines, ~65 s, byte-identical across two runs).

A loader that reads a table by field name keeps one base pointer and emits
"addi rX, rBASE, -N" per name, so no static xref sees the strings.  Solve the base
from the DISPLACEMENT SET alone: every (string address, displacement) pair implies
a candidate base, and the true base collects a vote from every name it explains,
so it wins outright.  My first cut took candidates from ONE displacement and
scored the unit loader at 52/226 against the right answer's 217/226 -- vote over
the whole set, not a probe.

Control passes with no prior knowledge: the tool recovers sub_82341A20 -> r30 =
0x82088F94 at 217/226, and independently recovers sub_8230D1F8 (129/132),
sub_822F9498 (90/91) and sub_822AE628 (81/108).  277 name-block-reading functions
image-wide, with the schema each names.

The analog block is SOLVED: sub_821A6CF0, r29 = 0x820A1630, 22/24.  In code order
it names ControlTweakName, YawMagForNormal, the 12 Tweak fields, the 8
AnalogRevice_* curves and GP_MAIN_GAME -- the whole schema in the object's own
order plus its pak.  r29 is built at 0x821A6D34 as addi r29, r11, 5680 =
0x820A0000 + 5680, matching the solved base exactly.  It is the same function that
reads PlayerParams.

Two of my own verdicts withdrawn: "referenced by nothing" and the softened "not
found by these routes".  The measurements behind them were right; the conclusions
were wrong.  The base was solvable from the data the whole time.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 21:22:50 +00:00
Claude (auto)
38e0a63946 re: the unit loader was already in the corpus (sub_82341A20), and 0-xref does not mean no reader
Item "what loads the 114 unit Maneuver records" -- the corpus already answers it,
and I should have found that before framing it as new.  My own error, corrected:
I claimed live-unit-definitions.md did not exist.  It does, at docs/re/, not under
structures/.  Looking in one directory is not looking.

sub_82341A20 is the unit-definition loader, documented in unit-struct-runtime.md
with a checked-in 177-line layout at
crates/sylpheed-formats/data/unit_definition_layout.txt.  Verified independently:
3969 instructions, 227 of its 236 "addi rX, r30, -N" displacements resolve to
strings -- the unit schema in order, Generic, UncertainName, IsDestructible,
NamePlate, Size_X/Y/Z ...  Maneuver sits at exactly 0x82088F94 - 13404.

Record keys are tag_hash, confirmed on the data side: the Maneuver record's key is
0x43FAA517 = tag_hash("Maneuver"), not name_hash (0x63A248D7), x114 records.

That explains the 0-xref phenomenon and softens my own verdict from last
iteration.  Maneuver's string has 0 xrefs AND a known reader: the loader takes its
address as a register-passed base plus a displacement, which no static xref can
see.  So the AnalogRevice/Tweak verdict goes from RED to AMBER -- a
base-plus-displacement sweep over the 2283 functions forming the 0x820A high half
reaches it 0 times, but run against the unit block as a control the same sweep
recovers only 5 references and misses sub_82341A20 itself.  The measurement
stands; "the reader is unreachable" does not.

Also fixed: player-tuning-tables.md credited live-unit-definitions.md with
documenting sub_822F9498.  It documents sub_82341A20, a different function.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 21:02:58 +00:00
Claude (auto)
6334f36dde re: the object is PlayerParams, and sub_822F9498 is ITS loader -- not the unit loader
Resolving every string sub_822F9498 references, in code order, gives 90, and they
are exactly the 890e1be4 object's schema in its record order: Misc (5 effect
names), SpecialAttack (+17), TacticalManeuver (+7), SpecialWeapon (+8), Booster
(+50).  It never names Generic, Maneuver, Explosion, Shield or StructureCount, and
the string Maneuver has 0 xrefs.  One call site, from sub_821A6CF0, which is
itself called once and references the literal PlayerParams.

CORRECTION to unit-datasheet-static.md, which calls sub_822F9498 the
unit-definition loader.  It loads one object: the player parameter table.  Its
AA_/AV_ interleave still stands as a struct layout, but the struct is
PlayerParams's Booster record, not each unit's Maneuver.  What loads the 114 unit
Maneuver records is now an open question -- it uses none of these strings.

Ranking all 114 units by how many of Booster's 50 values they reproduce: the top
five are exactly the five _Player units, all at 39/50; the sixth drops to 13/50;
none matches 50/50.  The five players agree with each other on all 50 -- the three
player ships share one flight model.  Booster stands alone on 10 fields; the
eleventh, AA_Yaw_Max, is 65.0 vs 65, formatting not a value (last iteration's
"11 differ" over-counted).

Still open, and sharper: nothing selects Booster.  PlayerParams is loaded once,
unconditionally, no branch -- yet the measured flight tracks the unit Maneuver
(1.25/1.20/1.28) and misses Booster badly.  Both cannot govern the same craft; the
next test is a runtime watch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 20:48:56 +00:00
Claude (auto)
92d1a39ce2 re: the player-tuning consumer -- found for half, BLOCKED for the other, two of my labels refuted
The consumer is sub_822F9498, the unit-definition loader the corpus already
documents.  The field-name block 0x8209F300-600 (BulletTimeVolumePC, FireBirdPower,
ChargeMaximum, Turn180RequiredTime, ShieldDoubler_Time, ...) is 36/36 referenced,
every reference inside that one function.

REFUTED, my own label: "Booster is a new schema".  Its 50 field names are a strict
subset of the unit Maneuver record -- 50/50 shared, 0 Booster-only.
CruisingVelocity occurs in exactly 115 records disc-wide: 114 Maneuver + this one.
The first check compared against Generic and scored 0/50, which read as brand new:
the wrong record of a multi-record object is not a control.

REFUTED: "Booster is the player craft's flight envelope".  flight-speed-law.md
measured ~125/~420/~1530; unit Maneuver 100/350/1200 gives consistent ratios
1.25/1.20/1.28, Booster 100/612/2100 gives 1.25/0.69/0.73.  Booster is a SECOND,
faster profile: 39 of 50 values identical to the player unit, and of the 11 that
move the three velocities scale by exactly x1.75 and two accelerations by x1.5.

BLOCKED: the AnalogRevice/Tweak reader.  That schema is its own contiguous block at
0x820A119C-0x820A1378 headed by ControlTweakName -- 0 of 28 strings has a code
xref and no instruction anywhere names a 0x820A1xxx operand, against a control of
36/36 on the neighbouring block.  Runtime watch needed.

Caught on the way: mov_stick_play/eye_stick_play exist twice; the referenced copy
belongs to sub_822AE628, which reads GP_HANGAR_ARSENAL's ControlTweak records (6 of
them, and none anywhere else) -- a clean, wrong answer if taken for ours.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 20:33:27 +00:00
Claude (auto)
c171a194d9 re: the GP_MAIN_GAME unnamed block -- 333 of 337 already owned, 4 genuinely new
All six language copies carry the identical 337 unnamed IDXD hashes.  Partitioned
by record-name shape (53 shapes), 333 map onto families the corpus already
documents: 131 weapon datasheets, 114 unit datasheets, 64 unit Faces tables, 10
unit message sets, 8 chatter rule tables, 5 enumerations/formations/placement.

114 is exactly the corpus's unit count (43 Craft + 71 Vessel) -- these are the
tables the corpus has always worked with, reached by SHAPE because they have no
names.  Naming them adds nothing.

The Enumerate object in each GP_MAIN_GAME_* is EMPTY, zero fields, which is why
route 2 named 1283 entries in DefTables and 0 here.

The 4 unclassified objects are new -> docs/re/structures/player-tuning-tables.md:
the analog stick response curves (8 axes, 11 samples + a named Count = 11, tested
8/8; yaw/roll/throttle are the identity ramp, the shaping is all on pitch and the
camera axes; adv_yaw is non-monotone and unexplained), the player craft's flight
envelope (Booster, 50 fields -- the player side of the AA_/AV_ pair documented
for NPCs -- plus TacticalManeuver, SpecialAttack + three gauge bands,
SpecialWeapon, Misc), the Stage 16 boss (identified by Shell_S16Boss_* ids;
Guardian HP 65000, Core 42000), and one unidentified Generic naming eff_n0071.

New regenerator main_game_unnamed.py, 112-line artefact, byte-identical across
two runs; the other twelve artefacts verify unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 20:17:10 +00:00
Claude (auto)
53084a11ca re: the Enumerate declaration tables close DefTables (130 -> 1413 of 1425 IDXD)
The string harvest could never name DefTables because those names are not
spelled out as "something.tbl" anywhere on the disc -- they are declared.

An IDXD object whose single record is named Enumerate is a DECLARATION TABLE:
its field NAMES are the names of other objects, each resolving as
name_hash("<field name>.tbl").  EnumLODSet_test.tbl declares 676,
EnumGameModel_test.tbl 360; the disc holds 144 such objects (138 in DefTables,
one in each GP_MAIN_GAME_*) declaring 1298 distinct names.

Route 2 names +1283 entries route 1 could not.  130 + 1283 + 12 = 1425, no
overlap, 99.2 % coverage.  Zero partials: of 5 suffixes x 6 prefixes, ('', .tbl)
scored 1036/1036 and every other combination scored 0.  Residual in full: 8
declaration tables nothing declares, 2 LOD sets (Model rou_e004 / rou_e013), 2
motion sets; 15 declared names have no pak entry at all.

REFUTED alongside it: the 40 XPR2 manifests are not the naming source -- their
82 Name= values and 82 DataFile/Source paths resolve 0 entries under any of the
5 suffixes.  They share the MODEL namespace only: 40 of the 82 appear as the
Model field value inside the tables.

Artefact +17/-18, every line paired, byte-identical across two runs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 19:59:20 +00:00
Claude (auto)
bcc23cba4a re: why each archive is low-coverage -- two different reasons, cleanly separated
Splitting every unnamed entry by magic turns the coverage percentages into an
explanation.

The three low-coverage UI archives have ZERO unnamed IDXD.  GP_HANGAR_ARSENAL
is 180 IDXD, 180 named, 0 unnamed -- its 1191 unnamed entries are 1149 T8aD/RATC
plus 42 LSTA, i.e. sprites.  GP_MISSION_SELECT and GP_DEBRIEFING_PILOTLOG hold
no IDXD objects at all.  So "22.6 % named" is misleading: every data table in
that pak is named, and these three are the same artwork-naming phenomenon as the
blocked 2D and READY_ROOM archives.

DefTables is the only genuine data gap: 1425 IDXD, 130 named, 1295 unnamed, in
17 record-name shapes -- Generic + Level_0..Level_3 (807, LOD sets) and Default +
EnumMotions + Generic + ReferenceFrames +/- Motion_break/dead/down (463, motion
sets).  One LOD and one motion table per model.  Level_0, EnumMotions and
ReferenceFrames appear in no document.

Control: the 100 % archives have no unnamed entry of any kind, and GP_MAIN_GAME_E
is 667/337 IDXD with zero unnamed artwork -- two failure modes, not a gradient.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 19:41:29 +00:00
Claude (auto)
a543b8da96 re: correction -- the naming sweep already covered the 24; it just never said so
archive_naming.py already harvests 6027 candidate names under 16 prefixes, and
testing its candidate set directly shows it names all 24 StageParameter_S<NN>
objects, 24/24.  The previous entry presented that naming as new -- it is not.
What was new was the identification (which object is which stage, the shared
_Tutorial table, IsBoss16Enable = S16), not the method.

The real gap, now closed: the sweep reported only per-archive percentages and
never emitted WHICH entry got which name, which is exactly why nobody could say
the settings objects were StageParameter_*.  It now prints the resolved name
families per archive -- 6573 named entries, 1631 families disc-wide.

Determinism caught again by the verify loop: the resolved map was built by
iterating a set, so collided hashes picked a different winner each run.  Now
iterated sorted().  Second time in two iterations -- any map built from a set
needs a sort.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 19:25:30 +00:00
Claude (auto)
589d09f2b5 re: all 24 settings objects named -- stage\StageParameter_S<NN>.tbl
The queued item (the 29-object resource manifest) turned out to be already
settled by challenge-mission-gate.md and stage-mission-tables.md, so the
iteration moved to the open question those docs make answerable.

Hashing EVERY string that appears in any IDXD object on the disc resolves all
24: StageParameter_S01..S16 (story), S24..S29 (challenge), StageParameter_
Tutorial shared by all six tutorials, and StageParameter_Test.  16+6+1+1 = 24,
which is exactly why the settings family is 24 against 29 stage records.

IsBoss16Enable confirmed independently: that object is StageParameter_S16.
GravityFactor is non-zero in S10 (700), S11 (400), S03 (250), S27 (250); the
three unscored objects are S24, S27, S28; the one without SplinterCell is _Test.

Refuted on the way: TOC order is not stage order (the TOC is hash-sorted and the
Boss16 object is 17th, not 16th), and neither the StageResource template nor a
real 19-field record names the settings table.  The AUTO_SETTINGS filenames are
a different scheme -- none of the 28 hashes to any of these 24.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 19:05:28 +00:00
Claude (auto)
5f764458cd re: the Phase_1/2/3 block of the settings family -- mostly per stage, not per phase
72 records, 94 distinct field names, 31 in every one (post-processing core, five
Fog*, three ScreenColor*, SpaceSize, five Supply*, UnderCommandSquadron, the
first BGOperate slot).  Optional families sit in clean tiers: Nebura_* and
ColorLayer* in 69/72, DOF_*/UnsharpMask_*/ExposureKey_* in 15/72, FinalPass* in
3/72.

REFUTED: BGOperateFrameCount is NOT the number of BGOperateFrameName_i slots --
36 of 72.  What holds is Count <= slots, 72/72: a fixed slot array with a live
count, the same shape as MessageCount under the 32-slot clamp.

68 of the fields common to all three phases NEVER differ in any of the 24
objects.  The phase block is a per-stage environment block copied three times;
what a phase change is actually for is the backdrop animation (BGOperate*) and
the supply/command squadron assignment.

MapPath is not in this family -- its 87 records per pack are 29 x 3, the
resource manifest's Phase_N.

Also fixed a non-determinism the verify loop caught: most_common() over a set
iteration ordered ties differently per run; now sorted by (-count, name).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 18:50:27 +00:00
Claude (auto)
26b7728ceb re: the rest of the stage-settings object -- cameras, player limits, difficulty
53 objects in GP_MAIN_GAME_E carry a Phase_1 record, in TWO families: 29 are
the resource manifest the corpus already owns (Phase_N = 4 fields) and 24 are
the settings table (Phase_N = 31-90 fields).  That answers the 24-vs-28 puzzle
left open by the scoring entry -- the 29 is a different table, not the settings.

Camera: three chase rigs in metres, 13 of 14 fields identical in every stage --
Nose (0, 4.5, 7), Near (0, 10, 40), Far (0, 15, 80), FOV 0.92; only CameraFar
varies, once.  Player: BulletLimit 512 / HomingLimit 256 / LaserLimit 32 and
the three 0.30 axis adjustments are constant, GravityFactor is non-zero in 4 of
24 stages, and IsBoss16Enable appears in exactly ONE object -- the first
per-stage handle for a family whose filenames do not resolve.

Difficulty_Easy/Normal/Hard is a SECOND difficulty record (8 damage and
guidance multipliers), separate from Score_*.

New doc structures/stage-settings-table.md; mission_scoring.py extended.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 18:35:05 +00:00
Claude (auto)
192d3bf67f re: mission scoring and the S/A/B/C/D rank thresholds
sub_8230D1F8 -- the loader that contaminated the AA_/AV_ offset search -- is the
stage-settings loader.  Reading its 122 field NAMES instead of its offsets found
the scoring block, which nothing in docs/re owned.

24 IDXD objects per language pack x 6 = 144, each holding Score_Easy /
Score_Normal / Score_Hard: 72 records per pack on one 22-field schema, no
variants.  Difficulty moves 10 of the 22 fields and never the five RankScore_*
thresholds -- the rank bar is per stage, difficulty scales the earning rate
(x0.5 / x1.0 / x2.0) and the penalties.  23 of 24 objects differ from the
commonest Normal record; 9 of 72 records zero the scoring entirely.

Not settled: which object is which stage.  None of AUTO_SETTINGS's 28 filenames
resolves to any of the 24 under 19 prefixes, and 24 vs 28 is unexplained.

New doc structures/mission-scoring.md, regenerator mission_scoring.py, artefact
data/mission-scoring.txt.  Twelve artefacts now.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 18:22:36 +00:00
Claude (auto)
1a1a8d98d9 re: the AA_/AV_ selection is BLOCKED for static RE, with the mechanism
Route 3 (call-graph bound): sub_821A6CF0 and sub_821AB650 each read 196/200/204
off THREE different base registers -- three unrelated objects, not the block.

Route 4 (data-flow bound): the definition object lives in the global at
0x828F358C; 18 functions touch it, 9 also touch block offsets.  The two best
are refuted -- sub_8230D1F8 is the rank/score loader storing
CraftScore_Adjustment, FFPenalty_Zessel_Maximum and RankScore_S/A/B at exactly
256/320/324/328/332, and sub_82398CC0 uses r19 as a float-constant pool.

So the offset region is shared by two unrelated objects and a constant pool:
offset-based discrimination is contaminated by construction, which is why it
has now failed three times.  Catching the selection needs a runtime watch.

Side finding, unowned by the corpus: sub_8230D1F8 is the rank/score loader.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 18:07:42 +00:00
Claude (auto)
4ffbd39a01 re: AA_/AV_ are one interleaved block; the selection is still unfound
sub_822F9498 (the unit-definition loader, called only from sub_821A6CF0) is the
only function referencing the AA_/AV_ name strings.  Mapping each name to the
stfs that follows gives an exact interleave: AV_ at X, AA_ at X+8, for all five
axes -- PitchPlus 196/204, PitchMinus 212/220, Yaw 228/236, Roll 244/252,
AxisMode 320/328.  A selector is an offset of 0 or 8, not two lookups.

The 20 strings exist twice in the image; the first block is referenced by
nothing.  Data control: the ten suffixes match exactly between families.

The selection stays a reading: functions loading two or more of 196/204/320/328
number 39 image-wide, and only two are call-graph-reachable from the loader.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 17:54:53 +00:00
Claude (auto)
36f97c6322 re: the player-facing chatter partition; correcting the wingman claim
The 13 tables that voice every player-facing event are all
PresetMessage_Katana_*, the PLAYER's own sets -- unit-group-table.md already
names the link (DeltaSaber_T_Player carries msg=MessageSet_Katana).  Exact
partition: 13 voice all 14 events, 131 voice none, 0 voice some, and no
non-Katana table voices any of them.  CharacterKATANA is the only speaker
exclusive to the set.

Corrects preset-message-rules.md, which called those the wingman tables.  The
wingman roster is owned by isl-condition-builtins.md (UNITS: Bird1-Sandra ...
Rhino2-Katana, Rhino3-Ellen) and needed no experiment.

Residual: Katana_09_S10-1 and Katana_14_S16-2 voice none of the 14.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 17:42:28 +00:00
Claude (auto)
b710c9b9f0 re: Interval and IntervalFluctuation are the cooldown; the chatter system closes
sub_82210C38 retires a line and is the only chatter function reading the rule's
+20/+24 as words: node[+16] = Fluctuation * rand01 + Interval frames, state
0x20.  Interval is a floor, Fluctuation a uniform additive jitter.

The queue node IS the cooldown timer -- the tick counts +16 down and only then
frees the slot, so the already-queued bail in sub_82210670 and the cooldown are
one mechanism: a speaker cannot repeat an event until its node expires.  A
lingering node still holds one of the 128 pool slots.

Also: +20 on the node is a ducking level, raised while a higher-priority line
plays and released on retirement.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 17:28:57 +00:00
Claude (auto)
f78565fe9e re: the bark queue, Priority ordering, and Yes = a one-shot line
sub_822109B0 allocates 68-byte nodes from a pool capped at 128; the Pattern
byte at +23 doubles as the occupancy flag.  Priority orders the pending list
descending; the >=10 front-push arm is dead because the disc's range is 1..9.

The tick sub_8220FC50 does rule[+36] |= (1 << node[+25]) & rule[+32], so a
line enters the used mask only if its Yes bit is set: Yes = one-shot, No =
repeatable.  388 one-shot lines on the disc.  Closes what reads +32.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 17:17:01 +00:00
Claude (auto)
8dc270091b re: the bark firing sites and the Probability roll
36 of the 64 event names are executable strings, so the string-xref join names
the eight sites that raise them; all eight share one entry, sub_8220FA98.
Probability is a per-cent roll -- the scaling constant at 0x820856F8 is exactly
100.0.  sub_82210670 suppresses an already-queued (speaker, rule) pair, treats
a message as eligible only if its bit is clear in the runtime mask at +36, and
picks uniformly, then hands Pattern/Priority/EffectiveTime to sub_822109B0.

Measured negative: the pick path never reads +20, +24 or +32.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 17:04:18 +00:00
Claude (auto)
6ee041c1bf re: the chatter phase merge is not additive; 224 authored variants never play
sub_82215A58 reads CrewCount + PresetMessage_Phase1/2/3 and reaches the loader
from ONE call site, so the three phase tables fold into one map keyed by the
event record name.  Every merge collides; sub_82213840 reconciles on the
message list plus +16/+17/+18/+20/+24/+28 (NOT the +32 Yes mask), and the
incumbent always wins.  Measured: 26432 collisions, 26208 identical, 224
different (189 differ only in the message list), 0 mask-only differences.

Refuted handle: intersecting functions by the object's offsets finds dozens of
unrelated layouts -- offset shape is not an identifier.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 16:50:42 +00:00
Claude (auto)
c4fd90baa9 re: the seven chatter fields, read off sub_82213980
The one function referencing all seven field-name strings is the rule table's
loader.  Interval / IntervalFluctuation / EffectiveTime are SECONDS, emitted
as *60 frame counts; Probability is a percentage and zero skips the record;
Pattern is a 4-arm enum of which only Sound and Window ship; the Yes/No pair
element is a u32 bitmask, which is why MessageCount is clamped to 32 (max on
disc is 26).  40-byte object layout recorded.  13 dead records characterised.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 16:35:56 +00:00
Claude (auto)
6a81268cf1 re: all 144 chatter rule tables named; the undeclared 8 are NOT tutorial
Three more naming routes: strip _msg from a message table (137/144, superset
of route 1, zero non-rule hits), predict the name from the Sperkers roster
(6/6, control 0/4), and sweep the naming grammar (1/144).  Union 144/144,
and every rule table has its _msg companion.

Refutes the reading left by 2c2af7a: the undeclared tables are two story-stage
tables and six TCAF fleet/ship tables, no tutorial content at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 16:22:19 +00:00
Claude (auto)
2c2af7ad78 re: the reactive-chatter rule table behind ORDOR_SQUADRON_EXTENDED
864 records = 144 rule tables per language pack x 6.  One schema for all
9216 event records; MessageCount*2 == positional count with zero mismatches.
Named 136/144 by two independent routes that agree as sets.  2388/2405
message ids join the settled sound-cue table.

Corrects squadron-orders.md: the executable misspells all four SQUADRON
entries of the 0x820AEEB0 enum as ORDOR_, and the disc data matches.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 16:07:17 +00:00
Sylpheed RE agent
d1368c12db re: the objective/tutorial marker records; S24-S29 are CHALLENGE, not story
Correction first.  stage-numbering-and-player-craft.md called S24-S29
'story' and reported 22 story stages.  challenge-mission-gate.md OWNS
that split and already had it: S01-S16 story, S18-S23 tutorials,
S24-S29 challenge.  A third confirmation fell out of this iteration:
AUTO_SETTINGS names stage01..16_settings.tbl, tutorial01..06 and
challenge01..06, and its field tags are 1-16, 18-23, 24-29 -- the
shipped stage numbers exactly.  The campaign is 16 story missions, not
22.  AUTO_SETTINGS and IGNORE are likewise already owned by
isl-condition-builtins.md, so nothing there was new either.

What is new: the 11 marker records hud-config.md listed as unread.  12
named fields each, portable HUD data.  Colour is ARGB, three per marker
-- objectives red 0xFFFF0000, guard/waypoint green 0xFF20FF20,
Acropolis/tanker cyan 0xFF00FFFF, sub-objective yellow 0xFFFFFF00, and
every TutorialMarker_* amber 0xFFFFA010.  Each carries a Normal sprite
and an Emphasis part with optional Sub companions; HPGauge 0/1/2;
RadarCursorType Circle/Rectangle/blank; BlinkCycle 0x800 on all eleven.
TutorialTarget is Yes on exactly the five TutorialMarker_* and No on the
six ObjectiveMarker_*.

Parameters binds the roles: TargetMarker -> ObjectiveMarker_Target,
HelpMarker -> ObjectiveMarker_SubObjective, TutoTargetMarker ->
TutorialMarker_Target, plus ReloadDispTime 0.1.
Enumerate_ObjectiveMarkers is the 11-name roster.

Artefact +13 lines / 0 deletions; the other nine regenerate
byte-identical.
2026-08-27 15:46:31 +00:00
Sylpheed RE agent
15c9e13c12 re: CoverArea is a 6-bit MOUNT mask; the three other turret leftovers closed
Six bits, never more.  Over all 835 turrets: 41 distinct values, max
0x3f, bits 6/7 set on none; per-bit 709/117/274/274/260/237.  0x00 (34)
= the empty Weapon_NULL hardpoints with YawLimit 0; 0x01 alone (439) =
craft hardpoints with YawLimit 2.5/1.0/0; 2-6 bits (362) = warship
mounts with YawLimit 45-180.

It tracks the MOUNT, not the weapon.  UN_e107_ADAN_AAFrigate has eight
identical AAFrigate_AAGun turrets: GN_GunXS_01..04 are 0x1d and 05..08
are 0x2d -- same gun, same YawLimit 120, different mask.  The Battleship
spreads five masks over GN_TGunL_01..05 / GN_TGunM_02..03, all firing
the same CAF_Ship_ASGun.

Which sector each bit denotes is NOT determined; six bits and the name
invite +-X/+-Y/+-Z or six hull faces, but nothing static fixes the
convention.  Not adopted.  Refuted on the way: bits 2 and 3 are not a
mutually-exclusive pair -- 188 turrets set both.

The 26 weapons no turret mounts are a coherent set: 11 _P player
variants, the nose/twin mounts, two _Child sub-munitions, the three
S16Boss_*, three Weapon_Test_*, and ADAN_Attacker_S_GunTurret.

Versatile_NNN ships nowhere -- 0 populated records across all 41
archives, only the ??? template row.

The two units one turret short: both extras are missile mounts with no
Frame.  Elan_EX4 = NoseGun + Missile, both mask 0x01, both frameless;
AAFrigate_EX4 = eight framed guns plus one Ship_AAMissile at 0x0c with
no Frame.  n=2, stated as the observed pattern, not a rule.

New artefact and regenerator; the other nine regenerate byte-identical.
2026-08-27 15:35:09 +00:00
Sylpheed RE agent
27602a3032 re: BLOCKED -- the 2D / READY_ROOM TOC names, three routes closed with controls
1. A different hash family.  The corpus knows three (idxd-tag-hash.md):
name_hash, tag_hash, ixud_hash.  Scoring all 5977 harvested names x 6
prefixes: GP_TITLE 8/16 and GP_PAUSE_MENU 6/11 under name_hash (the
positive controls), and tag_hash and ixud_hash explain NOTHING anywhere
-- including the paks name_hash does explain.  So they are not the TOC
function, and the unnameable pair is not keyed by a different one.
GP_MAIN_GAME_E2D stays at 0/711 under all three.

2. The executable.  sylpheed.db's strings table holds 7140 rows, of
which exactly two look like asset paths -- Data\gmicon002_2.t32 and
Data\gmicon006_2.t32, in a Data\ directory nothing else on the disc
uses -- and neither resolves in any archive.  The binary is not the name
source; it holds two strays and no table.

3. Name transformations -- 13 of them on the 419 config paths, all 0.

The container runs out here.  Those TOC keys hash names that exist on
neither the disc nor the executable in readable form.  The only lever
left is a dictionary attack using name_hash's shape (top byte = the
character-sum checksum), and that needs a plausible name corpus this
disc does not contain.  Noted as blocked rather than improvised around.

The port does not need these names: sprites and bundles are readable by
content (T8aD, RATC), and the config records already say which asset
each HUD element uses.  Only the archive-key to name mapping is missing.

Artefact +22 lines / 0 deletions; the other eight regenerate
byte-identical.
2026-08-27 15:22:13 +00:00
Sylpheed RE agent
15b97af170 re: which archives the disc can name -- 100% for menu paks, 0.0% for the 2D paks and READY_ROOM
Chasing more prefixes for the 419 HUD config paths would have been the
same mistake twice, so this censuses the whole disc: harvest every
plausible asset-name string from every archive (6027), hash each under
the 16 known path prefixes, and ask per archive what fraction of its TOC
that explains.  idxd-container.md and idxd-tag-hash.md own the hash;
neither says which archives are reachable by it.

The result is bimodal.  GP_TITLE 16/16, GP_PAUSE_MENU 11/11,
GP_STAGE_CLEAR 44/44, GP_CHALLENGE 151/151, GP_MOVIE_THEATER 56/56,
MiscBin 40/40, GP_GAMEOVER, GP_BUNK, GP_SYSTEM, GP_TUTORIAL and fonts
are at 100%; tables.pak 78/79, GP_DIALOG 139/140, the six language paks
115/117; GP_MAIN_GAME_* 751/1119.  Then the cliff: the six
GP_MAIN_GAME_*2D.pak at 0 of 711 each, and GP_READY_ROOM at 6 of 1106 --
the largest UI pak on the disc, not previously noted anywhere.

Eleven paks at 100% in the same run is the control that makes 0.0% a
finding rather than a failed guess.

So the 419 HUD paths are not missing assets: nothing in the 2D paks is
reachable by name from the disc's own strings at all.  Those TOC keys
hash names that are not written anywhere readable.

Also refuted first: the 419 values under 13 name transformations, every
one scoring 0 against the E2D 711 and against all 16630 entries.

Not settled: what those names are.  The lever left is the hash's shape
-- the top byte is the character-sum checksum -- but that needs a name
corpus the disc does not contain.

New structure doc, artefact and regenerator; the other eight regenerate
byte-identical.
2026-08-27 15:08:24 +00:00
Sylpheed RE agent
bb37b16847 re: the in-game HUD config -- 16 records, 419 asset paths, a per-stage ResourceTable
Checked first: no docs/re file mentions ArmsStatus, RangeFinder, Radar,
Sight, Wing, NamePlate or ResourceTable.  Only HudResource had been
opened; the other fifteen records had not.

The six IDXD entries of GP_MAIN_GAME_E2D.pak: two carry the 16-record
HUD config, two the 13-record ObjectiveMarker_*/TutorialMarker_* set,
one Face (52 portrait sprites), one ResourceTable.  Between them they
name 419 distinct asset paths -- the whole flight HUD -- with new
subdirectory prefixes throughout (ArmsSt, ActvArm, RangeF, Marker,
Manuva, Map, Speed, Radar, Sight, Wing, Hitmark, Lockon, Info).

The decisive control uses the config's own exact path strings, so no
guessing is left in the loop: 419 distinct .prt/.t32/.tbl values, ZERO
resolve as a pak entry under 10 prefixes, and the four config filenames
resolve to nothing either.  The .t32 sprites certainly exist -- 574 T8aD
in that pak.  So the 2D pak is not addressed by name_hash of the name
its config uses.

This supersedes the earlier framing: the 28 'dangling' .prt names were
never a missing-asset story; they are 28 of a set where none of the 419
resolves.

ResourceTable is 58 positional fields = 29 pairs, alternating
HudResource.tbl / HudMarkerResource.tbl, identical in all six language
paks, with exactly one override at pair index 25 -- HudResource_S26.tbl.
Reading: indexed by stage number minus 1, so index 25 is S26, the one
stage with its own HUD config.  Arithmetic exact, indexing unproven, not
adopted.

New structure doc, artefact and regenerator; the other seven regenerate
byte-identical.
2026-08-27 14:53:49 +00:00
Sylpheed RE agent
dcf4945db5 re: the variant reading is refuted; HudResource is the in-game screen config
Censused the 32 bundle-less in-game .prt names with the 68 as control:
4 of 32 appear as an element of some 2D bundle, against 58 of 68.
Being an element is normal for a part that exists; these are not
elements either.  Last iteration's 'not adopted' reading is refuted.
28 names are neither a bundle nor an element.

They are not unreferenced: a HudResource record inside the 2D paks' own
IDXD entries names them -- the in-game screen config, parallel to
tables.pak for menus.  GP_MAIN_GAME_E2D.pak has 6 IDXD entries; two
carry HudResource (24 named fields) beside ArmsStatus, Map, ArmsItem,
RangeFinder, Marker, Manuva, Number; two more carry the
ObjectiveMarker_*/TutorialMarker_* set.

Its values carry subdirectory prefixes -- Hitmark\, Lockon\ -- which my
hand-written prefix list never had.  So I rebuilt the negative the right
way round: harvested every IDXD field value containing a backslash, kept
the 60 commonest directories, and re-swept.  241/376 with the hand list;
241 with hand + harvested -- zero new resolutions.  After the language\
miss, this is the closure that counts: the prefix list came from the
disc, not from me.

Still open: whether the 28 are cut features or assembled at runtime from
sprites.  Nothing static separates those two.

Artefact +45 lines / 0 deletions; the other six regenerate
byte-identical.
2026-08-27 14:42:20 +00:00
Sylpheed RE agent
1132d95227 re: an embedded .prt part is a top-level RATC bundle, addressed by its element prefix
Read first: ui-rat-layout.md OWNS the RATC stack and already documents
the 60-byte element declaration table and 'one bundle = one
(context x language) build of that screen'.  What it does not say is how
a part is addressed when it is NOT a pak entry.

The elements of one bundle share a common name prefix, and that prefix
is the part name.  GP_MAIN_GAME_E2D.pak has 130 bundles with 114
distinct element prefixes -- pgmenu_btn00, pghud_wing, pgface,
pghud_range, pgmanuva_eff0 -- exactly the in-game part names.

The 100 in-game-only .prt names match a 2D bundle prefix 68 times; the
control, the 241 shipped parts, matches ZERO.  The two families are
disjoint on the test.  13 shipped parts match a bundle prefix in their
own screen pak, which is the expected shape.

Four of the five mission banners land here: pgmsg_start.prt is E2D
bundle 0x89fac252, a one-element bundle declaring pgmsg_start_sub.rat;
likewise _end_, _failed_, _restart_.  pgmsg_update has no bundle at all,
consistent with having no _sub.rat.

32 in-game names still have no 2D bundle.  They cluster into families
whose base name is a bundle, reading as a variant declared inside a
parent bundle -- a reading, not adopted.

Artefact +38 lines / 0 deletions; the other six regenerate
byte-identical.
2026-08-27 14:27:38 +00:00
Sylpheed RE agent
e4000d9cb5 re: who references a .prt decides whether it ships -- menu named, in-game embedded
Censused the whole 135 unresolved set instead of chasing the five.
Cross-tab over all 376 .prt names: referenced by tables.pak AND
resolves 241; neither 100; referenced but absent 35; resolves WITHOUT
being referenced by tables.pak -- ZERO.

So resolving implies a tables.pak reference, 0 counterexamples of 376.
tables.pak holds the menu screen configs and every part it names ships
as <lang3>\<name>.prt.  The 100 names that appear only inside the
GP_MAIN_GAME_*2D.pak bundles never ship as entries -- the in-flight HUD
is authored into the bundles, not loaded by name.  The five pgmsg_*.prt
were never a special case; the split is menu = named + shipped,
in-game = embedded.

It is not a prefix rule: 6 of the 135 pg* names do resolve (pgloading,
pgloading2, pgmsg_scr, pgpause, pgpause_ttrl, pgpbase), because
tables.pak names them.

The 35-name residual is one coherent family -- pgmenu_btn*/item*/pad,
pgfacewin*, pgtextwin, phinfo1-3, 02d, 02d_scr, psview_release: the
in-game pause menu and squadron-order overlay, named by a menu config
but drawn from the in-game bundles.  Consistent with the split, not
separately proved.

Artefact +44 lines / 0 deletions; the other six regenerate
byte-identical.
2026-08-27 14:13:41 +00:00
Sylpheed RE agent
94065d40ec re: where .prt screen parts live; the five mission banners are a real exception
Read first, as the rule now requires: ui-prm-primitives.md,
ui-rat-layout.md, ui-screen-runtime.md -- they own the .prt/RATC/T8aD
stack but describe element names inside bundles, not where parts live.

A .prt IS a pak entry, under a LANGUAGE directory.  Of 376 distinct
.prt names referenced on the disc, 241 resolve as archive entries --
220 under eng\ and jpn\, 38 under each of the other four.  prbase.prt
resolves under all six; palogo1.prt under two.

All five pgmsg_*.prt resolve under nothing, across 15 prefixes that
include every language dir, so this is not the missing-prefix mistake
again.

A lead, refuted by its own control: four of the five have a same-stem
_sub.rat sub-bundle inside every per-language 2D pak, and pgmsg_update
has none -- but only 4 of 376 .prt stems have a _sub.rat, and
prbase.prt, which does ship as an entry, has none.  Not the container
convention, not adopted.

An intermediate pass nearly reported 'the five appear only in the
manifest'; that was wrong because the token census was truncated by a
slice and the RATC hits were pgmsg_start_sub.rat, a longer token.

New artefact and regenerator; the other six regenerate byte-identical.
2026-08-27 14:01:24 +00:00
Sylpheed RE agent
9d25a3a582 re: correction -- the dialogue table was already in the corpus, and one negative was false
The previous commit announced message\MissionDialogMessage.tbl as a
find.  structures/ixud-localised-text.md OWNS it and already says
S02_P1_OBJECTIVE and friends are record names in an IDXD map,
message\MissionDialogMessage.tbl, whose positional fields list the
lowercase per-line IXUD names -- and that *_GRAPH is the odd one, a
single named field holding a texture.  I grepped the manifest doc and
not the text doc.  Third overclaim in four days; the rule is now to grep
the doc that owns the DATA, not only the doc that owns the FILE.

Worse, the headline negative was wrong.  MissionDialog_local_string.tbl
DOES resolve -- as language\MissionDialog_local_string.tbl, IXUD, in all
six GP_MAIN_GAME_* paks.  My 33 prefixes omitted language\, which is
precisely the convention ixud-localised-text.md records for language
paks.  A prefix sweep is only as good as its prefix list, and the list
should come from the corpus.

'LOSE has exactly one entry' was wrong too.  Per kind: HINT_PAUSE 4,
HINT 3, LOSE 4, OBJECTIVE 4 positional fields, and GRAPH 1 NAMED field
holding the .t32.  The single-field bucket was GRAPH, not LOSE.

What survives as new: the 11 non-MISSION field values read as a block;
the GP_TEST and TEXTS sibling records; the per-stage phase census; and
the five pgmsg_*.prt resolving nowhere under 34 prefixes x 41 archives,
now with two working controls in the same sweep.

Artefact 8 insertions / 3 deletions, every deleted line replaced by its
corrected form; the other five regenerate byte-identical.
2026-08-27 13:45:35 +00:00
Sylpheed RE agent
acbcd8d75c re: Stage\script.tbl's 11 non-MISSION fields, and the mission dialogue table
structures/mission-script-ssb.md owns this manifest and names its 40
fields but never read the 11 that are not MISSION<n> = StageNN.ssb.
They are DIALOG_MESSAGE, DIALOG_LOCAL_STRING, FONT (+size), TEXT_POS,
TEXT_LINES and the five pgmsg_*.prt.  Two sibling records were also
unread: GP_TEST (PATH = dat\GP_TEST\, a debug archive not on the disc)
and TEXTS (a second text style).

Probed 7 values x 33 prefixes x 41 archives.  One resolves:
message\MissionDialogMessage.tbl, in all six GP_MAIN_GAME_* paks --
200 records, 25280 bytes, every name S<NN>_P<n>_<KIND> with five kinds
40 each (HINT_PAUSE, HINT, OBJECTIVE, GRAPH, LOSE), fields positional
and tagged 0..3, each value a message key.  An index from (stage,
phase, kind) to the localised strings, on the same S<NN>_P<n> keying
the ISL corpus already uses.

Control: the 40 stage-phases span stages 1-16 and 24-29 -- a subset of
the 28 shipped, and the six with no hints are exactly 18-23, the
tutorials.  A fifth independent route to the story/tutorial split, and
it gives the phase count per stage.

The other six do not resolve, with the control in the same sweep:
MissionDialog_local_string.tbl and all five pgmsg_*.prt are not a pak
entry under any of the 33 prefixes, while
message\MissionDialogMessage.tbl and Stage\script.tbl both resolve in
6 archives.

name_hash is CASE-INSENSITIVE (message\ == Message\); tag_hash is not.

New structure doc, artefact and regenerator; the other five artefacts
regenerate byte-identical.
2026-08-27 13:37:31 +00:00
Sylpheed RE agent
ccdc226022 re: the stage numbering is consistent everywhere; the manifest still declares the cut S17
Correction first: the previous commit claimed the corpus 'did not know
the stage NUMBERS'.  It did -- structures/mission-script-ssb.md has
recorded S01-S16, S18-S29, S17 absent by a three-way convergence (hash
lookup, the table sweep, the loader's != 16 guard) since before that
pass.  What was actually new there is the player craft per stage and
ResourceID's ordinal tags; the numbering is a fourth independent route,
not a discovery.

The question this item asked: do the ISL StageNN.ssb files and the
UnitGroup_SNN.tbl tables use the same numbering, or is something
silently mis-labelled?  They are identical.  Stage\StageNN.ssb,
stage\UnitGroup_SNN.tbl and stage\Route_SNN.tbl all resolve for the
same 28 N with an EMPTY symmetric difference; AIParams is the same minus
18-23.  Nothing in the corpus is mis-keyed.  Formation_SNN and
Message_SNN resolve 0/40 -- those are field names, not files, and the
four families above are the controls that make that negative real.

New: Stage\script.tbl's SCRIPTS record lists 29 MISSION*.ssb fields --
MISSION1..MISSION29, including MISSION17 = Stage17.ssb -- and only 28 of
29 name a file that ships.  Control: MISSIONn -> Stage<n>.ssb with the
same n, 0/29 mismatched.  So S17 is a cut mission the shipping manifest
still names, which is why the loader carries an explicit != 16 guard
rather than just running 0..27.

Artefact +14 lines / 0 deletions; the other four regenerate
byte-identical.
2026-08-27 13:27:19 +00:00
Sylpheed RE agent
28a59d839f re: the disc's stage numbers, and the player's craft per stage
The corpus knew '28 stages' and 'six tutorials with no AIParams'.  It
did not know the NUMBERS, and they are not 1..28.  Hashing
stage\UnitGroup_S%02d.tbl for N=0..39 against GP_MAIN_GAME_E.pak:
1-16 and 18-29 ship, 17 does not -- 28 files -- and the six with no
AIParams_SNN.tbl are exactly 18-23.  So S01-S16 story, S17 absent,
S18-S23 tutorials, S24-S29 story: 22 + 6.

eng\GP_HANGAR_ARSENAL_3D.tbl's ResourceID record keys a player-craft
mesh by stage: Unit_St1_6 -> rou_f001 (DeltaSaber T), Unit_St7_16 ->
rou_f002 (W), plus six further fields tagged with the raw numbers
24,25,26,27,28,29 -> rou_f002 x4, rou_f004 (DeltaSaber A) at 28,
rou_f002.  The bare tags are the last six story-stage numbers.  Control
in the same record: tag_hash(name) == tag for 11/11 named fields, so
those six genuinely carry no name.

Cross-check from a different file: grouping the Arsenal pak's 168
stage-scoped entries by which _Player craft their loadout mounts gives
f001 = 6, f002 = 15, f004 = 1, tutorials 5+1 = 6, and every number
closes against ResourceID -- 6 = Unit_St1_6; 15 = Unit_St7_16 (10) plus
tags 24,25,26,27,29 (5); 1 = tag 28; 6+15+1+6 = 28.

So the player flies the DeltaSaber A in exactly one mission, S28, and
the DeltaSaber T only for the first six.

Not settled: the Arsenal entry filenames, so which of the 168 is S24 vs
S25 is constrained but not pinned.

New structure doc, artefact and regenerator; the other five artefacts
regenerate byte-identical.
2026-08-27 13:18:42 +00:00
Sylpheed RE agent
2b9709eaa0 re: who selects a loadout -- the pak entry does; the Arsenal pak is stage-scoped
Settled by elimination with two controls.  The 15 loadout names appear
as a field VALUE nowhere on the disc -- 0 occurrences across every pak
-- while the control Arbalest_155KG, which is referenced, appears 150
times as a value in the same pak.  They occur only as the field names of
the UNITS record.  They are not in the executable either; control:
WEAPONS (a section key) is, Arbalest_155KG is not.

So nothing references a loadout by name.  The selection is which pak
entry the Hangar loads.  GP_HANGAR_ARSENAL.pak has 180 IDXD entries in
10 shapes; 168 carry a UNITS roster = 28 stages x 6 languages, and the
other 12 = 2 x 6.  Each entry is one stage's whole Hangar config, and
its UNITS roster is the flight for that stage -- 5 distinct rosters over
15 / 5 / 5 / 2 / 1 stages, including one where Rhino2-Katana flies
alone.  Every count is a multiple of 6 and they sum to 28, the disc's
stage count by a fourth independent route.

Corrects the previous commit: it read the pak with setdefault, so
'15 loadouts / 24 allow-lists / STANDARD_ARM1 has 11 entries' is the
union of first-seen records, not one table.  Contents vary per entry --
PlayerSET_ARM1 has 5 distinct contents, STANDARD_ARM1 3, ExSET_NOSE 4.
The chain and both controls (60/60, 70/88) are per-record and
unaffected; only the per-list sizes were over-generalised.

Not settled: the entry filenames.  name_hash probing over 8 templates x
40 indices x 6 languages resolved 0, while the same probe's controls
<lang>\weapon.tbl and <lang>\strings.tbl resolved 12/12.

Artefact +40 lines / 1 changed heading; the other four regenerate
byte-identical.
2026-08-27 13:09:07 +00:00
Sylpheed RE agent
2b2d984f0c re: the Hangar loadout system -- loadout, allow-list, arsenal item
15 loadout records, one per flight position x pilot (Bird1-Sandra ..
Rhino4-Yoji), each with Arm1/Arm2/Arm3/Nose + UnitID.

The same trap as PlayerWeapon, one level up: Arm1/Arm2/Arm3/Nose do NOT
name items.  They name a per-slot ALLOW-LIST record -- one of 24 whose
only named field is Type (the slot kind) -- and the candidate items are
that record's positional, unnamed fields, in order.  Four hops:

  Rhino4-Yoji.Arm1 -> STANDARD_ARM1 -> [Falcon_9AM, Condor_105AM, ...]
  -> item.PlayerWeapon = Turret_NNN -> slot.WeaponID -> Weapon.ID

Controls: Arm1/2/3/Nose -> allow-list record 60/60; allow-list
positional entries -> arsenal item 70/88, and every one of the 18
misses is the single sentinel No_Equipment -- one of the four
WEAPONS-roster values with no item record, i.e. the empty-slot marker.

UnitID is two ID spaces at once: 5 rows name a unit Generic.ID (the
three -Katana rows are the player -- a _Player craft plus an extra,
empty PlayerUnit field), 8 name a character, resolving as Character +
the value into the 64-record character table.

Two values resolve to nothing, both single rows against 13 that do:
Rhino2-Ellen.UnitID = UN_f001_TCAF_DeltaSaber_W exists nowhere (checked
as a Generic.ID across every pak and as a record name, 0 hits) while
UN_f002_TCAF_DeltaSaber_W does -- consistent with a shipped typo,
reported not diagnosed -- and Rhino4-Brandon.UnitID = BRANDON has no
CharacterBRANDON among the 64.

New structure doc, artefact and regenerator; the other four artefacts
regenerate byte-identical.
2026-08-27 12:58:53 +00:00
Sylpheed RE agent
21e710a531 re: the 59-of-131 arsenal question is closed -- an item names a hardpoint
An Arsenal item does not reference a Weapon record.  It references a
Turret_NNN HARDPOINT SLOT on the player craft's own unit table, and the
slot is what carries the WeaponID.  Three hops:

  Arbalest_155KG.PlayerWeapon -> Turret_050  (a slot on
  UN_f001_TCAF_DeltaSaber_T_Player) -> .WeaponID ->
  Weapon_DSaber_P_wep_50_Cannon

Controls, both in the same loop: 0/59 distinct PlayerWeapon values are a
Weapon.ID; 59/59 are a Turret_NNN slot id; the full chain lands on a
Weapon.ID 59/59.  WingmanWeapon resolves identically.  The WEAPONS
roster's 59 = 55 item names + 4 empty-slot sentinels.

Wingmen fly a cheaper gun: following the same 59 slots across craft
variants, the _Player tables give each item its own weapon record (59
distinct) while the AI tables collapse all 59 onto 10 generic classes.
That is most of the 131.

Upgrades yesterday's 'hardpoint catalogue' reading from 21 to adopted,
proved from an independent file, and corrects its '10 distinct WeaponID'
figure -- that was the AI variant, not the player's.

Also adds an __main__ guard to unit_substructures.py so importing
pak_entries from it does not run its report; its artefact is unchanged
and still byte-identical.
2026-08-27 12:49:22 +00:00
Sylpheed RE agent
4a5e5d55e6 re: the destructible-subsystem model -- a unit's sub-records
The corpus has named these since unit-struct-runtime.md but never
opened them.  Per unit table: Turret_NNN 835 records (max 63 on one
unit), ShieldGenerator_NNN 46, Thruster_NNN 38, Hatch_NNN 26,
Bridge_NNN 25, plus one each of Shield/Mass/SE/Explosion/
StructureCount and NS_Body on 68 of 114.

Turret/ShieldGenerator/Thruster/Hatch/Bridge are ONE record shape: a
shared 19-field destructible-part base (ID, Name, ParentStructureID,
Frame = a mesh NODE name, NomalModel, CollisionModel, Radius, HP, the
four Is* flags, SpreadDamage, damaged/destroy motion + time, and the
three Effect_*), with per-kind extras.  Turrets add WeaponID,
AngularVelocity, YawLimit, PitchLimit_Elevation/_Depression, CoverArea,
IsAuto, HasBarrel and up to 80 CannonModel_NNN/CannonFrame_NNN.  Shield
generators, thrusters and bridges add PowerRatio.  Hatches add
SquadronID, LoadedCount, MaxAvailableCount, TakeoffInterval -- a
carrier's launch bay.

Control 1: StructureCount.<Kind>Count == #<Kind>_NNN records, over 684
comparisons -- 612 equal, 55 "0 declared, one blank placeholder"
(55/55 blank in Name AND NomalModel AND Frame), 11 differ, 6 kind
absent.  All 11 exceptions are Turret and all are declared < records.

Control 2: 835/835 Turret_NNN.WeaponID resolve to an ID in the
131-record Weapon datasheet, zero unresolved; 26 weapons are never
mounted on a turret.

Refuted in the same pass: "the DeltaSaber's 59 non-NULL hardpoints are
the 59-name WEAPONS arsenal roster".  The counts match exactly and the
sets overlap in 0 values -- two namespaces, one coincidence.

New structure doc, artefact and regenerator; other artefacts unchanged.
2026-08-27 12:39:50 +00:00
Sylpheed RE agent
39b3a4de7d re: correction -- Generic is a per-FILE header record, not a table
Yesterday's page said "Generic (394 per pak) is the unit datasheet".
Only 114 of the 394 are.  Every IDXD file carries exactly one Generic
record and its schema is set by what kind of file it is.  Partitioned
by field set, identically in all six GP_MAIN_GAME_*.pak:

  114  has HP                        -- a unit datasheet
  204  {Count} only                  -- a dialogue file
   64  {ID, Name, SideID, Unique}    -- a character (36 TCAF + 28 ADAN)
   10  {EnumAsteroidGroup}           -- an asteroid group
    2  degenerate

204+114+64+10+2 = 394, and 178 distinct Generic.ID = 114 unit + 64
character, the only two shapes carrying an ID.  That settles the
"178 IDs vs 394 records" question the previous entry left open.

Positive control in the same loop: for the 204 dialogue headers Count
equals the number of Message_NNN siblings, 204/204, zero mismatches.

Cross-check from the other side: Maneuver = 114, Effect = 114, the
carrying-entry sets are identical, every unit Generic has a Maneuver
sibling, and Generic.Type splits 43 Craft + 71 Vessel -- the same
43/71/114 unit-struct-runtime.md reached from live guest memory.

New tool generic_partition.py + artefact; ISL artefacts byte-identical.
2026-08-27 12:29:30 +00:00
Sylpheed RE agent
69e6f6bcd8 re: the static unit datasheet and the AI flight model
Same technique as the weapon tables -- field names are literal, so a
field-shape search over 190782 records finds the carriers.

Generic (394 per pak, 110 field names, 178 distinct IDs) is the unit
datasheet: HP, ShieldRatio, DefencePoint, AttackCraft/VesselPoint, the
five Resistance* terms, Size_X/Y/Z + radius, RadarRange 10000,
FCSRange 8500, MountedFCS, MountedShieldGenerator, Score/Damage/Mass,
NozzleCount with per-nozzle FX, and Model rou_e006 -- which ties the
sheet to the mesh names the corpus already decodes.

Maneuver (114) is the AI flight model: MaximumVelocity 1200,
CruisingVelocity 700, Acceleration 600, Deceleration 400,
SideThrustAcceleration 1000, Turn_AngularVelocity 180,
MaximumBank_Normal 60, per-axis DragFactor 3.0, afterburner and
reverse-thrust factors, plus nine named manoeuvres each with its own
timing/ratio/length bounds and AA_/AV_ rate pairs.  This is the static
source for what flight-speed-law.md measured at runtime.

Corrects a guess in the same pass: HP_CLASS/HP_ID are not ship stats --
they sit on records named for wingmen and ship classes alongside
HPGauge/RadarCursorType, i.e. a HUD gauge binding table.

New structure doc + artefact; ISL artefacts byte-identical.
2026-08-27 12:20:03 +00:00
Sylpheed RE agent
f2cacce59d re: the static weapon datasheet -- Weapon, Shell, AssortMissileParam
The corpus's weapon numbers came from a runtime capture; the static
tables are now read with no emulator.  The move: IDXD field names are
literal strings in the pool, so the stat-shaped keys already harvested
from the disassembly (MegaTons, GuidanceType, SpiralType, ...) can be
searched directly.  8648 records carry at least one, clustering on
three record names in every per-language GP_MAIN_GAME_*.pak:

  Weapon              131 records, 21 fields -- the launcher
  Shell               131 records, 37 fields -- the projectile
  AssortMissileParam    9 records, 20 fields -- missile guidance

131, not the 59 of the WEAPONS roster: that roster is the arsenal
menu's list, not the full set.

Controlled negative in the same pass: stageNN_settings.tbl does not
exist.  AUTO_SETTINGS names 28; four path prefixes give 0/116 while
both controls in the same loop are found.

Also decoded: EnumUnit (54 unit ids tying eNNN/fNNN mesh prefixes to
faction/class names), ArmsItemFile (59 weapon -> HUD icon mappings),
SETTINGS (sound config), Parameters (HUD markers).

New structure doc + artefact; ISL artefacts byte-identical.
2026-08-27 12:11:03 +00:00
Sylpheed RE agent
2a1236074b re: decode the config records -- render/input constants, and a correction
parse() + named() on each named record (data/config-records.txt).

The find, directly portable: Rendering carries the tone-mapping and
bloom numbers (Brightness 1.4, Contrast 1.0, ExposureKeyValue 0.18,
BrightPassThreshold 0.6, LuminanceMin/Max 0.15/3.6, BloomScale 0.5,
AfterimageScale 0.05, StarScale 1.20, GlareType 8), ControlTweak the
input constants (CameraSpeed 0.50, TargetSpeed 0.50, TargetMovingRange
-10..10, mov_trigger_play 20, eye/mov_stick_play 6000), and Camera
NoseCameraFOV 0.92 / ChangeTime 0.20.

Corrects my own last entry: WEAPONS is a 59-entry NAME ROSTER, not a
stat sheet, and UNITS is not units at all -- seven craft-slot/pilot
pairs, the wingman roster.  IGNORE is a 13-name blocklist over the
weapon enumeration.

AUTO_SETTINGS lists exactly 28 stageNN_settings.tbl files -- a third
independent confirmation of the 28-stage count.

The objective-shaped records are UI: SUB_OBJECTIVE is a HUD layout (two
variants ship), SQUADRON_ORDER_OBJECT is 29 .prt sprite names, MISSIONS
holds the challenge categories, STAGES has zero fields.

Docs + one new artefact; ISL artefacts byte-identical.
2026-08-27 12:02:13 +00:00
Sylpheed RE agent
bc48fa576d re: the config sections are real disc data -- 64/65, with a clean 0/54 control
Testing every literal key from data/config-keys.txt against the 3496
distinct IDXD record keys with tag_hash:

  section names (arg to sub_82448AA0) -> 64 / 65 present
  field  keys  (arg to sub_824482D0) ->  0 / 54 present

The two classes are each other's control and the dissociation is
total, confirming the model: sub_82448AA0 finds a RECORD keyed by
tag_hash(name); sub_824482D0 reads a FIELD whose name is a literal
string in the pool, so none of them can be a record key.

The single absent section is LANGUAGE -- the one real section in the
disc-root config.ini.  The same API serves both stores.

This deflates my own earlier framing: I had listed "64 sections whose
hardcoded fallbacks are worth pinning".  They are not fallbacks; all
but one are real data.  STAGENN_UNIT_MAX is the genuine exception,
because its name is built at runtime from the stage index.

The artefact now names the pak each record lives in -- notably
GP_HANGAR_ARSENAL.pak holds WEAPONS, UNITS, Camera, ControlTweak,
Rendering, AUTO_SETTINGS and IGNORE.

Docs + regenerated artefact; ISL artefacts byte-identical.
2026-08-27 11:54:36 +00:00
Sylpheed RE agent
49b5892df5 re: Stage30-33 do not exist; AIParams is missing for exactly the tutorials
Probing pak entry names with name_hash across all 41 paks:
UnitGroup_SNN.tbl exists for exactly S01-S16 and S18-S29 = 28 stages.
S17 and S30-S33 are absent, so the 33-entry STAGENN_UNIT_MAX table is
headroom over a 28-stage game.  Control is built in: 28 hits from 33
probes shows the loop resolves entry names.

Corroboration that fell out of it: AIParams_SNN.tbl exists for S01-S16
and S24-S29 but is missing for exactly S18-S23 -- the six stages the
corpus identified as tutorials by a completely different route (their
lone guard is request_next() != 1).  Two unrelated methods, same six.

Stage_SNN.tbl probes find 0, which is expected: stagetbl.py documents
that the per-stage record is not name-addressed.

Drops the [phase+356]/[phase+360] consumer question: 243 non-stack lwz
reads from those offsets image-wide, and the only ISL-region hits are
the two writers plus two vtable calls.  Not a cheap static question.

Docs only; artefacts byte-identical.
2026-08-27 11:40:29 +00:00
Sylpheed RE agent
731c3cd849 re: the section reader is a generic IDXD accessor; UNIT_MAX absence re-tested
Resolving the r4 string at every sub_82448AA0 / sub_824482D0 call site
gives 65 section names and 54 int keys (data/config-keys.txt) -- far
more than the 400-byte config.ini holds -- and BASE_INFO, SYSTEM,
MISSIONS and FONTS are all present as IDXD record keys on the disc.  So
it is a generic named-section accessor over the .tbl containers, not
"the INI reader" as I labelled it yesterday.

My first re-test was invalid: IDXD record keys are tag_hash, not
name_hash.  Scanning with name_hash returned 0 for STAGENN_UNIT_MAX and
0 for every control name too -- the control is what exposed it.

Redone with tag_hash: 7750/7750 entries parsed, 190782 records, 3496
distinct keys, controls BASE_INFO/SYSTEM/MISSIONS/FONTS found, and
STAGE01..33_UNIT_MAX -> 0 records.  With the literal search and
config.ini printed in full, the section exists nowhere on the disc, so
PLANE=200 / VESSEL=20 for every stage now rests on a controlled test.

Docs + one new artefact; ISL artefacts byte-identical.
2026-08-27 11:33:30 +00:00
Sylpheed RE agent
e46055ffae re: the config is an INI that ships nearly empty -- every stage takes 200/20
Corrects my own label: sub_824480D0/sub_82448AA0/sub_824482D0 is not an
XML reader.  The 40 <?xml pak entries are XPR2 resource manifests
(XBGMesh/Texture declarations).  The real config is a loose config.ini
at the disc root, header comment "application / game-part initial-
settings table", and every key this code touches is an INI section or
key name.

The shipped file is 400 bytes: [SYSTEM] (empty) and [LANGUAGE].  No
[STAGENN_UNIT_MAX] anywhere, and UNIT_MAX/VESSEL appear in 0 pak
entries -- so the lookup always misses and every stage runs on the
defaults PLANE=200, VESSEL=20.

The absence is controlled: the same scan decompressed 26443 entries
across 41 paks and found MSG_DEMO 192x, mapmesh_box_500km 162x and
<?xml exactly 40x, the census number.  Last iteration's null was
worthless -- it used s[8:] instead of s[10:] for the Z1 header and
searched the index rather than the .pNN data.  Using the corpus's own
reader was the whole difference.

Docs only; artefacts byte-identical.
2026-08-27 11:24:01 +00:00
Sylpheed RE agent
ad5270cfdd re: STAGENN_UNIT_MAX holds two per-stage caps, PLANE and VESSEL
Reading sub_82261F70 past the table lookup finishes the mechanism the
stage index feeds: the section name is built from the 0-based stage
index, looked up, and two ints land in the phase --

  [phase+356] = PLANE   (default 200)
  [phase+360] = VESSEL  (default 20)

with the defaults taken when a stage has no entry.  Key strings are
0x820A8B14 "PLANE" and 0x820A8B1C "VESSEL"; section and both keys are
the game's own names.  Surrounding reads (GP_SCRIPT, SCRIPTS,
Resource2D, TABLE) plus the pak census's 40 <?xm entries make this an
XML config reader.

Register discipline mattered twice: in sub_82261F70 r21 is the phase
and r31 is the STACK frame, and 0x82273400 lwz r11, 360(r11) in the
built-in switch is a vtable slot 90 call, not a phase read.

Not settled: the values (a crude zlib scan found no UNIT_MAX, but it
did not use the corpus's pak reader, so that is not evidence of
absence), and no consumer of the two fields was identified.

Docs only; artefacts byte-identical.
2026-08-27 11:14:17 +00:00
Sylpheed RE agent
17d1d2c999 re: X+12 is a 0-based STAGE INDEX -- proved by the game's own string table
Three iterations circled this.  Following the value rather than the
filename settles it in two hops:

  1. SilphScriptPhase's ctor sub_8225FEF8 never touches r7 (zero
     mentions).  The BASE ctor sub_822700C0 keeps it:
       or  r26, r7, r7   ->   stw r26, 152(r30)
  2. sub_82261F70 indexes a stack table of string pointers with that
     field.  The table is contiguous at 0x820A8880, 20 bytes/entry:
       index 0 = STAGE01_UNIT_MAX ... index 32 = STAGE33_UNIT_MAX,
       then PLANE at 33.

So X+12 = [phase+152] = a 0-based stage index, N -> STAGE(N+1), and the
gate is explained rather than described: <= 32 is the array bound (33
entries) and != 16 is STAGE17, the one stage number with no .ssb.

This retracts my own "not adopted": the stage-index reading was 1-of-1
but coincidence-shaped two iterations ago; 0-based is now PROVED by
index 5 -> STAGE06.

Caught a false positive: 0x82272D88 lwz r11, 152(r11) in the built-in
switch is a virtual call to slot 38 -- r11 is the vptr, not the phase.

Also names a third class: sub_822700C0 stamps 0x820A8E44 =
SilphScriptPhaseBase, so the hierarchy is Base <- ScriptPhase and
Base <- Demo.

Docs only; artefacts byte-identical.
2026-08-27 11:05:50 +00:00
Sylpheed RE agent
4feab6c579 re: a script load builds TWO phase objects, main at Y+4 and demo at Y+8
sub_8225FEF8's only caller is sub_82260568, which looks like a derived
ctor calling its base -- i.e. every phase is a Demo, which would make
the built-in dispatch inert.  Reading the constructor refutes it: it
allocates 10216 bytes and runs SilphScriptPhase's ctor on object A,
then initialises a second object B and stamps the Demo vptr on it.
They land at Y+4 and Y+8, which is why the load path resolves the
ScriptPhase through Y+4.

The Demo class's 109 overrides all point at three no-op stubs; only
slot 15, slot 111 (Update = sub_82275800), slot 112 and the destructor
survive.  DEMO is the corpus's established cutscene text family, so the
cutscene-runner reading is recorded as a reading, not a result.

Also notes a new handle for the stalled X+12 question: the kind is
passed straight into sub_82260568 as its third argument.

Docs only; artefacts byte-identical.
2026-08-27 10:14:00 +00:00
Sylpheed RE agent
044b98dcc4 re: the ScriptPhase vtable is 113 slots, and there is a SilphScriptPhaseDemo
RTTI at vtable[-1] names the class .?AVSilphScriptPhase@silph@@ and the
next class's COL at 0x820A8680 terminates the table at 113 entries.
The earlier ">=200 slots" reading used a bad terminator -- scanning for
a non-code word runs straight into the next vtable.  Cross-check: the
147 built-in stubs use 109 distinct slots, min 0 max 110.

The next vtable is .?AVSilphScriptPhaseDemo@silph@@, also 113 slots,
overriding 109 of them with mostly one shared stub -- a cut-down phase.
That explains why sub_822710D0 has two callers: sub_82263408 and
sub_82275800 are the two classes' Update, both slot 111.

Both "unread spawner callers" are placed: sub_82264058 is slot 0 of
SilphScriptPhase and sub_82273910 is slot 0 of the Demo class -- the two
destructors.

Also records a negative: the Stage%02d construction site I proposed last
iteration does not exist (no stage format string among the 43 short %d
strings; no precomputed name hash anywhere), so the stage-index lead
cannot be settled that way.

Docs only; artefacts byte-identical.
2026-08-27 10:05:01 +00:00
Sylpheed RE agent
6213825a2f re: the GamePart_MainGame vtable names the code region; a stage-index lead
sub_821A6CF0 is reached only by a tail b from a run of adjustor thunks
whose vtable is at 0x820A319C.  Slot 1 is 'addi r3, r0, 17; blr' -- the
factory id -- so RegisterToFactory<17, class silph::GamePart_MainGame>
names the class, and every thunk's lwz r3, 8(r3) makes slots 4-10
GamePart_MainGame::Impl methods (9 = the per-frame Update
sub_821AA1B0).  Several iterations' worth of inference is now a read.

Lead, not adopted: the disc has 28 scripts numbered Stage01-16 and
Stage18-29 -- exactly one gap, Stage17 -- and the gate excludes exactly
one value, 16, which lines up 0-based.  Control comparison: the refuted
GamePart-id reading had 4 missing values (1-of-4, worthless), this is
1-of-1.  Tighter but still coincidence-shaped.  The Stage%02d
construction site would settle it; not found this pass.

Docs only; artefacts byte-identical.
2026-08-27 09:56:34 +00:00
Sylpheed RE agent
2c5c8b11d9 re: the (16,32] gate is != 16 && <= 32, a validity check on a load parameter
Corrects this corpus's own notation: there is no lower bound, values
below 16 pass.

sub_8225EC78(X, kind, ...) -- the function carrying "script load
cancel" -- stores its second argument into X+12 and applies the same
two tests to it immediately, bailing out of the load.  So the gate is
the object's invariant, re-checked at every later touch, not a
selector.  42 sites image-wide have the shape, 34 of them in one code
region plus 3 in sub_8225EC78.

Object chain: X = [GamePart+104] is the current script instance
(installed at 0x821A78EC, previous one torn down via sub_8225EB60);
X+4 -> Y; Y+4 -> the ScriptPhase; Y+72 and [ScriptPhase+10152] get
the answer.

Refutes my own lead in the same pass: "X+12 is a GamePart id and 16 is
the unregistered one" fails its control -- 1, 2 and 18 are also absent
from the RegisterToFactory list, so 16 is one of four gaps.

Docs only; artefacts byte-identical.
2026-08-27 09:48:36 +00:00
Sylpheed RE agent
237dbce155 re: [phase+10152]/[phase+10156] are an answer and a request state
Built-ins 102/103/130 are one trio over two phase fields, read off
four consecutive vtable stubs (slots 70/71/72/97).  102's body:

  state==1 -> return 2                     (pending)
  state==2 -> special[0] = (answer==1); state=0; return 0
  else     -> state=1; answer=0; return 2  (issue)

2 and 0 are the dispatcher's thread codes, so 102 blocks the coroutine
until an answer arrives; 103 is the non-blocking form and 130 clears
the answer.  The answer is published by sub_821A9DC8 under the same
(16,32] gate that request_next's writer uses.

Flags an inherited name I could not verify: prompt_yes_no (102) has no
derivation recorded anywhere in the corpus, so 103 and 130 are left
unnamed rather than named off it.

Docs only; artefacts byte-identical.
2026-08-27 09:39:52 +00:00
Sylpheed RE agent
af61c162f3 isl: built-in 104 is request_next -- named from the game's own log string
Followed the writer, not the reader.  [phase+10160]'s only writer in
the image is one site in the mission frame loop sub_821AA1B0: it does
obj->get() on an object fetched from a registry by id 0x20FFFF02,
stores it to the phase, then clears the object -- read, publish, clear,
every frame.

The id namespace has exactly three members (0x20FFFF00/01/02), each
built at exactly 4 sites, and two of those are in sub_821D5178, which
gets 0x20FFFF01 and 0x20FFFF02 and logs both:

  GamePart_ReadyRoom::Impl::OnCommand - Wait() command is requested.
  Check flow control valiables. WAIT_MODE : %d, REQUEST_NEXT : %d

Argument order gives 0x20FFFF01 = WAIT_MODE, 0x20FFFF02 = REQUEST_NEXT.
PrepareScript corroborates: it sets WAIT_MODE=1, REQUEST_NEXT=0 before
an ISL script runs.  So the six tutorial stages' lone dominating
condition request_next() != 1 is the script waiting on the game part's
proceed flag.

6 artefact lines changed, all 6 pair exactly.
2026-08-27 09:31:01 +00:00
Sylpheed RE agent
b86c8c82ba isl: rename built-ins 8/9/93 to stopwatch_start/_elapsed/_stop
They are start / read / stop of one of 32 per-phase stopwatches, not
flag operations.  123-127 keep timer_* -- that is the mission timer,
five scalars at [phase+304..320], a different clock.

Artefact check: 84 lines changed across 5 files and all 84 pair
exactly with their old-name partners once column padding is
normalised (0 removed lines without an old name, 0 added lines
without a new one).  data/isl-timers.txt reproduces the same
675/675, 11.2 % control, 82/1 and identical histograms, which is
what shows the rename is cosmetic.

Also withdraws a label from the previous commit: sub_8230C398 is NOT
the message pump.  It runs every frame but drains nothing -- a state
machine on [0x828E1F8C] that only allocates, builds strings, looks up
and PUSHES.  And bus+8216 is weak evidence: sub_82254A08 is a generic
map find with ~120 sites, and the key looked up is a pointer, not a
tag.  The open handle is now the ring buffer at bus+4, not bus+8216.
2026-08-27 09:21:04 +00:00
Sylpheed RE agent
bf17f19f67 re: the timer message's consumer is a runtime registry, not a tag compare
The move I flagged last iteration -- "0xAB03E5BA is built once, so
nothing consumes it" -- is refuted by its own control: 25 of the 40
distinct 0xAB03xxxx tags in the image are built exactly once (62.5 %),
and no 4-aligned word anywhere in the image begins 0xAB03, so tags are
never stored statically.

What does stand:
  * sub_82175C20(bus+4, &msg) is a ring-buffer PUSH, not a dispatch.
  * The message class has a one-method vtable {scalar deleting dtor}
    with an MSVC RTTI locator at vtable[-1] -- no handler of its own.
  * The bus at [0x828F35DC] has a map at +8216, looked up via
    sub_82254A08 from sub_823001E8, whose only caller is the per-frame
    pump sub_8230C398.

So the item is NOT settled: the insert path into bus+8216 is unread,
and whether running out of time ends the mission is still unknown.

Docs only; ISL artefacts regenerate byte-identical.
2026-08-27 09:12:36 +00:00
Sylpheed RE agent
d79f59d24e re: timer_set's second argument is a SECOND countdown, not a warning threshold
mission-phase-timers.md left 180 open as "a limit and a warning
threshold is the obvious reading, but not established".  Reading
sub_822639B8 -- ScriptPhase::Update 0x82263528, same dt as the
stopwatch bank -- settles it the other way:

    if running: [+304] += dt
    if armed:   [+308] -= dt   while [+308] > 0
                else [+312] -= dt, clamped at 0

[+312] is never compared with [+308]; it is decremented, and only in
the A<=0 arm.  Two sequential countdowns.  Disc-wide the second
argument is 180 in all 29 timer_set sites while the first varies
(600 x19, 1200 x8, 900, 1800).

Built-ins 123-127 are vtable slots 90-94 on five scalars at
[phase+304..320].  125 and 126 have ZERO call sites in all 28 scripts:
the script arms, starts and stops this clock but never reads it.

Corrects mission-phase-timers.md, which merged this clock with
stopwatch 0 -- timer_resume starts [+304], set_flag(0) one instruction
later starts the stopwatch the timeline's kind=0 reads.

Docs only; all seven ISL artefacts regenerate byte-identical.
2026-08-27 09:03:18 +00:00
Sylpheed RE agent
3cad5d063f re: read_freg counts SECONDS -- the 32 "flags" are a stopwatch bank
The open question was the unit, not the array.  Following the writers of
[phase+88] settles it:

  * sub_822710D0(phase, dt), called from ScriptPhase::Update, does
    prev[i] = cur[i] then, only while [phase+120][i] == 1, cur[i] += dt --
    so +88/+104/+120 are current / previous / running, 32 entries each.
  * dt is seconds by a non-circular round trip: frames * (1/60) * 10000
    -> clamp 3200 -> * 1e-4, in the timing singleton at [0x828F35B4].
    The clamp is 0.32 s, a frame ceiling.
  * 675/675 timeline kinds are indices their own phase starts (control
    11.2 %), which is why kind is only ever 0 or 5.

Corrects isl-builtins.md twice: set_flag writes 0.0 not 1.0, and
clear_flag clears the running flag rather than the value.  Confirms its
grouping of 8/9/93 as one family.  Docs only -- all seven ISL artefacts
regenerate byte-identical.
2026-08-27 08:48:49 +00:00
Sylpheed RE agent
20b8412a38 re: read_freg reads the timeline's clock array -- "six clocks" was my mislabel
Built-in 9 is inline in its stub, with no vtable slot.  It bounds-checks
0 <= local[0] < 32 and then:

    822725E8  lwz    r10, 88(r31)          ; [phase+88]
    822725EC  rlwinm r11, r11, 2, 0, 29    ; i * 4
    822725F0  lfsx   f0, r11, r10          ; a FLOAT

isl-builtins.md already recorded read_freg(i) -> [phase+88][i].  What is new is that
this is exactly the array, and exactly the indexing, that the TIMELINE walker uses for
its `kind` field.  So kind is not a selector over six private clocks -- it is an index
into the same 32-entry float register file the script reads with read_freg(i).  A
clear condition like `read_freg(0) < 1200` and a timeline entry with kind = 0 read the
same register.  My "six clocks" framing came from the data only ever using indices 0
and 5, and it is corrected in isl-schedule.md.

Recorded as a nuance rather than a correction, because it is half-read: clear_flag
(93) clears a DIFFERENT 32-entry array, [phase+120], looping to 128 in steps of 4 --
not [phase+88].  set_flag (8) does bounds-check against [phase+88], but only its first
13 instructions were read, so the corpus's grouping of 8/9/93 as one "32-entry file"
family is neither confirmed nor overturned here.

This does NOT settle what the item actually asked: read_freg returns a float out of a
register file, and that the unit is seconds remains INFERRED from the 210/300/600/1200
gate values, exactly as it was before.  Also still open: what writes [phase+88], which
has 73 writers in the ISL region alone.

All artefacts regenerate byte-identical; documentation only.
2026-08-27 08:28:26 +00:00
Sylpheed RE agent
b63e6cc8af re: A/B in the trigger test -- searched, not settled, and the matrix reading excluded
Ran the "who writes this" move on the two 3-double vectors the trigger condition pulls
out of the unit record.  It did not settle them, and it is worth recording why.

99 sites across 40 functions write +32/+40/+48; 16 sites across 13 write +64/+72/+80;
six functions write both.  The most inviting is sub_821AD218, which is a float-to-double
widening copy of exactly 16 floats -- lfs from r4+0..60, stfd to r3+0..120.  Sixteen
floats is a 4x4 matrix, under which +32/+40/+48 is row 1 and +64/+72/+80 is row 2,
making A and B basis vectors rather than positions.  That is a tidy story and it is
wrong for this record.

The disproof was already in hand: sub_8226DAF8 reads `lwz r10, 100(r11)`, a WORD at
+100, which falls inside where a 16-double matrix's row 3 (+96..+120) would sit.  A
second writer, sub_8226A348 -- same built-in shape, [phase+324] indexed by local[4]
with the rec+4 liveness check -- confirms a mixed layout: words at +0/+4/+8/+12/+16 and
at +80, doubles only at +32/+40/+48.

So these offsets recur at the same positions in several unrelated object layouts, and
nothing found here ties a specific writer to the object the trigger actually reads.

A and B stay unnamed.  That is the second tidy reading rejected for them, after
previous/current-position.

All artefacts regenerate byte-identical; documentation only.
2026-08-27 08:21:58 +00:00
Sylpheed RE agent
c9d479f083 re: the kind-1 trigger condition is the same test with a pre-filter -- mechanism read end to end
sub_8226DC80 turns out to be sub_8226DAF8's twin.  Diffing them instruction by
instruction, 33 of the first 86 differ, and every difference before index 65 is a
register rename or a branch target.  Both:

  * resolve the route name (sub_8217FA08 -> sub_823012D8), leaving the route point in
    [r31+112/120/128];
  * index [phase+324] by payload+0 and require rec+100 != 0;
  * copy the two 3-double vectors out of the unit record.

They part at index 65.  Kind 0 calls vtable slot 60 immediately.  Kind 1 instead
computes an inline point-to-point distance first -- three fsub against the route
point, fmul plus two fmadd, fsqrt -- and rejects when it exceeds f31, the same radius:

    8226DDC4  fsqrt f0, f0
    8226DDC8  fcmpu cr6, f0, f31
    8226DDCC  bc    4, gt, return-0
    8226DDE8  lwz   r11, 60(r11)      ; then the SAME slot 60
    8226DDF0  bcctrl

So both trigger kinds run the identical point-to-segment test; kind 1 only adds a
cheap early-out against the same radius, and per the drain it does not spawn.

Diffing the two functions rather than reading the second one cost one query and made
the relationship obvious -- the same move that resolved the six ISL branch handlers.

All artefacts regenerate byte-identical; documentation only.

What remains on triggers is peripheral: which unit-record fields A and B are, slot 60
past the t < 0 rejection, payload+8, and the kind-1 path in the drain.
2026-08-27 08:14:24 +00:00
Sylpheed RE agent
950db19f1b re: the kind-0 trigger condition is a point-to-segment proximity test
sub_8226DAF8 past the route-name lookup: it resolves the name through sub_823012D8,
requires the unit record's rec+100 != 0, pulls TWO three-double vectors out of that
record -- rec+32/+40/+48 and rec+64/+72/+80 -- and then makes a virtual call:

    8226DC1C  lwz  r11, 0(r30)     ; the ScriptPhase vptr
    8226DC28  lwz  r11, 60(r11)    ; vtable slot 60
    8226DC30  bcctrl               ; test(phase, A, B, P = the route, f1 = node+16)

Slot 60 resolves to sub_82268068, which no built-in uses, so it is engine-internal.
Its arithmetic leaves no room for interpretation: per-axis fsub, fmul plus two fmadd,
fsqrt, giving |B - A|; reject if that is under 0.1 (the constant is literally 0.1, a
degenerate-segment guard); reject if |P - A| > f1 or |P - B| > f1; then compute
t = (P-A).(B-A) / |B-A|^2 and reject if t < 0.

That is a point-to-segment distance test, and it makes the node's double -- built-in
19's local[24] -- a RADIUS.  So a kind-0 trigger fires when a route point lies within
that radius of the segment between two vectors from the unit's record.

Recorded as unproven: WHAT A and B are.  A previous/current position pair would make
this the standard frame-rate-robust waypoint test, and it would be easy to write that
down -- which is exactly why it is not written down.  rec+64/72/80 and rec+32/40/48
are two positions and nothing here says which.

All artefacts regenerate byte-identical; documentation only.

Still unread: sub_8226DC80, the kind-1 condition, entirely; and slot 60 past the
t < 0 rejection, where a second constant at 0x820B0000+25192 is loaded.
2026-08-27 08:08:01 +00:00
Sylpheed RE agent
5670716566 re: both trigger kinds watch a ROUTE -- the unit-reaches-route mechanism
The kind-0 condition tester sub_8226DAF8 resolves a symbol as well, but from
payload+4 rather than +28:

    8226DB14  rlwinm r10, r5, 2, 0, 29   ; r5 = payload+4, x4
    8226DB24  lwz    r11, 244(r30)       ; [phase+244] = symbol table 1
    8226DB3C  lwzx   r10, r10, r11
    8226DB50  addi   r4, r11, 4          ; -> the name string

Both appenders write local[12] into whichever slot their own path reads -- built-in 19
into +4, built-in 25 into +4 and +28 -- so the same operand is the symbol either way,
and it resolves cleanly:

    built-in 19   local[12] -> symtab 1   79 / 79   all symbol type 1
    built-in 25   local[12] -> symtab 1   25 / 25   all symbol type 1

isl-builtins.md records symtab-1 type 1 as the Route_* names, and the samples agree:
Route_ADN106_p1F, Route_ADN1xxe1_p1F, Route_ADS101_p1F, Route_TCN004_p1S.

So both built-ins register a trigger on a UNIT reaching a ROUTE -- payload+0 is the
unit, local[12] the route.  They differ only in the payoff: built-in 19 additionally
carries a handler routine (local[32] -> payload+28, kind 0 -> spawn) while built-in 25
carries none (kind 1 -> no spawn).  That is a third independent agreement with
isl.py's SYM1_SLOTS, which lists slot 12 for both and was derived from operand ranges
alone.

It also joins up with the closed REMAINING OB work, where the counter was measured to
rise at a squadron's route arrival time -- route triggers are the mechanism that would
do that.  Recorded as a connection, not a demonstration: nothing here traces a trigger
to that counter.

All artefacts regenerate byte-identical; documentation only.

Still unread: what the conditions actually COMPARE -- only sub_8226DAF8's first ~34
instructions are read, up to the name lookup, and sub_8226DC80 not at all -- and
payload+8, a computed value passed to both testers.  A waypoint index would fit it,
which is exactly why it is not being called one.
2026-08-27 08:00:22 +00:00
Sylpheed RE agent
3a56cc6998 re: +24 is the trigger kind tag -- built-in 19 registers a routine, built-in 25 a route
Read straight off the drain, immediately after the pop:

    8226D7AC  lwz   r11, 92(r31)        ; payload+24
    8226D7B8  cmpli cr6, 0, r11, 0x1
    8226D7C4  bc    12, lt, 0x8226D89C  ; +24 == 0 -> the SPAWN path
    8226D7C8  bc     4, eq, 0x8226DA44  ; +24  > 1 -> bail
                                        ; +24 == 1 -> fall through, a NON-spawn path

So the field I recorded last iteration as "kind-tag shaped, but the drain was not
shown branching on it" does branch on it, three ways.

  +24 = 0, written by built-in 19: condition sub_8226DAF8, then bl 0x822737C8 --
      +28 is a code offset, matching the 79/79-vs-27.8%-control measurement.
  +24 = 1, written by built-in 25: condition sub_8226DC80, no spawn, and the path
      resolves +28 through [phase+244] -- symbol table 1:
          8226D834  lwz  r10, 112(r31)     ; payload+28
          8226D838  lwz  r11, 244(r29)     ; symtab 1
          8226D848  lwzx r10, r10, r11

And that resolution checks out: over all 25 call sites disc-wide, built-in 25's
local[12] resolves to a symbol-table-1 entry 25 times out of 25, and every name is a
route -- Route_ADS101_p1F, Route_ADT107_p1F, Route_TCN004_p1S, Route_ADT202A_p2S.
The 1-12 values that looked like a puzzle two iterations ago are simply small symbol
indices.  This independently confirms isl.py's SYM1_SLOTS, which lists slot 12 for
built-in 25 and was derived from operand ranges alone.

So there are two kinds of trigger: built-in 19 registers one whose payload is a
HANDLER ROUTINE, built-in 25 one whose payload is a ROUTE.

All artefacts regenerate byte-identical; documentation only.

Unread: both condition testers (sub_8226DAF8, sub_8226DC80), what the kind-1 path
does with the route past sub_8217FA08 / sub_8218CC88 / sub_8225FEA0, and the drain's
second spawn site at 0x8226DA10.
2026-08-27 07:53:26 +00:00
Sylpheed RE agent
4f1f5ac0a2 re: the unreached code -- five mechanisms eliminated, nothing in the file names it
Inverting the question -- read what an unreached routine DOES, rather than testing
another candidate mechanism -- sharpened this a lot without solving it.

The metric was partly inflated.  Of 24901 unreached instructions in 564 runs, 458 runs
are length 1, and 111 of those are a lone `ret` stranded after an end_coroutine, with
the next routine being a properly seeded entry.  Stage 02's first two "unreached
routines" are exactly that.  But those are only 458 instructions: the real gap is
24443 in 106 runs, the largest 1526 (Stage 26), 1496 and 1481 (Stage 29), 1069, 904.

And it is live mission logic.  Stage 26's 1526-instruction run stages arguments and
calls builtin57(ADN110, 1) and set_group_speed(ADN110, 2, 0) -- named units, named
built-ins.

New this iteration: kill_coroutine (built-in 5) is confirmed as a code-offset carrier,
which is what the corpus said -- its local[0] resolves 150/150 onto the instruction
stream against a 34.0% control, values 13740..221048.  And, like the others, it points
at zero unreached run-starts.

That is now five mechanisms.  Across 2757 code references -- start_coroutine 1765, the
phase timeline 675, built-in 19's trigger handlers 79, kill_coroutine 150, and the
0x1883 records 88 -- not ONE lands on an unreached run-start.  A categorical absence,
not a near miss.

Not a tool artefact: 1765 of 1765 start_coroutine sites resolve with 0 missed, 0
entries land outside their phase region, and 0 are discarded by the on-stream filter.

Combined with the earlier result that an unreached routine's offset appears nowhere in
the file as a word in any encoding, these routines are referenced by nothing inside the
.ssb.  Either they are unused in this build, or they are entered from outside it.  A
1526-instruction routine driving a named squadron sits badly with "unused", but that
discomfort is not evidence and the two readings are recorded without choosing.

No static test I have devised can decide it; a runtime probe logging sub_822737C8's
third argument through a mission would.

All artefacts regenerate byte-identical; documentation only.
2026-08-27 07:46:56 +00:00
Sylpheed RE agent
6402a34d9b re: two trigger appenders -- built-in 19's node carries the handler routine
Resolves the puzzle the previous commit left open: payload+28 is a 1-12 index, yet
the drain hands it to a spawner that wants a code offset.  The cause is that built-in
25 is not the only thing that appends a trigger node.

Searching for functions that write a double at +16 AND a word at +28 of the same
register finds exactly two: sub_8226A0D8 (built-in 25, vtable slot 28) which fills the
payload inline, and sub_8226E160, a free-list recycling variant whose sole caller
sub_82269ED0 is vtable slot 68 = BUILT-IN 19.

Built-in 19 passes local[4] -> node+0 (the unit), local[12] -> +4, local[24] -> +16 as
a double, and local[32] -> +28.  That last one is the field the drain spawns from, and
it is a genuine code offset:

  over all 79 call sites disc-wide
    land on the instruction stream    79 / 79 = 100.0%
    control, random 4-aligned offsets           27.8%
    values 12164 .. 111080, 73 distinct

Large, distinct, and every one resolves against a control that resolves barely a
quarter of the time.  So built-in 19 registers a trigger whose handler is a routine,
and the drain's `base + payload+28` spawn is that handler.  The two builders simply
put different things in the same slot.

Observed alongside: built-in 25 writes +24 = 1 and built-in 19 writes +24 = 0, which
is kind-tag shaped -- but the drain was not shown branching on it, so that is recorded
as an observation and not a reading.

All artefacts regenerate byte-identical; documentation only.

Still open: whether +24 selects between the node kinds, and this STILL does not
explain the unreached code -- 0 of the 79 handler offsets are unreached run-starts.
2026-08-27 07:38:24 +00:00
Sylpheed RE agent
9ee32f7940 re: decode the trigger node -- and withdraw two of my own claims about it
The reader is sub_8226E220, called by the per-frame drain sub_8226D740 as
(phase+272, six out-params).  It pops one node and copies seven payload fields out.
Built-in 25, the appender, writes exactly those seven offsets:

    +0   local[4]  -- the unit          +24  the constant 1
    +4   local[12]                      +28  local[12] again
    +8   computed                       +32  --
    +16  a DOUBLE from local[16]

Neither side was derived from the other, so the agreement is the check.  Container:
+16 head, +20 pending count -- which matches the count isl-builtins.md watched live at
phase+272+20 from a completely different direction -- and +24 a cursor.

WITHDRAWN, from the previous commit: "the drain spawns from [node+112]".  r31 = r1 -
256, the stack frame, so +112 is an output slot and not a node field.

REFUTED, the follow-up hypothesis that the trigger carries the routine offset and so
names the code nothing else starts: payload+28 is local[12], and across all 25 call
sites disc-wide those values are the small integers 1 through 12 -- 1 of 25 (4.0%)
land on the instruction stream against a 16.0% control, and none are unreached
run-starts.  Below chance.  My first version of that test used n=2, Stage 02 only; it
happened to agree, but two samples could not have supported it either way.

One correction in the other direction, to the corpus: isl-builtins.md withdrew
sub_8226E458's link to the trigger queue on the grounds that "the argument is
lwz r4, 324(r29), the unit array, not the trigger container".  That is the SECOND
argument; the first is r26 = phase + 272, set twelve instructions earlier.  The drain
does operate on the container.  What sub_8226E458 does to it remains unread, so only
the argument is corrected, not the conclusion.

All artefacts regenerate byte-identical; documentation only.

Still open: what local[12] indexes; what the drain actually spawns, since [stack+112]
is filled from payload+28 yet the spawner wants a code offset, so some step in that
chain is not what I read; and what starts the ~15% of unreached code.
2026-08-27 07:30:41 +00:00
Sylpheed RE agent
a3cb3339bc re: sub_822737C8 is the coroutine spawner; the record's third word is the MAIN ENTRY
Asking who calls the function the timeline calls enumerates every way an ISL routine
can begin.  sub_822737C8(phase, base, offset) computes base + offset early on, and has
seven real call sites: the phase initialiser sub_82270DF8, the built-in stub region
(start_coroutine), the timeline walker sub_822748D0, TWICE inside sub_8226D740 -- the
per-frame engine->script drain -- and two unread, sub_82273910 and sub_82264058.

CORRECTION to my own write-up: the 0x1883 record's third word is the phase's MAIN
ENTRY, not a "size".  The initialiser hands it straight to the spawner:

    8227101C  or  r5, r22, r22     ; the record's third word
    82271020  or  r4, r26, r26     ; the code base
    82271030  bl  0x822737C8

44 of 44 records land on the instruction stream (100%) against a 25.0% control, and
all three Stage-02 targets open with the identical prologue
`special[0]=0 ; local[0]=0 ; call builtin116(0)` -- a routine entry, not a length.
So the record is 0x1883, base, MAIN_ENTRY, 0, code_end, force_end_handler.

Seeding the main entries moves no coverage number: every one was already among the
CFG's entry points by another route.  This corrects a field's meaning, not the graph.

Lead recorded rather than claimed: both of the drain's spawns take their offset from
[node+112], the first field of a drained node to be located, and the best remaining
angle on the ~15% of code nothing appears to start.  It is NOT shown that those nodes
come from the trigger queue at phase+272 -- that is precisely the over-reach
isl-builtins.md already made and withdrew, so it is not asserted here.

All artefacts regenerate byte-identical; this is documentation only.
2026-08-27 07:22:51 +00:00
Sylpheed RE agent
4e948d0317 re: the timeline's consumer confirms the decode, and kind is a clock index
The schedule decode previously rested on structural checks alone.  The engine's own
reader is now found, statically.

The phase initialiser sub_82270DF8 takes the 0x1883 record's six words as arguments
and stores the table pointer:

    82270FDC  add  r10, r26, r21      ; code_base + entry_a
    82270FEC  stw  r10, 240(r30)      ; [phase+240]

[phase+240] has exactly two readers in the ISL region, and one -- sub_822748D0 -- is
called from sub_82263408, ScriptPhase::Update.  It walks the table and confirms every
field independently of my inference: N at table+4, kind at group+20, float t at
group+12, offset at group+4, and a 24-byte stride (three 8-byte records).

kind is NOT a flag.  `rlwinm r11, r11, 2, 0, 29` then `lfsx` -- it is multiplied by 4
and used to index float arrays at [phase+104] and [phase+88], so it selects WHICH
TIMER the entry is scheduled against; 0 and 5 are two of at least six.  An entry fires
only when its t lies between the two readings, the classic "which events did this
frame cross" test.  Which array is previous and which is current follows from the two
comparison directions and is stated as a reading of those branches, not measured.

Both arrays are fields of the phase object, so the clock is PER PHASE -- which is why
every phase's table restarts at t = 0.  That was an inference from the layout in the
previous commit; it is now a read.

All five artefacts regenerate byte-identical: this changes documentation only.

Still open and said so: what the six clocks are (only indices 0 and 5 appear in the
data, and neither array was traced to a source); sub_822737C8, which actually starts
the routine, is unread; and this is still not what starts the unreachable code, since
0 of the 675 targets are unreached run-starts.
2026-08-27 07:15:25 +00:00
Sylpheed RE agent
2463748a71 re: the trailing data table is a per-phase TIMELINE of scheduled routines
Decodes the table found at the end of every phase region.  Layout:

    int  N
    N x [ int offset ; float t ; int kind ]      -- 8-byte typed records,
                                                    tag 0x19 int, 0x1A float

1 + 3N matches the record count in every phase measured (Stage 02: 76/40/55
records for N = 25/13/18).

Checks, all independent of each other:
  schedule entries disc-wide                675
  0x1A float records disc-wide              675   (counted by a different route)
  offsets landing on the instruction stream 675/675 = 100.0%
  control, random 4-aligned offsets                  33.3%

The floats are seconds -- 0, 0.5, 1, 4, 5, 30, 50, 60, 90, 120, 150, 170, 180, 210,
240, 270, 300, 330, 360, 420, 570, 1020, 1080, 1140, 1170 -- and the targets are small
one-shot coroutines that set arguments, call one built-in and end_coroutine.  kind is
0 (556) or 5 (119) and is not identified.

Runtime cross-check, recorded as consistency rather than confirmation: the closed
REMAINING OB work measured Stage 02's squadron arrivals at t = 0, 120 and 210 s over
n=5 emulator runs, and all three appear in phase 1's static schedule, with 120 and 210
each appearing TWICE.  These are round numbers and phase 1 has ~22 distinct times over
0-1170, so presence alone is not unlikely; the doubling is the sharper detail and was
not predicted in advance.

New artefacts data/isl-stage02-schedule.txt and data/isl-schedule-all.txt with a
committed generator (isl_report.py schedule).  calls, phase-ends, conditions and
phase-guards all regenerate byte-identical.

Not settled and said so: kind is unread; the consumer is unread, so the decode rests
on the structural checks above; whether the clock is per-phase or per-mission is an
inference from the layout; and this is NOT what starts the unreachable code -- 0 of
the 675 targets are unreached run-starts, so that ~15% gap stands.
2026-08-27 07:07:35 +00:00
Sylpheed RE agent
02d3c9c82b re: entry_a is a code/data boundary, not an entry -- and the decoder was reading data as code
Disassembling the three Stage-02 entry_a targets shows opcodes 0x19 and 0x1A, and the
ISL dispatcher's table has 25 entries (cmplwi 0x18).  They are not instructions.  Each
phase region ENDS with a trailing data table of 8-byte typed records -- tag 0x19 = int,
tag 0x1A = IEEE float (0.0, 0.5, 1.0, 4.0) -- and entry_a is where it starts.

Confirmed across the disc: in 44 of 44 phases the first offset whose opcode exceeds
0x18 is exactly that phase's entry_a, with zero exceptions, and only two tags ever
appear (1394 x 0x19, 675 x 0x1A).  So the record is

    0x1883, base, size, 0, code_end, force_end_handler

one boundary and one entry, not two entries as the previous commit said.

That also retires this thread's own "82 of 88 land on a valid instruction = 93.2% vs a
38.6% control" as TOO WEAK a test: a data record has length 8 and passes "nonzero,
even".  The entry_b result stands on different evidence -- those targets were matched
against isl.call_sites(), an independent enumeration.

isl.linear_offsets was decoding all 2069 data records as instructions, 1.23% of the
stream.  Now each phase's walk stops at its boundary:

  decoded instructions   168251 -> 166182  (= 168251 - 2069, as predicted)
  opcode > 0x18               2069 -> 0
  call sites covered     25705/25705 -> 25705/25705
  exits unreachable                0 -> 0
  conditions unknown             400 -> 400

Recorded because the first attempt at the fix was worse than the bug: it destroyed 36%
of the stream (168251 -> 107596, exits 0 -> 74) because linear_offsets is ONE global
walk from the first phase base, so stopping at phase 1's table lost every later phase.
It has to skip the region and resume at the next base.  A count moving hard in the
wrong direction is the same signal as one that will not move.

Still open: the table's contents are undecoded -- its int values land on the
instruction stream 46/51 against a 29.5% chance rate, but 0 of them are unreached
run-starts, so this is not what starts the unreachable code either.
2026-08-27 06:59:58 +00:00
Sylpheed RE agent
5c4ae3ad7b re: the 0x1883 record carries entry points -- every phase exit is now reachable
data/isl-phase-guards-all.txt goes from 5 of 177 unreachable exits to 0.

The cheap first step failed, usefully.  An unreached routine's entry offset does NOT
appear as a word anywhere in the file, in any encoding: phase-relative 6.6% against an
11.5% control on reached offsets, absolute 1.6% vs 3.3%, and the /4 forms 0-1.6% vs
6.6-8.2%.  Every variant is at or below its control, which rules out the whole family
of "some instruction operand points at them".  It also rules out dead code: Stage 02's
3069 unreached instructions contain 485 calls, including start_coroutine x75,
squadron_attack x59, set_group_speed x42 and objective_marker x13.

The answer is the mission-level stream that isl-bytecode.md already partly read.  Each
0x1883 record is

    0x1883, base_delta, size, 0, entry_a, entry_b     ; entries PHASE-RELATIVE

Measured over all 28 stages, 82 of those 88 values land on a valid instruction --
93.2%, against a 38.6% chance rate for a random 4-aligned offset.  In Stage 02
entry_b is the phase's force-end handler: 0x1482C, 0x249F0 and 0x34A10, two of which
were exactly the unreachable exits, and the third being already reachable is the
consistency check.

Seeding them: exits unreachable 5 -> 0.  Those exits now report 0 necessary
conditions, which is what an engine-entered abort handler should look like.

Recorded because it is the same mistake twice: the first seeding attempt moved NOTHING
(reach 85.0% -> 85.0%, exits 5 -> 5).  dominating_conditions() builds its own entry set
and did not use the one I had patched -- fix-the-instance-not-the-class again, caught
only because an unchanged count is by now a standing signal.

Not settled and stated: reach went only 85.0% -> 85.2%, so what starts the other ~15%
of code is still unknown, and the negative above says it is not an operand in the file;
entry_a is unidentified; 6 of the 88 values do not land on an instruction.
2026-08-27 06:51:16 +00:00
Sylpheed RE agent
da6eb236ef re: the trigger-queue appender is built-in 25 -- the script appends, not the engine
isl-builtins.md carried the trigger queue at phase+272 a long way -- layout, live
count at +272+20, the generic appender sub_8226EAB8 -- and stopped at "what actually
appends a node is still unidentified", proposing a gdb watchpoint on a running
emulator.  It is a static question after all.

sub_8226EAB8 is a shared helper with 16 callers, so it cannot be identified by name.
But the corpus records that the trigger container EMBEDS its inner list at +12, so the
appender must hand it container+12.  Only two of the sixteen callers do, and one sets
its base 34 instructions earlier:

  8226A22C  addi r29, r24, 272     ; r24 = the ScriptPhase
  8226A2B4  addi r3,  r29, 12      ; -> phase + 272 + 12
            bl   0x8226EAB8

phase+272+12 is exactly the inner list whose count the corpus watched at +272+20.

sub_8226A0D8 is ScriptPhase vtable SLOT 28, which is what BUILT-IN 25's stub
tail-calls, and there are ZERO real `bl 0x8226A0D8` instructions in the image -- it is
reached only through the vtable.  So the queue is populated by the SCRIPT, retiring
the residual "the engine moves records into the phase's queue each frame" reading that
isl-builtins.md had already flagged as unsupported by its own call site.  Built-in 25's
arguments agree with two independently measured tables: UNIT_ARG (a unit at local[4])
and SYM1_SLOTS[12] (a symtab-1 name at local[12]).

REFUTED in the same pass, with a control: built-in 25 does NOT carry the coroutine
entry that a trigger fires, which would have seeded the 389 routines reachable from no
static entry.  Taking each staged operand as a code offset from the phase base, 0 of 8
land on an instruction boundary, against 38.7% for an arbitrary 4-aligned offset --
below chance, not above it.  And the scale is wrong regardless: built-in 25 has only
2 call sites in Stage 02.

So what starts the unreachable routines is still open, and it is not this.  A trigger
node's own layout and its consumer are unread, and built-in 25 stays unnamed.
2026-08-27 06:42:47 +00:00
Sylpheed RE agent
a2c9486b20 re: the sufficient side -- each phase exit now names the condition that FIRES it
Dominance said a phase cannot end unless X.  A port also needs "once X holds, it
must end", and that is a must-reach set: nodes from which END_PHASE is unavoidable,
as a least fixpoint where n qualifies when it has successors and ALL of them qualify.

The conservatism is deliberate and is the honest answer: a loop never enters the set,
because a poll loop reaches its exit only if the polled predicate eventually becomes
true, which is a liveness property rather than a graph one.

A dominating condition is a TRIGGER when the successor it takes on being satisfied
lies in that set.  Over all 28 stages: 732 dominating conditions, 234 triggers
(31.97%).  isl_report.py phase-guards now tags every line precond / TRIGGER.

The split lands where it should.  Stage 02's phase-1 objective exit is six
preconditions -- player alive, TCN004 destroyed, t <= 210, ADT102/ADT107/ADT113
destroyed -- and exactly ONE trigger: hp_pct_test(ADN101, 0) != 1.  Destroying ADN101
is what fires the phase.  That is a sentence a port can implement.

Per-exit distribution over 172 reachable exits: 89 have exactly one trigger, 42 have
none, 41 have several.  The 42 with none are not a failure -- they are the exits no
branch fires; Stage 02's 0x006260 ends on read_freg(0) < 1200, a timeout, and time
passing is not a property of the graph, so declining to call it a trigger is correct.

Recorded as a heuristic rather than a rule: "the first trigger is the point of no
return" holds for 33 of the 41 multi-trigger exits, with 8 counterexamples where a
precondition appears after a trigger.  The likely cause is that the listing is
ordered by file offset, which is not execution order -- coroutines and jumps let a
lower offset run later.  Not asserted.

calls, phase-ends and conditions all regenerate byte-identical; the two phase-guards
artefacts change only by gaining the tags.
2026-08-27 06:34:39 +00:00
Sylpheed RE agent
7bd6342061 re: read the three built-ins that appear inside clear conditions -- none named
isl-phase-guards produced the per-phase clear conditions, but three of the built-ins
in them were unread, leaving the conditions half-readable.  All three are read now
via the vtable at 0x820A84BC (control: 69 unit_state -> slot 184 -> 0x8226ADF0).
NONE is named: the corpus has withdrawn two names taken from usage shape, and what I
read does not support naming these.

builtin104 (slot 288, 0x8226BFE0) is a THREE-INSTRUCTION PURE GETTER:
    lwz r11, 10160(r3) ; stw r11, 164(r3) ; blr
So all six tutorial stages S18-S23 end on the value of a single engine-written word,
which is why their exits each have exactly one dominating condition and why
isl-builtins.md only ever saw 104 inside a poll loop.  That word has EXACTLY ONE
writer in the image -- sub_821AA1B0 at 0x821AAD9C, gated on a kind field being in
(16, 32] -- and r29 there is the return of a preceding call, so the meaning of the
value and of the gate are both unestablished.

builtin7 (slot 40) indexes [phase+324] by local[4], bails when rec+16 (the documented
unit state) is 1, 3 or 4, and resolves local[12] through [phase+244] -- symbol table 1.
isl.py's SYM1_SLOTS already listed slot 12 for built-in 7, derived from operand ranges
alone; reading the implementation shows the mechanism behind that.  Two independent
methods agreeing.  What it asks ABOUT the route is still unread.

builtin141 (slot 428) has the same unit-array entry and returns 0 when the unit is
gone; everything past 0x8226CA0C is unread.  Stage 16 calls it twice with arguments
differing in one position (0 vs -4000), which looks like a coordinate -- and looking
like one is exactly the evidence this corpus does not accept.

No artefact changes: the listings already printed builtin104/7/141 and still do.
All four Stage 02 artefacts regenerate byte-identical.
2026-08-27 06:27:00 +00:00
Sylpheed RE agent
b636ac9d4e re: phase-guards for all 28 stages, and a 6/6 cross-check from an unrelated method
isl_report.py now accepts a directory, so the dominance analysis runs over the whole
disc: data/isl-phase-guards-all.txt, 177 phase exits, of which only 5 (2.8%) are
reachable from no static entry.  CFG reach ranges 69.5% (S26) to 95.8% (S25), median
about 4 dominating conditions per exit.

The lopsided number in the per-stage table was the six TUTORIAL stages, S18-S23, each
with exactly ONE exit and exactly ONE dominating condition.  That could have been a
degenerate result, so I looked: it is the same condition in all six,

    END_PHASE  <-  builtin104() != 1

and isl-builtins.md reached built-in 104 from call-site USAGE alone -- "S18-S23 only,
followed by wait_s 39/39, preceded by end_coroutine 37/39, a textbook poll loop".
Usage said 104 is the tutorial's polled test; dominance says it is the tutorial's
clear condition.  Two unrelated methods, six for six.

Stage 16 -- the corpus outlier whose script may be compiled C++ -- resolves as well:
read_freg(0) < 600, player_gauge0_test, player_gauge1_test, and two builtin141 calls
differing in a single argument (0 vs -4000), which is the shape of a position or zone
test.  builtin141 is unread, so it is not named.

Stage 02's separate artefact regenerates byte-identical.

Also added: an RLIMIT_AS cap in isl_report's entry point.  The dominator pass
OOM-killed a run earlier on this 15 GB box; a bad input should now fail the process
rather than the machine.

Still not settled and stated in the doc: dominance gives necessary, not sufficient,
conditions; the 5 unreachable exits need the trigger queue at phase+272; builtin104,
builtin141 and builtin7 all appear in clear conditions and are unread.
2026-08-27 06:19:44 +00:00
Sylpheed RE agent
4f95b98813 re: the per-phase clear conditions, by dominance over the ISL CFG
Closes the backlog's "which condition guards each END_PHASE".  With the CFG from the
previous commit this is a graph query, not new machinery.

The obvious query is WRONG for this language, and I implemented it first: "one
successor reaches END_PHASE and the other does not" finds 1/62/1 guards across Stage
02's three phases, and the 1s are both the same read_freg(0) < 1200 timeout -- every
objective test missed.  The cause is the dominant idiom: a POLL LOOP's loop-back
branch also reaches the exit, one iteration later, so neither successor discriminates.
The asymmetric 1/62/1 is what exposed it; a uniform number would have read as
plausible.

Dominance has no such blind spot: a condition dominates an exit when every path from
an entry passes through it, so it is NECESSARY for the phase to end that way, and a
poll loop's test dominates its own exit by construction.  Iterative dominators
converge in 3 passes over 15670/18739 instructions (83.6%).

Result for Stage 02 -- every exit in all three phases is dominated by
unit_hp_pct(TCN001, Character_Player_Test) != 0, the player's ship being alive, which
falls out rather than being assumed.  Beyond that, phase 1's objective exit requires
hp_pct_test on ADT102, ADT107 and ADT113; phase 3's requires ADT301 and ADT302;
read_freg(0) gates at 210 / 300 and times out at 1200; random(3) and random(5)
dominate only the exits that pick one of several closing lines.

Two of the 15 exits are reachable from NO static entry, both FORCE_END_PHASE.  That
agrees with the independently measured 389 unreachable routines: they are started from
the trigger queue at phase+272, by data rather than code.

Practical note recorded: the first dominator run was OOM-killed -- 6743 nodes each
holding a Python set of up to 6743 elements.  Integer bitmasks run in seconds.

Not settled, and said so: dominance gives necessary, not sufficient, conditions; only
Stage 02's artefact is committed; one listed condition is still an unresolved
<unknown>; read_freg's units are inferred from the gate values, not read.

calls, phase-ends and conditions all regenerate byte-identical.
2026-08-27 06:12:41 +00:00
Sylpheed RE agent
5ea9e38b35 re: recover ISL conditions by CFG dataflow instead of a linear walk
The linear walk's 10% unknown was a floor imposed by the method: a block entered only
by a branch has a well-defined state, just not one a straight-line pass can see.
tools/re-capture/isl_cfg.py replaces it with a worklist fixpoint that joins each
block's state over its ACTUAL predecessors -- a value survives only if every
predecessor agrees.

Over all 28 stages:
  instructions reached by the CFG          85.0%
  condition sites, unknown LHS             756 (10.00%) -> 402 (5.32%)
  of those, never reached at all           389
  joined away (predecessors disagree)       13
  both resolve but DISAGREE                161   <- linear walk was wrong here

Those 161 are on top of the 889 the previous jmp fix caught.

Two zero-results on the way, both my own bug, both caught because the number looked
wrong rather than because a test failed:

  * The first CFG run reached only 36% of instructions and made things WORSE (35%
    unknown).  Cause: the phase bases reach almost nothing.  Most routines are
    COROUTINES the engine starts from its trigger queue, with no static predecessor,
    so every start_coroutine target has to be seeded as an entry.
  * That seeding then found ZERO entries in a file with 216 start_coroutine calls,
    because the target is staged in TWO steps -- special[0] = imm, then
    local[0] = special[0] -- and I matched only the direct-immediate form.

Reachability went 36% -> 64% -> 85% as each was fixed.

The 389 still unreached are an honest limit rather than a gap: nothing in the bytecode
starts them; they are entered from the trigger queue at phase+272, by data rather than
code, so no purely static analysis reaches them.

isl_report.py conditions now uses isl_cfg; calls and phase-ends regenerate
byte-identical.  Stage 02 unknowns drop from 71 to 25.
2026-08-27 06:02:37 +00:00
Sylpheed RE agent
58b404aef4 re: builtin80 is a command -- and finding that exposed an 11.75% bug in my tracker
Reading builtin80's body (0x82268460) to name it: it is NOT a predicate.  It
allocates a 20-byte object, stamps vtable 0x820A8CB0, magic 0xAB0311BA and the
unit's live object into it, pushes it onto a queue via the same helper push.i uses,
and returns 1 -- or 0 when the unit is absent.  A command.

That made the conditions listing impossible: it showed a six-way switch
`if builtin80(TCT206) == 0 … == 5` on a function returning 1 or 0.  Disassembling the
site shows two unconditional `jmp`s between the call and the compare, so 0x1B6C0 is
reached ONLY by a branch and its special[0] has nothing to do with builtin80.

op12 is unconditional -- the next instruction is never reached by fall-through -- and
the tracker walked through it exactly as it had walked through end_coroutine.  Last
iteration I fixed the instance and not the class, leaving 22x more bad sites in place
than the fix removed.

A/B over all 28 stages, 7563 sites, resetting at jmp as well:
  sites whose operands change              889  (11.75%)
  LHS unresolved, before -> after     34 (0.45%) -> 756 (10.00%)

So the previous commit's headline "0.0% unresolved" was a MISSING CHECK, not a strong
result: the linear walk always had some value to report, and reporting it was the bug.
10% is the honest figure and the other 90% is trustworthy for a reason.

Also corrected: isl-unit-args.md illustrated its diff with 0x1B6C0, which is one of
the bogus sites.  The UNIT_ARG result itself stands -- it came from reading
implementations, not from this listing -- but the example was picked from bad output.

Not done, and said so: recovering the 756 needs a dataflow join over each block's
actual predecessors, a CFG fixpoint rather than a linear pass.  The branch targets are
all known so the CFG is available; the analysis is not written.

calls and phase-ends regenerate byte-identical; conditions changes on 187 lines.
2026-08-27 05:53:20 +00:00
Sylpheed RE agent
126539b875 re: derive UNIT_ARG from the implementations -- 31 built-ins take a unit, not 31 of 55
isl.py's UNIT_ARG decides whether a built-in's slot-4 operand prints as a unit name
or a raw number.  It was inferred statistically from operand ranges and, by its own
comment, listed a slot "only when the ratio stayed below 1.0" -- conservative.

The vtable base makes it a lookup instead: every unit-taking built-in's implementation
opens with lwz 324(phase) / lwz 4(argbase) / rlwinm 2,0,29 / lwzx / lwz 4(rec).  Read
directly for all 147:

  implementation indexes [phase+324] by an argument   55
  of the statistical set's 31, confirmed              31  (zero false positives)
  UNIT_ARG claims a unit, implementation does not      0
  implementation says unit, UNIT_ARG missed it        24

The 24 include builtin80, group_ratio_pct, is_engaged, set_unit_flags,
squadron_trace, wait_units_ready and deploy_and_wait.  Hand-verified by reading
builtin7, 16, 80, 105, 117 and 136.

Recorded because it nearly passed: my FIRST control -- whether the additions' operands
resolve to a symbol-table-2 index -- is worthless.  The additions score 100.0%, but so
do the 31 baseline (100.0%) AND the 92 built-ins in neither set (99.3%).  Symtab 2 is
dense enough that almost any small integer lands in it.  A control the negative class
also passes is not evidence.

The control that discriminates is the tag word: a symbol operand is a two-word pair
whose first word is the constant 1, so slot0 == 1 exactly when slot 4 is a unit --
100.0% (13677 calls) / 100.0% (140) / 2.5% (2903).  A 40x separation.

Artefacts: isl-stage02.txt and -phase-ends.txt regenerate byte-identical; -conditions
changes on 28 sites, every diff line pairing, each a raw number becoming a unit name.

Left unnamed on purpose: all 24.  builtin80 returns a small enum (tested 0..4 in a
switch) but its body past the liveness check is unread; builtin103 is a predicate over
[phase+10152]/[phase+10156]; builtin105 tests a unit record's +16 against 4.
2026-08-27 05:45:10 +00:00
Sylpheed RE agent
1ac9aa1d7d re: fix the wrong ISL conditions -- the cause was a coroutine boundary, not the filter
The listing showed end_coroutine as the left-hand side of 34 comparisons disc-wide.
That is impossible -- it returns no value a script can test -- so it was the bug
reporting itself.

The recorded fix ("set special[0] only for built-ins that write [phase+164]") is
REFUTED.  end_coroutine's handler 0x82272624 is `addi r11,r0,1 ; addi r3,r0,3 ;
stw r11,164(r31)` -- it DOES write [phase+164], so that filter would have kept it.
Reading the handler before writing the filter is what caught this.

The real cause: end_coroutine returns 3, which DESTROYS the thread.  Execution does
not continue past it, so the instructions following it in the flat stream belong to
a different routine and every tracked value is stale.  The linear walk that makes
the decode possible is exactly what walks across that boundary.

A/B over all 28 stages, 7563 sites, resetting the tracker at end_coroutine:
  sites whose operands change            34  (0.45%)
  LHS = end_coroutine, before -> after   34 -> 0
  left as an explicit unknown            34  (0.45%)

The two counts being equal is the result: the leak was confined to exactly the sites
that displayed the impossible value, so the other 7529 conditions were never
affected.  Those 34 now print "<unknown: reached after a coroutine boundary>".

Not done, and said so: their RHS is still exact and the LHS is recoverable by seeding
the tracker at coroutine entries, whose targets are staged slot 0 of start_coroutine.

data/isl-stage02-conditions.txt regenerated; calls and phase-ends both byte-identical.
2026-08-27 05:36:02 +00:00
Sylpheed RE agent
eac5b3e22e re: resolve every ISL condition's comparand -- the clear conditions are readable
The deque ops are an EXPRESSION STACK: push the left operand, evaluate the right
(a built-in call, whose result lands in special[0]), pop the comparand back into
special[1], compare.  Tracking that through the linear decode is enough to recover
what each site tests.

Evidence the model is right, not just plausible:
  push vs pop across all 28 stages          1877 vs 1877
  files that underflow or end unbalanced    0 of 28
  Stage 02 pop.i sites followed by cmp.i    319 / 319
  ops immediately before a pop.i            call x313, cmp.a x6

isl.conditions() recovers 7563 condition sites disc-wide with 0.0% left as an
unresolved special[N]; 83.2% have a built-in call as the LHS and 99.7% compare
against a plain number.  Most-tested: hp_pct_test 1955, unit_state 1257,
unit_relation 796, dist_lt 450, unit_alive 413.

They read as conditions now:
  if unit_alive(TCN105) != 1
  if hp_pct_test(ADT308, 0) != 1
  if dist_lt(ADT308, TCN000, 15000) != 1      (world unit = 1 m, so 15 km)
  if unit_state(ADT308) == 1

data/isl-stage02-conditions.txt was a stale artefact with NO generator -- the thing
isl_report.py's docstring complained about.  It has one now (isl_report.py
conditions).  The calls and phase-ends artefacts both regenerate byte-identical, so
the change is additive.

Recorded rather than glossed: 15 of Stage 02's 965 sites (1.6%) attribute the LHS to
end_coroutine, which returns no value -- the tracker sets special[0] on EVERY call,
so those show a stale value and are wrong, not imprecise.  The fix is to set it only
for built-ins that write [phase+164], which the vtable work makes checkable.
2026-08-27 05:28:50 +00:00
Sylpheed RE agent
142b8d7200 re: close the ISL operand chain -- built-in result is special[0], via a vtable
Answers what the previous commit left open: naming the branches did not give a
clear condition, because that needs the operand chain feeding each compare.

First, a correction to my own work.  isl-bytecode.md -- which OWNS the opcode table
-- already named ops 21-24 push.i/push.f/pop.i/pop.f.  isl-branches.md, which I
wrote last iteration, said op21 and op23 were unread.  The stale file was mine.
Verified from the thunks rather than accepted: 21 pushes [phase+168] onto the deque
at phase+44, 22 pushes [phase+184] onto phase+64, and the 23/24 handlers touch only
r3+168 and r3+184.  So pop.i lands in special[1].

New: the 147-entry built-in table is a thin DISPATCH LAYER, not implementations.
Each stub resolves the local[] argument base and tail-calls a fixed ScriptPhase
vtable slot.  112 of 147 dispatch that way; 17 write [phase+164] inline; 0 write
+184.  Every named predicate is in the vtable group -- unit_state 184, unit_alive
188, hp_pct_test 64, dist_lt 56, is_engaged 252, timer_elapsed 372 -- which is the
control that the split separates engine queries from script bookkeeping.

The vtable is 0x820A84BC, derived from a known implementation rather than a stride:
MARK_LAST_PHASE is documented as [phase+300]=2; the function 0x8226B498 is exactly
that stub; it appears as a data word at exactly one address, 0x820A8570; built-in
39 uses slot 180.  The check NOT used in the derivation: built-in 40 mark_not_last
uses slot 176, and slot 176 holds the [phase+300]=1 stub.  Predicted and confirmed.
The db's own vptr_writes independently lists 0x820A84BC, written at 0x82261B80.

unit_state = slot 184 = 0x8226ADF0, which indexes [phase+324] by local[4] and writes
its answer to [phase+164] = special[0] at both exits.  The phase-3 poll loop now
reads end to end: unit_state(ADT308) -> special[0]; pop.i -> special[1]; cmp.i; beq.

isl.py names ops 21-24; the calls artefact regenerates with NO diff.

Left open and said so: the other 111 vtable slots, which comparand each site pushes,
the 35 non-vtable built-ins, and the vtable's length.
2026-08-27 05:21:10 +00:00
Sylpheed RE agent
bad96eb54a re: read the ISL branch handlers -- it is a condition-code machine
Closes the backlog item that was the last thing between the flat decode and a
per-phase clear condition, and closes isl-builtins.md's standing "op10 + op13 look
like a switch -- NOT confirmed".

op10 resolves two operands, issues a SIGNED cmp, and writes three condition bits to
a bitset at phase+24: bit 0 = EQ, bit 1 = GT, bit 2 = LT.  op11 is the same machine
for floats via fcmpu.  op13-op18 branch on those bits to [phase+232] + word@+4 --
the same phase-relative target form as the unconditional op12:

  13 bit0 set    beq      16 bits 2 then 0   ble
  14 bit0 clear  bne      18 bits 1 then 0   bge
  15 bit2 set    blt      17 bit1 set        bgt

13/14/15/17 are byte-identical apart from the bit index and the polarity.  All six
relations are present and each appears exactly once; that completeness is the check
that the reading is right, rather than the usage pattern -- which the item
explicitly warned against.

Operand order recorded because it is easy to reverse: LHS = (kind byte[1], word@+4),
RHS = (kind byte[0], word@+8).

Method note in the doc: the jump table at 0x822635FC holds THUNKS, and the handler
is the bl target inside each.  My first pass guessed handler addresses at a fixed
stride, landed mid-function, and produced a 20-line "difference" that was pure
misalignment.

isl.py names the ops; data/isl-stage02.txt is regenerated and every diff line pairs
exactly, only the op-name column changing (op10->cmp.i x5, op13->beq x4,
op14->bne x1).  data/isl-stage02-phase-ends.txt now shows the phase-3 poll loop
reading as one: unit_state(ADT308) -> op23 -> cmp.i -> beq back to 0xFEB4.

Left unnamed on purpose: op23 (0x82271C30) and op21 (0x82175C20).
2026-08-27 05:10:50 +00:00
Sylpheed RE agent
f41847701c re: the ISL stream is flat -- refute the "needs coroutine entry points" blocker
Two files (isl_report.py's docstring and structures/isl-builtins.md) recorded the
same blocker on a faithful per-phase condition listing: that it needs the coroutine
entry points from start_coroutine's operand.  Measured against isl.call_sites(),
which enumerates by scanning the encoding rather than by decoding and so is an
independent denominator:

  linear + jumps, stopping at ret (what the tool did)   133 / 2846 =   4.7%
  linear + jumps, continuing past ret                  2275 / 2846 =  79.9%
  ... + following start_coroutine (the recorded fix)   2355 / 2846 =  82.7%
  plain linear decode, no control flow at all          2846 / 2846 = 100.0%

Following the coroutine entries buys 2.8 points.  Disc-wide, a plain linear decode
from the first phase base reaches 25705/25705 call sites over all 28 stages, and
28/28 decode clean to code_end with no desync.

The real bug was isl.dis ending on `if op == 20: break`.  Op 20 is `ret`, but this
is a coroutine VM -- the thread suspends and resumes at the FOLLOWING instruction,
so code continues past it.  dis() now takes stop_at_ret (default True, preserving
the old output: data/isl-stage02.txt regenerates byte-identical) and
isl.linear_offsets() is the correct walk.

By-product, kept with its control: start_coroutine's target is staged slot 0 --
73/83 phase-1 sites land on a valid instruction, against a 38.7% chance rate for an
arbitrary 4-aligned offset.

New artefact data/isl-stage02-phase-ends.txt with a committed generator
(isl_report.py phase-ends).  It shows END_PHASE's call site is the WRONG place to
read a clear condition: all 12 Stage-02 sites sit in one stereotyped outro.  Not
settled, and stated as such: op10/op13/op14/op21/op23 are unread handlers, so the
condition in the poll loop upstream cannot be named yet.
2026-08-27 05:00:49 +00:00
Sylpheed RE agent
ef4e975a47 re: diff the legacy IDXD reader against the field table -- 17.54% wrong, with an exact predicate
Answers the backlog's open "first step: diff the two readers across the disc and
count disagreements", statically over every IDXD object.

Of 7750 objects and 738922 named fields whose true value is numeric, legacy
get_f32 is correct 39.42%, returns None (harmless) 43.04%, and returns a WRONG
NUMBER 17.54% (129612 fields).

The wrongness has an exact predicate: single-record objects 0 of 29822 wrong
(0.00%); multi-record objects 129612 of 709100 (18.28%).  The mechanism is in
get_raw itself -- it flattens the pool to a token list, finds the FIRST occurrence
of the key, and returns the preceding token, with no notion of records.  So every
record after the first inherits record 0's value: Weight truth=1.0 legacy=0.3,
Points truth=10000 legacy=4000.

Practical rule recorded: a get_f32 number from a single-record object is safe; from
a multi-record object only the first record is.

Withdrawn in the same document: my first sweep compared against "the string before
THIS field's own key" and reported 65.90% -- that is not what get_raw does, so the
figure is not the legacy reader's error rate.
2026-08-27 04:52:05 +00:00
Sylpheed RE agent
a8e387b564 backlog: mark the freeze entry's "first step" done, and rule out --log_mask=0
Two updates to the mission-freeze entry, both measured.

DONE: the entry's "first step, revised" was "make pilot.py shoot, then re-run
ob_flag.py ... the actual obstacle is that nothing the pilot does moves the
counter".  pilot.py now has SYLPH_WEAKEST=1 (target score scaled by remaining
hull) and the next run moved REMAINING OB 008 -> 007 concurrent with the live e010
floor dropping 16 -> 15.  The counter is fully solved; the freeze work now needs
only a frozen sample for the v2 wait probe.

DEAD END, with numbers: --log_mask=0 does not surface kernel call traces.  A full
Stage 02 run produced 199 MB at ~33 MB/min, and a 300k-line tail is 254127 A>
(Apu/XMA), 42444 d>, 2897 G>, 532 w>, with ZERO k> and only 14 K> lines per 58k of
boot.  XamShowSigninUI / KeWaitForSingleObject / NtWaitForSingleObject each appear
exactly once in the whole log -- an export listing, not call traces.  That
independently confirms the entry's own cost note: those calls are kHighFrequency
and silent without --log_high_frequency_kernel_calls=true.

The run also did not freeze (healthy TIME 00:24.28 -> 03:33.28), making it the
fourth consecutive non-freezing run.
2026-08-27 04:43:18 +00:00
Sylpheed RE agent
d1640548f9 re: the mission scripts were already found -- correct a stale 🔴 that cost three iterations
mission-phase-advance.md still carried "🔴 Not settled: where the script bytecode
lives".  It was settled, in a sibling file I had never opened:
structures/mission-script-ssb.md, which says in its own opening that this file
"recorded the bytecode as not on the disc under any obvious name.  It is on the
disc."

Verified independently before correcting: name_hash resolves Stage\script.tbl (the
manifest, 838 B compressed) and 28 Stage\StageNN.ssb records -- S01-S16, S18-S29 --
all in dat/GP_MAIN_GAME_S.pak, with Stage\Stage17.ssb absent, matching the loader
guard sub_8225EC78 (n == 16 || n > 32).

Recorded WHY the last three iterations' searches could not have worked, since that
is the reusable part: MISSION1..33 and MISSION_*_PRT are manifest FIELD KEYS, not
record names.  The records are named Stage\StageNN.ssb and the manifest maps
between them, so probing the record namespace with field-key names cannot hit --
2148 hashes over 41 paks and 26443 records returned zero for that reason alone.

Two side findings survive: the .embsec_ sections hold PPC code, not bytecode; and
the archive lookup keys by tag_hash (0x00FFFFDF) while the pak record index uses
name_hash -- not interchangeable.
2026-08-27 04:22:25 +00:00
Sylpheed RE agent
6b50fc0b84 re: the script lookup is a tag_hash directory search; pak route dead under both hashes
0x82448AA0 and 0x82448C50 are not strcmp.  Both pass the name to 0x82447DF0 and
use the result as a key: the first binary-searches a table of 16-byte records
(x16 for the end, /16 for the count, >>1 for the midpoint), the second packs the
hash into a three-word key and calls 0x8244E338.

Decoded 0x82447DF0 from the disassembly as ((sum of extsb bytes) & 0xFF) << 24 |
(rolling mod 0x00FFFFDF) -- i.e. tag_hash.  tools/re-capture/unitgroup.py::tag_hash
already documents itself as "a transcription of sub_82447DF0", so this was in the
corpus; the useful part is that it identifies which hash the ARCHIVE uses.

That exposed a defect in my previous sweep: it probed name_hash (0x00FFF9D7,
lowercased) only, while the archive keys by tag_hash (0x00FFFFDF, case-sensitive).
Re-ran with BOTH: 1380 names -> 2148 distinct hashes, 41 paks, 26443 records.
STILL ZERO.  The pak-record hypothesis is now refuted with the right hash rather
than merely unsupported.

Left open: the XEX's compressed/encrypted region (default.xex never decrypted
here), plus two untried static threads -- the second pair of SCRIPTS/GP_SCRIPT
references at 0x82262374 / 0x822622bc, in a different function, and tracing
[r31+80] back to whoever opened the archive being name-tested.
2026-08-27 04:15:33 +00:00
Sylpheed RE agent
13117df0bd re: close the pak route properly (41 paks) and read the script loader's own code
My earlier negative was under-scoped: it probed 35 paks, missing hidden/resource3d/
and dat/movie/.  Redone across every *.pak on the disc -- 41 archives, 26443
indexed records, 768 distinct name hashes -- still ZERO hits.  The scripts are not
a pak record under any of those names.

Located the three loader strings by VA (the .pe is a flat VA dump, VA = 0x82000000
+ offset) and pulled their xrefs from sylpheed.db:

  SCRIPTS   0x820a823c  <- 0x8225f1b8, 0x82262374
  GP_SCRIPT 0x820a8244  <- 0x8225f168, 0x822622bc
  MISSION1  0x820a8264  <- 0x8225eed8

Disassembling sub_8225EE20 shows both archive-name references call THE SAME
routine 0x82448AA0 with (object, string) and test the result with cmpi -- so the
loader COMPARES a name against an already-open archive rather than building a path
like dat\GP_SCRIPT.pak.  That fits GP_SCRIPT being a name an archive reports,
which is why no such file exists to find.

Next concrete steps recorded: identify 0x82448AA0 (strcmp/strstr/hash-compare) and
its neighbours 0x82448C50 / 0x8216F218, and trace [r31+80] -- the object being
name-tested -- back to whoever opened it.  That names the container.
2026-08-27 04:09:18 +00:00
Sylpheed RE agent
b425ddc01e re: refute the .embsec_ script candidate; narrow the pak route with 616 hash probes
Both candidates this file named for the missing phase-script bytecode were tested
statically, from the flat-VA .pe and the extracted paks.

REFUTED: the seven .embsec_ sections hold PPC CODE.  Parsed from the section table
(offset 592, 40 bytes apart) they total 129472 bytes, matching this file's own
"~130 KB" estimate, so they are the right sections -- but six of seven begin
7d8802a6 (mflr r12) and all carry the standard prologue (stwu r1,-N(r1), std
r30,-16(r1), bl).  That is not bytecode for a 147-builtin VM.

Also recorded: ".embsec_P" from `strings` is a false lead -- the name field is
exactly ".embsec_" and the P is byte 0x50 of the following VirtualSize (0x1350 =
4944, the fifth section's size).

CORRECTION: this file says grepping for MISSION_START_PRT "returns nothing".  That
grep was over the DISC EXTRACTION; all five MISSION_*_PRT names are present in the
executable image, in an .rdata table reading "SCRIPTS" "GP_SCRIPT" "script load
cancel\n" "MISSION1".."MISSION33".

NARROWED: the paks are name-hash addressed, so names can be probed rather than
eyeballed.  616 distinct hashes -- MISSION1..33 and the five MISSION_*_PRT under
prefixes SCRIPTS\, GP_SCRIPT\, scripts\, script\, SCRIPT\ and none, with suffixes
.prt/.PRT/.scr/.bin and none -- across all 35 paks: ZERO hits.

Left open: the XEX's compressed/encrypted region, or a name outside those guesses.
GP_SCRIPT is the strongest remaining thread -- referenced by code, absent from disc.
2026-08-27 04:01:34 +00:00
Sylpheed RE agent
2c44692ec6 re: THE DECREMENT IS MEASURED -- REMAINING OB is fully solved
SYLPH_WEAKEST=1 (00052b0) worked on its first live run: concentrating fire on the
already-damaged attacker produced the kill that four previous runs could not.

All gates enforced first -- FLIGHT confirmed, stage asserted, mission clock shown
advancing -- then:

  live e010 floor: 16 for samples 0-11, then 15 for samples 12-39 (one death)
  counter: 004 -> 008 at 02:06.33 (t=120 arrival, +4)
           008 -> 007 at 03:10.28  <-- DECREMENT, bracket (171.2s, 190.3s]
           007 -> 011 at 03:48.40 (t=210 arrival, +4 FROM 7, not from 8)
           011 held to 06:50.30 -- ceiling 11, where every prior run reached 12

Conclusive because: it fell by exactly 1 and not by 4, so the counter tracks CRAFT
not squadrons; exactly one attacker died and exactly one decrement occurred, in the
same window; and the CEILING moved with it -- a counter that merely read arrivals
would still have shown 012.  Control held: turrets fell 109 -> 92, seventeen
deaths, none of which moved it.

Settled reading: REMAINING OB is the number of objective-marked craft still alive,
the marked craft being exactly the members of the phase's A-route squadrons.  It
rises by a squadron's membership on that squadron's route arrival time and falls by
one per marked craft destroyed.  Every number was predicted from Route_S02.tbl and
UnitGroup_S02.tbl before it was measured.
2026-08-27 03:53:08 +00:00
Sylpheed RE agent
00052b0117 tools: add SYLPH_WEAKEST target scoring; record that assert_stage != in flight
pilot.py gains SYLPH_WEAKEST=1, which scales a target's score by its remaining
hull (pos+0x154) so the pilot finishes what is already hurt instead of
re-engaging whatever is nearest.  Motivated by b69cc23: over ~8 minutes the pilot
damaged 14 of 16 e010 attackers (hulls 360..500) and killed none, because 500 HP
spread across a squadron kills nobody.

STATUS: the flag is implemented and its targeting works -- 3105 of 3105 target
samples selected e010 -- but it is UNVERIFIED in combat, because the run it was
written for was lost.

That loss is the second half of this commit.  The run printed "READY ROOM / >>>
HUD / Stage 02 OK" and I began the experiment; there was no FLIGHT: line, because
the flight check failed three times and fell through silently while the next line
read like success.  The game was frozen on a near-black screen (screen_id `other`,
mean 10.8/2.8/2.1, frozen.py max_pixel_delta=0) and the pilot's every sample from
t=0.0 to t=406.1 is byte-identical with speed 0.

assert_stage.py could not have caught it: it reads the DEFINITION table, which is
populated when the STAGE loads, independently of whether the mission is running.
Recorded in nav-guards.md with the rule -- enforce the flight gate with a non-zero
exit, and run the three-crop TIME liveness check before any experiment.
2026-08-27 03:27:35 +00:00
Sylpheed RE agent
b69cc2325a re: quantify why the OB kill test keeps failing -- attackers are damaged, not missed
Fourth Stage 02 run: 520s of pilot with SYLPH_PREFER=e010 (198 fire=1), per-class
counts every 10s, OB+TIME throughout.

ARRIVAL TIMING n=4: 008 first at 02:05.00 (step bracket contains 120), 012 first
at 03:45.47 (bracket contains 210), then 012 held 3.5 more minutes with no fall
and no 016.

THE BLOCKER IS MEASURED, and it is not aim.  Reading hull at pos+0x154 for every
live e010 at the end: 14 of 16 are damaged, hulls 360..500, lowest 360/500 = 28%
gone.  The shots land; they just do not finish.  A kill needs roughly 3-4x longer
on one target than ~8 minutes of combat produced.

The contrast with the control shows the mechanism is hit points: e007 turrets
(HP 100) lost ~9 dead, 106 -> 97 live, while e010 attackers (HP 500) lost none and
the floor never moved off 16.

Decrement therefore still unproven after four runs, and "fly longer" is not a fix
since ~half of runs end early.  Options recorded for a redesign: point the pilot
at the WEAKEST attacker (hull is readable per entity; one is at 360/500), use
missiles (never deliberately fired), or wait on a scripted kill.
2026-08-27 03:07:33 +00:00
Sylpheed RE agent
1fc6338122 re: turret control CONFIRMED, arrival timing n=3, decrement still unproven
Third Stage 02 run, pilot with SYLPH_PREFER=e010 (312 fire=1 samples), per-class
live counts logged every 11s beside the HUD (new tools/re-capture/class_count.py).

CONTROL CONFIRMED: the live turret population fell 108 -> 101 -- seven e007 deaths
-- and REMAINING OB never decremented, only rose.  Previously this was inferred
from a run whose kill log happened to be turrets; it is now measured with the
classes counted directly.

ARRIVAL TIMING n=3: 004 -> 008 in (108.7s, 125.5s] and 008 -> 012 in (204.2s,
221.7s], both brackets containing the predicted 120 and 210.

The live e010 count sat at exactly 16 in 20 of 26 samples -- precisely phase 1's
e010 roster (ADT102/ADT107/ADT113/ADS151, each n=4) -- an independent runtime
corroboration of the static roster.

DECREMENT STILL UNPROVEN: the e010 floor never fell, so no marked attacker died
and the counter had no chance to move.  Three runs have failed to kill one.  The
blocker is combat effectiveness, not instrumentation.

Artifact recorded: six of 26 class samples read 17-28.  Spikes are always upward
and transient -- the tool dedups on a position triple read just after the pattern
scan, so an entity written between the two reads is counted twice.
2026-08-27 02:42:46 +00:00
Sylpheed RE agent
b0fc1e6e7b re: kill run -- arrival timing reproduced (n=2), decrement inconclusive, "live attackers" refuted
pilot.py with SYLPH_PREFER=e010, 300s, mission clock sampled throughout.

RISES REPRODUCED: 004 -> 008 in (114.8s, 134.7s] and 008 -> 012 in (196.4s,
213.3s], both brackets containing the predicted 120 and 210.  Second independent
run, so the arrival half is now n=2.

DECREMENT INCONCLUSIVE.  The pilot fought properly -- 424 fire=1 samples, 865 with
the target inside 1500 units, closest approach 79, target e010 throughout, hull
untouched -- and 13 ADAN died (129 -> 116).  The counter held 012 for 98s.  That is
NOT evidence against the decrement: the 13 dead were not identified by class, this
pilot's kills historically skew to turrets, and the live e010 count ROSE over the
run so attacker deaths cannot be inferred from it.  Recording it as inconclusive
rather than as a negative, which is the error this corpus has already logged twice.

POSITIVE RESULT: at TIME 05:56.85, both arms sampled together, REMAINING OB = 012
while live UN_e010_ADAN_Attacker_S = 23.  So the counter is NOT the live attacker
head-count; it tracks a subset, and 12 is exactly the three A-route squadrons'
membership (3 x n=4).  Phase 1 fields only 16 e010, so 23 live means later-phase or
F-route squadrons joined without touching the counter.

Anomaly recorded, not explained: the enumeration reports 2 _Player entities at two
distinct positions.
2026-08-27 02:18:29 +00:00
Sylpheed RE agent
92b9ad6b01 re: CONFIRMED -- the A-route arrival times predict the OB counter's steps exactly
Predicted from the disc alone, before the run: ADT102/ADT107/ADT113, each n=4,
arrive at t=0/120/210, so the counter reads 004, steps to 008 at t=120 and 012 at
t=210, and goes no higher in phase 1.

Measured with the mission clock sampled beside the counter throughout:

  004 held over six samples to 01:45.44
  008 first seen at 02:00.87  -> step bracketed in (105.4s, 120.9s]
  008 held over five samples to 03:17.55
  012 first seen at 03:30.66  -> step bracketed in (197.6s, 210.7s]
  012 held four more samples to 04:36.95, no 016

Both steps land on the predicted second, and the ceiling holds.  Three independent
features -- starting value, both step TIMES, and the ceiling -- came from
Route_S02.tbl and UnitGroup_S02.tbl with nothing fitted to runtime data.  So the
counter's rises ARE the A-route attacker squadrons arriving.

Status upgraded to CONFIRMED for the rises.  The DECREMENT half stays 🟡 and is
now the only open part: this run killed nothing, and the single observed 12 -> 11
remains one sample.
2026-08-27 01:55:07 +00:00
Sylpheed RE agent
d8baf92fb5 re: mission TIME is the liveness oracle, and it explains the earlier "0 movers"
The OB kill test did not run -- the guest froze about a minute into the mission --
but sampling the HUD's own mission clock beside the counter caught it instantly.
Twelve samples over ~4 minutes of wall time: the first reads TIME 00:43.24, every
one after reads 01:02.23 unchanged, with REMAINING OB 004 throughout.  frozen.py
agrees (max_pixel_delta=0), movers in the entity window are 0, and screen_id still
says `flight` with the process alive.

Worth a file because the mission clock defeats both traps this corpus has paid
for: it is the simulation's own counter, so neither a frozen world nor a finished
one advances it, where screen_id and pixel-churn are fooled by a GAME OVER screen
that animates.  Rule recorded: "X never changed" is only evidence if TIME changed
across the same window.

It also closes an open  from earlier this session -- entity-position-anchor-
refuted.md's "0 of 64 regions changed across 357 MB, cause unknown, not
reproduced".  Same signature, now reproduced: it was this freeze.

Still unrun for the fourth iteration: the kill test itself.  Not conceptually
blocked -- blocked by attrition, with this run dying at t~62s, before even the
t=120 arrival the trajectory prediction needs.
2026-08-27 01:34:09 +00:00
Sylpheed RE agent
8ebe4edff2 re: test what the 'A' route kind means across all 28 stages -- enriched, not "attackers"
Swept every Route_*_p<n><kind> in every stage table, resolving each squadron to
its unit via that stage's own UnitGroup, with F/S/M routes as the control:

  A/B routes   : 23 of 67 resolved are attacker/bomber = 34.3%
  F/S/M control: 27 of 928                             =  2.9%

A ~12x enrichment, so the signal is real and not a one-stage accident.  But "A
means the marked attackers" remains too strong: of the 67 resolved A/B routes, 35
carry UN_mn040_Asteroid_Big and 9 carry UN_n001_TTRL_Box (TTRL = tutorial).  A
reading covering all four is "things the mission wants shot" -- attackers in a
combat stage, asteroids in an asteroid stage, boxes in a tutorial -- which keeps A
as an objective marker rather than a unit-class marker, but that is a reading of
four unit types, not a measurement.

Records a near-miss: a truncated listing showed the first ~26 rows, dominated by
early stages, in which every resolved row was an attacker.  I nearly wrote "23/23,
unanimous".  The full set is 23/67 -- the tail of a sorted listing is not the
distribution.

The Stage 02 derivation is unaffected: its three A routes are still exactly the
three n=4 attacker squadrons.
2026-08-27 01:15:49 +00:00
Sylpheed RE agent
7e0c391b5c re: the 'A' route kind names the objective squadrons and predicts the whole OB trajectory
Pure static, from Route_S02.tbl.  Only THREE routes in the entire stage are kind
'A', all in phase 1 -- ADT102, ADT107, ADT113 -- and every one is a
UN_e010_ADAN_Attacker_S squadron of n=4.  The fourth phase-1 e010 squadron,
ADS151, is kind 'F'.

Their first-keyframe times are t=0, t=120 and t=210, so counting only the 'A'
squadrons predicts the counter exactly: 004 at entry, 008 at t=120, 012 at t=210,
and never above 012 in phase 1.  That is what every run has shown -- starting
value, step size and ceiling all fall out of the disc with nothing fitted.

It also explains the two awkward observations: the fifteen-minute hold at 012 (all
'A' squadrons arrived, and those kills were turrets), and why 016 never appeared.
The earlier prediction of a 016 cap is WITHDRAWN: ADS151 is an F route and is not
counted, so the ceiling is 12.

Kept 🟡: "A = marked attack objective" is inference from three routes in one
stage, consistent with the mission dialogue and with A appearing only in early
phases, but the letter's meaning is not proven and the kill test is still unrun.
2026-08-27 01:08:21 +00:00
Sylpheed RE agent
f5b030e3db re: phase-1's four attacker squadrons are measured -- all n=4, 16 craft
Replaces the 16/4 MEAN with a per-group measurement, closing the caveat that a
5/5/3/3 split would have made the counter's step-of-four a coincidence.

Route_S02.tbl names routes Route_<squadron>_p<phase><kind>, so the squadron->phase
link is readable off the disc.  Stage 02 has 37 phase-1 squadrons; intersecting
with the e010 roster gives exactly FOUR -- ADS151, ADT102, ADT107, ADT113 -- and
every one has n=4, summing to 16 craft.

The five e010 groups NOT in phase 1 (ADN204, ADN206, ADN208, ADN209, ADS251)
include both n=6 groups and the n=9 group.  That is why the stage-wide mean
misled: the odd-sized squadrons belong to later phases.  Restricted to phase 1 the
size is uniform.

Pure static, no emulator.  The hypothesis itself stays 🟡 -- the runtime
kill-one-e010 test is still the thing that would confirm or kill it.
2026-08-27 01:02:06 +00:00
Sylpheed RE agent
74314b457d re: test the OB hypothesis's weak point statically -- it holds, but a premise was wrong
Decoded stage\UnitGroup_S02.tbl with the existing unitgroup.py (pure static, no
emulator; 111 squadrons, roster self-check 111/111).

The "four per squadron" premise was 16/4, a mean, and I flagged that 5/5/3/3 would
make the step-of-four a coincidence.  Measured: Stage 02 has NINE e010_Attacker_S
squadrons totalling 45 members, sized 4,4,4,6,4,6,4,9,4 -- not uniform across the
stage, but SIX of the nine hold exactly 4, and phase 1's share is independently
recorded as 4 groups / 16 craft = 4 x 4.  So the phase-1 squadrons are the n=4
ones and the step-of-four is a real size, not an averaging artifact.

But a premise of the note is refuted: ADN110/111/112, which I cited as the polled
squadrons arriving at t=170, are UN_e007_ADAN_Turret with n=9 -- TURRET squadrons,
not attackers.  That arrival evidence is about turrets and is withdrawn from the
argument.

Records the tension it exposes: mission-wave-arrivals.md calls ADN110/111/112 the
squadrons the phase-1 clear condition polls, while mission-objectives-text.md has
the phase-1 objective as the marked attackers.  Unresolved.
2026-08-27 00:55:27 +00:00
Sylpheed RE agent
f041de8636 re: hypothesis for what REMAINING OB counts -- marked attackers, 4 per squadron
Assembled from existing captures, not newly measured, and marked 🟡 accordingly.

mission-objectives-text.md gives phase 1's roster as 4 e010_Attacker_S GROUPS
totalling 16 CRAFT -- four per squadron -- and the counter moves in steps of
exactly four (004 -> 008 -> 012).  The phase-1 objective is stated in the mission
dialogue as "the attackers with the orange markers", i.e. the e010s, not the
turrets that outnumber them 7:1.  mission-wave-arrivals.md pins the arrival of the
three polled squadrons at t=170 SECONDS, which is when the counter is seen to
climb.

That accounts for every number taken so far, including the two that looked
contradictory: the counter held at 012 while the hostile population fell by a
third because those kills were almost all turrets, and it decremented 12 -> 11 in
the one run whose pilot actually killed attackers.

Recorded with its own refutation tests (kill one e010 -> must fall by exactly 1;
kill a turret -> must not move; phase 1 should cap at 016) and with the weak point
stated: "four per squadron" is 16/4, a MEAN, and if the squadrons are 5/5/3/3 the
step-of-four pattern is coincidence.
2026-08-27 00:48:52 +00:00
Sylpheed RE agent
fbea9f9148 re: refute the OB entity-flag on the FULL enumeration -- the stationary-entity hole is closed
Every earlier refutation in this file carried the caveat that entities2.typed
types entities by their position CHANGING, so a stationary objective is invisible
to it.  This session's definition-pointer enumeration does not have that limit, so
the sweep was re-run against it (ob_flag_all.py, guarded route, stage asserted,
HUD cropped beside each sample):

  A: HUD 004, 147 entities -> 152 candidates
  B: HUD 012, 133 entities ->  15 candidates
  intersection: 1

The lone survivor pos+0x0250 = 239d6732 is the same offset AND identical value
this file already characterised as a per-group word.  Membership test: all 12
holders are UN_e010_ADAN_Attacker_S, 12 of 16 live attackers.  It is a squad
parameter, and it survived only because that population equalled the counter at
both samples.

Also reconfirms "not a class head-count" on 147 entities including capital ships.

Trap recorded: the first sweep reported 298 entities and a class with head-count
exactly 4 -- a perfect-looking hit that was pure artifact.  Deduping by ADDRESS
leaves the measured exact 2x duplication (pairs 0x1000 apart, byte-identical
positions) intact and doubles every population.  Dedup on the position VALUE.
2026-08-27 00:40:53 +00:00
Sylpheed RE agent
e3098f7316 re: LOCATE REMAINING OB at 0xbdb69668, and refine the ...9668 pattern
Ran this file's own transition filter on a guarded Stage 02 run (stage asserted),
reading the HUD from a crop taken at the same instant as each memory sample:
scan at HUD 004 -> 41537 candidates; filter at HUD 008 -> 8; verify across the
008 -> 012 transition, which was NOT selected on -> exactly ONE survivor.

That survivor, 0xbdb69668, tracked 4 -> 8 -> 12 against the HUD's 004 -> 008 ->
012.  The other seven collapsed into noise at the first unselected transition,
which is precisely what that rule exists to catch.

The file's "try 0xbdb59668 first, re-scan when it reads 0" rule worked verbatim:
it read a hard 0 here, and the re-scan cost about the predicted five minutes.

The ...9668 page-offset pattern is REFINED, not reinstated: the three located
addresses (0xbdb49668, 0xbdb59668, 0xbdb69668) are three ADJACENT 64 KB pages at
one offset, and in this run exactly one of 8192 probed pages held 12 -- the
counter -- making it a one-step lookup.  But the 2026-08-26 refutation stands as
measured (zero ...9668 VAs held the HUD value in that run), so this is a fast
heuristic to be HUD-checked, not a law.
2026-08-27 00:23:22 +00:00
Sylpheed RE agent
7e89249f46 re: refute the "counter sits at page offset 0x9668" prediction
The file proposed that the counter lives at a fixed offset inside an allocation
whose base moves by whole 64 KB pages, and stated the test itself: "a third scan
should again land on ...9668".  Ran it on a fresh guarded Stage 02 run.

Probing all 8192 pages of the form 0x????9668 across 0xa0000000-0xbfffffff: with
the HUD at 004, exactly two VAs held 4 (0xbc3f9668, 0xbe3f9668); with the HUD at
012, ZERO held 12.  Both candidates also failed the file's own transition rule --
over 252 s 0xbc3f9668 held a flat 4 and 0xbe3f9668 flickered 4/0 while the HUD
went 004 -> 012.

Both arms were sampled at the same instant (cropped HUD digits beside each memory
read), after a stale-screenshot comparison earlier in this session produced a
spurious 13-vs-004 mismatch.

Scope kept narrow: this refutes the page-offset prediction, not the confirmed
finding that 0xbdb59668 carries the counter in some runs.  The counter's address
in THIS run remains unknown -- no transition filter was run.
2026-08-26 23:57:25 +00:00
Sylpheed RE agent
6e081c5437 re: correct my own OB write-up -- I refuted a stale status line, not an open question
The section I added last iteration claimed to be testing an untested 🟡 and
concluded "the address is run-dependent, always re-derive".  Both halves were
wrong, and the evidence was already further down the SAME file:

 - cross-run stability was not untested -- the body records 0xbdb59668 carrying
   the counter in 3 of 5 measured runs and reading a hard 0 in the other two.  The
   stale claim was in the status line at the top, which I took at face value.
 - "always re-derive" is worse advice than the rule the file already gives: try
   0xbdb59668 first, check it against the HUD, re-scan only when it reads 0.

What my run genuinely adds is a SIXTH data point with a new failure mode: the
address read neither the counter nor 0 but 95748078, constant over four samples.
Status line rewritten to match the body (🟡 recurs but not universal), so the
document no longer contradicts itself.
2026-08-26 23:57:01 +00:00
Sylpheed RE agent
dd052287e9 re: REMAINING OB's address is run-dependent -- cross-run stability refuted
The doc carried a 🟡 saying cross-run stability was untested.  Tested now on a
fresh guarded Stage 02 run (stage asserted): the HUD reads "Remaining OB : 004"
while RAM at the documented 0xbdb59668 reads 95748078, constant over four samples
12s apart.  Not 4, not near 4, not moving.  So the address belongs to that run's
heap, as the corpus's own heap-reallocation warning predicts.

No constant-shift shortcut either: a BE u32 equal to 4 occurs 1654 times within
+-1 MB of the old address and 11202 times within +-16 MB, far too many to isolate
without the transition filter.  The durable result is the METHOD (ob_scan.py:
scan at one value, filter against live memory at a DIFFERENT value), not the
number.

Also fixes a contradiction in INDEX.md, which said in one row that the address is
"still " while another row linked the doc that had already CONFIRMED it.
2026-08-26 23:34:36 +00:00
Sylpheed RE agent
d2c81249cd re: WITHDRAW "the absence is structural" -- capital ships do have +0x130 blocks
My enumeration sourced entities from moving(), which requires a position to CHANGE
between two samples.  A capital ship holding station changes by exactly zero, so it
can never appear -- which is also why lo=0 and the whole-map scan did not help, and
why parked ArrowHead wingmen were missing.  The filter was the problem, not the
range, and not the game.

navigator.py already does it right and says so in its header: search for the
DEFINITION POINTER and take position = hit - 0x130, "which finds every entity
whether it is moving or not".  The existing mission_state.py scan on Stage 02
returns 149 entities with hull(+0x154) == definition HP for 129 of them:

  UN_f101_TCAF_Acropolis  (+5000, +0, +0)  hull=25000.0  HP=25000.0  frac=1.000
  UN_e105_ADAN_Cruiser                     hull=30000.0  HP=30000.0
  UN_e106_ADAN_Destroyer                   hull= 9150.0  HP=10000.0  (under fire)

So "an autopilot that must protect the Acropolis cannot locate it through the
+0x130 method at all" is withdrawn outright -- it can, and INDEX.md had already
recorded the Acropolis falling 25000 -> 23038 over 240s.

What survives: the two enumerations are still different structures in different
regions (0/116 vtable instances lie in the +0x130 region).  What does not survive
is any claim that the +0x130 model covers only four types.
2026-08-26 23:20:48 +00:00
Sylpheed RE agent
549134c415 re: the guarded route now verifies end-to-end, and the menu order is confirmed
With the sliver stopped at both ends (290cbe3 producer, 30e53f5 consumer) a full
guarded run reached Stage 02 and asserted it: "guard: menu confirmed (1279x675)"
then "OK: UN_f101_TCAF_Acropolis is in the 14 definitions", exit 0.

Read the menu the guard confirmed rather than continuing to assume it:
NEW GAME / LOAD GAME / TUTORIAL / OPTIONS / EXTRAS with the cursor on NEW GAME.
So dpad-down x1 = LOAD GAME is correct and dpad-down x2 = TUTORIAL, one press
further -- exactly consistent with the observed mis-selection when the guard was
being fed a sliver.  Capture committed as captures/main-menu-items.png.

Stated plainly: n=1 for "reproducible", and why the game window leaves the window
tree during a load was guarded against, not investigated.
2026-08-26 23:08:11 +00:00
Sylpheed RE agent
290cbe3497 tools: never crop to a window too narrow to be a game frame
The app owns TWO windows of class xenia_canary -- measured in the tree right now
as 10x10+10+10 and 1280x745+1+20.  Largest-by-area picks the game window while
both are present, but during a load or mode switch the game window is briefly
absent from the tree, the 10x10 helper wins by default, and the crop produces a
10x710 SLIVER.  That is exactly the grab screen_id classified as `menu` on
2026-08-26, which let a nav guard pass on garbage and loaded the wrong stage.

Ignore candidates narrower than 640 so nothing is selected in that case and the
existing fall-through hands back the raw root grab -- itself a valid full frame.

Verified on the selection logic directly: with both windows listed the pick is
unchanged (1280x745+1+45); with only the helper listed the old logic returned
10x10+10+10 and the new one selects nothing.  Pairs with 30e53f5, which rejects
such a frame at the consumer.
2026-08-26 23:02:08 +00:00
Sylpheed RE agent
540eefeae8 tools: add assert_stage.py and a menu guard; record that the guard was not enough
assert_stage.py checks the DEFINITION table against an expected stage marker and
earned its keep immediately: its first live run reported MISMATCH -- the capture
had a live flight HUD and would have been filed as Stage 02, but was the S01
tutorial.  That is exactly the failure that silently invalidated an earlier
cross-run comparison.

require_menu (launch_mission.sh) refuses to press until screen_id reads `menu`.
It is NOT sufficient, and this refutes my previous explanation: the run DID
confirm the menu and still loaded the tutorial.  The real cause was that the
guard's own capture was a 10x710 sliver which classified as `menu` -- fixed
separately in 30e53f5.

Left open: whether the menu guard suffices now that slivers are rejected (not
re-run), and why the capture was a sliver at all when the other shots in the same
run were 1279x675.
2026-08-26 22:56:22 +00:00
Sylpheed RE agent
30e53f599c tools: reject degenerate captures in screen_id instead of classifying them
Every statistic in screen_id is an AREA FRACTION, so a capture that is not a game
frame still produces clean numbers.  Measured 2026-08-26: a guard shot came back
10x710 -- a sliver -- and classified as `menu` with green=0.0000, white=0.0157.
The guard passed, the fixed key sequence went out anyway, and the run loaded a
TUTORIAL instead of the save's Stage 02.

This is the second time this failure has been paid for.  bin/screenshot's own
header records the first (2026-08-18): a second window of class "xenia_canary"
meant grabs came back as slivers and "a whole session's screen ids were noise".
That fix hardened the CAPTURE side only, so the same failure still reached the
oracles by any other path.  Reject it at the point the answer is consumed too:
features() now returns None below 640x360 and classify() reports `none`.

Verified: the 10x710 sliver -> `none`; readyroom, flight and the briefing capture
all still classify as before.
2026-08-26 22:55:58 +00:00
Sylpheed RE agent
67001f04fb re: pin the GHAD field base -- it is tag+4, not the tag+8 I guessed
Closes the caveat I flagged in c9dc8dd.  savegame.rs documents the chunk stream
(read off the title's own serializer at 0x822C00E8) as GDAA / phase string /
'GHAD' + 122 bytes / SHAB table, and FieldSpec.offset is "offset within the
122-byte GHAD block" -- so the base is the byte after the tag, tag+4.

Reading there yields three fields, two corroborated independently elsewhere in the
corpus: Stage +52 = 2, Points +24 = 4101 (weapon-datasheet-runtime.md's "4101 P"),
FlightTime +4 = 324773 (the 05:24.77 Stage-01 best time).  tag+8 reproduced only
the stage.  The payload's phase string reads GP_BUNK.

Left open: the .header mirror does not reproduce -- savegame.rs lists Stage at
header offset 0x14 but a BE u32 there reads 2097200 (00 20 00 30), which looks
like UTF-16 text.  Not chased; the payload reading does not depend on it.
2026-08-26 22:39:02 +00:00
Sylpheed RE agent
c9dc8dd4db re: the stage drift is navigation, not the save -- slot 01 is deterministically Stage 02
Refutes my own claim from the previous commit that slot 01 is an auto-save whose
restored mission drifts.  savedata has not been written since 2026-08-23; every run
today left it untouched (only the .gpd profile files moved), and there is exactly
one save, so a wandering save-list cursor cannot explain it either.

Decoded the save statically (GDHA + zlib at 0x92, 545 bytes): only SHAB record 0
carries clear data, records 1-15 are zeroed.  Three independent readings agree it
is Stage 02 -- the 324773 ms clear time is the 05:24.77 that SESSION-2026-08-11
documents for Stage 01, the 4101 points match weapon-datasheet-runtime.md's
"Stage 02, At Standby, 5% clear, 4101 P", and the GHAD block at tag+8 gives
+52 = 2.  Stage 02 is the Acropolis escort mission, which is exactly the roster the
last run produced.

So the odd run out was the S01 "Glasner Training Area" one -- a TUTORIAL, reached
because the nav issued dpad-down + A from a state never confirmed to be the main
menu.  Fix is two guards: verify screen_id reads `menu` immediately before the
dpad, and assert Acropolis is in the definition table before comparing rosters.

Caveats kept: the GHAD field base was not isolated by the search (7 candidates, 3
giving stage 2 -- tag+8 chosen because it agrees with the other two readings), and
the tutorial mis-selection was not demonstrated by re-running the emulator.
2026-08-26 22:32:30 +00:00
Sylpheed RE agent
a21927ab01 re: the two enumerations are different structures; only the vtable scan is complete
Same moment, same stage: the INST_VTABLE scan sees 116 objects across 14 types;
the moving+0x130 method sees 30 across 4.  Every capital ship, station, missile and
the objective-critical Acropolis reads ZERO in the +0x130 method.

The absence is structural, not a filter artifact -- both obvious explanations were
tested and failed.  Dropping the speed floor to 0 raised the count 30 -> 59 and
recovered the _Player but still only 4 types; scanning the WHOLE map with no floor
gives 181824 movers and still 4 types.  And 0/116 vtable instances lie inside the
window where the +0x130 blocks are found (instances 0xbc372cc0-0xbc9bc720, window
0xbd000000-0xbe000000), independently confirming these are separate allocations.

Consequence: an autopilot that must protect the Acropolis cannot find it via the
+0x130 method at all.

Also found: LOAD GAME -> slot 01 no longer restores the S01 training area but a
Stage-02-style escort mission (Acropolis, SchlosBase, cruisers, frigates).  Slot 01
is the AUTO-SAVE, so the restored mission moves as the save is written -- which
invalidates the earlier "101 vs 42" comparison outright, since those came from
different stages.

Corrects the previous note: entities2 prints its count AFTER dedup, so the 101 was
already deduplicated; the gap was the stage change, not duplication.
2026-08-26 22:24:35 +00:00
Sylpheed RE agent
dddff2007a re: refute the projectile hypothesis; the count inflation is 2x duplication + arrivals
Three checks kill "the fast turrets are shots":
 - persistence: 42/42 survived 15s, including 22/22 of the FAST (>300/s) ones;
 - hull at the documented pos+0x154 is a clean per-type constant -- 1000.0 for
   DeltaSaber_T, 100.0 for e007_Turret, with one turret at 79.0 (damaged);
 - the speed was never suspicious: isl-builtins.md already records e007's
   MaximumVelocity 500 / CruisingVelocity 280 and notes the data models turrets as
   mobile.  My premise "a turret is a fixed emplacement" was an inference from the
   NAME that the corpus had already contradicted from the DATA.

The inflated count is two measured effects, neither mis-typing: exact 2x
duplication (64 raw -> 32 deduped, every per-name count halving, commonest offset
gap 0x1000 = one page), and a growing population (42 -> 60 raw in 15s, matching
the documented wave arrivals).  So "101 vs 42" compared different mission times
with at least one count un-deduplicated.

Left open: whether the earlier 101 was raw or deduped (it is odd, so not a pure 2x
artifact); whether the 0x1000 spacing is a real second copy or page aliasing; and
which of the two enumerations is the entity list, since they describe different
sets.
2026-08-26 22:09:07 +00:00
Sylpheed RE agent
cc6d088573 re: the live entity readout works end-to-end; "0 movers" was a state
Fresh run, unmodified tools: entities2.py self 0x130 returns 74-101 typed live
entities with positions and speeds, a player entity, and the orientation matrix at
pos-0x70 with 16-byte stride -- exactly the documented layout.  Artifact committed
as docs/data/live-entities-2026-08-26.txt.

Last iteration's "zero moving triples across 357 MB" does not reproduce: sampled
seven times 4s apart, the whole map gives 47k-123k movers and the committed
ENT_VA window 10k-20k.  So the window is populated and the tooling is sound; what
differed between the sessions is NOT determined and is recorded as open, since the
earlier zero persisted for minutes across several commands.  Practical rule added:
check movers are non-zero before concluding anything from a memory probe.

New open discrepancy: the vtable scan reports 42 instances (20 turrets) while the
+0x130 method reports 101 (82 turrets), most of the excess moving at ~375/s -- and
a turret is a fixed emplacement.  Likely projectiles typed as their shooter, but
explicitly NOT asserted: no test here separates a projectile from a fast craft.
2026-08-26 21:54:23 +00:00
Sylpheed RE agent
f59c4e595c re: refute my own position anchor, withdraw the entities2 "defect", find the trap
Three results, two of them against my own earlier claims:

 - REFUTED: position = instance - 0x12c.  The corpus anchors on position (def ptr
   at +0x130, orientation at -0x70, hull at +0x154) and name_of reads the def ptr
   at instance+4, which predicts -0x12c.  Measured over all 42 named instances:
   almost every read is (0,0,0), three are garbage, and 0/42 move.  The vtable
   object and the position block are different structures.

 - WITHDRAWN: last iteration's claim that entities2.py's ENT_VA window is aimed at
   the definitions rather than the instances.  entities2 does not look for vtable
   objects at all -- it hunts moving position triples and checks +0x130, the
   documented anchor -- and it had worked minutes earlier (6914 movers, +0x130
   voted 92x; then 6634/138x).  I built the defect report on the single sample in
   between that returned zero.  The +0x29d0 position candidate falls with it.

 - FOUND: sampling 64 spots across the 505 extents (357 MB) twice, 2.5s apart,
   with the mission visibly running, only 5 change.  The megabyte holding all 42
   instances is byte-identical over seconds, as is its primary-VA counterpart.
   The mapping IS live (5 regions prove it); what is undetermined is whether the
   entity records are simply static or whether writes land in a different alias.
   Either way: verify a region changes before measuring through it.
2026-08-26 21:36:47 +00:00
Sylpheed RE agent
9770eadad4 re: withdraw "the speed cannot be used", and close the world-unit item
Last iteration I ruled the speed route out because the emulator is not real-time
so the wall-clock denominator is unknown.  Wrong: the game prints its own clock,
and flight-speed-law.md had already used it -- mission TIME across a wall interval
gives 1.26x, turning 443.6 units/wall-second into 352 per game-second against a
HUD 350.  Withdrawn in place, with the reasoning, rather than deleted.

Closes the backlog item: one world unit is one metre, by two independent routes.
2026-08-26 21:19:22 +00:00
Sylpheed RE agent
2d144ae7d0 re: RESTORE "one world unit is one metre" -- the objection conflated two families
The 2026-08-26 downgrade rested on two numbers, and grouping the CollisionSet
names by prefix dissolves both:

 - The 447 km "largest object" is rob_f002_cmesh, one of only FOUR rob_ meshes
   (44k-448k).  That family is not hull size: rob_f001_bdy_cmesh measures 50179
   and f001 is the player's own Delta Saber, whose disc record gives Size_Radius
   10.0.  A 50 km player fighter is impossible under every unit convention, so
   rob_ spans something else entirely and is not evidence about the world unit.
 - The "small" 133 m craft is small: rou_e010_cmesh = 133.2 is the 7th smallest
   of 78 rou_ meshes (median 636.2, max 9534.9).  Against its own family, the
   game calling e010 small AGREES with the metre.

Measured this run: the 500 km box is a cube of exactly 500000.0 units on every
axis (+-250000), and the second box mesh is exactly 100000.0 -- the same ruler.

Independent dynamic confirmation already in the corpus (flight-speed-law.md):
443.6 world units per wall-second, mission clock 1.26x wall, so 352 units per
game-second against a HUD reading of 350.
2026-08-26 21:18:54 +00:00
Sylpheed RE agent
042fb41fe3 backlog: record what the world-unit measurement now actually needs
The HUD is reachable and the control is paired; what blocks the measurement is
narrower and now named -- two tool defects (entities2.py's VA window covering the
definitions rather than the instances, gworld.py's 0x600 instance window) plus the
missing piece itself, a locked target whose HUD range and position can be read in
the same second.  Also records the speed shortcut as refuted so it is not retried.
2026-08-26 21:10:26 +00:00
Sylpheed RE agent
f384fc42ee re: world-unit attempt -- control paired, HUD reached, two tool defects found
Discharges the control owed by entities-live-roster.md: same process, title screen
scans 0/0 against 13/42 in flight.

Reaching a live HUD needed two steps no script had: START skips the post-take-off
cutscene, and a modal "tell you your objective?" dialog DIMS the frame (so the
classifier reads `other` and liveness looks like a stall) until Ⓑ/NO answers it.
After that, green 0.0145 -- inside the documented 1.3-1.5% flight band.

Two tool defects measured, not inferred:
 - entities2.py's ENT_VA_LO/HI (0xBD000000-0xBE000000) misses every live instance
   (they sit at 0xBC384CE0-0xBC9BAC20) and instead covers the DEFINITIONS.  Rescoped
   to the instance region, find_delta's +-0x400 radius yields zero votes.
 - gworld.py's WINDOW=0x600 is too small: no position-like triple moves inside the
   first 0x600 bytes of any of the 42 instances; 0x4000 finds one at +0x29d0.

The unit itself is NOT measured.  That needs a locked target so the HUD prints a
numeric range to an entity whose position can be read at the same moment; this run
never locked one.  The tempting shortcut -- 116.6 units per 0.6s wall-clock against
the HUD's 350 -- is recorded as refuted, because the emulator is not real-time.
2026-08-26 21:10:10 +00:00
Sylpheed RE agent
06f4dec741 backlog: the world-unit blocker was the ROUTE, not entities2.py's VA window
The entry blamed a stale committed VA window for "0 unit definitions".  Measured
2026-08-26: the scan works and returns 13 definitions + 42 named live instances
once the run reaches the mission via LOAD GAME -> slot 01 rather than via MISSION
SELECT.  What still blocks the measurement itself is narrower and now stated: the
flight HUD was not up (green 0.03% vs 1.3-1.5%), so there was no distance readout
to compare positions against.
2026-08-26 20:47:43 +00:00
Sylpheed RE agent
1f17726526 tools: classify the mission briefing instead of filing it as menu
The briefing map is cyan and satisfies every clause of the menu rule (b-r > 30,
r < 45, little white), with no earlier rule claiming it -- so it was labelled
`menu`.  That made wait_screen.sh report NEVER REACHED READY ROOM on a run that
had successfully done LOAD GAME -> slot 01 -> YES and was three screens further
on: a working route scored as a failed one, pointing the next debugging step at
an input path that was fine.

Cyan has b and g nearly equal (b-g ~ 5) where the menu's blue leads its green
(b-g ~ 32), so `r < 20 and g > 30 and b - g < 20` separates them; the r floor
keeps the title screen out.  The file's own docstring already carried the
briefing's mean as an aside -- it just never had a class.

Verified against all eight signatures the file documents (2 menu variants, title,
ready room, flight, 3 briefing measurements): no regressions, and the captured
briefing image now reads `briefing`.
2026-08-26 20:47:15 +00:00
Sylpheed RE agent
779bdde390 re: entity objects ARE spawned -- the vtables were stale-labelled, the ROUTE was wrong
gworld.py's DEF_VTABLE 0x820AF844 and INST_VTABLE 0x820AF030 resolve exactly as
written: 13 unit definitions and 42 live instances, every one name-resolved, with
52 moving triples.  The roster is coherent and stage-specific -- UN_S01_Asteroid_*
in the S01 training area, 2 e106 destroyers against 20 e007 turrets, 6 DeltaSaber_T
and exactly one _Player -- which is far stronger evidence than a hit count.

The constants were never stale.  structures/unit-struct-runtime.md records its own
provenance ("all six tutorials and Stage 02 loaded from save slot 01") and
launch_mission.sh already encoded that route: title -> LOAD GAME -> slot 01 -> YES
-> READY ROOM -> TAKE OFF.  LOAD GAME is the menu's SECOND item; the previous
iteration pressed the first (NEW GAME) and read the resulting 0/0 as evidence
about the constants.  One `step down` separated a dozen iterations from this.

Control still owed and stated in the doc: the title-screen arm was measured in
earlier processes, not this one.
2026-08-26 20:46:21 +00:00
Sylpheed RE agent
1bada3ba4c re: the working entity captures came from a SAVE SLOT, not mission select
Before deriving a new vtable I checked where gworld.py's constants came from.
They cite structures/unit-struct-runtime.md, which states its provenance:
"Captured 2026-07-29 ... all six tutorials and Stage 02 'Declaration of War'
loaded from save slot 01."

Loaded from a save slot -- not through MISSION SELECT, which is the route every
run this session has taken, and which needs the cleared-stage mask poke to offer
a stage at all.  So the constants may not be stale; they may just need the state
that route produces.  That is a cheaper question than writing a new vtable
finder, and it should be answered first.

First attempt inconclusive: driving the main menu's first item blind, five
presses deep, advanced the progress counter every time (3 -> 5 -> 6 -> 8 -> 10 ->
12, so the game responds) but left DEF_VTABLE / INST_VTABLE at 0/0 throughout.
Without a screen identity this is dead reckoning, and the first item may not be
the load-game entry -- newgame_path.sh documents it as NEW GAME with SELECT DATA
two screens further in.

Next: reach the save-slot screen deliberately rather than by counting presses,
and load slot 01.
2026-08-26 20:13:33 +00:00
Sylpheed RE agent
b56a4ab9f9 re: "follow the name pointer" is structurally dead for pak strings
Locating DeltaSaber at mission time works: 53 occurrences, all in the heap
(0xBC66..-0xBC6C..), none in the XEX image -- DeltaSaber_T.xpr, _Special,
_NoseGun, _Missile, _TwinGun.

But searching all of guest memory for a big-endian pointer to those addresses
returns ZERO references for every one tried.  That is the format, not a search
bug: those strings live in the mission pak's IDXD string pool, and this corpus
decoded that container long ago -- records reference names by OFFSET into the
pool, never by absolute pointer.  So there are no pointers to find, and "find the
name, follow what points at it" cannot work on pak data by construction.

This also narrows what the string counts proved.  The title-vs-mission control
stands (DeltaSaber 0 -> 81), so mission-specific DATA is loaded, which is more
than "the tables load at boot".  But these are asset-table strings, not live
entity objects, so they do not show entities have been spawned.  Honest split:
the mission's pak data is loaded; whether entity objects exist is still
unmeasured, because both probes tried -- gworld's vtable constants and
name-pointer following -- are respectively stale and structurally inapplicable.

Remaining routes, untried: derive the entity vtable from CODE via sylpheed.db's
vptr_writes table, which exists for exactly this; or find the entity list from
the mission update function rather than from data.
2026-08-26 20:06:10 +00:00
Sylpheed RE agent
432cc43885 re: the mission DOES load -- DEF_VTABLE=0 was a stale constant, not an empty game
Three iterations concluded "no units are loaded" from scan_vtable(DEF_VTABLE)
returning 0.  Withdrawn: the scan was measuring the wrong thing.

Counting unit-name strings in guest memory with the TITLE as a control (same boot
recipe, no mission):

    string        title    after take-off
    DeltaSaber        0        81
    rou_          1,981     7,680
    UN_              19       167
    e010             67       134
    ADAN          1,450     1,921

DeltaSaber is the player's craft: absent at the title, present 81 times after
take-off.  Every other count rises several-fold.  The stage content is
unambiguously in memory.

The control is what makes this a result.  Raw counts prove nothing on their own --
rou_ and ADAN are numerous at the title too, because the unit tables load at boot.
Only the title-vs-mission difference, and DeltaSaber's 0 -> 81 in particular,
separates "tables loaded" from "mission loaded".  My first reading skipped the
control and over-claimed; the control was run before publishing.

Withdrawn as a consequence: "no units are loaded"; and the inference that the
take-off lands in a never-ending cutscene (the movie accesses are real, but the
conclusion rested on DEF_VTABLE=0).  gworld.py's DEF_VTABLE 0x820AF844 and
INST_VTABLE 0x820AF030 do not locate entities in this build/state despite being
genuine vtables in sylpheed.db.

Next: derive the correct entity vtable.  The UN_ strings reachable by search are
UN_NOSE/UN_MOUNT attachment names in the XEX image, not runtime records -- so use
a name that only exists at mission time.  DeltaSaber is exactly that: find its
heap occurrences, find what points at them, read the referencing object's vtable.
2026-08-26 19:56:52 +00:00
Sylpheed RE agent
9e421b341c re: the post-take-off state is measurably neither a menu nor flight
wait_flight.sh carries the corpus's best discriminator, and it is a whole-image
statistic rather than a named pixel: the flight HUD paints green over the frame
at 1.3-1.5% green fraction, against <0.1% on every menu.

Measured on the post-take-off state: green=0.0022, mean (1.6, 45.6, 50.1).  That
is 0.22% -- an order of magnitude above a menu, an order of magnitude below
flight, on a dark blue-green frame.  screen_id.py files it under "menu" because
that is the closest bucket it has; the state is neither.

wait_flight.sh then tapped A for 280 seconds and never saw the HUD.

With the 12 \dat\movie accesses, the reading is consistent: this is the cutscene,
it does not end on A, and it does not reach the flight HUD within five minutes.

Open, cheapest first: how the movie path signals completion (the corpus already
decoded the movie handler for the intro video and found a software-decode
deadline); whether a mission can be started by a route that skips the launch
cinematic; or whether the cutscene simply needs far longer under lavapipe.

Oracle status overall: boot, menus and take-off are reliable and reproducible;
only the cutscene->mission step is unsolved -- a much smaller and better-defined
gap than the freeze it replaced.
2026-08-26 19:43:40 +00:00
Sylpheed RE agent
454be7b0de re: the take-off sequence lands in the launch CUTSCENE, not a hang
Logging file and kernel activity through the whole navigation and counting names:

    12  \dat\movie      <- heavy, repeated movie access
     3  \dat\sound
     2  Stage_S01       <- the stage resource is referenced
     1  XamShowDeviceSelectorUI

So after take-off the game plays the launch cutscene.  Under software rendering a
720p decode is very slow -- this corpus already raised INSTRUCTIONS_PER_MS 100x
because the movie handler has a 2000 ms software-decode deadline -- which is why
the state persists for minutes.

It is unambiguously live, by the widest margin yet measured:

                     frozen   menus   this state
    screen liveness  0.00%    0.22%   11.00%
    guest churn      0.000%   0.190%  0.671%

and the progress counter advances on input (17 -> 101 across the skip attempts),
so the pad reaches the game.

But it does not skip: four rounds of START / A / B advanced the counter every
time and left DEF_VTABLE / INST_VTABLE at 0/0.  The input registers, it just does
not end the sequence.

The remaining gap is now narrow and named: the take-off cutscene runs and is not
skippable by the obvious buttons.  Either it must play out, or the right skip
input is unknown, or it waits on something the movie path never delivers.
2026-08-26 19:30:21 +00:00
Sylpheed RE agent
915d6adbd9 re: the sign-in fix holds end-to-end, but no units are loaded yet
With the emulator launched correctly and the boot verified by the progress
counter first (counter > 0 = title reached; a flaky boot never leaves 0),
nav_to_flight.sh ran the whole sequence -- main menu at 20 s, mask poke, stage
select, briefing skip, ready room, take-off -- where every previous attempt
stalled.

The game is alive at the end of it: screen liveness 2.13% against 0.00% when
frozen, guest churn 0.288-2.932% against 0.000%.  An animating screen and a busy
guest at the point that used to be stone dead.

But no units.  DEF_VTABLE (0x820AF844) has no hits on a direct byte search, with
a working control in the same run: searching for the poked cleared-stage mask
0x0001FFFE finds it at 0x828F40C0, exactly where nav_to_flight wrote it.  So the
search is sound and the vtable is genuinely absent.  Both vtable constants were
also checked against sylpheed.db's vtables table and are real vtables, so they
are not stale.  Three minutes of waiting left DEF/INST at 0/0 with the counter
static at 17.

So the run reaches a live, animating state that is not a mission with spawned
units -- most likely the launch cinematic or a post-take-off screen.  The freeze
is gone; what still separates this from unit spawn is the remaining work.
2026-08-26 19:20:24 +00:00
Sylpheed RE agent
45c5ff8234 re: the sign-in fix must be applied at launch, not by the navigator
nav_to_flight.sh contains no run-canary invocation -- it drives an
already-running emulator -- so whether a profile is signed in depends entirely on
how the caller launched it.  Every nav_to_flight run in this session was launched
with a bare run-canary, which is why they froze.  Worth stating plainly because
the script prints "signing in profile B13EBABEBABEBABE", which comes from the
emulator's startup rather than from the script arranging it.

Refuted: "each screen loads its own GP_*.pak", a plausible route to a real screen
identity.  With full file logging the only names that appear are at boot
(sound.p0x, tables.p00, hidden) and nothing new opens as the UI moves between
screens -- the paks are opened once and read by offset, so file I/O carries no
per-screen signal.

Also recorded: one launch this iteration never reached the title -- progress
counter stuck at 0, black screen, guest churn 0.001%.  That is a DIFFERENT
failure from the sign-in freeze, which stalls after the main menu with the
counters already advanced, and it matches the boot flakiness
canary-scripted-input-traps.md already documents.  The counters distinguish
them: sign-in freeze stalls past the title, a bad boot never leaves 0.
2026-08-26 19:07:25 +00:00
Sylpheed RE agent
15e58d93b1 re: correct the "screen id" -- they are monotonic counters, not identities
The names in menu-state-in-memory.md are wrong for at least two of the three
words, and the test that shows it is going backwards.  Driving deep into the
menus, then three presses of B:

    0x828F38AC "cursor"   36  -> 38 -> 40 -> 41
    0x828F37B4 "misc"    971  -> 1067 -> 1068 -> 1068
    0x828A690C "screen"   56  -> 56, 56, 56

A cursor returns when you go back.  These only ever increase -- monotonic
counters, with 0x828F38AC advancing about 2 per input.  Driving forward produced
1,3,4,5,6,8,10,12,25,29,32,33,50,53,56 for the "screen id", which is an identity
sequence only if the game has 56+ screens and never revisits one -- exactly what
a counter also looks like.

Withdrawn: 0x828A690C as a screen IDENTITY (1 title, 3 main menu, 4 extras).  The
values are path-dependent; they matched across runs because the same key sequence
produces the same count, not because 3 means main menu.

Survives: all three advance if and only if the game responds, and are stable when
it does not.  That is a real input-progress signal, reproducible across runs and
both GPU backends, and it is what made blind navigation work.  Read it as "did
the game react?", never "which screen is this?".

It also retro-confirms the freeze diagnosis: with the sign-in fix the sequence
runs 3 -> 5, skipping 4 entirely, so "4 = extras" was never a screen -- it was the
count at which the game stopped responding.  The counter reading explains both
observations.

Open: no mission reached.  Counters at 56/41/1068, guest churn 0.190% (alive;
frozen is 0.000%), DEF_VTABLE and INST_VTABLE still 0.  Without a real screen
identity, navigation is dead reckoning; finding a genuine state enum is next, and
the snapshot-and-diff method can be repeated with these counters excluded.
2026-08-26 18:51:25 +00:00
Sylpheed RE agent
d712e5aa1a re: SOLVED -- the mission freeze is a modal sign-in dialog
Every Xam UI dialog goes through xeXamDispatchDialog, and the calling guest
thread blocks on fence.Wait() until it is dismissed.  With kernel logging on the
last call before the freeze is XamShowSigninUI(00000001, 00000001): the game
asks for a signed-in profile, Xenia opens a modal dialog, and nothing in a
scripted run ever dismisses an ImGui dialog.

That accounts for every symptom at once -- Main XThread futex-blocked at 0 ms CPU
rather than spinning, emulator alive, no guest progress, no faults, and
independence from both the GPU backend and the allocation outcome.

Fix: run-canary --logged_profile_slot_0_xuid=B13EBABEBABEBABE.  Same route, one
variable changed:

                        no profile      with profile
    screen id           4, forever      4 -> 5 -> 6 -> 8 -> 9 -> 10
    XamShowSigninUI     called          not called
    alloc failures      1               0
    guest throws        1               0
    guest churn         0.000%          1.006%

XamShowDeviceSelectorUI is then requested but storage_selection_dialog defaults
to false, so it is not shown and does not block.

Why it took so long: --log_mask=13, used by every script in this corpus,
DISABLES kernel logging -- the one category that names the dialog.  Without it
the freeze presents as a bare futex block, which is why it was attributed in
turn to a heap leak, rounding, MmQueryStatistics, a build regression, the route,
the savegame, shader compilation, an infinite guest loop and the software
rasterizer, each refuted in its own section.

Nearest miss worth recording: challenge-mission-gate.md reported a "Disc Read
Error" dialog on this failure.  That is XamShowDirtyDiscErrorUI, which calls
exit(1) -- "This is death, and should never return" -- so it cannot be our
freeze, whose process stays alive.  Checking that distinction is what led to the
dialog mechanism.

Open: no mission reached yet (DEF_VTABLE/INST_VTABLE still 0 at screen 10), so
the remaining screens are menus.  Ordinary navigation, not a blocker.
2026-08-26 18:41:09 +00:00
Sylpheed RE agent
18e9cecf21 re: WITHDRAW "the freeze requires the rasterizer" -- the control was not controlled
Driving the null backend blind, using the memory screen-id as the guide, to the
SAME screen the rendered run froze on (screen 4):

    guest churn   19 of 8,089,600 bytes in 4 s  (0.000%)
    Main XThread  0 ms CPU / 4 s, state S, wchan futex_do_wait
    log           AllocRange unable ... 134217728 bytes; guest throw

Frozen identically with no rasterizer at all.

The flaw is the comparison, not the data.  The earlier "control" measured the
null-GPU run while it was still in the MENUS and the lavapipe run AT the freeze,
then reported the difference as a backend effect.  The allocations I cited as
proof it had walked through the content load (114 -> 120) were menu-time
allocations.  A control needs both arms verified to be in the same state -- and
the memory screen-id built this same session is exactly what makes that
checkable, which I failed to use on the arm that mattered.

Withdrawn: "the freeze requires the rasterizer", "the fault is in the host
rendering path", and the hardware-Vulkan blocker that followed from it.  The
rendering question is no longer load-bearing.

Survives: the freeze happens on the first content load after the main menu, on
every route and BOTH GPU backends, with Main XThread futex-blocked at zero CPU
rather than spinning; and four runs froze with three different allocation
outcomes, so the 128 MB refusal is correlated but not necessary.

Also survives and is independently verified: the memory screen-id navigation,
now reproduced on a third run.  It is what exposed this error.

Open again: what is Main XThread waiting on?  Identify the futex and its holder,
carefully -- gdb perturbs this measurement.
2026-08-26 18:28:47 +00:00
Sylpheed RE agent
534da24658 re: find the menu screen id and cursor in guest memory
Menu navigation here has always been screenshot-driven, which is unusable under
--gpu=null -- the only configuration where the game does not hit the
software-rasterizer freeze.  Without a memory signal, the one backend that runs
is the one that cannot be steered.

Snapshotting 0x82800000+3 MB at each menu of a rendered run and keeping the
4-byte words that differ between screens and hold small integers leaves exactly
four of 786 432.  One has the property that matters -- it changes on a screen
transition and holds steady when only the highlight moves:

    0x828A690C  screen id     1 title, 3 main menu, 4 extras
    0x828F38AC  menu cursor   (second copy at 0x828F38BC)
    0x828F37B4  per-menu value

Verified on a fresh --gpu=null run with no display at all, driving the same keys
blind: title 1/2/12, main menu 3/4/45, after 4x down 3/12/45, extras 4/14/49 --
4/4 exact against the rendered run, across two runs and two GPU backends.  That
is the check that matters, since this corpus has already had to mark one runtime
address run-dependent.

tools/re-capture/menu_state.py reads them; `menu_state.py watch` prints on
change.

Open: the rest of the sequence into a mission.  Blind driving reached extras
(screen 4) and a further A did not move it, so MISSION SELECT needs a cursor
move first.  Screen ids beyond 4 are unmapped, and the rendered run freezes on
entering that screen -- so map ids up to the freeze, then step blind past it.
2026-08-26 18:21:04 +00:00
Sylpheed RE agent
5ecdb1c460 re: the freeze requires the rasterizer -- --gpu=null does not freeze
The cleanest control available, and it settles which side the bug is on.  Xenia
has a null graphics backend.  Same ISO, same route, same presses:

                          lavapipe            --gpu=null
    Main XThread CPU      0 ms / 3 s          3860 ms / 4 s
    state / wchan         S / futex_do_wait   R / not blocked
    guest memory churn    ~0 (18 B of 8.2 MB) 22.6% of 6 MB in 4 s
    allocations           stall               114 -> 120, no failures
    process CPU           265% (all llvmpipe) 330% (mostly guest)

With no rasterizer the guest does not freeze: it runs at close to a full core and
walks through the content load -- the exact point that hangs every rendered run.
So the fault is in the host rendering path, which explains why every game-side
hypothesis on that page was refuted in turn.

Blocked, and recorded as such rather than improvised around: deciding whether
lavapipe is deadlocked or merely taking hours over one pathological draw needs
hardware Vulkan, which this container does not have.  Six minutes of nine
saturated llvmpipe threads with no frame is suggestive, not conclusive.

Amber workaround: --gpu=null gives a live guest, and everything the backlog needs
-- entity positions, flight model, world unit -- is read from guest memory, not
pixels.  The gap is navigation: menu-walking is screenshot-driven, and blind A
presses advanced allocations 114 -> 120 but never reached a mission (DEF_VTABLE
and INST_VTABLE scans both 0).  Next step is to drive navigation from guest
memory instead of the screen, which is engineering rather than mystery.
2026-08-26 18:12:35 +00:00
Sylpheed RE agent
ffdccba655 re: WITHDRAW the infinite-loop conclusion -- gdb was the confound
The previous section concluded the guest spins forever in sub_82457780.
Measured without gdb, that is false.  Every observation behind it came from a
gdb-hosted run, and gdb intercepts every SIGSEGV -- which Xenia uses for guest
memory watches -- so it perturbs precisely what was being measured.

Same measurement, no debugger, per-thread /proc sampling parsed after the last
')':

    Main XThread          0 ms CPU per 3 s, 0 faults, state S, futex_do_wait
    GPU Commands         10 ms
    WSI swapchain queue 130 ms
    llvmpipe-0..9      ~1030 ms each per 6 s

Process at 265% of a core, essentially all in the software rasterizer.  The
guest thread is BLOCKED on a futex using zero CPU -- not spinning, and not in
sub_82457780.

Under gdb that same thread appeared as the top CPU consumer (890 ms / 4 s) and
appeared to be in guest JIT code, then in xe::ExceptionHandlerCallback.  That
was signal interception plus gdb re-stopping it between samples (state t,
wchan ptrace_stop).

Withdrawn: "an infinite copy loop in sub_82457780 is the freeze", and "Main
XThread is the top CPU consumer, so it is a spin not a block".  The opposite
holds.

Survives: the emit_source_annotations technique is sound and did prove the PC
was 0x824578A0 while the guest ran under gdb, so the guest does pass through
that copy loop.  The bne-exact-equality reading remains an accurate description
of the disassembly and a plausible hazard -- just not this freeze.

What the freeze actually is: guest blocked on a host futex at zero CPU, software
rasterizer saturated for six minutes without producing a frame, nothing
faulting, screen unchanged.  That is the guest waiting on a GPU operation that
never completes, with lavapipe spinning -- a HOST RENDERING problem, which is
also why every game-side hypothesis this session was refuted in turn.

Method rule earned the hard way and recorded on the page: do not diagnose a
performance or liveness question under a debugger.  Use it to read state at a
known stopping point, then re-measure timing and CPU detached.
2026-08-26 18:02:54 +00:00
Sylpheed RE agent
abed25fd67 re: prove the guest PC -- an infinite copy loop in sub_82457780
The previous section withdrew this localisation for resting on a stale
PPCContext.  It is now re-established by a sound method.

Xenia has a CPU cvar, emit_source_annotations, that makes MarkSourceOffset emit
"nop; nop; mov eax,<guest_address>; nop; nop" at EVERY guest instruction
boundary -- 90 90 B8 xx xx xx xx 90 90 in the code bytes.  So the guest PC reads
straight out of memory around rip, with no context and no offset guessing.

Dumping backwards from rip gives a clean run of them -- 0x82457874, 78, 7C, 80,
84 -- and the annotation immediately before rip is 0x824578A0.  That is
`sth r6, 0(r9)` inside sub_82457780: the exact instruction predicted from the
disassembly last time, now reached independently.

The loop advances r11 by 8 and terminates on `bne cr6` -- exact inequality.  If
r31 - r7 is not a positive multiple of 8, the comparison never becomes equal and
the loop never exits.

And the thread is genuinely running: over 4 s, Main XThread consumed 890 ms of
CPU, the top of any thread, with the process at ~238%.  A spin, not a block.

One bad metric recorded: I tried to show the loop marching through memory by
watching the last non-zero byte above 0x70200000.  It never moved -- but that
statistic saturates in an already-written region, so it could not have moved.
Not evidence either way, though it briefly looked like a refutation.

Open and now narrow: what makes r31 - r7 non-congruent to 0 mod 8.  r7 comes
from 0(r30), r31 is the container end, and r3 holds a fresh buffer from
0x824F7240 -- read those three from HOST registers at the freeze.
2026-08-26 17:50:57 +00:00
Sylpheed RE agent
ee2f737d78 re: audit the gdb session -- withdraw the sub_82457780 localisation
Went back to verify the instrumentation before building on it.  Some does not
hold.

Verified: the trailing comments in ppc_context.h are not struct offsets (the
struct opens with eight cr unions and fpscr), but rsi+0x20 for r[32] is
nonetheless correct -- confirmed empirically because the f[32] doubles begin at
+0x140 and 32*8 back from there lands on +0x20.

Withdrawn: the context is STALE mid-function.  The same dump shows r1 = 0, and a
guest stack pointer cannot be zero inside a function -- Xenia's JIT keeps live
guest registers in host registers and spills only at certain points.  So
r13 = 0x82457864 does not show where the guest is executing, and identifying
sub_82457780 as the stuck function is withdrawn as unproven.  The description of
what that code does (grow-and-copy, beq exit) is still accurate; it is just not
established that the guest is in it.

Three readings retracted for broken instrumentation:

  * "rip identical across samples ⇒ no progress" -- gdb stops the process
    between samples, and the JIT lays code out differently per run (a05be939 vs
    a05bc839 for the same instruction).
  * "zero minor faults on the stuck thread" and "utime=0" -- both parsed
    /proc/<pid>/task/<tid>/stat with awk positional fields, but comm is
    parenthesised AND contains spaces ("Main XThread (F"), so every index after
    it was wrong.  Printing state=XThread should have been the giveaway.
  * the SIGSEGV fault-storm refutation rested on the same parse; downgraded to
    unproven, though the process-wide ~300/s still argues against a tight loop.

Survives: the freeze reproduces without gdb; with the process stopped exactly
one thread is in userspace JIT code while a wchan census puts the rest in
futex_do_wait (55) or hrtimer_nanosleep (11); the instruction is a 16-bit store
through the membase to guest 0x701d0000, a stack-region address, first of a
four-store group; and that memory was unchanged over seconds of running time.

Next step is a host->guest code mapping from Xenia's own code cache rather than
inference from a stale context.
2026-08-26 17:38:49 +00:00
Sylpheed RE agent
a70b1d8724 re: locate the stuck thread -- a grow-and-copy in sub_82457780
Ran the freeze under the corpus's gdb wrapper (ptrace_scope is 1, so a debugger
must launch rather than attach, and its handle lines are needed because Xenia
uses SIGSEGV for guest memory watches).

One thread of 80 is in guest code: thread 50, Main XThread, at rip a05be939 in
JIT output.  All others are in a futex or clock_nanosleep.

Xenia's x64 backend keeps PPCContext in rsi, with r[32] at +0x20.  The guest
GPRs there give r8 = a3ac0000 (the 64 MB buffer from the doubling sequence),
r12 = a3ac0a18 inside it, and r13 = 82457864 -- guest code, sub_82457780.

That address sits in a grow-and-copy: size = count*8 clamped to 0x1FFFFFFF, a
call to 0x824F7240, then a loop copying halfwords eight bytes at a time.  The
host instruction it is stopped on is that copy's store:

    mov %r12w,(%rdi,%rax,1)     rdi = membase, rax = 0x701d0000

Sampled three times seconds apart with continue in between: rip, r13, r11 and
r31 identical every time.  No progress.

The loop's exit test is beq -- equality, not >= -- so an inconsistent start/end
pair never terminates it.  Recorded as amber: that is a reading of the
disassembly, not a demonstration.

Two of my own readings corrected:

  * "the guest spins at ~400% CPU" was ps's CUMULATIVE AVERAGE since process
    start, not an instantaneous rate.  Per-thread sampling puts Main XThread
    nowhere near the top.
  * a SIGSEGV fault storm fitted the constant rip nicely and is refuted: 1500
    minor faults in 5 s, zero major.

Caveat kept on the page: these are gdb-hosted observations and gdb intercepts
every SIGSEGV, so absolute timings are not the ungoverned ones.  The freeze is
not a gdb artefact -- it reproduces in every non-gdb run.
2026-08-26 17:28:31 +00:00
Sylpheed RE agent
d8f0a5e798 re: the allocation failure is definitively not the freeze -- four runs, three outcomes
Four runs now freeze at the first content load after the main menu with three
different allocation outcomes: stock (fails at 128 MB and throws), GPU-writeback
patched (no failure, stops at 32 MB), heavy XMA logging (no failure, stops at
16 MB), and no-savegame (no failure, no throw).  The memory account on this page
is accurate but describes a symptom; everything treating the 128 MB refusal as
the freeze is superseded.

The savegame is not the variable.  --content_root pointed at a profile-only tree
(the real content untouched) boots to the identical pools -- BC220000 Size
03A80000 and B50C0000 Size 070E0000, same addresses and sizes, same 43
allocations by the menu -- then freezes at the same press with zero failures.

Not slow shader compilation either.  Every earlier run called it frozen within
~30 s; this one was left six minutes at 0.00% liveness with CPU still 399-479%.

Correction made mid-measurement: the allocation counter creeping 119->124 looked
like "guest running, display stuck".  It is not.  A broad sample -- 400 extents,
8.2 MB -- changed 18 bytes in 3 seconds.  The guest is spinning, with one thread
allocating about once a minute.  The earlier "8 MB slab unchanged" reading was
weak for the same reason in reverse: one arbitrary region proves little.  Sample
broadly before calling a guest alive or dead, exactly as the screen needed two
frames rather than one.

Now excluded: the heap failure, the leak, rounding, MmQueryStatistics, a
heap-size knob, the exception cvar, a kernel-object wait, a build regression, the
route, the savegame, and shader compilation.
2026-08-26 17:15:03 +00:00
Sylpheed RE agent
5ed90577ee re: the 171 MB of boot allocations are the game's own pools, ~90% empty
Read from a LIVE guest (main menu, liveness 98.59%) rather than a frozen one,
sampling a page per megabyte:

    B50C0000  112.82 MB  89.7% zeros, header all zero
    BC220000   58.45 MB  90.8% zeros, header = pointers into itself

Neither holds content at the main menu and neither matches a disc file's size.
They are pre-reserved pools claimed at boot and mostly untouched.

BC220000's first words -- bc220010, bc79c9c0, bcd24c00, bc220040, bd17cda0,
bc79ccc0, all inside [BC220000, BFCA0000) -- interleaved with ordinals 6,0,4,5,
0,1,2,3, are an intrusive free-list.  That closes a loop from this morning: the
22 "Release failed because address is not a region start" messages were frees of
interior pointers into this exact block.  The game suballocates from its own
pool and hands the sub-pointers to MmFreePhysicalMemory.  Independent
confirmation that Xenia is right to refuse them.

It sharpens the contradiction rather than resolving it.  The budget adds up --
~171 MB of pools, ~379 MB live at the menu, 128 MB wanted for a content load,
about 495 MB of a 512 MB console against Xenia's fixed 16 MB GPU reservation,
which is the measured ~15 MB shortfall.  But the SAME emulator with the SAME
reservation ran a mission for 500 s on 08-10, and that run's evidence is entity
hull values sampled from guest memory at 2 Hz, not screenshots, so it is not the
frozen-frame illusion that fooled this session twice.

So the next thing to vary is the boot inputs -- profile, save data, config --
since B50C0000's request is 0x070D28B0, an odd computed size rather than a round
reservation, and so plausibly depends on something configurable.
2026-08-26 16:59:26 +00:00
Sylpheed RE agent
a9ce94ec83 re: MISSION SELECT is not special -- NEW GAME freezes too
Last iteration concluded the freeze is "entering MISSION SELECT" and made
avoiding that screen the next experiment.  Ran it; the conclusion was too
narrow.

First, a liveness metric that actually separates the states: two frames five
seconds apart, percentage of pixels changed.  The menu animates, so healthy is
99.80-99.97% and frozen is 0.00%.  No navigation script needed, and no
classifier.  Committed as tools/re-capture/route_liveness_probe.sh; this is what
should have been used from the first run.

Then the menu's FIRST item, NEW GAME, which never touches MISSION SELECT:

    main menu             99.80% alive
    after A on NEW GAME    7.33%
    after the next A       0.00%  -- frozen, and screen_id calls it "flight"

with the same 134217728-byte AllocRange failure in the log.

So the correct statement is broader: the game freezes on the first content load
after the main menu, whichever item is taken.  MISSION SELECT was just the route
every earlier run used.  "Avoid MISSION SELECT" is withdrawn -- there is nothing
to avoid, and that also puts the memory account back at the centre, since ~379
MB live plus a 128 MB content load fails on any route.

Worth repeating because it caught me twice: screen_id.py called a frozen frame
"flight" on a run that never left the menus.  Liveness first, classification
second.
2026-08-26 16:51:59 +00:00
Sylpheed RE agent
afc74ca9f4 re: it is not a stage-load freeze -- the game freezes entering MISSION SELECT
The whole page called this a stage-load/take-off freeze.  Wrong, and the
evidence was in each run's own screenshots.

nav_to_flight.sh saves a shot per step.  Compared CONSECUTIVELY:

    01-extras -> 02-missionselect   rmse 51.07, 99.42% changed
    02-missionselect -> 03-selected rmse  0.00,  0.00% changed
    03 -> 04 -> 05 -> 06 -> 07 -> 08-flight   all 0.00

Shots 02 through 08 are pixel-identical.  The screen stops at MISSION SELECT and
never changes.  Every later step the script reports -- selected, readyroom,
takeoff, flight -- is it pressing buttons at a frozen image while screen_id.py
classifies a static frame.

Withdrawn as a result:

  * "MISSION SELECT and the stage list worked, the failure came later during the
    take-off load".  It froze entering MISSION SELECT.
  * challenge-mission-gate.md §5.6 already said entering MISSION SELECT fails on
    a 128 MB request and shows "Disc Read Error".  It was right; I contradicted
    it because my navigation script claimed to have got further.
  * Every "reached flight" in this session is false, including runs where
    screen_id.py returned "flight".

Method note kept on the page: the first comparison ran every shot against shot 1
and returned rmse 51.07 / 99.42% for all seven, identical to two decimals across
seven different images.  That is a tell, not a result.  I then talked myself out
of it because md5sum showed eight distinct hashes -- which was PNG metadata.
Compare consecutive frames, and treat a suspiciously constant statistic as a bug
in the measurement.

Consequence for the oracle: the allocation failure happens while the display is
already frozen, so it is downstream, which fits the three runs that froze with
the allocation succeeding at 16, 32 and 128 MB.  MISSION SELECT is the thing to
avoid -- every route this session used goes EXTRAS -> MISSION SELECT, and
newgame_path.sh / tutorial_launch.sh reach a mission by other menus.
2026-08-26 16:42:13 +00:00
Sylpheed RE agent
1a6b9cac34 re: the stage-load freeze is not a regression -- an 08-17 binary reproduces it
This corpus has a 500-second Stage 02 flight from 2026-08-10, and the canary
tree gained the file-pad driver, the UI-draw capture, a threading_posix resume
fix and log_stuck_waits between then and now, with the running binary rebuilt on
08-24.  So a regression was the obvious suspicion.

It is wrong.  The container already keeps older builds under
/sylph-home/re/bin/, plus a host build from 08-17 that predates all of the 08-19
changes, and run-canary honours $XENIA_BIN -- so this cost no rebuild.  The
08-17 binary, same route, same point: 1 alloc failure, 1 guest throw.
Identical.

So bisecting the emulator is not the way in, and the 08-10 run did not differ by
binary either.  Whatever let it reach flight is in the route or the game state.

That sharpens the open question usefully.  Both of the two largest live blocks --
112.88 MB at B50C0000 and 58.50 MB at BC220000 -- are allocated at boot, before
any menu, so 171 MB of the 379.5 MB is fixed regardless of route.  The remaining
~208 MB is where a route difference could live, and the ledger can measure it:
capture live-bytes at the moment TAKE OFF is pressed for two navigation paths
and compare.
2026-08-26 16:19:57 +00:00
Sylpheed RE agent
a65649598e re: the freeze is not a kernel-object wait -- log_stuck_waits stays silent
The canary branch already carries log_stuck_waits, written for this exact
question: it names the object a guest thread keeps timing out on, and is silent
on a healthy run because a wait that gets satisfied never builds a streak.

Enabled it and ran to the freeze on the stock build.  Not one stuck-wait line,
alongside the usual AllocRange failure and guest throw.  So no guest thread is
parked on a kernel object that never gets signalled; combined with 389% CPU
across running threads, the guest is spinning in its own code -- which is what
the throw-that-returns predicts, since execution resumes after the throw and
runs into code that assumed it would not.

Operational note worth having: --log_stuck_waits=true on the command line is
ACCEPTED but the config file value wins, and the first run silently logged
"log_stuck_waits = false" while I thought it was on.  The startup dump prints
the effective value -- check the dump, not the flag.  The flag is cheap and
silent so it is now left enabled in the container config.
2026-08-26 16:10:07 +00:00
Sylpheed RE agent
b24264ef5c re: the allocation failure is NOT the freeze -- two patched builds say so
Rebuilt canary twice to test the freeze, and the result demotes my own headline.

1. The game does not size anything from MmQueryStatistics.  Xenia reports
   kernel_pages = 1 MB under a comment admitting the numbers are guessed, and
   the game really does call the export -- sub_82612420 converts
   total_physical_pages and title.available_pages to bytes, and its caller holds
   available-bytes in r23 while creating render surfaces.  Patched it to 32 MB,
   twice the shortfall, and rebuilt: 122 allocations, 484.5 MB ever, 379.5 MB
   live in 84 blocks, failing at "free 28969/131072" -- byte-identical to stock.
   A 32x change moved nothing.  Refuted; reverted.

2. The ~19 MB the ledger could not see is Xenia's own startup reservation:
   memory.cc:240 pins 16 MB of the parent heap for GPU writeback before the
   guest runs, and vC0000000's parent IS the 512 MB physical heap.  16 MB is
   more than the 14.84 MB shortfall.

3. Shrinking that reservation to 1 MB removes the failure completely -- zero
   AllocRange failures, zero guest throws -- AND THE GAME STILL FREEZES.
   Verified with a single emulator after killing the stale one: three frames at
   rmse 0.00, an 8 MB guest slab unchanged over 3 s, 389% CPU on 3 running
   threads.  The ledger shows it dying EARLIER, stopping at the 32 MB doubling
   step where stock reached 64 MB, so shrinking a live GPU region hangs the GPU
   instead.  Reverted.

So "the freeze is the refused 128 MB allocation" was too strong.  The refusal is
real and Xenia's 16 MB reservation is the swing factor that decides it, but the
guest hangs without it too, at an earlier point.  The refusal is one way this
stage load dies, not the cause.

Also recorded: --eh_dispatch, --mem_watch and --audio are all named in this
corpus's own scripts and docs and NONE exists in this build.  Each is silently
rejected, which blocks boot rather than warning.
2026-08-26 15:22:32 +00:00
Sylpheed RE agent
2656ac0d8e re: the heap ledger refutes my own leak hypothesis
Built the allocation ledger to test the leak I proposed this morning, and it
refutes it.

  * No leak.  The books balance: 379.5 MB live in 84 blocks plus 113.2 MB free
    is 492.7 MB of the console's 512, with ~19 MB in allocation paths the filter
    did not capture.  Nothing is missing that a leak would explain.

  * The 22 "leaked" releases are correct refusals.  All 22 failing frees are
    interior pointers into ONE allocation -- the 58.5 MB block at BC220000, at
    offsets from 5 to 30 MB.  The game sub-allocates out of a physical pool and
    frees the sub-blocks.  BaseHeap::Release frees whole regions, so honouring
    an interior pointer would free 58.5 MB, twenty-two times over.  The fix I
    was moving toward would have been a catastrophe.

  * Rounding is not it either: 480.67 MB requested against 484.50 MB granted is
    3.83 MB (0.8%), versus a 14.84 MB shortfall.

What actually happens is a doubling grow that holds both buffers: allocate 32 MB,
free the previous, allocate 64 MB, free the 32, then ask for 128 MB while still
holding the 64.  That needs 192 MB live for one buffer on top of ~315 MB held
elsewhere, and comes up 14.84 MB short.

No configuration fixes it.  There is no guest memory-size cvar; the 512 MB is
baked into the address map rather than a constant (the aliases at 0xA0000000 and
0xC0000000 are spaced exactly 0x20000000 apart, so growing the heap runs one
window into the next); and this tree has no eh_dispatch cvar, so
RtlRaiseException routes the guest's OOM throw to HandleCppException, which logs
and returns without unwinding.  That fall-through is the freeze.

Also worth recording: log_mask DISABLES categories (Kernel=1, Apu=2, Cpu=4,
Gpu=8), so the --log_mask=13 used throughout this corpus has been running with
the kernel log switched off.

The oracle is NOT fixed.  Ledger committed as data/heap-ledger-stage01.txt.
2026-08-26 12:22:40 +00:00
Sylpheed RE agent
419d45eae5 re: the leaking release is confirmed in Xenia's source, not inferred
The previous write-up argued the leak from adjacency -- 22 failed releases next
to the failed allocation.  memory.cc closes the loop outright:

  * the "parent free N/M pages" in the error is
    parent_heap_->unreserved_page_count() (memory.cc:1807);
  * unreserved_page_count_ is incremented in exactly ONE place, memory.cc:1445,
    inside BaseHeap::Release's page-table loop;
  * the failing path returns at memory.cc:1399, at the top of that same
    function, before the loop -- page table untouched, no free block inserted;
  * and PhysicalHeap::Release delegates to parent_heap_->Release, so the release
    that fails and the allocation that later comes up short are the same heap.

So every "address is not a region start" returns zero pages to the counter the
allocator consults, and those pages stay reserved for the life of the process.
That is control flow, not correlation.

Magnitude is still open and I am not claiming it: 512 - 113 = 399 MB missing
against only 22-23 failed releases would need ~18 MB average each, which is
implausible as the whole story.  Leaked releases are a contributor, maybe not
the dominant one.  An allocation ledger -- log every MmAllocatePhysicalMemoryEx
and MmFreePhysicalMemory with sizes and balance them -- would settle it, and is
a better use of a run than reproducing the freeze again.

Also recorded so nobody hunts for it: there is NO cvar for guest memory size.
memory.cc has only protect_zero / protect_on_release / scribble_heap and the
MMIO ones, and xboxkrnl_memory.cc says "We don't support separate devkit
memory, so just ignore this flag".  512 MB is hardcoded to the retail console,
so the freeze cannot be dodged by giving the emulator more -- a fix has to be
the release path itself.
2026-08-26 11:32:34 +00:00
Sylpheed RE agent
1ce7d7c54c re: the freeze's 512 MB is the emulated console's, not the host's
"Allocation failed" invites blaming the box, so I measured the box.  Nothing on
the host side is binding: /dev/shm is 2.0 G at 17% used with 1.7 G free (the
guest memory file is 4.5 G apparent but sparse, only 319 MB of real blocks),
host RAM has 12.3 G available of 15.9, and the container cgroup is at 2.9 G of a
7.0 G limit.

The log's own numbers say where it really is:

    131072 pages x 4 KB = 512 MB      <- the Xbox 360's unified memory, exactly
     28969 pages x 4 KB = 113 MB free
              0x08000000 = 128 MB requested

So "parent heap" is the emulated console's physical memory.  The game runs a
real 512 MB console down to 113 MB free and then asks for 128 MB; the container
is not involved.

Mechanism, recorded as amber rather than settled: 23 and 22 occurrences of
"BaseHeap::Release failed because address is not a region start" in the two
runs, adjacent to the failure (lines 1115-1176 against a failure at 1179).  A
release that cannot find its region start returns without freeing, so each leaks
guest physical pages.  That is inference from adjacency and count -- I have not
sampled free pages over time, which is the test that would settle it.

And the release failing at all is an emulator-side bug, not a game one: the
guest is freeing at an address Xenia's heap does not recognise as a region base.
2026-08-26 11:29:51 +00:00
Sylpheed RE agent
ef8d2ab0d8 re: the freeze is independent of the mask poke -- control run settles it
challenge-mission-gate.md §5.6 attributes the 128 MB heap failure to a careless
cleared-stage mask poke, concludes that poking only real story ids does not blow
the heap, and ends by asking for the control: repeat without the poke.

Ran it.  nav_to_flight.sh gains SYLPH_NO_POKE=1, which skips the write; only
Stage 1 is selectable without it, so the control changes stage too, which makes
the agreement stronger rather than weaker.

    poked    0x0001FFFE   Stage 02   frozen, 128 MB request refused
    control  untouched    Stage 01   frozen, 128 MB request refused

Both logs carry not merely the same error but the same numbers:

    requested 134217728 bytes, parent free 28969/131072 pages

28969 in both, across two stages and two boots.  So the poke does not cause it
and neither does the stage; the guest reproducibly arrives at a 128 MB request
with ~113 MB free.  An identical free-page count across independent runs also
says the allocation pattern is deterministic -- not a race, not host pressure.

The control was verified three ways, because the first attempt was confounded:
two emulators were alive at once (the previous one survived a pkill).  The mask
was read back as 0x0 from the live mapping, the log was confirmed to be this
run's, and the liveness test was repeated after killing the stale process so
exactly one emulator was running -- three frames at rmse 0.00, and an 8 MB slab
of guest RAM with 0 bytes changed over 3 s.
2026-08-26 11:26:44 +00:00
Sylpheed RE agent
78f097ef1f re: the mission freeze is a failed 128 MB guest allocation, not an emulator hang
Reproduced on a Stage 02 run and root-caused.  Two witnesses, both taken while
screen_id.py was calling the screen "flight": three screenshots over 8 s at rmse
0.00 with 0.00% of pixels changed, and a 4 MB slab of guest RAM with 0 bytes
changed over 2 s.  The emulator is not deadlocked -- 399% CPU over 15 running
threads.  It spins while the guest does nothing.

The log stops mid-stage-load on:

    PhysicalHeap::AllocRange unable to alloc physical memory in parent heap
        (requested 134217728 bytes, parent free 28969/131072 pages)
    MmAllocatePhysicalMemoryEx: Allocation failed  Size: 08000000
    Guest attempted to throw a C++ exception!

128 MB requested against ~113 MB free.  So it is not only fragmentation, which
is what "failed to find contiguous range" suggests on its own -- there was less
free memory in that heap than the request needed at all.  Preceded by repeated
"BaseHeap::Release failed because address is not a region start", which
challenge-mission-gate.md already notes leaks the range; a leak that repeats
through a session supplies the mechanism the freeze's variable onset (27, 45,
83, 183 s) needs.

This refutes a standing claim.  challenge-mission-gate.md §5.6 concludes that
poking only real story ids (0x0001FFFE) does not blow the heap.  This run poked
exactly that and hit the same 128 MB failure.  Bounded, though: there the
failure was on entering MISSION SELECT, here MISSION SELECT worked and the
failure came at the take-off load -- so the reading is that the poke value is
not what decides it.  That page's own open question, repeat without the poke,
is now the load-bearing experiment and is still unrun.

Also recorded: a frozen game passes the screen classifier.  A single-frame
statistic cannot distinguish flight from frozen-in-flight, which is why the
entity probes returned 0 definitions, 0 movers and 0 vtable hits with no sign
anything was wrong.  A second frame costs nothing and is decisive.
2026-08-26 11:14:24 +00:00
Sylpheed RE agent
c5b67b17f6 re: three boot-path failures that look like the emulator and are not
Spent a session getting to flight.  Each obstacle presents as "the emulator
died" and none of them is.

  * --audio prevents boot.  run-canary's header already says the flag is not a
    cvar in this tree and that an unknown argument blocks in a message box
    before logging starts.  Measured anyway, because the corpus also holds runs
    that passed it and booted: 3 trials each in BOTH orders, 67 565 bytes of log
    without the flag and 209 with -- and 209 is run-canary's own banner, not one
    line from xenia.  Order was reversed on purpose; this corpus has a standing
    lesson that an A/B from run order is noise.  Eight scripts on branch
    auto/idxd-unnamed-keys still pass it; main and this branch are clean, which
    reconciles August's successes with today's failures.

  * launch_mission.sh's skip_intro deadlocks.  It calls the attract loop a
    "movie" and refuses to tap, and waited out 600 s of unbroken movie verdicts
    before timing out.  nav_to_flight.sh, against the same running emulator,
    reached the main menu in 12 s and flight in 2 min 20 s by tapping A at the
    title.  The "wait it out" premise is wrong: the loop does not end.

  * "EMULATOR GONE at ~40 s" is this project's own Stop hook killing xenia when
    a Claude turn ends.  That is recorded further down this same file and I
    rediscovered it over three boots because I did not look.  Sequential tool
    calls within one turn are fine; ending the turn is what kills it.

The world unit is still unmeasured.  Flight was reached and the screen
classifier agrees, but entities2.py finds 0 unit definitions -- its committed VA
window does not match this run, the same run-dependent-address problem this file
documents for the OB counter.  Next attempt must hunt the range.
2026-08-26 11:02:15 +00:00
Sylpheed RE agent
b712972d09 docs: define the status markers, and fix six mislabelled or superseded entries
An audit of BACKLOG.md turned up a class of error with a single root cause: the
README defines only the CONFIRMED/PROBABLE/HYPOTHESIS confidence scale, while
the pages actually use a second vocabulary -- and 🔴 appears 98 times without
ever being defined.  It gets used for two different things, "refuted" and
"blocked", and three entries slid from one into the other.

README now defines /🟡//🔴//🚧 and states the rule the corpus was missing:
🔴 never means "we have not run it yet".  That is  or 🚧.  Its blocked sense is
only for a real limit of the box -- no push credentials, no hardware Vulkan, a
decision only the user can make -- and since the box can run the emulator,
script input, screenshot and read guest memory, "needs a run" is never blocked.
I made exactly this mistake on the world-unit item earlier today, which is what
prompted looking for others.

Fixed in BACKLOG.md:

  * the elimination test, marked 🔴 UNRUN and in fact run and refuted nine
    lines further down;
  * the frozen capture, marked 🔴 STILL UNRUN and in fact taken eleven lines
    down -- 🔴 wrong twice, since "the freeze did not happen this run" is a
    scheduling outcome and not a refutation;
  * a 🚧 STILL UNRUN item whose stated blocker (the boot-nav bug) is fixed;
  * the objective-counter heading, which asserts 0xbdb59668 as the answer while
    its own first body line refutes that address -- retitled to say what is
    actually solved, the method;
  * the paint-order "third measured permutation" question, answered inside its
    own entry by a third, fourth and fifth screen;
  * the UTF-16 endianness question -- resolved, and it is not a stale comment:
    localization.rs both documents LE and decodes with u16::from_le_bytes, so
    it is a code bug worth filing.

Also fixes the corpus's only dangling link (INDEX.md pointed at
structures/idxd-unnamed-keys.md, never written).
2026-08-26 10:29:15 +00:00
Sylpheed RE agent
76a998eed4 re: correct a mislabelled blocker -- the world unit needs a run, not a waiver
I wrote "blocked on the oracle" for the unit-to-metre conversion.  That was a
mislabel: red is for what the container cannot do, and run-canary works here.
What the km-name sweep actually established is narrower -- no STATIC test can
settle it, because the disc has exactly one size-bearing asset name.

The run is well-supported by tooling that already exists: findplayer.py
recovers the player position triple from motion, the HUD prints the distance to
the selected target in the game own units, and the same separation read both
ways is the conversion.  Recorded as amber with the experiment written out.
2026-08-26 10:07:01 +00:00
Sylpheed RE agent
0a64728fae re: retract the paint-key census and redo it over all 21 184 sprites
The census filtered pak entries whose own first four bytes are T8aD.  A sprite
is usually a child of a RATC bundle, and a bundle entry's magic is RATC, so a
top-level magic filter cannot see one:

    top-level T8aD entries (counted)    4 525 sprites,  45 keys
    T8aD inside RATC bundles (missed)  16 659 sprites, 204 keys
    both                               21 184 sprites, 216 keys

171 of the 216 keys exist only inside bundles.  The sharpest statement of the
error: that census never saw GP_TITLE.pak at all -- the pak holding both of the
screens this page's entire evidence comes from.

Retracted: "45 values", "the keys are pak-local", "each auxiliary pak occupies
its own narrow high-byte band".  On the full population 68/216 keys (31%, not
9%) cross a pak family and the per-pak ranges overlap heavily -- GP_BUNK
0x8000-0xa110, GP_TITLE 0x8000-0xc150, GP_LEADERBOARD 0x8000-0xf100.  The tidy
banding was an artifact of seeing one or two keys per pak.  So the key looks
like a shared vocabulary, which is the opposite of what I published.

Survives, now on the full population: the field is a u16 at +0x0A (upper half
zero 21 184/21 184), and it is an enumeration (216 values for 21 184 sprites).

Three wrong numbers on this page now, all the same shape -- a statistic computed
over a population I had not checked was the population in question.  Stated once
at the end of the section rather than three times: check the sampling frame
before the statistic.
2026-08-26 10:06:23 +00:00
Sylpheed RE agent
dbeab6aa9c re: flag that the paint-key census may have sampled the wrong sprite population
The census filters pak entries whose own first four bytes are T8aD.  The
sprites this page measures paint order on are children of a RATC bundle --
ui_layout.rs reaches them via ratc::parse, and a bundle entry's magic is RATC,
so a child T8aD never matches a top-level magic filter.

So the 45 keys may describe a population that only partly overlaps, or does not
overlap at all with, the one the page's two measured screens come from.  I do
not yet know which; the comparison is running.  Marking the section rather than
leaving the counts to be read as covering the screens' sprites.

Same failure shape as the 37/45 language-duplication note lower down the page:
a number computed over a population I had not checked was the population in
question.  Recording it as such.
2026-08-26 09:58:32 +00:00
Sylpheed RE agent
fdcb5e94e0 re: census the T8aD paint-order key -- it is pak-local, not a global vocabulary
The page rested on twelve values from two screens.  This walks all 4525 sprites
on the disc.

  * The field is a u16 at +0x0A.  The upper half of the 32-bit word the page
    reads is zero in 4525/4525.  Nothing above changes -- 0x00008100 sorts the
    same as 0x8100 -- but a future value with the high half set would mean
    something had been misread rather than that the layer got deeper.
  * It is an enumeration: 45 values for 4525 sprites, one of which (0x8100)
    covers 1188 of them.
  * The reading worth trying -- a global layer vocabulary shared across the UI
    -- is refuted.  Only 4 of 45 keys cross a pak family and 33 of 45 live only
    in GP_MAIN_GAME_2D; every other pak owns a narrow high-byte band (0x90-0x94
    for the in-game overlays, 0xa4 mission log, 0xb1-0xb2 save/load).  A screen
    that owns one or two keys is not ordering itself with them.

That supports "group id in the high bits, order in the low bits", which is what
the page already suspected, but it does NOT test it: paint order has been
measured on two screens and both are inside GP_MAIN_GAME_2D, so there is no
ground truth to check the split against.  Left amber.

The first number I got was 37/45 shared, which would have supported precisely
the wrong conclusion.  It came from counting paks instead of pak families: the
six GP_MAIN_GAME_*2D paks are the same screens in six languages and their key
sets are byte-for-byte identical.  Recorded on the page, because the shape
recurs -- a corpus with near-duplicate members manufactures agreement.
2026-08-26 09:50:29 +00:00
Sylpheed RE agent
97d02f0092 re: confirm the cmesh<->model link via GameResourceID; world unit is oracle-blocked
Two follow-ups on yesterday's^Wthis morning's CollisionSet write-up.

1. The _cmesh <-> render-model link, which I recorded as UNTESTED because
   matching stems against .xbg object names covered 4 of 158.  The disc keeps
   only one build manifest, so that corpus was never going to answer it.  The
   right corpus is the GameResourceID field of the DefTables / GP_MAIN_GAME
   records -- 480 distinct values.  Against those, with a control that shuffles
   the characters of each stem:

       ship/mob stems prefixed by a real resource id   108/112 = 96.4%
       same stems, characters shuffled  (control)        0/112 =  0.0%
       asteroid stems prefixed (expected none)           0/46

   So a CollisionSet entry is <GameResourceID>[_<part>]_cmesh.  The 0/46 on
   asteroids matters as much as the 108/112: a test that fired on everything
   would be the bound-check hazard again.

2. The world unit.  Sweeping every pak for a name carrying a kilometre figure
   returns mapmesh_box_500km.col/.rgn and nothing else -- 162 references, all to
   that one pair.  The reading rests on a single filename with no corroborating
   instance anywhere in the data, so no static test can settle it; marking it
   blocked on the oracle rather than leaving it as an open static question.

   My objection's premise did survive: rou_e010 is a real GameResourceID and
   e010_ADAN_Attacker_S is in the stage tables, so the 133-unit mesh does belong
   to a craft the game calls an attacker.  Whether the trailing _S means "small"
   is a further guess (there are _EX4 / _HF / _HF_Wayne variants), so it stays
   suggestive rather than evidence.
2026-08-26 09:41:46 +00:00
Sylpheed RE agent
d677bcfd80 re: decode CollisionSet_*.bin -- the per-object collision-mesh library
All 18 blobs are byte-identical: the per-stage naming is nominal, and every
stage points at one shared 1675148-byte library stored eighteen times.  That
identical size was the reason to open the item, and it turned out to be the
answer to it.

Record layout: {u32 size, u32 name_len, char name[name_len], u32 nv, u32 nt,
f32[3] x nv, u32[3] x nt}, next record at off + 8 + size.  The indices are u32
here where MCOL uses u16 -- two different serialisers in one archive.

What makes this a decode rather than a plausible reading: the walk consumes the
file to the byte over 158 variable-length records, with the size word predicted
from the two counts 158/158.  A wrong field would desynchronise within a few
records and could not land exactly on the end.  All indices in range 158/158;
98.24% of edges shared by exactly two triangles; 147/158 fully manifold.

158 meshes, 90 836 triangles: per-part ship proxies (_bdy/_brg/_eng/_wep/_sld,
the XBG7 sub-part vocabulary) plus 46 stage asteroid meshes whose prefixes are
exactly the stages that have an _AsteroidVolume_wp MCOL.

Two things this file makes me walk back:

  * The "1 unit = 1 metre" reading from mapmesh_box_500km is downgraded to
    amber.  The 500000 arithmetic stands, but it implies that a craft the game's
    own tables call "small" is 133 m and that rob_f002 is 447 km -- 89% of the
    arena width.  The format check survives; the interpretation has no
    independent support.
  * The _cmesh <-> render-model name link is recorded as UNTESTED, not
    confirmed: only one .xbg build manifest survives on the disc, so matching
    stems against object names covers 4 of 158, which is no coverage at all.
2026-08-26 09:31:22 +00:00
Sylpheed RE agent
80ae860e19 re: record the MiscBin name resolution in the backlog 2026-08-26 09:21:15 +00:00
Sylpheed RE agent
fa5a51ac78 re: name-resolve all 40 MiscBin entries, and demonstrate the REGN<->MCOL pairing
The names live outside MiscBin: they are the MapPath / MapMesh /
CollisionMeshes field values of the per-stage StageResource object (IDXD schema
3c9ae32e, in every GP_MAIN_GAME_<lang>.pak), and each hashes with the ordinary
pak name_hash straight to a TOC entry.  40/40 resolve, no collisions -- the 11
REGN as <stem>.rgn, the 11 MCOL as <stem>.col, and the 18 remaining blobs as
CollisionSet_S01..S16 / _Tutorial / _test.bin.  The .pe string table at 651540
was the way in: MapMesh and MapPath sit adjacent there.

This upgrades the pairing claim.  The first section of mcol-collision.md could
only say REGN and MCOL had matching *distributions* of bbox and cell size, and
flagged that as not an object-to-object link.  A phase record names one .rgn and
one .col, and all 11/11 pairs share a stem and agree exactly on both.

The names also check the format work from outside it: mapmesh_box_500km.col is
the object decoded here as 8 vertices and 12 triangles spanning exactly
+-250000, and its name says that cube is 500 km across -- so one world unit is
one metre, and a wrong stride could not have produced a box that measures what
its own filename claims.  70 of the 87 phases use it: most stages' only
collision is the arena wall, and _AsteroidVolume_ names the rest.

Still open: the 18 CollisionSet_*.bin are named but not decoded (all exactly
1675148 bytes), and CMapColliderBridge in the RTTI names the runtime consumer
without following it into the code.
2026-08-26 09:21:00 +00:00
Sylpheed RE agent
c0a7033295 re: MCOL solved -- a closed triangle collision mesh in a uniform grid
The 0x50 header word, which the first section of this page had dismissed as "a
large value", is two u16 counts: vertices and triangles.  They give the two
remaining blocks their stride, and every derived length is exact in 11/11 --
len(0x54) == align16(12*nv), len(0x58) == align16(6*nt), and nt equals the
bounding-sphere count decoded last iteration.

Checks that cannot pass by accident:

  * sphere i is the TIGHT bounding sphere of triangle i, 4768/4768, with
    max|v-c|/r median 0.99990 (a fixed 1.0001 epsilon), against a 1.32%
    random-triangle control;
  * the mesh is watertight -- every edge shared by exactly two triangles,
    7152/7152, zero degenerate triangles, zero unreferenced vertices;
  * the two smallest objects are 8 vertices and 12 triangles whose positions
    are the eight +-250000 corners of the map bbox: a bare bounding cube.

The cell lists are a correct broad phase: with an exact triangle/box SAT test
only 3 overlapping triangles in 18 577 entries are absent, so a query walking
one cell's list cannot miss a hit.  The 730 conservative extras bracket the
builder's own test between exact-SAT and AABB, which retires the 18 unexplained
"sphere misses" from the previous commit as that same margin.

mcol_probe.py gains `mesh` and `obj`; `verify` now runs all three checks and its
output is recorded in docs/re/data/mcol-verify.txt.
2026-08-26 09:12:36 +00:00
Sylpheed RE agent
4f0d21f50d re: MCOL's 0x5C block is bounding spheres at stride 16 -- the 0.75 was 12/16
The unexplained ~0.75 ratio left at the end of the last iteration was my own
stride.  I had read the block as 12-byte points because REGN's vertex section
is 12 bytes, and never checked it: len(0x5C) is not a multiple of 12 in 5 of
the 11 objects, so that stride was never arithmetically possible.

At stride 16 the relation is exact in 11/11 -- max u16 == len(0x5C)/16 - 1 --
and the record reads as {centre f32[3], radius f32}.  Powered test, since a
u16 is reached through a specific grid cell: the sphere it names reaches that
cell in 18 559/18 577 = 99.90%, against a 12.02% random-sphere control.  Both
fields carry signal (centre alone 26.75%, radius shuffled 70.19%).

The converse -- is the list *exactly* the intersecting set? -- is 0.38%, which
is the expected direction: a bounding sphere is conservative, so membership
implies overlap but not the reverse.  The tighter geometry is in 0x54/0x58,
still undecoded.  18 entries (0.10%) go the wrong way and are recorded as open.

tools/re-capture/regn_decode.py is copied unchanged from auto/regn-reader so
the probe's POF0 reader is the known-good one rather than a second copy.
2026-08-26 09:04:30 +00:00
Sylpheed RE agent
3274ee6d8e re: MCOL's u16 entries do not index the point block -- the bound-check was a false positive
Two tests. Counts modulo 3 are spread across all three residues (639/2063/1786),
so the u16 array is not a triangle list. And a B record is reached through a
specific cell, so a point it references should lie in that cell -- referenced
points score 0.79% against a 0.48% random-point control. Chance.

The contrast is the point. One section earlier the same u16 entries scored
18379/18379 (100%) on 'are these valid point indices'. I flagged that at the
time as the weak bound-check and recorded it as consistent rather than as a
finding. The caution was right: the powered version of the same question now
returns chance, and had the 100% been written up as the decode this page would
carry a confident false statement about MCOL's geometry.

Fifth appearance of the pattern across REGN and MCOL and the first time both
halves have been run on the same field, so the page now states it plainly: a
bound-check asks whether something could be an index, and the answer is set by
the size of the target collection rather than by the field's meaning.

Datum for the next attempt: the maximum u16 is consistently about 0.75x the
point count (923/1232, 1019/1360, 1163/1552, 59/80), too consistent to be
coincidence and not explained.
2026-08-26 08:52:44 +00:00
Sylpheed RE agent
8c57c0e621 re: MCOL array B decoded -- {count, u16 index array}, and the chain closes
Two corrections got there. MCOL has 9020 relocated words and only 4488 are
A-record pointers; I assumed the rest sat at B+8 mirroring A, refuted 0/4488.
Measuring their offset from the nearest preceding B record gives B+4 for 4488
(99.0%) and 44 before the first B -- exactly the four header pointers times 11
objects. Nothing unaccounted for.

So B is {u32 count, pointer at +4}. The pointers advance by exactly twice the
count: 4477/4477 (100.00%) over all 11 objects. That is a packed u16 array with
no padding, and it is the load-bearing evidence -- an exact arithmetic identity
over 4477 consecutive pairs. The companion check that those u16s are valid point
indices is the same weak bound-check flagged earlier and is recorded as
consistent rather than as evidence.

Chain: position -> cell -> A {cell index, count 1, ->B, sphere} -> B {count n,
->u16[n]} -> n indices into the point block. Same shape as REGN's cell -> item
-> refs -> geometry.

Open: the 0x54 and 0x58 blocks, which this chain never reaches, and what the
indexed points form.
2026-08-26 08:45:58 +00:00
Sylpheed RE agent
25ccb6723a re: MCOL A->B is 1:1 with count 1, matching REGN's cell-index convention
Filtering array A by the cell-index criteria and following each pointer: every A
record points at a distinct B record, 11/11, and every A count field is exactly
1, 11/11. Per object the A-record count is the number of occupied cells (110 to
575) and the counts sum to it exactly.

That is the same design REGN uses -- the corpus already records 'every occupied
cell has count exactly 1' there. Two sibling formats, one convention, and a
further independent confirmation of the A reading since the filter and the
cardinality are unrelated criteria.

Array B resisted, and both attempts failed in ways worth recording. The record
boundary was off by 8 again, producing records that start with the tail of the
previous structure -- the same mistake as the 0x74 check two iterations ago. And
the u16-index test had no power: the 0x5C block holds ~1232 points, so 'is this
u16 below the point count' passes for almost any small value, and duly reported
100% at seven offsets. Fourth time in these two formats that a bound-check
against a large collection produced a meaningless 100%.

Recorded what would have power instead: B records are 1:1 with occupied cells
and A carries that cell's bounding sphere, so a B field can be tested for
spatial consistency with that specific cell.
2026-08-26 08:38:46 +00:00
Sylpheed RE agent
c8d3893bb8 re: MCOL array A confirmed as the per-cell record -- 100% once split
Separating the two interleaved arrays by address and re-running the same three
criteria: array A (2509 records) gives byte3==1 at 100.00%, a valid cell index
at 100.00%, and the sphere reaching that cell at 99.92%. Array B (6467) gives
30.65% and 30.60% -- a different record type, and the control showing A's 100%
is not what any 32-byte block would score.

So array A is the per-cell record: cell index (x,y,z), count, pointer into array
B, bounding sphere -- the same role REGN's section 3 plays. That confirms the
earlier 50% was the interleaving artifact and not a half-working reading.

The split was crude, first-half-by-address giving 2509 vs 6467 rather than an
even cut, and A still came out at 100%. A rough partition isolating a perfect
population is stronger than a careful one isolating a good-ish population.

Array B's layout is still unread.
2026-08-26 08:32:41 +00:00
Sylpheed RE agent
900ca6a807 re: MCOL's 32-byte record is a cell entry, and there are two interleaved arrays
Read as 8 big-endian words, word 0 as four bytes is (x,y,z,1) -- a 3-D cell
index, matching the object's 5x5x5 grid. Words 4-6 are a position and word 7 a
positive scalar (a bounding sphere); word 1 is a count, word 2 the relocated
pointer. Every record pointer lands in the same region, 8976/8976, each a fixed
distance on with the same stride -- so there are two parallel arrays, A and B.

The 50% is the tell. Three independent criteria -- byte 3 == 1, valid cell
index, sphere reaches that cell -- all land on 50.0%, which says half the
records are not this type rather than that the reading half-works. The POF0 slot
list interleaves both arrays and I was testing B's records against A's layout.

So array A is a per-cell record, the same role REGN's section 3 plays. Array B
is unread, and the criteria have not yet been re-run on A alone -- if the
reading is right they should go to 100%.
2026-08-26 08:31:46 +00:00
Sylpheed RE agent
8df3624b70 re: MCOL's pointer layout, read from POF0
Used the known-good decoder rather than my own broken one. My version had three
errors: the delta stream starts at table+16 not +8, the tag bits are
0x40/0x80/0xC0 rather than 0/1/2, and slots carry the +16 fixup base. Sanity
check passes -- on REGN the tool returns header slots 0x70-0x84 exactly.

On MCOL, over all 11: the header-region relocated slots are exactly 0x54, 0x58,
0x5C and 0x74 (11/11); 0x5C resolves to 0x80, the first byte after the header
(11/11); and 0x74 resolves to eight bytes before the first array pointer (10/11).
So MCOL has four top-level pointers where REGN has six.

92.7% of gaps between consecutive relocated words are 32 bytes, in 7-127
contiguous runs per object, which with the 0x74 offset reads as arrays of
32-byte records each carrying a pointer at +8.

The four targets are two float blocks, a block of small ints / u16 pairs, and
counts followed by the record array -- the shape of a mesh, stated as a reading
of the shape since none of the blocks is decoded.

Open: the record layout, what the index block indexes, the one object where 0x74
does not land 8 before the array, and whether the 7.3% non-32 gaps are just run
boundaries.
2026-08-26 08:25:29 +00:00
Sylpheed RE agent
e31c3a1854 re: open MCOL -- same container as REGN, same map parameters
MCOL sits beside REGN in hidden/MiscBin.pak, 11 of each, never decoded. Over all
11: POF0 at data_size+16 11/11, bbox pad words 1.0/1.0/0.0 11/11, and
extent == max-min 11/11. So the header prefix is the same shape as REGN's and
the POF0 mechanism applies, which means the chunk+0x10 base and the loader's own
pointer list -- the two things that cracked REGN -- are available here too.

The map parameters are not merely similar but identical in distribution: bboxes
2/6/3 at 250k/50k/25k and the 0x40 triple 2 at 50000 and 9 at 10000, matching
REGN exactly. Eleven maps, each with an MCOL and a REGN over the same volume at
the same cell size. Noted that this matches distributions, not a demonstrated
object-to-object pairing.

Everything past 0x40 diverges from REGN and is open. Also recorded that my own
POF0 delta decoder is wrong here -- eight leading zero deltas -- and that the
working one is regn_decode.py on auto/regn-reader, which should be used rather
than re-derived. Re-deriving it is the mistake I made.
2026-08-26 08:18:17 +00:00
Sylpheed RE agent
7a351e641d re: REGN is excluded as the wave scheduler -- it is a navmesh
The original reason for investigating REGN was that a mission's enemy count
rises and falls, so a scheduler with parameters must exist somewhere, and a
per-map uniform grid is what such a thing would be indexed by.

Now that it is decoded that reasoning is answered: REGN is a tetrahedral
navigation mesh -- vertices, faces carrying plane equations and adjacency,
tetrahedra with portal costs between face pairs, and a grid indexing which tets
fall in each cell. Every section is accounted for by that structure, and there is
no time field, no unit reference and no trigger anywhere in it.

So the wave-scheduler search should treat REGN as excluded rather than unread.
The page's original hedge was right to keep the reading provisional, but the
reasoning it hedged was a guess from shape, and the shape belonged to
pathfinding -- which is what pointed the whole investigation here.

The arrival timetable in Route_S<NN>.tbl, keyframed per squadron per phase with
t in seconds, remains the only located part of the mechanism.
2026-08-26 08:11:36 +00:00
Sylpheed RE agent
6c47f8231c re: independently confirm the REGN face decode -- 133573/133573 against a 0.69% control
Re-derived the other branch's central check with my own code, my own reading of
the record and my own control, rather than accepting the number.

At base chunk+0x10, with the plane at intra-record +0 and three u16 vertex
indices at +32: every one of 133573 faces has all three named vertices
satisfying its own plane equation. Random-vertex control 2782/400719 = 0.69%.
100% against 0.69% is not a fit.

That also settles the record boundary from my side: the u16s describing a plane
sit after it in the same 48-byte record, so my earlier 'four zeros at the start
of each record' was those integers seen 16 bytes out of position, one record
late.

Two implementations, two independent guesses at the intra-record layout, the
same 100%.
2026-08-26 08:06:30 +00:00
Sylpheed RE agent
a2acd56407 re: backlog said REGN's coupling was unreachable statically -- it has been reached
I recorded the static coupling search as exhausted and needing PE code. The PE
work was done on auto/regn-reader and it solved the whole thing: REGN is a
tetrahedral navigation mesh, reached via the POF0 fixup table -- the loader's own
list of which words are pointers, so nothing needed guessing. Six sections;
position -> cell -> 32-byte item -> tet refs -> tetrahedron, with section 2 a
face carrying a plane plus its 3 vertices and the two tets either side.

Controlled checks: face through 3 of 4 tet vertices 253722/253722 against a
0.07-2.2% random control; portal cost equals face-centroid distance
380460/380460.

Also records against myself that the base is chunk+0x10 and my offsets here were
16 bytes early, that the plane arithmetic survives only because those fields
landed on the same bytes, and that my points-in-bbox count was never evidence --
a shift inside a homogeneous f32 array yields other floats from the same array.

Leaving the wrong conclusion in the backlog would have told the next reader the
avenue was closed when it was the one that worked.
2026-08-26 08:05:31 +00:00
Sylpheed RE agent
f3c4cbb7e9 re: resolved -- the REGN base is chunk+0x10 and my section-0 offsets were 16 bytes early
The other branch supplied concrete offsets: for 3506e972 its face record 0 and
its plane normal both begin at 0x1c700, and my chunk + offset_at_0x78 + 16 gives
0x1c700. Same bytes, different bookkeeping -- so the n.p+d result stands
unchanged and was never in dispute.

The base is chunk+0x10, on evidence with power: the loader does addi r3,r31,16;
at +0x10 the six POF0-relocated slots land exactly on 0x70-0x84, the six section
pointers, whereas at +0 they would relocate the u16 counts and leave two section
pointers unrelocated, which is non-functional; and section-0 record 0 reads as a
bbox corner at +0x10 and garbage at +0.

So my '13467/13467 points inside the bbox' was vacuous. Only 11 of 13467 read as
denormal at the wrong base -- the rest were still plausible coordinates, because
a 16-byte shift inside a packed array of f32 triples yields other floats from
the same array. Recorded the general form: a containment test cannot detect a
shift inside a homogeneous array, because the shifted values come from the same
distribution. For that class of error it is not a weak check, it is no check.

'Section 0 is a point list' happens to be right; the evidence I gave for it was
not evidence.
2026-08-26 08:00:18 +00:00
Sylpheed RE agent
0cbef2023f re: withdraw "everything was read 16 bytes early" — the planes were right
The coordinator refuted the broad form of my claim and was correct to. The
older reading put section 2 at chunk+offset with the normal at intra-record
+16, which addresses exactly the same bytes as chunk+0x10+offset with the
normal at +0: on 3506e972 both put face record 0's normal at 0x1c700. Its
unit-normal and n·p+d results were measuring real planes and stand.

What is actually wrong is the record BOUNDARY, and it decides which record the
four integer words belong to — the face adjacency, i.e. the field the whole
correlation search was looking for. Settled: requiring own==f and
tet[A].faces[i]==f gives 100.000% with the block from record f and 0.000% from
record f+1, on all 11 objects.

Also audits my own numbers for the same defect the coordinator found in the
point-in-bbox test: most of them are content tests that address identical bytes
under either convention and have no power over the origin. The four that do
have power are listed, including that POF0 at base+0 would relocate the u16
counts and leave the section pointers unrelocated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 07:58:45 +00:00
Sylpheed RE agent
81d6bb4bed re: record an open conflict with auto/regn-reader over the +0x10 base
That branch decodes REGN as a tetrahedral navmesh with strong checks and claims
the POF0 fixup base is chunk+0x10, so every offset on this page was read 16
bytes early.

I could not reproduce that on the one independently checkable thing here: the
plane list gives 133573/133573 unit normals at the unshifted base and 0/133573
at +16, and n.p+d = 0 holds to float round-off unshifted. A 16-byte shift
destroys it, so the blanket statement does not hold for this record.

Likely reconciliation is bookkeeping: a 48-byte face whose plane fields sit at a
different intra-record offset addresses the same bytes from a different origin.
That is a guess and I am not adopting either wording until checked.

Also confirmed: my own section-0 point test passes at 100% at BOTH bases, so it
never had power to distinguish them and should not be cited as validating the
offsets.
2026-08-26 07:55:12 +00:00
Sylpheed RE agent
50625b5a9e re: REGN is a tetrahedral navigation mesh — the cell→geometry link, decoded
Found the reader. It is the deserialiser, not a consumer, and it answers the
question twice: the fixup base is chunk+0x10 (82465198 addi r3,r31,16), so
every offset previously recorded was read 16 bytes early — which is why twenty
correlation tests sat at chance — and the POF0 table names every pointer word
in the file.

Six sections, not four. cell {count,item*} → item {n@+0x10, refs*@+0x14} →
array of pointers into section 1 → a 96-byte tetrahedron. Section 2 is a face:
plane, its three vertices, the two tetrahedra either side (0xFFFF = hull) and
their face slots.

All 11 objects: face passes through exactly 3 of its tet's 4 vertices in
253 722/253 722 (random control 0.07–2.2 %); portal cost == face-centroid
distance in 380 460/380 460; sphere reaches its cell 98.7–100 % vs 18–28 %
with transposed axes.

Refuted and kept: 'REGN'/'MCOL' are never built as constants in the executable
(0x474E occurs zero times in 1.87 M instructions), so no magic-dispatch site
exists; and "zero portal-pair float marks a hull edge" shows no lift at all.
Still open: the runtime consumer of the grid, the second portal float, and the
four flag bytes at tetrahedron +0x54.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 07:53:13 +00:00
Sylpheed RE agent
e5c6c27e6a re: regn_decode.py — read a REGN object through its own POF0 fixup table
The retail deserialiser (sub_82465110 / sub_82465138 / sub_82465200) relocates
a chunk with a fixup base of chunk+0x10, and the POF0 table it walks is an
exact list of which words are pointers. Decoding that table gives the pointer
graph with no guessing.

`verify` reproduces every number quoted in the doc, each against a control:
pointer-slot shape, reference-array packing, face/vertex incidence vs a random
face, cell agreement vs transposed axes, and the portal-cost identity.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 07:53:13 +00:00
Sylpheed RE agent
59c659460f re: correct my last two savegame sections -- they analysed the COMPRESSED stream
Both preceding sections split the file at 0x88 and called everything after it
the slot table. Wrong: this page already documents the container as GDHA + a
146-byte header + a zlib stream, and savegame.rs implements it. 0x88 is where
the Z1/zlib payload begins (5a31 = 'Z1', 78da = zlib), so the bytes I treated as
slot fields are deflate output.

Invalidated: the '6 pointer-shaped words in the slot region' are not words and
not pointers, so the false-positive rate I derived from them measured nothing.

Survives with different reasoning: 'the slot region is byte-identical' is true
because the two COMPRESSED streams are identical, which does imply identical
payloads -- and inflating both confirms it, 545 bytes byte-for-byte equal.

Stands and is better founded: all 12 differing words are in the header, which is
the only uncompressed region and therefore the only place a word-wise diff means
anything.

Done correctly, the result is a confirmation rather than a discovery: inflating
all three payloads reproduces the documented clear-ratio field stepping 5 -> 6
on the developed save.

The lesson: I found Z1/78da by inspection and nearly wrote it up as new. It was
already decoded four sections above where I was appending. Reading the whole
page first would have saved two wrong commits, and the wrongness was not
cosmetic -- a pointer census ran over deflate output and produced a
plausible-looking table.
2026-08-26 07:50:29 +00:00
Sylpheed RE agent
c1d01ad36a re: all savegame variation is in the header; slot region identical -- and my heuristic overstates
Splitting the same comparison at the documented slot-table boundary (save+136):
the header holds 34 words and ALL 12 differing words, while the slot table holds
35 words and ZERO differences. The slot region is byte-identical between the two
same-state saves, so the documented slot fields are untouched by the churn and a
slot-region diff is meaningful where a whole-file diff is not.

Also corrects my own framing from the previous commit. The slot region contains
6 words in the pointer value range that do NOT change across runs -- a pointer
captured from a moving heap would not sit still, so those are data whose values
merely land in 0x70-0x8F / 0xB0-0xBF. The value-range test alone does not
identify a pointer; a third of what it flags here is not one. What identifies a
pointer is being pointer-shaped AND varying, ideally with a shared constant
delta. So '26% of the file is heap addresses' overstates it and the defensible
figure is the 8 header words that are both.

Left the earlier section in place with the correction after it: a value-range
classifier on 32-bit words always has a false-positive rate, and quoting its raw
count as a fact about the format is the error.
2026-08-26 07:44:55 +00:00
Sylpheed RE agent
5d49f54c92 re: the save embeds live guest pointers, and a three-way diff of the captures is invalid
Comparing the two same-state saves (both 276 bytes), only 12 of 69 u32 words
differ, and nine of those carry guest addresses -- 0xBC/0xBD/0x70 prefixes, the
same regions the runtime work uses. Three differ by exactly 0x101080 and two by
exactly 0x300000: a shared constant offset is what a relocated heap does to a
pointer and not what data does.

The split is sharp: pointer-shaped words are 26% of the file and 44% of them
differ; non-pointer words are 8%. So a quarter of this structure is captured
heap addresses that change run to run regardless of play. That is the concrete
form of the page's existing 'much of the rest is uninitialised memory', now
demonstrated by constant deltas rather than inferred from odd-looking values.

Also records that my first pass was wrong: a byte-wise three-way diff reported
135 of 276 bytes differing, but game03 is 280 bytes with a ZERO-byte common
suffix, so offsets do not correspond past 0x00A. The valid equal-length pair
gives 30 bytes in 12 words. The inflated figure looked plausible -- half a save
changing is what dense state would do -- which is why it is written down.
2026-08-26 07:39:13 +00:00
Sylpheed RE agent
89ceb1e97e re: REGN cell payload indexes neither points nor planes -- static search exhausted
The previous test only tried section-1 targets. Closing that gap: the payload's
three index-shaped u32s, followed into the point list and the plane list and
checked for the target lying inside the referencing cell, all sit at the 0.203%
random control.

Two cells read 0.81%, 4x the baseline. I am not treating that as a lead: across
this and the previous iteration roughly twenty such tests have been run, and at
that count a single 4x enrichment on ~8000 trials is what noise looks like.
Calling it a signal would be the multiple-comparisons error a long hypothesis
sweep invites.

So REGN's header, grid, points, planes and cell index are decoded, section 1's
slot regions are censused, and the link between the grid and the geometry is not
reachable by any static test I can construct. The honest next step is the PE
code that reads a REGN object -- the same kind of work that cracked the .slb
packing phase -- rather than a twenty-first correlation.
2026-08-26 07:32:08 +00:00
Sylpheed RE agent
acd603fc76 re: the REGN cell index does not reference section 1 -- two powered tests
The natural coupling in a file with a uniform grid and a list of small volumes
is that the grid indexes the volumes. Tested by spatial agreement, it does not.

Test 1: every u16 in a cell's 32-byte payload, tried as a section-1 index and
checked for its centre lying inside the referencing cell. Every field sits at
the 0.138% random-control rate.

Test 2: every float triple in the payload, checked for lying inside its own
cell. 0.15-0.81%, also chance.

Recorded a worthless number from the same run rather than dropping it: those
triples lie inside the object's BOUNDING BOX in 100.00% at five different
offsets. The bbox spans the whole 500 km map so any mid-range triple passes, and
overlapping windows at +0 and +4 both scoring 100% is the tell -- a real field
would not survive a four-byte shift. Third time in this investigation that a
containment test against something large has produced a meaningless 100%.

Incidental and real: u32 slots at +0, +8 and +12 are below 0x10000 in 100% of
payload records while +4/+16/+20/+24 are in 11% and +28 never, so the record has
three index-shaped fields and four wide ones.
2026-08-26 07:26:45 +00:00
Sylpheed RE agent
c6014c5837 re: refute my own BVH reading of REGN section 1
Last iteration I offered 'position + scalar + integer links is the shape of a
BVH node' as a reading of the shape. Tested properly it fails.

Following every u16 half of every integer slot and checking child-sphere-inside-
parent-sphere gives 0.00% for every candidate -- but the informative number is
the RANDOM control, also 0.00%. No node's sphere contains any other node's
sphere anywhere in the file, so there is no nesting for an index to point at and
the hypothesis dies before the indices matter.

The reason is scale: slot 7 has a median of 3139 against a median inter-node
distance of 45457, 14x smaller, and a random other centre falls within it 0.40%
of the time. It is also smaller than the smallest grid cell on any map.

So slot 7 is a LOCAL scale, not a hierarchy radius. 63410 scattered centres each
with a sub-cell extent is the shape of many small independent volumes, which
would fit per-object collision hulls for asteroids and debris -- a reading, not
a measurement.

What this removes is a wrong frame: the file is not a tree, so tree-shaped tests
will keep returning nothing.
2026-08-26 07:20:39 +00:00
Sylpheed RE agent
b61351cf94 re: REGN section 1 slot census, and an index test with no power
96 bytes is 24 slots. Over all 63410 records: slots 4-6 hold values in the
header bbox range (a position), slot 7 is always positive 519..107600 (a radius
or extent), slots 8-11 are DENORMAL as floats -- 1.4e-45 upward -- so they are
integers a float reader would turn into near-zero garbage, and slots 12-23 are
six pairs with distinct even/odd distributions. Slots 2 and 3 are ~always zero.

Position + positive scalar + integer links is the shape of a BVH node, which
would fit a file carrying a point list and a plane list. That is a reading of
the shape and nothing more.

Recorded a failed test and why it failed: splitting the integer slots into u16
halves and checking them against each section's record count accepts ALL THREE
sections at ~100% for slots 8 and 9. A test that accepts every hypothesis
rejects none -- section 2 has tens of thousands of records, so the check
measures the section's size rather than the field's meaning. Slot 11's halves
are consecutive in 54%, which is suggestive and not a rule.

What would settle it is a test with power: follow a candidate index and check
the target is spatially consistent with the record's own position and radius.
2026-08-26 07:14:39 +00:00
Sylpheed RE agent
f481ccbb5f re: REGN sections 0/1/2 have strides, and section 2 is a plane list
The three sections recorded as undecoded are fixed-stride arrays and counts[0..2]
are their record counts: 12, 96 and 48 bytes. Section 1's remainder is exactly 0
in 11/11 objects and section 2's exactly 96 in 11/11, which is what makes these
strides rather than a coincidence of division.

Section 0 is a point list: 13467 of 13467 records lie inside their object's own
header bounding box.

Section 2 is a plane list, 12 f32: four zeros, a unit normal (|n|=1 in
133573/133573), a signed distance, a point inside the bbox (133573/133573), and
a trailing 1.0 (133573/133573). The decisive check is algebraic -- n.p + d must
vanish for a real plane, and over all 133573 records the relative residual has a
median of 2.29e-08 and a maximum of 2.15e-07. That is float round-off, not a fit.

So a REGN object carries a point list and a plane list beside its uniform grid,
which fits collision or region-boundary geometry and sits next to MCOL.

Still open: section 1 (96 B, 60631 records), what queries the planes, the zeros
at [0..3], and the constant 96-byte tail.
2026-08-26 07:07:10 +00:00
Sylpheed RE agent
7460d71e6e re: RATC +0x10 is a u16 pair too, and no bit is a clean predicate
The u16 at +0x10 is 0 in all 2985 bundles; the content is a 16-bit flag word at
+0x12 with 83 distinct values. Reading it as a u32 inflates the field and hides
that the header is built from u16 pairs -- the same shape +0x0c turned out to
have. All 16 bits are used, from 1.4% to 91.5%.

Cross-tabulated every bit against four properties measurable from the bundle:
multi-element, animated, window-starts-at-zero, 30fps. No bit is close to a
clean predicate. The strongest is bit 10 against window-at-zero, 0.79 vs 0.21 --
a real association but not a rule, and exactly the kind of moderate split that
invites over-reading.

Bit meanings stay open, but four candidate readings are excluded rather than
untried and the field is correctly sized. Every property visible in the file has
now been tried, so assigning meanings likely needs the game observed with
individual bundles loaded.
2026-08-26 06:59:08 +00:00
Sylpheed RE agent
92526469d9 re: the RATC +0x0c interval is authored, not a keyframe extent
Tested the alternative I recorded last iteration. Parsing every bundle's
keyframe times (2985/2985 parse), the derived-summary reading -- (high,low) ==
(min,max) keyframe time -- holds in 6 of 2985 (0.2%).

The apparent 34.2% match on 'high == min' is a coincidence of zeros: the minimum
keyframe time is 0 in 96% of bundles and high is 0 in 34.9%, so the 1022
'matches' are exactly the both-zero cases. Worth noting that last iteration I
declined to treat the high==0 share as support -- it turned out to be the
confound rather than the signal.

The interval is also narrow: (low-high)/(max-min) has a median of 0.019, about
2% of the keyframe span. It lies inside the keyframe range in 88.6%, entirely
after in 174 and entirely before in 68.

A short authored window is not the shape of a playback range or a whole-animation
loop region, so those readings weaken too. What it is stays open.
2026-08-26 06:52:44 +00:00
Sylpheed RE agent
49cd4996f2 re: RATC +0x0c is two u16s forming an ordered interval, not one number
Read as a u32 it looks meaningless -- 179 distinct values up to 248581842. The
raw values give it away (0x0007000F, 0x000F001A, 0x003C0064): two big-endian
u16s. Over all 2985 bundles, high < low in 2985/2985 with no equal and no
inverted cases, and both are bounded by the animation length at +0x08. Span
runs 1-1200, clustering on 1/10/30/8/20; low equals the animation length in 4%.

A strict ordering holding 2985 times rules out flags or a packed count. Which
interval it is stays yellow -- playback range, loop region and active window all
fit equally.

Recorded the alternative I did NOT test: that (high, low) is simply the min and
max keyframe time, making it a derived summary rather than an authored range.
First step written down. The 34.9% of bundles with high == 0 leans against it
but is not evidence on its own.
2026-08-26 06:46:50 +00:00
Sylpheed RE agent
13a97daf75 re: lead the .slb page with its settled state, not its history
The page is 724 lines of chronological record across 16 sections, several of
them superseded readings kept for their reasoning. A reader arriving fresh had
to read all of it to know what is currently true.

Adds a summary table at the top covering the twelve questions the page answers,
each with its confidence, plus the wave-enumeration recipe and an explicit note
that everything below is history. Also names the four mistakes recorded below so
a reader knows they are deliberate rather than stale.
2026-08-26 06:39:40 +00:00
Sylpheed RE agent
cd83b69389 re: refute the block-id chain -- 8 of 281 is coincidence
Last iteration I noticed BR09_04's block id names BR10_03, itself the next
exception entry, and flagged it as suggestive but untested. Tested: sorting all
282 by offset, entry[i].field equals entry[i+1]'s own id in 8 of 281 (2.8%),
indistinguishable from chance -- and BR09_04 was one of the eight.

Recorded rather than dropped because it is exactly the kind of pattern that
reads as a discovery when spotted in a sample of one.

The offset from the entry's own id is broadly spread, 7 to 14 and beyond with a
peak at 9-11, so the field is near the entry's id without standing in any fixed
relation to it.
2026-08-26 06:34:59 +00:00
Sylpheed RE agent
090bf0cf36 re: the scattered block bytes are an XACT-shaped cue record naming a real cue
Not noise -- a small structured record in three runs at about +1790, +3840 and
+7940. Across all 282: the leading 16-bit value names a real SOUNDS cue
282/282, the block carries two IEEE 1.0 floats in 281, and an 01/02/02/0x64 tail
in 281. Two unit floats and a 100 are the shape of volume, pitch and priority,
so this reads as an XACT cue record -- marked yellow, since every value is a
default and nothing varies enough to prove it.

The id is NOT this entry's own cue: it is consistently a different, higher one
(8501 -> 8504 = BR02_01; 5027 -> 5036 = VOICE_A_036), with a varying offset so
not a fixed stride. BR09_04's field names BR10_03, which is itself the next such
entry -- suggestive of a chain, but one observation and untested.

The practical point, which is settled: the block is a populated metadata record
naming a real cue, not padding.
2026-08-26 06:29:06 +00:00
Sylpheed RE agent
83493c848f re: w0 and w1 are the last two entries of the leading wave's seek table
Both, 282/282 exact: w0 == table[-2], w1 == table[-1]. The trailer's first two
words are a verbatim copy of the tail of the wave's own seek table.

That closes the question and kills the 'usable length or loop end' reading of
w0 -- it is just the previous seek point. It also explains why w1-w0 is always a
multiple of 512 without needing a separate fact: consecutive seek points advance
by whole 512-sample frames, so adjacent entries always differ by a whole number
of them. What I had recorded as a meaningful constraint was a property of the
table the values were copied from.

Neither word is the wave's true length -- w1 lands within 512 of the
PsuedoBytesPerSec length in only 9 of 282, consistent with the correction above.

Still unidentified: the ~40 scattered non-zero bytes elsewhere in the block.
2026-08-26 06:22:56 +00:00
Sylpheed RE agent
2320c56867 re: the seek table's sample total is not the wave length -- durations corrected
Chasing w0 I decoded leading waves and the output matched neither trailer word.
Following that into RIFF waves, where the extent is certain, the decoded sample
count exceeds the seek table's last cumulative sample by a median 9.7%.

The tiebreak is the bank's own PsuedoBytesPerSec: over 14 banks it agrees with
an actual FFmpeg decode to a mean of 0.007 s and with the seek-table duration
only to 0.287 s. Two independent quantities agree with each other and both
disagree with the seek total, so that last entry is the last SEEK POINT, not the
sample count.

Both artifacts regenerated with data_bytes / PsuedoBytesPerSec. Total audio is
408.3 minutes, not the 390.9 I published -- 4.3% in aggregate. Dialogue timings
all rise, e.g. 'They got Leader!' from 1.78/2.46 s to 2.25/2.93 s.

What made this hard to catch is worth recording: the seek total gave BGM of 2.4
minutes, chatter of 2.8 seconds and cutscenes of 11 minutes, and I cited that as
the chain validating itself. It validated the structure, not the scale -- a
uniform 10% error preserves every ratio I checked.
2026-08-26 06:17:07 +00:00
Sylpheed RE agent
62914fcfa0 re: name two fields inside the 12288-byte block's trailer
The trailer opens with two little-endian u32 words. w1 is the leading wave's
total sample count -- 282/282 exact, the same number as the last entry of that
wave's own seek table. And w0 < w1 with w1-w0 always a whole multiple of 512
(282/282), i.e. a whole number of XMA1 frames, 9 to 91 with a median of 14.

What w0 means is left at yellow: trailing the total by a whole number of frames
is the shape of a usable-length or loop-end field, but that is a reading of the
shape rather than a measurement, and nothing here separates the candidates.

Also corrects my own description: I called the region after the seek chunk 'zero
padding'. It is not padding -- about 48 non-zero bytes are scattered thinly
across the 12 KiB, roughly 17 per KiB in blocks 1, 3 and 7. That is a sparse
table, still unidentified, and the distinction matters to anyone skipping it.
2026-08-26 06:09:46 +00:00
Sylpheed RE agent
1b55444020 re: the 12288 region is a padded metadata block, and the assignment rule is 100%
Dumped it. In all 282 exceptions the region from the earlier wave's seek to the
first RIFF is exactly 12288 bytes: the seek chunk (240-260 bytes), then 47-57
non-zero trailer bytes, then zero padding out to the block size.

So an entry may hold a leading wave, a 12288-byte padded block, then its RIFF
wave -- and the assignment rule completes: first-or-second wave at/after the
entry offset is 7620/7620 = 100.00%, with ZERO unexplained.

This overturns my own refutation from one iteration ago. I proposed these were
leading segments, tested it as 'the seek should sit AT the first RIFF', got 0 of
282, and recorded the hypothesis refuted. It was right; my predicate was wrong
by exactly the padded block, a constant I had not yet found. A negative result is
only as good as the predicate it tests.

Still unidentified: what the 47-57 trailer bytes hold.
2026-08-26 06:04:40 +00:00
Sylpheed RE agent
bc5a645f3a re: enumerate all 9661 waves, and measure a bank-to-wave assignment rule
Scanning the flat stream for seek chunks satisfying size == 8+4*count with a
non-negative implied start yields 9661 waves and rejects NOTHING -- no false
positives in a gigabyte of audio. Each wave's extent is
[seek_pos - count*2048, seek_pos).

Assignment rule 'an entry names the first wave starting at or after its offset'
is 96.30% correct (7338/7620) against entries whose answer is independently
known from their RIFF.

The 282 exceptions are one class: the gap between the earlier wave's seek and
the first RIFF is EXACTLY 12288 bytes in all 282. I guessed they were leading
segments, which would put that seek AT the RIFF; refuted, 0 of 282.

Recorded at yellow. Correcting by 12288 would reach 100% on this set but is
curve-fitting until the region is identified, and the rule cannot yet unblock
headerless durations -- there is nothing to check it against there.
2026-08-26 05:58:29 +00:00
Sylpheed RE agent
1591481ca0 re: refuse a seek-based recovery of headerless bank durations
343 English headerless banks carry a valid seek chunk, which would lift dialogue
duration coverage from 62% to ~75%, with a median 3.25 s that looks exactly like
in-mission voice.

It is wrong. A seek sits immediately after its OWN data, so the implied wave
start is seek_pos - packets*2048 -- and for 277 of the 343 that start lies
BEFORE the entry begins, meaning the seek describes the previous bank's wave.
Applying it would attach a neighbour's duration to 81% of the recovered rows.
The 66 that start inside the entry are not proven to be that entry's wave
either; 'starts inside' is necessary, not sufficient.

Recorded as refused rather than applied with a caveat, because a plausible
median is exactly what has misled this work twice today. Recovering these needs
the bank-to-wave assignment settled first.
2026-08-26 05:52:02 +00:00
Sylpheed RE agent
652795e2b1 re: stage dialogue with per-line audio durations, in both languages
Joins the ISL script's built-in 64 call sites, the caption table and the sound
bank seek tables. Duration is the last cumulative sample over the sample rate --
read, never decoded. 1663 of 2683 call sites (62.0%) have an English duration,
2198 (81.9%) a Japanese one; the gap is the headerless banks, and which language
lacks one differs per line, so the columns are complementary.

Two columns deliberately: FILES names the jpn banks and the game substitutes the
language directory, so a line's English and Japanese recordings are different
files of different lengths -- 'They got Leader!' is 1.78 s in English and 2.46 s
in Japanese. The captions are English, so eng is the matching column. The first
version of this artifact had one column and would have attached Japanese timings
to English text; that is why it was regenerated.
2026-08-26 05:46:03 +00:00
Sylpheed RE agent
c8f6617620 re: full bank manifest -- 4114 banks, 390.9 minutes, and the durations self-validate
With the wave boundary exact and Channels read rather than assumed, every field
describing a bank can be read: path, cue, sound id, channels, rate, data bytes,
packets, samples, seconds. Duration comes from the last cumulative sample in the
seek table over the sample rate -- arithmetic, no decoding.

Nothing was fitted to an expectation, yet every category lands where its content
says it should: BGM mean 146 s, Movie mean 79 s with an 11-minute maximum,
Briefing 10.8 s, in-mission Voice 2.79 s never exceeding 7.3. If the boundary
rule, the channel field or the seek table were misread, the numbers would not
sort into those five shapes.

Incidental: 4 banks run at 44100 Hz against 48000 everywhere else, and the BGM
tracks are the stereo ones. 1021 of the 5135 FILES paths have no RIFF and are
excluded as headerless.
2026-08-26 05:39:16 +00:00
Sylpheed RE agent
0a0a0333dd slb: guard the wave-boundary identity with a test, and expose data_at
A wave runs to data_at + declared_size, and there is seek magic exactly there
whose little-endian packet count at +12 times 2048 equals the declared size --
7620/7620 disc-wide. That is the decoder-independent boundary and the thing that
proves the declared sizes honest, so it should not be able to regress silently.

The boundary routinely lies outside the entry's own comp_size window, so reading
it needs the flat segment stream rather than the entry slice; PakArchive gains a
small data_at(offset, len) accessor for that.

Test walks a bounded slice to stay fast -- 792 banks in this run, all holding.
8 disc tests pass.
2026-08-26 05:33:01 +00:00
Sylpheed RE agent
e19e2aa1e8 re: close both remaining .slb questions in the backlog -- both were my errors
The declared sizes are honest (seek magic at data_at + declared_size, 7620/7620)
and VOICE_TCAF_608 was stereo decoded as mono, not truncated. The four offsets
are a segment-packing phase, not a per-directory header size. Neither was closed
by finding something new; both were closed by correcting a mistake of mine.
2026-08-26 05:30:57 +00:00
Sylpheed RE agent
aa84aaf53f slb: read Channels instead of assuming mono -- and retract the TCAF_608 conclusion
I read the seek chunk's packet count big-endian; it is little-endian at seek+12,
with size == 8 + 4*count. And a seek sits immediately AFTER its own data, so an
entry's first seek usually belongs to the PREVIOUS bank (implied start -25232
for D_452, -145988 for TCAF_608). I was comparing an entry's first seek against
its first data -- different waves by construction, which is why no reading lined
up.

With that fixed, the declared sizes are honest: every RIFF-bearing entry on the
disc has seek magic at exactly data_at + declared_size with count*2048 ==
declared. 7620/7620, zero failures.

VOICE_TCAF_608 is not truncated. Its Channels is 2 and I decoded it as mono;
read as stereo it gives 6520176 bytes = 33.96 s, agreeing with both length
signals in the bank (33.88 s from cumulative samples, 33.97 s from
PsuedoBytesPerSec). 170 of 8021 banks (2.12%) are stereo -- exactly the rate of
my 1-in-60 outlier.

This is the mono/stereo trap already documented on this very page, met from the
other direction: I had written 'at two channels every bank yields one frame' and
then spent several passes blaming missing data for a one-frame decode.

Code fix: to_xma_riffs built the leading segment with a hard-wired mono fmt. It
now reads Channels from the bank's first RIFF. 7 disc tests pass.
2026-08-26 05:27:42 +00:00
Sylpheed RE agent
f1f701daf7 re: fix the slots-vs-lines wording at its actual location
The '3 or 4 wrapped lines' sentence is in isl-message-dialogue-link.md itself,
not in cutscene-message-table.md as my correction claimed. Corrected the
sentence in place -- a page has four line SLOTS, of which an English page fills
1 to 4 -- and fixed the misattribution in the correction note rather than
quietly repointing it.
2026-08-26 05:23:35 +00:00
Sylpheed RE agent
77a768c908 re: sweep all six language packs -- and Japanese is not missing captions
The five European packs have byte-for-byte identical MSG key sets: 18813 keys,
4091 ids, zero differences in any direction. Japanese has 14295 keys, which
looks like 4521 missing captions and is not: its id set matches to within 5, its
page count is the same (4765 vs 4758), and its mean lines per page is 1.52
against English's 1.85 with no 4-line pages at all. It carries the same dialogue
and wraps into fewer lines, which is what a language without inter-word spaces
does in a fixed-width box.

Also corrects my own wording in cutscene-message-table.md, which said a page
'holds 3 or 4 wrapped lines'. That counted key SLOTS, which MSG_DEMO records
allocate four at a time. Counting lines with text, an English page holds 1 to 4
(1574/2439/632/113). And it reconciles the 8800 figure used throughout: that is
distinct English keys with non-empty text, out of 18813 keys total.
2026-08-26 05:23:06 +00:00
Sylpheed RE agent
975d5b6519 re: the seek chunk's layout is readable, its packet count is not
Identified the structure: a little-endian size field after the tag, then
0x01000000, a varying word, a zero, then a strictly ascending table. Sizes 64 /
108 / 348 bytes for three sample banks.

Neither obvious reading gives the declared packet count. Entry count minus the
three header words is exactly right for VOICE_D_452 (13) and wrong for the other
two (24 vs 17, 84 vs 371) -- a one-of-three fit, recorded as FAILED rather than
as a rule with exceptions, because two measurements today already returned
plausible numbers for questions they could not answer.

The ascending values step by 1.5-2.2 million, far too large to be packet
indices, so they are on some other scale. Recorded what the next attempt should
know, including that auto/slb-loader chains seek packet counts successfully by
some field that is not either of the two tried here.
2026-08-26 05:16:12 +00:00
Sylpheed RE agent
a047e095c0 re: withdraw my own generalisation -- decoded length cannot locate a bank boundary
Generalising the TCAF_608 test across the 5296 over-declaring banks first looked
like a clean reversal: 59 of 60 gained audio (median 2.10x) from reading the full
declared size instead of stopping at the TOC window.

Then I tested whether the declared size is an honest boundary by reading TWICE
it. 33 of 40 yield >1.5x more audio, median ratio 1.64. The decoder keeps going
indefinitely, because XMA1 packets are self-contained -- the next bank's packets
decode perfectly well and are simply not this bank's. A byte count cannot tell
them apart.

So the 59/60 is withdrawn as evidence for anything. Same error as the unanchored
'data' search earlier on this page: a measurement returning a plausible number
for a question it cannot answer.

What survives: TCAF_608 is genuinely anomalous (86x more input, zero extra
output, where the typical bank yields without limit); the honest-size question
is unresolved and needs a boundary signal independent of the decoder, such as
the seek-chunk packet chain; and the only measured fact is that the declared
size exceeds the TOC window for 5296 of 7586 banks.
2026-08-26 05:10:58 +00:00
Sylpheed RE agent
606797e38b re: ran the decode test -- the bank data does not continue past the TOC window
Read VOICE_TCAF_608's full declared 759808 bytes out of the flat stream ignoring
window boundaries. It decodes to the same 896 bytes (0.01 s) as stopping at the
window end -- 86x more input, zero extra samples. Control VOICE_D_452, whose
declared size fits, decodes to 2.18 s normally. No segment join is crossed, so
the packing phase is not a confound.

So 'the bytes are outside the window but still in the stream' does not hold for
the one bank the other branch named as not truncated. I still withdraw my word
'truncated' as an over-claim about the other 5295 banks: I measured that their
declared size exceeds their window, not what lies beyond it, and have now tested
exactly one. Recorded the three claims at their actual strengths.
2026-08-26 05:04:50 +00:00
Sylpheed RE agent
59c68b2f61 re: the four .slb offsets are a segment-packing phase, not a header size
Reproduced independently: X = (cumulative start of the .pNN segment) mod 2048.
The running sums of the five sound.pNN file sizes are 0, 1392, 1468, 1600, 1728
-- exactly the four values measured, plus zero. The XMA grid is 2048-aligned
inside each segment file, the segments are not multiples of 2048 long, so each
join shifts the phase.

Two of my explanations on this page were wrong even though the measurements
stand. 'Varies by language and subdirectory' was correlation, not cause --
directories cluster into segments. And there is no header at all: those bytes
are the PREVIOUS bank's audio, which is why they had no length field and looked
like data.

Also records a disagreement I could not resolve. auto/slb-loader withdraws my
'69.8% over-declare' finding as declared-sizes-are-exact, but VOICE_TCAF_608
declares 759808 bytes of data with 55296 bytes to the next TOC entry. Both
readings agree the declared size exceeds the window; mine said 'truncated',
which I withdraw as an over-claim, but 'exact' requires a wave to span ~14 named
entries. Recorded with the experiment that would decide it.
2026-08-26 05:03:56 +00:00
Claude
a7d5a3bd12 re: the .slb data offset is the .pNN segment's 2048 grid phase
Closes the open question at the bottom of slb-data-offset.md.

  X = (cumulative start of the .pNN segment holding the wave) mod 2048

The XMA packet grid is 2048-aligned inside each individual segment file,
but the .pak TOC addresses entries in the flat concatenation at offsets
that are themselves multiples of 2048. The segment files are not multiples
of 2048 long, so each join shifts the grid by size % 2048 — and the four
disc-wide values are exactly the running sums:

  1392 = |p00| % 2048;  1468 = +76;  1600 = +132;  1728 = +128

Exact for 7620/7620 banks with a RIFF and 1163/1163 RIFF-less ones via
their seek chunk, 0 mismatches. Supersedes both heuristics (the 99.62 %
packet scan and the 99.97 % seek-residue rule) and dissolves the 28 ties.
The refutation test — an entry straddling a segment join must show two
phases in one file — passes on all 3 straddlers.

The leading bytes are the previous bank's audio, not a header: byte
diversity per offset is indistinguishable from a known packet (101.06 vs
101.90, no fixed field anywhere), the seek packet counts chain exactly
across consecutive entries, and the inter-entry bytes no TOC entry claims
are 1903/1928 non-zero.

Also recorded: the real bank header layout (id, block size 0x800, header
size in blocks, XMAWAVEFORMAT), and the loader search — a null result.
None of the four values exists as an immediate, a table or a float
anywhere in default.xex, which is what a pack-time artifact predicts.
Sound subsystem addresses mapped for the next pass.

Withdraws the 🟡 "most banks declare more data than they store" finding:
declared data sizes are exact (260/260), the bytes are just outside the
TOC window. Also withdraws my own "the header is unique, so nothing is
shared" inference — the windows tile, they do not overlap.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 05:01:26 +00:00
Claude
1f4524f9fe tools: slb_segment_phase.py — derive the .slb data offset exactly
Static, disc-only probe for why a .slb bank's XMA1 stream starts at
1392 / 1468 / 1600 / 1728 bytes in. Subcommands:

  phases  the sound.p00..p04 size table and the running sums mod 2048
  verify  predicted vs measured offset over every bank on the disc
  bank    structural dump of one entry (bank headers, waves, seek chunks)
  chain   the tiling arithmetic across consecutive entries

Reads nothing but dat/sound.pak, dat/sound.p00..p04 and tables.pak.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 05:01:26 +00:00
Sylpheed RE agent
67671e68d1 re: the flat entity count is explained -- arrivals are not spawns
Built-in 12 activate_unit returns 0 when the live object is NULL: it registers an
object that already exists and cannot create one. So a wave arrival is a craft
reaching a point on its route, not a new record, and a count of entity records
was never going to move.

Stage 02's UnitGroup totals Count = 116 across 111 squadrons, matching the live
probe's flat 116. The natural objection is that 116 might match by accident --
it does not: across all 28 stages the roster totals run 2..116, and 116 is the
MAXIMUM and unique to Stage 02, with nothing else within 5.

Three independent things agree: the handler cannot spawn, the roster totals 116,
and the live count sits at 116 while 22 of 113 routed squadrons have a route
starting after t=0.

Still n=1 for the per-member identity (only a Stage 02 save exists), and this
cannot settle timetable-vs-event -- that needs a position probe, not a count.
2026-08-26 04:59:32 +00:00
Sylpheed RE agent
d986a8564c re: withdraw the DisableInterval / delayed-route association -- it is a clustering artefact
The pooled cross-tab gave p ~= 0.02 for 'no Yes-flagged squadron has a delayed
route', recorded at a cautious yellow. Breaking it down by stage removes it:
all 55 Yes routes live in 2 stages, and 45 are in S14, which has NO delayed
route at all -- those 45 cannot be evidence either way. Only S04 contains both
a Yes squadron and a delayed route.

Within S04 the exact test is P = C(47,5)/C(57,5) = 0.37, expected 0.88 observed
0. Unremarkable. The pooled p treated 1023 routes as independent draws when they
are two clusters.

The doc had already flagged this as the kind of near-fit that misled the corpus
before; that caution was right and did not go far enough.
2026-08-26 04:53:13 +00:00
Sylpheed RE agent
f64600fa39 re: the wave timetable's t is SECONDS, measured against the live speed law
Route keyframes are (time, quat x4, pos x3), so each consecutive pair gives an
implied speed. The flight-speed law was measured live against a real-time clock
(cruise ~420, max ~1530 world units/s), which makes it an independent yardstick
in known units.

Across all 28 stage tables, 331 Route_* records and 1104 keyframe pairs: under
SECONDS not one pair exceeds the 1530 ceiling (median 147); under frames at 30
or 60 fps, 89% and 94% of the game's routes would demand impossible speeds.

Two corrections recorded. A first pass silently skipped non-numeric fields while
flattening records, shifting every later value -- it reported a median implied
speed of 65534, suspiciously 2^16, which is what exposed it. And the remaining
outliers were Frame_S<NN>_Asteroid records, not routes; the final measurement
filters by record kind rather than dropping whole stages.
2026-08-26 04:47:10 +00:00
Sylpheed RE agent
c7cf1e5c95 re: narrow what the .slb header is -- four things it is not
Probed the bytes before the data offset at all four values. Not a length field
(no word in the first 64 bytes matches the offset, the RIFF position or the entry
size, either endianness). Not a seek table or any ascending index (~50% of
consecutive BE word pairs are non-decreasing, i.e. random; all words distinct,
none zero). Not zero padding in general (5876 of 7358 have content). Not audio
being discarded (offset 0 wins 6 of 7358).

High-entropy content whose size is constant per language and subdirectory, with
no self-describing length -- which points at a loader that knows the size a
priori. Recorded the next step as static PE work rather than another archive
pass; the byte-level evidence here is about exhausted.
2026-08-26 04:37:59 +00:00
Sylpheed RE agent
6044ba49a0 slb: the seek chunk gives the data offset structurally, and breaks the 28 ties
The ties needed a different signal, not a longer scan. Banks carry one: a seek
chunk sitting on a packet boundary, so seek_pos % 2048 IS the data offset. On
the 6033 labelled banks with a seek before their first RIFF, 6031 agree
(99.97%) -- better than the packet scan and structural rather than statistical,
so scan_data_offset now tries it first.

On the scan's 28 ties it resolves 26 correctly and 0 wrongly (2 have no usable
seek). Combined rule scores 7354/7358 = 99.95%, up from 99.62%. 762 of the 1495
RIFF-less banks carry a seek, so the signal exists where it is needed.

Also ruled out, since a wrong offset was this page's whole subject: the header
is not audio being discarded. Adding 0 to the candidate set, it wins 6 of 7358.

7 disc tests pass.
2026-08-26 04:32:23 +00:00
Sylpheed RE agent
de5ea35452 re: the unvoiced script lines are not the resupply case -- 0 of 14 have a cutscene twin
Having found that the cue-less cutscene pages bind their audio through the movie
path, I tested the same explanation on the twelve script lines with no cue and
no bank. It fails: matching their caption text against every line in the
cutscene message table gives 0 of 14 hits, so nothing carries their audio by
another route.

They are ordinary story dialogue ('Katana, wait for me in hell'), which makes a
deliberate subtitle-only design unlikely -- but that is a judgement, not a
measurement, and why they are unvoiced is still open. Also noted: D_609 and
D_610 are equally unvoiced and simply are not among the twelve, because the
twelve counts script-called names and nothing calls them.
2026-08-26 04:24:37 +00:00
Sylpheed RE agent
f58cee368f re: the cue-less cutscene pages ARE the resupply lines, and they cross-check the movie work
16 of the 17 pages with an empty slot 4 belong to MSG_DEMO_600..604, whose
caption text matches, word for word, the five resupply lines documented from the
movie side as VOICE_D_450..454. Slot 4 is empty on purpose: that audio binds
through the movie path, not through a DEMO_nnn cue.

That also disposes of the '5 ids with more than one record' item -- each appears
once per stage that uses it, and the repeat counts equal the movie-slot counts
recorded independently for the matching banks: 3/3/4/4/2 both ways, 5 of 5. The
11 extra records over 138 ids are these repeats.

Worth more than closing two items: the cutscene message table and the movie
subtitle chain were decoded from different containers by different routes and
agree on the same five lines with the same multiplicities.

MSG_DEMO_500 remains: one page, KATANA, 2s, no text and no cue.
2026-08-26 04:23:52 +00:00
Sylpheed RE agent
946a10e7a7 re: check whether the scan's 99.62% transfers to the banks it is actually used on
The labelled set has a RIFF and the scan is unbounded, so it reads past the RIFF
there -- the headline number could have been borrowing discrimination that a
RIFF-less bank cannot offer. Confining the scan to the leading region gives
69.98%, which looks like exactly that problem.

It is not. Split by how much leading audio there is: on the 989 banks with >=24
packets of it, the scan is 100% correct with ZERO ties, whether or not the RIFF
is in range. The 69.98% is an artifact of short leading regions, where two or
three packets are not enough to separate candidates.

A RIFF-less bank is a whole pak entry, so 24 packets are always available. The
99.62% is conservative for the population the scan serves, not optimistic.
2026-08-26 04:18:53 +00:00
Sylpheed RE agent
bd5bd572c3 slb: the declared data size is an upper bound, not an exact one
Two comments claimed it is 'honest per sub-wave'. Measured: 5296 of 7586 banks
declare more than the entry holds and none declares exactly what it holds, so
the existing .min(slb.len()) clamp is load-bearing rather than defensive.
Comment-only change; 7 disc tests still pass.
2026-08-26 04:12:45 +00:00
Sylpheed RE agent
50849bd452 re: 69.8% of banks declare more data than they store -- and that explains TCAF_608
Of the 7586 banks with a RIFF and a data chunk after it, 5296 declare a data
size larger than the pak entry holds; 2290 declare less (the ordinary
multi-sub-wave case); NONE declare exactly what they hold. This contradicts the
decoder comment claiming the declared size 'is honest per sub-wave'. The code
clamps, so it is a documentation defect, not a crash.

It also closes the loose end from the offset work: eng\Voice\VOICE_TCAF_608,
the single bank where neither offset decoded, is 99% short -- there is nothing
there to decode.

Method note recorded: my first pass searched for 'data' from offset 0, which can
match by chance inside the leading audio region. Anchoring the search after the
first RIFF moved the count 5038 -> 5296.

Separately, the 55 'early RIFF' English banks are not an anomaly: all 55 sit at
exactly 1392 behind a zero-filled header -- a zero-length leading region, which
both the old and new code already handle correctly.
2026-08-26 04:11:45 +00:00
Sylpheed RE agent
0712a651b2 re: sound.pak is fully accounted for -- the last 2 entries were named in the table
static.slb (8970240 bytes, the shared SE bank) and Pj_Silph.xgs (533 bytes, XACT
global settings) both hash into the TOC. Their names come from the BANK_SE and
SETTINGS records of the very IDXD object this page documents -- I had printed
them at the top of the write-up and then reported the entries as unidentified.

9519 of 9519 now: 5100 jpn + 4382 eng + 35 root + these 2.
2026-08-26 04:09:33 +00:00
Sylpheed RE agent
e5ce7e4ba3 slb: the headerless path was decoding stereo at a fixed offset; both are wrong
1495 banks carry no RIFF and take a separate path that hardcoded both the
offset and stereo. Across a random 48-bank sample there was NOT ONE where the
old stereo-at-1392 pair beat the best mono offset; median gain 184x, individual
banks going from 0-4816 decoded bytes to 180000-380000. Stereo shows the same
stop-after-one-frame signature already recorded for the leading segment.

With no RIFF the offset cannot be derived, so scan_data_offset picks among the
four disc offsets by XMA1 packet-header plausibility. Validated on the LABELLED
set -- all 7358 banks that do have a RIFF, where the answer is forced: 7330
correct (99.62%), and all 28 misses are ties on the top score, never a wrong
unique winner. Ties fall back to 1392.

The winning offsets also reproduce, by directory, the distribution measured
independently from the RIFF-bearing banks. jpn\etc splits 1468/1600, so path
alone is not sufficient -- which is why this is a scan and not a lookup table.

7 disc tests pass (build-reborn test -p sylpheed-formats --test
slb_leading_segment_disc, SYLPHEED_DISC wired up).
2026-08-26 04:03:05 +00:00
Sylpheed RE agent
d6127a049e re: withdraw the 'Japanese banks are a separate format' item, one iteration later 2026-08-26 03:57:08 +00:00
Sylpheed RE agent
d15b3d8d85 slb: derive the leading-stream data offset instead of assuming 1392
HEADERLESS_DATA_OFFSET is the value the offset takes in <lang>\etc\, not a
property of the format. The leading stream is a whole number of 2048-byte XMA1
packets ending at the first RIFF, so its start is first_riff % XMA1_PACKET.
Disc-wide that takes four values -- 1392, 1468, 1600, 1728 -- varying by
language and subdirectory.

Verified by decoding, not by arithmetic: on a random 140-bank sample with a
non-empty leading region, the derived offset yields more audio in 85, identical
in 54 (the eng\etc controls, where it must and does reproduce the old
behaviour) and less in 1. Median gain among the improved is 70x --
eng\Voice\VOICE_TCAF_592 goes 1506 -> 97152 bytes, jpn 2910 -> 127178.

This withdraws my own claim from earlier today that the Japanese banks were a
different undecoded layout. They are the same format with a different offset;
I had treated a constant derived from one subdirectory as a property of the
format. The same error was hiding the identical defect in 1873 eng\Voice banks.
2026-08-26 03:56:50 +00:00
Sylpheed RE agent
3b67a451c2 tools(isl): apply the three new built-in names, with 28 flagged as probable
26 damage_unit -> set_unit_hp_pct, 29 -> set_unit_damage_taken_pct,
101 -> all_units_invulnerable, and 28 -> set_unit_damage_dealt_pct with an
inline note that its label rests on a single consumer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 03:43:56 +00:00
Sylpheed RE agent
fa3818c58c re: ISL built-ins 26 / 29 / 101 named end to end; 28's field pinned, label 🟡
The three unnamed built-ins are one family, and the chain from the dispatch
table to the field write is now followed for all of them:

  builtin -> ScriptPhase vtable slot -> interpreter command word 0xAB<op>BA
          -> command-table thunk -> opcode handler -> unit message 0xED08nnDE
          -> the GROUP pump sub_8232C4C0, which rebroadcasts to each child as
             0xED09nnDE
          -> the entity base handler sub_82398CC0, which writes the field.

  26 -> [unit+532]  = min(max(def.HP * pct, 0), def.HP)   = set_unit_hp_pct
  29 -> [unit+676]  = pct, a multiplier on damage TAKEN   = set_unit_damage_taken_pct
  28 -> [unit+672]  = pct, default 1.0                    = 🟡 damage DEALT
 101 -> the same message as 29 with a hard-wired 0.0, broadcast to every unit
        = all_units_invulnerable

`damage_unit` is WITHDRAWN for 26. The handler sets an absolute value rather
than subtracting one, and 100 heals to full -- which no damage primitive does.
It is pinned three ways: [unit+496] is the unit definition (the constructor
sub_82393868 fills it from the same std::map::find built-in 15 uses), [def+84]
is HP in unit_definition_layout.txt and the constructor seeds [unit+532] from
it, and crossing zero loads [def+584] = Delay and raises a flag, i.e. the
destruction sequence. So 0 destroys, with the datasheet's own death delay.

29 is the strongest of the three: [unit+676] has three independent readers
(sub_8237B020, sub_823800A8, sub_82398CC0) and every one multiplies a damage
amount immediately before it reduces [unit+532].

28 is deliberately left 🟡. The write and the 1.0 default are certain, but the
field has exactly ONE reader in the whole image -- the craft update's projectile
spawn, where it ends up as a multiplicative term in the damage message. That is
the mirror of 29 and it is tidy, which is exactly the shape that produced the
wrong names this file has already had to withdraw. What is not established is
that it reaches every weapon; the sibling damage sender sub_82388FF8 has no
+672 term at all.

Three usage tests, all measured over the 28 stages:
  * operand ceilings -- 26 is 97/97 inside [0,100] and 29 is 164/164, while 28
    (identical signature, identical x0.01 conversion) reaches 2000;
  * the craft cross-tab -- 26 splits cleanly into disposable props at 0,
    warships at 30-80 and the tutorial player craft at 100; 29 lands on the
    player, the tutorial boxes and the escorted TCAF hulls; 28 orders
    boss > ace > elite > line > prop;
  * the setup idiom -- activate_unit, then 15/29/28 as a speed/toughness/
    firepower trio, with 26 added wherever a unit must arrive pre-damaged.

Refutations attempted are recorded, including the two that turned into
confirmations (Stage 28 makes each tutorial box invulnerable with 29 and then
removes it with 26) and the offset-search trap that produced three false
readers, because projectiles have their own fields at 672 and 676.

Also corrected: the state guards. 26 rejects states 3 and 4; 28 and 29 reject
1, 3 and 4. This file said otherwise for both.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 03:43:56 +00:00
Sylpheed RE agent
1ba16da9a4 re(data): every call site of built-ins 26, 28, 29 and 101, with craft types
804 rows across all 28 stages, each joined to its squadron's craft through
`UnitGroup_S<NN>.tbl` -- 671/671 of the three unit-taking built-ins resolve,
none unknown. Regenerable from the committed tool.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 03:43:31 +00:00
Sylpheed RE agent
e9c06e2e87 tools: an ISL call-site dumper that carries the operands and the neighbours
`isl.py --calls` scans on the encoding, so it can see that a call happened but
not what it was called with -- the arguments live in the `local[]` staging of
the preceding instructions, not in the call. Naming a built-in needs those, and
it needs the calls either side, because the fixed idioms are what identify a
primitive (`116 -> 101 -> 100 -> 124 -> 93` is a phase teardown, and that is how
101 was named).

So this walks each stage's whole code region linearly, tracking both staging
forms, and prints stage/offset/id/operands plus the neighbouring call ids.
The linear walk is safe rather than assumed: it lands on every call site the
independent encoding scan finds, 2846/2846 in Stage 02.

`--craft` cross-tabs the operand against the squadron's craft type through
`unitgroup.py --all`, which is the test that separated built-in 15's classes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 03:43:31 +00:00
Sylpheed RE agent
a6fb570677 re: the '504 unnamed keys' is 504 entries, not 504 names -- it is 42 keys
Independently reproduced across all 33 paks: 7750 IDXD objects, 1485577 unnamed
field entries, 7094 distinct never-named keys splitting cleanly into 7052 in an
ordinal band (<=0x2198, 94.6% equal to their own field index) and 42 hash-shaped
(>=0x2677C), with ZERO keys in the gap between. The 42 carry exactly 504
entries -- six language copies of one object times two records.

So the preimage target was 42, not 504, and my earlier wording invited the
misreading. Cross-referenced to the idxd-unnamed-keys write-up, which shows the
42 belong to <lang>\script\ID.tbl and cannot be recovered from a 24-bit hash.
2026-08-26 03:40:55 +00:00
Sylpheed RE agent
eb75b62f82 re: INDEX row for the nameless IDXD field keys
Records the census, the 504-entries-not-504-names correction, and the null
result on cracking with the arithmetic that makes it a null result rather
than a shrug.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 03:37:05 +00:00
Sylpheed RE agent
e1e4f7bc1d re: the 42 hash-keyed nameless fields are an ISL script link map — names not cracked
All 42 live in six byte-identical copies of <lang>\script\ID.tbl inside
GP_READY_ROOM.pak, the artefact of the pass the executable announces as
"PrepareScript - isl script prescanning start". Two records, FILE and
OFFSET, carry the same 42 keys: symbol hash -> (.isb file, offset).

Confirmed independently, not inferred: 41 of the 42 keys occur verbatim as
little-endian words inside the .isb bytecode, and where they occur is a
coherent call graph -- main.isb calls the 30 stage/challenge/tutorial
symbols, those call the five helpers in function.isb, the tutorials call the
six in function_tutorial.isb, and main.isb's own symbol is called by nothing.
Ordinals do not appear as call targets in code; hashes do.

30 of the 42 names have their last characters PINNED by the hash's own
algebra. Bumping the character k places from the end moves the low 24 bits
by 256^k mod M and the top byte by 1, and the measured deltas are exactly
+0x01000001 across stage01..09, stage10..16 and challenge01..06, and
+0x01010000 across tutorial0101..0601. So those names end in the same digits
as their .isb filename. The prefixes are not recovered.

NULL RESULT on the names, and quantified rather than asserted. Seven attacks
(1.2M disc + executable strings, printf substitution, a hand guess list,
two-token composition over a mined vocabulary at S = 3.5e8, exhaustive
meet-in-the-middle preimage search) produced zero hits above the noise floor.
The search is not broken -- fed tag_hash("Stage01") it returns "Stage01" plus
three collisions, as 63^6 / 2^32 predicts. It returns nothing for the real
targets at <=6 characters, so the prefixes are longer, and at 7 characters
one target already has 1 176 measured preimages. A 24-bit modulus cannot
name an 8+ character identifier uniquely; only a corpus holding the actual
string can, and the disc does not hold it.

Refutations kept rather than deleted: the filename-stem hypothesis (direct),
the shared-prefix-plus-stem hypothesis (algebraic -- A's byte sum is pinned
twice and the two values disagree, 9 pairs, 9 contradictions), names inside
the .isb payloads (there are no ASCII names there at all), the hash constants
sitting in the executable (absent in both endiannesses), and IXUD holding
more of the same (8 unnamed tags, all ordinals 0..7).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 03:36:40 +00:00
Sylpheed RE agent
8af6a3f5d5 re: measure the nameless IDXD field keys — 504 is 504 entries, not 504 names
Walks every IDXD object on the disc (7 750 objects, 190 782 records,
2 757 039 field entries, zero parse failures) and separates the field keys
that carry no name in the pool from those that do.

The headline correction: the "504 fields disc-wide that are hash-keyed with
no name" is a count of field ENTRIES. It is 42 distinct keys x 6 language
copies of one object x 2 records (FILE and OFFSET) = 504. The brute-force
target was never 504 names.

7 094 distinct keys are never named anywhere. 7 052 of them are not hashes
at all: they are author-assigned element ids, equal to the field's own index
in 1 404 924 of 1 485 577 cases and hand-numbered with gaps elsewhere.
Refuted directly rather than assumed -- tag_hash("BGM_001") is 0xC662435B
while the key of the field valued "BGM_001.slb" is 0x000003E9.

The split is measured, not stipulated: every key that DOES carry a name and
has a zero checksum byte sits at 0x0002677C or above, the ordinal band tops
out at 0x2198, and not one key falls in the gap between.

tag_hash reproduces 1 271 462 / 1 271 462 named field keys disc-wide
(`idxd_unnamed_keys.py selftest`), which is the gate everything else rests on.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 03:36:23 +00:00
Sylpheed RE agent
e0648a815c re: index + a new open backlog item for the undecoded Japanese banks 2026-08-26 03:34:43 +00:00
Sylpheed RE agent
c55bc1d04d re: census of sound.pak, and the leading-region rule is English-only
Every FILES path hashes straight into the sound.pak TOC (5135/5135, path as
stored). The archive is then almost fully accounted for: 5100 jpn + 4382 eng +
35 BGM/JNGL = 9517 of 9519. Only eng and jpn resolve -- deu/fra/ita/esp give
zero, which is the archive-side measurement behind 'two voice sets shipped'.
718 Japanese banks have no English counterpart.

More importantly this bounds an existing result. The leading-region rule from
voice-bank-leading-region.md reproduces its five documented banks EXACTLY under
eng\etc\ (16384/2048/14336/45056/59392), so the pipeline is sound -- but it
holds for only 1571 of 4382 English banks and for ZERO of 5100 Japanese ones.
It is a property of some English banks, not of .slb in general, and the Japanese
layout is undecoded.

I nearly misread this as a broken extractor, because FILES lists jpn paths and
those gave non-multiples of 2048. Reproducing the recorded figures exactly under
eng is what distinguished a wrong assumption from a wrong reader.
2026-08-26 03:34:28 +00:00
Sylpheed RE agent
df48ccfe5b re: withdraw an overstatement -- the wrong-recording case was already settled
My sound-cue write-up said the table 'does not explain' the VOICE_D_452
rejection and implied it was still open. It is not: voice-bank-leading-region.md
settled it earlier -- the bank holds one generic line reused across four stages,
so the binding is correct and the expectation was wrong. The narrower true
statement, which is what the cue table actually adds, is that the reuse is a
movie-slot -> bank relation and not a cue-level one.
2026-08-26 03:29:41 +00:00
Sylpheed RE agent
511bfee410 re: index + backlog rows for the cue table and the cutscene message table 2026-08-26 03:24:30 +00:00
Sylpheed RE agent
b5fa88c0a2 re: the sound-cue table, and cutscene dialogue's speaker/portrait/timing
dat/tables.pak holds a 5798-entry SOUNDS record (cue name -> sound id) and a
5135-entry FILES record (.slb bank paths). Cue names are the join key, so a
script message id now resolves all the way to the bank that voices it:
MSG_VOICE_D_257 -> VOICE_D_257 -> 6945 -> jpn\etc\VOICE_D_257.slb.

The prefix rule is MSG_ -> VOICE_, not strip-MSG_. My first rule was the
latter; it left 88 names unresolved and I was about to write those families up
as text-only announcements, until VOICE_TCAF_592.slb turned up in FILES and
refuted it. Corrected rule resolves 1326 of 1338, and SOUNDS and FILES agree on
exactly the same 12 absentees.

Separately, MSG_DEMO_* is driven by its own IDXD tables in the language packs,
which carry speaker, portrait, on-screen seconds and audio cue per page. Field
count is 9*PageCount+2 for all 7 distinct PageCounts; 1252/1252 caption-key
slots match <ID>_<page>_<line>; the 78 multi-page records equal the 78 counted
independently from the caption side; 138 ids close exactly against the caption
table both ways.

Does not settle the known VOICE_D_452 wrong-recording case -- every cue id is
distinct, so bank sharing is not happening at this layer.
2026-08-26 03:24:15 +00:00
Sylpheed RE agent
8379bc57b8 re: backlog no longer lists multi-page captions as open 2026-08-26 03:11:51 +00:00
Sylpheed RE agent
a962daf518 re: caption pages are utterances, and the dialogue tool was truncating 356 names
Measured the <id>_<page>_<line> key structure: a page is one subtitle box of
3 or 4 wrapped lines, and successive pages are successive utterances by
possibly different speakers. 452 of 4091 ids span more than one page, up to 8.

That refutes the isl_dialogue.py committed two commits ago, which read only
page 000 -- 356 of the 1338 script message names are multi-page, so a quarter
of its output was truncated to the opening utterance. Tool now walks pages
until one is empty; Stage 02 sample regenerated (43 of 213 calls multi-page).

The 2683/2683 resolution figure is unaffected: it counted ids that have text,
and every id does have a page 000. What was wrong was the rendered text.
2026-08-26 03:11:38 +00:00
Sylpheed RE agent
de132c5dec re: index + backlog rows for the script-dialogue link 2026-08-26 03:09:46 +00:00
Sylpheed RE agent
7c4595cbf9 re: mission scripts are readable as dialogue (2683/2683 message calls resolve)
Built-in 64's slot-0 operand is a symbol-table-1 type-6 message id, and every
one of them now has caption text: 2683 of 2683 call sites across the 28 stage
scripts, 1338 distinct names, no residue of any kind.

This only became reachable once build_caption_text was switched to the IXUD
field table (537 -> 8800 lines); before that most of these names had nothing
to resolve to.

Adds isl_dialogue.py plus a committed Stage 02 sample. Does not settle which
recording plays for a given line, multi-page captions, or the other five
languages.
2026-08-26 03:09:10 +00:00
Sylpheed RE agent
253b960f56 re: INDEX row for IXUD predated all of today's IXUD work
The row still described ixud.rs as a cue reader and said nothing about the
record table or the caption families. Rewritten with the measured numbers:

* the IXUD record/field table is decoded and wired in -- IxudObject mirrors
  IdxdObject, uniform 16-byte records, 12-byte fields, every offset in CHARS,
  and the word at 0x08 is record 0's hash rather than a schema id. Verified
  1104/1104 objects, 1476/1476 records, 628165/628165 named fields.
* caption text goes 537 -> 8800 lines, which is 8800 of 8800 distinct keys,
  in two steps: generalising the key parser to all eight families took it to
  8074, and switching from token adjacency to record fields finished it.

The row also carries my correction rather than quietly dropping it: the
earlier "1.3% of the game's text" counted occurrences across blocks, and the
honest denominator is 8800 distinct keys, so the starting point was 6.1%.

And it keeps the DEMO control, which is the part a reader should remember:
token adjacency finds 537 lines in that family, fields find 541. The old
reader was dropping lines in the one family it was written for.

Status moved from 🟡/ to .

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 03:00:37 +00:00
Sylpheed RE agent
80a45bfbd7 re: wire the IXUD record table into the crate — captions go 537 to 8800 of 8800
ixud.rs now has an IdxdObject-shaped reader, IxudObject, and build_caption_text
reads captions as FIELDS instead of pairing them with whatever token follows in
the pool.

  build_demo_text      token adjacency   134 ids   537 lines
  build_caption_text   token adjacency  3721      8074
  build_caption_text   record fields    4085      8800  = all of them

Verified over the whole disc by tests/ixud_records_disc.rs: 1104/1104 objects
parse, 1476/1476 records and 628165/628165 named fields reproduce their
ixud_hash, 48 positional, zero failures. The header word at 0x08 is record 0's
hash, asserted per object -- there is no schema field, exactly as for IDXD. The
module doc described a 12-byte record directory and a "schema/type hash"; both
were wrong and are corrected.

I also have to correct my own number from the previous commit. "1.3% of the
game's text" counted OCCURRENCES: each family lives in 24-45 IXUD blocks and
the same key repeats across them. Distinct text-bearing MSG_* keys number 8800,
not 44579, and every one has the <id>_<page>_<line> shape. So the real coverage
was 537/8800 = 6.1%, and I overstated the gap about fivefold. Direction right,
magnitude wrong.

The DEMO control is the sharpest evidence for the change: token adjacency finds
537 lines there, the field reader 541. It was dropping lines even in the one
family it was written for -- which is why the test now asserts "must not lose
lines" rather than "must be identical".

Same lesson twice in one session: pool adjacency is a consequence of how
records are written, not a rule of the format.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 02:55:26 +00:00
Sylpheed RE agent
1b295e3cd2 re: read all eight caption families — 15x more text, and the same lesson twice
build_caption_text generalises the key parser from MSG_DEMO_* to all eight
families. The shapes are uniform and each family is 100% consistent with its
own: seven use MSG_<FAM>_<id>_<page>_<line>, and VOICE alone inserts a family
letter before the id.

                       ids    lines
  build_demo_text      134      537
  build_caption_text  3721     8074

The DEMO family comes out identical through both readers -- 537 lines either
way -- which is the control that generalising changed nothing that already
worked. Pinned by tests/caption_families_disc.rs, along with VOICE ids keeping
their family letter.

But this does NOT close the gap, and the write-up says so: 8074 against the
44579 text-bearing fields the record-level scan counts is about 18%.

The reason is the same lesson this session already learned once.
build_caption_text pairs a value with the key that happens to follow it in the
raw UTF-16 token stream -- the adjacency heuristic that was wrong for IDXD and
is wrong here for the same reason. ixud.rs has no record/field reader at all.
The IXUD record table IS decoded and verified disc-wide (1104/1104 objects,
628165/628165 fields reproducing their key) and was simply never wired into
the crate.

Next step recorded: give ixud.rs an IdxdObject-shaped reader and read captions
as fields rather than adjacent tokens. The decode exists; only the plumbing is
missing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 02:47:13 +00:00
Sylpheed RE agent
abe41d8c27 re: the crate reads 1.3% of the game's text — seven caption families unread
movie_subtitle handles MSG_DEMO_*, the cutscene captions. Counting every IXUD
block in GP_MAIN_GAME_E.pak, that is the SMALLEST of eight families:

  MSG_ADAN   23236 keys   9801 with text   ADAN combat chatter
  MSG_RHIN   21196        8509            Rhino squadron
  MSG_TCAF   17148        6728            TCAF
  MSG_VOICE  13060        6776            in-mission scripted dialogue
  MSG_BIRD   14036        5834            Bird squadron
  MSG_ADPL   12640        4127            ADAN pilots
  MSG_ACRO    4804        2244            Acropolis
  MSG_DEMO    1252         560            cutscene captions <- the only one read
  total     107372       44579

560 of 44579 text-bearing keys = 1.3%. I report the text-bearing column rather
than raw keys because only 41.5% of keys carry text -- the rest are the empty
line slots this container pads with, and counting those would flatter the
denominator.

MSG_VOICE_* is the family the message tables reference -- the dialogue whose
voice bindings this file now analyses in detail -- and nothing in crates/
parses it. So the corpus knows which bank plays for a line it cannot read.

First step recorded: build_demo_text already pairs a text value with the
MSG_DEMO_<demo>_<page>_<line> key that follows it, and the other seven
families use the same <id>_<page>_<line> shape, so generalising the key parser
is most of the work. With a warning attached: do NOT assume the id spaces
relate, since the voice-bank id and the caption id within one message page are
different numbers.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 02:40:10 +00:00
Sylpheed RE agent
188d5cfdd7 re: the genuinely-cut voice lines still have subtitles — and two id spaces differ
The 60 tokens absent from BOTH voice languages, itemised:

  45  VOICE_E_ family -- 44 numeric [0..43] plus the lettered VOICE_E_012B
  13  VOICE_C_ at 421, 423-426, 430, 432, 447-450, 470, 471 -- INSIDE the
      listed range [0..489], so interior gaps rather than a truncated tail
   2  VOICE_D_182 and _183, adjacent

These lines were written and captioned; only the audio is missing. Their
caption keys resolve to real text in the IXUD blocks:

  MSG_VOICE_C_355_000_00  "What are you doing? Quit wasting..."
  MSG_VOICE_C_367_000_00  "The final defense weapon is..."
  MSG_VOICE_C_347_000_00  (Japanese)
  MSG_VOICE_D_152_000_00  (Japanese)
  MSG_VOICE_E_044_000_00  (Japanese)

Three of the five sampled are still Japanese INSIDE the English pak --
captioned but never translated, matching the untranslated entries already
noted for the localised-text container.

Separately, a trap worth its own heading: within one message page the voice
bank token and the caption keys use DIFFERENT numbering.

  Message_106  voice VOICE_C_468   lines MSG_VOICE_C_385_000_00..02
  Message_129  voice VOICE_D_182   lines MSG_VOICE_D_152_000_00..02
  Message_044  voice VOICE_E_012B  ID    MSG_VOICE_E_044

Same family letter, different index space. Deriving one id from the other
will silently mis-pair audio with text -- which is the same class of mistake
as the demo-id voice binding this corpus already had to reject in-game.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 02:34:29 +00:00
Sylpheed RE agent
832c1a48aa re: most of the 268 missing voice banks are UNDUBBED, not cut — I had it wrong
Last iteration I read the scattered missing runs as cut dialogue. Checking the
other languages refutes that for most of them.

Only two voice languages exist: tables.pak holds eng\sounds.tbl (4418 banks)
and jpn\sounds.tbl (5136), and NO fra/deu/esp/ita. The other four
localisations are subtitle-only, which is itself a fact the corpus did not
record.

The English set is a strict subset of the Japanese one:

            eng     jpn
  listed    4418    5136
  resolved  4175    4383
  missing    268      60
  lacked by
  the other     0     718

Every id missing in Japanese is also missing in English, and NONE is missing
in Japanese but present in English. So the 268 split cleanly:

  208  exist in Japanese, not in English -- recorded in the original and
       never dubbed. A localisation gap, not cut content.
   60  absent from both -- genuinely cut, including all 44 VOICE_E_ (listed
       0 times in either language) plus 16 others.

Per family eng/jpn: A 683/771, B 383/423, C 366/474, D 387/546. The 718
Japanese-only banks include whole families English never had -- VOICE_ADAN_
(110), VOICE_ZZZZ_ (104), VOICE_TCAF_ (41), VOICE_RHIN_ (27).

So VOICE_E_ survives as the one genuinely-cut family, and my "short runs of
cut dialogue" reading was wrong: those lines are on the disc, in Japanese. The
previous text is kept above the correction.

This is why the "measured on English only" limit was worth writing down rather
than quietly generalising -- the second language changed the conclusion.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 02:28:17 +00:00
Sylpheed RE agent
6c951a9b02 re: 268 voice references name a bank that does not exist — one whole family
Following up the incidental gap from the DEMO_* work. Resolving every VOICE_*
token the way the engine does -- token, then eng\sounds.tbl for its directory,
then sound.pak -- gives 4175 resolved and 268 unresolved of 4443 distinct.

The gap is in the manifest, not the archive: the unresolved tokens are not
listed in sounds.tbl at all, and of the 4175 that ARE listed, every one is
present in the pak. Nothing is listed-then-missing.

By family, script-used ids against sounds.tbl-listed ids:

  VOICE_A_   696 [0..772]  listed 683   missing  14
  VOICE_B_   391 [0..422]  listed 383   missing   8
  VOICE_C_   429 [0..489]  listed 366   missing  63
  VOICE_D_   508 [0..598]  listed 387   missing 126
  VOICE_E_    44 [0..43]   listed   0   missing  44  <- the whole family

Two different shapes, and the distinction matters. A/B/C/D lose short runs
scattered through an otherwise dense range -- 11,12,13 / 56,57 / 181,182,183
/ 389-392 -- which is what cut dialogue looks like. VOICE_E_ is absent
entirely: 44 ids used, none listed, no VOICE_E_* anywhere in the manifest. A
whole speaker or category, cut.

For the port: a missing voice bank is a RETAIL condition, not a decode bug. A
reimplementation must tolerate an unresolvable VOICE_* reference, the same way
it must tolerate SUBTITLE_S12B.tbl resolving in no language.

Limit stated: measured on GP_MAIN_GAME_E.pak and eng\sounds.tbl only; whether
the other five languages drop the same ids is untested.

An earlier probe of mine checked only three sound.pak directories and would
have mis-attributed this; sounds.tbl lists five, including Briefing. Resolving
through the manifest rather than guessing directories is what makes the 268
trustworthy.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 02:22:46 +00:00
Sylpheed RE agent
197ae49123 re: DEMO_* in a message's voice slot is not a sound bank
An open question from the game_data migration: 132 dialogue lines carry an
unprefixed DEMO_* token where a VOICE_* one is expected, and whether DEMO_*
names a bank was untested. It does not.

  VOICE_*  4443 distinct, 4175 resolve in sound.pak
                          (2360 eng\Voice\, 1815 eng\etc\)
  DEMO_*    296 distinct,    0 resolve

Zero resolve under any of the three directories, and looking them up through
the convention -- VOICE_DEMO_nnn -- resolves 0 of 296 as well. So the slot is
not "the voice bank"; it can hold either a bank reference or a demo one.

The tokens run DEMO_000 to DEMO_310, 287 distinct in [0, 310] -- a dense
sequential id space, which is the shape of the demo id this file already uses
in MSG_DEMO_<demo>_<page>_<line> caption keys. I did NOT confirm the overlap
and say so: those keys live in IXUD wide-string blocks while my scan walked
IDXD objects only, so it found 0 of them. That needs an IXUD-aware sweep.

Incidental and left open: 268 of the 4443 VOICE_* tokens do not resolve
either. That is a separate gap and the DEMO_* finding does not explain it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 02:16:16 +00:00
Sylpheed RE agent
f095fe647f re: the stage-02 "defaulted field" capture is wrong for 99 of 101 fields
The last file still resting on the "defaulted on disc" premise. Re-checked
through the record table over every unit-like object on the disc -- 114 with
a Generic.Type of Craft or Vessel:

  present in ALL 114 objects  50
  present in some             49
  present in NONE              2   <- FiringRange, SELength

So only FiringRange and SELength are genuinely absent, out of 101 listed.
FiringRange was already recorded correctly elsewhere (runtime 0 is the
accessor's miss value); SELength joins it.

One pattern worth keeping rather than dismissing: 15 of the fields sit in
exactly 5 objects each -- AB_* (afterburner), PowerCut*, AA_/AV_AxisMode_*.
That is the Delta Saber family, so those fields are player-craft-only rather
than defaulted, which is a different fact than either the file or my
correction to it would suggest on its own.

The runtime values themselves stay -- they are real measurements. What is
withdrawn is the claim that the disc does not carry the fields.

That completes the sweep: weapon-datasheet-runtime.md, unit-struct-runtime.md,
live-unit-definitions.md, INDEX.md and now this capture all carried the same
wrong premise from the same blind reader.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 02:10:14 +00:00
Sylpheed RE agent
df8b63d35c re: INDEX still advertised the withdrawn "defaulted on disc" framing
The two headline rows for weapons and units were the last place still telling
a reader that these values are not on the disc. Both corrected in place, with
numbers I measured rather than adopted.

Weapons: re-reading captures/weapon-runtime-fields.csv through the record
table, 1514 of the 4393 rows labelled defaulted-on-disc DO have a value on
disc; 2879 are genuinely absent. I state that as an upper bound -- my matcher
counts a field when it appears in ANY record of the object, and a per-record
count gives ~1448. Either way the headline "4393 values the disc does not
carry" is wrong by about a third.

The spot checks are exact rather than aggregate: wep_05/wep_60
TriggerShotCount 4, wep_02 Power 100.0, wep_60 Power 1000.0 (refuting the
recorded "C band 150-500" bracket), wep_25 MaximumRange 4000.0,
wep_11/28/36/70 LoadingCount 6/5/5/0.

Units: the ~30-field player-craft table is on disc at exactly the values the
runtime "recovered", spread across the Generic / Shield / Mass / SE records
-- which is why a reader that could not name a record saw them as absent. And
"18 of 23 vessel records are missing at least one of Size_X/Y/Z/HP" is false:
0 of 114 objects with a Generic.Type (43 Craft + 71 Vessel) miss any of them.

Status markers moved from  and /🟡 to / so the rows no longer read as
settled-and-correct.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 02:04:11 +00:00
Sylpheed RE agent
2a9e25819d re: the unit "defaulted on disc" tables are wrong too — corrected
unit-struct-runtime.md lists ~30 player-craft fields as defaulted on disc with
values "recovered from the running game". All of them are on the disc at
exactly those values. Verified myself for UN_f001_TCAF_DeltaSaber_T:

  Size_Radius 10.0, FCSRange 500000.0, ResistanceToOptics/HQRatio/
  ShieldRatio/ThrusterRatio/ResistanceToShell/ResistanceToExplosion 1.0,
  MassScore 0                                    [Generic]
  ChargeSpeed 25.0, ChargeDelay_Break 10.0       [Shield]
  DryMass 100.0                                  [Mass]
  LowerHPThresholdRatio 0.3                      [SE]

The record column is the explanation: the values are spread across Generic,
Shield, Mass and SE. That is exactly why the old flat reader missed them --
it could not say which record it meant, so a per-record field read as absent.

Also wrong, and measured: "18 of the 23 vessel records are missing at least
one of Size_X/Y/Z/HP, nearly always Size_Y". Over every object carrying a
Generic.Type -- 43 Craft plus 71 Vessel, 114 objects -- ZERO are missing any
of the four.

live-unit-definitions.md's "Route B: 13 defaulted fields read out of the
running game" goes with it: all 13 Size_Y values are on disc, each exactly
equal to the value the runtime run "recovered". Nothing was recovered that
the disc did not hold; the run confirms only that the parser reads the disc.

As with the weapon sheet, the runtime captures stand as measurements. What is
withdrawn is the premise that they were reaching values the disc lacks.

Correction banners at the top of both; original text intact beneath.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 01:58:58 +00:00
Sylpheed RE agent
ddd0d5b4b7 re: the weapon DATA SHEET's "defaulted on disc" premise is false
Every field weapon-datasheet-runtime.md calls defaulted is on the disc. The
record table reads them directly; the old string-pool reader could not see a
value whose string is shared or reordered and reported it absent. Verified
against the disc myself rather than adopted:

  wep_05_ASMissile  TriggerShotCount  (defaulted) -> 4
  wep_60_ASMissile  TriggerShotCount  (defaulted) -> 4
  wep_02_Missile    Power             (defaulted) -> 100.0   [Shell]
  wep_60_ASMissile  Power             (defaulted) -> 1000.0  [Shell]
  wep_25_HBeam      MaximumRange      (defaulted) -> 4000.0  [Shell]
  wep_11/28/36/70   LoadingCount      (defaulted) -> 6/5/5/0

Two consequences recorded:

* The hypothesis "the title-code default for TriggerShotCount is 4" answers a
  question that does not exist -- both weapons state 4 explicitly.
* The bracket "wep_60_ASMissile's defaulted Power sits in the C band
  (150-500)" is refuted: it is 1000.0, outside the band. That also puts a
  question over the bucket model or over that row's identification.

What is NOT withdrawn: the runtime capture itself. Every number it read
matches the disc. What goes is the reason for taking it -- the Gallery panel
was not recovering values the disc lacks, it was reading values our parser
was missing. Still useful as an oracle, redundant as a source.

Correction banner at the top; the original document is left intact beneath it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 01:53:59 +00:00
Sylpheed RE agent
b273ee9f48 re: WITHDRAWN — "FormationID does not hash into the table". It does.
The refutation used the wrong hash function. A record key is tag_hash of the
record's own name, not the pak-TOC name_hash which lowercases and uses a
different modulus:

  tag_hash("Formation_4_Bird")  = 0x22a5eeed   <- exactly the record key
  name_hash("Formation_4_Bird") = 0x6286edad   <- the wrong hash, hence
                                                  the recorded "0 of 16"

Formation_4_Bird IS a record name in FormationSet_S02.tbl and every record in
that table keys by tag_hash of its name. Across all 28 UnitGroup_S<NN>.tbl the
same holds for field keys: 6960 of 6960 are tag_hash(field name), 0 are
name_hash. So FormationID resolves by a direct hash lookup and the roster
indirection is NOT required for it.

This is exactly the trap idxd-container.md warns about, and it cost this file
a spurious refutation. The original text is kept below the correction.

The squadron half of the claim still stands and is marked so: a squadron
record's key is not tag_hash of the squadron id either -- tag_hash("TCN001")
is 0x760cdeaa against keys running 0x659aff47 -- because the record's NAME is
not the squadron id. That roster indirection is real.

Also closes the file's standing "the key derivation stays open, a second hash
function is unidentified": that function is tag_hash, recovered since.

Found while checking a subagent's claim rather than taking it -- it reported
the hash correction, and verifying it turned up that the FormationID half was
a stronger result than reported.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 01:48:14 +00:00
Sylpheed RE agent
1707b6b74a re: recover the ISL interpreter's command table — 57 opcodes, committed
sub_822FE040 fills 1023 eight-byte slots at table+32 with a default and then
overwrites individual ones; slot = (N - 32) / 8 from each std r9, N(r31).
Symbolically executing it yields 57 populated slots, matching the count the
corpus recorded, now with the full opcode -> handler map committed as
docs/re/data/isl-command-table.txt and regenerable from
tools/re-capture/isl_cmdtab.py.

Nine opcodes point at 0x82391BA8, which is `li r3,1 ; blr` -- accept and
discard. 768, 769, 774, 775, 776, 791, 792, 793 and 805 are dead in this
build, which is why the built-ins posting them do nothing.

Opcodes 800-802's entries are thunks 8 bytes apart into 60-instruction
handlers that differ in exactly two words: a descriptor offset and a unit
message id.

  800  builtin 26        0xED0802DE
  801  builtin 28        0xED0803DE
  802  builtin 29, 101   0xED0804DE

That fixes the id format as 0xED08 nn DE, and the ids known from other work
fit it: opcode 514 -> 00DE, 803 -> 07DE, 999 -> 0FDE.

Stopped one link short of the semantics, and saying so: the pump's arm for
0xED0802DE does not apply an effect. It walks the unit's child list at
[unit+320]/[unit+324] and REBROADCASTS to each child as 0xED0902DE. So 0xED08
is the to-unit family and 0xED09 the to-child one, and the terminal effect is
further on. 26/28/29 remain unnamed.

The command table is the reusable part -- it answers "what does this opcode
reach" for every future built-in question, not just this family.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 01:44:49 +00:00
Sylpheed RE agent
8d1c9d1c77 re: pin the per-record game data — and correct the "SpaceSize is per phase" claim
A disc-gated test that fails if the loaders regress to one flat answer per
object. It checks the three structural claims that came with the migration
request, and one of them is overstated:

*  MainMissionBonus IS per difficulty — 18 of the 24 missions pay three
  different values, in a strict 1:2:4 easy:normal:hard ratio (all 24); the other
  6 pay 0 at every difficulty. The flat reader returned the Easy value.
*  RankScore_S/A/B/C/D repeat identically across the three Score_* records in
  24 of 24 missions, so the single flat answer happened to be right. The
  difficulty scaling lives in the earnings (CraftScore_Adjustment,
  KillBonus_Maximum, …), not in the thresholds.
* 🟡 SpaceSize is *stored* per phase (three Phase_N records), but only **1 of the
  24** missions varies it — the one that reads 250000/100000/100000. The other 23
  repeat a single value (500000 ×1, 100000 ×18, 50000 ×4). "Phase_1 = 250000,
  Phase_2/3 = 100000" is one mission's numbers, not a rule.

Also pinned: the Delta Saber's 63 Turret_NNN mounts at HP 100 each against a
TurretCount of 4; the SD-Battleship's per-component HP (bridge 10000, thruster
20000, shield generator 5000 at PowerRatio 0.25, hatch 100 launching
Squadron_Test2, anti-ship gun 1500) against a hull of 100000, and its 37 records
that define an HP; 418 hardpoints across the 23 capital ships, none missing HP;
131 weapons whose shell id always differs from the launcher id, 40 with a
ShellWake, and Weapon_NULL as the only one without a Shell.Power; and
fcs_range/shield_ratio/cruising_velocity/maximum_velocity/acceleration/
deceleration now set on 89 of 89 units, all of which the pool reader could miss.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 01:43:08 +00:00
Sylpheed RE agent
6ce1a87d49 re: rebuild game_data on the IDXD record table — 966 misses and 596 flattened reads
Measured first, over GP_MAIN_GAME_E.pak, comparing every named-field read the
six struct loaders performed against the record table: 4435 reads, 2872 agreed,
**966 returned None for a field that has a value**, **596 flattened a field that
several records carry**, 1 was wrong (a weapon whose TargetType is the empty
string read back as the neighbouring token "Skip"). The prior report of
4453/2887/974/591/1 is the same picture; the small differences are definitional
(I count a read as flattened only when the records disagree).

Every read now goes through IdxdObject::record, and the types say where a value
comes from:

* Weapon = the `Weapon` record (launcher) + the `Shell` record (projectile).
  Both carry an ID and a Name and — with `ShellWake` — an `Interval`, which the
  flat reader merged; they are separate fields now. Power/Velocity/ranges/
  LifeTime are Shell fields, which is why 427 weapon reads used to miss.
* CraftUnit/Vessel = `Generic` (hull) + `Maneuver` (flight model) +
  `StructureCount` (counts) + `Shield`, plus a new `hardpoints: Vec<Hardpoint>`
  — one entry per Turret_/Bridge_/Thruster_/Hatch_/ShieldGenerator_ record, each
  with its own HP. A flat HP could only ever be one of them.
* PlayerConfig = `Player`, plus `phases: Vec<PlayerPhase>` (SpaceSize/SupplyRange
  are per Phase_N) and `score: ByDifficulty<ScoreRules>` (MainMissionBonus is per
  Score_<difficulty>; the flat answer was the Easy one).
* Character faces come from the `Faces` record's field names (identical output to
  the old token scrape, 0 of 68 objects differ — now by construction).
* Stage = `StageResource` + `phases: Vec<StagePhase>`, and the packages it names.
* The `fields: BTreeMap` on every struct became `records: RecordSet`, which keeps
  the record boundary; `RecordSet::everywhere(field)` answers "which record".

The token-scraping loaders move too, and this is where the old reader was worst:

* Arsenal: options are the positional fields of the STANDARD_<slot> records. The
  scrape returned 16 nose options of which 8 were field keys and pilot names, and
  47 for arm3 of which 38 were junk, while missing Mine_B2A and No_Equipment.
  Now 8/12/9/9, all real weapons.
* Squadron: one record per squadron, members are Count*4 positional slots
  (unit, message set, n, pilot) — 1160 squadrons with ids and 2295 member tuples,
  against 28 idless squadrons and 47 members before. Agrees exactly with the
  independent Python decode in docs/re/structures/unit-group-table.md.
* DemoMessage: 11775 lines against 10263, every one with a speaker, a portrait,
  a delivery mode and a voice token, from fixed positional slots.
* PilotRoster: assignments are the records the `UNITS` record names, so each one
  now carries its unit id, its loadout and the player marker.
* UnitRoster: the roster is the field *names* of the single `EnumUnit` record.
* load_weapons selects on the records (Weapon + Shell) rather than on token[0],
  whose first byte is often a stray pool byte ("#Weapon", "%Weapon"). Same 131
  objects, no heuristic. GP_HANGAR_ARSENAL.pak holds none of them — the module
  doc's claim that player weapons live there was wrong.

schema:: constants keep their names and values but are documented for what they
are: record 0's name hash (PLAYER = Difficulty_Easy, UNIT = Maneuver, VESSEL =
Bridge_000, MESSAGE = Message_000), not a schema id.

Two things the migration exposes and does not fix, flagged in the docs instead:
load_units' bucket is 43 Type=Craft + 46 Type=Vessel objects (new `unit_type`
field lets a caller separate them), and StructureCount.TurretCount is not the
number of Turret_* records (the player's craft says 4 and has 63).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 01:42:53 +00:00
Sylpheed RE agent
62d8264cbb chore: drop zz_scratch_measure.rs, swept into 49a09a9 by accident
It is a throwaway measurement harness I wrote in this session ("TEMPORARY
measurement scratch — not for commit") that a concurrent commit picked up
along with its own files. It asserts nothing and duplicates work that is now
covered by tests/game_data_disc.rs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 01:42:28 +00:00
Sylpheed RE agent
510dcf29d8 re: built-ins 26/28/29 are one family — and damage_unit looks mis-named
Method-diffing settled the structure but not the semantics, so nothing is
named this time.

  26 damage_unit  vt76  sub_8226ACD0 (67)  opcode 800   97 sites
  28              vt84  sub_82268F98 (69)  opcode 801  410 sites
  29              vt88  sub_822690B0 (69)  opcode 802  164 sites
  101             vt276 sub_822691C8 (78)  opcode 802  133 sites (broadcast)

28 and 29 differ in two words only -- the opcode (0x21BA vs 0x22BA) and a
descriptor pointer 8 bytes apart -- and are otherwise instruction-identical.
All three take (unit, double). 26 differs from both by one guard: it rejects
only state 3, while 28 and 29 reject states 1 and 3.

The operand distributions separate them:

  26   97 sites   7 distinct   [0, 100]    0 x69, 80 x10, 100 x6
  28  410 sites  13 distinct   [0, 2000]   200 x116, 120 x76, 300 x74
  29  164 sites   6 distinct   [0, 100]    0 x64, 100 x53, 50 x38

26 and 29 are percentage-shaped; 28 is an absolute quantity an order of
magnitude larger.

That casts doubt on an existing name. damage_unit (26) passes 0 in 69 of its
97 calls -- dealing zero damage is a no-op, so 71% of its call sites would do
nothing, while SETTING a percentage-valued property to 0 is a natural thing
to do 69 times. 29 has the same shape. The name predates this session and I
have not withdrawn it, but it should not be relied on.

Where this stopped: the three commands' descriptors at 0x820A8D10/+8/+16
point into 0x8210E5xx, which is BELOW the disassembly DB's range (it starts
at 0x82150000) and holds no code, so that route cannot reach an execute
method. Opcodes 800-802's semantics need the interpreter command table
instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 01:38:08 +00:00
Sylpheed RE agent
679812f1c9 re: built-in 108 is deploy_squadron_ex — deploy plus a 1<<n selector
1146 sites in 22 stages, the second-largest unnamed built-in. Its method
sub_822646B8 (vtable slot 300) and built-in 2's sub_822642E0 (slot 12) are
190 and 199 instructions and differ in exactly one block. Diffed instruction
by instruction, 108 adds:

  lwz   r11, 16(r29)     the blob's slot-16 int
  cmpwi/blt/cmpwi/bgt    range-check n to [0, 31]
  slw   r21, r25, r11    r21 = 1 << n     <- a 32-bit selector
  ...                    default 1 when out of range

Everywhere built-in 2 passes its r21, 108 passes r20 and reserves r21 for the
mask, so the bit is an EXTRA argument to the same call rather than a
replacement. Both post the identical command word AB0100BA, opcode 256.

The operand is always a valid bit index: over all 1146 sites slot 16 is in
[0, 31], 1146 of 1146, none outside, so the out-of-range default never fires
in shipped content. Fifteen distinct values clustered at 16 (531x), 31
(165x), 20 (161x) and 2 (90x); 21 of 22 stages use more than one.

What the bit SELECTS is not established and the name does not claim it. There
is plainly a 32-bit space -- built-in 92 reportedly allocates a free bit by
OR-ing over live units, which would make 108 its "place in a named slot"
counterpart -- but I have not verified that, so the name says only what is
proven: the same deploy as built-in 2, with an extra selector.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 01:31:39 +00:00
Sylpheed RE agent
4764e3d5f2 re: built-in 12 is activate_unit — 517/517 on the ordering test
The highest-traffic unnamed built-in: 1197 sites across all 28 stages.
sub_822659F0 read directly:

* indexes [phase+324]'s record array by the slot-4 symbol;
* returns 0 immediately when the live object [record+4] is NULL, so it
  registers an object that already exists rather than spawning one;
* sets [record+16] = 2, the documented "active" state every unit predicate
  tests;
* stores sub_82301118's packed result into [record+20] (low 16) and
  [record+24] (high 16) -- member counts;
* posts opcode 513 (0xAB0201BA) either way; the slot-8 mode (1 in 999 sites,
  0 in 198) only decides whether cmd+20 is also set to 1.

The ordering test: if this activates a unit for the script, no predicate
should test a unit before it. Over all 28 stages, for every (stage, unit)
pair having both, activate_unit comes first 517 times and a predicate first
0 times. Recorded as file order rather than proven execution order --
coroutines can interleave -- but 517 with no exceptions is not a coincidence.
344 units are tested without ever being activated (live from mission start)
and 203 activated without being tested.

Also recorded: a tidy closure that FAILS. squad_survival_pct reports current
over initial and activate_unit snapshots counts, so the snapshot looks like
the baseline. It is not -- built-in 24 reads [record+16] for the state then
calls sub_823011B0 and sub_82301118 on the LIVE object, never touching
+20/+24. What reads those two fields is unidentified.

Named coverage is now 57 of 108 distinct ids and ~80% of call-site traffic.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 01:25:31 +00:00
Sylpheed RE agent
a0cfa68ed8 re: built-in 15 IS set_group_speed — and the turret anomaly was my own artefact
The member object sub_82348830 returns is the per-member unit DEFINITION, and
that identification is not a guess: the same spawn loop builds two aggregates
and each lands on a semantically apt field with the apt reducer.

  group +192   min, seeded FLT_MAX   member +164 = CruisingVelocity
  group +472   sum                   member  +84 = HP

A wrong struct would have to make both offsets land on apt fields AND pair
each with the apt reducer. Minimum of a speed, sum of hit points: a
formation's cruise limit and its total health.

The quantitative test over all 1360 sites, joining each to its craft's
definition:

  value <= the craft's MaximumVelocity    1355 / 1360 = 99.6%   (5 fail)
  value <= the craft's CruisingVelocity   1042 / 1360 = 76.6%   (318 fail)

The test discriminates -- the cruise bound breaks 318 times, the hull maximum
5 -- so the script sets a COMMANDED SPEED, free to exceed the cruise default
and bounded by what the hull can do.

The turret anomaly that stopped me naming this two iterations ago was my own
artefact. UN_e007_ADAN_Turret's definition carries MaximumVelocity 500 and
CruisingVelocity 280: the data models turrets as if mobile, so a script value
of 400 is legal and simply never manifests. I had assumed turrets have no
velocity fields and treated 13% of the traffic as a refutation.

Recorded as unsettled: the five overshoots are UN_e106_ADAN_Destroyer 200 vs
a 150 maximum (x2) and UN_e011_ADAN_Attacker_B_HF/_Wayne 500 vs 450 (x3).
Designer overrides or an engine clamp; not established.

Named set_group_speed. Default = the slowest member's CruisingVelocity;
mode 1 restores it, mode 3 sets it, mode 2 hands it a global constant.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 01:18:30 +00:00
Sylpheed RE agent
5693a1e373 re: built-in 15 targets a GROUP, and +192 is a MIN over its members
The spawn routine sub_8232B538 settles what the field is without needing its
consumer:

  8232B5B0  lfs  f0, 25184(r11)   -> 0x820B6260 = 3.4028235e38 (FLT_MAX)
  8232B5B4  stfs f0, 192(r30)     seed +192 with FLT_MAX
  8232B674  lfs  f13, 164(r3)     each member's own value
  8232B688  fsel f0, f11, f0, f13 f0 = min(f0, f13)
  8232B68C  stfs f0, 192(r30)
  8232BA28  stfs f0, 196(r30)     +196 starts equal to +192

The FLT_MAX seed is what makes it unambiguous: +192 is the MINIMUM of
[member+164] over the members.

So built-in 15's target is a GROUP, not an individual unit -- this file and
my own earlier notes both called it a unit. sub_82348830 hands back a
per-member object and the reduction runs over all of them.

A minimum over members is how a formation limit works: the group goes as fast
as its slowest ship. On that reading +192 is the natural cap and +196 the
effective one. And a CAP explains the turrets where a commanded speed could
not -- capping a static object at 400 is a no-op, so a designer can set it
uniformly from a template without consequence, and the asteroid's invariant 0
fits the same reading.

Recorded as unproven: mode 2's constant is [r27 + 13912] where r27 comes from
a runtime pointer at 0x8232C718, not a static base, so it cannot be resolved
from the image. If it is FLT_MAX then mode 2 is literally "uncapped" and the
three modes are set / uncap / restore. Tidy, and not shown.

The consumer is still not found. Also recorded: the image has NO RTTI at all
(0 of 1150 vtables), so class names are synthetic and the two constructors
writing vtable 0x820AF030 are 28 and 30 instructions touching neither field.
Bounding the group struct is the remaining prerequisite.

Built-in 15 stays unnamed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 01:10:17 +00:00
Sylpheed RE agent
3dc3fbadb8 re: built-in 15's write side verified; its consumer NOT found, filters recorded
Read directly at 0x8232C7CC, the unit message pump switches on [msg+36]:

  mode 1  [unit+196] = [unit+192]     restore the unit's stored default
  mode 2  [unit+196] = [r27+13912]    a global constant
  mode 3  [unit+196] = [msg+40]       the value the script passed

So the field has a per-unit default at +192 and a distinguished global value.
The shipped scripts only ever use mode 3 -- slot 8 is the double tag in
1961/1961 call sites -- so the two defaulting modes are dead in this build.

The consumer I did not find, and the failed approaches are recorded so the
next attempt does not repeat them:

* searching the flight/AI range for 196(rN) gives 170 hits, almost all on
  unrelated structs -- the N(rM) trap this file already warns about;
* narrowing to functions touching BOTH +192 and +196 still leaves 50;
* unit_definition_layout.txt names offsets 192/196 as AV_PitchMinus_Max/Min,
  but that is the unit DEFINITION object (vtable 0x820af844), not the spawned
  entity built-in 15 writes to. It does not apply, and taking it would have
  been an easy wrong turn -- angular-velocity names would have "explained"
  the turret anomaly while being about a different struct entirely.

What would settle it is pinning the spawned-entity struct first, then reading
+196's consumers within it.

Built-in 15 stays unnamed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 01:03:06 +00:00
Sylpheed RE agent
dd3c27f288 re: built-in 15 takes a craft-class scalar — but "set_speed" does not survive
15 is the biggest unnamed built-in: 1360 sites across 27 of 28 stages, with
an invariant (unit, double) operand shape. Joining every call site's unit to
its craft type through stage\UnitGroup_S<NN>.tbl resolves 1360 of 1360, none
unknown, and the values stratify hard:

  capital hulls (Destroyer, Carrier, CruiserEX, Cruiser, Acropolis,
                 BattleshipEX, ASFrigate)                        0-100
  mobile craft  (Elan 500, ArrowHead 400, DeltaSaber_T 400,
                 DeltaSaber_W 600, Attacker_S 250-400,
                 Phantom 300)                                  250-600
  asteroid      (mn040_Asteroid_Big, 74 sites)                  0, only 0

Capital ships crawl, fighters run, the player's craft is fastest at 600, and
a rock never moves. That is a speed's signature.

Except turrets break it. UN_e007_ADAN_Turret carries 400 x89 / 280 x17 /
380 x4 / 250 x2 and UN_e008_ADAN_TurretPlus 450 x62 / 500 x2 -- 176 of the
1360 sites, 13%, at fighter-class values on something that does not move.

The asteroid is what makes this sharp rather than dismissible: a genuinely
immobile object gets 0 every single time, so "immobile things get junk" does
not explain the turrets.

So the operand is a craft-class-dependent scalar with speed's range and
ordering, but set_speed is not a sound reading of it. Left UNNAMED. The
handler writes [unit+196], initialised from [unit+192] at spawn; reading its
consumer is what would settle it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 00:56:28 +00:00
Sylpheed RE agent
fa20db3db8 re: four ISL built-ins are tutorial-only, and 96 encodes the tutorial index
96, 97, 98 and 104 were unnamed. Measured over all 28 stages they form a
family that appears nowhere outside the six tutorials:

  96    8 sites   S18-S23 only
  97   38 sites   S18-S23 only, followed by start_coroutine 27/38
  98    0 sites   never called anywhere
  104  39 sites   S18-S23 only, followed by wait_s 39/39,
                  preceded by end_coroutine 37/39

104's adjacency is a textbook poll loop with no exceptions: coroutine starts,
tests the predicate, waits, repeats.

96's operand is the giveaway. Its eight payloads in stage order are
S18 -> 101, 102, 103; S19 -> 201; S20 -> 301; S21 -> 401; S22 -> 501;
S23 -> 601, i.e. (stage - 17) * 100 + section. Tutorial 1 has three sections
and the other five have one each, and the tutorial index is encoded in the
argument.

Names are deliberately NOT applied. tutorial_begin / tutorial_end /
tutorial_message_pending fit the shape, and [phase+340]/[phase+344] are said
to carry the state and payload, but I have not read that handler myself and
this file has already had to withdraw five names guessed from shape. The
distribution and the argument encoding are established and are what a port
needs; the labels can wait for someone to read the code.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 00:50:24 +00:00
Sylpheed RE agent
cb830e233e re: name the five mission-banner built-ins from call-site structure
77, 78, 81, 82 and 135 were unnamed. The engine has five contiguous strings
-- MISSION_START_PRT at 0x820A83F0, then _END_, _UPDATE_, _FAILED_,
_RESTART_ -- and five sequential ScriptPhase fields at +388/+392/+396/+400/
+404, stored in ascending order by one constructor region. Five names, five
fields, five unnamed built-ins.

Which is which is decided by call-site structure, measured over all 28
stages, and it is exact:

  39 MARK_LAST_PHASE   89 sites  -> 82 in 89/89
  82 banner_mission_failed        <- 39 in 89/89, then wait_s 89/89
  40 mark_not_last     50 sites  -> 78 (27) + 81 (17) + END_PHASE (6) = 50
  78 banner_mission_complete      <- 40 in 27/27
  81 banner_objective_update      <- 40 in 17/17
  77 banner_mission_start         22 sites in 22 stages, one per stage,
                                  after play_bgm
  135 banner_mission_restart      16 sites, after play_bgm, phase >= 2

39 -> 82 is a perfect pairing and 40's sites partition exactly three ways.

Stated as inferred rather than read: the string-to-field pairing itself comes
from both sequences ascending in the same order; my operand tracker did not
catch the string loads in that constructor. The ROLES above do not depend on
it.

76 is left unnamed on purpose. It has 38 sites = 22 + 16, exactly 77's count
plus 135's, and precedes them; its body sets [phase+332] = 1 and nothing in
the image reads that field. Suggestive arithmetic is not a name.

Flagged as a consequence: MARK_LAST_PHASE is followed by the FAILED banner in
89 of 89 sites and mark_not_last by END or UPDATE. So [phase+300] = 2 reads
less like "this is the last phase" than "end the mission now,
unsuccessfully" -- the existing names for that pair may be mis-framing it.

Artifact regenerated: docs/re/data/isl-stage02.txt.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 00:44:38 +00:00
Sylpheed RE agent
a32c00057e re: recover the .slb leading segment — mono, and scoped by measurement
to_xma_riffs now emits the leading headerless segment when it sits at a whole
number of XMA1 packets and carries a non-zero byte. VOICE_D_453 goes from a
0.14 s trailing fragment to a 45116-byte leading sub-wave that dominates it.

I withdrew this exact change earlier for two reasons. Both are now answered
rather than argued away:

* "It recovers no audio" -- it used the STEREO format. At two channels every
  bank yields exactly 1792 bytes, one frame, whatever its size. Mono yields up
  to 113x more.
* "It matches 1524 of 8021 RIFF-bearing entries" -- the byte-level reach is
  still 1524, but the audible reach is not. Across the 84 movie-bound banks
  the segment adds >1 s to exactly 7, the hokyu_*_H tankers on D_453/D_454 --
  precisely the broken ones -- and <=0.25 s to 66 of the rest. The largest
  non-resupply addition is S04A at +0.66 s on a 256 s movie.

The safety oracle is recorded with its limits: 8 of the 84 banks ALREADY
exceed their movie's duration before the change, by hundredths of a second,
so it cannot resolve differences at that scale. It establishes scoping, not
correctness. Callers clamp to the movie length regardless.

VOICE_D_451's all-zero leading region is skipped by the non-zero guard, so
the rule cannot prepend silence to a bank that does not need it. Pinned, as
is the packet arithmetic (n = 8, 1, 7, 22, 29) which has no tunable.

slb_disc, movie_subtitle_disc and movie_manifest_disc all still pass.

NOT verified by ear -- that needs a human.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 00:37:06 +00:00
Sylpheed RE agent
ea0eedda86 re: the .slb parts are sequential segments — and that closes the original mystery
The last open question was whether the leading XMA1-mono region duplicates the
RIFF sub-wave, which would make the earlier totals double-count. It does not.
Decoding both parts of every bank to mono PCM and measuring energy:

  bank         leading secs / RMS      riff secs / RMS
  VOICE_D_450     0.49 /    158          2.82 / 9898
  VOICE_D_451     0.01 /      0          1.58 / 9128
  VOICE_D_452     0.31 /    301          2.18 / 9061
  VOICE_D_453     2.12 /   9770          0.14 / 14462
  VOICE_D_454     3.07 /  10428          0.43 / 11639

Two shapes, and no bank holds the same content twice. In 450/451/452 the
leading region is silence or near-silence (RMS 0-301 against ~9000 for
speech) and the RIFF holds the line. In 453/454 the leading region holds the
line and the RIFF is a short loud tail fragment. Sequential segments of one
clip, so the totals stand and with them the 48 kHz fit.

This also closes the mystery that started the whole thread. The corpus
recorded 450 = 2.8 s, 451 = 1.6 s, 452 = 2.2 s as plausible but 453 = 0.14 s
and 454 = 0.43 s as "far too short". The decoder skips everything before the
first RIFF: for the first three that discards only silence, so they looked
fine; for the last two it discards the line itself and leaves the trailing
fragment. One rule, two outcomes, depending on which segment holds the
speech.

The fix is now well-posed in a way the withdrawn attempt was not: emit the
leading region only when it carries signal. That also avoids the 1524-bank
blast radius that sank the earlier version.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 00:26:24 +00:00
Sylpheed RE agent
b32006d2a8 re: RETRACTED "audio is missing" — a subtitle cue is a START time
The load-bearing error of the whole voice-bank thread, and it is mine. It
stood for three iterations across two write-ups that each called the result
proven.

I treated a subtitle cue as a timestamp that must fall INSIDE the voice clip,
and concluded a 0.07 s clip could not host a cue at 4.70 s. A cue is when the
line STARTS. The voice plays from the cue, so the clip only has to fit the
window between the cue and the end of the movie. Under that reading every
bank fits at plain 48 kHz:

  bank     samples  @48kHz    cue   window
  D_450     158967    3.31   4.00     5.30
  D_451      76084    1.59   3.70     5.60
  D_452     119562    2.49   0.00     8.34
  D_453     108608    2.26   4.70     4.60
  D_454     167828    3.50   0.00     9.50

2-3.5 s is also the right length for the lines. Nothing is missing, and the
17091-20563 Hz window from the previous commit is void with it -- its lower
bound came from the same misreading.

What survives, because it was measured rather than interpreted: the leading
region is XMA1 mono, the decode runs to the final frame, and cue values are
seconds.

Separately settled, and it is what exposed the error: each shared bank holds
ONE generic line. The 3-5 movies bound to a bank have IDENTICAL subtitle
text, 5 banks out of 5 -- "Rhino 3 has landed. Commencing resupply.",
"Resupply complete. You are cleared for take-off!", and so on.

That also explains the historical in-game rejection of hokyu_DS_s13A ->
VOICE_D_452 that started this whole thread. The line is generic, identical
for s02A/s07A/s08A/s13A. Someone expecting a stage-13-specific line would
hear the generic one and call it wrong -- while the binding is exactly right.
The disc said so; the subtitle text now says so independently.

Still open: whether the leading mono region is additional audio or an
alternate take, since the totals above add it to the RIFF sub-waves.

Artifact: examples/shared_bank_takes.rs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 00:20:57 +00:00
Sylpheed RE agent
b324278ae9 re: cue times ARE seconds — and "the rate does not converge" was my error
Two things settled, one of them a correction of my own claim from the last
iteration.

The cue unit is verified rather than assumed. parse_timing computes mm*60+ss,
but only if the token really is mm:ss.cc, so I checked against an independent
oracle: the movies are on the disc and a cue must land inside its own movie.
66 English movies with subtitle tracks, 0 cues land after the movie ends.
Centiseconds would have overflowed essentially all 66. The seconds reading
stands and the verdicts built on it survive.

"The sample rate does not converge" does not. I reported implied rates of
39742 / 20563 / 23108 Hz as irreconcilable. They are not estimates of the
same quantity -- each is a ONE-SIDED BOUND. The audio must be at least as
long as the last cue, so samples/cue is an UPPER bound; it cannot outlast its
movie, so samples/movie is a LOWER bound. Intersecting:

  bank         samples    cue   movie   lower Hz   upper Hz
  VOICE_D_450   158967   4.00    9.30      17091      39742
  VOICE_D_451    76084   3.70    9.30       8180      20563
  VOICE_D_453   108608   4.70    9.30      11677      23108

  => 17091-20563 Hz, non-empty. A single rate IS consistent.

I had been comparing them as competing point estimates, which is why they
looked contradictory.

What is still open, and stated as such: that window contains no standard XMA
rate. The lower bound assumes a whole bank plays inside one movie, and each of
these banks is bound to 3-5 movie slots -- so if a bank holds several takes
the lower bound is void, leaving rate <= 20563, which 22050 nearly meets.
Next step recorded: establish whether a shared bank is one line or several.

Artifact: examples/cue_unit_check.rs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 00:14:31 +00:00
Sylpheed RE agent
978ccf882a re: FFmpeg stops on the LAST frame — my "partial decode" claim was wrong
I had been discarding FFmpeg's stderr with Stdio::null(). It says exactly
what happens: an unimplemented "Reserved bit", then a negative bit-skip --
and the failing frame index is always the last one.

  bank         packets  frames  fr/packet  fails at
  VOICE_D_450        8    45.7       5.71        44
  VOICE_D_452        7    29.4       4.21        28
  VOICE_D_453       22   198.9       9.04       198
  VOICE_D_454       29   287.5       9.92       287

So the previous entry's "the decodes are visibly partial" is wrong, and it
was mine. I read "samples per input byte ranges 2.10-4.96" as truncation; it
is ordinary XMA1 variable bitrate. Only the final frame of each stream is
lost. The decode is essentially complete.

The sample rate still does not converge. I tried the obvious repair --
counting the whole bank, leading region plus RIFF sub-waves, since the two
split the audio very differently per bank. Two banks then agreed at a tidy
~2.1x ratio pointing near 22 kHz, and the third refuted it: implied rates are
39742, 20563 and 23108 Hz.

So the container is identified, the decode is essentially complete, and the
duration still does not reconcile -- which moves suspicion to the other side
of the comparison. movie_subtitle::track_voice_cues returns (u32, f32) and I
have been reading that f32 as SECONDS on the strength of the format notes
describing mm:ss.cc cue text. If it is centiseconds, a frame index or a
per-page offset, every "audio missing" verdict inherits the error. Recorded
as the next thing to check, and to be checked BEFORE any more audio work.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-26 00:00:59 +00:00
Sylpheed RE agent
116fd7d2ff re: the .slb leading region is XMA1 MONO — 113x more audio than stereo
Retried the format probe with the fmt chunk built to synth_xma1_fmt's exact
byte layout, and with the bank's own RIFF sub-wave decoded through the same
pipe as a CONTROL so a broken harness cannot masquerade as a result.

The channel count is the whole story:

  bank         lead B   channels=2   channels=1
  VOICE_D_450   16384         1792        46756
  VOICE_D_451    2048         1792          896   (all-zero region: control)
  VOICE_D_452   14336         1792        30154
  VOICE_D_453   45056         1792       203648
  VOICE_D_454   59392         1792       294440

channels=2 yields EXACTLY 1792 bytes for every bank regardless of size -- one
frame, then it stops. That constant is the tell. At channels=1 the same data
yields up to 113x more, and the control sub-wave decodes to 13568, so the
pipe works.

Why the previous probe got 0 bytes everywhere is now named: I read
synth_xma1_fmt(2, 2, 48000)'s second argument as a STREAM COUNT when it is a
CHANNEL MASK, and built the WAVEFORMATEX around that misreading.

Also recorded as a refutation, because it was tempting: solving for the
sample rate as decoded-samples / last-subtitle-cue does NOT converge. D_453
implies 21665 Hz -- close enough to 22050 that I nearly wrote it down -- but
D_450 implies 5844 Hz. No single rate explains both, and the decodes are
visibly partial (samples per input byte ranges 2.10-4.96 where a clean decode
would be near-constant).

So the container is identified and the duration is not. Next step recorded:
find why FFmpeg stops early, likely the hardcoded packet/subframe fields.

Artifact: examples/slb_fmt_probe.rs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-25 23:53:10 +00:00
Sylpheed RE agent
af320466bb re: the resupply banks really are missing audio — the subtitles prove it
The corpus said 0.14 s is "far too short for the spoken line". That is a
judgement about audio, and audio judgements cannot be made in this container.
The subtitle tracks settle it without listening: each carries cue START
times, and a subtitle that appears at t seconds cannot sit inside a clip
shorter than t.

FFmpeg-measured (not estimated from a compression ratio -- the first version
of this example used an 8:1 guess, which is not good enough to hang a
conclusion on):

  hokyu_LS_s02A  D_450  cue 4.00 s  audio 1.41 s  MISSING
  hokyu_LS_s09A  D_451  cue 3.70 s  audio 1.81 s  MISSING
  hokyu_LS_s02H  D_453  cue 4.70 s  audio 0.07 s  MISSING
  hokyu_DS_s13A  D_452  cue 0.00 s  audio 1.21 s  no signal
  hokyu_DS_s07H  D_454  cue 0.00 s  audio 0.21 s  no signal

Three of five are decisive; the other two have their only cue at 0.0 s and
say nothing either way. So something is genuinely missing from these banks --
established independently of the leading-region work, and measured rather
than felt.

The fmt-variation probe I recorded as the next step is INCONCLUSIVE and is
written up as such: 36 combinations over VOICE_D_453's 22-packet leading
region all produced 0 PCM bytes, including ones that should be equivalent to
the crate's own synth_xma1_fmt, which does parse. So the probe tested my
hand-built fmt chunk, not the hypothesis, and it is NOT evidence that the
region is non-XMA. The retry should use the crate's helper.

Artifact: examples/voice_len_vs_subs.rs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-25 23:45:26 +00:00
Sylpheed RE agent
49a09a9496 re: the "sibling default" rules are a dedup artefact — WITHDRAWN
The corpus recorded that some unit fields the disc leaves defaulted inherit
from a sibling: Size_Y from Size_X, FCSRange from RadarRange, DefencePoint
from AttackVesselPoint. Size_Y was marked the one to trust, on 9/9 support
across 7 independent ships, and it is restated in INDEX.md.

The premise is false. These fields are not defaulted -- they are on disc for
113-114 of 114 unit tables -- and Size_Y DIFFERS from Size_X in 90 of them.

The mechanism, cross-tabulating "legacy reader missed it" against "equal on
disc":

  pair                          seen+differ  seen+equal  miss+differ  miss+equal
  Size_Y / Size_X                    90          0            0           24
  FCSRange / RadarRange              54          0            1           58
  DefencePoint / AttackVesselPoint   51          0            1           61

seen+equal is 0 for all three: a value shared with a sibling is ALWAYS
invisible to the string-pool reader, because the pool stores each distinct
string once. And the reader almost never misses a value that differs. So
"the missing value equals the sibling's" was true BY CONSTRUCTION -- the rule
re-derived the very condition that made the field go missing. That is why the
support looked perfect: it could not fail on the cases it was fitted to.

The two miss+differ cells are its real wrong predictions, both named:
UN_e104_ADAN_Carrier DefencePoint is 0.2 (rule says 0.003), and
UN_e011_ADAN_Attacker_B_HF_Wayne FCSRange is 3000.0 (rule says 6000.0).

Retracted in unit-struct-runtime.md (original reasoning kept below the
correction), live-unit-definitions.md and INDEX.md. Pinned by a disc test
that asserts the seen+equal cells stay zero, so the mechanism itself is
guarded, not just the counts. Artifact: examples/sibling_rule_check.rs.

This one was found by my own check after the subagent assigned to it stalled.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-25 23:38:55 +00:00
Sylpheed RE agent
6fa6564be8 re: the .slb leading region is 1392+n*2048 — and my fix for it is withdrawn
The structure is exact. In all five resupply banks the first RIFF sits at
HEADERLESS_DATA_OFFSET + n*2048, where 1392 is a constant this crate already
had and 2048 is the XMA1 packet size: n = 8, 1, 7, 22, 29. No free parameter
to tune, and the raw bytes agree -- high entropy from offset 0, then a zero
run immediately before the RIFF. VOICE_D_451 is the control, its single
packet being all zeros.

So I made the obvious fix, emitting that region as a sub-wave, and then
withdrew it on two measurements:

* It does not recover audio. Coverage went 5.4% -> 89.9% for VOICE_D_453, but
  the emitted stream decodes through FFmpeg to 1792 PCM bytes -- silence --
  while the RIFF sub-waves from the same banks decode to 150-270 KB. Byte
  coverage was the wrong success metric and it looked like progress.
* It is not narrow. The rule matches 1524 of the 8021 RIFF-bearing entries in
  sound.pak, including RT* movie banks that decode correctly today. Landing
  it would have risked a wide regression in order to not-fix five banks.

to_xma_riffs is back to its previous behaviour, verified by re-measuring:
coverage is 5.4% / 9.7% again. The refuted attempt is recorded in the code
beside the branch it would have changed, so the next person does not
re-derive the arithmetic and re-make the change.

XMA1_PACKET is kept as a named constant because the blast-radius scan uses
it. Artifacts: examples/voice_bank_shape.rs (structure), voice_bank_dump.rs
(sub-waves for decoding), slb_hybrid_scan.rs (the 1524 count).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-25 23:30:42 +00:00
Sylpheed RE agent
fedb31a5f9 re: rebuild the movie manifest on the record table — the old counts measured pool strings
movie_manifest::parse now reads BASE_INFO's positional field keys (the game's
own cutscene ids, stage*100 + slot) and follows each to its record, instead of
scraping the string pool. The pool stores each distinct string once, so a
REPEAT reference produced no token and read as "no binding".

That single cause explains every wrong cell: 13 later references to
VOICE_D_450..454, two to SUBTITLE_hokyu_LS_s11A.tbl, and MS01A's share of
pwterop_s01a.prt. All 18 hokyu movies are bound, not five.

Counts, verified independently by me against the disc before recording:
104 cutscene SLOTS binding 101 distinct MOVIES; 99 slots / 96 movies with a
voice track, 99 / 96 with a subtitle, 22 / 22 with a telop. The docs' old
94 / 83 / 21 are exactly the counts of DISTINCT POOL STRINGS -- not wrong
measurements, measurements of the wrong thing. Three denominators were being
conflated; the new test pins all three.

Two assertions in movie_manifest_disc.rs were false and are corrected:
hokyu_DS_s13A binds VOICE_D_452 and resolves to eng\etc\VOICE_D_452.slb. The
in-game verdict that rejected that value tested an INFERENCE from a shared
demo id, on a decoder that discards 85-87% of banks in this class -- see
voice-bank-leading-region.md, committed earlier today.

The ~104 script ids are no longer open: they are literal positional keys,
each naming its record, and all 104 resolve. The old "counts differ by three,
positional pairing does not work" has a concrete cause -- three resupply
movies are bound by TWO slots each.

Also corrected: the naming convention has 3 subtitle exceptions (s24A/s27A
borrow s11A's track) and 18 voice exceptions, not one and five.

The legacy scraper is kept as a fallback for blobs with no record table, so
the synthetic unit fixtures still exercise it.

Artifacts: examples/movie_map_csv.rs regenerates the CSV, now slot-keyed
(104 rows; the movie-keyed version silently dropped one slot of each
duplicate). Disc tests green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-25 23:13:41 +00:00
Sylpheed RE agent
49c00e0955 re: the voice decoder discards up to 87% of a bank — "multi-subwave" refuted
The record table gives a DIRECT binding hokyu_DS_s13A -> VOICE_D_452, where
the corpus records the movie as unbound and movie_manifest_disc.rs asserts
None, citing an in-game verdict that this exact value was "the wrong
recording". That is the only place on the disc where a runtime observation
disagrees with the record table, so it was worth settling.

First, shape: these banks are SHARED. Five slots bind VOICE_D_452, five bind
451, four 450, four 453, three 454 -- 21 hokyu slots over five banks, and the
movies repeat too. Generic resupply cutscenes, not per-stage recordings.

The recorded explanation for 453 decoding to 0.14 s and 454 to 0.43 s was
that the banks are "likely multi-subwave / not cleanly sliced". Refuted: the
count of RIFF magics EQUALS the number of sub-waves recovered in all five
banks, and the last data chunk ends exactly at EOF in four of them. Nothing
between or after sub-waves is being missed.

The real defect: slb::to_xma_riffs finds audio by searching for the RIFF
magic, and a large region PRECEDES it. 87% of VOICE_D_453 and 85% of
VOICE_D_454 sit in front of the first RIFF -- 21-27% zero over 256 distinct
byte values, i.e. content, not padding. VOICE_D_451 is the control: its
leading region is 100% zero, 1 distinct value, real padding.

So the in-game verdict listened to a decode that had discarded most of the
bank, for exactly this bank class. It is evidence about the decoder, not
about the mapping. Note also that what was rejected was a value INFERRED
from a shared demo id; the record table supplies the same value as a stored
field, and only the inference was ever tested.

This does NOT establish the binding is right -- it removes the only recorded
evidence against it. What the leading region actually holds is undecoded, and
confirming the binding needs a human listening.

Artifact: examples/voice_bank_shape.rs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-25 23:03:33 +00:00
Sylpheed RE agent
75f0664bfe re: an ISL symbol operand is a (tag, index) pair — and two more names withdrawn
Verified rather than adopted: a subagent proposed that every even operand
slot is a type tag. Measured, the strong form is false and a precise form is
true.

TRUE: a SYMBOL operand is two words, a tag holding the constant 1 followed
by the index. Slot 0 is the integer 1 in 19899/19899 calls whose slot 4 is a
unit; slot 8 is tag-shaped in 100% of calls for every built-in taking a
second unit; slot 16 is 1 in 152/152 for built-in 128, the only one taking a
third. The 24 built-ins whose slot 0 is NOT the constant are exactly those
taking no symbol there. This explains the unit slots 4/12/20 rather than
replacing them.

FALSE as stated: slot 8 is a bare double for built-ins 4, 20, 24, 26, 28,
29, 90, 106 and 127, and built-in 75 carries five bare indices at 0/4/8/12/16
with no tags at all. Each built-in has a fixed signature and is 100%
self-consistent; none of the 34 with >=20 sites mixes the two.

Symbol table 1 has three types -- 1 routes (1362), 6 messages (2247), 7
effects (81) -- and its operand slots are type-pure, measured the same way.
Resolving them makes listings say what the script means:
`request_script_message(MSG_VOICE_D_257, ...)`, a fourth independent
confirmation of that name. Slots 24@4, 46@12 and 114@4 resolve 100% but MIX
types 6 and 1, so they are left unresolved rather than guessed.

Two more names withdrawn, neither replaced:
* 88 `camera_at` -- ZERO call sites in all 28 stages; never testable.
* 90 `camera_at_route` -- 8 sites, all Stage 02 phase 3, first operand is
  symtab-1 type 7 `eff_n0071`, an EFFECT name, in 8/8, with a per-missile
  Route_ADT301..308_p3M at slot 20. Not aimed at a camera.

Left unnamed on purpose: replacing a guessed name with another guess is how
the three names corrected earlier today went wrong.

Also flagged: 115 `named_event`'s only symbol operand is an eff_* name in
84/84 sites, so that name is suspect too. Not renamed pending a handler read.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-25 22:51:07 +00:00
Sylpheed RE agent
608a0cc710 re: measure the boot-nav title gate — and withdraw my own diagnosis
Last iteration I said launch_mission died because skip_intro only tests for
the title on a static frame, gated at rmse <= 1500, and that run logged 1503
and 1549 just above the cut. I also said the fix was NOT to nudge 1500 but
to measure both signals through a boot first. Measured, and the diagnosis
does not survive.

boot_trace.sh logs the two signals skip_intro decides on -- frame-to-frame
RMSE and the is_title.py green-glyph count -- through a clean boot with no
presses at all. One run, 29 samples over 484 s:

  8 samples had rmse <= 1500, so the gate OPENED eight times
  0 samples had glyph > 0, so the title was never seen

At t=145 s the RMSE was 1205, comfortably inside the threshold, and the
glyph test was called and answered zero. A frame can be perfectly static
without being the title -- the intro movie has long quiet stretches, three
reading RMSE exactly 0. So 1503/1549 were almost certainly movie frames too,
and raising the constant would have admitted two more of them.

What is left is narrower and honest: the interactive title never appeared,
rather than appearing and being missed by a threshold.

The limitation is recorded rather than buried: the tracer intended 1 s
sampling and achieved 16.9 s, because each iteration forks two screenshots,
ImageMagick compare and a fresh Python. So this does NOT prove the title
never appeared -- only that it was absent from 29 samples. A window shorter
than ~17 s falls between them. The recorded next step is to make the tracer
sample at the rate it claims before concluding anything stronger.

Artifact: docs/re/captures/boot-signal-trace.tsv.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-25 22:42:33 +00:00
Sylpheed RE agent
501298de81 re: the trigger container's "push" is a clear — and the live test is blocked
Third independent line for yesterday's built-in 100 rename, from the callee
this time. sub_8226E3B8 was labelled "push", which is what made built-in 100
look like push_trigger. It reads the element count, returns immediately when
the container is EMPTY, and otherwise walks the node list splicing nodes out
until it is empty. A push links one node; this unlinks all of them. It is
clear(). The append is sub_8226E160, reached from built-ins 19 and 25.

So the rename now rests on the handler, the usage (all 12 Stage 02 sites sit
in the phase terminator next to timer_stop / clear_flag(-1) /
MARK_LAST_PHASE), and the callee.

The dynamic half did NOT run, and the write-up says so. phase_watch.py now
samples [phase+272+20] (triggers queued) and [phase+216+8] (coroutines
alive) so a phase terminator's effect on the VM is visible in one line —
written here, never yet exercised against a live guest.

Boot-nav could not reach the title in 381 s. Diagnosed rather than retried:
skip_intro.sh only runs the title test on a static frame, gated at
rmse <= 1500, and this run measured 1503 at 104 s and 1549 at 139 s — just
above the cut — so is_title.py was never called and the one allowed press
was never spent. Recorded in BACKLOG with the explicit instruction NOT to
raise the constant: the first step is to log rmse and the glyph count
through a whole boot and look at the two distributions, because tuning a
threshold to make one run pass is fitting to a single sample.

Also reaped a stale lock: a gdb orphaned 2h14m earlier was holding
/tmp/xenia-canary.lock with an already-defunct emulator child.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-25 22:25:10 +00:00
Sylpheed RE agent
9a9921b759 re: three ISL built-in names were wrong, including the most-used one
All re-read twice — the handler, and the thing it calls — because each had
been named from its shape rather than its effect.

* id 11 `yield` -> `end_coroutine`. 0x82272624 is li r11,1 ; li r3,3 ;
  stw r11,164(r31), and the dispatcher's r3==3 arm erases the thread from
  the active list and returns it to the free list. It destroys the thread.
  2945 sites game-wide, 372 in Stage 02 — the most-used built-in there was.
* id 5 `await_label` -> `kill_coroutine(label)`. sub_82273B08 kills the
  thread parked at the target pc, or itself if the target is its own pc.
  It waits for nothing.
* id 100 `push_trigger` -> `reset_phase_threads`. It clears the trigger
  container and then frees every thread whose pc differs from the caller's
  — the opposite of pushing a trigger. Corroborated by usage: its 12 Stage
  02 sites all sit in the phase terminator, next to timer_stop,
  clear_flag(-1) and MARK_LAST_PHASE.

One name recovered from the game's own text: opcode 992 prints
"RequestScriptMessage %s" at 0x820A5700, so id 64 is request_script_message
(2683 sites).

Return codes documented properly: 1 = restart the coroutine from its entry
(previously not recorded at all), 3 = terminate. And the blocking set was
wrong in two places — it is 102, 120, 137, 142, 143. Id 97 does NOT block;
its handler ends `b 0x822724F8`, so it always returns 0.

Unit-operand resolution settled from DATA over all 28 stages rather than by
reading 147 handlers: a slot qualifies only if every value is a valid
symtab-2 index, it takes >=15 distinct values, AND its maximum reaches most
of the table — that last clause is what discriminates, since every small
integer is trivially "in range". 31 built-ins at slot 4, 8 at slot 12, one
at slot 20. It also refutes set_flag's slot 0, whose maximum overruns the
table, and the resolver now declines rather than inventing a name.

New and unexplained: symtab-2 holds two types, 2 and 8, and built-ins 95 and
128 take type 8 at slot 12 in 100% of their sites.

A downstream inference is withdrawn with it: the note reading the live
trigger counter attributed it to "the script arming watches as it goes" via
built-in 100. The measurement stands; the attribution does not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-25 22:09:05 +00:00
Sylpheed RE agent
9141ea2b24 re: audit the legacy IDXD reader against the real field table — and fix a test that encoded its error
With the record table decoded there is finally a ground truth to check the
old string-pool reader against. It infers `key -> value` from pool adjacency,
which is a consequence of how records are written, not a rule of the format.

Verified by hand against the disc, with an independent parser:

* `FCSRange` = 500000.0 — the module docs' own canonical example of a field
  "left at its default" that "omits the value string".
* `ShieldRatio` = 1.0, where `tests/pak_idxd_disc.rs` asserted None with the
  comment "a defaulted/omitted field must be None". That test encoded the
  false belief; it now keeps the None as a deliberate characterisation of the
  legacy reader, with the true value asserted beside it.
* `get_raw("Model")` on GP_HANGAR_ARSENAL returns the first record's model for
  every record — silent corruption, not an absent value. New test pins four
  records that disagree with it.

The cause is the flat API having no way to name a record: only 548 of 6325
objects have one. `HP` on the DeltaSaber answers 1000.0, the hull, while 63
Turret_* records each carry their own 100.0 (measured — a first draft said 34,
taken from a report rather than from the disc).

Disc-wide rates are recorded as single-source and labelled as such: get_raw
52% wrong, typed getters 38% miss, but 100% correct on single-record objects.

Also records a negative result: the 504 unnamed field keys were NOT recovered.
A 572464-string dictionary and 73191 variants gave 0/42. The key deltas do
prove the preimage ends with the two decimal digits.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-25 21:42:51 +00:00
Sylpheed RE agent
af32540190 re: decode the IDXD/IXUD record table — and there is no schema hash
The binary region in front of the string pool was the parser's oldest open
note ("Not yet decoded"). It is a uniform 16-byte record array sorted by
name hash, a field count, a 12-byte field array sorted by key, a pool size,
and the pool. The trailing `pool_size == file_len - pool_base` identity makes
the layout self-checking, which is what caught the first wrong version.

Verified over the WHOLE disc with zero failures: 7750/7750 IDXD objects,
190782/190782 records reproducing their stored tag_hash, 1271462/1271462
named fields reproducing their key. IXUD is the same container with
ixud_hash, UTF-16BE and every offset in chars — 1104/1104 objects,
628165/628165 fields, checked with an independent parser.

Field names are stored on disc, so no preimage search is needed: a field's
middle word points at its own name. Only 504 fields disc-wide are hash-keyed
with no name; the other 1485073 nameless fields are positional, keyed by a
literal integer (line slots, movie ids).

Two long-held beliefs are WITHDRAWN:

* The word at 0x08 is not a schema hash. It is record 0's name_hash — the
  format has no type field at all, and an object's kind is known only from
  the caller that loads it. It survived as "schema" because tables of one
  kind share their lowest-hashed record name. Caught by a test asserting
  every movie id names a real record: 1005 -> STAGE10_PHASE01 failed because
  tag_hash("STAGE10_PHASE01") IS 0x067025B9, that table's supposed schema id.
* The field's middle word is not an always-0xFFFFFFFF flags word. It is
  0xFFFFFFFF for 54% of fields, enough to look constant in a small sample;
  the tell was that it is constant per key ACROSS records, which a per-record
  flag cannot be but a per-name pointer must.

`schema_hash` keeps its name rather than churn 33 call sites, with corrected
docs. The first sweep globbed dat/** and missed hidden/DefTables.pak (1425
objects); the test now walks the whole disc root.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-25 21:40:23 +00:00
Sylpheed RE agent
34c99a1568 re: decode the IXUD localised text -- 534 entries, six languages
534 IXUD entries across 36 paks, 94 distinct keys, 92 resolved by name. Content
identified: mission objective/lose/hint panels, 91 bonus-objective banners,
scripted radio, cutscene dialogue, the reactive combat-chatter pool (MSG_ADAN
23,236 fields, MSG_RHIN 21,196, ...), resupply barks, speaker labels and the
hangar weapon Name/Desc/Condition.

Verified independently: language\MissionDialog_local_string.tbl decodes to 600
fields with the trailer identity holding, giving Stage 02's real objectives --
phase 1 'Shoot down all invading enemy fighters / while watching out for attacks
/ on the ACROPOLIS', phase 2 'Protect the Admiral's ship the CALIBAN', phase 3
'Destroy the interstellar cruise missiles'. That matches the script exactly: the
phase-1 condition polls three enemy squadrons and the escort's loss is the fail
branch, and phase 2 is positional, which is why no kill count appears there.

The .ssb symbol names (S02_P1_OBJECTIVE etc.) are NOT IXUD keys -- they are
records in an IDXD map, message\MissionDialogMessage.tbl, whose positional
fields list the lowercase per-line IXUD names; *_GRAPH holds a texture instead.

Structure: one record per object (except the six 63-record hangar files),
624,440 named fields, 55% of them empty line slots, no control characters, the
only escape a literal two-char \n in block-text tables, and records/fields in
ascending key order so lookup is a binary search. '[OB]' turns out to be a HUD
marker label in this text -- the same one the REMAINING OB work chased.

Six languages confirmed D/E/F/I/J/S; J omits the trailing empty line slot in 78
of 86 objects. Two keys remain unresolved, byte-identical across all six paks
(untranslated Japanese voice transcripts).

Also corrects mission-objectives-text.md: SUBOBJ_009 does have text, and
SUBOBJ_005 is two fields.
2026-08-25 21:01:55 +00:00
Sylpheed RE agent
dc2054f0be re: name the four squadron order classes
Three come from the game's own strings, verified present: 'Set Squadron order
attack/trace/escort' at 0x820AF17C/1A4/1CC, loaded once each at entry to the
unit message pump sub_8232C4C0 so the pairing is unambiguous. The printf is
compiled out (bl 0x82674028 = li r3,0; blr) but the format strings and arguments
survive, which is what makes them usable as labels. The same poster functions
serve both the message path and the interpreter-command path, so the identity
transfers to opcodes 517 trace / 784 attack / 790 escort, i.e. ISL built-ins
46/47/48, now named in isl.py.

The fourth, opcode 512, is recorded at LOWER confidence on purpose: those strings
name only three. Its label rests on the route-generation strings on its own
vtable, a symbol-table-1 (route) argument rather than the unit table, a route
point-count lookup, and the pre-existing move_order label -- four independent
lines, but none of them the game naming it.

Also records a trap: sub_82320B48's ORDER_{WINGMAN,SQUAD,SQUADRON}_{FORMATION,
ATTACK,COVER,EXTENDED} enum looks like it should map onto the order classes and
there is no evidence it does. It is a voice/comms axis; not connected.
2026-08-25 21:00:56 +00:00
Sylpheed RE agent
770ef2ea2b re: CONFIRMED -- the poke produced 'absent', not 'destroyed'
Settled from the disassembly, no run needed. Built-in 69's tail maps the
lifecycle lookup into [phase+164]: handle == 0 takes the early exit at
0x8226AF44 and returns 0, while the destroyed states return 2, 3 or 4 (and two
of those also normalise the record's +16). Those are different values.

So poking rec+4 = 0 made the predicate report ABSENT -- the same answer an
undeployed unit gives -- and never the answer the script branches on. The
condition was polling at 5 Hz throughout and correctly saw 'not here'.

Both null results are now fully explained, and neither was evidence about the
condition: the first poke wrote a field nobody reads, the second wrote the wrong
value into the right field. Simulating a kill needs the handle to stay valid
while the lifecycle lookup returns 3/4/5, i.e. the write belongs in whatever
sub_82301240 reads, not in the script's own record.

Also names built-ins 46/47/48 as squadron_trace / squadron_attack /
squadron_escort in isl.py.
2026-08-25 20:55:49 +00:00
Sylpheed RE agent
72e24acdfe re: WITHDRAWN -- the phase-1 condition IS polled, at 5 Hz
Tracing Stage 02 phase 1 by control flow (following op 12 jumps and the six
conditional branches) gives the whole chain: timer5 @ 4.0s -> trigger 0x2D30
(which also does timer_set/timer_resume) -> start_coroutine at 0x2FE8 ->
coroutine 0xBB9C, whose body is wait_s(0.2) ; yield -> ... -> 0xF524, the
ADN110/111/112 polls that latch set_flag(8).

Exactly one of phase 1's 79 coroutine entries reaches 0xF524 under a real
control-flow walk, and it is a 0.2s polling loop. So the condition is checked at
5 Hz from 4 seconds into the phase, and my 'the condition coroutine is not
polling' explanation -- which I had promoted to real evidence after the poke
experiments -- is withdrawn.

Two weaker methods gave wrong answers first and are named: linear decode fell
through into unrelated code and implicated 30-odd entries; 'nearest preceding
entry' implicated the right one only by proximity luck, 0x3988 bytes away. And I
matched start_coroutine operands across phases at first, which is meaningless
since the operand is relative to [phase+232] -- the same per-phase base trap that
broke isl.py's jump targets, hit again three iterations later.

The poke results now need a different explanation. The likeliest is that neither
poke produced the tested state: +16 is never read by built-in 69, and +4 = 0
takes the early-exit path an UNDEPLOYED unit also takes. Recorded as a hypothesis
with the experiment that would settle it.
2026-08-25 20:50:35 +00:00
Sylpheed RE agent
0b06e29d58 re: Stage 02 phase 1 read as a timeline; what arms the mission clock
Disassembling all 25 timer triggers in threshold order turns the phase into
readable script: fade, BGM, then squadron deployments at 30/60/90/120/170/210/
240s each as deploy + move_order + objective_marker, with radio messages
interleaved and a late block at 1020-1170s.

Answers the open question from the previous entry: timer 0 is armed by
timer_set(1200, 180) followed by timer_resume, fired at 4.0s on the phase-intro
clock (timer 5), which is already running when the phase begins. So the mission
clock has a 1200-second limit.

And it independently confirms the arrival measurement: the timer0 @ 170.0s
trigger deploys symbol index 0x01 = ADN110, the first of the three squadrons the
phase-1 condition polls. The live run measured those three going active at
~155-165s of mission time, and the route table also says 170. Three independent
sources agree -- the route table, the trigger table, and the running game.

Open: whether a coroutine can re-arm its own trigger, and timer_set's second
argument (180), for which 'warning threshold' is a guess rather than a finding.
2026-08-25 20:43:07 +00:00
Sylpheed RE agent
0cf75f998f re: phase coroutines are started by a per-frame timer table
sub_822748D0, called from ScriptPhase::Update every frame, walks the table at
[phase+240]: stride 24, each record carrying a coroutine offset (rec+4), an f32
threshold (rec+12) and a timer index (rec+20). It compares the previous and
current copies of the 32-entry float register file ([phase+104] and [phase+88])
and starts the coroutine at codebase+off on a RISING EDGE, prev <= t < cur, so
each fires once. Verified instruction by instruction.

The table comes from the mission-level begin_phase opcode 0x83's fourth operand
(the fifth is the phase end-event); sub_82270DF8 stores them at [phase+240] and
[phase+236]. On disc it is tagged constants -- 0x819 int, 0x81A float, both past
the ISL dispatcher's bound so they never execute. Verified on Stage 02: counts
25/13/18 with the tag triple correct in every record. Only timers 0 (1-1170s,
mission clock) and 5 (0-5s, phase intro) are used corpus-wide.

This explains the poke results. Two experiments set a squadron to 'destroyed'
and nothing happened; the leading explanation was that the condition coroutine
is not polling. It is not -- coroutines are started on a schedule by timer
crossings, so state written between firings is read by nobody.

It also reframes the arrival timetable: the routes' t=170 and these thresholds
are the same kind of thing. The mission is substantially a timeline, with unit
predicates deciding what happens at each scheduled point rather than when.

Open: which built-in arms or resets each timer.
2026-08-25 20:37:37 +00:00
Sylpheed RE agent
7a88d24e66 re: correct 'no mission counts kills' -- attribution IS used, globals are not
I wrote that no mission counts kills. Too strong. What holds is narrower: no
mission reads the two GLOBAL counters (33/34, verified 0 sites). The game does
track kills and missions test them through attribution:

  built-in 105 killed_by_player(unit) -- reads [squadron+600], which the
  unit-death handler sets from the damage message's _BY_PLAYER flag -- 117 sites
  across 14 stages (S02: 38, S01: 32), the 17th-most-used built-in in Stage 02.

  built-in 72 -- numerator is a per-(killer, victim) squadron KILL COUNTER,
  incremented in the unit-death handler; 13 sites, S01 only.

So 'did the player get that kill' is a common predicate; the game just never asks
'how many kills in total'. Evidence for 105 is the game's own strings: the death
handler branches into KILLED_BY_PLAYER vs KILLED, beside LOST_TURRET_BY_PLAYER,
SHIELD_DOWNED_BY_PLAYER and the rest.

Also withdraws the 'current / initial members' gloss on built-in 72: it is
100 x (units of B killed by A) / (declared members of B), attributed kills rather
than survivors, and reads 0 unless the script registered the (A,B) pair.

Built-in 55 resolved and shown to be effectively dead (0 sites corpus-wide);
built-in 75 confirmed as five HUD text lines via a [phase+376] dictionary.
2026-08-25 20:23:34 +00:00
Sylpheed RE agent
ac4f53b8a1 re: interpreter command table recovered; withdraw my 'constructors' reading
sub_822FE040 is a fully unrolled registration: 1023 slots filled with a default,
then 57 explicit writes, of which 48 are real handlers and nine are a shared
accept-and-discard stub. Return convention is nonzero = consumed, 0 = retry,
which is how the interpreter waits for a named unit to exist.

Opcode 995 is the ONLY handler touching the phase mirror [*(0x828F35F8)+236] --
the sole read and sole write in the table -- independently confirming why polling
that mirror saw nothing during phase 1. And no handler spawns or despawns a unit:
256 is the strongest deploy candidate but is unconfirmed because the message ids
are write-only in this image.

WITHDRAWN, verified wrong: I had recorded the writes to '+20' in sub_8226E7D8 /
sub_8226E930 as block initialisations by container constructors. At
0x8226E86C-0x8226E8E0 they do li r3,28 / bl 0x8230C160 then
lis r10,0xAB03 / ori r7,r10,0xE4BA / stw r7,4(r3): they build an INTERPRETER
COMMAND RECORD for opcode 996 and push it, i.e. AddSelector and RemoveSelector,
with a 32-entry cap. The stw to 20(r3) is the command record's +20, a different
object. Wrong twice: not constructors, and not that container.

Also flags that sub_8230C398 -- gated on *(0x82899CE0) == 16 at both call sites
-- looks like Stage 16's script compiled in C++, which 'debug defaults' does not
survive given the .ssb loader explicitly refuses mission 16.
2026-08-25 20:17:30 +00:00
Sylpheed RE agent
13ec75465b re: sub_8226E458 is a splice, but not shown to touch the trigger queue
Chasing which of sub_8226EAB8's 16 callers grows the trigger count: sub_8226E458
decrements a count at 8(r30) then calls the insert helper -- a splice -- and has
exactly one caller, inside sub_8226D740, the per-frame engine->script drain. That
suggests the engine moves records into the phase queue each frame.

But the call site refutes the neat reading: at 0x8226D780 the argument is
lwz r4, 324(r29), the UNIT ARRAY, not the trigger container. So I have not shown
sub_8226E458 touches the trigger queue at all, and 'the engine feeds triggers
each frame' is my inference rather than the disassembly's.

Recorded unresolved. Taking a function's shape for its purpose is what produced
the 'push' mislabel on sub_8226E3B8 and the ADN110-for-null pretty-print, each of
which cost an iteration.

Unaffected and solid: sub_8226EAB8 increments a count at +8 of its container, the
trigger container embeds its list at +12, so the watched word at phase+272+20 is
that inner list's count, and the guest was inside sub_8226EAB8 at the write.

Next: instead of guessing among 16 callers, re-run the watchpoint and read the
guest LR from the context -- the technique that named the writer will name its
caller.
2026-08-25 19:44:16 +00:00
Sylpheed RE agent
33013a9b84 re: the trigger-queue appender found -- sub_8226EAB8, count at inner+8
The watchpoint plus Canary's source settle it. At the write, the guest context
(rsi, per x64_emitter.cc:881) holds 0x8226EAE0, inside sub_8226EAB8. That
function is a generic list-node insert: it reads the count at 8(r30), guards
against 0x3FFFFFFF overflow, does addi r11,r11,1 / stw r11,8(r30), then links the
node. It has 16 callers, so it is a shared container helper.

That explains why two static searches missed it. The trigger container at
phase+272 EMBEDS an inner list object at +12 -- which is why the push does
'addi r31, r30, 12' -- and the inner object keeps its count at its own +8. So
272 + 12 + 8 = 292 = the watched word, and the instruction is stw r11, 8(r30)
with r30 = phase+284. Searching for 'stw rN, 20(rM)' could never have found it.

Also resolves the earlier open item on +12: it is the embedded list object, not
a list head pointer, which is why it read 0x000A0009 instead of an address.

Method note kept: the static hunt assumed the field's offset in the OUTER object
would appear in the writing instruction. A watchpoint is indifferent to the
addressing form, which is why it was the right tool after two failed offset
searches.
2026-08-25 19:34:15 +00:00
Sylpheed RE agent
121004ef3c re: the mission script is resident immediately -- two explanations refuted
ssb_watch.py polls the .ssb header, a symbol string and find_mission from the
moment flight is detected. On a normal run all three are present at the FIRST
sample (t=0.0, mission 0xBC79C960), so 'the script loads later than assumed' and
'the probe raced a load' are both out -- there is no window in which a healthy
mission flies without its script in memory.

A detail worth more than it looks: at that sample screen_id reported 'other',
not 'flight'. The script is loaded and the ScriptMission locatable BEFORE the
flight HUD appears, so residency is not gated on the HUD, and a run showing the
HUD without the script is in a state a healthy run never passes through.

That leaves the third candidate: the anomalous run's mission never loaded and
its IN FLIGHT was a misdetection -- consistent with it being frozen on a black
screen when checked afterwards. Not proven, but now the only surviving
explanation rather than one of three.
2026-08-25 19:16:13 +00:00
Sylpheed RE agent
761dcd004c re: the JIT context is in %rsi; but the re-run could not locate the mission
From Canary's own source (x64_emitter.cc:881) GetContextReg() returns rsi, so at
any JIT instruction %rsi is the PPCContext* -- which is also why the faulting
instruction read 0x110(%rsi), a guest register load. That is the way past the
watchpoint's ceiling: the guest register file is available at the write, and a
0x82xxxxxx word picked out of it resolves against sylpheed.db to name the caller.
trigger_watch.sh now dumps x/128wx  instead of a useless host backtrace.

The re-run then failed for an unrelated and unexplained reason: it reached
flight, the pilot bound, the guest was animating, and find_mission returned
NOTFOUND. Narrowed: the .ssb header is absent from guest memory (0 hits where
earlier runs hit immediately), ADN110 is absent too, but the manifest string
'Stage02.ssb' IS present at 0xBDA6C50B. So memory is readable and the manifest
is loaded while the script is not, in a mission that is flying.

No explanation offered. The cheap discriminator for next time is to poll for the
header from the moment flight starts and record when it appears, instead of
sampling once.
2026-08-25 19:03:33 +00:00
Sylpheed RE agent
c50ab7275f re: survey all 28 mission scripts -- kill counters are dead game-wide
All 28 StageNN.ssb decoded: 2,085,628 bytes, 25,705 call sites, 108 of 147
built-ins used. Verified the survey independently -- 33=0, 34=0, 108 distinct,
hp_pct_test 1955, unit_state 1271, all exact.

global_counter0/1 have ZERO call sites in all 28 stages. The handlers exist and
are wired; no mission calls them. So 'does a wave start after N kills?' is
answered for the whole game, not just Stage 02: no mission counts kills.

Two idioms change how the counts read. hp_pct_test(unit, 0.0) IS a destruction
test -- 1786 of 1955 calls (91%) pass 0.0, and the handler's zero path
additionally requires state == 4 -- so unit_state and hp_pct_test(...,0) are
interchangeable and stages just pick one. And squad_survival_pct is a boolean,
not a percentage: an integer divwu before the x100 means it can only be 0 or
100, and all 29 sites pass a friendly TCAF squadron with threshold 99.9, making
it 'has this escort lost anybody'. The game has no destroy-N%-of-a-squadron
objective.

Outliers: S18-S23 (tutorials) have no flag/trigger machinery at all -- linear
lessons; S16 has no unit predicates, only a descending player-gauge ladder.

Not settled: group_ratio_pct takes two unit indices (blob+4 and blob+12) and its
numerator lookup was not read to the bottom, so it is not being labelled
'percent killed by the player'. isl.py's UNIT_ARG omitted 71/72.
2026-08-25 18:53:56 +00:00
Sylpheed RE agent
da2d02db77 re: the trigger-count watchpoint fires, but the writer is JIT guest code
trigger_watch.sh + host_addr.py translate the guest VA to a host address
(0xBE14DEA4 -> host 0x1BE14DEA4) and watch it. It fires: Thread 50 'Main
XThread', old 0, new 16777216 -- which is big-endian 1 read little-endian, so
the count going 0 -> 1, independently confirming the field. The write happens on
the guest's own main thread, not an emulator worker.

But the writer cannot be named from the host stack: the faulting PC is
0xa0c65f23 with no symbol, executing 'mov 0x110(%rsi),%rbx', i.e. Xenia's
JIT-compiled guest code, and the frames above it are not host-unwindable. So the
watchpoint answers when and which thread, not which guest function.

Recorded as a ceiling of the method rather than retried blindly. The way past it
is that the JIT holds the guest context in a register (%rsi here), so the guest
PC is recoverable from the context block -- which needs Xenia's context layout
from the xenia-rs sources on this box, a separate tractable piece of work.
2026-08-25 18:52:35 +00:00
Sylpheed RE agent
28a4b1ead1 re: ISL opcodes decoded; the branch base is PER PHASE and isl.py was wrong
All 25 opcodes now have meanings. Ops 2/4/6/8 are integer compound assignment
(+= -= *= /=) and 3/5/7/9 the float versions; 10 and 11 are integer and float
compare writing three condition bits; 13-18 are je/jne/jl/jle/jg/jge; 21-24 are
push.i/push.f/pop.i/pop.f over deques at phase+44 and phase+64.

The shared-handler question is answered: the dispatcher leaves the opcode in r4
and the shared thunks never overwrite it, so those helpers take an extra opcode
argument and index a secondary table (0x82271448, 0x8227152C).

CORRECTION to my own tool and note: the branch/jump base is [phase+232], which
the phase initialiser sets to 0x24 + the phase's entry from the mission-level
stream -- 0xE4 / 0x14AA8 / 0x24B4C for Stage 02's three phases, not the file's
0x24. Measured on phase 1: base 0xE4 puts 525 of 525 branch targets on an
instruction boundary; base 0x24 manages 188. isl.py had been using 0x24 for
every phase, so its jump targets were wrong throughout. Fixed via
isl.phase_bases().

That also settles two things mission-script-ssb.md left open: offsets ARE
code-base-relative, and 0x1883's operand IS a code pointer -- the earlier worry
that some 'land on IEEE floats' was an artefact of adding the wrong base.
2026-08-25 18:52:09 +00:00
Sylpheed RE agent
415dd75a8b re: what appends a trigger node -- not found, recorded as such
Three approaches, none successful. sub_8226E160, earlier flagged as 'enqueue a
pending trigger', has exactly one caller and is a specific operation rather than
the general append. Writes to the count at +20 inside the container code number
only four, and all four are part of a block initialisation (stw to 0/8/12/16/20/
24 in consecutive instructions) in sub_8226E7D8 and sub_8226E930 -- constructors,
called from 0x8226E560 and from ScriptMission's own constructor at 0x822608A0.

So the increment that takes the count 0 -> 1 -> 2, which is measured live, does
not appear as a plain stw to 20(rM) anywhere in the container's code. It is
inlined, uses another addressing form, or lives somewhere I have not looked.

Recorded as not found rather than guessed: inferring from the shape of nearby
functions is exactly what produced the 'push' mislabel last iteration.

Names the approach that would settle it: a gdb watchpoint on
ScriptPhase+272+20 during a live mission. The address is known at runtime, the
count demonstrably changes within ~2 minutes, and a watchpoint reports the
writing instruction directly instead of inferring it from static shape.
2026-08-25 18:24:21 +00:00
Sylpheed RE agent
25a98fb204 re: trigger queue walks live -- structure confirmed, record contents refuted
Walked the container's list from +16 for 200s. The structure holds: +20 tracks
the node count, nodes chain through their first word, and entries appear as the
mission runs (0 -> 1 -> 2, then stable).

The record layout does not. I expected node+8 to hold small symbol indices,
which the pop's out-parameters made natural. Every field is a guest heap pointer
(0xBC..), so trigger records reference objects rather than table indices, and
those objects are unidentified.

Also records a false resolution I introduced: a line printed 'f4=0(ADN101)'
because the raw value is 0 and my formatter mapped index 0 to symbol-table-2's
first entry. ADN101 is not in that record -- the pretty-printer invented a name
for a null. A resolver must refuse values that were never indices.

And corrects the previous section: sub_8226E3B8 is a CLEAR, not a push. Its tail
decrements a counter, calls an erase helper, and loops while [+20] != 0. So
built-in 100 clears the queue then rebuilds the thread list, matching the
built-in table's own wording; 'push' was my label, not the disassembly's. Two
callers: vt2 (script) and 0x8226D420 (an engine site). What appends a node is
still unidentified.
2026-08-25 18:18:19 +00:00
Sylpheed RE agent
eeff706134 re: the trigger-queue pending count verified live -- 0 -> 1 -> 2
Read from a running Stage 02 mission: the container at ScriptPhase+272 has +20
moving 0 -> 1 -> 2 over the first two minutes while the phase ordinal stays 1.
So +20 is a real count of currently registered triggers -- the script arming
watches as it goes, and Stage 02 has 12 push_trigger sites -- and it is readable
from /dev/shm with no debugger. First direct view of what the script is waiting
for.

Corrects one field from the static reading: +12 is not (only) a list head. It
reads 0x000A0009, which is not a pointer, even though the push's
'addi r31, r30, 12' made list-head the obvious interpretation. Recorded as
unresolved rather than kept quietly.
2026-08-25 18:03:50 +00:00
Sylpheed RE agent
63d3ce089f re: trigger queue layout at phase+272, and a readable pending count
Chasing what makes the phase-1 condition re-evaluate, since the polls do not run
continuously.

Two method corrections: searching the VM range for '272(rN)' mostly returns
VTABLE slot offsets -- 0x82273174 lwz r11,272(r11) is followed by mtctr/bctrl,
a virtual call through slot 68, not an access to the phase field. And
[phase+272] is not a pointer to a queue but an EMBEDDED container: vt2
(sub_82265DD0) is 'addi r3,r3,272 ; b 0x8226E3B8', passing phase+272 as this.

Layout from the push/pop pair (sub_8226E3B8 from built-in 100, sub_8226E220
called every frame from sub_8226D740): +12 list head, +16 current node, +20
element count (zero = empty, tested first by the pop), +24 scratch. The pop
returns the record through out-parameters read from node+8: three u32s, a
double at +16, another u32 at +24 -- matching the six pointers sub_8226D740
passes in.

The actionable part is [phase+272+20], a live pending-trigger count readable
from /dev/shm. Watching it alongside [ScriptMission+40] should show when the
engine hands the script an event, which is when condition coroutines start --
the thing every phase experiment so far has been blind to.

Layout is from disassembly only; not yet verified live.
2026-08-25 17:54:34 +00:00
Sylpheed RE agent
67f3d621e6 re: the poke control PASSES -- writes reach the guest, hull is authoritative
Hammering settles what a single write could not: hull 0x44BB8000 (1500.0f),
944,387 writes of 1 over 15s, and afterwards the value HELD at 1 -- the game
stopped rewriting it. The screen left 'flight', the HUD is gone, the ship is
burning, and Natalie radios 'I've lost contact with Rhino 3!', the player's own
callsign. The game read the poked value and killed the player.

Established: writes to /dev/shm reach the running guest; hull at pos+0x154 is
authoritative, not a readout; and a single write loses a race against the game's
own continuous writes.

This upgrades two earlier results from inconclusive to genuine negatives. The
unit-record pokes were downgraded because I could not tell 'ignored' from 'never
arrived'. The write arrives -- and those pokes persisted untouched for 60s, so
the game genuinely saw state=4 and handle=0 on all three objective squadrons and
did nothing. That is real evidence the phase-1 condition coroutine is not polling
and its checks run only when a trigger starts them.

Withdrawn: last iteration's claim that the pilot's hull= is a different field or
scale. I read 1000.0f at pos+0x154 and inferred a mismatch with the logged 1500;
this run reads 1500.0f at the same offset. Same field, different value per run.
2026-08-25 17:48:47 +00:00
Sylpheed RE agent
77bc300692 re: the positive control ran, and is still inconclusive
poke_control.sh (self-retrying, succeeded on attempt 1 with no freeze) set the
player's hull at pos+0x154 to 1. Twelve seconds later the game had put
0x447A0000 back.

That establishes an asymmetry worth having: the hull word is continuously
rewritten by the game, while the unit-record fields held our value untouched for
60s. It separates fields the game maintains from fields nobody writes.

But it is not yet a control. The after-frame shows a red WARNING banner -- and
the before-frame already shows MISSILE ALERT, so the ship was under attack in
both and the banner is not attributable to the poke. A value being overwritten
proves the game writes that address, not that it read ours.

Correction: hull at pos+0x154 is 0x447A0000, a FLOAT 1000.0, not the 1500 the
pilot logs -- those are different fields or scales and should not be conflated.

The settling refinement: poke in a tight loop for several seconds so the value is
low whenever the game samples it. If hull is authoritative the ship dies and the
screen goes to GAME OVER; if it survives, the field is a readout.

The reusable part is the harness: boot -> verify animating -> locate -> act, with
a freeze at any step costing one retry rather than the iteration.
2026-08-25 17:36:00 +00:00
Sylpheed RE agent
cee5cffebf re: the freeze blocked the control run; tally it as the dominant cost
This iteration set out to run the positive control the poke results need -- poke
the player's hull, which the pilot logs every sample, and confirm the guest sees
it. It did not run: the bind took two attempts (first fwd_cos -0.94, second 1.0)
and by the time the player was being located the guest had frozen, with
entities2 reporting '0 moving triples' and frozen.py confirming
max_pixel_delta=0.

Session tally: froze at ~70/126/150/253/610-682s and this run; ran clean at
694s (ended by the game), 936s and 1064s. Roughly two in three freeze, each
costing a ~5min boot plus the window. The freeze has now truncated more
experiments than every other cause combined.

Restates that it is probably not ours: the mission-end freeze is recorded as
pre-existing in both our build and the official AppImage, and this session
produced a pilot-only freeze at 150s with no probe attached. The
probe-correlation lead is real but never became a clean split.

Consequence: experiments needing more than ~2 minutes of live mission should
checkpoint and resume, or detect the freeze and re-run themselves. Every tool
here witnesses the freeze; none survives it.
2026-08-25 17:23:20 +00:00
Sylpheed RE agent
0c930f203a re: corrected poke also does nothing -- and the method lacks a positive control
rec+4 = 0 (the early-exit path built-in 69 actually tests) sticks for 60s with
all three squadrons poked, and neither the phase nor +16 changes. +16 stayed 2
while the handle it derives from read zero.

Leading explanation: the phase-1 condition coroutine is not polling. Both pokes
persist untouched and nothing recomputes anything, which fits the polls at
0xF524 living in a coroutine that only runs when a trigger starts it
(built-in 1, fed from [phase+272]).

But the honest problem is bigger: two pokes have now produced no observable
effect and I cannot tell 'the game ignored the write' from 'the write never
reached the game'. The stick test only proves the value persisted in the
shared-memory FILE, not that Canary's guest sees it. That control should have
come first, so both null results are downgraded to INCONCLUSIVE rather than
evidence about the condition.

Next: poke something with a visible effect (player hull, a HUD counter) and
confirm it on screen before trusting any further null result. What still stands
from these runs is only what was observed rather than poked -- the arrivals,
ADN111's destruction, and the mission-over branch.
2026-08-25 17:11:16 +00:00
Sylpheed RE agent
b07b678ef0 re: unit_state reads rec+4 and rec+104, never rec+16
Disassembling built-in 69's handler at 0x8226ADF0 instead of trusting the
summary: it loads rec = base[idx], tests rec+4 against zero (early exit =
absent), passes rec+4 to sub_82301240 for the lifecycle lookup, and compares the
byte at rec+104. rec+16 is never touched on that path.

That explains the null poke result exactly -- I wrote into a field the condition
does not consult. +16 remains a faithful OBSERVABLE (the arrival and death
transitions were real) but it is a readout, not the input. It also explains why
rec+4 holds small consecutive integers: it is a handle that sub_82301240
resolves, not a pointer.

The corrected simulation of 'this squadron is gone' is rec+4 = 0, taking the
documented early exit.

Flags a general caution: this built-in table came from a subagent's reading of
handler behaviour, and this is the second per-offset field description that did
not survive the disassembly. The identifications of which built-in does what
have held up; the offset meanings are leads to verify.
2026-08-25 16:58:49 +00:00
Sylpheed RE agent
57f0238af2 re: poking all three squadrons to state 4 does NOT end phase 1
Ran the direct test instead of a seventh attempt at winning. All three objective
squadrons were live (state 2); the write to +16 sticks, and 60s later with all
three reading 4 -- the value a naturally-destroyed squadron takes, measured on
ADN111 -- [ScriptPhase+196] is still 0 and the ordinal still 1.

So 'phase 1 clears when ADN110/111/112 are destroyed' is not confirmed and its
simplest form is refuted. The bytecode reading (three unit_state polls then
set_flag(8)) stands; what does not follow is that flipping the field equals the
kill.

The persistence is the clue: built-in 69 normalises +16 when it polls, so a
running condition coroutine should have overwritten the poke within a frame. It
did not, which points at the condition being evaluated only when a trigger fires.

Also corrects the per-unit record layout: +4 is 26/27/28 for the three
squadrons -- small consecutive integers, NOT the 'live object pointer' the
built-in summary describes (an undeployed squadron has +4=0). +20 = 9 is exactly
their member count n from the roster, so the record is per-squadron and carries
its strength. My own probe printed 'obj=yes' by testing that word for non-zero
rather than pointer-ness, which made an index look like an object.
2026-08-25 16:53:30 +00:00
Sylpheed RE agent
dde74e73eb re: the win-the-mission route is not converging; poke the state instead
Two more attempts. Recording three things rather than another flat negative.

The pilot's gun-fire rate is 1.6% (81 of 4986 frames), but that is not the
blocker: the nose gun is Power 15 unguided while the main mount is Power 200
guided, and ~70 missiles went out in ~500s. The log's fire= field tracks only
the gun and invites the wrong conclusion.

SYLPH_KILL_TURRETS=1 was tried to align DEFEND with the objective and is
REFUTED as an improvement: 3387 of 11112 samples (30%) chased targets over
20,000 units away, because turrets are static and spread out, so the pilot
commits to distant ones and defends nothing. Escort still fell to 48.5% and no
additional objective squadron died.

The bounded scan delays freezes but does not remove them -- one run clean to
694s, one frozen at ~682s, against 3-of-3 inside 4 minutes unbounded. So 'the
sweeps were the cause' is too strong; they were a cost.

Six attempts, no phase advance. Names the cheaper experiment: guest memory is
writable, so set the two surviving squadrons' +16 to 4 with gpoke and watch
whether finished goes to 1 and the ordinal steps to 2. That tests the condition
directly, and a null result is equally informative.
2026-08-25 16:35:55 +00:00
Sylpheed RE agent
47af78d668 re: bounded scan fixes the freeze; mission-over branch confirmed on the oracle
Bounding the pointer scan to 0xBC000000-0xBD000000 (with a full-sweep fallback)
drops find_mission from a ~371MB walk to 0.7s. The run then went 694s with the
probe attached and NO freeze, against 3-of-3 frozen inside ~4 minutes with the
unbounded version. n=1, but the first probe-attached run to survive.

State encoding pinned to three points: 1 = not yet deployed, 2 = active,
4 = destroyed. ADN111 caught going 2 -> 4 at 433s while the active count fell
36 -> 27.

The phase ended at 694.9s WITHOUT the ordinal advancing, and every field matches
the branch read statically from sub_82260710: [phase+300]=2 (last-phase flag),
[mission+20]=0 (mission-over state), [phase+196]=1 (finished), [mission+40]=1
(unchanged). The static state machine is confirmed on the live oracle for the
mission-over half.

But this was a LOSS, not a clear: GAME OVER on screen, escort at 35.7%, pilot
DEAD at 676s, and two of the three objective squadrons still at state 2. So the
'destroy all three clears phase 1' prediction remains untested. What is
established is that the else-branch is the only route to phase 2 and needs
[phase+300] != 2 when the phase ends.

Five attempts, still no phase advance observed -- the obstacle is now keeping the
escort alive, not the freeze or the instrument.
2026-08-25 16:07:26 +00:00
Sylpheed RE agent
22b6541cd9 re: counter-example weakens the probe-causes-freeze correlation
A pilot-only run froze at t~150.7s (frozen.py: max_pixel_delta=0), found by
accident when the sweep-free experiment aborted at startup and the run flew with
no script probe attached. The tally is now 3-of-3 frozen with the probe versus
1-of-3 without, not 3-versus-0. Still a lean, but not the clean separation the
previous entry claimed, and marked down accordingly -- the fourth time a freeze
conclusion here has had to be softened by one more run.

The sweep-free test itself could not run: ScriptMission is re-allocated per run,
so the address from an earlier run (0xBC7A2A20) read back all zeros. The cheap
self-consistency check ([m+44] must equal the phase's [+244]) rejected it
instead of reporting garbage, which is the part that worked.

Names the cheaper replacement: a BOUNDED pointer scan. Every ScriptMission seen
so far sits in 0xBC79xxxx-0xBC7Axxxx, so ~32MB instead of ~371MB would cut the
sweep cost roughly tenfold. Not yet implemented.
2026-08-25 15:38:23 +00:00
Sylpheed RE agent
d086586a0f re: my own probe is now the strongest freeze correlate
phase_watch.py runs give a sharper tally than the earlier inducer test, because
the only difference between the groups is one tool: pilot alone went clean to
936s and 1064s (2 runs), while pilot + phase_watch froze at ~70s, ~253s and
~126s (3 runs). Same boot path, same pilot, same mission.

What phase_watch adds is find_mission(), which does TWO full sweeps of guest
memory at startup plus ~130 small reads every 5s. The sweeps are the same shape
as the heavy_read.py scans the earlier n=2 experiment implicated.

Still correlation, not cause -- 3 vs 2, and the earlier inducer test produced a
clean counter-example. But it is the best-supported version of 'the instrument
provokes the freeze' so far, and it is my instrument, which makes it actionable.

The cheap test is named: find_mission only needs to run once, so re-run the
watch with the sweeps replaced by an address from a prior run. Freezes stopping
implicates the sweeps; continuing implicates the per-sample reads.

Cost so far: three attempts at observing a phase advance, all truncated inside
four minutes.
2026-08-25 15:19:52 +00:00
Sylpheed RE agent
852256525d re: the arrival timetable confirmed live -- t is in SECONDS
ADN110/111/112 all carry first-keyframe time 170 on their _p1F routes, and in a
live run all three flipped from not-deployed to active at ~143s on the probe
clock. The probe zero sits roughly 10-20s into the mission, putting the arrival
at ~155-165s of mission time against a predicted 170.

That pins the unit as seconds: 170 frames at 30fps is 5.7s, so they would have
been active at the first sample and demonstrably were not. Three squadrons
sharing one timetable entry changing state in the same 5s window is not
coincidence. Flagged as a match rather than an exact measurement -- the probe
clock is not aligned to phase start.

Supersedes mission-arrival-watch.md's headline negative: 'no arrival has ever
been observed across six runs' was an instrument limit, not a fact about the
game. Its craft-counting analysis stays accurate -- craft counts conflate
deployment with attrition and cannot see an arrival at all.
2026-08-25 15:07:08 +00:00
Sylpheed RE agent
ab08c77388 re: state 1 = not yet deployed, and arrivals are directly observed
Watching [ScriptMission+40] and the three phase-1 objective squadrons together:
all of ADN110/111/112 flip state 1 -> 2 at ~143s, while records in state 2 climb
24 -> 35 over four minutes.

So state 1 means 'not yet deployed' for these, not 'gone'. The built-in table's
'1/3/4 = gone/dead/invalid' shorthand is incomplete, and reading state != 2 as
destroyed would have been wrong exactly as flagged last iteration.

This also answers a much older question: mission-arrival-watch.md and the wave
work recorded '0 confirmed arrivals' across many runs by watching the CRAFT
population. The script's own unit table shows arrivals plainly -- eleven records
enter state 2 within four minutes. The old negative measured the wrong
structure; craft counts conflate deployment with attrition, the per-unit state
field does not.

Both attempts froze (at ~70s and ~253s), so no phase advance was reached. The
freeze witness caught both immediately, which is why the truncation is visible
instead of a silently flat line.

New harness tools/re-capture/phase_watch.py.
2026-08-25 15:01:24 +00:00
Sylpheed RE agent
273690cf56 re: locate the live ScriptMission/ScriptPhase without a debugger
Unblocks the phase experiment, which was stuck because '38 enemies died' could
not say whether the right ones did. Chasing craft->squadron was the wrong angle:
the script VM keeps that table itself, indexed by the .ssb symbol-table-2 index.

Route: find the .ssb header in guest memory (0xAB840010 for a Stage 02 run),
code base = filebase + 0x24, scan for a word equal to it, then VALIDATE
arithmetically -- [ScriptMission+44] must equal filebase + symtab1 offset + 4.
Measured 0xAB874C94, predicted 0xAB874C94, exact. A second candidate that also
pointed at the code base failed that check and was discarded; without it either
would have looked plausible.

ScriptPhase+324 -> +4 is an array of 122 per-unit records -- exactly the size of
Stage 02's symbol table 2, an independent confirmation of the index space.

[ScriptMission+40] reads 1 in a phase-1 mission. The mirror at
[*(0x828F35F8)+236] that three earlier runs polled reads 0, because ChangePhase
only posts once the ordinal exceeds 1 -- so +40 is the real counter and is
reachable from /dev/shm with no debugger.

Flagged rather than asserted: the three objective squadrons read state=1 with a
LIVE object pointer in a mission where nothing has been shot, which does not fit
the built-in table's '1 = gone'. Reading state != 2 as destroyed would be a
plausible-but-wrong inference; the encoding needs pinning first.

New tool tools/re-capture/squadron_state.py, verified end to end against the
manual reading.
2026-08-25 14:25:46 +00:00
Sylpheed RE agent
cbfc7ab846 re: Turret-biased run verified live -- 38 kills, still no phase advance
Checked liveness before investing this time (animating, max_delta=254) and the
run was real: 823 distinct speeds, hull 1500->1485, escort 100%->30.7%, ADAN
population 127->89 (38 destroyed) over ~1064s, no GUEST FROZEN banner. The phase
mirror stayed 0 throughout.

But this still does not test the prediction. The claim is that phase 1 clears
when ADN110/111/112 -- 27 specific Turrets -- die; what was measured is that 38
enemies of some kind died. SYLPH_PREFER biases target choice but the pilot went
to DEFEND and spent the run on e010_ADAN_Attacker_S protecting the escort.

The missing instrument is per-squadron liveness: nothing maps a live craft back
to its roster squadron, the same gap roster-to-craft-link.md records as unsolved.
So 'no advance after 38 kills' is compatible with both 'the condition is wrong'
and 'the right 27 were never killed'. The decisive experiment is blocked on
attribution, not on flying time.
2026-08-25 14:11:50 +00:00
Sylpheed RE agent
9f2692eadd re: correction -- the script reads its own flags via built-in 9
Last iteration ended with 'what reads the flag file is unknown' after an offset
search failed. The framing was wrong: I was hunting an engine-side reader, but
the consumer is the script itself, through built-in 9 (read_freg), which loads
[phase+88][i] into the double result register.

Stage02.ssb calls read_freg 12 times -- the same count as set_flag (12) and
clear_flag (12) -- so the latch is symmetric and entirely inside the VM. That
closes the middle of the set_flag -> ... -> END_PHASE chain: one coroutine
latches, another reads it back and branches.

Also records an unconfirmed observation: op10 + op13 pairs with consecutive
small immediates and their own code offsets look like a case/branch dispatch.
Flagged as a pattern, not named -- neither handler has been read.
2026-08-25 13:47:56 +00:00
Sylpheed RE agent
6a192682e3 re: the Turret-biased run was void -- the guest was frozen from t=0
SYLPH_PREFER=Turret, escort at 100%, 1070s, phase field 0 throughout, 4029
Turret-targeting pilot samples. It looked like a clean negative -- 'hunting the
objective squadrons does not advance the phase' -- and it is worthless:
frozen.py reports max_pixel_delta=0, and the pilot's first sample at t=0.0
already has spd=0 with the same yaw/pitch/target/d=7186 it still had at 1070s.
The mission froze on entry to flight and nothing was ever shot, while screen_id
said 'flight' the whole time.

I was one step from writing this up as a fact about the game; running the freeze
test rather than trusting a plausible log is what caught it.

phase_probe.py now calls frozen.py every 60s and prints a GUEST FROZEN banner
inline, so a dead-world reading is labelled in the data instead of discovered
later. Note frozen.frozen() returns a TUPLE (is_frozen, max_delta) -- testing it
directly is always truthy and would have made the witness fire on every check.
Verified against the frozen guest.

Also recorded: the pilot log is itself a freeze witness -- identical
yaw/pitch/target across thousands of samples is a dead world, not patience.
2026-08-25 13:42:54 +00:00
Sylpheed RE agent
917083275d re: Stage 02 phase 1 watches 27 Turrets in ADN110/111/112
Resolving the three squadrons the phase-1 script condition polls against the
roster: all three are phase-1 (route-name map), all fly UN_e007_ADAN_Turret, all
n=9, all on Formation_ADAN_Turret09_30 -- so the condition watches 27 Turrets.

Three layers agree independently: route names say phase 1, the roster says what
they fly and how many, and the script polls exactly those three before latching
set_flag(8).

Also confirms on the data that 'Turret' is a craft type, not ship-mounted AA:
UN_e007_ADAN_Turret is flown by AI_ADAN_CraftSquadron_Rookie/_Veteran in a
30-slot craft formation, never AI_Structure.
2026-08-25 13:17:47 +00:00
Sylpheed RE agent
9352e079bf re: 936s live run -- no phase advance, and a script lose-branch GAME OVER
Polled the runtime phase mirror every 3s for 936s of hunting flight. It never
left 0, which the static work predicts rather than contradicts: ChangePhase is
only posted once the ordinal exceeds 1, so a phase-1 run never writes it. That
does make the mirror useless as a phase-1 readout -- it reads 0, not 1, and
[ScriptMission+40] needs a debugger.

The run ended in GAME OVER with the player's hull at full 1500, so a script
lose-branch fired rather than the player dying. The pilot log narrows it: the
tracked asset fell from 100.8% to 23.4% with ASSET-HIT events, while the pilot
avoided f101_TCAF_Acropolis 2588 times. 'Escort lost' is a measured candidate,
not a guess -- but the asset ended at 23.4%, not 0, so the threshold (or whether
another unit's loss ended it) is not established.

Two runs now (530s, 936s) and no phase advance observed. Since phase 1 polls
unit_state on ADN110/111/112 and the pilot hunts whatever is nearest rather than
the objective, biasing it at those squadrons is the next step.

Refuted: sub_8226D740 does NOT read the flag file. Its three addi rN,r31,120 are
fields of a local trigger-record buffer passed to sub_8226E220 as out-params;
the phase lives in r26/r29. Offsets 88 and 120 are too common image-wide (2091
and 5228 instructions) to isolate the register files by offset search, so what
reads the flags is still unknown and the set_flag -> END_PHASE chain has a gap.
2026-08-25 13:12:16 +00:00
Sylpheed RE agent
dcf37bcf93 re: a Stage 02 clear condition read end to end, with squadron names
Resolving symbol-table-2 indices turns the bytecode into mission logic. At
0xF524 Stage02.ssb polls unit_state on ADN110, ADN111 and ADN112, updates each
one's objective marker, then latches set_flag(8) -- exactly the
trigger/predicate/set_flag/END_PHASE shape predicted from the disassembly, now
observed in the mission's own code with names the roster tables already gave.

The 12 END_PHASE sites are outro sequences (wait_cmds_drained / fade_sound(3) /
builtin85(3) / wait_s(3) / END_PHASE / yield) -- the terminator, not the
decision.

Fixes a decode bug that hid every argument: the tracker only followed
local[i] = special[0], but the common form is an immediate written straight into
local[i] (k=01,03), so every unit predicate printed with NO arguments. The
disassembly looked complete while being empty exactly where it mattered.

Also records the live probe result: the phase mirror at [*(0x828F35F8)+236]
stayed 0 for ~530s of actively-hunting flight, no advance observed -- which is
what the static analysis predicts for phase 1, since ChangePhase only posts once
the ordinal exceeds 1.
2026-08-25 12:42:50 +00:00
Sylpheed RE agent
70cff9ca21 re: the 147 ISL built-ins characterised; Stage 02 gates on units, not counts
Table at 0x8227226C is 147 big-endian absolute VAs (verified structurally: it
ends exactly where the first handler begins, all targets inside sub_82272220).
Arguments are not in the instruction -- every handler does c_str() on
[phase+20], a packed blob, which is what the local[] staging fills. Return 2 =
yield; five built-ins block by skipping the pc advance.

Recovered the ScriptPhase state layout: 32-entry float and flag register files,
int/double result registers, the timer block, and the runtime unit array at
+324 indexed by symbol-table-2 index -- a direct hook from bytecode call sites
to the two .ssb symbol tables.

Spot-checked two claims against the disassembly rather than trusting them: id 4
loads a DOUBLE into the thread countdown and returns 2 (wait_s), and id 24 reads
current/initial squadron member counts (squad_survival_pct). Both exact.

Counting Stage02.ssb: unit_state 255, hp_pct_test 167, dist_lt 92, unit_alive
71, unit_relation 52 -- and squad_survival_pct, group_ratio_pct and the two
global counters are NOT called at all. So Stage 02's phases are gated on named
units (destroyed / HP / proximity), never on an aggregate count, even though the
kill-counter primitives exist in the VM. That answers the standing 'next wave
after N kills or after an event?' question for this stage: specific units, not a
number.

isl.py now names the built-ins, so the run-up to the first END_PHASE reads
wait_cmds_drained / fade_sound(3) / builtin85(3) / wait_s(3) / END_PHASE.

Not settled: 3 handlers unresolved (55, 75, 105); the 1024-slot interpreter
command table is only partly recovered.
2026-08-25 12:36:02 +00:00
Sylpheed RE agent
3f13a8ceef re: ISL operand kinds decoded; arguments are staged through local[]
Resolver table 0x82271D74 gives four kinds: 0 global[i], 1 immediate,
2 special[i] ([phase+164]/[phase+168]), 3 local[i] ([phase+20+i]). Byte[0] is
the rvalue kind, byte[1] the lvalue kind, so the recurring instruction pair is
argument staging -- values land in local[] at offsets 0,4,8,0xC and the next
call consumes them. A built-in's arguments are not in its own instruction.

Fixed a decode that would have been believed: immediates in set.f are DOUBLES
carried as two words (op 1 stores with stfd). Reading the high word as a float
gives 2.125 where the script means 3.0.

isl.py now tracks staging and prints call arguments, so the run-up to the first
END PHASE in Stage02 reads as builtin=64(0x42,2,1,9,1,-1) / 120 / 59(3) / 85(3)
/ 4(3) / 6. Three built-ins taking 3 just before the phase ends look like a
wait-seconds family -- flagged as unconfirmed until the built-in table is read.
2026-08-25 12:28:04 +00:00
Sylpheed RE agent
7b445916bc re: ISL bytecode encoding decoded; phase-end call sites located in Stage02
Read the encoding off the interpreter rather than guessing: instruction is a
big-endian u32 whose LOW byte is the opcode (25 of them, table 0x822635FC),
byte[2] is the instruction length -- every handler advances the pc by it -- and
bytes[0..1] are operand kinds. Op 12 is a jump whose operand is relative to the
code base [phase+232], which settles that offsets are code-base-relative for
this opcode. Op 19 is the built-in call: id in word@+4, and word@+8 is a
monotonically increasing STATEMENT id (0x245, 0x248, 0x24A, ...).

Confirmed by disassembling Stage02.ssb: the stream decodes cleanly from the code
base and routines terminate on ret exactly where expected.

Scanning the code region on the call encoding: 2846 call sites, 73 of the 147
built-ins used. The phase-control ones are located -- built-in 6 (end phase) at
12 sites, 62 at 3, 39 (mark last phase) at 8 -- so a phase has several exit
paths, as a mission with win and lose branches should.

New tool tools/re-capture/isl.py with --calls and --to (resync-into-target,
needed because instructions are variable-length so you cannot walk backwards).

Not settled: the 147 built-ins are uncharacterised, so this is structure without
meaning -- we can see THAT a phase ends, not WHAT was tested.
2026-08-25 12:21:44 +00:00
Sylpheed RE agent
aa5fe09d01 backlog: script bytecode found; ISL decode is the next payoff; nav fix only partly reliable 2026-08-25 12:13:18 +00:00
Sylpheed RE agent
c67b0585e8 re: the mission scripts found -- Stage\StageNN.ssb in GP_MAIN_GAME_<L>.pak
mission-phase-advance.md recorded the bytecode as not on the disc. It is; the
earlier grep failed because every pak entry is Z1+zlib, so no name is plaintext.

The loader resolves table KEYS, not a filename: GamePart name -> GP_SCRIPT ->
script.tbl (name_hash 0x75FE4656), whose SCRIPTS record is a 40-field manifest
mapping MISSION1..MISSION29 to StageNN.ssb. Verified: Stage02.ssb is 226,596
bytes, md5 aff69b5a..., byte-identical across all six language paks.

28 scripts exist; S17 is the missing one -- which agrees with the table sweep
(S17 in none of the five families) AND with the loader guard
sub_8225EC78 'if (n == 16 || n > 32) return', proving that mission number is
0-based. Three independent routes to the same conclusion.

Header decoded from ScriptMission::Load rather than guessed: version, code
offset 0x24, two symbol tables. Stage02 symtab1 = 326 symbols (197 message ids,
119 route names, 10 subobjectives), symtab2 = 122 (111 unit ids, 11 characters).

Two loops close: the 111 unit ids are exactly the 111 UnitGroup_S02 squadrons,
and the route names appear as SCRIPT SYMBOLS -- which is why nothing in the
executable parses Route_*_p<N>* names. The _pN convention is the script's.

Refuted: the .embsec_ sections are code (32,368 instructions, 108 functions),
not script; MiscBin.pak and DefTables.pak have zero hits.

Open: the 25 ISL opcodes and 147 built-ins are not decoded, and the
mission-level stream at +0x24 is only partly read.
2026-08-25 12:12:59 +00:00
Sylpheed RE agent
fba1cf1043 backlog: phase-advance solved, hashes located, nav bug fixed, tooling corrections 2026-08-25 12:01:53 +00:00
Sylpheed RE agent
c744adadfd re: run 3 did not freeze -- a counter-example to my own inducer hypothesis
Third run with the inducer on from flight start went 900s without freezing, so
the tally is 2 for / 1 against. Stated confound: the inducer was far weaker this
run (~47-50s per CPU pass vs ~4.2s in run 2, because two subagents were
saturating the box -- 22 passes in 15 min instead of ~150). That is consistent
with either reading, so it is not scored as a confirmation. What it does
establish is that the inducer is not sufficient, and that future runs must
report its measured rate rather than just that it was on.

Second healthy stability sample: 9 stable / 14 vary vs 12 / 13 in the first,
intersecting to only 8 threads. So the stable set is run-dependent and must be
measured within the run it is used in -- the same lesson as gdb thread numbering,
one level up.

The frozen half of the distribution experiment is still uncollected.
2026-08-25 11:56:00 +00:00
Sylpheed RE agent
e8fb9a41a6 re: phases are advanced by a compiled script VM, not by data
Closes the open question from mission-phase-membership.md. The static sweep
found no trigger because there is none in the data: each phase ends when its own
per-mission script says so.

[ScriptMission+40] is the 1/2/3 phase ordinal -- initialised at 0x822606B0 and
incremented at exactly one site, 0x822609F8-0x82260A00 (checked: only one stw to
40(rN) in the whole state machine). Its guard is [ScriptPhase+196] != 0, and
that flag has only two writers, vtable slots 0 and 1, reached only from built-ins
6 and 62 of the phase-script VM's 147-entry command table. Built-in 39 sets
[phase+300]=2, which ends the mission instead of advancing.

Refutes all four candidate triggers: no kill counter, timer, trigger volume or
message event appears on the path. Any such condition lives inside the script,
which is why three phases of one stage can have three different clear conditions.

CScriptInterpreter::ChangePhase (sub_822FF330, opcode 995) is the consumer, not
the cause; it writes a second phase mirror at [*(0x828F35F8)+236] -- a probe
target.

Not settled: the script bytecode is not on the disc under any obvious name.

Tooling correction: the .pe is not stale, it is a flat VA image (offset = VA -
0x82000000), verified 7/7 against the DB. Also: instructions.raw is an INTEGER,
and decoding it as a hex string silently compares nothing -- that nearly got
this correction recorded backwards.
2026-08-25 11:41:11 +00:00
Sylpheed RE agent
ed51d92100 docs: IXUD solved, extsb correction, name_hash exactness retraction 2026-08-25 11:29:09 +00:00
Sylpheed RE agent
f6508dd8ac re: locate both guest hash routines; IXUD solved; two corrections
Found the routines in the disassembly DB rather than guessing from data:
  sub_82447DF0  IDXD tag hash  (lbz+extsb, modulus 0x00FFFFDF, magic 0x2101)
  sub_82447E70  IXUD tag hash  (lhz, 64-bit, modulus 0xFFFFFF67 then 0x00FFFFDF)
Both transcribed instruction-for-instruction into Python and Rust.

IXUD SOLVED. It defeated every single-modulus search because it chains TWO
exact moduli -- the loop reduces mod 2^32-153 in 64-bit arithmetic and only the
result is folded mod 2^24-33. A polynomial mod M1 folded through M2 is not a
polynomial mod anything, which is exactly why the gcd test returned 1. Verified
independently: 86/86 record keys and 108,261/108,261 field tags in
GP_MAIN_GAME_E.pak, and NoRecord -> 0x1c6d9c96.

CORRECTION 1: tag_hash must SIGN-EXTEND each byte (extsb). My reconstruction
used unsigned bytes and matched all 1.27M disc names -- every one is ASCII --
while disagreeing on ~90% of random inputs with a byte >= 0x80 (verified:
18096/20000). The disc could never have caught this; only the disassembly did.

CORRECTION 2: name_hash's reduction is EXACT, not lossy. The module doc claimed
the missing conditional subtract made it something other than %. rlwinm r6,r6,
9,23,31 is just hi>>23, and with RECIP = floor(2^55/M)+1 that is Granlund-
Montgomery magic division -- 0 wrong at every quotient boundary across the full
32-bit domain. Retracted.

cargo test -p sylpheed-formats --lib hash: 10/10.
2026-08-25 11:28:35 +00:00
Sylpheed RE agent
fd26fec97f backlog: no freeze signature survives the control; boot-nav blind-tap bug found 2026-08-25 11:08:25 +00:00
Sylpheed RE agent
55bfcc77d3 re: tag_hash verified disc-wide; record keys follow a stronger own-name rule
Swept all 33 dat paks plus hidden/DefTables.pak: 7,750 IDXD objects, 190,782
records, 1,271,462 (name -> tag) pairs, 0 failures, 0 parse failures.

The key rule is stronger than first written: a record's key is tag_hash of its
own inline name, 190,782/190,782 -- no roster needed (roster-based recovery only
reaches ~30% of records). Verified independently here on UnitGroup_S02, 112/112.

The modulus is uniquely pinned, not just consistent: gcd of
poly256(name) - (tag & 0xFFFFFF) over all pairs is exactly 0x00FFFFDF, prime.

Case-sensitivity is load-bearing, not incidental: the disc has 17 name pairs
differing only in case (UNIT/Unit, TYPE/Type, STAGE/Stage, ...); name_hash
collides on all 17, tag_hash separates all 17.

Records the limit too: IXUD, the wide-string sibling, uses a DIFFERENT low-24
hash (0/19,808) though it keeps the same top-byte checksum (19,808/19,808). Its
offsets are counted in 16-bit chars (STR + 2*strsize == filesize) -- newly
decoded, all 534 parse. Its low bits are provably not a modular polynomial (gcd
= 1; exhaustive Barrett search over M in [2^20,2^25) finds nothing; fixed-position
deltas are non-constant), so it is lossy and needs the XEX.
2026-08-25 11:08:10 +00:00
Sylpheed RE agent
1772d94558 re: control run withdraws the T68/T69 signature too -- nothing survives
Six captures across one healthy run: 12 thread states stable, 13 vary. Every
thread previously reported as a freeze signature is in the VARIES set --
including T68/T69, which I had kept as 'what reproduces across both freezes'.
They park and unpark during ordinary play, landing on the same objects they hold
when frozen, so seeing them parked while frozen is not evidence.

Net: no thread-level freeze signature has survived. Both frozen diffs are
consistent with healthy variation, and one-sample-per-state was never capable of
separating them.

Still standing: the 12 stable threads hold the same object in all six captures
and none of them moved in either frozen capture -- so 'not a whole-emulator
stall' survives, now resting on the stable set being undisturbed rather than on
a count of unchanged threads.

Also noted: gdb thread numbers are not comparable across runs (this run has
T132-T142, earlier runs had T104-T106), so future cross-run work must key on the
object address or guest thread id.
2026-08-25 11:07:37 +00:00
Sylpheed RE agent
fa57338af8 backlog+roster: 387-vs-300 solved, phase map found, three candidates refuted 2026-08-25 10:45:53 +00:00
Sylpheed RE agent
c6c3d436f5 re: phase membership is in the Route record names; the ~300 craft are ~150
Route_S<NN>.tbl names every route Route_<squadron>_p<N><kind> -- 120/120 for
S02 -- so the squadron-to-phase map was on the disc all along, in a table read
until now only for fly-in geometry. 108/111 S02 squadrons map; the 3 misses are
typos in the route table and all are phase 2.

That resolves the 387-vs-300 gap after four failed attempts, by refuting its
premise: there were never 300 craft. Phase 1's member-level n multiset
{1x25, 2x1, 4x4, 9x12} matches the measured craft-per-record fan-in
{2x24, 4x1, 8x4, 18x12} bucket for bucket at exactly 2x, so 300 = 2*(151-1),
the -1 being the documented 41-vs-42 off-by-one.

The 2x is the probe double-counting, not real units: a 2x reading needs 16
Delta Sabers for 8 named pilots, and breaks the formation capacity bound in 20
of 37 phase-1 squadrons (16 of them on Formation_1_only, one slot) where the 1x
reading has 0 violations. All three measured craft totals being even is a
further tell.

Refuted: UnitGroup has no phase/spawn/delay field (1019 = sum(Count)*4 + 5*111,
every slot accounted for); DisableInterval is No for all 111 S02 squadrons;
stage\EnumSquadron_Test.tbl does not exist.

Confirms tag_hash independently: UnitGroup record key == tag_hash(squadron id),
1160/1160.

Still open: what advances a phase -- no static table encodes a trigger.
2026-08-25 10:45:24 +00:00
Sylpheed RE agent
cbf52ba9f9 re: recover the IDXD record-key / field-tag hash (8643/8643)
Closes the 4-byte record key. tag_hash is name_hash's shape -- byte-sum
checksum in the top byte over a 24-bit modular polynomial -- with two different
constants: modulus 0x00FFFFDF (2^24-33, prime) instead of 0x00FFF9D7, and no
lowercasing, so tags are case-sensitive. name_hash explains 0 of 8643.

Recovered from the tables rather than the executable: every inline field name
is a known (name -> tag) pair, and comparing names differing in one character
gives the per-position weights 1, 0x100, 0x10000, 0x21, 0x2100, ... -- a byte
leaving bit 24 re-enters as 33, i.e. reduction mod 2^24-33. Holds where it is
easy to get wrong (distance 8 and 9 carry correctly).

A record's key is the tag of its own name: FormationSet rosters 362/362,
UnitGroup rosters 281/281, S02 squadron names 111/111 -- so records can be
addressed by name without reading the roster first.

Implemented in Python (unitgroup.tag_hash) and Rust
(sylpheed_formats::hash::tag_hash) with 3 new unit tests carrying disc-derived
vectors; cargo test -p sylpheed-formats --lib hash is 8/8 green.

Not settled: the guest routine is unlocated, so this uses exact modular
arithmetic where the game may use a Barrett step without final fixup.
2026-08-25 10:38:18 +00:00
Sylpheed RE agent
6903b19a27 backlog: inducer n=2, T74/T75 signature withdrawn, T68/T69 survives 2026-08-25 10:10:27 +00:00
Sylpheed RE agent
820519cb0c re: second freeze at ~96s with the inducer on; T74/T75 signature withdrawn
Run 2 put the inducer on from flight start and froze ~96s in, against 670s
clean with it off -- n=2, contrast sharp, confounder (elapsed mission time)
still untouched.

Withdraws last iteration's 'T74/T75 move off a semaphore onto an event' as the
signature to chase: it does not reproduce. In run 2 they are on XEvent while
HEALTHY and stay there. The healthy state varies between instants, so a
one-sample-per-state diff cannot separate a freeze transition from ordinary
variation -- I read a difference of samples as a difference of states.

Reproduces across both: T68 and T69 go from not-waiting to waiting, T69 on a
semaphore and T68 on an event both times. And 21 of 24 threads unchanged in run
2 (17 of 24 in run 1), so 'not a whole-emulator stall' now has two independent
captures behind it.

Next: repeat the capture several times within one healthy run to establish which
thread states are stable before reading any frozen diff.
2026-08-25 10:10:14 +00:00
Sylpheed RE agent
78f0c8f3ea backlog: n settled as a unit count; in-table roster is the key-resolution rule 2026-08-25 10:04:34 +00:00
Sylpheed RE agent
de42fbd742 re: UnitGroup member field n is a unit count, bounded by the formation
FormationSet_S<NN>.tbl records are slot lists -- 1 + 8*FrameCount fields,
exactly. Resolving every squadron's FormationID and comparing gives
sum(n) <= FrameCount holding 1159/1160 across all 28 stages, 0 unresolved, with
539 filling the formation exactly. The single violation is a debug leftover
(S20, AI_Test / MessageSet_test, Formation_1_only with n=2) and is recorded.

The old 'n is not the _NN suffix of FormationID' observation was right but drew
the wrong conclusion: the suffix IS FrameCount, so n=9 against _30 just means 9
units in 9 of 30 slots.

Also: FormationID does not hash into its table (0/16). FormationSet carries a
name roster record -- no FrameCount, fields are (tag, name, '') with the tags
being the record keys -- the same convention as Enumerate_Squadrons. Second
occurrence of 'keys are resolved by an in-table roster, not by hashing'.

Does not close the 387-vs-300 gap, and the key derivation stays open.
2026-08-25 10:04:20 +00:00
Sylpheed RE agent
96fe3cab0d backlog: S18-S23 settled, S17 confirmed absent, GP_TUTORIAL.pak refuted 2026-08-25 09:35:16 +00:00
Sylpheed RE agent
cb7b0aa461 re: S18-S23 stage records were never missing; only S17 is absent
The 'S17-S23 have no stage record' gap was an artefact of enumerating by the
literal 'Stage_S<NN>'. Tutorial records omit it -- they name no per-stage .xpr
and pull AIParams/weapons/strings/subobjectives/nameplate/collision from a
shared _Tutorial set -- so all six were skipped. stagetbl.py Stage_S18 returns a
full six-record definition and always would have.

Counting distinct *_S<NN> names across all 1119 decompressed entries: UnitGroup
and Route cover 28 stages (S01-S16, S18-S23, S24-S29); Stage literal and
AIParams cover 22; SUBObjectiveSettings 16 (story only). S17 appears in none --
it is not a stage that lost its data, it does not exist.

Tutorial records carry the same Phase_1/2/3 structure as story stages, so the
tutorial is not a special mission type at the data layer.

Refuted en route: GP_TUTORIAL.pak does not hold the config -- 2 RATC entries,
zero IDXD, like GP_CHALLENGE.pak.
2026-08-25 09:34:58 +00:00
Sylpheed RE agent
479632c13b backlog: frozen capture taken, screen_id correction, instrument-provokes-freeze at n=1 2026-08-25 09:03:05 +00:00
Sylpheed RE agent
6900ebe5ca re: the frozen wait-object capture, and screen_id was never a freeze test
Caught the freeze by waiting for the event (frozen.py + in_flight) instead of
sleeping a guessed interval; freeze_waitobj.sh splits into boot/watch so the
wait is not capped by one Bash call. Verified hard: a frame minutes later is
byte-identical to the capture.

Healthy vs frozen, same run: 20 -> 24 wait frames, XEvent 19 -> 23,
XSemaphore 8 -> 7. The signature is per-thread -- 17 of 24 threads sit on the
exact object they were on, four previously-running threads park, and T74/T75
move off a semaphore onto an event. So the freeze is not a whole-emulator stall.

Also corrects the previous entry's test: screen_id reads 'flight' during a
freeze by design, which is why frozen.py exists. Re-testing the saved frames
says that run was genuinely healthy, but it was right by luck.

heavy_read.py added to test whether the instrument provokes the freeze: I/O is
free (371 MB in 0.1s, page cache), the cost is Python-level CPU. One data point
-- 670s clean, then frozen 54s after the inducer started -- recorded as n=1, not
as causation.
2026-08-25 09:02:48 +00:00
Sylpheed RE agent
93ccff50bb backlog: record the WaitMultiple withdrawal, the 30/30 read, and the missing freeze 2026-08-25 08:35:13 +00:00
Sylpheed RE agent
a9526749dc re: WaitMultiple read confirmed on a live capture; XTimer is a third type
23 wait frames, 30 objects, nothing unresolved -- the second deref turns every
former miss into a resolved object, as predicted. XEvent 20 / XSemaphore 9 /
XTimer 1; every WaitMultiple thread waits on a pair, and 78/79/80 and 64/65 are
worker groups sharing a handle.

%ebp does not survive as the count -- WaitMultiple reuses it at 8fc158 -- so the
array is bounded by reading until an entry stops resolving instead.

The frozen capture is still not taken: screen_id reads 'flight' at the second
capture and out to ~470s, so the mission never black-screened. The diff in the
data file is two healthy captures and is recorded as such.
2026-08-25 08:34:46 +00:00
Sylpheed RE agent
6f6ace5ec7 re: withdraw the 'unrestorable rbx' claim — the misses are WaitMultiple
The 8 threads whose [rbx] did not resolve to a vtable were never in
XObject::Wait. The backtrace grep matched WaitMultiple as a substring, and
there %rbx is the XObject** array (mov %rsi,%rbx) with the count in %ebp, so
[rbx] is objects[0] -- an object pointer, needing a second deref -- not a
vtable. The unwind restored rbx correctly for all 18.

freeze_waitobj.sh now takes the function and frame index from the backtrace and
applies the matching read, and captures twice in one run (healthy and after the
~270s black-screen) so the comparison is within-run. waitobj_report.py tabulates
both and diffs them, discarding any value info symbol cannot resolve.
2026-08-25 08:20:21 +00:00
Sylpheed RE agent
bfd4d55ce9 re: two wait object types, and the read validates itself
Re-extracting the same gdb capture per thread rather than by grep qualifies the
previous entry. Of eighteen threads whose frame 3 is XObject::Wait, eight have
[rbx] equal to the XEvent vtable plus sixteen, two equal to the XSemaphore
vtable plus sixteen, and eight hold a pointer into the mmap region that is not a
vtable at all. So the waits are on two distinct kernel types, XEvent and
XSemaphore, and the earlier claim that the object is an XEvent was right for the
majority but not the whole picture.

The eight non-vtable readings are the method checking itself rather than a
failure. A polymorphic object's first word is always a vtable pointer, so those
values are simply not this: rbx could not be restored for those frames and the
unwind returned whatever the register held. A reading counts only if [rbx] lands
in the binary's vtable range and resolves to a "vtable for" symbol; anything
else is discarded rather than interpreted. Ten of eighteen resolve and the rest
are honestly unknown.

That also settles the previous entry's worry that the 0x7ffc and 0x7ffd
addresses looked like stack. They are the shared mmap region, which holds thread
stacks and large allocations alike, so the vtable check rather than the address
range is what separates an object from a stack slot.

The follow-up run that would have added /proc/<pid>/maps classification and a
wider object dump never booted -- EMULATOR GONE at 0s, skip_intro exit 4 --
most likely a stale emulator or lockfile from the preceding gdb session, whose
process tree is parented differently and escaped the usual cleanup. So the map
classification, the multi-word object dump and the frozen-state capture are all
still unrun.
2026-08-25 08:10:56 +00:00
Sylpheed RE agent
a153c392f5 re: the waited object reads as an xe::kernel::XEvent
Executed the wait-object read on a live run under gdb. For every thread whose
frame 3 is XObject::Wait, rbx gives a pointer whose first quadword is
0x5555562db8f0, which is the PIE base plus 0xd878f0, and the symbol table has
vtable for xe::kernel::XEvent at 0xd878e0. The stored pointer is the vtable
symbol plus sixteen, since offset-to-top and RTTI come first, so it matches
exactly. The same vtable appears on every sampled waiting thread while the this
pointers differ, meaning many threads waiting on different XEvent instances. The
whole chain needs no DWARF and no rebuild, as the static groundwork predicted.

Two caveats, both recorded rather than smoothed over.

This is a healthy-play snapshot, not the freeze. The capture landed at 195 s of
flight with screen_id reporting flight and a non-black mean, because boot under
gdb costs about 300 s and the entire experiment has to fit inside one call --
a timeout kills the process group and takes the emulator with it, which lost an
earlier attempt outright. So this describes what threads wait on during normal
play, which is the control the frozen capture never had, but it is not the
frozen case.

And the this addresses look like host stack rather than heap. Either xenia
places these objects somewhere unusual, or rbx at frame 3 is not Wait's this
after the unwind and the vtable match is coincidence. An exact plus-sixteen
match on a known symbol is hard to get by accident, but the address range is not
what was expected. The settling check is to dump a few words at rbx and see
whether they look like an XEvent -- vtable, KernelState pointer, handle and type
fields -- or like saved registers.
2026-08-25 08:04:33 +00:00
Sylpheed RE agent
4ef15a1dbe re: route 1 is viable — XObject::Wait keeps this in %rbx and .eh_frame restores it
Pure static analysis, no run spent. XObject::Wait's prologue does mov %rdi,%rbx
at 8fbc9c, so the this pointer lives in a callee-saved register rather than a
stack slot. And the binary carries full unwind information: .eh_frame with
127231 FDEs, which survives in Release builds because C++ exceptions need it,
including an FDE covering 8fbc90 to 8fbde2 that tracks rbx explicitly.

Together those mean that from a thread frozen deep in pthread_cond_wait, moving
to the XObject::Wait frame and reading rbx yields the XObject being waited on --
gdb reconstructs callee-saved registers during the unwind from .eh_frame alone,
with no DWARF involved. Reading the first quadword at that pointer gives the
vtable, and vtable symbols are in the symtab, so the object's concrete type is
identifiable too.

This revises the previous entry, which listed route 1 as per-frame archaeology
that must be redone whenever the binary changes, and route 2, a RelWithDebInfo
rebuild, as what would make the question easy. Route 1 is neither expensive nor
fragile: two gdb commands per thread, no rebuild, and the oracle stays
byte-identical to the binary every other measurement in this corpus was taken
against.

Not yet executed on a frozen run, which is the next step and is now a small one.
2026-08-25 07:42:12 +00:00
Sylpheed RE agent
0120677722 re: freeze frontier — wait confirmed by ordinal, but info args will not work here
Two corrections to the plan the freeze file leaves for the next pass, both cheap
to establish and both worth having before a run is spent discovering them.

The export in the frozen frame is confirmed. The shim frame carries its ordinal
as a template parameter, 176 decimal, which is 0xB0, and xboxkrnl_table.inc line
190 maps that to KeWaitForSingleObject. The identification was previously read
off frame names; it is now pinned by ordinal.

The file's stated next step does not work on this binary. It says the wait
target is one info-args-equivalent away now that a debugger can be attached, but
readelf shows zero debug sections and only a symtab of 27807 entries. The
Release build is not stripped, which is why frames carry function names, but it
has no debug info at all, so gdb can name a frame and nothing inside it: no info
args, no p *this, no member access. The earlier note conflated "not stripped"
with "has debug info".

Two routes remain and their costs are now recorded. A prologue-guided stack read
can recover XObject::Wait's spilled this pointer from a known stack slot, needing
no rebuild and keeping the oracle byte-identical to the one every other
measurement in this corpus used, at the price of per-frame archaeology that must
be redone if the binary changes. Alternatively a RelWithDebInfo build via
build-canary makes this and every future freeze question straightforward, at the
price of a full compile and a binary that differs from the measured one.

Neither is attempted here. Recorded so the next pass picks with the costs
visible.
2026-08-25 07:36:56 +00:00
Sylpheed RE agent
eb7cedb13e re: a freeze with zero refused resumes; and fix the index that caused duplicate work
One new data point on the in-mission freeze. The refuted resume-spin lead rested
on refused resumes being normal -- thousands during gameplay, more in a healthy
run than a frozen one. Today's freeze log contains zero of them across 1147
lines, and the mission still froze at about 267 s with a black screen, against
2447 in an older log. So the warning is not even necessary for a freeze, let
alone sufficient, which closes the lead from the other side.

Also refuted today, before this file was found: the burst of BaseHeap::Release
failures at the end of the short log looks like a freeze signature and is not.
In the longer log the same failures span lines 1044 to 5210 and the log
continues for 2700 lines afterwards; they begin at mission load in both runs and
are routine.

The uncomfortable part is that both were already settled in
mission-freeze-resume-spin.md, as was the 0xbdb59668 address that a previous
iteration rediscovered independently. That is twice in one session that existing
work was redone.

The cause is mechanical rather than forgetfulness. docs/re/INDEX.md listed 20 of
43 notes and none of the recent ones, so searching the index for prior work on
the freeze returned nothing -- the corpus was searched, but the search was
blind. INDEX.md now carries a generated table of every note under docs/re/, 59
entries with title and status, and states outright that it should be searched
before starting an investigation. Regenerating it is a few lines of Python and
should be redone whenever notes are added.
2026-08-25 07:30:40 +00:00
Sylpheed RE agent
925006d38f re: the mission black-screens at ~4.5 min — that is what ends every run
A two-segment OB watch made the mechanism visible. Segment 1 ran clean, with the
witness reporting zero stalled samples of fifty and OB steady at 4 for 250
seconds. Segment 2, attaching to the same still-alive emulator, got "NOT IN A
MISSION (no unit definitions)".

The emulator was alive; the mission was gone. The pilot's hull reads 1500 in
every line, so the player never died. Its log stops at t=267 s with consecutive
byte-identical lines, the freeze signature. And the screen is now entirely
black, mean (0,0,0). This is the pre-existing mission-end black-screen freeze
already recorded in the corpus, caught in the act. Segment 1's clean witness is
not a contradiction, since it sampled to about 250 s and the freeze came at 267.

That corrects an earlier claim. mission-arrival-watch.md records the blocker as
removed because the emulator survives between calls, which is true, but it was
stated as though observation could be extended indefinitely within a turn. The
emulator surviving is not the mission surviving. The binding limit is the game's
own black-screen at roughly four and a half minutes and no amount of chaining
crosses it. The earlier chained run that reached 435 s of cumulative
verified-live observation did so across two segments of one mission, which fits:
its second segment froze at t=135 s.

The consequence is worth stating plainly. Any experiment needing more than about
four and a half minutes of one continuous mission is not doable on this build,
chained or not, and that includes clearing sixteen marked fighters at two kills
per five minutes. The freeze is therefore now the highest-value target in its own
right, since fixing it would unblock the arrival question, the phase-advance
question and the OB series together. It is also a long-standing known defect
rather than anything this work introduced.
2026-08-25 07:23:27 +00:00
Sylpheed RE agent
70b1e453a7 re: OB address is run-dependent; watcher hunts it, HUD reader gated on confidence
ob_watch.py verifies the address against a confidence-gated HUD reading before
reporting anything, and two consecutive fresh launches settle the question:
0xbdb59668 held 3165285888 against a HUD of 4 on one, and exactly 4 on the next.
The address is not stable across launches, the old note that it recurs in about
five runs of seven was right, and the gate did its job by refusing to report a
series from an address that did not describe that run. The watcher now hunts the
address on the current run when confirmation fails, using the same intersection
method, so it no longer depends on a lucky launch.

The HUD reader is also gated now. ob_read returns a best and second score per
digit and those were printed but never checked, which is how one misread
poisoned an intersection and produced a wrong refutation of big-endian u32. A
reading is accepted only if every digit scores at least 0.80 with a margin of at
least 0.05, the rule ob_read's own docstring states.

The measurement itself is a negative. With a clean witness, zero stalled samples
of fifty, OB held at 4 for 250 seconds while the pilot targeted e010 for 1964
ticks and fired on 1635 of them. Constant fire at the marked attackers and not
one decrement, so it destroyed none, which matches the roughly two marked kills
per five minutes measured earlier. The fire rate itself rose from 4.6 % of ticks
in an earlier diagnosis to 83 % here without producing more kills.

Recorded as unreproduced rather than explained away: the run that found the
address saw the counter rise 4, 8, 12 over five minutes, and that reading was
confirmed against the HUD. This run was flat over a comparable window. Both
observations are sound and they disagree, so the rise is not a stable property
of the mission's first five minutes and presumably depends on progress this run
never reached.
2026-08-25 06:02:33 +00:00
Sylpheed RE agent
d1154d7ca6 re: REMAINING OB found and verified — big-endian u32 at 0xbdb59668
Ported onto the shared harness, the HUD changed from 4 to 8 and the intersection
collapsed in a single step: one u32be survivor at 0xbdb59668, with the u16be and
u8 hits at 0xbdb5966a and 0xbdb5966b being the low half and low byte of that same
word.

Verified live rather than asserted. Reading screenshot and memory together three
times, the one legible HUD frame showed 012 against mem@0xbdb59668 = 12; the
other two frames were unreadable rather than mismatched. This independently
rediscovers the address the earliest sessions found by digit-transition hunting,
by a completely different method.

It also corrects an earlier conclusion. A previous run intersected HUD readings
of 4 and then 11, got zero survivors in every encoding, and that was written up
as eliminating big-endian u32 for the whole region. This run shows u32be holds
the counter, so the refutation was wrong. The likely cause is the input:
ob_digits.png has templates for 0 1 2 4 8 only, so values containing other
digits are misread rather than rejected, and "11" was probably one of those. A
single bad reading poisons an intersection permanently, because it removes the
true address and nothing later can restore it. The lesson is that an
intersection method needs individually verifiable inputs -- the reader's
confidence scores were printed but never gated on.

One observation reopens the arrival question in a useful way: the counter
increases, 4 then 8 then 12 across about five minutes, measured in memory so not
a digit misread. A count of remaining marked targets that rises means targets
are being added during the mission. That does not contradict the deployment
finding, since the roster is fixed at load, but it does mean the game marks new
objective targets as the mission proceeds. Watching this one address across a
whole mission is now the obvious next experiment and costs almost nothing.
2026-08-25 05:40:30 +00:00
Sylpheed RE agent
feb535a8fb re: build a shared probe harness so the same lessons stop being re-learned
Four probes were written from a blank file and each re-learned the same lessons
by losing a run: that a flat run cannot be told from a frozen guest without a
stall witness, that results held to the end of a run are destroyed by a turn
timeout, that a roster count which is not the stage's member count means a
different stage loaded and must be discarded, and that a run's witness state has
to be read before its numbers. Writing each lesson down did not stop the next
probe repeating it, because each probe started from nothing.

probeharness.py makes them structural. Probe(baseline=N) discovers the roster,
rescans up to five times and refuses to start if the count never reaches the
baseline. The witness is calibrated on construction, sampled by tick() and
reported by status() and summary(), so a probe cannot forget it, and when no
witness is found it reports UNVALIDATED rather than zero stalls. emit() flushes
on every line. craft(), strengths(), alive() and heap() supply the
roster-to-craft link, per-record liveness and the raw heap, so a new probe
writes only its own logic.

Verified rather than asserted: deploy_probe.py reimplements the per-record
deployment watch on top of it in about forty lines against wave7_probe's
hundred and fifty, and its first live run was clean -- 116 roster records, 32
witnesses at 10/s, zero stalled samples, seven losses tracked, and the TSV
written incrementally. Nothing about the result is new, which is the point: the
harness reproduces a known-good measurement.

The existing probes are deliberately not ported. They work, and rewriting them
would risk changing results other documents cite. New probes should use the
harness; old ones should be ported when they next need a change.
2026-08-25 05:22:24 +00:00
Sylpheed RE agent
738df50803 re: widen the OB scan to seven encodings; run inconclusive, probe lacked a witness
ob_by_hud.py now scans seven readings of the same bytes and keeps a separate
candidate set for each, as byte offsets: u32 big and little endian, u16 both
endiannesses at both alignments, and u8. The big-endian u32 reading had been
refuted, so widening rather than assuming is the point.

u32le is much the tightest at 154 candidates against u32be's 4452. That is a
hint about the encoding rather than a result, since a rarer bit pattern narrows
faster regardless of meaning.

The run is inconclusive. The HUD read 4 at every sample, so there was no second
value to collapse the sets against, and from t=136 the candidate counts are
byte-identical across five samples in all seven encodings, which is what a
frozen guest looks like -- nothing in 32 MB changed at all. The probe had no
stall witness, so the run cannot prove it either way. One is added now.

Worth stating plainly: this is the fourth probe written without a witness and
the third whose flat output could not be distinguished from a freeze. Each time
the fix gets applied to that one script. The durable fix is the shared probe
harness already noted in this file, and the lesson recurring four times is
itself the argument for building it.

What the hunt needs is unchanged: two HUD readings at different values in
non-stalled samples. The counter moves on kills, which lands back on the combat
limit, though the earlier 4 to 11 observation shows it does move.
2026-08-24 23:14:59 +00:00
Sylpheed RE agent
6fc082ca55 re: kill-free HUD route for REMAINING OB; big-endian u32 assumption refuted
The correlation route is gated on marked-fighter kills, which the pilot manages
at about two per five minutes. ob_read.py already reads the counter off the
screen, so ob_by_hud.py matches the displayed value against memory directly and
needs no kills at all: screenshot, read the digits, keep heap words equal to that
value, intersect across readings.

Four readings at value 4 narrowed 6156 candidates to 4312, the expected slow
drift. Then the HUD read 11 and the intersection collapsed to zero. A word
holding this counter must equal 4 at the first four samples and 11 at the last,
and none does, so within the entity heap read as big-endian u32 the counter does
not exist. It may be u16, u8, little-endian, or outside that region. Both
previous hunts assumed big-endian u32 there, so this eliminates the assumption
rather than merely failing to find anything.

The displayed value also went up, from 4 to 11 over about 340 seconds. A pure
countdown of remaining marked targets should not rise, and the deployment work
says phase 1 gains no new participants. Three readings are possible and none is
tested: the cell being read is not REMAINING OB, the digits are misread, or the
counter genuinely counts something that can increase. The two clean readings
scored 0.95 to 0.98 against their templates, but 4 and 11 use only digits that
are in the strip, which is exactly the selection effect that would hide a wrong
reading -- the template set covers 0 1 2 4 8 only, and most samples came back
unreadable.

Next is widening the scan to u16 and u8 and to little-endian, and beyond the
entity heap, which is a change to one function and costs no combat. Extending
ob_digits.png with the missing digits would also raise the sample yield, since
only two of eleven readings in a 480 s run were usable.
2026-08-24 22:58:55 +00:00
Sylpheed RE agent
7503066a91 re: OB correlation method proven; turret tracking refuted
With the value filter in place, an attach watching turret losses -- frequent,
unlike marked-fighter kills -- narrowed from 374 candidates to 2 to 0 across
three events. That is exactly how a correlation search should behave, and it
ends in a refutation rather than fizzling out: no plain u32 in the entity heap
decrements consistently with turret kills.

Witness checked first, per the rule adopted last iteration: 12 of 25 samples
stalled, but all four kill events fall in the early non-stalled stretch between
t=45 and t=177, so the events themselves are sound.

The negative fits the objective text rather than contradicting it. Stage 02
phase 1 asks to shoot down all invading enemy fighters, and turrets are not
fighters, so a counter of remaining marked targets should not move when a turret
dies. It also rules out the cheaper reading that REMAINING OB is a general kill
or enemy-remaining tally: it ignored ten turret deaths.

The method is proven and the search region is right, since 0xbdb59668 where the
counter was originally seen lies inside the scanned range. What is missing is
unchanged -- two or three e010 kill events in non-stalled samples -- against the
same two obstacles, the freeze rate and a pilot managing about two
marked-fighter kills per five minutes. One clean 220 s run this iteration
produced zero.
2026-08-24 22:39:59 +00:00
Sylpheed RE agent
95535d09da re: OB hunt second attempt — saving verified, filter fixed, attach was frozen
The incremental-save fix is verified. A fresh mission caught one e010 event at
t=241 s and wrote 1187 candidates to disk immediately; the turn timeout then
fired exactly as before, but this time the data survived. The session also
clears the candidate file at launch, since candidate offsets are only meaningful
within one emulator instance and resuming across launches would intersect
unrelated addresses.

The correlation itself was wrong though. It matched on the delta alone, so any
two float bit patterns whose integer representations differ by the loss count
qualified, and in a heap full of positions and velocities that is thousands of
words. The 1187 survivors were things like 1044450858, about 0.1f, and
3212461993, a negative float. Candidates must now also look like a counter --
a small non-negative integer in both samples -- which removes the noise by
construction instead of hoping the intersection washes it out.

The follow-up attach logged zero events across 520 s, which reads like the
combat-effectiveness limit again. It was not: 25 of its 26 samples were flagged
GUEST STALLED, so the guest was frozen for essentially the whole window. The
witness added last iteration did its job, and the lesson is about reading it --
the run summary quoted "0 events" first and the stall count only surfaced on a
deliberate check. A run's witness result should be the first thing looked at,
before any interpretation of what the run showed.

Still unfinished, with no address identified. What is needed is unchanged, two
or three e010 kill events in non-stalled samples, and the two obstacles are now
clearly separate: the freeze rate, and a pilot that manages about two
marked-fighter kills per five minutes.
2026-08-24 22:15:15 +00:00
Sylpheed RE agent
462c3dc019 re: record the REMAINING OB hunt in the backlog index
The previous commit landed the doc and the probes but its BACKLOG edit asserted
on text that had since moved, so the index did not mention the hunt at all.

That is the third time an assert-then-replace patch has silently failed this
way. The pattern is the problem: the assertion aborts the edit but the commit
that follows still succeeds, so a missing index entry looks like a clean run.
Locating the anchor first and verifying the insert afterwards, as done here, is
the habit to keep.
2026-08-24 21:50:34 +00:00
Sylpheed RE agent
f6f8930ee7 re: REMAINING OB by correlation — method works, run unfinished, two repeat defects
The objective text settles what the counter is, so the hunt can be a correlation
rather than a value scan: keep every word in the 32 MB heap that fell by the same
amount, in the same interval, as a named e010 loss. One event cut roughly eight
million words to 1056, so two or three more should leave a handful.

The run did not get them. The turn's timeout fired at t=219 s and the probe saved
its candidate set only at the end, so the 1056 were discarded and the follow-up
attach started from nothing. That is the same mistake already recorded in
guest-stalls.md, where an earlier probe deferred all analysis to the end and a
timeout killed it with 240 s of data in memory and nothing written. The lesson
was written down and then repeated in a new script four iterations later.

The attach had a second gap: 535 s with zero losses of any kind, which is
indistinguishable from a freeze, and ob_probe2 carried no stall witness so the
run cannot say which it was.

Both are fixed. Candidates are written after every event and SYLPH_OB_RESUME=1
reloads them so a chained attach keeps intersecting on the same mission, and the
witness from wave7_probe is carried here.

The underlying pattern is worth naming: each new probe starts from scratch and
re-earns the same lessons about saving incrementally and validating liveness. A
shared probe harness would stick where written-down lessons have not.

The hunt itself is unfinished. No address is identified, and finishing needs a
run that catches two or three marked-fighter kills, which is the same
combat-effectiveness limit already recorded -- about two per five minutes against
a dozen turrets.
2026-08-24 21:50:10 +00:00
Sylpheed RE agent
6bb8b14536 re: the per-phase objectives, from the game's own text
TextIndex in sylpheed-formats::localization already indexes per-phase objective
text under S<NN>_P<n>_Objective_<i>, with hints and lose conditions alongside.
Querying it settles what the wave work spent many iterations circling, and a
disc test now pins it.

Stage 02 phase 1 is "Shoot down all invading enemy fighters while watching out
for attacks on the ACROPOLIS", losing if the ACROPOLIS is sunk. Phase 2 is
"Protect the Admiral's ship the CALIBAN until it has entered the safe zone",
losing if the CALIBAN is sunk. Phase 3 is "Destroy the interstellar cruise
missiles headed for Schlos Base", matching phase 3's roster of exactly nine
ISCMissile and SUBOBJ_013.

That explains the phase advances and shows they are not all the same kind. Phase
1 is a kill-all-marked-fighters gate -- fighters, not the turrets every run
killed almost exclusively. Phase 2 is positional and ends when the CALIBAN
reaches a safe zone, so "what advances a phase" never had a single answer.

It also closes a loop to the first session. The hints say red mission markers
[OB] indicate your targets, so REMAINING OB -- the HUD counter hunted in the
earliest iterations and located at 0xbdb59668 -- is the count of remaining
marked objective targets, which is phase-1 progress itself. The old reframing
that the counter is not a roster was right, and this names what it is. It is
also the correct signal to watch for an advance: not deployed, which only
changes once the next phase deploys, but REMAINING OB reaching zero, which is
the cause.

Method lesson recorded because the reflex will recur: localization.rs has had
objectives, hints and lose_conditions for some time, and several iterations were
spent reconstructing the same information by hand from SUBObjective strings and
the guide script. The reconstruction was not wasted -- it produced the IXUD
UTF-16BE and language\ prefix findings independently -- but the corpus should
have been searched before the disc was, and grep -rl TextIndex crates/ would
have saved the detour.

Minor discrepancy noted, not investigated: ixud.rs documents the string pool as
UTF-16BE and localization.rs as UTF-16LE.
2026-08-24 21:24:38 +00:00
Sylpheed RE agent
be6207822d re: target preference works; the blocker is now combat effectiveness, not RE
pilot.py gained SYLPH_PREFER, a unit-name substring whose matches get their
target score multiplied by 0.05 while everything else is multiplied by 4.0. With
SYLPH_PREFER=e010 a clean 320 s run, zero stalls by the witness, killed eight
turrets and two Attacker_S.

The preference is real -- e010 kills went from roughly one across all previous
runs to two in a single run -- but it is weak. Turrets still outnumber attackers
four to one in the kill log, because target commitment and simple proximity keep
pulling the nose back to them, and phase 1 fields 108 turret craft against 16
attackers. Deployed stayed at 41 throughout, so no phase advance.

That quantifies the blocker. Clearing the marked attackers means destroying 16
craft, and at two per 320 s that is about 2560 seconds, roughly 43 minutes of
continuous verified-live flight across many chained attaches, against a freeze
rate of about two runs in five.

This is no longer a reverse-engineering problem. Everything needed to observe
the phase advance is built and validated -- the roster-to-craft link, the
liveness read, the stall witness, chained attaches and the discard rule. What is
missing is a pilot good enough to complete the mission objective, which is
game-playing work with an uncertain payoff.

The choice is recorded rather than made, because it is about how much effort one
confirmation is worth rather than a technical unknown: invest in the pilot,
accept the static answer where only the trigger is inferred rather than
observed, or attempt one very long chained run betting against the freeze rate.
2026-08-24 21:18:15 +00:00
Sylpheed RE agent
56e24ce9b1 re: the mission script is readable — phase 1 never advanced because we killed the wrong things
The localised string tables are decoded. language\*_local_string.tbl uses a
third archive prefix, language\, and is not IDXD: the magic is IXUD and the
strings are UTF-16 big-endian, which is why earlier ASCII-oriented dumps of
these files produced garbage.

That makes Stage 02's nine sub-objectives readable in words -- rescue the ally
destroyer, destroy all enemy fighters, sink all enemy warships, rescue the
Amalthea, keep the Caliban undamaged, sink the destroyer targeting the
Acropolis, destroy all enemy cruise missiles. Ninety-one objectives are defined
across the game.

The guide script then confirms the phase structure twice over. It reads as three
acts -- marked attackers and protecting the Acropolis, then a warship whose
engine, weapons and shield projector must be destroyed, then cruise missiles
taken down one by one -- and each act matches the per-phase roster composition
found independently. Phase 3 fields exactly nine UN_e201_ADAN_ISCMissile against
SUBOBJ_013's "You destroyed all enemy cruise missiles", an exact match between
two files decoded on different days by different routes.

And it gives a mundane explanation for every null result in the arrival work.
The phase-1 objective is stated outright in the script: the attackers with the
orange markers are the targets, meaning UN_e010_ADAN_Attacker_S, of which phase
1 fields four squadrons. Every loss line in every run log reads
UN_e007_ADAN_Turret, with e010 appearing twice in total, because SYLPH_HUNT was
built to make turrets targets and SYLPH_KEEPOUT was tuned for them. The 42 kills
were overwhelmingly the wrong 42.

Kept at 🟡 because the objective text is certain but the inference that this
particular objective gates the phase advance comes from the script's structure
rather than from measurement. The test is a pilot that prioritises e010 over
e007, which is a target-preference knob rather than new decoding, with the
existing probe watching deployed jump from 41 toward the phase-2 roster.
2026-08-24 21:02:10 +00:00
Sylpheed RE agent
4bb4e3691d re: message tables searched — phases are pervasive, the trigger is not in data
StageMessageSet_S02.tbl is resolved. stage-mission-tables.md had recorded it as
absent from the pak; it is present, and the archive prefix is message\ rather
than stage\ like its siblings. Sweeping prefix candidates over the full 16630
entry TOC found it immediately. It holds a voice mapping, a unit message set
pointer, and a list of 31 message tables including ScriptMessage_S02_msg.tbl and
the per-character PresetMessage sets.

message\UnitMessageSet_S02.tbl then gives an independent confirmation of the
phase structure. Its 26 records have exactly four named fields: CrewCount and
PresetMessage_Phase1, Phase2 and Phase3. Every message set carries a different
preset table per phase, so phases now appear in three unrelated table families --
the stage record, the route names, and the message sets. They are not a quirk of
one file.

The promising lead did not survive. ScriptMessage_S02_msg.tbl has 149 records
whose third positional field takes values like Killed and None, which looked
like a trigger. Enumerated across S01 and S02 it is a small closed vocabulary of
None, Emergency, Killed and Noise -- delivery categories for radio chatter,
meaning how a line is played rather than mission control flow -- and the fourth
field is 1 in every record, so it is not an argument either. Rejected.

That leaves no table anywhere carrying a phase-advance condition. Everything
found is a consequence of the phase and never its cause, which points at the
advance logic living in code. The only place left is the executable, and
default.xex is encrypted on disc, so the decrypted image exists only in guest
RAM and reaching it needs a run plus function-level work. Recorded as the honest
end of the static search rather than continuing to guess at table names.
2026-08-24 20:55:34 +00:00
Sylpheed RE agent
aac791e0e9 re: enemies come into play per phase, deployed at phase start
Cross-referencing Route_S02 with UnitGroup_S02 gives per-phase membership: phase
1 has 37 squadrons and 42 members, phase 2 has 34 and 36, phase 3 has 44 and 49,
with squadrons appearing in more than one phase so the totals overlap. The
measured deployed count, stable across many runs and from the first observable
instant, is 41 of 116 records.

41 against 42. The deployed set is the phase-1 roster, to within one record.

That makes every previously puzzling observation coherent. All 116 records exist
at mission load, confirmed on two stages, but only the current phase's squadrons
hold craft. Deployed is 41 before the ready room ends and never moves because
phase 1 deploys as a unit at load rather than on a schedule. No arrival was ever
observed in up to 435 s of verified-live flight with 42 kills because phase 1
never completed in any run, and phases 2 and 3 hold the other 85 members. And
the route times are within-phase fly-in timings rather than release times, which
fits the frames reading where Stage 01 phase 2's 1500, 1800 and 2100 are 50, 60
and 70 seconds at 30 Hz.

So enemies come into play per phase, deployed together at phase start, with each
squadron's route giving its entrance path and the moment it flies in.

The off-by-one is not explained and is recorded rather than rounded away. The
player's record may be counted differently, one squadron may hold craft in a way
the liveness read misses, or one phase-1 member may genuinely not deploy. A
one-record gap is exactly the size of the errors that have misled this corpus
before.

What advances a phase is now the single remaining question. It is directly
observable, since deployed would jump toward the phase-2 roster. It did not
happen in 435 s of verified-live flight with 42 craft destroyed, so it is
neither a short timer nor a small kill count. SUBObjectiveSettings names nine
objectives but carries only rewards and HUD strings, no trigger. Not yet
searched: StageMessageSet_S02.tbl, which has never resolved in the pak, and the
executable, whose decrypted image exists only in guest RAM.
2026-08-24 20:49:10 +00:00
Sylpheed RE agent
1f8a5b8dd5 re: deployment is resolved at mission load; withdraw the take-off hypothesis
The run meant to catch a deployment step at take-off removed the hypothesis
instead. One continuous series of 64 samples over about 380 s, started when the
roster first appeared and running through the ready room, the transition to
flight at +47 s, and several minutes of flight, shows deployed at 41 and craft
at 292 with not one change in either number.

So the previous iteration's suggestion that deployment is a single step at
take-off is withdrawn. It rested on ready room 39/276 against flight 41/300,
which came from different runs and was flagged 🟡 for that reason. Measured
within one run the values are identical before and after take-off, so that gap
was cross-run variance.

What this does establish is stronger. The roster is already deployed when the
roster first exists: 41 of 116 records hold craft before the ready room ends and
nothing changes afterwards. Together with the roster identity confirmed on two
stages and the earlier piloted runs, Stage 02 phase 1 has no observable
in-mission arrival at all -- not in the ready room, not at take-off, and not in
up to 435 s of verified-live flight during which the player destroyed 42 craft.

The route times still mean something, since 0/90/120/170/210/240 and Stage 01's
1500/1800/2100 are structured rather than noise. The frames reading remains the
best fit, and under it they are most likely fly-in animation timings applied at
load rather than release times.

Two things stay open and are recorded as such. Phases 2 and 3 have never been
tested: every run has stayed in phase 1 and phase advance was never located, so
a phase transition is now the most likely place an arrival could exist. And this
particular run had no pilot, so it adds nothing on event-gating; the piloted
435 s run already covers that ground.
2026-08-24 20:43:48 +00:00
Sylpheed RE agent
9daf9dc592 re: sample from the first moment — still flat, and the wait variant caught the ready room
early_probe defers everything expensive: no witness calibration, no per-record
labelling, and the heap scan uses bytes.find. Setup now completes in 0.5 to 0.8
seconds instead of about 25, so the first sample lands essentially at flight
detection.

It is still flat. Deployed reads 41 at flight+0.8 s and every sample after, with
the only change in 252 s being 41 to 40 when one squadron was wiped out. No
climb at any point. Under the frames reading that is expected, since the whole
phase-1 schedule would be finished within eight seconds of mission start.

The attempt to get ahead of flight detection did not work, and the reason is
worth recording. A second run started the probe before the launch and waited for
the roster to appear. It appeared with 116 records, but the numbers were
deployed 39 and craft 276, flat for the whole window. That is the ready room:
the roster is built before take-off, so waiting for it catches the pre-flight
scene, and the probe's window expired around the time flight actually began.
Waiting for the roster is not the same as catching mission start, and the test
as designed does not do what it claims.

The two runs together do suggest something, held at 🟡 because they are
different runs. The ready room shows 39 deployed and 276 craft while flight
shows 41 and 300, so two records and twenty-four craft appear between them. That
points at deployment being a single step at take-off rather than a schedule
unfolding during the mission.

Next is one run of about 400 s with the probe waiting for the roster, long
enough to span ready room, take-off and flight in a single continuous series,
which would show the 39 to 41 step directly or refute it.
2026-08-24 20:28:15 +00:00
Sylpheed RE agent
da94e705bd re: the 42 anomaly is Stage 01, and the route times are probably frames
The rescan-until-baseline retry settled the anomaly: five rescans over 50 s
stuck at 42, so not a load race. Inspecting the live mission rather than
guessing, the records include UN_S01_Asteroid_cmesh_* and UnitGroup_S01.tbl is
resident while UnitGroup_S02.tbl is not. The launch sometimes loads Stage 01
instead of Stage 02, and the discard rule caught every such run, which is why
none of them contaminated a result.

That accident supplies a test previously recorded as blocked. The roster
identity needed checking against a different stage, and no second save exists.
Stage 01 provides one: static 42 members and 13 distinct units against live 42
roster records and 13 definitions, with composition matching 20/20, 6/6, 5/5 and
2/2. One record per roster member now holds on two independent stages with
completely different rosters.

Stage 01's timetable also bears on the unit question, which has been open since
the schedule was found. Its phase 2 has entries at t=1500, 1800 and 2100. As
seconds that is 25 to 35 minutes into a single phase, with three arrivals spread
across the last ten minutes of it, which is not a plausible mission. At 30 Hz
they are 50, 60 and 70 seconds, which is exactly the shape of a paced phase.

If the unit is frames, Stage 02's phase-1 entries at 90, 120, 170, 210 and 240
are 3 to 8 seconds, every one of them before the probe's first sample, which
lands about 25 s after flight is detected. That would explain every null result
in this line of work at a stroke, and it fits what the probe has always seen:
deployed already at 41 by t=0 and never changing.

Kept at 🟡 -- it is an inference from the implausibility of one reading rather
than a measurement -- but it is now the leading explanation, ahead of
event-gating. The test is to sample at flight+2 s and compare with flight+30 s,
which needs the probe's enumeration and calibration reordered after a first
cheap sample rather than any new decoding.
2026-08-24 20:07:10 +00:00
Sylpheed RE agent
222f35a7a7 re: blocker removed — chained attaches extend the observation window
The previous iteration declared route entries t=210 and t=240 unreachable
because a shell call is capped at 595 s. That was wrong, and it rested on an
assumption I never checked: launch_mission.sh leaves the emulator running and it
survives between Bash calls within a turn. Verified directly -- 611 s elapsed
and still running after the launching call had returned.

attach_session.sh attaches a pilot and probe to an already-running mission, and
attaches chain, so the window is bounded by the turn rather than by one call.

On one continuous mission: 300 s of clean observation followed by an attach that
ran 135 s before freezing, giving 435 s of cumulative verified-live wall-clock.
The craft population fell from 300 to 258, so 42 were destroyed, and deployed
fell 41 to 38. Zero confirmed arrivals throughout -- the longest verified-live
observation so far. That covers the first four phase-1 route entries, t=90, 120,
170 and 210; t=240 needs 436 s and was missed by a second.

The coverage claim is qualified rather than asserted. The wall-to-game
conversion uses 55 %, from a 16.5/s frame rate against an assumed 30 Hz tick,
but the witness has measured 8, 11, 11, 21 and 24 per second across runs. At the
low end the factor is 0.27 and the same 435 s covers only about 117
game-seconds, reaching t=90 alone. So what is established is that nothing
arrived in 435 s of verified-live phase-1 flight with 42 kills; how many
scheduled arrival times that spans depends on a tick rate that is not yet
pinned, and pinning it is what would make the result sharp.

Also refuted: SYLPH_HZ=3, a lower pilot poll rate, produced the lowest
calibrated frame rate of any run at 8/s while landing the most kills at 26. The
pilot's polling is not the throttle and lowering it does not buy game time.
2026-08-24 19:54:32 +00:00
Sylpheed RE agent
8cc05692d6 re: blocker — the schedule's later entries cannot be reached in one turn
Computed the observation ceiling, which should have been done several iterations
ago instead of being approached one run at a time. A turn's shell call is capped
at 595 s; boot costs about 220 s, of which roughly 190 s is a title movie the
launch script waits out because tapping breaks the title; probe startup is about
25 s. That leaves roughly 350 s of observation, and at the measured 55 % of
real-time that is about 193 game-seconds.

Against the phase-1 route schedule: t=90 needs 164 s of wall-clock and t=120
needs 218 s, both already observed with no arrival. t=170 needs 309 s and is
reachable but only on a run that does not freeze. t=210 needs 382 s and t=240
needs 436 s, and neither fits in a turn at all.

So the tail of Stage 02's phase-1 schedule cannot be tested under this harness
however many runs are attempted. Recorded as a blocker rather than improvised
around. Two things would unblock it and neither is mine to decide: a safe way to
skip the title movie, which is more than half the budget and would roughly
double the window to about 297 game-seconds covering every entry, or a longer
shell timeout if the harness permits one.

This run is discarded: frozen at t=90 with 15 stalled samples and 3 losses,
correctly flagged and not interpreted. The freeze tally is now three clean of
five, lower than the three-of-four quoted last iteration.

What stands is unchanged: zero confirmed arrivals over 300 s of verified-live
flight, about 165 game-seconds, covering the t=90 and t=120 entries. That is the
strongest statement this harness can currently support.
2026-08-24 19:28:53 +00:00
Sylpheed RE agent
d4f6b7bf03 re: freezes are stochastic, not eliminated; calibration vectorised
Correcting the previous iteration's wording. It claimed on n=1 that disabling
the periodic rescan "removes the freeze". With more runs that is too strong: the
configuration is now clean at 210, 240 and 300 seconds and frozen at 60 on a
fourth run.

The tally across configurations: the heavy probe froze at 27, 45, 83, 183 and
255 seconds; the cheap probe with a 90 s rescan froze at 183; the cheap probe
with no rescan is clean in three runs of four past 200 s. That is a real and
large improvement in survival, but it is a change in probability rather than a
fix, and the documentation now says so.

The witness handled the frozen run correctly -- losses stop at t=45 and stalls
are flagged from t=60 onward, with no contradiction between them.

Separately, the candidate search was the last Python loop over eight million
words and most of the remaining startup cost. Replaced with two numpy vector
operations; 7250 candidates found and startup is no longer the bottleneck. Boot
now dominates: about 190 s of title movie plus 35 s to flight against a 595 s
turn cap leaves roughly 350 s of observation, so boot is the only remaining
lever if longer windows are needed.

Practical rule recorded: do not treat a single frozen run as evidence. Run,
check the witness, discard the frozen ones and keep the clean ones -- about
three in four are usable now, which is workable where it previously was not.
2026-08-24 19:13:06 +00:00
Sylpheed RE agent
14385ae170 re: clean run reproduces; startup was eating a third of the window
Second consecutive run with no stall flag on any sample, which confirms that
disabling the periodic rescan is what fixed the freezes. Nine losses, four
flicker increases all correctly rejected, and zero confirmed arrivals. The
trustworthy negative now extends to 240 s of verified-live flight, roughly 132
game-seconds, so nothing arrives past the route table's t=90 or t=120 entries
either.

The flicker rate is worth noting: about one spurious increase per minute. That
is the rate at which the old rule would have been manufacturing arrivals.

The arithmetic of the run exposed a waste. Boot finished at 249 s, the probe ran
240 s, and the turn's 595 s cap fired, leaving about 100 s unaccounted for
between them -- the witness calibration and the initial craft enumeration.

enumerate_craft was iterating every 4-byte word of 32 MB in Python, eight
million steps, to find fourteen fixed needles. Replaced with bytes.find() per
definition VA, which is the same search at C speed and is what the vtable scan
already did. Not yet run.

If that recovers most of the hundred seconds the observation window grows from
about 240 s to about 340 s, roughly 187 game-seconds, which would finally reach
the t=170 route entry -- the first of the schedule's later arrivals that no run
has yet been able to observe.
2026-08-24 18:31:51 +00:00
Sylpheed RE agent
d674fee457 re: the periodic rescan was the freeze driver; first trustworthy negative
Two changes this iteration, one failed and reverted, one that worked.

Trimming the witness calibration to eight spread 512 KB windows instead of the
full region found 17 candidates, none of them frame counters, and the witness
then reported 0 of 17 stalled on every sample of a run that recorded 13 losses.
That is a total contradiction, caught by the same internal check that exposed
the previous three witness failures. The frame-rate cluster is sparse and spread
sampling misses it. Reverted: two 32 MB reads once at startup is simply the
price of a witness that works.

The recurring cost was the periodic rescan, a 32 MB read every 90 to 180
seconds, and it exists only to catch craft appearing from nowhere -- which the
roster work already established does not happen, since every participant is
allocated at mission load and an arrival is a state change rather than an
allocation. Disabled.

The result is the first fully clean probed run: 3875 candidates, 32 witnesses at
11/s, no stall flag on any sample from t=0 to t=210, and eight losses spread
across it. Previous probed runs froze at 27, 45, 83, 183 and 255 seconds. This
one ended on the turn timeout. One run, so not proven, but together with the
clean no-probe control it points at recurring heavy reads rather than at memory
reading as such.

That also produces the first arrival result that means what it says. Every
earlier one carried a caveat -- a stalled guest, an unvalidated witness, a probe
degrading what it measured. This one has a validated witness reporting no
stalls, a demonstrably live guest, and a clean end: zero confirmed arrivals over
210 s of verified-live Stage 02 flight, roughly 115 game-seconds, while the
player destroyed eight craft.

It does not settle the question. The route table's t = 170, 210 and 240 entries
remain out of reach in a single turn. But it does establish that nothing arrives
in the first ~115 game-seconds of phase 1 under those conditions, which none of
the previous fifteen runs could honestly claim.
2026-08-24 18:16:49 +00:00
Sylpheed RE agent
f4561600c5 re: the stall witness is validated; the freeze is now the blocker
Fourth attempt at the witness, and it passes the check the previous three
failed. Searching the whole 32 MB region rather than a 4 MB window gives 6500
candidates and 32 witnesses in a clean 21/s cluster matching the measured frame
rate, and the flags have zero contradictions with the loss data: the single loss
falls in an unflagged sample and every flagged sample has none. The guest
genuinely froze at about 45 s and the instrument says so.

That leaves the freeze itself as the dominant problem. Onset across runs is 27,
45, 83, 183 and 255 seconds, median around 83, so the usable window is one to
four minutes and unpredictable. This run produced exactly one loss and is
useless for the arrival question even though the instrument finally works.
Fifteen runs have now gone into that question without an answer, largely for
this reason.

Worth being honest about the probe: "cheap" was never that cheap and the witness
made it worse. It still reads 32 MB to enumerate at startup and another 32 MB
every 90 s to rescan, and the full-region calibration added two more 32 MB
reads. The 45 s freeze came immediately after that calibration. Calibration now
samples eight spread windows of 512 KB -- 4 MB total, still covering the whole
span -- instead of reading everything twice, and the rescan interval is doubled
to 180 s. Not yet run.

Also this iteration: the 42-record anomaly recurred, the discard rule correctly
refused to interpret the run, and a rescan-until-baseline retry is now in place
so the next occurrence shows whether the count climbs toward 116 or sits at 42.
2026-08-24 17:50:58 +00:00
Sylpheed RE agent
1afe1b7134 re: fix the bind failure; witness search window was too narrow
Two harness problems, one fixed and verified, one diagnosed.

Bind: three consecutive attempts failed and the session aborted rather than
flying an unattended craft, which is last iteration's guard working as intended.
The cause is that entities2 self finds the player by motion between two samples,
so a craft sitting still at mission start is invisible. The session now holds the
throttle for three seconds before each attempt. Verified: the next run bound
immediately and produced 11 losses with zero confirmed arrivals, making fourteen
runs without one. One increase, 10 to 11, was surfaced and correctly not counted.

Witness: the frame-rate-band selection is still unreliable. It found only five
candidates in band and reported 0 of 5 moved in samples where craft were being
destroyed, and a counter genuinely running at 24/s cannot fail to advance across
a 15 s sample -- it would gain about 360. They are bursty counters that moved
during the 3 s calibration and then stopped.

The cause is mine. When the witness was bolted into wave7_probe the candidate
search was narrowed to a 4 MB window, while timer_probe searched the whole 32 MB
region and found 286 candidates with a clean cluster near 17/s. The narrowing
was never justified, it was just cheaper to write. Now searches the full region
once at startup, which is a one-off cost rather than per sample, and warns
explicitly when fewer than eight witnesses are available so a weak vote is
visible instead of silently trusted. Not yet run.

Worth recording: this is the third attempt at this witness, and all three
failures share one pattern -- a shortcut in selecting the witness rather than in
the voting logic. Each was caught only because the flagged samples contradicted
the loss events in the same output. That internal contradiction is the real
check and should survive into any future version.
2026-08-24 17:30:03 +00:00
Sylpheed RE agent
2b98105ac0 re: multi-witness vote works, its threshold did not, and freezes persist
The graded output is the improvement. Instead of a single bit it reports how
many witnesses agree, and the sequence tells a coherent story: 11, 9, 7, 4, 1,
then 0 of 31, with the drop to zero at t=183 s coinciding exactly with the last
loss and 106 s of nothing after it. That is a real freeze, identified.

The threshold was wrong though. Flagging a stall at "fewer than half" marked the
entire run stalled, including samples in which craft were destroyed, so 11 of 31
advancing is a healthy guest rather than a stalled one. The cause is the cluster
choice: the modal rate was 93/s, far above the ~16.5/s frame rate timer_probe
measured, and those are subsystem counters that tick in bursts and sit idle in
most 15 s windows even while the game runs. Picking the modal cluster was
convenient rather than principled.

Fixed to prefer the cluster whose rate falls in the frame-rate band of 8-40/s,
falling back to modal only if none exists, and to flag a stall only when zero
witnesses advance, which is the signal the data actually supports. Not yet run.

The uncomfortable part: this run used the cheap probe and still froze, at about
183 s. The previous iteration's "0 stalled samples" came from the unreliable
single-word witness and cannot stand as validation. What the evidence supports
now is that the no-probe control ran 300 s clean, the heavy probe froze at 27 to
255 s, and the cheap probe froze at 183 s -- one run on each arm. Cheap sampling
plausibly helps but does not remove the freeze, and it is equally possible the
freeze is stochastic and the control was lucky. Recorded as unresolved rather
than resolved in the probe's favour.

Practical consequence: the usable window is roughly three minutes per run,
sometimes less, whether or not the probe is cheap. Experiments needing longer
have to survive a freeze or be redesigned around one.
2026-08-24 17:09:43 +00:00
Sylpheed RE agent
015fb7d21e re: persistence rule works; the stall witness gives false positives
Run 12, cheap probe with a bound pilot: 16 losses over 290 s and zero confirmed
arrivals, making twelve runs without one. The persistence rule earned its place
immediately -- an increase of 13 to 15 was surfaced and correctly not counted,
since it does not start from zero. Under the previous rule it would have been
invisible, and a similar flicker straddling zero was nearly written up last
iteration as the first arrival.

The stall witness, on the other hand, is unreliable. Thirteen samples were
flagged GUEST STALLED while recording losses in those same samples, and a frozen
guest cannot destroy craft, so they are false positives and the run was healthy.

The cause is the selection rule: it took the first word in a 4 MB window whose
rate fell in a plausible band, and plenty of counters advance intermittently
without saying anything about whether frames are being rendered. timer_probe had
already solved this properly -- 286 candidates, a rate histogram with a dominant
cluster near 17/s -- and that lesson was not carried over when the witness was
bolted onto the probe.

Now fixed to a majority vote: collect every candidate, keep the modal-rate
cluster, sample up to 32 of them, and report a stall only when fewer than half
advance. It also prints RUN UNVALIDATED when no witness is found, because an
earlier run printed "stalled samples=0" alongside "tick witness: NONE", and a
witness that does not exist cannot report zero stalls. Not yet run.

Consequence worth flagging: the "0 stalled samples" that validated the cheap
probe last iteration came from this same unreliable witness and should be
re-confirmed under the majority rule. The pilot-log speed analysis that
established the stalls in the first place is unaffected.

Also fixed: the entity bind now retries three times and aborts if it never
takes, instead of silently flying an unattended craft -- one run was wasted that
way this iteration, producing no kills and no information.
2026-08-24 16:54:52 +00:00
Sylpheed RE agent
ce235e153b re: cheap probe verified; its one "arrival" is flicker
First run of wave7_probe: zero stalled samples across the whole run, against
three consecutive heavy-probe runs that stalled at roughly 27, 83 and 255 s. The
fix works. The guest is also visibly healthier -- 19 losses against 8 in a
heavy-probe run of comparable length -- so starving the emulator had been
suppressing the very activity the probe existed to watch.

It also reported the first arrival of the whole line of work, and that arrival
does not survive inspection. A record went 1 -> 0 at t=229, 0 -> 2 at t=259 and
2 -> 0 at t=274. Two craft appearing and vanishing within fifteen seconds is not
a wave.

The same log contains the giveaway: at t=60 a record read 13 and at t=75 the
same record read 14, an increase, with nothing printed, because the probe only
surfaced decreases. The hull-based liveness read flickers, and a flicker that
straddles zero was indistinguishable from an arrival under the old rule. The
count therefore stands at zero confirmed arrivals in eleven runs.

Two changes, neither yet exercised: every increase is printed rather than only
those from zero, and an increase from zero counts only if it persists across two
consecutive samples, with a candidate that returns to zero discarded as flicker.

Recorded because it was close: under the old rule this run would have been
written up as "first arrival observed", which would have been the
strongest-looking result so far and wrong.
2026-08-24 16:28:59 +00:00
Sylpheed RE agent
f92d60483a re: confirmed — the probe was causing the guest stalls
The control needed no new decoding: hunting pilot, keep-out 1400, 300 s, and no
memory probe at all, judged only from the pilot's own log. It ran healthy the
whole way -- 211, 257, 250, 241 and 200 distinct speed values in successive 60 s
windows, with no decline in the last one -- against probed runs that stalled at
roughly 27, 83 and 255 s and ended with a single distinct speed across 400 lines.

So the instrument was degrading the thing it measured. AGENT.md warned that a
full scan competes with the emulator for every core under lavapipe, and the
probes grew heavier each iteration until wave6 was reading the entire 32 MB
entity heap plus about 300 extra preads every 12 seconds while the game
rendered.

The control is n=1. The contrast is stark and the mechanism was predicted in
advance, so this is recorded as confirmed enough to act on rather than proven; a
second clean control would settle it.

What it costs is worth stating plainly. Every "no arrival" result so far was
collected while the probe was running, under conditions the game was struggling
with and in some runs after it had frozen. That does not make them wrong -- the
elimination test at t=163 s sits in a window the witness later showed was live --
but none of them is as strong as written, and the arrival question deserves
re-running with cheap sampling before silence is read as evidence.

wave7_probe.py is the fix: one full enumeration, then polling only the hull word
at each known craft base, about 1.2 KB per sample instead of 32 MB, with a full
rescan every 90 s to catch anything genuinely new. Three or four heavy scans per
run instead of twenty-five. Implemented but not yet run, so the claim that it
stops the stalling is untested and the next run must report the witness first.
2026-08-24 16:13:04 +00:00
Sylpheed RE agent
120a8ee50c re: the stall witness works, and stalls may be caused by the probe
First run with the tick witness. It flagged a stall from t=27 s and every sample
after, and the pilot's own telemetry -- which the probe never reads -- agrees:
35 distinct speed values across the whole log and exactly 1 in the last 400
lines, against 236 in the first 400 of a healthy run. All variation is in the
first ~50 s. The witness is validated.

It earned its keep on that same run. Without it the output reads as "no arrivals
across 313 seconds with 300 craft resident" -- clean, quotable and completely
worthless, because the game was frozen for 90 % of it. Rule adopted: a run whose
witness reports a stall is discarded, and every write-up states the witness
result. Flat samples are not evidence unless the witness says the guest was
advancing.

Stalls are frequent and early. The last three long runs stalled at roughly 255 s,
83 s (after the player died) and 27 s. That makes long observation windows
unreliable, and long windows are exactly what the arrival question needs.

Leading suspect is the probe itself, and it is recorded because it is
uncomfortable rather than despite it. AGENT.md warns that a full memory scan
competes with the emulator for every core under lavapipe, and these probes have
grown heavier each iteration: wave6 now reads the entire 32 MB entity heap plus
about 300 extra preads every 12 seconds while the game renders. If that is the
cause, the instrument has been degrading the thing it measures and the earlier
"no arrival" results were collected under conditions the game was struggling
with.

Next is a control that needs no new decoding: run the hunting pilot for 300 s
with no probe at all and judge from the pilot log alone. If it does not stall,
sampling has to get much cheaper -- narrow the scan to the roster region, sample
less often, or reread only the craft bases already located instead of rescanning
the heap.

The multi-squadron kill-threshold test did not run: the guest froze before
anything was destroyed, so there were no losses to threshold.
2026-08-24 15:57:09 +00:00
Sylpheed RE agent
932691b80d re: eliminate an enemy squadron — no wave follows; add a stall witness
SYLPH_KEEPOUT makes the hunt keep-out a knob instead of a hard-coded 600. At
1400 the pilot both kills and survives: hull 1500 and escorted asset 100 % for
the whole run, ENGAGE throughout, eight loss events against seven in the run
where it died at 83 s.

That let the elimination test actually run. An e007 Turret squadron was ground
18 -> 10 -> 8 -> 4 -> 2 -> 0, reaching zero at t=163 s with deployed falling 41
to 40 -- an enemy squadron destroyed outright for the first time in nine runs.

No arrival followed. ARRIVALS=0 at every sample including all those after t=163.
The hypothesis that a wave is released when a squadron is wiped out rather than
merely damaged does not survive its first test. Two further losses occurred at
t=176 and t=202, so the mission was demonstrably still live and still processing
kills; it simply produced no arrival. This refutes elimination-of-one-squadron
as the trigger. It does not refute event-gating generally -- a threshold across
several squadrons, an objective completion, or a specific squadron could all
still be the gate.

The valid window is smaller than the log suggests, about 90 s after the
elimination rather than 143 s, because the guest stalled around t=255 s. The
pilot's telemetry gives it away: the last 400 log lines contain one distinct
speed value against 236 in the first 400, with no timestamp gaps -- the process
kept logging while the game stopped advancing. Nothing in the probe output
distinguished a stall from a quiet mission, and the same ambiguity affects the
trailing flat samples of earlier runs.

wave6_probe now locates a counter advancing at frame rate, samples it each tick
and prints GUEST STALLED when it fails to advance, so future runs validate
themselves. Implemented but not yet exercised in a run.
2026-08-24 15:41:21 +00:00
Sylpheed RE agent
0578e8e72f re: player death bounds every run; fix a harness bug that shortened the windows
Correction first. The sed used to derive each session script from the last
stripped the probe's arguments, so wave5, census and wave6 sessions invoked
their probes with no arguments and every derived probe has been running on its
own defaults. The previous iteration's claim that the run was "cut at 240 s by
the turn timeout, not the planned 330 s" is therefore wrong: the probe used its
default of 240. The pilot got the requested duration while the probe watched for
a different one, and the numbers were plausible enough that it went unnoticed.
No earlier conclusion is invalidated -- the windows were real, just shorter than
intended and misattributed. All three sessions now pass SECS and EVERY.

First n -> 0 ever observed: the player's own record went 2 -> 0 at t=83 s and
deployed fell 41 to 40. The signal does register elimination, not just damage.
No arrival followed, which is weak evidence against the squadron-elimination
trigger since the record eliminated was the player rather than an enemy
squadron. Two other turret records dropped from 18 in the same sample; noted
without interpretation.

The important finding is what came after. For the remaining 220 seconds the
mission was frozen -- exactly 288 craft, zero losses, zero arrivals, across 18
consecutive samples. So the usable observation window is not the probe duration
but however long the player survives. A 340 s probe that loses its pilot at 83 s
yields 83 s of evidence and 257 s of nothing, and several earlier "nothing over
240 s" results may have been much shorter in practice than they look.

That also explains why pilot.py was written to survive rather than to shoot. The
SYLPH_HUNT mode added two iterations ago drops TURRET_KEEPOUT from 2500 to 600,
buying kills at the cost of exactly the survival the run depends on.

The elimination test itself did not run: the squadron reached 14, not 0, before
the pilot died. What is needed is a pilot that kills and survives -- hunt turrets
but keep the evade and retire behaviour, or a keep-out between the two extremes.
That is tuning, not discovery.
2026-08-24 15:25:17 +00:00
Sylpheed RE agent
83919ea0ae re: long run — a squadron ground 18 to 2, still no arrival
The per-record instrument works and resolves individual squadrons. Over 240 s
with the hunting pilot, seven loss events all landed on the same record,
tracking one e007 Turret squadron from 18 craft down to 2 while deployed held at
41 and the global count fell 300 to 284. Losses come in steps of 2 after an
opening drop of 4, which is unexplained and recorded rather than smoothed over.

This weakens the frame-rate explanation from the previous iteration. At ~16.5
fps against a 30 Hz tick, game time runs at about 55 % of wall-clock, so route
entries t = 90 and t = 120 land near 163 s and 218 s wall. The run reached 234 s
wall, roughly 129 game-seconds, passing both, and no arrival occurred at either.
"The runs were too short" no longer covers t = 90 and t = 120, though it still
covers 170, 210 and 240 -- the run was cut at 240 s by the turn timeout rather
than the planned 330 s, so t = 170 was never reached.

It also sharpens the event-gated model into something testable. The squadron
ended at 2, not 0, and no squadron has been eliminated in any run so far. If the
trigger is a squadron being wiped out rather than merely damaged, every
observation to date is explained: seven kills produced no arrival because they
never finished anything off.

Next is the cheapest decisive experiment yet available: run 60-90 s longer so
that squadron reaches zero and watch for a 0 -> n in the following samples. The
~210 s title movie at boot remains the binding constraint, leaving about 350 s of
observation per turn.
2026-08-24 15:09:33 +00:00
Sylpheed RE agent
cc5631a2f3 re: the mission clock is running — refute the stopped-clock explanation
The prime suspect from the previous iteration was that the phase scheduler
simply is not running, which would have made every arrival result meaningless.
It is refuted.

timer_probe.py takes three equally-spaced snapshots of the 32 MB game heap with
no pilot -- exactly the condition where nothing had been observed to change --
and keeps words whose two successive deltas are both positive and agree within
12 %, so linear rather than merely noisy. 286 words qualify, with a large
cluster advancing in lockstep at 16.5 per second.

That rate is not a coincidence: the existing performance notes put Canary
playback on this box at roughly 14-19 fps, and the dominant counter sits inside
that band, so these read as per-frame counters.

Which gives a mundane explanation for six arrival-free runs. If the scheduler is
frame-driven and the title targets 30 Hz, game time advances at about 55 % of
wall-clock here, so the 168, 190 and 240 second runs covered roughly 92, 105 and
132 seconds of game time. Route_S02 schedules phase-1 arrivals at 90, 120, 170,
210 and 240, so the longest run passed the first two and came nowhere near the
last three. No model has to be wrong for the observations to be empty.

Kept at 🟡, not promoted: two links are assumed rather than measured -- that
these counters are frame counters, and that the game's tick is 30 Hz. The values
also do not fit a naive frame count, since the cluster read 14193 about 255 s
after emulator start, which matches neither 16.5 nor 30 per second, so their
origin is genuinely unknown and no counter is claimed to be the mission clock.

Next is one long run, ~350 s of probe, watching for a 0 -> n transition near
t = 163 s and 218 s wall, where the t = 90 and t = 120 route entries land at 55 %
speed. If 350 s still yields nothing, the frame-rate explanation is itself
refuted and the event-gated model returns as the front-runner. The ~210 s title
movie at boot is the binding constraint on how much game time one turn can
observe.
2026-08-24 14:54:57 +00:00
Sylpheed RE agent
171f682b50 re: six runs with no arrival, and an accidental no-pilot control
wave6_probe refuses to interpret a run whose roster count is not the reproduced
baseline of 116, per the discard rule. This run passed and its first sample is
identical to the earlier link run: 116 records, 300 craft, 41 deployed, strength
histogram 2x24, 4x1, 8x4, 18x12. The deployment is deterministic at mission
start.

The pilot failed to bind, which accidentally supplied the control condition the
kill-versus-no-kill experiment needed. With nobody flying, not one craft was
destroyed in four minutes -- exactly 300 across all 22 samples -- against 16-20
losses in each piloted run. So losses are attributable to the player being in
the fight, and NPC crossfire destroys nothing by itself. That was an open
question two iterations ago.

Still no arrival. Zero 0 -> n transitions in either condition, across roughly
fifteen minutes of cumulative Stage 02 flight and windows up to 240 s. The 75
records holding no craft at mission start still hold none at the end. Against a
route table scheduling phase-1 arrivals at t = 90/120/170/210/240, that is now a
strong negative rather than a null result.

Three readings survive: the timetable's t is not seconds (at 30 Hz the whole
phase-1 schedule finishes inside 8 s, before any first sample); arrivals are
event-gated and no run supplied the trigger; or the mission is not advancing its
phase clock at all.

The third has never been checked and is the cheapest to eliminate, so it is now
the prime suspect. Nothing in six runs has confirmed that mission time advances:
the craft count freezes without a pilot, REMAINING OB has never read as a
counter, and no clock has been located. Every "no arrival" observation is
consistent with a scheduler that simply is not running, in which case the
arrival results so far are measuring a stopped clock.
2026-08-24 14:42:36 +00:00
Sylpheed RE agent
1280df7ce0 re: the 42-vs-116 outlier does not reproduce; losses are observable
Two further runs sampled the counts from the instant flight is detected. Both
give 116 roster records in the very first sample, flat thereafter, with 14 unit
definitions throughout. That is not a load race: a race would climb toward 116
from below, and it never does.

The leading explanation was arithmetic rather than behavioural -- census_probe
counts raw aligned vtable hits while wave5_probe counted distinct primary_va
values, which are different quantities if an offset can alias to a VA another
offset also claims. Measuring both in the same run refutes it: 116 raw hits and
116 distinct VAs in every sample.

So the outlier stands unexplained. Four runs now read 116/300, 116/300, 116/296
and one 42/170, with save drift, freezing, load race and VA aliasing each
individually refuted. Recorded rather than dropped, because a one-in-four
anomaly in the measurement apparatus is what later turns into a wrong
conclusion. Practical rule adopted: discard a run that disagrees with 116 rather
than interpreting it, and reproduce any finding in at least two runs.

Useful new fact: craft declines steadily and monotonically in both runs, 300 to
288 and 296 to 280, roughly 16-20 losses over 168 s under the hunting pilot.
Whether the player or NPC crossfire is responsible is not determined here, but
destruction is visible in this signal, which is half of what the kill-versus-
no-kill experiment needs.

Still not observed in any run: an arrival.
2026-08-24 14:27:31 +00:00
Sylpheed RE agent
f137d3eae3 re: per-record strength measured, but the run does not reproduce
The link from the previous iteration gives a per-record live strength, and the
measurement is internally sound: 11 records at 2, 2 at 8 and 11 at 12 sums to
exactly the 170 craft counted, with 24 of 42 records deployed.

It does not reproduce. The previous run saw 116 roster records and 300 craft
with strengths 2/4/8/18; this one sees 42 and 170 with 2/8/12 -- same disc, same
save slot, same launch script, same stage.

Two explanations were checked and both fail. The save has not drifted:
game01/savedata is unmodified since 2026-08-23 and only the profile .gpd files
were written today. The guest was not frozen: the pilot's telemetry over the
same 190 s shows speed varying across dozens of values and a live engagement
with an e007 Turret at 259-680 m.

So the discrepancy is real and unexplained, and until it is understood
per-record counts cannot be used as a time series. Recorded as the blocker
rather than worked around. The same within-run-only discipline already noted for
the global craft count now applies one level down.

Also recorded: zero arrivals and zero losses across 190 s. That is weak evidence
against clock-driven arrivals at t = 90/120/170 seconds, since a 0 -> n
transition should have appeared and none did. It is no test of the event-gated
model at all, because nothing was killed -- fc=0, no record lost strength, and
the player's hull never moved off 1500. The pilot closes to 259 m and misses, so
the gap is accuracy rather than engagement.
2026-08-24 14:04:24 +00:00
Sylpheed RE agent
46e511e791 re: find the roster-to-craft link — a pointer at roster_base + 0x08
The previous iteration was right that neither structure points at the other's
base, and wrong to stop there. Craft point eight bytes into a roster record.

Resolving the 5101 "pointers into the roster range" by distance to the nearest
roster base gives a distribution that is neither flat nor diffuse: +0x08 appears
exactly 300 times, matching the 300 live craft, while every other delta appears
at most twice and 4780 of 5104 land more than 0x4000 from any base.

A count matching a count is the coincidence that has already misled this corpus
three times, so the link was tested on something a coincidence cannot survive.
Each side's unit type is resolved by a different chain -- the craft's from its
definition pointer at +0x130, the record's from its +0x04 name chain -- and they
must agree. They do, on all 300, with zero disagreements, and the fan-out
histogram is [(1, 300)]: every craft has exactly one such pointer, none has zero
or two.

The fan-in is the useful half. Only 41 of the 116 roster records are referenced
at all; the other 75 have no craft. Craft per referenced record is discrete:
24 records with 2, one with 4, four with 8, and twelve with 18 -- summing to
exactly 300. The twelve holding 18 are all e007 Turret.

This is the observable four iterations have been hunting. A member with zero
craft is not deployed; a member's craft count is its live strength; so an
arrival is craft appearing for a record that had none and a kill is that count
falling. Both are attributable to a named squadron, because the record is a
UnitGroup member and UnitGroup names the squadron that Route_S02.tbl ties to a
phase and an arrival time. The chain from a memory observation to a static route
entry is complete.

Not claimed: that the 75 undeployed members are waiting on phases 2 and 3. That
is a plausible reading, but the run only shows they have no craft.
2026-08-24 13:48:39 +00:00
Sylpheed RE agent
ad18465519 re: record the pointer-link refutation in the backlog index
The previous commit landed the doc and the probe but its BACKLOG edit asserted
on stale text and did not apply, so the index still described the expansion rule
as merely open rather than recording which candidate had just been eliminated.
2026-08-24 13:31:28 +00:00
Sylpheed RE agent
176af18381 re: refute a direct pointer link between roster records and live craft
Fourth failed attempt to explain how 116 roster members become ~300 live craft,
recorded so it is not retried. Count (116), the member field n (387) and
formation slots (630) were rejected earlier; this rejects a head pointer.

link_probe.py scans the first 0x400 bytes of every object for an address of the
other kind. Roster record to craft base: 0 of 116. Craft to roster base: 0 of
300. Both directions empty.

The scan also measured the address spaces, which is the useful part. The two
structures live in different regions about 19 MB apart -- roster records at
0xbc372c00..0xbc9bc720, craft at 0xbdb2fd80..0xbdcd1d80 -- and the 14 unit
definitions match Stage 02's 14 distinct unit types exactly. The craft count is
not fixed: three runs at comparable mission times gave 296, 298 and 300, so the
population must be compared within a run and never across runs.

One thread is left open but explicitly not counted as evidence. Craft objects
hold 5101 words pointing somewhere into the roster VA range, which sounds like a
lot until you notice that range spans 6.5 MB and holds many allocations besides
the 116 records. The test that would settle it is the distance from each such
pointer to the nearest roster-record base: a spike at a single delta means a
link at base+X, a flat distribution kills the thread. Not run yet.

The association may not be a pointer at all -- an index, a hash, or a third
object such as a squadron instance would all look like this. Since an arrival is
already known to be a state change rather than an allocation, finding that
mediating structure matters more than finding a flag: it is what would let an
observed change be attributed to a named squadron and hence to a route.
2026-08-24 13:31:13 +00:00
Sylpheed RE agent
a7cc4d6408 re: sites are entities 1:1 — withdraw "n is craft-per-member"
The suspected confound turned out not to exist. Gaps between consecutive
same-unit definition-pointer sites are all >= 0x1000, with 274 of them exactly
0x1000, so entities are page-spaced and there are no near-adjacent pairs to
merge. Clustering at any threshold below 0x1000 gives ratio 1.00 for every unit
type, and hull is plausible on 298 of 298 clustered bases at delta 0x130. The
player shows two objects because there are two, not because one holds two
pointers.

That removes the excuse the previous iteration had used to keep the reading
alive, and the reading does not survive: sum(n) fits the turret row well (216
against 214, with kills already recorded), but DeltaSaber_T, Player and
Acropolis all come out at exactly twice their sum(n). An undershoot can be
blamed on phases 2-3 not having started; an overshoot cannot. n goes back to 
and the previous 🟡 is withdrawn. All the turret row establishes is that a
roster member expands into many craft, not that n is the factor.

Formation slot count was tested as the alternative and rejected outright: 630
turret slots against 214 live.

Side result worth keeping: a FormationSet record's FrameCount is its slot count,
and the name suffix usually agrees -- Turret07_30 -> 30, ArrowHead03_64 -> 64,
4_Bird -> 4 -- with one exception, AttackerS03_12 having 14 slots, so the suffix
is a label and not a guarantee.

Also recorded: the 298 live entities are not the 116 roster records. Both
structures exist at once, and the rule mapping one onto the other is the real
open question.

Probe caveat noted in the doc: entities2.moving() found no movers this run, so
the delta spectrum was empty and the clustering threshold was a fallback rather
than a measurement. It does not change the conclusion, since every gap exceeds
any threshold below 0x1000.
2026-08-24 13:13:43 +00:00
Sylpheed RE agent
356c4e70f8 re: motion-independent liveness probe; n looks like craft-per-member
entities2.moving() finds entities by displacement between two samples, so
anything stationary is invisible -- the entire explanation for the +/-10 swing
that made the previous run's count useless. liveness_probe.py enumerates by
definition pointer over the entity heap instead, moving or not, and reads hull
as f32 at position+0x154. The series is monotone rather than oscillating:
298 -> 280 over 164 s, with the decline matching the 18 disappearance events
exactly.

The hunting pilot does kill: one hull crossing caught directly, an
e010_ADAN_Attacker_S at t=57 s. The previous run's worry that SYLPH_HUNT shoots
but never destroys anything is settled.

Recorded as a non-result so the next run does not misread it: zero births in
164 s does NOT favour either wave model. The roster finding already established
that every participant is allocated at mission load, so neither a clock nor an
event model would produce an allocation. An arrival must be a state change on an
existing entity.

The mystery member field n now has a candidate meaning: the number of craft a
roster member spawns. Static sum(n) for turrets is 216 against 214 sites found,
with the count already falling before the first sample, where Count alone
predicts 21 -- off by an order of magnitude.

Not promoted, and the reason is a confound in my own measurement rather than the
data: the probe counts definition-pointer SITES, not entities. The player is one
member and yields two sites, and DeltaSaber_T yields exactly double its sum(n),
so some entity types hold several pointers to their definition. Until sites are
collapsed into distinct entities the turret match could be a coincidence between
a x1 multiplicity and a x1 ratio. The capital-ship rows undershoot for a
separate and expected reason: phases 2 and 3 have not started.
2026-08-24 13:01:19 +00:00
Sylpheed RE agent
08352d09ac re: add a hunting pilot; withdraw the "10 of 116 records are dynamic" result
The user asked for an actively hunting pilot, since a player who kills nothing
cannot trigger an event-gated wave and both previous runs used the survival
pilot.

pilot.py gains SYLPH_HUNT=1. The substantive change is which contacts ENGAGE may
shoot: it previously skipped every "hard" target -- "turrets and hulls are not
the objective" -- and stood off 2500 units from turrets, on the assumption that
an e007 Turret is an AA mount on a capital ship. It is a craft, one of the main
enemy types of the first six missions, and at 100 HP the cheapest kill on the
field. Under SYLPH_HUNT it is a target and the keep-out drops to 600. The run
confirms the pilot engages: steady ENGAGE, fire=1, committed to an
e010_ADAN_Attacker_S at ~2.2 km, hull and escorted asset untouched over 160 s.

Withdrawn: "only 10 of 116 records ever changed a byte in 170 s". This run
measured 41-56 records changing in every 10 s tick. The old figure does not
reproduce.

I cannot say why, because I changed two variables at once -- the record bound
(fixed 0x200 to bounded-by-next-record) and the pilot (survival to hunting).
Either explains it. That is a design error, and the honest outcome is a
retraction without a replacement explanation rather than a story that fits.
The conclusion it had supported is unaffected: the roster identity now rests on
the exact 10-of-10 unit-composition match measured independently.

Still open, and explicitly not concluded: the pilot's own entity scan shows ADAN
drifting 147 -> 129 -> 142, and the late rise has the shape of an arrival, but
the sample-to-sample swing is +/-10, the same size as the effect. AGENT.md warns
that polling faster than the guest updates manufactures a curve out of noise, so
no wave conclusion is drawn. The run probably did not kill anything either
(fc=0, asset at 100%), so it does not test the event-gated model. A stable
per-record liveness field and a working kill counter are both needed first;
REMAINING OB at 0xbdb59668 still does not read as a counter.
2026-08-24 12:45:43 +00:00
Sylpheed RE agent
2030324419 re: fix record labelling and measure the stride; confirm 116 records == roster
Both defects from the previous iteration are fixed by measuring instead of
assuming, and the fix immediately promotes a 🟡 result to .

Labelling: the previous probe assumed object+0x04 -> name_record+0x10 -> char*
and resolved 0 of 116. wave3_probe.py searches for the chain per record instead,
the way unit_discover.py does, and resolves 116 of 116 -- every one through the
pointer at +0x04 with the string at delta 0x00, not 0x10. The 0x10 belongs to
the definition object (vtable 0x820af844); the spawned-entity record
(0x820af030) uses 0x00. Carrying one over to the other cost the last run.

Stride: measured, not assumed. Gaps between consecutive records are min 32,
median 800, with common values 800, 640, 608, 576, 416 and 32. There is no fixed
record size, so the old RECLEN=0x200 window truncated large records and overran
small ones -- which is why its busiest fields were the last words of the window.
Future diffs must bound each record by the next record's address.

With labels available, the "116 records == 116 roster members" claim was tested
properly and is promoted from 🟡 to . The multiset of unit types matches the
static roster exactly: Turret 21/21, e106 Destroyer 19/19, f106 Destroyer 14/14,
f105 Cruiser 11/11, ASFrigate 9/9, ISCMissile 9/9, Attacker_S 9/9, e105 Cruiser
7/7, DeltaSaber_T 7/7, ArrowHead 6/6 -- 10 of 10 exact. A coincidental total is
possible; a coincidental distribution over ten unit types is not. The game
allocates one record per roster member at mission load.

Not settled: REMAINING OB at 0xbdb59668 held 95748078 unchanged all run. That
address is known to be run-dependent, and this was one of the misses, so the run
cannot say whether the pilot killed anything. Re-hunting it is a precondition
for the kill-versus-no-kill test, not an optional extra.
2026-08-24 12:35:02 +00:00
Sylpheed RE agent
fa9ff3b5c2 re: record the event-gated wave model as a competing hypothesis
The user proposed that a wave may be released by an event -- enemies destroyed,
an objective completed -- rather than at a fixed time, with the route's
first-keyframe time being a delay after the trigger. It fits the evidence better
than the clock model I had been assuming, and it explains a result I had left as
an open puzzle.

Both probe runs used the survival pilot, whose job is to stay alive rather than
shoot, and both saw no arrivals at all. A clock model says arrivals at
t = 90/120/170 happen regardless of what the player does. An event model says a
player who kills nothing triggers nothing, which is what was measured.
DisableInterval also gains a plausible referent: a pure timetable in the route
table would not need a per-squadron flag to switch a gating interval off.

Supporting evidence, cross-tabulated across all stages: no squadron with
DisableInterval=Yes has a delayed route (55 of 55 at t=0), while all 68 delayed
routes belong to squadrons with DisableInterval=No. If independent, about 3.7
such routes would be expected; P(zero) is roughly 0.02.

Kept at 🟡 and explicitly not promoted. A p ~ 0.02 association on a single
derived statistic is the same shape of near-fit as the "109 of 111" squadron
name base that already misled this corpus once. Name matching itself is sound --
for Stage 02, 108 of 111 squadrons have routes.

Records the test that separates the models: the same stage flown twice, killing
nothing versus killing aggressively, with REMAINING OB at 0xbdb59668 as the
cheap precondition check since it needs no new decoding. That test still depends
on the record-to-squadron labelling defect noted in the previous commit.
2026-08-24 12:25:54 +00:00
Sylpheed RE agent
e70ecaf382 re: withdraw the "turrets don't move" hedge — Turret is a craft, not a ship mount
The user pointed out that the "Turret" enemy of the first few missions is its
own craft type and must not be confused with the AA turrets mounted on capital
ships. The tables agree, and I should have checked them before hedging:

  UN_e007_ADAN_Turret      AI_ADAN_CraftSquadron_Rookie x70, _Veteran x19
  UN_e007_ADAN_Turret_EX4  AI_ADAN_CraftSquadron_Ace x4

No *_Turret unit ever uses AI_Structure, which is reserved for the things that
genuinely do not move -- asteroids, FloatingMine, SchlosBase, TransitPlatform,
SpaceFortress. AI_ADAN_CraftSquadron_* is Type=Squad in AIParams with the full
manoeuvre-weight block. The distribution matches "first few missions" too: 93
turret members across S01-S06, then nothing until S27.

This retracts the caveat on the 10-of-116 result. Removing it makes that
evidence stronger rather than weaker: the 106 inert records cannot be dismissed
as immobile scenery, because Stage 02's turrets are craft that should be
manoeuvring. The pre-allocated-roster reading survives a test it had been
allowed to dodge.

Lesson recorded in the doc: do not infer a unit's behaviour from its English
name -- check its AIID. The roster is self-describing and the answer was already
in the dump committed two iterations ago.
2026-08-24 12:24:20 +00:00
Sylpheed RE agent
8cc0b22967 re: diff inside the 116 entity records — no arrival flag found, two probe defects
Since the record count is flat, an arrival would have to flip a field inside a
record. This diffed all 116 records every 5 s for 170 s of Stage 02 flight.

The prediction under test -- groups of 3, 3, 3, 2, 1 records changing state at
t = 90, 120, 170, 210, 240 s -- did not appear. Changes are spread evenly across
ticks with no cluster at any predicted time and no field that transitions once
for exactly three records. Three explanations survive and this run cannot
separate them: the timetable's t is not seconds (at 30 Hz the whole phase-1
schedule finishes inside 8 s, before the first sample); arrival is not marked in
these records; or the mission was not in phase 1.

Supporting, not conclusive: only 10 of 116 records ever changed a byte, and 106
never changed at all. Live entities would be moving, so that fits the
pre-allocated roster reading -- but Stage 02's roster is turret-heavy and a
turret does not move while alive, so "inert" and "not yet arrived" are not
distinguishable here.

Two defects in my own probe, recorded rather than quietly fixed:

  - label() resolved to '?' for all 116 records, so nothing could be tied back
    to a squadron. That association is what would have made the result
    decisive -- "the three records that changed at t=90 are ADN110, ADN111,
    ADN112" is evidence; "records 18, 32, 99" is not. unit_discover.py already
    solves this and should be reused.
  - RECLEN=0x200 was assumed, not measured. The busiest fields are the last
    eight words of the window, which is what spilling into the next object
    looks like.

Method error kept: the first attempt deferred all analysis to the end and the
turn timeout killed it with 240 s of data in memory and nothing written. The
probe now streams transitions to disk and prints a partial ranking every 60 s.
With a 219 s cold-boot title movie, an end-only report has ~300 s of budget and
one bad estimate loses the whole run.
2026-08-24 12:21:48 +00:00
Sylpheed RE agent
27e0b6ebc1 re: find the arrival timetable; refute the entity-count proxy for waves
The routes' first-keyframe time is the arrival schedule. It is not always zero,
and grouping Stage 02's 120 route records by phase and first-frame time gives a
timetable: phase 1 releases 25 routes at t=0 then 3, 3, 3, 2, 1 at t = 90, 120,
170, 210, 240. Phase 2 has every one of its 37 routes at t=0, which is what
pins the meaning: t is measured from the start of its phase, not of the mission.
Entering a phase releases that phase's t=0 group and the rest follow on the
offsets.

That completes the data side of the question this line of work started from --
the schedule is data, split across UnitGroup (who) and Route (when, and the path
flown in), with no fixed enemy count anywhere.

Refuted: counting spawned-entity records does not reveal arrivals. One Stage 02
flight, 210 s sampled every 15 s, counting aligned 0x820af030 in an 8.3 MB span:
flat at 116 throughout, no step at 90, 120, 170, 210 or anywhere.

The reason looks more useful than the refutation. UnitGroup_S02's Count fields
sum to exactly 116 members, and there are exactly 116 records from the first
sample on, so the game most likely allocates one record per roster member at
mission load and a route arrival activates an existing record rather than
creating one. Kept at 🟡, not promoted: n=1, and the obvious refutation -- check
another stage's record count against its member sum (S01=42, S16=2, S29=95) --
needs a save for another stage, and only slot 01 / Stage 02 exists. Noted as the
blocker rather than worked around.

Not settled: whether the timetable's t is frames or seconds (at 30 Hz t=240 is
8 s; as seconds it is 4 min), and where an arrival is observable in memory. The
live flag is presumably a field inside those 116 records, which is a well-scoped
next probe now that the record set is bounded and located.

Operational note recorded: cold boot spent 204 s in the title movie, so a 300 s
probe overran the turn and the first attempt died with its output still in the
pipe. Log to a file rather than piping to tail.
2026-08-24 11:57:16 +00:00
Sylpheed RE agent
79253bc042 re: probe the runtime phase state — tables are resident, phase counter is not there
Static reading had gone as far as it could: the stage record splits a mission
into Phase_1..3 and every arrival route is phase-tagged, but nothing in the data
says what ends a phase. So this took it to the oracle -- one Stage 02 flight,
160 s under the survival pilot.

Confirmed, and this is the useful half: every string the static decode predicts
is present in live guest memory -- Phase_1, Phase_2, Route_ADN101_p1F,
SUBOBJ_010, AI_ADAN_CraftSquadron_Veteran, UnitGroup_S02.tbl. The game loads
exactly the tables the stage record names, under exactly the names we resolved,
and they can be located in RAM by content. That is the first dynamic
confirmation of the whole static table layer.

Refuted: the phase state is not adjacent to those strings. The probe reported
862 changed words around the anchors, which looks like a signal until you read
the values -- each word takes its predecessor's previous value and every value
points into the same region. It is one block shifted down four bytes, a single
memmove in a pointer list, occurring once between t=66s and t=89s. Diffing
around a string anchor was the cheap thing to try and it did not work.

Also recorded: a defect in my own probe. It scraped hit addresses with
0x([0-9a-f]{8}), but gmem.py find prints both the backing-file offset and the
guest VA, so half the anchors were file offsets read as addresses. Fixed to
match the va column only. It did not change the conclusion -- the anchor that
produced the shift was a real VA -- but a negative result from one of those
junk anchors would have been worthless.

Not settled: what advances a phase. Next handles are watching Route_ADN101_p1F
fire against entity positions, or working back from the SUBOBJ_*_Mes_L1 HUD
strings; the phase state is more likely near the known mutable REMAINING OB
counter than near the tables.
2026-08-24 11:31:20 +00:00
Sylpheed RE agent
cdb9e5a001 re: decode the stage table set — phases, routes, sub-objectives, AI parameters
Following the real stage record (not the _Test template dumped earlier) reaches
the whole mission-parameter layer, all of it in the same self-describing IDXD
container as the squadron roster.

The big one: a stage is divided into Phase_N blocks -- three for Stage 02, each
with its own map path, map mesh, asteroid definition and background -- and
Route_S<NN>.tbl holds the arrival paths, with records named

    Route_<squadron>_p<phase><kind>

tying a UnitGroup squadron id to a phase and to a time-stamped keyframed path of
(time, quat x4, pos x3). Route_ADN101_p1F is 3 frames at t = 0, 20, 30. The
identity len(fields) == FrameCount * 8 + 1 holds for 1449 of 1449 route records
across the 28 stages that have one, and 16/16 for FormationSet_S02.

Also decoded: SUBObjectiveSettings (per-objective bonus points by difficulty,
unlock item id, HUD strings) and AIParams (34 profiles, firing/guard/muster/
counter ranges plus 14 manoeuvre weights for Squad-type AI). The AIParams
numbers are exact original values from static RE and are portable as they are.

Adds tools/re-capture/stagetbl.py, which resolves a stage record by content and
can --follow every table it names, and commits two dumps as evidence.

Refuted and kept: the eight-value keyframe is the common case, not universal.
Formation_Fleet_01 has FrameCount=1 with 136 positional fields and
Formation_Fleet_02 has FrameCount=8 with 32, so a parser must not assume the
stride.

Corrects stage-definition-table.md, which was written from the _Test template
and is missing EnumerateSubobjective, EnumerateAIParams, BackGroundID and the
WingmanIconID fields the real record carries.

Not settled: what advances a phase -- the stage declares Phase_1..3 and routes
are phase-tagged, but nothing static says what ends one. That is a question for
the oracle, not for more static reading. Also open: the route-name kind letters
F/S/A/M/B, what activates a sub-objective, and StageMessageSet_S<NN>.tbl, which
does not resolve in GP_MAIN_GAME_E.pak.
2026-08-24 11:15:34 +00:00
Sylpheed RE agent
b39aabf5c3 re: decode the UnitGroup squadron roster, and correct yesterday's container reading
stage\UnitGroup_S<NN>.tbl is now fully readable. A squadron record is Count
member tuples -- (unit model, message set, n, identity/nameplate) -- followed by
five named fields: Count, SideID, AIID, FormationID, DisableInterval. The
property entries carry their own field names inline, so the tag hash never has
to be inverted.

Two independent self-checks validate it corpus-wide, 1160/1160 each across all
28 stage tables on the disc:

  - the length identity len(fields) == Count * 4 + 5, which is what pins the
    member-tuple width at 4 and the named-field count at 5;
  - agreement with the file's own Enumerate_Squadrons roster, which maps record
    key to squadron id independently of the per-record string offset.

Adds tools/re-capture/unitgroup.py (pure static, runs no emulator) with a
--all --check self-check mode, and commits the Stage 02 dump as evidence.

Corrections to the container layout written yesterday, all three wrong:

  - the 20-byte "(tag, 0, 0, count, size) section header" does not exist. It
    was the file's last 16-byte record followed by a plain npool word. The
    corrected layout is uniform across all 28 files; the old one failed on 9.
  - squadron ids do not use a separate string base. Every offset in the file is
    relative to the one string pool. The earlier "109 of 111" score was an
    artefact of the uniform 7-byte id stride and had silently shifted every
    name by three entries, which is why 17 TC*-named squadrons came out as
    SideID=ADAN. The roster record refuted it outright.
  - the roster is not always the last record; 9 stages put it elsewhere, so it
    is found by its missing Count.

Refuted and kept: the 4-byte record key is not the squadron id's name hash
(0 of 112).

Not settled: what the key encodes, the member tuple's third field n, and where
the arrival interval values live. DisableInterval is only a per-squadron flag
(Yes for 31 of 1160); the durations, triggers and arrival positions are not in
this file. Formation_*.tbl and EnumSquadron_Test.tbl are next.
2026-08-24 11:07:33 +00:00
Sylpheed RE agent
7482142abf re: find the mission wave data — stage records and the UnitGroup squadron roster
Sweeping the 811 unnamed IDXD objects in GP_MAIN_GAME_E.pak by schema turned up
schema 3c9ae32e: the per-stage definition record. 23 of them, one per stage,
each naming its background, resource package, collision set, message set,
nameplates, MapMesh/MapPath and EnumerateSquadron = UnitGroup_S<NN>.tbl.

That resolves two open threads at once:

  - MapPath = test.rgn hashes to 0x3506e972, a REGN object in MiscBin.pak, and
    MapMesh = test.col to 0x2cf7eb47, an MCOL object. REGN is a stage's map
    path data; MCOL is its collision mesh.

  - stage\UnitGroup_S02.tbl (0x019fd129, in all six language paks) is the
    Stage 02 squadron roster: 112 records, 112 squadron IDs, and a field
    vocabulary of FormationID / AIID / SideID / Count / DisableInterval, plus
    the unit model (UN_e010_ADAN_Attacker_S and friends, which match the XBG7
    mesh names we already decode), the MessageSet and the pilot character.

DisableInterval is the first direct evidence of the arrival-timing knob, which
is what the user's reframing predicted: the mission has a schedule with
parameters, not a fixed roster.

Container layout is only partly read. The 112x16 entry array was confirmed by
its boundary — keys increase for exactly 112 entries and break at 0x708, where
the next section header sits — not assumed. pak dump mislabels this file's
first key as its schema.

Refuted and recorded: the 16-byte record key is not the squadron ID's name
hash. name_hash("TCN001") = 0xd639f1a4 but the keys start 0x659aff47; 0 of 112
match.

Still open: the per-record payload fields, the meaning of the key, where the
interval values actually live, and the missing S17-S23 stage records.
2026-08-24 10:52:35 +00:00
Sylpheed RE agent
bd9e12c091 docs: update the REGN line - POF0 fixups in, two payload guesses out
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 10:40:01 +00:00
Sylpheed RE agent
2ab0f1647d formats: REGN carries a POF0 fixup table, and two payload readings are refuted
Every one of the 11 objects contains POF0 near the tail, at exactly
header[0x04] + 16 - an 11 of 11 relation. POF0 is a pointer-offset fixup table,
so the file is a serialised C++ object graph the loader patches on load, which
also explains why the offsets inside the cell index are absolute FILE offsets.
header[0x04] is therefore the size of the data area.

Two readings of the cell payload are recorded as refuted rather than dropped,
because both were tempting and both came from the smallest object alone: the f32
at record +0x1c is NOT a bounding-sphere radius (ratio to sqrt(3)*half-extent is
1.001 on that one object and 0.13-0.27 on the other ten), and a record's
(count, offset) pairs do NOT point at leaf arrays of count*4 bytes (0 of 11
objects clean). What survives is descriptive only: the payload is dominated by
float data, and the printable runs a string scan finds are float high-bytes
rather than text.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 10:39:47 +00:00
Sylpheed RE agent
051e165cc1 docs: rule DefTables and the XPR2 manifests out of the wave-scheduler search
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 10:32:02 +00:00
Sylpheed RE agent
0d3305f075 formats: REGN section 3 is the cell index, self-checked on all 11 objects
The fourth section is one 8-byte (count, offset) record per grid cell, followed
by the 32-byte records it points at. The check: the lowest offset any cell refers
to equals align16(offsets[3] + cells*8) on 11 of 11 objects - and the alignment
term is visible rather than assumed because the three 5x5x5 maps have 125*8 =
1000 bytes of index, which is not 16-aligned, so their payload starts 8 bytes
later than the six 10x10x10 maps'.

Two further invariants from the same sweep: every occupied cell has count exactly
1 (total items == occupied cells on all 11, so it is one record per cell rather
than a bucket list), and counts[4] equals occupied cells + 2 exactly on all 11 -
the +2 unexplained and recorded as such.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 10:31:42 +00:00
Sylpheed RE agent
3e9e765ce0 docs: open the wave-scheduler item - the counter is a schedule, not a roster
Raised by the user: a mission has no fixed enemy count, so what to look for is an
algorithm with parameters rather than a list of objectives - which also explains
why every per-entity flag search came back empty. Records the places searched so
far (MiscBin's REGN/MCOL, DefTables' three unnamed schemas), the false lead
(STAGES = Static.slb is a sound bank), and what is still unsearched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 10:23:18 +00:00
Sylpheed RE agent
3c1d1d99f9 formats: decode the REGN header - a per-map spatial grid, new to the corpus
hidden/MiscBin.pak was undocumented: 40 entries, none name-resolved, holding 11
REGN objects and 11 MCOL objects (plus 18 others). Eleven of each pairs them one
per map.

The REGN header decodes to a bbox, an extent, a cell size, grid dimensions, six
counts and four section offsets - and it self-checks twice over all 11 objects:
extent == cell x dims holds exactly 11 of 11, and counts[3] equals the cell count
(1000 on every 10x10x10 map, 125 on every 5x5x5). Two independent fields
reproducing the same grid is what makes it a decode rather than a guess.

Three map sizes exist: half-extent 250000 with 50 km cells, 50000 with 10 km
cells, and 25000 with 10 km cells on a 5x5x5 grid.

Written without overclaiming against the question that prompted it: a mission's
enemy count rises and falls, so a scheduler with parameters exists somewhere, and
a per-map cell grid is the kind of structure such a thing is indexed by - but the
four data sections are unread and nothing here shows spawn parameters.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 10:22:54 +00:00
Sylpheed RE agent
6a8c366a11 formats: correct the glyph class - a SECONDARY vtable of adjustor thunks
The first write-up called 0x820B2A64's 32 entries "methods" and the three xrefs
to it "construction sites". The DB's disassembly refutes both: every slot points
at a 16-byte adjustor thunk - lwz r11,-4(r3); subf r3,r11,r3; b <method> - which
is the PPC/MSVC thunk for a SECONDARY base subobject. So this is a secondary
vtable and the class uses multiple inheritance, and sub_823C3148, listed as a
constructor because it references the table, is actually the branch target of one
of the table's own thunks.

Following the 32 branch targets instead: 32 distinct addresses, 18 of them known
function starts, in two clusters (0x823c3xxx beside the thunks and 0x823e3xxx),
and NONE of them appears in any of the 1150 vtables the DB classified - so
nothing places this class in a named hierarchy, the disc's RTTI having no class
names. The two-cluster split is noted as a reading rather than a measurement.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 10:16:12 +00:00
Sylpheed RE agent
108eedf124 docs: record the HUD glyph quad decode in the backlog
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 10:07:57 +00:00
Sylpheed RE agent
00a06abc77 formats: decode the HUD glyph quad behind the objective counter
Following the pointers that move with REMAINING OB lands on four identical
objects whose first word is 0x820B2A64. Read live: that address holds 32
consecutive code pointers in 0x823c43b0..0x823c45a0 terminated by 0xfffffffc - a
vtable - and sylpheed.db has three ref xrefs to it from sub_823C3060,
sub_823C3148 and sub_823C31E0, i.e. three construction sites, the same pattern
the splash item vtable shows. No name: the disc's RTTI carries none.

The instance is a textured quad: a pixel size at +0x1c/+0x20 (34 x 42 for a
digit) and four vertices at +0x30/+0x48/+0x60/+0x78, each a colour and a UV pair.
The four UVs measured are the corners of an axis-aligned rectangle in order, all
0xffffffff.

The atlas size is a check rather than a guess: the UV rectangle is 0.0265 x
0.0547, and multiplying by 1280 x 768 gives 33.9 x 42.0 against the 34.0 x 42.0
stored in the object's own size fields - two independent fields agreeing to a
rounding step.

Also recorded: these objects churn. Re-reading an old pointer target a minute
later returned unrelated data, so this is a snapshot of a live pool.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 10:07:43 +00:00
Sylpheed RE agent
6636419e60 tools: warn that the Stop hook kills the emulator at every turn boundary
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 09:49:39 +00:00
Sylpheed RE agent
2fe76bd523 docs: close the EMULATOR GONE item - it was the Stop hook, and note the one-turn rule
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 09:49:20 +00:00
Sylpheed RE agent
ac17398515 formats+tools: the counter's neighbours are its own rendered digits
With the per-entity searches refuted at word and bit level, the question became
which object owns the counter. Sampling +-0x200 around it across a 4->8
transition: the control interval moved 0 of 256 words, and the step moved nine -
the counter plus four words holding ASCII '4' -> '8' NUL-padded, and four
pointers into 0xbcad2xxx that swap with them. Read live at HUD 008, all four
character slots hold '8'.

So the neighbourhood is the HUD's rendered text for this counter, which reframes
the address: it is the HUD widget's value rather than "the mission's own
objective counter" as this file called it.

Recorded against that, because it is already measured: there is no separate
mission-side copy moving on the same step. ob_hunt scans all of guest memory and
requires a match across two transitions, and it left exactly one address.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 09:48:57 +00:00
Sylpheed RE agent
01468b804a docs: the mystery SIGKILL was this project's own Stop hook
/work/.claude/settings.json defines a Stop hook that kill -9s every xenia_canary
at the end of each agent turn and prints "Stop hook killed N stale xenia
process(es)". So every run that "died mysteriously" died at a turn boundary,
which is why the timings looked random from inside the run (810 s, 54 s, 486 s)
and why nothing in the guest, the cgroup or the host explained them.

What survives is the measurement and not the story: the memory readings were real
and did refute memory pressure - cgroup at 6.5 of 7.0 GB, host 12 GB free at the
moment of a kill - but they were refuting a cause that was never in play.

The method lesson is recorded because it cost three iterations: when a process
dies at a SESSION boundary, check the harness before instrumenting the guest. A
failure mode was documented, a hypothesis raised, a per-poll memory sampler
written and committed, and host and cgroup counters read - all downstream of the
assumption that the kill came from outside the agent. The hook's own message had
been printed after every turn.

The operational rule that follows: an emulator experiment must COMPLETE INSIDE A
SINGLE TURN. Nothing survives the boundary, so runs cannot be left for the next
tick, and experiments whose evidence arrives in the first minutes are the ones
that work - which is exactly why the two-pass bit-level test succeeded where the
long freeze-watches did not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 09:48:23 +00:00
Sylpheed RE agent
bd69659a7d docs: close the bit-level flag question in the backlog
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 09:32:28 +00:00
Sylpheed RE agent
5a2faf09e5 docs: the bit-level OB flag is refuted - two transitions, no common survivor
The short-run strategy paid off: both counter steps happen in the first minutes
of flight, so one run yielded two complete passes before anything froze or was
killed. Pass 1 scanned at counter 4 and filtered on 4->8, leaving 29 of 710
candidates; pass 2 scanned at 8 and filtered on 8->12, leaving 2 of 197. The
intersection of the two survivor sets is EMPTY.

So no per-entity bit, in either polarity, anywhere in -0x400..+0xC00 around the
position triple, has a population that tracks REMAINING OB. With the word-level
refutation that closes the "objectives are marked in the entity object" family as
far as this method reaches.

Also recorded: 16 of pass 1's 29 survivors are the SAME word, pos+0x0250, with
sixteen different bits - the signature of a shared value rather than a flag, and
the same offset the word-level sweep found shared by 12 attackers. And a caveat
kept rather than smoothed: pass 2's B-sample HUD read 8 while RAM read 12.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 09:31:46 +00:00
Sylpheed RE agent
c90383dd07 docs: a third SIGKILL, traced live, refutes the memory explanation
Run 9 died the same way at t=486s, and the watcher's sampler had been running
throughout: at the kill the cgroup held 6.5 GB of its 7.0 GB limit, the host had
12 GB free, and oom_kill was 0. So the memory-pressure suspicion recorded
earlier is refuted - three kills, no OOM record on any, and the one with a full
trace shows headroom on both the cgroup and the host.

Also checked and clean: no stray process of mine was running, and the kills do
not fall on a clock (08:31, 09:00, 09:14 - gaps of ~29 and ~14 minutes).

Recorded as a container-level blocker rather than chased further, because nothing
in here can attribute a SIGKILL to its sender: no dmesg, no audit, and
ptrace_scope=1 limits strace to my own descendants. The practical response is to
stop designing experiments that need a long run - the counter transitions the OB
work needs all happen in the first minutes of flight.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 09:19:46 +00:00
Sylpheed RE agent
1b2bd3b22b docs+tools: the silent death is a SIGKILL from outside, and it is still not OOM
Run 8 died 54 s into its boot and this time bash named it: "176880 Killed" on the
run-canary line, which is SIGKILL. So the third failure mode is not an internal
fault - something outside the process is killing it.

And it is still not the OOM killer. Checked immediately after: oom_kill remained
0 and the allocation-stall counter did not move from 4421, so during run 8 the
cgroup never reached its limit (5.35 GB of 7 GiB), and the host had 13.8 GB
available. Two kills, no OOM evidence either time.

Rather than keep guessing after the fact, freeze_watch.sh now samples host
MemAvailable, cgroup memory.current and oom_kill on every poll and dumps the last
five samples when the process disappears - so the next occurrence carries its own
contemporaneous reading.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 09:07:11 +00:00
Sylpheed RE agent
bb81be7a02 docs: the v2 baseline holds on a second healthy run
Run 7 ended NO FREEZE within 1100s with the same shape as run 5: the same three
threads, the same ceiling of 13 distinct objects, and not one non-SUCCESS window
in 544 across both runs. The v1 stuck-wait side reported its usual single pair.

Run tally since v2 is three runs and no frozen sample - GAME OVER at ~22 min,
EMULATOR GONE at 810 s, and no freeze in 1100 s.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 09:00:18 +00:00
Sylpheed RE agent
e89ae8adff tools: write down the pgrep -f trap that has now cost four mistakes
An agent's commands run through a wrapper shell whose command line contains the
pattern being searched for, so `pgrep -f 'pilot\.py'` matches the shell running
that very pgrep, and `pkill -f 'fly_session|pilot\.py'` kills it mid-script. In
one session this killed two running scripts and twice made an "is it already
running?" guard answer yes because it had found itself. The [p]ilot bracket trick
does not help, because the real invocation appears on the wrapper's command line
too.

Recorded with what works instead: pgrep -x on the exact name, or ps -eo args
filtered against the snapshot-bash path that every wrapper carries, then kill by
explicit pid.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 08:52:24 +00:00
Sylpheed RE agent
58086feea2 docs: a third failure mode - the emulator dies silently, and OOM does not explain it
Run 6 ended neither frozen nor in GAME OVER: freeze_watch reported EMULATOR GONE
at 810s and the 1.1 MB Canary log simply stops mid-activity, with ZERO
occurrences of CRASH, Access Violation, GUEST-THROW, assert, Fatal or abort, and
no shutdown line.

Memory pressure is recorded as a suspect and explicitly not as a conclusion. The
cgroup had reached its 7 GiB ceiling (memory.peak == memory.max) with 4421
allocation stalls at the limit, and page cache - inflated by the disc-wide format
sweeps that read every .pak - was 4.68 GB of it. But memory.events:oom_kill is 0,
so the cgroup OOM killer did not do this, and nothing identifies what did.

Two operational notes fall out: /dev/shm/xenia_memory_* survives a dead run (342
MB resident) and run-canary only clears it at launch, and vm.drop_caches is not
writable here, so page cache can only be left to the kernel.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 08:35:23 +00:00
Sylpheed RE agent
5699b25fc4 docs: record the v2 probe's baseline in the freeze item, and what it rules out
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 08:32:06 +00:00
Sylpheed RE agent
a89cb27644 docs: drop a placeholder link that should never have been written
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 08:14:27 +00:00
Sylpheed RE agent
5fb4da4c43 docs+tools: the call-rate probe's baseline already refutes the timeout reading
v2 counts every KeWaitForSingleObject call per thread per second, tracks how many
distinct objects it saw, and logs the last result. Its HEALTHY baseline is a
result on its own: over a full 22-minute run the main thread cleared 500 calls/s
in 224 separate windows, peaking at 1235 calls/s over up to THIRTEEN distinct
objects, and the last result was X_STATUS_SUCCESS in all 314 windows. Not one
timeout.

So the game normally does hundreds of successful waits a second across many
objects - exactly the blind spot v1 could not see, and the reason a timeout-streak
counter reported the same single poller whether the game was frozen or healthy.

Stated as a consequence rather than a triumph: 500/s is NOT self-selecting, since
the main thread clears it constantly, so the freeze signal has to be a different
shape - a thread far above 1235/s, a new thread, or a window whose result is not
SUCCESS. That still needs a frozen sample; run 5 ended in GAME OVER at ~22 min
without freezing.

freeze_watch.sh now summarises the rate probe per thread when it captures.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 08:14:11 +00:00
Sylpheed RE agent
51c8e49f7b docs: the freeze item now has a captured freeze and a refuted hypothesis
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 07:40:32 +00:00
Sylpheed RE agent
3ecb3432f1 docs: the freeze was caught and the stuck-wait probe reports nothing new
The fourth run froze 9 seconds into the watcher's window, in flight, and the
probe built for that moment showed the healthy-run baseline and nothing else: one
pair, the same poller on the same object VA as every healthy run, only the thread
handle differing. No new (thread, object) pair appeared.

So the hypothesis the probe was built to catch is refuted - the freeze is not a
guest thread looping on KeWaitForSingleObject timeouts against ONE object - while
the CPU signature is unchanged from the gdb run: 1255 ticks over 10 s, 401 in the
TimerQueue thread and 292/280 in two guest threads.

What survives is stated as two specific blind spots of the instrument rather than
a shrug: the waits may cycle over DIFFERENT objects, which resets the streak and
makes them invisible to a same-object counter; or they may SUCCEED rather than
time out, which leaves a timeout counter nothing to count and would fit the
kernel-log evidence of a self-suspending worker cycling thousands of times
successfully.

Next is a v2 that counts calls per thread per second regardless of object or
result. The freeze lottery paid out on the first attempt this time.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 07:40:09 +00:00
Sylpheed RE agent
a40a0b7b01 docs: looking closer at the frame-rate word weakened it, and that is the result
The 16.16 reading of +0x04 rested on twelve bundles at 30.0. They are not twelve
witnesses: they are TWO bundles - pghud_range_main_em/emeff and
pghud_range_nose_em/emeff, both dur 30 - repeated across six language PAKs. A
30-frame flash is equally consistent with 0.5 s at 60 and 1 s at 30, so the
observation that "30.0 bundles are shorter" says almost nothing.

Worse for the clean fixed-point reading: the four bundles at 0x3C0001 would be
60.0000152 fps, which nobody authors. They are py_ranking_jump/py_ranking_next
dialogs, all dur 60. So the better reading is <rate:16>.<flag:16> - a rate-like
number in the high half and a small low field that is 0 on 2843 bundles and 1 on
four, meaning unknown.

Also stated: nothing in this container can settle it by timing, because the
emulator runs on software Vulkan far from real time - a stopwatch would measure
lavapipe rather than the game.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 07:34:56 +00:00
Sylpheed RE agent
f3612b982b docs: promote +0x08 to confirmed, keep the frame-rate reading amber
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 04:40:46 +00:00
Sylpheed RE agent
192a23a481 formats: +0x08 is the animation length - checked against the keyframe times
The rate/duration reading of the bundle header came from the values alone, so it
was checked against something the file states independently: the keyframe `time`
fields in the placement region.

Over the 2313 bundles that have both, the largest keyframe time is <= +0x08 in
EVERY one, none exceeds it, and 444 reach it exactly. The refutation attempt is
in the same data: a large unrelated constant would bound every time too, but the
max_time/+0x08 ratio would then pile up near zero - instead it peaks at exactly
1.0 with 520 bundles. Asserted, including the shape of that histogram, so the
bound cannot later be waved through as vacuous.

Stated precisely because the units are a separate claim: what is proven is that
+0x08 is the animation length in the SAME UNIT as a keyframe's time. That the
unit is frames still rests on the values (30/60/120/1200) and on the 16.16
reading of +0x04, which stays amber - the only new evidence for it is that the
twelve 30.0 bundles cap at +0x08 = 30 while the 2843 60.0 ones reach 1440, and
twelve bundles is not a demonstration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 04:40:28 +00:00
Sylpheed RE agent
3a9807f393 docs: close the screen-vs-fragment question against the header, and record what the header does say
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 04:26:19 +00:00
Sylpheed RE agent
27fcb69fdf formats: sweep the bundle header - no screen flag, but three of its words decode
The backlog asked what makes a bundle a screen rather than a fragment, and the
obvious suspect was the 32-byte header. Swept over all 2859 composable bundles
with a real declaration table.

The answer to the question is NO, and it is asserted rather than argued: no bit
of the flags word at +0x10 labels a screen. The best any bit manages is bit 13 -
403 bundles, 179 of them carrying a full-screen element, a 44% hit rate against a
12.8% base - and the commonest bit is set on 91% of everything. Enrichment, not a
marker.

The sweep found more than it was asked for, though. The header is not dead space:
+0x18 is 1280 on 2829 bundles and +0x1c is 720 on 2823 - the design resolution at
bundle level, the same pair the parser already reads out of a .rat record, and
asserted here. And +0x04 takes only three values, 0x3C0000 on 2843 and 0x1E0000
on 12, which are exactly 60.0 and 30.0 in 16.16 fixed point, with +0x08 taking
30/1200/120/60 - a frame rate and a duration in frames would fit a format whose
records are keyframe lists. That reading is marked amber: it comes from the
values alone and is not verified against an animation.

Also recorded, since the file will not say: element counts are min 1, median 2,
p75 5, p95 23, max 56, and only 365 bundles carry a full-screen element. The
population is mostly fragments and the separation is shape.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 04:26:06 +00:00
Sylpheed RE agent
92dd1da65a docs: the opt-link question has an answer and a stated coverage limit
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 04:09:30 +00:00
Sylpheed RE agent
d9a2229217 formats: what opt links, measured - a record-to-record reference, focus is one use of it
Two readings were on record and both were wrong in different directions: the
structure doc called it "normal state -> focused state" from a single example,
and the backlog called it "refuted as focus; unexplained otherwise".

Classified every link reachable from a declaration table: 1467 links, and ALL
1467 resolve to a RATC child of their own bundle, ALL are .rat -> .rat, none
dangle and none self-link. 1076 (73%) match the <stem>f focus pattern; the other
391 are chains between effect records - px_bunk_eff01 -> pjex_eff -> pjex_eff07,
pveff01 -> pjeff02 -> pjeff21 - which also explains why only 227 targets are
themselves declared elements: the middle of a chain is, the end is not.

So `opt ` is a record-to-record reference within the bundle, and focus is its
commonest use rather than its meaning.

Coverage is stated rather than glossed: the bundles hold 18718 raw `opt ` tags
against the 1467 classified, because opt_link reads the first tag of a DECLARED
element's record. Roughly 92% of occurrences sit deeper in the chains (or are
byte coincidences - the scan is unaligned) and are untested. The numbers are
asserted so the answer cannot drift back into an anecdote.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 04:09:18 +00:00
Sylpheed RE agent
f160b416ff docs: close the focused-state question against the declaration table, and log a third freeze-free run
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 03:50:58 +00:00
Sylpheed RE agent
2d28226b30 formats: the declaration entry does not mark a focused state - swept and asserted
The backlog's cheapest open question about the 60-byte entry: kind is a flags
word (0x10 untextured primitive, 0x4 repeated instance, 0x3002 button record), so
a focus bit would be the obvious answer and the name-pairing in
mark_focused_states would be a convention standing in for a real field.

It is not. Over every screen build on the disc: 54 name-paired focused/base
pairs, all 54 with IDENTICAL kind (all 0x0), NO bit ever set on the focused entry
and clear on its base, and the only words of the entry that ever differ are +48
and +52 - the pivot. The two entries differ in where the sprite sits and in
nothing else.

The test asserts all four numbers rather than just printing them, so the negative
is pinned instead of decaying back into a suspicion. Also noted: these buttons
carry kind = 0x0, so the documented 0x3002 "button record" belongs to the .rat
records and not to the .t32 sprites a menu draws.

Closes the declaration table, not the question - the focused state could still be
marked in the .rat record, the RATC child stream, or only in the game's code.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 03:50:40 +00:00
Sylpheed RE agent
68a3fed554 docs: note that catching a freeze is a per-run lottery, and the watcher that waits for it
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 03:14:25 +00:00
Sylpheed RE agent
0ef0e77b1e docs: the stuck-wait control holds on a second run, and the freeze did not come
A second 25-minute Stage 02 run under --log_stuck_waits ended NO FREEZE within
1500s, with the probe again reporting exactly one pair - thread F800004C on
object BE56BB5C - 24 lines against the first run's 27. The healthy-run control is
therefore measured twice and independently, so anything else the probe prints
during a freeze is signal.

Also recorded: two consecutive runs did not freeze, and the earlier freezes came
in a cluster. Roughly half the runs that reached flight ended early across the
session, but not evenly, so "wait for a freeze" is a ~30-minute lottery per run
rather than a dependable step. Instrument, watcher and control are in place; one
frozen sample is what is missing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 03:14:14 +00:00
Sylpheed RE agent
c06c6a8cf4 tools: wait for the freeze and capture the evidence at the moment it lands
Catching the in-mission freeze by hand costs a tool call every 25 s, and it
arrives anywhere between ten seconds and never - roughly half the runs that reach
flight freeze and the other half do not. freeze_watch.sh polls frozen.py, checks
the flight HUD is still on screen (the GAME OVER screen ANIMATES, so a run that
ended there is not a freeze and must not be scored as one), and on detection
snapshots what the stuck-wait probe has said.

Taking the probe snapshot at the moment of detection matters: the comparison
against the healthy-run baseline - one thread polling one Event at BE56BB5C - is
then made from the same instant rather than reconstructed afterwards from a log
that kept growing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 02:58:23 +00:00
Sylpheed RE agent
a58c8665e0 docs: the freeze item has an instrument and a control now
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 02:31:15 +00:00
Sylpheed RE agent
61acc81b3c docs: correct the canary commit hash after dropping an unrelated submodule change
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 02:31:03 +00:00
Sylpheed RE agent
4b6b72ca7b docs: the stuck-wait probe and its healthy-run baseline
The instrument is built (canary auto/re-wait-timeout-probe 83ad0eb3f, binary
archived at /sylph-home/re/bin/waitprobe): consecutive timeouts on the same
object, per thread, logged at 100 and then every 500 - self-selecting where the
global kHighFrequency flag was unusable.

The control is measured: a 25-minute Stage 02 run that never froze produced 27
lines, ALL one thread polling one Event at guest VA BE56BB5C with a ~30 ms
timeout, whose streak reached 8000 while the game ran perfectly. So the probe is
quiet, not silent - the first draft of the cvar help said silent and that is
corrected here - and the freeze signal will be a NEW (thread, object) pair rather
than the presence of output.

Not settled: the freeze did not reproduce this run, so there is no frozen sample
to compare against yet.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 02:30:29 +00:00
Sylpheed RE agent
315b7e1561 docs: point the freeze item at the targeted log line rather than the global flag
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 01:49:18 +00:00
Sylpheed RE agent
80dc6352b6 docs: logging the wait target is blocked by the flag's own cost
KeWaitForSingleObject is kHighFrequency, so it is silent unless
--log_high_frequency_kernel_calls=true - which confirms the corpus's earlier
"kHighFrequency waits are simply unlogged" note. With the flag the calls do
appear (1944 lines in 40 s), but the emulator slows so far that after SEVENTEEN
minutes the screen was still black and skip_intro had not seen even the intro
movie, with 175 MB of log written. A 2400 s boot budget did not help; the boot
was not going to finish.

Recorded with the numbers so nobody pays them twice, along with the two cheaper
routes: a targeted Canary log line that fires only after a wait has timed out N
times on the same thread (self-selecting, free on a healthy run, one small patch
plus a build that is already configured), or digging the PPCContext out of a
stack frame under gdb - possible but fragile, since the Release binary has symtab
only and no DWARF.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 01:49:05 +00:00
Sylpheed RE agent
99d94146b4 tools: give launch_mission a boot-timeout knob and an extra-flags hook
Both exist for one diagnostic. KeWaitForSingleObject is declared kHighFrequency,
so it is silent even at LOG_LEVEL=3 unless
--log_high_frequency_kernel_calls=true is passed - and that flag writes ~157 MB
in ten minutes and slows the emulator so far that the title had still not
appeared when skip_intro's fixed 600 s budget ran out. The run was then scored
"BOOT FAILED" when it was only slow.

EXTRA_FLAGS passes one-off diagnostic cvars without making them defaults;
SKIP_INTRO_TIMEOUT lets a deliberately-slowed run have the time it needs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 01:37:28 +00:00
Sylpheed RE agent
cd2704c2ac docs: the freeze entry gets a measurement instead of a plan
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 01:13:13 +00:00
Sylpheed RE agent
7150827e96 tools+docs: the freeze from inside - all 79 threads are in a wait, and two of them burn CPU cycling
The gdb route works, and the recipe is written down: run-canary execs $XENIA_BIN,
so a wrapper that execs "gdb --args <real binary>" keeps the lockfile, the flags
and the process name (gdb forks and execs the real binary, so ps -C xenia_canary
still finds the inferior) while satisfying ptrace_scope=1 by being the parent.
The Release binary is not stripped - 26595 symtab entries - so frames have names.
The handle SIGSEGV/SIGBUS/SIG32-35 lines are mandatory: xenia uses SIGSEGV for
guest memory watches and the RT signals for thread suspend.

A run froze after ~4 minutes of flight, screen still "flight" rather than GAME
OVER, and all 79 threads had backtraces. EVERY ONE is in a wait - guest threads
in KeWaitForSingleObject / NtWaitForSingleObjectEx / SelfSuspend, the GPU command
processor parked idle, the main thread in poll().

It is nevertheless burning 1253 ticks per 10 s: 403 in the TimerQueue thread
(nanosleep inside TimerThreadMain) and 290 + 274 in two guest threads that the
backtrace shows blocked in KeWaitForSingleObject. A thread genuinely blocked
cannot burn 28% of a core, so those two are CYCLING - a timed wait that expires
and is re-entered - with the timer thread servicing them hot. Three samples
minutes apart show identical frames.

That refines the earlier "the guest is spinning, not deadlocked": the CPU burn is
real but it is in the WAIT PATH inside the kernel layer, not in guest code. The
shape is an event that never gets signalled.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 01:13:01 +00:00
Sylpheed RE agent
b41727951e docs: fourth bit-level attempt - frozen again, but caught in a minute
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 00:41:59 +00:00
Sylpheed RE agent
a27dddf93d docs+tools: scope the guest-PC experiment properly, and let ob_bitflag find the counter itself
The freeze's named next step is more expensive than it looked, and the reasons
are recorded before anyone starts it: xenia's stack walker is a TWENTY-LINE STUB
on POSIX (Create logs "unimplemented" and returns nullptr), so
ThreadDebugInfo::guest_pc is never filled here; PPCContext carries no live PC
either. A reverse host->guest map is buildable - the code cache already learns
the mapping in OnCodePlaced - but that plus a way to sample another thread's RIP
is a real emulator feature, not a patch.

The cheap route that does exist: gdb is installed and ptrace_scope is 1, so
attaching to a running emulator is refused but launching it UNDER gdb is not -
run-canary execs $XENIA_BIN, so a wrapper that execs "gdb --args <real binary>"
keeps the lockfile and flags and makes gdb the parent. That would separate
"spinning in guest JIT code" from "spinning in a xenia loop", which is the fork
this is stuck on. Not attempted yet.

ob_bitflag now locates the counter itself from the three addresses measured so
far, instead of refusing when the default one is wrong. That costs no
transitions, and with roughly half of all runs ending early, transitions are the
scarce resource.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 00:41:47 +00:00
Sylpheed RE agent
6b3442ce8c docs: record the bit-level attempts and that run attrition is the real cost now
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 00:20:35 +00:00
Sylpheed RE agent
ca133c0220 tools+docs: the bit-level OB test, and a liveness hole - GAME OVER animates
ob_bitflag.py is the follow-on the word-level refutation named: for every 4-byte
offset in the window and every one of its 32 bits, count how many entities have
it set, keep the pairs whose count is exactly the counter, and require them to
match again after a transition. Both polarities, since an objective could be
marked by a bit that is CLEAR on it.

Three runs, no verification, and the reasons are recorded: run 1 gave 187 + 33
candidates at counter 4 and then reported "the counter never moved" for 700 s -
about a mission that had ENDED in GAME OVER partway through; run 2 hit the same
dead mission; run 3 had the counter at a different address (the guard refused,
correctly) and then froze after one filter.

The hole is closed. frozen() asks whether the guest is ANIMATING, and the GAME
OVER screen animates happily - mean colour (114,22,63) - so every liveness check
passed while the mission was over. frozen.in_flight() classifies the screen with
screen_id, and ob_hunt/ob_flag/ob_bitflag now abort with NO LONGER IN FLIGHT.
That is the second confident negative in this investigation that was really about
a dead world, so the rule is written down: before believing "X never happened",
show that the thing that would produce X was still running.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-24 00:20:04 +00:00
Sylpheed RE agent
f6febd1bd6 docs: close the flag hypothesis in the backlog, and name the bit-level follow-on
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 23:21:23 +00:00
Sylpheed RE agent
26421d5266 docs+tools: the per-entity OB flag is refuted, and the counter decrements when the player kills
With the pilot finally shooting, the experiment completed both halves. Sample A
at counter 12 over 120 entities gave 2 offsets where exactly 12 entities share a
value; the counter then went 12 -> 11 and NEITHER survived. So within +-0x400 of
an entity's position triple there is no 4-byte word whose shared-value population
tracks REMAINING OB.

The limits are recorded as part of the result, because they bound it: the test
asks which entities share an EXACT 32-bit value, so a single bit ORed into a word
that also carries health or a timer would never show up - a bit-level version of
the same differential is the follow-on. Anything outside the window, or on
entities that entities2 cannot see (it types by position CHANGING, so stationary
objectives are invisible), is untested too, and the populations differed a lot
between samples - 120 against 194.

Separately: REMAINING OB went 12 -> 11, the first decrement of this whole
investigation, while pilot.py logged 411 fire=1 samples and the HUD reached YOU
KILLED WARPLANES 0003. Stated carefully - it does NOT show the counter counts
kills, since an earlier run had the hostile population fall by a third with no
movement; it shows some kills close something the counter tracks.

Two robustness fixes: ob_flag retries an empty entity sample (one void run was
caused by exactly that), and Pad releases everything on interpreter exit - a
file-backed pad PERSISTS after its writer dies, so a tool killed mid-press would
leave a button held and the game would walk through menus on its own.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 23:21:08 +00:00
Sylpheed RE agent
e65b730af0 docs: the pilot fires - record it, and what it still does not do
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 22:39:46 +00:00
Sylpheed RE agent
5a9b16bf03 tools+docs: the pitch stick sign was inverted - the pilot fires now
Measured, not argued: command a 45-degree-off-the-nose error and watch whether it
shrinks, both axes, both sides, two pulse widths, with the opposite sign as a
control. Yaw's sign is correct (45 -> 11.3/32.7 at 0.6 s, 41.6/33.8 at 1.2 s).
Pitch's is inverted - the pilot's own sign GREW the error every time
(48.3/55.6/70.7/91.7) and the opposite shrank it every time (30.7/41.5/15.7/9.6).

A method artefact is recorded because it gave the opposite answer first: a 3 s
full-deflection pulse overshoots a 45-degree error so far that BOTH signs look
wrong (45 -> 164 and 45 -> 178). A long pulse cannot answer a sign question.

Verified against the game rather than by inspection. Before: fire=1 in 0 of 13521
samples, |aim yaw| pinned at 90.0, target 36-43 km away. After: 43 of 1732, aim
down to 2.3 degrees, range median 6.3 km, and the HUD's own ammunition counters
moving - NOSE BM 06000 -> 05723, MAIN MPM 00300 -> 00298.

Also fixed a leftover of the same FIFO era: pilot.py called pad.f.write("tap A
90") for the target-select double tap, which raised AttributeError once Pad
stopped having an `f`. Pad gained tap()/dpad(); ctrl_probe.py and target_probe.py
still use pad.f and now say so in place.

Still open: YOU KILLED is 0000 after 250 s of firing and REMAINING OB is still
012. The craft shoots, closes and selects; whether it destroys anything is next.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 22:39:33 +00:00
Sylpheed RE agent
70497b1606 docs: date the dead pad, so earlier flight results can be triaged instead of doubted
The uinput pad was replaced on 2026-08-13 (canary cbfad1ab3), which is when
flight_probe.Pad stopped reaching the game. flight-controls-runtime.md predates
it (2026-07-30) and is safe; flight-speed-law.md was last touched on the same
day and cannot be placed from dates alone; anything after it measured a craft
that was not being flown.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 22:10:21 +00:00
Sylpheed RE agent
dde23cac3c docs: record the dead-pad root cause and what it does not fix
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 22:09:51 +00:00
Sylpheed RE agent
0ddeffa9a6 tools+docs: the flight pad was writing into a dead file - every autopilot input went nowhere
flight_probe.Pad opened /tmp/sylph-vgamepad.fifo, "the vgamepad server's FIFO".
That server was removed when the uinput pad was replaced by Canary's --hid=file
driver, because a uinput device is not namespaced and scripted presses leaked to
the host's desktop. The FIFO is now an ordinary 91-byte file nothing reads, and
/tmp/xenia_pad.txt - the file the emulator polls - was 0 bytes while the
"autopilot" was flying. So every axis, trigger and button from pilot.py,
autopilot3.py, aim_probe.py and flight_probe.py went nowhere, silently.

The corpus already carried this trap for the SHELL scripts
(canary-scripted-input-traps.md, "every call here failed silently"). This class
was missed, and every flight tool imports it.

Verified against the oracle rather than by inspection, on runs confirmed
animating at both ends of every phase: before, full stick produced 0.00 degrees
of heading change over 4 s while the ship travelled 350-735 units, and the
attitude matrix at pos-0x70 was byte-identical; after, LX=-1 turns 12.72 degrees
and LX=+1 swings the flight direction from [1,0,0] to [0.13,-0.14,-0.98], with
the matrix moving 0.44/0.31 under stick and 0.0000 at neutral.

That also REFUTES the "stale attitude matrix" suspicion from the earlier pass -
pos-0x70 is live and tracks the ship; it only looked dead because nothing was
turning the ship.

Still open and said plainly: with the pad fixed a 150 s pilot run still fires 0
times, with |aim yaw| still exactly 90.0 and the target 36-43 km away. Steering
works now, so what is left is the stick SIGN against the pilot's error convention
or a target selection that commits to something too far to close. Both are
finally testable.

findrot_global/findself/findspeed/selfstate still write to the dead FIFO and now
say so in place; they are not repaired because none has been re-run since.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 22:09:27 +00:00
Sylpheed RE agent
d9cf899667 docs: record the pilot's proximal cause in the backlog
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 21:27:53 +00:00
Sylpheed RE agent
9c94a16d5a docs: the frozen-world memory delta, measured - 12x quieter, not silent
A frozen guest still changes ~30000 words a second (180952-235973 in 6 s, 0.2% of
the data) against 1863296 in 4 s (2.08%) while running. By region the frozen
activity concentrates in 0xbf900000/0xbfa00000, in 0xbe0-0xbe3 00000, and in the
0x704/0x705 pages that the boot log shows are guest thread STACKS - frame churn,
which is what a spinning thread looks like.

Stated plainly as not-yet-interpretable: there is no matched region breakdown for
the RUNNING case, because the control predates the summary. Those regions may
simply be the busiest ones at all times, and one run fixes that.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 21:27:53 +00:00
Sylpheed RE agent
813ae8c681 docs+tools: why the pilot never fires - the target is behind it, 100% of the time
fire=1 appears in 0 of 13521 samples. The gate that closes is measured rather
than guessed: over the 3004 samples that had a target, |aim yaw| is EXACTLY 90.0
degrees every time, which is the `if ez < 0` branch in sticks() - the target is
astern - with pitch near 180 and a range that grows 22km -> 49km and plateaus.
The craft flies away from what it is chasing for fifteen minutes and the turn
never completes.

What is NOT established is why, and the attempt is withdrawn rather than kept:
aim_probe.py reported the forward vector pinned at [-1,0,0] with 0.00 deg/s under
neutral, full-left and full-right stick, which looks like a stale attitude
matrix - but the guest had FROZEN partway through the probe, confirmed after the
fact by frozen.py and by the player position being identical across 3 s. A dead
world holds every matrix still. The probe is committed because it is the right
experiment; its numbers are not evidence.

One confusion resolved: today's entities2.py "0 moving triples" bind failures are
the freeze, not a tool defect - moving() types entities by position CHANGING, so
a frozen world yields nothing by construction.

live_delta.py gains a per-1MB-region summary; a flat list is useless at 236000
hits.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 21:27:27 +00:00
Sylpheed RE agent
cc865cd9da docs: revise the OB-flag item - the blocker is the pilot's trigger, not the freeze
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 21:05:27 +00:00
Sylpheed RE agent
81cb275173 docs: the OB-flag sample finally ran - 11 candidates, and a real negative about the counter
A run that stayed animating for 25 minutes gave the first proper sample: 161
entities against a counter of 12, so "not a class head-count" is now confirmed on
a sample seven times larger than the frozen run's. Eleven candidate offsets, with
who they group recorded - and the two biggest groups look wrong for an objective
flag on inspection: seven consecutive floats shared by 12 of 32 attackers read
like one shared AI/weapon parameter block, and +0x0238 = 4 is shared by twelve
TURRETS, which do not fit an objective that says "shoot down all invading enemy
fighters". None of it is verified.

The reason it is not verified is itself the result: REMAINING OB held at 012 for
fifteen minutes of LIVE flight - not a frozen world this time - so there was no
second sample. And it held while the hostile population fell from 132 to 93 ADAN
entities. That is a much stronger version of the earlier head-count argument: the
counter is not "hostiles left".

Also recorded, because it weakens every kill-based experiment in the corpus:
pilot.py never fires. fire=1 appears in 0 of 13521 samples. It flies, chases and
evades, and the trigger is never pulled, so kills are the wingmen's.

Freeze frequency updated: 3 of the 7 runs that reached flight.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 21:05:08 +00:00
Sylpheed RE agent
eb7fbecd40 tools: diff live guest memory against itself, and stop aborting on an unreadable HUD frame
live_delta.py exists for the freeze: a frozen run is guest code spinning, and
spinning code waits on guest memory, so the words still changing while frozen
should be the loop's own bookkeeping and whatever it polls. Measured control on a
RUNNING game, so the frozen number has something to be compared against:
1863296 words change in 4 s, 2.08% of the 357.7 MB of data extents. The sparse
4.6 GB costs nothing - the walk is over SEEK_DATA extents.

The freeze did not reproduce this run, so the frozen half of that measurement is
still owed.

ob_flag.py: an unreadable HUD frame is now retried rather than counted as a
mismatch. ob_read returns None on a frame it cannot read - an explosion across
the plate, a flash - and the sanity check treated that as "the address is wrong"
and aborted a perfectly good run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 20:53:07 +00:00
Sylpheed RE agent
99be872d1c docs: correct the freeze entry - the lead is dead, the spin is the finding
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 20:03:53 +00:00
Sylpheed RE agent
d8e9874984 docs: the resume-spin lead is refuted by its own control, and the freeze is a guest-side spin
The control this file never had: a run with the Kernel channel on, analysed WHILE
STILL FLYING, has 2738 refused resumes on one pair - more than the frozen run's
1171. The target's own lines show why. The game runs a self-suspending worker
(NtSuspendThread on itself, a manager thread resumes it, thousands of times), and
a self-suspended thread is not host-suspended, so the host Resume legitimately
returns false EVERY cycle: 3115 refusals against 3115 resumes.

The error is named rather than buried: the warning's commit says "~7 times in a
normal boot" and this file generalised that from boot to gameplay, where the
number is thousands. The 150x anomaly was an artefact of the baseline. The
zero-CPU threads go with it - the healthy run has four of those too.

What the instrumented reproduction DOES establish is sharper than the lead was.
The last kernel event in 690000 lines is "Thread F8000204 self-suspending", with
self-suspends 3116 against resumes 3115 - but the resumer never issues another
NtResumeThread at all, so nothing was dropped in flight; every thread stopped
together. And the guest is SPINNING, not deadlocked: over 10 s while frozen the
main thread is in state R gaining 409 ticks and guest threads gain ~680 in total
while making not one kernel call. So it is guest code waiting on something in
guest memory, and the next question is which guest PC.

Two corrections fall out: "the log stopped growing" is not a freeze detector (it
goes quiet for 25 s in normal flight), and 0xbdb59668 held the counter again -
4 of 6 runs now.

freeze_report.py makes the analysis repeatable, and refuses to answer "did this
thread ever run" when the Kernel channel was off rather than reporting a false NO.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 20:03:38 +00:00
Sylpheed RE agent
15d240bffb docs: promote the in-mission freeze - it is now blocking the item behind it
Two more attempts at the OB-flag experiment, both stopped by it: one froze before
the counter could move, the next had the counter at a different address (the
tool refused to run rather than report nonsense) and then froze as well.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 19:36:20 +00:00
Sylpheed RE agent
188e911ec7 docs: the in-mission freeze ends in 1171 refused resumes of one thread
A Stage 02 run froze at TIME 01:02 with a radio line caught mid-word. The
emulator was alive at ~200% CPU with its main thread in state R, two screenshots
six seconds apart were byte-identical, and the log ended in a spin: 1171 of the
run's 1200 "host resume was refused" lines are the single pair F80002AC ->
F8000240, starting at the line immediately after F80002AC is created, and the log
never grows again. The commit that added that warning records what normal looks
like - about 7 in a whole boot - so this is a 150x anomaly on one pair rather
than noise. F8000240 itself appears exactly once outside the spin, at creation,
and calls nothing.

Eleven of the frozen process's 79 host threads have zero CPU, four of them
consecutive late-created guest threads - the same signature as the lost resume
that c1b57f93b fixed for the title screen. That fix IS in this build, so either
there is a second window in that race or this only looks alike.

The inference is named as one: nothing here maps a guest handle to a host tid, so
"the zero-CPU threads are the ones being resumed" is a reading of two consistent
observations. And the refusals could equally be the game's reaction to a worker
stuck for another reason - log_mask=13 has the Kernel channel disabled, so not
one of F8000240's waits is visible. The next experiment is written down
concretely: reproduce with LOG_MASK=12 LOG_LEVEL=3 and map the handle to a tid.

Also caps ob_hunt's survivor listing at 40 - an aborted run printed all 21482 and
buried the line that mattered.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 19:36:03 +00:00
Sylpheed RE agent
639bedf68e docs: open the "what does REMAINING OB count" item, with the freeze noted as the blocker in front of it
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 19:10:57 +00:00
Sylpheed RE agent
fcd9fcecf8 docs+tools: a mission freeze that made an experiment lie, and the counter is not a class head-count
Three things from one Stage 02 run.

The address recurs a third time: HUD=4 RAM=4 at 0xbdb59668, so 3 of the 5 runs
measured put the counter exactly there.

The counter is NOT a live class head-count. With the counter at 4 the typed
entity list was 8 attackers, 7 friendly Delta Sabers, 7 turrets and the player -
no class has 4 members and no pair of them sums to 4. That sharpens the corpus's
existing "012 against 118 live ADAN" note from "not the hostile count" to "not
the count of any class this enumeration can see".

The flag experiment itself proves nothing, and why is the useful part. It found
20 offsets where exactly 4 of 23 entities agree, then reported "the counter never
moved" for 600 s. The guest had stopped advancing ten seconds into flight:
pilot.py logged 724 s of identical speed/yaw/pitch, and two screenshots six
seconds apart were byte-identical, max delta 0 over 863325 pixels - while
screen_id said "flight", the emulator burned 212% CPU and every liveness check
passed. So that was a fact about a dead world. Withdrawn along with it: the claim
in ob_session.sh that the counter climbs on its own in the first minutes, which
one advancing run supports and this one cannot.

frozen.py makes it a single call, checked in both directions (0 on the frozen
pair, 254 on two frames of a live run), and ob_hunt/ob_flag now say GUEST FROZEN
rather than waiting out their timeouts.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 19:10:41 +00:00
Sylpheed RE agent
2a844e6da9 docs: close the objective-counter address item in the backlog
Keeps the refutation next to the confirmation rather than replacing it, and
records the rule a future session actually needs: try 0xbdb59668, check it
against the HUD, re-scan when it reads 0.

What stays open under it is unchanged and is the part that matters for the
autopilot - what the counter counts, and whether an OB-badged entity carries a
flag in its entity object.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 18:39:40 +00:00
Sylpheed RE agent
dd39cdd03d docs: REMAINING OB is at 0xbdb59668 - confirmed to the corpus's own standard
One filter to select (004 -> 008, 35897 -> 7), a second on a transition it was
NOT selected by (008 -> 012, 7 -> 1), leaving exactly one address; then three
live paired readings, RAM/screenshot/RAM, all agreeing with the HUD.

The address is the same one run 1 reported. That does NOT reverse yesterday's
refutation and the entry says so explicitly: runs 2 and 3 read a hard 0 there on
an allocated page while the HUD counted, so "it is there in every run" stays
refuted. What is withdrawn is the stronger claim that the number was meaningless
- it recurs exactly, in 2 of the 4 runs measured, and run 3's amber candidate
0xbdb49668 sits one 64 KB page below it at the identical page offset 0x9668. The
practical rule is therefore: try 0xbdb59668, check it against the HUD, re-scan
when it reads 0.

Why run 3 failed and run 4 did not is also recorded, because it is a method
lesson rather than luck: the evidence was always in the first four minutes of the
stage, and the earlier runs simply could not look often enough - every HUD
reading cost a human round trip, so the 008 step went by between two of them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 18:39:22 +00:00
Sylpheed RE agent
abd72a2c24 tools: read REMAINING OB off the HUD, so the scan can follow the counter
Both hand attempts at this address managed exactly one filter and then lost the
counter, because every reading of the HUD costs a human round trip. ob_read.py
does it by template correlation over three fixed digit cells, and ob_hunt.py
uses that to run the whole method unattended: confirm the HUD on both sides of
the 0.9 s scan, then filter on each following transition, labelling the first as
the one it selected on and the rest as verification.

Two things measured rather than assumed:

* Normalisation is the point, not a nicety. The plate is translucent, so an
  explosion turns it orange - on the 008 reference frame a cyan-stroke mask finds
  6 of ~70 stroke columns and would silently read nothing. Per-cell mean/std
  normalisation reads it correctly at 0.843.
* The accept rule is two-sided because the margin is narrow: over 12 cells from
  four frames of known value the correct digit scores 0.843..1.000 and the best
  WRONG digit reaches 0.789 (0 and 8 are similar outlines). So a floor of 0.80
  AND a 0.05 margin over the runner-up, against a smallest observed correct
  margin of 0.093. A bare threshold fitted between those two numbers would be
  fitted to twelve samples.

Templates exist for 0 1 2 4 8 - the digits actually seen. Anything else reads as
"?" and callers must treat a "?" as unknown, never as a value. Rejects both
negatives tested: GAME OVER scores 0.05-0.18, the main menu 0.07-0.14.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 18:19:29 +00:00
Sylpheed RE agent
7146aed1a5 docs: fold the launcher repair and the clean scan into the backlog
The launcher item closes: unattended boot to Stage 02 flight works again, and the
entry now names all three defects that were in the way, including the
wait_flight pixel that was outside the HUD.

The objective-counter item stays open but is sharper: a HUD-clean scan exists,
and the surviving candidate that is hard to dismiss is named with the odds
against it being chance. The blocker is now the pilot's survival, not the method.

Also withdraws yesterday's claim that the "Auto-Save is active. OK?" dialog does
not return - the run that appeared to show that was tapping A every six seconds
while it waited, so it cannot distinguish "absent" from "dismissed unseen".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 18:04:53 +00:00
Sylpheed RE agent
2c5f653bce docs: a clean scan for REMAINING OB, and one candidate that is hard to dismiss
With the launcher repaired the scan could be run the way the method asks: the HUD
was confirmed at 004 immediately before AND after the 0.9 s scan, and the filter
was taken at a confirmed 012. That leaves 17 candidates from 39596.

One of them is 0xbdb49668 - the run-1 address minus exactly 0x10000, at the same
page offset 0x9668. Checked rather than admired: of all 39596 words holding 4 at
scan time, exactly TWO sit at that page offset, so a survivor landing there by
chance is a ~5e-5 event. The reading it suggests is testable - the counter sits
at a fixed offset in an allocation whose base moves by whole 64 KB pages.

Recorded as amber, not green, and the reason is written down: the later sample
that killed 10 of the 17 was taken during the GAME OVER flash, with no HUD on
screen, so the seven survivors are unrefuted rather than confirmed. The mission
ended because the escorted ACROPOLIS was lost at ~13 min while pilot.py sat in
EVADE/RETIRE; REMAINING OB held at 012 for the ten minutes before that, so there
was no transition to verify against.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 18:04:29 +00:00
Sylpheed RE agent
2888af5d7b tools: wait_flight was testing a pixel that is 40 rows outside the HUD
It tested one named pixel, (450,640), described as "inside the SHIELD bar". It is
not inside anything. `screenshot` crops to the GAME SURFACE, 1279x675, while
those constants were chosen for a 1280x720 window: measured on a frame that was
unmistakably in flight (TIME 01:18.05, REMAINING OB 004), the SHIELD bar's green
rows are 590-605 and the ARMOR bar's are 650-670, and (450,640) sits in the gap
between them reading (64,66,116) - blue.

That is why the corpus already carried "wait_flight.sh reported NEVER REACHED
FLIGHT while the game was plainly in flight" as an unexplained note, and it is
exactly the failure screen_id.py's own header warns about. Reproduced live here:
the script sat for five minutes reporting nothing while tapping A into the
cockpit every six seconds.

Now classified by whole-image statistics - screen_id.py's "flight" class, green
fraction 1.3-1.5% against <0.1% on every menu - which no crop or scale can move.
Two consecutive frames are required so an explosion in a cutscene cannot pass,
and the A tapping (which dismisses the objective card) STOPS once flight is seen,
because in the cockpit A is a weapon press and not a "continue".

Tested against the running game before being believed: 0 with "IN FLIGHT at 19s"
on the same session the old test had been failing on, and then in the unattended
launch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 18:02:27 +00:00
Sylpheed RE agent
8bea94a47f tools: launch_mission reaches Stage 02 flight unattended again - wait for screens, not seconds
Two separate reasons the scripted route stopped, both measured rather than guessed:

* LOAD -> READY ROOM is not 28 s. Both runs on 2026-08-23 overran it, so the next
  press was eaten by the transition and the run ended up in OPTIONS once and
  BRIEFINGS once. wait_screen.sh now waits for the screen, with an optional
  --tap that clears a dialog the caller cannot know about (a freshly restored
  profile inserts "Auto-Save is active. OK?" here).
* The READY ROOM is DRAWN long before it is USABLE: it comes up with a
  "Preparing to Sortie" spinner and TAKE OFF greyed out. The two states differ by
  1.7 units of blue whole-image, so screen_id.py cannot separate them and should
  not try. take_off_armed.py tests the label instead: 0.0000 bright pixels while
  preparing, 0.1633 once armed, on three captures from two runs. It carries its
  own position check - the always-enabled BRIEFINGS label below reads 0.1027 in
  all three, to four decimals, so if that reference is dark the boxes are off the
  labels and the answer is "unknown", not a confident wrong one.

screen_id.py gains a "readyroom" class from the same measurements; nothing else
reclassifies.

Verified end to end and unattended: boot -> title -> LOAD GAME -> slot 01 ->
READY ROOM -> TAKE OFF -> "IN FLIGHT at 34s", pilot bound and engaging.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 18:02:12 +00:00
Sylpheed RE agent
d412347c37 docs: the REMAINING OB address does not survive a run - refuted twice over
0xbdb59668 reads 0 in two independent Stage 02 runs while the HUD counts
004 -> 008 -> 012. Not an unmapped read: SEEK_DATA at that offset returns the
offset itself and the next hole is 5 MB later, so it is an allocated,
zero-filled word. The address was a per-run artefact, exactly as that file
already suspected it might be; the method is the durable result.

Re-finding it in the new run also failed, and both failures are recorded because
they are the instructive part. Two candidates were produced and both died on the
corpus's own rule -- verify across a transition you did not select on:
0xbc2377dc went 12 -> 18 while the HUD stayed 012 and read 3 two minutes later,
and 0xbd295b04 was plain noise.

One correction to the method note in that file: the scan is not slow. Over the
live /dev/shm image it takes 0.9 s. The real trap is that REMAINING OB climbs
004 -> 012 within about four minutes as waves spawn, so a scan is only valid if
the HUD is confirmed to hold the same value immediately before AND after it --
which is why the earlier 4-then-8 intersection came back empty.

What blocked finishing: with pilot.py retired at hull 340/1500 nothing was
killing objectives and the counter sat at 012 for five minutes, so there was no
later transition to filter on. What the counter counts, and whether an OB-badged
entity carries a flag, is untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 17:16:07 +00:00
Sylpheed RE agent
c905b3b6ad docs: the dynamic-RE state was gone - rebuild it, measured rather than remembered
The baseline binary, the profile, the Stage 02 save and the caches all live
outside both repos and had been wiped, and sylph-doctor reports 'all good'
without them. This is the recipe, verified by driving it to Stage 02 flight
rather than by inspecting files.

Three things worth having found:

* the baseline is an INCREMENTAL build, not a scratch one -- auto/upstream-baseline
  differs from the instrumented branch in 202 files and zero submodule gitlinks,
  so checking it out in the shared tree and rebuilding into the existing build dir
  takes ~9 min at -j4. git worktree prune first; a dead worktree still claims the
  branch.
* the baseline CANNOT make a profile (--create_profile_if_none is one of our 50
  commits) and the Sign In dialog cannot be typed into, so bootstrap the profile
  with the instrumented binary once and run the baseline against it.
* the Xbox content .header is NOT needed -- ContentManager::ListContent falls back
  to the directory name when ReadContentHeaderFile fails, checked in the source.
  The LOAD GAME screen then shows date, difficulty, flight time and clear ratio
  anyway, which proves those come out of savedata itself. No header was fabricated.

Trap paid for once: keep the file name xenia_canary. A copy saved as
xenia_canary.baseline-... survives every pkill -x in the toolkit and holds the
run-canary lockfile while ps -C xenia_canary shows nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 17:15:53 +00:00
Sylpheed RE agent
12d516729e tools: sample a candidate REMAINING OB address against the HUD in one run
ob_sample.py pairs a live read of a guest VA with a screenshot and a crop of the
HUD counter, and reads the word again AFTER the shot so a sample whose two reads
disagree can be thrown away instead of believed -- that race is what left the
first three-snapshot filter with zero survivors.

ob_session.sh is fly_session.sh plus that sampler, because the evidence this
needs is a transition, and an unattended craft is dead in about a minute. Its
header records that launch_mission.sh does not yet finish unattended on restored
state, so nobody discovers that halfway through a boot.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-23 17:15:36 +00:00
Sylpheed RE agent
a3f6ab617b tools+docs: find REMAINING OB in guest RAM, verified across unselected transitions
The mission's objective counter is a big-endian u32 at guest VA 0xbdb59668 in a
Stage 02 run on the upstream baseline. It was selected on an 18->24 transition
and then tracked 24->23->22 against the HUD on its own - four readings, two
changes it was not filtered on.

That last point is the whole discipline here, because the first attempt failed it.
An earlier differential over 19->18 also produced exactly one candidate,
0xbc22e83c, which matched the transition it was selected on and was still wrong:
read live it held 26 while the HUD showed 017. One matching transition is not
evidence.

A second trap is recorded too: a three-snapshot filter requiring 19 -> 19 -> 18
left ZERO survivors, because the value moves between the memory copy and the
screenshot that reads it. Filtering on the next DISTINCT value instead found the
counter on the first try.

ob_scan.py carries the method: scan one snapshot, then filter the candidate set
against live /dev/shm/xenia_memory_* at each new value, so only the first pass
needs a 4.8 GB copy.

Stated plainly as unsettled: the ADDRESS is from one run and cross-run stability
is untested, so the durable result is the method rather than the number. And what
the counter counts - whether every OB-badged entity is one of them, and whether
that badge is a flag in the entity object - is the follow-on the pilot actually
needs to CHOOSE targets rather than just know how many remain.
2026-08-19 19:01:24 +00:00
Sylpheed RE agent
09f189bf12 docs: a stock-upstream baseline runs Stage 02 crash-free
Built upstream/canary_experimental (a5a18f5c7, 82 commits ahead of our merge
base) clean, to answer whether the 0x82307128 mission crash is ours or theirs.

Two tooling gaps found building it: version.h is never generated (upstream's
xenia-build.py writes it, build-canary does not invoke it, and our tree only
builds because a stale copy from an old sylpheed-re build sits in the build dir),
and build-canary reports exit 0 on a FAILED build - only the missing binary gave
it away.

Pure upstream cannot be driven at all: --hid=file is ours and there is no uinput
here. So the honest baseline is upstream plus those three pad commits. That
baseline reproduced the LOST RESUME on its second boot - black screen, guest at
522% CPU, newest host thread at 00:00:00 CPU, zero kernel calls from its handle -
and upstream's threading_posix.cc still has the two-lock-scope publish. So pure
upstream is not a stable state; it carries a bug we had already fixed.

Adding ONLY the threading fix on top - still none of the RE instrumentation -
reaches Stage 02 FLIGHT: TIME 05:03.75, wingman Rhino 103, radio dialogue, and
REMAINING OB 020 with OB badges on targets. The run ended in a legitimate GAME
OVER (the unattended craft was shot down, as autopilot-memory-driven.md warns),
not a crash. Crash dumps for the whole run: 0, against 384 on the instrumented
branch at the briefing alone.

So the mission crash is not upstream's. Whether it is in our 50 commits or fixed
among upstream's 82 this run does not separate; that needs a bisect.

REMAINING OB is visible and moves (020->019->018->017) but is NOT found. A
three-snapshot BE-u32 filter left 0 survivors; a differential across widths gave
exactly one candidate, 0xbc22e83c, and it is REFUTED - read live it held 26 while
the HUD showed 017. The empty u32 filter is itself informative: the counter is
not a plain persistent big-endian word.
2026-08-19 18:44:23 +00:00
Sylpheed RE agent
003ba1012b tools: repair launch_mission.sh; Stage 02 now reaches the briefing and stops there
Finding REMAINING OB in RAM means getting to the escort stage, and
launch_mission.sh exists for exactly that. It carried three of the dead patterns
this session has been clearing: six vgamepad calls (the command no longer exists,
so the entire scripted route pressed nothing), the XUID E0300000EFBEA3D4 which
has no profile behind it and therefore opens the sign-in dialog that swallows
every keystroke, and the emulator's stdout going to /dev/null. wait_flight.sh had
a fourth, in the tap that skips the launch cinematic.

Fixed, the route works far past where it used to stop: boot -> title -> LOAD GAME
-> slot 01 -> READY ROOM -> TAKE OFF -> the Stage 02 briefing map, with ACROPOLIS
labelled on it.

And it stops there: 384 dumps at PC 0x82307128 and no flight HUD within 300s. The
briefing is still animating (two shots a minute apart differ by RMSE 2986), so
the emulator is not frozen - flight simply never starts.

One discriminator worth recording: the TUTORIAL reaches flight and has run
completely crash-free, while this path crashes during the briefing, which is
where the stage content loads. That is consistent with the cache-flush shape the
crash has always had, and points at WHAT is being loaded rather than at how far
the navigation gets.

REMAINING OB is therefore still not found - the mission was never entered.
2026-08-19 13:27:46 +00:00
Sylpheed RE agent
b7653216d6 docs: map where the autopilot's objective and action knowledge lives on the disc
The autopilot doc's own problem #2 is that the pilot ignores the mission
objective - its 300s run took no damage, killed one fighter, and watched
REMAINING OB RISE from 004 to 011. So this surveys where that knowledge is.

GP_TUTORIAL.pak yields the game's own list of what a player can do: BASIC
CONTROLS, HEADS-UP DISPLAY, RADAR, SUPPLY AND SPECIAL MOVES, RADIO ORDERS,
ADVANCED CONTROLS, with descriptions. Worth stating against what the pilot
actually does: pilot.py and navigator.py use move and attack only. Resupply,
special moves, radio orders to wingmen and advanced maneuvers are all unused, and
two of them bear directly on open problems - resupply is the untested RETIRE
mode, radio orders is an escort lever that flying skill does not substitute for.

tables.pak is a name-resolved config seam: 79 IDXD objects, 6275 distinct field
names, including DLG_MISSION_OBJECTIVE and DLG_STAGE_TITLE01..16 (so 16 stages).
GP_MAIN_GAME_<lang>.pak holds 131 weapon\ tables, 113 unit\ and 64 message\ -
the weapon ones being what problem #1 wants for real projectile speed and range.

But the objective TEXT is not text: DLG_MISSION_OBJECTIVE = pdscr099.prt, a
screen part, and the message tables resolve to face textures. All human-readable
text in this game is rendered sprite art, which is why the UI work has been
reading pixels all along. A per-mission objective table is an OCR job, not a
table dump - so the cheaper and more actionable route is the runtime REMAINING OB
counter, which is on the HUD and therefore in RAM.
2026-08-19 13:13:33 +00:00
Sylpheed RE agent
ef739ea80d docs: correct the empty ship capture - nothing is broken, there was no ship on screen
The previous commit framed the empty capture as "the 3D draws never reach
CaptureShipDrawForRE" and proposed a static comparison to find out why. The
comparison was done and it refutes the framing.

The discriminator for ship geometry is stride=24 prim=4 with a large vcount -
xbg7-mesh.md records a real one as stride=24 vcount=10891 indices=18 prim=4. Not
"positions outside the screen rectangle", which is what I used and is a bad test:
a UI sprite placed at (137,308) passes it.

By the correct test this run's capture has exactly one prim=4 draw, a 6-vertex
full-screen quad, and the 2.9 MB file I was comparing against has NO prim=4 draws
at all. That file is a UI capture: 1303 of its 1582 draws are stride=24 prim=13,
which two other docs in this corpus already identify as the UI sprite shader. The
"1300 3D draws" it appeared to contain were UI sprite coordinates counted by the
bad test.

So the capture recorded exactly what was on screen. The tutorial's opening is an
empty starfield, the player's own ship and a HUD - no capital ship. And
ship-placement-runtime-capture.md has always stated the procedure: play into the
mission, frame the ship side-on, press F10.

What remains is therefore not a code question but a gameplay one: reach a real
mission and frame a capital ship. Worth stating that the original capture was
taken interactively on HW Vulkan, and this container runs lavapipe.
2026-08-19 13:02:48 +00:00
Sylpheed RE agent
9b17506c1f tools+docs: a crash-free mission run, a capture that caught no geometry, and a correction
ship_capture_window.sh polls for the flight screen and presses F10 the moment it
appears rather than after a fixed sleep. One run gave three results.

First, the mission ran with ZERO crashes through t+152s - the first clean mission
run, where the three before it ended at 13243 / 11898 / 11497 - and it renders
and plays: player ship, starfield, full HUD, no dialog.

Second, the capture armed and wrote its file, so the mechanism works in-mission.

Third, and against expectation, the file holds NO ship geometry. 329KB, 181
deduped draws: 180 of them share a single vertex shader, all stride=28 vcount=3
prim=8 at full-screen coordinates, plus one full-screen quad, and not one draw
has positions outside the 1280x720 rectangle. The budget is not the limit -
kShipCaptureBudget is 8000 and only 181 distinct (vbase, WVP) pairs were seen -
and the scene was definitely drawing. The known-good capture from an earlier
session is 2.9 MB.

Fourth, a correction to the previous commit. It said the cache is "REFUTED as the
cure". Too strong: this run used the IDENTICAL complete cache as tut4 and
produced 0 crashes against tut4's 11497. What four runs support is that a
complete cache is not SUFFICIENT to prevent the storm and that run-to-run
variance dominates a 3-run comparison - not that the cache does nothing.

Next step is static: compare this capture's shape against the known-good one to
find why the 3D draws never reach CaptureShipDrawForRE.
2026-08-19 12:56:35 +00:00
Sylpheed RE agent
e85f6ac955 docs: the cache is refuted as the mission-crash cure, and a usable window found
This page's own reproduction - move the cache aside, get a throw; restore it, get
none - made "the cache is incomplete" the obvious explanation for the mission
crash. Three tutorial runs say otherwise.

  tut2  subdir 6 MISSING              641 crashes by t+24s   13243 total
  tut3  missing -> gained 6 in-run      2 crashes to t+80s   11898 total
  tut4  6 present from the start        2 crashes to t+56s   11497 total

The missing entry was real: \aab216c3\6 was requested and absent, and the game
wrote it during tut3 (11 -> 12 files). tut4 then ran with a complete cache and
stormed anyway. Every dump across all three runs is PC 0x82307128, one address,
no others. So the cache may gate the boot-time throw this page documents; it does
not gate the mission one.

What the runs did give is an opening. Both post-cache runs show the same shape -
exactly 2 crashes, then nothing, for 56-80 seconds of rendering, advancing
mission - where the first run was already at 641 by t+24s. That is where the
second capital-ship capture has to happen: F10 armed INSIDE the window. Not a
guarantee, since the storm began at t+24s, ~t+96s and ~t+56s across the three
runs, but two of three offered most of a minute.
2026-08-19 12:37:50 +00:00
Sylpheed RE agent
f9e6209308 docs: withdraw the resume-refused lead - kHighFrequency waits are simply unlogged
The 1663 refused resumes on thread F80001D8 looked like a second lost resume of
the kind fixed earlier today. They are not.

That thread DID execute - the log carries its XThread::Execute line, guest entry
82FFE6C0 - and then 137000 lines of silence. The silence proves nothing:
KeWaitForSingleObject and NtWaitForSingleObjectEx are declared kBlocking,
kHighFrequency, and PrintKernelCall skips every kHighFrequency export unless
--log_high_frequency_kernel_calls=true, which defaults to false. A thread parked
in a wait is invisible in these logs.

So the boring reading is the right one: the guest kicks a worker that is blocked
ON AN OBJECT, and Resume returns false because the thread is not SUSPENDED. That
is what a refused resume means, exactly as suspected before the count made it
look interesting.

Method note kept in the doc, because it is the reason one conclusion survives and
the other does not: the title-loader finding rested on 00:00:00 host CPU time
from ps -L and on the fix changing behaviour 5/5 against 1/5 - not on log
silence. In this codebase log silence alone is never sufficient evidence that a
thread is idle.

Also records a grep error of mine that briefly produced "zero kernel calls,
ever": the pattern ^[dikwF!]> missed the K> kernel prefix. Caught before it was
written down, and it changed the conclusion.
2026-08-19 12:12:40 +00:00
Sylpheed RE agent
402ee1dce4 docs: the mission path is reachable now - it freezes there instead of dying earlier
The backlog has said a second capital-ship capture needs the 0x82307128 crash
dealt with first, because the run died at SELECT DATA before any mission
existed. That is no longer where it stops. With the Canary threading fix making
the menu dependable, tutorial_launch.sh drives boot -> title -> menu -> TUTORIAL
and the mission LOADS AND RENDERS: flight HUD, "Go to the box on your screen",
warship counters, controller diagram.

It then freezes. 13243 crash dumps, every one at PC 0x82307128, guest thread 9,
Access Violation read at 0x10000000C, preceded by exactly ONE
RtlRaiseException(E06D7363) right after a HostPathDevice::ResolvePath() with
empty arguments - the cache-flush shape. Two screenshots 6s apart are identical
(RMSE 0), no new dumps accumulate, and the process still burns 400% CPU.

So: reachable, renders, unusable for a capture. The blocker moved rather than
lifted, which is worth recording precisely because the entry's stated reason is
now wrong.

One new lead, deliberately not claimed as a cause: the resume-refused diagnostic
added with the threading fix fires 1671 times on this path and 1663 of them are
the SAME thread, F80001D8, with 152 before the first crash - against about 7 on
the menu path. Resuming a non-suspended thread legitimately returns false and
"Resume to kick a worker" would produce exactly this, so it is a lead, not a bug.
It is cheap to settle: check whether that thread makes kernel calls between
refusals.

Cheapest next test named: this page already shows a COMPLETE on-disc cache
produces no throw at all, and the cache is 40MB with .partial/.cold-rebuilt
leftovers beside it. One boot to warm, one to re-test.
2026-08-19 12:05:23 +00:00
Sylpheed RE agent
ebc4e08b89 formats: two more measured paint orders, and the first independent confirmation
The three orders the derived rule was built from all live in GP_TITLE.pak, so
they cannot confirm it - the rule was fitted to them. These two are from
GP_SAVE_LOAD.pak, read off the running game now that the Canary threading fix
makes the main menu dependable.

The 9-element slot-list header composites EXACTLY as the sort predicts, on all 6
instances of it, and nothing about this screen was fed into the rule:

    measured  7 8 0 1 2 3 4 5 6
    derived   7 8 0 1 2 3 4 5 6

including TWO tied groups (0xb102 x2 and 0xb210 x5) that both come out in
declaration order, and the unkeyed pfeff00.prm fade quad last.

The 13-element save/load frame differs in exactly the two open questions and no
new ones: two unkeyed pfbase.tbm backgrounds paint FIRST where the sort puts the
keyless last - the splash's palogo_eff0.prm behaviour in a different file type,
so implied_layer_key now covers it - and the 0xb100 group of four paints
10,11,8,12 where declaration order is 8,10,11,12.

That second point is a SECOND screen with a mis-ordered tie, which is what the
question needed, and it immediately kills a candidate: 10 and 11 are kind=0x2002
while 8 and 12 are 0x0000, so "descending kind then declaration index"
reproduces 10,11,8,12 exactly - and then fails both title groups, where every
element of 0x8083 is kind 0 and where 0x80a0 would predict 2,3,4,5,0,1,7 against
a measured 0,2,4,7,1,3,5. Seven candidates refuted now.

16 disc tests green.
2026-08-19 11:46:09 +00:00
Sylpheed RE agent
714c74565f docs+tools: the threading fix measured 5/5 against 1/5, and a driver lock
resume_reliability.sh counts, per boot, whether the title's loader thread became
the CALLER of a kernel call. Counting boots where the press reached the title:
before the fix 1 of 5 ran the loader; after it, 5 of 5. Three of the post-fix
boots were driven far enough to classify the screen and all three reached the
MAIN MENU, each with the same shape - title at 216-252s, 40 kernel calls by the
loader, 6 ResolvePath reads. The failing boots before had zero of both, every
time.

Caveat kept in the doc: pre-fix runs were interleaved with compiles and a race is
load-sensitive, so the counts corroborate the mechanism rather than carry it.

Also fixes a self-inflicted confound. Two runs came back 'title_at_s=none' with
the emulator log ending in 'Killed', which I first read as the title detector
being too strict. It was not: an earlier copy of the script was still running
because 'pkill -f resume_reliability.sh' had matched the shell running it rather
than the script, and each of that copy's iterations kills the emulator BY NAME -
outside run-canary's lockfile. The harness now takes its own lockfile and refuses
a second copy, and the note records that pkill -f matches your own shell.
2026-08-19 11:12:46 +00:00
Sylpheed RE agent
0d8d2ab758 tools: a reliability harness, and a title test that also requires a static frame
resume_reliability.sh boots N times and records, per boot, whether the title's
loader thread was created and whether it ever became the CALLER of a kernel call.
That is the thing the threading fix targets, and it is binary; "did the menu
appear" is a worse test because it needs the game to get further and the attract
loop confuses it.

Two defects fixed on the way, both found by the harness disagreeing with itself:

  * is_title.py's glyph count alone still fired at 113s and 173s, during the
    intro movie, and the single press was wasted there. Measured, not guessed:
    on those runs NO loader thread was created at all, and the title handler
    always creates one - even on the boots where it then fails to run. The title
    check now also requires the frame to be STATIC, reusing the RMSE between the
    two grabs 0.6s apart that the movie branch already computes, and raises the
    glyph threshold to 800 (a real title measures ~1450-1520, the SQUARE ENIX
    logo 0).

  * `grep -ac ... || echo 0` corrupted every CSV field it guarded: grep exits 1
    on zero matches, so the fallback APPENDED a second value and the field came
    out "0 0".

Retrospective before/after from the logs already on disc, using ResolvePath after
the resume (logged at every verbosity, so it is comparable across all runs):
before the fix 1 of 5 boots that created a loader thread went on to load assets;
after it, 2 of 2. n is small and a proper count is running.
2026-08-19 10:59:35 +00:00
Sylpheed RE agent
ae3f37d3ec tools+docs: locate the lost resume, and a title test that is neither too narrow nor too loose
The stalled loader thread is a lost wakeup in Xenia's POSIX threading, fixed on
the canary branch as a60fe7d11 and written up here. A thread created suspended
publishes state_ and suspend_count_ in two separate lock scopes, and Resume()
waits only for state_ before testing suspend_count_ == 0 - so a resumer in the
gap drops the resume and the thread waits forever. The Linux XThread::Resume
discards that false, which is why the guest saw success.

On the first clean boot after the fix the loader thread is the CALLER on 20
kernel-call lines and issues 4 ResolvePath reads. Every failed boot before it had
exactly zero of both.

Stated plainly as not shown: that boots now reach the menu RELIABLY. One post-fix
boot, and it is confounded by the harness.

Which is the second half. skip_intro.sh's title test has now been wrong twice in
opposite directions: originally one absolute pixel (625,618) - a 1280x720
coordinate against the 1279x675 game surface, so it read the copyright line and
timed out with the title on screen - and then my replacement, screen_id.py, which
is too loose and called the SQUARE ENIX publisher logo "title" 151s into a boot,
spending the script's single press there. is_title.py now counts the green (A)
glyph over the whole frame: geometry-independent and specific, measured at 0
pixels on the logo and 1520 on a real title.
2026-08-19 10:41:43 +00:00
Sylpheed RE agent
24c1c940e6 tools+docs: the title's loader thread is created and never runs
Two findings, the second only visible because of the first.

1. The harness has always had kernel logging switched off. log_mask DISABLES
   categories (Kernel=1, Apu=2, Cpu=4, Gpu=8), so the long-standing --log_mask=13
   meant Kernel+Cpu+Gpu off; kernel calls also log at Debug while log_level
   defaults to Info. Seeing one needs BOTH LOG_MASK=12 and LOG_LEVEL=3, and no
   boot log this project has taken ever contained a kernel call. boot_menu.sh now
   takes LOG_MASK / LOG_LEVEL / EXTRA_FLAGS. A whole boot at Debug with Kernel on
   is 23 MB, so the default was costing far more than it saved.

2. With that on, a captured failure shows the (A) handler doing everything right:

     XamUserGetXUID(0, 7, ...)
     NtCreateEvent(...)
     ExCreateThread(..., entry=821748F0, ..., 00000001)
     ExCreateThread Active: Thread Initially Suspended,
     XThreadF80000CC (1F) Stack: 70880000-70900000
     NtResumeThread(F80000CC, ...)

   and the thread then never executing. Measured two independent ways: it makes
   ZERO kernel calls - it appears 13 times in the log and every one is as an
   ARGUMENT, never as the calling thread, while five other threads make 31905
   calls after the resume - and its host thread has 00:00:00 CPU time while the
   process sits at 546% and has burned 37 minutes of CPU in 6:46 wall.

   A spinning thread burns CPU. This one has not run at all.

So the chain is: press delivered -> handler runs -> thread created suspended ->
resumed -> never scheduled. Input, the cache-flush crash and the game's own logic
are all excluded. A lost resume is a race, which is the first explanation that
fits the ~1-in-3 success rate.

Not settled: where the resume is lost, and no successful boot has been captured
with kernel logging to compare against.
2026-08-19 10:19:06 +00:00
Sylpheed RE agent
723022f15c docs: the title does act on (A) - the loader thread it spawns is what stalls
"The title screen ignores (A)" is withdrawn. First-divergence across three boots
of the same binary says otherwise.

A slot-(1F) guest thread is spawned BY the press: exactly once per run,
immediately after the keydown, same stack base 70880000-70900000 in both runs
that got one, and never at all in the run that never accepted a press - which
rules out a periodic worker starting around the same time.

  prm6, reached the menu:      (A) at line 6498, (1F) at 6500, 6 ResolvePath after
  opt2, stuck on the title:    (A) at line 1287, (1F) at 1288, 0 ResolvePath after
  opt,  stalled before title:  no (A) ever, no (1F) thread at all

In the successful run the loader immediately reads six paths out of the on-disc
cache and the menu appears. In the failed run the same thread starts and performs
no file I/O ever again. Total ResolvePath for the three boots is 90/84/78 - the
successful run's extra six are exactly the ones after the press, so the boots are
otherwise identical in I/O.

Refuted as the cause: the cache-flush std::out_of_range. All four of today's runs
have zero GUEST-THROW, zero CRASH DUMP and zero Access Violation, and the guest
stays alive throughout with its keystroke-poll counter climbing past 15000.

Next probe is neither input nor the crash: what the (1F) thread waits on.
2026-08-19 10:03:41 +00:00
Sylpheed RE agent
63011eb9ed docs: a fourth paint order attempted; the title is intermittent, not input-bound
The tie-break needs a fourth measured screen with a tied group, and OPTIONS is
three d-pad steps from the main menu, avoiding the NEW GAME hang. It was not
reached.

Advancing past the title is intermittent: 1 success in 3 attempts today with the
same binary, profile and procedure.

Attempt 1 stalled before the title existed. The guest was alive - RSS steady at
598MB and the keystroke poll counter climbing past 15000 - but the screen was
black and screen_children.py found ONLY the splash object resident. So that
failure is not a presentation problem: the game never constructed the title.
Being able to separate "stalled before building the screen" from "built but not
presenting" is new, and it is what the reader is for.

Attempt 2 reached the title at 265s and refused four presses - (A) at 0.25s,
0.25s and 0.5s, START at 0.3s - every one of them logged as DELIVERED to the
guest. Third independent confirmation that input delivery is fine and the game
simply does not act on the press. Waiting longer does not help either: presses
were spread from 10s to about 4 minutes after the title appeared, while the
earlier successful run's press landed within seconds.

Records the log as evidence and states the next probe: it has to be guest-side
now, on whatever consumes the keystroke, since the input path is excluded.
2026-08-19 09:56:32 +00:00
Sylpheed RE agent
2efc0b7c59 docs: backlog — tie-break refuted six ways; residual is one blend on one screen 2026-08-19 09:34:29 +00:00
Sylpheed RE agent
a15e0ad39c formats: the tie-break, refuted six ways and measured down to the pixel
With the layer key and the primitives' implied keys in place, the tie-break -
how the game orders elements sharing a key - is all that is left between the
derived order and ground truth. Three measured screens now constrain it.

On the menu and the splash every tied group comes out in declaration order,
which is what the stable sort already gives. The title is the only screen that
discriminates, and nothing predicts it: 0x8083 x5 paints eff1, eff2, eff5, eff3,
eff4, and 0x80a0 x7 paints logo1 x3, tm, logo2 x3.

Refuted: declaration order; RATC child order; first keyframe time (52, 56, 62,
58, 60 - the measured order is not sorted by them); resting keyframe time;
resting X or Y (938, 938, 64, 788, 447); and T8aD header words +00, +04, +0c and
+10, which are either identical within a group or unsorted.

Child order is worth its own line: a strict improvement over declaration order
(7 misplaced positions on the title instead of 9, and it recovers the logo
grouping) and exactly right on the other two screens. NOT adopted, because on the
only screen that can tell them apart it is still wrong.

Adds a test that measures what the residual costs instead of assuming it. Of the
3 disagreeing pairs of drawn elements across all three screens, all 3 have
overlapping bounding boxes and 2 share opaque pixels: ptlogo_back2eff5 against
eff3 (22568 px) and eff4 (32395 px). The third pair, ptlogo2 vs ptlogo_tm,
overlaps by two columns and shares NO opaque pixel - the wordmark is transparent
there. A bounding-box test called that a defect; reading the alpha says it is
not, which is why the test reads pixels. The set is pinned, so a change that
makes it worse fails.

15 disc tests green.
2026-08-19 09:34:10 +00:00
Sylpheed RE agent
6c93d43446 docs: backlog — primitive layers tabled; the tie-break is now the only gap 2026-08-19 09:16:13 +00:00
Sylpheed RE agent
610b21c434 formats: a primitive layer is not in the file - measure it, table it, prove it
Two places checked for the key a .prm element sorts by, both empty:

  * the declaration entry's four unread words are constant across every element
    of all three measured screens (+28=0, +36=0xffffffff, +56=0, and +44 is a
    button ordinal 1-5, 0xffffffff elsewhere);
  * the bundle carries NO data for a primitive at all - the menu build declares
    pteff00.prm, pteff02.prm and pteff05.t32 and has zero RATC children for any
    of them, its 34 children being 21 T8aD sprites and 13 .rat records.

So the layer comes from the game code. But it is consistent, which is what makes
a per-name table a measurement rather than a fudge. Bracketing each unkeyed
element between the keys of its measured neighbours: pteff02.prm falls in
(0x8010, 0x8040) on BOTH screens it appears on, pteff00.prm is past the maximum
on both, and palogo_eff0.prm is below the minimum on the splash.

implied_layer_key records exactly those and nothing else; an unlisted primitive
keeps u32::MAX and still sorts last. With it, derived_paint_order produces the
same layer-key sequence as the order read off the running game on all three
measured screens - primitives included - and matches element-for-element on four
of the five bundle instances. The fifth is the title, differing only inside its
tied groups, which is a separate open question.

This does not make include_primitives safe by default: the 36 builds that come
out one colour are wiped by pzeff00.prm and pceff00.prm, never measured, hence
not in the table.

14 disc tests green.
2026-08-19 09:15:55 +00:00
Sylpheed RE agent
a1624af24e docs: input was never the blocker; the third permutation came from the main menu
The game receives the (A) press - measured, vk=5800 flags=0001, out of ~6500
polls. Input delivery was never broken; four harness bugs were, and with those
fixed the boot goes title -> one (A) -> main menu.

GP_DIALOG DIFFICULTY is still unreached (NEW GAME hits the standing black-screen
hang), but the main menu carries two primitives, which is what the question
needed. Records the slip in the instrumentation too: logging on XSUCCEEDED
counts X_ERROR_EMPTY as success and buried the two real keystrokes under 6499
empty polls.

Next step is now STATIC: a primitive's position is stable by role across three
measured screens but nothing in the file predicts it, so dump the full 60-byte
declaration entries - four words unread - and look for a word that sorts like
the measured orders.
2026-08-19 08:57:01 +00:00
Sylpheed RE agent
23565ec5ec formats: a third measured paint order — the main menu, with two primitives
Read off the running game with screen_children.py and identified by pivot
signature as GP_TITLE.pak ratc-index 8, the NEW GAME / LOAD GAME / TUTORIAL /
OPTIONS / EXTRAS screen:

  paint order: 1 3 4 2 5 8 9 6 7 15 10 11 12 13 14 0

It is the first measured screen carrying TWO primitives, and they land in
different places, which is the point. pteff02.prm (the 25% dim) paints 4th,
beneath the whole UI; pteff00.prm (the transition fade, resting transparent)
paints last. Both match their positions on the title screen exactly. So a
primitive's place is per-element and stable by role - backdrop first, dim at
slot 4, fade last - and there are now three permutations to test a derivation
against rather than two.

Wired into measured_paint_order, keyed by element names so both language builds
get it. The English build composited with --primitives edge-correlates at 0.9591
at shift (0,0) against a framebuffer capture taken in the same session - a third
screen confirming paint order, resting pose, fade alpha and primitives at once,
against a capture this project had not seen before.

13 disc tests green.
2026-08-19 08:56:21 +00:00
Sylpheed RE agent
20b0edc2c2 docs: backlog — the keystroke swallow is fixed and refuted as the cause 2026-08-19 08:21:27 +00:00
Sylpheed RE agent
e3ad6ad28c tools: fix four boot-harness bugs; find the swallowed-keystroke cause and refute it
Chasing why (A) does not advance the title turned up four harness bugs, one real
root cause, and a refutation of that root cause as the explanation.

FIXED, each verified:
  * boot_menu.sh sent the emulator stdout to /dev/null, which is why none of the
    rest was visible. It now keeps a log and prints its path.
  * skip_intro.sh probed pixel (625,618) for the green (A) glyph - a 1280x720
    coordinate. screenshot returns the 1279x675 game surface, where that point
    is on the copyright line and reads (8,17,31). Symptom: a 600s TIMEOUT with
    the title on screen. Now classified with screen_id.py; next boot printed
    "TITLE at 239s -> A".
  * skip_intro.sh no longer taps through the movies. Making that press real (the
    vgamepad fix) was a REGRESSION: the boot then reached the title in 90s and
    that title accepted nothing. menu_draw_capture.sh records the same finding
    independently.
  * boot_menu.sh signs in whichever profile exists instead of a hard-coded XUID.

ROOT CAUSE FOUND: naming a XUID with no profile behind it opens a sign-in
dialog, xam_dialogs_shown_ goes to 1, and IsUIActive() then discards every
XamInputGetKeystrokeEx. Measured 8,388,601 swallowed calls in one boot with the
pad log showing presses arriving normally throughout. Only B13EBABEBABEBABE
exists on disc; the script asked for E0300000EFBEA3D4. Signing in the real
profile takes the swallow count 8,388,601 -> 0.

AND REFUTED: that is not why the title is stuck. With a valid profile and zero
swallowed keystrokes, a single (A) on the title that ends the boot still does
nothing. The dialog bug was real and worth fixing; it is not the explanation.

Next probe is specific: --log_mask=13 records no kernel calls, so it is not even
known whether the game polls at the title. A log on the SUCCESSFUL
XamInputGetKeystrokeEx return, mirroring the one already on the swallow path,
answers it in one boot.
2026-08-19 08:21:08 +00:00
Sylpheed RE agent
99081a6198 docs: the boot log confirms the vgamepad failure verbatim
'skip_intro.sh: line 44: vgamepad: command not found' right after 'movie -> skip
A'. Every '-> skip A' and 'TITLE -> A' this harness has printed pressed nothing,
which is also why the intro movie ran to 434s: it was never skipped.
2026-08-19 07:49:37 +00:00
Sylpheed RE agent
efee309d8f docs+tools: screen_children.py, validated; the third paint order is blocked on input
screen_children.py walks every resident screen object (vtable 0x820b30b4), reads
the element array at +0x08 and the child array at +0x30, and prints the paint
permutation with each pivot. Validated in one run off the live title: it
reproduces BOTH previously measured orders character-for-character - the splash
[0,2,4,6,1,3,5] and the 24-element title permutation - so the next screen it is
pointed at can be trusted.

The screen that matters was not reached. GP_DIALOG DIFFICULTY carries exactly
one primitive and would say whether the game paints it first or last, which is
the bit that decides where primitives belong. But (A) does not advance the title
screen at holds of 0.10, 0.25 or 0.40s, and neither does START, with the pad
verified end to end: pad.py writes /tmp/xenia_pad.txt, the emulator runs with
--hid=file --pad_file pointing at it, and the mtime updates on every press. The
title is unambiguously the interactive one - PRESS (A) BUTTON is rendered.

Recorded as a blocker rather than worked around. Next step is specific: boot with
the emulator stdout kept and read the [RE-INPUT] IsUIActive log already present
on the canary branch, which would say whether the keystroke is being swallowed
emulator-side.
2026-08-19 07:48:58 +00:00
Sylpheed RE agent
01580d294e tools: two dead flags that made every emulator boot script fail silently
Trying to read a third paint order off the running game turned up two bugs in
the capture harness, both of which fail in ways that look like the game
misbehaving rather than the script being wrong.

1. `--audio` is not a cvar in this tree, and eight boot scripts passed it.
   Xenia calls ShowSimpleMessageBox from ParseLaunchArguments, BEFORE logging is
   initialised, so the symptom is a 10x10 window, no log, no guest memory and a
   dialog that blocks on XIfEvent forever - i.e. a hang deep in the emulator.
   run-canary`s own header documents this exact trap; the scripts predate it.
   Removed from all eight.

2. `vgamepad` no longer exists - the uinput pad was replaced by the --hid=file
   driver and pad.py - but skip_intro.sh still called it. The script runs
   without `set -e`, so the call failed silently and the title branch pressed
   nothing while still exiting 0. A caller was told "TITLE -> A" with the game
   sitting on the title screen. It now presses through pad.py and exits 6 if
   that fails.

The first bug is fixed and verified: the boot now reaches the title screen with
PRESS (A) BUTTON. The second is fixed but does NOT unblock the title - see the
next commit.
2026-08-19 07:48:16 +00:00
Sylpheed RE agent
71d9b73afe docs: backlog — primitives drawn opt-in; their paint order needs a runtime read 2026-08-19 07:21:48 +00:00
Sylpheed RE agent
eb86142e4f formats/cli: draw the .prm primitives, opt-in, because their paint order is unsolved
fill_quad composites an untextured primitive as a solid rectangle of the
keyframes fade colour, pivot x 2 in size, placed and scaled exactly as a sprite
is. Behind ComposeOptions::include_primitives and screen render --primitives.

On the title screen, whose paint order is ground truth, it is measurably right:
mean luminance 76.30 -> 63.72 against the captures 64.58, i.e. from +18% to
-1.3%, and mean absolute difference 16.07 -> 13.08. The background was ~40% too
bright; pteff02.prm, a 25% black dim, was what was missing. The wordmark is not
dimmed by it because the measured order paints that quad at slot 4, beneath the
logo. Edge correlation moves 0.9538 -> 0.9480, which is not informative here: a
uniform dim scales gradients uniformly so a normalised edge score barely sees it.

OFF BY DEFAULT, and that is the finding. A primitive has no T8aD header, so no
layer key, and derived_paint_order sorts the keyless to the end. GP_DIALOGs
pzeff00.prm is a single keyframe of opaque black at full screen; painted last it
wipes the build. Of the 125 builds that draw a visible primitive, 36 come out
>99% one colour with the derived order.

No constant default works, because the two screens read off the running game
disagree: the splash paints its primitive FIRST (the black backdrop) while the
title paints one at slot 4 and another LAST (the fade-out). Declaration order
fails the title too. A disc test measures the damage rather than asserting the
feature works, so the number moves when the ordering is solved.

Also records a false alarm worth keeping: a first pass reported 36 GP_DIALOG
builds at "100% black", which was a crude near-black pixel threshold and not a
black screen - those dialogs are dimmed 50% and perfectly legible. The genuinely
wiped builds are a different set.
2026-08-19 07:21:32 +00:00
Sylpheed RE agent
6b65f7ea91 docs: backlog — fade alpha applied; drawing the .prm quads is unblocked 2026-08-19 06:56:56 +00:00
Sylpheed RE agent
7fbdcc078c formats: apply the keyframe fade alpha, and fix the resting rule it exposed
blit modulated by tint only, which is 0xffffffff on essentially every keyframe,
so the fade word was decoded, stored and then discarded. Applying it as an ARGB
modulate takes the title composite's edge correlation against the framebuffer
capture of the running game from 0.4597 to 0.9538, both at zero shift. The white
wordmark with its blue outline, the trademark, the copyright and the orange
exploding planet all appear -- the planet because a full-screen blue effect that
rests at alpha 0 had been painting over it at full opacity.

ARGB is measured: across a fade-in the high byte walks 0x00 -> 0x80 -> 0xc0 ->
0xe0 -> 0xff while the low three stay ffffff, and the low 24 bits are 0xffffff on
5276 of the disc's 5453 resting keyframes.

A modulate can only remove pixels, so the risk is a blank screen. Measured: no-op
on 4060 of 5200 sprite elements, partial on 453, hides 687 -- transient HUD
indicators that should not be lit at rest -- and NO build is left with nothing
visible. A disc test asserts that, and that the no-op share stays high, which
also guards the resting rule against drifting onto ramp frames.

Both changes are in one commit because the second is invisible without the first.
Applying fade erased the word PAUSE, which the running-game capture plainly
shows: pgptitle.rat has three runs of two identical keyframes -- pre-roll, hold,
exit -- and the 'later run wins' tie-break grabbed the exit. A group carries the
screen's entry animation AND its exit, so a run ending on the last keyframe is
now excluded unless it is the only one. The title correlation is unchanged.
2026-08-19 06:56:38 +00:00
Sylpheed RE agent
2d2c281805 docs: backlog — resting pose settled; the unapplied fade alpha is next 2026-08-19 06:41:41 +00:00
Sylpheed RE agent
b5c44b7c2a formats: the resting pose is the hold, not the longest dwell
Element::rest() picked the keyframe with the largest gap to the next keyframe's
time. That reads a keyframe as a value held until the next one; it is the start
of a ramp toward it. A long gap after keyframe k means the screen spends that
time arriving at k+1, so the settled pose is at the far end of the gap.

The title wordmark zooms in over five frames and holds at (184,193) at 100% from
t=251 to t=264. The old rule picked the frame before the long gap: (179,186) at
101%, still mid-zoom.

Measured against the framebuffer capture of the running title screen, which is a
1:1 crop so frame coordinates map directly (confirmed: the copyright line lands
on row 669 in the capture and in both composites). Edge-correlated over the
wordmark box:

    plateau (landed)      best 0.4597 at shift (0,0)
    longest dwell (old)   best 0.1511 at shift (+3,+8), 0.1268 at (0,0)

The old composite scores 3x lower and only peaks after being moved, by about the
(-5,-7) that picking kf4 instead of kf5 predicts.

It also fixes six title elements the old rule rested at alpha 0x00 where the
capture plainly shows them, and pteff00.prm - the full-screen fade quad painted
last - which rested at opaque black. That was the blocker on .prm compositing.

Adds tools/re-capture/align_to_capture.py, which is how this was scored, and
turns the .prm test that deliberately asserted the old defect into a guard on
the fix.

Not settled and now the next item: compose ignores the keyframe fade alpha
entirely (blit modulates by tint only), which is why choosing the wrong keyframe
was invisible until now.
2026-08-19 06:41:24 +00:00
Sylpheed RE agent
eb61368d23 docs: backlog — .prm decoded; the resting rule is the new blocker 2026-08-19 06:30:23 +00:00
Sylpheed RE agent
393e549752 formats: settle what .prm elements are — untextured quads marked by kind bit 0x10
369 .prm elements exist in the disc's screen builds and every composite is
missing them. Swept statically:

  * none of the 369 has a RATC child of its own name — no payload, nothing to
    draw, so .prm is a primitive and not a sprite;
  * kind & 0x10 and a .prm name agree with ZERO exceptions in either direction
    over all 965 builds, so the format marks primitives as a decoded field and a
    port need not parse filenames;
  * 361 of 369 are exactly 1280x720 at scale 100% in the corner, and their
    keyframe 'fade' ARGB is overwhelmingly black at some alpha — these are the
    fade-to-black, dim-behind-menu and flash layers, i.e. the PRMD dim-quad the
    compositor's backdrop has been standing in for.

Refuted before believing: drawing them at Element::rest() is wrong. The title's
pteff00.prm is opaque -> transparent -> transparent -> opaque, a transition whose
resting pose is the transparent plateau; rest() picks by longest dwell and lands
on the opaque endpoint, which is painted LAST on that screen and would black out
the title. A test asserts that wrong answer deliberately so that fixing the
resting rule fails it and leads to the note.

No compositing change: the resting rule is not .prm-specific and has to be A/B'd
against the title framebuffer capture first.
2026-08-19 06:30:05 +00:00
Sylpheed RE agent
a87d5b5255 docs: backlog — two items solved, four opened by them 2026-08-19 06:04:10 +00:00
Sylpheed RE agent
a1c370e810 cli/formats: let the screen commands reach bundles with no .rat child
The developer-logo splash declares its sprites directly and has no .rat layout
child, so is_build rejected it and no screen command could render it — despite
it being one of only two screens whose paint order has been read off the running
game, and the one where the layer key explains the whole permutation.

Adds ui_layout::is_composable (a declaration table plus at least one element
resolving to a T8aD the bundle carries) and an opt-in --all on screen
list/info/render. Measured on the disc: 2859 RATC bundles, 965 pass is_build,
2751 pass is_composable, and 0 pass is_build without passing it — a strict
superset. It is opt-in because the 1786 extra bundles are mostly two-element
fragments (a button and its glow), and because widening the default would
renumber --build for every pak, invalidating the build indices the corpus's
notes cite by number.

The splash now renders 6/7 elements, painting its glows first in the order
measured off the game; a disc test pins that order.
2026-08-19 06:03:52 +00:00
Sylpheed RE agent
b28763db0b docs: record the _eff-vs-focused-state confusion and both refuted rules 2026-08-19 05:57:38 +00:00
Sylpheed RE agent
2708abb482 formats: a focused-state record needs the element it is the focused state of
`compose` skips focused-state records by default, so whatever that flag matches
vanishes from every composite. It matched a trailing `f` in the name and nothing
else, which is not the convention — it is a letter.

Measured across the disc: 2458 elements match the suffix and only 54 of them
have the base element they would be the focused version of; all 54 are
`pgmenu_btnNNf.t32`. The other 2404, spread over 864 bundles, are `_eff` glow
layers whose names end in the same letter — `pb_name_eff.t32` alone accounts for
1122 of them, and `palogo_gamearts_eff.t32` is one the draw capture shows the
game painting, before its logo, with its own layer key.

Requiring the pair recovers 587 glows in the composable builds. GP_OPTIONS is
the plainest case: without `pbmwindow_eff.t32` the panel had no window at all,
just two floating brackets (captures/ui-layout/options-with{out,}-its-glow-layer.png).

Two other candidate rules were tried and refuted first. The `opt` link is not a
focus link: 221 elements are opt-targets, only 2 of them suffix-match, and the
targets include `pjnet_bg.rat` and `pv_loading_loop1.rat`. And the suffix alone,
as shown above, is unrelated to anything.
2026-08-19 05:57:12 +00:00
Sylpheed RE agent
fb4f035f08 docs: record the paint-order key as landed, with what it left unsettled 2026-08-19 05:43:29 +00:00
Sylpheed RE agent
deff5a4d6d formats: check the derived paint order against the screens already verified
The layer-key order was adopted from two measured screens and then applied to
every build on the disc, so it owed a regression check against the screens the
corpus had already validated against the running game.

Rendered the tutorial PAUSE menu and the title main menu both ways and diffed:
3.8 % and 1.1 % of pixels differ, max delta 45/255 and 34/255, and the two
renders are indistinguishable in layout — the change is confined to blends where
translucent sprites overlap. No regression, but which order is more faithful on
those two screens is unsettled and recorded as such.

Adds a corpus-wide test asserting every composite's draw list is strictly
increasing in (layer key, declaration index), streaming one pak at a time so it
does not OOM alongside the other whole-disc tests. It reports the rule's reach:
341 of 965 builds are reordered, and it fails if that share collapses.
2026-08-19 05:43:08 +00:00
Sylpheed RE agent
d9ae42dd55 formats: derive the paint order from the sprite layer key
compose now sorts elements by the word at +0x08 of their sprite's T8aD header
instead of painting in declaration order, for every build except the two whose
measured order is hard-coded. That word is non-decreasing in the order the game
actually paints both measured screens, so every screen nobody has captured now
gets its layering from the file rather than from the declaration table, which is
provably not the paint order.

Verified with artifacts and both ways, not by a green build: the disc test
asserts the measured orders never invert the key and that the composite's key
sequence is sorted, and reading the word from +0x0c instead makes it fail; the
title composites identically; and GP_MISSION_SELECT — uncaptured — now composites
cleanly, committed as a capture.

Two things recorded rather than smoothed over: ties keep declaration order
because the game breaks them some other way that is not known, and the
developer-logo splash has no .rat child, so is_build rejects it and the
compositor never sees that bundle at all — its measured order is inert in
practice and screen render cannot draw it.
2026-08-19 05:28:05 +00:00
Sylpheed RE agent
aa1f49633e docs/re: point the paint-order item at the derivation
The entry said deriving the order from the bundle was open. A key has been found
— the T8aD header word at +0x08 — so the entry now says that, what it does not
cover (ties, bit meanings), and that it is not wired into the compositor yet.
2026-08-19 05:17:44 +00:00
Sylpheed RE agent
56ac88e128 docs/re: the paint order is sorted by a layer key in the T8aD sprite header
The word at +0x08 of a T8aD header — which this project's decoder never read,
taking width/height/tiles from +0x14 onward — is non-decreasing in the order the
game paints a screen, on BOTH screens whose order has been measured: 20 of the
title's 24 elements (the other four have no T8aD sprite) and 6 of the splash's 7.
No inversion anywhere.

On the splash it explains the whole permutation: the three _eff glows carry
0xa100 and their base logos 0xa110, so the glows paint first even though the
declaration table interleaves them.

This is the first FILE-DERIVABLE account of the paint order. Everything checked
before failed — declaration order and its reverse, the placement region, the RATC
child order, keyframe start and rest times, resting Y, the runtime record's
fields, and every other build's table.

Recorded as unsettled: the ties (two groups share a key and are painted in an
order that is not declaration order), what the bits actually mean (the values
look like flag words, and the two screens use different ranges), and the fact
that two screens is two screens — a third permutation either promotes this to a
rule or breaks it.
2026-08-19 05:17:28 +00:00
Sylpheed RE agent
a290791a30 docs/re: point the ship item at the end-to-end mission-path measurement
The entry said the crash blocks it; it now also says how far a run gets (past
SELECT DATA, into a cinematic) and that the crash's firing point moves, so nobody
goes looking for a menu route around it.
2026-08-19 05:08:59 +00:00
Sylpheed RE agent
fa48e39d90 docs/re: the mission path measured end to end — the cache crash is the blocker
Driving menu -> NEW GAME -> DIFFICULTY -> SELECT DATA -> slot 01 with plain flags
gets further than any run so far: SELECT DATA is reached with ZERO crashes (the
screen is alive — a log_ui_draws probe there records 140 draws over 8 frames),
and then choosing a slot lets the game proceed into a cinematic or load, where it
crashes at 0x82307128 — the same cache-flush std::map erase.

So the crash is intermittent in WHERE it fires, not whether: boot, SELECT DATA,
and now after slot selection. There is no safe path through the menus to be found
by choosing different options.

And it is the blocker for every mission-side experiment — the second capital-ship
capture included. Navigation is not the obstacle any more; it is scripted and
works.

Not settled and said so: how to get past it. --mem_watch=false does not (twice).
--eh_dispatch is still untested because no run with it on has reached a throw.
The black-screen hang is a separate intermittent failure with no diagnosis. A fix
is emulator guest-race work, not RE.
2026-08-19 05:08:46 +00:00
Sylpheed RE agent
d0e3dc6f33 docs/re: three more runs, and eh_dispatch is still untested — with the proof why
Drove NEW GAME deliberately (the menu's first item, no d-pad, which is what the
run that first hit the crash actually did). Three runs with --eh_dispatch on: no
crash, no throw, black screen before the save-slot screen. It is tempting to read
that as the flag working.

The run with --cache_throw_diag as well disproves it. That cvar logs the throw
BEFORE any dispatch is attempted, so a throw would appear whatever dispatch then
did — and there were zero. No guest exception happened, so neither flag ran any
code, so neither can explain the difference. What took those runs out is the
intermittent content-load hang, before the crash path was reached.

eh_dispatch therefore stays untested, and the entry says so with the reasoning
rather than banking a false pass.

Confirmed on the way: the screen after NEW GAME is DIFFICULTY — its whole-image
signature matches the earlier capture exactly — so the menu path is understood
even though the runs die after it.
2026-08-19 04:43:25 +00:00
Sylpheed RE agent
743ddda8b1 docs/re: eh_dispatch is untested, and the black screen follows a content load
The run with --eh_dispatch=true produced zero crashes and zero throws, which is
not the result it looks like: the crash path was never reached, so the flag is
UNTESTED. Recording that rather than banking a false pass.

What the run did establish, with screenshots: TUTORIAL leads to a lesson list
(a fifth screen nobody had captured, committed), and selecting BASIC CONTROLS
puts the guest on a permanent black screen — no crash, no GUEST-THROW, emulator
healthy. That gives the black screen a trigger it did not have before: it follows
a CONTENT LOAD, which also fits the earlier black screens.

So the two blockers on the way to a mission are distinct and both unbeaten: the
SELECT DATA crash when the path goes through save-slot enumeration, and the
black-screen hang on loading content. The eh_dispatch test needs a run that
actually reaches SELECT DATA.
2026-08-19 04:17:36 +00:00
Sylpheed RE agent
10b89c77a3 docs/re: restate the ship item's blocker as the crash, not navigation
Navigation is solved and scripted as far as the save-slot screen; what stops the
second capital-ship capture is the cache-flush crash firing at SELECT DATA. The
backlog entry now says that, so the next person does not re-solve navigation.
2026-08-19 04:01:09 +00:00
Sylpheed RE agent
72b0e23e3b docs/re: the cache-flush crash also fires at SELECT DATA, blocking every mission
Driving toward a mission for the second capital-ship capture works as far as two
screens nobody had captured — DIFFICULTY, then SELECT DATA — and then the guest
dies at PC 0x82307128, the SAME std::map/set erase as the boot-time throw, with
537 stacked dumps. Both screenshots are committed.

So the ship item's blocker is the crash, not navigation: the game reaches the
save-slot screen and dies enumerating save data. mem_watch stays eliminated —
this run had it off. And the save/cache path is the common factor across both
firings.

Stated as unsettled: whether a warm cache prevents this firing the way it
prevents the boot-time one. The cache WAS warm here, so it looks like no, but
that is a single run and the cold/warm A/B was only ever run against the
boot-time throw.
2026-08-19 04:00:53 +00:00
Sylpheed RE agent
24b68ed063 docs/re: flag the load confound behind the input and boot-time numbers
The container runs the emulator, the Canary build and the disc test suite on the
same 12 cores, and Xenia alone takes about 5.5 of them. Several (A) experiments
and most boot-duration numbers in this entry were measured while a cargo test or
a build was running; one boot under that load never reached the title in 400 s,
against 130-290 s unloaded.

It does not explain the (A) result by itself — the interleaved A/B alternated
arms under whatever load was present, and the latest title recorded accepted
while an earlier one refused — but "a starved guest misses the input window" is a
hypothesis the existing data cannot exclude, and it should have been flagged when
the numbers were taken rather than after. The same navigation is now being run on
an idle box.
2026-08-19 03:43:32 +00:00
Sylpheed RE agent
0b4dcc6e8f tools/docs: the second-capital-ship blocker is stale; retry boots, not presses
BACKLOG said the per-class ship generalisation needs "the capture-ship-placement
build of Canary". Checked: the ship capture is in the CURRENT build —
RequestShipCaptureFrame and CaptureShipDrawForRE are in command_processor.cc on
auto/re-ui-draw-order, and F10 wrote a 2.9 MB ship capture from this session's
binary. No separate build is needed.

What actually blocks it is navigation: a mission is behind the main menu, and (A)
at the title is accepted on about half of boots with nothing observable
predicting which. tutorial_launch.sh drives boot -> title -> menu -> TUTORIAL and
retries the WHOLE BOOT on refusal, because re-pressing the same title never
works — measured, not assumed.

First run of it: (A) refused, so it rebooted. That is the expected shape rather
than a failure of the harness.
2026-08-19 03:24:40 +00:00
Sylpheed RE agent
343864cc2f docs/re: note that the viewer shares the compositor path
Checked rather than assumed: the Bevy viewer's UI Screens browser calls the same
ui_layout::compose, so the measured paint order and the ghost skip reach the
window a person looks at, not just the CLI's screen render. Its element table
stays in declaration order, so the per-element visibility toggles still index
correctly.
2026-08-19 03:13:37 +00:00
Sylpheed RE agent
8c08c211a3 formats: narrow the ghost skip, and withdraw the claim that it fixed a bug
Checking the blast radius of the previous commit's `kind = 0x4` skip found 174
elements on the disc that are 0x4 with no non-0x4 element of the same sprite —
GP_READY_ROOM pak entry 75 is 56 elements and every one is 0x4, a list of real
icons a blanket skip would erase. So the rule is narrowed: skip an instance only
when its template is present, which is the case the title capture covers. The
title render is byte-identical before and after, so the narrowing changes nothing
the evidence covers.

And the claim that this "caught a regression" is WITHDRAWN, because measuring it
refuted it: none of those 174 elements is in a bundle `is_build` accepts, so none
of them ever reaches the compositor. The blanket skip would have been harmless in
practice. The narrow rule is a precaution, not a fix, and the comment says so.

The test that was going to assert the regression could not find such a build —
correctly, since none is composable. It is replaced by one that pins the fact
which makes the narrow rule safe: no composable build on the disc has a 0x4
element without its template, checked across 500+ builds. If that stops being
true it fails, instead of a screen quietly going empty.
2026-08-19 02:45:48 +00:00
Sylpheed RE agent
f66e9de430 docs/re: the paint-order item now has a landed fix, not just a finding
BACKLOG said the viewer paints in declaration order and the title composites
wrongly. That is no longer true for the two builds with measured orders, so the
entry says what landed, what it exposed (the kind=0x4 ghosts), and what is still
open: deriving the order for screens nobody has captured. The order is
deterministic, so a rule exists; it is not in any decoded structure, and the
loader is what is left to read.
2026-08-19 02:32:31 +00:00
Sylpheed RE agent
c3c79ad028 formats: composite in the MEASURED paint order, and drop the ghost instances
The compositor painted in declaration order, which the draw capture proved wrong
— the title's background is declared ninth and paints first. The order the game
uses is its runtime child list, and no decoded field reproduces it, so rather
than leave the screen composited wrongly this lands the orders that HAVE been
read off the running game and falls back to declaration order everywhere else.
Two builds are covered: GP_TITLE's title build and the GAME ARTS / SETA / studio
anima splash. Keyed by element names, which identify a build across paks and
language variants.

Rendering it exposed a second defect, and the same capture settles it: the
kind = 0x4 elements are motion-trail ghosts, not resting content. The bundle
declares three instances of each wordmark; the capture shows exactly ONE quad at
each wordmark's position. Drawing them at their resting keyframe put three
oversized PROJECT SYLPHEED copies across the composite. They are now skipped.

Verified with an artifact, not a green build: the composite is committed
(captures/title-composited-measured-order.png) and now reads as the title screen
— background, planet, ship, wordmark, TM, copyright, correctly layered.

The test is disc-gated and was checked BOTH ways: it passes as landed, and
disabling the order table makes it fail. It reads one pak rather than every
build on the disc — the first version used the all-builds helper and got the
test process OOM-killed running alongside the other three.
2026-08-19 02:32:14 +00:00
Sylpheed RE agent
f35a871baa docs/re: no ordering field in the runtime element record either
The next suspect was a role or depth field in the 48-byte element record, so
every undecoded word was dumped for all 24 title elements against its paint slot.

Clean negative: +0x08, +0x0C, +0x18, +0x1C and +0x2C are zero on 21 of 24
elements, and the three exceptions hold what looks like live animation state.
Nothing there orders anything.

One confirmation on the way: +0x04 is the declaration entry's `kind`, verified
against the file for all 24 — 0x10 on the two .prm elements, 0x4 on the four
repeat instances, 0x3000 on the two ptlogoall_eff, zero elsewhere. The record
mirrors the file here as the pivot and keyframe count already did. And `kind`
does not explain the order either: the paint order interleaves kinds freely.

So the ordering is in none of the decoded data — not the declaration entry, not
the placement region, not the runtime record. What is left is the loader that
appends to +0x30, which is worth reading precisely because the order is
deterministic. Stated without promising a static rule exists merely because one
could.
2026-08-19 02:20:01 +00:00
Sylpheed RE agent
72afde7115 docs/re: the paint order is deterministic, and the 7-element screen is the logo splash
Two things this iteration, both of which change what is worth doing next.

The order is DETERMINISTIC: two independent boots give byte-for-byte identical
permutations for both live screens. That kills the hypothesis that the child list
is built in I/O-completion order — which mattered, because a run-dependent list
would have made deriving a rule pointless. It is a pure function of the bundle.

And the 7-element bundle is identified by matching its pivots against every
7-element build on the disc: it is the developer-logo splash (GAME ARTS / SETA /
studio anima, GP_TITLE entries 11/14). With names attached, its paint order
0 2 4 6 1 3 5 reads as the full-screen .prm, then all three _eff glows, then all
three base logos — glow behind, logo on top.

That refines the earlier "grouped by sprite" reading and partly withdraws it:
here every element has its own sprite, so the grouping is by ROLE, not sprite
identity. The title build could not tell the two apart because its repeated logo
instances share both.

Still not derived: where the role comes from. No decoded field carries it, and
sorting on a "_eff" name suffix would be an odd thing for a loader to do.
2026-08-19 02:08:50 +00:00
Sylpheed RE agent
875ac2632d docs/re: two ground-truth paint permutations, and four more refutations
Deriving the child order from the bundle is what the port needs, so this collects
data rather than guessing. Both multi-element screen objects live on the title:

* a 7-element bundle paints 0 2 4 6 1 3 5 — evens then odds, a stable partition;
* the 24-element title build paints 9 11 12 10 13 6 20 19 14 15 18 16 17 0 2 4 7
  1 3 5 22 23 21 8.

Two things the pair establishes. The list is GROUPED, not shuffled: elements
sharing a sprite are contiguous (ptlogo1's three instances at slots 13-15,
ptlogo2's at 17-19, the back2eff family at 8-12). And the five single-keyframe
elements lead — exactly elements 9,10,11,12,13 have one keyframe, and the list
opens with all five, which is a file-visible property.

Refuted on this data: first-keyframe time and resting time (back2eff1 starts at
52 and rests at 56, yet paints after back2 at 66/80), declaration order and its
reverse, and the RATC child order (ptbase2 is child 4 and paints first while
ptlogo1 is child 0 and paints fourteenth).

Not settled, and said so: the rule. The group CONTENTS are explained; the group
ORDER is not, and two permutations are too few to build it.
2026-08-19 01:56:04 +00:00
Sylpheed RE agent
069165c03d docs/re: mark the paint-order item solved, and say what is left
The item that has been open through this whole run is answered: the paint order
is the screen object's reordered child array at +0x30, not any table in the file.
BACKLOG and INDEX now lead with that, and the investigation behind it is kept in
full underneath — most of it refutations, which is what made the answer findable.

What is left is stated in the same breath, because it is what the port needs:
deriving that order from the bundle without running the game. Until then the
viewer paints in declaration order and the title screen composites wrongly, which
is now a documented defect rather than a mystery.
2026-08-19 01:46:20 +00:00
Sylpheed RE agent
688851122d docs/re: SOLVED — the paint order is the screen object's child array at +0x30
The screen object holds a SECOND list of its elements, a reordering built at load
time, and that list is the paint order. It is 24 pointers at +0x30, each the
+0x00 field of one of the 48-byte element records, so both arrays hold the same
objects in different orders.

Checked against the draw capture rather than asserted: the seven elements the
capture can name sit at child slots 0, 6, 7, 13, 16, 17, 22 — strictly ascending,
in exactly the captured submission order. It also resolves the one sub-order no
static field could explain, the pair that decodes to the same 1133x280: slot 6 is
element 20 and slot 7 is element 19, so they paint 20-then-19, DESCENDING in
declaration terms. And the kind=0x4 repeat instances sit immediately after their
template, where the declaration table interleaves them.

Stated as unsolved, because the port cannot read a runtime array: deriving this
order from the bundle. The order is clearly structured rather than arbitrary —
elements sharing a sprite are adjacent and the full-screen effects lead — so it
is worth attacking, but it is not attacked here.
2026-08-19 01:46:00 +00:00
Sylpheed RE agent
4134c21766 docs/re: the UI screen object, found and verified in live guest memory
The item class's vtable (0x820b30b4) is a fixed 4-byte value at offset 0 of every
instance, so gmem.py finds the objects in the running game with no debugger and
no emulator change. Seven are live on the title screen.

Their first {ptr,count,capacity} triplet identifies them outright: counts of 24,
7 and 1 — exactly GP_TITLE's title build, loading overlay and PRESS (A) BUTTON
bundle. The two-bundle composition of the title screen, which the draw capture
had inferred, is visible directly in memory.

The element array at +0x08 is 48-byte entries in declaration order, and it was
checked against the file at five positions with no misses: elements 0, 1, 6, 9
and 21 carry pivots (451,50), (449,46), (320,160), (320,180) and (309,10) as
floats, matching build 4 exactly, and elements 0/1 carry keyframe count 8, which
is their count in the file.

Stated as the reason this was done and NOT achieved: the array is in declaration
order, which the capture already proves is not the paint order, so the renderer
walks something else or sorts. Three further triplets in the object are
unidentified; the 24-entry one at +0x14 is already ruled out as an index list
(its entries are pointers, not indices). The gain is that the question is now a
data question on a structure that dumps in seconds.
2026-08-19 01:35:13 +00:00
Sylpheed RE agent
82b4a9e523 docs/re: more UI map, no walk — and say that this is becoming a rabbit hole
Three facts, none of them the thing being looked for:

* the drawing API is enumerable — the renderer accessor the quad emitter uses has
  exactly 6 callers, four of them sibling quad emitters;
* sub_823C2990 is a FACTORY, not the singleton accessor its use in the top-level
  render function suggested: it allocates 4 bytes plus a 244-byte object from the
  heap at [0x828E2B14] and runs the chain that ends at the constructor installing
  vtable 0x820b30b4. So a UI item is 244 bytes;
* the 0x823C region holds both the item class and four of the RATC-fourcc
  loaders, so bundle parsing and item construction live together.

And the part worth writing down more than any of them: three iterations have
added map without answering the question, and each step has been a plausible next
query rather than a decisive test — the shape of a search that can run forever.
The decisive alternative is costed instead of started: a Canary memory watch on
the UI vertex buffers would report the guest PC that writes them, naming the
emitter and its caller outright. That is real emulator work, and it is now a
choice to weigh against leaving the paint order unresolved, which costs the port
one screen's fidelity and nothing else.
2026-08-19 01:23:12 +00:00
Sylpheed RE agent
4808751113 docs/re: find the guest's UI quad class from the layout the capture measured
The capture pinned the UI vertex exactly — prim 13, stride 24, float3 position at
+0, k_8_8_8_8 colour at +12, float2 UV at +16 — so the binary was searched for
code that writes that shape. Fifteen candidates; the tightest, sub_82250138, is
unambiguous: it asks a renderer for 4 vertices of primitive type 13, then writes
x/y/z, colour 0xFFFFFFFF at +12, u/v at +16/+20, stepping by 24. Every constant
matches something measured rather than assumed.

Its caller is a constructor that installs vtable 0x820A7264
(ANON_Class_AAFDBF89, 2 slots) — the UI quad/sprite class.

Recorded with equal weight: this is NOT what the search was for. The chain up
from it ends at sub_821A8578, which turns out to be an unrolled run of ~30
identical call triplets — a fixed subsystem sequence, not an element walk. No
function that iterates screen elements has been found, and nothing here bears on
the paint order yet.
2026-08-19 01:16:14 +00:00
Sylpheed RE agent
38df977b30 docs/re: alpha does not break the tie — but it validates the fade decode
The plan was to separate ptlogo_back2eff from ptlogo_back2eff5 (same 1133x280,
declaration 20 vs 18, either side of ptlogo_back2 at 19) by their resting fade
alphas. Carried out, and refuted: both quads come back at alpha FF, and reading
the bundle again, both elements REST at 255 under this project's own max-dwell
rule — back2eff5's longest hold is t=74 at 255, and the 192 I had quoted is a
later, shorter keyframe, not its resting value. The ambiguity stands.

What the colour did buy is worth more than the tie-break: the first check of the
fade/keyframe decode against the RUNNING GAME rather than against another parse.
Every static element draws at exactly the resting alpha the bundle predicts, and
the only two quads whose alpha moves between consecutive frames (88->88/86->87
and 3C->38) are the rotating effect pair and the PRESS (A) glow — the two things
visibly animating on screen. Nothing had confirmed before that the alpha channel
of a keyframe is what drives that throb.
2026-08-19 01:09:25 +00:00
Sylpheed RE agent
3dc6d9d32c docs/re: the loading screen never boots, and no other build explains the order
Two routes closed, both cheaply.

The loading screen was the best discriminator reachable WITHOUT navigation — its
full-screen element is declared 7th of 10, so the two candidate orders would look
completely different. It does not appear during the boot at all: 23 captures
armed across an entire boot are all intro-movie frames, two full-screen quads
apiece. It belongs to a content load, which is behind the main menu, which is
behind the (A) that works half the time.

And a loophole this document had left open is shut: the title's sprites were
matched to build 4 BY SIZE, so the game might have been running some other build
whose table lists them in the captured order. Only two bundles in GP_TITLE.pak
contain all seven sprites, and neither is ascending in the captured order.

Recorded rather than glossed: one sub-order is genuinely ambiguous, because
ptlogo_back2eff and ptlogo_back2eff5 decode to the same 1133x280 and sit at
declaration 20 and 18, either side of ptlogo_back2 at 19. The UI quad carries a
k_8_8_8_8 colour attribute the hook does not log yet, and the two rest at
different fade alphas — logging it would name them apart.
2026-08-19 00:58:46 +00:00
Sylpheed RE agent
2483b7c3c9 docs/re: my own flag hypothesis fails its test, and the input path is mapped
With the capture cvar gone from the launch line entirely, a fresh run reached the
boot title and (A) still did nothing. The flag is therefore not necessary for the
failure, and 0-of-7-with vs 4-of-6-without does not survive as an explanation.

The conclusion is the plain one: (A) at the title succeeds about half the time
and nothing measurable from outside predicts which. Five candidates are now
eliminated — Xenia UI swallowing the keystroke, the driver filter, a sign-in or
content call on the press, the age of the title, and the capture cvar.

So the guest side was traced instead. XamInputGetKeystrokeEx has exactly one
caller in the title, and that has one caller: entry_point -> main loop
(sub_8216EA68) -> per-frame input update (sub_822F1AA8) -> the pad poll
(sub_82457038). The poller's layout is readable and matches the runtime trace
(keystroke queue at +68/+72 drained with flags=3; current state at +36 compared
field-by-field against the previous at +52), and nothing in it is state-gated.
Whatever ignores (A) is a consumer further up — a real RE job, and that is where
the thread now stands, stated rather than guessed at again.
2026-08-19 00:47:53 +00:00
Sylpheed RE agent
6b26087bea tools: navigate to OPTIONS and capture there, with no launch flags
OPTIONS is the cheapest reachable screen whose full-screen background is
declared LATE (a static sweep of all 30 UI paks ranks GP_DEBRIEFING_PILOTLOG
26-of-29 and GP_BUNK 15-of-17 above it, but both need a mission). Only such a
screen can tell "declaration order" from "background first", which is the open
question in the paint-order item.

It launches without any capture cvar now that F10 arms unconditionally, so the
navigation is not carrying the one variable that correlates with the title
refusing (A).
2026-08-19 00:37:22 +00:00
Sylpheed RE agent
a04c133edc docs/re: boot time refuted as the gate on (A); the flag correlates with no mechanism
An interleaved A/B (arms alternating in one series, so cache warming cannot pose
as either variable) breaks the confound the last entry flagged — and not in the
direction the first table suggested. The LATEST title of every run recorded,
268 s, accepted (A); a 232 s title refused. "The title arrived late" is dead.

That leaves --log_ui_draws correlating at 0 of 7 against 4 of 5, with no
mechanism available: the cvar is read only when F10 arms a capture, F10 was never
pressed in those runs, the per-draw hook is one relaxed atomic load, and the two
arms' startup config dumps are byte-identical across 246 settings. A correlation
with no mechanism — and an arm that failed once — is not a finding; it is a
variable to remove. Removed: F10 now arms the capture unconditionally.

Also recorded: three boots this session ended on a permanent black screen, two of
them with no input at all. Not the pad, not the cache-flush crash, about one boot
in five.

Five candidate explanations for the (A) gate are now eliminated or refuted. The
next step is the guest's own state machine in sylpheed.db, not another guess from
outside.
2026-08-19 00:37:03 +00:00
Sylpheed RE agent
4990cf6eb3 docs/re: the boot-title recipe is 3/3 without my capture flag and 0/5 with it — and confounded
Using last iteration's recipe (first title, one (A)) four more times, it failed
every time. The nine-run table splits perfectly on --log_ui_draws: 3 of 3
successes without it, 0 of 5 with it. That would indict my own instrumentation
cvar, which is the one that captures paint order.

It is also perfectly confounded and the entry says so rather than picking the
exciting reading: every success had its title inside 147 s, every failure at
177 s or later. Flag and boot-time move together across all nine runs. A
mechanism is hard to see for the flag (it is read only when F10 arms a capture;
the per-draw hook is one relaxed atomic load), which argues for the boot-time
story — but "I cannot see how" is not evidence.

The separating experiment is named: ON runs with a fast boot. One that accepts
(A) exonerates the flag.

Also recorded: widening the menu signature last commit made screen_id classify
some boot frames as "menu", twice in one run before any menu existed. A dark-blue
movie frame and this menu are not separable from a single grab — navigation
scripts need the classification to hold across two grabs, which is a caller-side
fix, not a classifier one.
2026-08-19 00:00:43 +00:00
Sylpheed RE agent
b9062ea3bc tools: a working route to the menu, and the classifier that could not see it
screen_id.py called the main menu "other". Its menu rule required a near-white
fraction above 1.5%, measured in 2026-07; the menu reached from the boot title
measures 0.03% (mean 13,26,59 — dark, strongly blue, essentially green-free).
That is worse than a cosmetic miss: a script that waits for "menu" and never
sees it reports the navigation as failed while the menu is on screen, which is
exactly what happened here. Both measured signatures are now documented in the
code and both classify.

menu_draw_capture.sh now taps ONCE on the first title rather than up to 40 times:
repeating was measured to be useless (the attract title accepts nothing) and the
first title accepts a single press.

title_states_capture.sh is new — it captures the draw list in both title states
in one run, which is what refuted the "the attract title omits the button plate"
theory.
2026-08-18 23:10:29 +00:00
Sylpheed RE agent
9e16331155 docs/re: a second screen captured, and the two title states pinned down
Two results, one of which closes the navigation problem the last three
iterations kept hitting.

**The title states.** The hypothesis was that the attract-loop title is a
non-interactive presentation that omits the PRESS (A) plate. Half right:

* the state distinction is REAL — a single (A) on the title that ends the boot
  opens the main menu, 2 of 2 in independent runs, one of which never pressed
  F10; the title the attract loop returns to accepts nothing, not (A), START, B,
  BACK, X or Y, across dozens of delivered presses;
* the proposed tell is REFUTED — capturing the draws in both states in one run
  gives 13 quads at identical rects, ptbtn00 and ptbtn00f included. The two are
  identical to the renderer and different only to the guest.

So there is now a reliable route to the menu: first title after boot, one tap.

**The main menu's paint order**, captured with it. Its sprites are GP_TITLE
build 5's, and ptframe1/ptframe2/ptbtn01f land within 4 px of their declared
resting placements — an independent placement check on an untouched bundle.

It does NOT settle the ordering question, and the entry says so: build 5 lists
its background at indices 1-2, so declaration order and "background first"
predict the same sequence here. Same failure mode as GP_READY_ROOM/GP_OPTIONS.
What it does establish is that the title's disagreement is not a decode
artefact — same pak, same engine, one build that follows its table and one that
does not.
2026-08-18 23:10:04 +00:00
Sylpheed RE agent
4b0263b80f docs/re: the scripted route to the menu does not work — measured, not assumed
The previous entry named the next step: "tap (A) once per second and see whether
the success rate goes to 1". It was run. The answer is no.

40 presses at 1/s on the title change nothing, and neither does one each of
START, B, BACK, X and Y. Every press was delivered (the driver logs down/up for
each) and the new [RE-INPUT] diagnostic never fired, so nothing was swallowed —
the guest received them and ignored them.

Also recorded, because it destroys a run rather than merely wasting it: tapping
(A) through the boot (88 presses over the intro) ends on a permanent black
screen with no crash and no throw.

What is left is a difference in STATE, not in input: the one success came on a
title that appeared ~83 s into a warm boot, the failures on titles that appeared
after a full attract cycle. "The attract-loop title is a non-interactive
presentation" is written down as a hypothesis with the experiment that would
settle it — a log_ui_draws capture in each state, looking for ptbtn00.
2026-08-18 22:45:08 +00:00
Sylpheed RE agent
243835719f tools: a menu-capture session that encodes the two input traps
Boot -> title -> main menu -> arm the UI draw-order capture there. What it
encodes is what cost time to learn:

* it does NOT tap (A) during the boot. A version that did delivered 88 presses
  over the intro and left the guest on a black screen that never came back —
  no crash, no throw, emulator healthy. The intro is ~3.5 minutes and reaches
  the title on its own.
* it taps at the title until the screen stops being the title, rather than
  pressing once and concluding, because acceptance is intermittent.
* after F10 it clicks the game surface, because F10 also opens the emulator's
  menu bar and any Xenia UI makes IsUIActive() true, which swallows every later
  guest keystroke.

It does not currently reach the menu — see docs/re/canary-scripted-input-traps.md
for the measurements. Committed because the traps are worth keeping encoded, and
because the next experiment (capture the draws in each title state) needs the
same scaffolding.
2026-08-18 22:45:08 +00:00
Sylpheed RE agent
f5603aa107 docs/re: the main menu HAS been reached — the Ⓐ blocker is withdrawn
The previous entry called the title's Ⓐ a hard blocker. It is not: the main menu
is reached and screenshotted (NEW GAME / LOAD GAME / TUTORIAL / OPTIONS /
EXTRAS). What is true is narrower — Ⓐ advances the title only intermittently,
about one attempt in four, with the press verifiably delivered every time and no
Xenia UI active.

Three candidate causes were eliminated with measurements rather than argument:

* IsUIActive is now observable (Canary logs when it swallows a keystroke) and it
  never fires on the failing runs;
* the driver filter is fine — the game polls with flags=3 and the file pad
  reports Controller=1, so FilterDrivers keeps it;
* the game makes no content/user/signin call on the press at all — tracing every
  Xam call around it shows only input polling.

And two traps in my own measuring rig, which cost more than the bug and are
written down so nobody repeats them: a FIFO trace consumer that exits STALLS the
emulator (the guest stops polling — indistinguishable from a dead pad, and it
produced two runs of false evidence), and phase-A's kernel.return events carry a
placeholder return_value of literally 0, so "every keystroke call returns
SUCCESS" was an artifact of the logger, not a finding.
2026-08-18 22:18:51 +00:00
Sylpheed RE agent
44ff5454f9 docs/re: the cache-flush throw — a 100-second trigger, and two withdrawals
Chasing the title-screen crash into the emulator's own diagnostics turned it
from a mystery into a controlled experiment, and knocked over two things this
corpus said yesterday.

Withdrawn #1: "the fault address 0x1_0000000C is a pointer with a stale high
word". The crash dump prints r25 = 0x0000000C, clean. Xenia maps the guest's
4 GiB at host 0x1_00000000, so that IS guest address 12. The guest dereferenced
the small integer 12.

Withdrawn #2: "with --mem_watch=false the crash does not happen at all", which
named the crash-oracle handoff's suspect #1 as measured. It was confounded —
every --mem_watch=false run had also had a warm cache. Held cold, the throw
happens with the probe off (2 437 crash dumps). mem_watch is eliminated for this
crash.

What it actually is: the access violation is the guest's own `throw` RETURNING,
because this build logs guest C++ exceptions and continues rather than
unwinding. So the event is the throw, and with --cache_throw_diag=true the guest
names it: std::out_of_range, from the cache-manager flush, with a deque of 38
entries (37 distinct, one duplicated) against a 37-key map, every one of them
absent from the flush's snapshot but present in the live map — the TOCTOU race
the logger's own message describes.

And the new, useful part: the trigger is the on-disc cache. Complete cache, no
throw (2 runs). Directory moved aside or half-rebuilt, throw ~100 s into the boot
(3 runs, including one that threw with NO access violation behind it — which is
why crash dialogs are the wrong thing to count). A suspect in that bisection plan
now costs a `mv` and two minutes instead of a mission.
2026-08-18 21:44:22 +00:00
Sylpheed RE agent
d521c802ad docs/re: re-refute the time orderings on the right build, and kill a Y-sort
Two of the paint-order refutations were computed with build 7's keyframe times,
and build 7 is not the build the game runs. Re-checked on build 4: both still
fail, on the same element (ptlogo1 paints after ptlogo_back2 though it starts at
t=26 against t=66, and rests at t=42 against t=80).

A new candidate is recorded and refuted rather than left implicit, because it is
the kind that gets adopted on partial agreement: painter's order by resting Y
reproduces the captured order to within a single transposition — and is wrong
twice, on ptlogo_tm (drawn before a sprite 63 px higher) and on the background
(drawn first, though its resting Y would sort it fourth).

Also scoped, not walked: the guest-code avenue. The splash item vtable
0x820b30b4 is real (25 slots, three construction sites); RTTI carries no class
names anywhere in the binary; the format tags are fourcc immediates behind a
virtual call, not strings; and the obvious shortcut — searching for the 60-byte
declaration stride — leads to a time-string parser, not the table.
2026-08-18 21:05:39 +00:00
Sylpheed RE agent
ab8a0d9e6d docs/re: the title-screen crash is an STL map/set erase, 40 s from a cold boot
The crash PC from the previous iteration resolves, and it names itself:
sub_823070B0 references the string 'invalid map/set<T> iterator', builds it with
the string helpers and throws it — which is the guest C++ exception (E06D7363)
Xenia reports one line earlier. Its node offsets are MSVC's red-black tree node
exactly (_Left 0, _Parent 4, _Right 8, _Color 24, _Isnil 25), so this is a
std::map/set erase, and the fault is the first dereference after the iterator
check.

That matters beyond this blocker. The canary handoff's Ready-Room crash is the
same shape — a guest STL exception in a cache/save path — and its bisection plan
is priced at "one build plus one Ready-Room run" per suspect. If it is the same
defect, each suspect now costs 40 seconds, and suspect #1 is already measured:
--mem_watch=false removes it.

Stated as unresolved rather than guessed: the fault address 0x1_0000000C is a
32-bit value with bit 32 set, which fits BOTH a corrupt guest tree and a stale
high word on the emulator side. The measurement that separates them (read the
node from guest memory at the throw) is written down rather than assumed.

The second, unreproduced crash PC is identified too: an unrolled 4x16-bit copy
loop faulting on the STORE, i.e. a bad destination — a different failure.
2026-08-18 21:05:39 +00:00
Sylpheed RE agent
de038c54c0 docs/re: why a scripted run cannot get past the title — three traps and a blocker
Each of these presents as "the pad is dead", and each has a different cause.

1. **F10 opens the emulator's menu bar.** It is the RE capture hotkey AND the
   toolkit's menu key. With that menu open — or any Xenia UI window, the
   Profiles Menu included — `IsUIActive()` is true, so XamInputGetKeystrokeEx
   returns SUCCESS with a ZEROED keystroke before the pad driver is asked. The
   guest polls, gets nothing, and does nothing, with no error anywhere. Escape
   does not close it; a click on the game surface does. This cost most of an
   iteration.

2. **The file-pad was never the problem.** It delivers (vk=5800 down/up per tap)
   and A skips movies in the same runs.

3. **(A) at the title needs a signed-in profile.** Without one the guest calls
   XamShowSigninUI and Xenia's dialog appears — which is also the proof the
   guest reads the button. That dialog cannot be completed here: the gamertag is
   an ImGui text field and synthetic X key events never reach it. Hence Canary's
   new --create_profile_if_none.

And the blocker, stated as measured rather than as a theory: with the profile
signed in, (A) still does not advance the title across four boots. The
reproducible failure is next door — with the default --mem_watch=true the guest
crashes in the boot content path (read of 0x10000000C at 0x82307128, thread 9,
after a guest C++ throw), and with --mem_watch=false it does not. That is a
second, far cheaper reproduction of the Ready-Room crash the canary handoff
blames on the mem_watch probe: 40 s from a cold boot instead of after a mission.

One observation kept because it did not reproduce: a single (A) once crashed the
guest at a DIFFERENT PC (0x824578A0, thread 6). Two attempts to reproduce it
gave neither a crash nor a transition.

Next step named: those two PCs in xenia-rs/sylpheed.db, and the content exports
around the press.
2026-08-18 20:16:27 +00:00
Sylpheed RE agent
4101d9979f docs/re: the title paint order is build 4's, carries no depth, and is not in the file
Three findings and two corrections, all against the same capture.

Corrections first, because one of them invalidates numbers already written down:

* **the running screen is build 4, not build 7.** `screen info` defaults to the
  largest build; the game runs a different one, and the two disagree on sprite
  sizes (ptlogo1 822x100 vs 919x113 — the capture says 915x115). Every
  declaration index in the previous entry was against the wrong table. The
  conclusions survive unchanged; the indices are restated.
* **the ship capture does NOT skip these UI draws.** This entry justified a new
  hook partly on "it requires an f32x3 position stream, which a 2D quad does not
  have". These quads have one: armed on the title, xenia_ship_capture_01.log
  holds 1150 records of which 965 are the UI sprite shader. The new hook is
  still the right tool — frame brackets, submission order, texture identity —
  but it was justified with a wrong reason.

Findings:

* **no depth in the geometry.** Attribute 0 is k_32_32_32_FLOAT, so every quad
  carries a Z, and every Z is 0.00000. Submission order is the whole ordering.
* **no ordering field in the bundle.** Every word of build 4's declaration table
  dumped (committed as evidence): the unknowns are constant. The placement
  region's per-group lead word — never read before — is 0 for all 24 groups, and
  the region is followed straight by the RATC child stream, so no table hides
  behind it.
* **the batching rule**, which is what makes the order readable: a draw holds
  one or two quads, and a two-quad draw always holds two elements adjacent in
  paint order that share a texture page. The engine flushes when the page
  changes; it does not sort by page (pages alternate).

Reproduced in an independent run — same order, same rects — which is the second
observation the CONFIRMED bar asks for.
2026-08-18 20:16:05 +00:00
Sylpheed RE agent
088fcce724 tools: capture targets, and a Z-aware draw-order decoder
`ui_draw_capture.sh` grows three knobs the second iteration needed:

* ARM=early presses F10 before the title exists, so a long window contains the
  frames in which a screen is BUILT (it turns out none are — the title screen
  submits the same 11 draws every frame and never rebuilds);
* TARGET=menu taps A once on the title and arms on the main menu, skipping
  attract movies on the way — it does not get there, but the blocker it hits is
  documented rather than worked around;
* EXTRA_FLAGS passes emulator cvars through (--create_profile_if_none,
  --mem_watch=false, --log_level).

`ui_draw_order.py` follows the capture's new vertex format (x, y, z) and reports
the Z it now has.
2026-08-18 20:15:48 +00:00
Sylpheed RE agent
53642ee3d6 docs/re: the title screen's paint order, measured from the draw stream
BACKLOG's UI-ordering item ended at "the next step is no longer static — it is a
per-draw capture of the title screen showing the order the game submits". That
capture now exists.

The order, in submission order: a full-screen background layer, a rotated effect
pair, a second full-screen layer, ptlogo_back2eff + ptlogo_back2, ptlogo1 +
ptlogo_tm, ptlogo2, ptcopyright, and the PRESS (A) BUTTON plate — declaration
indices 13, 22|24, 23, 0, 11, 1, 28, and then two elements that are not in that
build at all.

Two more candidate orderings die on it (keyframe start time, resting-keyframe
time), and one structural fact reframes the item: the visible screen composites
TWO bundles — build 7 plus the one-element build 2 that is the button — so no
single build's element table can be the paint order whatever its order. The
button's quad lands at (384,551) against a declared rest of (383,550), which is
a one-pixel agreement between the static placement decode and the running game
on a bundle nothing had checked.

INDEX's "the screen's draw list is the bundle's declaration table (back-to-front)"
is demoted in place rather than edited away.

Still open and said so: the rule that produces the order; three of twelve quads
unidentified; one screen only.
2026-08-18 18:57:43 +00:00
Sylpheed RE agent
9ae8c23265 tools: drive and decode a UI draw-order capture
`ui_draw_capture.sh` boots to the title screen and arms Canary's new
`log_ui_draws` there — deliberately WITHOUT tapping A on the title, which is the
subject and which sends the guest into a save-data probe. ARM=early presses F10
before the title exists, for the frames in which a screen is built. A grab that
is not full-width is a hard error rather than something to classify.

`ui_draw_order.py` turns the capture into a named paint order: the UI shader
emits NDC, so a quad's pixel rect is exact, and the disc's sprites have
near-unique decoded sizes, so the rect's SIZE names the sprite. Matching is
nearest-within-6px with the distance printed, because a quad comes back a few
pixels under its sprite for a reason that is not yet measured.
2026-08-18 18:57:43 +00:00
Sylpheed RE agent
2b06c68412 tools: screenshot was grabbing a 10-pixel sliver, silently
The wrapper takes the xenia window's geometry from `xwininfo -root -children`
and crops the game surface out of it. Two things are wrong with that here, and
neither errors:

* `-children` lists only DIRECT children of the root, and openbox reparents the
  game window — so the real one is not in that list at all;
* the app owns more than one window of class "xenia_canary" — an off-screen 10x10
  helper as well as the 1280x745 game window — so `head -1` picked the 10x10 one.

Every grab came back a 10-pixel sliver, and nothing failed: `screen_id.py`
happily classified the sliver, the movie-skip heuristic fired on its noise, and
a whole session's worth of screen ids were meaningless. It also drove a stray
tap into the title screen's save-data probe, which crashed the guest.

Walk the full tree and take the largest xenia window by area, using its absolute
geometry (a reparented window's own +X+Y is relative to its frame).
2026-08-18 18:57:29 +00:00
Sylpheed RE agent
57a20e6472 docker: two ways to talk to the loose agent
Remote Control, so a detached run is not a one-way trip:

  ./sylph-agent remote   -> https://claude.ai/code/session_...
  ./sylph-agent attach   -> the container's own terminal

`loose` now starts Claude Code with `--remote-control <name>`, registering the
session with your account so you can chat with it from claude.ai or a phone.
The name is passed EXPLICITLY because the flag's value is optional — a bare
`--remote-control` swallows the /loop prompt that follows as the session name.
Off with SYLPH_REMOTE=0, renamed with SYLPH_REMOTE_NAME.

The pty is forced to 200x50. A detached `docker run -t` gives 80x24, and Claude
Code hard-wraps to the terminal width — which truncated the Remote Control URL
to ".../session_01..." in the one place you actually need to read it, and made
`docker logs` nearly unusable besides.

`remote` waits up to six minutes and reports progress: registration lands a
minute or two after launch, so answering "not found" immediately would be
answering a different question than the one being asked.

Verified: a loose run registers and `./sylph-agent remote` prints the full URL.
`attach` is NOT verified end to end from here — `docker attach` refuses a piped
stdin ("cannot attach stdin to a TTY-enabled container"), which is a property of
this harness rather than of the container, so it needs a real terminal to try.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 20:06:05 +02:00
Sylpheed RE agent
f5069b0f64 docs/re: neither existing Canary hook can capture the title screen's draw order
The paint-order item's remaining step is a per-draw capture, and the obvious
move is to reuse the RE instrumentation already compiled into the Canary build
on `sylpheed-re`. Read the source rather than assume: neither hook works here.

`--log_draws` de-dups by vertex-declaration fingerprint, so a screen's sprites
collapse to one record; it carries no texture identity and only first-seen
order. The F10 ship capture does keep submission order and would separate the
elements, but it returns early on any draw without an `f32x3` position stream —
the comment says "UI/effects — skip" in as many words.

So the item needs a new cvar-gated hook and a `build-canary` run. Stated in the
entry so the cost is known before someone starts, and so this reading is not
repeated.
2026-08-18 17:15:51 +00:00
Sylpheed RE agent
806c2029aa docs/re: the ship test's one-way gap was already half closed, and is now shut
Records both halves rather than just the fix: the "extra static placements
cannot fail it" line had been stale since `64d372c`, and the residual — a set of
names being blind to a duplicated instance — is what this round actually closed.

Also states what it does not close: the multiset is `e106`'s alone, so the
per-ship table this entry originally asked for still needs a runtime capture of
a second capital ship, i.e. the `capture-ship-placement` build of Canary.
2026-08-18 17:14:24 +00:00
Sylpheed RE agent
2376badd13 ship: pin the e106 ground truth as a multiset, not a set of names
`static_assembly_matches_runtime_capture` walks the capture's parts and looks
each up in ours, so an extra static placement was originally invisible to it —
that is how `e303_wep_01` sat here decoded 100x too large. `64d372c` closed that
with a set-of-names check; BACKLOG.md still says otherwise, and it is wrong.

One direction stayed open, though: a set cannot fail on a resource placed
*twice* when the capture lists it once. That is not hypothetical — a duplicated
instance is what a bad node walk emits, and the two legitimate duplicates here
(`e106_eng_01`, `e303_wep_01`) are why the test had to special-case counts at
all. So pin the whole multiset — 9 resources, 11 placements, against the
capture's 8 dedup'd parts — which subsumes both hand-written counts and fails on
an extra, a missing, or a duplicated placement.

Refuted before believing: perturbing the expected table (declaring
`e106_bdy_01` twice) fails with the real multiset on the left, so the assertion
runs against the disc rather than skipping for want of SYLPHEED_ISO.

The remaining `for want in &cap.parts` loop guards the table itself: editing it
cannot quietly drop a part the runtime actually draws.
2026-08-18 17:14:24 +00:00
Sylpheed RE agent
ee65631de8 docs/re: the title-screen oracle — what it settled, and what it refuted
`BACKLOG.md`'s "the declaration table is not a paint order on every screen" had
been reasoned about but never measured. It is now measured, and the entry says
so both ways.

Confirmed: a framebuffer capture of Canary on the title screen shows the
`PROJECT SYLPHEED` wordmarks over the full-screen background, so declaration
order is not the paint order there.

Refuted, and recorded rather than quietly dropped:

* the `0x10`-adjacency first step this entry proposed. The background *is* next
  to a `kind = 0x10` `.prm` on both screens that have one — but on opposite
  sides (`GP_TITLE` puts the background after it, `GP_MISSION_SELECT` before),
  so no rule keyed on the `.prm` orders both. `GP_READY_ROOM` and `GP_OPTIONS`
  cannot serve as the third and fourth witnesses: neither carries a `.prm` or a
  full-screen background at all.
* the placement-region group order and the RATC child order, as above.
* reverse declaration order, by the same capture — it would draw the background
  over the copyright line, which is visible.

And separated out: part of what the bad render showed was not ordering at all
but the pivot/scale defect fixed two commits back. `ui-rat-layout.md` gains that
rule, the correlation evidence, the unmeasured pivot-vs-centre gap, and the
disc-wide demotion of "pivot is exactly half the texture" (49 % agree, 37 % are
off by more than 16 px — `ptcopyright` is 694 px wide in the framebuffer and
`pivot*2` says 618).

Captures: `title-screen-oracle.png` (the game surface cropped out of the
emulator window at (1,45), 1:1, bottom 45 rows clipped by the display) and
`title-screen-composite-2026-08-18.png` beside it.
2026-08-18 16:32:52 +00:00
Sylpheed RE agent
d67c1da467 formats/cli: the bundle's other orderings, and a guard that neither is a paint order
The title screen needs an order that puts element 13 (`ptbase2.t32`, the
full-screen background) behind elements 0-5 (the wordmarks) — the capture shows
the wordmarks on top, so the declaration table is not it. Two other orderings
the bundle carries were the cheap candidates, and both are now dead:

* the **placement region** stores a keyframe group per element with an explicit
  element index, so it could be a second ordering. It is not — it equals the
  declaration order on every build on the disc. `UiBuild::placement_order`
  exposes it and `placement_region_order_is_never_a_second_ordering` pins it, so
  the refutation stays checkable instead of remembered.
* the **RATC child order** is the declaration order with the `.prm` elements
  absent — strictly less information, and no place to put the background other
  than where the table already puts it.

`screen info --geometry` prints both, plus each element's decoded sprite size
beside `pivot*2` and every keyframe's scale/position/time — the numbers a
placement hypothesis has to be tested against, and how the pivot/scale rule in
the previous commit was found.

`title_background_is_full_screen` pins that rule against the disc rather than a
synthetic sprite. `scaled_elements_are_a_small_and_mostly_undiscriminating_minority`
reports the scope honestly: 865 of 5 130 resting placements are scaled at all,
and only 213 of those could tell "about the pivot" from "about the sprite
centre" — which the capture did *not* settle, because `ptbase2`'s pivot is its
centre. It also counts how far `pivot*2` is from the decoded size disc-wide
(2 521 agree, 1 884 are off by more than 16 px), which demotes the "pivot is
exactly half the texture" result to a property of the tutorial bundle.
2026-08-18 16:32:40 +00:00
Sylpheed RE agent
18947218b4 formats: a UI element scales about its pivot, not its keyframe corner
The compositor read a keyframe as `top-left = (X,Y)`, `size = decoded · scale`
and ignored the declared pivot. That is right at 100 %, which is every element
the format was ever checked against — the pause menu, the ARSENAL chip ruler —
and wrong for every element that is scaled.

Measured against a framebuffer capture of Canary on the title screen.
`GP_TITLE.pak` build 7 element 13 is `ptbase2.t32`: 640x360, pivot (320,180),
one keyframe at (320,180) with scale 200 %. From the corner that is a 1280x720
rect at 320..1600 x 180..900 — a quarter-screen slab with the top-left quadrant
bare. Anchored at the pivot it is (0,0)..(1280,720), and the capture shows the
background art reaching all four edges. Normalised cross-correlation of the
composite against the capture, searched over +-40 px, peaks at (0,0): 0.90 on
the planet limb, 0.72 on the lower-left ship.

`ptcopyright.t32` calibrates the other half: unscaled, 694x20 at (293,655), and
the capture's glyph run is x 295..986 / y 700..718 once the 45 px of window
chrome is taken off. So the keyframe really is the top-left at 1:1.

Disc-wide this moves 865 of 5 130 resting placements. The pause menu's own
`pgpeff01` glow stops hanging off the menu frame to the bottom-right and
centres on it.

`ComposeOptions::backdrop` comes with it: the default dim slate stands in for
the PRMD dim-quad behind an in-mission screen, but comparing against a
framebuffer needs the black the game actually composites over, so
`screen render --black` can ask for it.
2026-08-18 16:20:26 +00:00
MechaCat02
82a80e4063 docker: turn the agent loose — detached /loop, with the first-run gates handled
`./sylph-agent loose [task]` starts Claude Code detached with
--dangerously-skip-permissions, running /loop on loop-task.md: work the RE
backlog one item at a time, commit to auto/* branches, never push, record
withdrawn results rather than deleting them. `logs`/`attach`/`stop` to watch and
end it. Runs -d WITHOUT --rm so the transcript survives the container exiting —
for an unattended run that is the only record of what happened.

Two things had to be fixed for an agent to survive being left alone.

MEMORY CONTINUITY. The project is now bind-mounted twice: at /work, and at its
own host path. Claude Code derives its per-project state key from the working
directory, so running at /work handed the agent an empty project instead of the
accumulated one. Verified: a loose run now reports MEMORY=yes and reads back the
same branch and backlog as the host.

FOUR INTERACTIVE GATES, each a silent permanent hang with nobody at the keyboard
-- no error, no log line, just a container that looks healthy and does nothing:

  theme picker      hasCompletedOnboarding + lastOnboardingVersion. Re-fires
                    whenever the container's Claude Code is a different version
                    to the host's, which is the normal case.
  folder trust      projects.<path>.hasTrustDialogAccepted
  bypass disclaimer answered in a pty by bin/claude-autonomous. It has no config
                    key by design -- it wants a person to accept once, and the
                    person did so by launching this.
  fullscreen upsell fullscreenUpsellSeenCount. This one fires MID-SESSION, after
                    the pty wrapper has already handed over, so it cannot be
                    answered the same way.

Config key names were read out of the shipped binary's own strings, not guessed.

The pty wrapper matches SINGLE WORDS. Claude Code draws its UI with
absolute-column escapes between words, so the prompt arrives as
`Yes,\x1b[13GI\x1b[15Gaccept` and a multi-word pattern never matches -- failing
in a way indistinguishable from the wrapper not running at all. It stops
matching once the session is live so nothing later is answered by accident.

~/.claude.json is now mounted read-only at a staging path and copied in, so the
container cannot rewrite the host config. Credentials stay shared read-write in
~/.claude, which is what token refresh and memory continuity need.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 17:10:09 +02:00
MechaCat02
23fdf12194 docker: a container the autonomous RE agent can be turned loose in
Builds and runs both halves of the project -- Canary as the oracle, Reborn as
the port -- plus the dynamic-RE toolkit. Claude Code runs with
--dangerously-skip-permissions as an unprivileged `agent` user, because that
flag is refused under root.

Capped at half the machine, computed at launch: --cpus nproc/2, --memory half
of MemTotal with --memory-swap equal to it (no swap escape hatch -- a swapping
build thrashes the host, which is the failure the cap exists to prevent), and
build parallelism derived INSIDE the container from available memory rather
than core count, since a full-parallel build of this tree has OOM-killed the
host outright.

Three things the old box got wrong are fixed rather than reproduced: a real
toolchain (so rebuild_canary.sh's hand-relinking is obsolete), numpy and Pillow
(whose absence silently disabled every image oracle and looked like a logic
bug), and a display owned by PID 1 (so Xvfb no longer "dies on its own every
few minutes" -- it was being reaped because nothing owned it).

Verified end to end, not by inspection: the image builds, sylph-doctor is green,
`build-canary` links xenia_canary inside the container, and that binary then
runs -- guest memory and the JIT code cache appear in /dev/shm within 4 s,
1 205 log lines, gmem.py reads guest RAM, pad.py drives the file pad, and
screenshot captures the display.

Five environment defects found and fixed on the way, each of which fails in a
way that points somewhere else entirely:

  * /dev/shm is `noexec` under Docker. Xenia maps its JIT code cache out of an
    shm file, so it died with "Unable to allocate code cache generated code
    storage / Cannot initalize processor" -- which reads as an address-space
    clash, not a mount flag. Now `--tmpfs /dev/shm:rw,exec`.
  * An unknown xenia flag HANGS rather than errors: ParseLaunchArguments calls
    ShowSimpleMessageBox before logging is initialised, and that SDL dialog
    blocks on XIfEvent forever. `--audio` (which the RE notes recommend) is not
    a cvar in this tree; the symptom was a 10x10 window and an empty log.
  * Named volumes come up root-owned unless their mount point exists in the
    image, so the first cmake configure failed on pkgRedirects.
  * Ubuntu 24.04 ships its own uid-1000 account, colliding with the host user.
  * Ubuntu's spirv-opt has no --canonicalize-ids, so the shader step dies ~500
    objects in; the launcher mounts the host's LunarG SDK instead of baking one
    in, which also keeps shader output byte-identical to a host build.

Known limits, stated rather than papered over: on an NVIDIA host without the
NVIDIA Container Toolkit there is no hardware Vulkan (/dev/dri alone does
nothing for NVIDIA), and under lavapipe the emulator runs correctly but was not
observed to reach a rendered frame within a couple of minutes. gdb needs `sudo`
inside the container because the host's yama ptrace_scope outranks SYS_PTRACE.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 23:05:48 +02:00
MechaCat02
d921ed2602 viewer: fix a startup panic in the Save File browser (Bevy B0002)
draw_save_ui held both an EventReader<RequestSaveOpen> (to know when the View
menu opened it) and an EventWriter<RequestSaveOpen> (so its own "Open savedata…"
button could re-trigger the file dialog). Bevy rejects a system that accesses
one event type both ways, and does so at schedule-validation time -- so the app
panicked on startup, before any window content.

The button now sets `SaveBrowser::request_open` and handle_save_open_request
treats that flag as equivalent to the event, with no path (dialog).

Audited every system in the viewer for the same shape; this was the only one.
Verified by running the binary to steady state rather than by compiling alone,
since a param conflict is invisible to the type checker.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 21:34:10 +02:00
MechaCat02
221702ef6b docs/re: the declaration table is not a paint order on every screen
Building the Explorer's UI Screens browser turned up a limit on a claim the
format doc states flatly. `ui-rat-layout.md` says the element declaration table
is the screen's back-to-front draw list, verified 11/11 on the tutorial pause
bundle -- and that reproduces exactly here. `GP_TITLE.pak` build 7 does not:
painting in declaration order puts the full-screen background art (element 13)
over the wordmarks (elements 0-5).

Ruled out rather than guessed at: there is no depth key in the 60-byte entry.
Across the title build's 30 entries every unknown word is constant, and `+36`
-- the one that varies -- is an instance index on the `kind = 0x4` repeated
entries, not a layer.

Recorded as an open item with the cheapest next step (composite two more
screens that have both a background and overlapping foreground, and check
whether their background sits adjacent to a `kind = 0x10` PRMD entry as the
title's does). The viewer paints in declaration order and does not paper over
it, so a screen whose background lands on top is showing this bug rather than
a decode failure.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 20:33:45 +02:00
MechaCat02
af0796c835 viewer: UI Screens and Save File browsers (View menu)
Two new floating browsers, both fed only by static data — the disc for screens,
a savedata file for saves — following the existing Game Data / Ships pattern
(request event → worker thread → IsoLoaderMsg → draw system).

UI Screens
  Pick a screen pak and one of its builds (a build = one context × language
  variant; GP_PAUSE_MENU holds six) and see the screen reassembled at 1280×720
  beside the element list it was derived from, so a wrong placement shows up as
  a wrong picture and the row next to it says why. Per-element visibility
  toggles, focused-state and loop-animation toggles, and an inspector showing
  each element's sprite, parent link, kind flags, pivot and keyframe count.

  Compositing decodes ~30 T8aD sprites, which is far too slow for the UI
  thread, so it runs on a worker under the same generation gate the XPR path
  uses — a toggle mid-decode discards the stale result rather than racing it.

  The panel deliberately surfaces two things it would be easy to hide: sprites
  that named a child the bundle does not hold, and builds recovered through the
  .rat fallback (where elements without a record are simply absent).

Save File
  Opens a savedata file (it lives in the emulator's content tree, not on the
  disc, so this is a file dialog rather than a disc path). Shows the container,
  the GHAD block with every field coloured by its actual confidence, the
  Arsenal develop blob, the per-stage SHAB records, and the header summary.

  Three things the panel states rather than glosses: the byte-identical
  round-trip, which is the check the whole layout rests on; that SHAB records
  are per-stage results and NOT the UI's save slots; and that the header
  summary is what the Details panel reads, so a payload edit that leaves it
  stale proves nothing about the field that was edited. Fields tested and
  refuted (+36, +56) are shown as refuted rather than dropped.

Also gates draw_game_data_ui / draw_ships_ui at their registration site. Both
are #[cfg(not(wasm32))] but were registered unconditionally, so the wasm leg of
`just ci` failed on an undefined name. NOTE: that leg still does not build, for
a pre-existing and unrelated reason — the workspace pins tokio with features =
["full"], which pulls mio, which refuses to compile for wasm32. Fixing that
means restructuring tokio's features per target and is left alone here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 19:15:15 +02:00
MechaCat02
dc96bace6f formats: consolidate ui_layout onto the declaration table; add a savegame parser
Two independent lines landed a `.rat` reading and neither was the whole
picture, so this merges them into one module and fixes what the merge exposed.

ui_layout — the screen is the BUNDLE, not the set of .rat records
------------------------------------------------------------------
`feat/ui-layout-preview` parsed `.rat` records; the autopilot stack documented
the RATC header and probed it in `examples/screen_layout.rs` but never landed a
library module. The `.rat`-only reading structurally cannot see an element that
has no record -- the `eff*` frame corners, the `deli*` dividers, `msg` -- which
is exactly what the committed real-vs-rebuilt capture shows missing. Rebuilt
around the header:

  * element declaration table at 0x20 (60-byte entries: name, parent index at
    +32, kind flags, pivot) = the back-to-front draw list;
  * the placement region after it = per-element keyframe groups.

Verified against the disc, each against a fact the docs state independently:
`pgpeff02a` -> parent 3 = `pgpeff02`; `pgp_ttrl_btn10` rests at (546,288); the
pause buttons sit at 268/337/407/478, the documented 70 px pitch; the Arsenal
carries X = -516. The tutorial PAUSE menu now composites 11/11 elements and
matches the real screen more closely than the earlier rebuild did.

Three defects found while validating, none of which any test would have caught:

  * the keyframe block is 40 bytes with X/Y/time at +28/+32/+36 and an
    alpha-ramping ARGB at +0 -- the fade, previously unread;
  * a group's data stops 4 bytes short of its last block's time slot, so that
    word is the NEXT group's element index. Reading it produced times like
    1869640736 and silently corrupted the max-dwell pick. Last-frame time is
    now `None`;
  * the `.rat` sprite-name field is not 16 bytes. Capping it there truncated
    `pgp_ttrl_title.t32` to `pgp_ttrl_title.t`, which resolved against nothing
    and dropped 4 of 11 tutorial elements from the composite.

Max-dwell also needed a tie-break: on equal gaps take the LATER frame, or
`pgpmsg` reports the y=645 fly-through instead of the y=605 it settles at.

savegame -- a Rust port of tools/re-capture/savegame.py
------------------------------------------------------
GDHA container, zlib payload, chunk stream (GDAA / phase / GHAD 122 B / 16x20 B
SHAB / trailer). Every GHAD word carries its own confidence rather than the
block being presented as solved: 6 named, 2 recorded as REFUTED (+36, +56 were
tested as difficulty and as stage and are neither), 7 still unknown.

Tested against the three real saves committed under docs/re/captures -- no disc
and no emulator needed. The load-bearing assertion is the byte-identical
round-trip; the develop differential is asserted as a property (spending 4000 P
moves +24 and not its twin +28, steps the clear ratio, and moves exactly two
blob entries), and the header summary is checked to agree with the payload it
mirrors -- the trap that makes the Details panel a bad oracle.

CLI: `screen list|info|render` and `save info`, so both are checkable headlessly
in the same spirit as `mesh render`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 18:59:27 +02:00
MechaCat02
6d5b13e395 Merge remote-tracking branch 'origin/archive/local-work-pre-movie-sync-2026-07-20'
# Conflicts:
#	crates/sylpheed-viewer/src/iso_loader.rs
2026-08-17 18:13:14 +02:00
MechaCat02
d5784ab0fc Merge remote-tracking branch 'origin/feat/ui-layout-preview' 2026-08-17 18:11:49 +02:00
6c9380ff13 re(flight): a per-frame sampler, and nav oracles that a menu bar cannot break
WIP toward the residual flight-speed-law question (does a 1 s burst reach the
steady angular rate, or is there a per-axis multiplier?). The write-up already
concluded that host-side polling cannot answer it and named a Canary-side hook
as the tool required; that hook now exists (--frame_probe_log, committed as
auto/re-frame-probe in xenia-canary-native) and this is the harness for it.

- `rebuild_canary.sh` -- the surgical rebuild the box can actually do, kept in
  the repo this time instead of in /tmp: compile only the changed objects, `ar`
  them into their archive, and re-run the link command lifted out of the
  generated ninja. A full `ninja` is impossible here (several TUs need dev
  headers the image lacks) and the build cache cannot be re-configured. 31 s.
- `frame_burst.py` -- points the probe at the player craft's transform block
  (pos-112, the three 16-byte-strided rows plus the position) and drives full
  stick holds, recording each hold's start and end in the same clock the probe
  stamps its lines with.
- `frame_session.sh` -- the whole run as ONE blocking foreground call, per the
  session-lifetime rule; REUSE=1 drives a Canary that is already up.
- `nav_to_flight.sh` -- fly_stage.sh's navigation, split out so a live emulator
  can be re-used. A boot to the title costs minutes under lavapipe and a run
  that only failed to NAVIGATE should not pay for it twice.

The navigation change is the one worth reading. Every screen oracle here tested
named pixels ("648,221 is white"), which is only valid while the game image sits
at a known place on the root window -- and it does not: xenia's GTK window has a
menu bar, so on this display the image is ~25 px lower and every constant reads
the wrong row. Nothing errors. One run sat 300 s in front of a plainly visible
MAIN MENU reporting "no main menu"; the next missed the title screen entirely
and let the attract movie loop for ten minutes.

So `screen_id.py` identifies screens by WHOLE-IMAGE statistics instead -- the
fraction of green UI-text pixels, the fraction of near-white pixels, and the
per-channel means -- which no vertical shift, scale or letterbox can move. It is
calibrated against known-good captures and classifies all of them correctly:
title, three different menu screens, in-flight, and four movie frames as
"other". `bin/screenshot` additionally crops the menu bar off saved evidence
shots, deriving the offset from the window's own height rather than a constant.

Not yet a finding: the run has not reached flight, so no rate has been measured.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 06:03:44 +00:00
4f6fcf36dc re(flight): the ramp test is inconclusive, and names the tool the residual needs
To separate "a 1 s burst never reaches the steady rate" from "a per-axis
multiplier", measure inside ONE hold: successive 0.25 s windows of a single 3 s
press, holding speed, attitude and starting conditions constant by construction.

    rep0 rate   16   59  325  209  130  322  238  110  310  151  183
    rep1 rate   52  181  231  246  236  169  368  195  181  255  182

Not usable. At 0.25 s the windows do not contain enough guest updates to average,
so rate and speed both swing 3x window to window -- the same aliasing that once
manufactured a rate-vs-speed curve, reappearing at finer resolution. The first
window is lowest in BOTH repeats, which is what a ramp would look like, but the
sequence never plateaus, so the signal cannot be separated from the sampling. No
claim either way.

Widening the window does not rescue it: 0.5 s averages well enough, but a hold
long enough to contain several 0.5 s windows bleeds speed -- and speed is the
variable under test. The two effects are entangled at this observation rate.

So the residual needs a different INSTRUMENT, not another script. Live-RAM polling
samples an unsynchronised snapshot; the question wants the craft's angular
velocity as the guest computes it, once per frame. That is a Canary-side hook --
the same shape as the existing F10 ship-capture patch -- and the rebuild toolchain
already makes it cheap. Recorded as the recommendation rather than attempted as a
seventh variation of the same measurement.
2026-08-13 23:53:23 +00:00
8b03a52aa6 re(flight): both axes at settled speeds -- shape confirmed, absolute scale is not
Burst design repeated with the HUD clock bracketed (01:01.86 -> 02:10.61, ~x1.26)
and roll measured in the same flight.

    pitch   87.0 @ 85    82.8 @ 308    47.0 @ 1077   min:max 1.85
    roll   125.0 @ 76   119.0 @ 291    82.6 @ 1023   min:max 1.51

SHAPE CONFIRMED on both axes: the rate interpolates between _Min (at minimum
speed) and _Max (at maximum speed), matching to 1.6% for pitch (vs PitchMinus
75/40 = 1.88) and 5.6% for roll (vs 200/125 = 1.60). Both ratios are
clock-independent so they stand regardless of the conversion, and pitch's
re-confirms ly+ = pitch-MINUS (reproducing 1.82 from the previous run).

ABSOLUTE SCALE DOES NOT MATCH, and not in the same direction:

    pitch  measured/predicted = 1.15, 1.21, 1.07   consistently OVER
    roll                      = 0.62, 0.64, 0.60   consistently UNDER, very flat

A clock error cannot explain this -- it would move both axes together and these
go opposite -- so the discrepancy is per-axis. Candidates, neither measured: a 1 s
burst not completing the angular-acceleration ramp (would under-read; fits roll's
flat 0.62, not pitch's excess), or an unidentified per-axis multiplier. Left as an
open question with the separating test named: 1 s versus 3 s bursts at the same
throttle, where the ramp hypothesis predicts the longer burst reads higher.

Also noted: the roll run correctly REFUSED on a WEAK row pin after the pitch
bursts left the craft tumbled -- and roll is immune to that labelling, which the
guard's own message says, so ALLOW_WEAK_PIN=1 is its documented exception. A guard
that states its own exception is worth having.
2026-08-13 23:42:28 +00:00
92d9683f3a re(flight): short bursts confirm the rate-vs-speed SHAPE, without needing the clock
The design the sweep could not provide: settle the throttle, measure the settled
speed, pitch for ONE second so speed barely moves inside the burst. Three
throttles, two repeats, row pin CONFIDENT, fresh flight.

    LT min      burst speed ~105    rate 113.6, 109.5 deg/wall-s
    cruise                  ~383         100.2,  88.4
    RT max                 ~1483          52.2,  70.5

Rate falls monotonically with speed -- 111.5 -> 94.3 -> 61.4 -- at three KNOWN,
SETTLED speeds instead of smeared across a bleeding one.

The decisive comparison needs no clock. Absolute rates depend on the run's clock
ratio, but the min:max RATIO cancels it:

    measured min:max            = 1.82
    AV_PitchMinus_Min/Max 75/40 = 1.88   ->  3.0% apart
    AV_PitchPlus_Min/Max 150/70 = 2.14   -> 15.1% apart

Two conclusions, neither resting on a clock measurement:
 - _Min/_Max really do mean "at minimum / at maximum speed", with the rate
   interpolating between them: shape confirmed to 3%.
 - ly+ drives pitch-MINUS, not plus. The craft has asymmetric pitch authority
   (75/40 down vs 150/70 up) and the ratio picks the pair cleanly.

Absolute magnitudes remain open: this run did not bracket the HUD clock, so
deg/GAME-second cannot be computed from it, and picking a ratio that makes the
numbers fit would be circular. The probe now screenshots the clock at both ends.

Also: fly_stage.sh now waits for the TAKE-OFF load too. Guarding only the stage
load left a run pressing A into a black screen and then reporting "player entity
not found" from a game that never reached flight.
2026-08-13 23:29:23 +00:00
a3f14710a4 re(flight): clean pitch sweep -- magnitudes agree, the interpolation law does not
Fresh flight, row pinning CONFIDENT (margin 0.413), one sweep and nothing before
it. axis_probe now REFUSES to measure on a WEAK pin (ALLOW_WEAK_PIN=1 overrides)
since it is a precondition, not a warning: roll is immune to the up/right
labelling but pitch and yaw are not.

Clock x1.26. Binned by speed, both in game units, against the linear
interpolation of AV_PitchPlus_Min 150 (at MinimumVelocity 100) to _Max 70 (at
MaximumVelocity 1200):

    speed ~435   measured 100.8   predicted 125.6
    speed ~572            113.8             115.7
    speed ~709            126.3             105.7
    speed ~846             83.1              95.7
    speed ~983             72.7              85.8

Supported: the magnitudes (73-126 measured vs 86-126 predicted) and a falling
high-speed end. NOT supported: the interpolation law. Scatter is +-25%, the two
fastest bins hold 1 and 2 windows (the first moments before the speed bled), and
the slowest bin misses in the wrong direction.

The flaw is structural, not statistical: a sweep DRIVEN by the speed bleeding
cannot dwell at either extreme, which is exactly where the law is most testable.

What would settle it: hold a settled throttle and pitch for ~1 SECOND, so speed
barely moves inside the burst and each burst gives one honest (speed, rate) point;
repeat at LT / neutral / RT for three clean points at known speeds. Recorded as
the next design rather than attempted as a fifth variation of the same sweep.
2026-08-13 23:12:50 +00:00
0a84c1358e re(flight): polling faster than the guest updates manufactures a clean curve
Fitting rate against instantaneous speed produced a tidy "rate rises with speed"
relationship, with speeds up to 4795 when the craft's maximum is 1200. It is
entirely an artefact: 20 Hz polling is faster than the guest updates these fields,
so a per-read delta is either exactly zero (no update yet) or a whole frame's
worth divided by a fraction of a frame. 111 of 352 reads were zero on BOTH
channels -- position and attitude update on the same frame, so the two are
perfectly correlated, and dividing each by the short wall dt produced the
correlation out of nothing.

Fix: aggregate over windows spanning many frames (0.5 s). A sum of |delta| over
such a window is right however the updates fall inside it.

This does NOT affect the swept-total probes (roll_axis.py, rate_probe.py) -- they
already summed over the whole dwell, immune for the same reason. Only per-sample
instantaneous rates were ever wrong, so no earlier number moves.

The windowed re-run is NOT yet claimed as a result. It gives plausible magnitudes
but still shows rate rising with speed, against the definition's PitchPlus_Min 150
> _Max 70, and it has two disqualifiers: it ran on an instance where the craft was
already tumbling from the previous sweep, so pinning reported "WEAK -- craft may
not be level", and the sweep started mid-range rather than at maximum. A clean
answer needs a fresh flight with pinning CONFIDENT and nothing before it. Since
what is in doubt is precisely what _Min/_Max mean, a measurement through a
doubtful instrument cannot settle it.

Both datasets kept, the bad one labelled, because the aliased curve is a good
example of what a manufactured correlation looks like.
2026-08-13 23:00:59 +00:00
6c7025851e re(flight): the rate probe measures a MOVING speed -- pitching bleeds it hard
Measured pitch with the rows properly pinned, against this craft's own disc caps
(AV_PitchPlus_Min 150, AV_PitchPlus_Max 70):

    min speed (LT)  1391.0 deg / 8.00 s, clock x1.326 -> 131.1 deg/game-s  vs 150
    max speed (RT)   989.0 deg / 8.05 s, clock x1.318 ->  93.2 deg/game-s  vs 70

A rate 33% ABOVE a cap is not a finding, it is a broken instrument. The HUD speed
is in the same bracketing screenshots that give the clock, so read it:

    slow phase   102 ->  18
    fast phase  1193 -> 589

The speed is NOT constant during the dwell -- pitching halves it in 8 seconds.
The cap is speed-dependent, so as the craft slowed its cap rose, and an 8-second
average necessarily lands between the max-speed cap and a mid-speed one. The 133%
is entirely the instrument.

This also weakens the roll result committed earlier: same method, so 120.9 vs
AV_Roll_Max 125 is CONSISTENT but is not a tight test -- the true cap could be
lower and still produce that average. Said plainly in the doc rather than left
standing as a clean confirmation. Min-speed figures are less affected; there is
little speed left to lose.

Proper fix, not yet done: dwell ~1-2 s so speed barely moves, or sample HUD speed
continuously and fit rate against INSTANTANEOUS speed -- which yields the whole
rate-vs-speed curve instead of two points.

Separately this is a flight-model finding: TURNING COSTS SPEED, steeply, with the
throttle still at maximum. A reimplementation treating the throttle as a speed the
craft simply holds will be wrong during manoeuvres.
2026-08-13 22:48:29 +00:00
dc46339b63 re(flight): rows pinned by world-Y, and nothing yaws -- the yaw gap is answered
The probe now pins which non-forward row is up and which is right, instead of
taking the D3D convention on faith:

    row world-Y means: [0.469, 0.883, -0.000]   forward = row 2
      -> up = row 1, right = row 0   CONFIDENT

That is the OPPOSITE of the assumption the previous run used, so that run's yaw
and pitch columns were swapped -- under the correct labels its ly+ reading of
154.1 deg/wall-s is PITCH, which is what a left-stick Y should do.

Measured with the unknown inputs FIRST, each passing a liveness check:

    rx  0.0  0.0  0.0      ry  0.0  0.0  0.0
    LB  0.0  0.0  0.0      RB  0.0  0.0  0.0
    lx  roll 209.8         ly  pitch 154.1

These zeros are trustworthy where the previous run's were not: the craft was
verified alive between inputs, and the probe aborted the moment it stopped moving
rather than reporting the clean zeros a destroyed craft produces (it did abort,
after lx+, which is why lx/ly are carried from the earlier run rather than
re-measured).

So NO PAD INPUT YAWS THE CRAFT. AV_Yaw_* (45/25) exists in the definitions but
nothing on the right stick or the shoulders drives it, which upgrades the old
"yaw: no input found" from a failure to find one into a measurement that the
remaining candidates do nothing. The natural reading is that yaw is a consequence
of banking rather than a commanded axis.

Not covered, and not claimed: the d-pad (tactical map) and the face buttons
(fire/weapon select). Neither is a plausible flight axis; neither was measured.
2026-08-13 22:35:36 +00:00
dfa769420d re(flight): axis probe pins the rows and checks liveness; fly_stage waits, not sleeps
Both fixes the previous run's caveats asked for, plus one the run itself forced.

axis_probe.py now:
 - PINS which non-forward row is up and which is right, by comparing world-Y
   across the rows in level flight, and says CONFIDENT or WEAK. entities2
   measures row 2 = forward against velocity, but the other two were labelled by
   the D3D convention, and yaw/pitch SWAP if that is wrong -- so the previous
   run's last two columns were named on an assumption.
 - checks the craft is ALIVE between inputs, and ABORTS with a message instead of
   reporting the clean zeros a destroyed craft produces. The first run ended on
   GAME OVER and only said so afterwards.
 - measures the UNKNOWN inputs (rx, ry, LB, RB) first while the craft is healthy,
   keeping the established lx/ly as controls at the end.

fly_stage.sh now WAITS for the stage load instead of sleeping a fixed guess. The
fixed sleeps worked until they didn't: one load ran long, the script pressed START
into a black screen, and every later step went to nothing while the screenshots
recorded a plausible-looking sequence. It now polls for a non-black frame and
aborts with a pointer to the log if the load hangs (PhysicalHeap::Release
failures) rather than continuing blind.

The probe itself did not run this iteration -- the stage load hung -- so there is
no new axis data, and none is claimed.
2026-08-13 22:22:19 +00:00
5f3c618b51 re(flight): axis probe -- lx is roll, ly drives one clean axis, rest not trustworthy
New probe (axis_probe.py) decomposes every held input into all THREE rotation
components at once, instead of measuring one axis at a time through a non-forward
matrix row -- the flaw that once made roll and pitch produce identical numbers.
For previous rows (f,u,w): roll = atan2(u.w_old, u.u_old), and forward's rotation
toward each of the other two rows gives the remaining pair.

Stage 02, file pad, full deflection on exactly one channel at a time:

    lx+   roll 209.8   b 0.9    c 10.1     deg/wall-s
    ly+   roll   0.0   b 154.1  c  0.0
    rx+   roll   0.0   b 0.0    c  0.0
    ry+   roll 161.1   b 87.0   c 37.1
    LB/RB all zero

What this supports: lx = ROLL, cleanly (~0 on both other channels), agreeing with
the independent roll measurement. ly drives ONE axis, cleanly.

What it does NOT support, and I am not claiming:
 - WHICH axis ly drives. entities2 measures row 2 = forward against velocity, but
   rows 0 and 1 are labelled up/right by the D3D convention rather than by
   evidence, and yaw/pitch SWAP if that is wrong. Roll is immune (rotation of
   either non-forward row in their shared plane is roll either way).
 - anything about rx/ry/LB/RB. The run ended on GAME OVER: full-deflection spin in
   a live combat mission gets the craft destroyed, and the only symptom is "0
   player candidates" AFTERWARDS, so late rows may be post-death. rx+ reading all
   zeros and ry+ reading mixed are exactly what a dying craft would produce.

So "yaw: no input found" is NOT resolved. Both gaps are now written into the
probe's header with what would fix them: a liveness check between inputs, and
pinning up-vs-right against world Y.
2026-08-13 22:10:23 +00:00
850b04c606 re(flight): roll DOES depend on speed -- the withdrawal is reversed
Measured about the FORWARD axis (roll_axis.py), which is the fix the withdrawal
specified: express the new up-vector in the old (up, right) basis and take
atan2(u.w_old, u.u_old), so the component along forward -- what pitch produces --
is dropped by construction. The old probe watched a non-forward matrix row, which
sees any rotation that moves it, and that is why two different stick axes produced
the same numbers.

Stage 02, file pad, single-axis holds (lx=32767 with every other channel exactly
0, trigger held in the same write since the pad state is written whole), 5 s
settles, 8 s dwells, each phase bracketed by HUD-clock screenshots:

  min speed (LT)  1 856.9 deg / 8.01 s wall, clock x1.247 -> 185.9 deg/game-s
                  vs AV_Roll_Min 200
  max speed (RT)  1 181.2 deg / 8.02 s wall, clock x1.218 -> 120.9 deg/game-s
                  vs AV_Roll_Max 125

So roll behaves exactly like pitch: the rate cap falls as speed rises and
_Min/_Max mean "at minimum / at maximum speed". The withdrawn claim that roll
shows NO speed dependence is reversed, and the axis rule is now confirmed on a
second axis instead of contradicted by it.

Both land just under their caps (93% and 97%), the right side for a rate limit.
The shortfall is NOT explained and no claim is made about it: candidates are the
craft not being exactly at min/max speed after 5 s, and the 20 Hz swept-angle sum
undercounting.

The tell that the old result was broken was two conditions agreeing too well. The
tell that this one is sound is that they disagree in the direction the definitions
predict, on two independently bracketed phases.
2026-08-13 21:57:10 +00:00
3d3d6726fa re(flight): START skips the briefing, and the player-entity lock is stage-specific
The scripted route now reaches FLIGHT unattended: boot -> main menu -> poke the
cleared-stage mask -> EXTRAS -> MISSION SELECT -> stage -> briefing -> READY ROOM
-> TAKE OFF -> flight, verified by a full HUD (TIME 00:52.54, speed 350, shields,
REMAINING OB 018) on stage 01.

Two things that cost a run each:

START skips the briefing; A does not. A pages through the brief, and ten A taps
still left the run sitting on a briefing screen -- twice, on two different
stages. One START press lands on the READY ROOM. fly_stage.sh now presses START.

entities2.py's `self` locked on `"Player" in name`, and that suffix is
STAGE-SPECIFIC: stage 02 fields UN_f002_TCAF_DeltaSaber_W_Player, but stage 01
fields UN_f001_TCAF_DeltaSaber_T with no suffix, so the filter found nothing
while the game was visibly flying and reporting 64 typed live entities. Falls
back to the craft class and prefers the instance that is actually moving (a
mission holds more than one). Recorded rather than worked around, because the
same assumption is embedded in several probes.
2026-08-13 21:45:54 +00:00
0b2f004ad0 re(flight): port the roll probe to the file pad, and drive fly_stage.sh to flight
roll_axis.py was written last session to re-measure roll ABOUT THE FORWARD AXIS --
the fix for the withdrawn result, whose defect was the MEASUREMENT (a non-forward
matrix row sees any rotation that moves it, and pitch moves it as much as roll)
and not the input device. It was never run, and it still drove vgamepad, which is
retired for leaking to the host.

Ported to the file pad. The important difference is not the device but the shape:
the file pad is a SNAPSHOT, not independent channels -- each write replaces the
whole state -- so holding a trigger while deflecting a stick has to be one write.
pad_state(**kw) does that. The values are also exact rather than whatever a
virtual stick quantises to, which is what makes a single-axis hold (lx=32767 with
everything else exactly 0) trustworthy for an axis-separation question.

fly_stage.sh now drives the rest of the route it previously stopped short of:
briefing (A: Continue) -> READY ROOM -> up to TAKE OFF -> flight, and notes why
it must get there at all (a snapshot at the briefing yields zero unit-definition
objects; they are instantiated at stage load proper).
2026-08-13 21:37:44 +00:00
faa7b3d611 re(challenge): the part id is never persisted -- differential search says stack only
No literal 26 exists anywhere, so the GamePart id is computed. That does not stop
it being found: the id is KNOWN at each screen from the GamePart table (EXTRAS = 5,
MISSION SELECT = 7), so snapshot both and intersect. New tool diff_words.py does
the classic differential search over the sparse guest image, and find_partslot.sh
drives the two screens and runs it.

Result: 171 MB scanned, exactly 4 addresses read 5 then 7 -- 0x708FFBEC,
0x708FFCBC, 0x708FFDAC, 0x708FFE20 -- and all four are guest STACK (the same run's
log puts thread stacks at 0x709...). So the requested part id exists only as a
stack argument in flight; there is no persistent field, which is consistent with
finding no literal store, and means there is nothing stable to poke.

That closes the last memory-and-menu route to the challenge missions. Reaching
them needs either the genuine in-game unlock (an in-mission attainment, per
AVSCRIPT_COMMAND_ATTAINMENT_CHALLENGE_MISSION_CARGO_SCORE) or an emulator-side
hook that forces the transition -- a code change, not a poke.

diff_words.py is worth keeping well beyond this question: it locates any field
whose address is unknown but whose value is known at two moments.
2026-08-13 21:31:47 +00:00
a79a1da183 re(challenge): withdraw the kind-field constructor attribution -- snapshot refutes it
Section 4 claimed the mission-kind field at obj+144 "is initialised to 0 in the
constructor sub_821783D8, alongside +148 = 0, +132 = 2, +136/+140 = -1". That
linked two code regions on nothing more than both touching +144/+148, and an
in-flight snapshot refutes it:

sub_821783D8 initialises the STATIC at 0x828F3EC0 -- its first act is
InitializeCriticalSection(obj, 256) -- and in the snapshot that object holds
0x000B1C8B at +144 and a float at +592. Not a kind, not flags. So the object that
owns the kind field is UNIDENTIFIED. Scanning the snapshot for it (kind in
{0,3,5,6} at +144, stage 10 at +148, pointers at +0/+52/+604) returns only
matches inside the executable's own static data; the value 10 at +148 is far too
common to discriminate.

The switch itself stands -- it is direct disassembly at two independent sites
(0x82184df0, 0x82185ed0) plus two grouping tests. Only the attribution was wrong,
and the wrong half is the half I inferred rather than read.

Confirmed on screen instead: launching a story stage from MISSION SELECT reaches
a READY ROOM carrying an "EXTRA" watermark -- the EXTRA config section, kind 3,
visible in the UI. Independent evidence the field means what section 4 says even
though its owning object is not pinned.
2026-08-13 21:21:49 +00:00
4cbce6bd21 re(units): S10 closes the story campaign for real -- 69 units, 7 204 values
roster_target had flagged a contradiction: S10, a STORY stage, still fielded an
unharvested unit, which the "story campaign complete at 68 units" claim did not
account for. S10 was simply never flown. Flying it settles it:
UN_e005_ADAN_ElanTypeQ_Margras is in the snapshot, 144 rows, and the merge takes
the file 68 -> 69 units, 9 393 -> 9 537 rows, 7 115 -> 7 204 defaulted-on-disc
values, with 154 disc cross-checks agreeing and 0 disagreeing. roster_target now
reports S10: 0 missing, and every remaining gap is a CHALLENGE stage
(S24/S25/S27/S28/S29).

A much simpler way to fly a story stage, replacing the save-editing route
(tools/re-capture/fly_stage.sh): poke ONE word -- 0x828F40C0 = 0x0001FFFE marks
stages 1-16 cleared -- and MISSION SELECT will launch any of them. Nothing is
written to disc, so there is no save to back up and restore, and TRAP 1 from the
old recipe (launch_mission.sh silently loading the last-used slot) cannot happen.

Route, learned by screenshotting each step: MISSION SELECT -> pick stage -> A ->
mission briefing (A: Continue) -> READY ROOM -> TAKE OFF -> flight. The READY
ROOM carries an "EXTRA" watermark, which is the static analysis's mission-KIND
= 3 ("EXTRA" config section, docs/re/challenge-mission-gate.md section 4) visible
on screen -- an independent confirmation of that field's meaning.

Also recorded: a snapshot taken at the BRIEFING yields 0 runtime objects. Unit
definitions are instantiated at stage load proper, so the snapshot has to wait
for flight; the briefing screen is too early.
2026-08-13 21:14:21 +00:00
aac017dd0d re(challenge): poking unlocks the whole story campaign; MISSION SELECT is story-only
Two results from the running game, one positive and one a clean negative.

POSITIVE: with word A = 0x0001FFFE (stages 1-16) every entry Stage01..Stage16 is
selectable, where the control run had only Stage01 and the rest greyed. Stage16
reads "Lonely Blue Planet - NO RECORD". So any story stage can be launched from
the menu by poking one word, with no save editing at all -- a simpler lever than
the GHAD stage-field patch used until now.

NEGATIVE: with word B = 0x3F (challenge stages 24-29 marked cleared) the list
still saturates at Stage16 -- the cursor stops there and further presses do
nothing. That matches the disc: the debriefing config declares exactly
px_deb_stage01..16, so the list is capped by data, not by the mask. The
challenge missions are NOT reachable through MISSION SELECT, and word B does not
feed it.

Also mapped, without finding the caller: the GP_DIALOG registry (tables.pak #41)
gives DLG_GO_CHALLENGE_MISSION_MENU = 41 and DLG_CHALLENGE_MISSION_AVAILABLE = 42
(0-based, in config order). No raw immediate 41/39/37 appears anywhere in the
GamePart code region, so dialogs are raised through a computed index and the
entry point to GamePart 26 is still unknown. New probe: examples/screen_configs.rs
dumps any tables.pak screen config by substring.
2026-08-13 20:54:57 +00:00
3ef2c438ae re(challenge): MISSION SELECT renders the cleared-stage mask, and a control run
With word A = 2 the screen lists Stage01 SELECTABLE, carrying a High Score and a
Best Time, and Stage02-Stage08 GREYED OUT. One cleared stage, one selectable
entry, at the bit index that names it -- the mask's meaning is now visible on
screen rather than inferred from disassembly.

Two runs, identical navigation, fresh boot each:
  control  word A 0x00000002  -> MISSION SELECT opens normally
  poked    word A 0xFFFFFFFF  -> MmAllocatePhysicalMemoryEx fails on 128 MB,
                                 guest throws, Xenia shows "Disc Read Error"
So last commit's heap failure was caused by the poke, and by a careless one:
0xFFFFFFFF claims stages that do not exist (0, 17, 24-31 in word A). Poking only
real story ids (0x0001FFFE = stages 1-16) does not blow the heap. That the list
screen changes behaviour with the mask is itself evidence word A feeds it.

Getting a trustworthy control took three tries, and every failure produced a
plausible wrong answer rather than an error:
  1. the title-glyph oracle fired during the ATTRACT MOVIE, so A was pressed at
     nothing and the run "reported 0 failures" for a screen it never reached;
  2. Xvfb keeps the previous instance's framebuffer until the new one draws, so
     a screenshot seconds after launch showed the OLD run -- "MAIN MENU reached
     after 1s", against a process that no longer existed;
  3. a single-pixel "is NEW GAME white?" test matched a white LOADING FLASH.
Fixes, all in challenge_probe.sh: blank the root and refuse screen oracles for
the first 40 s; identify a screen by a PATTERN of sampled points (white text AND
the dark panel behind it), not one pixel; require two consecutive samples. The
new oracle was validated offline against all six saved screenshots and accepts
exactly the one real main menu.

General lesson worth the words: a navigation oracle that can only fail by
returning the wrong screen will happily produce a whole run of confident,
meaningless results.
2026-08-13 20:44:27 +00:00
0bc790de52 docs(index): record the file-input pad and the live-write tool 2026-08-13 20:29:18 +00:00
023bb71cfd re(challenge): the cleared-stage mask is CONFIRMED on the running game
Booted the title and read the two gate words live:

    0x828F40C0 = 0x00000002     word A
    0x828F4814 = 0x00000000     word B

Word A = 2 = bit 1. The profile's save is Stage 02 "At Standby" -- stage 01
cleared -- so the mask is exactly one bit, at the index of the one cleared
stage, 1-BASED. Reproduced across two cold boots. That confirms against a known
progress state, on the real game:

  - the singleton is the static object at 0x828F4070, as derived statically;
  - word A is a cleared-stage bitmask (not achievements, not a stage number);
  - bit index = stage id, 1-based, so TimeAttack's REQUIREMENT 16 means "clear
    stage 16" -- the last story mission;
  - word B is the challenge half and is 0 on a story-only profile.

New tools: gpoke.py (live guest-memory WRITE, companion to gmem.py, prints
before/after for every word), pad.py (drives the new --hid=file pad; replaces
vgamepad, which leaked to the host through /dev/uinput), challenge_probe.sh
(one blocking session: boot, wait for title, drive in, poke, screenshot).

Poking both words did NOT surface a challenge entry in EXTRAS -- and that menu
was built 26 s after the poke, so it is not staleness. Entering MISSION SELECT
then failed, but the log names the real cause and it is not the gate:
MmAllocatePhysicalMemoryEx could not satisfy a 128 MB request (parent free
30633/131072 pages), the guest threw a C++ exception, and Xenia surfaced its
generic "Disc Read Error". It is preceded by "BaseHeap::Release failed because
address is not a region start" -- a failed release leaking the range. Recorded
as an emulator heap problem, with the control run (same navigation, no poke)
named as the next step.
2026-08-13 20:28:19 +00:00
f490fecefb re(challenge): both gate words are one ~1880-byte record, and it is not the save
Two results, one of which kills an operational hope I had been carrying.

THE RECORD IS MUCH BIGGER THAN I MEASURED. Reading 0x82175110 to the end: after
the two words, the 8-byte pair, the 184-byte memcpy and the 8 words at +200, it
copies 816 bytes at +232, 816 more at +1048, a sub-object at +1864 and a final
word at +1876. So the record spans +0..~+1880 = singleton +80..+1960, which means
word A is record +0 AND WORD B IS RECORD +1876. That retires last commit's "word
B has no known writer": there is no separate store because the whole record is
copied out, modified and assigned back as a unit (0x8216FF70 -> compare
0x822C3708, assign 0x82170650, then a worker 0x821700A8 -- note 168 decimal, not
hex, which I misread first time -- retrying a commit 0x822C33B8 up to five times).

IT IS NOT THE SAVEGAME, by two independent checks:
 - size: every real save on disk is a 276-byte container deflating to 545 bytes;
   the record is ~1880. It does not fit.
 - files: after many sessions the content tree holds only game0N/savedata and
   game0N/__thumbnail.png per slot plus three Headers/*.header. No second data
   file exists anywhere under the title id.
Also negative: the savegame object is *(*(this+4)) + 304, and the singleton's
holder address 0x828F48B0 is referenced NOWHERE outside the accessor, so this+4
is a different holder -- the save block is not a window into this record.

=> Hand-editing a save cannot unlock the challenge missions. The earlier
savegame-editing win does not extend here.

WHAT WOULD WORK. The singleton is a static object at 0x828F4070 (0x8216F650:
addis 0x828F + addi 16496), so the gate words sit at fixed guest addresses with
no scanning: word A = 0x828F40C0, word B = 0x828F4814. Canary maps guest RAM into
/dev/shm and the project already reads it live, so writing 0xFFFF / 0x3F there
while the title sits on a menu should open all six challenge missions without
playing the campaign -- the route to the last 42 EX units. Untested; needs a run.
2026-08-13 20:06:11 +00:00
20299c05a1 re(challenge): the gate is CLEARED STAGES -- my achievement reading was wrong
Third revision of this claim, and this one has the writer.

Word A (singleton +80) has exactly ONE writer in the image. Scanning all 22
callers of the +80 struct copier (0x82175110) for one that stores to the copy's
word 0 gives a single hit, 0x821C1820, inside GamePart_StageClear:

    if (this+1004 & 0x20000) skip           ; already recorded
    copy local = singleton->progress        ; src = obj+80
    local.word0 |= 1 << (this+84)
    if changed: singleton->set(local)       ; assign + async persist

and this+84 is the STAGE NUMBER, by two independent uses: 0x821C1760 indexes a
20-byte per-stage record array with it (this + (x+7)*20, three words plus an
8-byte timestamp -- the savegame's 16x20 SHAB shape), and 0x821C1EEC/0x821C1924
pass it as the index into the config key "STAGE" (0x820A2540), the debriefing's
px_deb_stage01..16 sprite list.

So a challenge mission's REQUIREMENT n means "stage n has been cleared", and the
<24 / >=24 split is the disc's own stage numbering: story 1-16 and tutorial 18-23
in word A, challenge 24-29 as word B bits 0-5. TimeAttack needs stage 16 -- the
last story mission -- and the other five chain off challenge stages 25-29. That
is what the raw numbers suggested in the first place.

REFUTED: "the bit space is the game's 24 achievements". ACHIEVEMENTS_REQUIREMENTS
having 24 entries and the gate splitting at 24 is a COINCIDENCE -- 24 is also the
first challenge stage's id. Nothing copies the Debriefing's masks (+208/+736/+740)
into +80, which is the check that should have preceded the claim.

The achievement work itself stands and is kept, retitled and rescoped in
structures/achievements.md: the XACH table (.pe 0x8FBCBC, 36-byte records,
1000G self-check), the 24 names/descriptions, the on-disc ACHIEVEMENTS_REQUIREMENTS
list, and the XACHIEVEMENT_DETAILS/XAM read path. It just does not gate the
challenge missions.

Still open: word B (+1956) has no known writer -- GamePart_StageClear's
unconditional 1 << x would land a challenge stage on word A bits 24-29, not word
B, so clearing a challenge mission must be recorded by another path (presumably
the one that also stores its Time/Points record).
2026-08-13 19:54:39 +00:00
90e0e66c7b re(achievements): earned state comes from XAM, and my bit numbering was wrong
Two findings, one of them a correction to the previous commit.

FOUND: GamePart_Debriefing enumerates XACHIEVEMENT_DETAILS from XAM
(0x8218F888). The records are 36 bytes -- confirmed by the arithmetic, not by
eye: the count is a byte count divided by 36 through the 0x38E38E39 multiply-high
magic plus srawi 3. Field +0 is used as a shift amount, field +32 is tested for
0x00020000, and the buffer sits behind a handle that is waited on (0x824AA330
with -1) then closed (0x824AA3E0). That is the XDK struct exactly, with
XACHIEVEMENT_DETAILS_ACHIEVED == 0x20000, via the
XamUserCreateAchievementEnumerator / XEnumerate pattern.

So the title does NOT persist earned achievements itself -- it asks the console.
The lever for achievement-gated content is the emulator's PROFILE data, not the
savegame. Independently, the singleton the challenge gate reads is not the object
holding the save block: re-scanning all 202 accessor call sites WITH function-
boundary stops touches only +80/+1956/+1960/+1964 and none of the known save
offsets (+304/+316/+320/+336/+380/+440). The earlier scan that seemed to find
them had register tracking bleeding into the next function -- a false positive I
am recording rather than quietly dropping.

CORRECTED: the previous commit claimed "bit n <-> achievement n+1" and treated
the Debriefing award pass and the challenge gate as the same bit space, both as
confirmed. Neither holds up:

- the only place the image is observed turning an achievement into a bit does
  1 << dwId with 1-based ids, so bit = the id and bit 0 is unused;
- the "list index is the bit index" step assumed 0x82448338's out-parameter is
  the loop ordinal. It is the child entry's first word out of a 12-byte array,
  and whether that is an ordinal, an id or a name hash is not pinned;
- the Debriefing's masks are its own fields (+208/+736/+740). Nothing observed
  copies them into the singleton's +80, and no writer of that bitmask has been
  found at all.

The requirement-type/id agreement still stands, but it confirms the LIST ORDER,
not the bit numbering. Consequence: if the join holds, TimeAttack's REQUIREMENT
16 is achievement 16 (Night Ravens Patch), not 17 (Solar System Defense Award).
Both read plausibly as a first-challenge gate, which is exactly why it needs
evidence and not the better story.

Also noted: +1960 is a third bitfield on the same singleton, and a what-changed
pass at 0x8219F3A4 diffs it and reports each newly set bit as bit + 64 -- so
there is a wider flag-id space whose bases are not yet worked out.
2026-08-13 19:43:35 +00:00
33ae20896e re(challenge): the gate's bit space is the game's 24 ACHIEVEMENTS
Static only. Last commit left "REQUIREMENT is a bit index into a progress
bitfield" with the space unidentified. It is the achievement space, and both
halves are now readable off the disc and the executable.

- GamePart_Debriefing (0x8218CF38-0x82191B18) awards them: sub_8218F9A8 walks
  the on-disc ACHIEVEMENTS_REQUIREMENTS list (tables.pak #16, schema 744c0519),
  and for entry index n tests bit n, evaluates the entry when clear, and sets
  the bit when satisfied. The list is literally ACHIEVEMENT01..ACHIEVEMENT24 --
  24 entries, which is exactly where the challenge gate splits word A from
  word B.

- The XEX carries the definitions: XACH at .pe 0x8FBCBC, 36-byte records
  {id, name_id, unlocked_desc_id, locked_desc_id, image_id u32, gamerscore u16,
  pad, flags u32, 16 zero bytes}, strings from one XSTR per language (English is
  table #5). tools/xach_dump.py parses it. SELF-CHECK: the 24 gamerscores sum to
  exactly 1000, the retail total -- a wrong stride does not land on a round 1000.

- The two sources agree on ORDER independently: the requirement types
  ShootDownAircrafts 1000/10000, ShootDownShips 100, ShootDownWeight MegaTons,
  GetAllWeapons and GetAllAchievements line up with ids 19-24 exactly as XACH
  names them. So bit n <-> achievement n+1 is evidence, not inference. (Those
  last two are requirement TYPES, not debug cheats, despite how they read.)

- Corollary: TimeAttack's REQUIREMENT 16 -- the one value that sits in direct
  value-before-key adjacency, so it survives IDXD dedup -- is bit 16 =
  achievement 17, "Solar System Defense Award", i.e. finish the story campaign.
  The other five values (25-29) are >= 24 and so index word B, a second flag
  space, plausibly a challenge-clear chain. Still 🟡.

REFUTED, from the last commit: the stores to +1956 in 0x822AF278 / sub_822C8748
are NOT this singleton. That object comes from 0x822CEB30, checks a +2652 flag
and stores string POINTERS at +1956/+2024 -- and a pointer ANDed with 1<<n is
meaningless as a gate. So nothing in the image writes this singleton's +1956
field-wise, and where the mask persists (save vs Xbox profile) is open. XEX
imports are by ordinal, so absent XamUser* strings are not evidence either way.
2026-08-13 19:32:43 +00:00
8ecd70f1bc re(challenge): the unlock is a bit test, and the mission table is on disc
Static only. Bounding each GamePart's code block by its factory creator thunk
(id -> creator recovered for 22 of 24 registrations at 0x8280C000-0x8280F800)
puts GamePart_ChallengeMission at 0x82187E60-0x8218CF10. Resolving every string
that block references gives the screen's config schema, and the record itself is
on disc -- tables.pak schema 54a10697, one copy per language, English entry #64.

Six missions: TimeAttack (record Time), ScoreAttack (record Points) and
Extra01..Extra04, each with MISSION_ID / REQUIREMENT / REQUIREMENT_DESC /
THUMBNAIL / STAGE_DESC / NEW_STAGE and a NORMAL_BUTTON / GRAY_BUTTON pair -- so
the screen always lists all six and greys out what is not earned.

THE GATE (0x82189970-0x821899D8), read off the code:

  REQUIREMENT absent      -> available
  REQUIREMENT == "Always" -> available
  else n = atoi(REQUIREMENT)
      n == 0              -> locked
      n <  24             -> test bit n      of the word at singleton+80
      n >= 24             -> test bit (n-24) of the word at singleton+1956

The singleton is 0x821707C0 (lazy, global 0x828F48BC). So availability is one
bit in a progress bitfield and REQUIREMENT is a bit INDEX -- not a stage number,
score or difficulty.

Values per mission are 🟡: the pool's numeric tokens are 16/25/26/27/29 and
24/28 already appear earlier as font metrics, so they would be deduped -- which
fits 24..29 but IDXD dedup makes positional pairing unsound here, so it is
recorded as a hypothesis, not a table.

Negative: the requirement TEXT is not in GP_CHALLENGE.pak (TextIndex over it =
0 entries; its only prose is embedded font copyright). Its PATH is a per-language
branch the loader does not currently reproduce.

Next: three stores to +1956 sit in 0x822C7DD0 / 0x822C8748, the same region as
the save serializer 0x822C00E8 -- if the bits are save-backed, a hand-written
save unlocks all six challenge missions and the last 42 units become one run.
2026-08-13 19:14:48 +00:00
10a96844ba re(challenge): the EX missions are a MODE, not a stage number
Static-only (no emulator, no pad input). Three findings, each with its own
evidence:

- The disc holds exactly 29 StageResource records in three families --
  S01-S16 story, S18-S23 tutorial (all bg=Original), S24-S29 challenge, plus
  Test. That is 16 + 6 + 6 + 1, matching weapon.tbl's stage01..16 /
  tutorial01..06 / challenge01..06 key set exactly. S17 does not exist.
  GP_CHALLENGE.pak has 0 IDXD objects -- it is the menu screen; challenge
  missions reuse GP_MAIN_GAME_E.pak's records.

- The GamePart id table is at 0x820A1630 (29 ids). Indices are confirmed by
  the image's own RegisterToFactory<N, class silph::GamePart_*> text, not by
  position: GP_CHALLENGE = 26, GP_TUTORIAL = 25, GP_BUNK = 10.

- The stage loader selects its config section from a mission-KIND field at
  object+144: 3 -> EXTRA, 5|6 -> CHALLENGE, else FILE (two independent sites,
  0x82184df0 and 0x82185ed0; two more classify {3,5,6} as one group). The
  constructor sets it to 0 and every write inside the class only clears it,
  and no immediate 3/5/6 store to it exists image-wide -- so the kind is
  supplied by the launching GamePart, never derived from the stage number.

That last point is a mechanism (unproven) for why patching the save's stage
field to 27 kills the load: the record is a challenge stage but the kind stays
FILE. Names an untried, zero-cost discriminator -- try stage 18-23.

Also flagged, not resolved: roster_target says S10 (a STORY stage) still
fields an unharvested unit, which contradicts the "story campaign complete"
claim by one unit.
2026-08-13 18:59:52 +00:00
eac92c3e44 re(flight): withdraw the roll result — the probe was measuring pitch as much as roll
Re-running roll with 5 s settles and in-run clock brackets:

  this run's clock: TIME 00:34.93 -> 00:45.97 = 11.04 s game in 7.98 s wall = 1.383
  min speed: 90.6 deg/wall-s /1.383 -> 65.5 deg/game-s  (AV_Roll_Min 200)
  max speed: 59.1            /1.383 -> 42.7             (AV_Roll_Max 125)

The corrected numbers are within a few per cent of the PITCH run's 67.8 and 40.9 —
two different stick axes cannot produce the same rates, so the probe is not
separating them. Cause: watching a non-forward matrix row sees any rotation that
moves that row, and pitch moves it as much as roll. The correct measure is rotation
ABOUT the forward axis (project the row onto the plane perpendicular to forward and
track that angle).

So "roll shows no speed dependence, unlike pitch" is withdrawn: it rested on 2 s
settles and a row that mixes axes, and the two runs disagree with each other
(144/150 then, 90.6/59.1 now). AV_Roll_{Min,Max} are not confirmed and the axis
question is open.

The clock ratio is now measured three times in three flights: 1.260, 1.311, 1.383 —
a property of the moment, not the machine, so every rate probe must bracket its own
phases.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 18:12:20 +00:00
8b10c0451f re(flight): the clock factor is universal — angular matches too once corrected in-run
pitch_gametime.py brackets each turn phase with HUD screenshots, so the mission
clock's own advance converts wall seconds to game seconds within the same run:

  this run's clock: TIME 00:33.68 -> 00:44.12 = 10.44 s game in 7.96 s wall = 1.311

  pitch @ min speed  88.9 deg/wall-s  /1.311 -> 67.8 deg/game-s  vs AV_PitchMinus_Min 75
  pitch @ max speed  53.6            /1.311 -> 40.9             vs AV_PitchMinus_Max 40

Both land on the definition (the slow phase 10% low, consistent with including the
AA_* ramp in an 8 s window), so the clock explanation covers angular motion as well:
every stated rate is per GAME second.

The ratio is not a machine constant — 1.260 in the earlier flight, 1.311 here — so it
must be measured in the same run as whatever it corrects. Bonus: the same shots show
the HUD reading 102 at full LT against MinimumVelocity 100.

Also documents the trap that cost three runs: a killed Canary leaves both its shm
image and its last frame on screen, so a dead emulator looks alive and the scans
report "0 moving triples" like a tooling bug. pgrep -x matches zombies, so
speed_law.require_live_emulator() checks the process state letter and refuses to
measure a corpse.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 17:56:16 +00:00
ae5f322c03 re(flight): the linear factor is the CLOCK — 1.260 measured against 1.267
Cleanest linear measurement: neutral throttle (HUD = CruisingVelocity 350), sticks
centred, 20 s of perfectly straight flight (displacement/path = 1.000):

  8 900 world units in 20.1 s -> 443.6 /s -> 1.267x the HUD's 350

And the game's own mission timer across a wall-clock interval:

  TIME 00:08.79 -> 00:46.97 = 38.18 s of game time in 30.29 s wall = 1.260

Same number. So the linear discrepancy is not a unit difference: the mission clock
runs ~1.26x faster than wall time under this emulator, and dividing world
displacement by WALL seconds inflates speed by exactly that. World units and
displayed speed share one unit; the definition velocities are per GAME second.

This supersedes the previous "world-unit vs displayed-speed" reading.

Left open (): settled turn rates measured 74.9/41.1 deg/s in wall time against
AV_PitchMinus_Min/Max 75/40, but the clock argument predicts ~94 for the first.
Either that agreement was luck inside a noisy sample (per-window rates spanned
61-96) or angular integration is frame-based where linear is time-based. The check
is to re-measure pitch and convert wall->game seconds with the clock ratio.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 17:26:27 +00:00
0cb81b6200 re(flight): the linear discrepancy is a world-unit vs displayed-speed difference
Screenshotting the HUD speed readout at each throttle step, beside the
position-derived measurement of the same moment:

  RT 0.00   HUD 350 (= CruisingVelocity)   position ~447   ratio 1.28
  RT 0.25   HUD 507                        position ~652   ratio 1.29
  RT 0.75   HUD 963                        position ~1141  ratio 1.19

So (a) the HUD speaks the definition's units — exactly CruisingVelocity at neutral,
963 at three-quarters against the 987 the interpolation predicts — confirming the
throttle law in the game's own numbers without any position sampling; and (b) world
displacement runs ~1.2x the displayed speed. Since settled angular rates need no such
factor, this is a unit difference between the position triple and the velocity
fields, not a clock effect: a reimplementation moving entities at MaximumVelocity in
world coordinates will be ~20% slow.

Also fixes speed_law.find_player: a mission holds more than one *_Player object and
at least one never moves, so the finder now samples each candidate twice and keeps
the one that displaces. Locking onto the static one is what produced a run of exact
zeros while the game was visibly flying.

🟡 The ratio is 1.19-1.29 rather than a clean constant and every sample was taken in
a firefight; pinning it wants a quiet map.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 17:06:35 +00:00
f6974ff3f0 re(flight): settled turn rates match the definition exactly — withdraw the time-base claim
Re-measured with 5 s of settle per phase and the speed recorded at the moment the
turn starts (flight_law3.py):

  pitch @ 130/s    74.9 deg/s   vs AV_PitchMinus_Min 75
  pitch @ 1821/s   41.1         vs AV_PitchMinus_Max 40
  roll  @ 110/s   129.5         vs AV_Roll_Min 200
  roll  @ 1722/s  149.3         vs AV_Roll_Max 125

Pitch lands on the definition's own numbers with NO scale factor, so the ~1.2x I
attributed to the emulated time base two iterations ago was an artefact of
differentiating during the AA_* acceleration ramp with too little settle. That
explanation is withdrawn: AV_* can be used verbatim.

What remains is only on the linear side — settled speeds still read high and vary
between runs (RT full: 1342 in one flight, 1821 in another, vs MaximumVelocity 1200),
consistent with a craft being shoved around in a firefight. The HUD reads exactly
CruisingVelocity at neutral. A clean linear measurement needs a quiet map; no cause
is claimed until then.

Roll re-measured with proper settles confirms the axis difference: no speed
dependence, both regimes near AV_Roll_Max, where pitch moved 75 -> 40.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 16:46:33 +00:00
6ebbbeff65 re(flight): LT mirrors RT, and roll does not depend on speed
One flight, two measurements (flight_law2.py, binding early so the moving-craft scan
can see the player).

LT curve: 436, 379, 289, 209, 126 units/s across LT 0.00 -> 1.00 — a straight ramp,
whose endpoints after the ~1.2 time-base factor are CruisingVelocity 350 and
MinimumVelocity 100. So the law is symmetric:

  RT: target = Cruising + RT * (Maximum - Cruising)
  LT: target = Cruising - LT * (Cruising - Minimum)

Roll (measured on a non-forward matrix row, since roll turns about the forward axis):
~144 deg/s at minimum speed and ~150 at maximum — no speed dependence, where pitch
dropped by a third to a half between the same regimes. After the time-base factor
that is ~121, i.e. AV_Roll_Max 125 in BOTH regimes.

So _Min/_Max does not mean the same thing for every axis: pitch interpolates with
speed, roll appears pinned at Max. A reimplementation applying one rule to all axes
would get low-speed roll wrong by ~60%.

Caveat recorded: the two roll phases were 2 s of settling apart, marginal for a
126 -> 1342 speed change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 16:30:19 +00:00
4dbd4f6cef re(flight): the throttle is analogue — target speed interpolates cruise -> maximum
RT is an analogue trigger, so "held" was one point on a curve. Walking it 0.00 ->
1.00 (throttle_curve.py) gives a straight ramp: 438, 626, 879, 1094, 1342 units/s.
Dividing by the ~1.2 time-base factor, the endpoints land on the definition's own
numbers (365 vs CruisingVelocity 350; 1118 vs MaximumVelocity 1200) and the midpoint
follows, so

  target speed = CruisingVelocity + RT * (MaximumVelocity - CruisingVelocity)

which refines the earlier "selects one of three targets" reading: those three are the
curve's endpoints.

It also refutes the standing afterburner hypothesis that full RT is the burner: the
curve is smooth through full deflection with no step, and the shield does not move.

The LT half is not measured yet — the entity scan needs the craft moving when it
runs, so a mission left idling drops out of it. Bind early.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 16:14:47 +00:00
fb1405b13f re(flight): the afterburner is not on A/B/X/LB/LS/RS — bounded negative, with a cheap HUD oracle
The definition describes the burner (AB_ConsumeShield_Begin 50, AB_ConsumeShield 10,
AB_AV_* turn caps well below normal) but names no input, and carries no AB velocity
field.

Three probes, all negative for A, B, X, LB (plus LS/RS on the first):
- ab_probe.py: hold RT for a max-speed baseline, then each candidate — speed stayed
  inside the baseline's own noise band every time.
- ab_state_probe.py: sample a window of the player object during each hold and
  report any float that falls — nothing fell.
- HUD oracle needing no offsets: count green pixels of the SHIELD bar on a freshly
  spawned craft. AB_ConsumeShield_Begin 50 should take a visible bite; the bar read
  156/156/156/157/157 across baseline and all four buttons.

So the burner needs a chord, an input this pad cannot reach, or belongs to another
craft/the AI. Recorded so the obvious buttons are not re-probed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 15:51:32 +00:00
86fcfcc8b0 re(flight): turn rates confirm AV_* are rate caps and _Min/_Max mean at min/max speed
turn_law.py pins the speed regime with a throttle, holds a stick axis and
differentiates the craft's own forward vector over 1-second windows.

  slow + nose down  ~87 deg/s   (AV_PitchMinus_Min 75)
  fast + nose down  ~54         (AV_PitchMinus_Max 40)
  slow + nose up   ~175         (AV_PitchPlus_Min 150)
  fast + nose up   ~136         (AV_PitchPlus_Max 70)

So agility falls with speed (_Min/_Max are at minimum/maximum speed, not rate
bounds) and pitching up is ~2x pitching down, exactly as the field pairs say.

Control mapping measured: LX is roll (forward vector barely moves, 3-5 deg/s), LY is
pitch (+1 = nose down per vgamepad's LY: -1 = up), and the right stick does not steer
at all.

The ~1.2x overshoot seen in the speed law appears again here (1.16-1.35x), and a
unit scale cannot explain both m/s and deg/s — a TIME BASE can: if the guest's
simulated second is shorter than the wall-clock second the probe measures against,
every rate reads high by the same factor. So the definition numbers are
self-consistent and these measurements confirm the shape of the law, not a scale.

Recorded 🟡: no yaw input found (AV_Yaw_* exists but neither stick yaws), which with
roll on LX and MaximumBank_Normal points at a bank-to-turn model.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 15:30:00 +00:00
1efc3f567d re(flight): the throttle is a target-speed selector, measured against the definition
speed_law.py locks onto the player entity once and samples its position while
holding each throttle input, differentiating over 1-second windows.

  no throttle  -> ~420   (CruisingVelocity 350)
  RT held      -> ~1 530 (MaximumVelocity 1200)
  LT held      -> ~125   (MinimumVelocity 100)
  release      -> back to cruise, from either direction

So the throttle SELECTS a target speed rather than adding thrust — which is what a
reimplementation would most likely have assumed from Acceleration/Deceleration
alone. Those govern the convergence rate instead: ~440 units/s^2 measured on
release (Deceleration 500) and ~470-560 under RT (Acceleration 600).

Recorded as 🟡: measured world speeds run ~1.2-1.3x the definition numbers in all
three regimes while the HUD shows the definition value exactly (350 at cruise), so
world coordinates are a constant multiple (~1.25) of the definition's velocity unit;
the spread is wider than the constant is precise because the craft manoeuvres while
sampled.

Three traps documented: RT/LT are analogue triggers (the button verb is a silent
no-op and the first run measured an unflown craft), per-sample differentiation
aliases against the guest's update rate (0, 1519, 1985, 0, 2681 for smooth flight),
and the player entity only enters the typed scan ~15 s in while the craft dies within
minutes if nobody flies it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 15:08:11 +00:00
6d0bdf0179 re(units): S08 closes the story stages (68 units), and GAME_CLEAR unlocks nothing
S08 gave the last two story-stage units — UN_be005_ADAN_SpaceFortress and
UN_mn500_ADAN_FloatingMine — plus six S08 asteroid collision meshes: 68 units,
9 393 rows, 7 115 defaulted-on-disc values. Every unit any story mission fields is
now read (689 more disc cross-checks, 0 disagreements).

Probe recorded as a negative: setting Game Status (GHAD +48 and its header mirror
+0x18) to 2 = STATE_GAME_CLEAR leaves both menus unchanged — the title still shows
NEW GAME / LOAD GAME / TUTORIAL / OPTIONS / EXTRAS and EXTRAS still shows only
MISSION SELECT / MOVIE THEATER / BACK. So the EX/challenge rosters are not gated on
that field.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 14:39:05 +00:00
0221217968 re(units): six targeted harvests — 60 of 110 units, 6 071 defaulted values
S03, S06, S09, S13, S15, S16 (plus the earlier run): 21 -> 60 units,
963 -> 6 071 defaulted-on-disc values, 8 241 rows. Disc cross-checks agree with
0 disagreements throughout.

- UN_e901_ADAN_Boss and UN_e910_core_ADAN_GeneratorCore are harvested (HP 10 000,
  Size_X 500, Size_Radius 250, MaximumVelocity 450) WITHOUT the boss ever being on
  screen: definitions are instantiated at stage load, so a unit only needs to be in
  the roster. That is far weaker than what the draw-capture work requires.
- The extra stages are not reachable via the save's stage field: stage 27 boots to
  the title and then the emulator exits during the load, as do the other S24..S29
  entries, while story stages 1-16 all load. The field addresses the story campaign
  only; EX/challenge rosters need the Challenge-mode menu path.

Remaining 50 units are almost all EX variants in S24/S25/S27/S28/S29; the only
story-stage stragglers are UN_mn500_ADAN_FloatingMine and
UN_be005_ADAN_SpaceFortress in S08.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 14:14:18 +00:00
bf6825e278 re(units): target missions by roster, harvest S09 — 36 units / 3 439 defaulted values
examples/roster_target.rs ranks stages by how many roster units are still
unharvested. The EnumUnit_S<NN> tables are found by hashing candidate TOC paths
(hash::TOC_NAME_SCHEMES) — UnitRoster::stage can only infer a tag when the roster
carries a UN_S<NN>_ prop, which most do not.

It picked S09 (10 missing). Flying it: 26 -> 36 units, 3 345 -> 4 785 rows,
2 351 -> 3 439 defaulted-on-disc values. New: e102_Battleship, e104_Carrier,
e107_AAFrigate, e011_Attacker_B, e008_TurretPlus, be001_TerrafoamingUnit,
e001_Elan_GR{,_Violeta}, f102_LightCarrier_Inv, f106_Destroyer_Inv.

Also settled: the definition objects are mission-independent. Eleven units appear in
more than one snapshot and four are not byte-identical, but compared through the
layout ZERO mapped fields differ — the 12 differing slots are all unmapped (offsets
4/8/16/20 and 0x250/0x268/0x300-0x308/0x330-0x338: object header and sub-object
pointers). So a harvested value is the definition, not a per-mission tweak, and the
earlier UN_f201_TCAF_Tanker flag resolves the same way. Cross-checks over three
snapshots: 1 052 agree, 0 disagree.

Third angle field found the same way (Through_AngleMaximum = 60 degrees in radians),
so the degrees<->radians rule covers any name containing "Angle".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 13:48:12 +00:00
14d5ed54d1 re(units): read definitions with the code-derived layout — defaulted values 1 059 -> 2 351
unit_runtime.py can only place fields the disc values (it scores triples against
disc records): 58 of 153 from one snapshot. data/unit_definition_layout.txt came
from the title's loader instead, so it places all 159 — including the fields no
disc record sets, which is the Route-B target.

tools/re-capture/unit_dump_layout.py reads every field of every live definition
object with that layout and keeps the discipline: a field the disc DOES value is a
check, not a new value. Over two snapshots (19 objects): 700 cross-checks agree,
0 disagree.

  fields placed per object            58 of 153 -> 159
  rows over those 19 units                  609 -> 2 736
  defaulted-on-disc values (whole file)   1 059 -> 2 351

Two traps recorded: the layout table's offsets are DECIMAL while the solver CSV
prints hex (parsing as hex fails the cross-check on everything — which is how it
announced itself), and angle fields can carry a prefix (AB_AA_PitchPlus is still an
angle, so the AV_/AA_ test must match anywhere in the name).

Flagged: UN_f201_TCAF_Tanker's object is not byte-identical between the two
missions — per-mission override or a runtime-mutated field; needs a third snapshot
to separate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 13:30:49 +00:00
d61baddaf0 re(units): stage select lifts the per-stage coverage limit — 21 -> 26 units
Unit definitions are instantiated per stage, so the parked "coverage grows by
visiting missions" limit was blocked on progress. With the save's stage field
solved, any story stage can be flown from a hand-edited save and snapshotted.

First harvest (a Night Ravens mission, 027 objectives, different roster):
  units with runtime values   21 -> 26
  csv rows                 1 827 -> 2 143
  defaulted-on-disc values   963 -> 1 059

New: UN_e004_ADAN_ElanPlus_N, UN_e006_ADAN_Vindicator_Margras,
UN_f002_TCAF_DeltaSaber_W, UN_f002_TCAF_DeltaSaber_W_Player (a second player craft)
and UN_mn040_Asteroid_Big (47 defaulted values, Size_Z = 2000).

Two traps recorded with the fix: launch_mission.sh presses A on the PRESELECTED
slot, which is the last-used one (03 here), so a probe in slot 01 is ignored — the
first attempt flew stage 02 again and the solver correctly found 0 new units; patch
every slot or confirm the highlighted slot by screenshot. And back up before
patching: the mid-run backup already held the probe, so slot 01 was restored from
the earlier pristine copy (md5 142b4f43…, verified byte-identical).

Token file for the solver: idxd_tokens <GP_MAIN_GAME_E.pak> Generic — 110 UN_*
records, the disc's full unit count.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 13:16:04 +00:00
d1b50dc1e6 re(xbg7): characterise the residual 47 misses — 99.25% of resources decode
examples/why_missed.rs reports the furthest gate per undecoded resource. Disc-wide
that is 47 rows / 43 distinct names, and they are mostly not ship geometry:

  30  e_rou_/_rou_ pose & proxy composites (24-vertex marker boxes, extent 0.010)
   6  .DAT particle composites in ptc_pack
   8  damage/LOD variants (e101_bdy_02_d, e901_wing_05_*_m, ...)
   3  props/other (g005 extent 0.196, _rou_f001_wep_05, e_rou_e005)

By gate: extent 32, coverage 8, winding 7. The extent bucket is almost entirely the
pose-proxy boxes, and lowering that floor was measured and refuted earlier. The one
genuinely interesting residual is e901_wing_05_L/R at winding 0.587/0.570 against the
0.70 floor — the signature of a thin double-sided sheet, unproven without a capture
with the boss on screen (flying stage 16 puts the container in memory but the unit
never appeared).

So: 6 247 / 6 294 resources decode (99.25 %), and every real mesh drawn in three
captured missions decodes at the GPU's own offsets with index runs matching byte for
byte (93/93 and 128/128).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 12:50:19 +00:00
3b35c48fce fix(xbg7): grouped selection prefers the candidate explaining the whole pool; per-sub-mesh decls on by default
With per-sub-mesh declarations enabled, n201_01 decoded as a 2-part fragment 4 bytes
off. Both starts validate for the pivot — 0x32BA718 at pad 2 (earlier in file order,
so first-match took it) and the capture-proven 0x32BA71C at pad 0 — so the pivot
alone cannot separate them; at the early one two of four sub-meshes fall out as
out-of-range.

anchor_grouped_meshes now builds each accepted candidate and keeps the one that
explains the most of the declared pool: it returns immediately when a candidate
explains all n sub-meshes, else keeps the best partial, so it can never decode less
than first-match did. n201_01 lands on all four capture-proven offsets
(0x32BA71C / 0x32BEFF4 / 0x32C416C / 0x32C536C) and its two sibling copies take their
own pools, so the twin collapse is gone.

XBG7_SUBMESH_DECLS is therefore on by default (=0 reverts):

  resources that never decode        85 -> 47
  resources decoding in no container 63 -> 30
  degenerate index runs               1 -> 1   (unchanged)
  cross-container minority decodes   96 -> 96  (unchanged)
  captured index runs, stage-02      93/93     (unchanged)
  captured index runs, stage-05     124/128 -> 128/128

The last line is the point: the buffers the capture could not name are the n201
family, and they now decode and match the GPU's indices byte for byte. Suite green
including twin_pairs_do_not_share_a_buffer, apart from the pre-existing
known-failing cross-container consistency test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 12:27:56 +00:00
4cdbbc2d48 feat(xbg7): read the per-sub-mesh vertex declarations (XBG7_SUBMESH_DECLS, off by default)
desc_dump shows each index marker is followed by its OWN element triples: n201_01
declares strides 24, 24, 24 and 28 (the last sub-mesh has a fourth element), which
matches the runtime capture's stride=28 on that draw and the four distinct vertex
shaders. parse_vertex_decl read the first declaration for the whole pool.

all_vertex_decls reads one per marker; anchor_grouped_meshes uses each sub-mesh's
own stride for the pool walk, the pivot validation and the read. debug_grouped_report
follows the same setting so the diagnostic cannot blame the wrong gate — at n201_01's
capture-proven pool start it now reports "pad 0: ACCEPTED" instead of a NaN position.

With XBG7_SUBMESH_DECLS=1: resources that never decode 85 -> 47, resources decoding
in no container 63 -> 30, capture oracles unchanged (93/93 index runs, 42/42 index
counts), consistency unchanged at 96.

Off by default because selection has not caught up: the three n201_0x copies then
settle on one pool (twin_pairs_do_not_share_a_buffer fails), production still picks
a start 4 bytes before sub-mesh #1 rather than the proven one even though the proven
start validates and is unclaimed, and four newly decoded ptc_pack .dat composites
carry degenerate triangles. Format truth is settled; choosing among candidates is
the remaining work.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 11:45:20 +00:00
27577538cb re(xbg7): a capture names a never-decoding family, and the cause is mixed strides in one pool
capture_ib_truth now proposes an identity for each drawn buffer our decoder cannot
place, by matching (vertex, index) counts against declared-but-not-decoded
resources. That identified n201_01/_02/_03 in Stage_S02.xpr — three of the 63
resources that decode in no container — and the capture pins all four sub-meshes:

  #0 vb 0x32BA71C ib 0x32B39FC 4464 idx 777 v stride 24
  #1 vb 0x32BEFF4 ib 0x32B5CDC 4464 idx 869 v stride 24
  #2 vb 0x32C416C ib 0x32B7FBC  576 idx 192 v stride 24
  #3 vb 0x32C536C ib 0x32B843C 4464 idx 869 v stride 28   <- different

The layout matches our assumptions exactly (tight index packing, last buffer flush
against vb0 so pad 0, span 27936 == align4-summed markers, contiguous vertex
buffers, max index == verts-1 everywhere). The defect is that sub-mesh #3 has a
different stride AND its own vertex shader: anchor_grouped_meshes parses one
declaration per resource and applies its stride to every sub-mesh, so it reads #3
out of phase (372 non-finite position components of 2607), and since the pivot is
the largest index count with ties going to the last marker, the pivot IS that
sub-mesh — so the whole resource is declined.

Also adds examples/miss_targets.rs (which container to aim a capture at): 63
resources decode nowhere, 58 of them in exactly one container, clustering as
Stage_S16 21 (e901_wing_05_*), ptc_pack 12, Base 6, then per-stage n2xx groups.
Flying stage 16 did not draw the e901 wings — the container is resident but the
unit must also be on screen.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 10:52:35 +00:00
92d36ec10f re(xbg7): a Stage-05 capture validates the pad fix out of sample, and shows how containers load
Ran ship_capture_close.sh in story stage 05 (reachable now via the save's stage
field) instead of stage 02: 3 logs, ~11 200 draws.

- Out-of-sample check of the pad-scoring fix: 124 of 124 index runs identical to
  the GPU's, 2 769 index elements, 0 differing — on draws the fix was not derived
  from (the stage-02 capture gave 93/93).
- A mission keeps SEVERAL stage containers resident: 65 drawn buffers place in
  Stage_S02.xpr at 0x17FE3FF4 (the shared TCAF/ADAN fleet) and 5 in Stage_S05.xpr
  at its own constant 0x1927B7F4 (the f101 ACROPOLIS, this mission's escort) —
  the first capture-truth rows for a container other than Stage_S02. Both sets
  have exact index counts and exact coverage.
- Why: f105_bdy_01 lives in 13 of the 22 stage containers and not in Stage_S05 at
  all, so the resident set follows the mission's unit roster, not the stage number.

Consequence for the residual misses: a different story stage is not a different
container. Aiming a capture at a specific container is a static question (resource
names per container x the stage's EnumUnit roster).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 10:20:00 +00:00
9fd5dcd37d re(savegame): GHAD +52 is the stage number — a hand-edited save plays Stage 05
Static analysis of the header builder (0x822870b4..0x82287128) shows the GDHA
container header carries a SUMMARY of the progress block, and the screens read
that: hdr+0x14 <- GHAD+52, hdr+0x18 <- GHAD+48, hdr+0x1c <- +24 (Points),
hdr+0x20 <- +4 (flight), hdr+0x24 <- the computed clear ratio, hdr+0x28 <- +12.
savegame_edit.py copies the donor header verbatim, so payload-only probes left a
stale summary — which is why +52 looked refuted.

Patching both (hdr[0x14] = 5, ghad[+52] = 5) makes the title read
"STAGE 05 - Star System Escape", load it, brief the Gallia Asteroid Group, and fly
an asteroid-field mission with a different objective and roster. So +52 is the
stage number, 1-based, and one u32 chooses the mission.

Also corrected in the GHAD table: +48 is the Game Status enum (0 = At Standby,
matching the screen's STATE_STAND_BY/STAGE_CLEAR/GAME_CLEAR list) and +12 is Times
Cleared, both via their header mirrors.

This unlocks runtime capture in containers other than Stage_S02 (the 85 XBG7
misses, box identity, ship-placement generalisation) and per-stage unit
definitions. Slot 01 was backed up and restored byte-identically.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 09:46:08 +00:00
0cf9220d24 re(savegame): stage-unlock probes are negative, and the Details panel is not a payload oracle
Four probes against slot 01 (the save the title-menu screens read), each backed up
and restored byte-identically:

- SHAB[1..7] filled -> MISSION SELECT still locks Stage02-08, cursor will not
  leave Stage01. The per-stage record table is not the unlock gate; the earlier
  SHAB[1] attempt used a throwaway slot and could not show this.
- GHAD +36/+52/+56 = 5 -> still locked, panel still STAGE 02 / EASY.
- GHAD +16 = 4 -> panel still STAGE 02, so it is not a 0-based stage index.
- live-RAM writes into the loaded GDHA header -> panel unchanged.

The useful part is why the panel cannot answer this: guest RAM at that screen holds
NO payload bytes (neither the 4101/4101/79 triple nor the 54-byte develop blob),
but it does hold the save's GDHA container header with a summary copy of Points,
flight time, clear ratio and a FILETIME. savegame_edit.py copies the donor header
verbatim, so a payload edit leaves that summary stale — "the panel did not change"
therefore cannot separate "not the stage field" from "the panel never reads the
payload". Future stage/difficulty probes must patch the header too, or be judged by
what the game does on load.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 09:18:46 +00:00
0776beb6f4 fix(xbg7): degenerate index runs 582 -> 1 (grouped path + prefer a clean candidate)
Two follow-ups to the pad-scoring fix, both driven by the same invariant (a
correctly located index run has no degenerate triangles):

- anchor_grouped_meshes picked its pad by first-match too; scoring the pivot run
  the same way cleared every remaining ptc_pack composite (f102/f104/e107).
- anchor_pool_mesh now prefers a degenerate-free candidate over an earlier dirty
  one. examples/better_home.rs showed the last two resources each had exactly one
  degenerate-free, pool-covering block, sitting later in file order than the
  lookalike we took. First-match order is kept for every clean hit, and a dirty
  block is still used if nothing clean exists, so coverage cannot regress.

  degenerate index runs, disc-wide:      582 -> 11 -> 1
  captured index runs identical:         93/93 (unchanged)
  resources decoded / misses:            6 209 / 85 (unchanged)
  index runs changed / anchors moved:    590 / 10 (_rou_f402_dead x8, e201_bdy_03_m x2)

Cross-container minority decodes 89 -> 96, and that is progress: all seven new
rows are _rou_f402_dead, which now has a majority (32x25x8) for the first time, so
its seven wrong copies are named instead of hidden behind "no majority".

The last dirty run (_rou_f402_dead in Stage_S09) is blocked by distinct assignment
— its clean block is claimed by e_rou_f003_Near, both 24-vertex bounding boxes. A
winding-floor escalation for that case was written, measured to fire for nothing,
and reverted; the reasoning is kept as a comment.

Regression threshold tightened to 1. Suite green with --include-ignored apart from
the pre-existing known-failing cross-container consistency test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 08:52:41 +00:00
33dc12c948 re(xbg7): render the before/after — the index shift shredded capital-ship hulls
ship_render --static on f101 (ACROPOLIS) with and without XBG7_PAD_FIRST_MATCH=1.
Identical triangle count, placements and bounds; the old wiring renders as a mess
of shards, the fixed one as a coherent hull. Kept as the evidence image for the
pad-scoring fix.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 07:28:40 +00:00
41b59faf5f fix(xbg7): the index run was one element late for 575 sub-meshes
Extending the capture comparison from index COUNTS to index VALUES
(`examples/capture_index_bytes.rs`, using the batch offsets the new ib logging
gives) showed 76 of 93 Stage_S02 index runs identical to the GPU's and 17
differing — every difference a shift by exactly one element, on buffers whose
index data sits at pad 2.

`anchor_pool_mesh` returned the FIRST pad that validated, and pad 0 is tried
first with the looser winding gate (0.70 vs 0.85). Read at pad 0, a pad-2 block
yields [true[1], true[2], …, garbage]: every index in range, the pool covered,
the positions right, the winding often just above 0.70 — so it validated, and
every triangle was mis-wired. Nothing count-based could see it.

The signature is decidable without the capture: a shifted run wires arbitrary
vertices, so triangles come out degenerate. 282 of 283 correctly anchored
Stage_S02 blocks have zero degenerate triangles, while the shifted readings carry
1–2 156. So score every validating pad by (degenerate triangles, then winding)
and keep the best. `XBG7_PAD_FIRST_MATCH=1` restores the old behaviour.

  captured index runs identical:            76/93  ->  93/93  (2 025 elements)
  decoded runs with a degenerate triangle:    579  ->     16  (disc-wide)
  sub-meshes whose index run changed:                    575  of 8 850
  resources decoded / vertex anchors / consistency:  unchanged (6 209 / same vb / 89)

Locked in by tests/mesh_disc.rs::decoded_index_runs_have_almost_no_degenerate_triangles.
Suite green with --include-ignored apart from the pre-existing known-failing
cross-container consistency test (the 24-vertex bounding-box class).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 07:26:52 +00:00
6d92e6c114 re(xbg7): the [index][vertex] layout is runtime-verified, and the indices= mystery was batching
The decoder's central unstated assumption — a block's index buffer sits
immediately before its vertex buffer (`vb - idx_count*2 - pad`, pad <= 3) — was
also the prime suspect for the residual anchor misses, since a capture-proven
`e106_eng_02_l` block was rejected outright. Measured it instead of assuming:

- extended the F10 ship capture to log each draw's index buffer (base, count,
  min/max index) and to key its de-dup on the index range, so every draw batch
  is recorded rather than only the first;
- `examples/capture_ib_truth.rs` places each drawn buffer in the container by its
  dumped positions and scores the capture against our decode.

Stage_S02, 42 drawn buffers placed: our idx_count == the sum of the draw's index
batches for 42/42, the batch union covers the vertex pool exactly for 42/42, and
all 30 single-block cases sit at pad <= 3 (20 at pad 0, 10 at pad 2). The other
12 are grouped pools, where one index pool serves the whole group. So the layout
holds, the decoded index count is exact, and eng_02_l died on the connectivity
gate (since replaced by the winding gate) — not on index location. The shipped
exact-coverage rule is independently confirmed.

The recorded "capture indices=21 vs our 246" disagreement was an artefact of the
old de-dup key: 21 was the first of two batches, 21 + 225 = 246. Any conclusion
from a pre-2026-08-13 capture's `indices=` or `vbase - ibase` is about one batch,
not about the block.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 05:57:44 +00:00
1b618f500f re: cross-validate the code-derived layout against the earlier RAM-binding solver
Two independent derivations of the unit definition layout -- the earlier
disc-value-to-RAM-word solver and this session's extraction from the loader's own
key strings -- agree on 25 of 25 shared offsets, 0 disagreements. The code-derived
table covers 159 fields against 27 confirmed, while the earlier one uniquely holds
ScorePoint (+0x08c) and MassScore (+0x094).

Also flags a tentative entry as an artefact: Slalom_CutoffRatio at +0x00c records
2.8026e-45 / 1.4013e-45, which are the denormal readings of the integers 2 and 1 --
the same trap that produced a false YawDragFactor hit while building the new map.
INDEX updated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 01:11:43 +00:00
7cd9fb8e0b feat(formats): land the verified unit-definition layout as data + API + test
The offset->field map read out of sub_82341A20 is now usable rather than just
documented: data/unit_definition_layout.txt carries the 159 fields with their
provenance and the two conventions (angles are degrees on disc and radians in the
object; a defaulted field keeps the accessor's 0.0 miss value), and
sylpheed_formats::unit_layout exposes fields()/field_at()/offset_of() so a memory
snapshot can be read by name.

tests/unit_layout_disc.rs replays the verification against the checked-in live
dump -- every mapped float of all 11 identified objects must equal its disc value,
angles compared in radians -- asserting 0 disagreements and >=400 agreements. It
needs no emulator. Full suite green: 11 binaries.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 00:44:26 +00:00
19b338d9d8 re(route-b): map generalises to units -- 406/0 across 11 objects, 121 defaulted fields
Applying the map to turrets, fighters, an attacker and a station (UNIT records
rather than VESSEL) gives 406 field values agreeing with the disc and 0
disagreeing across 11 objects, so one layout serves both schemas.

Two consequences the four-value signature could not give: the map IDENTIFIES an
object -- 0xbd3b6a00 is UN_bf001_TCAF_SchlosBase (14 agree, 0 disagree), not
UN_be005_ADAN_SpaceFortress (12 agree, 2 disagree on Color_R/Color_G) -- and
variants are provably identical, with ArrowHead/_EX4/_EX5 at 48/48, Turret/_EX4
at 51/51 and three DeltaSaber player variants at 49/49.

121 defaulted fields now have runtime values. FiringRange (0 in all 11) and
ShieldRatio (1 in all 6) look like true constants; HQRatio, ThrusterRatio and
AttackCraftPoint vary per unit and are recorded as such.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 00:12:45 +00:00
d674cddad4 re(route-b): full field map verified 154/154, and 101 defaulted fields read out
Anchoring each key string to the first store AFTER the accessor call makes the
extraction exact: 159 fields, 125 of them floats, and verify_fieldmap now checks
every one against the live objects -- 154 agree with the disc, 0 disagree.

The last ten mismatches taught the rule: angle fields are degrees on disc and
RADIANS at runtime (AV_PitchPlus_Max 3.6 -> 0.0628319; MaximumBank_Normal 60 ->
1.0472), an exact conversion across every angle field and unit.

101 defaulted fields now have runtime values: angle limits default to 3.6 degrees,
afterburner/turn-attack fields to 0, FiringRange to 0 (matching the loader's miss
value), Size_Y to that unit's Size_X, and the ratio family is per-unit rather than
constant (HQRatio 0.2 or 1, DefencePoint 0.1 or 0.25, ...).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 23:44:57 +00:00
0434afa6a6 re(route-b): the field map comes out of the code, and with it the Ratio family
The filler builds each key as addi r4, r30, -N with r30 = 0x82088f94, so every
store's field NAME is a string in the image. Pairing keys with the following stfs
gives the layout outright: Size_X/Y/Z at +48/52/56, HQRatio +88, ShieldRatio +92,
ThrusterRatio +96, resistances +116..132, Radar/FCS/FiringRange +672/676/680,
Attack/Defence points +692/696/700. verify_fieldmap.rs checks it against the live
dump: 62 fields agree with the disc, 0 disagree.

That yields the runtime value of each field for units whose record omits it --
HQRatio 0.2/1, ThrusterRatio 0.2/1, ShieldRatio 1, ResistanceToPlayer 1,
ResistanceToShell 0.1, ResistanceToExplosion 0.5, AttackVesselPoint 0.1,
AttackCraftPoint 0.1/0.5, DefencePoint 0.1/0.25, FiringRange 0, Size_Y = Size_X.
Recorded with the caveat that several show two values across units, so these are
per-unit runtime values rather than one global default.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 23:17:05 +00:00
2ed475868b re: Size_Y is genuinely defaulted, and the loader leaves it at 0.0
pool_window.rs shows the Destroyer's raw tokens: "200.0" "Size_X" "Size_Y"
"2000.0" "Size_Z" -- Size_Y is a bare key, so the 13 runtime values recovered
earlier are real defaults, not a reader artefact.

The filler reads each size field through sub_822FC5A8, which loads f31 from
0x8209fd28 = 0.0 at entry and returns it on a pool miss, then stores to +52.
Nothing else in the 15876-byte filler writes +52, so the definition leaves a
defaulted Size_Y at 0.0 and the runtime 200 is written later.

Two corrections: the +48/+52/+56 comparison block builds a size-class bitmask
against 1000.0 (constant 0x8209fd20), not a has-value mask; and both earlier
Size_Y derivation candidates are refuted (one is a conditional pick, the other
multiplies into a different struct).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 22:47:43 +00:00
be42e998bc re: the unit filler is sub_82341A20, and its 1.0 constant is not a default
Call graph completes: sub_82341048 allocates 880 bytes, calls the zero-filling
constructor, then sub_82341A20 (3969 instructions, 157 stfs, 892 calls) fills the
fields. A repeated lfs/stfs pattern writes the constant at 0x820854cc -- read from
the .pe as 1.0 -- into ten offsets, which looks like a blanket default.

Checking those offsets in the live objects first shows it is not: they hold
per-unit radian angles (10/15/16/30/40/45/60 degrees), so the 1.0 belongs to the
degree-to-radian conversion, not to a default. The ordinary default is 0 from the
constructor; non-zero defaults like Size_Y <- Size_X are specific derivations.
Incidental and useful: angle fields are radians at runtime, degrees on disc.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 22:18:56 +00:00
71adafe7f7 re: the definition constructor zero-fills -- defaults are derived after parsing
sub_8233FAF8 (0x8233faf8) writes vtable 0x820af844 -- the class dumped from a live
mission -- and every one of its 32 stores writes zero, out past +872. So defaults
are NOT constructor immediates, which was the hypothesis this tick set out to
test. Since a defaulted Size_Y nevertheless reads back non-zero at runtime (always
equal to Size_X), the value must be derived by a post-parse fixup -- the same
shape as the inheritance rule derived statically from sibling records.

Two candidate derivation sites (0x82217e18, 0x823081e8) are recorded as leads
only: the first is a conditional assignment rather than a copy, and offsets 48/52
are generic enough to belong to unrelated structures.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 21:49:48 +00:00
a3868fab5b re: the static route locates the reflective parser and reframes the defaults
sylpheed.db has the machinery named: IdxdLoad_Dispatch 0x824486c0, Idxd_Parse
0x82449640, Reflect_FindFieldIndex 0x8244a2f0, Reflect_SetField 0x8244a4b0.
FindFieldIndex looks a field up by NAME at runtime (strlen + compare), which
supports the reflective-read reading -- but Idxd_Parse is a text parser that
calls SetField on a target object, so a parsed record does exist and the earlier
'no flat record' hypothesis is weakened.

The useful consequence: Idxd_Parse only writes fields the text contains, so a
DEFAULTED field keeps whatever the constructor stored. The defaults are therefore
immediates in each definition class's constructor -- reachable by walking the call
graph above IdxdLoad_Dispatch -- not values to be scanned for in RAM.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 21:21:21 +00:00
bab2f9e6ca re: three probes for the parsed record all miss -- hypothesis: there is not one
Searched RAM for HP 25000 (200+ hits, too common), for the rare float
AttackVesselPoint 0.08 (64 hits, no window holds the unit's other values, one hit
is a code constant), and for the ID string itself (two copies, each pointed at
from -0x10, but the surrounding 256 words hold none of the numbers because that
region is the IDXD string pool where values are ASCII).

Hypothesis recorded as 🟡: IDXD is reflective and the engine likely reads values
from the pool by key on demand, caching only the per-frame ones -- which is
exactly what the 0x820af844 object contains at any dump depth. If so, a defaulted
field has no value anywhere in data and the default is applied by code at read
time, so Route B for the Ratio/Count family needs the read path or the static
DuckDB route, not more scanning. The 13 Size_Y values worked because Size_Y is
one of the cached fields.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 21:00:48 +00:00
f4f3bcb20a re: a deeper window does not help -- the 0x820af844 object is not the definition record
Dumping 512 words instead of 96 finds exactly the same 4 of ~50 disc values, for
all five correlated units, so the rest of the record is not in this structure.
Also corrects the reason recorded last commit: get_f32 is NOT unreliable on
default-heavy records -- it resolves 46-57 numeric fields per unit, which is what
made the emptiness of the correlation measurable in the first place. The one
automated hit (YawDragFactor -> +0x0c) remains false: +0x0c holds the integer 2
as a denormal and collided with the float 2.0.

Next probe is a RAM-wide search for a unit-distinctive value, not a bigger window.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 20:49:19 +00:00
892f1e3979 re: partial field map inside the live definition object, and why the automation failed
Anchoring on a unit whose disc record sets a field locates it in the live object:
Size_X/Y/Z (+0x30/34/38) and HP (+0x54) confirmed across five capital ships and
the player fighter; +0x74 (0.8) and +0x84 (0.97) are probable ThrusterRatio and
ResistanceParalyze but rest on a single anchoring unit; +0x40 varies per unit and
is unidentified.

live_offsets.rs automates the correlation and currently produces one hit which is
false -- YawDragFactor 2.0 collided with the integer 2 at +0x0c stored as a
denormal. Causes recorded: get_f32 is unreliable on default-heavy IDXD records
because the value-before-key pairing shifts, and 96 words only reaches +0x180
while RadarRange/FCSRange sit beyond it. Next run dumps 512 words.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 20:36:57 +00:00
3322202732 re(route-b): 13 defaulted Size_Y values read out of the running game
Live definition objects carry no name, so identity comes from the values:
unit_signatures.rs prints (HP, Size_X/Y/Z) for every disc unit and vessel, and a
live object is the record whose known fields it reproduces. 7 of 14 match a disc
record outright; the other 7 match nothing because their records default a field
-- 18 of 23 vessel records are missing at least one, nearly always Size_Y.

Matching each single-default record against the live object that reproduces its
remaining fields resolves 13 values (Destroyer 200, Cruiser 600/700, ASFrigate
80, ADAN Destroyer 300, Acropolis 400, ISCMissile 300, plus _Inv/_EX variants).
Every one equals that unit's Size_X, so the engine's own parsed definitions
confirm the statically-derived inheritance rule.

Limits stated: variants share one live object, and the ...Ratio/...Count family
is not resolved -- their offsets in the live object are still unknown.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 20:28:55 +00:00
a836f8e9ea re: live unit definitions from a running Stage 02, and the recipe that gets there
The runtime track is unblocked. Two environment facts cost most of the effort:
audio must be --audio --apu=sdl WITH SDL_AUDIODRIVER=dummy (muting with --apu=nop
fills the log with CreateDriver failures and the guest never passes the movie),
and everything must happen inside ONE blocking foreground call because anything
outliving its call is reaped. Boot is 25s to the flight HUD once caches are warm,
which is what makes launch-and-dump fit in one call. numpy/PIL are absent so the
analysis scripts do not run; gmem.py is stdlib and does.

Result: 14 live definition objects dumped (vtable 0x820af844, 96 words each).
The +0x54 column reads as HP across three orders of magnitude and +0x30..0x38 as
Size_X/Y/Z, corroborating the earlier single-snapshot layout, and five of the
fourteen have Size_Y == Size_X exactly, supporting the inheritance default rule.
Open: no name/id field in the first 96 words, so objects are not yet matched to
their disc records.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 20:18:32 +00:00
4d8c1b9457 docs: bring INDEX and BACKLOG up to the measured state
INDEX's XBG7 row still said 5480 decode (87.1%) and 125 of 681 shared resources
failing; it is now 6209 of 6294 (98.7%) with zero real meshes disagreeing across
containers and 46/46 capture-verified drawn buffers. The capital-ship row blamed
an XBG7 mis-decode for the user-reported assembly bug -- that mis-decode (a shared
turret ~100x too large) is fixed, so the BACKLOG entry is marked resolved at the
format layer, keeping the viewer-side pointers and the note that no metric caught
it: it took rendering the ship and looking.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 19:02:54 +00:00
33a7bf836b perf(mesh): memoise the whole-container decode
Filtering after the assignment made every subset query a full decode (~15s on a
50MB container). full_decode_cached memoises it per container -- fingerprint is
length plus three sampled 4KB windows, keyed with min_consistency, last four kept.
Decoding five ships from Stage_S02 in turn: 10.5s for the first, then 48us-1.4ms.
A stage now costs one decode rather than one per ship. Ten suites green, viewer
builds, and a spot-checked resource still lands on the same offset.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 18:57:50 +00:00
c79c0f6ccf fix(mesh): filtered decode must not depend on the requested subset
models_named pruned to the wanted set BEFORE distinct assignment, so collision
resolution saw a different resource population and returned different offsets:
27 of 356 resources in Stage_S02 decoded differently when asked for alone,
including real geometry (f001_bdy_30, f106_sld_02_l/m/d, f101_wep_01_l). Both the
viewer and assemble_ship decode subsets, so both could disagree with the
container's own answer. This was a regression from distinct assignment itself.

Fixed by filtering the OUTPUT: the assignment always runs over the whole
container. One-name, three-name and full decodes now agree exactly. Cost: a
single-resource query on a 50MB container goes from near-instant to ~15s;
per-container caching is the follow-up. Ten suites green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 18:47:41 +00:00
e8434541f2 re: monotone assignment refuted again -- box identity needs descriptor data
XBG7_MONOTONE=1 enforces descriptor order per (stride, vtx, idx) signature: 89
minority decodes, unchanged. Monotonicity constrains order WITHIN a container
while the disagreement is BETWEEN containers, which hold different numbers of
these boxes in different arrangements. The remaining 89 are bounding-box identity
ambiguities needing descriptor-level information, not another anchoring
heuristic. Knob kept, default off, measurement recorded; suites green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 18:30:58 +00:00
1d8b783d26 re: all 89 remaining disagreements are 24-vertex boxes -- no real mesh disagrees
Checking the vertex count of every minority decode: all 89 are 24-vertex
resources, including e101_wep_01_l, which last commit called the one real
geometry disagreement -- it is a 24-vert box too. So after this thread's fixes no
real mesh on the disc decodes differently in different containers; what remains
is structurally identical bounding boxes swapping identities, which needs an
ordering rule rather than a validity gate. Monotone anchoring was refuted before
but under a decoder with other defects, and is now cheap to judge: 89 -> 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 18:11:20 +00:00
cdaeac8e1b re: the consistency figure is mostly composite bounding boxes -- real disagreement is 1
consensus_check names the minority container instead of just flagging
disagreement: 89 minority decodes across 477 resources with a majority, and 88
are scene composites. A composite's descriptor carries a 24-vertex bounding box
(e_rou_e106 -> 22x22x22, e_rou_f106 -> 745x718x718); those boxes look
interchangeable so the assignment shuffles per container. 1141 of 6209 decoded
resources have scene nodes. Counting only real geometry the disc has ONE
disagreement: e101_wep_01_l in Stage_S25.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 18:02:06 +00:00
0625976ed7 re: the coverage fix moved 29 anchors, 22 of which carried another resource's geometry
Whole-disc comparison at XBG7_COVER_SLACK=4 vs =1: 29 resources changed anchor,
3 stopped decoding, 6207 unchanged. Of the moved ones present in several
containers, 0 matched the sibling consensus before and 9 after -- it moved them
onto the block their own copies agree on and none away. 22 of 29 were carrying
ANOTHER resource's geometry under their own name, including a shift chain in
Stage_S28 where n054_bdy_l held n056_bdy_l's mesh and n056_bdy_l held
n055_bdy_l's. Confidently-wrong-under-the-right-name is invisible to every
count-based metric; the coverage invariant is what separated them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 17:40:22 +00:00
095816834a re: CORRECTION -- the static assembler is not at fault, the pre-fix decoder was
Last iteration reported e303_wep_01 as inflated 30-110x by assemble_ship. Wrong:
the composite nodes carry scale 1.0 and orthonormal matrices, and under the
current decoder the turret places as 49x23x42 at +/-179. The 1600x2100x4800 box
only appears with XBG7_COVER_SLACK=4 -- the exact-coverage fix had already
repaired the turret too. The error came from comparing a deliberately pre-fix
render against post-fix measurements.

The containment screen's blindness therefore has a mundane cause: one mis-decode
inflated the envelope and hid another. Kept as a forward-looking invariant with
that caveat.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 17:28:18 +00:00
27fd9f186f re: containment screen fails too -- the static assembler inflates a shared turret
envelope_screen measures per-axis protrusion past the sibling envelope, the
relationship the eye used. It still does not flag e106_bdy_03, and dumping the
static assembly shows why: e303_wep_01's world box is 1600x2100x4800 around a
~400x400x2000 hull, so nothing can protrude. The decode is innocent -- that
resource is 49x23x42 in every container -- so the static assembler is inflating
it 30-110x per axis, non-uniformly. The screen is only meaningful once placement
is trustworthy, and the assembler now has a worse defect than the decoder had.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 17:20:52 +00:00
85ad540a7e re: the other four ships are clean; automating the slab check failed
Rendered f101/f105/f106/e105 statically -- all coherent, no stray masses, so the
e106 slab was specific. slab_screen.rs tries to automate the check by comparing
each part's min-axis extent to its ship median, but it produces identical flags
with and without the coverage fix at both 4x and 2.5x: it never sees
e106_bdy_03, the part it was built for. The eye used relationship (a mass apart
from the hull), not scale; a containment test is the right analogue and is not
built. Kept for what it does show: f002_bdy_22's 100000-unit tether and
t901_e01_D's mast are legitimate, and capture-verified f101_bdy_01 flags at 6x.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 07:22:43 +00:00
f340815db6 fix(mesh): require exact pool coverage -- found by rendering, not by a metric
Rendering assembled e106 from Stage_S02 showed the old slab: e106_bdy_03 spanning
600x1600x998 where three other containers give 276x236x941. Its anchor had slack
3, and the coverage gate tolerated up to three unreferenced tail vertices --
tolerance that was hiding a mis-anchor, since real blocks reach their last vertex
exactly (8580 of 8629). Requiring exact coverage moves it to the block the other
containers agree on and the slab disappears.

Costs 3 resources (6212 -> 6209), inconsistency 39 -> 38, capture oracle
unchanged at 46/46, suite green. Evidence: captures/e106-cover-slack-before-after.png

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 06:56:49 +00:00
9d33345560 re: the grouped pivot floor is not the blocker either -- anchor work plateaus at 98.7%
Sweeping XBG7_GROUPED_CONSISTENCY 0.85 -> 0.80 -> 0.75 decodes no additional
resources and leaves inconsistency at 39, while changing which geometry some
grouped models get -- strictly worse, so 0.85 stays. With the degeneracy/extent
fix already refuted, the remaining 82 misses are not a threshold away; they need
a structural answer (marker lists that do not map onto stored blocks), which
means a capture, not more tuning. Records the state of the four shipped changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 06:34:41 +00:00
2f0ce19f68 re: 98.7% coverage, and the scale-free degeneracy fix is refuted
Under the winding gate, coverage is 6212/6294 (98.7%) with 82 misses left,
attributed 42 degenerate/extent, 31 winding, 9 coverage, 0 connectivity. The
biggest bucket turns out NOT to be the blocker: replacing the absolute area test
with a scale-free collinearity test decodes no more resources and raises
inconsistency 39 -> 44, and dropping the extent floor to 0.05 adds two. Both stay
as opt-in knobs (XBG7_REL_DEGEN, XBG7_MIN_EXTENT) rather than defaults. Also
fixed another stale default label in edge_cap_sweep.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 06:19:38 +00:00
f835705b48 fix(mesh): winding consistency replaces the connectivity heuristic
Gate attribution put connectivity behind 153 of 225 misses -- the largest
blocker, and the gate already known to reject a capture-proven block. Testing it
against winding consistency (an objective topology signal: ~1.0 or ~0.0 for a
real mesh, ~0.5 for a mis-carve) shows a floor of 0.70 with connectivity inert
dominates the previous default on both axes: 6212 resources decoded (+143) with
39 shared inconsistencies (-17), capture oracle unchanged at 46/46, twin
invariant clean, coverage invariant intact. Suite green.

The 0.80 cliff (5770 resources, 0 inconsistent) is documented, not taken --
consistency is the weaker witness, and both points are one env var apart.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 05:47:01 +00:00
ee2b98341d fix(mesh): pre-pivot sub-meshes must cover their pool, not just index in range
The last two blocks that under-covered their vertex pool were f102_break.dat and
f104_break.dat in ptc_pack.xpr, each reading a neighbouring block's index buffer
against the wrong declaration (414 verts indexed to 404; 160 indexed to 79).
Their marker lists do not map onto the stored blocks -- only 2 of 9 and 4 of 10
sub-meshes decoded at all. Requiring coverage (max_idx + 4 >= vtx_count) for
pre-pivot sub-meshes drops exactly the mismatched pieces.

Every decoded sub-mesh disc-wide now covers its pool: 8580 at slack 0, 49 within
tolerance, none beyond, none negative. Coverage 6069/6294, inconsistency 56,
truth table 46/46 -- all unchanged. Suite green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 04:57:49 +00:00
6634d79c2e fix(mesh): grouped pools no longer emit sub-meshes with out-of-range indices
coverage_audit measures index coverage per decoded sub-mesh. 8586 of them
reference their last vertex exactly, so the 'buffer not covered' gate is well
founded -- but 18 had NEGATIVE slack: indices up to 364 vertices past the end of
their own buffer, emitted because anchor_grouped_meshes reads pre-pivot
sub-meshes unconditionally. Quality gates stay relaxed there (a tiny flat lead
part is legitimately poor) but index range is now required.

Coverage 6069/6294 unchanged, inconsistency 56 unchanged, truth table still 46/46
claimed, suite green; the vertex total drops by exactly the 1546 garbage verts.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 04:41:17 +00:00
c9c98cf942 re: coverage is 96.4% (6069/6294) -- and the 225 misses get a gate breakdown
undecoded.rs supplies the denominator the coverage numbers never had; the disc
holds 6294 XBG7 resources, 6069 decode, 225 are searched and missed, 0 lack a
descriptor. gate_histogram.rs attributes each miss to the furthest gate its best
candidate reached: 120 connectivity, 74 grouped-pool (different path), 15
degenerate/extent, 9 winding, 7 buffer-not-covered. Recorded as a work-list, not
a verdict -- a wrong candidate can pass more gates than the true block.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 04:22:43 +00:00
db8104ab99 re: n206 was fixed after all -- the audit was classifying by geometry, not buffer
debug_grouped_report (why_rejected) shows n206_02's alternative pool is ACCEPTED
at pad 0 under production gates and is in the candidate list -- and the decoder
does take it: n206_02 now anchors at 0x342d984. The 'still collapsed' reading
came from the audit comparing decoded geometry, and that offset holds a direct
(unmirrored) copy, so a separated pair still looked identical. The audit now
requires a SHARED BUFFER to call it a collapse: disc-wide 18 exact mirrors, 16
related, 0 collapses, 0 unrelated -- and the regression test drops its exception.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 04:10:05 +00:00
bb0c94c0ce fix(mesh): distinct assignment now covers grouped pools too
anchor_grouped_meshes takes the same taken set; a colliding grouped model is
re-placed whole past everything claimed, or keeps what it had. Runtime oracle
unchanged (45/46 exact, 0 unclaimed), coverage unchanged (6069), cross-container
inconsistency 62 -> 56, suite green.

It does not clear the n206 twin collapse: no alternative pool validates for the
loser, so that pair is a validator case (like eng_02_l before the cap move), not
a selection one. Also fixes a stale '0.28 (default)' label in edge_cap_sweep.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 03:53:10 +00:00
fd34d350f2 test: lock in the twin invariant -- 0 unrelated pairs disc-wide, 1 known collapse
twin_mirror_audit applies the capture-derived rule to all 166 containers: of 34
equal-count twin pairs, 18 are exact X-mirrors, 15 related another way, 1
identical, 0 unrelated. Two calibration fixes were needed first (authored halves
need a tolerance, and a mirrored pair may be stored in another vertex order).
The one collapse, n206_01/_02, is a grouped-pool pair -- the path distinct
assignment excludes -- so it names the next target. Added a disc-gated
regression test; refreshed the stale ignore message on the consistency test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 03:44:29 +00:00
55e1fe3783 re: the break-composite alarm is false -- the container stores 3-6 copies of each buffer
locate_draw counts copies of a captured buffer. The f105 parts _rou_f105_break
claims exist 3-6 times over (the 2336-vert one: 6 direct AND 6 mirrored), and
every live LOD checked is anchored on a direct, byte-identical copy -- so the
composite taking the drawn copy costs nothing. This also calibrates the oracle:
'exact' (anchored at the drawn offset) is stricter than correct, so 45/46 is a
lower bound. Open: a resource landing on a MIRRORED copy would be a real defect
invisible to every count-based metric.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 03:32:58 +00:00
7f1ac41563 fix(mesh): raise the connectivity cap 0.28 -> 0.42 on runtime evidence
Swept against the 46 capture-named Stage_S02 buffers with distinct assignment in
place: 0.28 anchors 40 exactly and leaves 4 unclaimed; 0.42 anchors 45 and leaves
none; nothing above 0.42 improves further, so it is the least permissive value
that captures the whole gain. 589 more resources decode, nothing is lost. The
cross-container consistency proxy worsens (46 -> 62) -- it cannot see a
systematic mis-anchor, and where the two disagree the capture wins. Suite green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 03:25:50 +00:00
d83bdd3b68 fix(mesh): distinct anchor assignment -- no two resources may claim one buffer
Selection was per-resource and greedy, so two resources could take one vertex
buffer while a valid one sat unused. A runtime capture proved that wrong for the
mirrored e106 hull twins: the container holds both halves and the engine draws
each from its own address. Now the first claimant keeps a buffer and later
resources re-anchor past everything already claimed (coverage can never regress;
grouped-pool models untouched).

Against the 46 capture-named Stage_S02 buffers: exact anchors 29 -> 40, unclaimed
12 -> 4. Disc-wide: 5480 resources decoded (unchanged), cross-container
inconsistency 125 -> 46.

The twins' mirror therefore lives in the DATA, not in the placement matrix: the
embedded e106_bdy_02 row and the two assertions encoding the old convention are
updated, each with the reason recorded. Full suite green incl. disc/ISO gates.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 03:16:51 +00:00
1815490d01 re: scored against the runtime, relaxing the cap is monotone -- the regressions are collisions
Against the 46 capture-named Stage_S02 buffers: 0.28 gives 29 exact anchors,
0.42 gives 31 with 0 lost and 2 fewer unclaimed. The 18 cross-container
consistency regressions are a different phenomenon: the decisive one is the
32-vertex buffer at 0x24ce5f4, unclaimed at 0.28 and claimed by BOTH f303_body_l
and e302_barrel_l at 0.42 -- a collision from greedy per-resource selection, not
a mis-placement. So the cap and the selection must change together.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 03:06:15 +00:00
b58f42e46d re: the capture is Stage_S02 -- provenance corrected, oracle grows to 46 buffers
capture_truth_scan places every drawn buffer across all 166 containers by modal
vbase-offset. Stage_S02 wins with 64 matches at one constant against Stage_S01's
16, and the logs carry f101/f105/f106/e105 -- a Stage-02 cast. Stage_S01 looked
consistent because the shared block is duplicated verbatim (twins 0x116F0 apart
in both), so the earlier structural findings hold; only the loaded-container
claim was wrong. The S02 table places 46 buffers, 12 claimed by nobody, and
resolves the six Stage_S01 mystery buffers as e105 parts.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 02:59:40 +00:00
df7a10749a re: calibrate the connectivity cap -- +612 resources against 18 consistency regressions
XBG7_EDGE_CAP / XBG7_SMALL_TRIS make the threshold sweepable (defaults unchanged,
full suite green). Above 0.417 the capture-proven eng_02_l anchors exactly right
and no e106 part regresses, and nothing that decoded at 0.28 is lost -- but ~250
existing anchors move silently and 18 shared resources lose cross-container
consistency. Not changed: the movers have no oracle yet.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 02:47:07 +00:00
845b63d7dd re: eng_02_l's real block is adjacent -- the connectivity heuristic rejects it
debug_find_index_buffer scans the container for an index buffer that validates
against a capture-proven vertex buffer. For eng_02_l nothing validates with the
connectivity test on; with it off the nearest hit is exact pad-0 adjacency
(vb-ib = 144 = 72*2). The block's mean_edge/diag is 0.417 against a 0.28 cap --
the documented false positive for coarse LODs, now caught with ground truth.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 02:31:53 +00:00
dde52c02e7 re: correct the 3.4% reuse figure -- the capture dedups by (vbase, transform)
Read off the emulator patch rather than inferred: CaptureShipDrawForRE dedups by
(vbase, WVP hash), so a buffer drawn repeatedly at one transform appears once.
The 3.4% counts multi-instance placement, not buffers serving several parts, and
is not the population evidence about sharing it was written up as. Also pins
vcount = buffer capacity and indices = that draw's num_indices.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 02:24:44 +00:00
51723af59a re: the twins' correct block validates -- distinctness is a real fix; eng_02_l is not
debug_try_anchor asks validate_block directly at the capture-proven offsets. Both
119-vert twin buffers are accepted by both twins (so the correct block lost the
first-match race, and a distinct assignment fixes it); the drawn 51-vert bridge
buffer is accepted by both bridge resources; eng_02_l's proven offset is rejected
outright, even with the pad widened to 64 -- a validator gap, not a selection one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 02:17:55 +00:00
76a433f75c re: the proven offsets ARE candidates -- these mis-anchors are selection failures
debug_vertex_run_starts exposes the anchor scan's candidate list: Stage_S01 has
15710 stride-24 starts and all three capture-proven offsets (0x3c55d8 twin,
0x40e418 bridge, 0x44a32c eng_02_l) are among them. anchor_pool_mesh takes the
first that validates, so an earlier lookalike wins. Scoped to the current
decoder's e106 cases; does not overturn the residual-51 finding.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 02:11:41 +00:00
26f1bdf6fd re: exact anchor offsets (GameMesh.vbuf_offset) -- correcting yesterday's defect table
The first read of the capture-truth table located our resources by searching the
container for their leading vertices, which reads much worse than reality: the
same leading run occurs at several offsets in one container. GameMesh now carries
the offset the anchor scan actually chose, so the comparison is exact -- 4 of the
ship's drawn buffers are anchored correctly, 2 are the twin collapse, and 2 are
real mis-anchors of a size we do decode (brg 51 verts, eng_02_l 44).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 02:04:55 +00:00
3245aca8cb re: the capture names file offsets -- the twin collapse is an anchoring error
Locating each drawn buffer's positions inside Stage_S01.xpr shows vbase - offset
is one constant (0x1A94FFF4, same in two runs), so a capture gives ground truth
at file-offset granularity. Read against our anchor scan it is a defect list:
full/_m resources starting at their own _l buffer, eight drawn buffers claimed by
nobody, and the bdy_01_l/bdy_02_l twins sharing one buffer while the container
carries both halves (0x3b3ee8 and its exact X-mirror at 0x3c55d8).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 01:55:13 +00:00
aa82d293a9 re: the capture logs are the sharing oracle -- engine reuses a buffer only 3.4% of the time
The raw Canary capture logs from the 2026-07 sessions are still on disc at
/sylph-home/re/shipcap/xenia_ship_capture_*.log, and their DRAW lines carry
vbase -- the GPU vertex base, i.e. ground truth for which buffer the engine draws
a part from. That is precisely the oracle the shared-decode question needs, and
it needs no new capture run.

Restricted to the ship-geometry stride 24, across the three logs: 6093 draws from
2291 distinct vbases, and only 77 vbases (3.4%) drawn more than once. So one
buffer serving several parts is the exception in the engine, while our decoder
has 19% of resources sharing geometry.

The measures differ (draws-per-buffer vs resources-per-geometry) so this is
evidence not proof, but it points the same way as e106_bdy_02 == e106_bdy_03_m:
much of that 19% is anchoring error rather than asset reuse.

Per-part use is already available: correlate_capture matches a draw to a resource
by vcount plus decoded positions, and the same match yields each part's vbase --
so two resources our decoder gives identical geometry can be checked directly,
different vbase meaning our shared decode is wrong.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 01:40:56 +00:00
6d96446735 re: the descriptor does not address geometry, and consistency does not prove reuse
Two follow-ups, both negative, both worth recording so the next attempt does not
repeat them.

The descriptor cannot replace the candidate scan. If entry.data_offset located
the geometry, anchoring would be direct addressing. Across e106's resources in
Stage_S01, anchored_vb - data_offset ranges from 1199052 to 4173988 with no
constant or stride; data_offset locates the DESCRIPTOR and none of the first six
descriptor words tracks the vertex pool.

And yesterday's "1241 of 1242 pairs identical in every container, so sharing is
legitimate reuse" is weaker than it read. A SYSTEMATIC error is invisible to a
consistency test, because it is consistent. The same dump shows e106_bdy_02 and
e106_bdy_03_m anchoring to the identical offset (1505556), and e106_bdy_01_l with
e106_brg_01_m (4251208) -- a hull half and a different body's medium LOD, a hull
half and a bridge LOD. Different parts; one of each pair is wrong.

Honest position: sharing is common (19%), some is certainly legitimate (a
mirrored twin pair genuinely shares one geometry) and some certainly is not, and
cross-container consistency cannot separate them. What can: comparing a shared
pair against a runtime capture.

Instrumentation reverted; only docs land here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 01:32:34 +00:00
f655c90b0c re: shared decodes are mostly legitimate reuse -- correcting yesterday's claim
I wrote that "two distinct resources sharing one decode is itself the bug" when
withdrawing the neighbourhood anchor. Measured, that is wrong.

1043 of 5480 decoded resources (19%) share geometry with another resource, and of
1242 related pairs that share a decode somewhere, 1241 are identical in EVERY
container they co-occur in. That is what legitimate asset reuse looks like, not a
defect. A mirrored pair like bdy_01/bdy_02 is SUPPOSED to share one geometry with
the reflection applied at placement -- which is precisely what apply_twin_mirrors
does.

So the twin-mirror regression was never about sharing. It is about WHICH of two
mirrored buffers is canonical: the disc holds an X+ and an X- version, the engine
treats one as the base, and the neighbourhood anchor moved these resources to the
nearer buffer, which is the other one. The fix is to pin the canonical buffer
with the capture as oracle, not to split the twins apart.

Exactly one pair is provably mis-anchored by this test: e105_bdy_02_l /
e105_brg_m, identical in 7 of the 15 containers holding both and different in the
rest -- two names cannot be the same geometry only sometimes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 01:25:44 +00:00
aedad811bc test: fix the stale default disc paths -- three tests could never run unaided
texture_disc.rs and mesh_disc.rs fall back to "/home/fabi/RE Project Sylpheed/..."
when SYLPHEED_ISO / SYLPHEED_RES3D are unset. The directory is "RE - Project
Sylpheed"; the fallbacks were missing the " - ", so stage_models_decode,
stage_models_sweep and stage_models_quality_audit failed with NotFound for anyone
who did not happen to know the env var. They now pass unaided (10/10 in
mesh_disc).

Found while auditing what the suite actually exercises after withdrawing the
anchor change: an ISO-gated test skipping, and an ignored test failing on a typo,
are both "green" in a normal run.

Full state, with SYLPHEED_ISO/DISC/RES3D set and --include-ignored: 82 lib tests
and every disc/ISO suite pass; the single failure is
shared_resources_decode_identically_in_every_container, which is deliberately
written as the target state and #[ignore]d.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 01:18:10 +00:00
64d372c7e8 revert(mesh): withdraw the neighbourhood anchor -- it regressed the e106 twin mirror
The neighbourhood anchor (f18d591) and its refinement (27a0701) took
cross-container inconsistency from 125 to 51 with coverage unchanged, and made
e106 render as a destroyer rather than a slab. Both are reverted.

ship::tests::static_assembly_matches_runtime_capture is gated on SYLPHEED_ISO, so
it SKIPS in an ordinary cargo test -- which is why the regression was invisible
in every suite run so far. With the ISO it fails:

  e106_bdy_01: static M row0 [-1.0, 0.0, 0.0] != captured [1.0, 0.0, 0.0]

e106_bdy_01 and _02 are a mirrored pair whose two buffers hold the same geometry
reflected in X, and BOTH resources currently decode to the SAME buffer (identical
counts, span and mean_x). apply_twin_mirrors picks which instance to reflect from
the sign of that mean_x, so which buffer wins flips the decision:

  before  both twins mean_x = -66.83  -> mirror bdy_02  (matches the capture)
  after   both twins mean_x = +66.83  -> mirror bdy_01  (contradicts it)

Neither is right -- two resources sharing one decode is itself the bug and the
mirror heuristic has been compensating. The capture is ground truth, so a change
that contradicts it does not ship. The real fix must give each twin its own
buffer first.

Kept from the attempt: this test now also asserts the SET of static placements
against the capture (allow-list {e303_wep_01} for vbase dedup), so extra
placements can finally fail it -- the direction it could never fail in before.

Docs, backlog, INDEX and the ignored test's message all corrected to say
diagnosed-not-fixed rather than fixed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 01:08:41 +00:00
0f9c95c52e re: selection fixes are exhausted for the last 51 -- the correct block is not a candidate
Implemented the proposed tiebreak as a last resort (take the attractor only if
nothing else validates), keyed first on "all coordinates multiples of 50" and
then on the sharper "all coordinates integral" -- the attractor reads
(42, -900, 2400), (-600, -500, -950) while real geometry carries fractions like
(24.55, 9.84, 4.46). Neither changed anything: still 51 inconsistent, 5480
decoded.

The null result is the finding. A mechanism that defers the attractor whenever
another candidate exists, and changes nothing, means no alternative candidate
validates for any of the 51: the correct block is not in the candidate list at
all. So the residual is not a selection problem and no reordering, scoring or
tiebreak will move it.

The frontier is vertex_run_starts -- the unit-normal run scan that builds the
candidate list -- which emits no start for these resources' real vertex buffers.

Both attempts reverted; only docs land here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 00:57:09 +00:00
343a26ecc0 re: the remaining mis-decodes share one attractor, not pairwise confusion
Tested the tempting explanation for the last 51 -- that _l/_dead variants share
their base's vertex and index counts, making them mutually confusable and
adjacent so locality cannot help. It is FALSE: 2714 variant/base pairs across
every container, and exactly zero share identical counts.

What is actually happening is better news. e010_bdy_01_l is 171 verts / 90 tris
and NO other resource in its container shares those counts, yet in Stage_S02 and
S26 it decodes to 1600x2100x4800 -- the same bounds e303_wep_01 (172/110)
produced before the fix. Differently-shaped resources land on the same place, so
the attractor is a region of round axis-aligned box data that validates for many
(vtx, idx) shapes at once, not "another mesh with my shape".

That also explains why the neighbourhood fix helped so broadly: it steers
resources away from one strong attractor rather than resolving many pairwise
confusions.

Candidate next step recorded with its risk: de-prefer candidates whose sampled
positions are all multiples of 50 with a large span. It must be a TIEBREAK
between validating candidates rather than a rejection -- an earlier sweep for
that signature also flagged legitimate e_rou_* composite proxies -- and it needs
the same gate (decoded must not fall from 5480).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 00:46:32 +00:00
27a0701e0d fix(mesh): refine the anchor map before using it -- inconsistency 63 -> 51
Pass 1's anchor map contains exactly the mistakes the neighbourhood is meant to
correct, so a resource sitting beside a mis-anchored neighbour inherits a bad
reference. Re-anchoring against the improving map and repeating converges
quickly: two rounds, and a third changes nothing (the loop exits early when a
round is a fixpoint).

  before                 decoded 5480/6294  inconsistent 125
  neighbourhood anchor   decoded 5480/6294  inconsistent  63
  + refining the map     decoded 5480/6294  inconsistent  51

Coverage still unchanged. The 51 that remain cluster in _l (LOD) and _dead
variants -- e001_l, e010_bdy_01_l, e106_eng_02_l, _rou_f302_base_dead,
e303_base_dead and friends. A plausible reading is that a variant shares its
base's vertex and index counts, making the two mutually confusable so that
locality cannot separate them; recorded as untested rather than asserted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 00:39:35 +00:00
f18d5919f7 fix(mesh): anchor XBG7 resources by neighbourhood -- inconsistency 125 -> 63, ships render right
anchor_pool_mesh took the FIRST candidate in file order from a container-global
vertex-run scan, so a resource could be handed another resource's block whenever
both shared (stride, vertex count, index count). Both blocks are real geometry and
both pass every quality gate, so only position separates them.

anchor_pool_mesh_near now tries candidates in order of distance from a reference,
and anchor_models_filtered runs two passes: pass 1 anchors first-match to learn
where resources land, pass 2 re-anchors each resource preferring the median anchor
of its +/-2 descriptor neighbours. Too few anchored neighbours -> keep pass 1, so
nothing regresses to guesswork.

  before  decoded 5480/6294  shared 681  inconsistent 125
  after   decoded 5480/6294  shared 681  inconsistent  63

Coverage unchanged, inconsistency halved. e303_wep_01 decodes to 49x23x42 in ALL
containers now, and e106 renders as a destroyer instead of a slab -- its two
shared turrets symmetric at X[-203,-154] and X[154,203]. That resolves the
user-reported "capital ships assemble wrong" for this cause.

The filtered path needed care: models_named (what the viewer uses) dropped
non-wanted resources, which would have left filtered decodes with no
neighbourhood and silently kept the old behaviour. Resources are now collected
regardless of the filter, but only the asked-for ones and their +/-2 neighbours
are decoded in pass 1, so a filtered decode stays proportional to what was asked.

63 cases remain; mesh_consistency_disc stays ignored and now records 63, not 125.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 00:31:20 +00:00
d660705c47 re: the correct anchor is already a candidate -- locality selects it, monotonicity does not
Enumerating every validating candidate for e303_wep_01 in Stage_S02 gives exactly
two:

  vb = 18403456   span 1600 x 2100 x 4800   <- taken, only because it is first
  vb = 52257440   span   49 x   23 x   42   <- correct, the size all 11 good
                                               containers give

So the scan is not missing anything; the wrong block merely appears earlier in
file order.

Monotonic anchoring is REFUTED as the constraint: recording each resource's
accepted anchor in descriptor order gives 25/47 increasing steps in Stage_S01 and
130/248 in Stage_S02, no better than chance. Locality holds instead -- in
Stage_S02 the descriptor neighbours anchor at 51974668 and 52218424, the correct
candidate is 52257440, and the wrongly-taken block sits at 18403456, two thirds
of the file from its own family.

Proposed rule, needing no new format knowledge: among validating candidates
prefer the one nearest the neighbouring resources' anchors, falling back to
first-match when there is no neighbour yet. It selects 52257440 here.

Not implemented: it moves the anchor for all 6294 resources, so it needs the
before/after measurement (decoded must not fall from 5480, inconsistent should
fall from 125) and then un-ignoring mesh_consistency_disc.

Instrumentation reverted; only docs land here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 00:16:22 +00:00
7c514e3a57 re: ROOT CAUSE of the silent XBG7 mis-decodes -- the candidate list is container-global
anchor_pool_mesh walks candidates from vertex_run_starts(bytes, data_base,
stride): ONE scan of the whole container per stride, shared by every resource of
that stride, and it accepts the first candidate that validates. Nothing ties the
chosen block to the resource it belongs to, so two resources sharing (stride,
vertex count, index count) are interchangeable to this search.

The wrong block is not distinguishable by quality. Tracing the accept for
e303_wep_01:

  Stage_S01  vb=4600480   span   49 x   23 x   42   passes 0.85 = true
  Stage_S02  vb=18403456  span 1600 x 2100 x 4800   passes 0.85 = true

Both clear the strict winding gate, because the wrongly-taken block IS real
coherent geometry -- another resource's. That rules out a family of fixes: no
threshold, scoring, or best-of-N selection can help, which is exactly why the
previous iteration's attempt changed nothing.

The search space must be constrained instead. Two candidates recorded, ordered by
how much new format knowledge they need: per-resource data extents from the XPR2
resource table, or monotonic anchoring (resource k's vb after resource k-1's) if
data blocks follow descriptor order -- testable against the 556 resources that
already decode consistently.

Instrumentation reverted; only docs land here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 00:08:38 +00:00
b5a4764567 re: the XBG7 silent mis-decode is in the per-block anchor, not the grouped-pool one
Tried the obvious fix: make anchor_grouped_meshes pick the best-scoring vb0
instead of the first candidate clearing the 0.85 gate. It changed nothing --
still 5480 of 6294 decoded, still 125 inconsistent -- and instrumenting the pivot
loop shows why: for e303_wep_01 that function never runs. The resource has a
single sub-mesh, so it goes through the per-block adjacency path
(anchor_pool_mesh) instead.

Recorded because the grouped-pool anchor is the prominent, well-documented one
and the natural first suspect; the fault is in the other path.

The change is reverted -- untargeted, unproven, and it added a scoring path for
no measured benefit. The idea behind it (several vb0 candidates can clear the
gate, and first-in-scan-order is an arbitrary tiebreak) still applies to whichever
anchor is actually at fault.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 00:00:37 +00:00
f34280476d test: lock today's format findings into disc-wide guards
Each finding today was originally missed by reasoning from a sample, so these
assert DISC-WIDE invariants rather than one hand-picked file.

  ui_surfaces_disc.rs (3 passing)
    every_t8ad_on_the_disc_decodes            -- all ~19216 surfaces; the old
                                                 256-grid model looked like 96%
    lsta_count_equals_sprites_plus_primitives -- header counts T8aD AND PRMD,
                                                 64/64, which is what made the
                                                 count look unreliable
    ratc_nesting_is_exactly_one_level         -- nested records are leaves; zero
                                                 grandchildren disc-wide

  mesh_consistency_disc.rs (1 ignored, deliberately)
    shared_resources_decode_identically_in_every_container

The mesh test is written as the TARGET state, not a snapshot of the bug: a
resource shared by several containers must decode to the same bounds, which today
fails for 125 of 681 shared resources. Fixing the anchor scan makes it pass;
un-ignoring it is then the last step rather than a rewrite. It only compares
decodes that agree on vertex/triangle counts, so "found different geometry" stays
a separate question from "placed the same geometry differently".

All suites green: 81 lib + the disc guards, with 2 ignored.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 23:50:20 +00:00
140856b3f9 docs(re): session log for 2026-08-11, and correct the capital-ship INDEX row
The capital-ship row still said placement was the open question. Today's work
says otherwise: placement is sound (hull static-exact, cross-id mounting is 2
pairs across 335 ships) and the user-reported "ships assemble wrong" is an XBG7
mis-decode instead. The row now says that and points at the backlog and the mesh
doc, and keeps the one-way-test problem visible.

SESSION-2026-08-11.md summarises the run for review, leading with the single item
that needs the user's decision (editing save slot 01, deliberately not done) and
separating findings, the two disc defects found, and the process notes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 23:41:58 +00:00
4097e8503f re: a detector for silent XBG7 mis-decodes -- 125 of 681 shared resources fail it
Following the e106 slab to its cause. e303_wep_01 decodes from 14 containers: in
11 it is a 49x23x42 turret with organic vertices, in Stage_S02/S08/S26 the same
resource -- identical 172 verts and 330 indices -- decodes to 1600x2100x4800 of
axis-aligned box corners with (0,0,1) normals. The anchor scan found a different
buffer that happens to share both counts, so every size-based check passes. That
is the "declined for location, not format" risk succeeding wrongly instead of
declining.

The detector needs no ground truth: a resource shared by several containers must
decode to the same bounds. 681 resources appear in >=2 containers and 125 decode
to different bounds while reporting identical vertex/triangle counts -- a lower
bound, since a resource wrong everywhere is invisible to it. _rou_f401 is
62x25x10 in 16 containers and 4738x3147x4738 in 2; _rou_e011_wep05 gives four
different spans across 8.

Majority vote would resolve 104 of the 125 (14 are exact 50/50 splits) and agrees
with ground truth in the one case that has a render and a runtime capture behind
it. Recorded as a heuristic, not applied: for _rou_e302_base_break the majority
is the LARGER span and nothing yet says which is right. Flag, do not silently
rewrite geometry on a vote.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 23:34:31 +00:00
c9916bcb41 re: CORRECTION -- the e106 slab is a container-dependent mis-decode, not a stray volume
One iteration ago I concluded assemble_ship was drawing a non-renderable
collision volume. That named the wrong cause, and decoding the same resource from
every container that holds it settles it:

  Stage_S01  172 verts 110 tris  49 x 23 x 42        <- a turret, correct
  Stage_S02  172 verts 110 tris  1600 x 2100 x 4800  <- wrong
  Stage_S08  same                1600 x 2100 x 4800  <- wrong
  Stage_S26  same                1600 x 2100 x 4800  <- wrong
  eleven others                  49 x 23 x 42        <- correct

Same resource, same vertex and triangle counts, correct in eleven containers and
wrong in three. So the placement is legitimate (e303_wep_01 is a small shared
turret cross-mounted on e101/e106), the original author's vbase-dedup explanation
of the capture's silence stands, and my "dedup would show one, not zero"
objection does not survive -- at its true size the turret is ordinary geometry.
The defect is in the mesh decoder.

The wider point: the decoder can produce wrong geometry WITHOUT declining. The
XBG7 audit counted 814 honest refusals; this is the other kind, silently 100x too
large. A screen for the signature (bounds exact multiples of 50, span > 1000)
flags 22-32 models each in S02/S03/S08/S26/S27, but it also catches legitimate
e_rou_* composite proxies, so that is a candidate list and not a bug count.

Next: diff the anchor scan's chosen vb0 for this resource between Stage_S01 and
Stage_S02 -- same resource, two outcomes -- and turn whatever distinguishes them
into a post-decode sanity check so a silent mis-decode becomes a decline.

Kept from the previous entry: the test can only fail one way, and cross-id
placement is genuinely narrow (2 pairs across 335 ships).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 23:25:55 +00:00
c5a5065389 re: the format layer is NOT exonerated -- assemble_ship draws a non-renderable volume
Did the visual the backlog kept naming as the next step, and it overturns the
conclusion. Rendering e106 from the static assembly and from the baked runtime
capture: the capture has 8 placements, the static assembly 11 -- the same 8 plus
e303_wep_01 x2 and a second e106_eng_01. In the render the destroyer sits inside
a white slab that dwarfs it.

That slab is e303_wep_01: 172 verts, 110 tris, bounds X[-1000,600] Y[-1050,1050]
Z[-2400,2400] = 1600 x 2100 x 4800. A real e106 turret (wep_02_01) is 269x179x417
with 772 tris; the bridge is 105x76x305. So it is not a turret -- it reads as a
collision/trigger volume, and whatever it is the game does not draw it: the e106
capture contains no e303_wep_01 at all.

Why it was missed: assemble_ship treats every rou_* composite node as drawable,
the cross-id mount is documented as intended, and the test asserts
count("e303_wep_01") == 2. The capture's silence was explained as vbase dedup,
but dedup would show one instance, not zero -- and the test walks capture parts
looking them up in the static output, so EXTRA static placements cannot fail it.
Same shape as the earlier include_external gap: a test that can only fail one way.

Scope stated carefully: sweeping 335 ships for "ship-scale span with <400 tris"
flags 20 ships and 58 placements over 28 resources, but ONLY the e106 case is
proven (render + capture absence + geometry). Others may be legitimately large
low-poly parts and need the same three checks.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 23:15:22 +00:00
5b8b5e0ab1 re: measure the XBG7 declined set, and close off the descriptor shortcut
"A few multi-stream / quantized bodies remain" declined understates it. Across
all 166 resource3d containers: 6294 XBG7 resources, 5480 decoded (87.1%), 814
declined, in 31 of 166 containers. By name the declined set is 492 e* (enemy
craft), 142 f*, 73 n*, 23 eff*, plus *_dead destroyed variants -- not hero
bodies.

Also recorded as a dead end, so nobody repeats it: the descriptor's third word
looks like (streams << 16) | format -- decoded g001..g003 carry 0x00010001 while
declined t170/t180 carry 0x00020004 -- but histogramming it disc-wide puts
decoded AND declined at every value (0x00010001 alone: 4479 decoded, 328
declined). Its low half tracks sub-mesh count. Decodability is not declared in
the descriptor; it is whether the unit-normal anchor scan can locate vb0, which
the code already tests. 229 declined resources carry the most common word[2] with
under 1 KB of data, i.e. small meshes with too little signal to anchor rather
than exotic formats.

No decode gained this iteration; the gap is now quantified and one attractive
wrong turn is closed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 23:03:58 +00:00
d9c69c566c lsta: the count was never unreliable -- it counts sprites AND PRMD primitives
The module said "a few entries disagree with the actual frame count, so we walk
by the T8aD magic instead". They do not disagree. An LSTA is a display list of
inline elements that are either T8aD sprites or PRMD primitives (the flat
coloured quad the UI bundles use to dim a scene), and the count at 0x04 counts
both: across all 64 lists on the disc, count == T8aD + PRMD, with no exceptions.

The six lists that looked wrong (GP_DEBRIEFING_PILOTLOG, GP_MISSION_SELECT, two
language builds each) each hold exactly one primitive, which is the whole of the
off-by-one.

Also measured after the T8aD rectangle-list fix: all 1281 sprite frames decode,
100%.

parse() still returns sprites and skips primitives -- that is the useful
behaviour -- but the docs now say so instead of blaming the header.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 22:52:02 +00:00
d9ffc223bc re: RATC "one level deep" is not a limitation -- measured, plus a second dangling asset
INDEX read as if the parser stopped at one level. Measurement says there is
nothing deeper: 2859 top-level bundles hold 18002 children at depth 1 and ZERO at
depth 2, with no blob failing to parse.

The children that are themselves RATC -- the .rat layout records -- are leaves.
They carry no child list and reference their siblings BY NAME (the sprite they
place, and via "opt " their focused variant): 3311 such leaves, every one
embedding sibling names, and 10144 of 10148 references resolve within the same
bundle. That is the same by-name convention used one level up (a screen config
naming .prt components) and one level up again (the movie table naming
<lang>.pak+<member>).

The 4 unresolved references are a disc defect, not a parse gap: pmbase.rat in
GP_STAGE_CLEAR.pak's four language builds places pmbase.t32, and that sprite
exists nowhere -- not a pak member, not a child of those bundles. Second such
dangling asset after SUBTITLE_S12B.tbl, so a reimplementation should skip a
missing sprite rather than call it a decode failure.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 22:44:19 +00:00
ac10d3bd19 t8ad: a surface is a list of sub-rectangles, not a 256 grid -- disc decode 96% -> 100%
The "~15% deferred variants" were not variants. Auditing every T8aD on the disc
gave 19216 surfaces, 18442 decoding (96.0%) and 774 failing in two clusters:
GP_DIALOG strips declaring 524x63 with a "tile count" of 1 or 2 instead of 3, and
small textures in the six *2D language paks whose pixels ran past the end of the
file.

Both fall out of the per-tile header, which is not opaque flags: it is four BE
u32 -- dst X, dst Y, width, height. A 15x18 icon stores a 13x18 rectangle at
(1,0); pdmes010 stores (59,6,256,54) and (315,6,149,54), the second beginning
exactly 16 + 256*54*4 bytes after the first. So 0x1c is a RECTANGLE COUNT and the
256-grid reading was an accident of most surfaces being stored as full-width
bands.

Parser rewritten to that model, still refusing to guess: a rectangle must fit the
declared surface and its pixels must fit the file, else None. Disc decode is now
19216/19216 = 100.00%.

Two test fixtures were built to the old model and are corrected rather than
worked around. lsta's t8ad_frame wrote NO offset-table entry, so the decoder read
"pixels" from inside the header -- the test passed only because it checked
dimensions alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 22:37:21 +00:00
9228255bfa re: validate the screen layout against the running game -- eight rows, +/-2 px
Booted to the ARSENAL and detected the eight teal category chips by colour, then
compared their row centres against the eight prbtn1..8.rat placements parsed from
eng\prmain_scr.prt. All eight fit

    screen_Y = placement_Y + pivotY(14) + chrome(45)

with residuals -1,-1,-1,-2,0,-2,-1,-2 px. The spacings are the real signature:
predicted 54,56,55,54,56,55,55 against observed 54,56,54,56,54,56,54 -- an
irregular alternating pattern, not a round number that could match by luck. The
one free parameter is the 45 px emulator window chrome, which is not part of the
game, and the residual is centroid measurement noise.

Confirms three things together: the declaration table is the element set, the
placement region gives real screen coordinates, and the pivot composes additively
(pivotY 14 = half the 28 px chip, so placement is top-left as documented).

Explicitly NOT confirmed: the max-dwell rule for animated elements. These buttons
are static -- every keyframe identical -- which is exactly what makes them a clean
ruler. That rule needs an element captured mid-slide.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 22:25:18 +00:00
35e2d1067f re: placement X/Y are SIGNED, the tail word is time, and the resting pose is max-dwell
Dumping the remaining Arsenal components turned up placements reading as
4294966780 and 4294966856 -- negative as i32 (-516, -440). X and Y are signed;
off-screen animation starts are negative, and read as unsigned a reimplementation
would draw them four billion pixels off.

The raw region then showed the rest of it. The trailing word is a TIME, and a
group is an in -> hold -> out animation: prselect_win1 runs
t=4:-516 -> 6:-71 -> 7:81 -> 8:127 -> 23:134 -> 24:134 -> 25:127 -> 27:81 ->
31:-71 -> 1:-516. So NEITHER the first nor the last keyframe is where the element
sits -- both are off-screen. The resting position is the max-dwell keyframe,
(127,155)..(134,155) here, which is where the weapon-list panel actually appears.

This corrects my own note from earlier today, which reported "final=" positions.
Static elements were unaffected (the pause menu's buttons have identical
keyframes) but animated ones were misread. screen_layout.rs now reports the
max-dwell pose plus the full timeline, and both captures are regenerated.

Also seen while dumping: kind 0x2, and a .sbo child (scrollbar) parented to the
list window.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 22:14:36 +00:00
de357ffdc9 re: the screen parse generalises to the ARSENAL, plus kind 0x4 and the component model
examples/screen_layout.rs dumps a bundle's declaration table and placement region
together. It reproduces the tutorial pause menu exactly and reads the ARSENAL the
same way -- 23 elements that match the running game: eight buttons prbtn1..8.rat
at X=242 evenly spaced (the config declares WEAPON_CATEGORIES = 8, and eight
categories are what the Arsenal shows), prexp3.t32 declared SEVEN times at X=726
34px apart (the DATA SHEET rows), prexp1 sliding (726,143)->(1286,143) with
prexp1a parented to it, and prmsg at (151,645).

Two additions to the format:

  kind = 0x4 marks a REPEATED INSTANCE of a sprite -- prexp3.t32 appears once
  with 0x0 then six times with 0x4, each with its own placement. So the element
  name is not a key; the declaration index is.

  A screen composes from named .prt components. GP_HANGAR_ARSENAL.pak has 510
  RATC entries because its config names components (Menu = prmain_scr.prt, etc.)
  and they resolve under the config's own PATH prefix: prmain_scr.prt is absent,
  eng\prmain_scr.prt is present -- the same <lang>+<member> convention the movie
  table uses. A sub-component reads identically: psselect_win1 declares 4
  elements, three parented to element 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 22:06:00 +00:00
f65387143e re: the UI screen gap is closed -- the bundle also carries every element's placement
Immediately after the declaration table the RATC bundle holds a placement region,
one group per element: a header of (u32 element index, u32 keyframe count)
followed by that many 40-byte keyframes carrying scale/tint/X/Y. The element
index is stated in the header, so the correspondence is read, not inferred from
order.

Tutorial pause bundle: 11 groups for 11 elements, every header index and count
matching the blocks present. The values are self-evidently right -- three menu
buttons at X=546 exactly 70 px apart (288/358/428), title (540,119), message
(451,545), and the eff* sprites carrying multi-position fly-ins.

Cross-checked: pgp_ttrl_btn10.rat places its own sprite at (546,288), identical
to its inline group, so the inline region is the same placement data and it
covers the eff*/deli*/msg elements that have no record of their own.

So a screen is fully reconstructible from its bundle alone: element list and
order from the declaration table, placement and animation from this region,
plus the sprites, with "opt " supplying focused states. This retires the caveat
added earlier today that only the order was known.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 21:58:21 +00:00
dd8770826b docs: INDEX — retire the stale 'loop1.rat not decoded' note
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 21:50:39 +00:00
8814df843b re: the RATC declaration table IS the screen's draw list, and +32 is a parent index
The doc's open gap was "where does the screen's draw list live" -- the eff/deli/msg
sprites have no .rat and loop1.rat turned out to be an animation. It is the
bundle's own declaration table, which is NOT the child listing: children are
grouped by type (all .t32, then all .rat) while the declaration table names
ELEMENTS in a plausible back-to-front order and contains exactly the missing set.
For the in-mission pause menu it lists 23 elements ending with loop1.rat, and it
omits the focused button variants -- those are reached via each base record's
"opt " link -- so it is the screen's element set, not an inventory.

Also corrects the entry layout: it is FIVE words after the name, not "u32 x4
flags", and word +32 is a PARENT ELEMENT INDEX (0xffffffff = none). pgpeff02a
carries 3 and element 3 is pgpeff02; pgpeff03a carries 5 and element 5 is
pgpeff03. Verified on two independent language builds, no out-of-range value in
any bundle, and the tutorial bundle (no "a" variants) is 0xffffffff throughout.

Bounded honestly: what is drawn and in what order is now answered; WHERE the
eff/deli/msg sprites land is not -- the declaration entry has a pivot but no X/Y.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 21:50:13 +00:00
8b7c9da067 re: clear mirror handling and the exhaust cones -- the viewer bug's static avenue is exhausted
Mirror handling: the concern was that apply() bakes R*(S*v)+T while the viewer
decides winding from det(m) alone and rotates normals by m alone, so a mirror
encoded as negative SCALE would reflect geometry without flipping winding. It
never happens -- across 1485 assembled parts in all 22 containers there are 22
mirrored parts, every one with det(m) < 0, and ZERO negative or non-uniform
scales. apply_twin_mirrors writes the reflection into m, so the flip always
fires and ignoring s for normals is harmless.

Exhaust cones: the only geometry the viewer invents, drawn at GN_Jet/GN_SJet
frames because the real engine geometry is recessed. Across 335 assembled ships,
192 with exhaust frames, not one cone sits outside its hull bounding box (10% of
span tolerance). Caveat kept in the doc: inside-the-box does not prove a cone is
correctly oriented or sized -- it rules out the reported symptom, no more.

Every mechanism the diagnosis proposed is now eliminated (include_external,
duplicate names, mirrors, cone placement). The visual comparison is no longer
the next step but the only remaining one, and if it agrees the original report
needs re-grounding against a specific ship and expectation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 21:41:20 +00:00
a7905d0f6b re: kill the duplicate-resource-name hypothesis for the viewer bug, and reconcile 22 vs 16 stages
build_ship_model resolves each placement with find(|m| m.name == p.resource) --
first match wins -- so a repeated resource name inside a container would draw
the wrong geometry. It cannot happen: decoding every XBG7 resource in all 22
stage containers gives 4603 resources and ZERO repeated names. Per ship it is
tighter still: e106 wants 9 distinct names and decodes exactly 9 models for 11
placements, e105 9 for 9, f105 5 for 6.

Two of the three remaining candidates for the user-reported viewer bug are now
gone (this and include_external), leaving mirror handling and the exhaust cones,
plus the untried visual comparison.

The container sweep also reconciles a number I flagged earlier: the disc has 22
stage CONTAINERS -- Stage_S01..S16 plus Stage_S24..S29 -- i.e. 16 story stages
plus 6 extra. The save screen's 16 STAGE sprites and the notes' "22 stages" are
both right and count different things; neither gets edited to match the other.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 21:31:18 +00:00
671891f9bf re: XPR2 channel order confirmed against the running game -- the second "dynamic unknown" closed
INDEX carried XPR2 as "colours unverified (dynamic item)". The failure mode that
matters is a swapped red/blue channel: invisible on greys, invisible without
ground truth, and it poisons every asset silently.

Method: pick an asset that is strongly asymmetric in hue and has known in-game
appearance. DeltaSaber_T.xpr (1024x1024 Dxt1, 11 mips) decodes to white/grey
panels with orange-rust accents; the game renders the same hull in the ARSENAL
scene.

  decoded texture : 1073 warm px, 0 cool, median saturated hue 23.3 deg
  in-game render  :  916 warm px, 124 cool, median saturated hue  9.3 deg

Both orange/red; a red<->blue swap would put the texture at ~200 deg. The cool
pixels are the blue UI behind the model and the 14 deg offset is the hangar's
red key light -- neither can move a hue by 180.

Stated as a bounded result, not "colours are correct": a hue-family comparison
cannot see a gamma/sRGB error, premultiplied alpha, or a per-channel scale, so
exact fidelity stays open and needs a flat unlit known-value surface. Cubemap
face ordering untouched. BG_Earth.xpr (A8R8G8B8 cubemap face) decoding to a
coherent nebula is corroboration for the de-tile step, not proof.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 21:21:55 +00:00
38f58ea559 re: the movie/subtitle/voice link is solved statically -- INDEX called it a dynamic unknown
INDEX carried IXUD as "movie<->track link unknown (dynamic item)". It is a
config record on the disc: tables.pak schema 0x067025b9, whose keys match the
movie game-part symbols in the executable (MOVIE / TELOP / SUBTITLE /
VOICETRACK, plus SUBTITLE_FONT / SUBTITLE_Y / SUBTITLE_HEIGHT next to
silph::GamePart_Movie::Impl::OnPrepare).

101 movies mapped: 94 with a subtitle, 83 with a voice track, 21 with a telop.
Every subtitle reference was looked up in dat/movie/<lang>.pak and 93 of 94
resolve -- SUBTITLE_S12B.tbl resolves in NONE of the six languages, a dangling
reference on the retail disc rather than a decode failure.

Naming is SUBTITLE_<base>.tbl and VOICE_<base>, with the staff roll and five
supply movies (VOICE_D_450..454) as documented exceptions, so a reimplementation
can resolve by convention and fall back to the table for those six.

Deliberately NOT published: the record's ~104 script ids paired to movies.
Positional pairing drifts by three -- visible at the tail, where
S24/S27_SUPPLY_ACROPOLIS would have to map to hokyu_LS_s24A/s27A and do not --
because the IDXD pool dedupes repeated values and an id whose movie was already
named contributes no token. The CSV is keyed by movie file instead.

For the intro-movie work: ADV.wmv has VOICE_ADV and no subtitle, no telop.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 21:10:08 +00:00
bb9e75fe6a re: the title-menu screens read the auto-save (slot 01), and the high score is computed not stored
Two results, both from a safe probe.

Slot 03 was given SHAB[0] = (a=1, b=12345, c=60000) -- a one-minute best time --
and MISSION SELECT still showed 4612 P EASY / 05:24.77 EASY, i.e. slot 01's
numbers. So the title-menu screens read the AUTO-SAVE, and any probe written to
a throwaway slot is invisible there. The previous SHAB identification is
unaffected: it matched slot 01's own bytes, which is what the screen renders.

The consequence is a decision, not a task: mapping the difficulty enum (a) or
testing b needs slot 01 itself edited -- the only save with real progress. Left
undone deliberately and flagged in the doc; a full content backup exists and
slot 01's bytes are archived, so it is recoverable, but it is the user's call.

Also: High Score 4612 is stored NOWHERE. It is absent from both .gpd profile
files, all three saves (compressed and inflated) and xconfig.settings, while the
best time 324773 is present in slot 01 alone. The score is computed at display
time, which is why SHAB[0].b = 4101 does not equal it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 21:00:16 +00:00
df45a1986b re: SHAB is the per-stage record table -- record 0 is Stage 01, and c is its best time
EXTRAS -> MISSION SELECT renders per-stage records and they match the table.
Stage 01 shows "High Score 4612 P EASY / Best Time 05:24.77 EASY" while SHAB[0]
is (2, 4101, 324773, FILETIME): 324773 ms IS 5m24.773s, the displayed best time.
Sixteen records, sixteen story stages -> the table is indexed by stage, and
record 0 is Stage 01, the only stage this save has cleared.

  a = 2       difficulty of the record (both entries tagged EASY)
  b = 4101    NOT the displayed high score 4612; equals GHAD +24/+28
  c = 324773  best clear time in ms -- confirmed against the screen
  d,e         FILETIME

This also explains the probe that filled SHAB[1] and changed nothing: the table
holds stage RECORDS, not progress, and Stage 02 has never been cleared so its
record is legitimately empty. Promotes the earlier "per-stage result record"
guess to confirmed for the layout.

Route: title -> EXTRAS -> MISSION SELECT. Only cleared stages are selectable (16
d-pad presses never left Stage 01) and the screen has its own difficulty
selector (Y).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 20:50:06 +00:00
98b3cf56ac re: name the save read path, and locate the save state at game_object+304
Disassembling around the serializer names its mirror and the whole read side:
deserializer 0x822C0380 (reads GDAA + phase string), GHAD reader 0x822BF858
(called with save+8, pairs with the writer 0x822BF678), SHAB reader 0x822BFD60
(save+136, count 16).

The GHAD reader is a plain unrolled byte copy with no clamping or validation
anywhere, which is why hand-written field values survive into the object exactly
as written.

The deserializer's only caller builds its argument as
lwz r11,4(r30) / lwz r11,0(r11) / addi r4,r11,304, so the save state lives at
+304 of a game object and every field now has a fixed absolute offset there --
Points at +336, clear ratio at +320, the develop blob at +380, the SHAB table at
+440. That is what a live-RAM read of the save state needs.

Recorded as a negative too: searching those absolute offsets does NOT find the
panel's populate code. lwz rX,336(rY) has 193 matches, and the windows holding
several right offsets at once are a vtable thunk table (slots 308-340) and a
float-parameter block. The stage/difficulty fields need the screen-side path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 20:24:47 +00:00
bc42c0970e re: the save screen's key table is compiled into the executable -- a 4th difficulty, 3 states, 16 stages
Static, no emulator. The LOAD/SAVE screen's config keys sit in .rdata as a
pointer array at 0x820a0074, identified as config keys (not loose strings) by
the Arsenal's own key list sitting a few hundred bytes earlier and matching its
pak record exactly.

  - GAME_VERY_HARD exists alongside EASY/NORMAL/HARD, while the stage tables
    only carry BonusPoint_EASY/NORMAL/HARD -- a fourth, unpriced tier.
  - Game Status is a 3-valued enum: STATE_STAND_BY (our save's "At Standby"),
    STATE_STAGE_CLEAR, STATE_GAME_CLEAR.
  - Exactly 16 STAGE sprite keys, matching weapon.tbl's stage01..16 (plus 6
    tutorials and 6 challenges). So 16 is the story-stage count and the "22
    stages" figure used elsewhere counts a different set -- flagged, not
    silently changed.

Also recovered: the panel's field names and positions (Points, FlightTime
%03d:%02d:%02d, ClearTimes, CompletionRate, RatioOfClear, Index, Label, Date,
BrokenData).

It does NOT name the stage/difficulty save fields, and it explains why the probe
sweep could not: the screen picks a sprite key BY INDEX through a config lookup,
so there is no lis/ori immediate to xref back to the selecting code.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 20:13:01 +00:00
2da9b8a9b8 re: the whole GHAD block is neither stage nor difficulty -- sixteen probe saves
Extends the previous refutation to a sweep. Probed with no effect on STAGE 02 or
Difficulty EASY: every scalar in the GHAD block (+0, +12, +16, +20, +28, +32,
+36 at 1/3/9, +40 u64, +48, +52, +56, +60, +64 raw), SHAB[0].a, and the SHAB
FILL COUNT in both directions -- record 1 filled with a copy of record 0, and
record 0 cleared. The "stage = filled-record count + 1" idea dies with it, and
so does the reading that made SHAB a per-stage result table by that route.

The panel does re-read each slot: slot 02 holds Points 4101 / Clear Ratio 5 %
and displays exactly that while its neighbours show 101 / 6 %.

Left: the phase string, the trailer, or the blob. Recorded caveat -- every save
on disc is genuinely Stage 02 EASY, so "field not found" and "panel does not vary
those two labels per slot" are not yet separated, and another probe round cannot
separate them. The next move is static: find the code that formats STAGE %02d
and read which offset it loads.

savegame_edit.py --set now packs an int into raw_* byte fields.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 20:01:26 +00:00
eac3435cc7 re: +36/+52/+56 are neither stage nor difficulty -- refuted with eleven probe saves
Those three fields all hold 2, and "difficulty or stage, undecidable from one
save" has been the reading since the format was parsed. Writing saves makes it
decidable and the answer is neither.

boot_menu.sh boots to the title menu WITHOUT loading anything, and LOAD GAME's
slot list renders each slot's Details panel from that slot's payload. Extra
slots can be fabricated (copy the directory plus a gameNN.header with its
UTF-16BE display string and ASCII name patched), so four probes fit in one boot,
read-only.

Probed: +36 at 1/3/9, +52 and +56 at 1/9, and +0, +16, +32, +48, +28,
SHAB[0].a. Every one left the panel at STAGE 02 / EASY / At Standby /
Times Cleared 0.

The negative is meaningful because the panel does read each payload -- slot 02
shows 5% clear ratio against the others' 6%, and Points tracked +24 exactly.
Two controls: patching a slot header to "STAGE09 HARD" changed nothing (the
display is payload-driven, not header text), and the row date follows the
container FILETIME.

Remaining candidates: +12, +20, +40 (u64), +60, +64, or the phase string.

Also here: savegame_edit.py --slot for SHAB records, and boot_menu.sh itself --
nav_probe.sh's boot loads a save, which with probe slots on disc loads a probe,
and a dropped d-pad step there put A on TAKE OFF and spent a boot loading a
mission.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 19:37:06 +00:00
6e0fbe52a6 docs: retire the last three sentences that predate the solved index space
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 19:01:43 +00:00
841156a95d re: the develop blob's index space is SOLVED -- it follows strings.tbl, cut items and all
Three hand-written probe saves settled what a mission payout could not.

  all 54 owned  -> every tab fills and the lists WRAP, giving exact counts
                   (GUN 8, BEAM 8, LASER 4, MPM 5, ASM 5, B/R 7, CANNON 6,
                   SPECIAL 6 = 43 displayed) and confirming the display order is
                   weapon.tbl's order item-for-item. It also killed the idea
                   that gate-unmet owned flags are suppressed: Maelstrom Bomb
                   (Stage 6) and Grav Cannon XGS (Stage 12) both rendered at a
                   Stage-2 save.
  only 9 + 39   -> STILETTO BG1 Developed (control) AND TOMAHAWK ALPHA RAIL GUN
                   Developed. The CANNON block starts at 39, not 38.
  only 39 + 48  -> JAMMING SYSTEM Developed. SPECIAL starts at 48, and with six
                   SPECIAL items the array closes exactly at 53, nothing spare.

So the blob is indexed by strings.tbl's item order -- the display order PLUS the
items only the localisation file carries (Adhesive Mine B2A, and three of
Ballista GSH / Designator LH / Smoke Marker Launcher / Bumble UV). They never
appear in the Arsenal but they hold slots, which is the entire reason the tail
looked broken. It also explains the retail save's five "unexplained" owned
entries at 33 and 45/46/47: cut items, shipped flagged owned, never rendered.

That weapon.tbl's No_Equipment..Wep_83 run is ALSO 54 long is a coincidence, and
it cost real time -- the two orders agree only to index 32. Recorded as such.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 19:01:14 +00:00
dc97dc215d docs: reconcile the 'free items are granted' wording with the 0 P purchase the probe caught
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 18:10:58 +00:00
f5f95be0a1 re: saves can be written back, which settles what the develop blob's 4s mean and refutes the tail
The container's derived fields turned out to be reproducible -- length+10 at
+0x30, payload length at +0x8c, adler32(payload) at +0x8e, everything else
copied -- and savegame_edit.py re-wraps a real save BYTE-IDENTICALLY, which is
the check that those three are the only ones. A hand-written save then loaded.

That replaced a blocked experiment (the tail question needed a mission payout,
and none of the currently developable items even sit in the disputed range) with
a direct one: write the blob, read the Arsenal.

  - controls: 4 at index 9 -> STILETTO BG1 Developed, 21 -> FALCON 9AM
    Developed. A hand-written 4 reaches the screen.
  - tail: 4 at 33 and 45 left their rows dashed (both on screen, not below the
    fold), and 38 left TOMAHAWK ALPHA RAIL GUN at "0 P" -- not owned. So the
    tail is not the weapon.tbl order continued.
  - clearing the real save's {22,26,39,46,47} cost the Tomahawk its Developed
    status, which puts its flag in that set (39 positionally) -- but a uniform
    +1 fails for SPECIAL, so no shift is asserted. Indices >=32 stay marked.

Two behaviours fell out. The title RE-DERIVES developable state on load and
announces it ("You can now develop Broad Sword ..."), so only the 4s are stored
state and a written 2 is pointless. And a no-cost item is bought for 0 P rather
than granted -- TOMAHAWK at "0 P" is what unowned looks like -- which is the
actual reason items read Developed in a save where nothing was spent.

Slot 03 was restored from its archived original (md5 verified); slots 01/02 were
never touched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 18:10:28 +00:00
f88f4f6847 re: the Arsenal develop economy, and the save blob's index space (first 32 confirmed)
GP_HANGAR_ARSENAL.pak's screen config points at weapon.tbl (item ids, in the
8-category display order) and strings.tbl (names, descriptions, and a
"Conditions to obtain" block per item). The id run No_Equipment .. Wep_83 is
exactly 54 long -- the save blob's length -- and all 60 conditions blocks are
extracted to a CSV: gates are stage completion, a predecessor item, or an ace
kill; costs run 3000-350000 P, and 20 items cost nothing once gated (which is
why items the player never bought read as Developed).

Predicting the save state from those conditions -- before looking at the blob --
says exactly six items are developable here, and the blob's six 2s sit on those
six, in weapon.tbl order, at indices 1/5/10/12/27/31. With the four obtained
items and the differential's own two transitions that is twelve concordances
over indices 0-31, nothing fitted. Broad Sword SG1 at index 5 needed scrolling
the GUN list to see, which is the only one the first screenshots missed.

The tail is NOT settled and is marked so: the Tomahawk is weapon.tbl index 38
and the screen shows it Developed, but blob[38] = 0, and the other tail 4s
(33/39/45/46/47) land on items the Arsenal shows as locked -- the SPECIAL tab is
entirely empty. A +1 shift does not repair it either. Settling it needs a second
development in a late category, which needs a mission payout.

Also recorded: IDXD string pools dedupe repeated values, so only the first
record of a table can be read from the token stream -- record 2 shows just its
unique values, record 6 no cost at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 17:38:45 +00:00
6ebf2ba00a re: capture the Hangar screen — it is a loadout picker, not a craft data sheet
The Hangar lists NOSE / MAIN WEAPON 1-3 with the mounted weapon and a derived
'Gross Weight: Light'. It shows no craft stat fields at all, so the Hangar is
not a route to the unit definition's defaulted fields — the Arsenal DATA SHEET
(weapons only) and the live-RAM route remain the only two.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 17:19:21 +00:00
f0ceda1c6f re: developing one Arsenal weapon separates Points from its twin and names the save blob
A third save, taken in the running game after developing exactly one item
(Light Machine Gun MG I, 4000 P) and declining the mount prompt, moves exactly
three things against the same-state save:

  +24  Points      4101 -> 101   and +28 does NOT move, which separates the
                                 pair one save could not tell apart; the
                                 Details panel then reads Points 101 P, so
                                 +24 is the spendable balance
  +8   clear ratio 5 -> 6        so the ratio counts collection, not stages
  +68  blob[1],[2]  2->4, 0->2   the item bought, and the successor the game
                                 announced as newly developable

One action, two blob transitions, two on-screen events in the same order — which
is what makes 0 locked / 2 developable / 4 developed a reading rather than a
guess, and rules out a plain owned-bitmask (it could not hold the middle state).
Which item each of the 54 indices is stays open: the arsenal id lists in
GP_HANGAR_ARSENAL.pak union to 35, and MG I sits at index 1, not 0.

Also recorded: inside a modal yes/no dialog a 60 ms d-pad tap is ignored (the
list menus accept it), and the develop confirm starts on NO while the mount
prompt right after it starts on YES.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 17:16:57 +00:00
5cc995e864 docs/re: INDEX — reflect the named save fields and the pure-state result 2026-08-11 06:02:31 +00:00
882dbc2ae2 re: a second save names Points, flight time and clear ratio — and shows the payload is pure state
Made a second save in-game (READY ROOM -> SYSTEM -> SAVE GAME -> empty slot) so
the format had a differential to read against. nav_probe.sh drives it: boot to
the READY ROOM, walk a scripted step list, screenshot after every step and stamp
every save file's md5, so the trail says which keypress wrote a save. That stamp
is what caught the first attempt failing -- the save confirm starts on YES,
unlike the load confirm which starts on NO, so the load flow's extra up-press
selected NO and wrote nothing.

Result 1: saving the same loaded state into a new slot produces a BYTE-IDENTICAL
545-byte payload. Only the GDHA header moves, and every byte that moves is either
the container FILETIME or one of the guest-pointer words -- which empirically
confirms those words are uninitialised padding rather than data. So the payload
holds no timestamp, no slot number and no name; a save's identity is entirely in
its content header.

Result 2: the LOAD/SAVE screen's Details panel prints Points 4101 P, Flight Time
000:05:24 and Clear Ratio 5 % for exactly this state, which names GHAD +24
(Points), +4 (flight time in ms -- 324773 ms = 5m24.773s) and +8 (clear ratio %).
Difficulty EASY and STAGE 02 both being 2, and three fields holding 2, is left
undecided on purpose: one save cannot separate them.

Result 3: the 16 SHAB records are not the UI's save slots. The UI has 20, slots
are separate gameNN files, and record 0's FILETIME stayed at 2026-07-23 in a save
written on 2026-08-11 -- so the table is part of the game state, most likely a
per-stage result record (one stage finished, one record filled), which the next
cleared stage will falsify or confirm.

The original save was backed up first and is untouched; the new save went to an
empty slot.
2026-08-11 06:02:21 +00:00
398e8ae0af re: the save file is a GDHA/zlib chunk stream — layout read off the title's own serializer
The whole retail save is 545 bytes: a GDHA container wrapping a zlib payload,
which is a chunk stream — 'GDAA', a length-prefixed game-phase name (GP_BUNK,
one of the title's GP_* screen ids), a 'GHAD' 122-byte progress block, a
count-prefixed table of 16 20-byte save slots, and a trailer.

None of it is guessed. The layout comes from the serializer at 0x822C00E8 and
its callee 0x822BF678: the writer primitive is 0x821885A8(stream, buf, len), the
slot count 16 and the 20-byte stride are literals, and the GHAD block is ten u32,
a u64, four u32, a raw 4-byte field and a raw 54-byte blob = 122 bytes, which is
exactly what the file carries. savegame.py re-serializes the parse and asserts
byte-identity.

Because the struct is written field-by-field with no packing changes, a payload
offset is also the offset in the live save object -- save+8 for GHAD, save+136
for the slot table, and 136+16*20 = 456 is the serializer's very next access.

Confirmed independently: slot 0's trailing two u32 decode as a FILETIME to
2026-07-23 20:07:23 UTC, and the content header's display string (written by the
game) says 07/23/2026 21:08 at UTC+1; the 15 empty slots hold 2006-01-01 rather
than zero.

Field meanings are left  on purpose -- the naming oracle is a diff of two
saves, and all four copies on disk are byte-identical, so a second save has to
be made first. Nothing has been written back to any save.
2026-08-11 05:40:26 +00:00
5c3e3dfe47 re: making turrets targets does not save the escort — the effect is inside the noise
The Stage-02 outcome write-up ended by naming "turrets near the asset must
become targets" as the fix worth ~50% of the escort damage. That was an
inference from a co-presence attribution, not a measurement. Run it and it
does not hold.

Also corrects the run labelling: pilot.py gained the SYLPH_KILL_TURRETS gate
two minutes before mission02 started, so mission02 was already a treatment run,
not a second baseline. Only mission01 (0 of 3968 pilot frames targeting a
turret) is the baseline.

At a common t=428s: baseline 46.9% escort hull, treatment 44.5% and 53.0%. The
two runs of the same arm differ by more than either differs from the baseline,
and the escort still reaches zero at t~590-670s in all three. So the
transferable finding is the power limit: one 430s flight cannot resolve an
effect below ~9 percentage points, and every single-run pilot conclusion,
including this one's, is inside it.

What does reproduce: the assault is scripted (onset 166/167/166s), and the
e007/e010 damage split is 50/50 in all three arms including the one that never
fires at a turret -- so that attribution measures the wave script, not us.

Also records that the viewer's include_external hypothesis in BACKLOG is dead
(it defaults true and is threaded through unchanged).
2026-08-11 05:29:52 +00:00
f4d59c5783 re: Stage 02 is lost at ~11 minutes, and the pilot's survival rule is what guarantees it
First session whose deliverable was the mission's ENDING rather than a
measurement (mission_run.sh, 500 s, hull of every entity at 2 Hz). The ACROPOLIS
is untouched to t=170 s then falls at ~53 HP/s with no let-up, reaching zero at
t=640-720 s — so "no mission completed" is not an artifact of the 240 s
time-boxes, and not of the 600 s cap on a blocking tool call. A longer session
would only watch the loss arrive.

Attributing the damage by co-presence, exactly two classes are ever near the
asset: e007 turrets (8483) and e010 bombers (8334). pilot.py treats turrets as
keep-out zones at 2500 units and never as targets — the rule that made it
survive — so roughly half the escort damage comes from the one class it is
designed to avoid. Survival and the objective are in direct conflict and the
pilot resolves it entirely for survival: WARSHIPS 0000, WARPLANES 0009,
REMAINING OB rising 004 -> 008, our hull untouched at 1500/1500 with 120
missiles spent. That is unspent risk budget, not a good run.

Also corrects launch_mission.sh: a harness-tracked BACKGROUND task does not keep
the display alive (lost 11 s in, at the turn boundary) — the
one-blocking-foreground-call rule stands.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 20:57:43 +00:00
9bc8c2d694 viewer: preview reassembled UI screens in the PAK browser
When an opened GP_*.pak RATC entry is a .rat build, compose the screen
(ui_layout::compose_build) and show it atop the RATC detail, labelled
'UI screen', with the sprite children below. Reuses the existing pak-open +
T8aD-decode path — no new browser/threading. Entry summaries mark UI builds.

cargo check -p sylpheed-viewer: clean.
2026-07-29 20:33:29 +02:00
c1da907135 formats: ui_layout — reassemble UI screens from .rat records
Parse each RATC bundle's .rat layout records (sprite name @0x20, placement
block [scale,tint,X,Y]) and composite the .t32 sprites back into the screen
image. Validated: GP_PAUSE_MENU rebuilds pixel-accurately (btn X=226,
Y=268/337/407/478 — the documented 70px pitch); focus records land 42px
left/8px up. Compositor confirmed by rendering the pause menu from disc alone.

Format doc: docs/re/structures/ui-rat-layout.md (agent RE).
2026-07-29 20:27:35 +02:00
MechaCat02
8ed8c85f18 [WIP] Audio codec probe/detection + CLI audio-info + viewer audio panel
Snapshot of local WIP before rebasing onto the movie/voice remote work.
- audio.rs: AudioCodec enum, AudioInfo::probe (RIFF/WAVE parse, raw-XMA2
  Shannon-entropy heuristic looks_like_raw_xma2), from_wav, riff_data_span.
- cli/main.rs: `audio-info` subcommand (cmd_audio_info).
- viewer ui.rs/iso_loader.rs: draw_audio_detail panel wiring.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 20:42:48 +02:00
MechaCat02
3e1040b472 [texture] Present Canary GPUTEXTUREFORMAT names + metadata; handle XPR5
- Port xenia-canary's complete 64-entry texture-format table (xenos.h enum +
  texture_info_formats.inl) into GPU_FORMATS + gpu_format_name()/desc() and
  X360TextureFormat::gpu_name()/desc().
- Viewer + CLI `texture info` now show the canonical k_… name, bpp and
  compressed flag (e.g. "k_DXT1 (Dxt1) · 4 bpp, compressed"); UnsupportedFormat
  now names the format instead of a bare hex code.
- Detect non-XPR2 containers up front: the game ships one XPR5 file
  (Common.xpr) alongside 165 XPR2 — report UnsupportedContainer("XPR5")
  cleanly instead of a "bad magic" parse error.
- Add a table-integrity test (index==code, spot-checks, decodable-variant
  consistency).

Static-RE scan of the 166 resource3d XPRs: only k_DXT1 (145), k_8_8_8_8 (19),
k_DXT5A (1) are used — all already decoded, so this is presentation, not new
decoders.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 21:09:55 +02:00
MechaCat02
390c67cb61 [viewer] Surface recovered TOC names in the PAK browser + refresh RE panel
- Move the IDXD->TOC-path resolver into sylpheed-formats as
  IdxdObject::recover_toc_path() so the CLI `pak list` and the GUI pack
  browser share one implementation (behaviour-preserving; CLI still
  resolves 308/1004 on GP_MAIN_GAME_E).
- PAK browser now shows each entry's recovered original path (via the
  name-hash) in green, in both the entry list and the IDXD detail heading.
- Replace the stale welcome "RE Notes" table (mesh/audio "unknown") with an
  accurate reverse-engineered-formats status grid + correct authorship.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 20:42:40 +02:00
760 changed files with 155079 additions and 2976 deletions

View File

@@ -103,6 +103,120 @@ enum Commands {
#[command(subcommand)]
cmd: MeshCommands,
},
/// Audio tools (identify WAV/XMA/XMA2 + metadata)
Audio {
#[command(subcommand)]
cmd: AudioCommands,
},
/// UI screen tools — reassemble a screen from its RATC bundle
Screen {
#[command(subcommand)]
cmd: ScreenCommands,
},
/// Save file (`savedata`) tools
Save {
#[command(subcommand)]
cmd: SaveCommands,
},
}
#[derive(Subcommand)]
enum ScreenCommands {
/// List the screen builds in a UI pak, with their element counts
List {
/// Path to a `GP_*.pak`
pak: PathBuf,
/// Widen the list from screen builds to **every composable bundle** —
/// including the ones with no `.rat` layout child, such as the
/// developer-logo splash. 2 859 RATC bundles on the disc compose; only
/// 965 are screen builds, and the rest are mostly two-element fragments
/// (a button and its glow), so this is off by default. **It renumbers
/// `--build`**, which is why it is a flag and not the default.
#[arg(long)]
all: bool,
},
/// Print one build's element declaration table and resting placements
Info {
/// Path to a `GP_*.pak`
pak: PathBuf,
/// Which build (index into `screen list`); default = the largest
#[arg(long)]
build: Option<usize>,
/// Widen the list from screen builds to **every composable bundle** —
/// including the ones with no `.rat` layout child, such as the
/// developer-logo splash. 2 859 RATC bundles on the disc compose; only
/// 965 are screen builds, and the rest are mostly two-element fragments
/// (a button and its glow), so this is off by default. **It renumbers
/// `--build`**, which is why it is a flag and not the default.
#[arg(long)]
all: bool,
/// Also print the bundle's other orderings and, per element, the decoded
/// sprite size beside the declared pivot and every keyframe's
/// scale/position/time — what a placement or paint-order hypothesis has
/// to be tested against.
#[arg(long)]
geometry: bool,
},
/// Composite one build to a PNG — the headless self-verify for the viewer
Render {
/// Path to a `GP_*.pak`
pak: PathBuf,
/// Output PNG
output: PathBuf,
/// Which build (index into `screen list`); default = the largest
#[arg(long)]
build: Option<usize>,
/// Draw the focused-state (`*f`) records over their base elements
#[arg(long)]
focus: bool,
/// Draw `loop*` sprite animations
#[arg(long)]
animated: bool,
/// Start the canvas black instead of the default dim slate — what the
/// game composites over on a screen carrying its own background, and so
/// what a framebuffer capture must be compared against.
#[arg(long)]
black: bool,
/// Draw the untextured `.prm` primitives (fade / dim / flash quads).
/// Off by default: they are decoded, but where they paint on a screen
/// without a measured order is unsolved — see
/// `docs/re/structures/ui-prm-primitives.md`.
#[arg(long)]
primitives: bool,
/// Widen the list from screen builds to **every composable bundle** —
/// including the ones with no `.rat` layout child, such as the
/// developer-logo splash. 2 859 RATC bundles on the disc compose; only
/// 965 are screen builds, and the rest are mostly two-element fragments
/// (a button and its glow), so this is off by default. **It renumbers
/// `--build`**, which is why it is a flag and not the default.
#[arg(long)]
all: bool,
},
}
#[derive(Subcommand)]
enum SaveCommands {
/// Parse a `savedata` file and print every field with its confidence
Info {
/// Path to a `savedata` file
file: PathBuf,
/// Also print the still-unidentified fields
#[arg(long)]
all: bool,
},
}
#[derive(Subcommand)]
enum AudioCommands {
/// Identify an audio file/stream and print its metadata
Info {
/// Path to a WAV / XMA / raw audio stream
file: PathBuf,
},
}
#[derive(Subcommand)]
@@ -219,9 +333,433 @@ async fn main() -> Result<()> {
cmd_pak_textures(&pak, &output, verbose)
}
},
Commands::Audio { cmd } => match cmd {
AudioCommands::Info { file } => cmd_audio_info(&file),
},
Commands::Screen { cmd } => match cmd {
ScreenCommands::List { pak, all } => cmd_screen_list(&pak, all),
ScreenCommands::Info {
pak,
build,
geometry,
all,
} => cmd_screen_info(&pak, build, geometry, all),
ScreenCommands::Render {
pak,
output,
build,
focus,
animated,
black,
all,
primitives,
} => cmd_screen_render(
&pak, &output, build, focus, animated, black, all, primitives,
),
},
Commands::Save { cmd } => match cmd {
SaveCommands::Info { file, all } => cmd_save_info(&file, all),
},
}
}
// ── UI screens ───────────────────────────────────────────────────────────────
/// Every RATC entry of a UI pak that parses as a screen build, with its bytes.
///
/// `all` widens the filter from `is_build` — which requires a `.rat` layout
/// child — to anything `compose` can draw. The developer-logo splash is the
/// reason: it declares its sprites directly, has no `.rat` child at all, and so
/// is invisible to every screen command without this. See
/// `docs/re/structures/ui-composable-bundles.md`.
fn screen_builds(pak: &Path, all: bool) -> Result<Vec<(usize, Vec<u8>)>> {
use sylpheed_formats::{pak::PakArchive, ui_layout};
let ar = PakArchive::open(pak).context("open pak")?;
let mut out = Vec::new();
for (i, e) in ar.entries().iter().enumerate() {
let Ok(bytes) = ar.read(e) else { continue };
let keep = if all {
ui_layout::is_composable(&bytes)
} else {
ui_layout::is_build(&bytes)
};
if keep {
out.push((i, bytes));
}
}
Ok(out)
}
/// Resolve `--build`: an explicit index into the build list, else the largest
/// build (a screen pak's biggest bundle is the full screen; the small ones are
/// language or context variants).
fn pick_build(builds: &[(usize, Vec<u8>)], want: Option<usize>) -> Result<usize> {
if builds.is_empty() {
anyhow::bail!("no screen builds in this pak");
}
match want {
Some(i) if i < builds.len() => Ok(i),
Some(i) => anyhow::bail!("build {i} out of range (0..{})", builds.len()),
None => Ok(builds
.iter()
.enumerate()
.max_by_key(|(_, (_, b))| b.len())
.map(|(i, _)| i)
.unwrap()),
}
}
fn cmd_screen_list(pak: &Path, all: bool) -> Result<()> {
use sylpheed_formats::ui_layout;
let builds = screen_builds(pak, all)?;
println!("{} screen build(s) in {}", builds.len(), pak.display());
for (i, (entry, bytes)) in builds.iter().enumerate() {
match ui_layout::parse_build(bytes) {
Some(b) => println!(
" [{i}] entry {entry:<3} {:>8} B {}x{} {} elements, {} sprites{}{}",
bytes.len(),
b.design_w,
b.design_h,
b.elements.len(),
b.sprites.len(),
b.context_hint
.as_deref()
.map(|c| format!(" context={c}"))
.unwrap_or_default(),
if b.from_fallback { " (fallback)" } else { "" },
),
None => println!(" [{i}] entry {entry:<3} {:>8} B (unparsed)", bytes.len()),
}
}
Ok(())
}
fn cmd_screen_info(pak: &Path, want: Option<usize>, geometry: bool, all: bool) -> Result<()> {
use sylpheed_formats::ui_layout;
let builds = screen_builds(pak, all)?;
let idx = pick_build(&builds, want)?;
let bytes = &builds[idx].1;
let b = ui_layout::parse_build(bytes).context("build did not parse")?;
println!(
"build [{idx}] {}x{} {} elements {} sprites{}",
b.design_w,
b.design_h,
b.elements.len(),
b.sprites.len(),
if b.from_fallback {
" (recovered from .rat records — the declaration table was unusable)"
} else {
""
}
);
println!(
"{:<3} {:<30} {:>7} {:>8} {:>12} {:>4} rest / keyframes",
"#", "element", "parent", "kind", "pivot", "kf"
);
for el in &b.elements {
// The resting pose is the max-dwell keyframe, not the first or the last.
let rest = match el.rest() {
None => "".to_string(),
Some(k) if el.keyframes.len() == 1 => format!("({},{})", k.x, k.y),
Some(k) => {
// The final frame of a group carries no time — print it as `-`
// rather than inventing one.
let t = |t: Option<u32>| t.map(|v| v.to_string()).unwrap_or_else(|| "-".into());
format!(
"rest ({},{}) t={} [{}]",
k.x,
k.y,
t(k.time),
el.keyframes
.iter()
.map(|f| format!("{}:{},{}", t(f.time), f.x, f.y))
.collect::<Vec<_>>()
.join(" ")
)
}
};
println!(
"{:<3} {:<30} {:>7} {:>8} {:>12} {:>4} {rest}",
el.index,
el.name,
el.parent.map(|p| p.to_string()).unwrap_or_else(|| "-".into()),
format!("{:#x}", el.kind),
format!("({},{})", el.pivot_x, el.pivot_y),
el.keyframes.len(),
);
if let Some(link) = &el.focus_link {
println!("{:<3} {:<30} → focus {link}", "", "");
}
}
if geometry {
print_geometry(&b, bytes);
}
Ok(())
}
/// The two other orderings the bundle carries, and each element's real drawn
/// size. Both were needed to settle the title screen's paint order — see
/// `docs/re/BACKLOG.md`.
fn print_geometry(b: &sylpheed_formats::ui_layout::UiBuild, bytes: &[u8]) {
println!();
let identity: Vec<usize> = (0..b.placement_order.len()).collect();
println!(
"placement-region group order: {:?}{}",
b.placement_order,
if b.placement_order == identity {
" (== declaration order)"
} else {
" (DIFFERS from declaration order)"
}
);
if let Some(kids) = sylpheed_formats::ratc::parse(bytes) {
println!(
"RATC child order: {:?}",
kids.iter()
.map(|c| format!("{}:{}", c.kind, c.name))
.collect::<Vec<_>>()
);
}
println!();
println!("geometry — decoded sprite size vs the declared pivot, and every keyframe");
println!(
"{:<3} {:<26} {:>11} {:>11} {:>5} keyframes t: x,y sx%,sy% a=alpha r=rot°",
"#", "sprite", "decoded", "pivot*2", "same"
);
for el in &b.elements {
let dims = el
.sprite
.as_ref()
.and_then(|s| b.sprites.get(s))
.and_then(|&(off, size)| sylpheed_formats::t8ad::parse(&bytes[off..off + size]))
.map(|img| (img.width, img.height));
let pv = (el.pivot_x * 2, el.pivot_y * 2);
let same = match dims {
Some(d) if d == pv => "yes",
Some(_) => "NO",
None => "-",
};
let kfs = el
.keyframes
.iter()
.map(|f| {
let rot = if f.rotation_deg != 0 {
format!(" r={}", f.rotation_deg)
} else {
String::new()
};
format!(
"{}: {},{} {}%,{}% a={}{rot}",
f.time.map(|v| v.to_string()).unwrap_or_else(|| "-".into()),
f.x,
f.y,
f.scale_x,
f.scale_y,
(f.fade >> 24) & 0xff
)
})
.collect::<Vec<_>>()
.join(" ");
println!(
"{:<3} {:<26} {:>11} {:>11} {:>5} {kfs}",
el.index,
el.sprite.as_deref().unwrap_or(""),
dims.map(|(w, h)| format!("{w}x{h}"))
.unwrap_or_else(|| "".into()),
format!("{}x{}", pv.0, pv.1),
same,
);
}
}
fn cmd_screen_render(
pak: &Path,
output: &Path,
want: Option<usize>,
focus: bool,
animated: bool,
black: bool,
all: bool,
primitives: bool,
) -> Result<()> {
use sylpheed_formats::ui_layout::{self, ComposeOptions};
let builds = screen_builds(pak, all)?;
let idx = pick_build(&builds, want)?;
let bytes = &builds[idx].1;
let b = ui_layout::parse_build(bytes).context("build did not parse")?;
let screen = ui_layout::compose(
&b,
bytes,
ComposeOptions {
include_focus: focus,
include_animated: animated,
backdrop: if black {
[0, 0, 0, 255]
} else {
ComposeOptions::default().backdrop
},
include_primitives: primitives,
},
None,
);
image::save_buffer(
output,
&screen.rgba,
screen.width,
screen.height,
image::ExtendedColorType::Rgba8,
)
.context("write PNG")?;
println!(
"build [{idx}]: drew {}/{} elements → {} ({}x{})",
screen.drawn.len(),
b.elements.len(),
output.display(),
screen.width,
screen.height
);
if !screen.missing.is_empty() {
println!(" sprites that did not resolve/decode: {:?}", screen.missing);
}
let undrawn: Vec<&str> = b
.elements
.iter()
.filter(|e| !screen.drawn.contains(&e.index))
.map(|e| e.name.as_str())
.collect();
if !undrawn.is_empty() {
println!(" not drawn ({}): {undrawn:?}", undrawn.len());
}
Ok(())
}
// ── save file ────────────────────────────────────────────────────────────────
fn cmd_save_info(file: &Path, all: bool) -> Result<()> {
use sylpheed_formats::savegame::{
self, Confidence, DevelopState, FieldKind, GHAD_LAYOUT,
};
let raw = std::fs::read(file).context("read save")?;
let save = savegame::parse(&raw).map_err(|e| anyhow::anyhow!("{e}"))?;
let mark = |c: Confidence| match c {
Confidence::Confirmed => "OK ",
Confidence::Probable => "~ ",
Confidence::Unknown => "? ",
Confidence::Refuted => "REF",
};
println!(
"container : GDHA, {} B header + {} B deflate → {} B payload",
save.header.bytes.len(),
raw.len() - save.header.bytes.len(),
save.payload.len()
);
println!(
"round-trip: {}",
if save.round_trips() {
"byte-identical"
} else {
"MISMATCH — the parse is wrong"
}
);
println!("phase : {}", save.phase);
println!("\nGHAD progress block:");
for f in GHAD_LAYOUT {
if !all && f.confidence == Confidence::Unknown && f.name.is_empty() {
continue;
}
let value = match f.kind {
FieldKind::Millis => save
.ghad_value(f)
.map(|v| format!("{v} ms ({})", savegame::fmt_millis(v as u32))),
FieldKind::Percent => save.ghad_value(f).map(|v| format!("{v} %")),
FieldKind::Raw | FieldKind::DevelopBlob => Some(
save.ghad_bytes(f)
.iter()
.map(|b| format!("{b:02x}"))
.collect::<Vec<_>>()
.join(" "),
),
_ => save.ghad_value(f).map(|v| format!("{v}")),
}
.unwrap_or_else(|| "-".into());
println!(
" {} +{:<3} {:<14} {}",
mark(f.confidence),
f.offset,
if f.name.is_empty() { "(unnamed)" } else { f.name },
value
);
if all && !f.note.is_empty() {
println!(" {}", f.note);
}
}
let dev = save.develop_state();
let owned = dev.iter().filter(|d| **d == DevelopState::Developed).count();
let ready = dev
.iter()
.filter(|d| **d == DevelopState::Developable)
.count();
println!(
"\nArsenal : {owned} developed, {ready} developable, {} locked (of {})",
dev.len() - owned - ready,
dev.len()
);
println!("\nper-stage records (SHAB — NOT the UI's save slots):");
for (i, r) in save.records.iter().enumerate() {
if !r.is_used() {
continue;
}
println!(
" stage {:02} difficulty~{} points?{} best {} ",
i + 1,
r.a,
r.b,
savegame::fmt_millis(r.best_time_ms)
);
}
println!("\nheader summary (what the in-game Details panel reads):");
for m in save.header.summary() {
println!(" +{:#04x} {:<14} {}", m.header_offset, m.name, m.value);
}
println!(" — a payload edit that leaves these stale shows no change on the panel,");
println!(" which is not evidence that the payload field was the wrong one.");
Ok(())
}
// ── audio info ───────────────────────────────────────────────────────────────
fn cmd_audio_info(file: &Path) -> Result<()> {
use sylpheed_formats::AudioInfo;
let bytes = std::fs::read(file).with_context(|| format!("Cannot read {}", file.display()))?;
let info = AudioInfo::probe(&bytes);
println!("{} {}", "Audio:".green().bold(), file.display());
println!(" Codec : {}", info.codec.label().yellow());
let opt = |v: Option<String>| v.unwrap_or_else(|| "".dimmed().to_string());
println!(" Channels : {}", opt(info.channels.map(|c| c.to_string())));
println!(" Sample rate: {}", opt(info.sample_rate.map(|r| format!("{r} Hz"))));
println!(" Bit depth : {}", opt(info.bits_per_sample.map(|b| format!("{b}-bit"))));
if let Some(d) = info.duration_secs {
println!(" Duration : {d:.2} s");
}
if let Some(p) = info.xma_packets {
println!(" XMA packets: {} (2048 B each)", p.to_string().yellow());
}
println!(" Size : {} bytes", info.size_bytes.to_string().yellow());
if info.codec.needs_decoder() {
println!(
" {} decode not supported (needs an XMA2 decoder + the sound-bank descriptor)",
"note:".dimmed()
);
}
Ok(())
}
// ── extract ────────────────────────────────────────────────────────────────
async fn cmd_extract(iso_path: &Path, output_dir: &Path) -> Result<()> {
@@ -376,9 +914,16 @@ fn cmd_texture_info(file: &Path) -> Result<()> {
let tex = X360Texture::from_xpr2(&bytes)
.with_context(|| format!("Failed to parse texture: {}", file.display()))?;
let d = tex.format.desc();
println!("{} {}", "Texture:".green().bold(), file.display());
println!(" Resolution : {}×{}", tex.width.to_string().yellow(), tex.height.to_string().yellow());
println!(" Format : {:?}", tex.format);
println!(
" Format : {} ({:?}) · {} bpp, {}",
tex.format.gpu_name().yellow(),
tex.format,
d.bpp,
if d.compressed { "compressed" } else { "uncompressed" },
);
println!(" Mip levels : {}", tex.mip_levels);
println!(" Data size : {} bytes", tex.data.len().to_string().yellow());
@@ -913,24 +1458,6 @@ fn decode_to_rgba8(tex: &sylpheed_formats::texture::X360Texture) -> Result<Vec<u
use sylpheed_formats::pak::inner_format_label as inner_label;
/// Try to recover an IDXD entry's original TOC path from its identity tokens.
/// Uses the entry's ID/Name/Model fields plus identifier-like pool tokens as
/// candidates for [`sylpheed_formats::hash::recover_toc_name`].
fn idxd_toc_name(obj: &IdxdObject, name_hash: u32) -> Option<String> {
let mut cands: Vec<&str> = Vec::new();
for key in ["ID", "Name", "Model"] {
if let Some(v) = obj.get_raw(key) {
cands.push(v);
}
}
for t in obj.tokens() {
if t.contains('_') || t.len() >= 5 {
cands.push(t.as_str());
}
}
sylpheed_formats::hash::recover_toc_name(name_hash, &cands)
}
fn cmd_pak_list(pak: &Path, idxd_only: bool) -> Result<()> {
let arc = PakArchive::open(pak).with_context(|| format!("opening {}", pak.display()))?;
println!(
@@ -958,7 +1485,7 @@ fn cmd_pak_list(pak: &Path, idxd_only: bool) -> Result<()> {
}
let (detail, name) = if is_idxd {
match IdxdObject::parse(&payload) {
Ok(o) => (o.identity(), idxd_toc_name(&o, e.name_hash)),
Ok(o) => (o.identity(), o.recover_toc_path(e.name_hash)),
Err(_) => (String::new(), None),
}
} else {

View File

@@ -12,7 +12,12 @@
ship e106 ref=e106_bdy_04
e106_bdy_01 1 0 0 0 1 0 0 0 1 -263.9948 -150.43242 1164.8667
e106_bdy_02 -1 0 0 0 1 0 0 0 1 264.0088 -150.41339 1164.8651
# 2026-08-12: this rotation was diag(-1,1,1) - the X-flip `correlate` bakes in
# when a draw's buffer is the mirror of the decoded one. With distinct anchor
# assignment the twins now decode to their OWN buffers (the container carries
# both halves), so the mirror lives in the data and the placement is plain
# identity. Re-emitting from the capture agrees: `1 0 0 0 1 0 0 0 1 264.04343`.
e106_bdy_02 1 0 0 0 1 0 0 0 1 264.0088 -150.41339 1164.8651
e106_bdy_03 1 0 0 0 1 0 0 0 1 0.0029247368 -34.517372 1075.878
e106_bdy_04 1 0 0 0 1 0 0 0 1 0 0 0
e106_brg_01 1 0 0 0 1 0 0 0 1 -0.005471501 153.31795 -164.03748

View File

@@ -0,0 +1,177 @@
# Unit / vessel definition object layout — offset, field, kind.
#
# Read out of the title's own loader, not inferred: `sub_82341A20` builds every
# key as `addi r4, r30, -N` with `r30 = 0x82088f94`, so the field NAME for each
# store is a string in the executable image; pairing each key with the first
# store after its accessor call gives the offset. The accessor for floats is
# `sub_822FC5A8`, which returns 0.0 when the pool has no value for the key.
#
# Verified against live objects dumped from a running Stage 02: 406 field values
# agree with the disc records and 0 disagree, over 11 objects spanning both the
# UNIT and VESSEL schemas (see docs/re/live-unit-definitions.md).
#
# Two conventions matter for a reimplementation:
# * angle fields are DEGREES on disc and RADIANS in this object;
# * a field the disc record omits is left at the accessor's miss value (0.0 for
# floats) unless something later derives it — `Size_Y` takes `Size_X`.
#
# offset kind field
24 str Model
28 str CollisionModel
48 f32 Size_X
52 f32 Size_Y
56 f32 Size_Z
64 f32 Color_R
68 f32 Color_G
72 f32 Color_B
80 f32 Size_Radius
84 f32 HP
88 f32 HQRatio
92 f32 ShieldRatio
96 f32 ThrusterRatio
100 word IsDestructible
104 str NamePlate
112 word MountedShieldGenerator
116 f32 ResistanceToOptics
120 f32 ResistanceToShell
124 f32 ResistanceToExplosion
128 f32 ResistanceToPlayer
132 f32 ResistanceParalyze
136 str HUDMarkerID
156 f32 MinimumVelocity
160 f32 MaximumVelocity
164 f32 CruisingVelocity
168 f32 Acceleration
172 f32 Deceleration
176 f32 AV_PitchPlus_Max
180 f32 AV_PitchPlus_Min
184 f32 AA_PitchPlus_Max
188 f32 AA_PitchPlus_Min
192 f32 AV_PitchMinus_Max
196 f32 AV_PitchMinus_Min
200 f32 AA_PitchMinus_Max
204 f32 AA_PitchMinus_Min
208 f32 AV_Yaw_Max
212 f32 AV_Yaw_Min
216 f32 AA_Yaw_Max
220 f32 AA_Yaw_Min
224 f32 AV_Roll_Max
228 f32 AV_Roll_Min
232 f32 AA_Roll_Max
236 f32 AA_Roll_Min
248 f32 SideThrustVelocity_Max
252 f32 SideThrustAcceleration
256 f32 MaximumBank_Normal
260 f32 YawDragFactor
264 f32 PitchDragFactor
268 f32 RollDragFactor
272 f32 DragFactorThreshold
276 f32 ArterBurner_Vc
280 f32 ReverseThrust_Vc
284 f32 ArterBurner_Acc
288 f32 ReverseThrust_Acc
292 f32 AccPitchFactor
296 f32 DecPitchFactor
300 f32 AV_AxisMode_Max
304 f32 AV_AxisMode_Min
308 f32 AA_AxisMode_Max
312 f32 AA_AxisMode_Min
316 f32 PowerCutConsumeShield
320 f32 PowerCutDeceleration
324 f32 AB_ConsumeShield_Begin
328 f32 AB_ConsumeShield
332 f32 AB_AV_PitchPlus
336 f32 AB_AA_PitchPlus
340 f32 AB_AV_PitchMinus
344 f32 AB_AA_PitchMinus
348 f32 AB_AV_Yaw
352 f32 AB_AA_Yaw
356 f32 AB_AV_Roll
360 f32 AB_AA_Roll
364 word SideRoll
368 f32 SideRoll_Time
372 f32 SideRoll_Length
376 word BarrelRoll
380 f32 BarrelRoll_CountMinimum
384 f32 BarrelRoll_CountMaximum
388 f32 BarrelRoll_Time
392 f32 BarrelRoll_Radius
396 word TurnAttack
400 f32 TurnAttack_CutoffRatio
404 f32 TurnAttack_DoubleRatio
408 f32 CutoffTimeMin
412 f32 CutoffTimeMax
416 f32 TurnAttack_DoubleTimeMin
420 f32 TurnAttack_DoubleTimeMax
424 word TurnAway
428 f32 Turn_AngularVelocity
432 f32 TurnAway_Time_Minimum
436 f32 TurnAway_Time_Maximum
440 word BoostAway
444 f32 BoostAway_Time_Minimum
448 f32 BoostAway_Time_Maximum
452 word HoldPosition
456 f32 HoldPosition_LengthMin
460 f32 HoldPosition_LengthMax
464 f32 HoldPosition_MinimumTime
468 f32 HoldPosition_MaximumTime
472 f32 HoldPosition_SideRatio
476 f32 HoldPosition_BackRatio
480 f32 HoldPosition_CutoffRatio
484 f32 HoldPosition_CancelTime
488 word Slalom
492 f32 Slalom_CutoffRatio
496 f32 Slalom_TurnCount_Min
500 f32 Slalom_TurnCount_Max
504 word Through
508 f32 Through_CutoffRatio
512 f32 Through_AngleMinimum
516 f32 Through_AngleMaximum
520 f32 Through_Time1Max
524 f32 Through_Time1Min
528 f32 Through_Time2Max
532 f32 Through_Time2Min
536 f32 Through_LengthMinimum
540 f32 Through_LengthMaximum
544 word SolidCutoff
548 f32 SolidCutoff_Ratio
552 f32 SolidCutoff_LengthMin
556 f32 SolidCutoff_LengthMax
560 f32 HomingResistAdjustment
564 f32 UsingChaffRatio
568 f32 MaxValue
572 f32 ChargeDelay
576 f32 ChargeDelay_Break
580 f32 ChargeSpeed
584 f32 Delay
588 f32 DelayAdjustment
596 str LowerHPFxModel
604 str ExplosionFxModel
612 str DestroyMotionName
620 str ExplosionMotionName
624 f32 DestroyMotionTime
628 f32 DryMass
632 f32 GrossMass
636 word Thruster
640 word SideThruster
644 word ExplosionSE
648 word JumpIn
652 word JumpOut
656 word ShipEnvironmentSE
660 word LowerHPSE
664 f32 LowerHPThresholdRatio
668 f32 SELength
672 f32 RadarRange
676 f32 FCSRange
680 f32 FiringRange
684 word MountedFCS
692 f32 AttackVesselPoint
696 f32 AttackCraftPoint
700 f32 DefencePoint
736 str ShieldRecoverEffectName
740 str ShieldHitEffectName
744 str JumpIn
748 str JumpOut
752 str FadeIn
756 str FadeOut
760 str Effect_Paralyze

View File

@@ -0,0 +1,55 @@
//! RE probe: the `ACHIEVEMENTS_REQUIREMENTS` config list.
//!
//! `GamePart_Debriefing` (`0x8218CF38`..`0x82191B18`) walks this list after a
//! mission (`sub_8218F9A8`): for each entry it takes the entry's **index** `n`,
//! tests bit `n` of an awarded-mask, and if the bit is clear it evaluates the
//! entry (`0x8218FAB0`) and sets the bit when satisfied. `GamePart_ChallengeMission`
//! then gates each challenge mission on a bit of the same space via its own
//! `REQUIREMENT` key. So this list *is* the achievement/bit numbering.
//!
//! Run: cargo run --release -p sylpheed-formats --example achievements_map -- <disc-root>
use sylpheed_formats::{idxd::IdxdObject, pak::PakArchive};
fn find(h: &[u8], n: &[u8]) -> bool {
h.windows(n.len()).any(|w| w == n)
}
fn main() {
let disc = std::env::args().nth(1).unwrap_or_else(|| {
std::env::var("SYLPHEED_DISC").expect("pass disc root or set SYLPHEED_DISC")
});
let dat = format!("{disc}/dat");
let mut paks: Vec<_> = std::fs::read_dir(&dat)
.expect("dat dir")
.filter_map(|e| e.ok())
.map(|e| e.path())
.filter(|p| p.extension().is_some_and(|x| x == "pak"))
.collect();
paks.sort();
for p in &paks {
let Ok(arc) = PakArchive::open(p) else { continue };
for (i, e) in arc.entries().iter().enumerate() {
let Ok(b) = arc.read(e) else { continue };
if !find(&b, b"ACHIEVEMENTS_REQUIREMENTS") {
continue;
}
let name = p.file_name().unwrap().to_string_lossy().to_string();
match IdxdObject::parse(&b) {
Ok(o) => {
let t = o.tokens();
println!(
"\n===== {name} entry #{i} schema {:08x} {} tokens =====",
o.schema_hash,
t.len()
);
for (j, tok) in t.iter().enumerate() {
println!(" {j:3} {tok}");
}
}
Err(err) => println!("\n===== {name} entry #{i}: not IDXD ({err}) ====="),
}
}
}
}

View File

@@ -0,0 +1,22 @@
//! Time repeated per-ship decodes of one container, as the viewer does.
use std::collections::HashSet;
use std::time::Instant;
use sylpheed_formats::mesh::Xbg7Model;
use sylpheed_formats::ship::{is_base_part, ship_id_of};
fn main() {
let a: Vec<String> = std::env::args().collect();
let bytes = std::fs::read(&a[1]).unwrap();
let names = sylpheed_formats::mesh::xbg7_resource_names(&bytes);
let ids: Vec<String> = {
let mut v: Vec<String> = names.iter().filter(|n| is_base_part(n))
.filter_map(|n| ship_id_of(n).map(|s| s.to_string())).collect();
v.sort(); v.dedup(); v.truncate(5); v
};
for id in &ids {
let want: HashSet<String> = names.iter()
.filter(|n| ship_id_of(n) == Some(id.as_str())).cloned().collect();
let t = Instant::now();
let got = Xbg7Model::models_named(&bytes, &want, &|| false);
println!("{id}: {} models in {:?}", got.len(), t.elapsed());
}
}

View File

@@ -0,0 +1,123 @@
//! Does a resource have a CLEANER home in its container than the one we picked?
//!
//! After the pad-scoring fix, only two resources on the disc still decode to an
//! index run with degenerate triangles — `e201_bdy_03_m` (2 containers) and
//! `_rou_f402_dead` (9). Degeneracy says the run does not fit the pool, so either
//! the vertex block is wrong or the candidate list never offered the right one.
//! This walks every candidate vertex-run start for the resource's declaration and
//! scores each `(start, pad)` the way the anchor now does — degenerate triangles
//! first, then winding, plus coverage — so the answer is one of:
//! * a strictly cleaner candidate exists (the selection is at fault),
//! * several are equally clean (genuinely ambiguous), or
//! * nothing is clean (the block is not in the candidate list at all).
//!
//! Usage: better_home <container.xpr> <resource-name>
use sylpheed_formats::mesh::{debug_resource_params, debug_vertex_run_starts, Xbg7Model};
fn be16(b: &[u8], at: usize) -> u32 {
((b[at] as u32) << 8) | b[at + 1] as u32
}
fn main() {
let a: Vec<String> = std::env::args().collect();
let bytes = std::fs::read(&a[1]).expect("container");
let name = &a[2];
let Some((markers, stride)) = debug_resource_params(&bytes, name) else {
eprintln!("no such XBG7 resource: {name}");
std::process::exit(1);
};
let (vc, ic) = markers[0];
println!("{name}: {} marker(s), first = {vc} verts / {ic} indices, stride {stride}", markers.len());
// Where did the decoder put it?
let ours = Xbg7Model::stage_models(&bytes)
.into_iter()
.find(|m| m.name == *name)
.and_then(|m| m.meshes.first().and_then(|s| s.vbuf_offset));
println!("our anchor: {ours:?}");
let starts = debug_vertex_run_starts(&bytes, stride);
println!("{} candidate vertex-run starts for stride {stride}", starts.len());
// Score every (start, pad): degenerate triangles, winding against the stored
// normals, and whether the run covers the pool exactly.
let mut rows: Vec<(usize, f32, usize, usize, usize, bool)> = Vec::new(); // degen, wind, start, pad, max_idx, covered
for &vb in &starts {
for pad in 0..=3usize {
if vb < ic * 2 + pad {
continue;
}
let ib = vb - ic * 2 - pad;
if ib + ic * 2 > bytes.len() || vb + vc * stride > bytes.len() {
continue;
}
let idx: Vec<u32> = (0..ic).map(|k| be16(&bytes, ib + k * 2)).collect();
let max_idx = *idx.iter().max().unwrap_or(&0) as usize;
if max_idx >= vc {
continue; // out of range — not a candidate at all
}
let pos: Vec<[f32; 3]> = (0..vc)
.map(|v| {
let at = vb + v * stride;
[
f32::from_be_bytes(bytes[at..at + 4].try_into().unwrap()),
f32::from_be_bytes(bytes[at + 4..at + 8].try_into().unwrap()),
f32::from_be_bytes(bytes[at + 8..at + 12].try_into().unwrap()),
]
})
.collect();
if pos.iter().any(|p| p.iter().any(|c| !c.is_finite() || c.abs() > 1e6)) {
continue;
}
let mut degen = 0usize;
let (mut agree, mut counted) = (0usize, 0usize);
for t in idx.chunks_exact(3) {
let (x, y, z) = (t[0] as usize, t[1] as usize, t[2] as usize);
if x == y || y == z || x == z {
degen += 1;
continue;
}
// Winding needs normals; use the geometric centroid normal as a
// stand-in so this stays declaration-agnostic: a consistent mesh
// has all faces pointing away from the centroid on a convex-ish
// hull. Weak, so degeneracy leads the sort.
let e1 = [pos[y][0] - pos[x][0], pos[y][1] - pos[x][1], pos[y][2] - pos[x][2]];
let e2 = [pos[z][0] - pos[x][0], pos[z][1] - pos[x][1], pos[z][2] - pos[x][2]];
let f = [
e1[1] * e2[2] - e1[2] * e2[1],
e1[2] * e2[0] - e1[0] * e2[2],
e1[0] * e2[1] - e1[1] * e2[0],
];
let cx: [f32; 3] = {
let mut c = [0.0f32; 3];
for p in &pos {
for k in 0..3 {
c[k] += p[k] / pos.len() as f32;
}
}
c
};
let out = [pos[x][0] - cx[0], pos[x][1] - cx[1], pos[x][2] - cx[2]];
counted += 1;
if f[0] * out[0] + f[1] * out[1] + f[2] * out[2] > 0.0 {
agree += 1;
}
}
let w = if counted == 0 { 0.0 } else { agree as f32 / counted as f32 };
rows.push((degen, w.max(1.0 - w), vb, pad, max_idx, max_idx + 1 == vc));
}
}
rows.sort_by(|a, b| a.0.cmp(&b.0).then(b.1.total_cmp(&a.1)));
println!("\n{} in-range candidates; best 12 by (degenerate, winding):", rows.len());
for (d, w, vb, pad, mx, cov) in rows.iter().take(12) {
let mark = if Some(*vb) == ours { " <-- ours" } else { "" };
println!(
" vb 0x{vb:07X} pad {pad} degen {d:>4} wind {w:.3} max_idx {mx}/{} {}{mark}",
vc - 1,
if *cov { "covers" } else { "SHORT" }
);
}
let clean = rows.iter().filter(|r| r.0 == 0 && r.5).count();
println!("\n{clean} candidates are degenerate-free AND cover the pool exactly");
}

View File

@@ -0,0 +1,53 @@
//! Does a resource's DESCRIPTOR carry its bounding box?
//!
//! The last cross-container disagreements are 24-vertex bound boxes swapping
//! identities; no anchoring rule can pin them (see docs). If the descriptor
//! states the box, that is the missing information. This decodes the resource,
//! takes the box its geometry actually spans, and searches the descriptor for
//! those float values.
//!
//! Usage: bounds_in_descriptor <container.xpr> <resource>...
use sylpheed_formats::mesh::{xbg7_descriptor_range, Xbg7Model};
use std::collections::HashSet;
fn main() {
let a: Vec<String> = std::env::args().collect();
let bytes = std::fs::read(&a[1]).expect("container");
let want: HashSet<String> = a[2..].iter().cloned().collect();
for m in Xbg7Model::models_named(&bytes, &want, &|| false) {
let (mut lo, mut hi) = ([f32::MAX; 3], [f32::MIN; 3]);
for s in &m.meshes {
for q in &s.positions {
for k in 0..3 {
lo[k] = lo[k].min(q[k]);
hi[k] = hi[k].max(q[k]);
}
}
}
let Some((d0, d1)) = xbg7_descriptor_range(&bytes, &m.name) else { continue };
println!(
"{} descriptor 0x{d0:x}..0x{d1:x} ({} bytes), box lo{:?} hi{:?}",
m.name,
d1 - d0,
lo.map(|v| v.round()),
hi.map(|v| v.round())
);
// Where in the descriptor does each bound value appear (±0.01)?
let targets: Vec<(&str, f32)> = vec![
("lo.x", lo[0]), ("lo.y", lo[1]), ("lo.z", lo[2]),
("hi.x", hi[0]), ("hi.y", hi[1]), ("hi.z", hi[2]),
];
for (label, v) in targets {
let mut at: Vec<usize> = Vec::new();
let mut o = d0;
while o + 4 <= d1 {
let f = f32::from_be_bytes(bytes[o..o + 4].try_into().unwrap());
if (f - v).abs() <= 0.01 * (1.0 + v.abs()) {
at.push(o - d0);
}
o += 4;
}
println!(" {label:5} {v:10.3} at descriptor offsets {:x?}", &at[..at.len().min(6)]);
}
}
}

View File

@@ -0,0 +1,38 @@
//! How many caption lines does `build_caption_text` actually recover?
use std::collections::BTreeMap;
use sylpheed_formats::{movie_subtitle, PakArchive};
fn main() {
let disc = std::env::var("SYLPHEED_DISC").expect("set SYLPHEED_DISC");
let pak = PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).expect("pak");
let all = movie_subtitle::build_caption_text(&pak);
let demo = movie_subtitle::build_demo_text(&pak);
let mut per: BTreeMap<&str, (usize, usize)> = BTreeMap::new();
for (id, lines) in &all {
let fam = id.split('_').next().unwrap();
let e = per.entry(fam).or_default();
e.0 += 1;
e.1 += lines.len();
}
println!("{:<8} {:>8} {:>9}", "family", "ids", "lines");
let (mut ids, mut lines) = (0, 0);
for (fam, (i, l)) in &per {
println!("{fam:<8} {i:>8} {l:>9}");
ids += i;
lines += l;
}
println!("{:<8} {:>8} {:>9}", "TOTAL", ids, lines);
println!(
"\nbuild_demo_text alone: {} ids, {} lines",
demo.len(),
demo.values().map(|v| v.len()).sum::<usize>()
);
// The DEMO family must come out identical either way — that is the control.
let demo_via_all: usize = all
.iter()
.filter(|(k, _)| k.starts_with("DEMO_"))
.map(|(_, v)| v.len())
.sum();
println!("DEMO via build_caption_text: {demo_via_all} lines");
}

View File

@@ -0,0 +1,222 @@
//! Where does a drawn block's INDEX buffer really live?
//!
//! Our XBG7 anchor scan only ever *assumes* the layout `[index buffer][vertex
//! buffer]` with a pad of at most 3 bytes between them (`anchor_pool_mesh`:
//! `ib = vb - idx_count*2 - pad`). Nothing on disc states it, and it is the gate
//! that rejected the capture-proven `e106_eng_02_l` block
//! (docs/re/captures/…): the block's index data was not where the decoder
//! looked. The F10 ship capture was extended on 2026-08-13 to log each draw's
//! index-buffer base, count and min/max index value, so the assumption is now
//! directly checkable:
//!
//! * `vbase - ibase` is the real gap in guest memory, and a stage container is
//! uploaded contiguously (see `shared_vbase_check`), so the same difference
//! holds in the file;
//! * `max index vs vcount` says whether a draw covers its whole vertex pool —
//! the "buffer not covered" miss class is a *sub-range draw* if it does not.
//!
//! Usage:
//! cargo run --release --example capture_ib_truth -- <Stage_SNN.xpr> <capture.log>...
use sylpheed_formats::mesh::{debug_resource_params, xbg7_resource_names, Xbg7Model};
use sylpheed_formats::ship_capture::{parse_capture, CapturedDraw};
use std::collections::HashMap;
fn q(v: f32) -> i64 {
(v as f64 * 1e4).round() as i64
}
fn main() {
let a: Vec<String> = std::env::args().collect();
if a.len() < 3 {
eprintln!("usage: capture_ib_truth <container.xpr> <capture.log>...");
std::process::exit(2);
}
let bytes = std::fs::read(&a[1]).expect("container");
// One entry per (log, vbase): the capture already de-dups per placement, and
// a buffer drawn at several transforms has the same index buffer each time.
let mut draws: Vec<CapturedDraw> = Vec::new();
let mut seen = std::collections::HashSet::new();
for log in &a[2..] {
let text = std::fs::read_to_string(log).expect("log");
for d in parse_capture(&text) {
// One entry per (log, vbase, index range): the engine issues SEVERAL
// draws over one vertex buffer, each with its own index sub-range, and
// it is their UNION that describes the block. (Captures taken before
// 2026-08-13 de-dup by (vbase, transform) and so hold only the first
// batch — such a log reads as a mysteriously short draw.)
let k = d.ib.map(|i| (i.ibase, i.icount)).unwrap_or((0, 0));
if d.ib.is_some() && d.pos.len() >= 4 && seen.insert((log.clone(), d.vbase, k)) {
draws.push(d);
}
}
}
eprintln!("{} drawn buffers with an index buffer", draws.len());
// ── Place the drawn buffers in the file: POSITION is f32×3 big-endian at
// vertex offset 0, so the dumped positions are a literal byte pattern.
let be = |at: usize| f32::from_be_bytes(bytes[at..at + 4].try_into().unwrap());
let mut index: HashMap<(i64, i64, i64), Vec<u32>> = HashMap::new();
let mut o = 0usize;
while o + 12 <= bytes.len() {
let (x, y, z) = (be(o), be(o + 4), be(o + 8));
if x.is_finite() && y.is_finite() && z.is_finite() && x.abs() < 1e6 && y.abs() < 1e6 && z.abs() < 1e6 {
index.entry((q(x), q(y), q(z))).or_default().push(o as u32);
}
o += 4;
}
let mut deltas: HashMap<i64, usize> = HashMap::new();
let mut hits: Vec<(i64, usize, &CapturedDraw)> = Vec::new(); // (delta, file offset, draw)
for d in &draws {
let k = (q(d.pos[0][0]), q(d.pos[0][1]), q(d.pos[0][2]));
for dx in -1..=1i64 {
for dy in -1..=1i64 {
for dz in -1..=1i64 {
let Some(cands) = index.get(&(k.0 + dx, k.1 + dy, k.2 + dz)) else { continue };
for &off in cands {
for stride in (12..=64).step_by(4) {
let ok = (1..4).all(|j| {
let at = off as usize + j * stride;
at + 12 <= bytes.len()
&& (0..3).all(|c| (be(at + c * 4) - d.pos[j][c]).abs() <= 1e-4)
});
if ok {
let delta = d.vbase as i64 - off as i64;
*deltas.entry(delta).or_default() += 1;
hits.push((delta, off as usize, d));
break;
}
}
}
}
}
}
}
let Some((&base_delta, &n)) = deltas.iter().max_by_key(|(_, n)| **n) else {
eprintln!("no draw could be placed in this container");
std::process::exit(1);
};
println!("container load constant: vbase - file_offset = 0x{base_delta:X} ({n} buffers agree)");
// ── Our decoder's view of the same container.
let models = Xbg7Model::stage_models(&bytes);
let mut by_off: HashMap<usize, Vec<(String, usize, usize)>> = HashMap::new();
for m in &models {
for sm in &m.meshes {
if let Some(off) = sm.vbuf_offset {
by_off
.entry(off)
.or_default()
.push((m.name.clone(), sm.positions.len(), sm.indices.len()));
}
}
}
println!("decoded {} resources, {} distinct vertex offsets\n", models.len(), by_off.len());
// Declared-but-not-decoded resources, indexed by their first marker's
// (vertex, index) counts. A drawn buffer our decoder cannot name is the one
// thing a capture can give the residual misses: ground truth for where the
// block actually is. Matching on counts is enough to propose an identity —
// then `debug_try_anchor` at that offset says which gate rejects it.
let decoded_names: std::collections::HashSet<String> =
models.iter().map(|m| m.name.clone()).collect();
let mut undecoded_by_counts: HashMap<(usize, usize), Vec<String>> = HashMap::new();
for n in xbg7_resource_names(&bytes) {
if decoded_names.contains(&n) {
continue;
}
if let Some((markers, _)) = debug_resource_params(&bytes, &n) {
if let Some(&(v, i)) = markers.first() {
undecoded_by_counts.entry((v, i)).or_default().push(n);
}
}
}
// ── The report: one row per drawn BUFFER, aggregating its index batches.
let mut per_buf: HashMap<u32, (usize, Vec<sylpheed_formats::ship_capture::CapturedIndexBuffer>, u32)> =
HashMap::new();
for (delta, voff, d) in &hits {
if *delta != base_delta {
continue;
}
let e = per_buf.entry(d.vbase).or_insert((*voff, Vec::new(), d.vcount));
let ib = d.ib.unwrap();
if !e.1.contains(&ib) {
e.1.push(ib);
}
}
let (mut pad0, mut pad_small, mut pad_off, mut unnamed) = (0usize, 0usize, 0usize, 0usize);
let (mut cover_exact, mut cover_short, mut idx_equal, mut idx_partial) = (0usize, 0usize, 0usize, 0usize);
let mut rows: Vec<(usize, String)> = Vec::new();
for (_, (voff, ibs, vcount)) in per_buf.iter() {
let batches = ibs.len();
let total: u32 = ibs.iter().map(|i| i.icount).sum();
let lo = ibs.iter().map(|i| i.ibase).min().unwrap() as i64 - base_delta;
let hi = ibs.iter().map(|i| i.ibase + i.icount * 2).max().unwrap() as i64 - base_delta;
let umax = ibs.iter().map(|i| i.imax).max().unwrap();
let gap = *voff as i64 - hi; // bytes from the end of the index data to the vertex buffer
let names = by_off.get(voff);
let dec_idx = names
.and_then(|v| v.iter().find(|(_, p, _)| *p as u32 == *vcount).map(|(_, _, i)| *i as u32));
// The decoder's assumption, scored: it expects the whole index buffer at
// `vb - 2*idx_count - pad`, pad ≤ 3.
let dec_pad = dec_idx.map(|i| *voff as i64 - (i as i64) * 2 - lo);
match dec_pad {
Some(0) => pad0 += 1,
Some(p) if (1..=3).contains(&p) => pad_small += 1,
Some(_) => pad_off += 1,
None => unnamed += 1,
}
if umax + 1 == *vcount {
cover_exact += 1;
} else {
cover_short += 1;
}
match dec_idx {
Some(i) if i == total => idx_equal += 1,
Some(_) => idx_partial += 1,
None => {}
}
rows.push((
*voff,
format!(
"vb 0x{:07X} v={:<6} batches {:<3} idx {:<6} span {:<7} gap {:<8} decpad {:<7} cover {:<10} {}",
voff,
vcount,
batches,
total,
hi - lo,
gap,
dec_pad.map(|p| p.to_string()).unwrap_or_else(|| "?".into()),
if umax + 1 == *vcount { "exact".to_string() } else { format!("{}/{}", umax, vcount - 1) },
names
.map(|v| v
.iter()
.map(|(n, p, i)| format!("{n}(v{p},i{i})"))
.collect::<Vec<_>>()
.join(" "))
.unwrap_or_else(|| {
// Nothing of ours sits here — is it a resource that never
// decodes? Propose it by (vertex, index) counts.
undecoded_by_counts
.get(&(*vcount as usize, total as usize))
.map(|v| format!("MISSED? {}", v.join(" ")))
.unwrap_or_else(|| "-".into())
})
),
));
}
rows.sort();
for (_, r) in &rows {
println!("{r}");
}
println!("\nplaced {} drawn buffers in this container", rows.len());
println!(
"DECODER layout assumption — whole index buffer at vb - 2*idx_count - pad: pad 0 {pad0} · pad 1..3 {pad_small} · elsewhere {pad_off} · not decoded here {unnamed}"
);
println!(
"index extent: our idx_count == sum of captured batches for {idx_equal} buffers, differs for {idx_partial}"
);
println!("vertex-pool coverage by the union of batches: exact {cover_exact} · short {cover_short}");
}

View File

@@ -0,0 +1,175 @@
//! Do our decoded triangle indices equal the ones the GPU actually read?
//!
//! `capture_ib_truth` established *where* a block's index buffer lives and that
//! its length matches ours. This asks the stronger question: are the index VALUES
//! the same, in the same order? The capture prints the first 24 indices of every
//! draw batch verbatim (`idx: …`), and a batch's `ibase` locates it inside the
//! block's index buffer — so for each drawn buffer we can line the captured run
//! up against `GameMesh::indices` at the right offset and compare element by
//! element. That tests the whole index path at once: the anchor, the u16
//! big-endian read, the triangle-list interpretation (a strip would disagree
//! immediately), and the sub-mesh carve.
//!
//! Usage:
//! cargo run --release --example capture_index_bytes -- <Stage_SNN.xpr> <capture.log>...
use sylpheed_formats::mesh::Xbg7Model;
use sylpheed_formats::ship_capture::{parse_capture, CapturedDraw};
use std::collections::HashMap;
fn q(v: f32) -> i64 {
(v as f64 * 1e4).round() as i64
}
fn main() {
let a: Vec<String> = std::env::args().collect();
if a.len() < 3 {
eprintln!("usage: capture_index_bytes <container.xpr> <capture.log>...");
std::process::exit(2);
}
let bytes = std::fs::read(&a[1]).expect("container");
let mut draws: Vec<CapturedDraw> = Vec::new();
let mut seen = std::collections::HashSet::new();
for log in &a[2..] {
let text = std::fs::read_to_string(log).expect("log");
for d in parse_capture(&text) {
let k = d.ib.map(|i| (i.ibase, i.icount)).unwrap_or((0, 0));
if d.ib.map_or(false, |i| i.head_len > 0) && d.pos.len() >= 4 && seen.insert((log.clone(), d.vbase, k)) {
draws.push(d);
}
}
}
eprintln!("{} draw batches with an index head", draws.len());
// Place each drawn buffer in the file by its dumped positions (see
// capture_ib_truth for the method) and keep the modal load constant.
let be = |at: usize| f32::from_be_bytes(bytes[at..at + 4].try_into().unwrap());
let mut index: HashMap<(i64, i64, i64), Vec<u32>> = HashMap::new();
let mut o = 0usize;
while o + 12 <= bytes.len() {
let (x, y, z) = (be(o), be(o + 4), be(o + 8));
if x.is_finite() && y.is_finite() && z.is_finite() && x.abs() < 1e6 && y.abs() < 1e6 && z.abs() < 1e6 {
index.entry((q(x), q(y), q(z))).or_default().push(o as u32);
}
o += 4;
}
let mut deltas: HashMap<i64, usize> = HashMap::new();
let mut hits: Vec<(i64, usize, &CapturedDraw)> = Vec::new();
for d in &draws {
let k = (q(d.pos[0][0]), q(d.pos[0][1]), q(d.pos[0][2]));
for dx in -1..=1i64 {
for dy in -1..=1i64 {
for dz in -1..=1i64 {
let Some(cands) = index.get(&(k.0 + dx, k.1 + dy, k.2 + dz)) else { continue };
for &off in cands {
for stride in (12..=64).step_by(4) {
let ok = (1..4).all(|j| {
let at = off as usize + j * stride;
at + 12 <= bytes.len()
&& (0..3).all(|c| (be(at + c * 4) - d.pos[j][c]).abs() <= 1e-4)
});
if ok {
*deltas.entry(d.vbase as i64 - off as i64).or_default() += 1;
hits.push((d.vbase as i64 - off as i64, off as usize, d));
break;
}
}
}
}
}
}
}
let Some((&base_delta, _)) = deltas.iter().max_by_key(|(_, n)| **n) else {
eprintln!("no draw could be placed in this container");
std::process::exit(1);
};
println!("load constant 0x{base_delta:X}");
// Our decode, indexed by vertex offset. A model may hold several sub-meshes;
// compare against the one whose vertex count matches the draw.
let models = Xbg7Model::stage_models(&bytes);
let mut by_off: HashMap<usize, Vec<(String, usize, Vec<u32>)>> = HashMap::new();
for m in &models {
for sm in &m.meshes {
if let Some(off) = sm.vbuf_offset {
by_off
.entry(off)
.or_default()
.push((m.name.clone(), sm.positions.len(), sm.indices.clone()));
}
}
}
// Where does each buffer's index data START? Take it from the capture, not
// from an assumed pad: `capture_ib_truth` established that a buffer's batches
// tile its index buffer exactly (sum of counts == our idx_count, span ==
// 2*count), so the lowest `ibase` over a buffer's batches IS the block's
// index start. Assuming `vb - 2*len` instead is wrong for the buffers that
// sit at pad 2 and shifts the whole comparison by one element.
let mut ib_start: HashMap<u32, i64> = HashMap::new();
for (delta, _, d) in &hits {
if *delta != base_delta {
continue;
}
let ibase = d.ib.unwrap().ibase as i64;
ib_start.entry(d.vbase).and_modify(|e| *e = (*e).min(ibase)).or_insert(ibase);
}
let (mut agree, mut disagree, mut unmatched, mut nooverlap) = (0usize, 0usize, 0usize, 0usize);
let mut bad: Vec<String> = Vec::new();
let mut checked_elems = 0usize;
let mut done: std::collections::HashSet<(u32, u32)> = std::collections::HashSet::new();
for (delta, voff, d) in &hits {
if *delta != base_delta {
continue;
}
let ib = d.ib.unwrap();
if !done.insert((d.vbase, ib.ibase)) {
continue;
}
let Some(cands) = by_off.get(voff) else {
unmatched += 1;
continue;
};
let Some((name, _, ours)) = cands.iter().find(|(_, p, _)| *p as u32 == d.vcount) else {
unmatched += 1;
continue;
};
// Element 0 of our index list is the block's index start, as observed.
let ours_start = ib_start[&d.vbase] - base_delta;
let batch_off = ib.ibase as i64 - base_delta - ours_start;
if batch_off < 0 || batch_off % 2 != 0 {
nooverlap += 1;
continue;
}
let first = (batch_off / 2) as usize;
let n = (ib.head_len as usize).min(ours.len().saturating_sub(first));
if n == 0 {
nooverlap += 1;
continue;
}
let mism = (0..n).find(|&k| ours[first + k] != ib.head[k]);
checked_elems += n;
match mism {
None => agree += 1,
Some(k) => {
disagree += 1;
if bad.len() < 8 {
bad.push(format!(
"{name} vb 0x{:07X} batch@{first}: ours {:?} != captured {:?} (first differs at {k})",
voff,
&ours[first..first + n.min(12)],
&ib.head[..n.min(12)]
));
}
}
}
}
println!(
"index runs compared: {agree} identical · {disagree} differing · {unmatched} no decoded resource · {nooverlap} batch outside our buffer"
);
println!("{checked_elems} index elements checked against the GPU");
for b in &bad {
println!(" MISMATCH {b}");
}
}

View File

@@ -0,0 +1,112 @@
//! Which container did each captured draw come from, and where in it?
//!
//! Extends the single-container `--map` check in `shared_vbase_check` to a whole
//! `resource3d` directory. For each container it indexes every 4-byte-aligned
//! position triple, looks up each draw's first dumped position, confirms the run
//! at a fixed stride, and reports the modal `vbase offset`. A container the
//! engine loaded shows one dominant constant; an unrelated one shows noise.
//!
//! The output is capture-named ground truth for anchors far beyond the one ship
//! `Stage_S01` gave us.
//!
//! Usage: capture_truth_scan <resource3d_dir> <capture.log>...
use sylpheed_formats::ship_capture::{parse_capture, parse_drawlog, CapturedDraw};
use std::collections::HashMap;
fn q(v: f32) -> i64 {
(v as f64 * 1e4).round() as i64
}
fn main() {
let a: Vec<String> = std::env::args().collect();
let dir = &a[1];
let mut draws: Vec<CapturedDraw> = Vec::new();
let mut seen = std::collections::HashSet::new();
for log in &a[2..] {
let text = std::fs::read_to_string(log).expect("log");
let mut d = parse_capture(&text);
if d.is_empty() {
d = parse_drawlog(&text);
}
for x in d {
// One entry per buffer; the logs are already deduped per transform.
if x.pos.len() >= 8 && x.vcount >= 20 && seen.insert((log.clone(), x.vbase)) {
draws.push(x);
}
}
}
eprintln!("{} distinct (log, vbase) draws to place", draws.len());
let mut files: Vec<_> = std::fs::read_dir(dir)
.unwrap()
.flatten()
.map(|e| e.path())
.filter(|p| p.extension().and_then(|s| s.to_str()) == Some("xpr"))
.collect();
files.sort();
let mut placed = 0usize;
for f in &files {
let Ok(bytes) = std::fs::read(f) else { continue };
// Index quantised position triples. Junk floats (NaN/huge) are skipped,
// which prunes most of a texture-heavy container.
let be = |at: usize| f32::from_be_bytes(bytes[at..at + 4].try_into().unwrap());
let mut index: HashMap<(i64, i64, i64), Vec<u32>> = HashMap::new();
let mut o = 0usize;
while o + 12 <= bytes.len() {
let (x, y, z) = (be(o), be(o + 4), be(o + 8));
if x.is_finite() && y.is_finite() && z.is_finite() && x.abs() < 1e6 && y.abs() < 1e6 && z.abs() < 1e6
{
index.entry((q(x), q(y), q(z))).or_default().push(o as u32);
}
o += 4;
}
let mut deltas: HashMap<i64, Vec<(u32, u32)>> = HashMap::new();
for d in &draws {
let k = (q(d.pos[0][0]), q(d.pos[0][1]), q(d.pos[0][2]));
// ±1 in each axis: the log rounds, our file value may round the
// other way at a tie.
for dx in -1..=1i64 {
for dy in -1..=1i64 {
for dz in -1..=1i64 {
let Some(cands) = index.get(&(k.0 + dx, k.1 + dy, k.2 + dz)) else {
continue;
};
for &off in cands {
for stride in (12..=64).step_by(4) {
let ok = (1..4).all(|j| {
let at = off as usize + j * stride;
at + 12 <= bytes.len()
&& (0..3).all(|c| {
(be(at + c * 4) - d.pos[j][c]).abs() <= 1e-4
})
});
if ok {
deltas
.entry(d.vbase as i64 - off as i64)
.or_default()
.push((d.vbase, d.vcount));
break;
}
}
}
}
}
}
}
let mut top: Vec<_> = deltas.into_iter().collect();
top.sort_by_key(|(_, v)| std::cmp::Reverse(v.len()));
if let Some((delta, hits)) = top.first() {
if hits.len() >= 3 {
placed += hits.len();
println!(
"{:<22} base=0x{:<10X} buffers={}",
f.file_name().unwrap().to_string_lossy(),
delta,
hits.len()
);
}
}
}
eprintln!("{placed} draws placed in a container");
}

View File

@@ -0,0 +1,81 @@
//! RE probe: what does the disc say about CHALLENGE / EXTRA missions?
//!
//! The title's stage-config reader (`0x82184b98`..`0x82184e94`) selects one of three
//! config sections by a mode field at `obj+144`:
//! mode == 3 -> "EXTRA"
//! mode == 5 or 6 -> "CHALLENGE"
//! otherwise -> "FILE"
//! so challenge missions are a *mode*, not a separate stage numbering. This probe
//! asks the disc which stage records exist and what GP_CHALLENGE.pak carries.
//!
//! Run: cargo run --release -p sylpheed-formats --example challenge_map -- <disc-root>
use sylpheed_formats::{idxd::IdxdObject, pak::PakArchive};
fn main() {
let disc = std::env::args().nth(1).unwrap_or_else(|| {
std::env::var("SYLPHEED_DISC").expect("pass disc root or set SYLPHEED_DISC")
});
println!("=== 1. StageResource records in GP_MAIN_GAME_E.pak ===");
let arc = PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).unwrap();
let mut rows = vec![];
for e in arc.entries() {
let Ok(b) = arc.read(e) else { continue };
let Ok(o) = IdxdObject::parse(&b) else { continue };
if o.schema_hash != 0x3c9ae32e {
continue;
}
let t = o.tokens();
let stage = t
.iter()
.find_map(|s| s.strip_prefix("EnumUnit_").map(|x| x.trim_end_matches(".tbl").to_string()))
.unwrap_or("?".into());
let bg = o.get_raw("BackGroundID").unwrap_or("?").to_string();
rows.push((stage, bg, t.len()));
}
rows.sort();
println!(" {} stage records", rows.len());
for (s, bg, n) in &rows {
println!(" {s:8} bg={bg:16} tokens={n}");
}
println!("\n=== 2. GP_CHALLENGE.pak contents ===");
match PakArchive::open(format!("{disc}/dat/GP_CHALLENGE.pak")) {
Ok(ch) => {
let mut by_schema: std::collections::BTreeMap<u32, usize> = Default::default();
let mut all_tokens: Vec<(u32, Vec<String>)> = vec![];
for e in ch.entries() {
let Ok(b) = ch.read(e) else { continue };
let Ok(o) = IdxdObject::parse(&b) else { continue };
*by_schema.entry(o.schema_hash).or_default() += 1;
all_tokens.push((o.schema_hash, o.tokens().iter().map(|s| s.to_string()).collect()));
}
println!(" {} entries, {} IDXD objects", ch.entries().len(), all_tokens.len());
for (h, n) in &by_schema {
println!(" schema {h:08x} x{n}");
}
for (h, t) in all_tokens.iter().take(12) {
println!(" -- {h:08x}: {:?}", &t[..t.len().min(60)]);
}
}
Err(e) => println!(" open failed: {e}"),
}
println!("\n=== 3. tokens mentioning Challenge / EX across the main pak ===");
let mut hits: std::collections::BTreeSet<String> = Default::default();
for e in arc.entries() {
let Ok(b) = arc.read(e) else { continue };
let Ok(o) = IdxdObject::parse(&b) else { continue };
for t in o.tokens() {
let l = t.to_ascii_lowercase();
if l.contains("challenge") || t.ends_with("_EX") || t.contains("_EX4") || t.contains("_EX5") {
hits.insert(format!("{:08x} {t}", o.schema_hash));
}
}
}
for h in hits.iter().take(120) {
println!(" {h}");
}
println!(" ({} distinct)", hits.len());
}

View File

@@ -0,0 +1,78 @@
//! RE probe: the GamePart_ChallengeMission screen config.
//!
//! The class's code range (`0x82187E60`..`0x8218CF10`, bounded by the factory
//! creator thunks either side) references these config keys:
//! MISSIONS, MISSION_ID, NEW_STAGE, REQUIREMENT, REQUIREMENT_DESC, "Always",
//! GRAY_BUTTON, NORMAL_BUTTON, THUMBNAIL, STAGE_DESC, TEXT_STAGE,
//! RECORD_TYPE, "Time", TEXT_RECORD, BASE_INFO
//! i.e. the challenge list is a *config record* with a per-mission REQUIREMENT.
//! It lives in `tables.pak` (one copy per language), not in GP_CHALLENGE.pak.
//!
//! Run: cargo run --release -p sylpheed-formats --example challenge_screen -- <disc-root>
use sylpheed_formats::{idxd::IdxdObject, pak::PakArchive};
const KEYS: &[&str] = &[
"MISSIONS",
"MISSION_ID",
"REQUIREMENT",
"REQUIREMENT_DESC",
"NEW_STAGE",
"GRAY_BUTTON",
"NORMAL_BUTTON",
"RECORD_TYPE",
"THUMBNAIL",
"STAGE_DESC",
];
fn find(h: &[u8], n: &[u8]) -> bool {
h.windows(n.len()).any(|w| w == n)
}
fn main() {
let disc = std::env::args().nth(1).unwrap_or_else(|| {
std::env::var("SYLPHEED_DISC").expect("pass disc root or set SYLPHEED_DISC")
});
let dat = format!("{disc}/dat");
let mut paks: Vec<_> = std::fs::read_dir(&dat)
.expect("dat dir")
.filter_map(|e| e.ok())
.map(|e| e.path())
.filter(|p| p.extension().is_some_and(|x| x == "pak"))
.collect();
paks.sort();
for p in &paks {
let Ok(arc) = PakArchive::open(p) else { continue };
for (i, e) in arc.entries().iter().enumerate() {
let Ok(b) = arc.read(e) else { continue };
if KEYS.iter().filter(|k| find(&b, k.as_bytes())).count() < KEYS.len() {
continue;
}
let name = p.file_name().unwrap().to_string_lossy().to_string();
match IdxdObject::parse(&b) {
Ok(o) => {
let toks = o.tokens();
// language tag: the PATH value, e.g. "dat\GP_CHALLENGE.pak+eng\"
let lang = toks
.iter()
.find(|t| t.contains("GP_CHALLENGE.pak+"))
.map(|t| t.to_string())
.unwrap_or_default();
println!(
"\n===== {name} entry #{i} schema {:08x} {} tokens [{lang}] =====",
o.schema_hash,
toks.len()
);
// Print the pool in order; the record is key/value interleaved and
// IDXD dedupes repeats, so pairing is read by eye, not asserted.
for (j, t) in toks.iter().enumerate() {
println!(" {j:3} {t}");
}
}
Err(e) => println!("\n===== {name} entry #{i}: IDXD parse failed: {e} ====="),
}
}
}
}

View File

@@ -0,0 +1,34 @@
//! How much of the "decoded geometry" belongs to SCENE COMPOSITES rather than to
//! real meshes? A composite (`rou_*`, `e_rou_*`) carries node transforms; the
//! anchor scan nevertheless finds a block for it, and that pseudo-geometry lands
//! in coverage and consistency counts.
//! Usage: composite_geometry <resource3d_dir>
use sylpheed_formats::mesh::{scene_world_nodes, Xbg7Model};
fn main() {
let dir = std::env::args().nth(1).expect("resource3d dir");
let mut files: Vec<_> = std::fs::read_dir(&dir)
.unwrap()
.flatten()
.map(|e| e.path())
.filter(|p| p.extension().and_then(|s| s.to_str()) == Some("xpr"))
.collect();
files.sort();
let (mut total, mut composite, mut composite_named) = (0usize, 0usize, 0usize);
for f in &files {
let Ok(bytes) = std::fs::read(f) else { continue };
for m in Xbg7Model::anchor_models_cancellable(&bytes, 0.0, &|| false) {
total += 1;
let has_nodes = !scene_world_nodes(&bytes, &m.name).is_empty();
let named = m.name.starts_with("rou_") || m.name.starts_with("e_rou_");
if has_nodes {
composite += 1;
}
if named {
composite_named += 1;
}
}
}
println!(
"{total} decoded; {composite} have scene nodes (a composite), {composite_named} are named rou_/e_rou_"
);
}

View File

@@ -0,0 +1,87 @@
//! Minority report: which container decodes a shared resource differently from
//! all the others?
//!
//! Cross-container consistency has been measured as "do the spans agree", which
//! only says a resource is inconsistent — not which copy is wrong. With three or
//! more copies the majority is the reference, and the minority names the
//! container AND the resource to look at. That is what caught the `n054`/`n056`
//! shift chain after the exact-coverage fix.
//!
//! Usage: consensus_check <resource3d_dir> [--list]
use sylpheed_formats::mesh::Xbg7Model;
use std::collections::{BTreeMap, HashMap};
fn main() {
let dir = std::env::args().nth(1).expect("resource3d dir");
let list = std::env::args().any(|a| a == "--list");
let mut files: Vec<_> = std::fs::read_dir(&dir)
.unwrap()
.flatten()
.map(|e| e.path())
.filter(|p| p.extension().and_then(|s| s.to_str()) == Some("xpr"))
.collect();
files.sort();
// name -> [(container, verts, tris, span)]
let mut seen: BTreeMap<String, Vec<(String, usize, usize, [i64; 3])>> = BTreeMap::new();
for f in &files {
let Ok(bytes) = std::fs::read(f) else { continue };
let where_ = f.file_name().unwrap().to_string_lossy().to_string();
for m in Xbg7Model::anchor_models_cancellable(&bytes, 0.0, &|| false) {
let (mut lo, mut hi) = ([f32::MAX; 3], [f32::MIN; 3]);
for s in &m.meshes {
for q in &s.positions {
for k in 0..3 {
lo[k] = lo[k].min(q[k]);
hi[k] = hi[k].max(q[k]);
}
}
}
if lo[0] == f32::MAX {
continue;
}
let v: usize = m.meshes.iter().map(|s| s.positions.len()).sum();
let t: usize = m.meshes.iter().map(|s| s.indices.len() / 3).sum();
let span = [
(hi[0] - lo[0]).round() as i64,
(hi[1] - lo[1]).round() as i64,
(hi[2] - lo[2]).round() as i64,
];
seen.entry(m.name.clone()).or_default().push((where_.clone(), v, t, span));
}
}
let (mut resources, mut minority) = (0usize, 0usize);
let mut rows: Vec<String> = Vec::new();
for (name, list_) in &seen {
// Only compare decodes that agree on how much geometry they found, and
// only where a majority can exist.
if list_.len() < 3 || !list_.iter().all(|e| e.1 == list_[0].1 && e.2 == list_[0].2) {
continue;
}
let mut votes: HashMap<[i64; 3], usize> = HashMap::new();
for e in list_ {
*votes.entry(e.3).or_default() += 1;
}
let (best, n) = votes.iter().max_by_key(|(_, n)| **n).unwrap();
if *n * 2 <= list_.len() {
continue; // no majority — cannot call anyone the odd one out
}
resources += 1;
for e in list_.iter().filter(|e| e.3 != *best) {
minority += 1;
rows.push(format!(
"{:<22} {name:<26} v={:<6} {:?} vs the other {n} containers' {:?}",
e.0, e.1, e.3, best
));
}
}
if list {
for r in &rows {
println!("{r}");
}
}
println!(
"{minority} minority decodes across {resources} resources that have a majority"
);
}

View File

@@ -16,7 +16,7 @@
//! Usage:
//! SYLPHEED_ISO=... cargo run --release --example correlate_capture -- \
//! <capture.log> <Stage_SNN> <ship_id> [ref_part_substr] [--emit]
//! e.g. SYLPHEED_ISO="/home/fabi/RE Project Sylpheed/Project Sylpheed - Arc of
//! e.g. SYLPHEED_ISO="/home/fabi/RE - Project Sylpheed/Project Sylpheed - Arc of
//! Deception (USA, Europe) (En,Ja).iso" \
//! cargo run --release --example correlate_capture -- \
//! xenia_ship_capture.log Stage_S01 e106 bdy_04 --emit

View File

@@ -0,0 +1,48 @@
//! Do real index buffers address their whole vertex pool?
//!
//! `validate_block` rejects a block whose indices reach fewer than `vtx_count4`
//! vertices ("buffer not covered"). That gate is the furthest-reached rejection
//! for a handful of resources that never decode — so the question is whether it
//! is well founded. This measures the slack on every block that DOES decode: if
//! real geometry always covers its pool, under-coverage is good evidence of a
//! wrong candidate and the gate stands.
use sylpheed_formats::mesh::Xbg7Model;
use std::collections::BTreeMap;
fn main() {
let dir = std::env::args().nth(1).expect("resource3d dir");
let mut files: Vec<_> = std::fs::read_dir(&dir)
.unwrap()
.flatten()
.map(|e| e.path())
.filter(|p| p.extension().and_then(|s| s.to_str()) == Some("xpr"))
.collect();
files.sort();
let mut hist: BTreeMap<i64, usize> = BTreeMap::new();
let mut worst: Vec<(i64, String)> = Vec::new();
for f in &files {
let Ok(bytes) = std::fs::read(f) else { continue };
for m in Xbg7Model::anchor_models_cancellable(&bytes, 0.0, &|| false) {
for sub in &m.meshes {
if sub.positions.is_empty() || sub.indices.is_empty() {
continue;
}
let max_idx = *sub.indices.iter().max().unwrap() as i64;
let slack = sub.positions.len() as i64 - 1 - max_idx;
*hist.entry(slack.min(20)).or_default() += 1;
if slack > 4 {
worst.push((slack, format!("{} in {}", m.name, f.file_name().unwrap().to_string_lossy())));
}
}
}
}
println!("unreferenced tail vertices (vtx_count 1 max index), over decoded sub-meshes:");
for (slack, n) in &hist {
println!(" {:>3}{} : {n}", slack, if *slack == 20 { "+" } else { " " });
}
worst.sort_by_key(|(s, _)| std::cmp::Reverse(*s));
for (s, w) in worst.iter().take(5) {
println!(" largest slack {s}: {w}");
}
}

View File

@@ -0,0 +1,59 @@
//! Are subtitle cue times really seconds?
//!
//! Every "audio missing" verdict in voice-bank-leading-region.md rests on
//! reading `track_voice_cues`' `f32` as seconds. `parse_timing` does compute
//! `mm*60 + ss` — but only if the token really is `mm:ss.cc`. The movies are on
//! the disc, so their true length is an independent oracle: a cue must land
//! INSIDE its own movie.
use std::process::Command;
use sylpheed_formats::{movie_subtitle, PakArchive};
fn movie_secs(path: &str) -> Option<f32> {
let out = Command::new("ffprobe")
.args([
"-v", "error", "-show_entries", "format=duration", "-of", "csv=p=0", path,
])
.output()
.ok()?;
String::from_utf8_lossy(&out.stdout).trim().parse().ok()
}
fn main() {
let disc = std::env::var("SYLPHEED_DISC").expect("set SYLPHEED_DISC");
let lang = PakArchive::open(format!("{disc}/dat/movie/eng.pak")).expect("eng.pak");
let mut checked = 0;
let mut over = 0;
let mut worst: Vec<(String, f32, f32)> = Vec::new();
let dir = format!("{disc}/dat/movie");
let Ok(rd) = std::fs::read_dir(&dir) else { return };
for e in rd.flatten() {
let p = e.path();
if p.extension().is_none_or(|x| x != "wmv") {
continue;
}
let base = p.file_stem().unwrap().to_string_lossy().to_string();
let cues = movie_subtitle::track_voice_cues(&lang, &base);
if cues.is_empty() {
continue;
}
let last = cues.iter().map(|(_, t)| *t).fold(0.0f32, f32::max);
let Some(secs) = movie_secs(&p.to_string_lossy()) else { continue };
checked += 1;
if last > secs {
over += 1;
worst.push((base, last, secs));
}
}
worst.sort_by(|a, b| (b.1 - b.2).partial_cmp(&(a.1 - a.2)).unwrap());
println!("movies with cues checked: {checked}");
println!("cues landing AFTER the movie ends: {over}");
for (m, last, secs) in worst.iter().take(8) {
println!(" {m:<28} last cue {last:>8.2}s movie {secs:>8.2}s");
}
println!(
"\nIf cues were centiseconds the last cue would be ~100x too big and \
essentially all {checked} would overflow."
);
}

View File

@@ -0,0 +1,67 @@
//! Does any unread declaration word POINT at the element's T8aD child?
//!
//! `pteff05.t32`/`pteff04.t32` resolve to no sprite because the `T8aD` they want
//! is registered under the name `8AX`. Elimination says `8AX` is the one they
//! mean -- one unresolved element, one unclaimed non-focus-state child, in 6 of
//! 6 title-side builds. Elimination is not a pointer, so: the 60-byte
//! declaration reads name[0..28], parent@32, kind@40, pivot@48/52. The words at
//! +28, +36, +44 and +56 are unread. If one of them indexes the RATC child
//! table, the RESOLVED elements are the control -- their child index is known,
//! so a candidate field must reproduce it for them before it may be believed for
//! the unresolved one.
//!
//! cargo run -p sylpheed-formats --example decl_word_probe -- <pak> [entry]
use sylpheed_formats::{pak, ratc, ui_layout};
const OFFS: [usize; 4] = [28, 36, 44, 56];
fn be32(b: &[u8], o: usize) -> u32 {
if o + 4 > b.len() { return 0; }
u32::from_be_bytes([b[o], b[o + 1], b[o + 2], b[o + 3]])
}
fn main() {
let path = std::env::args().nth(1).expect("usage: decl_word_probe <pak> [entry]");
let want: Option<usize> = std::env::args().nth(2).and_then(|s| s.parse().ok());
let ar = pak::PakArchive::open(&path).expect("open pak");
let entries: Vec<_> = ar.entries().to_vec();
// Per candidate offset, across every build: control hits / control total.
let mut hit = [0usize; 4];
let mut tot = 0usize;
for (i, e) in entries.iter().enumerate() {
if want.is_some_and(|w| w != i) { continue; }
let Ok(bytes) = ar.read(e) else { continue };
let Some(build) = ui_layout::parse_build(&bytes) else { continue };
let Some(kids) = ratc::parse(&bytes) else { continue };
// Index space to test against: the T8aD children, in child order.
let t8: Vec<&ratc::RatcChild> = kids.iter().filter(|c| c.kind == "T8aD").collect();
if build.elements.iter().all(|el| el.sprite.is_some() || el.kind & 0x10 != 0) {
continue;
}
println!("== entry {i} ({} elements, {} T8aD children)", build.elements.len(), t8.len());
for (n, c) in t8.iter().enumerate() { println!(" child[{n:2}] {}", c.name); }
for el in &build.elements {
if el.kind & 0x10 != 0 { continue; }
let d = &bytes[0x20 + el.index * 60..0x20 + (el.index + 1) * 60];
let words: Vec<u32> = OFFS.iter().map(|&o| be32(d, o)).collect();
// The control: for a RESOLVED element, which T8aD child is it?
let truth = el.sprite.as_ref()
.and_then(|s| t8.iter().position(|c| &c.name == s));
if let Some(t) = truth {
tot += 1;
for (k, w) in words.iter().enumerate() {
if *w as usize == t { hit[k] += 1; }
}
}
println!(
" [{:2}] {:26} sprite={:?} child={:?} +28={} +36={} +44={} +56={}",
el.index, el.name, el.sprite, truth,
words[0] as i32, words[1] as i32, words[2] as i32, words[3] as i32
);
}
}
println!("\nCONTROL: resolved elements whose child index a word reproduces, of {tot}:");
for (k, o) in OFFS.iter().enumerate() {
println!(" +{o:<3} {:3}/{tot}", hit[k]);
}
}

View File

@@ -0,0 +1,71 @@
//! Dump an XBG7 resource's descriptor around each index marker, to see whether a
//! grouped pool carries a declaration PER sub-mesh (the 2026-08-13 mixed-stride
//! finding says it must: n201's four sub-meshes use strides 24/24/24/28 and four
//! different vertex shaders).
//! Usage: desc_dump <container.xpr> <resource>
fn be32(b: &[u8], o: usize) -> u32 {
u32::from_be_bytes(b[o..o + 4].try_into().unwrap())
}
fn main() {
let a: Vec<String> = std::env::args().collect();
let bytes = std::fs::read(&a[1]).unwrap();
let want = &a[2];
// Walk the XPR2 directory by hand (the crate's reader is private).
// 16-byte directory entries at 0x10: type tag, data offset, descriptor size,
// name offset — all relative to 0x10 (see texture::Xpr2ResourceEntry).
let num = be32(&bytes, 12) as usize;
for i in 0..num {
let e = &bytes[0x10 + i * 16..0x10 + i * 16 + 16];
let tag = &e[0..4];
let data_off = be32(e, 4) as usize + 0x10;
let desc_size = be32(e, 8) as usize;
let name_off = be32(e, 12) as usize + 0x10;
if tag != b"XBG7" {
continue;
}
let name = {
let s = name_off.min(bytes.len());
let end = bytes[s..].iter().position(|&c| c == 0).unwrap_or(0) + s;
String::from_utf8_lossy(&bytes[s..end]).to_string()
};
if name != *want {
continue;
}
let desc = &bytes[data_off..(data_off + desc_size).min(bytes.len())];
println!("{name}: descriptor at 0x{data_off:X}, {} bytes", desc.len());
// markers: a == c*2, c%3==0, vertex count 32 bytes earlier
let mut rel = 0usize;
while rel + 8 <= desc.len() {
let (x, c) = (be32(desc, rel), be32(desc, rel + 4));
if c >= 3 && c % 3 == 0 && c < 400_000 && x == c * 2 && rel >= 32 {
let vc = be32(desc, rel - 32);
if (3..=200_000).contains(&vc) {
println!("\n marker @0x{rel:X}: {vc} verts / {c} indices — declaration triples after it:");
let mut r = rel + 8;
let mut stride = 0u32;
for _ in 0..12 {
if r + 12 > desc.len() {
break;
}
let (o, code, usage) = (be32(desc, r), be32(desc, r + 4), be32(desc, r + 8) >> 16);
if o == 0x00FF_0000 || code == 0xFFFF_FFFF || o > 0x1000 {
println!(" end marker @0x{r:X}: off=0x{o:X} code=0x{code:X}");
break;
}
println!(" off {o:>3} code 0x{:06X} usage {usage}", code & 0xFF_FFFF);
stride = stride.max(o + 4);
r += 12;
}
println!(" (min stride from element offsets: {stride})");
rel += 8;
continue;
}
}
rel += 4;
}
return;
}
eprintln!("resource not found");
}

View File

@@ -0,0 +1,80 @@
//! Calibrate the connectivity cap against the whole disc.
//!
//! `XBG7_EDGE_CAP` sets the cap; this reports, for one setting, how much
//! geometry decodes and how self-consistent it is across containers — the two
//! numbers any change to the cap has to trade off. Run it once per cap value.
use sylpheed_formats::mesh::Xbg7Model;
use std::collections::BTreeMap;
fn main() {
let dir = std::env::args().nth(1).expect("resource3d dir");
let mut files: Vec<_> = std::fs::read_dir(&dir)
.unwrap()
.flatten()
.map(|e| e.path())
.filter(|p| p.extension().and_then(|s| s.to_str()) == Some("xpr"))
.collect();
files.sort();
// name -> (verts, tris) -> spans seen, exactly as mesh_consistency_disc.rs.
let mut seen: BTreeMap<String, Vec<([i64; 3], usize, usize)>> = BTreeMap::new();
let (mut models, mut verts) = (0usize, 0usize);
for f in &files {
let Ok(bytes) = std::fs::read(f) else { continue };
for m in Xbg7Model::anchor_models_cancellable(&bytes, 0.0, &|| false) {
let (mut lo, mut hi) = ([f32::MAX; 3], [f32::MIN; 3]);
for s in &m.meshes {
for q in &s.positions {
for k in 0..3 {
lo[k] = lo[k].min(q[k]);
hi[k] = hi[k].max(q[k]);
}
}
}
if lo[0] == f32::MAX {
continue;
}
let v: usize = m.meshes.iter().map(|s| s.positions.len()).sum();
let t: usize = m.meshes.iter().map(|s| s.indices.len() / 3).sum();
models += 1;
verts += v;
if std::env::var("DUMP").is_ok() {
// Per-resource signature, so two cap settings can be diffed:
// a cap change that silently MOVES an existing anchor is the
// risk a coverage count cannot see.
println!(
"{}|{}|{v}|{t}|{}|{}|{}|{}",
f.file_name().unwrap().to_string_lossy(),
m.name,
m.meshes[0].vbuf_offset.unwrap_or(0),
(hi[0] - lo[0]).round() as i64,
(hi[1] - lo[1]).round() as i64,
(hi[2] - lo[2]).round() as i64
);
}
seen.entry(m.name.clone()).or_default().push((
[
(hi[0] - lo[0]).round() as i64,
(hi[1] - lo[1]).round() as i64,
(hi[2] - lo[2]).round() as i64,
],
v,
t,
));
}
}
let (mut shared, mut inconsistent) = (0usize, 0usize);
for (_, list) in &seen {
if list.len() < 2 || !list.iter().all(|e| e.1 == list[0].1 && e.2 == list[0].2) {
continue;
}
shared += 1;
if list.iter().any(|e| e.0 != list[0].0) {
inconsistent += 1;
}
}
println!(
"cap={} models={models} verts={verts} shared={shared} inconsistent={inconsistent}",
std::env::var("XBG7_EDGE_CAP").unwrap_or_else(|_| "library default".into())
);
}

View File

@@ -0,0 +1,106 @@
//! Does every part of a ship sit inside the envelope its siblings describe?
//!
//! `slab_screen` compared *scale* and could not see the `e106` slab. What the eye
//! used when the render exposed it was **relationship**: a blocky mass sitting
//! apart from the hull. This measures that — assemble a ship, and for each part
//! ask how far its world box protrudes beyond the box of all the OTHER parts,
//! relative to the ship's own size. A mis-anchored block sticks out; a real part,
//! however big, is part of the silhouette.
//!
//! Usage: envelope_screen <resource3d_dir> [protrusion_fraction]
use sylpheed_formats::mesh::Xbg7Model;
use sylpheed_formats::ship::{assemble_ship, ship_id_of};
use std::collections::{BTreeSet, HashSet};
fn main() {
let dir = std::env::args().nth(1).expect("resource3d dir");
let limit: f32 = std::env::args().nth(2).and_then(|s| s.parse().ok()).unwrap_or(0.35);
let mut files: Vec<_> = std::fs::read_dir(&dir)
.unwrap()
.flatten()
.map(|e| e.path())
.filter(|p| p.extension().and_then(|s| s.to_str()) == Some("xpr"))
.collect();
files.sort();
let mut flagged = 0usize;
for f in &files {
let Ok(bytes) = std::fs::read(f) else { continue };
let ids: BTreeSet<String> = Xbg7Model::anchor_models_cancellable(&bytes, 0.0, &|| false)
.iter()
.filter_map(|m| ship_id_of(&m.name).map(|s| s.to_string()))
.collect();
for id in &ids {
let placed = assemble_ship(&bytes, id, true);
if placed.len() < 3 {
continue;
}
let want: HashSet<String> = placed.iter().map(|p| p.resource.clone()).collect();
let models = Xbg7Model::models_named(&bytes, &want, &|| false);
// World box per placement.
let mut boxes: Vec<(String, [f32; 3], [f32; 3])> = Vec::new();
for p in &placed {
let Some(m) = models.iter().find(|m| m.name == p.resource) else { continue };
let (mut lo, mut hi) = ([f32::MAX; 3], [f32::MIN; 3]);
for s in &m.meshes {
for q in &s.positions {
let w = p.apply(*q);
for k in 0..3 {
lo[k] = lo[k].min(w[k]);
hi[k] = hi[k].max(w[k]);
}
}
}
if lo[0] != f32::MAX {
boxes.push((p.resource.clone(), lo, hi));
}
}
if boxes.len() < 3 {
continue;
}
for i in 0..boxes.len() {
// Envelope of every OTHER part.
let (mut lo, mut hi) = ([f32::MAX; 3], [f32::MIN; 3]);
for (j, b) in boxes.iter().enumerate() {
if i == j {
continue;
}
for k in 0..3 {
lo[k] = lo[k].min(b.1[k]);
hi[k] = hi[k].max(b.2[k]);
}
}
// Per-AXIS: a bow legitimately extends the ship along its long
// axis, so protrusion only means something measured against the
// envelope's size IN THAT AXIS. The e106 slab stuck ~1 500 out in
// Y where its siblings spanned ~800.
let mut worst = 0.0f32;
let mut worst_out = 0.0f32;
for k in 0..3 {
let size_k = hi[k] - lo[k];
if size_k <= 1.0 {
continue;
}
let out_k = (lo[k] - boxes[i].1[k]).max(boxes[i].2[k] - hi[k]).max(0.0);
if out_k / size_k > worst {
worst = out_k / size_k;
worst_out = out_k;
}
}
let (out, size) = (worst_out, 1.0f32);
let _ = size;
if worst > limit {
flagged += 1;
println!(
"{:<20} {:<22} protrudes {:>7.0} beyond its siblings ({:.0}% of the ship)",
f.file_name().unwrap().to_string_lossy(),
boxes[i].0,
out,
100.0 * worst
);
}
}
}
}
println!("{flagged} parts protrude more than {:.0}% of their ship's size", 100.0 * limit);
}

View File

@@ -0,0 +1,35 @@
//! Does a FILTERED decode agree with the full one?
//!
//! Distinct anchor assignment resolves collisions against the set of resources
//! being decoded — so `models_named` (a filtered subset, used by the ship
//! assembler and the viewer) can reach a different answer from a whole-container
//! decode. This measures that directly.
//! Usage: filter_consistency <container.xpr> [resource...]
use sylpheed_formats::mesh::Xbg7Model;
use std::collections::HashSet;
fn main() {
let a: Vec<String> = std::env::args().collect();
let bytes = std::fs::read(&a[1]).expect("container");
let full = Xbg7Model::anchor_models_cancellable(&bytes, 0.0, &|| false);
let names: Vec<String> = if a.len() > 2 {
a[2..].to_vec()
} else {
full.iter().map(|m| m.name.clone()).collect()
};
let mut differ = 0usize;
for n in &names {
let want: HashSet<String> = std::iter::once(n.clone()).collect();
let one = Xbg7Model::models_named(&bytes, &want, &|| false);
let (Some(f), Some(s)) = (full.iter().find(|m| &m.name == n), one.first()) else {
continue;
};
let off = |m: &Xbg7Model| m.meshes.first().and_then(|s| s.vbuf_offset).unwrap_or(0);
if off(f) != off(s) {
differ += 1;
if differ <= 10 {
println!("{n}: full decode at 0x{:x}, filtered at 0x{:x}", off(f), off(s));
}
}
}
println!("{differ} of {} resources decode differently when filtered", names.len());
}

View File

@@ -0,0 +1,27 @@
//! Where could a resource's index buffer be, given the vertex buffer a runtime
//! capture proves the engine drew from? Tests the anchor scan's adjacency
//! assumption against ground truth.
use sylpheed_formats::mesh::{debug_find_index_buffer, debug_resource_params};
fn main() {
let a: Vec<String> = std::env::args().collect();
let bytes = std::fs::read(&a[1]).unwrap();
for pair in a[2..].iter() {
let (name, off) = pair.split_once('@').unwrap();
let vb = usize::from_str_radix(off.trim_start_matches("0x"), 16).unwrap();
let params = debug_resource_params(&bytes, name);
let markers = params.as_ref().map(|(m, _)| m.clone()).unwrap_or_default();
println!("{name} @ 0x{vb:x} markers={markers:?}");
let hits = debug_find_index_buffer(&bytes, name, vb);
if hits.is_empty() {
println!(" no index buffer anywhere in the container validates this block");
}
let mut hits = hits;
hits.sort_by_key(|(_, d)| d.abs());
for (ib, d) in hits.iter().take(8) {
println!(" ib 0x{ib:x} vb-ib = {d} bytes");
}
if hits.len() > 8 {
println!("{} total", hits.len());
}
}
}

View File

@@ -0,0 +1,44 @@
//! Does the container hold an X-mirrored copy of a resource's decoded buffer?
//!
//! The twin invariant (see `twin_mirror_audit`) flags `…_01`/`…_02` pairs that
//! decode to unrelated geometry. If the mirror of one twin's buffer exists
//! somewhere else in the container, that offset is where the other twin belongs
//! and the pair is a mis-anchor; if it does not exist, the pair is simply not a
//! mirrored pair.
//!
//! Usage: find_mirror <container.xpr> <resource>...
use sylpheed_formats::mesh::Xbg7Model;
use std::collections::HashSet;
fn main() {
let a: Vec<String> = std::env::args().collect();
let bytes = std::fs::read(&a[1]).expect("container");
let want: HashSet<String> = a[2..].iter().cloned().collect();
let models = Xbg7Model::models_named(&bytes, &want, &|| false);
let be = |at: usize| f32::from_be_bytes(bytes[at..at + 4].try_into().unwrap());
for m in &models {
let pos: Vec<[f32; 3]> = m.meshes.iter().flat_map(|s| s.positions.clone()).take(8).collect();
if pos.len() < 8 {
continue;
}
let anchored = m.meshes[0].vbuf_offset.unwrap_or(0);
let (mut direct, mut mirror) = (Vec::new(), Vec::new());
for o in (0..bytes.len().saturating_sub(12 + 8 * 24)).step_by(4) {
for (flip, out) in [(1.0f32, &mut direct), (-1.0f32, &mut mirror)] {
if (0..8).all(|k| {
let at = o + k * 24;
(be(at) - flip * pos[k][0]).abs() <= 1e-4
&& (be(at + 4) - pos[k][1]).abs() <= 1e-4
&& (be(at + 8) - pos[k][2]).abs() <= 1e-4
}) {
out.push(o);
}
}
}
println!(
"{:<20} anchored 0x{anchored:x} direct copies {:x?} mirrored copies {:x?}",
m.name, direct, mirror
);
}
}

View File

@@ -7,10 +7,10 @@ fn main(){
let mut seen=std::collections::BTreeSet::new(); let mut shown=0;
println!("{} pilot-roster configs; distinct line-ups:", rosters.len());
for r in &rosters{
let key:String=r.pilots.iter().map(|(c,p)|format!("{c}:{p}")).collect::<Vec<_>>().join(",");
let key:String=r.pilots().iter().map(|(c,p)|format!("{c}:{p}")).collect::<Vec<_>>().join(",");
if seen.insert(key) && shown<8 {
shown+=1;
let flt:Vec<String>=r.pilots.iter().map(|(c,p)|format!("{c}={p}")).collect();
let flt:Vec<String>=r.pilots().iter().map(|(c,p)|format!("{c}={p}")).collect();
println!(" {}", flt.join(" "));
}
}

View File

@@ -0,0 +1,60 @@
//! Which gate stops the resources that never decode?
//! Usage: gate_histogram <resource3d_dir> [max_resources]
use sylpheed_formats::mesh::{debug_best_rejection, xbg7_resource_names, Xbg7Model};
use std::collections::{BTreeMap, HashSet};
fn main() {
let dir = std::env::args().nth(1).expect("resource3d dir");
let cap: usize = std::env::args().nth(2).and_then(|s| s.parse().ok()).unwrap_or(usize::MAX);
let mut files: Vec<_> = std::fs::read_dir(&dir)
.unwrap()
.flatten()
.map(|e| e.path())
.filter(|p| p.extension().and_then(|s| s.to_str()) == Some("xpr"))
.collect();
files.sort();
let mut hist: BTreeMap<usize, (usize, String)> = BTreeMap::new();
let mut done = 0usize;
for f in &files {
if done >= cap {
break;
}
let Ok(bytes) = std::fs::read(f) else { continue };
let names = xbg7_resource_names(&bytes);
if names.is_empty() {
continue;
}
let got: HashSet<String> = Xbg7Model::anchor_models_cancellable(&bytes, 0.0, &|| false)
.into_iter()
.map(|m| m.name)
.collect();
for n in names.iter().filter(|n| !got.contains(*n)) {
if done >= cap {
break;
}
done += 1;
if let Some((rank, why)) = debug_best_rejection(&bytes, n) {
let e = hist.entry(rank).or_insert((0, String::new()));
e.0 += 1;
if e.1.is_empty() {
e.1 = format!("{n}: {why}");
}
}
}
}
println!("furthest gate reached, over {done} resources that never decode:");
let label = |r: usize| match r {
0 => "no gate reached",
1 => "index out of range",
2 => "buffer not covered by indices",
3 => "degenerate / implausible positions",
4 => "connectivity (mean edge / diagonal)",
5 => "winding consistency",
9 => "grouped pool (different path)",
_ => "?",
};
for (r, (n, ex)) in &hist {
println!(" {:<38} {n:>5} e.g. {ex}", label(*r));
}
}

View File

@@ -0,0 +1,2 @@
fn main(){let a:Vec<String>=std::env::args().collect();let b=std::fs::read(&a[1]).unwrap();
for l in sylpheed_formats::mesh::debug_grouped_report(&b,&a[2],a[3].parse().unwrap()){println!("{l}")}}

View File

@@ -0,0 +1,32 @@
//! Dump a fingerprint of every decoded index run, so two decoder settings can be
//! diffed exactly. Usage: index_hash_dump <resource3d_dir>
use sylpheed_formats::mesh::Xbg7Model;
fn main() {
let dir = std::env::args().nth(1).expect("resource3d dir");
let mut files: Vec<_> = std::fs::read_dir(&dir)
.unwrap()
.flatten()
.map(|e| e.path())
.filter(|p| p.extension().and_then(|s| s.to_str()) == Some("xpr"))
.collect();
files.sort();
for f in &files {
let Ok(bytes) = std::fs::read(f) else { continue };
let w = f.file_name().unwrap().to_string_lossy().to_string();
for m in Xbg7Model::stage_models(&bytes) {
for (k, sm) in m.meshes.iter().enumerate() {
let mut h = 1469598103934665603u64;
for i in &sm.indices {
h = (h ^ *i as u64).wrapping_mul(1099511628211);
}
println!(
"{w} {} {k} vb={:?} n={} h={h:016x}",
m.name,
sm.vbuf_offset,
sm.indices.len()
);
}
}
}
}

View File

@@ -0,0 +1,134 @@
//! Is our index list one element late? A capture-free check of the `pad` choice.
//!
//! `anchor_pool_mesh` tries `ib = vb idx_count*2 pad` for `pad` in `0..=3`,
//! **pad 0 first and with the looser winding gate** (`XBG7_PAD0_CONSISTENCY`,
//! 0.70) than the pad ≥ 1 path (0.85). The 2026-08-13 index-byte comparison
//! against the runtime showed 17 draw batches whose captured indices equal our
//! decoded list shifted by exactly one element — all of them on buffers whose
//! real index data sits at pad 2. A one-element shift re-wires every triangle,
//! and it is invisible to every count-based metric (the count, the coverage and
//! the positions are all still right).
//!
//! This reproduces that finding from the container alone: for each resource it
//! reads the index run at pad 0 and at pad 2 and scores both on two properties a
//! correct triangle list has — **no degenerate triangles** (two equal indices)
//! and consistent winding against the stored normals.
//!
//! Usage: index_pad_check <container.xpr> [name-substring]
use sylpheed_formats::mesh::Xbg7Model;
fn be16(b: &[u8], at: usize) -> u32 {
((b[at] as u32) << 8) | b[at + 1] as u32
}
/// Degenerate triangles (a repeated index) and the fraction of triangles whose
/// face normal agrees with their vertices' stored normals.
fn score(idx: &[u32], pos: &[[f32; 3]], nrm: &[[f32; 3]]) -> (usize, f32, usize) {
let mut degen = 0usize;
let mut agree = 0usize;
let mut counted = 0usize;
for t in idx.chunks_exact(3) {
let (a, b, c) = (t[0] as usize, t[1] as usize, t[2] as usize);
if a == b || b == c || a == c {
degen += 1;
continue;
}
if a >= pos.len() || b >= pos.len() || c >= pos.len() {
continue;
}
let e1 = [pos[b][0] - pos[a][0], pos[b][1] - pos[a][1], pos[b][2] - pos[a][2]];
let e2 = [pos[c][0] - pos[a][0], pos[c][1] - pos[a][1], pos[c][2] - pos[a][2]];
let f = [
e1[1] * e2[2] - e1[2] * e2[1],
e1[2] * e2[0] - e1[0] * e2[2],
e1[0] * e2[1] - e1[1] * e2[0],
];
if nrm.len() <= a {
continue;
}
let n = nrm[a];
let dot = f[0] * n[0] + f[1] * n[1] + f[2] * n[2];
counted += 1;
if dot > 0.0 {
agree += 1;
}
}
let frac = if counted == 0 { 0.0 } else { agree as f32 / counted as f32 };
(degen, frac.max(1.0 - frac), counted)
}
fn main() {
let a: Vec<String> = std::env::args().collect();
let bytes = std::fs::read(&a[1]).expect("container");
let filter = a.get(2).cloned().unwrap_or_default();
let models = Xbg7Model::stage_models(&bytes);
println!(
"{:<22} {:>6} {:>6} {:<26} {:<26} verdict",
"resource", "verts", "idx", "pad 0 (what we decode)", "pad 2 (2 bytes earlier)"
);
let (mut better_p2, mut better_p0, mut tie) = (0usize, 0usize, 0usize);
let mut hist_ok: Vec<(usize, String, usize)> = Vec::new();
for m in &models {
if !filter.is_empty() && !m.name.contains(&filter) {
continue;
}
if m.meshes.len() != 1 {
continue; // single-block path only; grouped pools use another formula
}
let sm = &m.meshes[0];
let Some(vb) = sm.vbuf_offset else { continue };
let n = sm.indices.len();
if n < 6 || sm.normals.is_empty() || vb < n * 2 + 2 {
continue;
}
let read_at = |start: usize| -> Vec<u32> { (0..n).map(|k| be16(&bytes, start + k * 2)).collect() };
let l0 = read_at(vb - n * 2);
let l2 = read_at(vb - n * 2 - 2);
// Sanity: l0 must be what the decoder emitted, or the assumption that we
// took pad 0 is wrong for this resource and the comparison is meaningless.
let took_pad0 = l0 == sm.indices;
if !took_pad0 {
continue;
}
let (d0, c0, _) = score(&l0, &sm.positions, &sm.normals);
let (d2, c2, _) = score(&l2, &sm.positions, &sm.normals);
let max_i0 = l0.iter().max().copied().unwrap_or(0);
let max_i2 = l2.iter().max().copied().unwrap_or(0);
let v = sm.positions.len() as u32;
// A shifted list typically also overruns the pool by one vertex or leaves
// the last vertex unreferenced, so carry the coverage as evidence too.
let verdict = if d2 < d0 && c2 >= c0 {
better_p2 += 1;
"pad 2 is the real one"
} else if d0 < d2 || c0 > c2 {
better_p0 += 1;
"pad 0 fine"
} else {
tie += 1;
"indistinguishable"
};
if verdict == "pad 0 fine" {
hist_ok.push((d0, m.name.clone(), n));
}
if verdict != "pad 0 fine" || !filter.is_empty() {
println!(
"{:<22} {:>6} {:>6} degen {:>5} wind {:.3} max {:<5} degen {:>5} wind {:.3} max {:<5} {}",
m.name, v, n, d0, c0, max_i0, d2, c2, max_i2, verdict
);
}
}
println!("\npad 2 wins: {better_p2} · pad 0 wins: {better_p0} · indistinguishable: {tie}");
// Is "no degenerate triangle" a safe invariant for a correctly anchored
// block? Distribution over the resources where pad 0 is the better choice.
let zero = hist_ok.iter().filter(|(d, _, _)| *d == 0).count();
println!(
"of the {} pad-0-correct resources, {zero} have ZERO degenerate triangles",
hist_ok.len()
);
let mut worst: Vec<_> = hist_ok.iter().filter(|(d, _, _)| *d > 0).collect();
worst.sort_by_key(|(d, _, _)| std::cmp::Reverse(*d));
for (d, n, idx) in worst.iter().take(10) {
println!(" {n}: {d} degenerate of {} triangles", idx / 3);
}
}

View File

@@ -0,0 +1,90 @@
//! Where does each declared field sit inside the engine's live definition object?
//!
//! A live object carries no names, so the mapping has to be inferred: take a unit
//! whose disc record SETS a field, look for that value in the dumped words of the
//! matching live object, and keep the offsets that agree across several units.
//!
//! Run: live_offsets <disc-root> <live-dump.txt> <ID=0xVA> ...
//! e.g. … UN_f106_TCAF_Destroyer=0xbd3ee300 UN_f105_TCAF_Cruiser=0xbd40e200
use std::collections::BTreeMap;
use sylpheed_formats::idxd::IdxdObject;
use sylpheed_formats::pak::PakArchive;
fn main() {
let mut a = std::env::args().skip(1);
let disc = a.next().expect("disc root");
let dump = a.next().expect("live dump");
let pairs: Vec<(String, String)> = a
.filter_map(|s| s.split_once('=').map(|(i, v)| (i.to_string(), v.to_string())))
.collect();
// live: va -> offset -> f32
let mut live: BTreeMap<String, BTreeMap<usize, f32>> = BTreeMap::new();
let mut cur = String::new();
for line in std::fs::read_to_string(&dump).expect("dump").lines() {
if let Some(rest) = line.strip_prefix("=== ") {
cur = rest.trim().to_string();
continue;
}
let f: Vec<&str> = line.split_whitespace().collect();
if f.len() >= 4 && f[1].starts_with('+') {
if let (Ok(off), Ok(val)) =
(usize::from_str_radix(&f[1][1..], 16), f[3].parse::<f32>())
{
live.entry(cur.clone()).or_default().insert(off, val);
}
}
}
let pak = PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).expect("pak");
// key -> offset -> how many units agree
let mut votes: BTreeMap<String, BTreeMap<usize, usize>> = BTreeMap::new();
let mut units = 0usize;
for entry in pak.entries() {
let Ok(bytes) = pak.read(entry) else { continue };
let Ok(obj) = IdxdObject::parse(&bytes) else { continue };
let Some(id) = obj.get_raw("ID") else { continue };
let Some((_, va)) = pairs.iter().find(|(i, _)| i == id) else { continue };
let Some(words) = live.get(va) else { continue };
units += 1;
let mut numeric = 0usize;
let mut hit = 0usize;
for key in obj.tokens() {
let Some(v) = obj.get_f32(key) else { continue };
if !v.is_finite() || v == 0.0 {
continue; // zero matches everywhere and says nothing
}
numeric += 1;
let mut found = false;
for (off, w) in words {
if (*w - v).abs() <= v.abs() * 1e-6 {
*votes.entry(key.clone()).or_default().entry(*off).or_default() += 1;
found = true;
}
}
if found {
hit += 1;
}
}
eprintln!(" {id}: {numeric} numeric fields set on disc, {hit} found in the dumped window");
}
println!("{units} units correlated against their live objects\n");
println!("{:<28} {:>8} offsets agreeing (votes)", "field", "unique?");
for (key, offs) in &votes {
let best = offs.iter().max_by_key(|(_, n)| **n).unwrap();
if *best.1 < units.max(2) {
continue; // needs every correlated unit to agree
}
let list: Vec<String> = offs
.iter()
.filter(|(_, n)| **n == *best.1)
.map(|(o, n)| format!("+{o:03x}×{n}"))
.collect();
println!(
"{:<28} {:>8} {}",
key,
if list.len() == 1 { "unique" } else { "ambig" },
list.join(" ")
);
}
}

View File

@@ -0,0 +1,53 @@
//! How many copies of a captured buffer does a container hold?
//!
//! The twins showed that two resources decoding to one buffer can mean the
//! container really holds two (mirrored) copies and our scan found only one.
//! This asks that question for any draw: give it a `vbase`, and it reports every
//! offset whose leading vertices match the draw's dumped positions — directly, and
//! X-mirrored.
//!
//! Usage: locate_draw <container.xpr> <capture.log> <vbase-hex>...
use sylpheed_formats::ship_capture::{parse_capture, parse_drawlog};
fn main() {
let a: Vec<String> = std::env::args().collect();
let bytes = std::fs::read(&a[1]).expect("container");
let text = std::fs::read_to_string(&a[2]).expect("log");
let mut draws = parse_capture(&text);
if draws.is_empty() {
draws = parse_drawlog(&text);
}
let be = |at: usize| f32::from_be_bytes(bytes[at..at + 4].try_into().unwrap());
for want in &a[3..] {
let vb = u32::from_str_radix(want.trim_start_matches("0x"), 16).expect("hex vbase");
let Some(d) = draws.iter().find(|d| d.vbase == vb) else {
println!("vbase 0x{vb:08X}: not in this log");
continue;
};
let n = d.pos.len().min(8);
let mut direct = Vec::new();
let mut mirror = Vec::new();
for o in (0..bytes.len().saturating_sub(12 + 64 * 24)).step_by(4) {
for (flip, out) in [(1.0f32, &mut direct), (-1.0f32, &mut mirror)] {
let hit = (0..n).all(|k| {
let at = o + k * 24;
(be(at) - flip * d.pos[k][0]).abs() <= 1e-4
&& (be(at + 4) - d.pos[k][1]).abs() <= 1e-4
&& (be(at + 8) - d.pos[k][2]).abs() <= 1e-4
});
if hit {
out.push(o);
}
}
}
println!(
"vbase 0x{vb:08X} vcount={}: {} direct copy/copies {:x?}, {} mirrored {:x?}",
d.vcount,
direct.len(),
&direct[..],
mirror.len(),
&mirror[..]
);
}
}

View File

@@ -0,0 +1,86 @@
//! Which container should the next runtime capture aim at?
//!
//! The 2026-08-13 stage-05 capture showed that a mission keeps several containers
//! resident and that its OWN `Stage_SNN.xpr` is among them (the f101 ACROPOLIS was
//! drawn from `Stage_S05.xpr`), so a capture can be aimed by choosing the mission.
//! This ranks the targets: for every XBG7 resource on the disc it records the
//! containers where it decodes and the ones where it does not, then reports
//!
//! * resources that decode **nowhere** — the real gaps, where a capture is the
//! only ground truth left;
//! * per container, how many of those it holds (fly that mission);
//! * resources that miss in one container but decode in another — a defect worth
//! fixing, but the geometry is already recoverable, so a capture is not needed.
//!
//! Usage: miss_targets <resource3d_dir>
use sylpheed_formats::mesh::{xbg7_resource_names, Xbg7Model};
use std::collections::{BTreeMap, BTreeSet};
fn main() {
let dir = std::env::args().nth(1).expect("resource3d dir");
let mut files: Vec<_> = std::fs::read_dir(&dir)
.unwrap()
.flatten()
.map(|e| e.path())
.filter(|p| p.extension().and_then(|s| s.to_str()) == Some("xpr"))
.collect();
files.sort();
// resource -> (containers where it decodes, containers where it misses)
let mut ok: BTreeMap<String, BTreeSet<String>> = BTreeMap::new();
let mut miss: BTreeMap<String, BTreeSet<String>> = BTreeMap::new();
for f in &files {
let Ok(bytes) = std::fs::read(f) else { continue };
let where_ = f.file_name().unwrap().to_string_lossy().to_string();
let declared: BTreeSet<String> = xbg7_resource_names(&bytes).into_iter().collect();
if declared.is_empty() {
continue;
}
let decoded: BTreeSet<String> =
Xbg7Model::stage_models(&bytes).into_iter().map(|m| m.name).collect();
for n in &declared {
if decoded.contains(n) {
ok.entry(n.clone()).or_default().insert(where_.clone());
} else {
miss.entry(n.clone()).or_default().insert(where_.clone());
}
}
}
let never: BTreeMap<&String, &BTreeSet<String>> =
miss.iter().filter(|(n, _)| !ok.contains_key(*n)).collect();
let recoverable: Vec<&String> = miss.keys().filter(|n| ok.contains_key(*n)).collect();
println!("resources that decode NOWHERE: {}", never.len());
println!("resources that miss somewhere but decode elsewhere: {}\n", recoverable.len());
// Rank containers by how many never-decoding resources they hold.
let mut per_container: BTreeMap<&String, Vec<&String>> = BTreeMap::new();
for (n, wheres) in &never {
for w in wheres.iter() {
per_container.entry(w).or_default().push(n);
}
}
let mut ranked: Vec<_> = per_container.iter().collect();
ranked.sort_by_key(|(_, v)| std::cmp::Reverse(v.len()));
println!("container never-decoding resources it holds");
for (w, v) in ranked.iter().take(14) {
let sample: Vec<&str> = v.iter().take(6).map(|s| s.as_str()).collect();
println!("{:<26} {:>3} {}", w, v.len(), sample.join(", "));
}
// Exclusives: a never-decoding resource that only ONE container holds is only
// reachable through whatever loads that container.
let excl: Vec<(&&String, &&BTreeSet<String>)> =
never.iter().filter(|(_, w)| w.len() == 1).collect();
println!("\nof the never-decoding, {} live in exactly one container", excl.len());
let mut by_one: BTreeMap<&String, Vec<&String>> = BTreeMap::new();
for (n, w) in &excl {
by_one.entry(w.iter().next().unwrap()).or_default().push(n);
}
let mut b: Vec<_> = by_one.iter().collect();
b.sort_by_key(|(_, v)| std::cmp::Reverse(v.len()));
for (w, v) in b.iter().take(10) {
println!(" {:<24} {:>3} {}", w, v.len(), v.iter().take(5).map(|s| s.as_str()).collect::<Vec<_>>().join(", "));
}
}

View File

@@ -0,0 +1,27 @@
//! Regenerate `docs/re/captures/movie-subtitle-voice-map.csv` from the record
//! table. Keyed by cutscene SLOT, not by movie: three resupply movies are bound
//! by two slots each, and the old movie-keyed CSV silently dropped one of each.
use sylpheed_formats::{movie_manifest, IdxdObject, PakArchive};
fn main() {
let disc = std::env::var("SYLPHEED_DISC").expect("set SYLPHEED_DISC");
let arc = PakArchive::open(format!("{disc}/dat/tables.pak")).expect("tables.pak");
let manifest = arc
.entries()
.iter()
.filter_map(|e| arc.read(e).ok())
.find(|b| IdxdObject::is_idxd(b) && movie_manifest::is_manifest(b))
.expect("movie manifest");
println!("slot,movie,telop,subtitle,voicetrack");
for e in movie_manifest::parse(&manifest) {
println!(
"{},{}.wmv,{},{},{}",
e.slot,
e.movie,
e.telop.unwrap_or_default(),
e.subtitle.unwrap_or_default(),
e.voice_token.unwrap_or_default()
);
}
}

View File

@@ -0,0 +1,51 @@
//! Why does an element's sprite fail to resolve? Dump the two name spaces.
//!
//! `parse_build` resolves an element to a sprite by looking its DECLARED name up
//! in (a) the `.rat` record table, then (b) the `T8aD` child table. `pteff05.t32`
//! is in neither -- the `T8aD` it wants is registered as `8AX` -- so it resolves
//! to None and `compose` drops it without recording it as missing. This prints
//! both spaces, so the link between the two can be CHECKED rather than assumed.
//!
//! cargo run -p sylpheed-formats --example name_resolution -- <pak> <entry>
use sylpheed_formats::{pak, ui_layout};
fn main() {
let path = std::env::args().nth(1).expect("usage: name_resolution <pak> [entry]");
let want: Option<usize> = std::env::args().nth(2).and_then(|s| s.parse().ok());
let ar = pak::PakArchive::open(&path).expect("open pak");
let entries: Vec<_> = ar.entries().to_vec();
for (i, e) in entries.iter().enumerate() {
if want.is_some_and(|w| w != i) {
continue;
}
let Ok(bytes) = ar.read(e) else { continue };
let Some(build) = ui_layout::parse_build(&bytes) else { continue };
let unresolved: Vec<&ui_layout::Element> = build
.elements
.iter()
.filter(|el| el.sprite.is_none() && el.kind & 0x10 == 0)
.collect();
let claimed: std::collections::HashSet<&str> =
build.elements.iter().filter_map(|e| e.sprite.as_deref()).collect();
let unclaimed: Vec<&String> =
build.sprites.keys().filter(|k| !claimed.contains(k.as_str())).collect();
if want.is_none() && unresolved.is_empty() && unclaimed.is_empty() {
continue;
}
let un: Vec<&str> = unresolved.iter().map(|e| e.name.as_str()).collect();
let uc: Vec<String> = unclaimed
.iter()
.map(|k| format!("{k}({} B)", build.sprites[*k].1))
.collect();
println!(
"entry {i:3} {:2} elements UNRESOLVED {:?} UNCLAIMED {:?}",
build.elements.len(), un, uc
);
if want.is_some() {
for el in &build.elements {
let mark = if el.sprite.is_none() && el.kind & 0x10 == 0 { " <-- UNRESOLVED" } else { "" };
println!(" [{:2}] kind {:#06x} {:28} -> {:?}{mark}", el.index, el.kind, el.name, el.sprite);
}
}
}
}

View File

@@ -0,0 +1,20 @@
//! Dump a composite's scene nodes with their scale — the transform the static
//! assembler copies verbatim into a placement.
//! Usage: node_scale <container.xpr> <composite name>
use sylpheed_formats::mesh::scene_world_nodes;
fn main() {
let a: Vec<String> = std::env::args().collect();
let bytes = std::fs::read(&a[1]).expect("container");
for n in scene_world_nodes(&bytes, &a[2]) {
// Row norms of `m`: a rotation has all three at 1.0; anything else is a
// scale baked into the matrix, which `s` does not show.
let norm = |r: [f32; 3]| (r[0] * r[0] + r[1] * r[1] + r[2] * r[2]).sqrt();
println!(
"{:<26} s[{:7.3}{:7.3}{:7.3}] |m rows|[{:7.3}{:7.3}{:7.3}] t[{:9.1}{:9.1}{:9.1}]",
n.resource,
n.s[0], n.s[1], n.s[2],
norm(n.m[0]), norm(n.m[1]), norm(n.m[2]),
n.t[0], n.t[1], n.t[2]
);
}
}

View File

@@ -0,0 +1,129 @@
//! How much geometry on the disc was being read one index element late?
//!
//! The pad-scoring fix (2026-08-13) makes `anchor_pool_mesh` choose the pad whose
//! index run is cleanest instead of the first that validates. This measures the
//! blast radius: per container, how many single-block resources now decode from a
//! run that is NOT the old first-match (`pad 0`) one, and how many decoded runs
//! still contain degenerate triangles (the shift signature) afterwards.
//!
//! Usage: pad_shift_audit <resource3d_dir>
use sylpheed_formats::mesh::Xbg7Model;
fn be16(b: &[u8], at: usize) -> u32 {
((b[at] as u32) << 8) | b[at + 1] as u32
}
/// Fraction of triangles whose face normal agrees with the stored normals,
/// folded to `max(na, 1-na)` so both authored windings read as ≈1.
fn winding(idx: &[u32], pos: &[[f32; 3]], nrm: &[[f32; 3]]) -> f32 {
let (mut agree, mut n) = (0usize, 0usize);
for t in idx.chunks_exact(3) {
let (a, b, c) = (t[0] as usize, t[1] as usize, t[2] as usize);
if a == b || b == c || a == c || a.max(b).max(c) >= pos.len() || a >= nrm.len() {
continue;
}
let e1 = [pos[b][0] - pos[a][0], pos[b][1] - pos[a][1], pos[b][2] - pos[a][2]];
let e2 = [pos[c][0] - pos[a][0], pos[c][1] - pos[a][1], pos[c][2] - pos[a][2]];
let f = [
e1[1] * e2[2] - e1[2] * e2[1],
e1[2] * e2[0] - e1[0] * e2[2],
e1[0] * e2[1] - e1[1] * e2[0],
];
let nv = nrm[a];
n += 1;
if f[0] * nv[0] + f[1] * nv[1] + f[2] * nv[2] > 0.0 {
agree += 1;
}
}
if n == 0 {
return 1.0;
}
let fr = agree as f32 / n as f32;
fr.max(1.0 - fr)
}
fn degenerate(idx: &[u32]) -> usize {
idx.chunks_exact(3)
.filter(|t| t[0] == t[1] || t[1] == t[2] || t[0] == t[2])
.count()
}
fn main() {
let dir = std::env::args().nth(1).expect("resource3d dir");
let mut files: Vec<_> = std::fs::read_dir(&dir)
.unwrap()
.flatten()
.map(|e| e.path())
.filter(|p| p.extension().and_then(|s| s.to_str()) == Some("xpr"))
.collect();
files.sort();
let (mut total, mut moved, mut still_degen) = (0usize, 0usize, 0usize);
let mut weak = 0usize;
let mut weak_wind: Vec<(f32, f32, String)> = Vec::new();
let mut worst: Vec<(usize, String, String)> = Vec::new();
for f in &files {
let Ok(bytes) = std::fs::read(f) else { continue };
let where_ = f.file_name().unwrap().to_string_lossy().to_string();
let mut c_moved = 0usize;
for m in Xbg7Model::stage_models(&bytes) {
// Degeneracy is counted over EVERY sub-mesh (grouped pools included —
// they have their own pad choice); the pad-0 comparison below only
// applies to single-block resources, whose run starts at `vb - 2n`.
for sm in &m.meshes {
let d = degenerate(&sm.indices);
if d > 0 {
still_degen += 1;
worst.push((d, m.name.clone(), where_.clone()));
}
}
if m.meshes.len() != 1 {
continue;
}
let sm = &m.meshes[0];
let Some(vb) = sm.vbuf_offset else { continue };
let n = sm.indices.len();
if n < 6 || vb < n * 2 {
continue;
}
total += 1;
let pad0: Vec<u32> = (0..n).map(|k| be16(&bytes, vb - n * 2 + k * 2)).collect();
if pad0 != sm.indices {
moved += 1;
c_moved += 1;
// How strong was the evidence for moving? A pad-0 run with
// degenerate triangles is a decisive shift signature; one with
// none moved on winding alone, which is weaker.
if degenerate(&pad0) == 0 {
weak += 1;
// Record how far apart the two runs' winding is, to see
// whether these look like the same shift signature as the
// decisive cases (pad-0 winding drifting toward 0.5) or like
// noise between two equally clean readings.
let w0 = winding(&pad0, &sm.positions, &sm.normals);
let w2: f32 = {
let l2: Vec<u32> = (0..n).map(|k| be16(&bytes, vb - n * 2 - 2 + k * 2)).collect();
winding(&l2, &sm.positions, &sm.normals)
};
weak_wind.push((w0, w2, m.name.clone()));
}
}
}
if c_moved > 0 {
println!("{where_:<24} {c_moved} resources read from a non-pad-0 index run");
}
}
println!("\n{moved} of {total} single-block resources moved off the pad-0 run");
println!("{still_degen} decoded runs still contain a degenerate triangle");
println!("of the moved, {weak} had a pad-0 run with no degenerate triangle (moved on winding alone)");
let clear = weak_wind.iter().filter(|(w0, w2, _)| *w2 - *w0 > 0.15).count();
let close = weak_wind.iter().filter(|(w0, w2, _)| (*w2 - *w0).abs() <= 0.05).count();
println!(" of those: {clear} show the shift signature (pad-2 winding > pad-0 by >0.15), {close} are within 0.05 (noise)");
for (w0, w2, n) in weak_wind.iter().take(8) {
println!(" {n}: pad0 wind {w0:.3} -> pad2 {w2:.3}");
}
worst.sort_by_key(|(d, _, _)| std::cmp::Reverse(*d));
for (d, n, w) in worst.iter().take(15) {
println!(" {n} ({w}): {d} degenerate triangles");
}
}

View File

@@ -0,0 +1,136 @@
//! Does the DERIVED paint order reproduce the ones measured from the game?
//!
//! `compose` uses a measured order for the three builds that have one and falls
//! back to `derived_paint_order` (a sort on each sprite's layer key) everywhere
//! else. The doc comment claims the derived order "reproduces both measured
//! orders up to ties" — this checks that claim against all three, and says what
//! the ties actually cost.
//!
//! cargo run -p sylpheed-formats --example paint_order_audit -- <GP_TITLE.pak>
use sylpheed_formats::{pak, ui_layout};
fn measured(names: &[&str]) -> Option<(&'static str, Vec<usize>)> {
const TITLE: [&str; 24] = [
"ptlogo1.t32", "ptlogo2.t32", "ptlogo1.t32", "ptlogo2.t32", "ptlogo1.t32",
"ptlogo2.t32", "pteff01.t32", "ptlogo_tm.t32", "pteff00.prm", "ptbase2.t32",
"pteff04.t32", "ptloop01.rat", "ptloop02.rat", "pteff02.prm",
"ptlogo_back2eff1.t32", "ptlogo_back2eff2.t32", "ptlogo_back2eff3.t32",
"ptlogo_back2eff4.t32", "ptlogo_back2eff5.t32", "ptlogo_back2.t32",
"ptlogo_back2eff.t32", "ptcopyright.t32", "ptlogoall_eff.t32",
"ptlogoall_eff2.t32",
];
const SPLASH: [&str; 7] = [
"palogo_eff0.prm", "palogo_gamearts.t32", "palogo_gamearts_eff.t32",
"palogo_seta.t32", "palogo_seta_eff.t32", "palogo_anima.t32",
"palogo_anima_eff.t32",
];
const MENU: [&str; 16] = [
"pteff00.prm", "ptbase.t32", "pteff05.t32", "ptloop01.rat",
"ptloop02.rat", "pteff02.prm", "ptframe1.t32", "ptframe2.t32",
"pteff10.t32", "pteff12.t32", "ptbtn01.rat", "ptbtn02.rat",
"ptbtn03.rat", "ptbtn04.rat", "ptbtn05.rat", "ptmsg.t32",
];
if names == TITLE {
return Some(("title", vec![9,11,12,10,13,6,20,19,14,15,18,16,17,0,2,4,7,1,3,5,22,23,21,8]));
}
if names == SPLASH { return Some(("splash", vec![0,2,4,6,1,3,5])); }
if names == MENU {
return Some(("main menu", vec![1,3,4,2,5,8,9,6,7,15,10,11,12,13,14,0]));
}
None
}
fn main() {
let path = std::env::args().nth(1).expect("usage: paint_order_audit <pak>");
let ar = pak::PakArchive::open(&path).expect("open pak");
let mut checked = 0;
let entries: Vec<_> = ar.entries().to_vec();
for (i, e) in entries.iter().enumerate() {
let Ok(bytes) = ar.read(e) else { continue };
let Some(build) = ui_layout::parse_build(&bytes) else { continue };
let names: Vec<&str> = build.elements.iter().map(|e| e.name.as_str()).collect();
// Every build: how exposed is it to tie-breaking? A tie between
// OVERLAPPING elements is where a derived order can go visibly wrong.
let keys_all: Vec<u32> = build.elements.iter()
.map(|e| ui_layout::sprite_layer_key(&build, &bytes, e).unwrap_or(u32::MAX))
.collect();
let mut tie_pairs = 0;
for a in 0..keys_all.len() {
for b in (a + 1)..keys_all.len() {
if keys_all[a] == keys_all[b] && keys_all[a] != u32::MAX { tie_pairs += 1; }
}
}
// Of the tied pairs, how many OVERLAP? Only those can paint visibly
// differently under an arbitrary tie-break. Rect from the declared
// pivot (= half the sprite for a .t32) at the resting placement.
let rect = |e: &ui_layout::Element| -> Option<(i32,i32,i32,i32)> {
let kf = e.rest()?;
let (w, h) = ((e.pivot_x * 2) as i32, (e.pivot_y * 2) as i32);
if w == 0 || h == 0 { return None; }
Some((kf.x, kf.y, w, h))
};
let mut tie_overlap = 0;
for a in 0..keys_all.len() {
for b in (a + 1)..keys_all.len() {
if keys_all[a] != keys_all[b] || keys_all[a] == u32::MAX { continue; }
let (Some(ra), Some(rb)) = (rect(&build.elements[a]), rect(&build.elements[b]))
else { continue };
let ox = (ra.0 + ra.2).min(rb.0 + rb.2) - ra.0.max(rb.0);
let oy = (ra.1 + ra.3).min(rb.1 + rb.3) - ra.1.max(rb.1);
if ox > 0 && oy > 0 { tie_overlap += 1; }
}
}
let Some((label, want)) = measured(&names) else {
println!("entry {i:2} (no measured order) {} elements, {tie_pairs} tied pairs, \
{tie_overlap} of them OVERLAPPING", build.elements.len());
// Name them: these are the only pairs whose order can show.
for a in 0..keys_all.len() {
for b in (a + 1)..keys_all.len() {
if keys_all[a] != keys_all[b] || keys_all[a] == u32::MAX { continue; }
let (Some(ra), Some(rb)) = (rect(&build.elements[a]), rect(&build.elements[b]))
else { continue };
let ox = (ra.0 + ra.2).min(rb.0 + rb.2) - ra.0.max(rb.0);
let oy = (ra.1 + ra.3).min(rb.1 + rb.3) - ra.1.max(rb.1);
if ox > 0 && oy > 0 {
println!(" overlapping tie: [{a}] {} x [{b}] {} key {} rect {:?} / {:?} overlap {}x{}",
build.elements[a].name, build.elements[b].name, keys_all[a], ra, rb, ox, oy);
}
}
}
continue;
};
checked += 1;
let got = ui_layout::derived_paint_order(&build, &bytes);
let keys: Vec<u32> = build.elements.iter()
.map(|e| ui_layout::sprite_layer_key(&build, &bytes, e).unwrap_or(u32::MAX))
.collect();
let exact = got == want;
// How many adjacent pairs in the MEASURED order does derived get wrong,
// and of those, how many are between elements sharing a layer key (a
// tie the sort cannot resolve) versus a genuine key-order conflict?
let pos_got: Vec<usize> = {
let mut p = vec![0; got.len()];
for (r, &e) in got.iter().enumerate() { p[e] = r; }
p
};
let (mut inv, mut tied) = (0, 0);
for a in 0..want.len() {
for b in (a + 1)..want.len() {
let (x, y) = (want[a], want[b]);
if pos_got[x] > pos_got[y] {
inv += 1;
if keys[x] == keys[y] { tied += 1; }
}
}
}
println!("entry {i:2} {label:10} {} elements", want.len());
println!(" derived == measured : {}", if exact { "YES" } else { "NO" });
println!(" inverted pairs : {inv} (of which same-layer-key ties: {tied})");
if !exact {
println!(" measured: {want:?}");
println!(" derived : {got:?}");
println!(" keys : {keys:?}");
}
}
println!("\n{checked} build(s) with a measured order were checked");
}

View File

@@ -0,0 +1,13 @@
fn main(){
let a:Vec<String>=std::env::args().collect();
let bytes=std::fs::read(&a[1]).unwrap();
let filter=a.get(2).cloned().unwrap_or_default();
for n in sylpheed_formats::mesh::xbg7_resource_names(&bytes) {
if !filter.is_empty() && !n.contains(&filter) { continue }
if let Some((m,stride))=sylpheed_formats::mesh::debug_resource_params(&bytes,&n) {
let strides = sylpheed_formats::mesh::debug_decl_strides(&bytes, &n);
println!("{n:<28} decl-stride {stride} markers {:?} per-sub-mesh strides {:?}",
&m[..m.len().min(4)], strides);
}
}
}

View File

@@ -0,0 +1,52 @@
//! Does DRAWING NOTHING beat guessing, for an element with no held pose?
//!
//! A keyframe group is entry → hold → exit, and the exit ends invisible (on the
//! five port screens the final keyframe is invisible for 21/24, 8/16, 12/18, 2/3
//! and 6/7 elements). So the screen "as seen" is the HOLD — which is why
//! `rest_plateau` is the primary rule. An element with **no** plateau has no
//! hold, and `rest()` currently falls back to guessing an endpoint of a movement.
//!
//! This renders each screen twice — as-is, and with every plateau-less element
//! suppressed via `compose`'s `visible` mask — and correlates both against the
//! live capture. If suppression wins, the fallback should draw nothing.
//! Writes both composites as raw RGBA (`<out>/entryNN_{asis,suppressed}.raw`,
//! 1280x720) so the correlation is done outside — this crate has no image
//! decoder and the comparison is not worth a dependency.
use sylpheed_formats::{pak, ui_layout};
fn main() {
let pak_path = std::env::args().nth(1).expect("usage: <GP_TITLE.pak> <outdir> <entry>...");
let ar = pak::PakArchive::open(&pak_path).expect("open");
let entries: Vec<_> = ar.entries().to_vec();
let outdir = std::env::args().nth(2).expect("outdir");
std::fs::create_dir_all(&outdir).ok();
for spec in std::env::args().skip(3) {
let idx: usize = spec.parse().unwrap();
let bytes = ar.read(&entries[idx]).expect("read");
let Some(build) = ui_layout::parse_build(&bytes) else { continue };
// plateau-less = rest() had to guess: no two adjacent keyframes share a pose
// Default: suppress plateau-less elements. With SUPPRESS_SUBSTR set,
// suppress every element whose NAME contains it instead — used to test
// the entry→hold→exit model's prediction that the splash glows are all
// finished by the moment the logos are up.
let by_name = std::env::var("SUPPRESS_SUBSTR").ok();
let mask: Vec<bool> = build.elements.iter().map(|e| {
if let Some(sub) = &by_name {
return !e.name.to_lowercase().contains(sub.as_str());
}
let k = &e.keyframes;
(0..k.len().saturating_sub(1)).any(|i| {
k[i].fade == k[i+1].fade && k[i].scale_x == k[i+1].scale_x
&& k[i].scale_y == k[i+1].scale_y && k[i].x == k[i+1].x && k[i].y == k[i+1].y
})
}).collect();
let suppressed = mask.iter().filter(|m| !**m).count();
let opts = ui_layout::ComposeOptions::default();
let a = ui_layout::compose(&build, &bytes, opts, None);
let b = ui_layout::compose(&build, &bytes, opts, Some(&mask));
std::fs::write(format!("{outdir}/entry{idx:02}_asis.raw"), &a.rgba).unwrap();
std::fs::write(format!("{outdir}/entry{idx:02}_suppressed.raw"), &b.rgba).unwrap();
println!("entry {idx:2} {}x{} elements {:2} plateau-less suppressed {suppressed}",
a.width, a.height, build.elements.len());
}
}

View File

@@ -0,0 +1,29 @@
//! Raw token sequence around a key, for one record — the ground truth for
//! "is this field actually defaulted, or is our reader missing it?"
//! Run: pool_window <disc-root> <ID substring> <key>
use sylpheed_formats::idxd::IdxdObject;
use sylpheed_formats::pak::PakArchive;
fn main() {
let disc = std::env::args().nth(1).unwrap();
let want = std::env::args().nth(2).unwrap();
let key = std::env::args().nth(3).unwrap();
let pak = PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).unwrap();
for e in pak.entries() {
let Ok(b) = pak.read(e) else { continue };
let Ok(o) = IdxdObject::parse(&b) else { continue };
let Some(id) = o.get_raw("ID") else { continue };
if !id.contains(&want) { continue; }
let t = o.tokens();
println!("=== {id} get_f32({key}) = {:?}", o.get_f32(&key));
for (i, tok) in t.iter().enumerate() {
if tok == &key {
let lo = i.saturating_sub(6);
let hi = (i + 7).min(t.len());
for j in lo..hi {
println!(" [{j}]{} {:?}", if j == i { " <-- key" } else { " " }, t[j]);
}
println!();
}
}
}
}

View File

@@ -0,0 +1,6 @@
fn main() {
let a: Vec<String> = std::env::args().collect();
let bytes = std::fs::read(&a[1]).unwrap();
let vb: usize = a[3].parse().unwrap();
println!("{:?}", sylpheed_formats::mesh::debug_try_anchor(&bytes, &a[2], vb, 3));
}

View File

@@ -0,0 +1,118 @@
//! Scratch: confirm the `.rat` layout record offsets against a real screen pak.
//! Run: cargo run -p sylpheed-formats --example rat_inspect -- <GP_SCREEN.pak>
use sylpheed_formats::{pak::PakArchive, ratc, t8ad};
fn be32(b: &[u8], o: usize) -> u32 {
u32::from_be_bytes([b[o], b[o + 1], b[o + 2], b[o + 3]])
}
fn main() {
let path = std::env::args().nth(1).expect("pak path");
let ar = PakArchive::open(&path).expect("open pak");
println!("entries: {}", ar.len());
let mut decoded: Vec<Vec<u8>> = Vec::new();
for (i, e) in ar.entries().iter().enumerate() {
let d = ar.read(e).unwrap_or_default();
let magic = String::from_utf8_lossy(&d[..4.min(d.len())]).to_string();
let (nt, nr) = if ratc::is_ratc(&d) {
let k = ratc::parse(&d).unwrap_or_default();
(
k.iter().filter(|c| c.kind == "T8aD").count(),
k.iter()
.filter(|c| c.name.to_lowercase().ends_with(".rat"))
.count(),
)
} else {
(0, 0)
};
println!(
" entry[{:2}] {:>8} B magic={:4} t32={:2} rat={:2}",
i,
d.len(),
magic,
nt,
nr
);
decoded.push(d);
}
// Pick the entry with the most children as "build0".
let bi = (0..decoded.len())
.filter(|&i| ratc::is_ratc(&decoded[i]))
.max_by_key(|&i| ratc::parse(&decoded[i]).map(|k| k.len()).unwrap_or(0))
.unwrap();
println!("=> inspecting richest build: entry[{}]", bi);
let bundle = &decoded[bi];
let kids = ratc::parse(bundle).unwrap_or_default();
let t32: Vec<_> = kids.iter().filter(|c| c.kind == "T8aD").collect();
let rat: Vec<_> = kids
.iter()
.filter(|c| c.name.to_lowercase().ends_with(".rat"))
.collect();
println!(
"build0: {} children, {} t32 sprites, {} rat records",
kids.len(),
t32.len(),
rat.len()
);
for c in t32.iter().take(5) {
let b = &bundle[c.offset..(c.offset + c.size).min(bundle.len())];
if let Some(img) = t8ad::parse(b) {
println!(" T8aD {:28} {}x{}", c.name, img.width, img.height);
}
}
println!(" -- .rat records --");
for c in rat.iter().take(10) {
let r = &bundle[c.offset..(c.offset + c.size).min(bundle.len())];
if r.len() < 0x60 {
println!(" .rat {:20} (len {}, too short)", c.name, r.len());
continue;
}
let (dw, dh) = (be32(r, 0x18), be32(r, 0x1c));
let sprite = {
let s = &r[0x20..0x30.min(r.len())];
let end = s.iter().position(|&b| b == 0).unwrap_or(s.len());
String::from_utf8_lossy(&s[..end]).to_string()
};
let (pvx, pvy) = (be32(r, 0x50), be32(r, 0x54));
// scan for placement block [scaleX=100, scaleY=100, tint, X<dw, Y<dh]
let mut placement = None;
let mut o = 0x58;
while o + 20 <= r.len() {
if be32(r, o) == 100 && be32(r, o + 4) == 100 {
let (tint, x, y) = (be32(r, o + 8), be32(r, o + 12), be32(r, o + 16));
if x < dw && y < dh {
placement = Some((o, tint, x, y));
break;
}
}
o += 4;
}
println!(
" .rat {:22} {}x{} sprite={:16} pivot=({},{}) place={:x?}",
c.name, dw, dh, sprite, pvx, pvy, placement
);
}
// End-to-end: composite the build and write it out as a PNG.
if let Some(screen) = sylpheed_formats::ui_layout::compose_build(bundle, false) {
println!(
"compose: {}x{}, drew {} records: {:?}",
screen.width,
screen.height,
screen.drawn.len(),
screen.drawn
);
if let Some(out) = std::env::args().nth(2) {
// Dependency-free PPM (P6, RGB — alpha already composited over the backdrop).
let mut buf = format!("P6\n{} {}\n255\n", screen.width, screen.height).into_bytes();
for px in screen.rgba.chunks_exact(4) {
buf.extend_from_slice(&px[..3]);
}
std::fs::write(&out, buf).unwrap();
println!("wrote {out}");
}
} else {
println!("compose: build produced no image");
}
}

View File

@@ -0,0 +1,71 @@
//! Does a `.rat` leaf record decode with the same reader as a whole bundle?
//!
//! The port needs the position of `ptbtneff01.t32`, the focus ring, which is
//! declared *inside* the nested `ptbtn0Nf.rat` leaf and is therefore invisible
//! to anything that walks only a bundle's top-level elements.
//!
//! The leaf's first 32 bytes have the same shape as a bundle header --
//! `"RATC"`, `0x3c` declaration-entry size at `+4`, element count at `+20`,
//! design size at `+24`/`+28` -- so the hypothesis is that `parse_build` reads
//! it unchanged. The control is the BASE record `ptbtn0N.rat`, whose single
//! element's position is already known independently: the parent screen's
//! `screen info` reports `ptbtn01.rat` resting at (542,162).
//!
//! cargo run -p sylpheed-formats --example rat_leaf_placement -- <GP_TITLE.pak>
use sylpheed_formats::{pak, ratc, ui_layout};
fn main() {
let path = std::env::args().nth(1).expect("usage: … <pak>");
let ar = pak::PakArchive::open(&path).expect("open pak");
let entries: Vec<_> = ar.entries().to_vec();
for (ei, e) in entries.iter().enumerate() {
let Ok(bytes) = ar.read(e) else { continue };
let Some(kids) = ratc::parse(&bytes) else { continue };
// Only the title-family bundles carry ptbtn records.
if !kids.iter().any(|c| c.name == "ptbtn01f.rat") {
continue;
}
println!("=== pak entry {ei}");
for c in &kids {
if c.kind != "RATC" || !c.name.starts_with("ptbtn") {
continue;
}
let leaf = &bytes[c.offset..c.offset + c.size];
match ui_layout::parse_build(leaf) {
None => println!(" {:16} …parse_build says no", c.name),
Some(b) => {
println!(
" {:16} {}x{} {} element(s), fallback={}",
c.name,
b.design_w,
b.design_h,
b.elements.len(),
b.from_fallback
);
for el in &b.elements {
let r = el.rest();
println!(
" [{}] {:18} pivot ({:4},{:4}) rest ({:5},{:5}) kf {}",
el.index,
el.name,
el.pivot_x,
el.pivot_y,
r.map(|k| k.x).unwrap_or(-1),
r.map(|k| k.y).unwrap_or(-1),
el.keyframes.len()
);
for (i, k) in el.keyframes.iter().enumerate() {
println!(
" kf{i} t={:?} pos=({},{}) scale={}%,{}% a={} rot={} tint={:#010x}",
k.time, k.x, k.y, k.scale_x, k.scale_y,
k.fade >> 24, k.rotation_deg, k.tint
);
}
}
}
}
}
println!();
}
}

View File

@@ -0,0 +1,78 @@
//! Is a RATC child's name the printable run before its magic, or the `opt ` block?
//!
//! `ratc::parse` names each child by scanning backwards for the ASCII run that
//! precedes its magic. That is usually right, but it is a HEURISTIC, and the
//! real format states the name explicitly: immediately before each child sits
//!
//! "opt " | BE32 length | name | NUL | 3 bytes | <child magic>
//!
//! -- the same `opt ` block `ui_layout::opt_link` already decodes for a button's
//! focus link. When those 3 trailing bytes happen to be printable the heuristic
//! reads THEM as the name: the title screens' full-resolution background comes
//! out as `8AX` (bytes 38 41 58) instead of `pteff05.t32`, its element then
//! resolves to no sprite, and `compose` drops the screen's background.
//!
//! This compares the two readings for every RATC child in the paks given.
//!
//! cargo run -p sylpheed-formats --example ratc_child_names -- <pak>...
use sylpheed_formats::{pak, ratc};
/// The name stated by the `opt ` block that ends just before `at`.
fn opt_name(buf: &[u8], at: usize) -> Option<String> {
// The block is short; search back a bounded window for the tag.
let lo = at.saturating_sub(128);
let win = &buf[lo..at];
let pos = lo + win.windows(4).rposition(|w| w == b"opt ")?;
let len = u32::from_be_bytes(buf[pos + 4..pos + 8].try_into().ok()?) as usize;
if len == 0 || len > 64 || pos + 8 + len > at {
return None;
}
let s = String::from_utf8_lossy(&buf[pos + 8..pos + 8 + len]).to_string();
// It must be THIS child's block: name, NUL, then a short run to the magic.
if at - (pos + 8 + len) > 8 {
return None;
}
(!s.is_empty() && s.chars().all(|c| c.is_ascii_graphic())).then_some(s)
}
fn main() {
let mut children = 0usize;
let mut with_opt = 0usize;
let mut agree = 0usize;
let mut disagree: Vec<(String, usize, String, String)> = Vec::new();
for path in std::env::args().skip(1) {
let Ok(ar) = pak::PakArchive::open(&path) else { continue };
let short = path.rsplit('/').next().unwrap_or(&path).to_string();
let entries: Vec<_> = ar.entries().to_vec();
for (i, e) in entries.iter().enumerate() {
let Ok(bytes) = ar.read(e) else { continue };
let Some(kids) = ratc::parse(&bytes) else { continue };
for c in &kids {
children += 1;
let Some(o) = opt_name(&bytes, c.offset) else { continue };
with_opt += 1;
if o == c.name {
agree += 1;
} else {
disagree.push((short.clone(), i, c.name.clone(), o));
}
}
}
}
println!("RATC children scanned : {children}");
println!(" with an `opt ` block: {with_opt}");
println!(" scanned name AGREES : {agree}");
println!(" scanned name DIFFERS : {}", disagree.len());
let mut by_pair: std::collections::BTreeMap<(String, String), usize> = Default::default();
for (_, _, scanned, opt) in &disagree {
*by_pair.entry((scanned.clone(), opt.clone())).or_default() += 1;
}
println!("\ndistinct disagreements (scanned -> opt), with counts:");
for ((s, o), n) in &by_pair {
println!(" {s:24} -> {o:24} x{n}");
}
println!("\nfirst 20 occurrences:");
for (p, i, s, o) in disagree.iter().take(20) {
println!(" {p} entry {i:4} {s:20} -> {o}");
}
}

View File

@@ -0,0 +1,119 @@
//! The 60 RATC children that carry no `opt ` block — do they lack it, or is our
//! window too small?
//!
//! [`ratc::parse`] now prefers the name a child's own `opt ` block states, and
//! falls back to the old backwards printable-run scan when there is no block
//! within 128 bytes. That fallback fires for 60 of the disc's 18 002 children
//! (0.3 %), and the reach of the finding in `docs/re/structures/ratc-child-names.md`
//! stops there: "whether they genuinely lack the block or sit past the search
//! window is not established".
//!
//! This settles that. For every child with no accepted block it reports
//!
//! * whether an `opt ` tag exists at all further back, and how far;
//! * which guard rejected a tag that WAS in the window (length, gap, charset);
//! * the child's position in its bundle and its magic, in case the opt-less
//! ones are structurally distinct (e.g. always the first child);
//! * the raw bytes before the magic, so the fallback's answer can be judged.
//!
//! cargo run -p sylpheed-formats --example ratc_optless_children -- <pak>...
use sylpheed_formats::{pak, ratc};
/// Why a child has no accepted `opt ` name. Mirrors `ratc::opt_name`'s guards
/// one for one, so a rejection here is the same rejection the parser made.
#[derive(Debug, PartialEq, Eq, PartialOrd, Ord, Clone)]
enum Why {
/// No `opt ` tag in the 128-byte window, and none anywhere before it either.
NoTagAtAll,
/// No tag in the window, but one exists further back, this many bytes away.
TagBeyondWindow(usize),
/// Tag found, but its BE32 length is 0 or > 64.
BadLength(usize),
/// Tag found, name ends more than 8 bytes before the magic — a neighbour's.
GapTooBig(usize),
/// Tag found, the named bytes are not all printable ASCII.
NotGraphic,
}
/// The parser's own window.
const WINDOW: usize = 128;
fn classify(buf: &[u8], at: usize) -> Option<Why> {
let lo = at.saturating_sub(WINDOW);
let pos = match buf[lo..at].windows(4).rposition(|w| w == b"opt ") {
Some(p) => lo + p,
None => {
// Widen to the whole buffer before the child: is it merely far away?
return Some(match buf[..at].windows(4).rposition(|w| w == b"opt ") {
Some(p) => Why::TagBeyondWindow(at - p),
None => Why::NoTagAtAll,
});
}
};
let len = u32::from_be_bytes(buf.get(pos + 4..pos + 8)?.try_into().ok()?) as usize;
if len == 0 || len > 64 || pos + 8 + len > at {
return Some(Why::BadLength(len));
}
let gap = at - (pos + 8 + len);
if gap > 8 {
return Some(Why::GapTooBig(gap));
}
let s = String::from_utf8_lossy(&buf[pos + 8..pos + 8 + len]);
if s.is_empty() || !s.chars().all(|c| c.is_ascii_graphic()) {
return Some(Why::NotGraphic);
}
None // accepted — this child is not one of the 60
}
fn main() {
let mut total = 0usize;
let mut rows: Vec<(String, usize, usize, String, String, Why)> = Vec::new();
let mut first_child_of_bundle = 0usize;
for path in std::env::args().skip(1) {
let Ok(ar) = pak::PakArchive::open(&path) else { continue };
let short = path.rsplit('/').next().unwrap_or(&path).to_string();
let entries: Vec<_> = ar.entries().to_vec();
for (ei, e) in entries.iter().enumerate() {
let Ok(bytes) = ar.read(e) else { continue };
let Some(kids) = ratc::parse(&bytes) else { continue };
for (ci, c) in kids.iter().enumerate() {
total += 1;
let Some(why) = classify(&bytes, c.offset) else { continue };
if ci == 0 {
first_child_of_bundle += 1;
}
let lo = c.offset.saturating_sub(24);
let hex = bytes[lo..c.offset]
.iter()
.map(|b| format!("{b:02x}"))
.collect::<Vec<_>>()
.join(" ");
rows.push((short.clone(), ei, ci, c.name.clone(), hex, why));
}
}
}
println!("RATC children scanned : {total}");
println!(" with NO accepted `opt ` block: {}", rows.len());
println!(" ... of which are child #0 : {first_child_of_bundle}");
let mut by_why: std::collections::BTreeMap<String, usize> = Default::default();
for r in &rows {
let k = match &r.5 {
Why::TagBeyondWindow(_) => "TagBeyondWindow".to_string(),
Why::BadLength(_) => "BadLength".to_string(),
Why::GapTooBig(_) => "GapTooBig".to_string(),
other => format!("{other:?}"),
};
*by_why.entry(k).or_default() += 1;
}
println!("\nwhy, by cause:");
for (k, n) in &by_why {
println!(" {k:20} x{n}");
}
println!("\nevery occurrence (name is what the FALLBACK scan returned):");
for (p, ei, ci, name, hex, why) in &rows {
println!(" {p:28} entry {ei:4} child {ci:3} {name:24} {why:?}");
println!(" 24 bytes before the magic: {hex}");
}
}

View File

@@ -0,0 +1,60 @@
//! Which mission should the next unit harvest fly?
//!
//! Unit definitions are instantiated per stage, so coverage grows by visiting
//! missions — but only if the mission fields units we have not read yet. This
//! ranks every stage by how many of its roster units are missing from the
//! harvested CSV.
//!
//! Usage: roster_target <disc-root> <unit-runtime-fields.csv>
use sylpheed_formats::hash::name_hash;
use sylpheed_formats::idxd::IdxdObject;
use sylpheed_formats::pak::PakArchive;
use std::collections::{BTreeMap, BTreeSet};
fn main() {
let a: Vec<String> = std::env::args().collect();
let pak = PakArchive::open(format!("{}/dat/GP_MAIN_GAME_E.pak", a[1])).expect("pak");
let have: BTreeSet<String> = std::fs::read_to_string(&a[2])
.expect("csv")
.lines()
.skip(1)
.filter_map(|l| l.split(',').next().map(str::to_string))
.collect();
// Label each roster by STAGE: the TOC stores a hash of the original path, and
// the table names are known (`EnumUnit_SNN.tbl`), so hashing the candidates
// maps entries back to stages. Most rosters carry no `UN_S<NN>_` prop id, which
// is all `UnitRoster::stage` can infer from.
let mut by_hash: BTreeMap<u32, String> = BTreeMap::new();
// The TOC hashes a PATH, and these tables live under a directory — try the
// known schemes (see hash::TOC_NAME_SCHEMES) rather than the bare name.
for i in 1..=29 {
let stage = format!("S{i:02}");
for pre in ["", "unit\\", "battle\\", "stage\\", "enemy\\"] {
for suf in [".tbl", ""] {
by_hash.insert(name_hash(&format!("{pre}EnumUnit_{stage}{suf}")), stage.clone());
}
}
}
let mut rows: Vec<(usize, String, Vec<String>)> = Vec::new();
for e in pak.entries() {
let Some(stage) = by_hash.get(&e.name_hash).cloned() else { continue };
let Ok(b) = pak.read(e) else { continue };
let Ok(o) = IdxdObject::parse(&b) else { continue };
let mut units: Vec<String> = Vec::new();
for id in o.tokens().iter().filter(|s| s.starts_with("UN_")) {
let prop = id.contains("Asteroid") || id.contains("cmesh") || id.contains("_Box");
if !prop && !units.iter().any(|u| u == id) {
units.push(id.clone());
}
}
let missing: Vec<String> = units.iter().filter(|u| !have.contains(*u)).cloned().collect();
rows.push((missing.len(), stage, missing));
}
rows.sort_by_key(|(n, _, _)| std::cmp::Reverse(*n));
println!("units already harvested: {}\n", have.len());
println!("stage missing roster units not yet read");
for (n, stage, missing) in rows.iter().take(12) {
let s: Vec<&str> = missing.iter().take(6).map(|s| s.as_str()).collect();
println!("{stage:<8} {n:>5} {}", s.join(", "));
}
}

View File

@@ -0,0 +1,18 @@
//! RE probe: dump tables.pak screen-config records that mention a given token.
//! Usage: screen_configs <disc-root> <substring>
use sylpheed_formats::{idxd::IdxdObject, pak::PakArchive};
fn main(){
let a:Vec<String>=std::env::args().collect();
let needle=a.get(2).cloned().unwrap_or_else(||"CHALLENGE".into());
let arc=PakArchive::open(format!("{}/dat/tables.pak",a[1])).unwrap();
for (i,e) in arc.entries().iter().enumerate(){
let Ok(b)=arc.read(e) else{continue};
let Ok(o)=IdxdObject::parse(&b) else{continue};
let t=o.tokens();
if !t.iter().any(|s|s.to_lowercase().contains(&needle.to_lowercase())){continue}
let path=t.iter().find(|s|s.contains(".pak+")).cloned().unwrap_or_default();
if !path.is_empty() && !path.contains("+eng"){continue}
println!("\n=== entry #{i} schema {:08x} [{path}] {} tokens ===",o.schema_hash,t.len());
for (j,tok) in t.iter().enumerate(){println!(" {j:3} {tok}");}
}
}

View File

@@ -0,0 +1,111 @@
//! Dump a UI screen's full layout from one RATC bundle: the element declaration
//! table (what is drawn, in back-to-front order, with pivots and parent links)
//! plus the placement region that follows it (per element: a keyframe group of
//! scale / tint / X / Y).
//!
//! See docs/re/structures/ui-rat-layout.md. Run:
//! cargo run -p sylpheed-formats --example screen_layout -- <PAK> [0xHASH]
//! With no hash, the largest RATC entry in the pak is used.
use sylpheed_formats::pak::PakArchive;
fn be32(b: &[u8], o: usize) -> u32 {
u32::from_be_bytes([b[o], b[o + 1], b[o + 2], b[o + 3]])
}
fn main() {
let mut args = std::env::args().skip(1);
let pak = args.next().expect("usage: screen_layout <pak> [0xHASH]");
let want = args.next();
let arc = PakArchive::open(&pak).expect("open pak");
let bytes = match want {
Some(h) if h.starts_with("0x") => {
let h = u32::from_str_radix(h.trim_start_matches("0x"), 16).expect("hash");
arc.read_by_hash(h).expect("entry").expect("decompress")
}
Some(name) => arc.read_by_name(&name).expect("entry present").expect("decompress"),
None => arc
.entries()
.iter()
.filter_map(|e| arc.read(e).ok())
.filter(|b| b.len() > 16 && &b[..4] == b"RATC")
.max_by_key(|b| b.len())
.expect("no RATC entry"),
};
assert_eq!(&bytes[..4], b"RATC", "not a RATC bundle");
let count = be32(&bytes, 0x14) as usize;
let mut names = Vec::with_capacity(count);
let mut meta = Vec::with_capacity(count);
for i in 0..count {
let e = &bytes[0x20 + i * 60..0x20 + (i + 1) * 60];
let name = String::from_utf8_lossy(&e[..28])
.trim_end_matches('\0')
.trim_end_matches(char::from(0))
.to_string();
let parent = be32(e, 32);
let kind = be32(e, 40);
let (px, py) = (be32(e, 48), be32(e, 52));
names.push(name);
meta.push((parent, kind, px, py));
}
// Placement region: groups of (u32 element index, u32 keyframe count) then
// `count` 40-byte keyframes; the X/Y sit 12 bytes into the scale/tint block.
let mut pos = 0x20 + count * 60;
// X/Y are SIGNED: off-screen animation starts are negative (e.g. -516).
let mut placements: Vec<Vec<(i32, i32, u32)>> = vec![Vec::new(); count];
for _ in 0..count {
if pos + 8 > bytes.len() { break }
let idx = be32(&bytes, pos) as usize;
let frames = be32(&bytes, pos + 4) as usize;
if idx >= count || frames == 0 || frames > 4096 { break }
let first = pos + 28; // header + lead-in, verified on the pause bundles
let mut group = Vec::with_capacity(frames);
for k in 0..frames {
let blk = first + k * 40;
if blk + 20 > bytes.len() { break }
group.push((be32(&bytes, blk + 12) as i32, be32(&bytes, blk + 16) as i32, be32(&bytes, blk + 20)));
}
placements[idx] = group;
pos = first + frames * 40 - 20;
}
println!("{} elements", count);
println!("{:<3} {:<30} {:>7} {:>8} {:>12} {:>4} placement", "#", "element", "parent", "kind", "pivot", "kf");
for i in 0..count {
let (parent, kind, px, py) = meta[i];
let p = &placements[i];
// A group is an in → hold → out animation, so neither the first nor the
// last keyframe is where the element sits: report the MAX-DWELL one
// (longest gap to the next keyframe's time).
let shown = if p.is_empty() {
"".into()
} else if p.len() == 1 {
format!("({},{})", p[0].0, p[0].1)
} else {
let mut best = (0usize, 0u32);
for k in 0..p.len() - 1 {
let d = p[k + 1].2.saturating_sub(p[k].2);
if d > best.1 { best = (k, d) }
}
let r = p[best.0];
format!(
"rest ({},{}) t={}..{} [{}]",
r.0, r.1, r.2, p[best.0 + 1].2,
p.iter().map(|(x, y, t)| format!("{t}:{x},{y}")).collect::<Vec<_>>().join(" ")
)
};
println!(
"{:<3} {:<30} {:>7} {:>8} {:>12} {:>4} {}",
i,
names[i],
if parent == u32::MAX { "-".into() } else { parent.to_string() },
format!("{kind:#x}"),
format!("({px},{py})"),
p.len(),
shown
);
}
}

View File

@@ -0,0 +1,27 @@
//! Dump the menu's three SE cues as decodable `RIFF`s, to prove `se_wave_riff`
//! produces something ffmpeg actually accepts.
//!
//! cargo run -p sylpheed-formats --example se_wave_dump -- <outdir>
//! ffmpeg -i <outdir>/move.riff move.wav
use sylpheed_formats::media::{self, DirectorySource};
fn main() {
let out = std::env::args().nth(1).unwrap_or_else(|| "/tmp".into());
let disc = std::env::var("SYLPHEED_DISC").expect("SYLPHEED_DISC");
let src = DirectorySource::new(&disc);
for (name, off, pkts) in [("move", 0x1ec0usize, 4usize), ("back", 0x0ec0, 2), ("confirm", 0x5d6c0, 6)] {
match media::se_wave_riff(&src, "Static.slb", off, pkts, 1, 48000) {
Ok(riff) => {
let p = format!("{out}/{name}.riff");
std::fs::write(&p, &riff).unwrap();
println!("{p}: {} bytes ({pkts} packets at {off:#x})", riff.len());
}
Err(e) => println!("{name}: ERROR {e}"),
}
}
// The refusal path: a packet count the bank cannot satisfy.
match media::se_wave_riff(&src, "Static.slb", 0x1ec0, 1 << 20, 1, 48000) {
Ok(_) => println!("REFUSAL PATH FAILED — returned a short stream"),
Err(e) => println!("refusal path ok: {e}"),
}
}

View File

@@ -0,0 +1,58 @@
//! Does a shared resupply voice bank hold ONE line or SEVERAL takes?
//!
//! This decides whether the `samples/movie` lower bound on the sample rate is
//! real. The test is static and does not need audio judgement: the movies that
//! share a bank each have their OWN subtitle table. If those texts are identical
//! the bank plausibly holds one generic line; if they differ, one bank is
//! serving several distinct spoken lines.
use std::collections::BTreeMap;
use sylpheed_formats::{movie_subtitle, PakArchive};
fn main() {
let disc = std::env::var("SYLPHEED_DISC").expect("set SYLPHEED_DISC");
let lang = PakArchive::open(format!("{disc}/dat/movie/eng.pak")).expect("eng.pak");
let text = PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).expect("text pak");
let demo_text = movie_subtitle::build_demo_text(&text);
// bank → the movies bound to it, from the record table (see movie-subtitle-link).
let banks: BTreeMap<&str, Vec<&str>> = BTreeMap::from([
("VOICE_D_450", vec!["hokyu_LS_s02A", "hokyu_LS_s03A", "hokyu_LS_s06A"]),
(
"VOICE_D_451",
vec!["hokyu_LS_s09A", "hokyu_LS_s11A", "hokyu_LS_s15A", "hokyu_LS_s24A", "hokyu_LS_s27A"],
),
("VOICE_D_452", vec!["hokyu_DS_s02A", "hokyu_DS_s07A", "hokyu_DS_s08A", "hokyu_DS_s13A"]),
("VOICE_D_453", vec!["hokyu_LS_s02H", "hokyu_LS_s03H", "hokyu_LS_s06H", "hokyu_LS_s09H"]),
("VOICE_D_454", vec!["hokyu_DS_s07H", "hokyu_DS_s14H"]),
]);
for (bank, movies) in &banks {
println!("\n== {bank}");
let mut seen: BTreeMap<String, Vec<&str>> = BTreeMap::new();
for m in movies {
let cues = movie_subtitle::track_voice_cues(&lang, m);
let mut line = String::new();
for (demo, t) in &cues {
let txt = demo_text
.get(demo)
.map(|v| v.join(" "))
.unwrap_or_else(|| format!("<demo {demo}>"));
line.push_str(&format!("[{t:.2}] {txt} "));
}
if line.is_empty() {
line.push_str("<no cues>");
}
seen.entry(line.trim().to_string()).or_default().push(m);
}
for (line, ms) in &seen {
println!(" {:?}", ms);
println!(" {line}");
}
println!(
" -> {} distinct subtitle text(s) across {} movies",
seen.len(),
movies.len()
);
}
}

View File

@@ -0,0 +1,262 @@
//! Ask the runtime capture whether two resources that our decoder gives the
//! **same geometry** really are the same geometry.
//!
//! Our XBG7 anchor scan sometimes lands two different resource names on one
//! vertex buffer. Statics cannot separate "the container genuinely reuses a
//! buffer" from "the scan picked the wrong candidate" — but a capture can: the
//! engine uploads a buffer per resource and reuses one only 3.4 % of the time
//! (see docs/re/structures/xbg7-mesh.md), so a group of `k` resources our
//! decoder collapses onto one buffer should show up as `k` distinct `vbase`s
//! carrying that same vertex count and those same positions. Fewer means at
//! most one member of the group is really that geometry.
//!
//! Usage:
//! cargo run --release --example shared_vbase_check -- \
//! <Stage_SNN.xpr> <capture.log>...
use sylpheed_formats::mesh::Xbg7Model;
use sylpheed_formats::ship_capture::{parse_capture, parse_drawlog, CapturedDraw};
use std::collections::{BTreeMap, BTreeSet};
/// Quantised position key — the logs print 4 decimals, so compare at that scale.
fn key(p: [f32; 3]) -> (i64, i64, i64) {
(
(p[0] as f64 * 1e4).round() as i64,
(p[1] as f64 * 1e4).round() as i64,
(p[2] as f64 * 1e4).round() as i64,
)
}
/// Where in the container does a captured buffer live? POSITION is `f32×3` big
/// endian at vertex offset 0, so a draw's dumped positions are a literal byte
/// pattern: find the first one, then confirm the next few at a fixed stride.
/// This turns a capture into ground truth for a resource we mis-anchored.
fn locate_run(bytes: &[u8], pos: &[[f32; 3]]) -> Vec<(usize, usize)> {
if pos.len() < 4 {
return Vec::new();
}
// The log prints 4 decimals, so match on value with the printing tolerance
// rather than on bytes.
let be = |b: &[u8], at: usize| f32::from_be_bytes(b[at..at + 4].try_into().unwrap());
let same = |b: &[u8], at: usize, p: [f32; 3]| {
at + 12 <= b.len() && (0..3).all(|c| (be(b, at + c * 4) - p[c]).abs() <= 1e-4)
};
let mut out = Vec::new();
for o in (0..bytes.len().saturating_sub(12)).step_by(4) {
if !same(bytes, o, pos[0]) {
continue;
}
for stride in (12..=64).step_by(4) {
if (1..4).all(|k| same(bytes, o + k * stride, pos[k])) {
out.push((o, stride));
break;
}
}
}
out
}
fn main() {
let args: Vec<String> = std::env::args().collect();
if args.len() < 3 {
eprintln!("usage: shared_vbase_check <Stage_SNN.xpr> <capture.log>...");
std::process::exit(2);
}
let bytes = std::fs::read(&args[1]).expect("read container");
// Every draw from every log, keyed by vertex count.
// Keep the logs apart: each is its own emulator run, so a `vbase` only
// means something within one log.
let mut logs: Vec<(String, Vec<CapturedDraw>)> = Vec::new();
for log in args[2..].iter().filter(|a| !a.starts_with("--")) {
let text = std::fs::read_to_string(log).expect("read log");
let mut d = parse_capture(&text);
if d.is_empty() {
d = parse_drawlog(&text);
}
eprintln!("{log}: {} draws", d.len());
logs.push((log.rsplit('/').next().unwrap_or(log).to_string(), d));
}
// `--map`: is a draw's guest `vbase` just the container file offset plus a
// constant? If the container is uploaded contiguously it is — and then a
// capture names the exact offset of every buffer the engine drew, which is
// ground truth the anchor scan currently has to guess at.
if args.iter().any(|a| a == "--map") {
for (log, draws) in &logs {
let mut seen: BTreeSet<u32> = BTreeSet::new();
let mut delta: BTreeMap<i64, usize> = BTreeMap::new();
let mut unfound = 0usize;
for d in draws {
if d.pos.len() < 8 || d.vcount < 20 || !seen.insert(d.vbase) {
continue;
}
let at = locate_run(&bytes, &d.pos);
if at.is_empty() {
unfound += 1;
continue;
}
for (o, _) in at {
*delta.entry(d.vbase as i64 - o as i64).or_default() += 1;
}
}
let mut top: Vec<_> = delta.iter().collect();
top.sort_by_key(|(_, n)| std::cmp::Reverse(**n));
println!("{log}: {} distinct vbases located, {unfound} not in this container", seen.len() - unfound);
for (d, n) in top.iter().take(5) {
println!(" vbase - offset = 0x{:X} ×{n}", d);
}
}
return;
}
// Decode the container and group resources by the exact geometry they got.
let models = Xbg7Model::anchor_models_cancellable(&bytes, 0.0, &|| false);
// `--truth <base>`: with the container's guest load address (from `--map`),
// every draw names a file offset. Print it against the offset our anchor
// scan chose for each resource — a direct read-out of what we got wrong.
if let Some(a) = args.iter().find_map(|a| a.strip_prefix("--truth=")) {
let base = u32::from_str_radix(a.trim_start_matches("0x"), 16).expect("base");
// Where the anchor scan actually put each sub-mesh — exact, from the
// decoder, not inferred by searching for its leading vertices (the same
// leading run occurs at several offsets in a container, so a search
// cannot tell where a resource was anchored).
let mut ours: BTreeMap<usize, Vec<(String, usize)>> = BTreeMap::new();
for m in &models {
for sub in &m.meshes {
if let Some(o) = sub.vbuf_offset {
ours.entry(o).or_default().push((m.name.clone(), sub.positions.len()));
}
}
}
if args.iter().any(|a| a == "--anchors") {
println!("{:<12} where our decode put each resource", "file offset");
for (o, v) in &ours {
for (n, c) in v {
println!("0x{o:<10x} {n} ({c} verts)");
}
}
return;
}
let mut drawn: BTreeMap<usize, u32> = BTreeMap::new();
for (_, draws) in &logs {
for d in draws {
let off = d.vbase.wrapping_sub(base) as usize;
if off < bytes.len() && d.vcount >= 20 {
drawn.insert(off, d.vcount);
}
}
}
// Which resources have the drawn vertex count, wherever we put them?
// Right size + wrong place is a different bug from never finding it.
let mut by_count: BTreeMap<usize, Vec<String>> = BTreeMap::new();
for m in &models {
let n: usize = m.meshes.iter().map(|s| s.positions.len()).sum();
by_count.entry(n).or_default().push(m.name.clone());
}
// Is a capture-proven offset even a candidate the scan considers?
// Absent ⇒ the run scan misses it; present ⇒ selection picked another.
let starts: BTreeSet<usize> =
sylpheed_formats::mesh::debug_vertex_run_starts(&bytes, 24).into_iter().collect();
eprintln!("{} stride-24 candidate starts in this container", starts.len());
println!("{:<12} {:>7} {:>9} {:<44} our resources with that vcount", "file offset", "vcount", "candidate", "claimed by our decode");
for (off, vcount) in &drawn {
let who = ours
.get(off)
.map(|v| {
v.iter().map(|(n, c)| format!("{n}({c})")).collect::<Vec<_>>().join(", ")
})
.unwrap_or_else(|| "— NOBODY".into());
let same = by_count
.get(&(*vcount as usize))
.map(|v| v.join(", "))
.unwrap_or_else(|| "— none".into());
// Nearest resource we anchored at or before this offset — the
// likely owner of a buffer nobody claims.
let near = ours
.range(..=*off)
.next_back()
.map(|(o, v)| format!("{} @ -0x{:x}", v[0].0, off - o))
.unwrap_or_default();
let cand = if starts.contains(off) { "yes" } else { "NO" };
println!("0x{off:<10x} {vcount:>7} {cand:>9} {who:<44} {same:<34} {near}");
}
return;
}
let mut groups: BTreeMap<Vec<(i64, i64, i64)>, Vec<String>> = BTreeMap::new();
for m in &models {
let pos: Vec<(i64, i64, i64)> =
m.meshes.iter().flat_map(|s| s.positions.iter().copied()).map(key).collect();
if pos.is_empty() {
continue;
}
groups.entry(pos).or_default().push(m.name.clone());
}
let shared: Vec<_> = groups.iter().filter(|(_, n)| n.len() > 1).collect();
eprintln!(
"{} models, {} distinct geometries, {} shared by >1 resource",
models.len(),
groups.len(),
shared.len()
);
for (pos, names) in shared {
let vcount = pos.len() as u32;
// A draw belongs to this geometry if every dumped position is one of
// the decoded ones (the log dumps at most the first 64).
let want: BTreeSet<(i64, i64, i64)> = pos.iter().copied().collect();
println!("\n{} ({vcount} verts, {} resources)", names.join(""), names.len());
for (log, draws) in &logs {
let hits: Vec<&CapturedDraw> = draws.iter().filter(|d| d.vcount == vcount).collect();
let all: BTreeSet<u32> = hits.iter().map(|d| d.vbase).collect();
let matching: Vec<&&CapturedDraw> = hits
.iter()
.filter(|d| !d.pos.is_empty() && d.pos.iter().all(|p| want.contains(&key(*p))))
.collect();
let ok: BTreeSet<u32> = matching.iter().map(|d| d.vbase).collect();
// A buffer we do NOT match may still be the mirrored twin: same
// geometry with x negated. That is the case our assembler papers
// over with `apply_twin_mirrors`.
let mirrored: BTreeSet<u32> = hits
.iter()
.filter(|d| !ok.contains(&d.vbase))
.filter(|d| {
!d.pos.is_empty()
&& d.pos.iter().all(|p| want.contains(&key([-p[0], p[1], p[2]])))
})
.map(|d| d.vbase)
.collect();
println!(
" {log:32} draws={:<5} vbases@vcount={:<3} ours={} mirrored={} other={}",
hits.len(),
all.len(),
ok.len(),
mirrored.len(),
all.len() - ok.len() - mirrored.len()
);
// Where does each captured buffer live in the container? One
// representative draw per vbase is enough.
let mut done: BTreeSet<u32> = BTreeSet::new();
for d in &hits {
if d.pos.len() < 8 || !done.insert(d.vbase) {
continue;
}
let kind = if ok.contains(&d.vbase) {
"ours"
} else if mirrored.contains(&d.vbase) {
"mirror"
} else {
"other"
};
let at = locate_run(&bytes, &d.pos);
let shown: Vec<String> =
at.iter().take(4).map(|(o, s)| format!("0x{o:x}/stride{s}")).collect();
println!(
" vbase=0x{:08X} [{kind:6}] in container at: {}",
d.vbase,
if shown.is_empty() { "NOT FOUND".into() } else { shown.join(" ") }
);
}
}
}
}

View File

@@ -0,0 +1,61 @@
//! Does the legacy reader miss a field precisely when its value is shared?
//!
//! The corpus records four "sibling default" rules (`Size_Y` inherits `Size_X`,
//! `FCSRange` inherits `RadarRange`, …) used to recover values for units never
//! visited at runtime. If the rules are really a *deduplication artefact*, then
//! the legacy reader should report the field absent exactly when the two values
//! are equal on disc — and never otherwise.
use sylpheed_formats::{IdxdObject, PakArchive};
fn main() {
let disc = std::env::var("SYLPHEED_DISC").expect("set SYLPHEED_DISC");
let arc = PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).expect("pak");
let pairs = [
("Size_Y", "Size_X"),
("FCSRange", "RadarRange"),
("DefencePoint", "AttackVesselPoint"),
];
// legacy-absent x equal-on-disc, as a 2x2 table per pair.
let mut tally = [[[0usize; 2]; 2]; 3];
for e in arc.entries() {
let Ok(bytes) = arc.read(e) else { continue };
if !IdxdObject::is_idxd(&bytes) {
continue;
}
let Ok(obj) = IdxdObject::parse(&bytes) else { continue };
let Some(generic) = obj.record("Generic") else { continue };
if generic.get("Size_X").is_none() {
continue;
}
for (i, (field, sibling)) in pairs.iter().enumerate() {
let (Some(a), Some(b)) = (generic.get(field), generic.get(sibling)) else {
continue;
};
let legacy_absent = obj.get_f32(field).is_none();
tally[i][usize::from(legacy_absent)][usize::from(a == b)] += 1;
// The only cases where the rule PREDICTS WRONG: the reader misses
// the field and the two values differ, so "inherit the sibling"
// substitutes a number the disc contradicts.
if legacy_absent && a != b {
println!(
" rule-wrong: {:08x} {field} = {a} but {sibling} = {b} (ID {:?})",
e.name_hash,
obj.get_raw("ID")
);
}
}
}
println!("{:<14} {:>10} {:>10} {:>10} {:>10}", "pair", "seen+diff", "seen+eq", "MISS+diff", "MISS+eq");
for (i, (field, sibling)) in pairs.iter().enumerate() {
println!(
"{:<14} {:>10} {:>10} {:>10} {:>10}",
format!("{field}/{sibling}").chars().take(14).collect::<String>(),
tally[i][0][0],
tally[i][0][1],
tally[i][1][0],
tally[i][1][1]
);
}
println!("\n'MISS+diff' > 0 refutes 'the reader only misses shared values'.");
println!("'seen+eq' > 0 refutes 'a shared value is always invisible'.");
}

View File

@@ -0,0 +1,74 @@
//! Screen every ship family for a part that is wildly out of scale with its
//! siblings — the "slab" signature of a mis-anchored block.
//!
//! Rendering `e106` found such a part (`bdy_03`, 600×1600×998 beside parts of
//! ~250) that coverage, cross-container consistency, the capture oracle and the
//! twin invariant were all blind to. Eyeballing does not scale to 166 containers;
//! this does the same comparison numerically.
//!
//! Usage: slab_screen <resource3d_dir> [factor]
use sylpheed_formats::mesh::Xbg7Model;
use sylpheed_formats::ship::{is_base_part, ship_id_of};
use std::collections::BTreeMap;
fn main() {
let dir = std::env::args().nth(1).expect("resource3d dir");
let factor: f32 = std::env::args().nth(2).and_then(|s| s.parse().ok()).unwrap_or(4.0);
let mut files: Vec<_> = std::fs::read_dir(&dir)
.unwrap()
.flatten()
.map(|e| e.path())
.filter(|p| p.extension().and_then(|s| s.to_str()) == Some("xpr"))
.collect();
files.sort();
let mut flagged = 0usize;
for f in &files {
let Ok(bytes) = std::fs::read(f) else { continue };
let mut by_ship: BTreeMap<String, Vec<(String, f32)>> = BTreeMap::new();
for m in Xbg7Model::anchor_models_cancellable(&bytes, 0.0, &|| false) {
if !is_base_part(&m.name) {
continue;
}
let Some(id) = ship_id_of(&m.name) else { continue };
let (mut lo, mut hi) = ([f32::MAX; 3], [f32::MIN; 3]);
for s in &m.meshes {
for q in &s.positions {
for k in 0..3 {
lo[k] = lo[k].min(q[k]);
hi[k] = hi[k].max(q[k]);
}
}
}
if lo[0] == f32::MAX {
continue;
}
// Compare the part's SMALLEST axis span, not its diagonal. A long
// thin part (an antenna, a 100 000-unit tether on `f002`) is
// legitimately huge in one axis and would swamp a diagonal test; a
// mis-anchored block is bulky in all three, which is what the e106
// slab looked like (600×1600×998 beside siblings of ~250).
let thin = (hi[0] - lo[0]).min(hi[1] - lo[1]).min(hi[2] - lo[2]);
by_ship.entry(id.to_string()).or_default().push((m.name.clone(), thin));
}
for (id, parts) in &by_ship {
if parts.len() < 3 {
continue; // no meaningful median
}
let mut d: Vec<f32> = parts.iter().map(|(_, x)| *x).collect();
d.sort_by(|a, b| a.partial_cmp(b).unwrap());
let median = d[d.len() / 2];
for (name, diag) in parts {
if *diag > median * factor {
flagged += 1;
println!(
"{:<22} {name:<22} min-axis {diag:>8.0} vs ship median {median:>8.0} ({:.1}×)",
f.file_name().unwrap().to_string_lossy(),
diag / median
);
}
}
}
}
println!("{flagged} parts flagged at {factor}× the ship median");
}

View File

@@ -0,0 +1,113 @@
//! Does the `.slb` leading region decode as XMA1 under ANY plausible format?
//!
//! A first attempt at this hand-rolled the `fmt ` chunk and produced 0 PCM bytes
//! for all 36 combinations — including ones that should have matched the crate's
//! own working format. So it tested the chunk construction, not the hypothesis.
//! This version replicates `slb::synth_xma1_fmt`'s exact byte layout and starts
//! by reproducing its known result as a CONTROL; if the control does not match,
//! nothing below it means anything.
use std::io::Write;
use std::process::{Command, Stdio};
use sylpheed_formats::{movie_subtitle, slb, PakArchive};
/// Byte-for-byte `slb::synth_xma1_fmt` (private there). Note the second
/// parameter is a **channel mask**, not a stream count — mistaking it is what
/// made the first probe meaningless.
fn xma1_fmt(channels: u8, channel_mask: u16, rate: u32) -> Vec<u8> {
let mut fmt = Vec::with_capacity(40);
fmt.extend_from_slice(b"fmt ");
fmt.extend_from_slice(&32u32.to_le_bytes());
fmt.extend_from_slice(&0x0165u16.to_le_bytes()); // XMA1
fmt.extend_from_slice(&16u16.to_le_bytes()); // BitsPerSample
fmt.extend_from_slice(&0u16.to_le_bytes()); // EncodeOptions
fmt.extend_from_slice(&0u16.to_le_bytes()); // LargestSkip
fmt.extend_from_slice(&1u16.to_le_bytes()); // NumStreams
fmt.push(0); // LoopCount
fmt.push(3); // Version
fmt.extend_from_slice(&(rate * channels as u32 * 2).to_le_bytes());
fmt.extend_from_slice(&rate.to_le_bytes());
fmt.extend_from_slice(&0u32.to_le_bytes());
fmt.extend_from_slice(&0u32.to_le_bytes());
fmt.push(4); // SubframeData
fmt.push(channels);
fmt.extend_from_slice(&channel_mask.to_le_bytes());
fmt
}
fn riff(fmt: &[u8], data: &[u8]) -> Vec<u8> {
let mut out = Vec::with_capacity(12 + fmt.len() + 8 + data.len());
out.extend_from_slice(b"RIFF");
out.extend_from_slice(&((4 + fmt.len() + 8 + data.len()) as u32).to_le_bytes());
out.extend_from_slice(b"WAVE");
out.extend_from_slice(fmt);
out.extend_from_slice(b"data");
out.extend_from_slice(&(data.len() as u32).to_le_bytes());
out.extend_from_slice(data);
out
}
fn decode_bytes(r: &[u8]) -> usize {
let Ok(mut c) = Command::new("ffmpeg")
.args(["-v", "error", "-i", "pipe:0", "-f", "s16le", "pipe:1"])
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::null())
.spawn()
else {
return 0;
};
let buf = r.to_vec();
let mut stdin = c.stdin.take().unwrap();
std::thread::spawn(move || {
let _ = stdin.write_all(&buf);
});
c.wait_with_output().map(|o| o.stdout.len()).unwrap_or(0)
}
fn main() {
let disc = std::env::var("SYLPHEED_DISC").expect("set SYLPHEED_DISC");
let snd = PakArchive::open(format!("{disc}/dat/sound.pak")).expect("sound.pak");
let lang = PakArchive::open(format!("{disc}/dat/movie/eng.pak")).expect("eng.pak");
// The leading region decodes only as MONO. At channels=2 it yields 1792
// bytes; at channels=1, 203648 for VOICE_D_453. The decoded SAMPLE COUNT is
// independent of the declared rate (the rate only sets playback speed), so
// the subtitle cue can be used to solve for the real rate instead.
let bound = [
("hokyu_LS_s02A", 450u32),
("hokyu_LS_s09A", 451),
("hokyu_DS_s13A", 452),
("hokyu_LS_s02H", 453),
("hokyu_DS_s07H", 454),
];
println!(
"{:<16} {:>6} {:>9} {:>9} {:>10} {:>9} {:>12}",
"movie", "bank", "stereo B", "mono B", "samples", "cue s", "implied Hz"
);
for (movie, n) in bound {
let path = format!("eng\\etc\\VOICE_D_{n}.slb");
let Some(entry) = snd.find_by_name(&path) else { continue };
let bytes = snd.read(entry).expect("read");
let Some(first_riff) = bytes.windows(4).position(|w| w == b"RIFF") else { continue };
if first_riff <= slb::HEADERLESS_DATA_OFFSET {
println!("{movie:<16} {:>6} (no leading region)", format!("D_{n}"));
continue;
}
let lead = &bytes[slb::HEADERLESS_DATA_OFFSET..first_riff];
let stereo = decode_bytes(&riff(&xma1_fmt(2, 2, 48000), lead));
let mono = decode_bytes(&riff(&xma1_fmt(1, 0, 48000), lead));
let samples = mono / 2; // 16-bit mono
let cue = movie_subtitle::track_voice_cues(&lang, movie)
.iter()
.map(|(_, t)| *t)
.fold(0.0f32, f32::max);
let implied = if cue > 0.0 { samples as f32 / cue } else { f32::NAN };
println!(
"{movie:<16} {:>6} {stereo:>9} {mono:>9} {samples:>10} {cue:>9.2} {implied:>12.0}",
format!("D_{n}")
);
}
println!("\n'implied Hz' = decoded samples / the movie's last subtitle cue.");
println!("A consistent value near a standard rate is the real sample rate.");
}

View File

@@ -0,0 +1,31 @@
//! How many `.slb` banks would a "leading headerless stream" rule affect?
//!
//! The rule fires when the first `RIFF` sits at exactly
//! `HEADERLESS_DATA_OFFSET + n*XMA1_PACKET` with a non-zero leading region.
//! Before trusting it, count how many banks it would change — including the
//! `RT*` movie banks that already decode correctly today.
use sylpheed_formats::{slb, PakArchive};
fn main() {
let disc = std::env::var("SYLPHEED_DISC").expect("set SYLPHEED_DISC");
let snd = PakArchive::open(format!("{disc}/dat/sound.pak")).expect("sound.pak");
let (mut total, mut has_riff, mut hybrid, mut hybrid_nonzero) = (0, 0, 0, 0);
for e in snd.entries() {
let Ok(b) = snd.read(e) else { continue };
total += 1;
let Some(ri) = b.windows(4).position(|w| w == b"RIFF") else { continue };
has_riff += 1;
if ri > slb::HEADERLESS_DATA_OFFSET
&& (ri - slb::HEADERLESS_DATA_OFFSET) % slb::XMA1_PACKET == 0
{
hybrid += 1;
if b[slb::HEADERLESS_DATA_OFFSET..ri].iter().any(|x| *x != 0) {
hybrid_nonzero += 1;
}
}
}
println!(
"sound.pak entries {total}; with a RIFF {has_riff}; \
first RIFF at 1392+n*2048 {hybrid}; of those with a NON-ZERO leading region {hybrid_nonzero}"
);
}

View File

@@ -5,7 +5,7 @@ fn main(){
let sq=game_data::load_squadrons(&pak);
println!("{} squadron definitions", sq.len());
for s in sq.iter().filter(|s|s.side.as_deref()==Some("TCAF")).take(6){
let mem:Vec<String>=s.members.iter().map(|m|m.trim_start_matches("UN_").chars().take(18).collect()).collect();
println!(" {:8} {:22} {:24} x{} {:?}", s.id.clone().unwrap_or("?".into()), s.formation_id.clone().unwrap_or_default(), s.ai_id.clone().unwrap_or_default(), s.count.unwrap_or(0), mem);
let mem:Vec<String>=s.members.iter().map(|m|format!("{}/{}",m.unit.trim_start_matches("UN_").chars().take(18).collect::<String>(),m.pilot.clone().unwrap_or("·".into()))).collect();
println!(" {:8} {:22} {:24} x{} {:?}", s.id.clone(), s.formation_id.clone().unwrap_or_default(), s.ai_id.clone().unwrap_or_default(), s.count.unwrap_or(0), mem);
}
}

View File

@@ -0,0 +1,13 @@
use sylpheed_formats::mesh::{debug_resource_params, debug_vertex_run_starts};
fn main() {
let a: Vec<String> = std::env::args().collect();
let bytes = std::fs::read(&a[1]).unwrap();
let (markers, stride) = debug_resource_params(&bytes, &a[2]).expect("resource");
println!("{}: stride={stride} markers={markers:?}", a[2]);
let starts = debug_vertex_run_starts(&bytes, stride);
println!("{} candidate starts at stride {stride}", starts.len());
for off in &a[3..] {
let o = usize::from_str_radix(off.trim_start_matches("0x"), 16).unwrap();
println!(" 0x{o:x} in candidate list: {}", starts.binary_search(&o).is_ok());
}
}

View File

@@ -0,0 +1,7 @@
fn main(){let a:Vec<String>=std::env::args().collect();let b=std::fs::read(&a[1]).unwrap();
let stride:usize=a[2].parse().unwrap(); let want:usize=a[3].parse().unwrap();
let s=sylpheed_formats::mesh::debug_vertex_run_starts(&b,stride);
println!("{} candidate starts at stride {}", s.len(), stride);
println!("contains {:#x}: {}", want, s.contains(&want));
let near:Vec<String>=s.iter().filter(|&&o| o.abs_diff(want)<0x200).map(|o|format!("{o:#x}")).collect();
println!("nearby: {}", near.join(" "));}

View File

@@ -0,0 +1,34 @@
//! Per-sub-mesh vertex/index/coverage dump for one resource.
//! Usage: submesh_dump <container.xpr> <resource>...
use sylpheed_formats::mesh::{debug_resource_params, Xbg7Model};
use std::collections::HashSet;
fn main() {
let a: Vec<String> = std::env::args().collect();
let bytes = std::fs::read(&a[1]).expect("container");
let want: HashSet<String> = a[2..].iter().cloned().collect();
for m in Xbg7Model::models_named(&bytes, &want, &|| false) {
let markers = debug_resource_params(&bytes, &m.name).map(|(mk, _)| mk).unwrap_or_default();
println!("{}{} sub-meshes decoded, {} markers declared", m.name, m.meshes.len(), markers.len());
for (i, s) in m.meshes.iter().enumerate() {
let max_idx = s.indices.iter().max().copied().unwrap_or(0) as usize;
let (mut lo, mut hi) = ([f32::MAX; 3], [f32::MIN; 3]);
for q in &s.positions {
for k in 0..3 {
lo[k] = lo[k].min(q[k]);
hi[k] = hi[k].max(q[k]);
}
}
println!(
" #{i:<2} at 0x{:<9x} verts {:<6} idx {:<6} max_idx {:<6} slack {} span [{:.0} {:.0} {:.0}]",
s.vbuf_offset.unwrap_or(0),
s.positions.len(),
s.indices.len(),
max_idx,
s.positions.len() as i64 - 1 - max_idx as i64,
hi[0] - lo[0],
hi[1] - lo[1],
hi[2] - lo[2]
);
}
}
}

View File

@@ -0,0 +1,17 @@
//! Does the capture-proven offset validate for the resource that should own it?
use sylpheed_formats::mesh::debug_try_anchor;
fn main() {
let a: Vec<String> = std::env::args().collect();
let bytes = std::fs::read(&a[1]).unwrap();
// The production scan tries pads 0..=3; pass a bigger one to ask whether the
// block would validate at all with a wider index/vertex gap.
let max_pad: usize = std::env::var("MAX_PAD").ok().and_then(|v| v.parse().ok()).unwrap_or(3);
for pair in a[2..].iter() {
let (name, off) = pair.split_once('@').unwrap();
let off = usize::from_str_radix(off.trim_start_matches("0x"), 16).unwrap();
match debug_try_anchor(&bytes, name, off, max_pad) {
Some((v, i, pad)) => println!("{name:22} @ 0x{off:x} ACCEPTED v={v} idx={i} pad={pad}"),
None => println!("{name:22} @ 0x{off:x} rejected"),
}
}
}

View File

@@ -0,0 +1,119 @@
//! Do port/starboard twins decode to mirror images of each other?
//!
//! A runtime capture proved the container stores both halves of the `e106` hull
//! as separate, X-reflected buffers. That gives a **capture-free invariant**:
//! a `…_01`/`…_02` pair of equal vertex count should decode to geometry that is
//! an exact X-mirror — never to identical geometry (that is the collapse the
//! distinct-assignment fix targets), and never to something unrelated (that is a
//! mis-anchor no count-based metric can see).
//!
//! Usage: twin_mirror_audit <resource3d_dir>
use sylpheed_formats::mesh::Xbg7Model;
use std::collections::BTreeMap;
fn key(p: [f32; 3]) -> (i64, i64, i64) {
(
(p[0] * 1e3).round() as i64,
(p[1] * 1e3).round() as i64,
(p[2] * 1e3).round() as i64,
)
}
/// Elementwise equality with a tolerance. Truncating keys is too strict for a
/// mirrored pair: the halves are authored, not bit-negated, so they differ in
/// the last digits and an exact key test reports them as unrelated.
fn near(p: [f32; 3], q: [f32; 3]) -> bool {
(0..3).all(|c| (p[c] - q[c]).abs() <= 1e-3 * (1.0 + q[c].abs()))
}
fn main() {
let dir = std::env::args().nth(1).expect("resource3d dir");
let mut files: Vec<_> = std::fs::read_dir(&dir)
.unwrap()
.flatten()
.map(|e| e.path())
.filter(|p| p.extension().and_then(|s| s.to_str()) == Some("xpr"))
.collect();
files.sort();
let (mut same, mut mirrored, mut unrelated, mut related) = (0usize, 0usize, 0usize, 0usize);
let mut examples: Vec<String> = Vec::new();
for f in &files {
let Ok(bytes) = std::fs::read(f) else { continue };
let models = Xbg7Model::anchor_models_cancellable(&bytes, 0.0, &|| false);
let by_name: BTreeMap<&str, &Xbg7Model> =
models.iter().map(|m| (m.name.as_str(), m)).collect();
for m in &models {
let Some(stem) = m.name.strip_suffix("_01") else { continue };
let Some(t) = by_name.get(format!("{stem}_02").as_str()) else { continue };
let a: Vec<[f32; 3]> = m.meshes.iter().flat_map(|s| s.positions.clone()).collect();
let b: Vec<[f32; 3]> = t.meshes.iter().flat_map(|s| s.positions.clone()).collect();
if a.len() != b.len() || a.is_empty() {
continue;
}
let ident = a.iter().zip(&b).all(|(p, q)| near(*p, *q));
let mirr = a.iter().zip(&b).all(|(p, q)| near([-p[0], p[1], p[2]], *q));
// A pair that is neither may still be RELATED: mirrored on another
// axis, or the same point cloud in a different vertex order. Only a
// pair that is none of these is evidence of a mis-anchor.
let mirr_y = a.iter().zip(&b).all(|(p, q)| near([p[0], -p[1], p[2]], *q));
let mirr_z = a.iter().zip(&b).all(|(p, q)| near([p[0], p[1], -p[2]], *q));
let set = |v: &Vec<[f32; 3]>| {
let mut s: Vec<_> = v.iter().map(|p| key(*p)).collect();
s.sort_unstable();
s
};
let same_cloud = set(&a) == set(&b);
// …and the same point cloud after mirroring, for a pair whose
// halves are authored in different vertex order.
let mirrored_cloud = {
let am: Vec<[f32; 3]> = a.iter().map(|p| [-p[0], p[1], p[2]]).collect();
set(&am) == set(&b)
};
if !ident && !mirr && (mirr_y || mirr_z || same_cloud || mirrored_cloud) {
related += 1;
continue;
}
// Identical GEOMETRY is only a collapse if it also comes from the
// same buffer: a container may store a part twice unmirrored, and
// then two twins on two copies decode identically and correctly.
let shared_buffer = m.meshes[0].vbuf_offset.is_some()
&& m.meshes[0].vbuf_offset == t.meshes[0].vbuf_offset;
if ident && !shared_buffer {
related += 1;
continue;
}
if ident {
same += 1;
if examples.len() < 6 {
examples.push(format!(
"identical: {} / {}_02 in {}",
m.name,
stem,
f.file_name().unwrap().to_string_lossy()
));
}
} else if mirr {
mirrored += 1;
} else {
unrelated += 1;
if examples.len() < 6 {
examples.push(format!(
"unrelated: {} / {}_02 in {}",
m.name,
stem,
f.file_name().unwrap().to_string_lossy()
));
}
}
}
}
println!("twin pairs of equal vertex count: {}", same + mirrored + unrelated + related);
println!(" exact X-mirror (expected) : {mirrored}");
println!(" IDENTICAL, one buffer (collapse) : {same}");
println!(" related other way (Y/Z mirror, reordered): {related}");
println!(" unrelated (mis-anchor?) : {unrelated}");
for e in examples {
println!(" e.g. {e}");
}
}

View File

@@ -0,0 +1,56 @@
//! Which XBG7 resources never decode, and how big is that population?
//!
//! Coverage has been reported as "resources decoded" without a denominator. This
//! prints both, per container and in total, and names the misses so the gate
//! attribution (`why_rejected`) has a work list.
use sylpheed_formats::mesh::{debug_resource_params, xbg7_resource_names, Xbg7Model};
use std::collections::HashSet;
fn main() {
let dir = std::env::args().nth(1).expect("resource3d dir");
let show = std::env::args().nth(2).is_some();
let mut files: Vec<_> = std::fs::read_dir(&dir)
.unwrap()
.flatten()
.map(|e| e.path())
.filter(|p| p.extension().and_then(|s| s.to_str()) == Some("xpr"))
.collect();
files.sort();
let (mut total, mut decoded, mut no_decl) = (0usize, 0usize, 0usize);
let mut misses: Vec<String> = Vec::new();
for f in &files {
let Ok(bytes) = std::fs::read(f) else { continue };
let names = xbg7_resource_names(&bytes);
if names.is_empty() {
continue;
}
let got: HashSet<String> = Xbg7Model::anchor_models_cancellable(&bytes, 0.0, &|| false)
.into_iter()
.map(|m| m.name)
.collect();
for n in &names {
total += 1;
if got.contains(n) {
decoded += 1;
} else if debug_resource_params(&bytes, n).is_none() {
// No vertex declaration / no index markers: the anchor scan
// never even considers these, so they are a different question
// from "searched and not found".
no_decl += 1;
} else {
misses.push(format!("{}|{n}", f.file_name().unwrap().to_string_lossy()));
}
}
}
println!(
"XBG7 resources: {total} total, {decoded} decoded ({:.1}%), {no_decl} without a usable descriptor, {} searched-and-missed",
100.0 * decoded as f64 / total as f64,
misses.len()
);
if show {
for m in &misses {
println!("{m}");
}
}
}

View File

@@ -0,0 +1,31 @@
//! Every key/value a named unit record actually sets on disc.
//!
//! The live definition object in guest RAM has no field names; to find where a
//! field like `HQRatio` sits inside it, anchor on a unit whose disc record DOES
//! set that field and look for the value. This prints those anchors.
//! Run: unit_fields <disc-root> <substring of ID>...
use sylpheed_formats::idxd::IdxdObject;
use sylpheed_formats::pak::PakArchive;
fn main() {
let disc = std::env::args().nth(1).expect("disc root");
let want: Vec<String> = std::env::args().skip(2).collect();
let pak = PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).expect("pak");
for entry in pak.entries() {
let Ok(bytes) = pak.read(entry) else { continue };
let Ok(obj) = IdxdObject::parse(&bytes) else { continue };
let Some(id) = obj.get_raw("ID") else { continue };
if !want.iter().any(|w| id.contains(w.as_str())) {
continue;
}
println!("=== {id} (schema {:08x})", obj.schema_hash);
// The pool interleaves VALUE before KEY (see the module docs), so the
// pairs read (t[i] = value, t[i+1] = key).
let t = obj.tokens();
let mut i = 0;
while i + 1 < t.len() {
println!(" {:<30} {}", t[i + 1], t[i]);
i += 2;
}
}
}

View File

@@ -0,0 +1,33 @@
//! (HP, Size_X, Size_Y, Size_Z) for every craft unit and vessel on the disc.
//!
//! A live definition object in guest RAM carries HP at `+0x54` and the sizes at
//! `+0x30/34/38` but no name in its first words, so identity has to come from the
//! values themselves: this is the lookup table for that match.
//! Run: unit_signatures <disc-root>
use sylpheed_formats::game_data::{load_units, load_vessels};
use sylpheed_formats::PakArchive;
fn main() {
let disc = std::env::args().nth(1).expect("disc root");
let pak = PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).expect("pak");
println!("{:<34} {:>9} {:>8} {:>8} {:>8} kind", "id", "hp", "size_x", "size_y", "size_z");
for u in load_units(&pak) {
println!(
"{:<34} {:>9} {:>8} {:>8} {:>8} unit",
u.id.unwrap_or_default(),
u.hp.map(|v| v.to_string()).unwrap_or("".into()),
u.size_x.map(|v| v.to_string()).unwrap_or("".into()),
u.size_y.map(|v| v.to_string()).unwrap_or("".into()),
u.size_z.map(|v| v.to_string()).unwrap_or("".into()),
);
}
for v in load_vessels(&pak) {
println!(
"{:<34} {:>9} {:>8} {:>8} {:>8} vessel",
v.id.unwrap_or_default(),
v.hp.map(|x| x.to_string()).unwrap_or("".into()),
v.size_x.map(|x| x.to_string()).unwrap_or("".into()),
v.size_y.map(|x| x.to_string()).unwrap_or("".into()),
v.size_z.map(|x| x.to_string()).unwrap_or("".into()),
);
}
}

View File

@@ -0,0 +1,22 @@
use sylpheed_formats::idxd::IdxdObject;
use sylpheed_formats::pak::PakArchive;
fn main() {
let disc = std::env::args().nth(1).unwrap();
let want = std::env::args().nth(2).unwrap();
let pak = PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).unwrap();
for e in pak.entries() {
let Ok(b) = pak.read(e) else { continue };
let Ok(o) = IdxdObject::parse(&b) else { continue };
let Some(id) = o.get_raw("ID") else { continue };
if !id.contains(&want) { continue; }
println!("=== {id}");
let mut seen = std::collections::BTreeSet::new();
for k in o.tokens() {
if let Some(v) = o.get_f32(k) {
if v != 0.0 && seen.insert(k.clone()) {
println!(" {:<28} {:<12} 0x{:08x}", k, v, v.to_bits());
}
}
}
}
}

View File

@@ -0,0 +1,70 @@
//! Check the code-derived offset→field map against the live dump, then read out
//! the runtime value of every field the disc record leaves defaulted.
//! Run: verify_fieldmap <disc-root> <live-dump> <ID=0xVA>...
use std::collections::BTreeMap;
use sylpheed_formats::idxd::IdxdObject;
use sylpheed_formats::pak::PakArchive;
// Offsets read out of sub_82341A20's own key strings (stfs stores only).
// Offsets and names recovered from `sub_82341A20` itself: each key is built as
// `addi r4, r30, -N` (r30 = 0x82088f94, so the name is a string in the image),
// and the value lands in the first store into the object after the accessor
// call. 125 float-typed fields.
const MAP: &[(usize, &str)] = &[(48, "Size_X"), (52, "Size_Y"), (56, "Size_Z"), (64, "Color_R"), (68, "Color_G"), (72, "Color_B"), (80, "Size_Radius"), (84, "HP"), (88, "HQRatio"), (92, "ShieldRatio"), (96, "ThrusterRatio"), (116, "ResistanceToOptics"), (120, "ResistanceToShell"), (124, "ResistanceToExplosion"), (128, "ResistanceToPlayer"), (132, "ResistanceParalyze"), (156, "MinimumVelocity"), (160, "MaximumVelocity"), (164, "CruisingVelocity"), (168, "Acceleration"), (172, "Deceleration"), (176, "AV_PitchPlus_Max"), (180, "AV_PitchPlus_Min"), (184, "AA_PitchPlus_Max"), (188, "AA_PitchPlus_Min"), (192, "AV_PitchMinus_Max"), (196, "AV_PitchMinus_Min"), (200, "AA_PitchMinus_Max"), (204, "AA_PitchMinus_Min"), (208, "AV_Yaw_Max"), (212, "AV_Yaw_Min"), (216, "AA_Yaw_Max"), (220, "AA_Yaw_Min"), (224, "AV_Roll_Max"), (228, "AV_Roll_Min"), (232, "AA_Roll_Max"), (236, "AA_Roll_Min"), (248, "SideThrustVelocity_Max"), (252, "SideThrustAcceleration"), (256, "MaximumBank_Normal"), (260, "YawDragFactor"), (264, "PitchDragFactor"), (268, "RollDragFactor"), (272, "DragFactorThreshold"), (276, "ArterBurner_Vc"), (280, "ReverseThrust_Vc"), (284, "ArterBurner_Acc"), (288, "ReverseThrust_Acc"), (292, "AccPitchFactor"), (296, "DecPitchFactor"), (300, "AV_AxisMode_Max"), (304, "AV_AxisMode_Min"), (308, "AA_AxisMode_Max"), (312, "AA_AxisMode_Min"), (316, "PowerCutConsumeShield"), (320, "PowerCutDeceleration"), (324, "AB_ConsumeShield_Begin"), (328, "AB_ConsumeShield"), (332, "AB_AV_PitchPlus"), (336, "AB_AA_PitchPlus"), (340, "AB_AV_PitchMinus"), (344, "AB_AA_PitchMinus"), (348, "AB_AV_Yaw"), (352, "AB_AA_Yaw"), (356, "AB_AV_Roll"), (360, "AB_AA_Roll"), (368, "SideRoll_Time"), (372, "SideRoll_Length"), (380, "BarrelRoll_CountMinimum"), (384, "BarrelRoll_CountMaximum"), (388, "BarrelRoll_Time"), (392, "BarrelRoll_Radius"), (400, "TurnAttack_CutoffRatio"), (404, "TurnAttack_DoubleRatio"), (408, "CutoffTimeMin"), (412, "CutoffTimeMax"), (416, "TurnAttack_DoubleTimeMin"), (420, "TurnAttack_DoubleTimeMax"), (428, "Turn_AngularVelocity"), (432, "TurnAway_Time_Minimum"), (436, "TurnAway_Time_Maximum"), (444, "BoostAway_Time_Minimum"), (448, "BoostAway_Time_Maximum"), (456, "HoldPosition_LengthMin"), (460, "HoldPosition_LengthMax"), (464, "HoldPosition_MinimumTime"), (468, "HoldPosition_MaximumTime"), (472, "HoldPosition_SideRatio"), (476, "HoldPosition_BackRatio"), (480, "HoldPosition_CutoffRatio"), (484, "HoldPosition_CancelTime"), (492, "Slalom_CutoffRatio"), (496, "Slalom_TurnCount_Min"), (500, "Slalom_TurnCount_Max"), (508, "Through_CutoffRatio"), (512, "Through_AngleMinimum"), (516, "Through_AngleMaximum"), (520, "Through_Time1Max"), (524, "Through_Time1Min"), (528, "Through_Time2Max"), (532, "Through_Time2Min"), (536, "Through_LengthMinimum"), (540, "Through_LengthMaximum"), (548, "SolidCutoff_Ratio"), (552, "SolidCutoff_LengthMin"), (556, "SolidCutoff_LengthMax"), (560, "HomingResistAdjustment"), (564, "UsingChaffRatio"), (568, "MaxValue"), (572, "ChargeDelay"), (576, "ChargeDelay_Break"), (580, "ChargeSpeed"), (584, "Delay"), (588, "DelayAdjustment"), (624, "DestroyMotionTime"), (628, "DryMass"), (632, "GrossMass"), (664, "LowerHPThresholdRatio"), (668, "SELength"), (672, "RadarRange"), (676, "FCSRange"), (680, "FiringRange"), (692, "AttackVesselPoint"), (696, "AttackCraftPoint"), (700, "DefencePoint")];
fn main() {
let mut a = std::env::args().skip(1);
let disc = a.next().unwrap();
let dump = a.next().unwrap();
let pairs: Vec<(String, String)> =
a.filter_map(|s| s.split_once('=').map(|(i, v)| (i.into(), v.into()))).collect();
let mut live: BTreeMap<String, BTreeMap<usize, f32>> = BTreeMap::new();
let mut cur = String::new();
for line in std::fs::read_to_string(&dump).unwrap().lines() {
if let Some(r) = line.strip_prefix("=== ") { cur = r.trim().into(); continue; }
let f: Vec<&str> = line.split_whitespace().collect();
// dump columns: addr +off hex u32 f32 -- the float is f[4], not f[3]
if f.len() >= 5 && f[1].starts_with('+') {
if let (Ok(o), Ok(v)) = (usize::from_str_radix(&f[1][1..], 16), f[4].parse::<f32>()) {
live.entry(cur.clone()).or_default().insert(o, v);
}
}
}
let pak = PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).unwrap();
let (mut agree, mut disagree) = (0, 0);
let mut defaults: BTreeMap<String, Vec<(String, f32)>> = BTreeMap::new();
for e in pak.entries() {
let Ok(b) = pak.read(e) else { continue };
let Ok(o) = IdxdObject::parse(&b) else { continue };
let Some(id) = o.get_raw("ID") else { continue };
let Some((_, va)) = pairs.iter().find(|(i, _)| i == id) else { continue };
let Some(w) = live.get(va) else { continue };
for (off, key) in MAP {
let Some(got) = w.get(off) else { continue };
match o.get_f32(key) {
Some(want) => {
// Angle fields are stored in RADIANS at runtime and degrees on
// disc, so a match is either the raw value or its conversion.
let rad = want * std::f32::consts::PI / 180.0;
let ok = (want - got).abs() <= want.abs() * 1e-4
|| (rad - got).abs() <= rad.abs() * 1e-4;
if ok { agree += 1 } else {
disagree += 1;
println!(" MISMATCH {id} {key}: disc {want}, live +{off} = {got}");
}
}
None => defaults.entry((*key).into()).or_default().push((id.into(), *got)),
}
}
}
println!("\nmap check: {agree} fields agree with the disc, {disagree} disagree\n");
println!("runtime value where the disc record DEFAULTS the field:");
for (key, hits) in &defaults {
let vals: Vec<String> = hits.iter().map(|(_, v)| format!("{v}")).collect();
let uniq: std::collections::BTreeSet<&String> = vals.iter().collect();
println!(" {:<22} {:<28} ({} units)", key,
uniq.iter().map(|s| s.as_str()).collect::<Vec<_>>().join(", "), hits.len());
}
}

View File

@@ -0,0 +1,19 @@
//! Dump each resupply voice bank's sub-waves to `.xma` RIFFs for decoding.
use sylpheed_formats::{slb, PakArchive};
fn main() {
let disc = std::env::var("SYLPHEED_DISC").expect("set SYLPHEED_DISC");
let out = std::env::var("OUT_DIR_X").unwrap_or_else(|_| "/tmp/voicebanks".into());
std::fs::create_dir_all(&out).unwrap();
let snd = PakArchive::open(format!("{disc}/dat/sound.pak")).expect("sound.pak");
for n in 450..=454 {
let path = format!("eng\\etc\\VOICE_D_{n}.slb");
let Some(entry) = snd.find_by_name(&path) else { continue };
let bytes = snd.read(entry).expect("read");
for (i, r) in slb::to_xma_riffs(&bytes).iter().enumerate() {
let f = format!("{out}/VOICE_D_{n}_{i}.xma");
std::fs::write(&f, r).unwrap();
println!("{f} {} bytes", r.len());
}
}
}

View File

@@ -0,0 +1,85 @@
//! How many sub-waves does each resupply voice bank hold?
//!
//! The corpus records `VOICE_D_452` as the binding the game rejected in-game
//! ("wrong recording"), and separately notes that `VOICE_D_453`/`454` decode to
//! 0.14 s / 0.43 s — "far too short for the spoken line". Both observations are
//! explained if these banks are multi-sub-wave and the extractor plays only the
//! first. This prints the shape so that stops being a guess.
use sylpheed_formats::{slb, PakArchive};
fn main() {
let disc = std::env::var("SYLPHEED_DISC").expect("set SYLPHEED_DISC");
let snd = PakArchive::open(format!("{disc}/dat/sound.pak")).expect("sound.pak");
println!(
"{:<14} {:>7} {:>6} {:>6} {:>7} sub-wave data sizes",
"bank", "bytes", "RIFFs", "waves", "cover"
);
for n in 450..=454 {
for dir in ["etc", "Voice", "Movie"] {
let path = format!("eng\\{dir}\\VOICE_D_{n}.slb");
let Some(entry) = snd.find_by_name(&path) else { continue };
let bytes = snd.read(entry).expect("read");
let riffs = slb::to_xma_riffs(&bytes);
let sizes: Vec<usize> = riffs.iter().map(|r| r.len()).collect();
// How many RIFF magics does the bank actually contain, versus how
// many sub-waves the walker recovered? A gap means the walk stops
// early, and the missing bytes are the missing audio.
let magics = bytes.windows(4).filter(|w| *w == b"RIFF").count();
let covered: usize = sizes.iter().sum();
// What are the UNCOVERED bytes? If the tail past the last data
// chunk is all zero it is padding and the short duration is real;
// if it is high-entropy it is audio the parse is throwing away.
let last = bytes
.windows(4)
.rposition(|w| w == b"data")
.map(|i| {
let sz = u32::from_le_bytes(bytes[i + 4..i + 8].try_into().unwrap()) as usize;
(i + 8 + sz).min(bytes.len())
})
.unwrap_or(0);
// Where does the RIFF structure START? If it begins far into the
// file, the uncovered bytes are a leading region the parse skips,
// not a missed sub-wave.
let first_riff = bytes.windows(4).position(|w| w == b"RIFF").unwrap_or(0);
let datas = bytes.windows(4).filter(|w| *w == b"data").count();
// Is the leading region padding, or content? Padding is nearly all
// zero and uses few distinct byte values.
let head = &bytes[..first_riff];
let head_zero = head.iter().filter(|b| **b == 0).count();
let head_distinct = {
let mut seen = [false; 256];
for b in head {
seen[*b as usize] = true;
}
seen.iter().filter(|s| **s).count()
};
let tail = &bytes[last..];
let zeros = tail.iter().filter(|b| **b == 0).count();
// Dump the first bytes of the leading region so its structure is
// visible rather than guessed at.
let hex: String = head
.iter()
.take(48)
.map(|b| format!("{b:02x}"))
.collect::<Vec<_>>()
.join(" ");
println!(" head[0..48] {hex}");
println!(
"{:<14} {:>7} {:>6} {:>6} {:>6.1}% 1st RIFF @{:>6} data chunks {} head {:>5.1}% zero/{:>3} distinct tail {:>5} B ({:>5.1}% zero) {:?}",
format!("VOICE_D_{n}"),
bytes.len(),
magics,
riffs.len(),
100.0 * covered as f64 / bytes.len() as f64,
first_riff,
datas,
if head.is_empty() { 0.0 } else { 100.0 * head_zero as f64 / head.len() as f64 },
head_distinct,
tail.len(),
if tail.is_empty() { 0.0 } else { 100.0 * zeros as f64 / tail.len() as f64 },
sizes
);
break;
}
}
}

View File

@@ -0,0 +1,78 @@
//! Is audio actually missing from the resupply banks? Ask the subtitles.
//!
//! The `.slb` decode of `VOICE_D_453` yields 0.14 s, which "looks too short" —
//! but that judgement was an impression. The subtitle track for each movie
//! carries cue times, so it says independently how long the spoken line runs.
//! If the last cue lands near the decoded length, nothing is missing; if it lands
//! far past it, audio really is being lost.
use sylpheed_formats::{movie_subtitle, slb, PakArchive};
/// XMA1 at 48 kHz stereo, 16-bit → bytes per second of PCM.
const PCM_BYTES_PER_SEC: f32 = 48000.0 * 2.0 * 2.0;
/// Decode one sub-wave with FFmpeg and return its length in seconds. Measured,
/// not estimated from a compression ratio — the ratio guess was the first version
/// of this and it is not good enough to hang a conclusion on.
fn decoded_secs(riff: &[u8]) -> f32 {
use std::io::Write;
use std::process::{Command, Stdio};
let Ok(mut c) = Command::new("ffmpeg")
.args(["-v", "error", "-i", "pipe:0", "-f", "s16le", "pipe:1"])
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::null())
.spawn()
else {
return f32::NAN;
};
let buf = riff.to_vec();
let mut stdin = c.stdin.take().unwrap();
std::thread::spawn(move || {
let _ = stdin.write_all(&buf);
});
let out = c.wait_with_output().expect("ffmpeg");
out.stdout.len() as f32 / PCM_BYTES_PER_SEC
}
fn main() {
let disc = std::env::var("SYLPHEED_DISC").expect("set SYLPHEED_DISC");
let snd = PakArchive::open(format!("{disc}/dat/sound.pak")).expect("sound.pak");
let lang = PakArchive::open(format!("{disc}/dat/movie/eng.pak")).expect("eng.pak");
// movie → the bank the record table binds it to.
let bound = [
("hokyu_LS_s02A", 450),
("hokyu_LS_s09A", 451),
("hokyu_DS_s13A", 452),
("hokyu_LS_s02H", 453),
("hokyu_DS_s07H", 454),
];
println!(
"{:<18} {:>6} {:>10} {:>12} {:>12}",
"movie", "bank", "last cue s", "decoded s", "verdict"
);
for (movie, n) in bound {
let cues = movie_subtitle::track_voice_cues(&lang, movie);
let last = cues.iter().map(|(_, t)| *t).fold(0.0f32, f32::max);
let path = format!("eng\\etc\\VOICE_D_{n}.slb");
let Some(entry) = snd.find_by_name(&path) else { continue };
let bytes = snd.read(entry).expect("read");
// Sum the sub-waves' payloads as the decoder currently sees them.
let riffs = slb::to_xma_riffs(&bytes);
let secs: f32 = riffs.iter().map(|r| decoded_secs(r)).sum();
// One-directional: a subtitle that appears at t seconds cannot sit inside
// a clip shorter than t. A cue at 0.0 tells us nothing either way.
let verdict = if last == 0.0 {
"no cue signal"
} else if last > secs {
"AUDIO MISSING"
} else {
"consistent"
};
println!(
"{movie:<18} {:>6} {last:>10.2} {secs:>12.2} {verdict:>12} cues={}",
format!("D_{n}"),
cues.len()
);
}
}

View File

@@ -0,0 +1,19 @@
//! Why does a named resource never decode? Reports the furthest gate its best
//! candidate reached. Usage: why_missed <container.xpr> [name-substring]
use sylpheed_formats::mesh::{debug_best_rejection, xbg7_resource_names, Xbg7Model};
use std::collections::HashSet;
fn main() {
let a: Vec<String> = std::env::args().collect();
let bytes = std::fs::read(&a[1]).unwrap();
let filter = a.get(2).cloned().unwrap_or_default();
let decoded: HashSet<String> = Xbg7Model::stage_models(&bytes).into_iter().map(|m| m.name).collect();
for n in xbg7_resource_names(&bytes) {
if decoded.contains(&n) || (!filter.is_empty() && !n.contains(&filter)) {
continue;
}
match debug_best_rejection(&bytes, &n) {
Some((rank, why)) => println!("{n:<28} rank {rank} {why}"),
None => println!("{n:<28} (no candidate reached any gate / skipped before anchoring)"),
}
}
}

View File

@@ -0,0 +1,14 @@
//! Why does the decoder refuse a grouped-pool resource at a given pool start?
//! Usage: why_rejected <container.xpr> <resource>@<vb0-hex>...
use sylpheed_formats::mesh::debug_grouped_report;
fn main() {
let a: Vec<String> = std::env::args().collect();
let bytes = std::fs::read(&a[1]).expect("container");
for pair in &a[2..] {
let (name, off) = pair.split_once('@').expect("name@hex");
let vb0 = usize::from_str_radix(off.trim_start_matches("0x"), 16).expect("hex");
for line in debug_grouped_report(&bytes, name, vb0) {
println!("{line}");
}
}
}

View File

@@ -1,85 +1,405 @@
//! Audio format parsing — XMA and XWB handling.
//! Audio format parsing — WAV/PCM decode + Xbox 360 XMA/XMA2 recognition.
//!
//! Xbox 360 games use **XMA** (Xbox Media Audio) as their primary audio codec.
//! XMA is a proprietary Microsoft codec derived from WMA Pro.
//! ## What the game actually ships
//! Project Sylpheed's in-game audio lives in `dat/sound.pak` (IPFB): ~9500
//! entries of **raw, header-less XMA2** stream data — no per-entry `RIFF`/`fmt `
//! header, no compression wrapper (identical sound effects are byte-for-byte
//! duplicate entries). The per-stream format (channel count, sample rate, loop
//! points) is therefore **not** in the stream data; it lives in a separate
//! sound-bank descriptor that has not been reverse engineered yet. So this
//! module can *identify* those streams (and count their XMA packets) but cannot
//! yet decode them to PCM — that needs (a) the bank descriptor and (b) an XMA2
//! decoder.
//!
//! ## The Challenge
//! XMA decoding requires Microsoft's proprietary XMA decoder, which is only
//! available in the Windows DirectX runtime. There are two approaches:
//! ## What this module does today
//! - Fully parses **RIFF/WAVE PCM** (8/16/24-bit int, 32-bit float) → `GameAudio`
//! (interleaved `f32`), ready for playback/export of any converted audio.
//! - Reads metadata from **RIFF/WAVE XMA (`0x0165`) / XMA2 (`0x0166`)** headers
//! (channels, sample rate) — decode still unsupported.
//! - Recognizes **raw XMA2** stream blobs by entropy + packet alignment and
//! reports the 2048-byte packet count.
//!
//! ### Option A: Convert on extraction (recommended for Milestone 1)
//! Use `xma2encode.exe` (from Xbox 360 SDK) or `ffmpeg` (has partial XMA support)
//! to pre-convert all audio to OGG/WAV during the extraction step.
//!
//! ```bash
//! # Convert a single XMA file to WAV using ffmpeg
//! ffmpeg -i audio.xma output.wav
//!
//! # Or use VGMStream's test.exe for more accurate XMA decoding
//! ```
//!
//! ### Option B: XMA2 software decoder
//! The `xma2dec` project provides an open-source XMA2 decoder.
//! GitHub: https://github.com/koolkdev/xmalib (research-quality)
//!
//! ### XWB Wave Bank format
//! Xbox 360 games typically bundle audio into XWB (Xbox Wave Bank) files.
//! These are containers that hold multiple XMA streams.
//! Reference: https://github.com/microsoft/DirectXTK/tree/main/Audio
//! XMA framing constants are from xenia-canary `src/xenia/apu/xma_context.h`
//! (`kBytesPerPacket = 2048`, `kSamplesPerFrame = 512`, `kBytesPerSample = 2`).
use thiserror::Error;
// XMA framing (xenia-canary xma_context.h).
/// One XMA2 packet is 2048 bytes (4-byte header + 2044 bytes of frame data).
pub const XMA_BYTES_PER_PACKET: usize = 2048;
/// Decoded PCM samples produced per XMA frame, per channel.
pub const XMA_SAMPLES_PER_FRAME: u32 = 512;
// WAVE format tags.
const WAVE_FORMAT_PCM: u16 = 0x0001;
const WAVE_FORMAT_IEEE_FLOAT: u16 = 0x0003;
const WAVE_FORMAT_EXTENSIBLE: u16 = 0xFFFE;
const WAVE_FORMAT_XMA: u16 = 0x0165;
const WAVE_FORMAT_XMA2: u16 = 0x0166;
#[derive(Debug, Error)]
pub enum AudioError {
#[error("Unknown audio format magic: {0:?}")]
UnknownMagic([u8; 4]),
#[error("XMA decoding requires pre-conversion. See audio.rs for instructions.")]
XmaNotSupported,
#[error("Parse error: {0}")]
Parse(String),
#[error("not a recognized audio container")]
Unrecognized,
#[error("malformed audio: {0}")]
Malformed(&'static str),
#[error("codec needs a decoder this crate does not provide: {0:?}")]
NeedsDecoder(AudioCodec),
#[error("unsupported PCM sample format: tag {tag:#06x}, {bits} bits")]
UnsupportedPcm { tag: u16, bits: u16 },
}
/// An audio clip decoded to raw PCM, ready for Bevy's audio system.
/// Recognized audio container / codec.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum AudioCodec {
/// RIFF/WAVE integer PCM (`WAVE_FORMAT_PCM`).
Pcm,
/// RIFF/WAVE IEEE-float PCM (`WAVE_FORMAT_IEEE_FLOAT`).
PcmFloat,
/// RIFF/WAVE XMA (`0x0165`).
Xma,
/// RIFF/WAVE XMA2 (`0x0166`).
Xma2,
/// Header-less XMA2 stream (the game's `sound.pak` entries), identified
/// heuristically — no embedded channel/rate metadata.
RawXma2,
Unknown,
}
impl AudioCodec {
/// Does decoding this codec require support this crate does not (yet) have?
pub fn needs_decoder(&self) -> bool {
matches!(self, Self::Xma | Self::Xma2 | Self::RawXma2)
}
pub fn label(&self) -> &'static str {
match self {
Self::Pcm => "PCM",
Self::PcmFloat => "PCM float",
Self::Xma => "XMA (RIFF)",
Self::Xma2 => "XMA2 (RIFF)",
Self::RawXma2 => "XMA2 (raw stream)",
Self::Unknown => "unknown",
}
}
}
/// Non-decoding metadata describing an audio blob.
#[derive(Debug, Clone)]
pub struct AudioInfo {
pub codec: AudioCodec,
pub channels: Option<u16>,
pub sample_rate: Option<u32>,
pub bits_per_sample: Option<u16>,
/// PCM samples per channel, when derivable (WAV only).
pub samples_per_channel: Option<u64>,
pub duration_secs: Option<f32>,
pub size_bytes: usize,
/// 2048-byte XMA packet count, for XMA/raw-XMA streams.
pub xma_packets: Option<u32>,
}
impl AudioInfo {
fn empty(codec: AudioCodec, size: usize) -> Self {
Self {
codec,
channels: None,
sample_rate: None,
bits_per_sample: None,
samples_per_channel: None,
duration_secs: None,
size_bytes: size,
xma_packets: None,
}
}
/// Best-effort classification of an audio blob. Never fails: unrecognized
/// input yields [`AudioCodec::Unknown`].
///
/// Order matters: a `RIFF/WAVE` header is authoritative; only header-less,
/// high-entropy, packet-sized blobs fall through to the raw-XMA2 heuristic.
pub fn probe(bytes: &[u8]) -> Self {
if let Some(info) = parse_riff_wave(bytes) {
return info;
}
if looks_like_raw_xma2(bytes) {
let mut info = Self::empty(AudioCodec::RawXma2, bytes.len());
info.xma_packets = Some(bytes.len().div_ceil(XMA_BYTES_PER_PACKET) as u32);
return info;
}
Self::empty(AudioCodec::Unknown, bytes.len())
}
}
/// True when `bytes` is likely a raw, header-less XMA2 stream: no known audio
/// magic, but a large, near-incompressible (high-entropy) payload — the shape
/// of the game's `sound.pak` audio entries. Heuristic: not conclusive, but the
/// probe runs only after every structured format has been ruled out.
pub fn looks_like_raw_xma2(bytes: &[u8]) -> bool {
// XMA streams are always at least a couple of packets long.
if bytes.len() < XMA_BYTES_PER_PACKET * 2 {
return false;
}
if bytes.starts_with(b"RIFF") || bytes.starts_with(b"XMA2") {
return false; // handled by the RIFF path
}
shannon_entropy_bits(&bytes[..bytes.len().min(64 * 1024)]) >= 7.8
}
/// Shannon entropy in bits/byte over `data` (0.0..=8.0). Compressed audio sits
/// very close to 8; structured/text data is much lower.
fn shannon_entropy_bits(data: &[u8]) -> f64 {
if data.is_empty() {
return 0.0;
}
let mut counts = [0u32; 256];
for &b in data {
counts[b as usize] += 1;
}
let n = data.len() as f64;
counts
.iter()
.filter(|&&c| c > 0)
.map(|&c| {
let p = c as f64 / n;
-p * p.log2()
})
.sum()
}
// ── RIFF/WAVE ──────────────────────────────────────────────────────────────────
/// Parse a RIFF/WAVE header for metadata. Returns `None` if `bytes` is not a
/// `RIFF....WAVE` container. All RIFF fields are little-endian.
fn parse_riff_wave(bytes: &[u8]) -> Option<AudioInfo> {
if bytes.len() < 12 || &bytes[..4] != b"RIFF" || &bytes[8..12] != b"WAVE" {
return None;
}
let le16 = |o: usize| u16::from_le_bytes([bytes[o], bytes[o + 1]]);
let le32 =
|o: usize| u32::from_le_bytes([bytes[o], bytes[o + 1], bytes[o + 2], bytes[o + 3]]);
let mut pos = 12;
let (mut tag, mut channels, mut rate, mut bits) = (0u16, 0u16, 0u32, 0u16);
let mut data_bytes: Option<u64> = None;
let mut have_fmt = false;
while pos + 8 <= bytes.len() {
let id = &bytes[pos..pos + 4];
let size = le32(pos + 4) as usize;
let body = pos + 8;
match id {
b"fmt " if body + 16 <= bytes.len() => {
tag = le16(body);
channels = le16(body + 2);
rate = le32(body + 4);
bits = le16(body + 14);
// WAVE_FORMAT_EXTENSIBLE stores the real tag in the GUID's first
// two bytes, right after cbSize (+2) → +24 from the fmt body.
if tag == WAVE_FORMAT_EXTENSIBLE && body + 26 <= bytes.len() {
tag = le16(body + 24);
}
have_fmt = true;
}
b"data" => data_bytes = Some(size as u64),
_ => {}
}
// Chunks are word-aligned (pad byte when size is odd).
pos = body + size + (size & 1);
}
if !have_fmt {
return None;
}
let codec = match tag {
WAVE_FORMAT_PCM => AudioCodec::Pcm,
WAVE_FORMAT_IEEE_FLOAT => AudioCodec::PcmFloat,
WAVE_FORMAT_XMA => AudioCodec::Xma,
WAVE_FORMAT_XMA2 => AudioCodec::Xma2,
_ => AudioCodec::Unknown,
};
let mut info = AudioInfo::empty(codec, bytes.len());
info.channels = Some(channels).filter(|&c| c > 0);
info.sample_rate = Some(rate).filter(|&r| r > 0);
info.bits_per_sample = Some(bits).filter(|&b| b > 0);
match codec {
AudioCodec::Pcm | AudioCodec::PcmFloat => {
if let (Some(d), true) = (data_bytes, channels > 0 && bits > 0) {
let frame = channels as u64 * (bits as u64 / 8);
if frame > 0 {
let spc = d / frame;
info.samples_per_channel = Some(spc);
if rate > 0 {
info.duration_secs = Some(spc as f32 / rate as f32);
}
}
}
}
AudioCodec::Xma | AudioCodec::Xma2 => {
if let Some(d) = data_bytes {
info.xma_packets = Some((d / XMA_BYTES_PER_PACKET as u64) as u32);
}
}
_ => {}
}
Some(info)
}
// ── PCM decode → GameAudio ─────────────────────────────────────────────────────
/// An audio clip decoded to raw interleaved `f32` PCM (`L R L R …`).
#[derive(Debug, Clone)]
pub struct GameAudio {
/// Raw PCM samples (interleaved for stereo: L R L R ...)
pub samples: Vec<f32>,
pub channels: u16,
pub sample_rate: u32,
}
impl GameAudio {
/// Parse audio from raw bytes.
///
/// Currently only WAV/PCM is supported. For XMA files, pre-convert using:
/// `ffmpeg -i file.xma file.wav`
pub fn from_bytes(bytes: &[u8]) -> Result<Self, AudioError> {
// Check for WAV magic
if bytes.len() >= 4 && &bytes[..4] == b"RIFF" {
return Self::from_wav(bytes);
/// Decode a RIFF/WAVE **PCM** file (int 8/16/24-bit or 32-bit float) to
/// interleaved `f32`. XMA/XMA2 return [`AudioError::NeedsDecoder`].
pub fn from_wav(bytes: &[u8]) -> Result<Self, AudioError> {
let info = parse_riff_wave(bytes).ok_or(AudioError::Unrecognized)?;
if info.codec.needs_decoder() {
return Err(AudioError::NeedsDecoder(info.codec));
}
let channels = info.channels.ok_or(AudioError::Malformed("no channels"))?;
let rate = info.sample_rate.ok_or(AudioError::Malformed("no sample rate"))?;
let bits = info.bits_per_sample.ok_or(AudioError::Malformed("no bit depth"))?;
// XMA magic bytes
if bytes.len() >= 4 && (
&bytes[..4] == b"XWB\0" || // Wave bank
&bytes[..4] == b"XMA2" // Raw XMA2
) {
return Err(AudioError::XmaNotSupported);
}
// Locate the `data` chunk body.
let (off, len) = riff_data_span(bytes).ok_or(AudioError::Malformed("no data chunk"))?;
let data = &bytes[off..off + len];
Err(AudioError::UnknownMagic(
bytes[..4.min(bytes.len())].try_into().unwrap_or([0u8; 4])
))
}
fn from_wav(bytes: &[u8]) -> Result<Self, AudioError> {
// Minimal WAV parser for PCM files
// A proper implementation should use the `hound` crate:
// https://crates.io/crates/hound
//
// TODO: Add `hound = "3"` to Cargo.toml and implement properly
Err(AudioError::Parse(
"WAV parsing TODO: add `hound` crate and implement".to_string()
))
let samples: Vec<f32> = match (info.codec, bits) {
(AudioCodec::Pcm, 8) => data.iter().map(|&b| (b as f32 - 128.0) / 128.0).collect(),
(AudioCodec::Pcm, 16) => data
.chunks_exact(2)
.map(|c| i16::from_le_bytes([c[0], c[1]]) as f32 / 32768.0)
.collect(),
(AudioCodec::Pcm, 24) => data
.chunks_exact(3)
.map(|c| {
let v = ((c[2] as i32) << 16) | ((c[1] as i32) << 8) | c[0] as i32;
let v = (v << 8) >> 8; // sign-extend 24→32
v as f32 / 8_388_608.0
})
.collect(),
(AudioCodec::PcmFloat, 32) => data
.chunks_exact(4)
.map(|c| f32::from_le_bytes([c[0], c[1], c[2], c[3]]))
.collect(),
_ => return Err(AudioError::UnsupportedPcm { tag: 0, bits }),
};
Ok(Self { samples, channels, sample_rate: rate })
}
}
/// Byte span (offset, length) of the WAVE `data` chunk body, if present.
fn riff_data_span(bytes: &[u8]) -> Option<(usize, usize)> {
if bytes.len() < 12 || &bytes[..4] != b"RIFF" || &bytes[8..12] != b"WAVE" {
return None;
}
let le32 =
|o: usize| u32::from_le_bytes([bytes[o], bytes[o + 1], bytes[o + 2], bytes[o + 3]]) as usize;
let mut pos = 12;
while pos + 8 <= bytes.len() {
let size = le32(pos + 4);
let body = pos + 8;
if &bytes[pos..pos + 4] == b"data" {
let len = size.min(bytes.len().saturating_sub(body));
return Some((body, len));
}
pos = body + size + (size & 1);
}
None
}
#[cfg(test)]
mod tests {
use super::*;
/// Build a minimal 16-bit PCM WAV in memory (stereo, 2 frames).
fn tiny_wav() -> Vec<u8> {
let mut v = Vec::new();
let data: [i16; 4] = [1000, -1000, 32767, -32768]; // L R L R
let data_bytes: Vec<u8> = data.iter().flat_map(|s| s.to_le_bytes()).collect();
v.extend_from_slice(b"RIFF");
v.extend_from_slice(&(36 + data_bytes.len() as u32).to_le_bytes());
v.extend_from_slice(b"WAVE");
v.extend_from_slice(b"fmt ");
v.extend_from_slice(&16u32.to_le_bytes());
v.extend_from_slice(&WAVE_FORMAT_PCM.to_le_bytes());
v.extend_from_slice(&2u16.to_le_bytes()); // channels
v.extend_from_slice(&48000u32.to_le_bytes());
v.extend_from_slice(&(48000u32 * 2 * 2).to_le_bytes());
v.extend_from_slice(&4u16.to_le_bytes()); // block align
v.extend_from_slice(&16u16.to_le_bytes()); // bits
v.extend_from_slice(b"data");
v.extend_from_slice(&(data_bytes.len() as u32).to_le_bytes());
v.extend_from_slice(&data_bytes);
v
}
#[test]
fn probe_and_decode_pcm_wav() {
let wav = tiny_wav();
let info = AudioInfo::probe(&wav);
assert_eq!(info.codec, AudioCodec::Pcm);
assert_eq!(info.channels, Some(2));
assert_eq!(info.sample_rate, Some(48000));
assert_eq!(info.samples_per_channel, Some(2));
let audio = GameAudio::from_wav(&wav).unwrap();
assert_eq!(audio.channels, 2);
assert_eq!(audio.samples.len(), 4);
assert!((audio.samples[2] - 0.99997).abs() < 1e-3); // 32767/32768
}
#[test]
fn probe_xma2_riff_reports_metadata_not_decode() {
// Minimal RIFF/WAVE with an XMA2 fmt tag.
let mut v = Vec::new();
v.extend_from_slice(b"RIFF");
v.extend_from_slice(&200u32.to_le_bytes());
v.extend_from_slice(b"WAVE");
v.extend_from_slice(b"fmt ");
v.extend_from_slice(&16u32.to_le_bytes());
v.extend_from_slice(&WAVE_FORMAT_XMA2.to_le_bytes());
v.extend_from_slice(&2u16.to_le_bytes());
v.extend_from_slice(&44100u32.to_le_bytes());
v.extend_from_slice(&0u32.to_le_bytes());
v.extend_from_slice(&0u16.to_le_bytes());
v.extend_from_slice(&0u16.to_le_bytes());
let info = AudioInfo::probe(&v);
assert_eq!(info.codec, AudioCodec::Xma2);
assert_eq!(info.channels, Some(2));
assert!(info.codec.needs_decoder());
assert!(matches!(
GameAudio::from_wav(&v),
Err(AudioError::NeedsDecoder(AudioCodec::Xma2))
));
}
#[test]
fn raw_high_entropy_blob_reads_as_raw_xma2() {
// A pseudo-random 8 KB blob (no magic) → RawXma2 with a packet count.
let mut b = vec![0u8; 8192];
let mut x = 0x2545_F491u32;
for v in b.iter_mut() {
x ^= x << 13;
x ^= x >> 17;
x ^= x << 5;
*v = (x & 0xFF) as u8;
}
let info = AudioInfo::probe(&b);
assert_eq!(info.codec, AudioCodec::RawXma2);
assert_eq!(info.xma_packets, Some(4)); // 8192 / 2048
}
#[test]
fn structured_low_entropy_blob_is_unknown_not_audio() {
let b = b"IDXD............a bunch of readable ASCII text fields....".repeat(40);
assert_eq!(AudioInfo::probe(&b).codec, AudioCodec::Unknown);
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -20,9 +20,14 @@
//! i.e. the low 24 bits are a modular polynomial hash and the top byte is an
//! 8-bit additive checksum of the bytes. The reduction constant `0x8003_1493`
//! is the reciprocal of the modulus `0x00FF_F9D7` used by the `mulhwu`/`mullw`
//! Barrett step; there is **no** trailing conditional subtract, so the value is
//! defined by the exact op sequence (faithfully reproduced below), not by a
//! textbook `%`.
//! Barrett step.
//!
//! **The reduction is EXACT, not lossy.** An earlier version of this note said
//! the missing trailing conditional subtract made it something other than `%`.
//! It does not: `rlwinm r6,r6,9,23,31` is exactly `hi >> 23`, and with
//! `RECIP == floor(2^55/M) + 1` that is standard GranlundMontgomery magic
//! division. Checked at every quotient boundary (`k·M1, k·M, k·M+1`) across the
//! whole 32-bit domain: 0 wrong of 770. So the low 24 bits really are `A % M`.
//!
//! Verified against the real disc: `name_hash("files.tbl") == 0x8342_1153`
//! and `name_hash("eng\\weapon.tbl") == 0x900C_8DCD`, both of which are present
@@ -150,3 +155,129 @@ mod tests {
assert_eq!(name_hash(""), 0);
}
}
/// Barrett modulus of the **record/field tag** hash — a different constant from
/// [`MODULUS`], and the same one the IXUD hash folds down to.
const TAG_MODULUS: u32 = 0x00FF_FFDF; // 2^24 - 33, prime
/// The guest's divide magic for [`TAG_MODULUS`] (`floor(2^56/M) + 1`).
const TAG_MAGIC: u32 = 0x2101;
/// The IXUD loop modulus, applied in 64-bit arithmetic before [`TAG_MODULUS`].
const IXUD_M1: u64 = 0xFFFF_FF67; // 2^32 - 153
/// Hash an IDXD **record key / field tag** — `sub_82447DF0`.
///
/// This is *not* [`name_hash`]. Same shape — an 8-bit additive checksum over a
/// 24-bit modular polynomial — but two constants differ:
///
/// * modulus `0x00FF_FFDF` (2^24 33, prime), not `0x00FF_F9D7`;
/// * **no lowercasing**, so tags are case-sensitive. The disc relies on this:
/// 17 name pairs differ only in case (`UNIT`/`Unit`, `TYPE`/`Type`, …) and
/// `name_hash` collides on every one of them.
///
/// A record's key is the tag of its **own** name — 190,782/190,782 records
/// disc-wide — so records are addressable by name without reading a roster.
///
/// The guest **sign-extends** each byte (`extsb`), and that is load-bearing:
/// a version of this using unsigned bytes matched all 1.27M disc names, because
/// every one is ASCII, while disagreeing on ~90% of random inputs containing a
/// byte ≥ 0x80. Only the disassembly could catch that.
pub fn tag_hash(name: &str) -> u32 {
tag_hash_bytes(name.as_bytes())
}
/// [`tag_hash`] over raw bytes — the form that can express a non-UTF-8 name, and
/// the only way to exercise the `extsb` path.
pub fn tag_hash_bytes(bytes: &[u8]) -> u32 {
let mut a: u32 = 0;
let mut b: u32 = 0;
for &byte in bytes {
let c = byte as i8 as i32 as u32; // extsb
a = (a << 8).wrapping_add(c);
b = b.wrapping_add(c);
// Exact magic division by TAG_MODULUS, in the guest's add-correction form.
let hi = ((a as u64 * TAG_MAGIC as u64) >> 32) as u32;
let q = hi.wrapping_add(a.wrapping_sub(hi) >> 1) >> 23;
a = a.wrapping_sub(q.wrapping_mul(TAG_MODULUS));
}
((b << 24) & 0xFF00_0000) | (a & 0x00FF_FFFF)
}
/// Hash an **IXUD** record key / field tag — `sub_82447E70`.
///
/// IXUD is IDXD's wide-string sibling: identical container layout, but strings
/// are UTF-16BE and `strsize` and every string offset are counted in **16-bit
/// characters, not bytes** (`STR + 2·strsize == filesize`).
///
/// `units` is the name as big-endian UTF-16 code units.
///
/// It resisted every single-modulus search because it chains **two** exact
/// moduli: the loop reduces mod `2^32 153` in 64-bit arithmetic, and only the
/// final value is folded into 24 bits mod `2^24 33`. A polynomial mod `M1`
/// folded through `M2` is not a polynomial mod anything, which is why a gcd over
/// the observed pairs returns 1 and a Barrett sweep finds nothing.
///
/// The checksum byte sums the **full 16-bit code units**, not their low bytes —
/// indistinguishable on this disc, where every IXUD name is ASCII, but not in
/// general.
pub fn ixud_hash(units: &[u16]) -> u32 {
let mut a: u64 = 0;
let mut b: u32 = 0;
for &ch in units {
a = ((a << 16) + ch as u64) % IXUD_M1;
b = b.wrapping_add(ch as u32);
}
((b & 0xFF) << 24) | (a % TAG_MODULUS as u64) as u32
}
/// Convenience: [`ixud_hash`] for an ASCII/UTF-8 name.
pub fn ixud_hash_str(name: &str) -> u32 {
let units: Vec<u16> = name.encode_utf16().collect();
ixud_hash(&units)
}
#[cfg(test)]
mod tag_tests {
use super::tag_hash;
#[test]
fn known_tags_from_the_disc_tables() {
// Field names, from IDXD records in GP_MAIN_GAME_E.pak.
assert_eq!(tag_hash("SideID"), 0x1225_E093);
assert_eq!(tag_hash("ID"), 0x8D00_4944);
assert_eq!(tag_hash("Name"), 0x8161_7773);
// Formation record keys, from FormationSet_S02.tbl.
assert_eq!(tag_hash("Formation_4_Bird"), 0x22A5_EEED);
assert_eq!(tag_hash("Formation_1_only"), 0x6047_EECF);
assert_eq!(tag_hash("Formation_ADAN_Turret07_30"), 0x30CE_86BE);
}
#[test]
fn ixud_uses_a_different_hash_entirely() {
// Verified against real IXUD data: 86/86 record keys and 108,261/108,261
// field tags in GP_MAIN_GAME_E.pak.
assert_eq!(super::ixud_hash_str("NoRecord"), 0x1C6D_9C96);
assert_ne!(super::ixud_hash_str("NoRecord"), tag_hash("NoRecord"));
}
#[test]
fn tag_hash_sign_extends_high_bytes() {
// The guest uses extsb. Unsigned bytes agree on all-ASCII names but not
// here -- this input is the concrete counterexample.
let bytes = [0x4eu8, 0x3f, 0xcf, 0xa5, 0x0c, 0x86, 0x4c, 0x2b, 0x41, 0xcf];
assert_eq!(super::tag_hash_bytes(&bytes), 0x1AFF_849A);
}
#[test]
fn tags_are_case_sensitive_unlike_name_hash() {
// name_hash lowercases first; tag_hash must not.
assert_ne!(tag_hash("SideID"), tag_hash("sideid"));
}
#[test]
fn the_top_byte_is_the_byte_sum() {
for s in ["ID", "Formation_4_Bird", "SideID"] {
let sum = s.as_bytes().iter().map(|&b| b as u32).sum::<u32>() & 0xFF;
assert_eq!(tag_hash(s) >> 24, sum, "{s}");
}
}
}

View File

@@ -8,15 +8,44 @@
//! ## Layout (all fields big-endian)
//!
//! ```text
//! Offset Size Field
//! 0x00 4 Magic: "IDXD"
//! 0x04 4 count (number of top-level records; small)
//! 0x08 4 schema_hash (identifies the object type; custom hash family, preimage unknown)
//! 0x0C .. NODE / INDEX region — hash-keyed records + (for some schemas) auxiliary
//! binary tables. Not required for the string values below.
//! .. .. STRING POOL — the trailing, (almost) all-ASCII region.
//! Offset Size Field
//! 0x00 4 Magic: "IDXD"
//! 0x04 4 record_count n
//! 0x08 16*n records { u32 name_hash, u32 name_off, u32 field_begin, u32 field_end }
//! .. 4 field_count m (equals max(field_end))
//! .. 12*m fields { u32 key, u32 name_off, u32 value_off }
//! .. 4 pool_size (equals file_len pool_base)
//! .. .. STRING POOL — every `*_off` above is a byte offset from here.
//! ```
//!
//! Records are sorted ascending by `name_hash` and the guest binary-searches them
//! (`sub_82448AA0`); fields are sorted ascending by `key` and lower-bounded
//! (`sub_8244E338`). `name_hash` is [`crate::hash::tag_hash`] of the record's own
//! name — **not** the pak TOC hash: different modulus, and not lowercased.
//!
//! A field's `name_off` points at its own name, so **field names are on the disc**
//! and never have to be recovered from their hash; `key` is then `tag_hash(name)`.
//! A `name_off` of `0xFFFF_FFFF` means the field has no name, and its `key` is a
//! literal positional integer instead — a line-slot index, a movie id
//! ([`IdxdField::index`]).
//!
//! ## There is no schema field
//!
//! The word at `0x08` was long read as a `schema_hash` identifying the object
//! type. It is not: it is simply **record 0's `name_hash`**, the first cell of a
//! uniform 16-byte record array. `tag_hash(records[0].name)` reproduces it for
//! **all 7750** IDXD objects on the retail disc. It still works as a type
//! discriminator — tables of one kind share their lowest-hashed record name — so
//! [`IdxdObject::schema_hash`] is kept under its established name, but it
//! identifies a *record name*, not a schema. Nothing on disc names the type.
//!
//! Verified across the whole retail disc (`records_roundtrip_disc`,
//! `field_names_are_stored_disc`): all **7750** objects parse under this layout;
//! all **190782** records reproduce their stored `name_hash`; all **1271462**
//! named fields reproduce their `key` from their stored name; the remaining
//! **1485073** fields are unnamed with literal keys, and just **504** are
//! hash-keyed with no name stored.
//!
//! ## The string pool: **value-before-key, defaults omitted**
//!
//! Each property that has an explicit value is serialized as `<value>\0<key>\0` —
@@ -44,14 +73,34 @@
//! - [`IdxdObject::get_raw`] — the raw preceding token, no validation. Use for fields
//! you *know* are identifier-valued (`ID`, `Name`, `Type`, `Model`).
//!
//! ## Not yet decoded
//! ## Two readers, one object
//!
//! Default/omitted fields (many ratios, the `…Count` family) carry no value in the
//! pool — their values live in the binary node/index region (hash-keyed by the same
//! unrecovered custom hash) or come from schema defaults. Binding those requires
//! decoding that region per schema; until then the typed getters return `None` for
//! them (honest "unknown") rather than guessing. See `reference_ipfb_archive_format`
//! in the project notes.
//! The record/field table above is exact, so prefer it: [`IdxdObject::records`],
//! [`record`](IdxdObject::record), [`IdxdRecord::field`]. The older
//! *value-before-key string-pool* reader ([`get_f32`](IdxdObject::get_f32) and
//! friends, documented below) predates the decode and is kept because a large
//! part of the corpus is written in terms of it.
//!
//! ## The legacy string-pool reader: **value-before-key, defaults omitted**
//!
//! In the pool each property that has an explicit value is laid out as
//! `<value>\0<key>\0` — the value string comes *immediately before* its field
//! name. Numbers are stored as ASCII text (`"600.0"`, `"144"`), so they read out
//! directly:
//!
//! ```text
//! … "1000.0" "HP" "500000.0" "RadarRange" "10.0" "Size_X" …
//! value key value key value key
//! ```
//!
//! Fields left at their default omit the value string, appearing as a bare key.
//! So "the token before a key" is a real value *only when that token is itself
//! value-shaped*, which the typed getters enforce ([`get_f32`](IdxdObject::get_f32),
//! [`get_i64`](IdxdObject::get_i64), [`get_str`](IdxdObject::get_str) validate;
//! [`get_raw`](IdxdObject::get_raw) does not). The adjacency is a *consequence* of
//! the field table — each record's values are emitted next to their keys — not a
//! rule of the format, and it cannot see a field whose value string is shared or
//! reordered. Use the record API when correctness matters.
use thiserror::Error;
@@ -70,13 +119,75 @@ pub enum IdxdError {
/// A parsed IDXD object: its schema id plus the decoded string pool.
#[derive(Debug, Clone)]
pub struct IdxdObject {
/// Object-type id (which kind of definition this is). Custom hash; groups
/// entries by schema even though the preimage is unknown.
/// The word at `0x08`. Despite the name it is **record 0's `name_hash`**, not
/// a schema id — see the module docs. Retained because it does discriminate
/// object kinds in practice and the corpus is written in terms of it; prefer
/// [`records`](Self::records) when you want meaning rather than a bucket key.
pub schema_hash: u32,
/// The `count` header field (number of top-level records).
/// The `record_count` header field.
pub count: u32,
/// Ordered string-pool tokens (see module docs; value-before-key).
tokens: Vec<String>,
/// The decoded record/field table. `None` only when the binary region does
/// not parse — which no retail object does; see `records_roundtrip_disc`.
records: Option<Vec<IdxdRecord>>,
}
/// One named record of an [`IdxdObject`] — a row of the table.
#[derive(Debug, Clone)]
pub struct IdxdRecord {
/// The record's name, from the string pool.
pub name: String,
/// `tag_hash(name)`, as stored.
pub name_hash: u32,
/// This record's fields, in on-disc order (ascending `key`).
pub fields: Vec<IdxdField>,
}
/// One field of an [`IdxdRecord`] — a key/value cell.
#[derive(Debug, Clone)]
pub struct IdxdField {
/// `tag_hash(name)` when the field is named, else a literal positional
/// integer. See [`IdxdField::index`].
pub key: u32,
/// The field's own name, when it has one. `None` for positional fields.
pub name: Option<String>,
/// The field's value, from the string pool. Values are ASCII text, including
/// numbers (`"600.0"`).
pub value: String,
}
impl IdxdField {
/// The key read as a literal positional index. `None` for a named field.
///
/// Keyed off the stored name rather than off the key's magnitude, so it is
/// exact: a field is positional precisely when it has no name.
pub fn index(&self) -> Option<u32> {
self.name.is_none().then_some(self.key)
}
/// Whether this field is named `name`.
pub fn is_named(&self, name: &str) -> bool {
self.name.as_deref() == Some(name)
}
}
impl IdxdRecord {
/// The field named `name` (matched by `tag_hash`), if present.
pub fn field(&self, name: &str) -> Option<&IdxdField> {
let key = crate::hash::tag_hash(name);
self.fields.iter().find(|f| f.key == key)
}
/// The field stored under the literal integer key `index`, if present.
pub fn field_at(&self, index: u32) -> Option<&IdxdField> {
self.fields.iter().find(|f| f.key == index)
}
/// The value of the field named `name`.
pub fn get(&self, name: &str) -> Option<&str> {
self.field(name).map(|f| f.value.as_str())
}
}
impl IdxdObject {
@@ -92,10 +203,12 @@ impl IdxdObject {
let count = be32(bytes, 4);
let schema_hash = be32(bytes, 8);
let tokens = extract_string_pool(bytes);
let records = parse_records(bytes, count);
Ok(Self {
schema_hash,
count,
tokens,
records,
})
}
@@ -109,6 +222,25 @@ impl IdxdObject {
&self.tokens
}
/// The decoded record/field table, or `None` if the binary region did not
/// parse. Every retail object parses; a `None` here means a malformed or
/// synthetic buffer.
pub fn records(&self) -> Option<&[IdxdRecord]> {
self.records.as_deref()
}
/// The record named `name` (matched by `tag_hash`), if the table parsed.
pub fn record(&self, name: &str) -> Option<&IdxdRecord> {
let key = crate::hash::tag_hash(name);
self.records.as_ref()?.iter().find(|r| r.name_hash == key)
}
/// Record 0 — the one whose hash sits in the header word this crate calls
/// [`schema_hash`](Self::schema_hash).
pub fn first_record(&self) -> Option<&IdxdRecord> {
self.records.as_ref()?.first()
}
/// The raw token immediately preceding the first occurrence of `key`.
///
/// No validation: for a defaulted/omitted field this is the *neighbouring key*,
@@ -183,8 +315,91 @@ impl IdxdObject {
}
format!("schema {:08x}", self.schema_hash)
}
/// Recover this entry's original TOC path (e.g. `unit\rou_f001.tbl`) from its
/// identity/pool tokens by re-hashing candidates against `name_hash` with the
/// known path schemes. Returns `None` when no scheme reproduces the hash.
///
/// Shared by the CLI `pak list` and the GUI pack browser so both surface the
/// same recovered names. See [`crate::hash::recover_toc_name`].
pub fn recover_toc_path(&self, name_hash: u32) -> Option<String> {
let mut cands: Vec<&str> = Vec::new();
for key in ["ID", "Name", "Model"] {
if let Some(v) = self.get_raw(key) {
cands.push(v);
}
}
for t in self.tokens() {
if t.contains('_') || t.len() >= 5 {
cands.push(t.as_str());
}
}
crate::hash::recover_toc_name(name_hash, &cands)
}
}
/// Decode the record/field table (see module docs). Returns `None` — rather than
/// erroring — when any bound is inconsistent, so a malformed or synthetic buffer
/// still yields a usable string-pool reader.
fn parse_records(b: &[u8], count: u32) -> Option<Vec<IdxdRecord>> {
let n = count as usize;
if n == 0 || n > b.len() / 16 {
return None;
}
let recs_at = 0x08_usize;
let field_count_at = recs_at.checked_add(n.checked_mul(16)?)?;
if field_count_at + 4 > b.len() {
return None;
}
let m = be32(b, field_count_at) as usize;
let fields_at = field_count_at + 4;
let pool_size_at = fields_at.checked_add(m.checked_mul(12)?)?;
let pool = pool_size_at.checked_add(4)?;
if pool > b.len() {
return None;
}
// The trailing identity: the stored pool size is exactly what is left. This is
// what makes the layout self-checking — a wrong record stride lands here.
if be32(b, pool_size_at) as usize != b.len() - pool {
return None;
}
let string_at = |off: u32| -> Option<String> {
let start = pool.checked_add(off as usize)?;
if start >= b.len() {
return None;
}
let end = start + b[start..].iter().position(|&c| c == 0)?;
Some(String::from_utf8_lossy(&b[start..end]).into_owned())
};
let mut out = Vec::with_capacity(n);
for i in 0..n {
let r = recs_at + 16 * i;
let (begin, end) = (be32(b, r + 8) as usize, be32(b, r + 12) as usize);
if begin > end || end > m {
return None;
}
let mut fields = Vec::with_capacity(end - begin);
for j in begin..end {
let f = fields_at + 12 * j;
let name_off = be32(b, f + 4);
fields.push(IdxdField {
key: be32(b, f),
name: (name_off != NO_NAME).then(|| string_at(name_off)).flatten(),
value: string_at(be32(b, f + 8))?,
});
}
out.push(IdxdRecord {
name: string_at(be32(b, r + 4))?,
name_hash: be32(b, r),
fields,
});
}
Some(out)
}
/// A field `name_off` of `0xFFFF_FFFF` means the field is positional, not named.
const NO_NAME: u32 = 0xFFFF_FFFF;
/// Extract the trailing string pool. Finds the smallest offset whose suffix is
/// ≥98% printable-ASCII-or-NUL (the pool runs to end-of-buffer), then tokenises
/// into maximal printable runs (NUL *or* any non-printable byte separates tokens,
@@ -261,9 +476,9 @@ fn is_key_like(s: &str) -> bool {
fn parse_number(s: &str) -> bool {
!s.is_empty()
&& s.bytes().any(|b| b.is_ascii_digit())
&& s.bytes().enumerate().all(|(i, b)| {
b.is_ascii_digit() || b == b'.' || (i == 0 && (b == b'-' || b == b'+'))
})
&& s.bytes()
.enumerate()
.all(|(i, b)| b.is_ascii_digit() || b == b'.' || (i == 0 && (b == b'-' || b == b'+')))
}
#[inline]
@@ -337,7 +552,7 @@ mod tests {
fn resolved_fields_lists_explicit_values_only() {
let bytes = synth(&[
"rou_f001", "Model", // identifier value → excluded
"10.0", "Size_X", // explicit → included
"10.0", "Size_X", // explicit → included
"FCSRange", // defaulted (no value) → excluded
"Yes", "Mounted", // enum → included
]);

View File

@@ -15,16 +15,35 @@
//! start timecode instead of inline text; the actual string then lives in a
//! separate global table (not resolved here).
//!
//! ## Binary layout (as far as needed)
//! ## Binary layout — ✅ decoded and verified disc-wide
//!
//! ```text
//! 0x00 4 Magic "IXUD"
//! 0x04 4 version (1)
//! 0x08 4 schema/type hash (constant 0x6CC83E70)
//! .. .. record directory { key_hash u32, offset u32, len u32 } × n
//! .. .. UTF-16BE string pool, NUL-terminated entries
//! 0x00 4 Magic "IXUD"
//! 0x04 4 record_count n
//! 0x08 16*n records { u32 name_hash, u32 name_off, u32 field_begin, u32 field_end }
//! .. 4 field_count m
//! .. 12*m fields { u32 key, u32 name_off, u32 value_off }
//! .. 4 pool_size (in CHARS)
//! .. .. UTF-16BE string pool
//! ```
//! We don't need the directory to *present* the track — decoding the pool and
//! pairing tokens after the `SUBTITLE` header is enough and robust.
//!
//! **Every offset is in 16-bit chars, not bytes** — `pool_base + 2*off`. That
//! includes record and field *names*, not only values, and it is what makes the
//! `STR + 2*strsize == filesize` identity hold.
//!
//! `name_hash` is [`crate::hash::ixud_hash`] of the record's own name, and a
//! field's `key` is the same hash of its name; `name_off == 0xFFFF_FFFF` marks a
//! positional field whose key is a literal index. Verified over every IXUD object
//! on the disc: **1104/1104** objects parse, **1476/1476** records and
//! **628 165/628 165** named fields reproduce their stored hash.
//!
//! ❌ The layout this comment used to describe — a 12-byte record directory and a
//! "schema/type hash" at `0x08` — was wrong in the same way IDXD's was. There is
//! no schema field: the word at `0x08` is record 0's `name_hash`. See
//! `docs/re/structures/idxd-container.md`.
//!
//! The cue reader below still works on the string pool directly, which is fine
//! for presenting a track; [`IxudObject`] is the addressable route.
/// Magic at the start of every IXUD entry.
pub const IXUD_MAGIC: [u8; 4] = *b"IXUD";
@@ -185,3 +204,138 @@ mod tests {
assert!(parse(b"IDXD\0\0\0\0").is_none());
}
}
/// A parsed IXUD object: its records and their fields, addressable by name.
///
/// The counterpart of [`crate::IdxdObject`] for wide strings. Prefer this to the
/// token-pairing readers when you need a *specific* field: pool adjacency is a
/// consequence of how records are written, not a rule of the format.
#[derive(Debug, Clone)]
pub struct IxudObject {
/// The word at `0x08` — record 0's `name_hash`, not a schema id.
pub first_record_hash: u32,
records: Vec<IxudRecord>,
}
/// One named record of an [`IxudObject`].
#[derive(Debug, Clone)]
pub struct IxudRecord {
/// The record's name, decoded from UTF-16BE.
pub name: String,
/// `ixud_hash(name)`, as stored.
pub name_hash: u32,
/// This record's fields, in on-disc order.
pub fields: Vec<IxudField>,
}
/// One field of an [`IxudRecord`].
#[derive(Debug, Clone)]
pub struct IxudField {
/// `ixud_hash(name)` when named, else a literal positional index.
pub key: u32,
/// The field's own name, when it has one.
pub name: Option<String>,
/// The field's value.
pub value: String,
}
impl IxudRecord {
/// The field named `name` (matched by `ixud_hash`), if present.
pub fn field(&self, name: &str) -> Option<&IxudField> {
let key = crate::hash::ixud_hash_str(name);
self.fields.iter().find(|f| f.key == key)
}
/// The value of the field named `name`.
pub fn get(&self, name: &str) -> Option<&str> {
self.field(name).map(|f| f.value.as_str())
}
}
impl IxudObject {
/// Parse an IXUD object. `None` when the layout does not check out — the
/// trailing `pool_size` identity makes that self-verifying.
pub fn parse(b: &[u8]) -> Option<Self> {
if b.len() < 12 || b[0..4] != IXUD_MAGIC {
return None;
}
let be32 = |o: usize| u32::from_be_bytes([b[o], b[o + 1], b[o + 2], b[o + 3]]);
let n = be32(4) as usize;
if n == 0 || n > b.len() / 16 {
return None;
}
let field_count_at = 8usize.checked_add(n.checked_mul(16)?)?;
if field_count_at + 4 > b.len() {
return None;
}
let m = be32(field_count_at) as usize;
let fields_at = field_count_at + 4;
let pool_size_at = fields_at.checked_add(m.checked_mul(12)?)?;
let pool = pool_size_at.checked_add(4)?;
if pool > b.len() {
return None;
}
// pool_size counts CHARS, so this identity also proves the char scaling.
if (be32(pool_size_at) as usize).checked_mul(2)? != b.len() - pool {
return None;
}
let wstr = |off: u32| -> Option<String> {
if off == NO_NAME {
return None;
}
let start = pool.checked_add((off as usize).checked_mul(2)?)?;
let mut units = Vec::new();
let mut i = start;
while i + 1 < b.len() {
let u = u16::from_be_bytes([b[i], b[i + 1]]);
if u == 0 {
return Some(String::from_utf16_lossy(&units));
}
units.push(u);
i += 2;
}
None
};
let mut records = Vec::with_capacity(n);
for i in 0..n {
let r = 8 + 16 * i;
let (begin, end) = (be32(r + 8) as usize, be32(r + 12) as usize);
if begin > end || end > m {
return None;
}
let mut fields = Vec::with_capacity(end - begin);
for j in begin..end {
let f = fields_at + 12 * j;
fields.push(IxudField {
key: be32(f),
name: wstr(be32(f + 4)),
value: wstr(be32(f + 8))?,
});
}
records.push(IxudRecord {
name: wstr(be32(r + 4))?,
name_hash: be32(r),
fields,
});
}
Some(Self {
first_record_hash: be32(8),
records,
})
}
/// Every record, in on-disc order (ascending `name_hash`).
pub fn records(&self) -> &[IxudRecord] {
&self.records
}
/// The record named `name` (matched by `ixud_hash`).
pub fn record(&self, name: &str) -> Option<&IxudRecord> {
let key = crate::hash::ixud_hash_str(name);
self.records.iter().find(|r| r.name_hash == key)
}
}
/// A field `name_off` of `0xFFFF_FFFF` marks a positional field.
const NO_NAME: u32 = 0xFFFF_FFFF;

View File

@@ -37,6 +37,12 @@ pub mod t8ad;
// RATC nested resource bundle
pub mod ratc;
/// UI screen layout (`.rat` / RATC) — reassemble a screen from its pak.
pub mod ui_layout;
/// The retail `savedata` file: GDHA container, zlib payload, chunk stream.
pub mod savegame;
// LSTA sprite list (inline T8aD frames)
pub mod lsta;
@@ -61,6 +67,10 @@ pub mod movie_manifest;
pub mod movie_voice;
/// Assembling media whose bytes are not one archive entry — segment-spanning
/// reads, multi-sub-wave banks, and the continuous cutscene-voice stream.
pub mod media;
pub mod game_data;
pub mod localization;
@@ -68,13 +78,18 @@ pub mod localization;
// Whole-ship assembly from XBG7 part families (capital ships as split parts).
pub mod ship;
/// The runtime layout of a unit / vessel definition object, read out of the
/// title's loader and verified against a live mission (see the module docs).
pub mod unit_layout;
// Exact capital-ship placement from a runtime F10 ship-capture (ground truth).
pub mod ship_capture;
/// Re-export the most commonly used types at the crate root.
pub use audio::{AudioCodec, AudioInfo, GameAudio};
pub use font::FontInfo;
pub use idxd::{IdxdError, IdxdObject};
pub use ixud::{Cue, Subtitle};
pub use ixud::{Cue, IxudField, IxudObject, IxudRecord, Subtitle};
pub use movie_subtitle::{SubCue, SubLang};
pub use mesh::{GameMesh, Xbg7Model};
pub use ratc::RatcChild;

View File

@@ -1,9 +1,18 @@
//! `LSTA` — a sprite list: a header followed by N inline [`T8aD`](crate::t8ad)
//! frames concatenated back-to-back.
//! `LSTA` — a display list: a header followed by N inline elements concatenated
//! back-to-back, each either a [`T8aD`](crate::t8ad) sprite or a `PRMD`
//! primitive (a flat coloured quad — the same primitive the UI bundles use to
//! dim a scene).
//!
//! A `count` lives at `@0x04`, but a few entries disagree with the actual frame
//! count, so we walk by the `T8aD` magic instead (robust) and decode each frame
//! from its slice up to the next frame (or end).
//! The `count` at `@0x04` is **exact, and counts both kinds**: across all 64
//! lists on the disc, `count == T8aD frames + PRMD primitives` with no
//! exceptions (measured 2026-08-11). An earlier note here said "a few entries
//! disagree with the actual frame count" — they do not; that comparison was
//! counting sprites against a total that includes primitives.
//!
//! [`parse`] walks by magic and returns the **sprites**, deliberately skipping
//! `PRMD` entries, so its result length is `count` only for lists that hold no
//! primitives. Six lists do (in `GP_DEBRIEFING_PILOTLOG`, `GP_MISSION_SELECT`),
//! each with exactly one.
use crate::t8ad::{self, T8adImage, T8AD_MAGIC};
@@ -15,8 +24,9 @@ pub fn is_lsta(bytes: &[u8]) -> bool {
bytes.len() >= 4 && bytes[0..4] == LSTA_MAGIC
}
/// Decode all inline T8aD frames. Frames that don't decode (unsupported T8aD
/// variant) are skipped. Returns `None` only for non-LSTA input.
/// Decode the inline T8aD sprites, skipping `PRMD` primitives. Returns `None`
/// only for non-LSTA input. Every one of the 1 281 sprite frames on the disc
/// decodes (measured 2026-08-11, after the T8aD rectangle-list fix).
pub fn parse(bytes: &[u8]) -> Option<Vec<T8adImage>> {
if !is_lsta(bytes) {
return None;
@@ -48,12 +58,21 @@ pub fn parse(bytes: &[u8]) -> Option<Vec<T8adImage>> {
mod tests {
use super::*;
/// A faithful one-rectangle T8aD frame: base header, a 1-entry offset table,
/// then the rectangle header (dst 0,0, size w×h) and its pixels. (Before
/// 2026-08-11 this fixture wrote no offset-table entry at all and the decoder
/// read "pixels" from inside the header — the test only ever checked the
/// dimensions, so it passed anyway.)
fn t8ad_frame(w: u32, h: u32) -> Vec<u8> {
let mut b = vec![0u8; 64];
let mut b = vec![0u8; 0x2c];
b[0..4].copy_from_slice(&T8AD_MAGIC);
b[0x14..0x18].copy_from_slice(&w.to_be_bytes());
b[0x18..0x1c].copy_from_slice(&h.to_be_bytes());
b[0x1c..0x20].copy_from_slice(&1u32.to_be_bytes());
b.extend_from_slice(&0x30u32.to_be_bytes()); // offset table → rect at 0x30
for v in [0u32, 0, w, h] {
b.extend_from_slice(&v.to_be_bytes()); // dst X, dst Y, width, height
}
b.extend_from_slice(&vec![0x80u8; (w * h * 4) as usize]);
b
}

View File

@@ -0,0 +1,361 @@
//! Assembling media that does **not** sit in one place on the disc.
//!
//! Most assets are one archive entry and are read with [`crate::pak`] alone.
//! Audio is not, and this module owns every case where the bytes of one playable
//! thing have to be gathered from somewhere other than a single entry:
//!
//! * **An entry spans segment files.** A `.pak` TOC offset addresses the
//! *concatenated* `.p00….pNN` stream, so one entry routinely straddles two
//! files on disc. [`DiscSource::read_segment_range`] is the seam for that.
//! * **A bank holds several sub-waves.** A `.slb` is an XACT bank; its sub-waves
//! are either alternate takes or sequential segments of one line, and only
//! concatenating them all and clamping to the known length gets both right.
//! * **A cutscene voice is not in its own bank.** The movie voices are one
//! continuous XMA stream chunked into `VOICE_*.slb` TOC entries whose
//! boundaries do **not** match the cutscene cues. A cue routinely spans two
//! chunks, so *a `.slb` need not hold the track its name claims*.
//! [`resolve_movie_voice_region`] resolves a movie to a byte region of the
//! stream instead, which is the only reading that produces the right audio.
//!
//! ## Why this lives in `sylpheed-formats` and not in a viewer
//!
//! It used to live in the Bevy viewer, which meant the one piece of logic most
//! likely to be re-derived incorrectly was in the crate least likely to be
//! reused. Anything that reads the disc — the viewer, a CLI, an asset exporter
//! for a port — needs the same answers, and there must be one implementation of
//! them.
//!
//! ## What deliberately stays out
//!
//! Decoding. This module returns **XMA `RIFF`s**, not PCM: turning XMA into
//! samples means shelling out to FFmpeg, which is a native-only dependency and
//! a policy decision for the consumer. The seam is "here are the bytes that
//! belong together" — everything up to that point is disc knowledge, everything
//! after it is a codec choice.
use crate::pak::PakArchive;
use crate::slb::VoiceLang;
/// Where disc bytes come from. Implemented over an extracted directory, an ISO,
/// or anything else that can serve the same three questions.
///
/// It is a trait rather than a concrete type because the callers differ in ways
/// this module should not know about: a viewer reads from whichever source the
/// user opened, a headless exporter reads from a fixed extract, and a test reads
/// from a fixture.
pub trait DiscSource {
/// Read a whole file by disc-relative path, e.g. `dat/sound.pak`.
fn read_file(&self, path: &str) -> Result<Vec<u8>, String>;
/// Open an IPFB archive by disc-relative path, with its `.pNN` segments.
fn open_pak(&self, path: &str) -> Result<PakArchive, String>;
/// Read `len` bytes at `offset` into the concatenated `<stem>.p00….pNN`
/// stream, where `stem` is a disc-relative path without extension
/// (`dat/sound`). The range may cross a segment boundary; that is the point.
fn read_segment_range(&self, stem: &str, offset: u64, len: usize) -> Result<Vec<u8>, String>;
}
/// Read one `sound.pak` bank by name-hash, taking only its byte range from the
/// segments rather than inflating the 1.07 GB archive.
pub fn read_sound_bank<S: DiscSource + ?Sized>(
source: &S,
name_hash: u32,
) -> Result<Vec<u8>, String> {
let toc = source.read_file("dat/sound.pak")?;
let entries = PakArchive::parse_toc(&toc).map_err(|e| e.to_string())?;
let idx = entries
.binary_search_by_key(&name_hash, |e| e.name_hash)
.map_err(|_| "not present in sound.pak".to_string())?;
let e = &entries[idx];
source.read_segment_range("dat/sound", e.offset as u64, e.comp_size as usize)
}
/// The XMA `RIFF`s of one named bank, in the order they must be concatenated.
///
/// Every sub-wave is returned, not just the first. The two bank shapes need
/// this for opposite reasons: a **segment** bank (`VOICE_RT07A` = 24 s + 14 s +
/// 11 s ≈ the 50 s movie) is only complete when all of them are joined, and an
/// **alternate-take** bank (`VOICE_S00A`, whose sub-wave 0 already spans the
/// whole movie) is trimmed by the caller's length clamp. Taking sub-wave 0 alone
/// dropped two thirds of the dialogue on segment banks — that was a real bug.
pub fn sound_bank_riffs<S: DiscSource + ?Sized>(
source: &S,
clip_name: &str,
) -> Result<Vec<Vec<u8>>, String> {
let bytes = read_sound_bank(source, crate::hash::name_hash(clip_name))?;
Ok(riffs_of(&bytes))
}
/// One sound-effect wave out of a **delimiter-less** bank, as a decodable `RIFF`.
///
/// `Static.slb` — where the menu's cues live — has no `RIFF`, no `seek` chunk and
/// no XACT container: it is a packed run of whole 2048-byte XMA1 packets. So
/// [`sound_bank_riffs`] finds nothing to split on, and a wave is defined *only*
/// by `(offset, packet_count)`. Both come from the running game, not from the
/// file: launch Canary with `--xma_param_probe=true`, trigger the sound, and the
/// log prints the stream's packet count and first 32 bytes; searching those bytes
/// in the bank gives the offset. ⚠️ The file order is **not** cue-id order, so the
/// index cannot be counted out — see `docs/re/menu-audio-cues.md`.
///
/// The three cues a menu needs, all mono 48 kHz:
///
/// | event | offset | packets |
/// |---|---|---|
/// | d-pad move | `0x1ec0` | 4 |
/// | Ⓑ back | `0x0ec0` | 2 |
/// | Ⓐ confirm | `0x5d6c0` | 6 |
///
/// Returns an error rather than a short stream if the bank does not actually
/// hold `packet_count` whole packets at `offset` — a truncated XMA stream decodes
/// to plausible-sounding garbage, which is the failure worth refusing.
pub fn se_wave_riff<S: DiscSource + ?Sized>(
source: &S,
bank: &str,
offset: usize,
packet_count: usize,
channels: u8,
rate: u32,
) -> Result<Vec<u8>, String> {
let len = packet_count * crate::slb::XMA1_PACKET;
// Read only the packets asked for, not the whole bank. That is not just an
// efficiency point: `Static.slb` is 8.97 MB and is the ONE entry in
// `sound.pak` whose declared extent runs past the end of the extracted
// segments (by 616 768 B), so reading it whole fails outright on a disc
// extraction that is short at the tail. Every cue we need sits in the first
// few hundred KB. See `docs/re/menu-audio-cues.md`.
let toc = source.read_file("dat/sound.pak")?;
let entries = PakArchive::parse_toc(&toc).map_err(|e| e.to_string())?;
let hash = crate::hash::name_hash(bank);
let idx = entries
.binary_search_by_key(&hash, |e| e.name_hash)
.map_err(|_| format!("{bank}: not present in sound.pak"))?;
let e = &entries[idx];
if offset + len > e.comp_size as usize {
return Err(format!(
"{bank}: {packet_count} packets at {offset:#x} need {len} bytes, \
but the bank declares only {} bytes",
e.comp_size
));
}
let packets =
source.read_segment_range("dat/sound", e.offset as u64 + offset as u64, len)?;
Ok(crate::slb::xma1_wave_riff(&packets, channels, rate))
}
/// The XMA `RIFF`s of a continuous byte region of the voice stream, as returned
/// by [`resolve_movie_voice_region`].
pub fn voice_region_riffs<S: DiscSource + ?Sized>(
source: &S,
start: u64,
end: u64,
) -> Result<Vec<Vec<u8>>, String> {
let bytes = source.read_segment_range("dat/sound", start, (end - start) as usize)?;
Ok(riffs_of(&bytes))
}
/// Sub-wave `RIFF`s of a bank's bytes, with the single-stream fallback.
///
/// Some banks — the data-before-header `\etc\` radio clips — defeat the
/// multi-sub-wave scanner, and the robust single-stream reader handles them. An
/// empty result here means genuinely undecodable, not "scanner confused".
fn riffs_of(bytes: &[u8]) -> Vec<Vec<u8>> {
let riffs = crate::slb::to_xma_riffs(bytes);
if riffs.is_empty() {
crate::slb::to_xma_riff_best(bytes).into_iter().collect()
} else {
riffs
}
}
/// Resolve a movie's voice bank **name** through the manifest in `tables.pak`.
///
/// Only the manifest's DIRECT bindings are trusted. Extending this to unbound
/// resupply movies by shared demo line was tried and verified WRONG — it played
/// the wrong recording — so an unbound movie stays unvoiced rather than play a
/// guess. `None` therefore means "this cutscene has no voice-over", which is a
/// real answer for most `hokyu_*` movies.
pub fn resolve_movie_voice_clip<S: DiscSource + ?Sized>(
source: &S,
movie: &str,
lang: VoiceLang,
) -> Option<String> {
let pak = source.open_pak("dat/tables.pak").ok()?;
let manifest = find_manifest(&pak)?;
let sounds = pak
.read_by_name(&format!("{}\\sounds.tbl", lang.code_pub()))?
.ok()?;
crate::movie_manifest::resolve_voice_entry(&manifest, &sounds, movie, lang)
}
/// The manifest has no stable name, so it is found by shape among the entries.
fn find_manifest(pak: &PakArchive) -> Option<Vec<u8>> {
pak.entries().iter().find_map(|e| {
pak.read(e)
.ok()
.filter(|b| crate::movie_manifest::is_manifest(b))
})
}
/// Voice token for a hokyu (resupply) cutscene the manifest leaves unbound.
///
/// Only 5 of the 18 hokyu movies carry an explicit `VOICETRACK`; the rest reuse
/// those recordings. The selector is the cutscene's **demo id** (from its
/// subtitle track), NOT the ship category: `hokyu_LS_s02A` and `hokyu_LS_s11A`
/// are both LS/carrier but use demos 600 vs 601, whose lines differ. So the map
/// is derived from the 5 bound hokyu — each of which has both a subtitle demo id
/// and a `VOICETRACK` — and the target movie's demo id is looked up in it.
pub fn hokyu_voice_token<S: DiscSource + ?Sized>(
source: &S,
movie: &str,
lang: VoiceLang,
manifest: &[u8],
) -> Option<String> {
use crate::movie_subtitle as ms;
if !movie.starts_with("hokyu_") {
return None;
}
let lang_pak = source
.open_pak(&format!("dat/movie/{}.pak", lang.code_pub()))
.ok()?;
let want = ms::track_voice_cues(&lang_pak, movie).first().map(|&(d, _)| d)?;
crate::movie_manifest::parse(manifest)
.into_iter()
.find_map(|e| {
let tok = e.voice_token.filter(|_| e.movie.starts_with("hokyu_"))?;
ms::track_voice_cues(&lang_pak, &e.movie)
.iter()
.any(|&(d, _)| d == want)
.then_some(tok)
})
}
/// Resolve a movie's cutscene voice to a continuous `[start, end)` byte region
/// of the voice stream — **the reading that produces the right audio**.
///
/// The chain is movie → cue token (manifest) → sound id (master registry) →
/// region (scan the stream for two trailers). Each cue ends at an inline
/// `(sound_id, 0x11, …)` trailer, so cue *N* is the bytes between trailer *N-1*
/// and trailer *N*.
///
/// Returns `None` for movies whose voice is not a `\Movie\` bank — the hokyu
/// `\etc\` clips — which the caller then resolves the per-clip way via
/// [`resolve_movie_voice_clip`].
pub fn resolve_movie_voice_region<S: DiscSource + ?Sized>(
source: &S,
movie: &str,
lang: VoiceLang,
) -> Option<(u64, u64)> {
use crate::{hash::name_hash, movie_manifest, movie_voice};
let code = lang.code_pub();
let tpak = source.open_pak("dat/tables.pak").ok()?;
let manifest = find_manifest(&tpak)?;
let token = movie_manifest::voice_token(&manifest, movie)
.or_else(|| hokyu_voice_token(source, movie, lang, &manifest))?;
// token → sound id, via the large per-language IDXD entry carrying the
// `<lang>\Movie\VOICE_*.slb` paths. Located by content, like the manifest.
let marker = format!("{code}\\Movie\\VOICE_ADV.slb");
let registry = tpak.entries().iter().find_map(|e| {
tpak.read(e)
.ok()
.filter(|b| b.windows(marker.len()).any(|w| w == marker.as_bytes()))
})?;
let id = *movie_voice::registry_voice_ids(&registry).get(&token)?;
// Physical anchor: the TOC offset of this token's own `.slb` chunk. That is a
// start point NEAR the cue's trailers, not the cue itself — the cue may sit
// before or after it, which is the whole reason a region is needed. The
// token's subdirectory varies by kind.
let stoc = source.read_file("dat/sound.pak").ok()?;
let entries = PakArchive::parse_toc(&stoc).ok()?;
let anchor = ["Movie", "etc", "Voice"].iter().find_map(|dir| {
let h = name_hash(&format!("{code}\\{dir}\\{token}.slb"));
entries
.binary_search_by_key(&h, |e| e.name_hash)
.ok()
.map(|i| entries[i].offset as u64)
})?;
// Scan both directions from the anchor. The window must span the largest
// bank (ADV ≈ 3.6 MB) or the predecessor trailer falls outside it.
let win_start = anchor.saturating_sub(2 * 1024 * 1024) & !3;
let window = source
.read_segment_range("dat/sound", win_start, 8 * 1024 * 1024)
.ok()?;
let end_local = movie_voice::find_descriptor(&window, id)?;
let end = win_start + end_local as u64;
// Start = the predecessor trailer. Prefer the exact `id-1`; where the id
// sequence has a gap (VOICE_D_453 → 454) fall back to the nearest trailer
// below — but only within one bank (~1.5 MB), else this is the first cue in
// its block and the audio starts at the anchor itself.
let start = movie_voice::find_descriptor(&window, id.wrapping_sub(1))
.or_else(|| movie_voice::find_descriptor_before(&window, end_local))
.map(|o| win_start + o as u64)
.filter(|&s| s < end && end - s < 1_500_000)
.unwrap_or(anchor);
Some((start, end))
}
/// A [`DiscSource`] over an **extracted** disc directory.
///
/// Provided here rather than left to each caller because every headless
/// consumer — the CLI, the disc tests, an asset exporter for a port — wants
/// exactly this and would otherwise re-derive the segment-spanning read, which
/// is the part that is easy to get subtly wrong.
#[cfg(not(target_arch = "wasm32"))]
pub struct DirectorySource {
root: std::path::PathBuf,
}
#[cfg(not(target_arch = "wasm32"))]
impl DirectorySource {
pub fn new(root: impl Into<std::path::PathBuf>) -> Self {
Self { root: root.into() }
}
}
#[cfg(not(target_arch = "wasm32"))]
impl DiscSource for DirectorySource {
fn read_file(&self, path: &str) -> Result<Vec<u8>, String> {
std::fs::read(self.root.join(path)).map_err(|e| format!("{path}: {e}"))
}
fn open_pak(&self, path: &str) -> Result<PakArchive, String> {
PakArchive::open(self.root.join(path)).map_err(|e| format!("{path}: {e}"))
}
/// Walks `<stem>.p00`, `.p01`, … skipping whole segments until the offset is
/// inside one, then reads across as many as the length needs. A range that
/// straddles a boundary is the normal case, not an edge case.
fn read_segment_range(&self, stem: &str, offset: u64, len: usize) -> Result<Vec<u8>, String> {
use std::io::{Read, Seek, SeekFrom};
let mut out = Vec::with_capacity(len);
let (mut skip, mut need) = (offset, len);
for i in 0..100u32 {
if need == 0 {
break;
}
let path = self.root.join(format!("{stem}.p{i:02}"));
let Ok(meta) = std::fs::metadata(&path) else { break };
let seg_len = meta.len();
if skip >= seg_len {
skip -= seg_len;
continue;
}
let mut f = std::fs::File::open(&path).map_err(|e| e.to_string())?;
f.seek(SeekFrom::Start(skip)).map_err(|e| e.to_string())?;
let take = need.min((seg_len - skip) as usize);
let start = out.len();
out.resize(start + take, 0);
f.read_exact(&mut out[start..]).map_err(|e| e.to_string())?;
need -= take;
skip = 0;
}
if need != 0 {
return Err(format!("segment range short by {need} bytes"));
}
Ok(out)
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -100,6 +100,61 @@ const FIELD_KEYS: [&str; 4] = ["MOVIE", "VOICETRACK", "SUBTITLE", "TELOP"];
/// mission/phase; otherwise (a blob without the key array) the kind is inferred
/// from the movie name and `slot` is empty.
pub fn parse(bytes: &[u8]) -> Vec<MovieEntry> {
if let Some(rows) = parse_records(bytes) {
return rows;
}
parse_string_pool(bytes)
}
/// Read the manifest out of the IDXD **record table** — the authoritative path.
///
/// `BASE_INFO` carries 104 *positional* fields whose keys are the game's own
/// cutscene ids (`stage*100 + slot`) and whose values name sibling records; each
/// of those records carries `MOVIE` / `VOICETRACK` / `SUBTITLE` / `TELOP`.
///
/// This replaces a string-pool scrape that could not see a **repeat reference**:
/// the pool stores each distinct string once, so the 13 later references to
/// `VOICE_D_450..454`, the two to `SUBTITLE_hokyu_LS_s11A.tbl` and `MS01A`'s
/// share of `pwterop_s01a.prt` produced no token and read as "no binding". That
/// is why `hokyu_DS_s13A` was recorded as having no voice-over.
///
/// Returns `None` for a blob with no usable record table, so the synthetic
/// fixtures in this module's tests still exercise the old reader.
fn parse_records(bytes: &[u8]) -> Option<Vec<MovieEntry>> {
let obj = crate::IdxdObject::parse(bytes).ok()?;
let base = obj.record("BASE_INFO")?;
// Ids ascend; the game's own order is play order.
let mut ids: Vec<(u32, &str)> = base
.fields
.iter()
.filter_map(|f| f.index().map(|i| (i, f.value.as_str())))
.collect();
if ids.is_empty() {
return None;
}
ids.sort_unstable();
let mut out = Vec::with_capacity(ids.len());
for (_, slot) in ids {
let Some(rec) = obj.record(slot) else { continue };
let Some(movie) = rec.get("MOVIE") else { continue };
let (kind, mission, phase) = classify_slot(slot);
out.push(MovieEntry {
slot: slot.to_string(),
kind,
mission,
phase,
movie: movie.trim_end_matches(".wmv").to_string(),
voice_token: rec.get("VOICETRACK").map(after_plus),
subtitle: rec.get("SUBTITLE").map(after_plus),
telop: rec.get("TELOP").map(after_plus),
});
}
(!out.is_empty()).then_some(out)
}
/// The legacy string-pool reader. Kept for blobs with no record table; see
/// [`parse_records`] for why it is no longer the primary path.
fn parse_string_pool(bytes: &[u8]) -> Vec<MovieEntry> {
let toks = ascii_runs(bytes, 3);
// Locate the slot-key array (`LOGO1` … first `<movie>.wmv`) and the value
@@ -334,7 +389,11 @@ mod tests {
assert_eq!(m[0].subtitle.as_deref(), Some("SUBTITLE_S13A.tbl"));
assert_eq!(m[1].voice_token.as_deref(), Some("VOICE_D_450"));
assert_eq!(m[1].kind, MovieKind::Supply);
assert_eq!(m[2].voice_token, None, "hokyu_DS_s13A has no voice-over");
// NB: in the synthetic blob below this movie really has no token. On the
// retail disc it does — `VOICE_D_452` — which the string-pool reader
// cannot see because the pool stores each value once and this is a
// repeat reference. The disc test pins the real value.
assert_eq!(m[2].voice_token, None);
assert_eq!(m[2].kind, MovieKind::Supply);
}

View File

@@ -25,8 +25,10 @@ use crate::pak::PakArchive;
/// Subtitle language. `pak_code` selects `dat/movie/<code>.pak`; `game_code`
/// selects `dat/GP_MAIN_GAME_<code>.pak` (the caption text).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum SubLang {
/// Default only because the disc's own default is English.
#[default]
English,
Japanese,
German,
@@ -238,6 +240,76 @@ pub fn build_demo_text(text_pak: &PakArchive) -> BTreeMap<u32, Vec<String>> {
.collect()
}
/// Every caption family in the pack, not just the cutscene one.
///
/// `build_demo_text` reads `MSG_DEMO_*` — 560 text-bearing keys, the **smallest**
/// of eight families. The other seven carry the combat chatter and the in-mission
/// scripted dialogue: 44 019 more lines, or **98.7 %** of the game's text.
///
/// Key shapes, measured over every IXUD block in `GP_MAIN_GAME_E.pak`:
///
/// | family | shape | text-bearing keys |
/// |---|---|---|
/// | `ACRO` `ADAN` `ADPL` `BIRD` `DEMO` `RHIN` `TCAF` | `MSG_<FAM>_<id>_<page>_<line>` | 37 803 |
/// | `VOICE` | `MSG_VOICE_<letter>_<id>_<page>_<line>` | 6 776 |
///
/// `VOICE` is the only family with a letter before the id, and every family is
/// 100 % consistent with its own shape.
///
/// Returns `"<FAM>_<id>" → ordered lines`, e.g. `"ADAN_600"`, `"VOICE_A_150"`.
///
/// ⚠️ The id here is the **caption** id. It is *not* the voice-bank id: a message
/// page binding `VOICE_C_468` carries lines keyed `MSG_VOICE_C_385_*`. Same
/// family letter, different index space — do not derive one from the other.
pub fn build_caption_text(text_pak: &PakArchive) -> BTreeMap<String, Vec<String>> {
let mut by_id: BTreeMap<String, BTreeMap<(u32, u32), String>> = BTreeMap::new();
for entry in text_pak.entries() {
let Ok(bytes) = text_pak.read(entry) else {
continue;
};
let Some(obj) = crate::IxudObject::parse(&bytes) else {
continue;
};
// Read the caption as a FIELD -- its key is the `MSG_*` name and its
// value is the text. The token-adjacency version of this recovered 8074
// lines against the 44579 fields that actually carry text, because pool
// adjacency is a consequence of how records are written rather than a
// rule of the format. Same mistake the IDXD reader made.
for rec in obj.records() {
for f in &rec.fields {
let Some(key) = f.name.as_deref() else { continue };
let Some((id, page, line)) = caption_key(key) else {
continue;
};
if f.value.trim().is_empty() {
continue;
}
by_id.entry(id).or_default().insert((page, line), clean(&f.value));
}
}
}
by_id
.into_iter()
.map(|(d, m)| (d, m.into_values().collect()))
.collect()
}
/// `MSG_<FAM>_<id>_<page>_<line>` → `("<FAM>_<id>", page, line)`, with `VOICE`'s
/// extra family letter folded into the id.
fn caption_key(t: &str) -> Option<(String, u32, u32)> {
let rest = t.strip_prefix("MSG_")?;
let mut parts: Vec<&str> = rest.split('_').collect();
// Trailing <page>_<line> are always numeric.
let line: u32 = parts.pop()?.parse().ok()?;
let page: u32 = parts.pop()?.parse().ok()?;
// What remains is <FAM> or <FAM>_<letter>, then the numeric id.
let id: u32 = parts.pop()?.parse().ok()?;
if parts.is_empty() {
return None;
}
Some((format!("{}_{id:03}", parts.join("_")), page, line))
}
/// Parse a timing track's IXUD payload into `(token, start, end)` triples, where
/// `token` is either a `MSG_DEMO_<d>` reference or an inline caption string.
///

View File

@@ -190,6 +190,17 @@ impl PakArchive {
Ok(Self::from_parts(index, Vec::new())?.entries)
}
/// A raw slice of the concatenated segment data, by absolute offset.
///
/// Entry windows are **not** wave boundaries in `sound.pak`: a `.slb` wave
/// runs to `data_at + declared_size`, which routinely overruns the entry's
/// own `comp_size` (see docs/re/structures/slb-data-offset.md). Reading the
/// boundary marker therefore needs the flat stream, not the entry slice.
/// Returns `None` if the range falls outside the loaded data.
pub fn data_at(&self, offset: usize, len: usize) -> Option<&[u8]> {
self.data.get(offset..offset.checked_add(len)?)
}
/// All TOC entries, in stored order (ascending `name_hash`).
pub fn entries(&self) -> &[PakEntry] {
&self.entries
@@ -214,16 +225,34 @@ impl PakArchive {
}
/// The raw stored bytes for an entry (still `"Z1"`-wrapped / compressed).
///
/// One entry on the retail disc declares more bytes than the segments hold:
/// `sound.pak`'s `Static.slb` (the SFX bank) claims 8 970 240 bytes at the
/// highest offset in the archive, 616 768 past the end of `sound.p04`. It is
/// not corruption and it is not our extraction — `sound.p04` is byte-for-byte
/// the size the ISO's own directory record gives, and a sweep of **every**
/// `.pak` on the disc finds this one entry and no other. So the last entry's
/// `comp_size` is an allocation size, not a stored size.
///
/// A short read is therefore allowed **only** for the highest-offset entry,
/// which is the shape the evidence supports. Any other overrun is still an
/// error: that would be real damage, and clamping it would hide the damage
/// behind a half-decoded asset.
pub fn stored_bytes(&self, entry: &PakEntry) -> Result<&[u8], PakError> {
let start = entry.offset as usize;
let end = start + entry.comp_size as usize;
self.data
.get(start..end)
.ok_or(PakError::OffsetOutOfRange {
offset: entry.offset,
size: entry.comp_size,
data_len: self.data.len(),
})
if let Some(b) = self.data.get(start..end) {
return Ok(b);
}
let is_tail = self.entries.iter().all(|e| e.offset <= entry.offset);
if is_tail && start < self.data.len() {
return Ok(&self.data[start..]);
}
Err(PakError::OffsetOutOfRange {
offset: entry.offset,
size: entry.comp_size,
data_len: self.data.len(),
})
}
/// Decompress an entry to its raw payload bytes. Handles the `"Z1"` container

View File

@@ -62,7 +62,7 @@ pub fn parse(bytes: &[u8]) -> Option<Vec<RatcChild>> {
for (idx, &(off, kind)) in offs.iter().enumerate() {
let next = offs.get(idx + 1).map(|&(o, _)| o).unwrap_or(bytes.len());
children.push(RatcChild {
name: name_before(bytes, off),
name: opt_name(bytes, off).unwrap_or_else(|| name_before(bytes, off)),
kind: kind.to_string(),
offset: off,
size: next.saturating_sub(off),
@@ -71,9 +71,51 @@ pub fn parse(bytes: &[u8]) -> Option<Vec<RatcChild>> {
Some(children)
}
/// The name a child's own `opt ` block states, if it has one.
///
/// The real format is explicit. Immediately before each child sits
///
/// ```text
/// "opt " | BE32 length | name | NUL | 3 bytes | <child magic>
/// ```
///
/// -- the same `opt ` block `ui_layout`'s focus link already reads. Prefer it,
/// because [`name_before`] is a heuristic and those 3 trailing bytes are
/// sometimes printable, in which case the heuristic reads THEM as the name.
/// Measured disc-wide: of 18 002 RATC children, 17 942 carry an `opt ` block,
/// 17 918 of which agree with the scan and **24 do not** -- every one of the 24
/// a 3-byte tail (`8AX` x22, `'OX` x2) beating a real name. On the title screens
/// that cost the whole background: `pteff05.t32` came out as `8AX`, its element
/// then resolved to no sprite, and `compose` silently dropped it. See
/// `docs/re/structures/ratc-child-names.md`.
fn opt_name(bytes: &[u8], off: usize) -> Option<String> {
let lo = off.saturating_sub(128);
let win = &bytes[lo..off];
let pos = lo + win.windows(4).rposition(|w| w == b"opt ")?;
let len = u32::from_be_bytes(bytes.get(pos + 4..pos + 8)?.try_into().ok()?) as usize;
if len == 0 || len > 64 || pos + 8 + len > off {
return None;
}
// It must be THIS child's block: the name, its NUL and a short run to the
// magic. Anything further away is a neighbour's block, so fall back.
if off - (pos + 8 + len) > 8 {
return None;
}
let s = String::from_utf8_lossy(&bytes[pos + 8..pos + 8 + len]).to_string();
(!s.is_empty() && s.chars().all(|c| c.is_ascii_graphic())).then_some(s)
}
/// The nearest name string preceding `off`: the *last* printable run (len ≥ 3)
/// in the 96 bytes before the child magic. A few record-header bytes usually sit
/// between the name and the magic, so an exact-adjacency scan isn't enough.
///
/// Fallback only -- [`opt_name`] is the stated name. 60 of the disc's 18 002
/// children have no `opt ` block and still rely on this, and all 60 are
/// accounted for: they are the ten frames of the disc's only `.tan` frame
/// sequence, in six language copies of one `GP_READY_ROOM` bundle, where a
/// single `opt ` block names the whole run. See
/// `docs/re/structures/ratc-tan-frame-sequence.md` -- and note that this means
/// `parse` OVER-reports there, listing frames as children.
fn name_before(bytes: &[u8], off: usize) -> String {
let start = off.saturating_sub(96);
let window = &bytes[start..off];
@@ -101,6 +143,49 @@ fn name_before(bytes: &[u8], off: usize) -> String {
mod tests {
use super::*;
/// The `opt ` block wins over a printable tail.
///
/// This is the `pteff05.t32` / `8AX` case, byte for byte: the name is stated
/// with an explicit length, then a NUL, then three payload bytes that happen
/// to spell `8AX` in ASCII. The old backwards printable-run scan returned
/// `8AX` here, which is what dropped the background from every menu screen.
#[test]
fn opt_block_beats_a_printable_tail() {
let mut b = RATC_MAGIC.to_vec();
b.extend_from_slice(&[0u8; 28]);
b.extend_from_slice(b"opt ");
b.extend_from_slice(&11u32.to_be_bytes()); // len("pteff05.t32")
b.extend_from_slice(b"pteff05.t32\0");
b.extend_from_slice(b"8AX"); // payload, printable by accident
let off = b.len();
b.extend_from_slice(b"T8aD");
b.extend_from_slice(&[0u8; 16]);
let kids = parse(&b).expect("parse");
assert_eq!(kids.len(), 1);
assert_eq!(kids[0].name, "pteff05.t32");
assert_eq!(kids[0].offset, off);
// And the heuristic on its own really would have said `8AX` -- so this
// test fails for the right reason if the preference is ever reversed.
assert_eq!(name_before(&b, off), "8AX");
}
/// No `opt ` block: 60 of the disc's 18 002 children are like this, and they
/// must keep working off the scan.
#[test]
fn falls_back_to_the_scan_without_an_opt_block() {
let mut b = RATC_MAGIC.to_vec();
b.extend_from_slice(&[0u8; 28]);
b.extend_from_slice(b"plain.t32");
b.extend_from_slice(&[0x0e, 0x10, 0xa4]);
let off = b.len();
b.extend_from_slice(b"T8aD");
b.extend_from_slice(&[0u8; 16]);
let kids = parse(&b).expect("parse");
assert_eq!(kids[0].name, "plain.t32");
}
#[test]
fn lists_named_children() {
let mut b = RATC_MAGIC.to_vec();

View File

@@ -0,0 +1,604 @@
//! `savedata` — the retail save file.
//!
//! The whole save is **545 bytes**. It lives in the console's content tree as a
//! single file and there is no second one, so everything the game remembers
//! between sessions is in here:
//!
//! ```text
//! <content>/<XUID>/535107D4/00000001/game01/savedata
//! ```
//!
//! # Container
//!
//! ```text
//! 'GDHA' <146-byte header> <zlib stream, 78 DA>
//! ```
//!
//! Much of the header is **uninitialised memory** — words like `0x828F3DA8` are
//! guest virtual addresses that leaked out of the struct's padding — so it is not
//! byte-reproducible and must not be read as meaningful. Three header fields
//! *are* derived from the payload ([`Header::derived`]), and a handful more are a
//! **summary copy** of payload fields ([`Header::summary`]) — which matters more
//! than it sounds: the in-game Details panel reads the *summary*, not the
//! payload, so editing a payload field alone leaves a stale panel beside it. A
//! probe that judges "did that field change the display?" without patching the
//! mirror cannot tell *"wrong field"* from *"the panel never reads the payload"*.
//!
//! # Payload
//!
//! A chunk stream, read off the title's own serializer at `0x822C00E8` (its
//! writer primitive is `0x821885A8(stream, buf, len)`) rather than guessed, and
//! verified by a byte-identical round-trip:
//!
//! ```text
//! 'GDAA' payload magic
//! u32 len, char[len] current game phase, e.g. "GP_BUNK"
//! 'GHAD' + 122 bytes the progress block
//! u32 16, 16 x ('SHAB' + 5 x u32) the per-stage record table
//! u32 4, "BUNK", 'NETA', u32 trailer
//! ```
//!
//! The in-memory struct is written field-by-field with no packing changes, so a
//! payload offset **is** the offset in the live save object: GHAD at `save+8`,
//! the record table at `save+136`, and the serializer's next access after the
//! table is `lwz r11,456(save)` — exactly `136 + 16*20`.
//!
//! # Confidence
//!
//! The container and the chunk layout are ✅ `CONFIRMED`. Individual GHAD fields
//! are **not** uniformly known — eleven are still ❔ — so every field carries its
//! own [`Confidence`] and a viewer is expected to show it rather than present the
//! lot as solved. See `docs/re/structures/savegame-format.md`.
use std::io::Read;
/// How well a field is understood. Mirrors the `docs/re` convention.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Confidence {
/// Named and verified against the game.
Confirmed,
/// A well-supported reading, not proven.
Probable,
/// Unidentified.
Unknown,
/// A reading that was tested and **refuted** — recorded so it is not retried.
Refuted,
}
/// How to render a GHAD word.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum FieldKind {
/// Plain 32-bit count / enum / id.
U32,
/// 64-bit word.
U64,
/// Milliseconds, displayed as `m:ss.mmm`.
Millis,
/// Percent.
Percent,
/// Raw bytes.
Raw,
/// The per-item Arsenal development blob.
DevelopBlob,
}
/// One documented field of the GHAD progress block.
#[derive(Debug, Clone, Copy)]
pub struct FieldSpec {
/// Offset within the 122-byte GHAD block.
pub offset: usize,
/// Byte width.
pub size: usize,
pub kind: FieldKind,
pub confidence: Confidence,
/// Short name, or `""` when the field is unidentified.
pub name: &'static str,
/// What is known — including what was ruled out.
pub note: &'static str,
}
/// The GHAD block's layout, read off `0x822BF678`: ten u32, one u64
/// (`ld r11,40(r30)`), four u32, a raw 4-byte field and a raw 54-byte blob.
/// 10·4 + 8 + 4·4 + 4 + 54 = **122**, exactly what the file carries — the layout
/// is closed, with nothing unaccounted for.
pub const GHAD_LAYOUT: &[FieldSpec] = &[
FieldSpec { offset: 0, size: 4, kind: FieldKind::U32, confidence: Confidence::Unknown, name: "", note: "" },
FieldSpec { offset: 4, size: 4, kind: FieldKind::Millis, confidence: Confidence::Confirmed, name: "FlightTime", note: "Milliseconds. 324773 ms against the panel's `Flight Time 000:05:24`." },
FieldSpec { offset: 8, size: 4, kind: FieldKind::Percent, confidence: Confidence::Confirmed, name: "ClearRatio", note: "Percent. Not a stage counter: developing one Arsenal weapon stepped it 5 → 6, so it counts collection too." },
FieldSpec { offset: 12, size: 4, kind: FieldKind::U32, confidence: Confidence::Probable, name: "TimesCleared", note: "Mirrored to header +0x28; the panel prints `Times Cleared`." },
FieldSpec { offset: 16, size: 4, kind: FieldKind::U32, confidence: Confidence::Unknown, name: "", note: "" },
FieldSpec { offset: 20, size: 4, kind: FieldKind::U32, confidence: Confidence::Unknown, name: "", note: "" },
FieldSpec { offset: 24, size: 4, kind: FieldKind::U32, confidence: Confidence::Confirmed, name: "Points", note: "The spendable balance. Separated from +28 by a develop differential: spending 4000 P moved only this field (4101 → 101)." },
FieldSpec { offset: 28, size: 4, kind: FieldKind::U32, confidence: Confidence::Probable, name: "PointsTotal?", note: "Not the displayed Points — it did NOT move when 4000 P was spent. A lifetime/earned total is the obvious read; unproven until a save is taken after earning." },
FieldSpec { offset: 32, size: 4, kind: FieldKind::U32, confidence: Confidence::Unknown, name: "", note: "" },
FieldSpec { offset: 36, size: 4, kind: FieldKind::U32, confidence: Confidence::Refuted, name: "", note: "Tested as difficulty and as stage; probe saves refuted both." },
FieldSpec { offset: 40, size: 8, kind: FieldKind::U64, confidence: Confidence::Unknown, name: "", note: "" },
FieldSpec { offset: 48, size: 4, kind: FieldKind::U32, confidence: Confidence::Probable, name: "GameStatus", note: "Mirrored to header +0x18; 0 renders `At Standby`, matching the screen's STATE_STAND_BY / STATE_STAGE_CLEAR / STATE_GAME_CLEAR list." },
FieldSpec { offset: 52, size: 4, kind: FieldKind::U32, confidence: Confidence::Confirmed, name: "Stage", note: "1-based stage number. Set it to 5 (with the header mirror) and the game reads `STAGE 05 — Star System Escape`, loads it, and flies it." },
FieldSpec { offset: 56, size: 4, kind: FieldKind::U32, confidence: Confidence::Refuted, name: "", note: "See +36 — refuted." },
FieldSpec { offset: 60, size: 4, kind: FieldKind::U32, confidence: Confidence::Unknown, name: "", note: "" },
FieldSpec { offset: 64, size: 4, kind: FieldKind::Raw, confidence: Confidence::Unknown, name: "", note: "The trailer's u32 carries the same value." },
FieldSpec { offset: 68, size: 54, kind: FieldKind::DevelopBlob, confidence: Confidence::Confirmed, name: "DevelopState", note: "One byte per Arsenal item. Index space is strings.tbl's item order, cut items included, closing at 53." },
];
/// Size of the GHAD progress block.
pub const GHAD_SIZE: usize = 122;
/// Records in the per-stage table.
pub const RECORD_COUNT: usize = 16;
/// `u32`s per record, after the `SHAB` tag.
pub const RECORD_FIELDS: usize = 5;
/// Length of the Arsenal development blob.
pub const DEVELOP_BLOB_LEN: usize = 54;
/// State of one Arsenal item in the development blob.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum DevelopState {
/// `0` — not yet available.
Locked,
/// `2` — available to develop now. **Re-derived at load**, not stored state.
Developable,
/// `4` — owned.
Developed,
/// Anything else, which no save has yet shown.
Other(u8),
}
impl DevelopState {
pub fn from_byte(b: u8) -> Self {
match b {
0 => Self::Locked,
2 => Self::Developable,
4 => Self::Developed,
other => Self::Other(other),
}
}
}
/// One `SHAB` record — a per-stage result, record 0 being Stage 01.
///
/// These are **not** the UI's 20 save slots, which is what the count of 16
/// first suggested.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct StageRecord {
/// 🟡 difficulty.
pub a: u32,
/// ❔ a points figure — equals GHAD `+24`/`+28`, so not the displayed high
/// score (which is computed at display time and stored nowhere).
pub b: u32,
/// ✅ best clear time in milliseconds — 324 773 against the screen's
/// `05:24.77`.
pub best_time_ms: u32,
/// Windows FILETIME, high then low word.
pub time_hi: u32,
pub time_lo: u32,
}
impl StageRecord {
/// Whether the record holds a result at all.
pub fn is_used(&self) -> bool {
self.a != 0 || self.b != 0 || self.best_time_ms != 0
}
/// The record's FILETIME as 100 ns ticks since 1601, or `None` if unset.
pub fn filetime_ticks(&self) -> Option<u64> {
let t = (u64::from(self.time_hi) << 32) | u64::from(self.time_lo);
(t != 0).then_some(t)
}
}
/// The GDHA container header.
#[derive(Debug, Clone)]
pub struct Header {
/// The header bytes, verbatim.
pub bytes: Vec<u8>,
}
/// A header word that mirrors a payload field, with where it came from.
#[derive(Debug, Clone, Copy)]
pub struct Mirror {
pub header_offset: usize,
pub value: u32,
/// The GHAD offset this copies, if it is a GHAD field.
pub ghad_offset: Option<usize>,
pub name: &'static str,
}
impl Header {
fn be32(&self, o: usize) -> u32 {
if o + 4 > self.bytes.len() {
return 0;
}
u32::from_be_bytes([
self.bytes[o],
self.bytes[o + 1],
self.bytes[o + 2],
self.bytes[o + 3],
])
}
/// The summary copy the in-game Details panel actually reads.
///
/// This is the trap the format doc records: a payload edit that leaves these
/// stale shows no change on the panel, which is *not* evidence that the
/// payload field was the wrong one.
pub fn summary(&self) -> Vec<Mirror> {
vec![
Mirror { header_offset: 0x14, value: self.be32(0x14), ghad_offset: Some(52), name: "Stage" },
Mirror { header_offset: 0x1c, value: self.be32(0x1c), ghad_offset: Some(24), name: "Points" },
Mirror { header_offset: 0x20, value: self.be32(0x20), ghad_offset: Some(4), name: "FlightTime" },
Mirror { header_offset: 0x24, value: self.be32(0x24), ghad_offset: Some(8), name: "ClearRatio" },
Mirror { header_offset: 0x28, value: self.be32(0x28), ghad_offset: Some(12), name: "TimesCleared" },
]
}
/// The three header fields derived from the payload — all that stands between
/// a parse and a hand-written save the title will load.
pub fn derived(&self) -> Vec<Mirror> {
vec![
Mirror { header_offset: 0x30, value: self.be32(0x30), ghad_offset: None, name: "deflate length + 10" },
Mirror { header_offset: 0x8c, value: u32::from(u16::from_be_bytes([self.bytes.get(0x8c).copied().unwrap_or(0), self.bytes.get(0x8d).copied().unwrap_or(0)])), ghad_offset: None, name: "payload length" },
Mirror { header_offset: 0x8e, value: self.be32(0x8e), ghad_offset: None, name: "adler32(payload)" },
]
}
}
/// A fully parsed save.
#[derive(Debug, Clone)]
pub struct SaveGame {
pub header: Header,
/// The inflated payload, verbatim — the round-trip compares against this.
pub payload: Vec<u8>,
/// Current game phase, e.g. `GP_BUNK`. One of the title's screen ids.
pub phase: String,
/// The 122-byte progress block.
pub ghad: Vec<u8>,
/// The per-stage record table.
pub records: Vec<StageRecord>,
/// Trailer: `(name, tag, value)`.
pub trailer: (String, String, u32),
}
/// What went wrong reading a save.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum SaveError {
NotGdha,
NoZlibStream,
Inflate(String),
NotGdaa,
Truncated(&'static str),
/// A chunk tag was not where the serializer puts it.
BadTag { expected: &'static str, at: usize },
/// Bytes left over after the trailer — the layout is supposed to close
/// exactly, so this means the parse is wrong, not that the file has extras.
TrailingBytes(usize),
}
impl std::fmt::Display for SaveError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::NotGdha => write!(f, "not a GDHA container"),
Self::NoZlibStream => write!(f, "no zlib stream in the container"),
Self::Inflate(e) => write!(f, "inflate failed: {e}"),
Self::NotGdaa => write!(f, "payload is not GDAA"),
Self::Truncated(what) => write!(f, "payload truncated reading {what}"),
Self::BadTag { expected, at } => write!(f, "expected {expected} at {at:#x}"),
Self::TrailingBytes(n) => write!(f, "{n} bytes left after the trailer"),
}
}
}
impl std::error::Error for SaveError {}
fn be32(b: &[u8], o: usize) -> Option<u32> {
b.get(o..o + 4)
.map(|s| u32::from_be_bytes([s[0], s[1], s[2], s[3]]))
}
/// Split a `GDHA` container into its header and inflated payload.
pub fn unwrap_container(raw: &[u8]) -> Result<(Header, Vec<u8>), SaveError> {
if raw.len() < 4 || &raw[0..4] != b"GDHA" {
return Err(SaveError::NotGdha);
}
let at = raw
.windows(2)
.position(|w| w == [0x78, 0xDA])
.ok_or(SaveError::NoZlibStream)?;
let mut out = Vec::new();
flate2::read::ZlibDecoder::new(&raw[at..])
.read_to_end(&mut out)
.map_err(|e| SaveError::Inflate(e.to_string()))?;
Ok((
Header {
bytes: raw[..at].to_vec(),
},
out,
))
}
/// Parse an inflated payload.
pub fn parse_payload(payload: &[u8]) -> Result<SaveGame, SaveError> {
if payload.len() < 4 || &payload[0..4] != b"GDAA" {
return Err(SaveError::NotGdaa);
}
let mut o = 4usize;
let nlen = be32(payload, o).ok_or(SaveError::Truncated("phase length"))? as usize;
o += 4;
let phase = payload
.get(o..o + nlen)
.map(|s| String::from_utf8_lossy(s).to_string())
.ok_or(SaveError::Truncated("phase name"))?;
o += nlen;
if payload.get(o..o + 4) != Some(b"GHAD") {
return Err(SaveError::BadTag { expected: "GHAD", at: o });
}
o += 4;
let ghad = payload
.get(o..o + GHAD_SIZE)
.ok_or(SaveError::Truncated("GHAD block"))?
.to_vec();
o += GHAD_SIZE;
let count = be32(payload, o).ok_or(SaveError::Truncated("record count"))? as usize;
o += 4;
let mut records = Vec::with_capacity(count);
for _ in 0..count {
if payload.get(o..o + 4) != Some(b"SHAB") {
return Err(SaveError::BadTag { expected: "SHAB", at: o });
}
o += 4;
let mut v = [0u32; RECORD_FIELDS];
for slot in v.iter_mut() {
*slot = be32(payload, o).ok_or(SaveError::Truncated("record"))?;
o += 4;
}
records.push(StageRecord {
a: v[0],
b: v[1],
best_time_ms: v[2],
time_hi: v[3],
time_lo: v[4],
});
}
let tlen = be32(payload, o).ok_or(SaveError::Truncated("trailer length"))? as usize;
o += 4;
let tname = payload
.get(o..o + tlen)
.map(|s| String::from_utf8_lossy(s).to_string())
.ok_or(SaveError::Truncated("trailer name"))?;
o += tlen;
let ttag = payload
.get(o..o + 4)
.map(|s| String::from_utf8_lossy(s).to_string())
.ok_or(SaveError::Truncated("trailer tag"))?;
o += 4;
let tval = be32(payload, o).ok_or(SaveError::Truncated("trailer value"))?;
o += 4;
if o != payload.len() {
return Err(SaveError::TrailingBytes(payload.len() - o));
}
Ok(SaveGame {
header: Header { bytes: Vec::new() },
payload: payload.to_vec(),
phase,
ghad,
records,
trailer: (tname, ttag, tval),
})
}
/// Parse a whole `savedata` file.
pub fn parse(raw: &[u8]) -> Result<SaveGame, SaveError> {
let (header, payload) = unwrap_container(raw)?;
let mut save = parse_payload(&payload)?;
save.header = header;
Ok(save)
}
impl SaveGame {
/// Re-serialize the payload.
///
/// This is the correctness check for the whole layout: the serializer writes
/// the struct field-by-field with no packing, so a correct parse must
/// reproduce the payload **byte for byte**. See [`SaveGame::round_trips`].
pub fn serialize_payload(&self) -> Vec<u8> {
let mut out = Vec::with_capacity(self.payload.len());
out.extend_from_slice(b"GDAA");
out.extend_from_slice(&(self.phase.len() as u32).to_be_bytes());
out.extend_from_slice(self.phase.as_bytes());
out.extend_from_slice(b"GHAD");
out.extend_from_slice(&self.ghad);
out.extend_from_slice(&(self.records.len() as u32).to_be_bytes());
for r in &self.records {
out.extend_from_slice(b"SHAB");
for v in [r.a, r.b, r.best_time_ms, r.time_hi, r.time_lo] {
out.extend_from_slice(&v.to_be_bytes());
}
}
let (name, tag, val) = &self.trailer;
out.extend_from_slice(&(name.len() as u32).to_be_bytes());
out.extend_from_slice(name.as_bytes());
out.extend_from_slice(tag.as_bytes());
out.extend_from_slice(&val.to_be_bytes());
out
}
/// Whether re-serializing reproduces the payload exactly.
pub fn round_trips(&self) -> bool {
self.serialize_payload() == self.payload
}
/// Read a GHAD field as an unsigned integer. `None` for the raw/blob fields.
pub fn ghad_value(&self, spec: &FieldSpec) -> Option<u64> {
match spec.size {
4 => be32(&self.ghad, spec.offset).map(u64::from),
8 => self
.ghad
.get(spec.offset..spec.offset + 8)
.map(|s| u64::from_be_bytes(s.try_into().unwrap())),
_ => None,
}
}
/// Raw bytes of a GHAD field.
pub fn ghad_bytes(&self, spec: &FieldSpec) -> &[u8] {
self.ghad
.get(spec.offset..spec.offset + spec.size)
.unwrap_or(&[])
}
/// The per-item Arsenal development state.
///
/// Index space is `strings.tbl`'s item order — the Arsenal's display order
/// *plus* the cut items only the localisation file lists — closing at 53.
/// `weapon.tbl`'s id list is **not** the index space; that it is also 54 long
/// is a coincidence, and the two agree only to index 32.
pub fn develop_state(&self) -> Vec<DevelopState> {
self.ghad
.get(68..68 + DEVELOP_BLOB_LEN)
.map(|b| b.iter().copied().map(DevelopState::from_byte).collect())
.unwrap_or_default()
}
/// Convenience: the 1-based stage number (GHAD `+52`).
pub fn stage(&self) -> Option<u32> {
be32(&self.ghad, 52)
}
/// Convenience: the spendable points balance (GHAD `+24`).
pub fn points(&self) -> Option<u32> {
be32(&self.ghad, 24)
}
/// Convenience: total flight time in milliseconds (GHAD `+4`).
pub fn flight_time_ms(&self) -> Option<u32> {
be32(&self.ghad, 4)
}
/// Convenience: clear ratio in percent (GHAD `+8`).
pub fn clear_ratio_pct(&self) -> Option<u32> {
be32(&self.ghad, 8)
}
}
/// Format a millisecond count the way the game's panels do (`m:ss.mmm`).
pub fn fmt_millis(ms: u32) -> String {
let (m, rem) = (ms / 60_000, ms % 60_000);
format!("{}:{:02}.{:03}", m, rem / 1000, rem % 1000)
}
#[cfg(test)]
mod tests {
use super::*;
/// Build a payload with the exact chunk stream the serializer writes.
fn synth_payload(phase: &str, stage: u32, points: u32) -> Vec<u8> {
let mut ghad = vec![0u8; GHAD_SIZE];
ghad[4..8].copy_from_slice(&324_773u32.to_be_bytes()); // flight time
ghad[8..12].copy_from_slice(&5u32.to_be_bytes()); // clear ratio
ghad[24..28].copy_from_slice(&points.to_be_bytes());
ghad[52..56].copy_from_slice(&stage.to_be_bytes());
ghad[68] = 4; // item 0 developed
ghad[69] = 2; // item 1 developable
let mut p = Vec::new();
p.extend_from_slice(b"GDAA");
p.extend_from_slice(&(phase.len() as u32).to_be_bytes());
p.extend_from_slice(phase.as_bytes());
p.extend_from_slice(b"GHAD");
p.extend_from_slice(&ghad);
p.extend_from_slice(&(RECORD_COUNT as u32).to_be_bytes());
for i in 0..RECORD_COUNT {
p.extend_from_slice(b"SHAB");
let vals = if i == 0 {
[2u32, 4101, 324_773, 0x01DC_0000, 0x1234_5678]
} else {
[0, 0, 0, 0, 0]
};
for v in vals {
p.extend_from_slice(&v.to_be_bytes());
}
}
p.extend_from_slice(&4u32.to_be_bytes());
p.extend_from_slice(b"BUNK");
p.extend_from_slice(b"NETA");
p.extend_from_slice(&0x0915_0000u32.to_be_bytes());
p
}
#[test]
fn parses_the_chunk_stream() {
let p = synth_payload("GP_BUNK", 2, 4101);
let s = parse_payload(&p).unwrap();
assert_eq!(s.phase, "GP_BUNK");
assert_eq!(s.ghad.len(), GHAD_SIZE);
assert_eq!(s.records.len(), RECORD_COUNT);
assert_eq!(s.trailer, ("BUNK".into(), "NETA".into(), 0x0915_0000));
assert_eq!(s.stage(), Some(2));
assert_eq!(s.points(), Some(4101));
assert_eq!(s.flight_time_ms(), Some(324_773));
assert_eq!(s.clear_ratio_pct(), Some(5));
}
#[test]
fn round_trips_byte_identically() {
// The layout is closed, so a correct parse must reproduce the payload
// exactly. This is the property the whole spec rests on.
let p = synth_payload("GP_BUNK", 2, 4101);
let s = parse_payload(&p).unwrap();
assert!(s.round_trips());
assert_eq!(s.serialize_payload(), p);
}
#[test]
fn record_zero_is_stage_one() {
let s = parse_payload(&synth_payload("GP_BUNK", 2, 4101)).unwrap();
assert!(s.records[0].is_used());
assert!(!s.records[1].is_used());
assert_eq!(s.records[0].best_time_ms, 324_773);
assert!(s.records[0].filetime_ticks().is_some());
assert!(s.records[1].filetime_ticks().is_none());
}
#[test]
fn develop_blob_alphabet() {
let s = parse_payload(&synth_payload("GP_BUNK", 2, 4101)).unwrap();
let d = s.develop_state();
assert_eq!(d.len(), DEVELOP_BLOB_LEN);
assert_eq!(d[0], DevelopState::Developed);
assert_eq!(d[1], DevelopState::Developable);
assert_eq!(d[2], DevelopState::Locked);
}
#[test]
fn ghad_layout_is_closed() {
// 10*4 + 8 + 4*4 + 4 + 54 = 122, exactly what the file carries. If this
// ever fails the layout has grown a gap and the round-trip is a lie.
let total: usize = GHAD_LAYOUT.iter().map(|f| f.size).sum();
assert_eq!(total, GHAD_SIZE);
let mut at = 0;
for f in GHAD_LAYOUT {
assert_eq!(f.offset, at, "gap before {:#x}", f.offset);
at += f.size;
}
}
#[test]
fn rejects_a_non_save() {
assert_eq!(parse(b"not a save at all").unwrap_err(), SaveError::NotGdha);
assert_eq!(parse_payload(b"XXXX").unwrap_err(), SaveError::NotGdaa);
}
#[test]
fn trailing_bytes_are_an_error_not_a_shrug() {
let mut p = synth_payload("GP_BUNK", 2, 4101);
p.push(0);
assert_eq!(parse_payload(&p).unwrap_err(), SaveError::TrailingBytes(1));
}
#[test]
fn millis_format_matches_the_panel() {
assert_eq!(fmt_millis(324_773), "5:24.773");
}
}

View File

@@ -615,18 +615,64 @@ mod tests {
}
}
}
// Multi-instance coverage the capture couldn't see (vbase dedup).
let count = |res: &str| placed.iter().filter(|p| p.resource == res).count();
assert_eq!(count("e106_eng_01"), 2, "both engine nacelles placed");
assert_eq!(count("e303_wep_01"), 2, "both shared turrets placed");
// The mirrored starboard hull reflects (det < 0), the port one doesn't.
// ── Extras: the direction this test could not originally fail in. ──
// The loop above walks the CAPTURE's parts and looks each up in ours, so
// a static placement with no counterpart is invisible to it — which is
// how a resource decoded 100x too large (`e303_wep_01`, 2026-08-12) sat
// here unnoticed. Pin the whole **multiset** rather than the set of
// names: a set cannot fail on a *duplicated* instance of a resource the
// capture already lists, and duplication is exactly what a bad node walk
// produces. This subsumes the per-resource counts that used to be
// asserted separately below.
//
// Two deviations from the capture are expected, and only these two:
// * `e303_wep_01` x2 — a turret shared with `e101`, cross-mounted here.
// Absent from the capture because the F10 dump dedups by vertex base
// and both instances draw the one buffer.
// * `e106_eng_01` x2 — the mirrored engine nacelles, same reason.
let mut counts: std::collections::BTreeMap<&str, usize> = Default::default();
for p in &placed {
*counts.entry(p.resource.as_str()).or_default() += 1;
}
let expected: std::collections::BTreeMap<&str, usize> = [
("e106_bdy_01", 1),
("e106_bdy_02", 1),
("e106_bdy_03", 1),
("e106_bdy_04", 1),
("e106_brg_01", 1),
("e106_eng_01", 2),
("e106_eng_02", 1),
("e106_wep_02_01", 1),
("e303_wep_01", 2),
]
.into_iter()
.collect();
assert_eq!(
counts, expected,
"the static assembly's placement multiset drifted from the e106 ground truth"
);
// …and every name the capture lists must be in that table, so editing the
// table cannot quietly drop a part the runtime actually draws.
for want in &cap.parts {
assert!(
expected.contains_key(want.part.as_str()),
"{} is in the runtime capture but not in the expected multiset",
want.part
);
}
// NEITHER hull reflects: the twins' geometry is mirrored on the disc,
// so both placements are proper rotations. This flipped on 2026-08-12 —
// while both twins decoded to one buffer, `apply_twin_mirrors` had to
// synthesise the reflection here; a runtime capture showed the container
// holds both halves, and distinct anchor assignment now hands each twin
// its own (see docs/re/structures/xbg7-mesh.md).
let det = |m: &[[f32; 3]; 3]| {
m[0][0] * (m[1][1] * m[2][2] - m[1][2] * m[2][1])
- m[0][1] * (m[1][0] * m[2][2] - m[1][2] * m[2][0])
+ m[0][2] * (m[1][0] * m[2][1] - m[1][1] * m[2][0])
};
let one = |res: &str| placed.iter().find(|p| p.resource == res).unwrap();
assert!(det(&one("e106_bdy_02").m) < 0.0, "starboard hull mirrored");
assert!(det(&one("e106_bdy_02").m) > 0.0, "starboard hull plain (mirror is in the data)");
assert!(det(&one("e106_bdy_01").m) > 0.0, "port hull plain");
}

View File

@@ -49,6 +49,33 @@ pub struct CapturedDraw {
/// First few LOCAL vertex positions dumped with the draw (buffer order).
/// Used to disambiguate same-vcount twins (mirrored port/starboard parts).
pub pos: Vec<[f32; 3]>,
/// The draw's INDEX buffer, when the capture recorded one (`ib base=…`,
/// added 2026-08-13): guest base address, index count, and the min/max index
/// value the emulator read out of guest memory. `None` for older logs and
/// for auto-index draws. This is ground truth for two things the offline
/// decoder can only assume — where a block's index buffer lives relative to
/// its vertex buffer, and how much of the vertex pool a draw really covers.
pub ib: Option<CapturedIndexBuffer>,
}
/// The index buffer a captured draw used. See [`CapturedDraw::ib`].
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct CapturedIndexBuffer {
/// Guest base address of the index data.
pub ibase: u32,
/// Number of indices the draw issued (== `VGT_DRAW_INITIATOR.num_indices`).
pub icount: u32,
/// Lowest index value in the buffer.
pub imin: u32,
/// Highest index value in the buffer — with `vcount` this says whether the
/// draw covers its whole vertex pool or only a sub-range.
pub imax: u32,
/// The first indices, verbatim (the capture prints up to 24). Byte-level
/// ground truth for the offline index decode: a matched block's decoded
/// index prefix must equal this run.
pub head: [u32; 24],
/// How many of `head` the capture actually carried.
pub head_len: u8,
}
/// A ship part to match against the capture. `part` is the **base** part name
@@ -100,13 +127,16 @@ pub fn parse_capture(text: &str) -> Vec<CapturedDraw> {
let mut vcount = 0u32;
let mut pos: Vec<[f32; 3]> = Vec::new();
let mut consts: Vec<(usize, [f64; 4])> = Vec::new();
let mut ib: Option<CapturedIndexBuffer> = None;
let flush = |vbase: u32,
vcount: u32,
pos: &mut Vec<[f32; 3]>,
ib: &mut Option<CapturedIndexBuffer>,
consts: &[(usize, [f64; 4])],
out: &mut Vec<CapturedDraw>| {
let pos = std::mem::take(pos);
let ib = ib.take();
if vbase == 0 {
return;
}
@@ -115,18 +145,47 @@ pub fn parse_capture(text: &str) -> Vec<CapturedDraw> {
return; // no WorldView for this draw — skip it
};
if let Some((r, t)) = normalize_wvp([c0, c1, c2]) {
out.push(CapturedDraw { vbase, vcount, r, t, pos });
out.push(CapturedDraw { vbase, vcount, r, t, pos, ib });
}
};
for line in text.lines() {
let l = line.trim();
if let Some(rest) = l.strip_prefix("DRAW ") {
flush(vbase, vcount, &mut pos, &consts, &mut out);
flush(vbase, vcount, &mut pos, &mut ib, &consts, &mut out);
consts.clear();
let f = |k: &str| rest.split_whitespace().find_map(|t| t.strip_prefix(k));
vbase = f("vbase=0x").and_then(|s| u32::from_str_radix(s, 16).ok()).unwrap_or(0);
vcount = f("vcount=").and_then(|s| s.parse().ok()).unwrap_or(0);
} else if let Some(rest) = l.strip_prefix("ib base=0x") {
// `ib base=0x… count=N fmt=u16 endian=E len=L delta_vb=D min=a max=b idx: …`
let f = |k: &str| rest.split_whitespace().find_map(|t| t.strip_prefix(k));
let base = rest
.split_whitespace()
.next()
.and_then(|s| u32::from_str_radix(s, 16).ok());
if let (Some(ibase), Some(icount)) = (base, f("count=").and_then(|s| s.parse().ok())) {
let mut head = [0u32; 24];
let mut head_len = 0u8;
if let Some((_, list)) = l.split_once("idx:") {
for tok in list.split_whitespace() {
let Ok(v) = tok.parse::<u32>() else { break };
if head_len as usize >= head.len() {
break;
}
head[head_len as usize] = v;
head_len += 1;
}
}
ib = Some(CapturedIndexBuffer {
ibase,
icount,
imin: f("min=").and_then(|s| s.parse().ok()).unwrap_or(0),
imax: f("max=").and_then(|s| s.parse().ok()).unwrap_or(0),
head,
head_len,
});
}
} else if l.starts_with("pos:") || l.starts_with("positions:") {
pos = parse_pos_line(l, 8);
} else if l.starts_with("vsconst") {
@@ -147,7 +206,7 @@ pub fn parse_capture(text: &str) -> Vec<CapturedDraw> {
}
}
}
flush(vbase, vcount, &mut pos, &consts, &mut out);
flush(vbase, vcount, &mut pos, &mut ib, &consts, &mut out);
out
}
@@ -242,7 +301,9 @@ pub fn parse_drawlog(text: &str) -> Vec<CapturedDraw> {
let get = |i: usize| consts.iter().find(|(k, _)| *k == i).map(|(_, v)| *v);
let (Some(c0), Some(c1), Some(c2)) = (get(0), get(1), get(2)) else { return };
if let Some((r, t)) = normalize_wvp([c0, c1, c2]) {
out.push(CapturedDraw { vbase: base, vcount: size / stride, r, t, pos });
// The draw-logger format carries an index base too, but it de-dups
// by vertex declaration, so it never lines up per part — left None.
out.push(CapturedDraw { vbase: base, vcount: size / stride, r, t, pos, ib: None });
}
};
@@ -717,10 +778,17 @@ mod tests {
assert_eq!(e106.reference, "e106_bdy_04");
assert_eq!(e106.parts.len(), 8, "all 8 e106 parts placed");
let get = |p: &str| e106.parts.iter().find(|x| x.part == p).unwrap();
// Port/starboard hull pair: X = ∓264, the starboard copy mirrored.
// Port/starboard hull pair: X = ∓264, **both plain**. The mirror is
// baked into the disc data, not into the placement: a runtime capture
// shows the container carrying two 119-vertex buffers whose contents are
// exact X-reflections, each drawn from its own address (see
// docs/re/structures/xbg7-mesh.md). Until 2026-08-12 both twins decoded
// to ONE buffer and this row carried diag(-1,1,1) to compensate; with
// distinct anchor assignment they decode to their own, and re-emitting
// from the capture produces identity here.
assert!((get("e106_bdy_01").t[0] + 264.0).abs() < 0.1);
assert!((get("e106_bdy_02").t[0] - 264.0).abs() < 0.1);
assert_eq!(get("e106_bdy_02").m[0][0], -1.0);
assert_eq!(get("e106_bdy_02").m[0][0], 1.0);
assert_eq!(get("e106_bdy_01").m[0][0], 1.0);
// Bridge: centreline, above and aft of the hull reference.
let brg = get("e106_brg_01");

View File

@@ -23,15 +23,31 @@
/// Fixed offset of the raw XMA1 stream in a headerless `.slb` (no `RIFF`).
pub const HEADERLESS_DATA_OFFSET: usize = 1392;
/// XMA1 packet size. A headerless stream is always a whole number of these, which
/// is how a leading stream is told apart from arbitrary bytes before a `RIFF`.
pub const XMA1_PACKET: usize = 2048;
/// Voice language for cutscene audio. Only English and Japanese voice exist on
/// the disc (subtitles cover more languages, voice does not).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum VoiceLang {
/// The default only because the disc's own default audio track is English;
/// nothing else about the code should assume it.
#[default]
English,
Japanese,
}
impl VoiceLang {
pub const ALL: [VoiceLang; 2] = [VoiceLang::English, VoiceLang::Japanese];
pub fn label(self) -> &'static str {
match self {
VoiceLang::English => "English",
VoiceLang::Japanese => "Japanese",
}
}
fn code(self) -> &'static str {
match self {
VoiceLang::English => "eng",
@@ -63,14 +79,122 @@ pub struct VoiceClip {
pub display: String,
}
/// Enumerate the voice/dialog clips named in a decompressed `sounds.tbl` (the
/// IDXD in `tables.pak`). Extracts every `<lang>\{Voice,etc,Movie,Briefing}\…`
/// path ending in `.slb` for `lang`, parsed into `(name, speaker, display)`.
pub fn list_voice_clips(sounds_tbl: &[u8], lang: VoiceLang) -> Vec<VoiceClip> {
/// What kind of audio a `sounds.tbl` entry names.
///
/// The split is the on-disc path shape, not a guess: the 36 language-independent
/// banks sit at the table root (`BGM_###.slb`, `JNGL_00#.slb`, `Static.slb`),
/// while everything else is under `<lang>\<dir>\`.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub enum AudioCategory {
/// `BGM_###.slb` — 32 music tracks, language-independent.
Music,
/// `JNGL_00#.slb` — 3 short jingles (mission clear / fail stings).
Jingle,
/// `Static.slb` — the sound-effect bank, one 9 MB multi-wave bank.
Sfx,
/// `<lang>\Voice\` — in-mission radio chatter, by speaker.
Radio,
/// `<lang>\etc\` — the other spoken lines (cutscene dialogue, system).
Dialogue,
/// `<lang>\Movie\VOICE_<movie>.slb` — a cutscene's continuous voice track.
MovieVoice,
/// `<lang>\Briefing\BR<NN>_<MM>.slb` — mission briefing lines.
Briefing,
/// A `.slb` whose path matched no known shape.
Other,
}
impl AudioCategory {
pub const ALL: [AudioCategory; 8] = [
AudioCategory::Music,
AudioCategory::Jingle,
AudioCategory::Sfx,
AudioCategory::Radio,
AudioCategory::Dialogue,
AudioCategory::MovieVoice,
AudioCategory::Briefing,
AudioCategory::Other,
];
pub fn label(self) -> &'static str {
match self {
AudioCategory::Music => "Music",
AudioCategory::Jingle => "Jingles",
AudioCategory::Sfx => "Sound effects",
AudioCategory::Radio => "Radio",
AudioCategory::Dialogue => "Dialogue",
AudioCategory::MovieVoice => "Movie voice",
AudioCategory::Briefing => "Briefing",
AudioCategory::Other => "Other",
}
}
/// True for the categories that are spoken lines — the set
/// [`list_voice_clips`] returns.
pub fn is_voice(self) -> bool {
matches!(
self,
AudioCategory::Radio
| AudioCategory::Dialogue
| AudioCategory::MovieVoice
| AudioCategory::Briefing
)
}
/// True when the bank is language-independent, so it appears whichever
/// `<lang>\sounds.tbl` is read.
pub fn is_shared(self) -> bool {
matches!(
self,
AudioCategory::Music | AudioCategory::Jingle | AudioCategory::Sfx
)
}
fn classify(name: &str) -> AudioCategory {
let leaf = name.rsplit('\\').next().unwrap_or(name);
if !name.contains('\\') {
return if leaf.starts_with("BGM_") {
AudioCategory::Music
} else if leaf.starts_with("JNGL_") {
AudioCategory::Jingle
} else if leaf.eq_ignore_ascii_case("Static.slb") {
AudioCategory::Sfx
} else {
AudioCategory::Other
};
}
match name.rsplit('\\').nth(1) {
Some("Voice") => AudioCategory::Radio,
Some("etc") => AudioCategory::Dialogue,
Some("Movie") => AudioCategory::MovieVoice,
Some("Briefing") => AudioCategory::Briefing,
_ => AudioCategory::Other,
}
}
}
/// One playable bank named in `sounds.tbl`, with the category its path implies.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct AudioEntry {
pub clip: VoiceClip,
pub category: AudioCategory,
}
/// Enumerate **every** `.slb` bank named in a decompressed `sounds.tbl` (the
/// IDXD in `tables.pak`): the language-independent music/jingle/SFX banks at
/// the table root, plus every `<lang>\…` spoken line.
///
/// Measured on the retail disc: `eng\sounds.tbl` names 4 418 banks (36 shared +
/// 2 382 Radio + 1 821 Dialogue + 101 Briefing + 78 Movie voice) and
/// `jpn\sounds.tbl` names 5 136 (the same 36 shared + 5 100 Japanese lines).
/// Every one of the 36 shared names resolves to a `sound.pak` TOC entry under
/// [`crate::hash::name_hash`], which is the check that they are real banks and
/// not stale table text.
pub fn list_audio_entries(sounds_tbl: &[u8], lang: VoiceLang) -> Vec<AudioEntry> {
let prefix = format!("{}\\", lang.code());
let mut seen = std::collections::BTreeSet::new();
let mut out = Vec::new();
// Scan for printable-ASCII runs; keep those that look like a voice path.
// Scan for printable-ASCII runs; keep those that name a `.slb`.
let mut i = 0;
while i < sounds_tbl.len() {
let start = i;
@@ -79,15 +203,14 @@ pub fn list_voice_clips(sounds_tbl: &[u8], lang: VoiceLang) -> Vec<VoiceClip> {
}
if i - start >= 6 {
if let Ok(s) = std::str::from_utf8(&sounds_tbl[start..i]) {
// Every spoken-line category, so the standalone player covers them
// all: in-mission radio (`\Voice\`, `\etc\`) and bound movie voices
// (`\Movie\`) all carry `VOICE_`; mission-briefing lines live in
// `\Briefing\` as `BR<NN>_<MM>.slb` (no `VOICE` in the name).
let is_voice = s.contains("VOICE") || s.contains("\\Briefing\\");
if s.starts_with(&prefix) && s.ends_with(".slb") && is_voice {
if seen.insert(s.to_string()) {
out.push(parse_voice_clip(s));
}
// Take this language's entries plus the root (shared) banks; a
// path under the OTHER language would be a table artefact.
let mine = s.starts_with(&prefix) || !s.contains('\\');
if mine && s.ends_with(".slb") && seen.insert(s.to_string()) {
out.push(AudioEntry {
category: AudioCategory::classify(s),
clip: parse_voice_clip(s),
});
}
}
}
@@ -96,6 +219,21 @@ pub fn list_voice_clips(sounds_tbl: &[u8], lang: VoiceLang) -> Vec<VoiceClip> {
out
}
/// Enumerate just the spoken-line clips — [`list_audio_entries`] restricted to
/// [`AudioCategory::is_voice`].
///
/// In-mission radio (`\Voice\`, `\etc\`) and bound movie voices (`\Movie\`) all
/// carry `VOICE_`; mission-briefing lines live in `\Briefing\` as
/// `BR<NN>_<MM>.slb` and carry no `VOICE` at all, which is why the category —
/// i.e. the directory — decides this and not the filename.
pub fn list_voice_clips(sounds_tbl: &[u8], lang: VoiceLang) -> Vec<VoiceClip> {
list_audio_entries(sounds_tbl, lang)
.into_iter()
.filter(|e| e.category.is_voice())
.map(|e| e.clip)
.collect()
}
fn parse_voice_clip(name: &str) -> VoiceClip {
// `<lang>\<cat>\VOICE_<SPK>_<NNN>.slb` or `..\VOICE_<movie>.slb`.
let stem = name
@@ -127,20 +265,183 @@ fn parse_voice_clip(name: &str) -> VoiceClip {
/// that yields the full track for segment banks while the clamp drops the
/// duplicate takes for alternate-take banks. (Dynamic RE via Canary file-I/O
/// tracing confirmed the movie→voice binding; this fixes the *decode* of `RT*`.)
/// A bank's channel count, read from its first `RIFF` sub-wave.
///
/// `XMASTREAMFORMAT.Channels` sits at `RIFF + 49`. **2.12 % of banks are stereo**
/// (170 of 8 021), and decoding one of those as mono yields a single frame and
/// stops — the same signature already recorded for the leading segment. So the
/// channel count has to be read, not assumed. Returns `None` when there is no
/// `RIFF` to read it from.
fn riff_channels(slb: &[u8]) -> Option<u8> {
let ri = find(slb, b"RIFF", 0)?;
slb.get(ri + 49).copied().filter(|c| *c == 1 || *c == 2)
}
/// The four data offsets that occur on the disc, in ascending order.
///
/// Measured over all 7 358 banks whose offset is *known* (they carry a `RIFF`,
/// so the offset is forced to `first_riff % XMA1_PACKET`): no other value
/// occurs. They are all of the form `1392 + 4k`.
pub const DATA_OFFSET_CANDIDATES: [usize; 4] = [1392, 1468, 1600, 1728];
/// How plausible a candidate offset is, judged by XMA1 packet headers alone.
///
/// Each 2048-byte packet opens with a big-endian header: 6 bits frame count,
/// 15 bits frame-offset-in-bits, 3 bits metadata, 8 bits packet-skip. At the
/// true offset those fields stay in range packet after packet; one byte off and
/// they do not. Returns the fraction of the first `LIMIT` packets that look
/// sane, so 1.0 is a clean stream.
fn packet_plausibility(slb: &[u8], start: usize) -> f32 {
const LIMIT: usize = 24;
let (mut seen, mut ok, mut pos) = (0usize, 0usize, start);
while pos + XMA1_PACKET <= slb.len() && seen < LIMIT {
let h = u32::from_be_bytes([slb[pos], slb[pos + 1], slb[pos + 2], slb[pos + 3]]);
let frame_offset_bits = (h >> 11) & 0x7FFF;
let metadata = (h >> 8) & 0x7;
let packet_skip = h & 0xFF;
if frame_offset_bits as usize <= XMA1_PACKET * 8 && metadata <= 1 && packet_skip <= 8 {
ok += 1;
}
seen += 1;
pos += XMA1_PACKET;
}
if seen == 0 {
0.0
} else {
ok as f32 / seen as f32
}
}
/// The data offset implied by the bank's `seek` chunk, if it has one.
///
/// A bank's `seek` chunk lands on a packet boundary, so `seek_pos % XMA1_PACKET`
/// *is* the data offset. Measured on the 6 033 labelled banks that have a
/// `seek` before their first `RIFF`: **6 031 agree (99.97 %)**, 2 disagree.
/// This is structural rather than statistical, which is why it is tried first.
fn seek_chunk_offset(slb: &[u8]) -> Option<usize> {
let pos = find(slb, b"seek", 0)?;
let residue = pos % XMA1_PACKET;
DATA_OFFSET_CANDIDATES.contains(&residue).then_some(residue)
}
/// Recover a bank's data offset when there is no `RIFF` to derive it from.
///
/// Two independent signals, tried in order of how well each is evidenced:
///
/// 1. **the `seek` chunk's position** mod the packet size — 99.97 % on the
/// labelled set, and structural rather than statistical;
/// 2. **packet-header plausibility** — pick the candidate whose XMA1 headers
/// stay in range over the first 24 packets. Alone this is 99.62 %, and all
/// 28 of its misses are ties rather than wrong unique winners.
///
/// Together, on the 7 358 banks where the answer *is* known from the `RIFF`
/// position: **7 354 correct (99.95 %)**. The `seek` residue resolves 26 of the
/// scan's 28 ties correctly and none of them wrongly; the other 2 have no
/// usable `seek`. Falls back to [`HEADERLESS_DATA_OFFSET`] when neither signal
/// decides.
pub fn scan_data_offset(slb: &[u8]) -> usize {
if let Some(off) = seek_chunk_offset(slb) {
return off;
}
let mut best = (HEADERLESS_DATA_OFFSET, -1.0f32);
let mut tied = false;
for &c in &DATA_OFFSET_CANDIDATES {
if c >= slb.len() {
continue;
}
let score = packet_plausibility(slb, c);
if score > best.1 {
best = (c, score);
tied = false;
} else if (score - best.1).abs() < f32::EPSILON {
tied = true;
}
}
if tied {
HEADERLESS_DATA_OFFSET
} else {
best.0
}
}
/// Where a bank's leading headerless packet stream starts.
///
/// The stream is a whole number of 2048-byte XMA1 packets ending at the first
/// `RIFF`, so its start is simply `first_riff % XMA1_PACKET`. Disc-wide that
/// lands on 1392, 1468, 1600 or 1728 depending on language and subdirectory —
/// [`HEADERLESS_DATA_OFFSET`] is just the `<lang>\etc\` case. Measured over a
/// 140-bank sample, deriving the offset instead of assuming 1392 recovers a
/// median **70×** more decoded audio and never less except in one bank where
/// neither offset decodes (see `docs/re/structures/slb-data-offset.md`).
pub fn leading_data_offset(first_riff: usize) -> usize {
first_riff % XMA1_PACKET
}
pub fn to_xma_riffs(slb: &[u8]) -> Vec<Vec<u8>> {
let mut out = Vec::new();
if find(slb, b"RIFF", 0).is_none() {
// Headerless single-stream bank.
if let Some(data) = slb.get(HEADERLESS_DATA_OFFSET..) {
let first_riff = find(slb, b"RIFF", 0);
if first_riff.is_none() {
// Headerless single-stream bank. Two things here were wrong, and the
// corrections are measured (docs/re/structures/slb-data-offset.md):
//
// * the offset is not the constant — with no `RIFF` to derive it from,
// scan the four candidates by packet plausibility;
// * the stream is **mono**. At two channels a 48-bank sample yielded
// 0..4 816 bytes; at one, 180 000..380 000. There was not one bank
// where the old stereo/1392 pair beat the scanned mono pair, and the
// median gain was 184x.
let start = scan_data_offset(slb);
if let Some(data) = slb.get(start..) {
if !data.is_empty() {
out.push(build_riff(&synth_xma1_fmt(2, 2, 48000), data));
out.push(build_riff(&synth_xma1_fmt(1, 0, 48000), data));
}
}
return out;
}
// HYBRID banks: a headerless packet stream followed by RIFF sub-waves, two
// SEQUENTIAL SEGMENTS of one clip. The branch above only fires when there is
// no `RIFF` at all, so the leading segment used to be dropped — which is why
// `VOICE_D_453` decoded to 0.14 s: its line is in that segment and only the
// trailing fragment survived.
//
// The boundary is arithmetic, not a magic: XMA1 packets are 2048 bytes, so a
// leading stream is a whole number of packets ending at the first `RIFF`.
// Its START is therefore `first_riff % XMA1_PACKET` — **not** the constant
// `HEADERLESS_DATA_OFFSET`, which is only the value that offset happens to
// take in `<lang>\etc\`. Disc-wide it takes four values (1392, 1468, 1600,
// 1728), varying by language and subdirectory, and assuming 1392 starts the
// decode mid-packet everywhere else. See docs/re/structures/slb-data-offset.md.
// It decodes as **mono** — at two channels every bank yields exactly 1792
// bytes, one frame, whatever its size.
//
// An earlier version of this was withdrawn for two good reasons, both now
// answered: it recovered no audio (it used the stereo format), and it
// matched 1524 of the 8021 RIFF-bearing entries. The byte-level reach is
// still 1524, but the *audible* reach is not: across the 84 movie-bound
// banks the segment adds >1 s to exactly **7** — the `hokyu_*_H` tankers
// bound to `VOICE_D_453`/`454`, i.e. precisely the broken ones — and
// ≤0.25 s to 66 of the rest. Callers clamp to the movie length anyway.
if let Some(ri) = first_riff {
let start = leading_data_offset(ri);
if ri > start {
if let Some(data) = slb.get(start..ri) {
if data.iter().any(|b| *b != 0) {
// Channels come from the bank's own `fmt `, not a constant:
// 170 of 8 021 banks are stereo and decode to one frame if
// forced to mono.
let ch = riff_channels(slb).unwrap_or(1);
let mask = if ch == 2 { 2 } else { 0 };
out.push(build_riff(&synth_xma1_fmt(ch, mask, 48000), data));
}
}
}
}
let mut pos = 0usize;
while let Some(ri) = find(slb, b"RIFF", pos) {
// Parse this sub-wave's fmt + data (declared size is honest per sub-wave).
// Parse this sub-wave's fmt + data. The declared `data` size is an
// UPPER bound, not an exact one: 5 296 of 7 586 banks declare more than
// the entry holds and none declares exactly what it holds, so the clamp
// below is load-bearing (docs/re/structures/slb-data-offset.md).
let Some(fi) = find(slb, b"fmt ", ri) else { break };
let Some(fsz) = le32(slb, fi + 4) else { break };
let Some(fmt_end) = fi.checked_add(8).and_then(|v| v.checked_add(fsz as usize)) else {
@@ -174,10 +475,12 @@ pub fn to_xma_riff(slb: &[u8]) -> Option<Vec<u8>> {
if let Some(ri) = find(slb, b"RIFF", 0) {
// RIFF layout: a `.slb` is an XACT bank of one or more sub-waves, each
// `[seek][RIFF: fmt + Dmmy pad + data][declared_size XMA bytes]`. Take the
// FIRST sub-wave, bounded by its **declared `data` size** (which is
// honest per sub-wave). Decoding to end-of-file instead would append the
// later sub-waves — for multi-take story movies those are ALTERNATE takes,
// which is what made S10S16 play the wrong audio.
// FIRST sub-wave, bounded by its **declared `data` size** which is an
// upper bound only (69.8 % of banks over-declare it, so the clamp
// matters), but still the right boundary to cut at. Decoding to
// end-of-file instead would append the later sub-waves — for multi-take
// story movies those are ALTERNATE takes, which is what made S10S16
// play the wrong audio.
let fi = find(slb, b"fmt ", ri)?;
let fsz = le32(slb, fi + 4)? as usize;
let fmt_end = fi.checked_add(8)?.checked_add(fsz)?;
@@ -232,6 +535,28 @@ pub fn to_xma_riff_best(slb: &[u8]) -> Option<Vec<u8>> {
(!data.is_empty()).then(|| build_riff(&synth_xma1_fmt(2, 2, 48000), data))
}
/// Wrap a run of **raw XMA1 packets** as a standalone, decodable `RIFF/WAVE`.
///
/// For a bank with no internal delimiters — `Static.slb` is a packed run of whole
/// 2048-byte packets with no `RIFF`, no `seek` and no `WAVE` — a wave is defined
/// *only* by `(offset, packet count)`, both of which come from the running game
/// (`--xma_param_probe`). There is nothing in the file to parse, so the header
/// has to be synthesized, and that is the step worth doing exactly once, here,
/// rather than in each consumer.
///
/// `packets` must be a whole number of [`XMA1_PACKET`] bytes; anything else is a
/// short read and produces a stream the decoder will run off the end of.
/// The `channel_mask` follows the same convention as the rest of this module:
/// `1` for mono, `2` for stereo.
///
/// The three menu cues in `docs/re/menu-audio-cues.md` are
/// `(0x1ec0, 4)` d-pad move, `(0x0ec0, 2)` Ⓑ back and `(0x5d6c0, 6)` Ⓐ confirm,
/// all mono 48 kHz.
pub fn xma1_wave_riff(packets: &[u8], channels: u8, rate: u32) -> Vec<u8> {
let mask = if channels == 1 { 1 } else { 2 };
build_riff(&synth_xma1_fmt(channels, mask, rate), packets)
}
/// A minimal `fmt ` chunk carrying an XMA1 `XMAWAVEFORMAT` (one stream).
fn synth_xma1_fmt(channels: u8, channel_mask: u16, rate: u32) -> Vec<u8> {
let mut fmt = Vec::with_capacity(40);
@@ -356,6 +681,56 @@ mod tests {
);
}
#[test]
fn list_audio_entries_categorises_root_banks_and_keeps_them_language_shared() {
// The three root banks carry no language component, so BOTH sounds.tbl
// files name them; a language filter that only accepted `<lang>\` would
// silently drop all the music, which is what it used to do.
let mut tbl = Vec::new();
for s in [
"BGM_001.slb",
"JNGL_002.slb",
"Static.slb",
"eng\\Voice\\VOICE_ADAN_010.slb",
"eng\\etc\\VOICE_D_450.slb",
"eng\\Movie\\VOICE_S13A.slb",
"eng\\Briefing\\BR01_01.slb",
] {
tbl.extend_from_slice(s.as_bytes());
tbl.push(0);
}
let by = |lang| {
list_audio_entries(&tbl, lang)
.into_iter()
.map(|e| (e.clip.name, e.category))
.collect::<Vec<_>>()
};
let eng = by(VoiceLang::English);
let want = [
("BGM_001.slb", AudioCategory::Music),
("JNGL_002.slb", AudioCategory::Jingle),
("Static.slb", AudioCategory::Sfx),
("eng\\Voice\\VOICE_ADAN_010.slb", AudioCategory::Radio),
("eng\\etc\\VOICE_D_450.slb", AudioCategory::Dialogue),
("eng\\Movie\\VOICE_S13A.slb", AudioCategory::MovieVoice),
("eng\\Briefing\\BR01_01.slb", AudioCategory::Briefing),
];
assert_eq!(eng.len(), want.len());
for (n, c) in want {
assert!(
eng.iter().any(|(en, ec)| en == n && *ec == c),
"{n} not categorised as {c:?}"
);
}
// Reading the Japanese table yields the shared banks and none of the
// English lines.
let jpn = by(VoiceLang::Japanese);
assert_eq!(jpn.len(), 3, "only the shared banks: {jpn:?}");
assert!(jpn.iter().all(|(_, c)| c.is_shared()));
// And the voice view is exactly the non-shared half.
assert_eq!(list_voice_clips(&tbl, VoiceLang::English).len(), 4);
}
#[test]
fn rebuilds_riff_from_headerless() {
let mut slb = vec![0u8; HEADERLESS_DATA_OFFSET];

View File

@@ -1,20 +1,28 @@
//! `T8aD` — the game's 2D UI/HUD texture format.
//!
//! A 32bpp **A8R8G8B8** (Xbox byte order) surface stored as **256×256 raster
//! tiles in row-major order** — each tile prefixed by a 16-byte tile header, edge
//! tiles clipped to the image bounds. Fully reversed 2026-07-17 from the file
//! header and verified against the running game (title screen).
//! A 32bpp **A8R8G8B8** (Xbox byte order) surface stored as a list of
//! **arbitrary sub-rectangles**, each with its own destination origin and size.
//! Reversed 2026-07-17 (verified against the running game's title screen) and
//! **corrected 2026-08-11**, when the per-tile header turned out to carry the
//! rectangle's placement rather than being opaque flags.
//!
//! ```text
//! 0x00 4 Magic "T8aD"
//! 0x14 4 width (BE u32)
//! 0x14 4 width (BE u32) the full surface
//! 0x18 4 height (BE u32)
//! 0x1c 4 tile count (BE u32) = ceil(w/256) * ceil(h/256)
//! 0x2c tiles*4 offset table: absolute byte offset of each row-major tile
//! <off> 16 per-tile header (flags + tile w/h), then:
//! <off+16> tile_w * tile_h * 4 bytes of A8R8G8B8 pixels, row-major
//! 0x1c 4 rectangle count (BE u32) — NOT ceil(w/256)*ceil(h/256)
//! 0x2c count*4 offset table: absolute byte offset of each rectangle
//! <off> 16 rectangle header, four BE u32: dst X, dst Y, width, height
//! <off+16> width * height * 4 bytes of A8R8G8B8 pixels, row-major
//! ```
//!
//! Most surfaces happen to be stored as full-width 256-tall bands, which is why
//! treating the file as a 256×256 grid decoded 96 % of the disc correctly. It is
//! not the model, though: a dialogue strip declares 524×63 and stores **one**
//! 173×25 rectangle at (175,20), and `pdmes010` stores two — (59,6,256,54) and
//! (315,6,149,54), the second beginning exactly `16 + 256*54*4` bytes after the
//! first. Anything the rectangles do not cover stays transparent.
//!
//! Surfaces ≤256px wide are a single tile column, so the first tile's pixels sit
//! at `44 + tiles*4 + 16 = 64` — which is why the old "type→header size 64/84/…"
//! rule (header = 44 + tiles*20) happened to decode small textures correctly: for
@@ -30,6 +38,18 @@ pub struct T8adImage {
pub width: u32,
pub height: u32,
pub rgba: Vec<u8>,
/// The header word at `+0x04`. A flag word; **bit `0x02`** is a candidate
/// blend selector — it separates the title's effect sprites from its ordinary
/// ones exactly, and disc-wide it toggles independently of the rest of the
/// word in 27.1 % of 19 216 sprites. See
/// `docs/re/structures/ui-paint-order-key.md`.
///
/// ⚠️ **Additive was tested and REFUTED.** Blending bit-`0x02` sprites
/// additively moved every measure against the title capture the wrong way:
/// whole-frame mean diff +0.55 → +1.04, swoosh-band mean +1.83 → +3.98, band
/// edge-correlation 0.6971 → 0.5578. The bit is real and independent, but it
/// does not select an additive blend. Carried, not acted on.
pub flags: u32,
}
/// Whether `bytes` starts with the T8aD magic.
@@ -43,18 +63,11 @@ fn be32(b: &[u8], off: usize) -> u32 {
}
/// Side of the square storage tile, in texels, and the per-tile header size.
const TILE: usize = 256;
const TILE_HDR: usize = 16;
/// Bytes of per-rectangle header before its pixels: dst X, dst Y, w, h.
const RECT_HDR: usize = 16;
/// Decode a T8aD surface from a slice whose first bytes ARE the magic. Returns
/// `None` for non-T8aD input or a variant we can't decode as RGBA (never guesses).
///
/// Layout (reversed from the header + verified against the running game):
/// a 44-byte base header, then a `tiles`-entry big-endian u32 **offset table** at
/// `0x2c`, where `tiles` = the field at `0x1c` = `ceil(w/256) * ceil(h/256)`.
/// Each entry is the absolute byte offset of a **row-major** 256×256 tile; every
/// tile is a 16-byte tile header followed by `tile_w*tile_h*4` A8R8G8B8 pixels,
/// edge tiles clipped to the image bounds.
/// `None` if any rectangle fails to fit the surface or the file — never guesses.
pub fn parse(bytes: &[u8]) -> Option<T8adImage> {
if !is_t8ad(bytes) || bytes.len() < 0x40 {
return None;
@@ -64,45 +77,50 @@ pub fn parse(bytes: &[u8]) -> Option<T8adImage> {
if !(1..=4096).contains(&width) || !(1..=4096).contains(&height) {
return None;
}
let tiles = be32(bytes, 0x1c) as usize;
let cols = width.div_ceil(TILE);
let rows = height.div_ceil(TILE);
// The field at 0x1c must be the tile count; otherwise it's a variant we don't
// decode (e.g. DXT / palettized) — defer rather than misdecode.
if tiles == 0 || tiles != cols * rows {
let rects = be32(bytes, 0x1c) as usize;
if rects == 0 || rects > 4096 {
return None;
}
const TABLE: usize = 0x2c;
if bytes.len() < TABLE + tiles * 4 {
if bytes.len() < TABLE + rects * 4 {
return None;
}
// Anything no rectangle covers stays transparent.
let mut rgba = vec![0u8; width * height * 4];
for ty in 0..rows {
for tx in 0..cols {
let tile = ty * cols + tx;
let pixels = be32(bytes, TABLE + tile * 4) as usize + TILE_HDR;
let tw = TILE.min(width - tx * TILE);
let th = TILE.min(height - ty * TILE);
if pixels + tw * th * 4 > bytes.len() {
return None; // truncated / not the layout we expect
}
for row in 0..th {
let mut s = pixels + row * tw * 4;
let mut d = ((ty * TILE + row) * width + tx * TILE) * 4;
for _ in 0..tw {
// A8R8G8B8 → RGBA8.
rgba[d] = bytes[s + 1];
rgba[d + 1] = bytes[s + 2];
rgba[d + 2] = bytes[s + 3];
rgba[d + 3] = bytes[s];
s += 4;
d += 4;
}
for r in 0..rects {
let at = be32(bytes, TABLE + r * 4) as usize;
if at + RECT_HDR > bytes.len() {
return None;
}
let dx = be32(bytes, at) as usize;
let dy = be32(bytes, at + 4) as usize;
let rw = be32(bytes, at + 8) as usize;
let rh = be32(bytes, at + 12) as usize;
// Never guess: a rectangle must fit the surface and its pixels the file.
if rw == 0 || rh == 0 || dx + rw > width || dy + rh > height {
return None;
}
let pixels = at + RECT_HDR;
if pixels + rw * rh * 4 > bytes.len() {
return None;
}
for row in 0..rh {
let mut s = pixels + row * rw * 4;
let mut d = ((dy + row) * width + dx) * 4;
for _ in 0..rw {
// A8R8G8B8 → RGBA8.
rgba[d] = bytes[s + 1];
rgba[d + 1] = bytes[s + 2];
rgba[d + 2] = bytes[s + 3];
rgba[d + 3] = bytes[s];
s += 4;
d += 4;
}
}
}
Some(T8adImage {
flags: be32(bytes, 4),
width: width as u32,
height: height as u32,
rgba,
@@ -123,7 +141,11 @@ mod tests {
b[0x18..0x1c].copy_from_slice(&h.to_be_bytes());
b[0x1c..0x20].copy_from_slice(&1u32.to_be_bytes()); // 1 tile
b.extend_from_slice(&0x30u32.to_be_bytes()); // offset table: tile 0 @ 0x30
b.extend_from_slice(&[0u8; 16]); // 16-byte tile header → pixels at 0x40
// rectangle header: dst (0,0), size w×h
b.extend_from_slice(&0u32.to_be_bytes());
b.extend_from_slice(&0u32.to_be_bytes());
b.extend_from_slice(&w.to_be_bytes());
b.extend_from_slice(&h.to_be_bytes());
for i in 0..(w * h) {
b.extend_from_slice(&[(i & 0xff) as u8, 0x24, 0x63, 0xB2]); // A, R, G, B
}
@@ -143,8 +165,8 @@ mod tests {
}
#[test]
fn assembles_row_major_tiles_via_offset_table() {
// 300×1 → 2 tiles: (0,0)=256×1 red, (1,0)=44×1 blue, each +16-byte header.
fn assembles_rectangles_via_offset_table() {
// 300×1 → 2 rectangles: (0,0) 256×1 red, then (256,0) 44×1 blue.
let (w, h): (u32, u32) = (300, 1);
let mut b = vec![0u8; 0x2c];
b[0..4].copy_from_slice(&T8AD_MAGIC);
@@ -155,9 +177,9 @@ mod tests {
let off1 = off0 + 16 + 256 * 4; // tile-0 header + its 256 pixels
b.extend_from_slice(&(off0 as u32).to_be_bytes());
b.extend_from_slice(&(off1 as u32).to_be_bytes());
b.extend_from_slice(&[0u8; 16]);
for v in [0u32, 0, 256, 1] { b.extend_from_slice(&v.to_be_bytes()) } // dst(0,0) 256×1
b.extend_from_slice(&[0xFF, 0xFF, 0, 0].repeat(256)); // A,R,G,B red
b.extend_from_slice(&[0u8; 16]);
for v in [256u32, 0, 44, 1] { b.extend_from_slice(&v.to_be_bytes()) } // dst(256,0) 44×1
b.extend_from_slice(&[0xFF, 0, 0, 0xFF].repeat(44)); // A,R,G,B blue
let img = parse(&b).expect("decodes");
assert_eq!((img.width, img.height), (300, 1));
@@ -166,10 +188,11 @@ mod tests {
}
#[test]
fn rejects_wrong_tilecount_and_short() {
// tile-count field that isn't ceil(w/256)*ceil(h/256) → None
fn rejects_out_of_range_rect_and_short() {
// a rectangle that does not fit the declared surface → None, never guess
let mut b = synth(2, 2);
b[0x1c..0x20].copy_from_slice(&7u32.to_be_bytes());
let at = 0x2c + 4;
b[at + 8..at + 12].copy_from_slice(&99u32.to_be_bytes()); // width 99 > 2
assert!(parse(&b).is_none());
// truncated pixel data → None
let b = synth(64, 64);

View File

@@ -52,7 +52,10 @@ pub enum TextureError {
#[error("No TX2D texture resource found in XPR2 file")]
NoTextureFound,
#[error("Unsupported texture format: 0x{0:02X}")]
#[error("Unsupported XPR container '{0}' (this reader handles XPR2 only)")]
UnsupportedContainer(String),
#[error("Unsupported texture format: {} (0x{:02X})", gpu_format_name(*.0), .0)]
UnsupportedFormat(u8),
#[error("Buffer too small: need {needed} bytes, have {have}")]
@@ -125,6 +128,159 @@ impl X360TextureFormat {
pub fn block_size(&self) -> usize {
if self.is_block_compressed() { 4 } else { 1 }
}
/// Canary's canonical `k_…` GPUTEXTUREFORMAT name (e.g. `k_DXT1`).
pub fn gpu_name(&self) -> &'static str {
gpu_format_name(*self as u8)
}
/// The full Canary format descriptor (bpp, block dims, compression).
pub fn desc(&self) -> &'static GpuFormatDesc {
// Every enum value has a valid entry in the 0..=63 table.
&GPU_FORMATS[*self as usize]
}
}
// ── GPUTEXTUREFORMAT reference table (ported from xenia-canary) ─────────────────
/// One row of Xenia's GPU texture-format table. `bpp` is bits per pixel; for a
/// block-compressed format one "block" covers `block_w × block_h` texels.
/// Source: xenia-canary `src/xenia/gpu/xenos.h` (`enum class TextureFormat`) +
/// `src/xenia/gpu/texture_info_formats.inl` (`FORMAT_INFO(...)`).
#[derive(Debug, Clone, Copy)]
pub struct GpuFormatDesc {
/// 6-bit GPUTEXTUREFORMAT code (GPUFC dword_1 bits[5:0]).
pub code: u8,
/// Canary's canonical name, e.g. `k_8_8_8_8`, `k_DXT4_5`.
pub name: &'static str,
pub block_w: u8,
pub block_h: u8,
pub bpp: u16,
pub compressed: bool,
}
impl GpuFormatDesc {
/// Bytes per block (or per pixel when `block_w == block_h == 1`).
pub fn bytes_per_block(&self) -> usize {
self.block_w as usize * self.block_h as usize * self.bpp as usize / 8
}
}
const fn d(
code: u8,
name: &'static str,
block_w: u8,
block_h: u8,
bpp: u16,
compressed: bool,
) -> GpuFormatDesc {
GpuFormatDesc { code, name, block_w, block_h, bpp, compressed }
}
/// The complete GPUTEXTUREFORMAT table (codes 0..=63), verbatim from
/// xenia-canary. Lets us name/describe *any* texture format the game uses —
/// even ones this crate can't yet decode — instead of a bare hex code.
#[rustfmt::skip]
pub const GPU_FORMATS: [GpuFormatDesc; 64] = [
d(0, "k_1_REVERSE", 1, 1, 1, false),
d(1, "k_1", 1, 1, 1, false),
d(2, "k_8", 1, 1, 8, false),
d(3, "k_1_5_5_5", 1, 1, 16, false),
d(4, "k_5_6_5", 1, 1, 16, false),
d(5, "k_6_5_5", 1, 1, 16, false),
d(6, "k_8_8_8_8", 1, 1, 32, false),
d(7, "k_2_10_10_10", 1, 1, 32, false),
d(8, "k_8_A", 1, 1, 8, false),
d(9, "k_8_B", 1, 1, 8, false),
d(10, "k_8_8", 1, 1, 16, false),
d(11, "k_Cr_Y1_Cb_Y0_REP", 2, 1, 16, true),
d(12, "k_Y1_Cr_Y0_Cb_REP", 2, 1, 16, true),
d(13, "k_16_16_EDRAM", 1, 1, 32, false),
d(14, "k_8_8_8_8_A", 1, 1, 32, false),
d(15, "k_4_4_4_4", 1, 1, 16, false),
d(16, "k_10_11_11", 1, 1, 32, false),
d(17, "k_11_11_10", 1, 1, 32, false),
d(18, "k_DXT1", 4, 4, 4, true),
d(19, "k_DXT2_3", 4, 4, 8, true),
d(20, "k_DXT4_5", 4, 4, 8, true),
d(21, "k_16_16_16_16_EDRAM", 1, 1, 64, false),
d(22, "k_24_8", 1, 1, 32, false),
d(23, "k_24_8_FLOAT", 1, 1, 32, false),
d(24, "k_16", 1, 1, 16, false),
d(25, "k_16_16", 1, 1, 32, false),
d(26, "k_16_16_16_16", 1, 1, 64, false),
d(27, "k_16_EXPAND", 1, 1, 16, false),
d(28, "k_16_16_EXPAND", 1, 1, 32, false),
d(29, "k_16_16_16_16_EXPAND", 1, 1, 64, false),
d(30, "k_16_FLOAT", 1, 1, 16, false),
d(31, "k_16_16_FLOAT", 1, 1, 32, false),
d(32, "k_16_16_16_16_FLOAT", 1, 1, 64, false),
d(33, "k_32", 1, 1, 32, false),
d(34, "k_32_32", 1, 1, 64, false),
d(35, "k_32_32_32_32", 1, 1, 128, false),
d(36, "k_32_FLOAT", 1, 1, 32, false),
d(37, "k_32_32_FLOAT", 1, 1, 64, false),
d(38, "k_32_32_32_32_FLOAT", 1, 1, 128, false),
d(39, "k_32_AS_8", 4, 1, 8, true),
d(40, "k_32_AS_8_8", 2, 1, 16, true),
d(41, "k_16_MPEG", 1, 1, 16, false),
d(42, "k_16_16_MPEG", 1, 1, 32, false),
d(43, "k_8_INTERLACED", 1, 1, 8, false),
d(44, "k_32_AS_8_INTERLACED", 4, 1, 8, true),
d(45, "k_32_AS_8_8_INTERLACED", 1, 1, 16, true),
d(46, "k_16_INTERLACED", 1, 1, 16, false),
d(47, "k_16_MPEG_INTERLACED", 1, 1, 16, false),
d(48, "k_16_16_MPEG_INTERLACED", 1, 1, 32, false),
d(49, "k_DXN", 4, 4, 8, true),
d(50, "k_8_8_8_8_AS_16_16_16_16", 1, 1, 32, false),
d(51, "k_DXT1_AS_16_16_16_16", 4, 4, 4, true),
d(52, "k_DXT2_3_AS_16_16_16_16", 4, 4, 8, true),
d(53, "k_DXT4_5_AS_16_16_16_16", 4, 4, 8, true),
d(54, "k_2_10_10_10_AS_16_16_16_16", 1, 1, 32, false),
d(55, "k_10_11_11_AS_16_16_16_16", 1, 1, 32, false),
d(56, "k_11_11_10_AS_16_16_16_16", 1, 1, 32, false),
d(57, "k_32_32_32_FLOAT", 1, 1, 96, false),
d(58, "k_DXT3A", 4, 4, 4, true),
d(59, "k_DXT5A", 4, 4, 4, true),
d(60, "k_CTX1", 4, 4, 4, true),
d(61, "k_DXT3A_AS_1_1_1_1", 4, 4, 4, true),
d(62, "k_8_8_8_8_GAMMA_EDRAM", 1, 1, 32, false),
d(63, "k_2_10_10_10_FLOAT_EDRAM", 1, 1, 32, false),
];
/// Canary's canonical name for a 6-bit GPUTEXTUREFORMAT code, or
/// `"unknown(NN)"` when out of the 0..=63 range.
pub fn gpu_format_name(code: u8) -> &'static str {
GPU_FORMATS
.get(code as usize)
.map(|f| f.name)
.unwrap_or("unknown")
}
/// The full descriptor for a GPUTEXTUREFORMAT code, if in range.
pub fn gpu_format_desc(code: u8) -> Option<&'static GpuFormatDesc> {
GPU_FORMATS.get(code as usize)
}
/// Identify the XPR container variant from the 4-byte magic (`XPR0`/`XPR2`/
/// `XPR5`/…), so callers can report `XPR5` clearly instead of "bad magic".
pub fn xpr_container_kind(bytes: &[u8]) -> Option<String> {
if bytes.len() >= 4 && &bytes[..3] == b"XPR" {
Some(String::from_utf8_lossy(&bytes[..4]).into_owned())
} else {
None
}
}
/// Guard the XPR2-only decode paths: turn a non-XPR2 container into a clear
/// [`TextureError::UnsupportedContainer`] before binrw reports a generic
/// magic mismatch. (The game also ships some `XPR5` packages, e.g. Common.xpr.)
fn ensure_xpr2(bytes: &[u8]) -> Result<(), TextureError> {
match xpr_container_kind(bytes) {
Some(k) if k == "XPR2" => Ok(()),
Some(k) => Err(TextureError::UnsupportedContainer(k)),
None => Ok(()), // let the normal magic check produce BadMagic
}
}
// ── XPR2 container format ─────────────────────────────────────────────────────
@@ -261,6 +417,7 @@ impl X360Texture {
/// Decode the `want`-th texture resource (`TX2D` / `TXCM`, directory order).
pub fn from_xpr2_index(bytes: &[u8], want: usize) -> Result<Self, TextureError> {
use std::io::Cursor;
ensure_xpr2(bytes)?;
let mut cur = Cursor::new(bytes);
// Parse header — validates "XPR2" magic, reads 3 × u32 (total 16 bytes)
@@ -353,6 +510,7 @@ impl X360Texture {
/// `BG_Acheron`: `data_size == 6 × 0x400000` and all 6 faces decode cleanly.)
pub fn cube_faces_from_xpr2(bytes: &[u8]) -> Result<Option<Cubemap>, TextureError> {
use std::io::Cursor;
ensure_xpr2(bytes)?;
let mut cur = Cursor::new(bytes);
let header = Xpr2Header::read(&mut cur)?;
let mut entries = Vec::new();
@@ -640,6 +798,25 @@ fn compact_bits(mut x: u32) -> u32 {
mod tests {
use super::*;
#[test]
fn gpu_format_table_is_index_aligned() {
// The hand-transcribed Canary table must stay index==code for all 64.
for (i, f) in GPU_FORMATS.iter().enumerate() {
assert_eq!(f.code as usize, i, "GPU_FORMATS[{i}] has code {}", f.code);
}
// Spot-check the formats the game actually uses (from xenos.h + .inl).
assert_eq!(gpu_format_name(6), "k_8_8_8_8");
assert_eq!(gpu_format_name(18), "k_DXT1");
assert_eq!(gpu_format_name(59), "k_DXT5A");
assert_eq!(gpu_format_name(200), "unknown");
// Every decodable enum variant resolves to a compressed/uncompressed
// descriptor consistent with its own is_block_compressed().
for code in [6u8, 7, 18, 19, 20, 49, 59] {
let fmt = X360TextureFormat::from_u8(code).unwrap();
assert_eq!(fmt.desc().compressed, fmt.is_block_compressed(), "code {code}");
}
}
#[test]
fn morton_decode_corners() {
assert_eq!(morton_decode(0), (0, 0));

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,105 @@
//! The runtime layout of a unit / vessel definition object.
//!
//! The game parses an [`crate::idxd`] record into a fixed 880-byte object whose
//! field offsets are **not** guessable from the disc data: the record is a
//! reflective key/value pool, and the loader assigns each key to a member by
//! name. This table is that assignment, read out of the loader itself
//! (`sub_82341A20` — every key is built as `addi r4, r30, -N`, so the field name
//! for each store is a string in the executable image), and verified against
//! objects dumped from a running mission: **406 values agree with the disc
//! records, 0 disagree**, over 11 objects covering both schemas.
//!
//! Why a reimplementation wants it:
//!
//! - it names the field behind every word of a live definition object, so a
//! memory snapshot can be read directly;
//! - it says which fields a record leaves **defaulted**, and what the loader
//! leaves there — the float accessor returns `0.0` on a pool miss;
//! - it carries two conventions that are invisible on disc: **angles are degrees
//! in the data and radians in the object**, and **`Size_Y` takes `Size_X`**
//! when omitted.
//!
//! See `docs/re/live-unit-definitions.md` for the derivation and the measured
//! default values.
/// How a field is stored in the definition object.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Kind {
/// IEEE-754 single, big-endian.
F32,
/// Pointer to a string.
Str,
/// 32-bit word (bool / enum / count / id).
Word,
}
/// One field of the definition object.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Field {
/// Byte offset from the start of the object.
pub offset: usize,
/// Field name, exactly as the disc record spells it.
pub name: &'static str,
/// Storage kind.
pub kind: Kind,
}
const TABLE: &str = include_str!("../data/unit_definition_layout.txt");
/// Every mapped field, in offset order.
pub fn fields() -> Vec<Field> {
TABLE
.lines()
.filter(|l| !l.trim_start().starts_with('#') && !l.trim().is_empty())
.filter_map(|l| {
let mut it = l.split_whitespace();
let offset = it.next()?.parse().ok()?;
let kind = match it.next()? {
"f32" => Kind::F32,
"str" => Kind::Str,
_ => Kind::Word,
};
// `name` is a &'static str because TABLE is 'static.
let name = it.next()?;
let name: &'static str = TABLE.get(
TABLE.find(name).map(|s| s..s + name.len())?,
)?;
Some(Field { offset, name, kind })
})
.collect()
}
/// The field at `offset`, if one is mapped there.
pub fn field_at(offset: usize) -> Option<Field> {
fields().into_iter().find(|f| f.offset == offset)
}
/// The offset of `name`, if it is mapped.
pub fn offset_of(name: &str) -> Option<usize> {
fields().into_iter().find(|f| f.name == name).map(|f| f.offset)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn table_parses_and_is_ordered() {
let f = fields();
assert!(f.len() > 150, "expected the full map, got {}", f.len());
assert!(f.windows(2).all(|w| w[0].offset < w[1].offset), "offsets must be strictly increasing");
}
#[test]
fn known_fields_sit_where_the_loader_puts_them() {
// Spot-checks from the verified map; these four also anchor the
// identification of a live object (see docs/re/live-unit-definitions.md).
assert_eq!(offset_of("Size_X"), Some(48));
assert_eq!(offset_of("Size_Y"), Some(52));
assert_eq!(offset_of("Size_Z"), Some(56));
assert_eq!(offset_of("HP"), Some(84));
assert_eq!(offset_of("HQRatio"), Some(88));
assert_eq!(field_at(96).map(|f| f.name), Some("ThrusterRatio"));
assert_eq!(field_at(48).map(|f| f.kind), Some(Kind::F32));
}
}

View File

@@ -0,0 +1,85 @@
//! Caption recovery across all eight `MSG_*` families.
//!
//! `build_demo_text` reads only `MSG_DEMO_*`, the smallest family.
//! `build_caption_text` generalises the key parser to all eight.
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use sylpheed_formats::{movie_subtitle, PakArchive};
fn disc_root() -> Option<PathBuf> {
if let Ok(p) = std::env::var("SYLPHEED_DISC") {
let p = PathBuf::from(p);
if p.join("dat").is_dir() {
return Some(p);
}
}
let d = Path::new(
"/home/fabi/RE - Project Sylpheed/Project Sylpheed - Arc of Deception (USA, Europe) (En,Ja)",
);
d.join("dat").is_dir().then(|| d.to_path_buf())
}
macro_rules! skip_without_disc {
($root:ident) => {
let Some($root) = disc_root() else {
eprintln!("SKIP: set SYLPHEED_DISC");
return;
};
};
}
#[test]
fn all_eight_caption_families_are_read() {
skip_without_disc!(root);
let pak = PakArchive::open(root.join("dat/GP_MAIN_GAME_E.pak")).expect("pak");
let all = movie_subtitle::build_caption_text(&pak);
let mut per: BTreeMap<String, usize> = BTreeMap::new();
for (id, lines) in &all {
*per.entry(id.split('_').next().unwrap().to_string()).or_default() += lines.len();
}
let fams: Vec<&str> = per.keys().map(String::as_str).collect();
assert_eq!(
fams,
["ACRO", "ADAN", "ADPL", "BIRD", "DEMO", "RHIN", "TCAF", "VOICE"],
"all eight families must appear"
);
let total: usize = all.values().map(|v| v.len()).sum();
// 8800 is ALL of them: every distinct text-bearing MSG_* key on the disc has
// the <id>_<page>_<line> shape, and the field reader recovers 8800 of 8800.
assert_eq!(total, 8800, "recovered caption lines");
assert_eq!(all.len(), 4085, "recovered caption ids");
// `VOICE` is the only family with a letter before the id; its ids must keep it.
assert!(all.contains_key("VOICE_A_150"), "VOICE ids keep their family letter");
}
/// The control: generalising must not lose anything the DEMO-only reader had.
/// It does not — it gains, because token adjacency was dropping lines there too.
#[test]
fn demo_family_is_not_lost_by_generalising() {
skip_without_disc!(root);
let pak = PakArchive::open(root.join("dat/GP_MAIN_GAME_E.pak")).expect("pak");
let demo = movie_subtitle::build_demo_text(&pak);
let all = movie_subtitle::build_caption_text(&pak);
let old: usize = demo.values().map(|v| v.len()).sum();
let new: usize = all
.iter()
.filter(|(k, _)| k.starts_with("DEMO_"))
.map(|(_, v)| v.len())
.sum();
// The token-adjacency reader misses 4 DEMO lines that the field reader gets,
// so the record route is strictly better even on the family it was written
// for. It must never be WORSE.
assert_eq!(old, 537, "build_demo_text, token adjacency");
assert_eq!(new, 541, "build_caption_text, record fields");
assert!(new >= old, "the record route must not lose lines");
// …and 16x more text overall than the DEMO-only path saw.
let total: usize = all.values().map(|v| v.len()).sum();
assert!(total > old * 16, "expected a large gain, got {total} vs {old}");
}

View File

@@ -0,0 +1,309 @@
//! Real-disc tests for the record-table game-data loaders. Skipped without
//! `SYLPHEED_DISC`.
//!
//! These pin the values that only became readable once the loaders moved off the
//! string-pool reader (`docs/re/idxd-legacy-reader-audit.md`): the per-hardpoint
//! stats, the per-difficulty scoring and the per-phase play space. A regression
//! to the flat reader collapses each of them to one answer and fails here.
use std::collections::BTreeSet;
use sylpheed_formats::game_data::{self as gd, Difficulty, HardpointKind};
use sylpheed_formats::PakArchive;
fn main_pak() -> Option<PakArchive> {
let disc = std::env::var("SYLPHEED_DISC").ok()?;
PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).ok()
}
fn skip() {
eprintln!("SKIP: set SYLPHEED_DISC");
}
/// The player's craft carries 63 `Turret_NNN` records — one per equippable
/// weapon — each with its own `HP`, while `StructureCount.TurretCount` says 4.
/// The flat reader had one `HP` slot for the whole object.
#[test]
fn craft_hardpoints_are_per_record() {
let Some(pak) = main_pak() else { return skip() };
let units = gd::load_units(&pak);
let saber = units
.iter()
.find(|u| u.id.as_deref() == Some("UN_f001_TCAF_DeltaSaber_T"))
.expect("Delta Saber present");
assert_eq!(saber.hp, Some(1000.0), "hull HP");
assert_eq!(saber.turret_count, Some(4), "StructureCount.TurretCount");
let turrets: Vec<_> = saber
.hardpoints
.iter()
.filter(|h| h.kind == HardpointKind::Turret)
.collect();
assert_eq!(turrets.len(), 63, "Turret_NNN records");
assert!(turrets.iter().all(|t| t.hp == Some(100.0)), "every mount has HP 100");
// Each mount names its own weapon model, `rou_f001_wep_NN`.
let mount = turrets.iter().find(|t| t.index == 33).expect("Turret_033");
assert_eq!(mount.model.as_deref(), Some("rou_f001_wep_33"));
assert_eq!(mount.id.as_deref(), Some("Turret_033"));
}
/// A capital ship's components each carry their own HP — five distinct values on
/// the SD-Battleship, where the flat reader returned only the hull's 100000.
#[test]
fn vessel_hardpoints_are_per_record() {
let Some(pak) = main_pak() else { return skip() };
let vessels = gd::load_vessels(&pak);
let bs = vessels
.iter()
.find(|v| v.id.as_deref() == Some("UN_e101_ADAN_SDBattleship"))
.expect("SD-Battleship present");
assert_eq!(bs.hp, Some(100000.0));
let hp_of = |kind: HardpointKind, idx: u32| {
bs.hardpoints
.iter()
.find(|h| h.kind == kind && h.index == idx)
.and_then(|h| h.hp)
};
assert_eq!(hp_of(HardpointKind::Bridge, 0), Some(10000.0));
assert_eq!(hp_of(HardpointKind::Thruster, 0), Some(20000.0));
assert_eq!(hp_of(HardpointKind::ShieldGenerator, 0), Some(5000.0));
assert_eq!(hp_of(HardpointKind::Hatch, 0), Some(100.0));
// Distinct HP values across the ship's components.
let distinct: BTreeSet<String> = bs
.hardpoints
.iter()
.filter_map(|h| h.hp.map(|v| format!("{v}")))
.collect();
assert!(distinct.len() >= 4, "distinct component HP values: {distinct:?}");
// A main gun names the weapon it fires and the shield generator its share.
let asgun = bs
.hardpoints
.iter()
.find(|h| h.weapon_id.as_deref() == Some("Weapon_ADAN_Ship_ASGun"))
.expect("an anti-ship gun");
assert_eq!(asgun.hp, Some(1500.0));
assert_eq!(
bs.hardpoints
.iter()
.find(|h| h.kind == HardpointKind::ShieldGenerator)
.and_then(|h| h.power_ratio),
Some(0.25)
);
// The launch bay names the squadron it scrambles.
assert_eq!(
bs.hardpoints
.iter()
.find(|h| h.kind == HardpointKind::Hatch)
.and_then(|h| h.squadron_id.clone()),
Some("Squadron_Test2".to_string())
);
}
/// `MainMissionBonus` lives in `Score_Easy` / `Score_Normal` / `Score_Hard`, one
/// value each. The flat reader returned the Easy one for every mission.
#[test]
fn main_mission_bonus_is_per_difficulty() {
let Some(pak) = main_pak() else { return skip() };
let cfgs = gd::load_player_configs(&pak);
assert_eq!(cfgs.len(), 24);
let mut distinct3 = 0;
let mut doubling = 0;
let mut all_equal = 0;
for c in &cfgs {
let (e, n, h) = (
c.score.easy.main_mission_bonus,
c.score.normal.main_mission_bonus,
c.score.hard.main_mission_bonus,
);
let set: BTreeSet<_> = [e, n, h].into_iter().flatten().collect();
match set.len() {
3 => distinct3 += 1,
1 => all_equal += 1,
_ => {}
}
if let (Some(e), Some(n), Some(h)) = (e, n, h) {
if n == 2 * e && h == 4 * e {
doubling += 1;
}
}
}
eprintln!("MainMissionBonus: 3-distinct {distinct3}/24, all-equal {all_equal}, doubling {doubling}");
assert_eq!(distinct3, PIN_BONUS_DISTINCT3);
assert_eq!(doubling, PIN_BONUS_DOUBLING);
// The first config's own numbers.
let c = &cfgs[0];
assert_eq!(c.score.easy.main_mission_bonus, PIN_C0_EASY);
assert_eq!(c.score.normal.main_mission_bonus, PIN_C0_NORMAL);
assert_eq!(c.score.hard.main_mission_bonus, PIN_C0_HARD);
}
/// The rank thresholds repeat identically across the three `Score_*` records —
/// so the one flat answer happened to be right, but only by coincidence.
#[test]
fn rank_scores_repeat_across_difficulties() {
let Some(pak) = main_pak() else { return skip() };
let cfgs = gd::load_player_configs(&pak);
let mut identical = 0;
for c in &cfgs {
let row = |s: &gd::ScoreRules| {
(s.rank_score_s, s.rank_score_a, s.rank_score_b, s.rank_score_c, s.rank_score_d)
};
if row(&c.score.easy) == row(&c.score.normal) && row(&c.score.normal) == row(&c.score.hard) {
identical += 1;
}
}
eprintln!("rank scores identical across difficulties: {identical}/{}", cfgs.len());
assert_eq!(identical, cfgs.len());
assert_eq!(cfgs[0].score.normal.rank_score_s, Some(10000));
assert_eq!(cfgs[0].score.normal.rank_score_d, Some(1000));
// The difficulty scaling that *is* per record.
assert_eq!(cfgs[0].difficulty_f32(Difficulty::Easy, "DamageAdjustment"), PIN_EASY_DMG);
assert_eq!(cfgs[0].difficulty_f32(Difficulty::Hard, "ShieldDamageAdjustment"), PIN_HARD_SHIELD);
}
/// `SpaceSize` lives in `Phase_1` / `Phase_2` / `Phase_3`, one value each.
#[test]
fn space_size_is_per_phase() {
let Some(pak) = main_pak() else { return skip() };
let cfgs = gd::load_player_configs(&pak);
let mut varying = 0;
let mut p1_250k = 0;
for c in &cfgs {
assert_eq!(c.phases.len(), 3, "every mission has three Phase_N records");
let sizes: Vec<Option<f32>> = c.phases.iter().map(|p| p.space_size).collect();
let shapes: BTreeSet<String> = sizes.iter().map(|v| format!("{v:?}")).collect();
if shapes.len() > 1 {
varying += 1;
}
if sizes[0] == Some(250000.0) {
p1_250k += 1;
}
}
eprintln!("SpaceSize: varying across phases {varying}/24, phase 1 = 250000 in {p1_250k}");
assert_eq!(varying, PIN_SPACE_VARYING);
assert_eq!(p1_250k, PIN_SPACE_P1_250K);
let c = &cfgs[0];
assert_eq!(c.space_size(1), PIN_C0_SPACE1);
assert_eq!(c.space_size(2), PIN_C0_SPACE2);
assert_eq!(c.space_size(3), PIN_C0_SPACE3);
}
/// Fields the string-pool reader returned `None` for now have values, disc-wide.
#[test]
fn previously_unreadable_stats_are_populated() {
let Some(pak) = main_pak() else { return skip() };
let units = gd::load_units(&pak);
let vessels = gd::load_vessels(&pak);
let n = |f: fn(&gd::CraftUnit) -> Option<f32>| units.iter().filter(|u| f(u).is_some()).count();
eprintln!(
"units={} fcs={} shield_ratio={} cruise={} maxvel={} accel={} decel={} shield_max={}",
units.len(),
n(|u| u.fcs_range),
n(|u| u.shield_ratio),
n(|u| u.cruising_velocity),
n(|u| u.maximum_velocity),
n(|u| u.acceleration),
n(|u| u.deceleration),
n(|u| u.shield_max),
);
for (label, count) in [
("fcs_range", n(|u| u.fcs_range)),
("shield_ratio", n(|u| u.shield_ratio)),
("cruising_velocity", n(|u| u.cruising_velocity)),
("maximum_velocity", n(|u| u.maximum_velocity)),
("acceleration", n(|u| u.acceleration)),
("deceleration", n(|u| u.deceleration)),
] {
assert_eq!(count, units.len(), "{label} set on every unit");
}
assert!(vessels.iter().all(|v| v.fcs_range.is_some() && v.shield_ratio.is_some()));
// Hardpoint HP is readable for every vessel component.
let hp_missing = vessels
.iter()
.flat_map(|v| &v.hardpoints)
.filter(|h| h.hp.is_none())
.count();
assert_eq!(hp_missing, 0);
let total_hardpoints: usize = vessels.iter().map(|v| v.hardpoints.len()).sum();
eprintln!("vessel hardpoints: {total_hardpoints}");
assert_eq!(total_hardpoints, PIN_VESSEL_HARDPOINTS);
}
/// The launcher and its projectile are two records with two ids; the tracer has a
/// third `Interval`.
#[test]
fn weapon_launcher_and_shell_stay_separate() {
let Some(pak) = main_pak() else { return skip() };
let ws = gd::load_weapons(&pak);
assert_eq!(ws.len(), 131);
let differing = ws
.iter()
.filter(|w| w.id.is_some() && w.shell_id.is_some() && w.id != w.shell_id)
.count();
eprintln!("weapons whose shell id differs from the launcher id: {differing}/{}", ws.len());
assert_eq!(differing, PIN_WEAPON_SHELL_IDS);
// A player missile the corpus quotes through the flat reader — the launcher
// fields agree, and `MaximumRange` (a `Shell` field) reads the same.
let m26 = ws
.iter()
.find(|w| w.id.as_deref() == Some("Weapon_DSaber_P_wep_26_Missile"))
.expect("Weapon_DSaber_P_wep_26_Missile is in GP_MAIN_GAME_E.pak, not the hangar pak");
assert_eq!(m26.target_type.as_deref(), Some("Vessel,Craft"));
assert_eq!(m26.loading_count, Some(144));
assert_eq!(m26.interval, Some(3.0));
assert_eq!(m26.mass, Some(0.77));
assert_eq!(m26.trigger_shot_count, Some(12));
assert_eq!(m26.max_range, Some(10000.0));
let with_wake = ws.iter().filter(|w| w.records.record("ShellWake").is_some()).count();
eprintln!("weapons with a ShellWake record: {with_wake}");
assert_eq!(with_wake, PIN_WEAPON_WAKE);
// `Power` is a `Shell` field: every weapon but one has it.
let no_power: Vec<&str> =
ws.iter().filter(|w| w.power.is_none()).filter_map(|w| w.id.as_deref()).collect();
assert_eq!(no_power, ["Weapon_NULL"], "only the placeholder weapon has no Shell.Power");
}
/// `RecordSet::everywhere` is the API that answers "which record did that come
/// from" — the question the flat map could not express.
#[test]
fn everywhere_enumerates_the_records_that_define_a_field() {
let Some(pak) = main_pak() else { return skip() };
let vessels = gd::load_vessels(&pak);
let bs = vessels
.iter()
.find(|v| v.id.as_deref() == Some("UN_e101_ADAN_SDBattleship"))
.unwrap();
let hps = bs.records.everywhere("HP");
eprintln!("SDBattleship records defining HP: {}", hps.len());
assert_eq!(hps.len(), PIN_BS_HP_RECORDS);
assert!(hps.iter().any(|(r, v)| *r == "Generic" && *v == "100000.0"));
assert!(hps.iter().any(|(r, _)| r.starts_with("Turret_")));
}
// Pinned measurements over `GP_MAIN_GAME_E.pak`.
/// 18 of the 24 missions pay a different main-objective bonus per difficulty;
/// the other 6 pay 0 at every difficulty.
const PIN_BONUS_DISTINCT3: usize = 18;
/// …and in all 24 the ratio is easy : normal : hard = 1 : 2 : 4.
const PIN_BONUS_DOUBLING: usize = 24;
const PIN_C0_EASY: Option<i64> = Some(1000);
const PIN_C0_NORMAL: Option<i64> = Some(2000);
const PIN_C0_HARD: Option<i64> = Some(4000);
const PIN_EASY_DMG: Option<f32> = Some(1.5);
const PIN_HARD_SHIELD: Option<f32> = Some(0.5);
/// `SpaceSize` is stored per phase, but only **one** of the 24 missions actually
/// varies it across its phases (250000 → 100000 → 100000). The other 23 repeat
/// one value: 500000 once, 100000 eighteen times, 50000 four times.
const PIN_SPACE_VARYING: usize = 1;
const PIN_SPACE_P1_250K: usize = 1;
const PIN_C0_SPACE1: Option<f32> = Some(250000.0);
const PIN_C0_SPACE2: Option<f32> = Some(100000.0);
const PIN_C0_SPACE3: Option<f32> = Some(100000.0);
/// Destructible components across the 23 capital ships, each with its own HP.
const PIN_VESSEL_HARDPOINTS: usize = 418;
/// Every weapon's projectile carries an id of its own (`Shell_…` vs `Weapon_…`).
const PIN_WEAPON_SHELL_IDS: usize = 131;
const PIN_WEAPON_WAKE: usize = 40;
/// Records of the SD-Battleship that define an `HP`: the hull plus 36 components.
const PIN_BS_HP_RECORDS: usize = 37;

View File

@@ -0,0 +1,348 @@
//! The IDXD record/field table, checked against every IDXD object on the disc.
//!
//! The binary region in front of the string pool was undecoded for a long time
//! (`idxd.rs` used to say so). It is a record array plus a field array, and the
//! decisive evidence is that every record reproduces its own stored hash and
//! every named field reproduces its own key: the layout has no free parameters
//! left once that holds 1.4 million times.
//!
//! `first_header_word_is_record0_hash` is the one that demotes `schema_hash`.
use std::path::{Path, PathBuf};
use sylpheed_formats::hash::tag_hash;
use sylpheed_formats::{IdxdObject, PakArchive};
fn disc_root() -> Option<PathBuf> {
if let Ok(p) = std::env::var("SYLPHEED_DISC") {
let p = PathBuf::from(p);
if p.join("dat").is_dir() {
return Some(p);
}
}
let default = Path::new(
"/home/fabi/RE - Project Sylpheed/Project Sylpheed - Arc of Deception (USA, Europe) (En,Ja)",
);
default.join("dat").is_dir().then(|| default.to_path_buf())
}
macro_rules! skip_without_disc {
($root:ident) => {
let Some($root) = disc_root() else {
eprintln!("SKIP: extracted disc not found (set SYLPHEED_DISC to enable)");
return;
};
};
}
/// Every `.pak` on the disc, recursively.
///
/// Deliberately the whole disc root, not `dat/`: `hidden/DefTables.pak` holds
/// another 1425 IDXD objects, and an earlier version of this sweep missed them.
fn all_paks(root: &Path) -> Vec<PathBuf> {
let mut out = Vec::new();
let mut stack = vec![root.to_path_buf()];
while let Some(dir) = stack.pop() {
let Ok(rd) = std::fs::read_dir(&dir) else {
continue;
};
for e in rd.flatten() {
let p = e.path();
if p.is_dir() {
stack.push(p);
} else if p.extension().is_some_and(|x| x == "pak") {
out.push(p);
}
}
}
out.sort();
out
}
#[test]
fn records_roundtrip_disc() {
skip_without_disc!(root);
let (mut objects, mut hashed, mut unparsed) = (0usize, 0usize, 0usize);
let mut bad: Vec<String> = Vec::new();
for pak in all_paks(&root) {
let Ok(ar) = PakArchive::open(&pak) else {
continue;
};
for entry in ar.entries() {
let Ok(bytes) = ar.read(entry) else { continue };
if !IdxdObject::is_idxd(&bytes) {
continue;
}
let Ok(obj) = IdxdObject::parse(&bytes) else {
continue;
};
objects += 1;
let Some(recs) = obj.records() else {
unparsed += 1;
bad.push(format!(
"{}:{:08x} record region did not parse",
pak.display(),
entry.name_hash
));
continue;
};
assert_eq!(
recs.len(),
obj.count as usize,
"{}:{:08x} record count",
pak.display(),
entry.name_hash
);
for r in recs {
hashed += 1;
if r.name_hash != tag_hash(&r.name) {
if bad.len() < 10 {
bad.push(format!(
"{}:{:08x} record {:?} stored {:08x} != tag_hash {:08x}",
pak.display(),
entry.name_hash,
r.name,
r.name_hash,
tag_hash(&r.name)
));
}
unparsed += 1;
}
}
// Records are stored sorted by hash so the guest can binary-search.
assert!(
recs.windows(2).all(|w| w[0].name_hash <= w[1].name_hash),
"{}:{:08x} records not sorted by hash",
pak.display(),
entry.name_hash
);
}
}
eprintln!("IDXD objects {objects}, hashed records {hashed}, failures {unparsed}");
assert!(
bad.is_empty(),
"{} failures, first few:\n{}",
unparsed,
bad.join("\n")
);
// Guard against the sweep silently finding nothing.
assert_eq!(objects, 7750, "IDXD object total changed");
assert_eq!(hashed, 190_782, "record total changed");
}
/// A concrete, human-checkable row: the movie table keys its cutscene ids as
/// *literal integers*, which the string-pool reader could never have told apart
/// from field names.
#[test]
fn movie_table_ids_are_literal_field_keys() {
skip_without_disc!(root);
let ar = PakArchive::open(root.join("dat/tables.pak")).expect("tables.pak");
let mut found = false;
for entry in ar.entries() {
let Ok(bytes) = ar.read(entry) else { continue };
if !IdxdObject::is_idxd(&bytes) {
continue;
}
let obj = IdxdObject::parse(&bytes).unwrap();
if obj.schema_hash != 0x0670_25B9 {
continue;
}
let base = obj.record("BASE_INFO").expect("BASE_INFO record");
// The five named fields the movie GamePart reads.
assert_eq!(base.get("PATH"), Some("dat\\movie\\"));
assert_eq!(base.get("VERSION"), Some("0x060329"));
// …and the numeric ids, which are keys, not names.
assert_eq!(
base.field_at(105).map(|f| f.value.as_str()),
Some("STAGE01_PHASE01")
);
assert_eq!(
base.field_at(205).map(|f| f.value.as_str()),
Some("STAGE02_PHASE01")
);
// Each id names a record that carries the actual file names.
let rec = obj.record("STAGE02_PHASE01").expect("STAGE02_PHASE01");
assert_eq!(rec.get("MOVIE"), Some("RT02A.wmv"));
assert_eq!(rec.get("VOICETRACK"), Some("VOICE_RT02A"));
// Every literal-keyed BASE_INFO field must name a real record.
let ids: Vec<u32> = base.fields.iter().filter_map(|f| f.index()).collect();
assert!(ids.len() > 90, "only {} literal ids", ids.len());
for f in base.fields.iter().filter(|f| f.index().is_some()) {
assert!(
obj.record(&f.value).is_some(),
"id {} -> {:?} has no record",
f.key,
f.value
);
}
found = true;
break;
}
assert!(
found,
"movie table (schema 067025b9) not found in tables.pak"
);
}
/// The header word this crate calls `schema_hash` is record 0's name hash. If
/// that were a coincidence it would not survive 6325 objects.
#[test]
fn first_header_word_is_record0_hash() {
skip_without_disc!(root);
let (mut checked, mut mismatched) = (0usize, 0usize);
for pak in all_paks(&root) {
let Ok(ar) = PakArchive::open(&pak) else {
continue;
};
for entry in ar.entries() {
let Ok(bytes) = ar.read(entry) else { continue };
if !IdxdObject::is_idxd(&bytes) {
continue;
}
let Ok(obj) = IdxdObject::parse(&bytes) else {
continue;
};
let Some(first) = obj.first_record() else {
continue;
};
checked += 1;
if tag_hash(&first.name) != obj.schema_hash {
mismatched += 1;
if mismatched <= 5 {
eprintln!(
"{}:{:08x} header {:08x} != tag_hash({:?}) {:08x}",
pak.display(),
entry.name_hash,
obj.schema_hash,
first.name,
tag_hash(&first.name)
);
}
}
}
}
eprintln!("checked {checked}, mismatched {mismatched}");
assert_eq!(mismatched, 0);
assert_eq!(checked, 7750);
}
/// Field names are stored on disc; the key is `tag_hash` of the stored name.
/// Unnamed fields carry a literal positional key instead.
#[test]
fn field_names_are_stored_disc() {
skip_without_disc!(root);
let (mut named, mut positional, mut hash_keyed_unnamed, mut bad) =
(0usize, 0usize, 0usize, 0usize);
for pak in all_paks(&root) {
let Ok(ar) = PakArchive::open(&pak) else {
continue;
};
for entry in ar.entries() {
let Ok(bytes) = ar.read(entry) else { continue };
if !IdxdObject::is_idxd(&bytes) {
continue;
}
let Ok(obj) = IdxdObject::parse(&bytes) else {
continue;
};
let Some(recs) = obj.records() else { continue };
for r in recs {
for f in &r.fields {
match &f.name {
Some(n) => {
named += 1;
if tag_hash(n) != f.key {
bad += 1;
if bad <= 5 {
eprintln!(
"{}:{:08x} {:?}.{:?} key {:08x} != {:08x}",
pak.display(),
entry.name_hash,
r.name,
n,
f.key,
tag_hash(n)
);
}
}
}
None if f.key < 0x1_0000 => positional += 1,
None => hash_keyed_unnamed += 1,
}
}
}
}
}
eprintln!(
"named {named}, positional {positional}, hash-keyed-but-unnamed {hash_keyed_unnamed}, bad {bad}"
);
assert_eq!(bad, 0, "named fields must reproduce their key");
assert_eq!(named, 1_271_462);
assert_eq!(positional, 1_485_073);
// A small residue keeps a hash-shaped key with no stored name: the only
// fields on the disc whose name still has to be recovered by preimage search.
assert_eq!(hash_keyed_unnamed, 504);
}
/// The "sibling default" rules are a string-pool deduplication artefact.
///
/// The corpus recorded that `Size_Y` inherits `Size_X` (and three similar pairs)
/// for units the disc leaves "defaulted". The fields are not defaulted — they are
/// on disc — and the old reader missed them precisely when the value was shared
/// with the sibling, which is the condition the rule then "predicted".
#[test]
fn sibling_default_rules_are_a_dedup_artefact() {
skip_without_disc!(root);
let arc = PakArchive::open(root.join("dat/GP_MAIN_GAME_E.pak")).expect("pak");
let pairs = [
("Size_Y", "Size_X"),
("FCSRange", "RadarRange"),
("DefencePoint", "AttackVesselPoint"),
];
// [pair][legacy_absent][equal_on_disc]
let mut tally = [[[0usize; 2]; 2]; 3];
for e in arc.entries() {
let Ok(bytes) = arc.read(e) else { continue };
if !IdxdObject::is_idxd(&bytes) {
continue;
}
let Ok(obj) = IdxdObject::parse(&bytes) else { continue };
let Some(generic) = obj.record("Generic") else { continue };
if generic.get("Size_X").is_none() {
continue;
}
for (i, (field, sibling)) in pairs.iter().enumerate() {
let (Some(a), Some(b)) = (generic.get(field), generic.get(sibling)) else {
continue;
};
tally[i][usize::from(obj.get_f32(field).is_none())][usize::from(a == b)] += 1;
}
}
// The field is present on disc far more often than it "differs", so the
// premise "these fields are defaulted" is simply false.
assert_eq!(tally[0][0][0] + tally[0][1][0], 90, "Size_Y differs from Size_X");
// The mechanism: a value shared with the sibling is ALWAYS invisible to the
// legacy reader. If this cell were ever non-zero the dedup story would be
// incomplete.
for (i, (field, sibling)) in pairs.iter().enumerate() {
assert_eq!(
tally[i][0][1], 0,
"{field}/{sibling}: legacy reader saw a value it shares with its sibling"
);
}
// …and it almost never misses a value that differs. Those few cells are
// exactly where the rule predicts the wrong number.
assert_eq!(tally[0][1][0], 0, "Size_Y: rule never wrong");
assert_eq!(tally[1][1][0], 1, "FCSRange: UN_e011_ADAN_Attacker_B_HF_Wayne");
assert_eq!(tally[2][1][0], 1, "DefencePoint: UN_e104_ADAN_Carrier");
}

View File

@@ -0,0 +1,95 @@
//! The IXUD record/field table, checked against every IXUD object on the disc.
//!
//! The decode was verified with a standalone parser hours before it existed in
//! the crate; this is the same check through `IxudObject`.
use std::path::{Path, PathBuf};
use sylpheed_formats::hash::ixud_hash_str;
use sylpheed_formats::{IxudObject, PakArchive};
fn disc_root() -> Option<PathBuf> {
if let Ok(p) = std::env::var("SYLPHEED_DISC") {
let p = PathBuf::from(p);
if p.join("dat").is_dir() {
return Some(p);
}
}
let d = Path::new(
"/home/fabi/RE - Project Sylpheed/Project Sylpheed - Arc of Deception (USA, Europe) (En,Ja)",
);
d.join("dat").is_dir().then(|| d.to_path_buf())
}
macro_rules! skip_without_disc {
($root:ident) => {
let Some($root) = disc_root() else {
eprintln!("SKIP: set SYLPHEED_DISC");
return;
};
};
}
fn all_paks(root: &Path) -> Vec<PathBuf> {
let mut out = Vec::new();
let mut stack = vec![root.to_path_buf()];
while let Some(dir) = stack.pop() {
let Ok(rd) = std::fs::read_dir(&dir) else { continue };
for e in rd.flatten() {
let p = e.path();
if p.is_dir() {
stack.push(p);
} else if p.extension().is_some_and(|x| x == "pak") {
out.push(p);
}
}
}
out.sort();
out
}
#[test]
fn ixud_records_roundtrip_disc() {
skip_without_disc!(root);
let (mut objects, mut records, mut named, mut positional, mut bad) = (0, 0, 0, 0, 0);
for pak in all_paks(&root) {
let Ok(ar) = PakArchive::open(&pak) else { continue };
for entry in ar.entries() {
let Ok(bytes) = ar.read(entry) else { continue };
if bytes.len() < 4 || bytes[0..4] != *b"IXUD" {
continue;
}
let Some(obj) = IxudObject::parse(&bytes) else {
bad += 1;
continue;
};
objects += 1;
// The header word is record 0's hash, not a schema id.
assert_eq!(
obj.first_record_hash,
obj.records()[0].name_hash,
"{}: header word is record 0's hash",
pak.display()
);
for r in obj.records() {
records += 1;
assert_eq!(r.name_hash, ixud_hash_str(&r.name), "{}", pak.display());
for f in &r.fields {
match &f.name {
Some(n) => {
named += 1;
assert_eq!(f.key, ixud_hash_str(n), "{}", pak.display());
}
None => positional += 1,
}
}
}
}
}
eprintln!("objects {objects}, records {records}, named {named}, positional {positional}");
assert_eq!(bad, 0, "every IXUD object must parse");
assert_eq!(objects, 1104);
assert_eq!(records, 1476);
assert_eq!(named, 628_165);
assert_eq!(positional, 48);
}

View File

@@ -0,0 +1,88 @@
//! Real-disc tests for media assembly. Skipped without `SYLPHEED_DISC`.
//!
//! This logic used to live in the Bevy viewer, where it had no test at all. It
//! is the trickiest reading on the disc — a cutscene's voice is a byte region of
//! a continuous stream, not the bank its name points at — so it gets pinned here
//! before anything else is built on top of it.
use std::path::PathBuf;
use sylpheed_formats::media::{self, DirectorySource, DiscSource};
use sylpheed_formats::slb::VoiceLang;
fn disc() -> Option<DirectorySource> {
let p = PathBuf::from(std::env::var("SYLPHEED_DISC").ok()?);
p.join("dat").is_dir().then(|| DirectorySource::new(p))
}
/// A segment-spanning read returns the same bytes as slicing the whole archive.
///
/// The control that matters: `sound.pak`'s data is five segments, so a TOC
/// offset late in the archive addresses a position no single file has. If the
/// walk were off by a segment this would return plausible-looking wrong bytes
/// rather than fail, which is exactly why it is asserted against the archive's
/// own read rather than against a length.
#[test]
fn segment_range_matches_the_archive_read() {
let Some(src) = disc() else {
eprintln!("SKIP: set SYLPHEED_DISC");
return;
};
let name = "BGM_020.slb";
let hash = sylpheed_formats::hash::name_hash(name);
let via_range = media::read_sound_bank(&src, hash).expect("segment range read");
let toc = src.read_file("dat/sound.pak").unwrap();
let entries = sylpheed_formats::PakArchive::parse_toc(&toc).unwrap();
let e = entries
.iter()
.find(|e| e.name_hash == hash)
.expect("BGM_020 in the TOC");
assert_eq!(via_range.len(), e.comp_size as usize);
// And it decodes, which a misaligned read would not do.
let riffs = media::sound_bank_riffs(&src, name).expect("riffs");
assert!(!riffs.is_empty(), "no sub-waves recovered");
}
/// A movie's voice resolves to a byte region, and the region is sane.
///
/// `RT01A` is one of the cutscenes whose voice spans more than one `.slb`
/// chunk — the case that motivated regions over per-bank reads in the first
/// place. The assertions are deliberately about *shape* (ordered, non-empty,
/// smaller than one bank) rather than exact offsets, because the offsets are
/// disc facts we have no independent oracle for here; a regression that
/// reversed or emptied the region would still be caught.
#[test]
fn movie_voice_resolves_to_a_region_that_decodes() {
let Some(src) = disc() else {
eprintln!("SKIP: set SYLPHEED_DISC");
return;
};
let (start, end) = media::resolve_movie_voice_region(&src, "RT01A", VoiceLang::English)
.expect("RT01A has a bound voice track");
assert!(start < end, "region is inverted: {start}..{end}");
assert!(end - start > 4096, "region is implausibly small");
assert!(end - start < 1_500_000, "region spans more than one bank");
let riffs = media::voice_region_riffs(&src, start, end).expect("region riffs");
assert!(!riffs.is_empty(), "region decoded to no audio");
}
/// An unbound movie stays unvoiced rather than borrowing a neighbour's clip.
///
/// This is a *negative* the corpus paid for: extending resolution to unbound
/// resupply movies by shared demo line played the WRONG recording. The guard
/// keeps that door shut.
#[test]
fn manifest_binding_is_the_only_route() {
let Some(src) = disc() else {
eprintln!("SKIP: set SYLPHEED_DISC");
return;
};
// A movie the manifest does not bind must resolve to nothing, not to a guess.
assert_eq!(
media::resolve_movie_voice_clip(&src, "no_such_movie_xyz", VoiceLang::English),
None
);
}

View File

@@ -0,0 +1,144 @@
//! Cross-container consistency for XBG7 geometry.
//!
//! A geometry resource shared by several stage containers must decode to the
//! same bounds in each. This needs no ground truth, and on 2026-08-11 it found
//! **125 of 681** shared resources decoding to different bounds while reporting
//! identical vertex and triangle counts — the anchor scan locating a different
//! buffer of the same size (see `docs/re/structures/xbg7-mesh.md`).
//!
//! The test is `#[ignore]`d because the decoder does not satisfy it yet. It is
//! written as the *target* state so that fixing the anchor scan makes it pass,
//! rather than as a snapshot of the bug.
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use sylpheed_formats::mesh::Xbg7Model;
fn disc_root() -> Option<PathBuf> {
if let Ok(p) = std::env::var("SYLPHEED_DISC") {
let p = PathBuf::from(p);
if p.join("dat").is_dir() {
return Some(p);
}
}
let default = Path::new(
"/home/fabi/RE - Project Sylpheed/Project Sylpheed - Arc of Deception (USA, Europe) (En,Ja)",
);
if default.join("dat").is_dir() {
return Some(default.to_path_buf());
}
None
}
/// Rounded (w, h, d) of a model's own geometry.
fn span(m: &Xbg7Model) -> Option<[i64; 3]> {
let (mut lo, mut hi) = ([f32::MAX; 3], [f32::MIN; 3]);
for s in &m.meshes {
for q in &s.positions {
for k in 0..3 {
lo[k] = lo[k].min(q[k]);
hi[k] = hi[k].max(q[k]);
}
}
}
if lo[0] == f32::MAX {
return None;
}
Some([
(hi[0] - lo[0]).round() as i64,
(hi[1] - lo[1]).round() as i64,
(hi[2] - lo[2]).round() as i64,
])
}
#[test]
#[ignore = "known-failing: 62 of 714 shared resources decode inconsistently (was 125 of 681; distinct anchor assignment + the 0.42 connectivity cap fixed the rest). Note this metric is the WEAKER witness — a systematic mis-anchor is consistent — see docs/re/structures/xbg7-mesh.md"]
fn shared_resources_decode_identically_in_every_container() {
let Some(root) = disc_root() else {
eprintln!("SKIP: extracted disc not found (set SYLPHEED_DISC to enable)");
return;
};
let dir = root.join("hidden/resource3d");
let mut files: Vec<PathBuf> = std::fs::read_dir(&dir)
.expect("resource3d/")
.flatten()
.map(|e| e.path())
.filter(|p| p.extension().and_then(|s| s.to_str()) == Some("xpr"))
.collect();
files.sort();
// name -> (verts, tris) -> set of spans seen
let mut seen: BTreeMap<String, Vec<([i64; 3], usize, usize, String)>> = BTreeMap::new();
for f in &files {
let Ok(bytes) = std::fs::read(f) else { continue };
let where_ = f.file_name().unwrap().to_string_lossy().to_string();
for m in Xbg7Model::anchor_models_cancellable(&bytes, 0.0, &|| false) {
let Some(sp) = span(&m) else { continue };
let v: usize = m.meshes.iter().map(|s| s.positions.len()).sum();
let t: usize = m.meshes.iter().map(|s| s.indices.len() / 3).sum();
seen.entry(m.name.clone()).or_default().push((sp, v, t, where_.clone()));
}
}
let mut bad: Vec<String> = Vec::new();
for (name, list) in &seen {
if list.len() < 2 {
continue;
}
// Only compare decodes that agree on how much geometry they found;
// a differing vertex/triangle count is a different question.
if !list.iter().all(|e| e.1 == list[0].1 && e.2 == list[0].2) {
continue;
}
let spans: std::collections::BTreeSet<[i64; 3]> = list.iter().map(|e| e.0).collect();
if spans.len() > 1 && bad.len() < 10 {
bad.push(format!("{name}: {spans:?}"));
}
}
assert!(
bad.is_empty(),
"{} shared resources decode to different bounds; first: {bad:?}",
bad.len()
);
}
/// Port/starboard twins must not decode to the *same* buffer.
///
/// A runtime capture showed the container stores both halves of the `e106` hull
/// as separate X-reflected buffers, so a `…_01`/`…_02` pair of equal vertex
/// count should come out mirrored (or related by another axis / vertex order) —
/// never identical, which is the collapse distinct assignment fixes. Since that
/// fix reached grouped pools too, this holds for **every** twin pair on the disc
/// — including `n206`, which was the last exception.
#[test]
fn twin_pairs_do_not_share_a_buffer() {
let Some(root) = disc_root() else {
eprintln!("SKIP: extracted disc not found (set SYLPHEED_DISC to enable)");
return;
};
let mut files: Vec<PathBuf> = std::fs::read_dir(root.join("hidden/resource3d"))
.expect("resource3d/")
.flatten()
.map(|e| e.path())
.filter(|p| p.extension().and_then(|s| s.to_str()) == Some("xpr"))
.collect();
files.sort();
let mut collapsed: Vec<String> = Vec::new();
for f in &files {
let Ok(bytes) = std::fs::read(f) else { continue };
let models = Xbg7Model::anchor_models_cancellable(&bytes, 0.0, &|| false);
let by_name: BTreeMap<&str, &Xbg7Model> =
models.iter().map(|m| (m.name.as_str(), m)).collect();
for m in &models {
let Some(stem) = m.name.strip_suffix("_01") else { continue };
let Some(t) = by_name.get(format!("{stem}_02").as_str()) else { continue };
let (a, b) = (m.meshes[0].vbuf_offset, t.meshes[0].vbuf_offset);
if a.is_some() && a == b {
collapsed.push(format!("{}/{stem}_02 in {}", m.name, f.file_name().unwrap().to_string_lossy()));
}
}
}
assert!(collapsed.is_empty(), "twin pairs sharing one buffer: {collapsed:?}");
}

View File

@@ -17,7 +17,7 @@ fn res3d_dir() -> Option<PathBuf> {
}
}
let default =
PathBuf::from("/home/fabi/RE Project Sylpheed/sylph_extract/hidden/resource3d");
PathBuf::from("/home/fabi/RE - Project Sylpheed/sylph_extract/hidden/resource3d");
default.is_dir().then_some(default)
}
@@ -321,7 +321,7 @@ fn hero_ship_grouped_pool_decodes() {
fn stage_models_decode() {
use sylpheed_formats::mesh::Xbg7Model;
let dir = std::env::var("SYLPHEED_RES3D").unwrap_or_else(|_| {
"/home/fabi/RE Project Sylpheed/sylph_extract/hidden/resource3d".to_string()
"/home/fabi/RE - Project Sylpheed/sylph_extract/hidden/resource3d".to_string()
});
let path = format!("{dir}/Stage_S10.xpr");
let bytes = std::fs::read(&path).expect("read Stage_S10");
@@ -359,7 +359,7 @@ fn stage_models_sweep() {
use sylpheed_formats::mesh::Xbg7Model;
use std::time::Instant;
let dir = std::env::var("SYLPHEED_RES3D").unwrap_or_else(|_| {
"/home/fabi/RE Project Sylpheed/sylph_extract/hidden/resource3d".to_string()
"/home/fabi/RE - Project Sylpheed/sylph_extract/hidden/resource3d".to_string()
});
let mut names: Vec<_> = std::fs::read_dir(&dir)
.unwrap()
@@ -386,7 +386,7 @@ fn stage_models_sweep() {
fn stage_models_quality_audit() {
use sylpheed_formats::mesh::Xbg7Model;
let dir = std::env::var("SYLPHEED_RES3D").unwrap_or_else(|_| {
"/home/fabi/RE Project Sylpheed/sylph_extract/hidden/resource3d".to_string()
"/home/fabi/RE - Project Sylpheed/sylph_extract/hidden/resource3d".to_string()
});
let bytes = std::fs::read(format!("{dir}/Stage_S07.xpr")).unwrap();
let models = Xbg7Model::stage_models(&bytes);
@@ -424,3 +424,59 @@ fn stage_models_quality_audit() {
assert!(huge < models.len() / 20, "few huge (skybox-plane) models");
assert!(worst_deg < 0.35, "no model should be mostly-degenerate");
}
/// A correctly located index run has **no degenerate triangles**. That is the
/// signature the 2026-08-13 pad-scoring fix keys on: an index list read one
/// element late still passes every count-based gate but wires arbitrary
/// vertices, which produces triangles with a repeated index. Before the fix
/// **579** decoded runs on the disc carried such triangles (and a runtime
/// capture confirmed 17 of 93 index batches disagreed with the GPU); after it and
/// the two follow-ups (pad scoring in the grouped path, and preferring a
/// degenerate-free candidate over an earlier dirty one), exactly **one** does.
/// Locking the
/// number in, because the defect is invisible to coverage and to the anchor
/// oracle: every count stays correct while the geometry is mis-wired.
#[test]
#[ignore = "requires extracted disc models — set SYLPHEED_RES3D"]
fn decoded_index_runs_have_almost_no_degenerate_triangles() {
let Some(dir) = res3d_dir() else {
eprintln!("SKIP: resource3d dir not found (set SYLPHEED_RES3D)");
return;
};
let mut files: Vec<PathBuf> = std::fs::read_dir(&dir)
.expect("resource3d/")
.flatten()
.map(|e| e.path())
.filter(|p| p.extension().and_then(|s| s.to_str()) == Some("xpr"))
.collect();
files.sort();
let mut offenders: Vec<(String, String, usize)> = Vec::new();
for f in &files {
let Ok(bytes) = std::fs::read(f) else { continue };
let where_ = f.file_name().unwrap().to_string_lossy().to_string();
for m in Xbg7Model::stage_models(&bytes) {
for sm in &m.meshes {
let d = sm
.indices
.chunks_exact(3)
.filter(|t| t[0] == t[1] || t[1] == t[2] || t[0] == t[2])
.count();
if d > 0 {
offenders.push((m.name.clone(), where_.clone(), d));
}
}
}
}
// The one at default settings: `_rou_f402_dead` in `Stage_S09`, whose
// degenerate-free block is claimed by `e_rou_f003_Near` — both 24-vertex
// bounding boxes, the identity class that needs descriptor-level data. (With
// `XBG7_SUBMESH_DECLS=1` four newly decoded `ptc_pack` `.dat` composites join
// it; that knob is off by default.) Anything else here is a regression.
assert!(
offenders.len() <= 1,
"{} decoded index runs contain degenerate triangles (expected ≤ 1): {:?}",
offenders.len(),
&offenders[..offenders.len().min(20)]
);
}

Some files were not shown because too many files have changed in this diff Show More