Reads the live world out of guest RAM and drives the pad from it. Working:
loop-rate memory reads, whole-RAM float scanning with numpy (1270 orthonormal
3x3 blocks in 6.2 s), entity enumeration by unit type (116 live instances in
Stage 02), pad control written straight into the vgamepad FIFO (the CLI spawns
a process per command and its tap/hold sleep inside the server, so neither is
usable in a control loop), unattended mission entry, and the Hangar loadout --
the "Recommended" control is AUTO SELECT, which at 5 % progress is a no-op
because only two weapons are developed and both are already mounted.
Not working, and the reason the craft is not yet flown: the class 0x820af030
is NOT the live entity. It has one object per spawned thing and carries the
unit-ID string, which is why it looked like the entity list, but every one of
its 384 words is constant across a 29 s in-flight capture. No transform lives
in it or one pointer hop from it. Input correlation (hard left yaw vs hard
right, looking for a turn axis that reverses) does find self-like objects at
cos = -0.99, but they cluster in what looks like a camera volume rather than
the craft, and with no definition pointer near them the trick of learning one
entity's layout and applying it to the rest has nothing to anchor on -- so the
33418 moving triples in a firefight cannot be split into enemies, friendlies
and bullets, and there is nothing to aim at.
Two dead ends are recorded so they are not repeated: RT is not the throttle
(the two-state speed scan therefore found nothing), and comparing orientation
matrices 2 s apart is outside the small-angle regime, which is what produced
"angular velocities" of 30000.
Also corrects the claim in unit-struct-runtime.md that 0x820af030 holds live
state. The definition class 0x820af844 and every value derived from it are
unaffected.
autopilot2.py (a PD controller using body angular velocity from consecutive
rotation matrices) is committed but has never had a valid config to run
against, and is marked as untested.
Captures the five remaining tutorials (grab_tutorial.sh, one cold boot each)
and re-solves over seven snapshots from seven separate emulator runs. Coverage
18 -> 21 units, confirmed fields 22 -> 27 (Size_Y, MassScore, MinimumVelocity,
AV_PitchPlus_Min, AV_PitchMinus_Min join the zero-contradiction set).
The multi-run union exposed something the single-run check could not: the same
unit's object is NOT byte-identical between runs. unit_runtime.py --crosscheck
pins down why -- exactly 15 words differ, 13 of them holding guest heap
pointers, and none of them is a solved or interpolated field offset. So every
value reported is run-invariant, which is a stronger statement than the
within-run identity check that came before it.
The two non-pointer stragglers are a real caveat, now documented rather than
smoothed over: +0x2c8 reads 8000.0 for UN_f001_TCAF_DeltaSaber_T_Ttrl in two
tutorials and 10000.0 in a third, with a 0/1 flag at +0x2d0. A couple of words
in the object are set per stage, so it is mostly but not entirely the parsed
table. Unidentified, marked NEEDS-HUMAN.
Coverage honesty: all six tutorials together add only 3 units the missions do
not already have -- they reuse one training box, one drone and the player
craft. Further coverage needs story progress, not more tutorials.
Two navigation facts encoded in grab_tutorial.sh, both learned by breaking
them: the main menu is not input-ready for ~10 s after the title tap and early
d-pad presses are dropped (which sends the A to NEW GAME); and NEW GAME is not
a shortcut to Stage 01 -- it gates on DIFFICULTY then plays the prologue. A
NEW GAME excursion to the READY ROOM leaves game01/savedata byte-identical,
verified by diff against a backup.
Counted the definition objects at three points in Stage 02 -- just after
take-off, ~12 min into combat, and after GAME OVER: 14 objects, same 14 IDs
each time. Capturing a stage therefore costs one load and one snapshot; the
mission does not have to be played or survived.
Craft stats were the last parked Route-B target: the Hangar exposes only a
weight class, and in menus the flight-model object is not instantiated at all.
It is instantiated in-mission, so this reads it there.
The definition class is discovered rather than assumed (unit_discover.py):
scanning for pointers to the disc ID strings and tallying the word behind each
pointer site picks out vtable 0x820af844 -- one object per unit ID, name-record
pointer at +0x04, string at +0x10, exactly the Weapon shape. The neighbouring
class 0x820af030 is the spawned entity, not the definition; the two are told
apart by one-object-per-ID and by 14/14 objects being byte-identical across two
snapshots 12 minutes and one firefight apart.
22 fields bind with zero contradictions on >=3 distinct values. Beyond that,
the Maneuver sub-record turns out to be laid out in schema declaration order,
4 bytes per field, base 0x9c with a two-slot gap after AA_Roll_Min -- 29
independently solved anchors fit two exact bases with no conflicts. That rule
pins five fields NO unit table ever values (PitchDragFactor, RollDragFactor,
DragFactorThreshold, ArterBurner_Acc, DecPitchFactor); PitchDrag == RollDrag ==
the disc's YawDrag on 18/18 units corroborates the interpolation.
Angles are float32 radians at runtime and degrees on disc.
Coverage is 18 of 110 units: unlike weapons, unit definitions are instantiated
per stage, so it grows by visiting missions. The AI-behaviour tail of Maneuver
is explicitly NOT resolved and marked tentative in the CSV.
Captured from Xenia Canary (BASIC CONTROLS tutorial + Stage 02 from save 01).
Canary backs the whole guest address space with one shared-memory file, so the
game's RAM is readable from the host while it runs -- no debugger, no emulator
patch. That turns the parked Route-B question (fields the IDXD omits because
they sit at their default) from "unreachable" into a table lookup.
gmem.py maps guest VAs into /dev/shm/xenia_memory_* via Xenia's fixed table and
searches only the allocated extents, so a full-RAM scan is ~0.2 s.
weapon_runtime.py finds the parsed objects by scanning for their vtable, then
*solves* the struct layout instead of guessing it: every (field, offset,
encoding) triple is scored against the disc records, and a binding is accepted
only with zero contradictions -- and marked confirmed only when >=10 records
agree on >=3 distinct values, because a field whose samples are all one number
matches any offset holding that constant.
Result: Weapon (0xc0) and Shell (0x200) mapped, 4393 values the disc does not
carry, for all 126 weapons at 5 % save progress. Cross-checks hold -- the
Arsenal DATA SHEET read 4 for wep_05/wep_60 Max. Lock Ons and the memory read
agrees; the in-flight HUD's 06000/00300 match LoadingCount; all 252 objects are
byte-identical across a screen change, so this is definition data, not state.
Two findings fell out: `IsCharging = Yes` switches LoadingCount and
TriggerShotCount to float32 (it partitions the 8 float-encoded records exactly),
and two .tbl entries declare Shell_TCAF_Ship_AAGun with conflicting Power -- the
runtime keeps the one that omits it.
Where the defaulted values *come from* (the IDXD's undecoded binary node region
vs code defaults) is not settled here; they vary per weapon, so they are not one
constructor constant.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Second screen, independent of the pause work: GP_TITLE.pak's ptbtn01..05.rat give
X=542 for all five items and Y=162/242/322/402/482, an 80px pitch where the pause
menu used 70. Measured against the main-menu screenshot the sprite tops sit at a
constant +46px for all five, which is exactly the 45px of Xenia window chrome plus
one — so the record's Y is the sprite's top edge in the guest framebuffer, to the
pixel, on a screen the format was not derived from.
GP_TITLE also splits by sub-screen the way the pause pak splits by context:
ptbtn00 (PRESS A BUTTON), ptbtn01..05 (main menu), ptbtn11..13 (EXTRAS submenu),
pgloading_* (loading screen).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
u32 count at 0x14, then fixed 60-byte entries of [name | 4 flag words | pivotX |
pivotY | 0]. The table lists both .t32 sprites and .rat records, so it is the
screen's element list.
Verified: for all 7 .t32 entries in the tutorial pause bundle the declared pivot
is exactly half the decoded texture's dimensions, 7/7 with no mismatch. That also
settles what the pair at 0x50/0x54 of a .rat record is — a .rat simply opens with
one of these declarations.
The language split is visible inside one bundle: the English build's .t32
declarations carry correct English pivots while its .rat declarations carry
Japanese-derived ones, so the sprite table is regenerated per language and the
layout layer is inherited from the Japanese master.
Adds tools/re-capture/ratc_decls.py (parse the table, check pivots against the
decoded textures).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Follow-up decoding on the layout record:
- 0x50/0x54 is the sprite PIVOT: exactly half the texture size on 4 of 5 plain
sprites. It is a rotation/scale centre, not a draw offset — X/Y remains the
top-left, which is what actually reproduces the screenshot.
- The records are byte-identical across the English and Japanese bundles: layout
is authored once and only the sprites are swapped. That explains pgpbtn01's
pivot implying a 226px texture that neither shipped language has.
- loop1.rat is NOT the screen composition — it is a looping sprite animation
(3-name table + ~30 keyframes at one position). The eff*/deli*/msg draw list is
therefore still unfound, and that gap is now stated plainly.
- The 380-byte top-level entries are PRMD primitives: a colour plus four explicit
corner coordinates for a full-screen quad — the pause dim overlay. The format
is tag-driven ('opt ', 'PRMD', 'end '), not a fixed struct.
Also demotes one piece of evidence. The screenshot is of the TUTORIAL pause menu,
so only pgp_ttrl_* can be checked against it; the in-mission records mapping onto
the same screenshot under a constant offset works only because both builds share
the 70px pitch and proves nothing. In-mission coordinates are now marked
unverified.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Each UI sprite <name>.t32 ships with a <name>.rat companion, and that companion
is the layout record: big-endian u32, a 1280x720 design space at 0x18, and a
placement block of scaleX/scaleY/tint/X/Y. Animated elements repeat the block
with a trailing time field (keyframes); an 'opt ' tag carries a length-prefixed
link to the focused-state record.
Verified in that order, records first: across the four pause buttons exactly one
field varies, stepping 268/338/408/478 at a constant 70px pitch with X fixed at
226; the three items visible in a screenshot of the running game measure
346.0/415.5/485.5, mapping onto those numbers under one constant offset to
within 0.5px. The tutorial build's records are outright framebuffer coordinates,
and compositing its sprites there reproduces the screenshot pixel-accurately.
Also records two mistakes the A/B caught, since a static-only reading would have
shipped both: texture width does not identify a language (Japanese fits English
widths), and the in-mission and tutorial pause menus are separate sprite sets
rather than one set re-packed into slots.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The movie cutscene subtitle + voice pipeline, driven by the ADVERTISE_MOVIE
manifest (the authoritative movie -> subtitle -> voice index). Also flushes
several sessions of local WIP (async viewer loading, grouped-pool XBG7/hero-ship
decode, drawlog tooling). See docs/HANDOFF-movie-voice-subtitles-2026-07-19.md.
Subtitles (movie_subtitle.rs):
- Full movie->track->text chain; join multi-line captions sharing one timing
(fixes S13A dropped "Look at it father" line); overlap-safe active_cues();
Latin-1 accents preserved.
Voice (slb.rs): XACT .slb -> XMA1 RIFF; take the FIRST sub-wave bounded by its
declared data size (fixes S10-S16 alternate-take garble); list_voice_clips.
Manifest (movie_manifest.rs): parse ADVERTISE_MOVIE (0x5B983A08) for the real
movie->voice binding (not always VOICE_<movie>; e.g. hokyu -> VOICE_D_* in etc\).
Resolve the token's sound.pak path via sounds.tbl. DIRECT bindings only — the
demo-id shared-clip fallback for unbound hokyu movies was verified WRONG in-game
and reverted (unbound hokyu stay unvoiced; correct join key is an OPEN problem).
Viewer: manifest-driven voice (movie player toggle + solo button), standalone
"Voice Lines" browser, stacked caption overlay.
Tests: 46 formats-lib + 11 viewer-lib + movie_manifest/movie_subtitle/slb disc
tests; full workspace green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Revert the mis-guided "triangle STRIP" reading (a937779): XBG7 index
buffers are triangle LISTS (prim=4 in the GPU capture; stored-normal
agreement 1.000 as a list vs ~0.49 as a strip). Reinstate the
winding-consistency gate.
Crack the grouped-pool layout used by the hero ship and ~150 detailed
models. A resource's sub-meshes share one index pool (buffers 4-byte
aligned, in descriptor-marker order) followed by one vertex pool (each
vtx_count*stride, same order); the vertex pool is 4-byte aligned after
the index pool. The whole resource is derived from one anchored pivot:
ib0 = vb0 - span - pad (pad in 0..=3, alignment)
ib[i] = align4(ib[i-1] + idx_count[i-1]*2)
vb[i] = vb[i-1] + vtx_count[i-1]*stride
vb0 is found via the unit-normal vertex-run scan; the alignment is
confirmed by validating the LARGEST sub-mesh (most reliable), after
which the rest are read/validated. A single marker reduces this to the
existing adjacency anchor, so grouped generalises it.
Results (cross-checked against a Canary GPU draw-log capture of
DeltaSaber_T.xpr):
- DeltaSaber_T f001 = body + 7 detail parts = 8650 tris, every sub-mesh
0-degenerate / full-coverage / winding-agreement 1.000.
- All 19 previously-declined weapon models now decode (they hit pad 2
and/or lead with a tiny bracket that broke a markers[0] pivot). Corpus
audit: 0/146 geometry files fail (was 19 -> flat-texture in the viewer).
- Stages unchanged (single-marker path is byte-identical; grouped falls
back to the old anchor on failure). 7/7 disc tests green incl. the
strict stage quality audit; new hero_ship_grouped_pool_decodes test.
Viewer: route by count_xbg7; --only matches an exact model name (so the
neutral pose renders without the mnv*/turn180 animation poses). Fix the
albedo matcher: match a sub-model to its _col map by entity stem
(e007_bdy_01 -> e007_col) instead of a full-name prefix, lifting stage
sub-model texturing from ~25% to ~95% (the rest were flat grey). Colour
correctness (channel order/sRGB) remains a separate dynamic-RE item.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Stage containers (hidden/resource3d/Stage_S*.xpr) are collections of up to
~450 enemy/prop sub-models, not single meshes. Xbg7Model::stage_models decodes
them: each resource is a [12-byte header][index buffer][vertex buffer] block
(index count = descriptor marker, vertex count = u32 32 bytes before it) whose
on-disc offset is NOT stored, so it is located by content — one O(file) pass per
stride finds vertex-buffer starts (unit NORMAL at +12 whose previous slot isn't)
and each resource is pinned to the candidate whose indices validate and produce
non-degenerate, well-connected triangles. A connectivity gate (mean triangle
edge <= 0.28x the bbox diagonal) rejects spiky mis-anchors. ~4993 sub-models
decode across the 22 stages.
The same insight fixes the weapon single-model layout: it is [12-byte header]
[index][vertex] too, not [index][12-byte gap][vertex]. Reading indices from the
block start turned the 12 header bytes into 6 junk indices (2 leading degenerate
triangles — the recurring stray-triangle artifact) and dropped the last 6 real
indices. Skipping the header leaves vertex offsets identical (coverage unchanged
at 36/166) and corrects the triangle list. Verified on wep_00/03/04.
Adds `sylpheed-cli mesh {info,render}` — a headless software rasterizer that
writes a shaded PNG (orthographic, z-buffered, two-sided), so recovered geometry
can be verified without the GUI. Stages render as a normalised thumbnail grid;
--only filters sub-models.
Tests: stage_models_{decode,sweep,quality_audit}; docs/re updated (xbg7-mesh.md
evidence log + INDEX.md).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Dynamic-RE follow-up: capture Canary's GPU vertex-fetch + draw calls and
feed the ground truth back into the static decoder.
The GPU capture confirmed the reverse-engineered layout exactly — meshes
draw as triangle LISTs (prim=4) with pos f32x3 @0, normal f16x4 @0x0C,
uv f16x2 @0x14 — and revealed the XBG7 vertex format is NOT fixed-stride:
models omit elements (stride 20 = pos+normal, no UV; 24 = pos+normal+uv).
- mesh.rs: parse the descriptor's vertex declaration ({offset, format-code,
usage} triples; 0x2A23B9=pos f32x3, 0x1A2360=normal f16x4, 0x2C235F=uv
f16x2) to drive per-model stride + element offsets, instead of assuming
stride 24. Coverage 25 -> 36 fully-validated models (e.g. the Stage_S*
props, which are pos+normal only). Same index-range + unit-normal safety
gates; complex/mismatched layouts still declined.
- Endianness note (documented): the capture's fetch endian=k8in32 describes
the GPU's guest-memory copy, NOT the .xpr file bytes — reading the file
with k8in32 breaks the normals (|n|->1.33); naive big-endian per element
is correct (the game rearranges vertex data on load).
- tests: a coverage-regression test (>=35 models) + the existing weapon /
body-declined disc tests still pass.
Confirmed against a Canary draw-log capture; see docs/re/structures/
xbg7-mesh.md (evidence log + declaration table).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reverse-engineer the single-stream XBG7 geometry layout (clean-room: hex
inspection + geometric validation of the retail disc's
hidden/resource3d/*.xpr, no game code copied) and present models as
textured 3D meshes in the explorer.
Format (docs/re/structures/xbg7-mesh.md): XBG7 geometry resources sit
inside XPR2 containers alongside TX2D textures. For ~25 single-stream
models (weapons, simple props) the data section is a sequence of
sub-meshes, each an u16-BE triangle-list index buffer followed (after a
fixed 12-byte header) by a stride-24 vertex buffer whose declaration is
in the descriptor: POSITION f32x3 @0x00, NORMAL f16x4 @0x0C, TEXCOORD
f16x2 @0x14. Sub-mesh (vtx,idx) counts come from descriptor tuples.
The +12 vertex offset is pinned by the recovered normals being exactly
unit-length (align16 lands 4 bytes early and silently corrupts every
field). A safety gate rejects any model whose indices are out of range
or whose mean |normal| is not ~1, declining garbage (Stage_S*
placeholders, the complex multi-stream hero-ship body) rather than
mis-decoding it.
- mesh.rs: Xbg7Model::from_xpr2 -> GameMesh { positions, normals, uvs,
indices }; standalone f16->f32; unit + real-disc tests (weapon decodes
to 215v/364t with unit normals + in-range UVs; DeltaSaber body
declined).
- texture.rs: from_xpr2_index / texture_names so the viewer can pick a
model's _col albedo map.
- viewer: loose .xpr with decodable XBG7 spawns Bevy meshes (real normals,
double-sided) textured with the albedo, framed by the orbit camera; the
central egui panel goes transparent so the 3D scene shows through.
Complex multi-stream body meshes (DeltaSaber f004, other vertex layouts)
remain undecoded and are cleanly declined — next target is dynamic RE.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolved the "colours look odd" channel-order/sRGB question for k_8_8_8_8
(T8aD 2D UI textures) with no emulator run:
- sRGB: Canary's Vulkan host-format table maps plain k_8_8_8_8 to
R8G8B8A8_UNORM and contains no R8G8B8A8_SRGB anywhere — gamma is a
separate explicit EDRAM path. So these are raw linear bytes; apply no
sRGB/gamma decode.
- Channel order: measured 789 real disc textures — byte0 is the opaque
alpha (==0xFF-dominant) in 97% — so on-disk is ARGB. Cross-checked
against xenia-rs decode_k8888_tiled, the Canary-validated M1/M2 path,
which nets the same [A,R,G,B]->[R,G,B,A]. sylpheed-formats is correct.
Promotes the k8888 entry HYPOTHESIS -> CONFIRMED; INDEX updated. XPR2
skybox tiling + viewer display-gamma remain separate open items.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Establish docs/re/ as the spec-side of the clean-room:
- README: confidence tiers (HYPOTHESIS/PROBABLE/CONFIRMED), promotion
requires NEW independent evidence, append-only evidence logs,
demote-on-contradiction, and a clean-room firewall (describe behaviour +
cite addresses; never paste decompiled code). Documents the toolchain
(sylpheed.db + zq.py, xenia-rs probes, Wine-Canary oracle) and the
join-by-guest-PC method with the VA-equality caveat.
- Function/structure templates + a seeded INDEX (9 already-reversed formats).
- First entry: texture-color-k8888 — CONFIRMED plain k_8_8_8_8 is linear
UNORM (no sRGB), channel order still to be measured.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
