Syplheed-Reborn/docs/re/INDEX.md

RE knowledge index

Confidence: ✅ CONFIRMED · 🟡 PROBABLE · ❔ HYPOTHESIS. See README.

Formats we've already reversed are, for now, documented by their parser + disc round-trip tests (the executable spec) rather than a prose file — the "Spec" column points there. Promote to a prose structures/…md file when a format needs behavioural notes beyond layout.

Data structures / formats

Format	Conf.	Spec (parser + tests)	Notes
IPFB .pak archive	✅	sylpheed-formats/src/pak.rs + tests/pak_idxd_disc.rs	header + 12-byte TOC, Z1/zlib payloads
name-hash (TOC keys)	✅	sylpheed-formats/src/hash.rs	Barrett-reduction hash; recovers original paths
IDXD object/table	✅	sylpheed-formats/src/idxd.rs	self-describing; ship/weapon stats verified vs known values
XPR2 texture + cubemap	🟡/✅	sylpheed-formats/src/texture.rs + colour check	de-tile + A8R8G8B8 and DXT1. Channel order ✅ confirmed against the running game: the Delta Saber's decoded atlas is orange-dominant (median saturated hue 23.3°, zero cool pixels) and the game renders the same hull at 9.3° — a red↔blue swap would sit at ≈200°. Exact fidelity (gamma/sRGB curve, premultiplied alpha, per-channel scale) is 🟡 untested, since a hue comparison cannot see it; cubemap face ordering ❔
T8aD 2D texture	✅	sylpheed-formats/src/t8ad.rs	100 % of the disc decodes (19 216/19 216, measured). The "~15 % deferred variants" were a wrong model, not a variant: a surface is a list of arbitrary sub-rectangles, each with a 16-byte header of dst X, dst Y, width, height, not a 256×256 grid — 0x1c is the rectangle count. Uncovered area stays transparent. Colours ✅ CONFIRMED (k8888)
RATC bundle	✅	sylpheed-formats/src/ratc.rs	child listing confirmed. "One level deep" is not a limitation — there is nothing deeper: 2 859 bundles hold 18 002 children at depth 1 and 0 at depth 2, with no parse failures. Nested RATC blobs are leaf records that reference siblings by name (opt , the sprite name): 3 311 leaves, all embedding sibling names, 10 144 of 10 148 references resolve. The 4 that do not are one dangling asset — pmbase.rat → pmbase.t32 in GP_STAGE_CLEAR.pak's four language builds, and pmbase.t32 is on the disc nowhere
LSTA sprite list	✅	sylpheed-formats/src/lsta.rs	A display list of inline elements: T8aD sprites and PRMD primitives. The count at 0x04 is exact and counts both — count == T8aD + PRMD for 64/64 lists on the disc, which retires the old "a few entries disagree" note (it compared sprites against a total including primitives). All 1 281 sprite frames decode after the T8aD rectangle-list fix
IXUD subtitle	🟡/✅	sylpheed-formats/src/ixud.rs + movie link	timed cues. The movie↔subtitle↔voice link is solved — statically, from the movie config record in tables.pak (schema 0x067025b9), not from the running game as this row previously assumed: 101 movies mapped, 94 with subtitles, 83 with voice, 21 with a telop overlay. 93 of 94 subtitle refs resolve in the language paks; SUBTITLE_S12B.tbl is missing from all six languages — a dangling reference on the disc. Naming is SUBTITLE_<base>.tbl / VOICE_<base> with six documented exceptions. The record's ~104 script ids are ❔ — positional pairing drifts by three because the IDXD pool dedupes repeated values
Fonts (ttf/otf/ttc)	✅	sylpheed-formats/src/font.rs	standard OpenType, parsed via ttf-parser
XBG7 mesh	✅/🟡	sylpheed-formats/src/mesh.rs + tests/mesh_disc.rs (xbg7)	6 294 resources, 6 209 decode (98.7 %), 82 searched-and-missed (2026-08-12, up from 5 480 / 87.1 %). Five evidence-driven fixes got there: distinct anchor assignment (no two resources may claim one buffer — proved by a capture showing the container holds both mirrored e106 hull halves), the connectivity cap replaced by a winding-consistency gate at 0.70, structural requirements on pre-pivot sub-meshes (index range, then exact pool coverage), and filtering after the assignment so a subset query cannot differ from the full decode. Validated against a runtime capture that names the file offset of every buffer the engine drew: 46/46 drawn buffers claimed, 45 anchored exactly. No real mesh now decodes differently in different containers — all 89 remaining cross-container disagreements are interchangeable 24-vertex bounding boxes, which no anchoring rule can pin (monotone order re-tested and refuted). Remaining misses attribute to the degeneracy/extent gate (42), winding (31) and coverage (9); the first was probed and its "obvious" fix refuted. Every decoded sub-mesh covers its own vertex pool. The [index buffer][vertex buffer] layout is now runtime-verified (2026-08-13): with the F10 capture extended to log each draw's index buffer, all 42 drawn Stage_S02 buffers match our decoded index count exactly, all 42 have their index union cover the pool exactly, and the 30 single-block cases all sit at pad ≤ 3 — so e106_eng_02_l's old rejection was the connectivity gate, not a misplaced index buffer. The indices= mystery was the capture keeping only the first of several index batches per buffer. And comparing index VALUES found the biggest silent defect yet: the anchor took the first pad that validated, so a block whose index data sits at pad 2 was read one element late — 76/93 captured runs matched, all 17 differences a one-element shift. Scoring pads by degenerate triangles + winding fixes it: 93/93 captured runs now match byte for byte, disc-wide degenerate runs 582 → 1 (the grouped path had the same bug; and two resources were anchored on a degenerate lookalike earlier in file order), 590 of 8 850 sub-meshes re-wired with 10 vertex anchors moved, resources decoded unchanged at 6 209. Cross-container minority decodes 89 → 96 — because the decoder improved: _rou_f402_dead now has a majority (32×25×8) so its seven wrong copies are named instead of hidden. One dirty run remains, blocked by distinct assignment on a 24-vertex box. Then the descriptor gave up its last structural secret: it declares a vertex layout per sub-mesh (n201_01 → strides 24/24/24/28, capture-confirmed), and grouped selection must prefer the candidate explaining the whole pool rather than the first whose pivot validates — together they take never-decoding resources 85 → 47 (6 247 / 6 294 = 99.25 % decode), put n201_01 on all four capture-proven offsets and raise the stage-05 capture oracle to 128/128. The residual 47 is 30 pose/proxy composites (0.010-unit marker boxes), 6 .DAT particle composites, 8 damage/LOD variants and 3 props — not a threshold away
Capital-ship part placement	✅	sylpheed-formats/src/ship.rs (static) + runtime capture	Placement is sound (hull static-exact against the e106 capture; cross-id mounting genuinely narrow, 2 pairs across 335 ships). The XBG7 mis-decode this row used to blame for "ships assemble wrong" — a shared turret ~100× too large in some containers — is fixed (2026-08-12, the exact-coverage requirement): e303_wep_01 now decodes 49×23×42 everywhere and places at ±179 on the e106 hull, and no real mesh disagrees across containers. A composite-node audit confirmed the assembler itself never applied a bad scale (all nodes scale 1.0, orthonormal). Still open: static_assembly_matches_runtime_capture walks capture parts only, so extra static placements cannot fail it
Weapon fields defaulted on disc	✅	runtime struct · DATA SHEET route	Solved. Canary maps guest RAM into /dev/shm, so the parsed Weapon/Shell objects are readable live; their layout is solved against disc ground truth (zero contradictions over 100+ records). All 126 weapons, exact numbers, no story progress needed — 4 393 values the disc does not carry. Supersedes the letter-bucket limit of the DATA SHEET route, which now serves as the independent cross-check
Unit (craft/vessel) fields defaulted on disc	✅/🟡	runtime struct	The parsed unit\UN_*.tbl definition object, vtable 0x820af844, ≥0x380 bytes, one per unit — discovered, not assumed (unit_discover.py), and distinguished from the spawned-entity class 0x820af030 by being one-per-ID and byte-constant within a run. Across runs only pointer words move — --crosscheck proves no reported field offset is run-dependent (two words, +0x2c8/+0x2d0, are stage-dependent and remain unidentified). 27 fields ✅ (21 units, 7 runs); the Maneuver block is schema declaration order, 4 bytes/field, base 0x9c with a two-slot gap after AA_Roll_Min (29 anchors, 0 conflicts), which also pins 5 fields no disc record ever values. Angles are radians at runtime, degrees on disc. Re-derived independently 2026-08-13 from the loader's own key strings (sub_82341A20; the field name for each store is a string in the image): 159 fields, agreeing with this solver on 25 of 25 shared offsets, verified at 406 values matching the disc and 0 disagreeing over 11 live objects spanning UNIT and VESSEL — landed as data/unit_definition_layout.txt + sylpheed_formats::unit_layout + a no-emulator test, with 121 defaulted fields read out (live-unit-definitions). Unlike weapons, unit definitions are instantiated per stage, so coverage (21/110) grows by visiting missions — but a defaulted field is not a global constant: Size_Y provably inherits Size_X (7 independent units, 6 distinct values), and three more sibling rules are recorded ❔, recovering 65 values in units never visited — values
Arsenal develop economy	✅/❔	arsenal-develop-economy + conditions	The Arsenal reads weapon.tbl (item ids, in the 8-category display order) and strings.tbl (names, descriptions, and a "Conditions to obtain" block per item) out of GP_HANGAR_ARSENAL.pak. All 60 conditions are extracted: gates are stage completion, a predecessor item, or an ace kill; costs run 3 000–350 000 P and 20 items are free once gated. weapon.tbl's first record reproduces the in-game DATA SHEET exactly (Range D / Power E / Speed – / Weight 0.3 = Light / 4000 P) — later records are unreadable from the string pool alone because IDXD dedupes repeated values. Used to identify the save blob's index space, now solved: the blob follows strings.tbl's order — the display order plus the cut items only the localisation file lists (Adhesive Mine B2A, Ballista GSH, …) — pinned by four hand-written probe saves (9 Stiletto, 21 Falcon, 39 Tomahawk, 48 Jamming System) and closing exactly at index 53. weapon.tbl's id list is not the index space; that it is also 54 long is a coincidence, and the two agree only to index 32. The retail save's five unexplained owned entries are the cut items, shipped owned and never rendered
UI screen layout (.rat)	✅/🟡	ui-rat-layout	One pak per UI screen; each RATC = one (context × language) build; every <name>.t32 sprite has a <name>.rat layout record (BE u32; 1280×720 design space; scale/tint/X/Y, keyframes for animated elements, opt link to the focused state). The tutorial PAUSE menu and the title main menu both rebuild pixel-accurately from the disc. loop1.rat is decoded — it is a looping sprite animation, not a composition. The screen's draw list is the RATC bundle's own declaration table (elements in back-to-front order, including the eff*/deli*/msg sprites that have no .rat, and excluding focused button variants reached via opt ); its entry also carries a parent element index at +32. A screen is fully reconstructible from its bundle: the placement region right after the declaration table gives every element a keyframe group (header = element index + keyframe count, then 40-byte blocks of scale/tint/X/Y), including the .rat-less sprites — verified 11/11 on the tutorial pause bundle, with pgp_ttrl_btn10's inline (546,288) matching its own record exactly
Save file (savedata)	✅/❔	savegame-format + tools/re-capture/savegame.py	GDHA container, zlib payload, chunk stream (GDAA / phase name / GHAD 122 B progress block / 16×20 B slot table / trailer). Container and layout read off the title's own serializer 0x822C00E8 and verified by a byte-identical round-trip; the whole save is 545 B. Payload offsets are also the live save object's offsets (save+8 GHAD, save+136 slots). A second save made in-game names Points (+24), flight time in ms (+4) and clear ratio % (+8) off the game's own Details panel; the payload is a pure function of game state (same state saved twice = byte-identical, only the header FILETIME and its uninitialised pointer padding move), and the 16 SHAB records are not the UI's 20 save slots. Difficulty vs stage is undecided — three fields hold 2. A third save, taken after developing exactly one Arsenal weapon (Light Machine Gun MG I, 4000 P), moves exactly three things: +24 Points 4101→101 (which separates it from +28, that did not move), +8 clear ratio 5→6 (so the ratio counts collection, not only stages), and two entries of the 54-byte blob — 2→4 for the item bought and 0→2 for the successor the game announced as newly developable, giving the blob its alphabet ✅ 0 locked / 2 developable / 4 developed (only the 4s are stored — 2 is re-derived at load). Saves can also be written back: three derived header fields (length at +0x30, payload length at +0x8c, adler32 at +0x8e) are all that stand between a parse and a hand-written save that the title loads, and savegame_edit.py re-wraps a real save byte-identically. That turned the blob's index space from blocked-on-story-progress into four probe saves — see the economy note

Runtime / dynamic-capture technique

Technique	Conf.	Spec	Notes
Scripted input without a virtual controller	✅	tools/re-capture/pad.py + Canary --hid=file	The old vgamepad path created its device through /dev/uinput, which is not namespaced — a pad made inside the container registers with the HOST's input stack, so every scripted press leaked to the desktop. Canary now carries a header-only driver that reads pad state from a text file (--hid=file --pad_file=…): no kernel device, nothing leaves the container, and analogue values are exact. ⚠️ The trap: 360 menus poll XamInputGetKeystrokeEx, not GetState — with GetKeystroke stubbed the pad looks completely dead on a title screen while its own log shows the press arriving. Implemented edge-triggered, no auto-repeat (scripted input wants one event per press). Proven end to end: boot → title → main menu → EXTRAS from the file alone
Live guest-memory write	✅	tools/re-capture/gpoke.py	Write companion to gmem.py, same guest-VA → /dev/shm map; prints before/after per word. Used to confirm the challenge gate's cleared-stage mask on the running game
Live guest-memory read	✅	tools/re-capture/gmem.py	Canary backs the guest address space with /dev/shm/xenia_memory_*; guest VAs map in through Xenia's fixed table. Full-RAM search ~0.2 s (sparse, SEEK_DATA). No debugger, no emulator patch, game keeps running
IDXD object layout solver	✅	tools/re-capture/weapon_runtime.py	Scan RAM for a class's vtable → enumerate its objects → brute-force (field, offset, encoding) against the disc records. Accepts a binding only on zero contradictions. Generalizes to any IDXD-backed definition
Live entity state, anchored on the definition	✅	tools/re-capture/own_state.py · autopilot	An undamaged craft holds its definition's own numbers, so a solved definition field locates the matching live field without a value scan: definition HP (1500) → hull at position+0x154, confirmed by a trace across a death (30/60/90 per hit, negative at 0). Reusable for any live counter whose maximum the definition carries
Mission / escort state, every entity's hull	✅	tools/re-capture/mission_state.py · escort state	hull = position + 0x154 is a property of the entity class, not of the player object: at t=0 it equals each entity's own definition HP across 7 classes and 5 distinct HP values (turret 100, fighter 500, destroyer 10000, cruiser 30000, ACROPOLIS 25000), falls under fire (780 damage events in 240 s), goes negative at death, and the object then leaves the heap. So an escort objective is scoreable live — UN_f101_TCAF_Acropolis measured at 25000 → 23038 over 240 s, attack starting only at t≈170 s. REMAINING OB counts objectives, not hostiles (012 on the HUD vs 118 live ADAN); its address is still ❔
In-flight control mapping	✅/🟡	tools/re-capture/fire_probe.sh · controls	Measured by holding each pad input and photographing the HUD ammo counters: RB = nose gun (6000→5956 in 4 s, ~11 rounds/s, HEAT rises), Y = main mount (missiles, 300→299), d-pad = tactical map overlay, nothing else moves a counter. No target-cycle input exists — the TARGET marker is present with nothing pressed, so targeting is automatic and a missile lock is time-on-target. That, not target choice or ballistics, is what caps lethality at 2 kills per 98 missiles
Throttle → speed law	✅/🟡	flight-speed-law	The throttle selects a TARGET SPEED, it does not add thrust: no input settles at ~420 (CruisingVelocity 350), RT at ~1 530 (MaximumVelocity 1200), LT at ~125 (MinimumVelocity 100), and releasing either returns to cruise. Acceleration/Deceleration govern the convergence rate (measured ~440–560 units/s² against 500/600). 🟡 measured world speeds run ≈1.2–1.3× the definition numbers while the HUD shows the definition value exactly, so world units are a constant (~1.25) multiple of the definition's velocity unit
Input → dynamics calibration	✅	tools/re-capture/ctrl_probe.py · binq.py	Hold each pad input in turn and measure the craft's speed as displacement/s of its own position triple — no speed field needed first. Settled the throttle: RT accelerates, LT brakes, and the setting persists (488 → 1510 → 174 units/s), overturning an earlier field-scan conclusion

Functions / code paths

Function	Conf.	Reimpl.	Summary
Achievement table + XAM read path	✅/🟡	achievements	The XEX's XACH resource (.pe 0x8FBCBC, 36-byte records) defines 24 achievements summing to 1000G — the retail total, which self-checks the stride and field offsets. GamePart_Debriefing (0x8218CF38–0x82191B18) does two things: it walks the on-disc ACHIEVEMENTS_REQUIREMENTS list (tables.pak #16, entries ACHIEVEMENT01…24, in achievement-id order — its ShootDownAircrafts/ShootDownShips/ShootDownWeight/GetAllWeapons/GetAllAchievements types line up with ids 19–24 exactly as XACH names them), evaluating each and setting a bit; and it enumerates XACHIEVEMENT_DETAILS from XAM — 36-byte records confirmed by the 0x38E38E39+srawi 3 divide-by-36, dwId at +0, dwFlags & 0x00020000 (…_ACHIEVED) at +32, behind a waited-then-closed async handle. So earned state comes from the console profile, not the 545-byte save — and the masks it builds are 1 << dwId, i.e. bit = the 1-based id. The challenge missions do NOT gate on this — an earlier claim here, refuted by finding +80's sole writer: it is GamePart_StageClear setting 1 << stage, so that word is a cleared-stage mask and the shared "24" was a coincidence. GetAllAchievements/GetAllWeapons are requirement types, not debug cheats
Challenge-mission gate + stage-config switch	✅	challenge-mission-gate	GamePart_ChallengeMission gates each of the six challenge missions on a CLEARED-STAGE bit: REQUIREMENT absent or "Always" → available, else n = atoi(v) and it tests bit n of singleton +80 (n < 24) or bit n-24 of +1956 (n ≥ 24) — which is the disc's own stage numbering (story 1–16 and tutorial 18–23 below 24, challenge 24–29 above). Word A's sole writer is 0x821C1820 in GamePart_StageClear, doing 1 << (this+84) where this+84 is the stage number (it also indexes a 20-byte per-stage record array and the debriefing STAGE sprite list). So TimeAttack needs stage 16 — the last story mission — and the rest chain off challenge stages 25–29. The six-mission table is on disc in tables.pak (schema 54a10697). Separately, the stage loader picks its config section from a mission-kind field at object+144: 3 → EXTRA, 5/6 → CHALLENGE, anything else → FILE; two further sites treat {3,5,6} as one class. Constructed as 0 (sub_821783D8) and only ever cleared inside the class, so the kind comes from the launching GamePart, not from the stage number — which is a mechanism (🟡, unproven) for why patching the save's stage field to a challenge stage kills the load. Same note carries the GamePart id table (0x820A1630, 29 ids, GP_CHALLENGE = 26, cross-checked against the image's own RegisterToFactory<26, …> text) and the disc's three stage families — S01–S16 story, S18–S23 tutorial, S24–S29 challenge, plus Test, matching weapon.tbl's 16 + 6 + 6 key set exactly. GP_CHALLENGE.pak holds 0 IDXD objects — it is the menu screen; challenge missions reuse GP_MAIN_GAME_E.pak's stage records