Files
Syplheed-Reborn/docs/re/structures/unit-struct-runtime.md
Claude (auto-RE) 72365b217a re: memory-driven autopilot -- infrastructure, and an honest account of what is not solved
Reads the live world out of guest RAM and drives the pad from it. Working:
loop-rate memory reads, whole-RAM float scanning with numpy (1270 orthonormal
3x3 blocks in 6.2 s), entity enumeration by unit type (116 live instances in
Stage 02), pad control written straight into the vgamepad FIFO (the CLI spawns
a process per command and its tap/hold sleep inside the server, so neither is
usable in a control loop), unattended mission entry, and the Hangar loadout --
the "Recommended" control is AUTO SELECT, which at 5 % progress is a no-op
because only two weapons are developed and both are already mounted.

Not working, and the reason the craft is not yet flown: the class 0x820af030
is NOT the live entity. It has one object per spawned thing and carries the
unit-ID string, which is why it looked like the entity list, but every one of
its 384 words is constant across a 29 s in-flight capture. No transform lives
in it or one pointer hop from it. Input correlation (hard left yaw vs hard
right, looking for a turn axis that reverses) does find self-like objects at
cos = -0.99, but they cluster in what looks like a camera volume rather than
the craft, and with no definition pointer near them the trick of learning one
entity's layout and applying it to the rest has nothing to anchor on -- so the
33418 moving triples in a firefight cannot be split into enemies, friendlies
and bullets, and there is nothing to aim at.

Two dead ends are recorded so they are not repeated: RT is not the throttle
(the two-state speed scan therefore found nothing), and comparing orientation
matrices 2 s apart is outside the small-angle regime, which is what produced
"angular velocities" of 30000.

Also corrects the claim in unit-struct-runtime.md that 0x820af030 holds live
state. The definition class 0x820af844 and every value derived from it are
unaffected.

autopilot2.py (a PD controller using body angular velocity from consecutive
rotation matrices) is committed but has never had a valid config to run
against, and is marked as untested.
2026-07-29 19:04:33 +00:00

14 KiB
Raw Blame History

Runtime Unit struct (craft / vessel definitions) — read from live guest memory

Confidence: CONFIRMED for the 27 fields marked below (each binding is reproduced by 1021 independent disc records, on ≥3 distinct values, with zero contradictions), plus the Maneuver block's declaration-order layout (29 anchors, two bases, no conflicts). 🟡 PROBABLE for the fields interpolated between confirmed anchors. 🟡/ for the thin single-value bindings, which are listed but must not be trusted yet.

Captured 2026-07-29 from Xenia Canary running the retail disc in the sylph-re container: all six tutorials and Stage 02 "Declaration of War" loaded from save slot 01. 21 of the disc's 110 units, over 7 snapshots from 7 separate emulator runs.

Why this exists

Craft stats were the one Route-B target the previous two passes could not reach. The Hangar exposes only Gross Weight (a class, not a number), and the menu route has no surface at all for the flight model. From the memory side, menus were equally useless: only the player craft's name string is resident there — the definition objects do not exist until a mission loads.

They do exist in-mission. This documents their layout and the values the disc leaves defaulted.

Finding the class (discovered, not assumed)

tools/re-capture/unit_discover.py takes no vtable as input. It locates every disc unit-ID string in a memory snapshot, finds every aligned word pointing at string_va - d for a range of d, and tallies the word at pointer_site - k across distinct unit IDs. One (d, k, word) combination wins by a wide margin:

δ (name-record → string) ID pointer at word distinct units
0x10 object +0x04 0x820af030 4
0x10 object +0x04 0x820af844 4

— i.e. exactly the Weapon shape: the object holds a name-record pointer at +0x04, and the ID string sits at name_record + 0x10.

The two vtables are two different things, and telling them apart matters:

vtable what it is evidence
0x820af030 spawned entity record — one per spawned thing, but not live state (see below) 12 objects for 4 IDs; the same ID appears many times (one per box in the scene); irregular spacing
0x820af844 parsed definition — the .tbl exactly one object per distinct unit ID; minimum spacing 0x380

Correction (2026-07-29, from the autopilot work): 0x820af030 was described here as holding live state. It does not — across a 29 s in-flight capture all 384 words of it are constant. It is one record per spawned thing, but the flying entity's transform is somewhere else entirely. See autopilot-memory-driven. Nothing below depends on it; the definition class 0x820af844 is unaffected.

Only 0x820af844 is used. It is the runtime image of the .tbl:

  • Within one run it is byte-identical across two snapshots taken ~12 minutes apart with combat in between (14/14 objects, 0 differing bytes) — definition data, not live state.
  • Across runs the same unit is not byte-identical, and that had to be explained rather than waved away. unit_runtime.py --crosscheck compares every unit that appears in more than one snapshot (5 of them, over 7 runs): exactly 15 words differ, and 13 of them hold guest pointers (0x8xxxxxxx/0xbxxxxxxx — heap addresses, which move per process). No solved or interpolated field offset is among the 15 — every value reported here is run-invariant.
  • The two non-pointer stragglers, +0x2c8 and +0x2d0, are stage-dependent: for one and the same unit (UN_f001_TCAF_DeltaSaber_T_Ttrl) +0x2c8 reads 8000.0 in two tutorials and 10000.0 in a third, with +0x2d0 a 0/1 flag beside it. So the object is mostly but not entirely the parsed table — a couple of words are set per stage. Unidentified; NEEDS-HUMAN.

One .tblone object. A unit table is several sub-records (Generic, Maneuver, Shield, Explosion, Mass, Effect, SE, Turret_00N), and they are all flattened into that single ≥0x380-byte object — unlike weapons, where Weapon and Shell are separate arrays.

Solving the layout

Same discipline as weapon_runtime.py: score every (field, byte-offset, encoding) triple against the disc records and accept a binding only on zero contradictions, requiring ≥3 distinct values so a field whose samples are all one number cannot match any offset holding that constant.

python3 tools/re-capture/unit_runtime.py unit_tokens.txt snap_a.bin snap_b.bin --csv

Snapshots are unioned — each mission instantiates only the units in its own stage, so coverage grows by visiting stages. cp --sparse=always a copy of /dev/shm/xenia_memory_* first (~2 s); the running emulator pegs every core under lavapipe and makes repeated live reads flaky.

Encoding note — angles are radians at runtime

Every AV_* / AA_* / *Bank* / Turn_AngularVelocity field is stored as float32 radians, while the disc writes degrees. The solver needed a rad encoding (degrees(f32)) to bind them at all; 18 units agree on AV_PitchPlus_Max alone. A reimplementation reading the .tbl must convert.

Confirmed layout

= ≥10 disc records agree on ≥3 distinct values, zero contradict.

offset enc field agree distinct
+0x030 f32 Size_X 21 13
+0x034 f32 Size_Y 12 7
+0x038 f32 Size_Z 19 13
+0x040 f32 Color_R 21 5
+0x044 f32 Color_G 19 6
+0x048 f32 Color_B 17 5
+0x050 f32 Size_Radius 12 10
+0x054 f32 HP 19 10
+0x074 f32 ResistanceToOptics 11 3
+0x08c f32 ScorePoint 21 9
+0x094 f32 MassScore 10 7
+0x09c f32 MinimumVelocity 11 3
+0x0a0 f32 MaximumVelocity 19 8
+0x0a4 f32 CruisingVelocity 17 7
+0x0a8 f32 Acceleration 18 5
+0x0ac f32 Deceleration 17 5
+0x0b0 rad AV_PitchPlus_Max 18 7
+0x0b4 rad AV_PitchPlus_Min 10 6
+0x0c4 rad AV_PitchMinus_Min 10 5
+0x0f8 f32 SideThrustVelocity_Max 14 3
+0x238 f32 MaxValue (Shield) 13 6
+0x244 f32 ChargeSpeed (Shield) 13 6
+0x270 f32 DestroyMotionTime 19 7
+0x2a0 f32 RadarRange 18 9
+0x2a4 f32 FCSRange 12 8
+0x2b4 f32 AttackVesselPoint 14 8
+0x2bc f32 DefencePoint 12 7

HQRatio +0x058, ShieldRatio +0x05c, ThrusterRatio +0x060, ResistanceToShell +0x078, ResistanceToExplosion +0x07c, ResistanceToPlayer +0x080, ResistanceParalyze +0x084, BridgeCount +0x070 (i32), DryMass +0x274, GrossMass +0x278, LowerHPThresholdRatio +0x298, AttackCraftPoint +0x2b8 bind with zero contradictions on fewer records or fewer distinct values — 🟡 PROBABLE. The full solver output is in captures/unit-runtime-fields.csv (1 827 values, conf column).

The Maneuver block is laid out in schema declaration order

This is the strongest structural result and it is independent of any single field's agreement count.

schema_order.py merges the Maneuver field-name order from all 113 unit tables by topological sort over their pairwise "k[i] precedes k[i+1]" constraints. The merge is acyclic and every one of the 113 tables is a subsequence of the merged 102-field order — so that order is the schema's.

Against it, the solved offsets fall into two exact runs:

declaration indices offset rule anchors that fit
0 … 20 (MinimumVelocityAA_Roll_Min) 0x09c + 4·i 21 / 21
21 … 32 (SideThrustVelocity_MaxAccPitchFactor) 0x0a4 + 4·i 8 / 8

One 4-byte slot per field, with a two-slot gap after AA_Roll_Min (0x0f00x0f4, purpose unknown). 29 independently-derived anchors, zero conflicts, across a 0x9c0x125 span.

Fields NO disc record ever values — 🟡 PROBABLE

Five Maneuver fields are declared by the schema but left at their default by every one of the 110 unit tables, so no amount of disc analysis can ever reach them. The declaration-order rule pins them between confirmed anchors (YawDragFactor +0x104ArterBurner_Vc +0x114, and ReverseThrust_Vc +0x118ReverseThrust_Acc +0x120):

offset field craft (f00*, e0*) capital ships (*1**, e2*) inert (SchlosBase, Box)
+0x108 PitchDragFactor 3 2 0
+0x10c RollDragFactor 3 2 0
+0x110 DragFactorThreshold 0.5 0.5 0
+0x11c ArterBurner_Acc 2 2 0
+0x128 DecPitchFactor 30 30 0

Corroboration beyond the interpolation, checked over all 18 units:

  • PitchDragFactor == RollDragFactor == YawDragFactor holds 18/18 — and YawDragFactor is disc-supplied (3.0 for craft, 2.0 for warships), so two interpolated offsets reproduce a known number, per unit, every time.
  • DecPitchFactor == AccPitchFactor holds 17/18. The exception is UN_e015_ADAN_Puppy (AccPitchFactor 1, DecPitchFactor 0.5) — which defaults both on disc, so it is two independent fields that happen to be set equal elsewhere, not a broken binding.

UN_e015_ADAN_Puppy also breaks the craft/warship bucketing above for DecPitchFactor (0.5, not 30); the per-unit values are in the CSV.

The tail of the Maneuver block (the AI-behaviour fields — SideRoll_*, BarrelRoll_*, TurnAttack_*, HoldPosition_*, Slalom_*, Through_*, SolidCutoff_*, and the AxisMode / AB_* sub-block) is NOT resolved. Those fields are declared by only a handful of tables and almost always with a single distinct value, so the solver's bindings there are coincidences: it placed Slalom_CutoffRatio at +0x00c and TurnAttack_DoubleTimeMin at +0x110, both of which the declaration-order rule contradicts. They are marked tentative in the CSV. NEEDS-HUMAN / needs more coverage — more stages would give those fields distinct values and settle it.

The player craft, UN_f001_TCAF_DeltaSaber_T_Player

30 of its fields are defaulted on disc. Notable recovered values:

field value note
Size_Radius 10
FCSRange 500000 — same as RadarRange
ChargeSpeed (shield) 25
ResistanceToOptics 1
HQRatio / ShieldRatio / ThrusterRatio 1 / 1 / 1 🟡
ResistanceToShell / ResistanceToExplosion 1 / 1 🟡
DryMass 100 🟡 (GrossMass 250 is on disc)
LowerHPThresholdRatio 0.3 🟡
ChargeDelay_Break 10 🟡
MassScore 0 🟡

Every AI-behaviour field the solver bound reads 0 for the player craft, which is the expected shape (the player is not AI-driven) — but see the caveat above: those offsets are not settled, so treat the zeros as consistent, not proven.

The …Ratio family that the Route-B target list parked is 1.0 for almost every unit, with real exceptions that only the runtime shows: UN_e105_ADAN_Cruiser HQRatio = 0.2, UN_bf001_TCAF_SchlosBase and UN_e106_ADAN_Destroyer ThrusterRatio = 0.2, UN_n001_TTRL_Box ShieldRatio = 0.3.

Coverage and how to extend it

21 of 110 units. Unlike weapons — where one snapshot held all 126 — unit definitions are instantiated per stage, so coverage is bounded by the stages reachable from the save (slot 01 is at 5 %, Stage 02). unit_runtime.py unions any number of snapshots and re-solves, and more units directly promote the 🟡 bindings to by adding distinct values — the six tutorials took the confirmed set from 22 fields to 27.

The tutorials are nearly exhausted as a source: all six together contribute only 3 units the missions do not already have (UN_f001_TCAF_DeltaSaber_T_Ttrl, UN_e015_ADAN_Puppy_2, UN_f001_TCAF_DeltaSaber_T_Player_Ttrl2) — they reuse one training box, one target drone and the player craft. Real coverage now needs real missions, i.e. story progress on the save.

tools/re-capture/grab_tutorial.sh captures one tutorial per invocation (cold boot → menu → Nth entry → snapshot, ~2.5 min). It cold-boots for each because backing out of a loaded mission via PAUSE → BACK TO MENU wedges the emulator. Two timing facts it encodes, both learned the hard way: the main menu is not input-ready for ~10 s after the title tap, and d-pad presses before that are silently dropped — which sends the A to NEW GAME instead of TUTORIAL. And NEW GAME is not a cheap way to reach Stage 01: it gates on a DIFFICULTY menu and then plays the prologue movie.

A NEW GAME excursion as far as the READY ROOM leaves 535107D4/00000001/game01/savedata byte-identical — only the profile .gpd achievement files change — so it does not endanger the 5 % save. Verified by diff against a backup, not assumed.

A stage's whole unit set is parsed at load, not as waves spawn — checked by counting the definition objects at three points in Stage 02: immediately after take-off, ~12 minutes in mid-combat, and after GAME OVER. 14 objects, the same 14 IDs, every time. So capturing a stage costs one load and one snapshot; there is no need to play it, and no need to survive it.

Stages captured so far: Ttrl (BASIC CONTROLS), Stage 02.