Claude (auto-RE)
72365b217a
Reads the live world out of guest RAM and drives the pad from it. Working: loop-rate memory reads, whole-RAM float scanning with numpy (1270 orthonormal 3x3 blocks in 6.2 s), entity enumeration by unit type (116 live instances in Stage 02), pad control written straight into the vgamepad FIFO (the CLI spawns a process per command and its tap/hold sleep inside the server, so neither is usable in a control loop), unattended mission entry, and the Hangar loadout -- the "Recommended" control is AUTO SELECT, which at 5 % progress is a no-op because only two weapons are developed and both are already mounted. Not working, and the reason the craft is not yet flown: the class 0x820af030 is NOT the live entity. It has one object per spawned thing and carries the unit-ID string, which is why it looked like the entity list, but every one of its 384 words is constant across a 29 s in-flight capture. No transform lives in it or one pointer hop from it. Input correlation (hard left yaw vs hard right, looking for a turn axis that reverses) does find self-like objects at cos = -0.99, but they cluster in what looks like a camera volume rather than the craft, and with no definition pointer near them the trick of learning one entity's layout and applying it to the rest has nothing to anchor on -- so the 33418 moving triples in a firefight cannot be split into enemies, friendlies and bullets, and there is nothing to aim at. Two dead ends are recorded so they are not repeated: RT is not the throttle (the two-state speed scan therefore found nothing), and comparing orientation matrices 2 s apart is outside the small-angle regime, which is what produced "angular velocities" of 30000. Also corrects the claim in unit-struct-runtime.md that 0x820af030 holds live state. The definition class 0x820af844 and every value derived from it are unaffected. autopilot2.py (a PD controller using body angular velocity from consecutive rotation matrices) is committed but has never had a valid config to run against, and is marked as untested.
275 lines
14 KiB
Markdown
275 lines
14 KiB
Markdown
# Runtime `Unit` struct (craft / vessel definitions) — read from live guest memory
|
||
|
||
**Confidence: ✅ CONFIRMED** for the 27 fields marked ✅ below (each binding is
|
||
reproduced by 10–21 independent disc records, on ≥3 distinct values, with
|
||
**zero** contradictions), plus the Maneuver block's declaration-order layout
|
||
(29 anchors, two bases, no conflicts). 🟡 PROBABLE for the fields interpolated
|
||
between confirmed anchors. 🟡/❔ for the thin single-value bindings, which are
|
||
listed but must not be trusted yet.
|
||
|
||
Captured 2026-07-29 from Xenia Canary running the retail disc in the sylph-re
|
||
container: **all six tutorials** and **Stage 02 "Declaration of War"** loaded
|
||
from save slot 01. 21 of the disc's 110 units, over 7 snapshots from 7 separate
|
||
emulator runs.
|
||
|
||
## Why this exists
|
||
|
||
Craft stats were the one Route-B target the previous two passes could not
|
||
reach. The Hangar exposes only `Gross Weight` (a class, not a number), and
|
||
[the menu route](../weapon-datasheet-runtime.md) has no surface at all for the
|
||
flight model. From the memory side, menus were equally useless: **only the
|
||
player craft's name string is resident there — the definition objects do not
|
||
exist until a mission loads.**
|
||
|
||
They do exist in-mission. This documents their layout and the values the disc
|
||
leaves defaulted.
|
||
|
||
## Finding the class (discovered, not assumed)
|
||
|
||
[`tools/re-capture/unit_discover.py`](../../../tools/re-capture/unit_discover.py)
|
||
takes no vtable as input. It locates every disc unit-ID string in a memory
|
||
snapshot, finds every aligned word pointing at `string_va - d` for a range of
|
||
`d`, and tallies the word at `pointer_site - k` across **distinct** unit IDs.
|
||
One `(d, k, word)` combination wins by a wide margin:
|
||
|
||
| δ (name-record → string) | ID pointer at | word | distinct units |
|
||
|---|---|---|---|
|
||
| `0x10` | object `+0x04` | `0x820af030` | 4 |
|
||
| `0x10` | object `+0x04` | `0x820af844` | 4 |
|
||
|
||
— i.e. exactly the `Weapon` shape: the object holds a name-record pointer at
|
||
`+0x04`, and the ID string sits at `name_record + 0x10`.
|
||
|
||
The two vtables are **two different things**, and telling them apart matters:
|
||
|
||
| vtable | what it is | evidence |
|
||
|---|---|---|
|
||
| `0x820af030` | **spawned entity record** — one per spawned thing, but **not live state** (see below) | 12 objects for 4 IDs; the same ID appears many times (one per box in the scene); irregular spacing |
|
||
| `0x820af844` | **parsed definition** — the `.tbl` | exactly one object per distinct unit ID; minimum spacing `0x380` |
|
||
|
||
**Correction (2026-07-29, from the autopilot work):** `0x820af030` was
|
||
described here as holding live state. It does not — across a 29 s in-flight
|
||
capture **all 384 words of it are constant**. It is one record per spawned
|
||
thing, but the flying entity's transform is somewhere else entirely. See
|
||
[autopilot-memory-driven](../autopilot-memory-driven.md). Nothing below depends
|
||
on it; the definition class `0x820af844` is unaffected.
|
||
|
||
Only `0x820af844` is used. It is the runtime image of the `.tbl`:
|
||
|
||
* **Within one run** it is byte-identical across two snapshots taken ~12 minutes
|
||
apart with combat in between (14/14 objects, 0 differing bytes) — definition
|
||
data, not live state.
|
||
* **Across runs** the same unit is *not* byte-identical, and that had to be
|
||
explained rather than waved away. `unit_runtime.py --crosscheck` compares
|
||
every unit that appears in more than one snapshot (5 of them, over 7 runs):
|
||
exactly **15 words differ**, and 13 of them hold guest pointers
|
||
(`0x8xxxxxxx`/`0xbxxxxxxx` — heap addresses, which move per process).
|
||
**No solved or interpolated field offset is among the 15** — every value
|
||
reported here is run-invariant.
|
||
* The two non-pointer stragglers, `+0x2c8` and `+0x2d0`, are **stage-dependent**:
|
||
for one and the same unit (`UN_f001_TCAF_DeltaSaber_T_Ttrl`) `+0x2c8` reads
|
||
`8000.0` in two tutorials and `10000.0` in a third, with `+0x2d0` a 0/1 flag
|
||
beside it. So the object is *mostly* but not *entirely* the parsed table —
|
||
a couple of words are set per stage. Unidentified; **NEEDS-HUMAN**.
|
||
|
||
One `.tbl` → **one** object. A unit table is several sub-records (`Generic`,
|
||
`Maneuver`, `Shield`, `Explosion`, `Mass`, `Effect`, `SE`, `Turret_00N`), and
|
||
they are all flattened into that single ≥`0x380`-byte object — unlike weapons,
|
||
where `Weapon` and `Shell` are separate arrays.
|
||
|
||
## Solving the layout
|
||
|
||
Same discipline as
|
||
[`weapon_runtime.py`](../../../tools/re-capture/weapon_runtime.py): score every
|
||
`(field, byte-offset, encoding)` triple against the disc records and accept a
|
||
binding only on **zero contradictions**, requiring ≥3 distinct values so a
|
||
field whose samples are all one number cannot match any offset holding that
|
||
constant.
|
||
|
||
```bash
|
||
python3 tools/re-capture/unit_runtime.py unit_tokens.txt snap_a.bin snap_b.bin --csv
|
||
```
|
||
|
||
Snapshots are **unioned** — each mission instantiates only the units in its own
|
||
stage, so coverage grows by visiting stages. `cp --sparse=always` a copy of
|
||
`/dev/shm/xenia_memory_*` first (~2 s); the running emulator pegs every core
|
||
under lavapipe and makes repeated live reads flaky.
|
||
|
||
### Encoding note — angles are radians at runtime
|
||
|
||
Every `AV_*` / `AA_*` / `*Bank*` / `Turn_AngularVelocity` field is stored as
|
||
**float32 radians**, while the disc writes **degrees**. The solver needed a
|
||
`rad` encoding (`degrees(f32)`) to bind them at all; 18 units agree on
|
||
`AV_PitchPlus_Max` alone. A reimplementation reading the `.tbl` must convert.
|
||
|
||
## Confirmed layout
|
||
|
||
✅ = ≥10 disc records agree on ≥3 distinct values, zero contradict.
|
||
|
||
| offset | enc | field | agree | distinct |
|
||
|---|---|---|---:|---:|
|
||
| `+0x030` | f32 | `Size_X` | 21 | 13 |
|
||
| `+0x034` | f32 | `Size_Y` | 12 | 7 |
|
||
| `+0x038` | f32 | `Size_Z` | 19 | 13 |
|
||
| `+0x040` | f32 | `Color_R` | 21 | 5 |
|
||
| `+0x044` | f32 | `Color_G` | 19 | 6 |
|
||
| `+0x048` | f32 | `Color_B` | 17 | 5 |
|
||
| `+0x050` | f32 | `Size_Radius` | 12 | 10 |
|
||
| `+0x054` | f32 | `HP` | 19 | 10 |
|
||
| `+0x074` | f32 | `ResistanceToOptics` | 11 | 3 |
|
||
| `+0x08c` | f32 | `ScorePoint` | 21 | 9 |
|
||
| `+0x094` | f32 | `MassScore` | 10 | 7 |
|
||
| `+0x09c` | f32 | `MinimumVelocity` | 11 | 3 |
|
||
| `+0x0a0` | f32 | `MaximumVelocity` | 19 | 8 |
|
||
| `+0x0a4` | f32 | `CruisingVelocity` | 17 | 7 |
|
||
| `+0x0a8` | f32 | `Acceleration` | 18 | 5 |
|
||
| `+0x0ac` | f32 | `Deceleration` | 17 | 5 |
|
||
| `+0x0b0` | rad | `AV_PitchPlus_Max` | 18 | 7 |
|
||
| `+0x0b4` | rad | `AV_PitchPlus_Min` | 10 | 6 |
|
||
| `+0x0c4` | rad | `AV_PitchMinus_Min` | 10 | 5 |
|
||
| `+0x0f8` | f32 | `SideThrustVelocity_Max` | 14 | 3 |
|
||
| `+0x238` | f32 | `MaxValue` (Shield) | 13 | 6 |
|
||
| `+0x244` | f32 | `ChargeSpeed` (Shield) | 13 | 6 |
|
||
| `+0x270` | f32 | `DestroyMotionTime` | 19 | 7 |
|
||
| `+0x2a0` | f32 | `RadarRange` | 18 | 9 |
|
||
| `+0x2a4` | f32 | `FCSRange` | 12 | 8 |
|
||
| `+0x2b4` | f32 | `AttackVesselPoint` | 14 | 8 |
|
||
| `+0x2bc` | f32 | `DefencePoint` | 12 | 7 |
|
||
|
||
`HQRatio +0x058`, `ShieldRatio +0x05c`,
|
||
`ThrusterRatio +0x060`, `ResistanceToShell +0x078`,
|
||
`ResistanceToExplosion +0x07c`, `ResistanceToPlayer +0x080`,
|
||
`ResistanceParalyze +0x084`, `BridgeCount +0x070` (i32),
|
||
`DryMass +0x274`, `GrossMass +0x278`,
|
||
`LowerHPThresholdRatio +0x298`, `AttackCraftPoint +0x2b8` bind with zero
|
||
contradictions on fewer records or fewer distinct values — 🟡 PROBABLE. The
|
||
full solver output is in
|
||
[`captures/unit-runtime-fields.csv`](../captures/unit-runtime-fields.csv)
|
||
(1 827 values, `conf` column).
|
||
|
||
## The Maneuver block is laid out in schema declaration order
|
||
|
||
This is the strongest structural result and it is independent of any single
|
||
field's agreement count.
|
||
|
||
[`schema_order.py`](../../../tools/re-capture/schema_order.py) merges the
|
||
`Maneuver` field-name order from **all 113 unit tables** by topological sort
|
||
over their pairwise "k[i] precedes k[i+1]" constraints. The merge is acyclic and
|
||
**every one of the 113 tables is a subsequence of the merged 102-field order** —
|
||
so that order is the schema's.
|
||
|
||
Against it, the solved offsets fall into two exact runs:
|
||
|
||
| declaration indices | offset rule | anchors that fit |
|
||
|---|---|---|
|
||
| 0 … 20 (`MinimumVelocity` … `AA_Roll_Min`) | `0x09c + 4·i` | 21 / 21 |
|
||
| 21 … 32 (`SideThrustVelocity_Max` … `AccPitchFactor`) | `0x0a4 + 4·i` | 8 / 8 |
|
||
|
||
One 4-byte slot per field, with a **two-slot gap after `AA_Roll_Min`**
|
||
(`0x0f0`–`0x0f4`, purpose unknown). 29 independently-derived anchors, zero
|
||
conflicts, across a 0x9c–0x125 span.
|
||
|
||
### Fields NO disc record ever values — 🟡 PROBABLE
|
||
|
||
Five `Maneuver` fields are declared by the schema but left at their default by
|
||
*every* one of the 110 unit tables, so no amount of disc analysis can ever
|
||
reach them. The declaration-order rule pins them between confirmed anchors
|
||
(`YawDragFactor +0x104` … `ArterBurner_Vc +0x114`, and
|
||
`ReverseThrust_Vc +0x118` … `ReverseThrust_Acc +0x120`):
|
||
|
||
| offset | field | craft (`f00*`, `e0*`) | capital ships (`*1**`, `e2*`) | inert (`SchlosBase`, `Box`) |
|
||
|---|---|---:|---:|---:|
|
||
| `+0x108` | `PitchDragFactor` | 3 | 2 | 0 |
|
||
| `+0x10c` | `RollDragFactor` | 3 | 2 | 0 |
|
||
| `+0x110` | `DragFactorThreshold` | 0.5 | 0.5 | 0 |
|
||
| `+0x11c` | `ArterBurner_Acc` | 2 | 2 | 0 |
|
||
| `+0x128` | `DecPitchFactor` | 30 | 30 | 0 |
|
||
|
||
Corroboration beyond the interpolation, checked over all 18 units:
|
||
|
||
* `PitchDragFactor == RollDragFactor == YawDragFactor` holds **18/18** — and
|
||
`YawDragFactor` is *disc-supplied* (3.0 for craft, 2.0 for warships), so two
|
||
interpolated offsets reproduce a known number, per unit, every time.
|
||
* `DecPitchFactor == AccPitchFactor` holds **17/18**. The exception is
|
||
`UN_e015_ADAN_Puppy` (`AccPitchFactor` 1, `DecPitchFactor` 0.5) — which
|
||
defaults *both* on disc, so it is two independent fields that happen to be
|
||
set equal elsewhere, not a broken binding.
|
||
|
||
`UN_e015_ADAN_Puppy` also breaks the craft/warship bucketing above for
|
||
`DecPitchFactor` (0.5, not 30); the per-unit values are in the CSV.
|
||
|
||
The tail of the `Maneuver` block (the AI-behaviour fields — `SideRoll_*`,
|
||
`BarrelRoll_*`, `TurnAttack_*`, `HoldPosition_*`, `Slalom_*`, `Through_*`,
|
||
`SolidCutoff_*`, and the `AxisMode` / `AB_*` sub-block) is **NOT resolved**.
|
||
Those fields are declared by only a handful of tables and almost always with a
|
||
single distinct value, so the solver's bindings there are coincidences: it
|
||
placed `Slalom_CutoffRatio` at `+0x00c` and `TurnAttack_DoubleTimeMin` at
|
||
`+0x110`, both of which the declaration-order rule contradicts. They are marked
|
||
`tentative` in the CSV. **NEEDS-HUMAN / needs more coverage** — more stages
|
||
would give those fields distinct values and settle it.
|
||
|
||
## The player craft, `UN_f001_TCAF_DeltaSaber_T_Player`
|
||
|
||
30 of its fields are defaulted on disc. Notable recovered values:
|
||
|
||
| field | value | note |
|
||
|---|---|---|
|
||
| `Size_Radius` | 10 | ✅ |
|
||
| `FCSRange` | 500000 | ✅ — same as `RadarRange` |
|
||
| `ChargeSpeed` (shield) | 25 | ✅ |
|
||
| `ResistanceToOptics` | 1 | ✅ |
|
||
| `HQRatio` / `ShieldRatio` / `ThrusterRatio` | 1 / 1 / 1 | 🟡 |
|
||
| `ResistanceToShell` / `ResistanceToExplosion` | 1 / 1 | 🟡 |
|
||
| `DryMass` | 100 | 🟡 (`GrossMass` 250 is on disc) |
|
||
| `LowerHPThresholdRatio` | 0.3 | 🟡 |
|
||
| `ChargeDelay_Break` | 10 | 🟡 |
|
||
| `MassScore` | 0 | 🟡 |
|
||
|
||
Every AI-behaviour field the solver bound reads **0** for the player craft,
|
||
which is the expected shape (the player is not AI-driven) — but see the caveat
|
||
above: those offsets are not settled, so treat the zeros as consistent, not
|
||
proven.
|
||
|
||
The `…Ratio` family that the Route-B target list parked is **1.0 for almost
|
||
every unit**, with real exceptions that only the runtime shows:
|
||
`UN_e105_ADAN_Cruiser` `HQRatio` = 0.2, `UN_bf001_TCAF_SchlosBase` and
|
||
`UN_e106_ADAN_Destroyer` `ThrusterRatio` = 0.2,
|
||
`UN_n001_TTRL_Box` `ShieldRatio` = 0.3.
|
||
|
||
## Coverage and how to extend it
|
||
|
||
21 of 110 units. Unlike weapons — where one snapshot held all 126 — **unit
|
||
definitions are instantiated per stage**, so coverage is bounded by the stages
|
||
reachable from the save (slot 01 is at 5 %, Stage 02). `unit_runtime.py` unions
|
||
any number of snapshots and re-solves, and more units directly promote the 🟡
|
||
bindings to ✅ by adding distinct values — the six tutorials took the confirmed
|
||
set from 22 fields to 27.
|
||
|
||
The tutorials are nearly exhausted as a source: all six together contribute only
|
||
3 units the missions do not already have (`UN_f001_TCAF_DeltaSaber_T_Ttrl`,
|
||
`UN_e015_ADAN_Puppy_2`, `UN_f001_TCAF_DeltaSaber_T_Player_Ttrl2`) — they reuse
|
||
one training box, one target drone and the player craft. **Real coverage now
|
||
needs real missions**, i.e. story progress on the save.
|
||
|
||
`tools/re-capture/grab_tutorial.sh` captures one tutorial per invocation
|
||
(cold boot → menu → Nth entry → snapshot, ~2.5 min). It cold-boots for each
|
||
because backing out of a loaded mission via PAUSE → BACK TO MENU wedges the
|
||
emulator. Two timing facts it encodes, both learned the hard way: the main menu
|
||
is **not input-ready for ~10 s** after the title tap, and d-pad presses before
|
||
that are silently dropped — which sends the A to `NEW GAME` instead of
|
||
`TUTORIAL`. And `NEW GAME` is not a cheap way to reach Stage 01: it gates on a
|
||
DIFFICULTY menu and then plays the prologue movie.
|
||
|
||
A NEW GAME excursion as far as the READY ROOM leaves
|
||
`535107D4/00000001/game01/savedata` byte-identical — only the profile `.gpd`
|
||
achievement files change — so it does not endanger the 5 % save. Verified by
|
||
diff against a backup, not assumed.
|
||
|
||
**A stage's whole unit set is parsed at load, not as waves spawn** — checked by
|
||
counting the definition objects at three points in Stage 02: immediately after
|
||
take-off, ~12 minutes in mid-combat, and after GAME OVER. 14 objects, the same
|
||
14 IDs, every time. So capturing a stage costs one load and one snapshot; there
|
||
is no need to play it, and no need to survive it.
|
||
|
||
Stages captured so far: `Ttrl` (BASIC CONTROLS), Stage 02.
|