re: read the defaulted weapon stats out of live guest memory

Canary backs the whole guest address space with one shared-memory file, so the
game's RAM is readable from the host while it runs -- no debugger, no emulator
patch. That turns the parked Route-B question (fields the IDXD omits because
they sit at their default) from "unreachable" into a table lookup.

gmem.py maps guest VAs into /dev/shm/xenia_memory_* via Xenia's fixed table and
searches only the allocated extents, so a full-RAM scan is ~0.2 s.

weapon_runtime.py finds the parsed objects by scanning for their vtable, then
*solves* the struct layout instead of guessing it: every (field, offset,
encoding) triple is scored against the disc records, and a binding is accepted
only with zero contradictions -- and marked confirmed only when >=10 records
agree on >=3 distinct values, because a field whose samples are all one number
matches any offset holding that constant.

Result: Weapon (0xc0) and Shell (0x200) mapped, 4393 values the disc does not
carry, for all 126 weapons at 5 % save progress. Cross-checks hold -- the
Arsenal DATA SHEET read 4 for wep_05/wep_60 Max. Lock Ons and the memory read
agrees; the in-flight HUD's 06000/00300 match LoadingCount; all 252 objects are
byte-identical across a screen change, so this is definition data, not state.

Two findings fell out: `IsCharging = Yes` switches LoadingCount and
TriggerShotCount to float32 (it partitions the 8 float-encoded records exactly),
and two .tbl entries declare Shell_TCAF_Ship_AAGun with conflicting Power -- the
runtime keeps the one that omits it.

Where the defaulted values *come from* (the IDXD's undecoded binary node region
vs code defaults) is not settled here; they vary per weapon, so they are not one
constructor constant.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-29 05:29:34 +00:00
parent e3f3d55d29
commit 9daac6f1f0
7 changed files with 8053 additions and 1 deletions

View File

@@ -21,9 +21,16 @@ Promote to a prose `structures/…md` file when a format needs behavioural notes
| Fonts (ttf/otf/ttc) | ✅ | `sylpheed-formats/src/font.rs` | standard OpenType, parsed via ttf-parser |
| XBG7 mesh | 🟡/❔ | `sylpheed-formats/src/mesh.rs` + `tests/mesh_disc.rs` ([xbg7](structures/xbg7-mesh.md)) | weapons/props: declaration-driven variable stride (36 models), GPU-confirmed. **Stage containers: 5662 sub-models across 22 stages** via content-anchored grouped pools (`stage_models`). Quantized hero bodies (DeltaSaber `f004`) still declined |
| Capital-ship part placement | 🟡 | `sylpheed-formats/src/ship.rs` (static) + [runtime capture](ship-placement-runtime-capture.md) | hull placement static-exact; external parts approximate statically. **Runtime capture** (Canary F10 → VS-constant WorldView) gives ground truth — validated on `e106` destroyer; not yet baked into the viewer |
| Weapon fields defaulted on disc | ✅/🟡 | [runtime DATA SHEET](weapon-datasheet-runtime.md) | The Arsenal Gallery-Mode panel reads the live record: `Ammo Capacity` = `LoadingCount` ✅, `Max. Lock Ons` = `TriggerShotCount` ✅. Recovers `TriggerShotCount` for `wep_05`/`wep_60` (both **4**); brackets defaulted `Power`/`MaximumRange` via the letter classes. 9 of ~61 weapons reachable at 5 % save progress |
| Weapon fields defaulted on disc | ✅ | [runtime struct](structures/weapon-struct-runtime.md) · [DATA SHEET route](weapon-datasheet-runtime.md) | **Solved.** Canary maps guest RAM into `/dev/shm`, so the parsed `Weapon`/`Shell` objects are readable live; their layout is solved against disc ground truth (zero contradictions over 100+ records). All 126 weapons, exact numbers, no story progress needed — [4 393 values](captures/weapon-runtime-fields.csv) the disc does not carry. Supersedes the letter-bucket limit of the DATA SHEET route, which now serves as the independent cross-check |
| UI screen layout (`.rat`) | ✅/🟡 | [ui-rat-layout](structures/ui-rat-layout.md) | One pak per UI screen; each RATC = one (context × language) build; every `<name>.t32` sprite has a `<name>.rat` **layout record** (BE u32; 1280×720 design space; scale/tint/X/Y, keyframes for animated elements, `opt ` link to the focused state). **The tutorial PAUSE menu and the title main menu both rebuild pixel-accurately from the disc.** `loop1.rat` (screen-level draw order) not yet decoded |
## Runtime / dynamic-capture technique
| Technique | Conf. | Spec | Notes |
|-----------|-------|------|-------|
| Live guest-memory read | ✅ | [`tools/re-capture/gmem.py`](../../tools/re-capture/gmem.py) | Canary backs the guest address space with `/dev/shm/xenia_memory_*`; guest VAs map in through Xenia's fixed table. Full-RAM search ~0.2 s (sparse, `SEEK_DATA`). No debugger, no emulator patch, game keeps running |
| IDXD object layout solver | ✅ | [`tools/re-capture/weapon_runtime.py`](../../tools/re-capture/weapon_runtime.py) | Scan RAM for a class's vtable → enumerate its objects → brute-force `(field, offset, encoding)` against the disc records. Accepts a binding only on **zero** contradictions. Generalizes to any IDXD-backed definition |
## Functions / code paths
_None documented yet — populated during the dynamic-RE phase._

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,294 @@
# Runtime `Weapon` / `Shell` structs — read from live guest memory
**Confidence: ✅ CONFIRMED** for the fields marked ✅ below (each binding is
reproduced by 10125 independent disc records with **zero** contradictions);
🟡 for the thin ones. Captured 2026-07-29 from Xenia Canary running the retail
disc, save slot 01 (Stage 02, 5 % progress), READY ROOM and ARSENAL.
## Why this exists
`weapon\Weapon_*.tbl` is an [IDXD](../INDEX.md) record whose string pool **omits
every field left at its default**. That parked a long list of stats as
unreadable — the `…Ratio` / `…Count` family, `Power`, `MaximumRange`. The
[Arsenal DATA SHEET route](../weapon-datasheet-runtime.md) recovered a few of
them but only as letter buckets (Range/Damage as `A``E`), and only for the 9
weapons unlocked at 5 % progress.
This reads the values **directly out of the running game's memory** instead.
All 126 weapons, all fields, exact numbers, in one pass — and it needs no story
progress, because the definitions are parsed at load time whether or not the
player has unlocked the weapon.
## The lever: Canary maps guest RAM into `/dev/shm`
Xenia Canary backs the entire guest address space with one shared-memory file,
`/dev/shm/xenia_memory_<id>`. It is a plain file: **the guest's RAM is readable
from the host with `open`/`seek`/`read`, live, no debugger and no emulator
patch.** Guest VAs map into it through Xenia's fixed table (`memory.cc`), which
[`tools/re-capture/gmem.py`](../../../tools/re-capture/gmem.py) implements:
```bash
python3 tools/re-capture/gmem.py find "Weapon_DSaber_P_wep_01" # search all of RAM
python3 tools/re-capture/gmem.py words 0xbccce500 48 # dump as BE u32/f32
```
The file is sparse (~212 MB resident of 4.5 GB), and the scan uses
`SEEK_DATA`/`SEEK_HOLE`, so a full-RAM search costs ~0.2 s.
## Finding the objects
1. The title's **schema field-name pool** is in the XEX at `0x82086a30`, in
declaration order: `EnumWeapon`, type `Weapon` (`ID`, `Name`, `TargetType`,
`CartridgeModelName`), then type `Shell` (`ID`, `Name`, `MovementType`, …
`Explosion_MaxDamageRadius`). This is exactly the on-disc field order. No
pointer to these strings exists anywhere in RAM — PPC builds the addresses
with `lis`/`ori` immediate pairs — so the schema descriptor cannot be found
by pointer-chasing; the layout has to be solved instead (below).
2. Each parsed record becomes **two C++ objects**, a `Weapon` and its `Shell`,
each in its own contiguous array, each identified by its **vtable pointer**:
| class | vtable VA | stride | count |
|-------|-----------|--------|-------|
| `Weapon` | `0x820af548` | `0xc0` | 126 |
| `Shell` | `0x820af58c` | `0x200` | 126 |
The vtable VAs are static (XEX `.data`); the array base addresses are heap
and are **not** assumed — the tool locates every object by scanning RAM for
the vtable word.
3. Object `+0x04` points at a 0x40-byte **name record**; the ID string sits at
`+0x10` inside it. That is what keys each object back to its disc record.
`Weapon``Shell` pairing is by ID (`Weapon_X``Shell_X`) — the two arrays
are in **different orders**, so index-pairing would be wrong.
## Solving the layout (the part that makes this evidence, not guesswork)
[`tools/re-capture/weapon_runtime.py`](../../../tools/re-capture/weapon_runtime.py)
brute-forces every `(field, byte offset, encoding)` triple and scores it against
the disc: how many records does this offset *reproduce*, and how many does it
*contradict*? A binding is accepted only with **zero contradictions**, and is
marked ✅ only when ≥10 records agree on ≥3 distinct values.
That threshold matters: a field whose disc samples are all the same number
matches any offset holding that constant, so agreement count alone is not
evidence — the number of **distinct** values pinned down is. Two fields landing
on one offset is impossible in a real struct, so collisions are resolved to the
better-evidenced field and the loser is reported as unsolved. Bindings that
contradict more than 20 % of their samples are discarded outright rather than
reported on their agreeing subset.
Encodings found: `f32`, `u32`, `deg` (**angles are stored in radians**;
`SprayAngle`, `AngularVelocity` and `SplitCone` are degrees on disc), and `cnt`
— see the next section.
### `IsCharging` switches the counter encoding ✅
`LoadingCount` and `TriggerShotCount` at `+0x28` / `+0x44` are **int32 on 118
records and float32 on 8**. The 8 are exactly the records with
`IsCharging = Yes` — an exact partition, found by testing every disc field/value
pair against the float-encoded set. A charging weapon drains its magazine
continuously, so its counter needs a fraction.
Any reimplementation must branch on `IsCharging` when reading these two fields;
reading them as int unconditionally yields `1125515264` instead of `150`.
## Struct maps
See [`docs/re/captures/weapon-runtime-fields.csv`](../captures/weapon-runtime-fields.csv)
for the full machine-readable table — 7 182 rows, every confirmed field × every
one of the 126 records, tagged `disc` or `defaulted-on-disc`. **4 393 of those
values were not readable from the disc at all.**
### `Weapon` (`0xc0` bytes)
| offset | enc | field | agree | distinct | conf |
|--------|-----|-------|------:|---------:|------|
| `+0x010` | u32 | `ReticleType` | 62 | 13 | ✅ |
| `+0x01c` | u32 | `SpecialWeaponType` | 73 | 6 | ✅ |
| `+0x028` | cnt | `LoadingCount` | 121 | 33 | ✅ |
| `+0x02c` | f32 | `Interval` | 125 | 28 | ✅ |
| `+0x030` | f32 | `ReadyInterval` | 120 | 10 | ✅ |
| `+0x034` | f32 | `Heating` | 114 | 30 | ✅ |
| `+0x038` | f32 | `Cooling` | 106 | 21 | ✅ |
| `+0x03c` | f32 | `Mass` | 122 | 42 | ✅ |
| `+0x040` | f32 | `HitRatio` | 102 | 5 | ✅ |
| `+0x044` | cnt | `TriggerShotCount` | 114 | 17 | ✅ |
| `+0x048` | f32 | `TriggerShotInterval` | 99 | 11 | ✅ |
| `+0x050` | deg | `SprayAngle` | 96 | 17 | ✅ |
| `+0x06c` | f32 | `LockIntervalSingle` | 61 | 9 | ✅ |
| `+0x070` | f32 | `LockIntervalMulti` | 28 | 9 | ✅ |
| `+0x09c` | f32 | `MaximumCharging` | 3 | 3 | 🟡 |
Also identified structurally, not by the solver: `+0x00` vtable, `+0x04` name
record, `+0x0c` `TargetType` as a bitmask (`Vessel|Craft|Structure` = 7),
`+0x14`/`+0x18` name hashes, `+0x7c`/`+0x80` muzzle-flash FX name record + hash.
Unsolved (no consistent offset): `Cracker_ShotInterval`, `Cracker_SubShotCount`,
`MinimumCharging`, `MultiTargetCount`.
### `Shell` (`0x200` bytes)
| offset | enc | field | agree | distinct | conf |
|--------|-----|-------|------:|---------:|------|
| `+0x00c` | cnt | `DeleteFadeSpeed` | 39 | 2 | 🟡 |
| `+0x010` | cnt | `GuidanceType` | 9 | 4 | 🟡 |
| `+0x014` | cnt | `SpiralType` | 3 | 1 | 🟡 |
| `+0x020` | f32 | `Length` | 107 | 3 | ✅ |
| `+0x024` | f32 | `Volume` | 19 | 3 | ✅ |
| `+0x028` | f32 | `ShellMass` | 110 | 34 | ✅ |
| `+0x03c` | f32 | `HP` | 28 | 2 | 🟡 |
| `+0x040` | f32 | `LifeTime` | 94 | 43 | ✅ |
| `+0x044` | f32 | `FadeInTime` | 13 | 2 | 🟡 |
| `+0x048` | f32 | `FadeOutTime` | 7 | 1 | 🟡 |
| `+0x04c` | f32 | `Velocity` | 106 | 15 | ✅ |
| `+0x050` | f32 | `MinimumVelocity` | 11 | 2 | 🟡 |
| `+0x054` | f32 | `MaximumVelocity` | 29 | 7 | ✅ |
| `+0x058` | deg | `AngularVelocity` | 49 | 19 | ✅ |
| `+0x05c` | f32 | `Acceleration` | 9 | 4 | 🟡 |
| `+0x064` | f32 | `BeginGuidanceTimeAdjust` | 24 | 2 | 🟡 |
| `+0x068` | f32 | `EndGuidanceTime` | 19 | 4 | ✅ |
| `+0x070` | f32 | `Spiral_BeginTime` | 2 | 2 | 🟡 |
| `+0x074` | f32 | `Spiral_BeginTimeAdjust` | 25 | 2 | 🟡 |
| `+0x08c` | f32 | `MinimumRange` | 43 | 7 | ✅ |
| `+0x090` | f32 | `MaximumRange` | 117 | 21 | ✅ |
| `+0x0a0` | f32 | `Color_R` | 18 | 4 | ✅ |
| `+0x0a4` | f32 | `Color_G` | 121 | 3 | ✅ |
| `+0x0a8` | f32 | `Color_B` | 100 | 2 | 🟡 |
| `+0x0b4` | f32 | `Radius` | 89 | 10 | ✅ |
| `+0x0b8` | f32 | `Power` | 101 | 37 | ✅ |
| `+0x0bc` | f32 | `FailedDamageRatio` | 2 | 2 | 🟡 |
| `+0x0c0` | f32 | `PlayerLaserPower` | 11 | 6 | ✅ |
| `+0x0fc` | f32 | `ChaffResistRatio` | 24 | 1 | 🟡 |
| `+0x104` | f32 | `ChargingSizeRatio` | 3 | 1 | 🟡 |
| `+0x10c` | f32 | `SphereRadiusBegin` | 9 | 6 | 🟡 |
| `+0x110` | f32 | `SphereRadiusTurn` | 9 | 8 | 🟡 |
| `+0x114` | f32 | `SphereRadiusEnd` | 10 | 8 | ✅ |
| `+0x118` | f32 | `ExplosionTurnTime` | 3 | 2 | 🟡 |
| `+0x11c` | f32 | `ExplosionLifeTime` | 7 | 6 | 🟡 |
| `+0x120` | f32 | `ExplosionDamage_Maximum` | 4 | 4 | 🟡 |
| `+0x124` | f32 | `ExplosionDamage_OuterEdge` | 8 | 6 | 🟡 |
| `+0x128` | f32 | `Explosion_MaxDamageRadius` | 8 | 3 | 🟡 |
| `+0x130` | f32 | `SplitTime_Maximum` | 2 | 2 | 🟡 |
| `+0x134` | deg | `SplitCone` | 2 | 2 | 🟡 |
| `+0x148` | cnt | `NodeCount` | 39 | 4 | ✅ |
| `+0x164` | f32 | `Deceleration` | 9 | 2 | 🟡 |
Unsolved: `BeginGuidanceTime`, `EndGuidanceTimeAdjust`, `Spiral_EndTime`,
`SplitTime_Minimum`, `StartingVelocity`, `Straight1Type`, `st1_df_pitch`,
`pitch0`, and the `sp_qu_*` / `sp_sp_*` / `sp_zg_*` spiral-motion family. Those
are declared by too few records (or by none that the solver could separate) —
they need a state where the shells are actually in flight.
## The answer to the parked Route-B question
Player-weapon fields that are **defaulted on disc**, now read exactly (`·` = the
disc carries the value already):
| wep | LoadingCount | TriggerShotCount | MaximumRange | Power | Heating | Cooling | ReadyInterval | HitRatio | SprayAngle |
|---|---|---|---|---|---|---|---|---|---|
| 01 | · | **1** | · | · | · | · | · | · | · |
| 02 | · | · | · | **100** | · | · | · | **1** | · |
| 03 | · | · | · | · | · | · | · | · | **1** |
| 05 | · | **4** | · | · | · | · | · | · | · |
| 08 | · | · | · | · | · | · | · | **1** | · |
| 09 | · | · | · | · | **0.02** | · | · | · | **1** |
| 11 | **6** | · | · | · | · | · | · | · | · |
| 12 | · | · | · | · | **0** | · | · | **1** | · |
| 13 | · | · | · | **300** | · | · | · | · | · |
| 14 | · | · | · | · | · | · | · | · | **1** |
| 19 | · | · | · | · | · | **0.2** | · | · | **1** |
| 24 | · | **1** | · | · | · | · | · | · | **1** |
| 25 | · | · | **4000** | · | · | · | · | · | · |
| 26 | · | · | · | **500** | · | · | · | · | · |
| 27 | · | **1** | · | · | · | · | · | · | · |
| 28 | **5** | · | · | · | · | · | · | · | · |
| 29 | · | · | · | **100** | · | · | · | · | · |
| 30 | · | **4** | · | · | · | · | · | · | · |
| 36 | **5** | · | · | · | · | · | · | · | · |
| 37 | · | **1** | · | · | · | · | · | · | **1** |
| 38 | · | · | · | · | · | · | · | **1** | · |
| 39 | · | **1** | · | · | · | · | · | · | **1** |
| 40 | · | · | · | · | · | · | **0.1** | · | · |
| 48 | · | · | · | · | · | · | · | · | **1** |
| 50 | · | · | · | · | · | · | · | · | **1** |
| 52 | · | · | · | · | · | · | · | **1** | · |
| 53 | · | **1** | · | · | · | · | · | · | · |
| 54 | · | · | · | · | · | · | · | · | **1** |
| 55 | · | · | · | · | **0** | · | · | **1** | · |
| 56 | · | · | · | · | · | · | · | **1** | · |
| 57 | · | · | · | · | **0** | · | · | **1** | · |
| 58 | · | · | **10000** | · | · | · | · | · | **1** |
| 59 | · | · | · | · | · | · | · | **1** | **1** |
| 60 | · | **4** | · | **1000** | · | · | · | · | · |
| 62 | · | **1** | · | · | **0** | · | · | · | · |
| 66 | · | · | · | · | · | · | · | **1** | · |
| 67 | · | · | · | · | · | · | · | **1** | · |
| 68 | · | · | · | · | · | · | · | **1** | · |
| 69 | · | · | · | · | · | · | · | **1** | · |
| 70 | **0** | · | · | · | · | · | · | **1** | · |
| 71 | · | · | · | **10** | · | · | · | **1** | · |
| 81 | · | · | · | **300** | · | · | · | · | · |
| 82 | · | · | **10000** | · | · | · | · | · | **1** |
| 84 | · | · | · | · | · | · | · | · | **0.1** |
| 85 | · | **1** | · | · | **0** | · | · | · | · |
`HitRatio` is **1.0 for every one of the 24 records that default it** — the
whole `…Ratio` family that blocked Route B is simply "no penalty".
`wep_82`'s defaulted field is `PlayerLaserPower` = **12000** (its `Power`,
12000, is on disc).
## Cross-checks
- **Against the independent screenshot route.** The Arsenal DATA SHEET
([weapon-datasheet-runtime.md](../weapon-datasheet-runtime.md)) established
`Max. Lock Ons == TriggerShotCount`, and read **4** for `wep_05` and `wep_60`
off the panel. The memory read gives **4** for both — two unrelated methods,
same numbers.
- **Against the in-flight HUD.** `NOSE BM 06000` / `MAIN MPM 00300` matches
`wep_01` `LoadingCount = 6000` and `wep_02` `= 300` at `+0x28`.
- **Stability.** All 252 objects are **byte-identical** between the READY ROOM
and the ARSENAL, so these are load-time definition data, not transient state.
- **Self-consistency.** 121 records agree on `LoadingCount`'s offset across 33
distinct values with zero contradictions; `Power`, 101 records / 37 distinct
values. A wrong offset cannot do that.
## Incidental findings
- **A duplicate, conflicting disc record.** Two `.tbl` entries declare
`Shell_TCAF_Ship_AAGun`; one sets `Power = 60.0`, the other omits it. The
runtime object holds **5**, i.e. the *omitting* declaration won. Any
reimplementation loading both will silently pick one — this says which.
- **Not every disc record is instantiated.** 131 disc records produced 126
objects; the 5 with no runtime object are context variants that this save
never loads — `Weapon_TCAF_DeltaSaber_NoseGun_Ttrl` / `_Laser_Ttrl`
(tutorial), `_NoseGun_None`, `Weapon_TCAF_Ship_AAGun_EX5`,
`Weapon_ADAN_Attacker_S_GunTurret`. Running the tutorial should instantiate
the `_Ttrl` pair. No runtime object lacked a disc record.
- Defaulted `Power` values vary per weapon (1, 10, 100, 300, 500, 1000), so they
are **not** one constructor constant. Where they come from — the IDXD's
undecoded binary node/index region, or per-type code defaults — is **not
determined here** (`NEEDS-HUMAN` / follow-up). Either way the runtime values
above are ground truth, and they are now a decoding oracle for that region.
## Reproduce
```bash
export HOME=/sylph-home/re SDL_AUDIODRIVER=dummy
run-canary --audio --apu=sdl --log_mask=13 --logged_profile_slot_0_xuid=E0300000EFBEA3D4 &
tools/re-capture/skip_intro.sh # -> title
# LOAD GAME -> slot 01 -> YES -> READY ROOM (weapon tables load with the save)
cargo run --release -p sylpheed-formats --example idxd_tokens -- \
<disc>/dat/GP_MAIN_GAME_E.pak > /tmp/wep_tokens.txt
python3 tools/re-capture/weapon_runtime.py /tmp/wep_tokens.txt # report
python3 tools/re-capture/weapon_runtime.py /tmp/wep_tokens.txt --csv # full table
```
## What this unlocks
`gmem.py` + "scan for the vtable, solve the layout against disc ground truth" is
**not weapon-specific**. The same three steps apply to any IDXD-backed
definition whose fields the disc defaults — craft/`UNIT` stats, which the Hangar
UI exposes only as a `Gross Weight` class and which
[the menu route could not reach at all](../weapon-datasheet-runtime.md), are the
obvious next target.