feat(formats): land the verified unit-definition layout as data + API + test

The offset->field map read out of sub_82341A20 is now usable rather than just
documented: data/unit_definition_layout.txt carries the 159 fields with their
provenance and the two conventions (angles are degrees on disc and radians in the
object; a defaulted field keeps the accessor's 0.0 miss value), and
sylpheed_formats::unit_layout exposes fields()/field_at()/offset_of() so a memory
snapshot can be read by name.

tests/unit_layout_disc.rs replays the verification against the checked-in live
dump -- every mapped float of all 11 identified objects must equal its disc value,
angles compared in radians -- asserting 0 disagreements and >=400 agreements. It
needs no emulator. Full suite green: 11 binaries.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

crates/sylpheed-formats/data/unit_definition_layout.txt

@@ -0,0 +1,177 @@
+# Unit / vessel definition object layout — offset, field, kind.
+#
+# Read out of the title's own loader, not inferred: `sub_82341A20` builds every
+# key as `addi r4, r30, -N` with `r30 = 0x82088f94`, so the field NAME for each
+# store is a string in the executable image; pairing each key with the first
+# store after its accessor call gives the offset. The accessor for floats is
+# `sub_822FC5A8`, which returns 0.0 when the pool has no value for the key.
+#
+# Verified against live objects dumped from a running Stage 02: 406 field values
+# agree with the disc records and 0 disagree, over 11 objects spanning both the
+# UNIT and VESSEL schemas (see docs/re/live-unit-definitions.md).
+#
+# Two conventions matter for a reimplementation:
+#   * angle fields are DEGREES on disc and RADIANS in this object;
+#   * a field the disc record omits is left at the accessor's miss value (0.0 for
+#     floats) unless something later derives it — `Size_Y` takes `Size_X`.
+#
+# offset  kind  field
+24      str   Model
+28      str   CollisionModel
+48      f32   Size_X
+52      f32   Size_Y
+56      f32   Size_Z
+64      f32   Color_R
+68      f32   Color_G
+72      f32   Color_B
+80      f32   Size_Radius
+84      f32   HP
+88      f32   HQRatio
+92      f32   ShieldRatio
+96      f32   ThrusterRatio
+100     word  IsDestructible
+104     str   NamePlate
+112     word  MountedShieldGenerator
+116     f32   ResistanceToOptics
+120     f32   ResistanceToShell
+124     f32   ResistanceToExplosion
+128     f32   ResistanceToPlayer
+132     f32   ResistanceParalyze
+136     str   HUDMarkerID
+156     f32   MinimumVelocity
+160     f32   MaximumVelocity
+164     f32   CruisingVelocity
+168     f32   Acceleration
+172     f32   Deceleration
+176     f32   AV_PitchPlus_Max
+180     f32   AV_PitchPlus_Min
+184     f32   AA_PitchPlus_Max
+188     f32   AA_PitchPlus_Min
+192     f32   AV_PitchMinus_Max
+196     f32   AV_PitchMinus_Min
+200     f32   AA_PitchMinus_Max
+204     f32   AA_PitchMinus_Min
+208     f32   AV_Yaw_Max
+212     f32   AV_Yaw_Min
+216     f32   AA_Yaw_Max
+220     f32   AA_Yaw_Min
+224     f32   AV_Roll_Max
+228     f32   AV_Roll_Min
+232     f32   AA_Roll_Max
+236     f32   AA_Roll_Min
+248     f32   SideThrustVelocity_Max
+252     f32   SideThrustAcceleration
+256     f32   MaximumBank_Normal
+260     f32   YawDragFactor
+264     f32   PitchDragFactor
+268     f32   RollDragFactor
+272     f32   DragFactorThreshold
+276     f32   ArterBurner_Vc
+280     f32   ReverseThrust_Vc
+284     f32   ArterBurner_Acc
+288     f32   ReverseThrust_Acc
+292     f32   AccPitchFactor
+296     f32   DecPitchFactor
+300     f32   AV_AxisMode_Max
+304     f32   AV_AxisMode_Min
+308     f32   AA_AxisMode_Max
+312     f32   AA_AxisMode_Min
+316     f32   PowerCutConsumeShield
+320     f32   PowerCutDeceleration
+324     f32   AB_ConsumeShield_Begin
+328     f32   AB_ConsumeShield
+332     f32   AB_AV_PitchPlus
+336     f32   AB_AA_PitchPlus
+340     f32   AB_AV_PitchMinus
+344     f32   AB_AA_PitchMinus
+348     f32   AB_AV_Yaw
+352     f32   AB_AA_Yaw
+356     f32   AB_AV_Roll
+360     f32   AB_AA_Roll
+364     word  SideRoll
+368     f32   SideRoll_Time
+372     f32   SideRoll_Length
+376     word  BarrelRoll
+380     f32   BarrelRoll_CountMinimum
+384     f32   BarrelRoll_CountMaximum
+388     f32   BarrelRoll_Time
+392     f32   BarrelRoll_Radius
+396     word  TurnAttack
+400     f32   TurnAttack_CutoffRatio
+404     f32   TurnAttack_DoubleRatio
+408     f32   CutoffTimeMin
+412     f32   CutoffTimeMax
+416     f32   TurnAttack_DoubleTimeMin
+420     f32   TurnAttack_DoubleTimeMax
+424     word  TurnAway
+428     f32   Turn_AngularVelocity
+432     f32   TurnAway_Time_Minimum
+436     f32   TurnAway_Time_Maximum
+440     word  BoostAway
+444     f32   BoostAway_Time_Minimum
+448     f32   BoostAway_Time_Maximum
+452     word  HoldPosition
+456     f32   HoldPosition_LengthMin
+460     f32   HoldPosition_LengthMax
+464     f32   HoldPosition_MinimumTime
+468     f32   HoldPosition_MaximumTime
+472     f32   HoldPosition_SideRatio
+476     f32   HoldPosition_BackRatio
+480     f32   HoldPosition_CutoffRatio
+484     f32   HoldPosition_CancelTime
+488     word  Slalom
+492     f32   Slalom_CutoffRatio
+496     f32   Slalom_TurnCount_Min
+500     f32   Slalom_TurnCount_Max
+504     word  Through
+508     f32   Through_CutoffRatio
+512     f32   Through_AngleMinimum
+516     f32   Through_AngleMaximum
+520     f32   Through_Time1Max
+524     f32   Through_Time1Min
+528     f32   Through_Time2Max
+532     f32   Through_Time2Min
+536     f32   Through_LengthMinimum
+540     f32   Through_LengthMaximum
+544     word  SolidCutoff
+548     f32   SolidCutoff_Ratio
+552     f32   SolidCutoff_LengthMin
+556     f32   SolidCutoff_LengthMax
+560     f32   HomingResistAdjustment
+564     f32   UsingChaffRatio
+568     f32   MaxValue
+572     f32   ChargeDelay
+576     f32   ChargeDelay_Break
+580     f32   ChargeSpeed
+584     f32   Delay
+588     f32   DelayAdjustment
+596     str   LowerHPFxModel
+604     str   ExplosionFxModel
+612     str   DestroyMotionName
+620     str   ExplosionMotionName
+624     f32   DestroyMotionTime
+628     f32   DryMass
+632     f32   GrossMass
+636     word  Thruster
+640     word  SideThruster
+644     word  ExplosionSE
+648     word  JumpIn
+652     word  JumpOut
+656     word  ShipEnvironmentSE
+660     word  LowerHPSE
+664     f32   LowerHPThresholdRatio
+668     f32   SELength
+672     f32   RadarRange
+676     f32   FCSRange
+680     f32   FiringRange
+684     word  MountedFCS
+692     f32   AttackVesselPoint
+696     f32   AttackCraftPoint
+700     f32   DefencePoint
+736     str   ShieldRecoverEffectName
+740     str   ShieldHitEffectName
+744     str   JumpIn
+748     str   JumpOut
+752     str   FadeIn
+756     str   FadeOut
+760     str   Effect_Paralyze

crates/sylpheed-formats/src/lib.rs

@@ -68,6 +68,10 @@ pub mod localization;
 // Whole-ship assembly from XBG7 part families (capital ships as split parts).
 pub mod ship;
 
+/// The runtime layout of a unit / vessel definition object, read out of the
+/// title's loader and verified against a live mission (see the module docs).
+pub mod unit_layout;
+
 // Exact capital-ship placement from a runtime F10 ship-capture (ground truth).
 pub mod ship_capture;
 

crates/sylpheed-formats/src/unit_layout.rs

@@ -0,0 +1,105 @@
+//! The runtime layout of a unit / vessel definition object.
+//!
+//! The game parses an [`crate::idxd`] record into a fixed 880-byte object whose
+//! field offsets are **not** guessable from the disc data: the record is a
+//! reflective key/value pool, and the loader assigns each key to a member by
+//! name. This table is that assignment, read out of the loader itself
+//! (`sub_82341A20` — every key is built as `addi r4, r30, -N`, so the field name
+//! for each store is a string in the executable image), and verified against
+//! objects dumped from a running mission: **406 values agree with the disc
+//! records, 0 disagree**, over 11 objects covering both schemas.
+//!
+//! Why a reimplementation wants it:
+//!
+//! - it names the field behind every word of a live definition object, so a
+//!   memory snapshot can be read directly;
+//! - it says which fields a record leaves **defaulted**, and what the loader
+//!   leaves there — the float accessor returns `0.0` on a pool miss;
+//! - it carries two conventions that are invisible on disc: **angles are degrees
+//!   in the data and radians in the object**, and **`Size_Y` takes `Size_X`**
+//!   when omitted.
+//!
+//! See `docs/re/live-unit-definitions.md` for the derivation and the measured
+//! default values.
+
+/// How a field is stored in the definition object.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum Kind {
+    /// IEEE-754 single, big-endian.
+    F32,
+    /// Pointer to a string.
+    Str,
+    /// 32-bit word (bool / enum / count / id).
+    Word,
+}
+
+/// One field of the definition object.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub struct Field {
+    /// Byte offset from the start of the object.
+    pub offset: usize,
+    /// Field name, exactly as the disc record spells it.
+    pub name: &'static str,
+    /// Storage kind.
+    pub kind: Kind,
+}
+
+const TABLE: &str = include_str!("../data/unit_definition_layout.txt");
+
+/// Every mapped field, in offset order.
+pub fn fields() -> Vec<Field> {
+    TABLE
+        .lines()
+        .filter(|l| !l.trim_start().starts_with('#') && !l.trim().is_empty())
+        .filter_map(|l| {
+            let mut it = l.split_whitespace();
+            let offset = it.next()?.parse().ok()?;
+            let kind = match it.next()? {
+                "f32" => Kind::F32,
+                "str" => Kind::Str,
+                _ => Kind::Word,
+            };
+            // `name` is a &'static str because TABLE is 'static.
+            let name = it.next()?;
+            let name: &'static str = TABLE.get(
+                TABLE.find(name).map(|s| s..s + name.len())?,
+            )?;
+            Some(Field { offset, name, kind })
+        })
+        .collect()
+}
+
+/// The field at `offset`, if one is mapped there.
+pub fn field_at(offset: usize) -> Option<Field> {
+    fields().into_iter().find(|f| f.offset == offset)
+}
+
+/// The offset of `name`, if it is mapped.
+pub fn offset_of(name: &str) -> Option<usize> {
+    fields().into_iter().find(|f| f.name == name).map(|f| f.offset)
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn table_parses_and_is_ordered() {
+        let f = fields();
+        assert!(f.len() > 150, "expected the full map, got {}", f.len());
+        assert!(f.windows(2).all(|w| w[0].offset < w[1].offset), "offsets must be strictly increasing");
+    }
+
+    #[test]
+    fn known_fields_sit_where_the_loader_puts_them() {
+        // Spot-checks from the verified map; these four also anchor the
+        // identification of a live object (see docs/re/live-unit-definitions.md).
+        assert_eq!(offset_of("Size_X"), Some(48));
+        assert_eq!(offset_of("Size_Y"), Some(52));
+        assert_eq!(offset_of("Size_Z"), Some(56));
+        assert_eq!(offset_of("HP"), Some(84));
+        assert_eq!(offset_of("HQRatio"), Some(88));
+        assert_eq!(field_at(96).map(|f| f.name), Some("ThrusterRatio"));
+        assert_eq!(field_at(48).map(|f| f.kind), Some(Kind::F32));
+    }
+}

crates/sylpheed-formats/tests/unit_layout_disc.rs

@@ -0,0 +1,96 @@
+//! The definition-object layout must keep agreeing with the disc records.
+//!
+//! `unit_layout` was read out of the game's loader and checked against objects
+//! dumped from a running Stage 02. This pins that check: every mapped float
+//! field of every identified object must equal the value its disc record sets.
+//! It needs no emulator — the dump is checked in under `docs/re/captures/`.
+
+use std::collections::BTreeMap;
+
+use sylpheed_formats::idxd::IdxdObject;
+use sylpheed_formats::pak::PakArchive;
+use sylpheed_formats::unit_layout::{fields, Kind};
+
+/// Live objects identified in the dump, with the disc record each one is.
+/// `bf001` is here because full-record agreement is what identified it: the
+/// four-value signature also fitted `UN_be005_ADAN_SpaceFortress`, which
+/// disagrees on `Color_R`/`Color_G`.
+const IDENTIFIED: &[(&str, &str)] = &[
+    ("UN_f106_TCAF_Destroyer", "0xbd3ee300"),
+    ("UN_f105_TCAF_Cruiser", "0xbd40e200"),
+    ("UN_e105_ADAN_Cruiser", "0xbd3fd800"),
+    ("UN_e106_ADAN_Destroyer", "0xbd3e6f80"),
+    ("UN_f101_TCAF_Acropolis", "0xbd3e1f00"),
+    ("UN_bf001_TCAF_SchlosBase", "0xbd3b6a00"),
+    ("UN_e007_ADAN_Turret", "0xbd3dea80"),
+    ("UN_e010_ADAN_Attacker_S", "0xbd3faa80"),
+    ("UN_f003_TCAF_ArrowHead", "0xbd3dbd00"),
+    ("UN_f001_TCAF_DeltaSaber_T", "0xbd3dfc00"),
+    ("UN_f001_TCAF_DeltaSaber_T_Player", "0xbd3da100"),
+];
+
+const DUMP: &str = include_str!("../../../docs/re/captures/stage02-live-unit-definitions-deep.txt");
+
+fn disc_root() -> Option<String> {
+    if let Ok(p) = std::env::var("SYLPHEED_DISC") {
+        if std::path::Path::new(&p).join("dat").is_dir() {
+            return Some(p);
+        }
+    }
+    let d = "/home/fabi/RE - Project Sylpheed/Project Sylpheed - Arc of Deception (USA, Europe) (En,Ja)";
+    std::path::Path::new(d).join("dat").is_dir().then(|| d.to_string())
+}
+
+/// `va -> offset -> value`, from the dump's `addr +off hex u32 f32` columns.
+fn live() -> BTreeMap<String, BTreeMap<usize, f32>> {
+    let mut out: BTreeMap<String, BTreeMap<usize, f32>> = BTreeMap::new();
+    let mut cur = String::new();
+    for line in DUMP.lines() {
+        if let Some(rest) = line.strip_prefix("=== ") {
+            cur = rest.trim().to_string();
+            continue;
+        }
+        let f: Vec<&str> = line.split_whitespace().collect();
+        if f.len() >= 5 && f[1].starts_with('+') {
+            if let (Ok(off), Ok(val)) = (usize::from_str_radix(&f[1][1..], 16), f[4].parse::<f32>())
+            {
+                out.entry(cur.clone()).or_default().insert(off, val);
+            }
+        }
+    }
+    out
+}
+
+#[test]
+fn mapped_fields_match_the_disc_records() {
+    let Some(disc) = disc_root() else {
+        eprintln!("SKIP: extracted disc not found (set SYLPHEED_DISC to enable)");
+        return;
+    };
+    let pak = PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).expect("main pak");
+    let live = live();
+    let floats: Vec<_> = fields().into_iter().filter(|f| f.kind == Kind::F32).collect();
+
+    let (mut agree, mut bad) = (0usize, Vec::new());
+    for entry in pak.entries() {
+        let Ok(bytes) = pak.read(entry) else { continue };
+        let Ok(obj) = IdxdObject::parse(&bytes) else { continue };
+        let Some(id) = obj.get_raw("ID") else { continue };
+        let Some((_, va)) = IDENTIFIED.iter().find(|(i, _)| *i == id) else { continue };
+        let Some(words) = live.get(*va) else { continue };
+        for f in &floats {
+            let (Some(want), Some(got)) = (obj.get_f32(f.name), words.get(&f.offset)) else {
+                continue;
+            };
+            // Angle fields are degrees on disc and radians in the object.
+            let rad = want * std::f32::consts::PI / 180.0;
+            if (want - got).abs() <= want.abs() * 1e-4 || (rad - got).abs() <= rad.abs() * 1e-4 {
+                agree += 1;
+            } else {
+                bad.push(format!("{id}.{} disc {want} vs live +{} = {got}", f.name, f.offset));
+            }
+        }
+    }
+    assert!(bad.is_empty(), "{} fields disagree: {bad:?}", bad.len());
+    assert!(agree >= 400, "expected ≥400 agreeing fields, got {agree}");
+}

docs/re/live-unit-definitions.md

@@ -456,3 +456,19 @@ runtime. `FiringRange` (0 in all 11) and `ShieldRatio` (1 in all 6 that default
 it) look like genuine constants; `HQRatio` (0.2 or 1 over 9 units),
 `ThrusterRatio` (0.2 or 1 over 10) and `AttackCraftPoint` (0, 0.1, 0.5 or 1 over
 9) are clearly per-unit and must not be recorded as one value.
+
+## Landed in the crate
+
+The map is no longer only an analysis artefact:
+
+- **`data/unit_definition_layout.txt`** — the 159 fields (offset, kind, name),
+  with the provenance and the two conventions (degrees→radians; a defaulted field
+  keeps the accessor's miss value) in its header.
+- **`sylpheed_formats::unit_layout`** — `fields()`, `field_at(offset)`,
+  `offset_of(name)`, so a memory snapshot can be read by name.
+- **`tests/unit_layout_disc.rs`** — replays the whole check against the
+  checked-in dump: every mapped float of all 11 identified objects must equal its
+  disc value (angles compared in radians). Asserts 0 disagreements and ≥400
+  agreements, and needs **no emulator** — the capture is in the repo.
+
+Full suite green: 11 test binaries, workspace builds.