9 Commits

Author SHA1 Message Date
6a41525c41 re: generalize the two capture tools beyond weapons
idxd_tokens took the sub-record type list as an argument already but still
hard-filtered entries to those declaring a `Weapon_*` id, so it could not be
pointed at another schema. Filter on the requested type names instead -- it now
splits `EnumUnit` (Generic/Maneuver/Shield/Explosion/Frame/Turret/...) too.

gmem: honour $GMEM_FILE. `cp --sparse=always /dev/shm/xenia_memory_* snap.bin`
takes ~2 s and reads identically, which matters because a running Canary pegs
every core under lavapipe and makes repeated live reads stall unpredictably.

Groundwork for the craft (UNIT) stats. Not a finding yet: in menus only the
player craft's *name* is resident -- the flight-model object is not instantiated
until a mission loads, so that needs an in-mission snapshot.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 06:05:13 +00:00
9daac6f1f0 re: read the defaulted weapon stats out of live guest memory
Canary backs the whole guest address space with one shared-memory file, so the
game's RAM is readable from the host while it runs -- no debugger, no emulator
patch. That turns the parked Route-B question (fields the IDXD omits because
they sit at their default) from "unreachable" into a table lookup.

gmem.py maps guest VAs into /dev/shm/xenia_memory_* via Xenia's fixed table and
searches only the allocated extents, so a full-RAM scan is ~0.2 s.

weapon_runtime.py finds the parsed objects by scanning for their vtable, then
*solves* the struct layout instead of guessing it: every (field, offset,
encoding) triple is scored against the disc records, and a binding is accepted
only with zero contradictions -- and marked confirmed only when >=10 records
agree on >=3 distinct values, because a field whose samples are all one number
matches any offset holding that constant.

Result: Weapon (0xc0) and Shell (0x200) mapped, 4393 values the disc does not
carry, for all 126 weapons at 5 % save progress. Cross-checks hold -- the
Arsenal DATA SHEET read 4 for wep_05/wep_60 Max. Lock Ons and the memory read
agrees; the in-flight HUD's 06000/00300 match LoadingCount; all 252 objects are
byte-identical across a screen change, so this is definition data, not state.

Two findings fell out: `IsCharging = Yes` switches LoadingCount and
TriggerShotCount to float32 (it partitions the 8 float-encoded records exactly),
and two .tbl entries declare Shell_TCAF_Ship_AAGun with conflicting Power -- the
runtime keeps the one that omits it.

Where the defaulted values *come from* (the IDXD's undecoded binary node region
vs code defaults) is not settled here; they vary per weapon, so they are not one
constructor constant.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 05:29:34 +00:00
e3f3d55d29 re: ui layout generalizes — the title main menu rebuilds too
Second screen, independent of the pause work: GP_TITLE.pak's ptbtn01..05.rat give
X=542 for all five items and Y=162/242/322/402/482, an 80px pitch where the pause
menu used 70. Measured against the main-menu screenshot the sprite tops sit at a
constant +46px for all five, which is exactly the 45px of Xenia window chrome plus
one — so the record's Y is the sprite's top edge in the guest framebuffer, to the
pixel, on a screen the format was not derived from.

GP_TITLE also splits by sub-screen the way the pause pak splits by context:
ptbtn00 (PRESS A BUTTON), ptbtn01..05 (main menu), ptbtn11..13 (EXTRAS submenu),
pgloading_* (loading screen).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 19:44:29 +00:00
9a86a09f8d re: ui — the RATC bundle header is the screen's element declaration table
u32 count at 0x14, then fixed 60-byte entries of [name | 4 flag words | pivotX |
pivotY | 0]. The table lists both .t32 sprites and .rat records, so it is the
screen's element list.

Verified: for all 7 .t32 entries in the tutorial pause bundle the declared pivot
is exactly half the decoded texture's dimensions, 7/7 with no mismatch. That also
settles what the pair at 0x50/0x54 of a .rat record is — a .rat simply opens with
one of these declarations.

The language split is visible inside one bundle: the English build's .t32
declarations carry correct English pivots while its .rat declarations carry
Japanese-derived ones, so the sprite table is regenerated per language and the
layout layer is inherited from the Japanese master.

Adds tools/re-capture/ratc_decls.py (parse the table, check pivots against the
decoded textures).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 19:40:57 +00:00
aaa4ea60b3 re: ui .rat — pivot field, language-independence, PRMD, and a tightened claim
Follow-up decoding on the layout record:

- 0x50/0x54 is the sprite PIVOT: exactly half the texture size on 4 of 5 plain
  sprites. It is a rotation/scale centre, not a draw offset — X/Y remains the
  top-left, which is what actually reproduces the screenshot.
- The records are byte-identical across the English and Japanese bundles: layout
  is authored once and only the sprites are swapped. That explains pgpbtn01's
  pivot implying a 226px texture that neither shipped language has.
- loop1.rat is NOT the screen composition — it is a looping sprite animation
  (3-name table + ~30 keyframes at one position). The eff*/deli*/msg draw list is
  therefore still unfound, and that gap is now stated plainly.
- The 380-byte top-level entries are PRMD primitives: a colour plus four explicit
  corner coordinates for a full-screen quad — the pause dim overlay. The format
  is tag-driven ('opt ', 'PRMD', 'end '), not a fixed struct.

Also demotes one piece of evidence. The screenshot is of the TUTORIAL pause menu,
so only pgp_ttrl_* can be checked against it; the in-mission records mapping onto
the same screenshot under a constant offset works only because both builds share
the 70px pitch and proves nothing. In-mission coordinates are now marked
unverified.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 19:38:04 +00:00
621b9dd6e9 re: crack the UI layout record — the retail UI rebuilds from the disc
Each UI sprite <name>.t32 ships with a <name>.rat companion, and that companion
is the layout record: big-endian u32, a 1280x720 design space at 0x18, and a
placement block of scaleX/scaleY/tint/X/Y. Animated elements repeat the block
with a trailing time field (keyframes); an 'opt ' tag carries a length-prefixed
link to the focused-state record.

Verified in that order, records first: across the four pause buttons exactly one
field varies, stepping 268/338/408/478 at a constant 70px pitch with X fixed at
226; the three items visible in a screenshot of the running game measure
346.0/415.5/485.5, mapping onto those numbers under one constant offset to
within 0.5px. The tutorial build's records are outright framebuffer coordinates,
and compositing its sprites there reproduces the screenshot pixel-accurately.

Also records two mistakes the A/B caught, since a static-only reading would have
shipped both: texture width does not identify a language (Japanese fits English
widths), and the in-mission and tutorial pause menus are separate sprite sets
rather than one set re-packed into slots.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 19:05:21 +00:00
07eff85819 re: in-flight HUD — independent confirmation of the ammo mapping
The tutorials need no save and are ~1 min from a cold boot, so they are the
cheapest route into a live flight scene. The HUD prints the equipped weapons as
NOSE BM 06000 / MAIN MPM 00300 — matching wep_01's LoadingCount 6000 and wep_02's
300, with the weapons identified from the HANGAR loadout rather than from the
ammo number. That makes it a genuinely independent confirmation of
Ammo Capacity == LoadingCount, from a different renderer in a different mode.

Also inventories the HUD elements (heat gauges, shield/armor pools, target armor
gauge, per-weapon RANGE markers) and notes that the RANGE gauge draws each
weapon's MaximumRange on a scale — a route to a real number where the Arsenal
panel only gives a letter class.

Adds autopilot.py (waypoint-arrow chaser). It detects reliably but oscillates;
recorded as such rather than as a working tool.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 18:50:46 +00:00
c3b5f3f401 re: state the real (non-circular) evidence for Ammo Capacity == LoadingCount
The 'N/N exact matches' framing was circular: each UI entry was identified BY its
ammo value. Replaced with what actually carries the claim — tab-uniqueness of the
forced match, the independent Max. Lock Ons agreement on two of them, and one
entry (STILETTO BG I) identified from the HANGAR loadout rather than from ammo.
Also downgrades LIGHT MACHINE GUN MG I to PROBABLE: three guns share ammo 3000.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 18:09:03 +00:00
ba33c533da re: runtime DATA SHEET reads the live weapon record (Route B, first capture)
Drove the retail game headless (Canary + lavapipe, vgamepad) to the Ready Room
Arsenal and read the Gallery-Mode DATA SHEET for every weapon the 5%-progress
save reveals.

Two UI->field mappings are CONFIRMED against weapons whose values ARE on disc:
  Ammo Capacity  == LoadingCount      (9/9 exact matches)
  Max. Lock Ons  == TriggerShotCount  (FALCON 9AM 12 == wep_02; BUZZARD 22 == wep_04)

That makes the panel an oracle for fields the disc leaves defaulted, and it is
shown for undeveloped weapons too, so no points need spending. Recovered
TriggerShotCount for wep_05 and wep_60 (both 4, on-disc absent), and bracketed
the defaulted Power / MaximumRange via the Range/Damage letter classes.

Ruled out first: the defaults are not in hidden/DefTables.pak (no WEAPON-schema
objects there) nor in the localized GP_MAIN_GAME_* duplicates.

Also adds the two analysis examples that produce the shopping list of defaulted
fields, the screenshot harness used to drive the menus, and the evidence PNGs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 18:01:25 +00:00
46 changed files with 9005 additions and 452 deletions

1
.gitignore vendored
View File

@@ -20,3 +20,4 @@ Thumbs.db
# Trunk build output
dist/
__pycache__/

View File

@@ -0,0 +1,50 @@
//! Scratch analysis: for one IDXD key, list every object that DECLARES it and
//! whether it carries a value on disc or is left at the title-code default.
//!
//! Run: cargo run -p sylpheed-formats --example default_owners -- <KEY> [KEY...]
use sylpheed_formats::idxd::IdxdObject;
use sylpheed_formats::pak::PakArchive;
fn is_number(s: &str) -> bool {
let s = s.strip_suffix(['f', 'F']).unwrap_or(s);
let t = s.strip_prefix(['-', '+']).unwrap_or(s);
!t.is_empty() && t.chars().all(|c| c.is_ascii_digit() || c == '.')
}
fn is_key(s: &str) -> bool {
!s.is_empty()
&& s.chars().all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '-')
&& s.chars().next().is_some_and(|c| c.is_ascii_alphabetic() || c == '_')
&& !is_number(s)
}
fn is_value(s: &str) -> bool {
is_number(s) || !is_key(s)
}
fn main() {
let root = std::env::var("SYLPHEED_DISC")
.unwrap_or_else(|_| "/home/fabi/RE - Project Sylpheed/sylph_extract".into());
let wanted: Vec<String> = std::env::args().skip(1).collect();
let arc = PakArchive::open(std::path::Path::new(&root).join("dat/GP_MAIN_GAME_E.pak")).unwrap();
for e in arc.entries() {
let Ok(bytes) = arc.read(e) else { continue };
let Ok(obj) = IdxdObject::parse(&bytes) else { continue };
let toks = obj.tokens().to_vec();
let mut hits: Vec<String> = vec![];
for (i, t) in toks.iter().enumerate() {
if !wanted.iter().any(|w| w == t) {
continue;
}
let valued = i > 0 && is_value(&toks[i - 1]);
hits.push(if valued {
format!("{t}={}", toks[i - 1])
} else {
format!("{t}=<DEFAULT>")
});
}
if !hits.is_empty() {
println!("0x{:08x} {:<44} {}", obj.schema_hash, obj.identity(), hits.join(" "));
}
}
}

View File

@@ -0,0 +1,135 @@
//! Scratch analysis: which IDXD fields are DECLARED but left at their default on
//! disc, per schema. Those defaults live in title code, so they can only be read
//! from the running game — this prints the shopping list for that dynamic capture.
//!
//! Run: cargo run -p sylpheed-formats --example defaulted_fields -- <disc-root>
use std::collections::{BTreeMap, BTreeSet};
use sylpheed_formats::idxd::IdxdObject;
use sylpheed_formats::pak::PakArchive;
fn is_number(s: &str) -> bool {
let s = s.strip_suffix(['f', 'F']).unwrap_or(s);
let t = s.strip_prefix(['-', '+']).unwrap_or(s);
!t.is_empty() && t.chars().all(|c| c.is_ascii_digit() || c == '.')
}
/// Same shape-test the parser uses: an identifier-looking token that is not a
/// value literal is a field-name key.
fn is_key(s: &str) -> bool {
!s.is_empty()
&& s.chars()
.all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '-')
&& s.chars().next().is_some_and(|c| c.is_ascii_alphabetic() || c == '_')
&& !is_number(s)
}
fn is_value(s: &str) -> bool {
is_number(s) || !is_key(s)
}
fn main() {
let root = std::env::args()
.nth(1)
.unwrap_or_else(|| "/home/fabi/RE - Project Sylpheed/sylph_extract".into());
let paks: Vec<String> = std::env::args().skip(2).collect();
let paks = if paks.is_empty() {
vec![
"dat/GP_MAIN_GAME_E.pak".to_string(),
"dat/GP_HANGAR_ARSENAL.pak".to_string(),
]
} else {
paks
};
for pak_rel in &paks {
let path = std::path::Path::new(&root).join(pak_rel);
let Ok(arc) = PakArchive::open(&path) else {
eprintln!("-- skip {pak_rel} (open failed)");
continue;
};
println!("\n================ {pak_rel} ================");
// schema -> key -> (n_set, n_defaulted, distinct values, example owners)
type Stat = (usize, usize, BTreeSet<String>, Vec<String>);
let mut per_schema: BTreeMap<u32, (usize, BTreeMap<String, Stat>)> = BTreeMap::new();
for e in arc.entries() {
let Ok(bytes) = arc.read(e) else { continue };
let Ok(obj) = IdxdObject::parse(&bytes) else {
continue;
};
let ident = obj.identity();
let toks = obj.tokens().to_vec();
let entry = per_schema.entry(obj.schema_hash).or_default();
entry.0 += 1;
// Track which keys this object declares, and whether each is valued.
let mut seen_here: BTreeMap<String, Option<String>> = BTreeMap::new();
for (i, t) in toks.iter().enumerate() {
if !is_key(t) {
continue;
}
let prev = if i == 0 { None } else { Some(&toks[i - 1]) };
let valued = prev.map(|p| is_value(p)).unwrap_or(false);
let v = if valued { Some(toks[i - 1].clone()) } else { None };
seen_here.entry(t.clone()).or_insert(v);
}
for (k, v) in seen_here {
let s = entry.1.entry(k).or_default();
match v {
Some(val) => {
s.0 += 1;
if s.2.len() < 12 {
s.2.insert(val);
}
}
None => {
s.1 += 1;
if s.3.len() < 6 {
s.3.push(ident.clone());
}
}
}
}
}
for (schema, (n_obj, keys)) in per_schema {
if n_obj < 2 {
continue;
}
let name = match schema {
0x0426_e81d => "PLAYER",
0x6ab4_825a => "WEAPON",
0x43fa_a517 => "UNIT",
0x3c5b_0549 => "VESSEL",
0xbd86_d41c => "CHARACTER",
0x3c9a_e32e => "STAGE",
0xb412_e6d8 => "MESSAGE",
_ => "?",
};
let defaulted: Vec<_> = keys
.iter()
.filter(|(_, s)| s.1 > 0)
.collect();
if defaulted.is_empty() {
continue;
}
println!("\n--- schema 0x{schema:08x} {name} ({n_obj} objects) ---");
println!(
"{:<30} {:>5} {:>5} {}",
"KEY", "set", "dflt", "values seen (≤12) | owners defaulting"
);
for (k, (n_set, n_def, vals, owners)) in defaulted {
let vv: Vec<&str> = vals.iter().map(|s| s.as_str()).collect();
println!(
"{:<30} {:>5} {:>5} {} | {}",
k,
n_set,
n_def,
vv.join(","),
owners.join(",")
);
}
}
}
}

View File

@@ -0,0 +1,108 @@
//! RE diagnostic: emit every `weapon\Weapon_*.tbl` in a pak as machine-readable
//! records, split at the sub-object boundaries the schema declares.
//!
//! An IDXD `.tbl` for a weapon holds `count` sub-records — a `Weapon` and its
//! `Shell` — flattened into one string pool. `sylpheed-cli pak dump` shows them
//! merged, which is ambiguous (both declare `ID`/`Name`). The title's schema
//! name pool gives the split point: the pool contains the literal type names
//! (`Weapon`, `Shell`), and each sub-record's fields follow its type name.
//!
//! Here we split on a type-name token appearing as a *key* position, so each
//! record is emitted with its own ID and field set:
//!
//! ```text
//! REC <tbl-hash> <index> <TypeName> <ID>
//! F <key> <value>
//! ```
//!
//! Run: cargo run --release -p sylpheed-formats --example idxd_tokens -- <pak> [types...]
use sylpheed_formats::idxd::IdxdObject;
use sylpheed_formats::pak::PakArchive;
fn is_number(s: &str) -> bool {
let s = s.strip_suffix(['f', 'F']).unwrap_or(s);
let t = s.strip_prefix(['-', '+']).unwrap_or(s);
!t.is_empty() && t.chars().all(|c| c.is_ascii_digit() || c == '.')
}
/// `Yes`/`No`-style scalar literals are values, not field names, even though
/// they are identifier-shaped.
fn is_enum_value(s: &str) -> bool {
matches!(s, "Yes" | "No" | "YES" | "NO" | "On" | "Off" | "ON" | "OFF")
}
fn is_key_like(s: &str) -> bool {
!s.is_empty()
&& s.chars().all(|c| c.is_ascii_alphanumeric() || c == '_')
&& s.chars().next().is_some_and(|c| c.is_ascii_alphabetic() || c == '_')
&& !is_number(s)
&& !is_enum_value(s)
}
fn main() {
let mut args = std::env::args().skip(1);
let pak_path = args.next().expect("usage: idxd_tokens <pak> [TypeName...]");
let types: Vec<String> = {
let v: Vec<String> = args.collect();
if v.is_empty() {
vec!["Weapon".into(), "Shell".into()]
} else {
v
}
};
let pak = PakArchive::open(&pak_path).expect("open pak");
for entry in pak.entries() {
let Ok(bytes) = pak.read(entry) else { continue };
let Ok(obj) = IdxdObject::parse(&bytes) else { continue };
let toks = obj.tokens();
// Only entries that actually declare one of the requested sub-record
// types (the pool-start heuristic can prefix one stray byte, so match a
// short suffix rather than equality -- same rule as the splitter below).
if !toks
.iter()
.any(|t| types.iter().any(|ty| t == ty || (t.ends_with(ty.as_str()) && t.len() <= ty.len() + 2)))
{
continue;
}
if std::env::var("SYLPH_RAW").is_ok() {
println!("RAW {:08x}", entry.name_hash);
for t in toks {
println!("T {t}");
}
}
// Sub-record boundaries: a bare type-name token. The first one is the
// schema's own declaration (`EnumWeapon`-style header lives in title
// code, not here), so we take every occurrence in order.
let mut bounds: Vec<(usize, &str)> = Vec::new();
for (i, t) in toks.iter().enumerate() {
// The pool-start heuristic can glue one stray binary byte onto the
// first token ("YWeapon"), so match a short suffix, not equality.
if let Some(ty) = types
.iter()
.find(|ty| t == *ty || (t.ends_with(ty.as_str()) && t.len() <= ty.len() + 2))
{
bounds.push((i, ty.as_str()));
}
}
for (n, &(start, ty)) in bounds.iter().enumerate() {
let end = bounds.get(n + 1).map(|b| b.0).unwrap_or(toks.len());
let slice = &toks[start..end];
// The record's own ID value is the token straight after its type
// name (`Shell` -> `Shell_DSaber_P_wep_01_Beam`). The `ID`/`Name`
// *keys* are interned once in the pool, so only the first record
// shows them -- position-of-key lookup would miss the rest.
let id = slice.get(1).map(|s| s.as_str()).unwrap_or("?");
println!("REC {:08x} {n} {ty} {id}", entry.name_hash);
for i in 1..slice.len() {
let (k, v) = (slice[i].as_str(), slice[i - 1].as_str());
if is_key_like(k) && (!is_key_like(v) || is_number(v)) {
println!("F {k} {v}");
}
}
}
}
}

View File

@@ -1,118 +0,0 @@
//! Scratch: confirm the `.rat` layout record offsets against a real screen pak.
//! Run: cargo run -p sylpheed-formats --example rat_inspect -- <GP_SCREEN.pak>
use sylpheed_formats::{pak::PakArchive, ratc, t8ad};
fn be32(b: &[u8], o: usize) -> u32 {
u32::from_be_bytes([b[o], b[o + 1], b[o + 2], b[o + 3]])
}
fn main() {
let path = std::env::args().nth(1).expect("pak path");
let ar = PakArchive::open(&path).expect("open pak");
println!("entries: {}", ar.len());
let mut decoded: Vec<Vec<u8>> = Vec::new();
for (i, e) in ar.entries().iter().enumerate() {
let d = ar.read(e).unwrap_or_default();
let magic = String::from_utf8_lossy(&d[..4.min(d.len())]).to_string();
let (nt, nr) = if ratc::is_ratc(&d) {
let k = ratc::parse(&d).unwrap_or_default();
(
k.iter().filter(|c| c.kind == "T8aD").count(),
k.iter()
.filter(|c| c.name.to_lowercase().ends_with(".rat"))
.count(),
)
} else {
(0, 0)
};
println!(
" entry[{:2}] {:>8} B magic={:4} t32={:2} rat={:2}",
i,
d.len(),
magic,
nt,
nr
);
decoded.push(d);
}
// Pick the entry with the most children as "build0".
let bi = (0..decoded.len())
.filter(|&i| ratc::is_ratc(&decoded[i]))
.max_by_key(|&i| ratc::parse(&decoded[i]).map(|k| k.len()).unwrap_or(0))
.unwrap();
println!("=> inspecting richest build: entry[{}]", bi);
let bundle = &decoded[bi];
let kids = ratc::parse(bundle).unwrap_or_default();
let t32: Vec<_> = kids.iter().filter(|c| c.kind == "T8aD").collect();
let rat: Vec<_> = kids
.iter()
.filter(|c| c.name.to_lowercase().ends_with(".rat"))
.collect();
println!(
"build0: {} children, {} t32 sprites, {} rat records",
kids.len(),
t32.len(),
rat.len()
);
for c in t32.iter().take(5) {
let b = &bundle[c.offset..(c.offset + c.size).min(bundle.len())];
if let Some(img) = t8ad::parse(b) {
println!(" T8aD {:28} {}x{}", c.name, img.width, img.height);
}
}
println!(" -- .rat records --");
for c in rat.iter().take(10) {
let r = &bundle[c.offset..(c.offset + c.size).min(bundle.len())];
if r.len() < 0x60 {
println!(" .rat {:20} (len {}, too short)", c.name, r.len());
continue;
}
let (dw, dh) = (be32(r, 0x18), be32(r, 0x1c));
let sprite = {
let s = &r[0x20..0x30.min(r.len())];
let end = s.iter().position(|&b| b == 0).unwrap_or(s.len());
String::from_utf8_lossy(&s[..end]).to_string()
};
let (pvx, pvy) = (be32(r, 0x50), be32(r, 0x54));
// scan for placement block [scaleX=100, scaleY=100, tint, X<dw, Y<dh]
let mut placement = None;
let mut o = 0x58;
while o + 20 <= r.len() {
if be32(r, o) == 100 && be32(r, o + 4) == 100 {
let (tint, x, y) = (be32(r, o + 8), be32(r, o + 12), be32(r, o + 16));
if x < dw && y < dh {
placement = Some((o, tint, x, y));
break;
}
}
o += 4;
}
println!(
" .rat {:22} {}x{} sprite={:16} pivot=({},{}) place={:x?}",
c.name, dw, dh, sprite, pvx, pvy, placement
);
}
// End-to-end: composite the build and write it out as a PNG.
if let Some(screen) = sylpheed_formats::ui_layout::compose_build(bundle, false) {
println!(
"compose: {}x{}, drew {} records: {:?}",
screen.width,
screen.height,
screen.drawn.len(),
screen.drawn
);
if let Some(out) = std::env::args().nth(2) {
// Dependency-free PPM (P6, RGB — alpha already composited over the backdrop).
let mut buf = format!("P6\n{} {}\n255\n", screen.width, screen.height).into_bytes();
for px in screen.rgba.chunks_exact(4) {
buf.extend_from_slice(&px[..3]);
}
std::fs::write(&out, buf).unwrap();
println!("wrote {out}");
}
} else {
println!("compose: build produced no image");
}
}

View File

@@ -0,0 +1,55 @@
//! Scratch analysis: inventory one UI screen's pak — every entry, and for RATC
//! entries the children they bundle. Optionally write an entry's decompressed
//! bytes out for hex inspection.
//!
//! Run: cargo run -p sylpheed-formats --example ui_screen -- <PAK> [--dump 0xHASH out.bin]
use sylpheed_formats::pak::{self, PakArchive};
use sylpheed_formats::ratc;
fn main() {
let mut args = std::env::args().skip(1);
let pak = args.next().expect("usage: ui_screen <pak> [--dump 0xHASH out.bin]");
let arc = PakArchive::open(&pak).expect("open pak");
let rest: Vec<String> = args.collect();
if rest.first().map(|s| s == "--dump").unwrap_or(false) {
let h = u32::from_str_radix(rest[1].trim_start_matches("0x"), 16).unwrap();
let bytes = arc.read_by_hash(h).expect("entry present").expect("decompress");
std::fs::write(&rest[2], &bytes).unwrap();
println!("wrote {} bytes to {}", bytes.len(), rest[2]);
return;
}
if rest.first().map(|s| s == "--child").unwrap_or(false) {
// --child 0xHASH <child-name> <out>: carve one RATC child out by its
// listed offset/size, so a 165-byte layout record can be hexdumped alone.
let h = u32::from_str_radix(rest[1].trim_start_matches("0x"), 16).unwrap();
let bytes = arc.read_by_hash(h).expect("entry present").expect("decompress");
let kids = ratc::parse(&bytes).expect("ratc");
let k = kids.iter().find(|k| k.name == rest[2]).expect("child not found");
std::fs::write(&rest[3], &bytes[k.offset..k.offset + k.size]).unwrap();
println!("wrote {} B ({} @ {:#x}) to {}", k.size, k.name, k.offset, rest[3]);
return;
}
println!("{pak}: {} entries", arc.len());
for e in arc.entries() {
let Ok(bytes) = arc.read(e) else {
println!(" {:08x} <decompress failed>", e.name_hash);
continue;
};
let label = pak::inner_format_label(&bytes);
println!(" {:08x} {:>9} B {}", e.name_hash, bytes.len(), label);
if ratc::is_ratc(&bytes) {
if let Some(kids) = ratc::parse(&bytes) {
for k in &kids {
println!(" - {:<40} {:>9} B {}", k.name, k.size, k.kind);
}
if kids.is_empty() {
println!(" (no children found)");
}
}
}
}
}

View File

@@ -37,9 +37,6 @@ pub mod t8ad;
// RATC nested resource bundle
pub mod ratc;
/// UI screen layout (`.rat`) — reassemble a screen from its pak.
pub mod ui_layout;
// LSTA sprite list (inline T8aD frames)
pub mod lsta;

View File

@@ -1,282 +0,0 @@
//! `.rat` UI-screen layout — reassemble a UI screen from its pak.
//!
//! A UI screen ships as one pak (`GP_TITLE`, `GP_PAUSE_MENU`, …). Inside it,
//! each large [RATC](crate::ratc) bundle is one *(context × language)* **build**
//! of the screen, holding its `<name>.t32` sprites and `<name>.rat` layout
//! records side by side. Each `.rat` record is itself a RATC-tagged blob that
//! places one sprite; this module parses those records and composites the
//! sprites back into the screen image.
//!
//! Format reverse-engineered in `docs/re/structures/ui-rat-layout.md` and
//! validated here against `GP_PAUSE_MENU.pak` / `GP_TITLE.pak`: the pause menu's
//! `pgpbtn00/01/04/15.rat` read X=226, Y=268/337/407/478 (the documented 70 px
//! pitch), and focus records land 42 px left / 8 px up of their base.
use crate::{ratc, t8ad};
use std::collections::HashMap;
fn be32(b: &[u8], o: usize) -> u32 {
u32::from_be_bytes([b[o], b[o + 1], b[o + 2], b[o + 3]])
}
/// One sprite placement parsed from a `.rat` record.
#[derive(Debug, Clone)]
pub struct Placement {
/// The `.rat` record's own name (e.g. `pgpbtn00.rat`).
pub record: String,
/// The `.t32` sprite this record places (from record offset 0x20).
pub sprite: String,
/// Top-left position in the design space (X/Y from the placement block).
pub x: u32,
pub y: u32,
/// Scale in percent (100 = 1:1).
pub scale_x: u32,
pub scale_y: u32,
/// RGBA tint (`0xffffffff` = untinted).
pub tint: u32,
/// A `*f.rat` focus-state record (draws the selection art).
pub focused: bool,
/// A `loopN.rat` / keyframed record — its first frame is taken.
pub animated: bool,
}
/// A parsed UI build: one screen layout (one context × language).
pub struct UiBuild {
/// Design-space dimensions, normally 1280×720.
pub design_w: u32,
pub design_h: u32,
/// Every `.rat` placement in draw order.
pub placements: Vec<Placement>,
/// Sprite name → (offset, size) of its `T8aD` child within the bundle.
pub sprites: HashMap<String, (usize, usize)>,
/// A guessed context from the sprite naming (e.g. `"tutorial"`), if any.
pub context_hint: Option<String>,
}
/// Whether `bundle` is a RATC build (has at least one `.rat` layout child).
pub fn is_build(bundle: &[u8]) -> bool {
ratc::is_ratc(bundle)
&& ratc::parse(bundle).is_some_and(|kids| {
kids.iter()
.any(|c| c.name.to_ascii_lowercase().ends_with(".rat"))
})
}
/// Parse one `.rat` record (a RATC-tagged placement blob) → a [`Placement`].
fn parse_record(name: &str, rec: &[u8]) -> Option<Placement> {
if rec.len() < 0x58 || rec[0..4] != *b"RATC" {
return None;
}
let dw = be32(rec, 0x18);
let dh = be32(rec, 0x1c);
if dw == 0 || dh == 0 || dw > 8192 || dh > 8192 {
return None;
}
// Sprite name: NUL-terminated ASCII at 0x20 (up to 16 bytes).
let sname = {
let s = &rec[0x20..0x30.min(rec.len())];
let end = s.iter().position(|&b| b == 0).unwrap_or(s.len());
String::from_utf8_lossy(&s[..end]).trim().to_string()
};
if sname.is_empty() {
return None;
}
// Placement block: [scaleX=100, scaleY=100, tint, X, Y] — the first such run
// whose X/Y fall inside the design space (records are tag-driven/variable, so
// this anchor is more robust than a fixed offset). See the format doc.
let mut placement = None;
let mut o = 0x58;
while o + 20 <= rec.len() {
if be32(rec, o) == 100 && be32(rec, o + 4) == 100 {
let (tint, x, y) = (be32(rec, o + 8), be32(rec, o + 12), be32(rec, o + 16));
if x < dw && y < dh {
placement = Some((tint, x, y));
break;
}
}
o += 4;
}
let (tint, x, y) = placement?;
let lname = name.to_ascii_lowercase();
Some(Placement {
record: name.to_string(),
sprite: sname,
x,
y,
scale_x: 100,
scale_y: 100,
tint,
focused: lname.ends_with("f.rat"),
animated: lname.contains("loop"),
})
}
/// Parse a build bundle into its placements and sprite table.
pub fn parse_build(bundle: &[u8]) -> Option<UiBuild> {
let kids = ratc::parse(bundle)?;
let mut sprites = HashMap::new();
let mut placements = Vec::new();
for c in &kids {
let lname = c.name.to_ascii_lowercase();
let end = (c.offset + c.size).min(bundle.len());
if c.kind == "T8aD" {
sprites.insert(c.name.clone(), (c.offset, end - c.offset));
} else if lname.ends_with(".rat") {
if let Some(p) = parse_record(&c.name, &bundle[c.offset..end]) {
placements.push(p);
}
}
}
if placements.is_empty() {
return None;
}
let (design_w, design_h) = placements
.iter()
.find_map(|_| {
// design dims are constant across records; re-read the first record
kids.iter()
.find(|c| c.name.to_ascii_lowercase().ends_with(".rat"))
.map(|c| {
let r = &bundle[c.offset..(c.offset + c.size).min(bundle.len())];
(be32(r, 0x18), be32(r, 0x1c))
})
})
.unwrap_or((1280, 720));
let context_hint = sprites
.keys()
.find_map(|n| n.contains("ttrl").then(|| "tutorial".to_string()));
Some(UiBuild {
design_w,
design_h,
placements,
sprites,
context_hint,
})
}
/// A composited screen image ready to display.
pub struct ComposedScreen {
pub width: u32,
pub height: u32,
/// Row-major RGBA8.
pub rgba: Vec<u8>,
/// Names of the records actually drawn.
pub drawn: Vec<String>,
}
/// Composite a build into its screen image.
///
/// Draws every base placement (title + menu items). Animated `loop*` records are
/// skipped (they're decorations without a static position); `*f` focus records
/// are skipped unless `include_focus`. Sprites are alpha-blended at their
/// top-left with their tint applied.
pub fn compose_build(bundle: &[u8], include_focus: bool) -> Option<ComposedScreen> {
let build = parse_build(bundle)?;
let (w, h) = (build.design_w, build.design_h);
// A dim backdrop stands in for the PRMD dim-quad + live 3D scene.
let mut canvas = vec![0u8; (w * h * 4) as usize];
for px in canvas.chunks_exact_mut(4) {
px.copy_from_slice(&[14, 14, 20, 255]);
}
let mut drawn = Vec::new();
for p in &build.placements {
if p.animated || (p.focused && !include_focus) {
continue;
}
let Some(&(off, size)) = build.sprites.get(&p.sprite) else {
continue;
};
let Some(img) = t8ad::parse(&bundle[off..off + size]) else {
continue;
};
blit(&mut canvas, w, h, &img, p);
drawn.push(p.record.clone());
}
Some(ComposedScreen {
width: w,
height: h,
rgba: canvas,
drawn,
})
}
/// Alpha-blend one sprite onto the canvas at its placement, with tint + scale.
fn blit(canvas: &mut [u8], cw: u32, ch: u32, img: &t8ad::T8adImage, p: &Placement) {
let (sw, sh) = (img.width, img.height);
if sw == 0 || sh == 0 {
return;
}
let (dw, dh) = (sw * p.scale_x / 100, sh * p.scale_y / 100);
let (tr, tg, tb, ta) = (
(p.tint >> 24) & 0xff,
(p.tint >> 16) & 0xff,
(p.tint >> 8) & 0xff,
p.tint & 0xff,
);
for oy in 0..dh {
let ty = p.y + oy;
if ty >= ch {
break;
}
let syi = (oy * sh / dh).min(sh - 1);
for ox in 0..dw {
let tx = p.x + ox;
if tx >= cw {
break;
}
let sxi = (ox * sw / dw).min(sw - 1);
let si = ((syi * sw + sxi) * 4) as usize;
let sr = img.rgba[si] as u32 * tr / 255;
let sg = img.rgba[si + 1] as u32 * tg / 255;
let sb = img.rgba[si + 2] as u32 * tb / 255;
let sa = img.rgba[si + 3] as u32 * ta / 255;
if sa == 0 {
continue;
}
let di = ((ty * cw + tx) * 4) as usize;
for (k, sc) in [sr, sg, sb].into_iter().enumerate() {
let dc = canvas[di + k] as u32;
canvas[di + k] = ((sc * sa + dc * (255 - sa)) / 255) as u8;
}
canvas[di + 3] = 255;
}
}
}
#[cfg(test)]
mod tests {
use super::*;
/// A synthetic `.rat` record: RATC header, sprite name at 0x20, a
/// `[100,100,tint,X,Y]` placement block.
fn synth_record(sprite: &str, x: u32, y: u32) -> Vec<u8> {
let mut r = vec![0u8; 0x58];
r[0..4].copy_from_slice(b"RATC");
r[0x18..0x1c].copy_from_slice(&1280u32.to_be_bytes());
r[0x1c..0x20].copy_from_slice(&720u32.to_be_bytes());
let nb = sprite.as_bytes();
r[0x20..0x20 + nb.len()].copy_from_slice(nb);
for v in [100u32, 100, 0xffff_ffff, x, y] {
r.extend_from_slice(&v.to_be_bytes());
}
r
}
#[test]
fn parses_placement_block() {
let r = synth_record("pgpbtn00.t32", 226, 268);
let p = parse_record("pgpbtn00.rat", &r).unwrap();
assert_eq!(p.sprite, "pgpbtn00.t32");
assert_eq!((p.x, p.y), (226, 268));
assert_eq!(p.tint, 0xffff_ffff);
assert!(!p.focused);
}
#[test]
fn flags_focus_and_rejects_out_of_range() {
let f = parse_record("pgpbtn00f.rat", &synth_record("ring.t32", 184, 260)).unwrap();
assert!(f.focused);
// X beyond design space → no placement found.
assert!(parse_record("bad.rat", &synth_record("x.t32", 9000, 10)).is_none());
}
}

View File

@@ -160,9 +160,8 @@ pub enum PakContent {
T8ad(ImageRgba),
/// An LSTA sprite list — inline T8aD frames.
Lsta(Vec<ImageRgba>),
/// A RATC bundle — its listed children (T8aD children carry a decoded
/// image), plus the reassembled UI screen when the bundle is a `.rat` build.
Ratc(Vec<RatcEntry>, Option<ImageRgba>),
/// A RATC bundle — its listed children (T8aD children carry a decoded image).
Ratc(Vec<RatcEntry>),
/// Plain-text / XML payload, with its encoding label.
Text { text: String, encoding: String },
}
@@ -193,13 +192,6 @@ impl ImageRgba {
rgba: img.rgba,
}
}
fn from_composed(s: sylpheed_formats::ui_layout::ComposedScreen) -> Self {
Self {
width: s.width,
height: s.height,
rgba: s.rgba,
}
}
fn pixels(&self) -> usize {
(self.width as usize) * (self.height as usize)
}
@@ -1477,10 +1469,7 @@ fn classify_content(payload: &[u8]) -> PakContent {
})
.collect();
if !entries.is_empty() {
// If this bundle is a `.rat` UI build, reassemble the screen.
let ui_screen = sylpheed_formats::ui_layout::compose_build(payload, false)
.map(ImageRgba::from_composed);
return PakContent::Ratc(entries, ui_screen);
return PakContent::Ratc(entries);
}
}
}

View File

@@ -672,9 +672,7 @@ fn draw_pak_browser(ui: &mut egui::Ui, pak: &mut PakView) {
PakContent::Png(img) => draw_png_detail(ui, row, img, img_tex),
PakContent::T8ad(img) => draw_t8ad_detail(ui, row, img, img_tex),
PakContent::Lsta(frames) => draw_lsta_detail(ui, row, frames, img_tex),
PakContent::Ratc(children, ui_screen) => {
draw_ratc_detail(ui, row, children, ui_screen.as_ref(), img_tex)
}
PakContent::Ratc(children) => draw_ratc_detail(ui, row, children, img_tex),
PakContent::Text { text, encoding } => {
draw_text_detail(ui, row, text, encoding)
}
@@ -701,10 +699,7 @@ fn row_kind(row: &crate::iso_loader::PakRow) -> String {
PakContent::Png(img) => format!("PNG {}×{}", img.width, img.height),
PakContent::T8ad(img) => format!("T8aD {}×{}", img.width, img.height),
PakContent::Lsta(f) => format!("LSTA · {} sprite(s)", f.len()),
PakContent::Ratc(c, screen) => {
let s = if screen.is_some() { " · UI screen" } else { "" };
format!("RATC · {} item(s){s}", c.len())
}
PakContent::Ratc(c) => format!("RATC · {} item(s)", c.len()),
PakContent::Text { .. } => "text".into(),
PakContent::None => row.identity.clone(),
}
@@ -944,47 +939,21 @@ fn draw_ratc_detail(
ui: &mut egui::Ui,
row: &crate::iso_loader::PakRow,
children: &[crate::iso_loader::RatcEntry],
ui_screen: Option<&ImageRgba>,
img_tex: &mut ImgCache,
) {
ui.horizontal(|ui| {
ui.heading("RATC bundle");
ui.separator();
ui.label(format!("{} item(s)", children.len()));
if ui_screen.is_some() {
ui.separator();
ui.strong("🖼 UI screen");
}
ui.separator();
ui.weak("colours unverified");
});
ui.separator();
// Cache the reassembled screen (if any) as texture 0, then child thumbnails.
let mut refs: Vec<&ImageRgba> = Vec::new();
if let Some(s) = ui_screen {
refs.push(s);
}
let child_base = refs.len();
refs.extend(children.iter().filter_map(|c| c.image.as_ref()));
let refs: Vec<&ImageRgba> = children.iter().filter_map(|c| c.image.as_ref()).collect();
ensure_textures(ui, row.hash, &refs, img_tex);
// The reassembled screen, scaled to fit the panel width.
if ui_screen.is_some() {
if let Some(tex) = img_tex.as_ref().and_then(|(_, t)| t.first()) {
ui.label("Reassembled from this screen's .rat layout records:");
let sz = tex.size_vec2();
let scale = (ui.available_width() / sz.x.max(1.0)).min(1.0);
ui.add(egui::Image::new(egui::load::SizedTexture::new(
tex.id(),
[sz.x * scale, sz.y * scale],
)));
ui.weak("Frame/glow decorations (no .rat) and the live 3D background are omitted.");
ui.separator();
}
}
let mut img_i = child_base;
let mut img_i = 0;
for c in children {
ui.horizontal(|ui| {
if c.image.is_some() {

View File

@@ -21,6 +21,15 @@ Promote to a prose `structures/…md` file when a format needs behavioural notes
| Fonts (ttf/otf/ttc) | ✅ | `sylpheed-formats/src/font.rs` | standard OpenType, parsed via ttf-parser |
| XBG7 mesh | 🟡/❔ | `sylpheed-formats/src/mesh.rs` + `tests/mesh_disc.rs` ([xbg7](structures/xbg7-mesh.md)) | weapons/props: declaration-driven variable stride (36 models), GPU-confirmed. **Stage containers: 5662 sub-models across 22 stages** via content-anchored grouped pools (`stage_models`). Quantized hero bodies (DeltaSaber `f004`) still declined |
| Capital-ship part placement | 🟡 | `sylpheed-formats/src/ship.rs` (static) + [runtime capture](ship-placement-runtime-capture.md) | hull placement static-exact; external parts approximate statically. **Runtime capture** (Canary F10 → VS-constant WorldView) gives ground truth — validated on `e106` destroyer; not yet baked into the viewer |
| Weapon fields defaulted on disc | ✅ | [runtime struct](structures/weapon-struct-runtime.md) · [DATA SHEET route](weapon-datasheet-runtime.md) | **Solved.** Canary maps guest RAM into `/dev/shm`, so the parsed `Weapon`/`Shell` objects are readable live; their layout is solved against disc ground truth (zero contradictions over 100+ records). All 126 weapons, exact numbers, no story progress needed — [4 393 values](captures/weapon-runtime-fields.csv) the disc does not carry. Supersedes the letter-bucket limit of the DATA SHEET route, which now serves as the independent cross-check |
| UI screen layout (`.rat`) | ✅/🟡 | [ui-rat-layout](structures/ui-rat-layout.md) | One pak per UI screen; each RATC = one (context × language) build; every `<name>.t32` sprite has a `<name>.rat` **layout record** (BE u32; 1280×720 design space; scale/tint/X/Y, keyframes for animated elements, `opt ` link to the focused state). **The tutorial PAUSE menu and the title main menu both rebuild pixel-accurately from the disc.** `loop1.rat` (screen-level draw order) not yet decoded |
## Runtime / dynamic-capture technique
| Technique | Conf. | Spec | Notes |
|-----------|-------|------|-------|
| Live guest-memory read | ✅ | [`tools/re-capture/gmem.py`](../../tools/re-capture/gmem.py) | Canary backs the guest address space with `/dev/shm/xenia_memory_*`; guest VAs map in through Xenia's fixed table. Full-RAM search ~0.2 s (sparse, `SEEK_DATA`). No debugger, no emulator patch, game keeps running |
| IDXD object layout solver | ✅ | [`tools/re-capture/weapon_runtime.py`](../../tools/re-capture/weapon_runtime.py) | Scan RAM for a class's vtable → enumerate its objects → brute-force `(field, offset, encoding)` against the disc records. Accepts a binding only on **zero** contradictions. Generalizes to any IDXD-backed definition |
## Functions / code paths

Binary file not shown.

After

Width:  |  Height:  |  Size: 195 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 233 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 218 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 188 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 167 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 220 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 98 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 51 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 104 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 117 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 112 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 53 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 53 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 45 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 44 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 102 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 159 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 54 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 53 KiB

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,182 @@
# `.rat` — the UI element layout / animation record
**Status:**`CONFIRMED` for placement (2026-07-28). The retail UI can be
**reassembled from the disc**: the tutorial PAUSE menu rebuilds pixel-accurately from its
sprites placed at the coordinates in their `.rat` records — no fitting, no manual nudging.
![real vs rebuilt](captures/ui-layout/pause-tutorial-real-vs-rebuilt.png)
*Left: the running game (Canary screenshot). Right: rebuilt from `GP_PAUSE_MENU.pak` alone.
The remaining differences are the animated frame/glow sprites (`*eff*`) that were not
placed, and the live 3D background.*
## Screen composition
The UI is **one pak per screen**`GP_TITLE`, `GP_PAUSE_MENU`, `GP_READY_ROOM`,
`GP_SAVE_LOAD`, `GP_MISSION_SELECT`, `GP_OPTIONS`, `GP_SYSTEM`, `GP_TUTORIAL`,
`GP_STAGE_CLEAR`, `GP_MOVIE_THEATER`, `GP_DEBRIEFING_PILOTLOG`, `GP_LEADERBOARD`,
`GP_MISSION_LOG`, `GP_BUNK`, `GP_DIALOG`, `GP_GAMEOVER`, `GP_CHALLENGE`,
`GP_HANGAR_ARSENAL`.
Inside a screen pak, each top-level [RATC](../INDEX.md) bundle is **one (context × language)
build of that screen**. Its own header is the screen's **element declaration table**, and
the elements themselves follow as children:
| child | what it is |
|---|---|
| `<name>.t32` | the sprite ([T8aD](texture-color-k8888.md)) |
| `<name>.rat` | that sprite's **layout record** (this document) |
| `<screen>loop1.rat` | a looping sprite animation (see below) |
### The bundle header — element declaration table
```
0x14 u32 entry count
0x20 entry[count], 60 bytes each:
+0 char[28] element name, NUL-padded ("pgp_ttrl_eff10.t32", "pgp_ttrl_btn10.rat")
+28 u32 ×4 flags (0xffffffff / 0xffffffff / 0 / 0xffffffff on every entry seen)
+48 u32 pivot X
+52 u32 pivot Y
+56 u32 0
```
The table lists **both** sprites and `.rat` records — it is the screen's element list.
`pgp_ttrl` declares 11: six `eff*`, `msg`, and four `.rat`s (`title`, `btn10..12`).
**Verified:** for all 7 `.t32` entries the declared pivot is *exactly* half the decoded
texture's dimensions — `eff10` 408×120 → 204,60; `eff21` 428×360 → 214,180; `msg` 381×38 →
190,19; and so on, 7/7 with no mismatch (`tools/re-capture/ratc_decls.py`).
`GP_PAUSE_MENU.pak`'s six bundles are `{in-mission, tutorial} × {English, Japanese}`, with
the two in-mission builds each present twice at identical size. The purpose of that
duplicate is **NEEDS-HUMAN** (resolution or aspect variant?), and where the other four
shipped languages live is likewise unresolved — this pak holds only EN and JP.
Naming is transparent: `pgp` = pause screen, `pgp_ttrl_` = its tutorial context, `btnNN` =
menu item, `btnNNf` = that item's **focused** sprite, `eff*` = frame/glow decoration,
`deli*` = the item divider, `title`, `msg`.
## Record layout
Big-endian u32 throughout (Xbox 360), and **tag-driven**: 4-char ASCII tags (`opt `,
`PRMD`, `end `) mark sections, so a record is a stream of blocks rather than a fixed struct.
A minimal record (a static button) is 165 bytes:
```
0x00 "RATC" magic — a RATC bundle reused as a data record
0x08 u32 payload size
0x14 u32 entry count (loop1.rat: 3, matching its 3 sprite names)
0x18 u32 design width = 1280
0x1c u32 design height = 720
0x20 char[16] the sprite this record places, e.g. "pgpbtn00.t32"
0x50 u32 pivot X = texture width / 2
0x54 u32 pivot Y = texture height / 2
...
── placement block ──
u32 scale X = 100 (percent)
u32 scale Y = 100
u32 tint = 0xffffffff (RGBA, white = untinted)
u32 X ← top-left position
u32 Y ←
u32 time (keyframe records only)
...
"opt " u32 len char[len] link to another record, e.g. "pgpbtn00f.rat"
```
- **X/Y is the sprite's top-left**, not its centre: compositing at these coordinates
reproduces the screenshot, which drawing centred on them would not.
- **The pivot at 0x50/0x54 is half the texture size** — 4 of the 5 plain sprites match
exactly (`pgpbtn00` 86×42 → 43,21; `pgpbtn05` 221×42 → 110,21; `pgptitle` 202×73 →
101,36; `pgpbtn04` 172×43 → 86,22). It is a rotation/scale centre, not a draw offset.
- **Animated elements are a keyframe list.** `pgptitle.rat` (752 B) is the same placement
block repeated with a varying trailing `time` field — the PAUSE title's fly-in.
- **`opt `** carries a length-prefixed record name. On `pgpbtn00.rat` it points at
`pgpbtn00f.rat`, i.e. *normal state → focused state*. Focused records place their sprite
42 px left and 8 px up of the base, because the focused art includes the selection ring
that hangs off the left edge; their pivot is a constant (21,25) rather than half-size.
- `<screen>loop1.rat` is **not** a screen composition — it is a looping sprite animation:
a 3-name table (`pgpeff34/35/36.t32`) plus ~30 keyframes all at one position.
- The small (380 B) top-level entries are **`PRMD` primitives**, not sprites: a colour and
four explicit corner coordinates `(0,0) (1280,0) (0,720) (1280,720)` — the full-screen
quad that dims the scene behind the pause menu — terminated by `end `.
### The records are language-independent
`pgpbtn00.rat` is **byte-identical** in the English and Japanese bundles. The layout is
authored once and only the `.t32` sprites are swapped, which has two consequences:
- The baked pivot belongs to *whichever build the record was authored from*, not to the
sprite actually shipped beside it. That is why `pgpbtn01`'s pivot (113 → a 226 px wide
texture) matches neither the English sprite (207) nor the Japanese one (148).
- The split is visible inside a single bundle: in the **English** tutorial build, every
`.t32` declaration carries the correct English pivot (7/7), while the `.rat` declarations
carry Japanese-derived ones (`btn10` → 43,21 = 86/2, the *Japanese* sprite). So the
`.t32` table is regenerated per language and the `.rat` layer is inherited from the
Japanese master.
- **Do not infer anything about a language from a texture size** — see the traps below.
## Evidence
Positions were read out of the records and checked against a screenshot of the running
game, twice, in that order — the records were never fitted to the picture.
1. **Differential.** Across `pgpbtn00/01/04/05.rat`, exactly one field varies and it steps
`268 → 338 → 408 → 478` — a constant 70 px pitch — while the field before it is 226 in
all four. A vertical menu: constant X, evenly spaced Y.
2. **Absolute placement — the decisive test.** The *tutorial* build's records give
546/288, 546/358, 546/428 and 540/119. Compositing its sprites at exactly those numbers,
with no offset and no fitting, reproduces the screenshot (image above).
3. **Pivot.** 4 of 5 plain sprites carry exactly half their texture's dimensions at
0x50/0x54 (above).
> A caution on step 2, because the first pass here got it subtly wrong: the screenshot is
> of the **tutorial** pause menu, so only the `pgp_ttrl_*` records can be checked against
> it. The in-mission records (X = 226) also map onto the same screenshot under a single
> constant offset — but that only works because both builds share the 70 px pitch, and it
> proves nothing. The in-mission coordinates remain **unverified**: confirming them needs a
> screenshot of a pause during an actual mission.
## It generalizes — the title screen
The same method run against `GP_TITLE.pak` reproduces the **main menu**, which is a
different screen with a different item count and a different pitch:
![main menu real vs rebuilt](captures/ui-layout/title-mainmenu-real-vs-rebuilt.png)
`ptbtn01..05.rat` give X = 542 for all five and Y = 162 / 242 / 322 / 402 / 482 — an
**80 px** pitch, where the pause menu used 70. Measured against the screenshot, the sprite
tops land at a constant **+46 px** for all five (one reads 45, a 1-px edge-detection
wobble), and 46 is exactly the 45 px of Xenia window chrome plus one. So the record's Y is
the sprite's top edge in the guest framebuffer, to the pixel, on a second screen.
`GP_TITLE.pak` also splits by sub-screen the way the pause pak splits by context:
`ptbtn00` alone (the `PRESS Ⓐ BUTTON` prompt), `ptbtn01..05` (main menu), `ptbtn11..13`
(the EXTRAS submenu), plus `pgloading_*` for the loading screen.
## Two traps this caught
Both were mistakes made during this analysis, caught by comparing against the real game —
recording them because a static-only reading would have shipped them:
- **Texture width does not identify a language.** English `RESUME` (166 px) and Japanese
`再開` (86 px) differ hugely, but Japanese `通信ログ` (148 px) is within a few px of an
English label. The first language assignment made here was wrong; rendering the sprites
is the only reliable check. (The records being language-independent makes this worse:
a record's baked pivot implies a texture width that matches *no* shipped sprite.)
- **The in-mission and tutorial pause menus are different sprite sets, not one set
re-packed.** In-mission is `RESUME / RADIO LOG / OPTIONS / BACK TO TITLE` (4 items,
`pgpbtnNN`); the tutorial is `RESUME / OPTIONS / BACK TO MENU` (3 items,
`pgp_ttrl_btn1N`). Matching the 3-item screenshot against the 4-item set suggests a
runtime slot-packing rule that does not exist.
## Next
- **The `eff*` / `deli*` / `msg` placements are still missing** — those sprites have no
`.rat` of their own, and `loop1.rat` turned out to be an animation, not a composition.
So the screen's draw list lives somewhere not yet found (the parent RATC's own header
region, or title code). That is the gap between the rebuild above and a complete screen.
- The same method should now unroll the other screens directly; `GP_HANGAR_ARSENAL.pak`
(789 T8aD + 510 RATC) is the big one, and the ARSENAL `DATA SHEET` panel documented in
[weapon-datasheet-runtime.md](../weapon-datasheet-runtime.md) is a ready-made oracle for it.
- Tooling: `crates/sylpheed-formats/examples/ui_screen.rs` (inventory a screen pak, carve a
named RATC child), `sylpheed-cli pak textures` (decode every sprite).

View File

@@ -0,0 +1,294 @@
# Runtime `Weapon` / `Shell` structs — read from live guest memory
**Confidence: ✅ CONFIRMED** for the fields marked ✅ below (each binding is
reproduced by 10125 independent disc records with **zero** contradictions);
🟡 for the thin ones. Captured 2026-07-29 from Xenia Canary running the retail
disc, save slot 01 (Stage 02, 5 % progress), READY ROOM and ARSENAL.
## Why this exists
`weapon\Weapon_*.tbl` is an [IDXD](../INDEX.md) record whose string pool **omits
every field left at its default**. That parked a long list of stats as
unreadable — the `…Ratio` / `…Count` family, `Power`, `MaximumRange`. The
[Arsenal DATA SHEET route](../weapon-datasheet-runtime.md) recovered a few of
them but only as letter buckets (Range/Damage as `A``E`), and only for the 9
weapons unlocked at 5 % progress.
This reads the values **directly out of the running game's memory** instead.
All 126 weapons, all fields, exact numbers, in one pass — and it needs no story
progress, because the definitions are parsed at load time whether or not the
player has unlocked the weapon.
## The lever: Canary maps guest RAM into `/dev/shm`
Xenia Canary backs the entire guest address space with one shared-memory file,
`/dev/shm/xenia_memory_<id>`. It is a plain file: **the guest's RAM is readable
from the host with `open`/`seek`/`read`, live, no debugger and no emulator
patch.** Guest VAs map into it through Xenia's fixed table (`memory.cc`), which
[`tools/re-capture/gmem.py`](../../../tools/re-capture/gmem.py) implements:
```bash
python3 tools/re-capture/gmem.py find "Weapon_DSaber_P_wep_01" # search all of RAM
python3 tools/re-capture/gmem.py words 0xbccce500 48 # dump as BE u32/f32
```
The file is sparse (~212 MB resident of 4.5 GB), and the scan uses
`SEEK_DATA`/`SEEK_HOLE`, so a full-RAM search costs ~0.2 s.
## Finding the objects
1. The title's **schema field-name pool** is in the XEX at `0x82086a30`, in
declaration order: `EnumWeapon`, type `Weapon` (`ID`, `Name`, `TargetType`,
`CartridgeModelName`), then type `Shell` (`ID`, `Name`, `MovementType`, …
`Explosion_MaxDamageRadius`). This is exactly the on-disc field order. No
pointer to these strings exists anywhere in RAM — PPC builds the addresses
with `lis`/`ori` immediate pairs — so the schema descriptor cannot be found
by pointer-chasing; the layout has to be solved instead (below).
2. Each parsed record becomes **two C++ objects**, a `Weapon` and its `Shell`,
each in its own contiguous array, each identified by its **vtable pointer**:
| class | vtable VA | stride | count |
|-------|-----------|--------|-------|
| `Weapon` | `0x820af548` | `0xc0` | 126 |
| `Shell` | `0x820af58c` | `0x200` | 126 |
The vtable VAs are static (XEX `.data`); the array base addresses are heap
and are **not** assumed — the tool locates every object by scanning RAM for
the vtable word.
3. Object `+0x04` points at a 0x40-byte **name record**; the ID string sits at
`+0x10` inside it. That is what keys each object back to its disc record.
`Weapon``Shell` pairing is by ID (`Weapon_X``Shell_X`) — the two arrays
are in **different orders**, so index-pairing would be wrong.
## Solving the layout (the part that makes this evidence, not guesswork)
[`tools/re-capture/weapon_runtime.py`](../../../tools/re-capture/weapon_runtime.py)
brute-forces every `(field, byte offset, encoding)` triple and scores it against
the disc: how many records does this offset *reproduce*, and how many does it
*contradict*? A binding is accepted only with **zero contradictions**, and is
marked ✅ only when ≥10 records agree on ≥3 distinct values.
That threshold matters: a field whose disc samples are all the same number
matches any offset holding that constant, so agreement count alone is not
evidence — the number of **distinct** values pinned down is. Two fields landing
on one offset is impossible in a real struct, so collisions are resolved to the
better-evidenced field and the loser is reported as unsolved. Bindings that
contradict more than 20 % of their samples are discarded outright rather than
reported on their agreeing subset.
Encodings found: `f32`, `u32`, `deg` (**angles are stored in radians**;
`SprayAngle`, `AngularVelocity` and `SplitCone` are degrees on disc), and `cnt`
— see the next section.
### `IsCharging` switches the counter encoding ✅
`LoadingCount` and `TriggerShotCount` at `+0x28` / `+0x44` are **int32 on 118
records and float32 on 8**. The 8 are exactly the records with
`IsCharging = Yes` — an exact partition, found by testing every disc field/value
pair against the float-encoded set. A charging weapon drains its magazine
continuously, so its counter needs a fraction.
Any reimplementation must branch on `IsCharging` when reading these two fields;
reading them as int unconditionally yields `1125515264` instead of `150`.
## Struct maps
See [`docs/re/captures/weapon-runtime-fields.csv`](../captures/weapon-runtime-fields.csv)
for the full machine-readable table — 7 182 rows, every confirmed field × every
one of the 126 records, tagged `disc` or `defaulted-on-disc`. **4 393 of those
values were not readable from the disc at all.**
### `Weapon` (`0xc0` bytes)
| offset | enc | field | agree | distinct | conf |
|--------|-----|-------|------:|---------:|------|
| `+0x010` | u32 | `ReticleType` | 62 | 13 | ✅ |
| `+0x01c` | u32 | `SpecialWeaponType` | 73 | 6 | ✅ |
| `+0x028` | cnt | `LoadingCount` | 121 | 33 | ✅ |
| `+0x02c` | f32 | `Interval` | 125 | 28 | ✅ |
| `+0x030` | f32 | `ReadyInterval` | 120 | 10 | ✅ |
| `+0x034` | f32 | `Heating` | 114 | 30 | ✅ |
| `+0x038` | f32 | `Cooling` | 106 | 21 | ✅ |
| `+0x03c` | f32 | `Mass` | 122 | 42 | ✅ |
| `+0x040` | f32 | `HitRatio` | 102 | 5 | ✅ |
| `+0x044` | cnt | `TriggerShotCount` | 114 | 17 | ✅ |
| `+0x048` | f32 | `TriggerShotInterval` | 99 | 11 | ✅ |
| `+0x050` | deg | `SprayAngle` | 96 | 17 | ✅ |
| `+0x06c` | f32 | `LockIntervalSingle` | 61 | 9 | ✅ |
| `+0x070` | f32 | `LockIntervalMulti` | 28 | 9 | ✅ |
| `+0x09c` | f32 | `MaximumCharging` | 3 | 3 | 🟡 |
Also identified structurally, not by the solver: `+0x00` vtable, `+0x04` name
record, `+0x0c` `TargetType` as a bitmask (`Vessel|Craft|Structure` = 7),
`+0x14`/`+0x18` name hashes, `+0x7c`/`+0x80` muzzle-flash FX name record + hash.
Unsolved (no consistent offset): `Cracker_ShotInterval`, `Cracker_SubShotCount`,
`MinimumCharging`, `MultiTargetCount`.
### `Shell` (`0x200` bytes)
| offset | enc | field | agree | distinct | conf |
|--------|-----|-------|------:|---------:|------|
| `+0x00c` | cnt | `DeleteFadeSpeed` | 39 | 2 | 🟡 |
| `+0x010` | cnt | `GuidanceType` | 9 | 4 | 🟡 |
| `+0x014` | cnt | `SpiralType` | 3 | 1 | 🟡 |
| `+0x020` | f32 | `Length` | 107 | 3 | ✅ |
| `+0x024` | f32 | `Volume` | 19 | 3 | ✅ |
| `+0x028` | f32 | `ShellMass` | 110 | 34 | ✅ |
| `+0x03c` | f32 | `HP` | 28 | 2 | 🟡 |
| `+0x040` | f32 | `LifeTime` | 94 | 43 | ✅ |
| `+0x044` | f32 | `FadeInTime` | 13 | 2 | 🟡 |
| `+0x048` | f32 | `FadeOutTime` | 7 | 1 | 🟡 |
| `+0x04c` | f32 | `Velocity` | 106 | 15 | ✅ |
| `+0x050` | f32 | `MinimumVelocity` | 11 | 2 | 🟡 |
| `+0x054` | f32 | `MaximumVelocity` | 29 | 7 | ✅ |
| `+0x058` | deg | `AngularVelocity` | 49 | 19 | ✅ |
| `+0x05c` | f32 | `Acceleration` | 9 | 4 | 🟡 |
| `+0x064` | f32 | `BeginGuidanceTimeAdjust` | 24 | 2 | 🟡 |
| `+0x068` | f32 | `EndGuidanceTime` | 19 | 4 | ✅ |
| `+0x070` | f32 | `Spiral_BeginTime` | 2 | 2 | 🟡 |
| `+0x074` | f32 | `Spiral_BeginTimeAdjust` | 25 | 2 | 🟡 |
| `+0x08c` | f32 | `MinimumRange` | 43 | 7 | ✅ |
| `+0x090` | f32 | `MaximumRange` | 117 | 21 | ✅ |
| `+0x0a0` | f32 | `Color_R` | 18 | 4 | ✅ |
| `+0x0a4` | f32 | `Color_G` | 121 | 3 | ✅ |
| `+0x0a8` | f32 | `Color_B` | 100 | 2 | 🟡 |
| `+0x0b4` | f32 | `Radius` | 89 | 10 | ✅ |
| `+0x0b8` | f32 | `Power` | 101 | 37 | ✅ |
| `+0x0bc` | f32 | `FailedDamageRatio` | 2 | 2 | 🟡 |
| `+0x0c0` | f32 | `PlayerLaserPower` | 11 | 6 | ✅ |
| `+0x0fc` | f32 | `ChaffResistRatio` | 24 | 1 | 🟡 |
| `+0x104` | f32 | `ChargingSizeRatio` | 3 | 1 | 🟡 |
| `+0x10c` | f32 | `SphereRadiusBegin` | 9 | 6 | 🟡 |
| `+0x110` | f32 | `SphereRadiusTurn` | 9 | 8 | 🟡 |
| `+0x114` | f32 | `SphereRadiusEnd` | 10 | 8 | ✅ |
| `+0x118` | f32 | `ExplosionTurnTime` | 3 | 2 | 🟡 |
| `+0x11c` | f32 | `ExplosionLifeTime` | 7 | 6 | 🟡 |
| `+0x120` | f32 | `ExplosionDamage_Maximum` | 4 | 4 | 🟡 |
| `+0x124` | f32 | `ExplosionDamage_OuterEdge` | 8 | 6 | 🟡 |
| `+0x128` | f32 | `Explosion_MaxDamageRadius` | 8 | 3 | 🟡 |
| `+0x130` | f32 | `SplitTime_Maximum` | 2 | 2 | 🟡 |
| `+0x134` | deg | `SplitCone` | 2 | 2 | 🟡 |
| `+0x148` | cnt | `NodeCount` | 39 | 4 | ✅ |
| `+0x164` | f32 | `Deceleration` | 9 | 2 | 🟡 |
Unsolved: `BeginGuidanceTime`, `EndGuidanceTimeAdjust`, `Spiral_EndTime`,
`SplitTime_Minimum`, `StartingVelocity`, `Straight1Type`, `st1_df_pitch`,
`pitch0`, and the `sp_qu_*` / `sp_sp_*` / `sp_zg_*` spiral-motion family. Those
are declared by too few records (or by none that the solver could separate) —
they need a state where the shells are actually in flight.
## The answer to the parked Route-B question
Player-weapon fields that are **defaulted on disc**, now read exactly (`·` = the
disc carries the value already):
| wep | LoadingCount | TriggerShotCount | MaximumRange | Power | Heating | Cooling | ReadyInterval | HitRatio | SprayAngle |
|---|---|---|---|---|---|---|---|---|---|
| 01 | · | **1** | · | · | · | · | · | · | · |
| 02 | · | · | · | **100** | · | · | · | **1** | · |
| 03 | · | · | · | · | · | · | · | · | **1** |
| 05 | · | **4** | · | · | · | · | · | · | · |
| 08 | · | · | · | · | · | · | · | **1** | · |
| 09 | · | · | · | · | **0.02** | · | · | · | **1** |
| 11 | **6** | · | · | · | · | · | · | · | · |
| 12 | · | · | · | · | **0** | · | · | **1** | · |
| 13 | · | · | · | **300** | · | · | · | · | · |
| 14 | · | · | · | · | · | · | · | · | **1** |
| 19 | · | · | · | · | · | **0.2** | · | · | **1** |
| 24 | · | **1** | · | · | · | · | · | · | **1** |
| 25 | · | · | **4000** | · | · | · | · | · | · |
| 26 | · | · | · | **500** | · | · | · | · | · |
| 27 | · | **1** | · | · | · | · | · | · | · |
| 28 | **5** | · | · | · | · | · | · | · | · |
| 29 | · | · | · | **100** | · | · | · | · | · |
| 30 | · | **4** | · | · | · | · | · | · | · |
| 36 | **5** | · | · | · | · | · | · | · | · |
| 37 | · | **1** | · | · | · | · | · | · | **1** |
| 38 | · | · | · | · | · | · | · | **1** | · |
| 39 | · | **1** | · | · | · | · | · | · | **1** |
| 40 | · | · | · | · | · | · | **0.1** | · | · |
| 48 | · | · | · | · | · | · | · | · | **1** |
| 50 | · | · | · | · | · | · | · | · | **1** |
| 52 | · | · | · | · | · | · | · | **1** | · |
| 53 | · | **1** | · | · | · | · | · | · | · |
| 54 | · | · | · | · | · | · | · | · | **1** |
| 55 | · | · | · | · | **0** | · | · | **1** | · |
| 56 | · | · | · | · | · | · | · | **1** | · |
| 57 | · | · | · | · | **0** | · | · | **1** | · |
| 58 | · | · | **10000** | · | · | · | · | · | **1** |
| 59 | · | · | · | · | · | · | · | **1** | **1** |
| 60 | · | **4** | · | **1000** | · | · | · | · | · |
| 62 | · | **1** | · | · | **0** | · | · | · | · |
| 66 | · | · | · | · | · | · | · | **1** | · |
| 67 | · | · | · | · | · | · | · | **1** | · |
| 68 | · | · | · | · | · | · | · | **1** | · |
| 69 | · | · | · | · | · | · | · | **1** | · |
| 70 | **0** | · | · | · | · | · | · | **1** | · |
| 71 | · | · | · | **10** | · | · | · | **1** | · |
| 81 | · | · | · | **300** | · | · | · | · | · |
| 82 | · | · | **10000** | · | · | · | · | · | **1** |
| 84 | · | · | · | · | · | · | · | · | **0.1** |
| 85 | · | **1** | · | · | **0** | · | · | · | · |
`HitRatio` is **1.0 for every one of the 24 records that default it** — the
whole `…Ratio` family that blocked Route B is simply "no penalty".
`wep_82`'s defaulted field is `PlayerLaserPower` = **12000** (its `Power`,
12000, is on disc).
## Cross-checks
- **Against the independent screenshot route.** The Arsenal DATA SHEET
([weapon-datasheet-runtime.md](../weapon-datasheet-runtime.md)) established
`Max. Lock Ons == TriggerShotCount`, and read **4** for `wep_05` and `wep_60`
off the panel. The memory read gives **4** for both — two unrelated methods,
same numbers.
- **Against the in-flight HUD.** `NOSE BM 06000` / `MAIN MPM 00300` matches
`wep_01` `LoadingCount = 6000` and `wep_02` `= 300` at `+0x28`.
- **Stability.** All 252 objects are **byte-identical** between the READY ROOM
and the ARSENAL, so these are load-time definition data, not transient state.
- **Self-consistency.** 121 records agree on `LoadingCount`'s offset across 33
distinct values with zero contradictions; `Power`, 101 records / 37 distinct
values. A wrong offset cannot do that.
## Incidental findings
- **A duplicate, conflicting disc record.** Two `.tbl` entries declare
`Shell_TCAF_Ship_AAGun`; one sets `Power = 60.0`, the other omits it. The
runtime object holds **5**, i.e. the *omitting* declaration won. Any
reimplementation loading both will silently pick one — this says which.
- **Not every disc record is instantiated.** 131 disc records produced 126
objects; the 5 with no runtime object are context variants that this save
never loads — `Weapon_TCAF_DeltaSaber_NoseGun_Ttrl` / `_Laser_Ttrl`
(tutorial), `_NoseGun_None`, `Weapon_TCAF_Ship_AAGun_EX5`,
`Weapon_ADAN_Attacker_S_GunTurret`. Running the tutorial should instantiate
the `_Ttrl` pair. No runtime object lacked a disc record.
- Defaulted `Power` values vary per weapon (1, 10, 100, 300, 500, 1000), so they
are **not** one constructor constant. Where they come from — the IDXD's
undecoded binary node/index region, or per-type code defaults — is **not
determined here** (`NEEDS-HUMAN` / follow-up). Either way the runtime values
above are ground truth, and they are now a decoding oracle for that region.
## Reproduce
```bash
export HOME=/sylph-home/re SDL_AUDIODRIVER=dummy
run-canary --audio --apu=sdl --log_mask=13 --logged_profile_slot_0_xuid=E0300000EFBEA3D4 &
tools/re-capture/skip_intro.sh # -> title
# LOAD GAME -> slot 01 -> YES -> READY ROOM (weapon tables load with the save)
cargo run --release -p sylpheed-formats --example idxd_tokens -- \
<disc>/dat/GP_MAIN_GAME_E.pak > /tmp/wep_tokens.txt
python3 tools/re-capture/weapon_runtime.py /tmp/wep_tokens.txt # report
python3 tools/re-capture/weapon_runtime.py /tmp/wep_tokens.txt --csv # full table
```
## What this unlocks
`gmem.py` + "scan for the vtable, solve the layout against disc ground truth" is
**not weapon-specific**. The same three steps apply to any IDXD-backed
definition whose fields the disc defaults — craft/`UNIT` stats, which the Hangar
UI exposes only as a `Gross Weight` class and which
[the menu route could not reach at all](../weapon-datasheet-runtime.md), are the
obvious next target.

View File

@@ -0,0 +1,232 @@
# Weapon DATA SHEET — runtime capture (Route B)
**Status:** 🟡 first dynamic capture, 2026-07-28. The Arsenal's *Gallery Mode* panel is a
direct runtime readout of the IDXD weapon record, which makes it an oracle for the fields
the disc leaves **defaulted**. Two field mappings are ✅ `CONFIRMED`; two defaulted values
are recovered at 🟡 `PROBABLE`. Captured from the retail game under Xenia Canary
(software Vulkan, headless) — see [the container recipe](#how-this-was-captured).
## Problem
`sylpheed-formats::game_data` reads the combat tables out of `dat/GP_MAIN_GAME_E.pak`, but
**a field left at its default value carries no value on disc** — the key is present in the
IDXD string pool with no value token in front of it. Those defaults live in title code, so
a static read can only ever say "not set", never *what* the game uses. That is a real hole
for the reimplementation: e.g. `Weapon_DSaber_P_wep_01_Beam` — the Delta Saber's starting
beam gun — has no `TriggerShotCount` and no `Power` on disc.
Ruled out first: the defaults are **not** hiding in another pak. `hidden/DefTables.pak`
contains no `WEAPON`-schema (`0x6ab4825a`) objects at all, and the other `GP_MAIN_GAME_*`
paks are localized duplicates of the English one.
The two scratch analyses that produced the shopping list live next to the crate:
`crates/sylpheed-formats/examples/defaulted_fields.rs` (per-schema: which keys are declared
but defaulted, and by whom) and `examples/default_owners.rs` (per-key: every owner, valued
or `<DEFAULT>`).
> Caveat on those tools: they classify a token as a *value* only if it is numeric or
> non-identifier-shaped. **Boolean/enum-valued fields therefore read as `<DEFAULT>`
> spuriously** (`Yes`, `Homing`, `Single`, `Burst` are identifier-shaped). Every finding
> below concerns numeric fields, where the classification is sound.
## Finding — the DATA SHEET reads the record
In **ARSENAL → (weapon type) → Y (Gallery Mode)** each entry shows a `DATA SHEET`, and it
is shown for weapons that have **not** been developed yet — only fully hidden (dashed)
entries are withheld. The same panel appears in **HANGAR → (hard point)**, with an extra
`Weapon Type` row.
| DATA SHEET row | IDXD field | Confidence | Evidence |
|---|---|---|---|
| `Ammo Capacity` | `LoadingCount` | ✅ CONFIRMED | every one of the 10 identified entries lands on a `LoadingCount` that exists in its own weapon-type tab — and see the circularity note below |
| `Max. Lock Ons` | `TriggerShotCount` | ✅ CONFIRMED | FALCON 9AM = 12 vs `wep_02`'s 12; BUZZARD 10AM = 22 vs `wep_04`'s 22 (two distinctive values, two independent records) |
| `Hard Point` | mount slot | ✅ CONFIRMED | matches the HANGAR slot the weapon is mountable/equipped on (`STILETTO BG I` → NOSE, `FALCON 9AM` → MAIN WEAPON1) |
| `Range Class` | bucket of `MaximumRange` | 🟡 PROBABLE | monotone in the on-disc metres, see the bracket table below |
| `Damage Class` | bucket of `Power` | 🟡 PROBABLE | monotone in the on-disc power, see below |
| `Weight Class` | bucket of the hangar-table `Weight` | ❔ HYPOTHESIS | only one clean pair so far (`wep_33`, Weight 0.3 → "Light") |
| `Speed Class` | ❔ | ❔ | present only on missiles (MPM = A, ASM = B); no on-disc pairing established |
| `Sight Homing`, `Lock on Overlap` | ❔ (`Available` / ``) | ❔ | the plausible on-disc partners (`Homing`, `OverlapLockon`) are identifier-valued and not yet decoded; **NEEDS-HUMAN** |
### Why the `Ammo Capacity` evidence is not circular
Each UI entry was **identified** by matching its `Ammo Capacity` against the records in
that weapon-type tab, so "the ammo matches" cannot on its own prove the mapping. What does:
1. **The match is forced and unique.** For 10 of the 11 entries, exactly one record in that
tab carries that number (`600` occurs three times overall — `wep_04`, `wep_24`, `wep_55`
— but in three different tabs: MPM, BEAM, B/R). An unrelated quantity would not land on
a valid, tab-unique `LoadingCount` eleven times running.
2. **A second, independent field then agrees.** FALCON 9AM and BUZZARD 10AM were pinned by
ammo alone, and their `Max. Lock Ons` (12, 22) then matched those same records'
`TriggerShotCount` (12, 22) — values that play no part in the identification. A wrong
identification would have to be wrong twice, consistently.
3. **One entry is identified without ammo at all.** STILETTO BG I is described in-game as
"the initially mounted Delta Saber beam gun" and is the weapon the HANGAR shows equipped
on the nose; `wep_01_Beam` is the corresponding record, and its `LoadingCount` 6000 is
what the panel shows.
The weakest identification is LIGHT MACHINE GUN MG I: three guns share `LoadingCount = 3000`
(`wep_09`, `wep_33`, `wep_83`). `wep_33` is picked on `Weight Class = Light` (it has by far
the smallest `Mass`, 1.5 vs 3.6 / 33.0) — 🟡 PROBABLE, not certain.
## Finding — recovered defaults
| Weapon | UI name | Field | On disc | **Runtime** | Conf. |
|---|---|---|---|---|---|
| `Weapon_DSaber_P_wep_05_ASMissile` | TERRIER SMH | `TriggerShotCount` | *(defaulted)* | **4** | 🟡 |
| `Weapon_DSaber_P_wep_60_ASMissile` | HOUND SMH | `TriggerShotCount` | *(defaulted)* | **4** | 🟡 |
Both defaulted weapons read **4**, which is consistent with a single title-code default of
`TriggerShotCount = 4` rather than two per-weapon constants — but two samples cannot tell
those apart. ❔ HYPOTHESIS: *the title-code default for `TriggerShotCount` is 4.* It would
be confirmed by a third weapon that defaults the field and also reads 4 (candidates that
were still locked in this save: `wep_27`, `wep_30`, and the seven Beams), or refuted by one
that reads anything else.
`Power` and `MaximumRange` defaults are **not** exactly recoverable from this panel — it
shows only the letter bucket. They are bracketed instead (below).
## Captured rows
All values are from one session on save slot 01 (Stage 02, "At Standby", 5 % clear, 4101 P);
`✓` marks a value that matches the on-disc record exactly.
| UI name | Record | Rng | Dmg | Spd | Weight | Ammo | Lock | Homing | Overlap | Hard point |
|---|---|---|---|---|---|---|---|---|---|---|
| LIGHT MACHINE GUN MG I | `wep_33_Gun` 🟡 | D | E | | Light | 3000 ✓ | | | | NOSE WEAPON (Nose) |
| BROAD SWORD SG I | *see below* | E | D | | Light | 200 | | | | NOSE WEAPON (Nose) |
| STILETTO BG I | `wep_01_Beam` | E | E | | Light | 6000 ✓ | | | | NOSE WEAPON (Nose) |
| DAGGER BG2 | `wep_37_Beam` | D | E | | Light | 4000 ✓ | | | | NOSE WEAPON (Nose) |
| PILUM BP | `wep_24_Beam` | B | D | | Light | 600 ✓ | | | | MAIN WEAPON1 (Fore) |
| FALCON 9AM | `wep_02_Missile` | D | D | A | Heavy | 300 ✓ | 12 ✓ | Available | | MAIN WEAPON1 (Fore) |
| BUZZARD 10AM | `wep_04_Missile` | C | D | A | Heavy | 600 ✓ | 22 ✓ | | Available | MAIN WEAPON1 (Fore) |
| DART 23 ROCKET | `wep_55_Rocket` | C | D | | Heavy | 600 ✓ | | | | MAIN WEAPON1 (Fore) |
| TERRIER SMH | `wep_05_ASMissile` | D | C | B | Heavy | 45 ✓ | **4** | Available | Available | MAIN WEAPON2 (Rear) |
| HOUND SMH | `wep_60_ASMissile` | B | C | B | Medium | 18 ✓ | **4** | Available | Available | MAIN WEAPON2 (Rear) |
| TOMAHAWK ALPHA RAIL GUN | `wep_03_Cannon` | C | C | | Medium | 75 ✓ | | | | MAIN WEAPON3 (Lower) |
**BROAD SWORD SG I is unidentified — NEEDS-HUMAN.** `Ammo Capacity 200` narrows it to
`wep_38` / `wep_41` / `wep_42_Shotgun` (all `LoadingCount = 200`); "SG" and the GUN tab fit
a shotgun. `wep_42` is excluded by range (2500 m would not share class E with `wep_38`/
`wep_41`'s 3000 m *if* the class is a pure range bucket), leaving `wep_38` vs `wep_41`,
which the panel cannot separate. Its `Damage Class D` also does not fit the `Power` bracket
below (all three shotguns are `Power ≤ 16`, i.e. bucket E), so either the identification or
the "Damage Class = bucket of `Power`" model is wrong for shotguns.
### Letter-class brackets
Sorting the identified rows by their on-disc numbers gives monotone, non-overlapping bands:
```
Range Class E: 3000 (wep_01)
D: 3500 · 4000 · 4000 · 4000 (wep_33, wep_37, wep_02, wep_05)
C: 4500 · 5000 · 5000 (wep_55, wep_04, wep_03)
B: 6500 · 6500 (wep_24, wep_60)
Damage Class E: 10 · 14 · 16 (wep_33, wep_01, wep_37)
D: 70 · 75 · 100 (wep_24, wep_04, wep_55)
C: 200 · 400 (wep_03, wep_05)
```
Two consequences for the reimplementation:
- The **thresholds are not pinned** — only bracketed (e.g. the D/C range boundary lies in
(4000, 4500]). More weapons, or a static read of the title-code table, would pin them.
- They **bracket the defaulted numbers**: `wep_02_Missile`'s defaulted `Power` sits in the
D band (≈ 17…150 by the observed edges) and `wep_60_ASMissile`'s in the C band
(≈ 150…500). 🟡 PROBABLE, and only as good as the bucket model.
## How this was captured
Container recipe (`sylph-container/mission.md`), with two corrections worth keeping:
- **Skip the intro movie with A.** The brief warns it crashes; it does not. Skipping cuts
boot-to-main-menu from ~13 min to **~1 min** under lavapipe. (Thanks: user tip.)
- **The title screen falls back to the attract loop within a few seconds**, so a
screenshot→look→tap cycle always misses it. Poll the framebuffer and tap in the same
process. Both are automated in `scratchpad/skip_intro.sh` (movie detected by frame-to-
frame RMSE; title by the green Ⓐ glyph at pixel 625,618).
- Under lavapipe the game polls input at its own low frame rate: **a 60 ms d-pad tap is
dropped roughly half the time**; 200 ms is reliable and 300 ms starts to auto-repeat.
- The Hangar hard-point weapon carousel is cycled with d-pad **down**, not left/right.
Path: title → A → LOAD GAME → slot 01 → *Load game?* **YES** → READY ROOM → ARSENAL →
type tab (LB/RB) → **Y** for the DATA SHEET → d-pad down through the list.
## Open / next
- Only **9 weapons of ~61** are revealed at 5 % completion, and none of the four whose
`LoadingCount` is defaulted (`wep_11`, `wep_28`, `wep_36`, `wep_70`) is among them.
Progressing the save (or a later save) is what unlocks the rest — the panel itself
already shows undeveloped weapons, so no points need to be spent.
- **Craft (`UNIT`-schema) defaults are not reachable this way.** The Hangar exposes exactly
one craft-level runtime number, `Gross Weight` (a class, "Light"). The defaulted craft
fields (`ShieldRatio`, `BarrelRoll_Count*`, `HoldPosition_*Ratio`, `Slalom_TurnCount_Max`,
`UsingChaffRatio`, …) are AI/flight-model constants with no UI surface; they would need
in-flight behavioural measurement or a guest-memory read, not a menu screenshot.
- The `Sight Homing` / `Lock on Overlap` on-disc partners are still unidentified.
Screenshots for every row above: `/sylph-home/re/caps/` in the container.
Evidence PNGs are committed under [`captures/weapon-datasheet/`](captures/weapon-datasheet/)
(64-colour quantized for size; the numbers stay legible).
---
# Addendum — the in-flight HUD (2026-07-28)
Reached via **TUTORIAL → BASIC CONTROLS / HEADS-UP DISPLAY** from the title menu (no
save needed). Evidence: [`captures/hud-runtime/`](captures/hud-runtime/).
## The HUD corroborates the ammo mapping, independently
The Delta Saber's HUD prints its two equipped weapons as `NOSE <TYPE> <AMMO>` and
`MAIN <TYPE> <AMMO>`. With the loadout known from the HANGAR (nose = STILETTO BG I, main =
FALCON 9AM) the HUD reads **`NOSE BM 06000`** and **`MAIN MPM 00300`** — exactly
`wep_01_Beam`'s `LoadingCount = 6000` and `wep_02_Missile`'s `300`.
This matters because it is **not** the Arsenal panel: the weapons were identified from the
HANGAR loadout, and the numbers come from a different renderer in a different game mode. It
is a genuinely independent confirmation of `Ammo Capacity == LoadingCount`.
The type tags (`BM`, `MPM`) are the same short codes as the Arsenal type tabs.
## HUD element inventory
| HUD element | Backing data | Conf. |
|---|---|---|
| `NOSE` / `MAIN` + type tag + 5-digit ammo | equipped weapon, `LoadingCount` | ✅ |
| `HEAT` / `L.HEAT` bar under each weapon | the `Heating` / `Cooling` pair | 🟡 |
| Speed readout + throttle scale (`0`, `100`, `A/B`) | craft velocity | ✅ |
| `SHIELD` and `ARMOR` bars (two separate pools) | craft `HP` + a shield pool | 🟡 |
| Target reticle: name / type / distance + ring **Armor Gauge** | target unit record | 🟡 |
| `RANGE` gauge with `MAIN` and `NOSE` tick markers | each equipped weapon's `MaximumRange`, plotted against target distance | 🟡 |
| `YOU KILLED: WARSHIPS nnnn` + a second counter | score / `ScorePoint` | ❔ |
The `RANGE` gauge is the interesting one for future work: it draws a **per-weapon marker on
a distance scale**, so a weapon whose `MaximumRange` is defaulted on disc (`wep_25`,
`wep_58`, `wep_82`) would have its value *drawn* rather than bucketed into a letter — if the
scale can be calibrated against two weapons with known ranges, that recovers a real number
where the Arsenal panel only gives a class.
## Craft velocity
Full afterburner (RT) peaked at **1193** against `UN_f001_TCAF_DeltaSaber_T`'s on-disc
`MaximumVelocity = 1200`. 🟡 PROBABLE — one observation, and the readout may have been
still climbing. Cruise sat at 350, which is *not* the record's `CruisingVelocity` (700), so
the number is the current throttle setting, not a named constant; don't read more into it.
## Notes for the next session
- The tutorials need **no save game** and are reachable in ~1 min from a cold boot, which
makes them the cheapest way back into a live flight scene.
- `tools/re-capture/autopilot.py` chases the yellow off-screen waypoint arrow (colour +
shape, `-sample` not `-resize`, ~0.65 s per detection). It **finds the arrow reliably but
oscillates** — the proportional gain is too high for the craft's turn rate. It needs a
damping/derivative term before it can actually fly a waypoint.
- The HEADS-UP DISPLAY tutorial reaches a scripted targeting segment where the ship stops
moving (target distance pinned) and the on-screen controller highlights **A**; tapping and
holding A did not advance it. **NEEDS-HUMAN**: what input that segment wants.
- Canary is unstable here: it died twice mid-session (once loading BEAM `Resource3D`, once
hanging on tutorial teardown) with no crash dump. Re-launch is cheap; just don't assume a
long session survives.

View File

@@ -0,0 +1,20 @@
# Runtime-capture harness (sylph-re container)
Screenshot-driven scripts for reading the running retail game's menus under Xenia
Canary + lavapipe, headless. They assume the container helpers `screenshot`,
`vgamepad`, `pad` are on `$PATH` and `HOME=/sylph-home/re`.
| Script | What it does |
|---|---|
| `skip_intro.sh` | Boot → main menu, unattended. Taps A only while the intro movie is actually playing (frame-to-frame RMSE), then once at the `PRESS Ⓐ BUTTON` title. Static logo screens are left alone, so a stray tap can never land on NEW GAME. |
| `wait_title.sh` | Older variant: wait for the title (green Ⓐ glyph at px 625,618) and tap A. Superseded by `skip_intro.sh`. |
| `step.sh` | One Arsenal navigation step (`down`/`up`/`next`/`prev`/`none`) + a compact capture: weapon list stacked over the `DATA SHEET`. |
| `sweep.sh` | Walk a whole weapon-type list, capturing **only** rows that show a `DATA SHEET` — locked rows (a "Conditions to Develop" panel) are detected by the brightness of the `Range Class` label box and skipped. |
| `type.sh` | Change weapon-type tab N times (RB) and report the header strip. |
| `hp.sh` / `cyc.sh` | Hangar hard-point carousel: `cyc.sh` steps it (d-pad **down**, not left/right) and captures the Name + `DATA SHEET`. |
**Input timing under lavapipe:** the game polls input at its own low frame rate, so a
60 ms d-pad tap is dropped roughly half the time. 200 ms is reliable; 300 ms starts to
auto-repeat (two rows per press).
Findings produced with these: [`docs/re/weapon-datasheet-runtime.md`](../../docs/re/weapon-datasheet-runtime.md).

View File

@@ -0,0 +1,95 @@
#!/usr/bin/env python3
"""Fly the Delta Saber toward the tutorial waypoint by chasing the yellow
off-screen direction arrow.
Detection: no PIL/numpy in the box, so the frame comes from `convert ... txt:-`.
Two things make it fast AND correct:
* `-sample` (point sampling, not `-resize`/`-scale` averaging) — a 1-px-thin
glyph keeps its true colour, so the exact colour predicate still fires while
the dump shrinks ~9x (3.5s -> 0.65s).
* shape, not just colour — the instruction-frame bars and the throttle marker
are the same dim yellow, so the arrow is the largest blob that is roughly as
tall as it is wide, with the fixed throttle marker blacklisted by position.
A ~1.1 s control loop is fast enough to converge; the earlier ~6 s loop was not.
"""
import re, subprocess, sys, time
X0, Y0, W, H = 80, 60, 820, 640 # flight view (the arrow also rides the bottom edge)
K = 3.03 # 1/0.33 sample factor
CX, CY = 640, 405 # crosshair
THROTTLE = (382, 456) # fixed yellow decoy on the speed gauge
PX = re.compile(r"^(\d+),(\d+):.*?srgb\((\d+),(\d+),(\d+)\)")
def pad(*a):
subprocess.run(["/opt/sylph/vgamepad.py", *a], check=False)
def arrow(png="/tmp/ap.png"):
subprocess.run(["/opt/sylph/screenshot.sh", png], check=False,
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
out = subprocess.run(["convert", png, "-crop", f"{W}x{H}+{X0}+{Y0}", "+repage",
"-sample", "33%", "txt:-"], capture_output=True, text=True).stdout
pts = set()
for l in out.splitlines()[1:]:
m = PX.match(l)
if not m:
continue
x, y, r, g, b = (int(v) for v in m.groups())
if r >= 60 and g >= 48 and b <= 0.35 * g and (r - g) <= 0.45 * r:
pts.add((x, y))
seen, best = set(), None
for p in pts:
if p in seen:
continue
st, c = [p], []
seen.add(p)
while st:
x, y = st.pop(); c.append((x, y))
for dx in (-1, 0, 1):
for dy in (-1, 0, 1):
q = (x + dx, y + dy)
if q in pts and q not in seen:
seen.add(q); st.append(q)
xs = [q[0] for q in c]; ys = [q[1] for q in c]
cx, cy = X0 + sum(xs) / len(c) * K, Y0 + sum(ys) / len(c) * K
if abs(cx - THROTTLE[0]) < 30 and abs(cy - THROTTLE[1]) < 30:
continue
if len(c) < 8:
continue
if best is None or len(c) > best[2]:
best = (cx, cy, len(c))
return best
def main():
steps = int(sys.argv[1]) if len(sys.argv) > 1 else 25
centred = 0
for i in range(steps):
a = arrow()
if a is None:
print(f"{i:2d}: no arrow -> boost (target ahead)")
pad("trig", "RT", "0.55"); time.sleep(1.2); pad("trig", "RT", "0")
centred += 1
if centred >= 6:
print(" arrival likely — stopping"); return 0
continue
x, y, n = a
dx, dy = x - CX, y - CY
if abs(dx) < 70 and abs(dy) < 70:
centred += 1
print(f"{i:2d}: arrow ({x:.0f},{y:.0f}) centred -> boost")
pad("trig", "RT", "0.55"); time.sleep(1.2); pad("trig", "RT", "0")
continue
centred = 0
lx = max(-1.0, min(1.0, dx / 200))
ly = max(-1.0, min(1.0, dy / 160))
print(f"{i:2d}: arrow ({x:.0f},{y:.0f}) n={n} -> LX {lx:+.2f} LY {ly:+.2f}")
pad("axis", "LX", f"{lx:.2f}"); pad("axis", "LY", f"{ly:.2f}")
time.sleep(0.45)
pad("axis", "LX", "0"); pad("axis", "LY", "0")
print("steps exhausted")
return 1
sys.exit(main())

9
tools/re-capture/cyc.sh Executable file
View File

@@ -0,0 +1,9 @@
#!/usr/bin/env bash
# Step the Hangar weapons-container carousel right and capture the Name+DATA SHEET.
set -u
export HOME=/sylph-home/re
OUT=/sylph-home/re/caps; mkdir -p "$OUT"
vgamepad dpad right; sleep 0.30; vgamepad dpad center; sleep 3.0
screenshot /tmp/h.png >/dev/null 2>&1
convert /tmp/h.png -crop 530x480+700+95 +repage "$OUT/$1.png"
echo "$OUT/$1.png"

168
tools/re-capture/gmem.py Normal file
View File

@@ -0,0 +1,168 @@
#!/usr/bin/env python3
"""Read the *live* guest memory of a running Xenia Canary.
Canary backs the whole guest address space with a single shared-memory file,
`/dev/shm/xenia_memory_<pid-ish>`, so the guest's RAM is readable from the host
with no debugger, no emulator patch and no pause: open the file, seek, read.
The file is one flat image of Xenia's *physical* backing store; guest virtual
addresses map into it through Xenia's fixed table (memory.cc `map_info`).
`va_to_off()` implements that table, so callers work in guest VAs.
Sub-commands
find <pattern> search every allocated extent, print guest VAs
read <va> [len] hexdump guest memory
words <va> [n] dump n big-endian u32 / f32 pairs
`<pattern>` is a python literal-ish string: plain text, or `hex:0011aabb`.
Numbers accept 0x form. The scan uses SEEK_DATA so the ~4.6 GB of sparse holes
cost nothing.
"""
import os
import re
import sys
import struct
# (guest_va_lo, guest_va_hi_inclusive, file_offset_of_lo) — Xenia memory.cc.
MAP = [
(0x00000000, 0x3FFFFFFF, 0x00000000),
(0x40000000, 0x7EFFFFFF, 0x40000000),
(0x7F000000, 0x7F0FFFFF, 0x00000000),
(0x7F100000, 0x7FFFFFFF, 0x00100000),
(0x80000000, 0x8FFFFFFF, 0x80000000),
(0x90000000, 0x9FFFFFFF, 0x80000000),
(0xA0000000, 0xBFFFFFFF, 0x100000000),
(0xC0000000, 0xDFFFFFFF, 0x100000000),
(0xE0000000, 0xFFFFFFFF, 0x100000000),
]
def va_to_off(va):
for lo, hi, base in MAP:
if lo <= va <= hi:
return base + (va - lo)
raise ValueError(f"va {va:#x} outside the guest map")
def off_to_vas(off):
"""All guest VAs that alias this file offset (the map is many-to-one)."""
out = []
for lo, hi, base in MAP:
if base <= off <= base + (hi - lo):
out.append(lo + (off - base))
return out
def primary_va(off):
"""The most useful VA for an offset: physical 0xA0000000+ / xex 0x80000000+."""
vas = off_to_vas(off)
return vas[0] if vas else None
def mem_path():
# A snapshot (`cp --sparse=always /dev/shm/xenia_memory_* snap.bin`, ~2 s)
# reads identically and does not contend with the running emulator, which
# pegs every core under lavapipe. Point $GMEM_FILE at one to work offline.
env = os.environ.get("GMEM_FILE")
if env:
return env
if len(sys.argv) > 1 and sys.argv[1].startswith("/dev/shm/"):
return sys.argv.pop(1)
cands = [f"/dev/shm/{n}" for n in os.listdir("/dev/shm") if n.startswith("xenia_memory_")]
if not cands:
sys.exit("no /dev/shm/xenia_memory_* — is Canary running?")
if len(cands) > 1:
sys.exit(f"several memory files, pass one explicitly: {cands}")
return cands[0]
def extents(fd, size):
"""Yield (start, end) of the file's allocated (non-hole) ranges."""
pos = 0
while pos < size:
try:
data = os.lseek(fd, pos, os.SEEK_DATA)
except OSError:
return
try:
hole = os.lseek(fd, data, os.SEEK_HOLE)
except OSError:
hole = size
yield (data, hole)
pos = hole
def parse_pattern(s):
if s.startswith("hex:"):
return bytes.fromhex(s[4:])
return s.encode("latin-1")
def cmd_find(f, size, args):
pat = parse_pattern(args[0])
limit = int(args[1]) if len(args) > 1 else 64
hits = 0
CHUNK = 1 << 24
for start, end in extents(f.fileno(), size):
pos = start
carry = b""
carry_at = start
while pos < end:
f.seek(pos)
buf = f.read(min(CHUNK, end - pos))
if not buf:
break
blob = carry + buf
base = carry_at
for m in re.finditer(re.escape(pat), blob):
off = base + m.start()
va = primary_va(off)
print(f"{off:#013x} va {va:#010x}" if va is not None else f"{off:#013x} va ?")
hits += 1
if hits >= limit:
return
keep = len(pat) - 1
carry = blob[-keep:] if keep else b""
carry_at = base + len(blob) - len(carry)
pos += len(buf)
if hits == 0:
print("(no hits)", file=sys.stderr)
def cmd_read(f, size, args):
va = int(args[0], 0)
n = int(args[1], 0) if len(args) > 1 else 256
f.seek(va_to_off(va))
data = f.read(n)
for i in range(0, len(data), 16):
row = data[i : i + 16]
hexs = " ".join(f"{b:02x}" for b in row)
txt = "".join(chr(b) if 0x20 <= b < 0x7F else "." for b in row)
print(f"{va + i:08x} {hexs:<47} |{txt}|")
def cmd_words(f, size, args):
va = int(args[0], 0)
n = int(args[1], 0) if len(args) > 1 else 32
f.seek(va_to_off(va))
data = f.read(n * 4)
for i in range(0, len(data) - 3, 4):
(u,) = struct.unpack_from(">I", data, i)
(fl,) = struct.unpack_from(">f", data, i)
fs = f"{fl:.6g}" if -1e30 < fl < 1e30 else ""
print(f"{va + i:08x} +{i:04x} {u:#010x} {u:>12} {fs}")
def main():
path = mem_path()
if len(sys.argv) < 2:
sys.exit(__doc__)
cmd, args = sys.argv[1], sys.argv[2:]
size = os.path.getsize(path)
with open(path, "rb") as f:
{"find": cmd_find, "read": cmd_read, "words": cmd_words}[cmd](f, size, args)
if __name__ == "__main__":
main()

11
tools/re-capture/hp.sh Executable file
View File

@@ -0,0 +1,11 @@
#!/usr/bin/env bash
# Cycle the Hangar hard-point weapon carousel one step right and capture the
# Name + DATA SHEET panel (the mountable-weapon list for that hard point).
set -u
export HOME=/sylph-home/re
OUT=/sylph-home/re/caps; mkdir -p "$OUT"
[ "${1:-}" = "none" ] || { vgamepad dpad right; sleep 0.20; vgamepad dpad center; }
sleep 2.5
screenshot /tmp/h.png >/dev/null 2>&1
convert /tmp/h.png -crop 495x460+705+95 +repage "$OUT/$2.png"
echo "$OUT/$2.png"

View File

@@ -0,0 +1,41 @@
#!/usr/bin/env python3
"""Parse a RATC bundle's header SPRITE DECLARATION TABLE: u32 count at 0x14, then
fixed 60-byte entries of [name, NUL-padded | 4 u32 flags | pivotX | pivotY | 0].
Check each pivot against half the decoded texture's real dimensions."""
import struct, sys, glob, re, os
def decls(d):
n = struct.unpack_from(">I", d, 0x14)[0]
out = []
for i in range(n):
o = 0x20 + i * 60
if o + 60 > len(d): break
name = d[o:o+28].split(b"\0")[0].decode("ascii", "replace")
px, py = struct.unpack_from(">II", d, o + 48)
out.append((name, px, py))
return n, out
def texmap(prefix):
t = {}
for f in glob.glob(f"pause-tex/{prefix}_*.png"):
m = re.match(rf".*/{prefix}_(.+)\.t32_(\d+)x(\d+)\.png", f)
if m: t[m.group(1) + ".t32"] = (int(m.group(2)), int(m.group(3)))
return t
for path, prefix in [(sys.argv[1], sys.argv[2])]:
d = open(path, "rb").read()
n, ds = decls(d)
tm = texmap(prefix)
print(f"{os.path.basename(path)}: count={n}, parsed={len(ds)}")
ok = bad = miss = 0
for name, px, py in ds:
if name in tm:
w, h = tm[name]
hit = (px == w // 2 and py == h // 2)
ok, bad = ok + hit, bad + (not hit)
flag = "OK " if hit else "MISMATCH"
print(f" {flag} {name:28s} tex {w:4d}x{h:<4d} half {w//2:4d},{h//2:<4d} decl {px:4d},{py:<4d}")
else:
miss += 1
print(f" ? {name:28s} (no decoded texture) decl {px:4d},{py:<4d}")
print(f" => pivot == half(texture): {ok} ok, {bad} mismatch, {miss} unchecked")

27
tools/re-capture/skip_intro.sh Executable file
View File

@@ -0,0 +1,27 @@
#!/usr/bin/env bash
# Drive the boot sequence to the main menu without a human in the loop.
# - movie playing (two grabs 0.6s apart differ a lot) -> tap A to skip
# - static screen -> if the green "PRESS (A) BUTTON" glyph is there, tap A and stop
# Static logo screens are left alone, so a stray tap can never land on NEW GAME.
set -u
export HOME=/sylph-home/re
# The X root keeps the DEAD session's last frame, so a fresh launch would be
# detected as "already at the title". Blank it, and wait for the new window.
xsetroot -solid black 2>/dev/null || true
until xdotool search --name "Xenia-canary" >/dev/null 2>&1; do sleep 1; done
deadline=$(( SECONDS + ${1:-900} ))
while [ $SECONDS -lt $deadline ]; do
screenshot /tmp/f1.png >/dev/null 2>&1; sleep 0.6
screenshot /tmp/f2.png >/dev/null 2>&1
d=$(compare -metric RMSE /tmp/f1.png /tmp/f2.png null: 2>&1 | sed 's/ .*//' | cut -d. -f1)
d=${d:-0}
read -r r g b < <(convert /tmp/f2.png -format "%[fx:int(255*p{625,618}.r)] %[fx:int(255*p{625,618}.g)] %[fx:int(255*p{625,618}.b)]" info:)
if [ "$g" -gt 130 ] && [ $((g - r)) -gt 45 ] && [ $((g - b)) -gt 45 ]; then
echo "TITLE at ${SECONDS}s -> A"; vgamepad tap A 250; exit 0
fi
if [ "$d" -gt 1500 ]; then
echo "movie (rmse $d) at ${SECONDS}s -> skip A"; vgamepad tap A 250; sleep 3
fi
sleep 1
done
echo "TIMEOUT"; exit 1

22
tools/re-capture/step.sh Executable file
View File

@@ -0,0 +1,22 @@
#!/usr/bin/env bash
# One Arsenal navigation step + a compact capture: the weapon list (which row is
# selected) stacked over the DATA SHEET numeric rows. Everything else is dropped.
# Usage: step.sh <down|up|next|prev|none> <tag> [settle_s]
set -u
export HOME=/sylph-home/re
OUT=/sylph-home/re/caps; mkdir -p "$OUT"
case "${1}" in
down) vgamepad dpad down; sleep 0.20; vgamepad dpad center ;;
up) vgamepad dpad up; sleep 0.20; vgamepad dpad center ;;
next) vgamepad hold RB 0.35 ;;
prev) vgamepad hold LB 0.35 ;;
none) : ;;
esac
sleep "${3:-2.5}"
raw="$OUT/$2.raw.png"
screenshot "$raw" >/dev/null 2>&1
convert "$raw" -crop 500x420+150+180 +repage /tmp/_list.png
convert "$raw" -crop 500x200+700+100 +repage /tmp/_sheet.png
convert /tmp/_list.png /tmp/_sheet.png -background black -append "$OUT/$2.png"
rm -f "$raw"
echo "$OUT/$2.png"

21
tools/re-capture/sweep.sh Executable file
View File

@@ -0,0 +1,21 @@
#!/usr/bin/env bash
# Walk down a weapon-type list N rows; capture ONLY rows that show a DATA SHEET
# (locked rows show a "Conditions to Develop" panel instead — detected by the
# brightness of the "Range Class" label box, ~5 when absent, >>20 when present).
set -u
export HOME=/sylph-home/re
OUT=/sylph-home/re/caps; mkdir -p "$OUT"
pfx=$1; n=${2:-8}
for ((i=1;i<=n;i++)); do
vgamepad dpad down; sleep 0.20; vgamepad dpad center; sleep 3.0
screenshot /tmp/s.png >/dev/null 2>&1
m=$(convert /tmp/s.png -crop 150x24+730+116 +repage -colorspace Gray -format "%[fx:int(255*mean)]" info:)
if [ "$m" -gt 45 ]; then
convert /tmp/s.png -crop 500x420+150+180 +repage /tmp/_l.png
convert /tmp/s.png -crop 500x200+700+100 +repage /tmp/_s.png
convert /tmp/_l.png /tmp/_s.png -background black -append "$OUT/$pfx-r$i.png"
echo "row $i: DATA SHEET -> $OUT/$pfx-r$i.png"
else
echo "row $i: locked (mean $m)"
fi
done

11
tools/re-capture/type.sh Executable file
View File

@@ -0,0 +1,11 @@
#!/usr/bin/env bash
# Change Arsenal weapon type N times (RB) and report the header strip only.
set -u
export HOME=/sylph-home/re
OUT=/sylph-home/re/caps; mkdir -p "$OUT"
n=${1:-1}
for ((i=0;i<n;i++)); do vgamepad hold RB 0.35; sleep 2.5; done
sleep 1.5
screenshot /tmp/hdr.png >/dev/null 2>&1
convert /tmp/hdr.png -crop 420x50+130+148 +repage "$OUT/hdr.png"
echo "$OUT/hdr.png"

21
tools/re-capture/wait_title.sh Executable file
View File

@@ -0,0 +1,21 @@
#!/usr/bin/env bash
# Poll the framebuffer for the "PRESS (A) BUTTON" title screen (the green A glyph
# at ~625,618) and tap A the instant it appears — the title auto-returns to the
# attract loop after a few seconds, which is why a human-paced tap misses it.
set -u
export HOME=/sylph-home/re
TMP=/tmp/title-probe.png
deadline=$(( SECONDS + ${1:-900} ))
while [ $SECONDS -lt $deadline ]; do
if screenshot "$TMP" >/dev/null 2>&1; then
read -r r g b < <(convert "$TMP" -format "%[fx:int(255*p{625,618}.r)] %[fx:int(255*p{625,618}.g)] %[fx:int(255*p{625,618}.b)]" info:)
if [ "$g" -gt 130 ] && [ $((g - r)) -gt 45 ] && [ $((g - b)) -gt 45 ]; then
echo "TITLE detected (rgb $r,$g,$b) at ${SECONDS}s — tapping A"
vgamepad tap A 200
exit 0
fi
fi
sleep 1
done
echo "TIMEOUT: title not seen"
exit 1

View File

@@ -0,0 +1,303 @@
#!/usr/bin/env python3
"""Solve the runtime `Weapon`/`Shell` struct layouts against disc ground truth,
then read the fields the disc leaves defaulted.
The game parses every `weapon\\Weapon_*.tbl` IDXD record into two C++ objects, a
`Weapon` and its `Shell`. Each class lives in its own contiguous array in guest
RAM and is identifiable by its vtable pointer. Fields the IDXD omits (the
Route-B "defaulted" list) still hold their real values in those objects — put
there by the constructor before the parser overwrites what the disc supplies.
Method (no guessing):
1. read every disc record's explicitly-valued fields, split per sub-record
(`sylpheed-formats --example idxd_tokens`);
2. read every runtime object out of the live emulator (`gmem`), keyed by the
object's own ID string;
3. for each (field, byte-offset, encoding) triple, count how many weapons the
offset reproduces and how many it contradicts. An offset that agrees on
many and contradicts none is a confirmed binding;
4. print the map, then the values at those offsets for the weapons whose disc
record omits the field.
Step 3 is the whole safeguard: a wrong offset cannot silently agree with ~100
independent records.
Usage: weapon_runtime.py <tokens.txt> [--md]
"""
import math
import os
import re
import struct
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import gmem # noqa: E402
# vtable VA -> (class name, object stride)
CLASSES = {
0x820AF548: ("Weapon", 0xC0),
0x820AF58C: ("Shell", 0x200),
}
NAME_STR_OFF = 0x10 # the name string sits +0x10 into a name record
# ---------------------------------------------------------------- disc side
def read_tokens(path):
"""({class: {ID: {field: value}}}, {(class, ID): [field...]}) from the dump.
A handful of IDs are declared by more than one `.tbl`. Where two such
declarations disagree on a field, that field is *ambiguous on disc* — the
runtime holds whichever definition won, so it must not be scored as a
contradiction. Those are collected separately, not silently merged.
"""
seen, ambiguous = {}, {}
cur = None
for line in open(path):
if line.startswith("REC "):
_, _, _, cls, rid = line.split()
cur = (cls, rid)
seen.setdefault(cur, [])
seen[cur].append({})
elif line.startswith("F ") and cur is not None:
_, k, v = line.rstrip("\n").split(" ", 2)
seen[cur][-1][k] = v
out = {}
for (cls, rid), defs in seen.items():
merged = {}
for d in defs:
merged.update(d)
if len(defs) > 1:
bad = {
k
for k in {k for d in defs for k in d}
if len({d.get(k) for d in defs}) > 1
}
if bad:
ambiguous[(cls, rid)] = sorted(bad)
for k in bad:
merged.pop(k, None)
out.setdefault(cls, {})[rid] = merged
return out, ambiguous
# ------------------------------------------------------------- runtime side
def scan_vtable(f, size, vt):
pat = struct.pack(">I", vt)
hits = []
for a, b in gmem.extents(f.fileno(), size):
f.seek(a)
blob = f.read(b - a)
for m in re.finditer(re.escape(pat), blob):
off = a + m.start()
if off % 4 == 0:
hits.append(off)
return sorted(set(hits))
def runtime_objects(f, size):
"""{class: {ID: raw_bytes}} plus {class: [(va, ID)]} in array order."""
objs, order = {}, {}
for vt, (cls, stride) in CLASSES.items():
objs[cls], order[cls] = {}, []
for off in scan_vtable(f, size, vt):
f.seek(off)
raw = f.read(stride)
(nameptr,) = struct.unpack_from(">I", raw, 4)
try:
f.seek(gmem.va_to_off(nameptr + NAME_STR_OFF))
name = f.read(64).split(b"\0")[0].decode("latin-1")
except (ValueError, UnicodeDecodeError):
continue
objs[cls][name] = raw
order[cls].append((gmem.primary_va(off), name))
return objs, order
# ------------------------------------------------------------------- solver
ENC = {
"f32": lambda raw, o, ch: struct.unpack_from(">f", raw, o)[0],
"u32": lambda raw, o, ch: float(struct.unpack_from(">I", raw, o)[0]),
"deg": lambda raw, o, ch: math.degrees(struct.unpack_from(">f", raw, o)[0]),
# A charging weapon drains its magazine continuously, so the two counter
# fields are float32 on `IsCharging = Yes` records and int32 on the rest.
# Discovered from the runtime: `IsCharging` partitions the 8 float-encoded
# records exactly (see docs/re/weapon-struct-runtime.md).
"cnt": lambda raw, o, ch: (
struct.unpack_from(">f", raw, o)[0] if ch else float(struct.unpack_from(">I", raw, o)[0])
),
}
def near(a, b):
if a == b:
return True
return abs(a - b) <= 2e-4 * max(abs(a), abs(b), 1e-6)
def solve(disc_cls, run_cls, stride, charging):
"""{field: (off, enc, n_agree, [mismatch...])}"""
numeric = {}
for rid, fields in disc_cls.items():
if rid not in run_cls:
continue
for k, v in fields.items():
try:
numeric.setdefault(k, {})[rid] = float(v.rstrip("fF"))
except ValueError:
pass
solved = {}
for field, samples in numeric.items():
if len(samples) < 2:
continue
cands = []
for off in range(0, stride - 3, 4):
for enc, fn in ENC.items():
agree, bad = 0, []
for rid, want in samples.items():
got = fn(run_cls[rid], off, rid in charging)
if math.isfinite(got) and near(got, want):
agree += 1
else:
bad.append((rid, want, got))
if agree >= 2:
cands.append((len(bad), -agree, off, enc, agree, bad))
if not cands:
continue
cands.sort()
nbad, _, off, enc, agree, bad = cands[0]
# A binding that contradicts a large slice of the evidence is not a
# binding at all -- it is a coincidence on the agreeing subset.
if len(bad) > 0.2 * len(samples):
continue
# Discriminating power: a field whose on-disc samples are all the same
# value matches any offset holding that constant, so such a binding is
# a coincidence waiting to happen. Count the distinct values that the
# *agreeing* records pin down.
distinct = len({round(v, 6) for rid, v in samples.items() if rid not in {b[0] for b in bad}})
solved[field] = (off, enc, agree, bad, distinct)
# Two fields cannot share one byte offset. Where the solver lands two on the
# same (offset, enc), keep the better-evidenced one and drop the other.
best_at = {}
for field, (off, enc, agree, bad, distinct) in solved.items():
key = (off, enc)
score = (distinct, agree, -len(bad))
if key not in best_at or score > best_at[key][1]:
best_at[key] = (field, score)
winners = {f for f, _ in best_at.values()}
return {f: v for f, v in solved.items() if f in winners}, numeric
def report(cls, disc_cls, run_cls, stride, out, charging):
solved, numeric = solve(disc_cls, run_cls, stride, charging)
p = out.append
p(f"\n### `{cls}` — runtime struct ({stride:#x} bytes/object)\n")
p("Confidence: ✅ = ≥10 disc records agree on ≥3 distinct values, none contradict.")
p("🟡 = consistent but thin evidence. ⚠️ = the runtime contradicts the disc "
"(listed below the table).\n")
p("| offset | enc | field | agree | distinct values | contradict | conf |")
p("|--------|-----|-------|------:|----------------:|-----------:|------|")
contradictions = []
for field, (off, enc, agree, bad, distinct) in sorted(solved.items(), key=lambda kv: kv[1][0]):
if bad:
conf = "⚠️"
contradictions.append((field, off, enc, bad))
elif agree >= 10 and distinct >= 3:
conf = ""
else:
conf = "🟡"
p(f"| `+{off:#05x}` | {enc} | `{field}` | {agree} | {distinct} | {len(bad)} | {conf} |")
unsolved = sorted(set(numeric) - set(solved))
if unsolved:
p(f"\nNumeric fields with no consistent offset (unsolved): "
f"{', '.join('`'+u+'`' for u in unsolved)}")
for field, off, enc, bad in contradictions:
p(f"\n**`{field}` (`+{off:#05x}`) — runtime disagrees with disc on "
f"{len(bad)} record(s):**\n")
for rid, want, got in sorted(bad)[:24]:
p(f"- `{rid}`: disc `{want:g}` → runtime `{got:g}`")
p(f"\n#### `{cls}` values the disc defaults, read from the live objects\n")
for field, (off, enc, agree, bad, distinct) in sorted(solved.items(), key=lambda kv: kv[1][0]):
if bad or agree < 10 or distinct < 3:
continue # only report from ✅ bindings
missing = [
(rid, ENC[enc](run_cls[rid], off, rid in charging))
for rid in sorted(disc_cls)
if rid in run_cls and field not in disc_cls[rid]
]
if not missing:
continue
vals = sorted({round(v, 6) for _, v in missing})
p(f"\n**`{field}`** (`+{off:#05x}`, {enc}) — defaulted by {len(missing)} of "
f"{len(run_cls)} records")
if len(vals) == 1:
p(f"\n> all = **{vals[0]:g}**")
else:
p("")
for rid, v in missing:
p(f"- `{rid}` = **{v:g}**")
return solved
def main():
tokens = sys.argv[1] if len(sys.argv) > 1 else "/tmp/wep_tokens.txt"
disc, ambiguous = read_tokens(tokens)
path = gmem.mem_path()
size = os.path.getsize(path)
with open(path, "rb") as f:
objs, order = runtime_objects(f, size)
# `IsCharging = Yes` switches the two counter fields to float32; the Shell
# inherits the flag from its Weapon (same ID suffix).
charging = {
rid for rid, f in disc.get("Weapon", {}).items() if f.get("IsCharging") == "Yes"
}
charging |= {"Shell" + rid[len("Weapon"):] for rid in set(charging)}
out = []
for cls, stride in [(c, s) for c, s in CLASSES.values()]:
d, r = disc.get(cls, {}), objs.get(cls, {})
matched = set(d) & set(r)
out.append(f"\n<!-- {cls}: {len(r)} runtime objects, {len(d)} disc records, "
f"{len(matched)} matched by ID -->")
report(cls, d, r, stride, out, charging)
if "--csv" in sys.argv:
import csv
w = csv.writer(sys.stdout)
w.writerow(["class", "id", "field", "offset", "enc", "value", "source", "conf"])
for cls, stride in [(c, st) for c, st in CLASSES.values()]:
d, r = disc.get(cls, {}), objs.get(cls, {})
solved, _ = solve(d, r, stride, charging)
for field, (off, enc, agree, bad, distinct) in sorted(
solved.items(), key=lambda kv: kv[1][0]
):
conf = "confirmed" if (not bad and agree >= 10 and distinct >= 3) else "tentative"
for rid in sorted(r):
val = ENC[enc](r[rid], off, rid in charging)
src = "disc" if field in d.get(rid, {}) else "defaulted-on-disc"
w.writerow([cls, rid, field, f"{off:#05x}", enc, f"{val:g}", src, conf])
return
if ambiguous:
out.append("\n### Disc records declared twice, with conflicting values\n")
out.append("These fields are ambiguous *on disc*; the runtime shows which "
"declaration won. They are excluded from the scoring above.\n")
for (cls, rid), fields in sorted(ambiguous.items()):
out.append(f"- `{cls}` `{rid}`: {', '.join('`'+f+'`' for f in fields)}")
print("\n".join(out))
if __name__ == "__main__":
main()