6 Commits

Author SHA1 Message Date
530555de9f re: the weapons are the control — sibling-default inheritance is unit-schema-specific, not engine-wide
Four rules from 21 units invites the coincidence objection, so run the identical
sweep against the Weapon/Shell capture, which has COMPLETE coverage (126
records). It finds no sibling rule at all: the one 100%-agreement candidate has a
single distinct value and is really a constant default. Weapon defaults vary per
record exactly as unit defaults do, so "defaults are computed" is general while
"defaults come from a sibling field" is not.

Size_Y <- Size_X survives, and is now checked at the raw-token level rather than
through the sub-record merge: e105, f105 and f101 each declare Size_X/Size_Z/
Size_Radius and no Size_Y, and each reads back its own Size_X at runtime. The
two two-unit hypotheses are demoted to coincidence-not-excluded.

Also records a negative for planning: Stage 01, the only other reachable stage,
adds four uncaptured units that are variants of already-captured ones, so it
would re-measure rather than test the rules.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 20:09:14 +00:00
69b4a2e569 re: a unit field left unset on disc is not a global constant — Size_Y inherits Size_X
Attacks the 21/110 unit-coverage limit from the cheap side: if a defaulted field
always took one runtime value, the captured units would pin it for all 110. Only
6 of 24 confirmed defaulted fields behave that way. The other 18 vary per unit,
so the default is computed.

Asking which OTHER field of the same unit holds that value — counting only
non-zero cases, and checking the two fields sit at different offsets so the
layout solver cannot be aliasing them — gives four rules. Size_Y <- Size_X is
solid: seven unrelated ships (e105 600, e106 300, e108 80, e201 300, f101 400,
f105 700, f106 200) each omit it on disc and each shows its own Size_X live,
while the two fields differ freely when both are on disc. Size_Radius fits both
min(X,Z) and the median of the three axes and cannot yet be separated;
e010_ADAN_Attacker_S is what rules out the simpler Size_X rule. FCSRange and
DefencePoint rest on two independent units each and ship as HYPOTHESIS.

Applied across the disc the rules recover 65 (unit, field) values in units that
have never been visited. Falsification test recorded: load any uncaptured stage
and compare one predicted value.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 19:44:24 +00:00
ca500c171e ship: index-less brg/eng/sld parts never matched their GN frame — 34 ships assembled without a bridge
Tier 3 matched a part to its hardpoint by trailing index, so `e105_brg`
compared "01" == "" against GN_Bridge_01 and fell through silently. The runtime
capture is what exposed it: the game draws the bridge and places it at
[0, 70, -1850] rel e105_bdy_01, and assemble_ship emitted nothing there.

With no index to match on, take the lowest-numbered frame of the category.
Diffing assemble_ship part counts across every container: 34 (stage, ship)
entries gain parts — e102 +2 (bridge and engine), e104 +1, e105 +1, Stages
02-29. ship_audit is unchanged, so nothing regressed, and the capture now agrees
to dT 0.03 / dR 0.000.

Also fixes the diff itself: correlate_frames compared static against a rotation
sampled from the first block, which can belong to another INSTANCE of the class.
Scoped to the position-agreeing cluster, e105_eng_01 goes 1.711 -> 0.000 and
both e106 nacelles to 0.000. The one remaining rotation delta (e106_wep_02_01,
0.134) is a turret whose rotation varies by 0.182 between blocks that agree on
its position — the runtime disagrees with itself more than with the assembler.
The new rotVar column makes that distinction visible.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 19:16:06 +00:00
3d9f21f030 re: control the range, segment the frames — f105/e105/e106 all match static assembly
A capture at controlled range (ship_capture_close.sh: lock a capital ship, close
on it, F10 per range band) finally draws capital-ship hulls at full detail. Two
correctness fixes were needed before the numbers meant anything:

* one F10 log is ~14 frames with no delimiter, and WV_ref^-1 . WV_p only cancels
  the camera within one frame — segment_frames splits on vertex-buffer
  recurrence, and correlate_frames cross-checks the blocks against each other
  instead of trusting a single shot;
* aggregate by consensus, not median: a stage holds several ships of one class
  sharing vertex buffers, so a block can mix two instances.

Result: f105, e105 and e106 reproduce assemble_ship to <=0.43 units in
translation and 0.000 in rotation for every part that does not move. The e106
rules generalise, and the viewer bug report now points at the viewer. Narrow
leftovers: e105_brg is missing from assemble_ship, e105_eng_01 rotation differs
by 1.711.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 18:19:26 +00:00
1d4b35df0f re: the Stage-02 capture drew no capital ship at all — invert the match, then control range
Inverting the capture↔part question (invert_capture over one container,
vcount_index over all 166) identifies every large draw in the 2026-07-31
capture: the player's own DeltaSaber (10891 verts), its weapon packs, the
backdrop and particles. Of f101/e105/e106 only 1-3 of 15-37 resources have a
drawn vcount, each a 44-225-vertex far-LOD/effect piece whose count collides
with dozens of unrelated resources.

So the zero-correlation was not an LOD-list gap, not over-strict position
validation and not a different draw path: the ships were too far away to be
drawn. approach_capture.py flies at a locked capital ship and presses F10 per
range band, stamping each capture with its distance.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 18:04:58 +00:00
Claude (auto-RE)
bbfeb1c387 re: ship-placement generalisation — static audit + first Stage-02 F10 capture (WIP)
Working the BACKLOG item "capital ships assemble wrong in the viewer".

- ship_audit over all 22 stage containers: only ONE outlier ship
  (f002_bdy_05, Stage_S03/S27, dist 6540 vs cluster spread 1071), so static
  assembly is not grossly broken class-wide. Recurring MULTIKEY joint tracks on
  f104/f105/f106/e102 are the standing hypothesis for class-specific error —
  e106, the one validated ship, has none.
- new tools/re-capture/ship_capture_session.sh: one blocking session that boots
  Stage 02 and fires N F10 ship-captures with screenshots. Boot to in-flight was
  24 s; 3 of 5 presses produced logs (2964/3111/3668 draws).
- NEGATIVE, unexplained: correlate_capture matched ZERO parts for f101/f105/
  f106/e105. Documented with the collected facts and the next step (invert the
  match: largest capture vcounts -> which decoded part has that count).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 05:16:20 +00:00
14 changed files with 1162 additions and 5 deletions

View File

@@ -0,0 +1,280 @@
//! Correlate a capture **per frame** and cross-check the frames against each
//! other — the placement is only believable if independent frames agree.
//!
//! `correlate_capture` treats one capture log as one set of draws. It is not:
//! an F10 press dumps ~14 frames with no delimiter, and `WV_ref⁻¹ · WV_p` only
//! cancels the camera within a single frame (see
//! [`sylpheed_formats::ship_capture::segment_frames`]). With a moving camera the
//! mixed-frame answer is wrong, and — worse — it is wrong *silently*.
//!
//! So: segment, correlate each frame independently, then report per part the
//! median translation and the spread across frames. A part whose spread is a
//! few units is measured; a part that swings by hundreds is not, whatever the
//! single-shot number said.
//!
//! Usage:
//! SYLPHEED_ISO=... cargo run --release --example correlate_frames -- \
//! <capture.log> <Stage_SNN> <ship_id> [ref_part_substr] [--min-parts N]
use sylpheed_formats::mesh::{xbg7_resource_names, Xbg7Model};
use sylpheed_formats::ship::{is_base_part, ship_id_of};
use sylpheed_formats::ship_capture::{
correlate, parse_capture, parse_drawlog, segment_frames, PartKey,
};
use sylpheed_formats::xiso::open_iso;
use std::collections::{BTreeMap, HashSet};
use std::path::Path;
fn median(mut v: Vec<f32>) -> f32 {
v.sort_by(|a, b| a.partial_cmp(b).unwrap());
let n = v.len();
if n % 2 == 1 { v[n / 2] } else { 0.5 * (v[n / 2 - 1] + v[n / 2]) }
}
fn main() {
let args: Vec<String> = std::env::args().collect();
let positional: Vec<&String> = args[1..].iter().filter(|a| !a.starts_with("--")).collect();
if positional.len() < 3 {
eprintln!("usage: correlate_frames <capture.log> <Stage_SNN> <ship_id> [ref_part] [--min-parts N]");
std::process::exit(2);
}
let (log, stage, id) = (positional[0], positional[1], positional[2]);
let ref_sub = positional.get(3).map(|s| s.as_str()).unwrap_or("bdy_01");
let min_parts: usize = args
.iter()
.position(|a| a == "--min-parts")
.and_then(|i| args.get(i + 1))
.and_then(|s| s.parse().ok())
.unwrap_or(3);
let iso = std::env::var("SYLPHEED_ISO").expect("SYLPHEED_ISO");
let text = std::fs::read_to_string(log).expect("read log");
let mut draws = parse_capture(&text);
if draws.is_empty() {
draws = parse_drawlog(&text);
}
let frames = segment_frames(&draws);
println!("{} draws → {} camera-consistent blocks", draws.len(), frames.len());
let bytes = {
let rt = tokio::runtime::Builder::new_current_thread().enable_all().build().unwrap();
rt.block_on(async {
let mut r = open_iso(Path::new(&iso)).await.unwrap();
r.read_file(&format!("hidden/resource3d/{stage}.xpr")).await.unwrap()
})
};
let names = xbg7_resource_names(&bytes);
let base_parts: Vec<String> = names
.iter()
.filter(|n| is_base_part(n) && ship_id_of(n) == Some(id.as_str()))
.cloned()
.collect();
let mut want: HashSet<String> = base_parts.iter().cloned().collect();
for p in &base_parts {
for suf in ["_m", "_l", "_d"] {
let c = format!("{p}{suf}");
if names.contains(&c) {
want.insert(c);
}
}
}
let models = Xbg7Model::models_named(&bytes, &want, &|| false);
let positions_of = |name: &str| -> Option<Vec<[f32; 3]>> {
let m = models.iter().find(|m| m.name == name)?;
Some(m.meshes.iter().flat_map(|s| s.positions.iter().copied()).collect())
};
// part -> [T per frame], and how many frames placed it at all.
let mut samples: BTreeMap<String, Vec<[f32; 3]>> = BTreeMap::new();
let mut rots: BTreeMap<String, Vec<[[f32; 3]; 3]>> = BTreeMap::new();
let mut used_frames = 0usize;
for (fi, fr) in frames.iter().enumerate() {
let mut keys: Vec<PartKey> = Vec::new();
for part in &base_parts {
let variants =
[part.clone(), format!("{part}_m"), format!("{part}_l"), format!("{part}_d")];
let union: Vec<[f32; 3]> =
variants.iter().filter_map(|v| positions_of(v)).flatten().collect();
for cand in &variants {
if let Some(pos) = positions_of(cand) {
let vcount = pos.len() as u32;
if fr.iter().any(|d| d.vcount == vcount) {
keys.push(PartKey { part: part.clone(), vcount, ref_pos: union.clone() });
}
}
}
}
let Some(ship) = correlate(id, fr, &keys, ref_sub) else { continue };
if ship.parts.len() < min_parts {
continue;
}
// Placements are expressed in the REFERENCE part's frame, so blocks that
// fell back to a different reference (because the requested one was not
// drawn in that block) are in a different coordinate system entirely.
// Averaging them together is what makes an otherwise clean result look
// like it disagrees by exactly the distance between the two references.
if !ship.reference.contains(ref_sub) {
println!(" block {fi:2}: skipped — reference fell back to {}", ship.reference);
continue;
}
used_frames += 1;
println!(
" block {fi:2} ({:4} draws): ref={} parts={}",
fr.len(),
ship.reference,
ship.parts.len()
);
for p in &ship.parts {
samples.entry(p.part.clone()).or_default().push(p.t);
rots.entry(p.part.clone()).or_default().push(p.m);
}
}
if used_frames == 0 {
println!("\nno block placed {min_parts}+ parts — the ship is not drawn close enough");
return;
}
// Aggregate by CONSENSUS, not by average. A stage holds several ships of the
// same class, they share vertex buffers, and a block can therefore contain
// one instance's full-LOD part next to another instance's `_m` copy — two
// different buffers, so nothing splits them, and the recovered translation
// then belongs to whichever instance the correlator happened to pick. Those
// are outliers by thousands of units, so a mean or a median over all blocks
// is meaningless; the largest cluster of blocks that agree with each other
// is the placement, and the rest are honestly reported as other instances.
const TOL: f32 = 25.0; // float noise in the WV products, measured ≤0.4
let cluster = |ts: &Vec<[f32; 3]>| -> (Vec<usize>, usize) {
let mut best: Vec<usize> = Vec::new();
for seed in ts {
let near: Vec<usize> = ts
.iter()
.enumerate()
.filter(|(_, t)| (0..3).all(|i| (t[i] - seed[i]).abs() < TOL))
.map(|(i, _)| i)
.collect();
if near.len() > best.len() {
best = near;
}
}
let out = ts.len() - best.len();
(best, out)
};
println!("\nacross {used_frames} blocks — consensus T (largest agreeing cluster):");
let mut agree = 0usize;
let mut consensus: BTreeMap<String, ([f32; 3], [[f32; 3]; 3])> = BTreeMap::new();
for (part, ts) in &samples {
let (cl_idx, outliers) = cluster(ts);
let cl: Vec<[f32; 3]> = cl_idx.iter().map(|&i| ts[i]).collect();
let med = [
median(cl.iter().map(|t| t[0]).collect()),
median(cl.iter().map(|t| t[1]).collect()),
median(cl.iter().map(|t| t[2]).collect()),
];
let spread: Vec<f32> = (0..3)
.map(|a| {
let v: Vec<f32> = cl.iter().map(|t| t[a]).collect();
v.iter().cloned().fold(f32::MIN, f32::max) - v.iter().cloned().fold(f32::MAX, f32::min)
})
.collect();
let verdict = if cl.len() < 2 {
"single block — unverified"
} else {
agree += 1;
"AGREES"
};
// How much the ROTATION varies between blocks that agree on position.
// A part bolted to the hull reads 0 here; a part that is articulating
// (turret aiming, engine gimballing) does not — which is what separates
// "the assembler has the rotation wrong" from "the part moved".
let all_ms = rots.get(part).cloned().unwrap_or_default();
let ms: Vec<[[f32; 3]; 3]> =
cl_idx.iter().filter_map(|&i| all_ms.get(i).copied()).collect();
// Keep a rotation from INSIDE the cluster: the first sample overall can
// belong to another instance, and diffing static against that reads as a
// rotation error that is really an instance mix-up.
consensus.insert(
part.clone(),
(med, ms.first().copied().unwrap_or([[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]])),
);
let rot_var = ms
.iter()
.flat_map(|a| ms.iter().map(move |b| (a, b)))
.map(|(a, b)| {
(0..3)
.flat_map(|i| (0..3).map(move |j| (i, j)))
.map(|(i, j)| (a[i][j] - b[i][j]).abs())
.fold(0.0f32, f32::max)
})
.fold(0.0f32, f32::max);
println!(
" {part:18} {:2}/{:2} blocks T=[{:9.1}{:9.1}{:9.1}] spread=[{:6.2}{:6.2}{:6.2}] rotVar={rot_var:5.3} {verdict}{}",
cl.len(), ts.len(), med[0], med[1], med[2], spread[0], spread[1], spread[2],
if outliers > 0 { format!(" (+{outliers} other-instance)") } else { String::new() }
);
}
println!("\n{agree}/{} parts reproduce across blocks", samples.len());
// `--static <Stage_SNN.xpr>`: diff the offline assembler against this
// ground truth. Static placements are in ship space, so both sides are
// re-expressed in the reference part's frame before comparing — and the
// rotation is compared too, because "wrong orientation" is half of the
// reported viewer symptom and a translation-only check cannot see it.
let Some(si) = args.iter().position(|a| a == "--static") else { return };
let Some(spath) = args.get(si + 1) else { return };
let sbytes = std::fs::read(spath).expect("read stage container");
// `include_external = true` — the engine cluster, the bridge and cross-id
// turrets live in SEPARATE composites (`e_rou_e106_eng`, 3 nodes) that the
// primary-composite pass does not reach. With `false` an e106 assembles as
// 5 parts and the runtime capture's bridge/nacelles read as "not produced by
// assemble_ship", which is a property of the caller, not of the format.
let scene = sylpheed_formats::ship::assemble_ship(&sbytes, id, true);
let Some(sref) = scene.iter().find(|p| p.resource.contains(ref_sub)) else {
println!("\nstatic: no part matching '{ref_sub}' — cannot align frames");
return;
};
// The reference is placed axis-aligned in every ship seen so far; if that
// ever stops holding, the rotation would have to be unwound here too.
println!("\nstatic vs runtime (both relative to {}):", sref.resource);
let mut worst_t = 0.0f32;
let mut worst_r = 0.0f32;
for (part, (med, rm)) in &consensus {
let (med, rm) = (*med, *rm);
// A part may be instanced (mirrored twins share a resource name); take
// the static copy that lands nearest the captured one.
let cands: Vec<&sylpheed_formats::mesh::ScenePart> =
scene.iter().filter(|p| &p.resource == part).collect();
if cands.is_empty() {
println!(" {part:18} — not produced by assemble_ship");
continue;
}
let rel = |p: &sylpheed_formats::mesh::ScenePart| {
[p.t[0] - sref.t[0], p.t[1] - sref.t[1], p.t[2] - sref.t[2]]
};
let best = cands
.iter()
.min_by(|a, b| {
let d = |p: &sylpheed_formats::mesh::ScenePart| {
let r = rel(p);
(0..3).map(|i| (r[i] - med[i]).powi(2)).sum::<f32>()
};
d(a).partial_cmp(&d(b)).unwrap()
})
.unwrap();
let r = rel(best);
let dt: Vec<f32> = (0..3).map(|i| r[i] - med[i]).collect();
let dtm = dt.iter().map(|v| v.abs()).fold(0.0f32, f32::max);
let drm = (0..3)
.flat_map(|i| (0..3).map(move |j| (i, j)))
.map(|(i, j)| (best.m[i][j] - rm[i][j]).abs())
.fold(0.0f32, f32::max);
worst_t = worst_t.max(dtm);
worst_r = worst_r.max(drm);
let mark = if dtm < 1.0 && drm < 0.02 { "MATCH" } else { "DIFFERS" };
println!(
" {part:18} static=[{:9.1}{:9.1}{:9.1}] dT={dtm:7.2} dR={drm:6.3} {mark}",
r[0], r[1], r[2]
);
}
println!("\nworst dT={worst_t:.2} worst dR={worst_r:.3} ({} static parts, {} captured)",
scene.len(), samples.len());
}

View File

@@ -0,0 +1,156 @@
//! Invert the capture↔part match: instead of asking, per ship part, "is there a
//! draw with this vertex count?", ask of the **capture's** biggest draws "which
//! decoded resource in this stage container has that vertex count?".
//!
//! This is the diagnostic for the 2026-07-31 negative result (Stage_S02 capture,
//! zero parts correlated). It separates three hypotheses:
//! 1. LOD/variant vcount not covered by the correlator's variant list
//! → the big draws DO map to named resources, just not to the `_m`/`_l`/`_d`
//! set the correlator tries;
//! 2. position validation over-rejects
//! → the vcounts match the very parts we asked for (so the vcount key was
//! fine and the rejection happened later);
//! 3. a different draw path (instanced/batched/merged buffers)
//! → the big draws match NO resource in the container at all.
//!
//! Usage:
//! SYLPHEED_ISO=... cargo run --release --example invert_capture -- \
//! <capture.log> <Stage_SNN> [top_n] [--all]
//! `--all` lists every capture vcount, not just the `top_n` (default 40) largest.
use sylpheed_formats::mesh::{xbg7_resource_names, Xbg7Model};
use sylpheed_formats::ship_capture::{parse_capture, parse_drawlog};
use sylpheed_formats::xiso::open_iso;
use std::collections::{HashMap, HashSet};
use std::path::Path;
fn main() {
let args: Vec<String> = std::env::args().collect();
let positional: Vec<&String> = args[1..].iter().filter(|a| !a.starts_with("--")).collect();
let all = args.iter().any(|a| a == "--all");
if positional.len() < 2 {
eprintln!("usage: invert_capture <capture.log> <Stage_SNN> [top_n] [--all]");
std::process::exit(2);
}
let (log, stage) = (positional[0], positional[1]);
let top_n: usize = positional.get(2).and_then(|s| s.parse().ok()).unwrap_or(40);
let iso = std::env::var("SYLPHEED_ISO").expect("SYLPHEED_ISO");
let text = std::fs::read_to_string(log).expect("read log");
let mut draws = parse_capture(&text);
if draws.is_empty() {
draws = parse_drawlog(&text);
println!("parsed {} draws (draw-logger format)", draws.len());
} else {
println!("parsed {} draws (F10 capture format)", draws.len());
}
// Decode EVERY geometry resource in the stage container, not just one ship's.
let bytes = {
let rt = tokio::runtime::Builder::new_current_thread().enable_all().build().unwrap();
rt.block_on(async {
let mut r = open_iso(Path::new(&iso)).await.unwrap();
r.read_file(&format!("hidden/resource3d/{stage}.xpr")).await.unwrap()
})
};
let names = xbg7_resource_names(&bytes);
println!("{stage}.xpr: {} XBG7 resources", names.len());
let want: HashSet<String> = names.iter().cloned().collect();
let models = Xbg7Model::models_named(&bytes, &want, &|| false);
println!("decoded {} models", models.len());
// vcount -> resource names with that many vertices.
let mut by_vcount: HashMap<u32, Vec<String>> = HashMap::new();
for m in &models {
let v: usize = m.meshes.iter().map(|s| s.positions.len()).sum();
by_vcount.entry(v as u32).or_default().push(m.name.clone());
}
// Per-submesh counts too: a draw may be one sub-mesh of a multi-mesh resource.
let mut by_sub_vcount: HashMap<u32, Vec<String>> = HashMap::new();
for m in &models {
for (i, s) in m.meshes.iter().enumerate() {
if m.meshes.len() > 1 {
by_sub_vcount
.entry(s.positions.len() as u32)
.or_default()
.push(format!("{}#{i}", m.name));
}
}
}
// Capture vcounts, de-duped by (vbase, vcount) so a re-drawn part counts once
// per distinct buffer.
let mut draw_count: HashMap<u32, usize> = HashMap::new();
let mut bufs: HashMap<u32, HashSet<u32>> = HashMap::new();
for d in &draws {
*draw_count.entry(d.vcount).or_default() += 1;
bufs.entry(d.vcount).or_default().insert(d.vbase);
}
let mut vcounts: Vec<u32> = draw_count.keys().copied().collect();
vcounts.sort_unstable_by(|a, b| b.cmp(a));
let matched_draws: usize = draws
.iter()
.filter(|d| by_vcount.contains_key(&d.vcount) || by_sub_vcount.contains_key(&d.vcount))
.count();
println!(
"\n{} distinct vcounts; {}/{} draws have a vcount present in {stage}.xpr ({:.1}%)",
vcounts.len(),
matched_draws,
draws.len(),
100.0 * matched_draws as f64 / draws.len().max(1) as f64
);
let shown = if all { vcounts.len() } else { top_n.min(vcounts.len()) };
println!("\nlargest capture vcounts (draws / distinct vbufs) → matching resources:");
for &v in vcounts.iter().take(shown) {
let n = draw_count[&v];
let b = bufs[&v].len();
let mut hit: Vec<String> = by_vcount.get(&v).cloned().unwrap_or_default();
let sub: Vec<String> = by_sub_vcount.get(&v).cloned().unwrap_or_default();
hit.extend(sub.into_iter().map(|s| format!("{s} (sub)")));
let label = if hit.is_empty() {
"— no resource".to_string()
} else {
let mut h = hit.clone();
h.sort();
h.truncate(6);
format!("{}{}", h.join(", "), if hit.len() > 6 { ", …" } else { "" })
};
println!(" vcount {v:6} draws {n:4} bufs {b:3} {label}");
}
// `--ship <id>`: every resource of one ship family, with its vertex count and
// whether the capture drew it — this is what shows an all-`_l` (far-LOD) frame.
if let Some(i) = args.iter().position(|a| a == "--ship") {
if let Some(id) = args.get(i + 1) {
let mut rows: Vec<(String, u32, usize)> = models
.iter()
.filter(|m| m.name.contains(id.as_str()))
.map(|m| {
let v = m.meshes.iter().map(|s| s.positions.len()).sum::<usize>() as u32;
(m.name.clone(), v, draw_count.get(&v).copied().unwrap_or(0))
})
.collect();
rows.sort_by(|a, b| a.0.cmp(&b.0));
let drawn = rows.iter().filter(|r| r.2 > 0).count();
println!("\n{id} resources in {stage}.xpr ({drawn}/{} with a drawn vcount):", rows.len());
for (name, v, n) in rows {
println!(" {name:28} vcount {v:6} {}", if n > 0 { format!("DRAWN ×{n}") } else { "".into() });
}
}
}
// The other direction, for orientation: the container's biggest resources and
// whether the capture ever drew that many vertices.
let mut sizes: Vec<(u32, String)> = models
.iter()
.map(|m| (m.meshes.iter().map(|s| s.positions.len()).sum::<usize>() as u32, m.name.clone()))
.collect();
sizes.sort_unstable_by(|a, b| b.0.cmp(&a.0));
println!("\nlargest resources in {stage}.xpr → drawn in the capture?");
for (v, name) in sizes.iter().take(top_n.min(sizes.len())) {
let n = draw_count.get(v).copied().unwrap_or(0);
println!(" {name:28} vcount {v:6} {}", if n > 0 { format!("DRAWN ×{n}") } else { "not drawn".into() });
}
}

View File

@@ -0,0 +1,98 @@
//! Global "which resource has N vertices?" index over every `.xpr` container in
//! an extracted `resource3d` directory, answered for the vcounts a capture log
//! actually drew.
//!
//! Companion to `invert_capture`: that one asks the question inside a single
//! stage container, this one asks it across ALL containers — so a draw whose
//! geometry lives in `Common.xpr`, a `rou_*` weapon pack or a `BG_*` backdrop is
//! still identified instead of coming back "no resource".
//!
//! Usage:
//! cargo run --release --example vcount_index -- <resource3d_dir> <capture.log> [top_n]
//! cargo run --release --example vcount_index -- <resource3d_dir> --vcounts 10891,6000
use sylpheed_formats::mesh::{xbg7_resource_names, Xbg7Model};
use sylpheed_formats::ship_capture::{parse_capture, parse_drawlog};
use std::collections::{HashMap, HashSet};
fn main() {
let args: Vec<String> = std::env::args().collect();
if args.len() < 3 {
eprintln!("usage: vcount_index <resource3d_dir> <capture.log|--vcounts a,b,c> [top_n]");
std::process::exit(2);
}
let dir = &args[1];
// Which vertex counts are we asking about, and how often was each drawn?
let mut draw_count: HashMap<u32, usize> = HashMap::new();
let mut bufs: HashMap<u32, HashSet<u32>> = HashMap::new();
if args[2] == "--vcounts" {
for v in args[3].split(',').filter_map(|s| s.trim().parse::<u32>().ok()) {
draw_count.insert(v, 0);
}
} else {
let text = std::fs::read_to_string(&args[2]).expect("read log");
let mut draws = parse_capture(&text);
if draws.is_empty() {
draws = parse_drawlog(&text);
}
eprintln!("parsed {} draws", draws.len());
for d in &draws {
*draw_count.entry(d.vcount).or_default() += 1;
bufs.entry(d.vcount).or_default().insert(d.vbase);
}
}
let top_n: usize = args.get(3).and_then(|s| s.parse().ok()).unwrap_or(usize::MAX);
// Decode every container once; keep only the vcount → names mapping.
let mut by_vcount: HashMap<u32, Vec<String>> = HashMap::new();
let mut files: Vec<std::path::PathBuf> = std::fs::read_dir(dir)
.expect("read dir")
.filter_map(|e| e.ok().map(|e| e.path()))
.filter(|p| p.extension().is_some_and(|e| e == "xpr"))
.collect();
files.sort();
let mut total_res = 0usize;
for f in &files {
let Ok(bytes) = std::fs::read(f) else { continue };
let names = xbg7_resource_names(&bytes);
if names.is_empty() {
continue;
}
let want: HashSet<String> = names.iter().cloned().collect();
let models = Xbg7Model::models_named(&bytes, &want, &|| false);
let container = f.file_stem().unwrap().to_string_lossy().to_string();
for m in &models {
total_res += 1;
let whole: usize = m.meshes.iter().map(|s| s.positions.len()).sum();
by_vcount.entry(whole as u32).or_default().push(format!("{container}:{}", m.name));
if m.meshes.len() > 1 {
for (i, s) in m.meshes.iter().enumerate() {
by_vcount
.entry(s.positions.len() as u32)
.or_default()
.push(format!("{container}:{}#{i}", m.name));
}
}
}
}
eprintln!("indexed {} resources from {} containers", total_res, files.len());
let mut vcounts: Vec<u32> = draw_count.keys().copied().collect();
vcounts.sort_unstable_by(|a, b| b.cmp(a));
println!("\nvcount draws bufs resources anywhere in resource3d/");
for v in vcounts.into_iter().take(top_n) {
let n = draw_count[&v];
let b = bufs.get(&v).map(|s| s.len()).unwrap_or(0);
let hit = by_vcount.get(&v).cloned().unwrap_or_default();
let label = if hit.is_empty() {
"— NONE".to_string()
} else {
let mut h = hit.clone();
h.sort();
let shown = h.len().min(8);
format!("{}{}", h[..shown].join(", "), if h.len() > shown { format!(", … ({} total)", h.len()) } else { String::new() })
};
println!("{v:6} {n:5} {b:4} {label}");
}
}

View File

@@ -349,9 +349,21 @@ pub fn assemble_ship(bytes: &[u8], id: &str, include_external: bool) -> Vec<Scen
let Some((_, gncat)) = CATS.iter().find(|(c, _)| *c == cat) else {
continue;
};
if let Some(frame) =
frames.iter().find(|f| f.resource.contains(gncat) && trailing_index(&f.resource) == idx)
{
// An index-less part (`e105_brg`, against a `GN_Bridge_01` frame) used to
// compare `"01" == ""` and fall through, so the bridge was silently
// dropped from the assembly while the game draws it — caught by a runtime
// capture, which places `e105_brg` at the `GN_Bridge_01` frame exactly.
// With no index to match on, take the lowest-numbered frame of the
// category; an indexed part still matches its own index only.
let mut cands: Vec<&ScenePart> = frames
.iter()
.filter(|f| {
f.resource.contains(gncat)
&& (idx.is_empty() || trailing_index(&f.resource) == idx)
})
.collect();
cands.sort_by_key(|f| trailing_index(&f.resource).parse::<u32>().unwrap_or(u32::MAX));
if let Some(frame) = cands.first().copied() {
placed.push(ScenePart { resource: part.clone(), m: frame.m, t: frame.t, s: frame.s });
placed_res.insert(part.clone());
}

View File

@@ -183,6 +183,41 @@ pub const SHIP_VS_HASH: &str = "0xC7F781F4C1D58054";
/// with `vs=`[`SHIP_VS_HASH`] are kept (the ship shader), so HUD/skybox draws are
/// ignored. (The player fighter shares ONE buffer across its fin draws and so
/// collapses to a single entry here — fine, capital ships are the target.)
/// Split a capture into blocks that are guaranteed to share one camera.
///
/// **Why this is not optional.** One F10 press dumps a flat list of draws with
/// no frame delimiter, and it spans ~14 frames (the same vertex buffer recurs
/// that many times). The placement math is `WV_ref⁻¹ · WV_p`, which cancels the
/// camera **only when both draws come from the same frame** — mix frames and
/// the residual is the camera's motion between them. With a static ship and a
/// static camera that error is invisible, which is how the single validated
/// `e106` capture passed; closing on a cruiser at ~760 u/s it is hundreds of
/// units, and two frames of the same ship then disagree about where its parts
/// are (measured 2026-08-10: `f105_bdy_02` at `[488, 736, -620]` vs
/// `[0, 0, -1090]`).
///
/// The split rule is the recurrence itself: a vertex buffer that appears again
/// starts a new block. Splitting too eagerly is harmless (a block is still one
/// camera, just with fewer parts in it) and it separates two instances of the
/// same class as a bonus; failing to split is what corrupts the result.
pub fn segment_frames(draws: &[CapturedDraw]) -> Vec<Vec<CapturedDraw>> {
let mut out: Vec<Vec<CapturedDraw>> = Vec::new();
let mut cur: Vec<CapturedDraw> = Vec::new();
let mut seen: std::collections::HashSet<u32> = std::collections::HashSet::new();
for d in draws {
if !seen.insert(d.vbase) {
out.push(std::mem::take(&mut cur));
seen.clear();
seen.insert(d.vbase);
}
cur.push(d.clone());
}
if !cur.is_empty() {
out.push(cur);
}
out
}
pub fn parse_drawlog(text: &str) -> Vec<CapturedDraw> {
let mut out = Vec::new();
let mut seen: std::collections::HashSet<u32> = std::collections::HashSet::new();

View File

@@ -8,7 +8,21 @@ unknown, what evidence exists, and what the first step would be. Move an item in
## Capital ships assemble wrong in the viewer
**Reported:** 2026-07-30, by the user. **Status:** ❔ open, not investigated.
**Reported:** 2026-07-30, by the user. **Status:** 🔎 **diagnosed 2026-08-10 — the
format layer is exonerated.** Runtime captures of three classes (`f105`, `e105`,
`e106`) at controlled range reproduce `assemble_ship` to ≤0.43 units in translation
and to 0.000 in rotation for every part that does not move; see
[`ship-placement-capture-generalisation.md`](ship-placement-capture-generalisation.md)
§4. So look at **the viewer**: first that it passes `include_external = true`
(`iso_loader.rs:4012` — with `false` an e106 loses its bridge and both nacelles,
5 parts instead of 11), then its own transform stack.
One real format-side bug was found on the way and is **fixed**: index-less parts
(`e105_brg`) never matched their `GN_Bridge_01` hardpoint, so 34 (stage, ship) entries
`e102`, `e104`, `e105` across Stages 0229 — assembled without a bridge. The other
apparent exception (`e105_eng_01` rotation) was an aggregation artefact and is 0.000.
The original report and its reasoning follow.
The reborn viewer builds capital ships from the split XBG7 parts via
`sylpheed-formats::ship::assemble_ship`, and they come out **wrong** — parts in the

View File

@@ -22,7 +22,7 @@ Promote to a prose `structures/…md` file when a format needs behavioural notes
| XBG7 mesh | 🟡/❔ | `sylpheed-formats/src/mesh.rs` + `tests/mesh_disc.rs` ([xbg7](structures/xbg7-mesh.md)) | weapons/props: declaration-driven variable stride (36 models), GPU-confirmed. **Stage containers: 5662 sub-models across 22 stages** via content-anchored grouped pools (`stage_models`). Quantized hero bodies (DeltaSaber `f004`) still declined |
| Capital-ship part placement | 🟡 | `sylpheed-formats/src/ship.rs` (static) + [runtime capture](ship-placement-runtime-capture.md) | hull placement static-exact; external parts approximate statically. **Runtime capture** (Canary F10 → VS-constant WorldView) gives ground truth — validated on `e106` destroyer; not yet baked into the viewer |
| Weapon fields defaulted on disc | ✅ | [runtime struct](structures/weapon-struct-runtime.md) · [DATA SHEET route](weapon-datasheet-runtime.md) | **Solved.** Canary maps guest RAM into `/dev/shm`, so the parsed `Weapon`/`Shell` objects are readable live; their layout is solved against disc ground truth (zero contradictions over 100+ records). All 126 weapons, exact numbers, no story progress needed — [4 393 values](captures/weapon-runtime-fields.csv) the disc does not carry. Supersedes the letter-bucket limit of the DATA SHEET route, which now serves as the independent cross-check |
| Unit (craft/vessel) fields defaulted on disc | ✅/🟡 | [runtime struct](structures/unit-struct-runtime.md) | The parsed `unit\UN_*.tbl` definition object, vtable `0x820af844`, ≥`0x380` bytes, one per unit — **discovered, not assumed** (`unit_discover.py`), and distinguished from the spawned-entity class `0x820af030` by being one-per-ID and byte-constant within a run. Across runs only pointer words move — `--crosscheck` proves **no reported field offset is run-dependent** (two words, `+0x2c8`/`+0x2d0`, are stage-dependent and remain unidentified). 27 fields ✅ (21 units, 7 runs); the `Maneuver` block is **schema declaration order, 4 bytes/field, base `0x9c` with a two-slot gap after `AA_Roll_Min`** (29 anchors, 0 conflicts), which also pins 5 fields *no* disc record ever values. Angles are **radians at runtime, degrees on disc**. Unlike weapons, unit definitions are instantiated **per stage**, so coverage (21/110) grows by visiting missions — [values](captures/unit-runtime-fields.csv) |
| Unit (craft/vessel) fields defaulted on disc | ✅/🟡 | [runtime struct](structures/unit-struct-runtime.md) | The parsed `unit\UN_*.tbl` definition object, vtable `0x820af844`, ≥`0x380` bytes, one per unit — **discovered, not assumed** (`unit_discover.py`), and distinguished from the spawned-entity class `0x820af030` by being one-per-ID and byte-constant within a run. Across runs only pointer words move — `--crosscheck` proves **no reported field offset is run-dependent** (two words, `+0x2c8`/`+0x2d0`, are stage-dependent and remain unidentified). 27 fields ✅ (21 units, 7 runs); the `Maneuver` block is **schema declaration order, 4 bytes/field, base `0x9c` with a two-slot gap after `AA_Roll_Min`** (29 anchors, 0 conflicts), which also pins 5 fields *no* disc record ever values. Angles are **radians at runtime, degrees on disc**. Unlike weapons, unit definitions are instantiated **per stage**, so coverage (21/110) grows by visiting missions — but a defaulted field is **not** a global constant: `Size_Y` provably inherits `Size_X` (7 independent units, 6 distinct values), and three more sibling rules are recorded ❔, recovering 65 values in units never visited — [values](captures/unit-runtime-fields.csv) |
| UI screen layout (`.rat`) | ✅/🟡 | [ui-rat-layout](structures/ui-rat-layout.md) | One pak per UI screen; each RATC = one (context × language) build; every `<name>.t32` sprite has a `<name>.rat` **layout record** (BE u32; 1280×720 design space; scale/tint/X/Y, keyframes for animated elements, `opt ` link to the focused state). **The tutorial PAUSE menu and the title main menu both rebuild pixel-accurately from the disc.** `loop1.rat` (screen-level draw order) not yet decoded |
## Runtime / dynamic-capture technique

Binary file not shown.

After

Width:  |  Height:  |  Size: 562 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 543 KiB

View File

@@ -0,0 +1,247 @@
# Capital-ship placement — does the `e106` result generalise? (WIP, 2026-07-31)
**Status:** 🚧 **WIP, time-boxed session.** Two results so far: a static audit across all
22 stage containers (done, below) and a first in-mission F10 capture run in Stage 02
(done — three capture logs, but **no capital-ship part correlated**; see "Open").
Context: [`BACKLOG.md`](BACKLOG.md) — "Capital ships assemble wrong in the viewer",
reported 2026-07-30. The oracle and the correlator already exist
([`ship-placement-runtime-capture.md`](ship-placement-runtime-capture.md)); the open
question is whether the rules derived from the one validated ship (`e106`, Stage_S01)
hold for other classes.
## 1. Static audit across all stages (offline, reproducible)
```
cargo run --release --example ship_audit -- ../sylph_extract/hidden/resource3d
```
87 lines of output, of which:
- **Only two OUTLIER lines, and they are the same ship twice**:
`Stage_S03`/`Stage_S27`, `f002_bdy_05` centroid `[-1398 6251 918]`, `dist=6540`
vs a cluster spread of `1071`. Every other assembled ship in every other stage
has all parts inside its own cluster.
→ **The user-visible breakage is NOT a gross static-placement outlier for most
classes.** Whatever is wrong in the viewer is either subtler than "part flung far
away" (wrong rotation, wrong mirror, missing part) or lives in the viewer, not in
`assemble_ship`. `f002_bdy_05` is a genuine, separate, reproducible static bug.
- **MULTIKEY**: joint tracks with more than one keyframe, which `read_trs9`'s
single-key read does not model. Recurring rigs: `e_rou_f104` (3), `e_rou_f105` (2),
`e_rou_f106` (2), `e_rou_e102` (6), `e_rou_e108_Missile_open` (2), `e_rou_e501` (1),
and the `e901` boss with 215 tracks per pose. Several of these (`Missile_open`, the
`e901_attack*` poses) are obviously *animation* and harmless for a static pose; the
plain hull rigs `f104`/`f105`/`f106`/`e102` are **not** obviously animation and are
the best hypothesis for a class-specific assembly error. ❔ **HYPOTHESIS — not
verified.** `e106`, the one validated ship, has **no** multikey tracks, which is
exactly how a rule that only works for single-key rigs could have passed unnoticed.
Raw audit output is reproducible with the command above (not checked in; it is
deterministic from the disc).
## 2. First in-mission capture run (Stage 02)
New tool: [`tools/re-capture/ship_capture_session.sh`](../../tools/re-capture/ship_capture_session.sh)
— one blocking session (per the session-lifetime rule): boot → Stage 02 in flight →
N× {screenshot, F10, small yaw}. Each F10 writes its own
`xenia_ship_capture_NN.log` next to the binary.
Run 2026-07-31, 5 presses requested:
- **Boot to in-flight took 24 s** (`skip_intro.sh` skipped the movie at 1 s and 6 s,
title at 11 s, HUD shield bar at 24 s) — much faster than the ~100 s in the notes.
- **3 of 5 F10 presses produced a log** (`_01``_03`, 2964 / 3111 / 3668 draws).
Logs (810 MB each) and the screenshots are at `/sylph-home/re/shipcap/`; not
committed for size. One screenshot is checked in as
[`captures/shipcap-stage02-launch.png`](captures/shipcap-stage02-launch.png).
- The screenshot confirms the capture frames are real in-mission combat frames
(HUD live, `REMAINING OB 004`, ACROPOLIS + a Destroyer labelled on screen, a
capital-ship hull filling the bottom of the frame).
### Result: no correlation yet ❌
```
correlate_capture xenia_ship_capture_03.log Stage_S02 <id> bdy_01
```
for `f101` (ACROPOLIS), `f105`, `f106`, `e105` reports *"no draw matches any LOD
(culled/off-screen?)"* for essentially every part — only two speculative LOD tries
(`f101_bdy_03` vcount 90 `[l]`, `e105_wep_01` vcount 60 `[l]`) and **zero accepted
matches**.
That is a **negative result, and it is not yet explained**. Facts collected:
- The capture is not empty or degenerate: 3668 draws in `_03`, top shaders
`0xDA51B0745ABF85D2` (1258), `0xE0BAFB4F520FE441` (1091), `0xEEA84C59D7F95371` (770).
None is the `e106` ship-shader hash from the 2026-07-26 capture; the F10 path does
not filter by hash, so this alone is not the cause.
- Large vertex counts *are* present (3024, 2772, 1736, 1612, 1240 …), so capital-ship-
sized geometry is being drawn.
Candidate explanations, **untested**:
1. the Stage-02 capital ships on screen are drawn from LOD/damage variants
(`_d00`, `_m`, `_l`) whose vcounts the correlator's variant list does not cover;
2. the position-validation step rejects otherwise-correct vcount hits (the capture
dumps ≤64 positions — a set-membership test against the wrong variant fails);
3. the ships in view at launch are drawn by a *different* draw path than `e106` in
Stage_S01 (e.g. instanced/batched), so no single draw equals one part.
**First step next session:** take the largest few vcounts in the capture and ask which
decoded part in `Stage_S02.xpr` has that count (invert the match), instead of asking
per-part whether a draw exists. That distinguishes (1)/(2) from (3) immediately.
## 3. The inverted match — the ships were never drawn (2026-08-10) ✅ explained
The inversion was run and it settles the negative result. Two new tools:
```
cargo run --release --example invert_capture -- <capture.log> Stage_S02 [top_n] [--ship f101]
cargo run --release --example vcount_index -- ../sylph_extract/hidden/resource3d <capture.log>
```
`invert_capture` asks, of the capture's own vertex counts, which resource in one stage
container has that count; `vcount_index` asks the same across **all 166 containers**
(5480 resources), so a draw whose geometry lives in `Common.xpr`, a `rou_*` weapon pack
or a `DeltaSaber_*` player-craft pack is identified instead of coming back "unknown".
On `xenia_ship_capture_03.log` (3668 draws, Stage 02):
| capture vcount | draws | what it is |
|---|---|---|
| 10891 | 28 | **`DeltaSaber_T:f001`** — the player's own craft |
| 6000 | 14 | `Stage_S02:n006_02` — backdrop |
| 1096 / 1008 / 841 / 215 / 127 | 14112 | `rou_f001_wep_*` — the player's weapons |
| 417 / 279 / 201 / 167 | 104448 | `Base:j00*`, `ptc_pack:*` — HUD/particles |
| 8 / 4 / 3 / 1 | 317590 | particle quads |
- **Not one capital-ship hull part appears.** Per ship: `f101` **1 of 15** resources had a
drawn vcount (`f101_bdy_03_l`, 90 verts), `e105` 3 of 37, `e106` 3 of 34 — and each of
those hits is a 44225-vertex `_l`/`_b` piece whose count also collides with dozens of
unrelated resources, i.e. probably not even the ship.
- The 3668 draws span **~14 frames** per F10 press and use only **10 distinct vertex
shaders**, and the player's own craft is captured at **full detail with its `c0..c2`
WVP rows** — so the capture path itself is healthy and unfiltered.
- The screenshot ([`captures/shipcap-stage02-launch.png`](captures/shipcap-stage02-launch.png))
agrees once read carefully: the hull "filling the bottom of the frame" is the **player's
own craft** in the chase view. The nearest contact on the HUD is a wingman's engine trail.
**So hypothesis (3) is dead, and (1)/(2) never applied.** The correlator's message
"no draw matches any LOD (culled/off-screen?)" was literally true: the ships were far
enough away that the renderer drew nothing of them. `correlate` additionally cannot
anchor without the reference part, and `f101_bdy_01` was never drawn at any LOD.
**The variable that was never controlled is RANGE.** New tooling closes that gap:
[`tools/re-capture/approach_capture.py`](../../tools/re-capture/approach_capture.py)
locks onto a capital ship (definition size-radius ≥ 150 = not a fighter), flies at it
with navigator.py's drift compensation and CPA avoidance, firing disabled, and presses
F10 as each range band is crossed (8000 / 6000 / 4500 / 3000 / 2000 / 1400 / 900),
stamping every capture with its distance in `approach-bands.jsonl`. Driver:
[`ship_capture_close.sh`](../../tools/re-capture/ship_capture_close.sh). Besides giving
the correlator a full-detail frame, the stamped bands measure the game's own **LOD
ladder** per part, which the reborn renderer needs anyway.
## 4. Controlled-range capture — three classes verified (2026-08-10) ✅
`ship_capture_close.sh 240` ran one blocking session: boot → Stage 02 in flight →
lock the `f105` cruiser → close on it at full throttle, F10 at each range band.
Six bands fired (7375 / 5937 / 4394 / 2994 / 1944 / 1259 units, stamped in
`approach-bands.jsonl`); **3 of 6 presses produced a log** — the same 3-of-N as the
earlier session, so a press during a previous 8 MB dump is still lost. Logs at
`/sylph-home/re/shipcap-close/` (not committed, ~9 MB each).
The difference from every earlier capture is immediate: `f105_bdy_01` (10926 verts),
`bdy_02`, `bdy_03`, `eng_01`, `sld_01` are all **drawn at full detail**, and the far
log additionally caught the `e105` and `e106` hulls.
### 4a. One log is ~14 frames, and mixing them silently corrupts the result
`WV_ref⁻¹ · WV_p` cancels the camera **only within one frame**. The capture log has no
frame delimiter, so `correlate_capture` was mixing ~14 frames; with the camera closing
at ~760 u/s that is not a small error — two logs of the same cruiser disagreed by
1090 units on `f105_bdy_02`. New `ship_capture::segment_frames` splits the log wherever
a vertex buffer recurs (over-splitting is harmless — a block is still one camera;
under-splitting is what corrupts), and new
[`correlate_frames`](../../crates/sylpheed-formats/examples/correlate_frames.rs)
correlates each block independently and **cross-checks the blocks against each other**.
Two aggregation rules had to be right, and both were wrong first:
- **Only blocks with the requested reference part count.** A block that fell back to
another reference expresses its parts in a different frame — averaging them in
produces a "disagreement" of exactly the distance between the two references.
- **Consensus, not median.** A stage holds several ships of one class; they share
vertex buffers, and a block can hold one instance's full-LOD part beside another's
`_m` copy (different buffers, so nothing splits them). The largest cluster of
mutually-agreeing blocks is the placement; the rest are reported as
`(+N other-instance)` rather than averaged into nonsense.
With that, every part reproduces across independent frames to **≤1 unit** (typical
spread 0.030.2).
### 4b. Static assembly matches the runtime on all three classes ✅
`correlate_frames … --static <Stage_S02.xpr>` diffs `assemble_ship` against the capture,
translation **and rotation**, both re-expressed in the reference part's frame:
| ship | rig | parts compared | worst dT | worst dR |
|---|---|---|---|---|
| `f105` TCAF cruiser | 1 engine, mirrored `sld` pair | 5 | **0.12** | **0.000** |
| `e105` ADAN cruiser | 6 hull bodies, bridge, engine | 7 | **0.05** | 1.711 (`eng_01` only) |
| `e106` ADAN destroyer | 2 nacelles + centre, turret | 8 | **0.43** | 0.098 (`eng`/`wep` only) |
**So the `e106` rules DO generalise.** Translation is exact for every part of every
class — 20 of 21 comparisons under 0.5 units. This is the answer the BACKLOG item asked
for, and it is the opposite of the assumption in it: `assemble_ship` is right, so the
viewer's "capital ships assemble wrong" is the viewer's own transform stack (the
backlog's own "worth ruling out first, cheaply").
Both first-pass exceptions were chased down, and neither survives as an open question:
-**`e105_brg` was genuinely missing — a real assembler bug, now fixed.** Tier 3
matched a part to its `GN_*` hardpoint by trailing index, so an index-less part
(`e105_brg`) compared `"01" == ""` against `GN_Bridge_01` and fell through silently.
With no index to match on, take the lowest-numbered frame of the category. The
runtime is the check: `e105_brg` now assembles at `[0.0, 70.0, -1850.0]` relative to
`e105_bdy_01`, **dT 0.03, dR 0.000** against the capture.
Reach measured by diffing `assemble_ship` part counts over all containers before and
after: **34 (stage, ship) entries gain parts**`e102` +2 (bridge *and* engine),
`e104` +1, `e105` +1, across Stages 0229. Every one of those ships was assembling
without its bridge. `ship_audit` is unchanged (still exactly the `f002_bdy_05`
outlier), so nothing regressed.
- ✅ **The rotation deltas were an artefact of my own aggregation, plus one real
articulation.** The static diff was comparing against a rotation taken from the
first sampled block, which can belong to *another instance* of the class; scoping it
to the position-agreeing cluster drops `e105_eng_01` from dR 1.711 to **0.000** and
both `e106` nacelles to **0.000**. What remains is `e106_wep_02_01` at dR 0.134 — and
that part's rotation varies by **0.182 between blocks that agree on its position**,
i.e. the runtime disagrees with itself more than it disagrees with the assembler.
It is a turret aiming, not an assembly error. `correlate_frames` now prints that
`rotVar` column precisely so "the part moved" cannot be mistaken for "the rotation
is wrong".
Final numbers, three classes, 21 parts: **worst dT 0.43, worst dR 0.000** for every
part that is not articulating.
`include_external` matters and is a caller-side trap: with `false` an `e106` assembles
as **5** parts and with `true` as **11** — the engine cluster, the bridge and the
cross-id `e303_wep_01` turrets live in separate composites (`e_rou_e106_eng`, 3 nodes)
that the primary-composite pass never reaches. The viewer takes it as a parameter
(`iso_loader.rs:4012`); if it is ever passed `false`, ships lose their engines and
bridge — which looks exactly like "assembles wrong".
## Honest summary
- ✅ Static assembly is **not** grossly broken across stages — 1 outlier ship
(`f002_bdy_05`), reproducible.
- 🟡 A concrete, testable hypothesis for class-specific breakage exists (multikey joint
tracks on `f104`/`f105`/`f106`/`e102`; `e106` has none).
- ✅ The earlier zero-match was **range**, not a format or correlator bug (§3): the
captures were taken where no capital-ship geometry is drawn at all.
- ✅ With range controlled (§4), **three classes**`f105`, `e105`, `e106` — reproduce
static assembly to **≤0.43 units** in translation, cross-checked across independent
frames. The `e106`-derived rules generalise; the MULTIKEY hypothesis in §1 is *not*
needed to explain anything observed so far (`f105` has 2 multikey tracks and still
matches exactly).
- ✅ One real assembler bug found and fixed by this route: index-less `brg`/`eng`/`sld`
parts never matched their `GN_*` frame, so **34 (stage, ship) entries** assembled
without a bridge (and `e102` also without its engine). Verified against the capture.
- ▶ Next: the viewer itself (`include_external`, node-instance recursion) — the format
layer is now measured, not assumed. A per-ship regression table over the checked-in
captures would keep it that way.

View File

@@ -272,3 +272,84 @@ take-off, ~12 minutes in mid-combat, and after GAME OVER. 14 objects, the same
is no need to play it, and no need to survive it.
Stages captured so far: `Ttrl` (BASIC CONTROLS), Stage 02.
## A defaulted unit field is not a global constant — some inherit from a sibling
**Confidence: 🟡 for `Size_Y`, ❔ for the rest. Analysis 2026-08-10, offline, from
[`captures/unit-runtime-fields.csv`](../captures/unit-runtime-fields.csv).**
The coverage limit above (21 of 110 units, growing only with story progress) is
worth attacking from the other side first: *if* a field the disc leaves unset
always took the same runtime value, the 21 captured units would pin that default
for all 110 and no further missions would be needed.
**It does not.** Restricting to the 150 values that are both ✅ CONFIRMED and
come from a field the disc leaves defaulted, only 6 of 24 fields have a single
value across every unit that defaults them (`HP`→10, `MassScore`→0,
`MaximumVelocity`→0, `RadarRange`→0, `DestroyMotionTime`→0, `Size_Z`→0.1). The
other 18 take several distinct values — so the default is computed per unit.
Where from? For each defaulted value, ask which *other* field of the same unit
holds exactly that value. Counting only cases where the value is **non-zero**
(otherwise `0 == 0` inflates every pair) and checking that the two fields are at
**different offsets** (so the match is not the layout solver aliasing them):
| defaulted field | takes the value of | support | independent units |
|---|---|---|---|
| `Size_Y` (`0x034`) | `Size_X` (`0x030`) | 9/9 | **7**, 6 distinct values |
| `Size_Radius` (`0x050`) | `min(Size_X, Size_Z)` | 4/4 | 4, 3 distinct values |
| `FCSRange` (`0x2a4`) | `RadarRange` (`0x2a0`) | 4/4 | 2 |
| `DefencePoint` (`0x2bc`) | `AttackVesselPoint` (`0x2b4`) | 6/6 | 2 |
`Size_Y ← Size_X` is the one to trust: seven unrelated ships (`e105` 600,
`e106` 300, `e108` 80, `e201` 300, `f101` 400, `f105` 700, `f106` 200) each omit
`Size_Y` on disc and each shows its own `Size_X` at runtime. When both fields
*are* on disc they differ freely (14 distinct `Size_Y` values against 13 of
`Size_X`), so this is a default rule, not one value stored twice.
`Size_Radius`'s formula is **not yet separable**: `min(Size_X, Size_Z)` and "the
median of the three axes" fit all four units identically. `UN_e010_ADAN_Attacker_S`
is what rules out the simpler `Size_Radius ← Size_X` (X=100, Y=40, Z=50, radius
**50**). The last two rules rest on two independent units each and are ❔ —
recorded so they can be falsified, not relied on.
**Why it matters for the reimplementation:** filling a missing `Size_Y` with `0`
or with a global constant gives the game's largest hulls a wrong lateral extent
(`f105` 700, `e105` 600, `f101` 400 — all defaulted on disc). Applied across the
disc, the rules recover **65 (unit, field) values in units that have never been
visited**: `Size_Y` in 21 of the 21 units that omit it, `Size_Radius` in 22 of 26,
`FCSRange` in 14 of 56, `DefencePoint` in 8 of 60.
### Cross-check against the weapons: this is NOT an engine-wide mechanism
The obvious worry is that four rules from 21 units are coincidence. The
`Weapon`/`Shell` capture is the control: **complete coverage, 126 records**, with
the same "defaulted on disc" classification. Running the identical sweep there
(confirmed rows, non-zero values, offsets required to differ) finds **no sibling
rule at all** — the single 100 %-agreement candidate (`Shell.Length ←
`Shell.Volume`, 5 records) has one distinct value, i.e. it is really the constant
`Length → 10` coinciding with `Volume = 10`. Weapon defaults vary per record just
as unit defaults do (10 of 14 `Weapon` fields, 16 of 17 `Shell` fields), so the
phenomenon is general; the *sibling* explanation is not.
So `Size_Y ← Size_X` is **specific to the unit schema** (plausibly the size block
defaulting its axes), not a property of IDXD default resolution. Two consequences:
the rule cannot be justified by appeal to a general mechanism, and the two
two-unit hypotheses (`FCSRange`, `DefencePoint`) lose the support they would have
borrowed from one — treat them as **coincidence-not-excluded** until a new stage
tests them.
`Size_Y ← Size_X` itself survives this scrutiny, and was re-checked at the raw
token level rather than through the sub-record merge: `UN_e105_ADAN_Cruiser`,
`UN_f105_TCAF_Cruiser` and `UN_f101_TCAF_Acropolis` each declare `Size_X`,
`Size_Z` and `Size_Radius` and **no `Size_Y` at all**, and each reads back its own
`Size_X` (600 / 700 / 400) at runtime.
**How to falsify:** the rules predict a specific number for units in stages not
yet captured. Load any new stage, snapshot, and compare — one disagreement kills
the rule. Note what is *not* a useful test: Stage 01, the only other reachable
stage, adds just four uncaptured units (`e010`/`e106` variants) whose predictions
are the same numbers their already-captured base variants gave, so it would
re-measure rather than test. A real test needs a stage with unfamiliar classes,
i.e. story progress — which is now the *only* thing story progress is needed for
here.

View File

@@ -0,0 +1,144 @@
#!/usr/bin/env python3
"""Fly TO a capital ship and dump a draw capture at several ranges.
Why this exists: the 2026-07-31 Stage-02 capture correlated **zero** parts, and
inverting the match (`cargo run --example invert_capture`) showed why — at the
captured frames no capital-ship hull was drawn at all. The only large draw was
the player's own craft (`DeltaSaber_T:f001`, 10891 verts); of `f101`/`e105`/
`e106` only a handful of tiny far-LOD/effect pieces appeared. The ships were
simply too far away. Pressing F10 wherever the craft happens to be is therefore
not a capture strategy.
So: pick a capital ship, fly at it, and press F10 as each distance band is
crossed. That gives (a) frames where the full-detail hull is actually drawn —
what the correlator needs — and (b) as a by-product, the game's own **LOD
ladder**, because each capture is stamped with the range it was taken at.
Firing is disabled (the target is usually a friendly), and navigator.py's
closest-point-of-approach avoidance is inherited unchanged, so closing on a hull
does not end in a collision.
Usage: approach_capture.py <config.json> [seconds] [--target REGEX] [--dry]
Env: SYLPH_CAPTURE_WIN xdotool window id to send F10 to (unset = no capture)
SYLPH_CAPTURE_OUT where to write the band log and screenshots
"""
import json
import math
import os
import re
import subprocess
import sys
import time
from collections import Counter
import numpy as np
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import navigator # noqa: E402
from navigator import Navigator, ang, norm # noqa: E402
from flight_probe import Pad # noqa: E402
# Ranges (guest units) at which to dump a capture, largest first. Chosen to
# straddle the plausible LOD switches: the far-LOD pieces seen in the 2026-07-31
# capture were drawn at whatever range the craft sat at, and the one validated
# capture (e106, Stage_S01) had the ship close.
BANDS = [8000.0, 6000.0, 4500.0, 3000.0, 2000.0, 1400.0, 900.0]
# A capital ship, not a fighter: the definition's own size radius says which.
CAPITAL_RADIUS = 150.0
class Approach(Navigator):
# Never shoot: the approach target is usually the escorted asset, and a
# negative cone makes the inherited fire gate unsatisfiable.
FIRE_CONE = -1.0
HOLD = 700.0 # stop closing inside this; the capture is already made
def __init__(self, W, pad, target_re=None, dry=False, log=sys.stdout,
win=None, out=None):
super().__init__(W, pad, dry=dry, log=log)
self.target_re = re.compile(target_re, re.I) if target_re else None
self.win = win
self.out = out or "/sylph-home/re/shipcap"
self.locked = None # (off, name) — stay on one ship
self.pending = list(BANDS)
self.captures = []
self.throttle = None
# -------------------------------------------------------------- target
def pick(self, me_p, me_v, fwd, ents, me_off):
"""The chosen capital ship — locked once, so the run is one approach."""
cands = [e for e in ents
if e[0] != me_off and "Player" not in e[1] and e[4] >= CAPITAL_RADIUS
and (self.target_re is None or self.target_re.search(e[1]))]
if not cands:
return None
if self.locked is not None:
same = [e for e in cands if e[0] == self.locked]
if same:
e = same[0]
return (e[0], e[1], e[2], e[2] - me_p, float(np.linalg.norm(e[2] - me_p)))
# First lock: the biggest ship that is not absurdly far.
cands.sort(key=lambda e: (-e[4], float(np.linalg.norm(e[2] - me_p))))
e = cands[0]
self.locked = e[0]
print(f"LOCK {e[1]} radius={e[4]:.0f} d={np.linalg.norm(e[2]-me_p):.0f}",
file=self.log, flush=True)
return (e[0], e[1], e[2], e[2] - me_p, float(np.linalg.norm(e[2] - me_p)))
# ------------------------------------------------------------- capture
def capture(self, band, dist, name):
idx = len(self.captures) + 1
shot = f"{self.out}/approach-{idx:02d}.png"
if self.win:
subprocess.run(["screenshot", shot], capture_output=True)
subprocess.run(["xdotool", "key", "--window", self.win, "F10"],
capture_output=True)
rec = {"index": idx, "band": band, "distance": round(dist, 1),
"target": name, "shot": shot, "t": round(time.time(), 3)}
self.captures.append(rec)
print(f"CAPTURE {idx:02d} band={band:.0f} d={dist:.0f} {name}",
file=self.log, flush=True)
with open(f"{self.out}/approach-bands.jsonl", "a") as f:
f.write(json.dumps(rec) + "\n")
# ---------------------------------------------------------------- loop
def step(self, t, dt, prev_vhat):
msg, vhat = super().step(t, dt, prev_vhat)
# Distance to the locked ship drives both the throttle and the captures.
ents = self.W.sample(t)
me = next((e for e in ents if "Player" in e[1]), None)
tgt = next((e for e in ents if e[0] == self.locked), None) if self.locked else None
if me is None or tgt is None:
return msg, vhat
d = float(np.linalg.norm(tgt[2] - me[2]))
# Throttle: RT to close, LT to hold off once we are as near as we want.
want = 1 if d > self.HOLD * 2 else (-1 if d < self.HOLD else 0)
if want != self.throttle and not self.dry:
self.pad.trig("RT", 1.0 if want > 0 else 0.0)
self.pad.trig("LT", 1.0 if want < 0 else 0.0)
self.throttle = want
while self.pending and d <= self.pending[0]:
band = self.pending.pop(0)
self.capture(band, d, tgt[1])
return f"{msg} | d={d:7.0f} thr={want:+d} left={len(self.pending)}", vhat
def main():
cfg = json.load(open(sys.argv[1]))
secs = float(sys.argv[2]) if len(sys.argv) > 2 and not sys.argv[2].startswith("-") else 240.0
target = None
if "--target" in sys.argv:
target = sys.argv[sys.argv.index("--target") + 1]
W = navigator.World(cfg)
a = Approach(W, Pad(), target_re=target, dry="--dry" in sys.argv,
win=os.environ.get("SYLPH_CAPTURE_WIN"),
out=os.environ.get("SYLPH_CAPTURE_OUT"))
a.run(secs)
print(f"CAPTURES {json.dumps(a.captures)}", flush=True)
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,45 @@
#!/usr/bin/env bash
# ONE blocking session: boot -> Stage 02 in flight -> fly AT a capital ship and
# dump an F10 draw capture at each distance band (approach_capture.py).
#
# Supersedes ship_capture_session.sh for correlation work: that one pressed F10
# wherever the craft happened to be, and the 2026-07-31 run proved that captures
# nothing — inverting the match showed no capital-ship hull was drawn in any of
# those frames, only the player's own craft and particles. Range is the variable
# that matters, so range is what this controls.
#
# Usage: ship_capture_close.sh [seconds] [out_dir] [target_regex]
set -u
export HOME=/sylph-home/re SDL_AUDIODRIVER=dummy DISPLAY=:98
export PYTHONPATH=/sylph-home/.local/lib/python3.12/site-packages
SD="$(cd "$(dirname "$0")" && pwd)"
SECS="${1:-240}"
OUT="${2:-/sylph-home/re/shipcap-close}"
TARGET="${3:-}"
BINDIR="/home/fabi/RE - Project Sylpheed/xenia-canary-native/build/bin/Linux/Release"
CFG=/tmp/nav-close.json
mkdir -p "$OUT"
rm -f "$BINDIR"/xenia_ship_capture_*.log "$OUT"/approach-bands.jsonl
"$SD/launch_mission.sh" fly || { echo "BOOT FAILED"; exit 1; }
python3 "$SD/entities2.py" self 0x130 "$CFG" || { echo "BIND FAILED"; exit 1; }
echo "--- config: $(cat "$CFG")"
# F10 goes to the emulator window through XTEST; the window must be focused.
win="$(xdotool search --class -- xenia | tail -1)"
[ -z "$win" ] && win="$(xdotool search --name -- Xenia | tail -1)"
echo "WINDOW=$win"
[ -n "$win" ] && { xdotool windowactivate "$win" 2>/dev/null; xdotool windowfocus "$win" 2>/dev/null; }
export SYLPH_CAPTURE_WIN="$win" SYLPH_CAPTURE_OUT="$OUT"
if [ -n "$TARGET" ]; then
python3 "$SD/approach_capture.py" "$CFG" "$SECS" --target "$TARGET" 2>&1 | tail -80
else
python3 "$SD/approach_capture.py" "$CFG" "$SECS" 2>&1 | tail -80
fi
sleep 2
cp -v "$BINDIR"/xenia_ship_capture_*.log "$OUT"/ 2>/dev/null
echo "--- bands ---"; cat "$OUT/approach-bands.jsonl" 2>/dev/null
grep -c '^DRAW' "$OUT"/xenia_ship_capture_*.log 2>/dev/null
echo "CLOSE CAPTURE SESSION DONE"

View File

@@ -0,0 +1,45 @@
#!/usr/bin/env bash
# ONE blocking session: boot -> Stage 02 in flight -> sweep the view and press
# F10 several times, so each press dumps xenia_ship_capture_NN.log with whatever
# capital ships are on screen at that moment.
#
# Why a sweep and not a single press: the 2026-07-26 e106 capture missed the
# bridge because it was culled at that camera angle. Several presses at
# different headings cost nothing (the capture is a one-frame draw dump) and
# each one is independently correlatable.
#
# Usage: ship_capture_session.sh [presses] [out_dir]
set -u
export HOME=/sylph-home/re SDL_AUDIODRIVER=dummy DISPLAY=:98
SD="$(cd "$(dirname "$0")" && pwd)"
PRESSES="${1:-6}"
OUT="${2:-/sylph-home/re/shipcap}"
BINDIR="/home/fabi/RE - Project Sylpheed/xenia-canary-native/build/bin/Linux/Release"
SHOTS=/sylph-home/re/shots
mkdir -p "$OUT" "$SHOTS"
rm -f "$BINDIR"/xenia_ship_capture_*.log
"$SD/launch_mission.sh" fly || { echo "BOOT FAILED"; exit 1; }
# F10 goes to the emulator window through XTEST; the window must be focused.
win="$(xdotool search --class -- xenia | tail -1)"
[ -z "$win" ] && win="$(xdotool search --name -- Xenia | tail -1)"
echo "WINDOW=$win"
[ -n "$win" ] && { xdotool windowactivate "$win" 2>/dev/null; xdotool windowfocus "$win" 2>/dev/null; }
for i in $(seq 1 "$PRESSES"); do
screenshot "$SHOTS/shipcap-$i.png" >/dev/null 2>&1
if [ -n "$win" ]; then xdotool key --window "$win" F10; else xdotool key F10; fi
sleep 3
# Yaw a little between presses so a culled part gets another chance, and the
# craft keeps closing on the friendly formation (the capital ships).
vgamepad axis LX 0.45; sleep 1.2; vgamepad axis LX 0.0
sleep 3
done
sleep 2
cp -v "$BINDIR"/xenia_ship_capture_*.log "$OUT"/ 2>/dev/null
cp -v "$SHOTS"/shipcap-*.png "$OUT"/ 2>/dev/null
grep -c '^DRAW' "$OUT"/xenia_ship_capture_*.log 2>/dev/null
echo "CAPTURE SESSION DONE"