re: control the range, segment the frames — f105/e105/e106 all match static assembly

A capture at controlled range (ship_capture_close.sh: lock a capital ship, close
on it, F10 per range band) finally draws capital-ship hulls at full detail. Two
correctness fixes were needed before the numbers meant anything:

* one F10 log is ~14 frames with no delimiter, and WV_ref^-1 . WV_p only cancels
  the camera within one frame — segment_frames splits on vertex-buffer
  recurrence, and correlate_frames cross-checks the blocks against each other
  instead of trusting a single shot;
* aggregate by consensus, not median: a stage holds several ships of one class
  sharing vertex buffers, so a block can mix two instances.

Result: f105, e105 and e106 reproduce assemble_ship to <=0.43 units in
translation and 0.000 in rotation for every part that does not move. The e106
rules generalise, and the viewer bug report now points at the viewer. Narrow
leftovers: e105_brg is missing from assemble_ship, e105_eng_01 rotation differs
by 1.711.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

crates/sylpheed-formats/examples/correlate_frames.rs

@@ -0,0 +1,256 @@
+//! Correlate a capture **per frame** and cross-check the frames against each
+//! other — the placement is only believable if independent frames agree.
+//!
+//! `correlate_capture` treats one capture log as one set of draws. It is not:
+//! an F10 press dumps ~14 frames with no delimiter, and `WV_ref⁻¹ · WV_p` only
+//! cancels the camera within a single frame (see
+//! [`sylpheed_formats::ship_capture::segment_frames`]). With a moving camera the
+//! mixed-frame answer is wrong, and — worse — it is wrong *silently*.
+//!
+//! So: segment, correlate each frame independently, then report per part the
+//! median translation and the spread across frames. A part whose spread is a
+//! few units is measured; a part that swings by hundreds is not, whatever the
+//! single-shot number said.
+//!
+//! Usage:
+//!   SYLPHEED_ISO=... cargo run --release --example correlate_frames -- \
+//!       <capture.log> <Stage_SNN> <ship_id> [ref_part_substr] [--min-parts N]
+
+use sylpheed_formats::mesh::{xbg7_resource_names, Xbg7Model};
+use sylpheed_formats::ship::{is_base_part, ship_id_of};
+use sylpheed_formats::ship_capture::{
+    correlate, parse_capture, parse_drawlog, segment_frames, PartKey,
+};
+use sylpheed_formats::xiso::open_iso;
+use std::collections::{BTreeMap, HashSet};
+use std::path::Path;
+
+fn median(mut v: Vec<f32>) -> f32 {
+    v.sort_by(|a, b| a.partial_cmp(b).unwrap());
+    let n = v.len();
+    if n % 2 == 1 { v[n / 2] } else { 0.5 * (v[n / 2 - 1] + v[n / 2]) }
+}
+
+fn main() {
+    let args: Vec<String> = std::env::args().collect();
+    let positional: Vec<&String> = args[1..].iter().filter(|a| !a.starts_with("--")).collect();
+    if positional.len() < 3 {
+        eprintln!("usage: correlate_frames <capture.log> <Stage_SNN> <ship_id> [ref_part] [--min-parts N]");
+        std::process::exit(2);
+    }
+    let (log, stage, id) = (positional[0], positional[1], positional[2]);
+    let ref_sub = positional.get(3).map(|s| s.as_str()).unwrap_or("bdy_01");
+    let min_parts: usize = args
+        .iter()
+        .position(|a| a == "--min-parts")
+        .and_then(|i| args.get(i + 1))
+        .and_then(|s| s.parse().ok())
+        .unwrap_or(3);
+
+    let iso = std::env::var("SYLPHEED_ISO").expect("SYLPHEED_ISO");
+    let text = std::fs::read_to_string(log).expect("read log");
+    let mut draws = parse_capture(&text);
+    if draws.is_empty() {
+        draws = parse_drawlog(&text);
+    }
+    let frames = segment_frames(&draws);
+    println!("{} draws → {} camera-consistent blocks", draws.len(), frames.len());
+
+    let bytes = {
+        let rt = tokio::runtime::Builder::new_current_thread().enable_all().build().unwrap();
+        rt.block_on(async {
+            let mut r = open_iso(Path::new(&iso)).await.unwrap();
+            r.read_file(&format!("hidden/resource3d/{stage}.xpr")).await.unwrap()
+        })
+    };
+    let names = xbg7_resource_names(&bytes);
+    let base_parts: Vec<String> = names
+        .iter()
+        .filter(|n| is_base_part(n) && ship_id_of(n) == Some(id.as_str()))
+        .cloned()
+        .collect();
+    let mut want: HashSet<String> = base_parts.iter().cloned().collect();
+    for p in &base_parts {
+        for suf in ["_m", "_l", "_d"] {
+            let c = format!("{p}{suf}");
+            if names.contains(&c) {
+                want.insert(c);
+            }
+        }
+    }
+    let models = Xbg7Model::models_named(&bytes, &want, &|| false);
+    let positions_of = |name: &str| -> Option<Vec<[f32; 3]>> {
+        let m = models.iter().find(|m| m.name == name)?;
+        Some(m.meshes.iter().flat_map(|s| s.positions.iter().copied()).collect())
+    };
+
+    // part -> [T per frame], and how many frames placed it at all.
+    let mut samples: BTreeMap<String, Vec<[f32; 3]>> = BTreeMap::new();
+    let mut rots: BTreeMap<String, Vec<[[f32; 3]; 3]>> = BTreeMap::new();
+    let mut used_frames = 0usize;
+    for (fi, fr) in frames.iter().enumerate() {
+        let mut keys: Vec<PartKey> = Vec::new();
+        for part in &base_parts {
+            let variants =
+                [part.clone(), format!("{part}_m"), format!("{part}_l"), format!("{part}_d")];
+            let union: Vec<[f32; 3]> =
+                variants.iter().filter_map(|v| positions_of(v)).flatten().collect();
+            for cand in &variants {
+                if let Some(pos) = positions_of(cand) {
+                    let vcount = pos.len() as u32;
+                    if fr.iter().any(|d| d.vcount == vcount) {
+                        keys.push(PartKey { part: part.clone(), vcount, ref_pos: union.clone() });
+                    }
+                }
+            }
+        }
+        let Some(ship) = correlate(id, fr, &keys, ref_sub) else { continue };
+        if ship.parts.len() < min_parts {
+            continue;
+        }
+        // Placements are expressed in the REFERENCE part's frame, so blocks that
+        // fell back to a different reference (because the requested one was not
+        // drawn in that block) are in a different coordinate system entirely.
+        // Averaging them together is what makes an otherwise clean result look
+        // like it disagrees by exactly the distance between the two references.
+        if !ship.reference.contains(ref_sub) {
+            println!("  block {fi:2}: skipped — reference fell back to {}", ship.reference);
+            continue;
+        }
+        used_frames += 1;
+        println!(
+            "  block {fi:2} ({:4} draws): ref={} parts={}",
+            fr.len(),
+            ship.reference,
+            ship.parts.len()
+        );
+        for p in &ship.parts {
+            samples.entry(p.part.clone()).or_default().push(p.t);
+            rots.entry(p.part.clone()).or_default().push(p.m);
+        }
+    }
+
+    if used_frames == 0 {
+        println!("\nno block placed {min_parts}+ parts — the ship is not drawn close enough");
+        return;
+    }
+    // Aggregate by CONSENSUS, not by average. A stage holds several ships of the
+    // same class, they share vertex buffers, and a block can therefore contain
+    // one instance's full-LOD part next to another instance's `_m` copy — two
+    // different buffers, so nothing splits them, and the recovered translation
+    // then belongs to whichever instance the correlator happened to pick. Those
+    // are outliers by thousands of units, so a mean or a median over all blocks
+    // is meaningless; the largest cluster of blocks that agree with each other
+    // is the placement, and the rest are honestly reported as other instances.
+    const TOL: f32 = 25.0; // float noise in the WV products, measured ≤0.4
+    let cluster = |ts: &Vec<[f32; 3]>| -> (Vec<[f32; 3]>, usize) {
+        let mut best: Vec<[f32; 3]> = Vec::new();
+        for seed in ts {
+            let near: Vec<[f32; 3]> = ts
+                .iter()
+                .filter(|t| (0..3).all(|i| (t[i] - seed[i]).abs() < TOL))
+                .copied()
+                .collect();
+            if near.len() > best.len() {
+                best = near;
+            }
+        }
+        let out = ts.len() - best.len();
+        (best, out)
+    };
+    println!("\nacross {used_frames} blocks — consensus T (largest agreeing cluster):");
+    let mut agree = 0usize;
+    let mut consensus: BTreeMap<String, [f32; 3]> = BTreeMap::new();
+    for (part, ts) in &samples {
+        let (cl, outliers) = cluster(ts);
+        let med = [
+            median(cl.iter().map(|t| t[0]).collect()),
+            median(cl.iter().map(|t| t[1]).collect()),
+            median(cl.iter().map(|t| t[2]).collect()),
+        ];
+        consensus.insert(part.clone(), med);
+        let spread: Vec<f32> = (0..3)
+            .map(|a| {
+                let v: Vec<f32> = cl.iter().map(|t| t[a]).collect();
+                v.iter().cloned().fold(f32::MIN, f32::max) - v.iter().cloned().fold(f32::MAX, f32::min)
+            })
+            .collect();
+        let verdict = if cl.len() < 2 {
+            "single block — unverified"
+        } else {
+            agree += 1;
+            "AGREES"
+        };
+        println!(
+            "  {part:18} {:2}/{:2} blocks  T=[{:9.1}{:9.1}{:9.1}]  spread=[{:6.2}{:6.2}{:6.2}]  {verdict}{}",
+            cl.len(), ts.len(), med[0], med[1], med[2], spread[0], spread[1], spread[2],
+            if outliers > 0 { format!("  (+{outliers} other-instance)") } else { String::new() }
+        );
+    }
+    println!("\n{agree}/{} parts reproduce across blocks", samples.len());
+
+    // `--static <Stage_SNN.xpr>`: diff the offline assembler against this
+    // ground truth. Static placements are in ship space, so both sides are
+    // re-expressed in the reference part's frame before comparing — and the
+    // rotation is compared too, because "wrong orientation" is half of the
+    // reported viewer symptom and a translation-only check cannot see it.
+    let Some(si) = args.iter().position(|a| a == "--static") else { return };
+    let Some(spath) = args.get(si + 1) else { return };
+    let sbytes = std::fs::read(spath).expect("read stage container");
+    // `include_external = true` — the engine cluster, the bridge and cross-id
+    // turrets live in SEPARATE composites (`e_rou_e106_eng`, 3 nodes) that the
+    // primary-composite pass does not reach. With `false` an e106 assembles as
+    // 5 parts and the runtime capture's bridge/nacelles read as "not produced by
+    // assemble_ship", which is a property of the caller, not of the format.
+    let scene = sylpheed_formats::ship::assemble_ship(&sbytes, id, true);
+    let Some(sref) = scene.iter().find(|p| p.resource.contains(ref_sub)) else {
+        println!("\nstatic: no part matching '{ref_sub}' — cannot align frames");
+        return;
+    };
+    // The reference is placed axis-aligned in every ship seen so far; if that
+    // ever stops holding, the rotation would have to be unwound here too.
+    println!("\nstatic vs runtime (both relative to {}):", sref.resource);
+    let mut worst_t = 0.0f32;
+    let mut worst_r = 0.0f32;
+    for (part, med) in &consensus {
+        let med = *med;
+        // A part may be instanced (mirrored twins share a resource name); take
+        // the static copy that lands nearest the captured one.
+        let cands: Vec<&sylpheed_formats::mesh::ScenePart> =
+            scene.iter().filter(|p| &p.resource == part).collect();
+        if cands.is_empty() {
+            println!("  {part:18} — not produced by assemble_ship");
+            continue;
+        }
+        let rel = |p: &sylpheed_formats::mesh::ScenePart| {
+            [p.t[0] - sref.t[0], p.t[1] - sref.t[1], p.t[2] - sref.t[2]]
+        };
+        let best = cands
+            .iter()
+            .min_by(|a, b| {
+                let d = |p: &sylpheed_formats::mesh::ScenePart| {
+                    let r = rel(p);
+                    (0..3).map(|i| (r[i] - med[i]).powi(2)).sum::<f32>()
+                };
+                d(a).partial_cmp(&d(b)).unwrap()
+            })
+            .unwrap();
+        let r = rel(best);
+        let dt: Vec<f32> = (0..3).map(|i| r[i] - med[i]).collect();
+        let dtm = dt.iter().map(|v| v.abs()).fold(0.0f32, f32::max);
+        let rm = rots.get(part).map(|v| v[0]).unwrap_or([[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]]);
+        let drm = (0..3)
+            .flat_map(|i| (0..3).map(move |j| (i, j)))
+            .map(|(i, j)| (best.m[i][j] - rm[i][j]).abs())
+            .fold(0.0f32, f32::max);
+        worst_t = worst_t.max(dtm);
+        worst_r = worst_r.max(drm);
+        let mark = if dtm < 1.0 && drm < 0.02 { "MATCH" } else { "DIFFERS" };
+        println!(
+            "  {part:18} static=[{:9.1}{:9.1}{:9.1}]  dT={dtm:7.2}  dR={drm:6.3}  {mark}",
+            r[0], r[1], r[2]
+        );
+    }
+    println!("\nworst dT={worst_t:.2}  worst dR={worst_r:.3}  ({} static parts, {} captured)",
+             scene.len(), samples.len());
+}

crates/sylpheed-formats/src/ship_capture.rs

@@ -183,6 +183,41 @@ pub const SHIP_VS_HASH: &str = "0xC7F781F4C1D58054";
 /// with `vs=`[`SHIP_VS_HASH`] are kept (the ship shader), so HUD/skybox draws are
 /// ignored. (The player fighter shares ONE buffer across its fin draws and so
 /// collapses to a single entry here — fine, capital ships are the target.)
+/// Split a capture into blocks that are guaranteed to share one camera.
+///
+/// **Why this is not optional.** One F10 press dumps a flat list of draws with
+/// no frame delimiter, and it spans ~14 frames (the same vertex buffer recurs
+/// that many times). The placement math is `WV_ref⁻¹ · WV_p`, which cancels the
+/// camera **only when both draws come from the same frame** — mix frames and
+/// the residual is the camera's motion between them. With a static ship and a
+/// static camera that error is invisible, which is how the single validated
+/// `e106` capture passed; closing on a cruiser at ~760 u/s it is hundreds of
+/// units, and two frames of the same ship then disagree about where its parts
+/// are (measured 2026-08-10: `f105_bdy_02` at `[488, 736, -620]` vs
+/// `[0, 0, -1090]`).
+///
+/// The split rule is the recurrence itself: a vertex buffer that appears again
+/// starts a new block. Splitting too eagerly is harmless (a block is still one
+/// camera, just with fewer parts in it) and it separates two instances of the
+/// same class as a bonus; failing to split is what corrupts the result.
+pub fn segment_frames(draws: &[CapturedDraw]) -> Vec<Vec<CapturedDraw>> {
+    let mut out: Vec<Vec<CapturedDraw>> = Vec::new();
+    let mut cur: Vec<CapturedDraw> = Vec::new();
+    let mut seen: std::collections::HashSet<u32> = std::collections::HashSet::new();
+    for d in draws {
+        if !seen.insert(d.vbase) {
+            out.push(std::mem::take(&mut cur));
+            seen.clear();
+            seen.insert(d.vbase);
+        }
+        cur.push(d.clone());
+    }
+    if !cur.is_empty() {
+        out.push(cur);
+    }
+    out
+}
+
 pub fn parse_drawlog(text: &str) -> Vec<CapturedDraw> {
     let mut out = Vec::new();
     let mut seen: std::collections::HashSet<u32> = std::collections::HashSet::new();

docs/re/BACKLOG.md

@@ -8,7 +8,18 @@ unknown, what evidence exists, and what the first step would be. Move an item in
 
 ## Capital ships assemble wrong in the viewer
 
-**Reported:** 2026-07-30, by the user. **Status:** ❔ open, not investigated.
+**Reported:** 2026-07-30, by the user. **Status:** 🔎 **diagnosed 2026-08-10 — the
+format layer is exonerated.** Runtime captures of three classes (`f105`, `e105`,
+`e106`) at controlled range reproduce `assemble_ship` to ≤0.43 units in translation
+and to 0.000 in rotation for every part that does not move; see
+[`ship-placement-capture-generalisation.md`](ship-placement-capture-generalisation.md)
+§4. So look at **the viewer**: first that it passes `include_external = true`
+(`iso_loader.rs:4012` — with `false` an e106 loses its bridge and both nacelles,
+5 parts instead of 11), then its own transform stack. Two narrow format-side leftovers
+remain: `e105_brg` is never produced by `assemble_ship`, and `e105_eng_01`'s rotation
+differs by 1.711.
+
+The original report and its reasoning follow.
 
 The reborn viewer builds capital ships from the split XBG7 parts via
 `sylpheed-formats::ship::assemble_ship`, and they come out **wrong** — parts in the

docs/re/ship-placement-capture-generalisation.md

@@ -138,16 +138,92 @@ stamping every capture with its distance in `approach-bands.jsonl`. Driver:
 the correlator a full-detail frame, the stamped bands measure the game's own **LOD
 ladder** per part, which the reborn renderer needs anyway.
 
+## 4. Controlled-range capture — three classes verified (2026-08-10) ✅
+
+`ship_capture_close.sh 240` ran one blocking session: boot → Stage 02 in flight →
+lock the `f105` cruiser → close on it at full throttle, F10 at each range band.
+Six bands fired (7375 / 5937 / 4394 / 2994 / 1944 / 1259 units, stamped in
+`approach-bands.jsonl`); **3 of 6 presses produced a log** — the same 3-of-N as the
+earlier session, so a press during a previous 8 MB dump is still lost. Logs at
+`/sylph-home/re/shipcap-close/` (not committed, ~9 MB each).
+
+The difference from every earlier capture is immediate: `f105_bdy_01` (10926 verts),
+`bdy_02`, `bdy_03`, `eng_01`, `sld_01` are all **drawn at full detail**, and the far
+log additionally caught the `e105` and `e106` hulls.
+
+### 4a. One log is ~14 frames, and mixing them silently corrupts the result
+
+`WV_ref⁻¹ · WV_p` cancels the camera **only within one frame**. The capture log has no
+frame delimiter, so `correlate_capture` was mixing ~14 frames; with the camera closing
+at ~760 u/s that is not a small error — two logs of the same cruiser disagreed by
+1090 units on `f105_bdy_02`. New `ship_capture::segment_frames` splits the log wherever
+a vertex buffer recurs (over-splitting is harmless — a block is still one camera;
+under-splitting is what corrupts), and new
+[`correlate_frames`](../../crates/sylpheed-formats/examples/correlate_frames.rs)
+correlates each block independently and **cross-checks the blocks against each other**.
+
+Two aggregation rules had to be right, and both were wrong first:
+- **Only blocks with the requested reference part count.** A block that fell back to
+  another reference expresses its parts in a different frame — averaging them in
+  produces a "disagreement" of exactly the distance between the two references.
+- **Consensus, not median.** A stage holds several ships of one class; they share
+  vertex buffers, and a block can hold one instance's full-LOD part beside another's
+  `_m` copy (different buffers, so nothing splits them). The largest cluster of
+  mutually-agreeing blocks is the placement; the rest are reported as
+  `(+N other-instance)` rather than averaged into nonsense.
+
+With that, every part reproduces across independent frames to **≤1 unit** (typical
+spread 0.03–0.2).
+
+### 4b. Static assembly matches the runtime on all three classes ✅
+
+`correlate_frames … --static <Stage_S02.xpr>` diffs `assemble_ship` against the capture,
+translation **and rotation**, both re-expressed in the reference part's frame:
+
+| ship | rig | parts compared | worst dT | worst dR |
+|---|---|---|---|---|
+| `f105` TCAF cruiser | 1 engine, mirrored `sld` pair | 5 | **0.12** | **0.000** |
+| `e105` ADAN cruiser | 6 hull bodies, bridge, engine | 7 | **0.05** | 1.711 (`eng_01` only) |
+| `e106` ADAN destroyer | 2 nacelles + centre, turret | 8 | **0.43** | 0.098 (`eng`/`wep` only) |
+
+**So the `e106` rules DO generalise.** Translation is exact for every part of every
+class — 20 of 21 comparisons under 0.5 units. This is the answer the BACKLOG item asked
+for, and it is the opposite of the assumption in it: `assemble_ship` is right, so the
+viewer's "capital ships assemble wrong" is the viewer's own transform stack (the
+backlog's own "worth ruling out first, cheaply").
+
+Two real exceptions, both narrow:
+
+- **`e105_brg` is never produced by `assemble_ship`**, at either `include_external`
+  setting, although the runtime draws it and places it at `[0.0, 70.0, -1850.0]`
+  relative to `e105_bdy_01` (4/4 blocks, spread ≤0.13). ❔ A genuine missing part.
+- **Rotation differs only on parts that move**: `e106_wep_02_01` (turret, dR 0.093),
+  `e106_eng_01`/`eng_02` (dR ~0.097) and `e105_eng_01` (dR **1.711**). A turret aiming
+  and a nacelle gimballing at capture time is expected and is not an assembly error;
+  1.711 on `e105_eng_01` is too large for that and is ❔ **unexplained — NEEDS-HUMAN**
+  (either a static rotation bug on that one part, or the part is articulated).
+
+`include_external` matters and is a caller-side trap: with `false` an `e106` assembles
+as **5** parts and with `true` as **11** — the engine cluster, the bridge and the
+cross-id `e303_wep_01` turrets live in separate composites (`e_rou_e106_eng`, 3 nodes)
+that the primary-composite pass never reaches. The viewer takes it as a parameter
+(`iso_loader.rs:4012`); if it is ever passed `false`, ships lose their engines and
+bridge — which looks exactly like "assembles wrong".
+
 ## Honest summary
 
 - ✅ Static assembly is **not** grossly broken across stages — 1 outlier ship
   (`f002_bdy_05`), reproducible.
 - 🟡 A concrete, testable hypothesis for class-specific breakage exists (multikey joint
   tracks on `f104`/`f105`/`f106`/`e102`; `e106` has none).
-- ❌ The runtime oracle has **not** reproduced on a second ship class yet — but the
+- ✅ The earlier zero-match was **range**, not a format or correlator bug (§3): the
-  reason is now known and is not a format or correlator bug: **the captures were taken
+  captures were taken where no capital-ship geometry is drawn at all.
-  at ranges where no capital-ship geometry is drawn at all** (§3, verified by inverting
+- ✅ With range controlled (§4), **three classes** — `f105`, `e105`, `e106` — reproduce
-  the match against all 5480 decoded resources). Do not assume the `e106` rules
+  static assembly to **≤0.43 units** in translation, cross-checked across independent
-  generalise; equally, do not read the zero-match as evidence against them.
+  frames. The `e106`-derived rules generalise; the MULTIKEY hypothesis in §1 is *not*
-- ▶ Next: run `ship_capture_close.sh` so the capture happens with a hull actually on
+  needed to explain anything observed so far (`f105` has 2 multikey tracks and still
-  screen, then re-run `correlate_capture` on the closest band.
+  matches exactly).
+- ❌ Still open, both narrow and both evidenced: `e105_brg` is missing from
+  `assemble_ship`, and `e105_eng_01`'s rotation differs by 1.711 — NEEDS-HUMAN.
+- ▶ Next: chase those two, and point the viewer investigation at the viewer
+  (`include_external`, node-instance recursion), not at the format layer.