fabi/Syplheed-Reborn
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

re: runtime-capture pipeline for exact capital-ship part placement

Browse Source 
Static external-part placement (ship::assemble_ship) is only approximate — a GN_*
frame is the mount pivot, and the real offset lives in a runtime detail rig. So
capture ground truth from Canary instead.

Finding: the guest vertex buffer holds LOCAL coords (byte-identical to the .xpr),
so capital-ship parts are placed entirely in the vertex shader. The per-part
transform is in the VS float constants — c0..c2 are the WorldViewProjection rows;
normalising each by its norm (= projection x/y/z scales, sy/sx = 16:9) yields the
rigid WorldView (rows orthonormal, det +1). Expressing every part relative to a
reference part cancels the camera: rel_p = WV_ref⁻¹·WV_p, a pure ship-space rigid
transform.

- examples/correlate_capture.rs: parse xenia_ship_capture.log, decode the ship's
  parts, match by vertex count, recover each part's ship-relative placement.
- docs/re/ship-placement-runtime-capture.md: full method + the capture protocol
  (Canary branch capture-ship-placement, F10 hotkey) + validated e106 result.

Validated on e106 (ADAN Destroyer, Stage_S01): all 7 drawn parts matched by
vertex count; bdy_01/bdy_02 recovered as a PORT/STARBOARD pair (X = ∓264) that
static had overlapping; engines correctly aft; extent 872×670×2181. Only brg_01
(bridge) missing — culled at that camera angle, needs a bridge-facing capture.

HANDOFF: next = re-capture bridge, bake a per-ship placement table the viewer
prefers over assemble_ship, then capture the cruiser / ACROPOLIS / TCAF ships.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-07-24 10:50:26 +02:00
parent 79c78cc7e1
commit a68405216f
3 changed files with 313 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

207
crates/sylpheed-formats/examples/correlate_capture.rs Normal file
View File

@@ -0,0 +1,207 @@
//! Recover exact capital-ship part placement from a Canary `xenia_ship_capture.log`
//! (F10 snapshot: per-draw guest vertex-buffer + up to 48 vertex-shader float
//! constants).
//!
//! FINDING (2026-07-24): the captured vertex BUFFER holds LOCAL coordinates —
//! byte-identical to the `.xpr` — so capital-ship parts are placed entirely in the
//! **vertex shader**. The per-part transform is in the VS constants: `c0..c2` are
//! the **WorldViewProjection** matrix rows. Their norms are `(sx, sy, 1)` = the
//! projection x/y scales (sy/sx = 16:9 aspect); dividing each row by its norm
//! yields the rigid **WorldView** (verified: rows orthonormal, det +1). The camera
//! View cancels when we express every part relative to a reference part:
//! rel_p = WV_ref⁻¹ · WV_p (a pure ship-space rigid transform)
//! Applying `rel_p` to part `p`'s local geometry assembles the ship exactly, in the
//! reference part's frame — ground truth, no static guessing.
//!
//! Usage:
//! SYLPHEED_ISO=... cargo run --release --example correlate_capture -- \
//! <capture.log> <Stage_SNN> <ship_id> [ref_part_substr]
//! e.g. `... /tmp/xenia_ship_capture.log Stage_S01 e106 bdy_04`
use sylpheed_formats::mesh::{xbg7_resource_names, Xbg7Model};
use sylpheed_formats::ship::{is_base_part, ship_id_of};
use sylpheed_formats::xiso::open_iso;
use std::collections::HashSet;
use std::path::Path;
type M3 = [[f64; 3]; 3];
struct Draw {
vbase: u32,
vcount: u32,
/// Rigid WorldView: R (rows) + T (view-space translation).
r: M3,
t: [f64; 3],
ok: bool,
}
fn parse(text: &str) -> Vec<Draw> {
let mut out = Vec::new();
let mut vbase = 0u32;
let mut vcount = 0u32;
let mut consts: Vec<(usize, [f64; 4])> = Vec::new();
let mut flush = |vbase: u32, vcount: u32, consts: &[(usize, [f64; 4])], out: &mut Vec<Draw>| {
if vbase == 0 {
return;
}
// c0..c2 = WVP rows; normalise each to unit → rigid WorldView.
let get = |i: usize| consts.iter().find(|(k, _)| *k == i).map(|(_, v)| *v);
let (Some(c0), Some(c1), Some(c2)) = (get(0), get(1), get(2)) else {
out.push(Draw { vbase, vcount, r: [[0.0; 3]; 3], t: [0.0; 3], ok: false });
return;
};
let rows = [c0, c1, c2];
let mut r = [[0.0; 3]; 3];
let mut t = [0.0; 3];
let mut ok = true;
for (i, row) in rows.iter().enumerate() {
let n = (row[0] * row[0] + row[1] * row[1] + row[2] * row[2]).sqrt();
if n < 1e-6 {
ok = false;
break;
}
r[i] = [row[0] / n, row[1] / n, row[2] / n];
t[i] = row[3] / n;
}
out.push(Draw { vbase, vcount, r, t, ok });
};
for line in text.lines() {
let l = line.trim();
if let Some(rest) = l.strip_prefix("DRAW ") {
flush(vbase, vcount, &consts, &mut out);
consts.clear();
let f = |k: &str| rest.split_whitespace().find_map(|t| t.strip_prefix(k));
vbase = f("vbase=0x").and_then(|s| u32::from_str_radix(s, 16).ok()).unwrap_or(0);
vcount = f("vcount=").and_then(|s| s.parse().ok()).unwrap_or(0);
} else if l.starts_with("vsconst") {
for cap in l.split("c").skip(1) {
// "<i>=(x,y,z,w) ..."
if let Some((idx, rest)) = cap.split_once('=') {
if let Ok(i) = idx.trim().parse::<usize>() {
let nums: Vec<f64> = rest
.trim_start_matches('(')
.split(')')
.next()
.unwrap_or("")
.split(',')
.filter_map(|x| x.trim().parse().ok())
.collect();
if nums.len() == 4 {
consts.push((i, [nums[0], nums[1], nums[2], nums[3]]));
}
}
}
}
}
}
flush(vbase, vcount, &consts, &mut out);
out
}
fn mt_apply(r: &M3, t: &[f64; 3], v: [f64; 3]) -> [f64; 3] {
[
r[0][0] * v[0] + r[0][1] * v[1] + r[0][2] * v[2] + t[0],
r[1][0] * v[0] + r[1][1] * v[1] + r[1][2] * v[2] + t[1],
r[2][0] * v[0] + r[2][1] * v[1] + r[2][2] * v[2] + t[2],
]
}
fn transpose(m: &M3) -> M3 {
[
[m[0][0], m[1][0], m[2][0]],
[m[0][1], m[1][1], m[2][1]],
[m[0][2], m[1][2], m[2][2]],
]
}
fn mmul(a: &M3, b: &M3) -> M3 {
let mut o = [[0.0; 3]; 3];
for i in 0..3 {
for j in 0..3 {
o[i][j] = (0..3).map(|k| a[i][k] * b[k][j]).sum();
}
}
o
}
fn main() {
let args: Vec<String> = std::env::args().collect();
if args.len() < 4 {
eprintln!("usage: correlate_capture <capture.log> <Stage_SNN> <ship_id> [ref_part_substr]");
std::process::exit(2);
}
let (log, stage, id) = (&args[1], &args[2], &args[3]);
let ref_sub = args.get(4).map(|s| s.as_str()).unwrap_or("bdy_04");
let iso = std::env::var("SYLPHEED_ISO").expect("SYLPHEED_ISO");
let draws = parse(&std::fs::read_to_string(log).expect("read log"));
eprintln!("parsed {} draws ({} with WorldView)", draws.len(), draws.iter().filter(|d| d.ok).count());
let bytes = {
let rt = tokio::runtime::Builder::new_current_thread().enable_all().build().unwrap();
rt.block_on(async {
let mut r = open_iso(Path::new(&iso)).await.unwrap();
r.read_file(&format!("hidden/resource3d/{stage}.xpr")).await.unwrap()
})
};
let names = xbg7_resource_names(&bytes);
let parts: Vec<String> =
names.iter().filter(|n| is_base_part(n) && ship_id_of(n) == Some(id.as_str())).cloned().collect();
let want: HashSet<String> = parts.iter().cloned().collect();
let models = Xbg7Model::models_named(&bytes, &want, &|| false);
// Match each part to a captured draw by vertex count; keep its WorldView.
struct Placed {
part: String,
r: M3,
t: [f64; 3],
local: Vec<[f64; 3]>,
}
let mut placed: Vec<Placed> = Vec::new();
let mut used: HashSet<u32> = HashSet::new();
for part in &parts {
let Some(m) = models.iter().find(|m| &m.name == part) else { continue };
let local: Vec<[f64; 3]> =
m.meshes.iter().flat_map(|s| s.positions.iter()).map(|p| [p[0] as f64, p[1] as f64, p[2] as f64]).collect();
let vc = local.len() as u32;
if let Some(d) = draws.iter().find(|d| d.ok && d.vcount == vc && !used.contains(&d.vbase)) {
used.insert(d.vbase);
placed.push(Placed { part: part.clone(), r: d.r, t: d.t, local });
} else {
println!("{part:20} (no matching captured draw — occluded/culled?)");
}
}
// Reference part → ship frame. rel_p = WV_ref⁻¹ · WV_p (camera cancels).
let Some(rf) = placed.iter().find(|p| p.part.contains(ref_sub)).or(placed.first()) else {
eprintln!("no parts placed");
return;
};
let rt_ref = transpose(&rf.r);
let ref_t = rf.t;
println!("\nreference = {} → ship-relative placement (M rows, T):", rf.part);
let mut lo = [f64::MAX; 3];
let mut hi = [f64::MIN; 3];
for p in &placed {
// rel R = Rᵀ_ref · R_p ; rel T = Rᵀ_ref · (T_p − T_ref)
let rel_r = mmul(&rt_ref, &p.r);
let dt = [p.t[0] - ref_t[0], p.t[1] - ref_t[1], p.t[2] - ref_t[2]];
let rel_t = mt_apply(&rt_ref, &[0.0; 3], dt);
// Assembled centroid (validation) + overall extent.
let mut c = [0.0; 3];
for v in &p.local {
let w = mt_apply(&rel_r, &rel_t, *v);
for k in 0..3 {
c[k] += w[k];
lo[k] = lo[k].min(w[k]);
hi[k] = hi[k].max(w[k]);
}
}
let n = p.local.len().max(1) as f64;
println!(
" {:18} T=[{:8.1}{:8.1}{:8.1}] centroid=[{:8.1}{:8.1}{:8.1}]",
p.part, rel_t[0], rel_t[1], rel_t[2], c[0] / n, c[1] / n, c[2] / n
);
}
println!(
"\nassembled extent = [{:.0} {:.0} {:.0}] ({} parts placed)",
hi[0] - lo[0], hi[1] - lo[1], hi[2] - lo[2], placed.len()
);
}

1
docs/re/INDEX.md
View File

@@ -20,6 +20,7 @@ Promote to a prose `structures/…md` file when a format needs behavioural notes
| IXUD subtitle | 🟡 | `sylpheed-formats/src/ixud.rs` | timed cues; **movie↔track link unknown** (dynamic item) |
| Fonts (ttf/otf/ttc) | ✅ | `sylpheed-formats/src/font.rs` | standard OpenType, parsed via ttf-parser |
| XBG7 mesh | 🟡/❔ | `sylpheed-formats/src/mesh.rs` + `tests/mesh_disc.rs` ([xbg7](structures/xbg7-mesh.md)) | weapons/props: declaration-driven variable stride (36 models), GPU-confirmed. **Stage containers: 5662 sub-models across 22 stages** via content-anchored grouped pools (`stage_models`). Quantized hero bodies (DeltaSaber `f004`) still declined |
| Capital-ship part placement | 🟡 | `sylpheed-formats/src/ship.rs` (static) + [runtime capture](ship-placement-runtime-capture.md) | hull placement static-exact; external parts approximate statically. **Runtime capture** (Canary F10 → VS-constant WorldView) gives ground truth — validated on `e106` destroyer; not yet baked into the viewer |
## Functions / code paths

105
docs/re/ship-placement-runtime-capture.md Normal file
View File

@@ -0,0 +1,105 @@
# Capital-ship part placement — runtime capture (ground truth)
**Status:** pipeline working end-to-end for one ship (`e106` ADAN Destroyer);
correlator recovers exact ship-relative transforms. Not yet baked into the viewer.
## Problem
Capital ships ship as **split XBG7 parts** (hull bodies, bridge, engines, turrets)
in `hidden/resource3d/Stage_SNN.xpr`. Reconstructing the whole ship needs each
part's placement. Static reverse-engineering of the composite scene graph
(`e_rou_<id>`, `GN_*` frames) got the **hull right but external parts only
approximately** — a `GN_*` frame is the mount *pivot*, and the real outboard /
rotational offset lives in a detail sub-rig or in runtime code not recoverable
statically. See `sylpheed-formats::ship::assemble_ship` (the static best-effort).
## Finding
The **guest vertex buffer holds LOCAL coordinates** — byte-identical to the
`.xpr` (verified: `e106_bdy_04`'s first vertex `(-20.0000,153.1539,45.9997)`
matches the decode exactly). So parts are placed **entirely in the vertex
shader**; the buffer carries no placement. (Contrast: static *stage* geometry is
pre-transformed into world space and byte-matches the `.xpr` at file offsets —
that's a different case.)
The per-part transform is in the **vertex-shader float constants**. For the ship
shader (`vs=0xC7F781F4C1D58054`), constants `c0..c2` are the **WorldViewProjection**
matrix rows:
- Their norms are `(sx, sy, 1)` = the projection x/y scales; `sy/sx ≈ 1.78 = 16:9`
aspect. Dividing each row by its norm gives the rigid **WorldView** (verified:
normalized rows orthonormal to 1e-4, `det = +1`).
- The camera View cancels when expressing every part relative to a reference part:
`rel_p = WV_ref⁻¹ · WV_p` = `(Rᵀ_ref·R_p , Rᵀ_ref·(T_p − T_ref))` — a pure
ship-space rigid transform. Applying `rel_p` to part `p`'s local geometry
assembles the ship exactly, in the reference part's frame.
(`c4..c14` also vary per part — likely a normal matrix / a second WVP variant /
WorldView split; not needed, `c0..c2` suffice.)
## Capture instrumentation
Branch **`capture-ship-placement`** in the `xenia-canary-native` worktree (off the
`instrument-current`-descended native branch, which already has GPU `log_draws`).
- **F10** in the emulator window → `xe::gpu::RequestShipCaptureFrame()`.
- Dumps the next ≤8000 draws (de-duped by vertex-buffer address) to
**`xenia_ship_capture.log`** in the binary's dir
(`build/bin/Linux/Release/`). Per draw:
```
DRAW vbase=0x… stride=… vcount=… indices=… prim=… vs=0x…
pos: (x,y,z) … up to 64 LOCAL vertex positions (== .xpr)
vsconst base=N: c0=(x,y,z,w) c1=… … first 48 VS float4 constants
```
- Files: `src/xenia/gpu/command_processor.{cc,h}` (`CaptureShipDrawForRE`,
`RequestShipCaptureFrame`), `src/xenia/app/emulator_window.cc` (F10 key).
- Run: `run-canary-native.sh` (HW Vulkan, interactive). Play into the mission,
frame the ship side-on, press F10.
## Correlator
`sylpheed-formats/examples/correlate_capture.rs`:
```
SYLPHEED_ISO=… cargo run --release --example correlate_capture -- \
<capture.log> <Stage_SNN> <ship_id> [ref_part_substr]
```
Parses the log, decodes the ship's base parts, matches each part to a draw by
**vertex count** (unique per part), recovers WorldView from `c0..c2`, and prints
each part's ship-relative transform (reference-part frame) + assembled extent.
## Validated result — `e106` ADAN Destroyer (Mission 1 / Stage_S01, side view)
Vertex-count → part map (all matched exactly): bdy_01/02 = 1261, bdy_03 = 815,
bdy_04 = 1633, eng_01 = 583, eng_02 = 491, wep_02_01 = 1002, brg_01 = 202.
Ship-relative placement (reference = `bdy_04`, aft hull):
| part | T (ship-relative) | note |
|---|---|---|
| bdy_04 | (0, 0, 0) | reference, aft hull |
| eng_01 | (131, −133, −132) | **aft** (−Z), starboard |
| eng_02 | (0, −49, −140) | aft, centre |
| bdy_01 | (**−264**, −150, 1165) | **port** hull, forward |
| bdy_02 | (**+264**, −150, 1165) | **starboard** hull, forward |
| bdy_03 | (0, −35, 1076) | forward keel/spine |
| wep_02_01 | (0, 49, 1009) | forward weapon |
Assembled extent **872 × 670 × 2181** (a coherent ~2181-long destroyer).
**Key wins vs static:** `bdy_01`/`bdy_02` are a **port/starboard pair** (X = ∓264),
which static had overlapping at one point; engines land correctly aft; the ship is
not the 1894-tall tower the static Y put out.
**Missing:** `brg_01` (bridge, vcount 202) was **culled/occluded** at that camera
angle (0 draws) — needs a second capture framing the superstructure.
## Next steps
1. **Bridge:** re-capture with the bridge visible → fills `brg_01`.
2. **Bake:** promote the correlator to a module; emit a per-ship placement table
(`ship_id → [(part, R, T)]`) as a checked-in data file; have the viewer's
`build_ship_model` prefer the captured table over `assemble_ship` when present.
3. **Other ships:** same capture for the cruiser (`e105`, Stage_S02), ACROPOLIS
(`f101`), TCAF cruiser/destroyer (`f105`/`f106`). One side-on F10 each.
4. **Turrets:** the shared `e3NN`/`e4NN` gun models appear as their own draws in the
capture too — correlate them to place turrets (static never resolved these).