re(xbg7): the [index][vertex] layout is runtime-verified, and the indices= mystery was batching
The decoder's central unstated assumption — a block's index buffer sits immediately before its vertex buffer (`vb - idx_count*2 - pad`, pad <= 3) — was also the prime suspect for the residual anchor misses, since a capture-proven `e106_eng_02_l` block was rejected outright. Measured it instead of assuming: - extended the F10 ship capture to log each draw's index buffer (base, count, min/max index) and to key its de-dup on the index range, so every draw batch is recorded rather than only the first; - `examples/capture_ib_truth.rs` places each drawn buffer in the container by its dumped positions and scores the capture against our decode. Stage_S02, 42 drawn buffers placed: our idx_count == the sum of the draw's index batches for 42/42, the batch union covers the vertex pool exactly for 42/42, and all 30 single-block cases sit at pad <= 3 (20 at pad 0, 10 at pad 2). The other 12 are grouped pools, where one index pool serves the whole group. So the layout holds, the decoded index count is exact, and eng_02_l died on the connectivity gate (since replaced by the winding gate) — not on index location. The shipped exact-coverage rule is independently confirmed. The recorded "capture indices=21 vs our 246" disagreement was an artefact of the old de-dup key: 21 was the first of two batches, 21 + 225 = 246. Any conclusion from a pre-2026-08-13 capture's `indices=` or `vbase - ibase` is about one batch, not about the block. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
This commit is contained in:
196
crates/sylpheed-formats/examples/capture_ib_truth.rs
Normal file
196
crates/sylpheed-formats/examples/capture_ib_truth.rs
Normal file
@@ -0,0 +1,196 @@
|
|||||||
|
//! Where does a drawn block's INDEX buffer really live?
|
||||||
|
//!
|
||||||
|
//! Our XBG7 anchor scan only ever *assumes* the layout `[index buffer][vertex
|
||||||
|
//! buffer]` with a pad of at most 3 bytes between them (`anchor_pool_mesh`:
|
||||||
|
//! `ib = vb - idx_count*2 - pad`). Nothing on disc states it, and it is the gate
|
||||||
|
//! that rejected the capture-proven `e106_eng_02_l` block
|
||||||
|
//! (docs/re/captures/…): the block's index data was not where the decoder
|
||||||
|
//! looked. The F10 ship capture was extended on 2026-08-13 to log each draw's
|
||||||
|
//! index-buffer base, count and min/max index value, so the assumption is now
|
||||||
|
//! directly checkable:
|
||||||
|
//!
|
||||||
|
//! * `vbase - ibase` is the real gap in guest memory, and a stage container is
|
||||||
|
//! uploaded contiguously (see `shared_vbase_check`), so the same difference
|
||||||
|
//! holds in the file;
|
||||||
|
//! * `max index vs vcount` says whether a draw covers its whole vertex pool —
|
||||||
|
//! the "buffer not covered" miss class is a *sub-range draw* if it does not.
|
||||||
|
//!
|
||||||
|
//! Usage:
|
||||||
|
//! cargo run --release --example capture_ib_truth -- <Stage_SNN.xpr> <capture.log>...
|
||||||
|
use sylpheed_formats::mesh::Xbg7Model;
|
||||||
|
use sylpheed_formats::ship_capture::{parse_capture, CapturedDraw};
|
||||||
|
use std::collections::HashMap;
|
||||||
|
|
||||||
|
fn q(v: f32) -> i64 {
|
||||||
|
(v as f64 * 1e4).round() as i64
|
||||||
|
}
|
||||||
|
|
||||||
|
fn main() {
|
||||||
|
let a: Vec<String> = std::env::args().collect();
|
||||||
|
if a.len() < 3 {
|
||||||
|
eprintln!("usage: capture_ib_truth <container.xpr> <capture.log>...");
|
||||||
|
std::process::exit(2);
|
||||||
|
}
|
||||||
|
let bytes = std::fs::read(&a[1]).expect("container");
|
||||||
|
|
||||||
|
// One entry per (log, vbase): the capture already de-dups per placement, and
|
||||||
|
// a buffer drawn at several transforms has the same index buffer each time.
|
||||||
|
let mut draws: Vec<CapturedDraw> = Vec::new();
|
||||||
|
let mut seen = std::collections::HashSet::new();
|
||||||
|
for log in &a[2..] {
|
||||||
|
let text = std::fs::read_to_string(log).expect("log");
|
||||||
|
for d in parse_capture(&text) {
|
||||||
|
// One entry per (log, vbase, index range): the engine issues SEVERAL
|
||||||
|
// draws over one vertex buffer, each with its own index sub-range, and
|
||||||
|
// it is their UNION that describes the block. (Captures taken before
|
||||||
|
// 2026-08-13 de-dup by (vbase, transform) and so hold only the first
|
||||||
|
// batch — such a log reads as a mysteriously short draw.)
|
||||||
|
let k = d.ib.map(|i| (i.ibase, i.icount)).unwrap_or((0, 0));
|
||||||
|
if d.ib.is_some() && d.pos.len() >= 4 && seen.insert((log.clone(), d.vbase, k)) {
|
||||||
|
draws.push(d);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
eprintln!("{} drawn buffers with an index buffer", draws.len());
|
||||||
|
|
||||||
|
// ── Place the drawn buffers in the file: POSITION is f32×3 big-endian at
|
||||||
|
// vertex offset 0, so the dumped positions are a literal byte pattern.
|
||||||
|
let be = |at: usize| f32::from_be_bytes(bytes[at..at + 4].try_into().unwrap());
|
||||||
|
let mut index: HashMap<(i64, i64, i64), Vec<u32>> = HashMap::new();
|
||||||
|
let mut o = 0usize;
|
||||||
|
while o + 12 <= bytes.len() {
|
||||||
|
let (x, y, z) = (be(o), be(o + 4), be(o + 8));
|
||||||
|
if x.is_finite() && y.is_finite() && z.is_finite() && x.abs() < 1e6 && y.abs() < 1e6 && z.abs() < 1e6 {
|
||||||
|
index.entry((q(x), q(y), q(z))).or_default().push(o as u32);
|
||||||
|
}
|
||||||
|
o += 4;
|
||||||
|
}
|
||||||
|
let mut deltas: HashMap<i64, usize> = HashMap::new();
|
||||||
|
let mut hits: Vec<(i64, usize, &CapturedDraw)> = Vec::new(); // (delta, file offset, draw)
|
||||||
|
for d in &draws {
|
||||||
|
let k = (q(d.pos[0][0]), q(d.pos[0][1]), q(d.pos[0][2]));
|
||||||
|
for dx in -1..=1i64 {
|
||||||
|
for dy in -1..=1i64 {
|
||||||
|
for dz in -1..=1i64 {
|
||||||
|
let Some(cands) = index.get(&(k.0 + dx, k.1 + dy, k.2 + dz)) else { continue };
|
||||||
|
for &off in cands {
|
||||||
|
for stride in (12..=64).step_by(4) {
|
||||||
|
let ok = (1..4).all(|j| {
|
||||||
|
let at = off as usize + j * stride;
|
||||||
|
at + 12 <= bytes.len()
|
||||||
|
&& (0..3).all(|c| (be(at + c * 4) - d.pos[j][c]).abs() <= 1e-4)
|
||||||
|
});
|
||||||
|
if ok {
|
||||||
|
let delta = d.vbase as i64 - off as i64;
|
||||||
|
*deltas.entry(delta).or_default() += 1;
|
||||||
|
hits.push((delta, off as usize, d));
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let Some((&base_delta, &n)) = deltas.iter().max_by_key(|(_, n)| **n) else {
|
||||||
|
eprintln!("no draw could be placed in this container");
|
||||||
|
std::process::exit(1);
|
||||||
|
};
|
||||||
|
println!("container load constant: vbase - file_offset = 0x{base_delta:X} ({n} buffers agree)");
|
||||||
|
|
||||||
|
// ── Our decoder's view of the same container.
|
||||||
|
let models = Xbg7Model::stage_models(&bytes);
|
||||||
|
let mut by_off: HashMap<usize, Vec<(String, usize, usize)>> = HashMap::new();
|
||||||
|
for m in &models {
|
||||||
|
for sm in &m.meshes {
|
||||||
|
if let Some(off) = sm.vbuf_offset {
|
||||||
|
by_off
|
||||||
|
.entry(off)
|
||||||
|
.or_default()
|
||||||
|
.push((m.name.clone(), sm.positions.len(), sm.indices.len()));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
println!("decoded {} resources, {} distinct vertex offsets\n", models.len(), by_off.len());
|
||||||
|
|
||||||
|
// ── The report: one row per drawn BUFFER, aggregating its index batches.
|
||||||
|
let mut per_buf: HashMap<u32, (usize, Vec<sylpheed_formats::ship_capture::CapturedIndexBuffer>, u32)> =
|
||||||
|
HashMap::new();
|
||||||
|
for (delta, voff, d) in &hits {
|
||||||
|
if *delta != base_delta {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let e = per_buf.entry(d.vbase).or_insert((*voff, Vec::new(), d.vcount));
|
||||||
|
let ib = d.ib.unwrap();
|
||||||
|
if !e.1.contains(&ib) {
|
||||||
|
e.1.push(ib);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let (mut pad0, mut pad_small, mut pad_off, mut unnamed) = (0usize, 0usize, 0usize, 0usize);
|
||||||
|
let (mut cover_exact, mut cover_short, mut idx_equal, mut idx_partial) = (0usize, 0usize, 0usize, 0usize);
|
||||||
|
let mut rows: Vec<(usize, String)> = Vec::new();
|
||||||
|
for (_, (voff, ibs, vcount)) in per_buf.iter() {
|
||||||
|
let batches = ibs.len();
|
||||||
|
let total: u32 = ibs.iter().map(|i| i.icount).sum();
|
||||||
|
let lo = ibs.iter().map(|i| i.ibase).min().unwrap() as i64 - base_delta;
|
||||||
|
let hi = ibs.iter().map(|i| i.ibase + i.icount * 2).max().unwrap() as i64 - base_delta;
|
||||||
|
let umax = ibs.iter().map(|i| i.imax).max().unwrap();
|
||||||
|
let gap = *voff as i64 - hi; // bytes from the end of the index data to the vertex buffer
|
||||||
|
let names = by_off.get(voff);
|
||||||
|
let dec_idx = names
|
||||||
|
.and_then(|v| v.iter().find(|(_, p, _)| *p as u32 == *vcount).map(|(_, _, i)| *i as u32));
|
||||||
|
// The decoder's assumption, scored: it expects the whole index buffer at
|
||||||
|
// `vb - 2*idx_count - pad`, pad ≤ 3.
|
||||||
|
let dec_pad = dec_idx.map(|i| *voff as i64 - (i as i64) * 2 - lo);
|
||||||
|
match dec_pad {
|
||||||
|
Some(0) => pad0 += 1,
|
||||||
|
Some(p) if (1..=3).contains(&p) => pad_small += 1,
|
||||||
|
Some(_) => pad_off += 1,
|
||||||
|
None => unnamed += 1,
|
||||||
|
}
|
||||||
|
if umax + 1 == *vcount {
|
||||||
|
cover_exact += 1;
|
||||||
|
} else {
|
||||||
|
cover_short += 1;
|
||||||
|
}
|
||||||
|
match dec_idx {
|
||||||
|
Some(i) if i == total => idx_equal += 1,
|
||||||
|
Some(_) => idx_partial += 1,
|
||||||
|
None => {}
|
||||||
|
}
|
||||||
|
rows.push((
|
||||||
|
*voff,
|
||||||
|
format!(
|
||||||
|
"vb 0x{:07X} v={:<6} batches {:<3} idx {:<6} span {:<7} gap {:<8} decpad {:<7} cover {:<10} {}",
|
||||||
|
voff,
|
||||||
|
vcount,
|
||||||
|
batches,
|
||||||
|
total,
|
||||||
|
hi - lo,
|
||||||
|
gap,
|
||||||
|
dec_pad.map(|p| p.to_string()).unwrap_or_else(|| "?".into()),
|
||||||
|
if umax + 1 == *vcount { "exact".to_string() } else { format!("{}/{}", umax, vcount - 1) },
|
||||||
|
names
|
||||||
|
.map(|v| v
|
||||||
|
.iter()
|
||||||
|
.map(|(n, p, i)| format!("{n}(v{p},i{i})"))
|
||||||
|
.collect::<Vec<_>>()
|
||||||
|
.join(" "))
|
||||||
|
.unwrap_or_else(|| "-".into())
|
||||||
|
),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
rows.sort();
|
||||||
|
for (_, r) in &rows {
|
||||||
|
println!("{r}");
|
||||||
|
}
|
||||||
|
println!("\nplaced {} drawn buffers in this container", rows.len());
|
||||||
|
println!(
|
||||||
|
"DECODER layout assumption — whole index buffer at vb - 2*idx_count - pad: pad 0 {pad0} · pad 1..3 {pad_small} · elsewhere {pad_off} · not decoded here {unnamed}"
|
||||||
|
);
|
||||||
|
println!(
|
||||||
|
"index extent: our idx_count == sum of captured batches for {idx_equal} buffers, differs for {idx_partial}"
|
||||||
|
);
|
||||||
|
println!("vertex-pool coverage by the union of batches: exact {cover_exact} · short {cover_short}");
|
||||||
|
}
|
||||||
@@ -49,6 +49,27 @@ pub struct CapturedDraw {
|
|||||||
/// First few LOCAL vertex positions dumped with the draw (buffer order).
|
/// First few LOCAL vertex positions dumped with the draw (buffer order).
|
||||||
/// Used to disambiguate same-vcount twins (mirrored port/starboard parts).
|
/// Used to disambiguate same-vcount twins (mirrored port/starboard parts).
|
||||||
pub pos: Vec<[f32; 3]>,
|
pub pos: Vec<[f32; 3]>,
|
||||||
|
/// The draw's INDEX buffer, when the capture recorded one (`ib base=…`,
|
||||||
|
/// added 2026-08-13): guest base address, index count, and the min/max index
|
||||||
|
/// value the emulator read out of guest memory. `None` for older logs and
|
||||||
|
/// for auto-index draws. This is ground truth for two things the offline
|
||||||
|
/// decoder can only assume — where a block's index buffer lives relative to
|
||||||
|
/// its vertex buffer, and how much of the vertex pool a draw really covers.
|
||||||
|
pub ib: Option<CapturedIndexBuffer>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The index buffer a captured draw used. See [`CapturedDraw::ib`].
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
|
pub struct CapturedIndexBuffer {
|
||||||
|
/// Guest base address of the index data.
|
||||||
|
pub ibase: u32,
|
||||||
|
/// Number of indices the draw issued (== `VGT_DRAW_INITIATOR.num_indices`).
|
||||||
|
pub icount: u32,
|
||||||
|
/// Lowest index value in the buffer.
|
||||||
|
pub imin: u32,
|
||||||
|
/// Highest index value in the buffer — with `vcount` this says whether the
|
||||||
|
/// draw covers its whole vertex pool or only a sub-range.
|
||||||
|
pub imax: u32,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// A ship part to match against the capture. `part` is the **base** part name
|
/// A ship part to match against the capture. `part` is the **base** part name
|
||||||
@@ -100,13 +121,16 @@ pub fn parse_capture(text: &str) -> Vec<CapturedDraw> {
|
|||||||
let mut vcount = 0u32;
|
let mut vcount = 0u32;
|
||||||
let mut pos: Vec<[f32; 3]> = Vec::new();
|
let mut pos: Vec<[f32; 3]> = Vec::new();
|
||||||
let mut consts: Vec<(usize, [f64; 4])> = Vec::new();
|
let mut consts: Vec<(usize, [f64; 4])> = Vec::new();
|
||||||
|
let mut ib: Option<CapturedIndexBuffer> = None;
|
||||||
|
|
||||||
let flush = |vbase: u32,
|
let flush = |vbase: u32,
|
||||||
vcount: u32,
|
vcount: u32,
|
||||||
pos: &mut Vec<[f32; 3]>,
|
pos: &mut Vec<[f32; 3]>,
|
||||||
|
ib: &mut Option<CapturedIndexBuffer>,
|
||||||
consts: &[(usize, [f64; 4])],
|
consts: &[(usize, [f64; 4])],
|
||||||
out: &mut Vec<CapturedDraw>| {
|
out: &mut Vec<CapturedDraw>| {
|
||||||
let pos = std::mem::take(pos);
|
let pos = std::mem::take(pos);
|
||||||
|
let ib = ib.take();
|
||||||
if vbase == 0 {
|
if vbase == 0 {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
@@ -115,18 +139,33 @@ pub fn parse_capture(text: &str) -> Vec<CapturedDraw> {
|
|||||||
return; // no WorldView for this draw — skip it
|
return; // no WorldView for this draw — skip it
|
||||||
};
|
};
|
||||||
if let Some((r, t)) = normalize_wvp([c0, c1, c2]) {
|
if let Some((r, t)) = normalize_wvp([c0, c1, c2]) {
|
||||||
out.push(CapturedDraw { vbase, vcount, r, t, pos });
|
out.push(CapturedDraw { vbase, vcount, r, t, pos, ib });
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
for line in text.lines() {
|
for line in text.lines() {
|
||||||
let l = line.trim();
|
let l = line.trim();
|
||||||
if let Some(rest) = l.strip_prefix("DRAW ") {
|
if let Some(rest) = l.strip_prefix("DRAW ") {
|
||||||
flush(vbase, vcount, &mut pos, &consts, &mut out);
|
flush(vbase, vcount, &mut pos, &mut ib, &consts, &mut out);
|
||||||
consts.clear();
|
consts.clear();
|
||||||
let f = |k: &str| rest.split_whitespace().find_map(|t| t.strip_prefix(k));
|
let f = |k: &str| rest.split_whitespace().find_map(|t| t.strip_prefix(k));
|
||||||
vbase = f("vbase=0x").and_then(|s| u32::from_str_radix(s, 16).ok()).unwrap_or(0);
|
vbase = f("vbase=0x").and_then(|s| u32::from_str_radix(s, 16).ok()).unwrap_or(0);
|
||||||
vcount = f("vcount=").and_then(|s| s.parse().ok()).unwrap_or(0);
|
vcount = f("vcount=").and_then(|s| s.parse().ok()).unwrap_or(0);
|
||||||
|
} else if let Some(rest) = l.strip_prefix("ib base=0x") {
|
||||||
|
// `ib base=0x… count=N fmt=u16 endian=E len=L delta_vb=D min=a max=b idx: …`
|
||||||
|
let f = |k: &str| rest.split_whitespace().find_map(|t| t.strip_prefix(k));
|
||||||
|
let base = rest
|
||||||
|
.split_whitespace()
|
||||||
|
.next()
|
||||||
|
.and_then(|s| u32::from_str_radix(s, 16).ok());
|
||||||
|
if let (Some(ibase), Some(icount)) = (base, f("count=").and_then(|s| s.parse().ok())) {
|
||||||
|
ib = Some(CapturedIndexBuffer {
|
||||||
|
ibase,
|
||||||
|
icount,
|
||||||
|
imin: f("min=").and_then(|s| s.parse().ok()).unwrap_or(0),
|
||||||
|
imax: f("max=").and_then(|s| s.parse().ok()).unwrap_or(0),
|
||||||
|
});
|
||||||
|
}
|
||||||
} else if l.starts_with("pos:") || l.starts_with("positions:") {
|
} else if l.starts_with("pos:") || l.starts_with("positions:") {
|
||||||
pos = parse_pos_line(l, 8);
|
pos = parse_pos_line(l, 8);
|
||||||
} else if l.starts_with("vsconst") {
|
} else if l.starts_with("vsconst") {
|
||||||
@@ -147,7 +186,7 @@ pub fn parse_capture(text: &str) -> Vec<CapturedDraw> {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
flush(vbase, vcount, &mut pos, &consts, &mut out);
|
flush(vbase, vcount, &mut pos, &mut ib, &consts, &mut out);
|
||||||
out
|
out
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -242,7 +281,9 @@ pub fn parse_drawlog(text: &str) -> Vec<CapturedDraw> {
|
|||||||
let get = |i: usize| consts.iter().find(|(k, _)| *k == i).map(|(_, v)| *v);
|
let get = |i: usize| consts.iter().find(|(k, _)| *k == i).map(|(_, v)| *v);
|
||||||
let (Some(c0), Some(c1), Some(c2)) = (get(0), get(1), get(2)) else { return };
|
let (Some(c0), Some(c1), Some(c2)) = (get(0), get(1), get(2)) else { return };
|
||||||
if let Some((r, t)) = normalize_wvp([c0, c1, c2]) {
|
if let Some((r, t)) = normalize_wvp([c0, c1, c2]) {
|
||||||
out.push(CapturedDraw { vbase: base, vcount: size / stride, r, t, pos });
|
// The draw-logger format carries an index base too, but it de-dups
|
||||||
|
// by vertex declaration, so it never lines up per part — left None.
|
||||||
|
out.push(CapturedDraw { vbase: base, vcount: size / stride, r, t, pos, ib: None });
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ Promote to a prose `structures/…md` file when a format needs behavioural notes
|
|||||||
| LSTA sprite list | ✅ | `sylpheed-formats/src/lsta.rs` | A display list of inline elements: **T8aD sprites and `PRMD` primitives**. The `count` at `0x04` is **exact and counts both** — `count == T8aD + PRMD` for **64/64** lists on the disc, which retires the old "a few entries disagree" note (it compared sprites against a total including primitives). **All 1 281 sprite frames decode** after the T8aD rectangle-list fix |
|
| LSTA sprite list | ✅ | `sylpheed-formats/src/lsta.rs` | A display list of inline elements: **T8aD sprites and `PRMD` primitives**. The `count` at `0x04` is **exact and counts both** — `count == T8aD + PRMD` for **64/64** lists on the disc, which retires the old "a few entries disagree" note (it compared sprites against a total including primitives). **All 1 281 sprite frames decode** after the T8aD rectangle-list fix |
|
||||||
| IXUD subtitle | 🟡/✅ | `sylpheed-formats/src/ixud.rs` + [movie link](movie-subtitle-link.md) | timed cues. **The movie↔subtitle↔voice link is solved — statically**, from the movie config record in `tables.pak` (schema `0x067025b9`), not from the running game as this row previously assumed: [101 movies mapped](captures/movie-subtitle-voice-map.csv), 94 with subtitles, 83 with voice, 21 with a telop overlay. 93 of 94 subtitle refs resolve in the language paks; **`SUBTITLE_S12B.tbl` is missing from all six languages** — a dangling reference on the disc. Naming is `SUBTITLE_<base>.tbl` / `VOICE_<base>` with six documented exceptions. The record's ~104 **script ids** are ❔ — positional pairing drifts by three because the IDXD pool dedupes repeated values |
|
| IXUD subtitle | 🟡/✅ | `sylpheed-formats/src/ixud.rs` + [movie link](movie-subtitle-link.md) | timed cues. **The movie↔subtitle↔voice link is solved — statically**, from the movie config record in `tables.pak` (schema `0x067025b9`), not from the running game as this row previously assumed: [101 movies mapped](captures/movie-subtitle-voice-map.csv), 94 with subtitles, 83 with voice, 21 with a telop overlay. 93 of 94 subtitle refs resolve in the language paks; **`SUBTITLE_S12B.tbl` is missing from all six languages** — a dangling reference on the disc. Naming is `SUBTITLE_<base>.tbl` / `VOICE_<base>` with six documented exceptions. The record's ~104 **script ids** are ❔ — positional pairing drifts by three because the IDXD pool dedupes repeated values |
|
||||||
| Fonts (ttf/otf/ttc) | ✅ | `sylpheed-formats/src/font.rs` | standard OpenType, parsed via ttf-parser |
|
| Fonts (ttf/otf/ttc) | ✅ | `sylpheed-formats/src/font.rs` | standard OpenType, parsed via ttf-parser |
|
||||||
| XBG7 mesh | ✅/🟡 | `sylpheed-formats/src/mesh.rs` + `tests/mesh_disc.rs` ([xbg7](structures/xbg7-mesh.md)) | **6 294 resources, 6 209 decode (98.7 %), 82 searched-and-missed** (2026-08-12, up from 5 480 / 87.1 %). Five evidence-driven fixes got there: **distinct anchor assignment** (no two resources may claim one buffer — proved by a capture showing the container holds both mirrored `e106` hull halves), the connectivity cap replaced by a **winding-consistency gate at 0.70**, **structural requirements on pre-pivot sub-meshes** (index range, then exact pool coverage), and **filtering after the assignment** so a subset query cannot differ from the full decode. Validated against a runtime capture that names the file offset of every buffer the engine drew: **46/46 drawn buffers claimed, 45 anchored exactly**. **No real mesh now decodes differently in different containers** — all 89 remaining cross-container disagreements are interchangeable 24-vertex bounding boxes, which no anchoring rule can pin (monotone order re-tested and refuted). Remaining misses attribute to the degeneracy/extent gate (42), winding (31) and coverage (9); the first was probed and its "obvious" fix refuted. Every decoded sub-mesh covers its own vertex pool |
|
| XBG7 mesh | ✅/🟡 | `sylpheed-formats/src/mesh.rs` + `tests/mesh_disc.rs` ([xbg7](structures/xbg7-mesh.md)) | **6 294 resources, 6 209 decode (98.7 %), 82 searched-and-missed** (2026-08-12, up from 5 480 / 87.1 %). Five evidence-driven fixes got there: **distinct anchor assignment** (no two resources may claim one buffer — proved by a capture showing the container holds both mirrored `e106` hull halves), the connectivity cap replaced by a **winding-consistency gate at 0.70**, **structural requirements on pre-pivot sub-meshes** (index range, then exact pool coverage), and **filtering after the assignment** so a subset query cannot differ from the full decode. Validated against a runtime capture that names the file offset of every buffer the engine drew: **46/46 drawn buffers claimed, 45 anchored exactly**. **No real mesh now decodes differently in different containers** — all 89 remaining cross-container disagreements are interchangeable 24-vertex bounding boxes, which no anchoring rule can pin (monotone order re-tested and refuted). Remaining misses attribute to the degeneracy/extent gate (42), winding (31) and coverage (9); the first was probed and its "obvious" fix refuted. Every decoded sub-mesh covers its own vertex pool. **The `[index buffer][vertex buffer]` layout is now runtime-verified** (2026-08-13): with the F10 capture extended to log each draw's index buffer, all **42** drawn `Stage_S02` buffers match our decoded index count exactly, all 42 have their index union cover the pool exactly, and the 30 single-block cases all sit at `pad ≤ 3` — so `e106_eng_02_l`'s old rejection was the connectivity gate, not a misplaced index buffer. The `indices=` mystery was the capture keeping only the **first of several index batches** per buffer |
|
||||||
| Capital-ship part placement | ✅ | `sylpheed-formats/src/ship.rs` (static) + [runtime capture](ship-placement-runtime-capture.md) | Placement is **sound** (hull static-exact against the `e106` capture; cross-id mounting genuinely narrow, 2 pairs across 335 ships). The XBG7 mis-decode this row used to blame for "ships assemble wrong" — a shared turret ~100× too large in some containers — is **fixed** (2026-08-12, the exact-coverage requirement): `e303_wep_01` now decodes 49×23×42 everywhere and places at ±179 on the `e106` hull, and no real mesh disagrees across containers. A composite-node audit confirmed the assembler itself never applied a bad scale (all nodes scale 1.0, orthonormal). Still open: `static_assembly_matches_runtime_capture` walks capture parts only, so **extra** static placements cannot fail it |
|
| Capital-ship part placement | ✅ | `sylpheed-formats/src/ship.rs` (static) + [runtime capture](ship-placement-runtime-capture.md) | Placement is **sound** (hull static-exact against the `e106` capture; cross-id mounting genuinely narrow, 2 pairs across 335 ships). The XBG7 mis-decode this row used to blame for "ships assemble wrong" — a shared turret ~100× too large in some containers — is **fixed** (2026-08-12, the exact-coverage requirement): `e303_wep_01` now decodes 49×23×42 everywhere and places at ±179 on the `e106` hull, and no real mesh disagrees across containers. A composite-node audit confirmed the assembler itself never applied a bad scale (all nodes scale 1.0, orthonormal). Still open: `static_assembly_matches_runtime_capture` walks capture parts only, so **extra** static placements cannot fail it |
|
||||||
| Weapon fields defaulted on disc | ✅ | [runtime struct](structures/weapon-struct-runtime.md) · [DATA SHEET route](weapon-datasheet-runtime.md) | **Solved.** Canary maps guest RAM into `/dev/shm`, so the parsed `Weapon`/`Shell` objects are readable live; their layout is solved against disc ground truth (zero contradictions over 100+ records). All 126 weapons, exact numbers, no story progress needed — [4 393 values](captures/weapon-runtime-fields.csv) the disc does not carry. Supersedes the letter-bucket limit of the DATA SHEET route, which now serves as the independent cross-check |
|
| Weapon fields defaulted on disc | ✅ | [runtime struct](structures/weapon-struct-runtime.md) · [DATA SHEET route](weapon-datasheet-runtime.md) | **Solved.** Canary maps guest RAM into `/dev/shm`, so the parsed `Weapon`/`Shell` objects are readable live; their layout is solved against disc ground truth (zero contradictions over 100+ records). All 126 weapons, exact numbers, no story progress needed — [4 393 values](captures/weapon-runtime-fields.csv) the disc does not carry. Supersedes the letter-bucket limit of the DATA SHEET route, which now serves as the independent cross-check |
|
||||||
| Unit (craft/vessel) fields defaulted on disc | ✅/🟡 | [runtime struct](structures/unit-struct-runtime.md) | The parsed `unit\UN_*.tbl` definition object, vtable `0x820af844`, ≥`0x380` bytes, one per unit — **discovered, not assumed** (`unit_discover.py`), and distinguished from the spawned-entity class `0x820af030` by being one-per-ID and byte-constant within a run. Across runs only pointer words move — `--crosscheck` proves **no reported field offset is run-dependent** (two words, `+0x2c8`/`+0x2d0`, are stage-dependent and remain unidentified). 27 fields ✅ (21 units, 7 runs); the `Maneuver` block is **schema declaration order, 4 bytes/field, base `0x9c` with a two-slot gap after `AA_Roll_Min`** (29 anchors, 0 conflicts), which also pins 5 fields *no* disc record ever values. Angles are **radians at runtime, degrees on disc**. **Re-derived independently 2026-08-13 from the loader's own key strings** (`sub_82341A20`; the field name for each store is a string in the image): **159 fields**, agreeing with this solver on **25 of 25 shared offsets**, verified at **406 values matching the disc and 0 disagreeing** over 11 live objects spanning UNIT and VESSEL — landed as `data/unit_definition_layout.txt` + `sylpheed_formats::unit_layout` + a no-emulator test, with **121 defaulted fields** read out ([live-unit-definitions](live-unit-definitions.md)). Unlike weapons, unit definitions are instantiated **per stage**, so coverage (21/110) grows by visiting missions — but a defaulted field is **not** a global constant: `Size_Y` provably inherits `Size_X` (7 independent units, 6 distinct values), and three more sibling rules are recorded ❔, recovering 65 values in units never visited — [values](captures/unit-runtime-fields.csv) |
|
| Unit (craft/vessel) fields defaulted on disc | ✅/🟡 | [runtime struct](structures/unit-struct-runtime.md) | The parsed `unit\UN_*.tbl` definition object, vtable `0x820af844`, ≥`0x380` bytes, one per unit — **discovered, not assumed** (`unit_discover.py`), and distinguished from the spawned-entity class `0x820af030` by being one-per-ID and byte-constant within a run. Across runs only pointer words move — `--crosscheck` proves **no reported field offset is run-dependent** (two words, `+0x2c8`/`+0x2d0`, are stage-dependent and remain unidentified). 27 fields ✅ (21 units, 7 runs); the `Maneuver` block is **schema declaration order, 4 bytes/field, base `0x9c` with a two-slot gap after `AA_Roll_Min`** (29 anchors, 0 conflicts), which also pins 5 fields *no* disc record ever values. Angles are **radians at runtime, degrees on disc**. **Re-derived independently 2026-08-13 from the loader's own key strings** (`sub_82341A20`; the field name for each store is a string in the image): **159 fields**, agreeing with this solver on **25 of 25 shared offsets**, verified at **406 values matching the disc and 0 disagreeing** over 11 live objects spanning UNIT and VESSEL — landed as `data/unit_definition_layout.txt` + `sylpheed_formats::unit_layout` + a no-emulator test, with **121 defaulted fields** read out ([live-unit-definitions](live-unit-definitions.md)). Unlike weapons, unit definitions are instantiated **per stage**, so coverage (21/110) grows by visiting missions — but a defaulted field is **not** a global constant: `Size_Y` provably inherits `Size_X` (7 independent units, 6 distinct values), and three more sibling rules are recorded ❔, recovering 65 values in units never visited — [values](captures/unit-runtime-fields.csv) |
|
||||||
|
|||||||
50
docs/re/captures/stage-s02-index-buffer-truth.txt
Normal file
50
docs/re/captures/stage-s02-index-buffer-truth.txt
Normal file
@@ -0,0 +1,50 @@
|
|||||||
|
container load constant: vbase - file_offset = 0x17FE3FF4 (93 buffers agree)
|
||||||
|
decoded 356 resources, 422 distinct vertex offsets
|
||||||
|
|
||||||
|
vb 0x13FD6E8 v=25448 batches 8 idx 52077 span 104154 gap 2 decpad 2 cover exact f101_bdy_01(v25448,i52077)
|
||||||
|
vb 0x192961C v=13172 batches 2 idx 24816 span 49632 gap 0 decpad 0 cover exact f101_bdy_02(v13172,i24816)
|
||||||
|
vb 0x1BF40F4 v=261 batches 3 idx 498 span 996 gap 41476 decpad 41476 cover exact e_rob_f001(v24,i36) e_rou_f003_Near(v24,i36) e_rou_f101_wep_01(v24,i36) e_rou_f105(v24,i36) _rou_f105_break(v261,i498) e_rou_f106(v24,i36) e_rou_f302_barrel(v24,i36) e_rou_f302_base(v24,i36) e_rou_f303_barrel(v24,i36) e_rou_f303_base(v24,i36) e_rou_e007_Far(v24,i36) e_rou_e007_Near(v24,i36) e_rou_e010_Far(v24,i36) e_rou_e010_Near(v24,i36) e_rou_e105(v24,i36) e_rou_e105_wep_01(v24,i36) e_rou_e106(v24,i36) e_rou_e106_eng(v24,i36) e_rou_e106_wep_02_01(v24,i36) e_rou_e106_wep_02_joint(v24,i36) e_rou_e108_Missile_open(v24,i36) e_rou_e201(v24,i36) e_rou_e302_barrel(v24,i36) e_rou_e302_base(v24,i36) e_rou_e303_barrel_Near(v24,i36) e_rou_e303_base_Near(v24,i36) e_rou_e501(v24,i36)
|
||||||
|
vb 0x1C0F67C v=82 batches 1 idx 144 span 288 gap 133672 decpad 133672 cover exact _rou_f105_break(v82,i144)
|
||||||
|
vb 0x1C1D92C v=220 batches 3 idx 384 span 768 gap 179872 decpad 179872 cover exact _rou_f105_break(v220,i384)
|
||||||
|
vb 0x1C1EDCC v=80 batches 2 idx 132 span 264 gap 184888 decpad 184888 cover exact _rou_f105_break(v80,i132)
|
||||||
|
vb 0x1C1F54C v=148 batches 3 idx 288 span 576 gap 186232 decpad 186232 cover exact _rou_f105_break(v148,i288)
|
||||||
|
vb 0x1C2032C v=156 batches 1 idx 246 span 492 gap 189292 decpad 189292 cover exact _rou_f105_break(v156,i246)
|
||||||
|
vb 0x1C2953C v=84 batches 1 idx 156 span 312 gap 220012 decpad 220012 cover exact _rou_f105_break(v84,i156)
|
||||||
|
vb 0x1C2AF1C v=122 batches 3 idx 351 span 702 gap 224882 decpad 224882 cover exact _rou_f105_break(v122,i351)
|
||||||
|
vb 0x1C2BA8C v=24 batches 2 idx 36 span 72 gap 227736 decpad 227736 cover exact _rou_f105_break(v24,i36)
|
||||||
|
vb 0x1ECB4C0 v=2446 batches 4 idx 5466 span 10932 gap 0 decpad 0 cover exact f105_bdy_01_m(v2446,i5466)
|
||||||
|
vb 0x227AC94 v=1692 batches 3 idx 3651 span 7302 gap 2 decpad 2 cover exact f106_bdy_02_l(v1692,i3651)
|
||||||
|
vb 0x22C2EF0 v=686 batches 2 idx 1506 span 3012 gap 0 decpad 0 cover exact f106_bdy_03_l(v686,i1506)
|
||||||
|
vb 0x22C8F50 v=1558 batches 3 idx 4098 span 8196 gap 0 decpad 0 cover exact f106_bdy_03_m(v1558,i4098)
|
||||||
|
vb 0x22ED420 v=261 batches 4 idx 561 span 1122 gap 2 decpad 2 cover exact f106_eng_01_l(v261,i561)
|
||||||
|
vb 0x22F0430 v=1162 batches 4 idx 3018 span 6036 gap 0 decpad 0 cover exact f106_eng_01_m(v1162,i3018)
|
||||||
|
vb 0x2326C9C v=254 batches 2 idx 495 span 990 gap 2 decpad 2 cover exact f106_sld_01_l(v254,i495)
|
||||||
|
vb 0x233B3FC v=254 batches 2 idx 495 span 990 gap 2 decpad 2 cover exact f106_sld_02_l(v254,i495)
|
||||||
|
vb 0x233D700 v=627 batches 3 idx 1434 span 2868 gap 0 decpad 0 cover exact f106_sld_02_m(v627,i1434)
|
||||||
|
vb 0x24CE5F4 v=32 batches 1 idx 60 span 120 gap 0 decpad 0 cover exact f303_body_l(v32,i60)
|
||||||
|
vb 0x28EA1D4 v=156 batches 2 idx 228 span 456 gap 0 decpad 0 cover exact e105_bdy_01_l(v156,i228)
|
||||||
|
vb 0x28F7514 v=107 batches 3 idx 180 span 360 gap 0 decpad 0 cover exact e105_bdy_02_l(v107,i180)
|
||||||
|
vb 0x291E1E8 v=185 batches 2 idx 285 span 570 gap 2 decpad 2 cover exact e105_bdy_03_l(v185,i285)
|
||||||
|
vb 0x29A4788 v=181 batches 3 idx 318 span 636 gap 0 decpad 0 cover exact e105_bdy_04_l(v181,i318)
|
||||||
|
vb 0x29B78B8 v=93 batches 2 idx 117 span 234 gap 2 decpad 2 cover exact e105_bdy_05_l(v93,i117)
|
||||||
|
vb 0x2A22574 v=41 batches 1 idx 51 span 102 gap 2 decpad 2 cover exact e105_bdy_06_l(v41,i51)
|
||||||
|
vb 0x2A47BAC v=77 batches 1 idx 111 span 222 gap 2 decpad 2 cover exact e105_brg_l(v77,i111)
|
||||||
|
vb 0x2A9FDA0 v=76 batches 1 idx 90 span 180 gap 0 decpad 0 cover exact e105_eng_01_l(v76,i90)
|
||||||
|
vb 0x2ACE840 v=60 batches 1 idx 90 span 180 gap 0 decpad 0 cover exact e105_wep_01_l(v60,i90)
|
||||||
|
vb 0x2D1FEE8 v=119 batches 2 idx 246 span 492 gap 0 decpad 0 cover exact e106_bdy_01_l(v119,i246)
|
||||||
|
vb 0x2D315D8 v=119 batches 2 idx 246 span 492 gap 0 decpad 0 cover exact e106_bdy_02_l(v119,i246)
|
||||||
|
vb 0x2D492C4 v=146 batches 2 idx 324 span 648 gap 0 decpad 0 cover exact e106_bdy_03_l(v146,i324)
|
||||||
|
vb 0x2D7363C v=179 batches 2 idx 399 span 798 gap 2 decpad 2 cover exact e106_bdy_04_l(v179,i399)
|
||||||
|
vb 0x2D7A418 v=51 batches 2 idx 126 span 252 gap 0 decpad 0 cover exact e106_brg_01_l(v51,i126)
|
||||||
|
vb 0x2DB0CCC v=58 batches 2 idx 96 span 192 gap 0 decpad 0 cover exact e106_eng_01_l(v58,i96)
|
||||||
|
vb 0x2DB632C v=44 batches 2 idx 72 span 144 gap 0 decpad 0 cover exact e106_eng_02_l(v44,i72)
|
||||||
|
vb 0x2DC305C v=82 batches 2 idx 168 span 336 gap 0 decpad 0 cover exact e106_wep_02_01_l(v82,i168)
|
||||||
|
vb 0x329077C v=6000 batches 1 idx 6000 span 12000 gap 0 decpad 0 cover exact n006_02(v6000,i6000)
|
||||||
|
vb 0x3434030 v=4 batches 1 idx 6 span 12 gap 24 decpad 24 cover exact n301_02B(v4,i6)
|
||||||
|
vb 0x3434090 v=4 batches 1 idx 6 span 12 gap 108 decpad 108 cover exact n301_02B(v4,i6)
|
||||||
|
vb 0x34340F0 v=4 batches 1 idx 6 span 12 gap 192 decpad 192 cover exact n301_02B(v4,i6)
|
||||||
|
|
||||||
|
placed 42 drawn buffers in this container
|
||||||
|
DECODER layout assumption — whole index buffer at vb - 2*idx_count - pad: pad 0 20 · pad 1..3 10 · elsewhere 12 · not decoded here 0
|
||||||
|
index extent: our idx_count == sum of captured batches for 42 buffers, differs for 0
|
||||||
|
vertex-pool coverage by the union of batches: exact 42 · short 0
|
||||||
@@ -1354,3 +1354,45 @@ Not settled: `e106_brg_01_b_02` ≡ `e106_brg_01_l` (51 verts). A second 51-vert
|
|||||||
holds three near-identical 51-vertex runs, so the pair has no oracle yet.
|
holds three near-identical 51-vertex runs, so the pair has no oracle yet.
|
||||||
`n006_01A` ≡ `n006_01B` shows a single `vbase` in all three logs — consistent
|
`n006_01A` ≡ `n006_01B` shows a single `vbase` in all three logs — consistent
|
||||||
with real reuse, but equally with only one of the two being on screen.
|
with real reuse, but equally with only one of the two being on screen.
|
||||||
|
|
||||||
|
## ✅ The `[index buffer][vertex buffer]` layout is RUNTIME-VERIFIED (2026-08-13)
|
||||||
|
|
||||||
|
Every anchor decision in this decoder rests on one assumption nothing on disc
|
||||||
|
states: a block's index buffer sits **immediately before** its vertex buffer, at
|
||||||
|
`vb − idx_count*2 − pad` with `pad ≤ 3` (`anchor_pool_mesh`). It was also the
|
||||||
|
prime suspect for the residual misses — the note above recorded a capture-proven
|
||||||
|
`e106_eng_02_l` block that the decoder *rejected*, and "the index buffer is
|
||||||
|
somewhere else" would have explained it.
|
||||||
|
|
||||||
|
It is now measured, not assumed. The F10 ship capture in `xenia-canary-native`
|
||||||
|
was extended to log each draw's index buffer (`ib base=… count=… min=… max=…`,
|
||||||
|
`CaptureShipDrawForRE`), and `examples/capture_ib_truth.rs` scores it against our
|
||||||
|
decode of the same container (`Stage_S02`, load constant `0x17FE3FF4`,
|
||||||
|
[full table](../captures/stage-s02-index-buffer-truth.txt)):
|
||||||
|
|
||||||
|
| measurement, 42 drawn buffers placed in the container | result |
|
||||||
|
|---|---|
|
||||||
|
| our `idx_count` == sum of the draw's index batches | **42 / 42** |
|
||||||
|
| union of batches covers the vertex pool exactly (`max_idx == vcount−1`) | **42 / 42** |
|
||||||
|
| index data at `vb − 2*idx_count − pad`, `pad ≤ 3` (single-block path) | **30 / 30** (20 at pad 0, 10 at pad 2) |
|
||||||
|
| the remaining 12 | all **grouped pools** — one index pool for the whole group, so the per-sub-mesh distance is larger by construction (`_rou_f105_break` sub-meshes, `n301_02B`) |
|
||||||
|
|
||||||
|
So the layout assumption is **correct**, the index count we decode is **exactly**
|
||||||
|
what the engine indexes, and `eng_02_l`'s rejection was the connectivity gate
|
||||||
|
(since replaced by the winding gate), not the index location. The coverage rule
|
||||||
|
shipped earlier (`XBG7_COVER_SLACK = 1`, "a real block reaches its last vertex")
|
||||||
|
is independently confirmed: every drawn buffer's index union ends exactly at
|
||||||
|
`vcount−1`.
|
||||||
|
|
||||||
|
### The trap that hid this: the capture kept only the FIRST batch
|
||||||
|
|
||||||
|
The engine issues **several draws over one vertex buffer**, each indexing a
|
||||||
|
sub-range (2–9 batches here; the player craft's 10 891-vertex buffer takes 9).
|
||||||
|
The capture de-duped by `(vbase, WVP transform)`, so it recorded one batch per
|
||||||
|
placement — which is exactly the recorded mystery *"the capture's `indices=` field
|
||||||
|
disagrees with the descriptor index count (119-vert twin draws log `indices=21`
|
||||||
|
vs our 246)"*. Not a disagreement: `21` was the first of two batches, and
|
||||||
|
`21 + 225 = 246`. Both twins now read `batches 2 · idx 246 · span 492 · gap 0`.
|
||||||
|
Mixing the index range into the de-dup key (same commit) makes every batch
|
||||||
|
appear. **Any conclusion drawn from a pre-2026-08-13 capture's `indices=` value,
|
||||||
|
or from `vbase − ibase`, is about one batch and not about the block.**
|
||||||
|
|||||||
Reference in New Issue
Block a user