fabi/Syplheed-Reborn
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(formats): declaration-driven XBG7 decode (variable stride)

Browse Source 
Dynamic-RE follow-up: capture Canary's GPU vertex-fetch + draw calls and
feed the ground truth back into the static decoder.

The GPU capture confirmed the reverse-engineered layout exactly — meshes
draw as triangle LISTs (prim=4) with pos f32x3 @0, normal f16x4 @0x0C,
uv f16x2 @0x14 — and revealed the XBG7 vertex format is NOT fixed-stride:
models omit elements (stride 20 = pos+normal, no UV; 24 = pos+normal+uv).

- mesh.rs: parse the descriptor's vertex declaration ({offset, format-code,
  usage} triples; 0x2A23B9=pos f32x3, 0x1A2360=normal f16x4, 0x2C235F=uv
  f16x2) to drive per-model stride + element offsets, instead of assuming
  stride 24. Coverage 25 -> 36 fully-validated models (e.g. the Stage_S*
  props, which are pos+normal only). Same index-range + unit-normal safety
  gates; complex/mismatched layouts still declined.
- Endianness note (documented): the capture's fetch endian=k8in32 describes
  the GPU's guest-memory copy, NOT the .xpr file bytes — reading the file
  with k8in32 breaks the normals (|n|->1.33); naive big-endian per element
  is correct (the game rearranges vertex data on load).
- tests: a coverage-regression test (>=35 models) + the existing weapon /
  body-declined disc tests still pass.

Confirmed against a Canary draw-log capture; see docs/re/structures/
xbg7-mesh.md (evidence log + declaration table).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-07-12 18:42:21 +02:00
parent ef6e448268
commit 4096b2d2a5
4 changed files with 244 additions and 68 deletions
Download Patch File Download Diff File Expand all files Collapse all files

193
crates/sylpheed-formats/src/mesh.rs
View File

@@ -17,23 +17,25 @@
//!
//! ## The "simple" layout decoded here (CONFIRMED)
//!
//! For single-stream models (weapons, simple props — ~40 of the 166 disc
//! models) the data section is a straight sequence of sub-meshes, each:
//! For single-stream models (weapons, simple props — 36 of the 166 disc models)
//! the data section is a straight sequence of sub-meshes, each:
//!
//! ```text
//! [ index buffer : idx_count × u16 big-endian ] triangle list
//! [ 12-byte vertex-buffer header (contents undecoded) ]
//! [ vertex buffer : vtx_count × 24 bytes ] (declaration below)
//! offset 0x00 POSITION : f32 × 3 big-endian (model units)
//! offset 0x0C NORMAL : f16 × 4 big-endian (x, y, z, w; unit)
//! offset 0x14 TEXCOORD : f16 × 2 big-endian (u, v)
//! [ vertex buffer : vtx_count × stride bytes ] (declaration-driven)
//! (pad to 16 bytes → next sub-mesh)
//! ```
//!
//! The per-vertex element offsets / usages come from a **vertex declaration**
//! in the descriptor (usage `0x00` POSITION, `0x03` NORMAL, `0x05` TEXCOORD),
//! identical across the decoded models. Correct alignment is pinned by the
//! recovered normals being exactly unit-length.
//! The vertex layout is **not fixed** — it comes from a **vertex declaration**
//! in the descriptor: a table of `{offset, format-code, usage}` triples (usage
//! `0x00` POSITION `f32×3`, `0x03` NORMAL `f16×4`, `0x05` TEXCOORD `f16×2`).
//! Models omit UV or use fewer elements, so stride varies (20 = pos+normal,
//! 24 = pos+normal+uv, …). Each element is read in naive big-endian component
//! order. Correct alignment is pinned by the recovered normals being exactly
//! unit-length. **Cross-checked against a Canary GPU vertex-fetch capture**,
//! which confirmed the primitive type (triangle list), formats, and offsets —
//! see `docs/re/structures/xbg7-mesh.md`.
//!
//! `vtx_count` / `idx_count` come from per-sub-mesh records in the descriptor:
//! a `[vtx_count:u32][0:u32][idx_count:u32][tail:u32]` tuple (big-endian), read
@@ -130,6 +132,13 @@ impl Xbg7Model {
let name = read_cstr(bytes, xbg.name_offset as usize + DIR_BASE)
.unwrap_or_else(|| "XBG7".to_string());
// Parse the vertex declaration (element offsets/formats + stride). The
// XBG7 layout is NOT fixed-stride — models omit UV or use fewer elements
// (stride 20 = pos+normal, stride 24 = pos+normal+uv, …). Confirmed
// against a Canary GPU vertex-fetch capture (see docs/re/xbg7-mesh.md).
let decl = parse_vertex_decl(&bytes[desc..desc_end])
.ok_or(MeshError::UnsupportedLayout)?;
// Extract the ordered list of sub-mesh (vtx_count, idx_count) records.
let records = submesh_records(&bytes[desc..desc_end]);
if records.is_empty() {
@@ -147,7 +156,7 @@ impl Xbg7Model {
// header (a normal length of exactly 1.0 pins this offset across
// every decoded model — `align`-based guesses landed 4 bytes early).
let vb = ie + VERTEX_BUFFER_GAP;
let ve = vb + vtx_count * VERTEX_STRIDE;
let ve = vb + vtx_count * decl.stride;
if ve > bytes.len() {
return Err(MeshError::UnsupportedLayout);
}
@@ -162,43 +171,53 @@ impl Xbg7Model {
indices.push(i);
}
// Vertex layout (stride 24), per the descriptor's vertex declaration
// (usage codes: 0x00 POSITION, 0x03 NORMAL, 0x05 TEXCOORD):
// +0x00 POSITION f32 × 3 (big-endian)
// +0x0C NORMAL f16 × 4 (x, y, z, w; use xyz — unit length)
// +0x14 TEXCOORD f16 × 2 (u, v)
// Vertices per the declaration. The .xpr stores each element in
// naive big-endian component order (f32 / f16 read at consecutive
// offsets) — the GPU's `k8in32` fetch endianness applies to the
// rearranged guest-memory copy, not to these file bytes.
let mut positions = Vec::with_capacity(vtx_count);
let mut normals = Vec::with_capacity(vtx_count);
let mut uvs = Vec::with_capacity(vtx_count);
let mut normal_len_sum = 0.0f32;
for v in 0..vtx_count {
let o = vb + v * VERTEX_STRIDE;
let x = bef(bytes, o);
let y = bef(bytes, o + 4);
let z = bef(bytes, o + 8);
let o = vb + v * decl.stride;
// POSITION: f32×3 big-endian.
let p = o + decl.pos_offset;
let x = bef(bytes, p);
let y = bef(bytes, p + 4);
let z = bef(bytes, p + 8);
if !(x.is_finite() && y.is_finite() && z.is_finite()) {
return Err(MeshError::UnsupportedLayout);
}
positions.push([x, y, z]);
let nx = half(bytes, o + 0x0C);
let ny = half(bytes, o + 0x0E);
let nz = half(bytes, o + 0x10);
normal_len_sum += (nx * nx + ny * ny + nz * nz).sqrt();
normals.push([nx, ny, nz]);
// NORMAL: f16×4 (use xyz).
if let Some(no) = decl.normal_offset {
let nb = o + no;
let nx = half(bytes, nb);
let ny = half(bytes, nb + 2);
let nz = half(bytes, nb + 4);
normal_len_sum += (nx * nx + ny * ny + nz * nz).sqrt();
normals.push([nx, ny, nz]);
}
let u = half(bytes, o + 0x14);
let vv = half(bytes, o + 0x16);
uvs.push([u, vv]);
// TEXCOORD: f16×2.
if let Some(uo) = decl.uv_offset {
let ub = o + uo;
uvs.push([half(bytes, ub), half(bytes, ub + 2)]);
}
}
// Sanity gate: a correctly-aligned vertex buffer in this layout has
// unit-length normals. If the mean is far off, the model does not
// fit the simple layout (wrong padding / different format) — decline
// rather than emit garbage. (Catches e.g. `Stage_S*` placeholders.)
let mean_normal_len = normal_len_sum / vtx_count.max(1) as f32;
if !(0.5..=2.0).contains(&mean_normal_len) {
return Err(MeshError::UnsupportedLayout);
// Sanity gate: when the declaration has a normal element, a correctly
// aligned vertex buffer yields unit-length normals. A mean far from 1
// means the layout does not fit (wrong stride / offset) — decline
// rather than emit garbage.
if decl.normal_offset.is_some() {
let mean = normal_len_sum / vtx_count.max(1) as f32;
if !(0.5..=2.0).contains(&mean) {
return Err(MeshError::UnsupportedLayout);
}
}
meshes.push(GameMesh {
@@ -217,13 +236,111 @@ impl Xbg7Model {
// ── Layout constants ────────────────────────────────────────────────────────
/// Bytes per vertex: `pos f32×3 (12) + normal f16×4 (8) + uv f16×2 (4)`.
const VERTEX_STRIDE: usize = 24;
/// Fixed byte gap between the end of a sub-mesh's index buffer and the start of
/// its vertex buffer (a small vertex-buffer header — contents not yet decoded).
const VERTEX_BUFFER_GAP: usize = 12;
// ── Vertex declaration ───────────────────────────────────────────────────────
/// The vertex layout for one XBG7 resource, parsed from the descriptor's
/// declaration table (shared by all its sub-meshes).
struct VertexDecl {
/// Bytes per vertex.
stride: usize,
/// Byte offset of the POSITION element (`f32×3`) within a vertex.
pos_offset: usize,
/// Byte offset of the NORMAL element (`f16×4`), if present.
normal_offset: Option<usize>,
/// Byte offset of the TEXCOORD element (`f16×2`), if present.
uv_offset: Option<usize>,
}
/// Known element format codes → element size in bytes (from the GPU capture:
/// POSITION `f32×3`, NORMAL `f16×4`, TEXCOORD `f16×2`).
fn decl_code_size(code: u32) -> Option<usize> {
match code {
0x2A_23B9 => Some(12), // f32×3 (POSITION)
0x1A_2360 => Some(8), // f16×4 (NORMAL)
0x2C_235F => Some(4), // f16×2 (TEXCOORD)
_ => None,
}
}
/// Parse the XBG7 vertex declaration: a table of `{offset:u32, code:u32,
/// usage<<16:u32}` big-endian triples that follows the `(index_bytes,
/// index_count)` marker, terminated by an `offset == 0x00FF0000` /
/// `code == 0xFFFFFFFF` sentinel. Usage codes: `0` POSITION, `3` NORMAL,
/// `5` TEXCOORD. Stride is the max element extent; unknown element sizes are
/// inferred from the next element's offset.
fn parse_vertex_decl(desc: &[u8]) -> Option<VertexDecl> {
// Locate the (index_bytes, index_count) marker: index_bytes == count*2.
let mut mk = None;
let mut rel = 0usize;
while rel + 40 <= desc.len() {
let a = be32(desc, rel);
let c = be32(desc, rel + 4);
if c >= 3 && c % 3 == 0 && c < 200_000 && a == c * 2 {
mk = Some(rel);
break;
}
rel += 4;
}
let mk = mk?;
// Read declaration triples.
let mut elems: Vec<(usize, u32, u32)> = Vec::new(); // (offset, code, usage)
let mut r = mk + 8;
for _ in 0..16 {
if r + 12 > desc.len() {
break;
}
let off = be32(desc, r);
let code = be32(desc, r + 4);
let usage = be32(desc, r + 8) >> 16;
if off == 0x00FF_0000 || code == 0xFFFF_FFFF {
break;
}
if off as usize > 0x1000 {
break; // out-of-range offset — not a real element
}
elems.push((off as usize, code & 0x00FF_FFFF, usage));
r += 12;
}
if elems.is_empty() {
return None;
}
let mut stride = 0usize;
for (i, &(off, code, _)) in elems.iter().enumerate() {
let size = decl_code_size(code).unwrap_or_else(|| {
if i + 1 < elems.len() {
elems[i + 1].0.saturating_sub(off)
} else {
4
}
});
stride = stride.max(off + size);
}
if stride == 0 || stride > 256 {
return None;
}
let pos_offset = elems
.iter()
.find(|&&(_, c, u)| c == 0x2A_23B9 || u == 0)
.map(|&(o, _, _)| o)
.unwrap_or(0);
let normal_offset = elems.iter().find(|&&(_, _, u)| u == 3).map(|&(o, _, _)| o);
let uv_offset = elems.iter().find(|&&(_, _, u)| u == 5).map(|&(o, _, _)| o);
Some(VertexDecl {
stride,
pos_offset,
normal_offset,
uv_offset,
})
}
// ── Descriptor sub-mesh record scan ─────────────────────────────────────────
/// Scan an XBG7 descriptor for the ordered list of per-sub-mesh

36
crates/sylpheed-formats/tests/mesh_disc.rs
View File

@@ -66,6 +66,42 @@ fn weapon_model_decodes_to_expected_geometry() {
);
}
#[test]
#[ignore = "requires extracted disc models — set SYLPHEED_RES3D"]
fn declaration_driven_decode_covers_expected_model_count() {
let Some(dir) = res3d_dir() else {
return;
};
let mut ok = 0usize;
let mut total = 0usize;
for entry in std::fs::read_dir(&dir).unwrap() {
let path = entry.unwrap().path();
if path.extension().and_then(|e| e.to_str()) != Some("xpr") {
continue;
}
total += 1;
let bytes = std::fs::read(&path).unwrap();
if let Ok(model) = Xbg7Model::from_xpr2(&bytes) {
if !model.meshes.is_empty() {
// Every decoded model must be self-consistent (indices in range,
// and unit normals where present).
for m in &model.meshes {
assert!(
m.indices.iter().all(|&i| (i as usize) < m.positions.len()),
"{path:?}: index out of range"
);
}
ok += 1;
}
}
}
eprintln!("XBG7 declaration-driven decode: {ok}/{total} models");
// Variable-stride declaration parsing lifted coverage vs the old fixed
// stride-24 decoder (25 → 36 fully-validated models; multi-sub-mesh models
// whose later sub-mesh offset isn't yet handled are still declined whole).
assert!(ok >= 35, "coverage regressed: only {ok}/{total} decoded");
}
#[test]
#[ignore = "requires extracted disc models — set SYLPHEED_RES3D"]
fn complex_body_mesh_is_declined_not_garbage() {