Files
Syplheed-Reborn/docs/re/structures/xbg7-mesh.md
MechaCat02 2053f31d17 feat(formats,cli): decode XBG7 stage containers + fix weapon layout
Stage containers (hidden/resource3d/Stage_S*.xpr) are collections of up to
~450 enemy/prop sub-models, not single meshes. Xbg7Model::stage_models decodes
them: each resource is a [12-byte header][index buffer][vertex buffer] block
(index count = descriptor marker, vertex count = u32 32 bytes before it) whose
on-disc offset is NOT stored, so it is located by content — one O(file) pass per
stride finds vertex-buffer starts (unit NORMAL at +12 whose previous slot isn't)
and each resource is pinned to the candidate whose indices validate and produce
non-degenerate, well-connected triangles. A connectivity gate (mean triangle
edge <= 0.28x the bbox diagonal) rejects spiky mis-anchors. ~4993 sub-models
decode across the 22 stages.

The same insight fixes the weapon single-model layout: it is [12-byte header]
[index][vertex] too, not [index][12-byte gap][vertex]. Reading indices from the
block start turned the 12 header bytes into 6 junk indices (2 leading degenerate
triangles — the recurring stray-triangle artifact) and dropped the last 6 real
indices. Skipping the header leaves vertex offsets identical (coverage unchanged
at 36/166) and corrects the triangle list. Verified on wep_00/03/04.

Adds `sylpheed-cli mesh {info,render}` — a headless software rasterizer that
writes a shaded PNG (orthographic, z-buffered, two-sided), so recovered geometry
can be verified without the GUI. Stages render as a normalised thumbnail grid;
--only filters sub-models.

Tests: stage_models_{decode,sweep,quality_audit}; docs/re updated (xbg7-mesh.md
evidence log + INDEX.md).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 20:56:35 +02:00

12 KiB
Raw Blame History

XBG7 — mesh geometry (inside XPR2 model containers)

  • Confidence: 🟡 PROBABLE for the single-stream layout (below); HYPOTHESIS / undecoded for the complex multi-stream body layout.
  • Parser in: sylpheed-formats/src/mesh.rs (Xbg7Model::from_xpr2), tests tests/mesh_disc.rs. Container parsing reused from src/texture.rs (Xpr2Header / Xpr2ResourceEntry).
  • Applies to: ship / weapon / prop models in hidden/resource3d/*.xpr (166 files).
  • Method: clean-room — static hex inspection of the retail disc + geometric validation of the recovered triangles (non-degenerate area, indices in range, bbox matches the descriptor's stored size). No game code decompiled or copied.

Where XBG7 lives

Models are ordinary XPR2 containers (see the XPR2 texture doc / texture.rs). The 16-byte resource-directory entries (from file offset 0x10) carry TX2D texture resources and one or more XBG7 geometry resources:

entry = [ tag:4 ][ data_offset:u32 ][ descriptor_size:u32 ][ name_offset:u32 ]   (big-endian)

Offsets are relative to the directory base 0x10. The XBG7 descriptor (at data_offset+0x10, descriptor_size bytes) is a scene / material / node graph — it holds node names (rou_f001_mnt1_root, Light, …), a bounding value (0x41F00000 = 30.0 ≈ the ship's ~30-unit length), material names matching the TX2D channels (_col albedo, _spc specular, _gls gloss, _lum luminance), and per-sub-mesh records. The vertex / index buffers live in the container's shared data section (from header_size).

Sub-mesh records (in the descriptor)

Read in file order by a sliding 4-byte scan; each is a big-endian tuple:

[ vtx_count:u32 ][ 0:u32 ][ idx_count:u32 ][ tail:u32 ]
   3..=65535        ==0     mult. of 3        1..=64

(For rou_f001_wep_00: vtx_count=215, idx_count=1092 — matches the recovered geometry exactly.)

The single-stream data layout — 🟡 PROBABLE (decoded, GPU-cross-checked)

For 36 of the 166 models (weapons, simple props) the data section is a straight sequence of sub-meshes, carved from header_size in record order:

per sub-mesh block:
  [ 12-byte header  (contents undecoded) ]
  [ index buffer  : idx_count × u16 BE ]        triangle list (prim=4, GPU-confirmed)
  [ vertex buffer : vtx_count × stride bytes ]  ← declaration-driven
  (pad to 16 bytes → next sub-mesh)

The 12-byte header precedes the index buffer (the same block shape as stage resources — see below); the vertex buffer follows the indices with no further gap. (Earlier this was mis-modelled as [index][12-byte gap][vertex], which put the vertex buffer at the identical offset but read the index buffer 12 bytes too early — turning the 12 header bytes into 6 junk indices = 2 leading degenerate triangles (a stray-triangle artifact) and dropping the last 6 real indices. Skipping the header fixes the triangle list with no change to vertex coverage.)

Vertex declaration. The layout is not fixed-stride. The descriptor holds a declaration table (right after the (index_bytes, index_count) marker) of {offset:u32, format-code:u32, usage<<16:u32} big-endian triples, terminated by offset == 0x00FF0000 / code == 0xFFFFFFFF:

usage element format code format size
0x00 POSITION 0x2A23B9 f32×3 12 B
0x03 NORMAL 0x1A2360 f16×4 (use xyz) 8 B
0x05 TEXCOORD 0x2C235F f16×2 (u,v) 4 B

Stride = max element extent. Models omit elements → variable stride (20 = pos+normal, no UV; 24 = pos+normal+uv). Each element is read in naive big-endian component order (the raw file bytes; see the endianness note below). Assuming a fixed stride-24 was why the old decoder mis-aligned and declined the pos+normal-only models.

Alignment pinned by the normals. The vertex buffer starts at index_end + 12 (a fixed 12-byte header — NOT align16, which lands 4 bytes early on most models). The correct offset is the unique one where recovered normals are exactly unit-length.

Safety gate: every index is validated < vtx_count, and when the declaration has a normal element the mean recovered |normal| must be ≈1 ([0.5, 2.0]). A model failing either is rejected (MeshError::UnsupportedLayout) rather than emitting garbage.

Endianness — file bytes are naive-BE, k8in32 is a red herring

The Canary GPU capture reports every vertex stream with fetch endian = 2 (k8in32, each 32-bit word byte-reversed). This describes the guest-memory copy the GPU fetches — not the .xpr file bytes. Reading the file with a k8in32 transform breaks the normals (mean |normal| → 1.33); the plain per-element big-endian read yields exactly unit normals. So the game rearranges the vertex data between the on-disc .xpr and the uploaded buffer; the decoder reads the file directly and must use naive BE.

Stage containers — multi-resource, grouped pools (content-anchored)

hidden/resource3d/Stage_S*.xpr are not single models but collections of enemy / prop sub-models — up to ~400 XBG7 resources each (e.g. Stage_S07 = 378). Their data layout differs from the weapon files:

  • Each resource is a block [12-byte header][index buffer][vertex buffer]. Unlike weapons there is no 12-byte gap between index and vertex — the vertex buffer directly follows the indices.
  • The index count is the descriptor's (index_bytes, index_count) marker (the total for the resource — a resource may have several sub-meshes summing to it), not the first sub-mesh record. The vertex count is a u32 stored 32 bytes before the marker.
  • The blocks are scattered through the data section, interleaved with the container's texture data, in an allocation order that is not directory order and is not stored in any descriptor field we could find (the descriptor holds sizes — rel 160 ≈ index_bytes+3, rel 164 = 0x1000_0000 | (vertex_bytes+2) — but no data offset). Reconstructing that allocation order is unsolved.

Because the offset is not stored, each resource's block is located by content: parser Xbg7Model::stage_models does one O(file) pass per distinct stride to find every vertex-buffer start (an offset whose NORMAL — f16×4 at vertex +12 — is unit length while the previous stride slot's is not, i.e. a run boundary; ~one candidate per block, not millions), then pins each resource to the unique candidate where the index_count indices ending just before it are all < vertex_count, reference nearly all vertices, and yield non-degenerate triangles with real extent. This is fast (≤ ~2 s on a 70 MB stage) and unambiguous (no two resources collide). Blocks that fail validation — the few quantized hero bodies — are skipped, never emitted as garbage.

Coverage: 5662 sub-models decode across the 22 stage files (e.g. Stage_S07 366/378, Stage_S10 7/9 — including the main enemy bodies e003/e005, their LODs, weapons, and props). The viewer (spawn_stage_models) lays the decoded sub-models out as a side-by-side "cast sheet", skipping the few huge skybox-plane resources (300 k-unit quads). Cross-checked: e003 = 2383 v / 1436 t bbox 23.5×9.5×32.5; e005 = 2566 v / 1507 t; both 0 degenerate.

Not yet decoded — the complex body layout

The hero-ship body meshes (DeltaSaber_A/_T/_W.xpr resource f004, and ~100 other models) still decline. Two open sub-problems: (1) multi-sub-mesh models whose first sub-mesh decodes but a later one's inter-mesh offset isn't yet handled (the align16 advance is a guess) — these are declined whole; (2) the big body meshes, where the data section does not start with an index buffer and the geometry sits at descriptor-addressed offsets. NB the GPU capture showed every rendered mesh is single-stream (just wider strides, e.g. 44 bytes = pos + f32×3 + colour + 2×f16×4), so the body is likely single-stream-with-a-richer-declaration rather than the "separate streams" first guessed — it was simply not rendered in the captured session (menu only). A capture taken in a mission (where DeltaSaber renders) would hand over its exact declaration directly. DeltaSaber_A's 5 XBG7 blocks are f004 (body) + _rou_f004_mnv01_L/_R, _mnv02, _turn180 (maneuver / pose).

Evidence log

  • 2026-07-12 — rou_f001_wep_00.xpr: XPR2 dir = 1×XBG7 (f001_wep_00) + 3×TX2D (_col/_gls/_spc). Data section starts with a u16-BE index run (max 214), then stride-24 vertices. Descriptor record [215,0,1092,4] at desc +0x2B0; index-buffer byte size 0x888 (=2184=1092×2) at desc +0x1A0. Recovered mesh = 215 v / 364 t, 362 non-degenerate, median tri area 0.015 — coherent. → single-stream layout PROBABLE.
  • 2026-07-12 — descriptor-driven sequential carving over all 166 models: 42 carve under the index-only check. Real 3D extent confirmed on rou_f001_wep_04 (bbox 3.7×1.2×3.5).
  • 2026-07-12 (refine) — attribute ranges differed per model (wep_00 UV≈attr0/2, wep_04 attr4/5, wep_03 attr1≈±60000 = garbage) → refuted the fixed "6 half attrs, UV=attr0/2" reading. Found the descriptor vertex declaration (offsets 0x0C/0x14, usages 0x03 NORMAL / 0x05 TEXCOORD). Solved the vertex-base offset with a unit-normal validator: index_end + 12 gives median |normal| = 1.000 on every weapon model (wep_00/02/03/04), vs align16 landing 4 bytes early. UVs then land in [0,1]. Adding the normal gate: 25 models decode clean + normal-valid (the rest — incl. Stage_S* degenerate blobs — correctly declined).
  • 2026-07-12 (DYNAMIC) — added a cvar-gated draw logger to Canary (command_processor.cc::LogDrawForRE, cvar log_draws), captured the Ready Room / Briefings. GPU ground truth confirmed the static layout exactly: primitive prim=4 = triangle LIST (settles the list-vs-strip question), and a stream with f32x3 @offset0 + f16x4 @3dw + f16x2 @5dw, stride 6 dwords = 24 bytes — matching POSITION@0, NORMAL@0x0C, TEXCOORD@0x14. Revealed stride varies (24, 20, 28, 44 …) and that all streams are single-stream → parsed the declaration for variable stride: coverage 25 → 36 (e.g. wep_05 is pos+normal, stride 20, no UV — previously mis-aligned). Also confirmed the endianness note above: fetch endian=2 (k8in32) is the guest copy; file stays naive-BE. Stage_S* now decode (stride 20).
  • 2026-07-12 (STAGE) — Stage_S*.xpr decoded as multi-resource containers. Found each geometry block is [12B hdr][index buffer][vertex buffer], index count = descriptor marker (total, e.g. e003 = 4308 spanning 2 sub-meshes), vertex count = u32 at marker32 (e003 = 2383 → verified by max-index 2382 and 0 degenerate tris, bbox 23.5×9.5×32.5). Blocks are scattered among texture data with no stored offset (descriptor rel 160 = idx_bytes+3, rel 164 = 0x1000_0000 | (vtx_bytes+2) are sizes, not offsets; block starts e.g. e003@0x10230, e003_l@0x5d000, e005_l@0x9b000 are not directory-ordered). Solved by content anchoring: a single per-stride pass finds vertex-run starts (unit NORMAL at +12 whose previous slot isn't), then match each resource by strict index+triangle validation. 5662 sub-models decode across 22 stages (S07 366/378), incl. the previously-declined main bodies e005 (2566 v) and weapons — ≤2 s on 70 MB. Parser Xbg7Model::stage_models, tests stage_models_{decode,sweep,quality_audit}.
  • 2026-07-12 — DeltaSaber_A.xpr body: data does not begin with indices; plain-f32×3 runs with ship-scale extent (span ≈2734, matching bbox 30.0) found only at high offsets (data+0x28634C, …) → multi-stream, undecoded.