Files
Syplheed-Reborn/docs/re/structures/xbg7-mesh.md
Claude (auto-RE) dde52c02e7 re: correct the 3.4% reuse figure -- the capture dedups by (vbase, transform)
Read off the emulator patch rather than inferred: CaptureShipDrawForRE dedups by
(vbase, WVP hash), so a buffer drawn repeatedly at one transform appears once.
The 3.4% counts multi-instance placement, not buffers serving several parts, and
is not the population evidence about sharing it was written up as. Also pins
vcount = buffer capacity and indices = that draw's num_indices.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 02:24:44 +00:00

44 KiB
Raw Blame History

XBG7 — mesh geometry (inside XPR2 model containers)

  • Confidence: 🟡 PROBABLE for the single-stream layout (below); HYPOTHESIS / undecoded for the complex multi-stream body layout.
  • Parser in: sylpheed-formats/src/mesh.rs (Xbg7Model::from_xpr2), tests tests/mesh_disc.rs. Container parsing reused from src/texture.rs (Xpr2Header / Xpr2ResourceEntry).
  • Applies to: ship / weapon / prop models in hidden/resource3d/*.xpr (166 files).
  • Method: clean-room — static hex inspection of the retail disc + geometric validation of the recovered triangles (non-degenerate area, indices in range, bbox matches the descriptor's stored size). No game code decompiled or copied.

Where XBG7 lives

Models are ordinary XPR2 containers (see the XPR2 texture doc / texture.rs). The 16-byte resource-directory entries (from file offset 0x10) carry TX2D texture resources and one or more XBG7 geometry resources:

entry = [ tag:4 ][ data_offset:u32 ][ descriptor_size:u32 ][ name_offset:u32 ]   (big-endian)

Offsets are relative to the directory base 0x10. The XBG7 descriptor (at data_offset+0x10, descriptor_size bytes) is a scene / material / node graph — it holds node names (rou_f001_mnt1_root, Light, …), a bounding value (0x41F00000 = 30.0 ≈ the ship's ~30-unit length), material names matching the TX2D channels (_col albedo, _spc specular, _gls gloss, _lum luminance), and per-sub-mesh records. The vertex / index buffers live in the container's shared data section (from header_size).

Sub-mesh records (in the descriptor)

Read in file order by a sliding 4-byte scan; each is a big-endian tuple:

[ vtx_count:u32 ][ 0:u32 ][ idx_count:u32 ][ tail:u32 ]
   3..=65535        ==0     mult. of 3        1..=64

(For rou_f001_wep_00: vtx_count=215, idx_count=1092 — matches the recovered geometry exactly.)

The single-stream data layout — 🟡 PROBABLE (decoded, GPU-cross-checked)

For 36 of the 166 models (weapons, simple props) the data section is a straight sequence of sub-meshes, carved from header_size in record order:

per sub-mesh block:
  [ 12-byte header  (contents undecoded) ]
  [ index buffer  : idx_count × u16 BE ]        triangle list (prim=4, GPU-confirmed)
  [ vertex buffer : vtx_count × stride bytes ]  ← declaration-driven
  (pad to 16 bytes → next sub-mesh)

The 12-byte header precedes the index buffer (the same block shape as stage resources — see below); the vertex buffer follows the indices with no further gap. (Earlier this was mis-modelled as [index][12-byte gap][vertex], which put the vertex buffer at the identical offset but read the index buffer 12 bytes too early — turning the 12 header bytes into 6 junk indices = 2 leading degenerate triangles (a stray-triangle artifact) and dropping the last 6 real indices. Skipping the header fixes the triangle list with no change to vertex coverage.)

Vertex declaration. The layout is not fixed-stride. The descriptor holds a declaration table (right after the (index_bytes, index_count) marker) of {offset:u32, format-code:u32, usage<<16:u32} big-endian triples, terminated by offset == 0x00FF0000 / code == 0xFFFFFFFF:

usage element format code format size
0x00 POSITION 0x2A23B9 f32×3 12 B
0x03 NORMAL 0x1A2360 f16×4 (use xyz) 8 B
0x05 TEXCOORD 0x2C235F f16×2 (u,v) 4 B

Stride = max element extent. Models omit elements → variable stride (20 = pos+normal, no UV; 24 = pos+normal+uv). Each element is read in naive big-endian component order (the raw file bytes; see the endianness note below). Assuming a fixed stride-24 was why the old decoder mis-aligned and declined the pos+normal-only models.

Alignment pinned by the normals. The vertex buffer starts at index_end + 12 (a fixed 12-byte header — NOT align16, which lands 4 bytes early on most models). The correct offset is the unique one where recovered normals are exactly unit-length.

Safety gate: every index is validated < vtx_count, and when the declaration has a normal element the mean recovered |normal| must be ≈1 ([0.5, 2.0]). A model failing either is rejected (MeshError::UnsupportedLayout) rather than emitting garbage.

Endianness — file bytes are naive-BE, k8in32 is a red herring

The Canary GPU capture reports every vertex stream with fetch endian = 2 (k8in32, each 32-bit word byte-reversed). This describes the guest-memory copy the GPU fetches — not the .xpr file bytes. Reading the file with a k8in32 transform breaks the normals (mean |normal| → 1.33); the plain per-element big-endian read yields exactly unit normals. So the game rearranges the vertex data between the on-disc .xpr and the uploaded buffer; the decoder reads the file directly and must use naive BE.

Stage containers — multi-resource, grouped pools (content-anchored)

hidden/resource3d/Stage_S*.xpr are not single models but collections of enemy / prop sub-models — up to ~400 XBG7 resources each (e.g. Stage_S07 = 378). Their data layout differs from the weapon files:

  • Each resource is a block [12-byte header][index buffer][vertex buffer]. Unlike weapons there is no 12-byte gap between index and vertex — the vertex buffer directly follows the indices.
  • The index count is the descriptor's (index_bytes, index_count) marker (the total for the resource — a resource may have several sub-meshes summing to it), not the first sub-mesh record. The vertex count is a u32 stored 32 bytes before the marker.
  • The blocks are scattered through the data section, interleaved with the container's texture data, in an allocation order that is not directory order and is not stored in any descriptor field we could find (the descriptor holds sizes — rel 160 ≈ index_bytes+3, rel 164 = 0x1000_0000 | (vertex_bytes+2) — but no data offset). Reconstructing that allocation order is unsolved.

Because the offset is not stored, each resource's block is located by content: parser Xbg7Model::stage_models does one O(file) pass per distinct stride to find every vertex-buffer start (an offset whose NORMAL — f16×4 at vertex +12 — is unit length while the previous stride slot's is not, i.e. a run boundary; ~one candidate per block, not millions), then pins each resource to the unique candidate where the index_count indices ending just before it are all < vertex_count, reference nearly all vertices, and yield non-degenerate triangles with real extent. This is fast (≤ ~2 s on a 70 MB stage) and unambiguous (no two resources collide). Blocks that fail validation — the few quantized hero bodies — are skipped, never emitted as garbage.

Coverage: 5662 sub-models decode across the 22 stage files (e.g. Stage_S07 366/378, Stage_S10 7/9 — including the main enemy bodies e003/e005, their LODs, weapons, and props). The viewer (spawn_stage_models) lays the decoded sub-models out as a side-by-side "cast sheet", skipping the few huge skybox-plane resources (300 k-unit quads). Cross-checked: e003 = 2383 v / 1436 t bbox 23.5×9.5×32.5; e005 = 2566 v / 1507 t; both 0 degenerate.

Not yet decoded — the complex body layout

The hero-ship body meshes (DeltaSaber_A/_T/_W.xpr resource f004, and ~100 other models) still decline. Two open sub-problems: (1) multi-sub-mesh models whose first sub-mesh decodes but a later one's inter-mesh offset isn't yet handled (the align16 advance is a guess) — these are declined whole; (2) the big body meshes, where the data section does not start with an index buffer and the geometry sits at descriptor-addressed offsets. NB the GPU capture showed every rendered mesh is single-stream (just wider strides, e.g. 44 bytes = pos + f32×3 + colour + 2×f16×4), so the body is likely single-stream-with-a-richer-declaration rather than the "separate streams" first guessed — it was simply not rendered in the captured session (menu only). A capture taken in a mission (where DeltaSaber renders) would hand over its exact declaration directly. DeltaSaber_A's 5 XBG7 blocks are f004 (body) + _rou_f004_mnv01_L/_R, _mnv02, _turn180 (maneuver / pose).

Evidence log

  • 2026-07-12 — rou_f001_wep_00.xpr: XPR2 dir = 1×XBG7 (f001_wep_00) + 3×TX2D (_col/_gls/_spc). Data section starts with a u16-BE index run (max 214), then stride-24 vertices. Descriptor record [215,0,1092,4] at desc +0x2B0; index-buffer byte size 0x888 (=2184=1092×2) at desc +0x1A0. Recovered mesh = 215 v / 364 t, 362 non-degenerate, median tri area 0.015 — coherent. → single-stream layout PROBABLE.
  • 2026-07-12 — descriptor-driven sequential carving over all 166 models: 42 carve under the index-only check. Real 3D extent confirmed on rou_f001_wep_04 (bbox 3.7×1.2×3.5).
  • 2026-07-12 (refine) — attribute ranges differed per model (wep_00 UV≈attr0/2, wep_04 attr4/5, wep_03 attr1≈±60000 = garbage) → refuted the fixed "6 half attrs, UV=attr0/2" reading. Found the descriptor vertex declaration (offsets 0x0C/0x14, usages 0x03 NORMAL / 0x05 TEXCOORD). Solved the vertex-base offset with a unit-normal validator: index_end + 12 gives median |normal| = 1.000 on every weapon model (wep_00/02/03/04), vs align16 landing 4 bytes early. UVs then land in [0,1]. Adding the normal gate: 25 models decode clean + normal-valid (the rest — incl. Stage_S* degenerate blobs — correctly declined).
  • 2026-07-12 (DYNAMIC) — added a cvar-gated draw logger to Canary (command_processor.cc::LogDrawForRE, cvar log_draws), captured the Ready Room / Briefings. GPU ground truth confirmed the static layout exactly: primitive prim=4 = triangle LIST (settles the list-vs-strip question), and a stream with f32x3 @offset0 + f16x4 @3dw + f16x2 @5dw, stride 6 dwords = 24 bytes — matching POSITION@0, NORMAL@0x0C, TEXCOORD@0x14. Revealed stride varies (24, 20, 28, 44 …) and that all streams are single-stream → parsed the declaration for variable stride: coverage 25 → 36 (e.g. wep_05 is pos+normal, stride 20, no UV — previously mis-aligned). Also confirmed the endianness note above: fetch endian=2 (k8in32) is the guest copy; file stays naive-BE. Stage_S* now decode (stride 20).
  • 2026-07-12 (STAGE) — Stage_S*.xpr decoded as multi-resource containers. Found each geometry block is [12B hdr][index buffer][vertex buffer], index count = descriptor marker (total, e.g. e003 = 4308 spanning 2 sub-meshes), vertex count = u32 at marker32 (e003 = 2383 → verified by max-index 2382 and 0 degenerate tris, bbox 23.5×9.5×32.5). Blocks are scattered among texture data with no stored offset (descriptor rel 160 = idx_bytes+3, rel 164 = 0x1000_0000 | (vtx_bytes+2) are sizes, not offsets; block starts e.g. e003@0x10230, e003_l@0x5d000, e005_l@0x9b000 are not directory-ordered). Solved by content anchoring: a single per-stride pass finds vertex-run starts (unit NORMAL at +12 whose previous slot isn't), then match each resource by strict index+triangle validation. 5662 sub-models decode across 22 stages (S07 366/378), incl. the previously-declined main bodies e005 (2566 v) and weapons — ≤2 s on 70 MB. Parser Xbg7Model::stage_models, tests stage_models_{decode,sweep,quality_audit}.
  • 2026-07-17 — triangle-LIST re-confirmed; a strip interlude refuted; winding-consistency gate added. A 2026-07 change had briefly re-read the index buffers as triangle strips (to "fill holes"). Refuted objectively with a new XVERIFY diagnostic that compares both readings by stored-normal agreement (each triangle's cross-product face normal vs the sum of its vertices' stored normals): the LIST reading gives agreement 1.000 on every clean weapon (wep_00/03/04/19 = only possible with correct topology + winding), the STRIP reading ~0.49 (random). The strip reading also over-generated ~2.5× the triangles (wep_00: 938 vs 364) — a hole-filling garbage soup. Reverted to LIST in both paths (from_xpr2, read_pool_mesh), matching the prim=4 GPU capture. Added an objective winding-consistency gate max(na, 1-na): a correct carve is internally consistent (agreement ≈1.0, or ≈0.0 for inverted-but-consistent winding — a real single-sided mesh), a mis-carve scatters to the ≈0.5 middle. from_xpr2 declines sub-meshes below 0.90 (e.g. wep_23 na=0.398 → declined instead of a spike-mess); the single-model content-anchor fallback gates at 0.85; the large multi-resource stage path stays ungated (its enemy meshes span a continuous 0.51.0 consistency range — a hard gate there dropped ~48/314 legit S07 blocks). Routing fixed: decode_models (CLI) and the viewer now route by count_xbg7 (1 → validated records-based list decode, fallback to strict-gated anchor; >1 → stage anchor) instead of the old "whichever decoder yields more verts" rule — that rule let stage content-anchoring win on single-model weapon files and fabricate phantom blocks (a wep_00 clone appearing inside wep_19), duplicates, and spike-mess anchors. Weapons now: 33 clean-decode / 26 declined (declined = genuinely multi-stream or un-carvable, shown as nothing rather than garbage); stage coverage unchanged (S07 314). expand_triangle_strip retained as an XVERIFY-only diagnostic.
  • 2026-07-12 — DeltaSaber_A.xpr body: data does not begin with indices; plain-f32×3 runs with ship-scale extent (span ≈2734, matching bbox 30.0) found only at high offsets (data+0x28634C, …) → multi-stream, undecoded.
  • 2026-07-18 — GROUPED-POOL layout cracked → the hero ship (Delta Saber) fully decodes. The detailed models (DeltaSaber_*.xpr + ~100 others) were declined for location, not format — their vertex format is the standard stride-24 triangle list. A resource's several sub-meshes don't interleave [idx][vtx] per block; they share two grouped pools: an index pool (buffers concatenated in descriptor-marker order, each 4-byte aligned) followed by a vertex pool (each sub-pool vtx_count × stride, same order), with the index pool ending exactly where the vertex pool begins. So the whole resource pivots on one unknown, the first vertex-pool start vb0 (= index-pool end, found by the unit-normal vertex-run scan); everything else is derived: ib0 = vb0 span, ib[i] = align4(ib[i-1] + idx_count[i-1]·2), vb[i] = vb[i-1] + vtx_count[i-1]·stride. Reversed statically from DeltaSaber_T.xpr and cross-checked against a Canary GPU draw-log capture (mission ship = DeltaSaber_T.xpr, found via the --log_file_io kernel hook): f001 = body (idx@data+0xC = 0x5500C, vtx@0x61ACC, 10891 v / 8187 t) + 7 detail parts (fins/cockpit/wingtips, markers at descriptor 0x3BEC…0x58FC) = 8650 tris, and every sub-mesh decodes at 0 degenerate / full coverage / winding-agreement 1.000. This is the layout the per-block adjacency anchor (ib = vb idx_bytes) rendered as a spiky phantom (it read 24561 indices starting 2782 B too late, agree 0.64, 1277 degenerate). Insight: a single index marker reduces the grouped model to index_end = vb0, i.e. the existing adjacency ib = vb idx_bytes — so grouped generalises the single-block anchor (n=1 is identical). Implemented as anchor_grouped_meshes (mesh.rs): anchor_models routes resources with

    1 index marker to it (validated per sub-mesh; on failure falls back to the old first-marker adjacency anchor so stage coverage never regresses); single-marker stages/props keep the exact prior path. The shared acceptance test is factored into validate_block (the connectivity heuristic is relaxed for derived grouped parts, which are pinned by in-range + consistency, so small flat fins aren't mis-rejected). Render self-check: sylpheed-cli mesh render DeltaSaber_T.xpr --only f001 (exact-name match excludes the _rou_f001_mnv* animation poses) → clean complete fighter. Test hero_ship_grouped_pool_decodes. Colours/UVs still pending the running-game oracle.

  • 2026-07-18 (refinement) — 4-byte vertex-pool alignment + weapon recovery. The grouped-pool rule "index pool ends exactly where the vertex pool begins" is really "the vertex pool is 4-byte aligned after the index pool": vb0 = align4(ib0 + span), so 0..=3 bytes of padding can sit between them. DeltaSaber's index pool ended already-aligned (pad 0), which hid this; 19 weapon/*_hangar models (single- and multi-marker: wep_08/11/34/58/62/69/81/83…) have pad 2 and so decoded to nothing — the viewer then showed them as a flat 2D texture instead of a model. Fix: both anchors try pad ∈ 0..=3 (ib = vb idx_bytes pad for the single-block adjacency anchor; ib0 = vb0 span pad for the grouped pivot), validated — a wrong pad reads shifted indices → agreement collapses < 0.85, so only the true pad passes. pad>0 in the ungated stage path is gated at a strict 0.85 to avoid a false anchor; pad 0 keeps its exact prior behaviour (stages unchanged). Result: all 19 now decode as clean models (e.g. wep_34 1243 v / 1233 t, a long-barrelled gun-pod; wep_08 3 sub-meshes / 478 t). Viewer routing already falls through from_xpr2anchor_models(0.85) for single-XBG7 files, so the recovered grouped/padded weapons now preview as meshes.
  • 2026-07-18 (refinement 2) — pivot on the largest sub-mesh; all 19 recovered. Three weapons (wep_81, wep_81_hangar, wep_30_hangar) still declined because the grouped pivot validated markers[0], which for these is a tiny elongated lead bracket that fails the connectivity gate even when perfectly placed. Fixed by pivoting the alignment check on the largest marker (max index count) — the sub-mesh whose triangle-quality/connectivity signature most reliably confirms (ib0, vb0). Once the pivot validates, markers up to it are read unconditionally (a legitimately tiny/flat lead part may fail the quality gates yet still be real), and markers after it stay validated so a stray trailing marker ends the chain. Result: all 19 previously-declined weapons decode (wep_81 460 t missile w/ tail fins; wep_30_hangar 334 t). DeltaSaber unchanged (its body IS the largest marker → same pivot). 7/7 disc tests green, stage quality audit unchanged.

The declined set, measured (2026-08-11)

The module note said "a few multi-stream / quantized bodies remain" declined. Measured across all 166 hidden/resource3d/*.xpr:

  • 6 294 XBG7 resources, 5 480 decoded (87.1 %), 814 declined, in 31 of 166 containers. Worst: Stage_S09 64/380, Stage_S06 58/324, ptc_pack 57/136.
  • The declined set is not "a few hero bodies". By name prefix it is 492 e* (enemy craft), 142 f*, 73 n*, 23 eff*, plus destroyed variants (_rou_f402_dead, _rou_f302_base_dead) and one weapon (_rou_e011_wep04).

A shortcut that does not work

The resource descriptor's third word looked like a format/stream flag — decoded g001…g003 carry 0x00010001 while declined t170/t180 carry 0x00020004, which reads temptingly as (streams << 16) | format. It is not that. Histogramming it over the whole disc puts decoded and declined resources at every value:

word[2]      decoded  declined
0x00010001      4479       328
0x00010002       126        27
0x00010003       120       112
0x00010004       142        84
…               …          …

Its low half runs 1…0x52 and tracks sub-mesh count, not vertex format. So decodability is not declared in the descriptor — it is a property of whether the unit-normal anchor scan can locate vb0, which is exactly what the current code already tests. Anyone attacking this should not spend time on the descriptor: 229 of the declined resources even carry the most common 0x00010001 with under 1 KB of data, i.e. they are small meshes the scan has too little signal to anchor, not exotic formats.

Silent mis-decodes: a detector, and how many there are (2026-08-11)

The declined set is the honest failure mode — 814 resources the decoder refuses. This is the other kind: geometry that decodes without complaint and is wrong.

The case that exposed it

e303_wep_01 decodes from fourteen containers. In eleven it is a 49 × 23 × 42 turret with organic vertices (24.55, 0.00, 4.46 …). In Stage_S02, S08 and S26 the same resource — identical 172 vertices and 330 indices — decodes to 1600 × 2100 × 4800 of axis-aligned box corners:

Stage_S01   [   24.55    0.00    4.46]  [  24.55   9.84   2.91]   normals varied
Stage_S02   [   42.00 -900.00 2400.00]  [-600.00 -500.00 -500.00]  normals (0,0,1)
            [ -600.00 -500.00 -950.00]  [-600.00 -950.00 -950.00]  ← box face corners

The anchor scan located a different buffer that happens to share the vertex and index counts, so every size-based check it makes passes. This is exactly the "declined only for location, not format" risk the module notes describe — except here it does not decline, it succeeds wrongly.

The detector: cross-container bounds consistency

A resource shared by several containers must decode to the same bounds. That needs no ground truth, and it measures the problem:

  • 681 resources appear in ≥2 containers.
  • 125 of them decode to different bounds while reporting identical vertex and triangle counts — a lower bound on silent mis-decodes (a resource wrong in every container is invisible to this test).

Examples: _rou_f401 decodes as 62×25×10 in 16 containers and 4738×3147×4738 in 2; _rou_e011_wep05 produces four different spans across 8 containers.

Repair candidate, and its limits

Taking the majority span across containers resolves 104 of the 125; 14 are exact 50/50 splits that a vote cannot decide. It agrees with the ground truth in the one case that has independent evidence — e303_wep_01, where the 11-container majority is the turret the render and the runtime capture both support.

🟡 It is a heuristic and is otherwise unvalidated. For _rou_e302_base_break the majority is the larger span (685×1206×1444, 8 of 15) and nothing yet says which is right. Use the detector to flag; do not silently rewrite geometry on a vote.

ROOT CAUSE: the candidate list is container-global

Traced 2026-08-12. anchor_pool_mesh (the per-block path, which is the one that handles single-sub-mesh resources like e303_wep_01) walks a candidate list built by vertex_run_starts(bytes, data_base, stride)one scan of the whole container per stride, shared by every resource of that stride. It accepts the first candidate that validates.

So a resource is anchored to whatever block matches its signature first in file order, and nothing ties that block to the resource it belongs to. Two resources sharing (stride, vertex count, index count) are interchangeable to this search.

The wrong block is not distinguishable by quality. Tracing the accept for e303_wep_01:

Stage_S01   ACCEPT vb=4600480   pad=0  span=  49 ×  23 ×  42   passes 0.85 = true
Stage_S02   ACCEPT vb=18403456  pad=0  span=1600 × 2100 × 4800 passes 0.85 = true

Both clear the strict winding-consistency gate, because the wrongly-taken block is real, coherent geometry — just another resource's. That rules out a whole family of fixes: no threshold, no scoring, no "pick the best candidate" changes this, and the earlier attempt to add best-of-N selection in the grouped-pool anchor duly changed nothing.

The search space has to be constrained instead — and the fix is now pinned down.

The correct block is already in the candidate list. Enumerating every validating candidate for e303_wep_01 in Stage_S02 gives exactly two:

vb = 18 403 456   span 1600 × 2100 × 4800   ← what the decoder takes, only because it is first
vb = 52 257 440   span   49 ×   23 ×   42   ← correct: the same size all 11 good containers give

So nothing needs to be found that the scan is missing; the wrong one merely appears earlier in file order.

Locality picks the right one. Recording each resource's accepted anchor in descriptor order shows that global monotonicity is refuted — only 25 of 47 steps increase in Stage_S01 and 130 of 248 in Stage_S02, i.e. no better than chance. But neighbourhood holds strongly: in Stage_S02 this resource's descriptor neighbours anchor at 51 974 668 and 52 218 424, its correct candidate is 52 257 440, and the block it wrongly takes is at 18 403 456 — two thirds of the file away from its own family.

Proposed rule: among candidates that validate, prefer the one nearest the anchors of the neighbouring resources (equivalently: decode in descriptor order and prefer candidates close to the previous resource's anchor), falling back to first-match when there is no neighbour yet. That needs no new format knowledge, and it selects 52 257 440 here.

⚠️ WITHDRAWN (2026-08-12) — it halved inconsistency but regressed the twin mirror

anchor_pool_mesh_near tries candidates in order of distance from a reference, and anchor_models_filtered runs two passes: pass 1 anchors first-match to learn where resources land, then pass 2 re-anchors each resource preferring its neighbourhood — the median anchor of its ±2 descriptor neighbours. A resource with too few anchored neighbours keeps pass 1's result, so nothing regresses to guesswork.

decoded shared inconsistent e106 mirror
shipped (today) 5 480 / 6 294 681 125 matches capture
neighbourhood anchor 5 480 / 6 294 681 63 flipped
+ refining the map 5 480 / 6 294 681 51 flipped

Refining matters because pass 1's anchor map contains the very mistakes the neighbourhood is meant to correct, so a resource beside a mis-anchored neighbour inherits a bad reference. Re-anchoring against the improving map and repeating converges quickly — two rounds, with a third changing nothing.

On its own metric this looked complete: coverage unchanged, inconsistency halved, e303_wep_01 decoding to 49 × 23 × 42 in all containers, and e106 rendering as a destroyer instead of a slab (before · after).

It was reverted anyway. ship::tests::static_assembly_matches_runtime_capture is gated on SYLPHEED_ISO and therefore skips in an ordinary cargo test; run with the ISO it fails:

e106_bdy_01: static M row0 [-1.0, 0.0, 0.0] != captured [1.0, 0.0, 0.0]

Why: e106_bdy_01 and e106_bdy_02 are a mirrored pair whose two vertex buffers hold the same geometry reflected in X, and both resources currently decode to the same buffer — identical vertex count, identical span, identical mean_x. apply_twin_mirrors decides which instance to reflect from the sign of that mean_x, so which of the two buffers gets picked flips the decision:

before the change   both twins decode with mean_x = 66.83  → mirror bdy_02  (matches the capture)
after  the change   both twins decode with mean_x = +66.83  → mirror bdy_01  (contradicts it)

The runtime capture is ground truth, so a change that contradicts it does not ship.

Correction (measured after the fact): the first write-up of this said "two distinct resources sharing one decode is itself the bug". That is wrong. Sharing is normal here — 1 043 of 5 480 decoded resources (19 %) share geometry with another resource, and of 1 242 related pairs, 1 241 are identical in every container they co-occur in, which is what legitimate asset reuse looks like. A mirrored pair like bdy_01/bdy_02 is supposed to share one geometry, with the reflection applied at placement — exactly what apply_twin_mirrors does.

What actually matters is which of two mirrored buffers is canonical. The disc holds both an X+ and an X version; the engine treats one as the base, and apply_twin_mirrors was tuned against that. The neighbourhood anchor moved these resources to the nearer buffer, which is the other one — hence the flip. So the real fix is not "give each twin its own buffer" but pin which buffer is canonical, with the capture as the oracle.

Exactly one pair is provably mis-anchored by this test: e105_bdy_02_l / e105_brg_m share a decode in 7 of the 15 containers holding both and differ in the rest — two names cannot be the same geometry only sometimes.

The filtered path needed care. models_named (what the viewer's ship rendering uses) drops non-wanted resources, which would leave a filtered decode with no neighbourhood at all — and silently keep the old behaviour. Resources are now collected regardless of the filter, but only the asked-for ones and their ±2 neighbours are decoded in pass 1, so a filtered decode stays proportional to what was asked for.

51 remain, and they cluster in _l (LOD) and _dead variants — e001_l, e010_bdy_01_l, e011_bdy_01_l, e016_l, e104_bdy_05_l, e106_eng_02_l, e501_01_l, _rou_f301_base_dead, _rou_f302_base_dead, e303_base_dead.

A tempting explanation, tested and false. The obvious reading is that a variant shares its base's vertex and index counts, so the two are mutually confusable and adjacent, defeating locality. Checked across every container: 2 714 variant/base pairs, and exactly zero share identical counts.

What is actually happening: one region is a universal false positive. e010_bdy_01_l is 171 verts / 90 tris and no other resource in its container shares those counts — yet in Stage_S02 and S26 it decodes to 1600 × 2100 × 4800, the same bounds e303_wep_01 (172 verts / 110 tris) produced before the fix. Differently-shaped resources are landing on the same place. So the attractor is not "another mesh with my shape" but a region of round, axis-aligned box data that validates for many different (vtx, idx) shapes at once — every index lands in range and the triangles are coherent boxes.

That also explains why the neighbourhood fix helped so broadly: it steers resources away from a single strong attractor rather than resolving many pairwise confusions.

Selection-based fixes are exhausted — tested. The proposed tiebreak was implemented as a last resort (accept the attractor only if nothing else validates), first keyed on "all coordinates multiples of 50" and then on the sharper "all coordinates integral" — the attractor reads (42, 900, 2400), (600, 500, 950) while real geometry carries fractions like (24.55, 9.84, 4.46). Neither changed anything: still 51.

That null result is itself the answer. A mechanism that defers the attractor whenever another candidate exists, changing nothing, means no alternative candidate validates for any of the 51 — the correct block is not in the candidate list at all. Both attempts were reverted rather than kept.

So the residual is not a selection problem, and no reordering, scoring or tiebreak will move it. The frontier is vertex_run_starts — the unit-normal run scan that builds the candidate list — which does not emit a start for these resources' real vertex buffers. That is where the remaining 51 live.

The ignored test mesh_consistency_disc.rs still asserts the target state and now records 63 rather than 125; the remaining cases are where the neighbourhood is itself wrong or absent.

Where the mis-decode is not: the grouped-pool anchor

An attempt to fix it by making anchor_grouped_meshes choose the best-scoring vb0 (rather than the first candidate clearing the 0.85 gate) changed nothing — still 5 480 of 6 294 decoded and still 125 inconsistent — and instrumenting the pivot loop shows why: for e303_wep_01 it never runs. The resource has a single sub-mesh, so it is decoded by the per-block adjacency path (anchor_pool_mesh), not the grouped-pool anchor.

So the silent mis-decode lives in the per-block anchor. That is worth knowing before anyone else spends time on the grouped-pool pivot, which is the more prominent and better-documented of the two and the natural first suspect.

The change was reverted: it was untargeted, unproven, and added a scoring path with no demonstrated benefit. (Its one reusable idea — that several vb0 candidates can clear the gate and first-in-scan-order is an arbitrary tiebreak — still applies to whichever anchor turns out to be at fault.)

Two follow-ups on the anchor (2026-08-12)

The descriptor does not address the geometry. If it did, the whole candidate-scan could be replaced by direct addressing. It cannot: across e106's resources in Stage_S01, anchored_vb entry.data_offset ranges from 1 199 052 to 4 173 988 with no constant or stride. data_offset locates the descriptor, and nothing in the first six descriptor words tracks the vertex pool. The scan is necessary.

Cross-container agreement does not prove legitimate reuse. The earlier measurement — 1 241 of 1 242 related pairs identical in every container — was read as "sharing is normal". It is weaker than that: a systematic error is invisible to a consistency test, because it is consistent. The same dump shows e106_bdy_02 and e106_bdy_03_m anchoring to the identical offset (1 505 556), and e106_bdy_01_l with e106_brg_01_m (4 251 208) — a hull half and a different body's medium LOD, or a hull half and a bridge LOD. Those are different parts; one of each pair must be wrong.

So the honest position is: sharing is common (19 %), some of it is certainly legitimate (a mirrored twin pair genuinely shares one geometry), and some is certainly not — and cross-container consistency cannot tell them apart. A test that can: compare a shared pair against a runtime capture, which is ground truth for what the engine actually draws.

The capture answers it at population level — and the logs are still on disc

/sylph-home/re/shipcap/xenia_ship_capture_*.log (kept from the 2026-07 capture sessions) carry the raw per-draw lines the baked table was distilled from:

DRAW vbase=0x150CCAC0 stride=28 vcount=1 indices=1 prim=1 vs=0x…
DRAW vbase=0x150CCAC0 stride=24 vcount=10891 indices=18 prim=4 vs=0x…

vbase is the GPU vertex base — ground truth for which buffer the engine draws a part from, which is exactly the oracle the sharing question needs. Over the three logs, restricted to the ship-geometry stride 24:

  • 6 093 draws from 2 291 distinct vbases
  • only 77 vbases (3.4 %) are drawn more than once

⚠️ That 3.4 % measures less than it first appears — corrected 2026-08-12. The capture code (command_processor.cc, CaptureShipDrawForRE) de-duplicates by (vbase, WVP-transform hash), so a buffer drawn many times at one transform — which is what a mesh split into per-material sub-draws looks like — appears once. The figure therefore counts buffers drawn at several placements (multi-instance parts), not buffers serving several parts. It is not the population-level evidence about sharing it was first written up as; the bdy_01_l/bdy_02_l result below is direct evidence and stands on its own.

Two more field semantics, read off the same patch rather than guessed: vcount = fetch.size × 4 / stride is the buffer's capacity, not the draw's vertex usage (which is why it matches a decoded resource's vertex count so exactly), and indices is VGT_DRAW_INITIATOR.num_indices, that draw's index count. So the 119-vertex twin logging indices=21 against our 246-index marker most likely means the engine issues the mesh as several sub-range draws and the log keeps the first — likely, not proven.

How to use it per-part: correlate_capture already matches a draw to a resource by vcount plus decoded positions. The same match yields, for each part, the vbase the engine used — so two resources that our decoder gives the same geometry can be checked directly: different vbase in the capture ⇒ our shared decode is wrong. That is the per-part oracle any future anchor work should be validated against, and it needs no new capture run.

It was run — and the capture gives file-offset ground truth

examples/shared_vbase_check.rs does the per-part check above, and then goes one step further than planned. Three results, in order of strength.

1. A draw's vbase is the container file offset plus a constant. Vertex POSITION is f32×3 big-endian at vertex offset 0, so a draw's dumped positions are a value pattern that can be searched for in the .xpr itself. Doing that for every draw in xenia_ship_capture_01/02.log and histogramming vbase offset:

xenia_ship_capture_01.log: 11 distinct vbases located, 208 not in this container
   vbase - offset = 0x1A94FFF4  ×8      ← same constant in log 02

The 208 "not in this container" are draws whose geometry lives in Common.xpr, a weapon pack or a backdrop — expected. The eight that do belong to Stage_S01 share one constant, and the same constant in a second run, so the container is uploaded contiguously and a capture names the exact file offset of every buffer the engine drew. Log 03 loaded the container at a different address, so the constant is per-run, not baked.

2. Read against our anchor scan, that is a defect list. GameMesh now carries vbuf_offset — the offset the anchor scan actually placed a sub-mesh at — so the comparison is exact (captures/stage-s01-capture-truth-offsets.txt):

drawn offset vcount our resource anchored there our resources with that vcount
0x3b3ee8 119 e106_bdy_01_l, e106_bdy_02_l bdy_01_l, bdy_02_l
0x3c55d8 119 nobody bdy_01_l, bdy_02_l
0x3dd2c4 146 e106_bdy_03_l bdy_03_l
0x40763c 179 e106_bdy_04_l bdy_04_l
0x40e418 51 nobody (ours sit 0x5d0 earlier) brg_01_b_02, brg_01_l
0x444ccc 58 e106_eng_01_l eng_01_l
0x44a32c 44 nobody eng_02_l
0x45705c 82 e106_wep_02_01_l wep_02_01_l
0x38788 0x4b8b8 0xb6574 0xdbbac 0x133da0 0x162840 181, 93, 41, 77, 76, 60 — nobody mostly none (other objects in the stage)

Four of the ship's drawn buffers are anchored exactly right. Two are the twin collapse below. Two are mis-anchors of a size we do have: the engine's 51-vertex bridge buffer is at 0x40e418 while both our 51-vertex bridge resources sit at 0x40de48, and its 44-vertex eng_02_l is at 0x44a32c while ours is elsewhere entirely. The remaining six belong to other objects in the stage (n041, n042, e303), only two of which we decode at the right size.

A trap worth recording. The first version of this table located our resources by searching the container for their leading vertices instead of asking the decoder, and it read much worse — full and _m resources appearing to start inside their own _l buffer. That was an artifact: the same leading vertex run occurs at several offsets in one container (e106_bdy_03's first eight positions occur at four, bdy_01's at three). That multiplicity is itself the reason the anchor scan is ambiguous — but it makes a position search useless for asking where a resource was anchored. Hence vbuf_offset.

3. The twin pair is an anchoring error, and the mirror is in the data. For e106_bdy_01_le106_bdy_02_l the capture shows two 119-vertex buffers per run, 0x3b3ee8 and 0x3c55d8; the first is byte-for-byte what we decode, and the second is its exact X-reflection (every dumped position matches ours with x negated). So the container carries both halves as separate baked geometry, the engine draws each from its own buffer, and our decoder returning one buffer for both names is the defect — which correlate's mirror flag and ship::apply_twin_mirrors have been compensating for downstream all along.

That settles the question this section opened with, for this pair: not legitimate reuse. It also pins what the withdrawn neighbourhood-anchor fix could not: 0x3b3ee8 stays with whichever twin we already decode there, and the other twin must move to 0x3c55d8. The invariant is checkable without a capture — mirrored twins must decode to X-reflected buffers, never identical ones.

4. Root cause, for these three: selection, not the run scan. mesh::debug_vertex_run_starts exposes the candidate list the anchor scan works from. Stage_S01 yields 15 710 stride-24 candidate starts, and all three capture-proven offsets are in it0x3c55d8 (the mirrored twin), 0x40e418 (the drawn bridge buffer) and 0x44a32c (eng_02_l). The scan sees the right offsets; anchor_pool_mesh walks the list in ascending order and takes the first that validates, so an earlier lookalike wins — our bridge resources sit 0x5d0 before the buffer the engine drew.

This is scoped: it says the current decoder's e106 mis-anchors are selection failures. It does not overturn the earlier finding that the residual 51 under the withdrawn neighbourhood fix had no validating candidate at all — a different population, and the two can both be true.

What the twins suggest as the fix: selection is per-resource and greedy, so two resources can and do claim one buffer while a validating buffer sits unused. An assignment that is distinct by construction — each candidate used at most once — resolves the twin case by shape rather than by heuristic. Whether the proven offsets actually validate for their resources is the next thing to test; if they do, distinctness alone is the fix.

5. Do the proven offsets validate? Two of three — and that splits the fix. mesh::debug_try_anchor(bytes, name, vb, max_pad) asks validate_block directly (examples/try_anchor.rs):

resource proven offset verdict
e106_bdy_01_l / e106_bdy_02_l 0x3b3ee8 and 0x3c55d8 accepted for both, at both (v=119, idx=246, pad=0)
e106_brg_01_l / e106_brg_01_b_02 0x40e418 accepted for both (v=51, idx=126, pad=0)
e106_eng_02_l 0x44a32c rejected — and still rejected with the pad widened to 64

So the twin case is exactly what it looked like: the correct block is perfectly acceptable and simply lost the first-match race, and a distinct assignment (each candidate buffer claimed by at most one resource) fixes it — the two resources have two accepted offsets between them. The bridge pair is weaker: 0x40e418 is accepted by both, our current 0x40de48 is accepted too, so distinctness would separate them but not choose correctly.

eng_02_l is a different failure: the offset the engine drew from is not acceptable at all, so no selection policy can reach it. That is the "residual" class this file describes above, now with one member pinned to a concrete offset for the first time.

An open discrepancy, recorded not explained. The capture's DRAW lines carry an indices= field that does not agree with the descriptor's index count: the 119-vertex twin draws log indices=21 where our marker says 246, and the 44-vertex draw logs indices=12. Whether that field is an index count of a sub-range, a different unit, or a Xenia-side artifact is unknown — it may matter for eng_02_l, whose block validation is exactly what an index-count mismatch would break.

Not settled: e106_brg_01_b_02e106_brg_01_l (51 verts). A second 51-vertex vbase exists in the logs but is not from this container, and the container holds three near-identical 51-vertex runs, so the pair has no oracle yet. n006_01An006_01B shows a single vbase in all three logs — consistent with real reuse, but equally with only one of the two being on screen.