Locating each drawn buffer's positions inside Stage_S01.xpr shows vbase - offset is one constant (0x1A94FFF4, same in two runs), so a capture gives ground truth at file-offset granularity. Read against our anchor scan it is a defect list: full/_m resources starting at their own _l buffer, eight drawn buffers claimed by nobody, and the bdy_01_l/bdy_02_l twins sharing one buffer while the container carries both halves (0x3b3ee8 and its exact X-mirror at 0x3c55d8). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
648 lines
39 KiB
Markdown
648 lines
39 KiB
Markdown
# XBG7 — mesh geometry (inside XPR2 model containers)
|
||
|
||
- **Confidence:** 🟡 `PROBABLE` for the single-stream layout (below); ❔ `HYPOTHESIS` / undecoded
|
||
for the complex multi-stream body layout.
|
||
- **Parser in:** `sylpheed-formats/src/mesh.rs` (`Xbg7Model::from_xpr2`), tests
|
||
`tests/mesh_disc.rs`. Container parsing reused from `src/texture.rs` (`Xpr2Header` /
|
||
`Xpr2ResourceEntry`).
|
||
- **Applies to:** ship / weapon / prop models in `hidden/resource3d/*.xpr` (166 files).
|
||
- **Method:** clean-room — static hex inspection of the retail disc + geometric validation of the
|
||
recovered triangles (non-degenerate area, indices in range, bbox matches the descriptor's stored
|
||
size). No game code decompiled or copied.
|
||
|
||
## Where XBG7 lives
|
||
|
||
Models are ordinary **`XPR2`** containers (see the XPR2 texture doc / `texture.rs`). The 16-byte
|
||
resource-directory entries (from file offset `0x10`) carry `TX2D` texture resources **and** one or
|
||
more `XBG7` geometry resources:
|
||
|
||
```
|
||
entry = [ tag:4 ][ data_offset:u32 ][ descriptor_size:u32 ][ name_offset:u32 ] (big-endian)
|
||
```
|
||
|
||
Offsets are relative to the directory base `0x10`. The `XBG7` *descriptor* (at `data_offset+0x10`,
|
||
`descriptor_size` bytes) is a **scene / material / node graph** — it holds node names
|
||
(`rou_f001_mnt1_root`, `Light`, …), a bounding value (`0x41F00000` = 30.0 ≈ the ship's ~30-unit
|
||
length), material names matching the `TX2D` channels (`_col` albedo, `_spc` specular, `_gls` gloss,
|
||
`_lum` luminance), and per-sub-mesh records. The **vertex / index buffers** live in the container's
|
||
shared **data section** (from `header_size`).
|
||
|
||
## Sub-mesh records (in the descriptor)
|
||
|
||
Read in file order by a sliding 4-byte scan; each is a big-endian tuple:
|
||
|
||
```
|
||
[ vtx_count:u32 ][ 0:u32 ][ idx_count:u32 ][ tail:u32 ]
|
||
3..=65535 ==0 mult. of 3 1..=64
|
||
```
|
||
|
||
(For `rou_f001_wep_00`: `vtx_count=215`, `idx_count=1092` — matches the recovered geometry exactly.)
|
||
|
||
## The single-stream data layout — 🟡 PROBABLE (decoded, GPU-cross-checked)
|
||
|
||
For 36 of the 166 models (weapons, simple props) the data section is a straight sequence of
|
||
sub-meshes, carved from `header_size` in record order:
|
||
|
||
```
|
||
per sub-mesh block:
|
||
[ 12-byte header (contents undecoded) ]
|
||
[ index buffer : idx_count × u16 BE ] triangle list (prim=4, GPU-confirmed)
|
||
[ vertex buffer : vtx_count × stride bytes ] ← declaration-driven
|
||
(pad to 16 bytes → next sub-mesh)
|
||
```
|
||
|
||
The 12-byte header precedes the **index** buffer (the same block shape as stage
|
||
resources — see below); the vertex buffer follows the indices with no further
|
||
gap. (Earlier this was mis-modelled as `[index][12-byte gap][vertex]`, which put
|
||
the vertex buffer at the identical offset but read the index buffer 12 bytes too
|
||
early — turning the 12 header bytes into 6 junk indices = **2 leading degenerate
|
||
triangles** (a stray-triangle artifact) and dropping the last 6 real indices.
|
||
Skipping the header fixes the triangle list with no change to vertex coverage.)
|
||
|
||
**Vertex declaration.** The layout is **not fixed-stride**. The descriptor holds a declaration table
|
||
(right after the `(index_bytes, index_count)` marker) of `{offset:u32, format-code:u32, usage<<16:u32}`
|
||
big-endian triples, terminated by `offset == 0x00FF0000` / `code == 0xFFFFFFFF`:
|
||
|
||
| usage | element | format code | format | size |
|
||
|-------|----------|-------------|--------|------|
|
||
| 0x00 | POSITION | `0x2A23B9` | f32×3 | 12 B |
|
||
| 0x03 | NORMAL | `0x1A2360` | f16×4 (use xyz) | 8 B |
|
||
| 0x05 | TEXCOORD | `0x2C235F` | f16×2 (u,v) | 4 B |
|
||
|
||
Stride = max element extent. Models **omit elements** → variable stride (20 = pos+normal, no UV;
|
||
24 = pos+normal+uv). Each element is read in **naive big-endian component order** (the raw file
|
||
bytes; see the endianness note below). Assuming a fixed stride-24 was why the old decoder mis-aligned
|
||
and declined the pos+normal-only models.
|
||
|
||
**Alignment pinned by the normals.** The vertex buffer starts at `index_end + 12` (a fixed 12-byte
|
||
header — NOT `align16`, which lands 4 bytes early on most models). The correct offset is the unique
|
||
one where recovered normals are exactly unit-length.
|
||
|
||
**Safety gate:** every index is validated `< vtx_count`, and when the declaration has a normal
|
||
element the mean recovered `|normal|` must be ≈1 (`[0.5, 2.0]`). A model failing either is rejected
|
||
(`MeshError::UnsupportedLayout`) rather than emitting garbage.
|
||
|
||
### Endianness — file bytes are naive-BE, `k8in32` is a red herring ✅
|
||
|
||
The Canary GPU capture reports every vertex stream with fetch `endian = 2` (`k8in32`, each 32-bit
|
||
word byte-reversed). This describes the **guest-memory** copy the GPU fetches — **not** the `.xpr`
|
||
file bytes. Reading the file with a `k8in32` transform breaks the normals (mean `|normal|` → 1.33);
|
||
the plain per-element big-endian read yields exactly unit normals. So the game **rearranges** the
|
||
vertex data between the on-disc `.xpr` and the uploaded buffer; the decoder reads the file directly
|
||
and must use naive BE.
|
||
|
||
## Stage containers — multi-resource, grouped pools ✅ (content-anchored)
|
||
|
||
`hidden/resource3d/Stage_S*.xpr` are not single models but **collections of
|
||
enemy / prop sub-models** — up to ~400 `XBG7` resources each (e.g. `Stage_S07` =
|
||
378). Their data layout differs from the weapon files:
|
||
|
||
- Each resource is a block `[12-byte header][index buffer][vertex buffer]`.
|
||
Unlike weapons there is **no 12-byte gap between index and vertex** — the
|
||
vertex buffer directly follows the indices.
|
||
- The **index count** is the descriptor's `(index_bytes, index_count)` marker
|
||
(the *total* for the resource — a resource may have several sub-meshes summing
|
||
to it), **not** the first sub-mesh record. The **vertex count** is a `u32`
|
||
stored **32 bytes before** the marker.
|
||
- The blocks are **scattered through the data section, interleaved with the
|
||
container's texture data**, in an allocation order that is **not** directory
|
||
order and is **not stored** in any descriptor field we could find (the
|
||
descriptor holds sizes — `rel 160 ≈ index_bytes+3`, `rel 164 =
|
||
0x1000_0000 | (vertex_bytes+2)` — but no data offset). Reconstructing that
|
||
allocation order is unsolved.
|
||
|
||
Because the offset is not stored, each resource's block is located by
|
||
**content**: parser `Xbg7Model::stage_models` does one `O(file)` pass per
|
||
distinct stride to find every **vertex-buffer start** (an offset whose NORMAL —
|
||
`f16×4` at vertex `+12` — is unit length while the *previous* stride slot's is
|
||
not, i.e. a run boundary; ~one candidate per block, not millions), then pins each
|
||
resource to the unique candidate where the `index_count` indices ending just
|
||
before it are all `< vertex_count`, reference nearly all vertices, and yield
|
||
non-degenerate triangles with real extent. This is fast (≤ ~2 s on a 70 MB
|
||
stage) and unambiguous (no two resources collide). Blocks that fail validation —
|
||
the few quantized hero bodies — are **skipped**, never emitted as garbage.
|
||
|
||
**Coverage:** 5662 sub-models decode across the 22 stage files (e.g. `Stage_S07`
|
||
366/378, `Stage_S10` 7/9 — including the main enemy bodies `e003`/`e005`, their
|
||
LODs, weapons, and props). The viewer (`spawn_stage_models`) lays the decoded
|
||
sub-models out as a side-by-side "cast sheet", skipping the few huge skybox-plane
|
||
resources (300 k-unit quads). Cross-checked: `e003` = 2383 v / 1436 t bbox
|
||
23.5×9.5×32.5; `e005` = 2566 v / 1507 t; both 0 degenerate.
|
||
|
||
## Not yet decoded — ❔ the complex body layout
|
||
|
||
The hero-ship *body* meshes (`DeltaSaber_A/_T/_W.xpr` resource `f004`, and ~100 other models) still
|
||
decline. Two open sub-problems: (1) **multi-sub-mesh models** whose *first* sub-mesh decodes but a
|
||
later one's inter-mesh offset isn't yet handled (the `align16` advance is a guess) — these are
|
||
declined whole; (2) the big body meshes, where the data section does not start with an index buffer
|
||
and the geometry sits at descriptor-addressed offsets. NB the GPU capture showed **every** rendered
|
||
mesh is *single-stream* (just wider strides, e.g. 44 bytes = pos + f32×3 + colour + 2×f16×4), so the
|
||
body is likely single-stream-with-a-richer-declaration rather than the "separate streams" first
|
||
guessed — it was simply not rendered in the captured session (menu only). A capture taken *in a
|
||
mission* (where `DeltaSaber` renders) would hand over its exact declaration directly. `DeltaSaber_A`'s
|
||
5 XBG7 blocks are `f004` (body) + `_rou_f004_mnv01_L/_R`, `_mnv02`, `_turn180` (maneuver / pose).
|
||
|
||
## Evidence log
|
||
|
||
- 2026-07-12 — `rou_f001_wep_00.xpr`: XPR2 dir = 1×XBG7 (`f001_wep_00`) + 3×TX2D
|
||
(`_col/_gls/_spc`). Data section starts with a u16-BE index run (max 214), then stride-24
|
||
vertices. Descriptor record `[215,0,1092,4]` at desc `+0x2B0`; index-buffer byte size `0x888`
|
||
(=2184=1092×2) at desc `+0x1A0`. Recovered mesh = 215 v / 364 t, 362 non-degenerate, median tri
|
||
area 0.015 — coherent. → single-stream layout **PROBABLE**.
|
||
- 2026-07-12 — descriptor-driven sequential carving over all 166 models: **42 carve** under the
|
||
index-only check. Real 3D extent confirmed on `rou_f001_wep_04` (bbox 3.7×1.2×3.5).
|
||
- 2026-07-12 (refine) — attribute ranges differed per model (`wep_00` UV≈attr0/2, `wep_04`
|
||
attr4/5, `wep_03` attr1≈±60000 = garbage) → **refuted the fixed "6 half attrs, UV=attr0/2"
|
||
reading.** Found the descriptor **vertex declaration** (offsets 0x0C/0x14, usages 0x03 NORMAL /
|
||
0x05 TEXCOORD). Solved the vertex-base offset with a **unit-normal validator**: `index_end + 12`
|
||
gives median `|normal| = 1.000` on every weapon model (`wep_00/02/03/04`), vs `align16` landing 4
|
||
bytes early. UVs then land in `[0,1]`. Adding the normal gate: **25 models decode clean +
|
||
normal-valid** (the rest — incl. `Stage_S*` degenerate blobs — correctly declined).
|
||
- 2026-07-12 (DYNAMIC) — added a cvar-gated draw logger to Canary
|
||
(`command_processor.cc::LogDrawForRE`, cvar `log_draws`), captured the Ready Room / Briefings.
|
||
**GPU ground truth confirmed the static layout exactly**: primitive `prim=4` = **triangle LIST**
|
||
(settles the list-vs-strip question), and a stream with `f32x3 @offset0` + `f16x4 @3dw` +
|
||
`f16x2 @5dw`, stride 6 dwords = 24 bytes — matching `POSITION@0, NORMAL@0x0C, TEXCOORD@0x14`.
|
||
Revealed **stride varies** (24, 20, 28, 44 …) and that all streams are **single-stream** →
|
||
parsed the declaration for variable stride: coverage **25 → 36** (e.g. `wep_05` is pos+normal,
|
||
stride 20, no UV — previously mis-aligned). Also confirmed the endianness note above: fetch
|
||
`endian=2` (k8in32) is the *guest* copy; file stays naive-BE. `Stage_S*` now decode (stride 20).
|
||
- 2026-07-12 (STAGE) — `Stage_S*.xpr` decoded as multi-resource containers. Found each geometry
|
||
block is `[12B hdr][index buffer][vertex buffer]`, index count = descriptor marker (total, e.g.
|
||
`e003` = 4308 spanning 2 sub-meshes), vertex count = `u32` at marker−32 (`e003` = 2383 → verified
|
||
by max-index 2382 and 0 degenerate tris, bbox 23.5×9.5×32.5). Blocks are scattered among texture
|
||
data with **no stored offset** (descriptor rel 160 = idx_bytes+3, rel 164 = `0x1000_0000 |
|
||
(vtx_bytes+2)` are sizes, not offsets; block starts e.g. `e003`@0x10230, `e003_l`@0x5d000,
|
||
`e005_l`@0x9b000 are not directory-ordered). Solved by **content anchoring**: a single per-stride
|
||
pass finds vertex-run starts (unit NORMAL at +12 whose previous slot isn't), then match each
|
||
resource by strict index+triangle validation. **5662 sub-models decode across 22 stages** (S07
|
||
366/378), incl. the previously-declined main bodies `e005` (2566 v) and weapons — ≤2 s on 70 MB.
|
||
Parser `Xbg7Model::stage_models`, tests `stage_models_{decode,sweep,quality_audit}`.
|
||
- 2026-07-17 — **triangle-LIST re-confirmed; a strip interlude refuted; winding-consistency gate
|
||
added.** A 2026-07 change had briefly re-read the index buffers as triangle *strips* (to "fill
|
||
holes"). Refuted objectively with a new `XVERIFY` diagnostic that compares both readings by
|
||
**stored-normal agreement** (each triangle's cross-product face normal vs the sum of its vertices'
|
||
stored normals): the LIST reading gives agreement **1.000** on every clean weapon (`wep_00/03/04/19`
|
||
= only possible with correct topology + winding), the STRIP reading **~0.49** (random). The strip
|
||
reading also over-generated ~2.5× the triangles (wep_00: 938 vs 364) — a hole-filling garbage soup.
|
||
Reverted to LIST in both paths (`from_xpr2`, `read_pool_mesh`), matching the `prim=4` GPU capture.
|
||
Added an objective **winding-consistency gate** `max(na, 1-na)`: a correct carve is internally
|
||
consistent (agreement ≈1.0, or ≈0.0 for inverted-but-consistent winding — a real single-sided
|
||
mesh), a mis-carve scatters to the ≈0.5 middle. `from_xpr2` declines sub-meshes below 0.90 (e.g.
|
||
`wep_23` na=0.398 → declined instead of a spike-mess); the single-model **content-anchor fallback**
|
||
gates at 0.85; the large multi-resource **stage** path stays ungated (its enemy meshes span a
|
||
continuous 0.5–1.0 consistency range — a hard gate there dropped ~48/314 legit S07 blocks).
|
||
**Routing fixed:** `decode_models` (CLI) and the viewer now route by `count_xbg7` (1 → validated
|
||
records-based list decode, fallback to strict-gated anchor; >1 → stage anchor) instead of the old
|
||
"whichever decoder yields more verts" rule — that rule let stage content-anchoring win on
|
||
single-model weapon files and fabricate **phantom** blocks (a `wep_00` clone appearing inside
|
||
`wep_19`), duplicates, and spike-mess anchors. Weapons now: 33 clean-decode / 26 declined (declined
|
||
= genuinely multi-stream or un-carvable, shown as nothing rather than garbage); stage coverage
|
||
unchanged (S07 314). `expand_triangle_strip` retained as an `XVERIFY`-only diagnostic.
|
||
- 2026-07-12 — `DeltaSaber_A.xpr` body: data does **not** begin with indices; plain-`f32×3` runs
|
||
with ship-scale extent (span ≈27–34, matching bbox 30.0) found only at high offsets
|
||
(`data+0x28634C`, …) → multi-stream, **undecoded**.
|
||
- 2026-07-18 — **GROUPED-POOL layout cracked → the hero ship (Delta Saber) fully decodes.** The
|
||
detailed models (`DeltaSaber_*.xpr` + ~100 others) were declined for **location**, not format —
|
||
their vertex format is the standard stride-24 triangle list. A resource's *several* sub-meshes
|
||
don't interleave `[idx][vtx]` per block; they share **two grouped pools**: an **index pool**
|
||
(buffers concatenated in descriptor-marker order, each **4-byte aligned**) followed by a **vertex
|
||
pool** (each sub-pool `vtx_count × stride`, same order), with the index pool ending **exactly**
|
||
where the vertex pool begins. So the whole resource pivots on one unknown, the first vertex-pool
|
||
start `vb0` (= index-pool end, found by the unit-normal vertex-run scan); everything else is
|
||
derived: `ib0 = vb0 − span`, `ib[i] = align4(ib[i-1] + idx_count[i-1]·2)`,
|
||
`vb[i] = vb[i-1] + vtx_count[i-1]·stride`. Reversed statically from `DeltaSaber_T.xpr` and
|
||
**cross-checked against a Canary GPU draw-log capture** (mission ship = `DeltaSaber_T.xpr`, found
|
||
via the `--log_file_io` kernel hook): `f001` = body (idx@`data+0xC` = 0x5500C, vtx@0x61ACC, 10891 v
|
||
/ 8187 t) + **7 detail parts** (fins/cockpit/wingtips, markers at descriptor 0x3BEC…0x58FC) =
|
||
**8650 tris**, and **every sub-mesh decodes at 0 degenerate / full coverage / winding-agreement
|
||
1.000**. This is the layout the per-block adjacency anchor (`ib = vb − idx_bytes`) rendered as a
|
||
**spiky phantom** (it read 24561 indices starting 2782 B too late, agree 0.64, 1277 degenerate).
|
||
Insight: a single index marker reduces the grouped model to `index_end = vb0`, i.e. the existing
|
||
adjacency `ib = vb − idx_bytes` — so grouped **generalises** the single-block anchor (n=1 is
|
||
identical). Implemented as `anchor_grouped_meshes` (mesh.rs): `anchor_models` routes resources with
|
||
>1 index marker to it (validated per sub-mesh; on failure falls back to the old first-marker
|
||
adjacency anchor so stage coverage never regresses); single-marker stages/props keep the exact
|
||
prior path. The shared acceptance test is factored into `validate_block` (the connectivity
|
||
heuristic is relaxed for *derived* grouped parts, which are pinned by in-range + consistency, so
|
||
small flat fins aren't mis-rejected). Render self-check: `sylpheed-cli mesh render DeltaSaber_T.xpr
|
||
--only f001` (exact-name match excludes the `_rou_f001_mnv*` animation poses) → clean complete
|
||
fighter. Test `hero_ship_grouped_pool_decodes`. Colours/UVs still pending the running-game oracle.
|
||
- 2026-07-18 (refinement) — **4-byte vertex-pool alignment + weapon recovery.** The grouped-pool
|
||
rule "index pool ends exactly where the vertex pool begins" is really "the vertex pool is **4-byte
|
||
aligned** after the index pool": `vb0 = align4(ib0 + span)`, so 0..=3 bytes of padding can sit
|
||
between them. DeltaSaber's index pool ended already-aligned (pad 0), which hid this; **19
|
||
weapon/`*_hangar` models** (single- and multi-marker: `wep_08/11/34/58/62/69/81/83…`) have pad 2
|
||
and so decoded to *nothing* — the viewer then showed them as a flat 2D texture instead of a model.
|
||
Fix: both anchors try `pad ∈ 0..=3` (`ib = vb − idx_bytes − pad` for the single-block adjacency
|
||
anchor; `ib0 = vb0 − span − pad` for the grouped pivot), validated — a wrong pad reads shifted
|
||
indices → agreement collapses < 0.85, so only the true pad passes. pad>0 in the ungated stage path
|
||
is gated at a strict 0.85 to avoid a false anchor; pad 0 keeps its exact prior behaviour (stages
|
||
unchanged). Result: all 19 now decode as clean models (e.g. `wep_34` 1243 v / 1233 t, a
|
||
long-barrelled gun-pod; `wep_08` 3 sub-meshes / 478 t). Viewer routing already falls through
|
||
`from_xpr2` → `anchor_models(0.85)` for single-XBG7 files, so the recovered grouped/padded weapons
|
||
now preview as meshes.
|
||
- 2026-07-18 (refinement 2) — **pivot on the largest sub-mesh; all 19 recovered.** Three weapons
|
||
(`wep_81`, `wep_81_hangar`, `wep_30_hangar`) still declined because the grouped pivot validated
|
||
`markers[0]`, which for these is a tiny *elongated* lead bracket that fails the connectivity gate
|
||
even when perfectly placed. Fixed by pivoting the alignment check on the **largest** marker (max
|
||
index count) — the sub-mesh whose triangle-quality/connectivity signature most reliably confirms
|
||
`(ib0, vb0)`. Once the pivot validates, markers up to it are read unconditionally (a legitimately
|
||
tiny/flat lead part may fail the quality gates yet still be real), and markers after it stay
|
||
validated so a stray trailing marker ends the chain. Result: **all 19 previously-declined weapons
|
||
decode** (`wep_81` 460 t missile w/ tail fins; `wep_30_hangar` 334 t). DeltaSaber unchanged (its
|
||
body IS the largest marker → same pivot). 7/7 disc tests green, stage quality audit unchanged.
|
||
|
||
## The declined set, measured (2026-08-11)
|
||
|
||
The module note said "a few multi-stream / quantized bodies remain" declined.
|
||
Measured across all 166 `hidden/resource3d/*.xpr`:
|
||
|
||
- **6 294 XBG7 resources, 5 480 decoded (87.1 %), 814 declined**, in **31 of 166**
|
||
containers. Worst: `Stage_S09` 64/380, `Stage_S06` 58/324, `ptc_pack` 57/136.
|
||
- The declined set is **not** "a few hero bodies". By name prefix it is **492
|
||
`e*`** (enemy craft), **142 `f*`**, **73 `n*`**, **23 `eff*`**, plus destroyed
|
||
variants (`_rou_f402_dead`, `_rou_f302_base_dead`) and one weapon
|
||
(`_rou_e011_wep04`).
|
||
|
||
### A shortcut that does not work
|
||
|
||
The resource descriptor's third word looked like a format/stream flag — decoded
|
||
`g001…g003` carry `0x00010001` while declined `t170`/`t180` carry `0x00020004`,
|
||
which reads temptingly as `(streams << 16) | format`. **It is not that.**
|
||
Histogramming it over the whole disc puts decoded *and* declined resources at
|
||
every value:
|
||
|
||
```
|
||
word[2] decoded declined
|
||
0x00010001 4479 328
|
||
0x00010002 126 27
|
||
0x00010003 120 112
|
||
0x00010004 142 84
|
||
… … …
|
||
```
|
||
|
||
Its low half runs 1…0x52 and tracks sub-mesh count, not vertex format. **So
|
||
decodability is not declared in the descriptor** — it is a property of whether
|
||
the unit-normal anchor scan can locate `vb0`, which is exactly what the current
|
||
code already tests. Anyone attacking this should not spend time on the
|
||
descriptor: 229 of the declined resources even carry the *most* common
|
||
`0x00010001` with under 1 KB of data, i.e. they are small meshes the scan has too
|
||
little signal to anchor, not exotic formats.
|
||
|
||
## Silent mis-decodes: a detector, and how many there are (2026-08-11)
|
||
|
||
The [declined set](#the-declined-set-measured-2026-08-11) is the *honest* failure
|
||
mode — 814 resources the decoder refuses. This is the other kind: geometry that
|
||
decodes without complaint and is wrong.
|
||
|
||
### The case that exposed it
|
||
|
||
`e303_wep_01` decodes from fourteen containers. In eleven it is a **49 × 23 × 42**
|
||
turret with organic vertices (`24.55, 0.00, 4.46` …). In `Stage_S02`, `S08` and
|
||
`S26` the *same* resource — **identical 172 vertices and 330 indices** — decodes
|
||
to **1600 × 2100 × 4800** of axis-aligned box corners:
|
||
|
||
```
|
||
Stage_S01 [ 24.55 0.00 4.46] [ 24.55 9.84 2.91] normals varied
|
||
Stage_S02 [ 42.00 -900.00 2400.00] [-600.00 -500.00 -500.00] normals (0,0,1)
|
||
[ -600.00 -500.00 -950.00] [-600.00 -950.00 -950.00] ← box face corners
|
||
```
|
||
|
||
The anchor scan located a **different buffer that happens to share the vertex and
|
||
index counts**, so every size-based check it makes passes. This is exactly the
|
||
"declined only for *location*, not format" risk the module notes describe — except
|
||
here it does not decline, it succeeds wrongly.
|
||
|
||
### The detector: cross-container bounds consistency
|
||
|
||
A resource shared by several containers must decode to the same bounds. That
|
||
needs no ground truth, and it measures the problem:
|
||
|
||
- **681 resources appear in ≥2 containers.**
|
||
- **125 of them decode to different bounds while reporting identical vertex and
|
||
triangle counts** — a lower bound on silent mis-decodes (a resource wrong in
|
||
*every* container is invisible to this test).
|
||
|
||
Examples: `_rou_f401` decodes as `62×25×10` in 16 containers and `4738×3147×4738`
|
||
in 2; `_rou_e011_wep05` produces **four** different spans across 8 containers.
|
||
|
||
### Repair candidate, and its limits
|
||
|
||
Taking the **majority span** across containers resolves **104 of the 125**; 14 are
|
||
exact 50/50 splits that a vote cannot decide. It agrees with the ground truth in
|
||
the one case that has independent evidence — `e303_wep_01`, where the 11-container
|
||
majority is the turret the render and the runtime capture both support.
|
||
|
||
🟡 **It is a heuristic and is otherwise unvalidated.** For
|
||
`_rou_e302_base_break` the majority is the *larger* span (`685×1206×1444`, 8 of
|
||
15) and nothing yet says which is right. Use the detector to flag; do not silently
|
||
rewrite geometry on a vote.
|
||
|
||
### ROOT CAUSE: the candidate list is container-global
|
||
|
||
Traced 2026-08-12. `anchor_pool_mesh` (the **per-block** path, which is the one
|
||
that handles single-sub-mesh resources like `e303_wep_01`) walks a candidate list
|
||
built by `vertex_run_starts(bytes, data_base, stride)` — **one scan of the whole
|
||
container per stride**, shared by every resource of that stride. It accepts the
|
||
**first** candidate that validates.
|
||
|
||
So a resource is anchored to *whatever block matches its signature first in file
|
||
order*, and nothing ties that block to the resource it belongs to. Two resources
|
||
sharing `(stride, vertex count, index count)` are interchangeable to this search.
|
||
|
||
**The wrong block is not distinguishable by quality.** Tracing the accept for
|
||
`e303_wep_01`:
|
||
|
||
```
|
||
Stage_S01 ACCEPT vb=4600480 pad=0 span= 49 × 23 × 42 passes 0.85 = true
|
||
Stage_S02 ACCEPT vb=18403456 pad=0 span=1600 × 2100 × 4800 passes 0.85 = true
|
||
```
|
||
|
||
Both clear the strict winding-consistency gate, because the wrongly-taken block
|
||
**is** real, coherent geometry — just another resource's. That rules out a whole
|
||
family of fixes: no threshold, no scoring, no "pick the best candidate" changes
|
||
this, and the earlier attempt to add best-of-N selection in the grouped-pool
|
||
anchor duly changed nothing.
|
||
|
||
**The search space has to be constrained instead — and the fix is now pinned
|
||
down.**
|
||
|
||
**The correct block is already in the candidate list.** Enumerating *every*
|
||
validating candidate for `e303_wep_01` in `Stage_S02` gives exactly two:
|
||
|
||
```
|
||
vb = 18 403 456 span 1600 × 2100 × 4800 ← what the decoder takes, only because it is first
|
||
vb = 52 257 440 span 49 × 23 × 42 ← correct: the same size all 11 good containers give
|
||
```
|
||
|
||
So nothing needs to be found that the scan is missing; the wrong one merely
|
||
appears earlier in file order.
|
||
|
||
**Locality picks the right one.** Recording each resource's accepted anchor in
|
||
descriptor order shows that global **monotonicity is refuted** — only 25 of 47
|
||
steps increase in `Stage_S01` and 130 of 248 in `Stage_S02`, i.e. no better than
|
||
chance. But *neighbourhood* holds strongly: in `Stage_S02` this resource's
|
||
descriptor neighbours anchor at **51 974 668** and **52 218 424**, its correct
|
||
candidate is **52 257 440**, and the block it wrongly takes is at **18 403 456** —
|
||
two thirds of the file away from its own family.
|
||
|
||
**Proposed rule:** among candidates that validate, prefer the one **nearest the
|
||
anchors of the neighbouring resources** (equivalently: decode in descriptor order
|
||
and prefer candidates close to the previous resource's anchor), falling back to
|
||
first-match when there is no neighbour yet. That needs no new format knowledge,
|
||
and it selects `52 257 440` here.
|
||
|
||
### ⚠️ WITHDRAWN (2026-08-12) — it halved inconsistency but regressed the twin mirror
|
||
|
||
`anchor_pool_mesh_near` tries candidates in order of distance from a reference,
|
||
and `anchor_models_filtered` runs **two passes**: pass 1 anchors first-match to
|
||
learn where resources land, then pass 2 re-anchors each resource preferring its
|
||
**neighbourhood** — the median anchor of its ±2 descriptor neighbours. A resource
|
||
with too few anchored neighbours keeps pass 1's result, so nothing regresses to
|
||
guesswork.
|
||
|
||
| | decoded | shared | inconsistent | e106 mirror |
|
||
|---|---|---|---|---|
|
||
| shipped (today) | 5 480 / 6 294 | 681 | **125** | ✅ matches capture |
|
||
| neighbourhood anchor | 5 480 / 6 294 | 681 | 63 | ❌ flipped |
|
||
| + refining the map | 5 480 / 6 294 | 681 | 51 | ❌ flipped |
|
||
|
||
**Refining matters** because pass 1's anchor map contains the very mistakes the
|
||
neighbourhood is meant to correct, so a resource beside a mis-anchored neighbour
|
||
inherits a bad reference. Re-anchoring against the improving map and repeating
|
||
converges quickly — two rounds, with a third changing nothing.
|
||
|
||
On its own metric this looked complete: coverage unchanged, inconsistency
|
||
halved, `e303_wep_01` decoding to 49 × 23 × 42 in *all* containers, and `e106`
|
||
rendering as a destroyer instead of a slab
|
||
([before](../captures/e106-static-assembly-volume-bug.png) ·
|
||
[after](../captures/e106-static-assembly-fixed.png)).
|
||
|
||
**It was reverted anyway.** `ship::tests::static_assembly_matches_runtime_capture`
|
||
is gated on `SYLPHEED_ISO` and therefore skips in an ordinary `cargo test`; run
|
||
with the ISO it fails:
|
||
|
||
```
|
||
e106_bdy_01: static M row0 [-1.0, 0.0, 0.0] != captured [1.0, 0.0, 0.0]
|
||
```
|
||
|
||
**Why:** `e106_bdy_01` and `e106_bdy_02` are a mirrored pair whose two vertex
|
||
buffers hold the same geometry reflected in X, and **both resources currently
|
||
decode to the *same* buffer** — identical vertex count, identical span, identical
|
||
`mean_x`. `apply_twin_mirrors` decides which instance to reflect from the sign of
|
||
that `mean_x`, so which of the two buffers gets picked flips the decision:
|
||
|
||
```
|
||
before the change both twins decode with mean_x = −66.83 → mirror bdy_02 (matches the capture)
|
||
after the change both twins decode with mean_x = +66.83 → mirror bdy_01 (contradicts it)
|
||
```
|
||
|
||
The runtime capture is ground truth, so a change that contradicts it does not
|
||
ship.
|
||
|
||
**Correction (measured after the fact):** the first write-up of this said "two
|
||
distinct resources sharing one decode is itself the bug". **That is wrong.**
|
||
Sharing is normal here — 1 043 of 5 480 decoded resources (19 %) share geometry
|
||
with another resource, and of 1 242 related pairs, **1 241 are identical in every
|
||
container they co-occur in**, which is what legitimate asset reuse looks like.
|
||
A mirrored pair like `bdy_01`/`bdy_02` is *supposed* to share one geometry, with
|
||
the reflection applied at placement — exactly what `apply_twin_mirrors` does.
|
||
|
||
What actually matters is **which of two mirrored buffers is canonical**. The disc
|
||
holds both an X+ and an X− version; the engine treats one as the base, and
|
||
`apply_twin_mirrors` was tuned against that. The neighbourhood anchor moved these
|
||
resources to the *nearer* buffer, which is the other one — hence the flip. So the
|
||
real fix is not "give each twin its own buffer" but **pin which buffer is
|
||
canonical**, with the capture as the oracle.
|
||
|
||
**Exactly one pair is provably mis-anchored by this test**: `e105_bdy_02_l` /
|
||
`e105_brg_m` share a decode in 7 of the 15 containers holding both and differ in
|
||
the rest — two names cannot be the same geometry only sometimes.
|
||
|
||
**The filtered path needed care.** `models_named` (what the viewer's ship
|
||
rendering uses) drops non-wanted resources, which would leave a filtered decode
|
||
with no neighbourhood at all — and silently keep the old behaviour. Resources are
|
||
now collected regardless of the filter, but only the asked-for ones and their ±2
|
||
neighbours are decoded in pass 1, so a filtered decode stays proportional to what
|
||
was asked for.
|
||
|
||
**51 remain**, and they cluster in `_l` (LOD) and `_dead` variants — `e001_l`,
|
||
`e010_bdy_01_l`, `e011_bdy_01_l`, `e016_l`, `e104_bdy_05_l`, `e106_eng_02_l`,
|
||
`e501_01_l`, `_rou_f301_base_dead`, `_rou_f302_base_dead`, `e303_base_dead`.
|
||
|
||
**A tempting explanation, tested and false.** The obvious reading is that a
|
||
variant shares its base's vertex and index counts, so the two are mutually
|
||
confusable *and* adjacent, defeating locality. Checked across every container:
|
||
**2 714 variant/base pairs, and exactly zero share identical counts.**
|
||
|
||
**What is actually happening: one region is a universal false positive.**
|
||
`e010_bdy_01_l` is 171 verts / 90 tris and **no other resource in its container
|
||
shares those counts** — yet in `Stage_S02` and `S26` it decodes to
|
||
**1600 × 2100 × 4800**, the *same* bounds `e303_wep_01` (172 verts / 110 tris)
|
||
produced before the fix. Differently-shaped resources are landing on the same
|
||
place. So the attractor is not "another mesh with my shape" but a region of
|
||
**round, axis-aligned box data** that validates for many different `(vtx, idx)`
|
||
shapes at once — every index lands in range and the triangles are coherent boxes.
|
||
|
||
That also explains why the neighbourhood fix helped so broadly: it steers
|
||
resources away from a single strong attractor rather than resolving many
|
||
pairwise confusions.
|
||
|
||
**Selection-based fixes are exhausted — tested.** The proposed tiebreak was
|
||
implemented as a *last resort* (accept the attractor only if nothing else
|
||
validates), first keyed on "all coordinates multiples of 50" and then on the
|
||
sharper **"all coordinates integral"** — the attractor reads `(42, −900, 2400)`,
|
||
`(−600, −500, −950)` while real geometry carries fractions like
|
||
`(24.55, 9.84, 4.46)`. **Neither changed anything: still 51.**
|
||
|
||
That null result is itself the answer. A mechanism that defers the attractor
|
||
whenever another candidate exists, changing nothing, means **no alternative
|
||
candidate validates for any of the 51** — the correct block is *not in the
|
||
candidate list at all*. Both attempts were reverted rather than kept.
|
||
|
||
So the residual is **not** a selection problem, and no reordering, scoring or
|
||
tiebreak will move it. The frontier is `vertex_run_starts` — the unit-normal
|
||
run scan that builds the candidate list — which does not emit a start for these
|
||
resources' real vertex buffers. That is where the remaining 51 live.
|
||
|
||
The ignored test
|
||
[`mesh_consistency_disc.rs`](../../crates/sylpheed-formats/tests/mesh_consistency_disc.rs)
|
||
still asserts the target state and now records 63 rather than 125; the remaining
|
||
cases are where the neighbourhood is itself wrong or absent.
|
||
|
||
### Where the mis-decode is *not*: the grouped-pool anchor
|
||
|
||
An attempt to fix it by making `anchor_grouped_meshes` choose the **best-scoring**
|
||
`vb0` (rather than the first candidate clearing the 0.85 gate) changed **nothing**
|
||
— still 5 480 of 6 294 decoded and still 125 inconsistent — and instrumenting the
|
||
pivot loop shows why: for `e303_wep_01` it never runs. **The resource has a single
|
||
sub-mesh, so it is decoded by the per-block adjacency path
|
||
(`anchor_pool_mesh`), not the grouped-pool anchor.**
|
||
|
||
So the silent mis-decode lives in the **per-block** anchor. That is worth knowing
|
||
before anyone else spends time on the grouped-pool pivot, which is the more
|
||
prominent and better-documented of the two and the natural first suspect.
|
||
|
||
The change was reverted: it was untargeted, unproven, and added a scoring path
|
||
with no demonstrated benefit. (Its one reusable idea — that several `vb0`
|
||
candidates can clear the gate and first-in-scan-order is an arbitrary tiebreak —
|
||
still applies to whichever anchor turns out to be at fault.)
|
||
|
||
## Two follow-ups on the anchor (2026-08-12)
|
||
|
||
**The descriptor does not address the geometry.** If it did, the whole
|
||
candidate-scan could be replaced by direct addressing. It cannot: across `e106`'s
|
||
resources in `Stage_S01`, `anchored_vb − entry.data_offset` ranges from
|
||
**1 199 052 to 4 173 988** with no constant or stride. `data_offset` locates the
|
||
*descriptor*, and nothing in the first six descriptor words tracks the vertex
|
||
pool. The scan is necessary.
|
||
|
||
**Cross-container agreement does not prove legitimate reuse.** The earlier
|
||
measurement — 1 241 of 1 242 related pairs identical in every container — was
|
||
read as "sharing is normal". It is weaker than that: a **systematic** error is
|
||
invisible to a consistency test, because it is consistent. The same dump shows
|
||
`e106_bdy_02` and `e106_bdy_03_m` anchoring to the identical offset
|
||
(`1 505 556`), and `e106_bdy_01_l` with `e106_brg_01_m` (`4 251 208`) — a hull
|
||
half and a *different* body's medium LOD, or a hull half and a bridge LOD. Those
|
||
are different parts; one of each pair must be wrong.
|
||
|
||
So the honest position is: **sharing is common (19 %), some of it is certainly
|
||
legitimate (a mirrored twin pair genuinely shares one geometry), and some is
|
||
certainly not** — and cross-container consistency cannot tell them apart. A test
|
||
that can: compare a shared pair against a runtime capture, which is ground truth
|
||
for what the engine actually draws.
|
||
|
||
### The capture answers it at population level — and the logs are still on disc
|
||
|
||
`/sylph-home/re/shipcap/xenia_ship_capture_*.log` (kept from the 2026-07 capture
|
||
sessions) carry the raw per-draw lines the baked table was distilled from:
|
||
|
||
```
|
||
DRAW vbase=0x150CCAC0 stride=28 vcount=1 indices=1 prim=1 vs=0x…
|
||
DRAW vbase=0x150CCAC0 stride=24 vcount=10891 indices=18 prim=4 vs=0x…
|
||
```
|
||
|
||
`vbase` is the GPU vertex base — **ground truth for which buffer the engine draws
|
||
a part from**, which is exactly the oracle the sharing question needs. Over the
|
||
three logs, restricted to the ship-geometry stride 24:
|
||
|
||
- **6 093 draws from 2 291 distinct vbases**
|
||
- **only 77 vbases (3.4 %) are drawn more than once**
|
||
|
||
So in the engine, one buffer serving several parts is the **exception**. Our
|
||
decoder currently has **19 %** of resources sharing geometry with another. The
|
||
two figures are not the same measure — draws-per-buffer versus
|
||
resources-per-geometry — so this is evidence rather than proof, but it points the
|
||
same way as the `bdy_02` ≡ `bdy_03_m` case: **much of that 19 % is anchoring
|
||
error, not asset reuse.**
|
||
|
||
**How to use it per-part:** `correlate_capture` already matches a draw to a
|
||
resource by `vcount` plus decoded positions. The same match yields, for each
|
||
part, the `vbase` the engine used — so two resources that our decoder gives the
|
||
same geometry can be checked directly: different `vbase` in the capture ⇒ our
|
||
shared decode is wrong. That is the per-part oracle any future anchor work should
|
||
be validated against, and it needs no new capture run.
|
||
### ✅ It was run — and the capture gives file-offset ground truth
|
||
|
||
`examples/shared_vbase_check.rs` does the per-part check above, and then goes one
|
||
step further than planned. Three results, in order of strength.
|
||
|
||
**1. A draw's `vbase` *is* the container file offset plus a constant.** Vertex
|
||
POSITION is `f32×3` big-endian at vertex offset 0, so a draw's dumped positions
|
||
are a value pattern that can be searched for in the `.xpr` itself. Doing that for
|
||
every draw in `xenia_ship_capture_01/02.log` and histogramming `vbase − offset`:
|
||
|
||
```
|
||
xenia_ship_capture_01.log: 11 distinct vbases located, 208 not in this container
|
||
vbase - offset = 0x1A94FFF4 ×8 ← same constant in log 02
|
||
```
|
||
|
||
The 208 "not in this container" are draws whose geometry lives in `Common.xpr`,
|
||
a weapon pack or a backdrop — expected. The eight that do belong to `Stage_S01`
|
||
share **one** constant, and the *same* constant in a second run, so the container
|
||
is uploaded contiguously and **a capture names the exact file offset of every
|
||
buffer the engine drew**. Log 03 loaded the container at a different address, so
|
||
the constant is per-run, not baked.
|
||
|
||
**2. Read against our anchor scan, that is a defect list**
|
||
([`captures/stage-s01-capture-truth-offsets.txt`](../captures/stage-s01-capture-truth-offsets.txt)):
|
||
|
||
| file offset | drawn vcount | claimed by our decode |
|
||
|---|---|---|
|
||
| `0x3b3ee8` | 119 | `e106_bdy_01_l`(119), `e106_bdy_02_l`(119) |
|
||
| `0x3c55d8` | 119 | — **nobody** |
|
||
| `0x3dd2c4` | 146 | `e106_bdy_03`(815), `e106_bdy_03_l`(146) |
|
||
| `0x40763c` | 179 | `e106_bdy_04_l`(179) ✅ |
|
||
| `0x40e418` | 51 | `e106_brg_01`(202), `e106_brg_01_m`(92) |
|
||
| `0x444ccc` | 58 | `e106_eng_01_l`(58) ✅ |
|
||
| `0x45705c` | 82 | `e106_wep_02_01_l`(82), `e106_wep_02_01_m`(294) |
|
||
| `0x38788` `0x4b8b8` `0xb6574` `0xdbbac` `0x133da0` `0x162840` `0x44a32c` | 181, 93, 41, 77, 76, 60, 44 | — **nobody** |
|
||
|
||
Two resources land exactly (`bdy_04_l`, `eng_01_l`). Everywhere else a **full or
|
||
`_m` resource starts at the offset of its own `_l` buffer** — `bdy_03` decodes
|
||
815 vertices beginning where the engine's 146-vertex LOD begins — and eight
|
||
drawn buffers are claimed by no resource at all. This is the anchor scan taking
|
||
the first candidate that validates, seen directly rather than inferred.
|
||
|
||
**3. The twin pair is an anchoring error, and the mirror is in the data.** For
|
||
`e106_bdy_01_l` ≡ `e106_bdy_02_l` the capture shows **two** 119-vertex buffers
|
||
per run, `0x3b3ee8` and `0x3c55d8`; the first is byte-for-byte what we decode,
|
||
and the second is its **exact X-reflection** (every dumped position matches ours
|
||
with `x` negated). So the container carries both halves as separate baked
|
||
geometry, the engine draws each from its own buffer, and our decoder returning
|
||
one buffer for both names is the defect — which `correlate`'s mirror flag and
|
||
`ship::apply_twin_mirrors` have been compensating for downstream all along.
|
||
|
||
That settles the question this section opened with, for this pair: **not
|
||
legitimate reuse.** It also pins what the withdrawn neighbourhood-anchor fix
|
||
could not: `0x3b3ee8` stays with whichever twin we already decode there, and the
|
||
other twin must move to `0x3c55d8`. The invariant is checkable without a capture
|
||
— *mirrored twins must decode to X-reflected buffers, never identical ones*.
|
||
|
||
Not settled: `e106_brg_01_b_02` ≡ `e106_brg_01_l` (51 verts). A second 51-vertex
|
||
`vbase` exists in the logs but is **not** from this container, and the container
|
||
holds three near-identical 51-vertex runs, so the pair has no oracle yet.
|
||
`n006_01A` ≡ `n006_01B` shows a single `vbase` in all three logs — consistent
|
||
with real reuse, but equally with only one of the two being on screen.
|