Two follow-ups, both negative, both worth recording so the next attempt does not repeat them. The descriptor cannot replace the candidate scan. If entry.data_offset located the geometry, anchoring would be direct addressing. Across e106's resources in Stage_S01, anchored_vb - data_offset ranges from 1199052 to 4173988 with no constant or stride; data_offset locates the DESCRIPTOR and none of the first six descriptor words tracks the vertex pool. And yesterday's "1241 of 1242 pairs identical in every container, so sharing is legitimate reuse" is weaker than it read. A SYSTEMATIC error is invisible to a consistency test, because it is consistent. The same dump shows e106_bdy_02 and e106_bdy_03_m anchoring to the identical offset (1505556), and e106_bdy_01_l with e106_brg_01_m (4251208) -- a hull half and a different body's medium LOD, a hull half and a bridge LOD. Different parts; one of each pair is wrong. Honest position: sharing is common (19%), some is certainly legitimate (a mirrored twin pair genuinely shares one geometry) and some certainly is not, and cross-container consistency cannot separate them. What can: comparing a shared pair against a runtime capture. Instrumentation reverted; only docs land here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
555 lines
34 KiB
Markdown
555 lines
34 KiB
Markdown
# XBG7 — mesh geometry (inside XPR2 model containers)
|
||
|
||
- **Confidence:** 🟡 `PROBABLE` for the single-stream layout (below); ❔ `HYPOTHESIS` / undecoded
|
||
for the complex multi-stream body layout.
|
||
- **Parser in:** `sylpheed-formats/src/mesh.rs` (`Xbg7Model::from_xpr2`), tests
|
||
`tests/mesh_disc.rs`. Container parsing reused from `src/texture.rs` (`Xpr2Header` /
|
||
`Xpr2ResourceEntry`).
|
||
- **Applies to:** ship / weapon / prop models in `hidden/resource3d/*.xpr` (166 files).
|
||
- **Method:** clean-room — static hex inspection of the retail disc + geometric validation of the
|
||
recovered triangles (non-degenerate area, indices in range, bbox matches the descriptor's stored
|
||
size). No game code decompiled or copied.
|
||
|
||
## Where XBG7 lives
|
||
|
||
Models are ordinary **`XPR2`** containers (see the XPR2 texture doc / `texture.rs`). The 16-byte
|
||
resource-directory entries (from file offset `0x10`) carry `TX2D` texture resources **and** one or
|
||
more `XBG7` geometry resources:
|
||
|
||
```
|
||
entry = [ tag:4 ][ data_offset:u32 ][ descriptor_size:u32 ][ name_offset:u32 ] (big-endian)
|
||
```
|
||
|
||
Offsets are relative to the directory base `0x10`. The `XBG7` *descriptor* (at `data_offset+0x10`,
|
||
`descriptor_size` bytes) is a **scene / material / node graph** — it holds node names
|
||
(`rou_f001_mnt1_root`, `Light`, …), a bounding value (`0x41F00000` = 30.0 ≈ the ship's ~30-unit
|
||
length), material names matching the `TX2D` channels (`_col` albedo, `_spc` specular, `_gls` gloss,
|
||
`_lum` luminance), and per-sub-mesh records. The **vertex / index buffers** live in the container's
|
||
shared **data section** (from `header_size`).
|
||
|
||
## Sub-mesh records (in the descriptor)
|
||
|
||
Read in file order by a sliding 4-byte scan; each is a big-endian tuple:
|
||
|
||
```
|
||
[ vtx_count:u32 ][ 0:u32 ][ idx_count:u32 ][ tail:u32 ]
|
||
3..=65535 ==0 mult. of 3 1..=64
|
||
```
|
||
|
||
(For `rou_f001_wep_00`: `vtx_count=215`, `idx_count=1092` — matches the recovered geometry exactly.)
|
||
|
||
## The single-stream data layout — 🟡 PROBABLE (decoded, GPU-cross-checked)
|
||
|
||
For 36 of the 166 models (weapons, simple props) the data section is a straight sequence of
|
||
sub-meshes, carved from `header_size` in record order:
|
||
|
||
```
|
||
per sub-mesh block:
|
||
[ 12-byte header (contents undecoded) ]
|
||
[ index buffer : idx_count × u16 BE ] triangle list (prim=4, GPU-confirmed)
|
||
[ vertex buffer : vtx_count × stride bytes ] ← declaration-driven
|
||
(pad to 16 bytes → next sub-mesh)
|
||
```
|
||
|
||
The 12-byte header precedes the **index** buffer (the same block shape as stage
|
||
resources — see below); the vertex buffer follows the indices with no further
|
||
gap. (Earlier this was mis-modelled as `[index][12-byte gap][vertex]`, which put
|
||
the vertex buffer at the identical offset but read the index buffer 12 bytes too
|
||
early — turning the 12 header bytes into 6 junk indices = **2 leading degenerate
|
||
triangles** (a stray-triangle artifact) and dropping the last 6 real indices.
|
||
Skipping the header fixes the triangle list with no change to vertex coverage.)
|
||
|
||
**Vertex declaration.** The layout is **not fixed-stride**. The descriptor holds a declaration table
|
||
(right after the `(index_bytes, index_count)` marker) of `{offset:u32, format-code:u32, usage<<16:u32}`
|
||
big-endian triples, terminated by `offset == 0x00FF0000` / `code == 0xFFFFFFFF`:
|
||
|
||
| usage | element | format code | format | size |
|
||
|-------|----------|-------------|--------|------|
|
||
| 0x00 | POSITION | `0x2A23B9` | f32×3 | 12 B |
|
||
| 0x03 | NORMAL | `0x1A2360` | f16×4 (use xyz) | 8 B |
|
||
| 0x05 | TEXCOORD | `0x2C235F` | f16×2 (u,v) | 4 B |
|
||
|
||
Stride = max element extent. Models **omit elements** → variable stride (20 = pos+normal, no UV;
|
||
24 = pos+normal+uv). Each element is read in **naive big-endian component order** (the raw file
|
||
bytes; see the endianness note below). Assuming a fixed stride-24 was why the old decoder mis-aligned
|
||
and declined the pos+normal-only models.
|
||
|
||
**Alignment pinned by the normals.** The vertex buffer starts at `index_end + 12` (a fixed 12-byte
|
||
header — NOT `align16`, which lands 4 bytes early on most models). The correct offset is the unique
|
||
one where recovered normals are exactly unit-length.
|
||
|
||
**Safety gate:** every index is validated `< vtx_count`, and when the declaration has a normal
|
||
element the mean recovered `|normal|` must be ≈1 (`[0.5, 2.0]`). A model failing either is rejected
|
||
(`MeshError::UnsupportedLayout`) rather than emitting garbage.
|
||
|
||
### Endianness — file bytes are naive-BE, `k8in32` is a red herring ✅
|
||
|
||
The Canary GPU capture reports every vertex stream with fetch `endian = 2` (`k8in32`, each 32-bit
|
||
word byte-reversed). This describes the **guest-memory** copy the GPU fetches — **not** the `.xpr`
|
||
file bytes. Reading the file with a `k8in32` transform breaks the normals (mean `|normal|` → 1.33);
|
||
the plain per-element big-endian read yields exactly unit normals. So the game **rearranges** the
|
||
vertex data between the on-disc `.xpr` and the uploaded buffer; the decoder reads the file directly
|
||
and must use naive BE.
|
||
|
||
## Stage containers — multi-resource, grouped pools ✅ (content-anchored)
|
||
|
||
`hidden/resource3d/Stage_S*.xpr` are not single models but **collections of
|
||
enemy / prop sub-models** — up to ~400 `XBG7` resources each (e.g. `Stage_S07` =
|
||
378). Their data layout differs from the weapon files:
|
||
|
||
- Each resource is a block `[12-byte header][index buffer][vertex buffer]`.
|
||
Unlike weapons there is **no 12-byte gap between index and vertex** — the
|
||
vertex buffer directly follows the indices.
|
||
- The **index count** is the descriptor's `(index_bytes, index_count)` marker
|
||
(the *total* for the resource — a resource may have several sub-meshes summing
|
||
to it), **not** the first sub-mesh record. The **vertex count** is a `u32`
|
||
stored **32 bytes before** the marker.
|
||
- The blocks are **scattered through the data section, interleaved with the
|
||
container's texture data**, in an allocation order that is **not** directory
|
||
order and is **not stored** in any descriptor field we could find (the
|
||
descriptor holds sizes — `rel 160 ≈ index_bytes+3`, `rel 164 =
|
||
0x1000_0000 | (vertex_bytes+2)` — but no data offset). Reconstructing that
|
||
allocation order is unsolved.
|
||
|
||
Because the offset is not stored, each resource's block is located by
|
||
**content**: parser `Xbg7Model::stage_models` does one `O(file)` pass per
|
||
distinct stride to find every **vertex-buffer start** (an offset whose NORMAL —
|
||
`f16×4` at vertex `+12` — is unit length while the *previous* stride slot's is
|
||
not, i.e. a run boundary; ~one candidate per block, not millions), then pins each
|
||
resource to the unique candidate where the `index_count` indices ending just
|
||
before it are all `< vertex_count`, reference nearly all vertices, and yield
|
||
non-degenerate triangles with real extent. This is fast (≤ ~2 s on a 70 MB
|
||
stage) and unambiguous (no two resources collide). Blocks that fail validation —
|
||
the few quantized hero bodies — are **skipped**, never emitted as garbage.
|
||
|
||
**Coverage:** 5662 sub-models decode across the 22 stage files (e.g. `Stage_S07`
|
||
366/378, `Stage_S10` 7/9 — including the main enemy bodies `e003`/`e005`, their
|
||
LODs, weapons, and props). The viewer (`spawn_stage_models`) lays the decoded
|
||
sub-models out as a side-by-side "cast sheet", skipping the few huge skybox-plane
|
||
resources (300 k-unit quads). Cross-checked: `e003` = 2383 v / 1436 t bbox
|
||
23.5×9.5×32.5; `e005` = 2566 v / 1507 t; both 0 degenerate.
|
||
|
||
## Not yet decoded — ❔ the complex body layout
|
||
|
||
The hero-ship *body* meshes (`DeltaSaber_A/_T/_W.xpr` resource `f004`, and ~100 other models) still
|
||
decline. Two open sub-problems: (1) **multi-sub-mesh models** whose *first* sub-mesh decodes but a
|
||
later one's inter-mesh offset isn't yet handled (the `align16` advance is a guess) — these are
|
||
declined whole; (2) the big body meshes, where the data section does not start with an index buffer
|
||
and the geometry sits at descriptor-addressed offsets. NB the GPU capture showed **every** rendered
|
||
mesh is *single-stream* (just wider strides, e.g. 44 bytes = pos + f32×3 + colour + 2×f16×4), so the
|
||
body is likely single-stream-with-a-richer-declaration rather than the "separate streams" first
|
||
guessed — it was simply not rendered in the captured session (menu only). A capture taken *in a
|
||
mission* (where `DeltaSaber` renders) would hand over its exact declaration directly. `DeltaSaber_A`'s
|
||
5 XBG7 blocks are `f004` (body) + `_rou_f004_mnv01_L/_R`, `_mnv02`, `_turn180` (maneuver / pose).
|
||
|
||
## Evidence log
|
||
|
||
- 2026-07-12 — `rou_f001_wep_00.xpr`: XPR2 dir = 1×XBG7 (`f001_wep_00`) + 3×TX2D
|
||
(`_col/_gls/_spc`). Data section starts with a u16-BE index run (max 214), then stride-24
|
||
vertices. Descriptor record `[215,0,1092,4]` at desc `+0x2B0`; index-buffer byte size `0x888`
|
||
(=2184=1092×2) at desc `+0x1A0`. Recovered mesh = 215 v / 364 t, 362 non-degenerate, median tri
|
||
area 0.015 — coherent. → single-stream layout **PROBABLE**.
|
||
- 2026-07-12 — descriptor-driven sequential carving over all 166 models: **42 carve** under the
|
||
index-only check. Real 3D extent confirmed on `rou_f001_wep_04` (bbox 3.7×1.2×3.5).
|
||
- 2026-07-12 (refine) — attribute ranges differed per model (`wep_00` UV≈attr0/2, `wep_04`
|
||
attr4/5, `wep_03` attr1≈±60000 = garbage) → **refuted the fixed "6 half attrs, UV=attr0/2"
|
||
reading.** Found the descriptor **vertex declaration** (offsets 0x0C/0x14, usages 0x03 NORMAL /
|
||
0x05 TEXCOORD). Solved the vertex-base offset with a **unit-normal validator**: `index_end + 12`
|
||
gives median `|normal| = 1.000` on every weapon model (`wep_00/02/03/04`), vs `align16` landing 4
|
||
bytes early. UVs then land in `[0,1]`. Adding the normal gate: **25 models decode clean +
|
||
normal-valid** (the rest — incl. `Stage_S*` degenerate blobs — correctly declined).
|
||
- 2026-07-12 (DYNAMIC) — added a cvar-gated draw logger to Canary
|
||
(`command_processor.cc::LogDrawForRE`, cvar `log_draws`), captured the Ready Room / Briefings.
|
||
**GPU ground truth confirmed the static layout exactly**: primitive `prim=4` = **triangle LIST**
|
||
(settles the list-vs-strip question), and a stream with `f32x3 @offset0` + `f16x4 @3dw` +
|
||
`f16x2 @5dw`, stride 6 dwords = 24 bytes — matching `POSITION@0, NORMAL@0x0C, TEXCOORD@0x14`.
|
||
Revealed **stride varies** (24, 20, 28, 44 …) and that all streams are **single-stream** →
|
||
parsed the declaration for variable stride: coverage **25 → 36** (e.g. `wep_05` is pos+normal,
|
||
stride 20, no UV — previously mis-aligned). Also confirmed the endianness note above: fetch
|
||
`endian=2` (k8in32) is the *guest* copy; file stays naive-BE. `Stage_S*` now decode (stride 20).
|
||
- 2026-07-12 (STAGE) — `Stage_S*.xpr` decoded as multi-resource containers. Found each geometry
|
||
block is `[12B hdr][index buffer][vertex buffer]`, index count = descriptor marker (total, e.g.
|
||
`e003` = 4308 spanning 2 sub-meshes), vertex count = `u32` at marker−32 (`e003` = 2383 → verified
|
||
by max-index 2382 and 0 degenerate tris, bbox 23.5×9.5×32.5). Blocks are scattered among texture
|
||
data with **no stored offset** (descriptor rel 160 = idx_bytes+3, rel 164 = `0x1000_0000 |
|
||
(vtx_bytes+2)` are sizes, not offsets; block starts e.g. `e003`@0x10230, `e003_l`@0x5d000,
|
||
`e005_l`@0x9b000 are not directory-ordered). Solved by **content anchoring**: a single per-stride
|
||
pass finds vertex-run starts (unit NORMAL at +12 whose previous slot isn't), then match each
|
||
resource by strict index+triangle validation. **5662 sub-models decode across 22 stages** (S07
|
||
366/378), incl. the previously-declined main bodies `e005` (2566 v) and weapons — ≤2 s on 70 MB.
|
||
Parser `Xbg7Model::stage_models`, tests `stage_models_{decode,sweep,quality_audit}`.
|
||
- 2026-07-17 — **triangle-LIST re-confirmed; a strip interlude refuted; winding-consistency gate
|
||
added.** A 2026-07 change had briefly re-read the index buffers as triangle *strips* (to "fill
|
||
holes"). Refuted objectively with a new `XVERIFY` diagnostic that compares both readings by
|
||
**stored-normal agreement** (each triangle's cross-product face normal vs the sum of its vertices'
|
||
stored normals): the LIST reading gives agreement **1.000** on every clean weapon (`wep_00/03/04/19`
|
||
= only possible with correct topology + winding), the STRIP reading **~0.49** (random). The strip
|
||
reading also over-generated ~2.5× the triangles (wep_00: 938 vs 364) — a hole-filling garbage soup.
|
||
Reverted to LIST in both paths (`from_xpr2`, `read_pool_mesh`), matching the `prim=4` GPU capture.
|
||
Added an objective **winding-consistency gate** `max(na, 1-na)`: a correct carve is internally
|
||
consistent (agreement ≈1.0, or ≈0.0 for inverted-but-consistent winding — a real single-sided
|
||
mesh), a mis-carve scatters to the ≈0.5 middle. `from_xpr2` declines sub-meshes below 0.90 (e.g.
|
||
`wep_23` na=0.398 → declined instead of a spike-mess); the single-model **content-anchor fallback**
|
||
gates at 0.85; the large multi-resource **stage** path stays ungated (its enemy meshes span a
|
||
continuous 0.5–1.0 consistency range — a hard gate there dropped ~48/314 legit S07 blocks).
|
||
**Routing fixed:** `decode_models` (CLI) and the viewer now route by `count_xbg7` (1 → validated
|
||
records-based list decode, fallback to strict-gated anchor; >1 → stage anchor) instead of the old
|
||
"whichever decoder yields more verts" rule — that rule let stage content-anchoring win on
|
||
single-model weapon files and fabricate **phantom** blocks (a `wep_00` clone appearing inside
|
||
`wep_19`), duplicates, and spike-mess anchors. Weapons now: 33 clean-decode / 26 declined (declined
|
||
= genuinely multi-stream or un-carvable, shown as nothing rather than garbage); stage coverage
|
||
unchanged (S07 314). `expand_triangle_strip` retained as an `XVERIFY`-only diagnostic.
|
||
- 2026-07-12 — `DeltaSaber_A.xpr` body: data does **not** begin with indices; plain-`f32×3` runs
|
||
with ship-scale extent (span ≈27–34, matching bbox 30.0) found only at high offsets
|
||
(`data+0x28634C`, …) → multi-stream, **undecoded**.
|
||
- 2026-07-18 — **GROUPED-POOL layout cracked → the hero ship (Delta Saber) fully decodes.** The
|
||
detailed models (`DeltaSaber_*.xpr` + ~100 others) were declined for **location**, not format —
|
||
their vertex format is the standard stride-24 triangle list. A resource's *several* sub-meshes
|
||
don't interleave `[idx][vtx]` per block; they share **two grouped pools**: an **index pool**
|
||
(buffers concatenated in descriptor-marker order, each **4-byte aligned**) followed by a **vertex
|
||
pool** (each sub-pool `vtx_count × stride`, same order), with the index pool ending **exactly**
|
||
where the vertex pool begins. So the whole resource pivots on one unknown, the first vertex-pool
|
||
start `vb0` (= index-pool end, found by the unit-normal vertex-run scan); everything else is
|
||
derived: `ib0 = vb0 − span`, `ib[i] = align4(ib[i-1] + idx_count[i-1]·2)`,
|
||
`vb[i] = vb[i-1] + vtx_count[i-1]·stride`. Reversed statically from `DeltaSaber_T.xpr` and
|
||
**cross-checked against a Canary GPU draw-log capture** (mission ship = `DeltaSaber_T.xpr`, found
|
||
via the `--log_file_io` kernel hook): `f001` = body (idx@`data+0xC` = 0x5500C, vtx@0x61ACC, 10891 v
|
||
/ 8187 t) + **7 detail parts** (fins/cockpit/wingtips, markers at descriptor 0x3BEC…0x58FC) =
|
||
**8650 tris**, and **every sub-mesh decodes at 0 degenerate / full coverage / winding-agreement
|
||
1.000**. This is the layout the per-block adjacency anchor (`ib = vb − idx_bytes`) rendered as a
|
||
**spiky phantom** (it read 24561 indices starting 2782 B too late, agree 0.64, 1277 degenerate).
|
||
Insight: a single index marker reduces the grouped model to `index_end = vb0`, i.e. the existing
|
||
adjacency `ib = vb − idx_bytes` — so grouped **generalises** the single-block anchor (n=1 is
|
||
identical). Implemented as `anchor_grouped_meshes` (mesh.rs): `anchor_models` routes resources with
|
||
>1 index marker to it (validated per sub-mesh; on failure falls back to the old first-marker
|
||
adjacency anchor so stage coverage never regresses); single-marker stages/props keep the exact
|
||
prior path. The shared acceptance test is factored into `validate_block` (the connectivity
|
||
heuristic is relaxed for *derived* grouped parts, which are pinned by in-range + consistency, so
|
||
small flat fins aren't mis-rejected). Render self-check: `sylpheed-cli mesh render DeltaSaber_T.xpr
|
||
--only f001` (exact-name match excludes the `_rou_f001_mnv*` animation poses) → clean complete
|
||
fighter. Test `hero_ship_grouped_pool_decodes`. Colours/UVs still pending the running-game oracle.
|
||
- 2026-07-18 (refinement) — **4-byte vertex-pool alignment + weapon recovery.** The grouped-pool
|
||
rule "index pool ends exactly where the vertex pool begins" is really "the vertex pool is **4-byte
|
||
aligned** after the index pool": `vb0 = align4(ib0 + span)`, so 0..=3 bytes of padding can sit
|
||
between them. DeltaSaber's index pool ended already-aligned (pad 0), which hid this; **19
|
||
weapon/`*_hangar` models** (single- and multi-marker: `wep_08/11/34/58/62/69/81/83…`) have pad 2
|
||
and so decoded to *nothing* — the viewer then showed them as a flat 2D texture instead of a model.
|
||
Fix: both anchors try `pad ∈ 0..=3` (`ib = vb − idx_bytes − pad` for the single-block adjacency
|
||
anchor; `ib0 = vb0 − span − pad` for the grouped pivot), validated — a wrong pad reads shifted
|
||
indices → agreement collapses < 0.85, so only the true pad passes. pad>0 in the ungated stage path
|
||
is gated at a strict 0.85 to avoid a false anchor; pad 0 keeps its exact prior behaviour (stages
|
||
unchanged). Result: all 19 now decode as clean models (e.g. `wep_34` 1243 v / 1233 t, a
|
||
long-barrelled gun-pod; `wep_08` 3 sub-meshes / 478 t). Viewer routing already falls through
|
||
`from_xpr2` → `anchor_models(0.85)` for single-XBG7 files, so the recovered grouped/padded weapons
|
||
now preview as meshes.
|
||
- 2026-07-18 (refinement 2) — **pivot on the largest sub-mesh; all 19 recovered.** Three weapons
|
||
(`wep_81`, `wep_81_hangar`, `wep_30_hangar`) still declined because the grouped pivot validated
|
||
`markers[0]`, which for these is a tiny *elongated* lead bracket that fails the connectivity gate
|
||
even when perfectly placed. Fixed by pivoting the alignment check on the **largest** marker (max
|
||
index count) — the sub-mesh whose triangle-quality/connectivity signature most reliably confirms
|
||
`(ib0, vb0)`. Once the pivot validates, markers up to it are read unconditionally (a legitimately
|
||
tiny/flat lead part may fail the quality gates yet still be real), and markers after it stay
|
||
validated so a stray trailing marker ends the chain. Result: **all 19 previously-declined weapons
|
||
decode** (`wep_81` 460 t missile w/ tail fins; `wep_30_hangar` 334 t). DeltaSaber unchanged (its
|
||
body IS the largest marker → same pivot). 7/7 disc tests green, stage quality audit unchanged.
|
||
|
||
## The declined set, measured (2026-08-11)
|
||
|
||
The module note said "a few multi-stream / quantized bodies remain" declined.
|
||
Measured across all 166 `hidden/resource3d/*.xpr`:
|
||
|
||
- **6 294 XBG7 resources, 5 480 decoded (87.1 %), 814 declined**, in **31 of 166**
|
||
containers. Worst: `Stage_S09` 64/380, `Stage_S06` 58/324, `ptc_pack` 57/136.
|
||
- The declined set is **not** "a few hero bodies". By name prefix it is **492
|
||
`e*`** (enemy craft), **142 `f*`**, **73 `n*`**, **23 `eff*`**, plus destroyed
|
||
variants (`_rou_f402_dead`, `_rou_f302_base_dead`) and one weapon
|
||
(`_rou_e011_wep04`).
|
||
|
||
### A shortcut that does not work
|
||
|
||
The resource descriptor's third word looked like a format/stream flag — decoded
|
||
`g001…g003` carry `0x00010001` while declined `t170`/`t180` carry `0x00020004`,
|
||
which reads temptingly as `(streams << 16) | format`. **It is not that.**
|
||
Histogramming it over the whole disc puts decoded *and* declined resources at
|
||
every value:
|
||
|
||
```
|
||
word[2] decoded declined
|
||
0x00010001 4479 328
|
||
0x00010002 126 27
|
||
0x00010003 120 112
|
||
0x00010004 142 84
|
||
… … …
|
||
```
|
||
|
||
Its low half runs 1…0x52 and tracks sub-mesh count, not vertex format. **So
|
||
decodability is not declared in the descriptor** — it is a property of whether
|
||
the unit-normal anchor scan can locate `vb0`, which is exactly what the current
|
||
code already tests. Anyone attacking this should not spend time on the
|
||
descriptor: 229 of the declined resources even carry the *most* common
|
||
`0x00010001` with under 1 KB of data, i.e. they are small meshes the scan has too
|
||
little signal to anchor, not exotic formats.
|
||
|
||
## Silent mis-decodes: a detector, and how many there are (2026-08-11)
|
||
|
||
The [declined set](#the-declined-set-measured-2026-08-11) is the *honest* failure
|
||
mode — 814 resources the decoder refuses. This is the other kind: geometry that
|
||
decodes without complaint and is wrong.
|
||
|
||
### The case that exposed it
|
||
|
||
`e303_wep_01` decodes from fourteen containers. In eleven it is a **49 × 23 × 42**
|
||
turret with organic vertices (`24.55, 0.00, 4.46` …). In `Stage_S02`, `S08` and
|
||
`S26` the *same* resource — **identical 172 vertices and 330 indices** — decodes
|
||
to **1600 × 2100 × 4800** of axis-aligned box corners:
|
||
|
||
```
|
||
Stage_S01 [ 24.55 0.00 4.46] [ 24.55 9.84 2.91] normals varied
|
||
Stage_S02 [ 42.00 -900.00 2400.00] [-600.00 -500.00 -500.00] normals (0,0,1)
|
||
[ -600.00 -500.00 -950.00] [-600.00 -950.00 -950.00] ← box face corners
|
||
```
|
||
|
||
The anchor scan located a **different buffer that happens to share the vertex and
|
||
index counts**, so every size-based check it makes passes. This is exactly the
|
||
"declined only for *location*, not format" risk the module notes describe — except
|
||
here it does not decline, it succeeds wrongly.
|
||
|
||
### The detector: cross-container bounds consistency
|
||
|
||
A resource shared by several containers must decode to the same bounds. That
|
||
needs no ground truth, and it measures the problem:
|
||
|
||
- **681 resources appear in ≥2 containers.**
|
||
- **125 of them decode to different bounds while reporting identical vertex and
|
||
triangle counts** — a lower bound on silent mis-decodes (a resource wrong in
|
||
*every* container is invisible to this test).
|
||
|
||
Examples: `_rou_f401` decodes as `62×25×10` in 16 containers and `4738×3147×4738`
|
||
in 2; `_rou_e011_wep05` produces **four** different spans across 8 containers.
|
||
|
||
### Repair candidate, and its limits
|
||
|
||
Taking the **majority span** across containers resolves **104 of the 125**; 14 are
|
||
exact 50/50 splits that a vote cannot decide. It agrees with the ground truth in
|
||
the one case that has independent evidence — `e303_wep_01`, where the 11-container
|
||
majority is the turret the render and the runtime capture both support.
|
||
|
||
🟡 **It is a heuristic and is otherwise unvalidated.** For
|
||
`_rou_e302_base_break` the majority is the *larger* span (`685×1206×1444`, 8 of
|
||
15) and nothing yet says which is right. Use the detector to flag; do not silently
|
||
rewrite geometry on a vote.
|
||
|
||
### ROOT CAUSE: the candidate list is container-global
|
||
|
||
Traced 2026-08-12. `anchor_pool_mesh` (the **per-block** path, which is the one
|
||
that handles single-sub-mesh resources like `e303_wep_01`) walks a candidate list
|
||
built by `vertex_run_starts(bytes, data_base, stride)` — **one scan of the whole
|
||
container per stride**, shared by every resource of that stride. It accepts the
|
||
**first** candidate that validates.
|
||
|
||
So a resource is anchored to *whatever block matches its signature first in file
|
||
order*, and nothing ties that block to the resource it belongs to. Two resources
|
||
sharing `(stride, vertex count, index count)` are interchangeable to this search.
|
||
|
||
**The wrong block is not distinguishable by quality.** Tracing the accept for
|
||
`e303_wep_01`:
|
||
|
||
```
|
||
Stage_S01 ACCEPT vb=4600480 pad=0 span= 49 × 23 × 42 passes 0.85 = true
|
||
Stage_S02 ACCEPT vb=18403456 pad=0 span=1600 × 2100 × 4800 passes 0.85 = true
|
||
```
|
||
|
||
Both clear the strict winding-consistency gate, because the wrongly-taken block
|
||
**is** real, coherent geometry — just another resource's. That rules out a whole
|
||
family of fixes: no threshold, no scoring, no "pick the best candidate" changes
|
||
this, and the earlier attempt to add best-of-N selection in the grouped-pool
|
||
anchor duly changed nothing.
|
||
|
||
**The search space has to be constrained instead — and the fix is now pinned
|
||
down.**
|
||
|
||
**The correct block is already in the candidate list.** Enumerating *every*
|
||
validating candidate for `e303_wep_01` in `Stage_S02` gives exactly two:
|
||
|
||
```
|
||
vb = 18 403 456 span 1600 × 2100 × 4800 ← what the decoder takes, only because it is first
|
||
vb = 52 257 440 span 49 × 23 × 42 ← correct: the same size all 11 good containers give
|
||
```
|
||
|
||
So nothing needs to be found that the scan is missing; the wrong one merely
|
||
appears earlier in file order.
|
||
|
||
**Locality picks the right one.** Recording each resource's accepted anchor in
|
||
descriptor order shows that global **monotonicity is refuted** — only 25 of 47
|
||
steps increase in `Stage_S01` and 130 of 248 in `Stage_S02`, i.e. no better than
|
||
chance. But *neighbourhood* holds strongly: in `Stage_S02` this resource's
|
||
descriptor neighbours anchor at **51 974 668** and **52 218 424**, its correct
|
||
candidate is **52 257 440**, and the block it wrongly takes is at **18 403 456** —
|
||
two thirds of the file away from its own family.
|
||
|
||
**Proposed rule:** among candidates that validate, prefer the one **nearest the
|
||
anchors of the neighbouring resources** (equivalently: decode in descriptor order
|
||
and prefer candidates close to the previous resource's anchor), falling back to
|
||
first-match when there is no neighbour yet. That needs no new format knowledge,
|
||
and it selects `52 257 440` here.
|
||
|
||
### ⚠️ WITHDRAWN (2026-08-12) — it halved inconsistency but regressed the twin mirror
|
||
|
||
`anchor_pool_mesh_near` tries candidates in order of distance from a reference,
|
||
and `anchor_models_filtered` runs **two passes**: pass 1 anchors first-match to
|
||
learn where resources land, then pass 2 re-anchors each resource preferring its
|
||
**neighbourhood** — the median anchor of its ±2 descriptor neighbours. A resource
|
||
with too few anchored neighbours keeps pass 1's result, so nothing regresses to
|
||
guesswork.
|
||
|
||
| | decoded | shared | inconsistent | e106 mirror |
|
||
|---|---|---|---|---|
|
||
| shipped (today) | 5 480 / 6 294 | 681 | **125** | ✅ matches capture |
|
||
| neighbourhood anchor | 5 480 / 6 294 | 681 | 63 | ❌ flipped |
|
||
| + refining the map | 5 480 / 6 294 | 681 | 51 | ❌ flipped |
|
||
|
||
**Refining matters** because pass 1's anchor map contains the very mistakes the
|
||
neighbourhood is meant to correct, so a resource beside a mis-anchored neighbour
|
||
inherits a bad reference. Re-anchoring against the improving map and repeating
|
||
converges quickly — two rounds, with a third changing nothing.
|
||
|
||
On its own metric this looked complete: coverage unchanged, inconsistency
|
||
halved, `e303_wep_01` decoding to 49 × 23 × 42 in *all* containers, and `e106`
|
||
rendering as a destroyer instead of a slab
|
||
([before](../captures/e106-static-assembly-volume-bug.png) ·
|
||
[after](../captures/e106-static-assembly-fixed.png)).
|
||
|
||
**It was reverted anyway.** `ship::tests::static_assembly_matches_runtime_capture`
|
||
is gated on `SYLPHEED_ISO` and therefore skips in an ordinary `cargo test`; run
|
||
with the ISO it fails:
|
||
|
||
```
|
||
e106_bdy_01: static M row0 [-1.0, 0.0, 0.0] != captured [1.0, 0.0, 0.0]
|
||
```
|
||
|
||
**Why:** `e106_bdy_01` and `e106_bdy_02` are a mirrored pair whose two vertex
|
||
buffers hold the same geometry reflected in X, and **both resources currently
|
||
decode to the *same* buffer** — identical vertex count, identical span, identical
|
||
`mean_x`. `apply_twin_mirrors` decides which instance to reflect from the sign of
|
||
that `mean_x`, so which of the two buffers gets picked flips the decision:
|
||
|
||
```
|
||
before the change both twins decode with mean_x = −66.83 → mirror bdy_02 (matches the capture)
|
||
after the change both twins decode with mean_x = +66.83 → mirror bdy_01 (contradicts it)
|
||
```
|
||
|
||
The runtime capture is ground truth, so a change that contradicts it does not
|
||
ship.
|
||
|
||
**Correction (measured after the fact):** the first write-up of this said "two
|
||
distinct resources sharing one decode is itself the bug". **That is wrong.**
|
||
Sharing is normal here — 1 043 of 5 480 decoded resources (19 %) share geometry
|
||
with another resource, and of 1 242 related pairs, **1 241 are identical in every
|
||
container they co-occur in**, which is what legitimate asset reuse looks like.
|
||
A mirrored pair like `bdy_01`/`bdy_02` is *supposed* to share one geometry, with
|
||
the reflection applied at placement — exactly what `apply_twin_mirrors` does.
|
||
|
||
What actually matters is **which of two mirrored buffers is canonical**. The disc
|
||
holds both an X+ and an X− version; the engine treats one as the base, and
|
||
`apply_twin_mirrors` was tuned against that. The neighbourhood anchor moved these
|
||
resources to the *nearer* buffer, which is the other one — hence the flip. So the
|
||
real fix is not "give each twin its own buffer" but **pin which buffer is
|
||
canonical**, with the capture as the oracle.
|
||
|
||
**Exactly one pair is provably mis-anchored by this test**: `e105_bdy_02_l` /
|
||
`e105_brg_m` share a decode in 7 of the 15 containers holding both and differ in
|
||
the rest — two names cannot be the same geometry only sometimes.
|
||
|
||
**The filtered path needed care.** `models_named` (what the viewer's ship
|
||
rendering uses) drops non-wanted resources, which would leave a filtered decode
|
||
with no neighbourhood at all — and silently keep the old behaviour. Resources are
|
||
now collected regardless of the filter, but only the asked-for ones and their ±2
|
||
neighbours are decoded in pass 1, so a filtered decode stays proportional to what
|
||
was asked for.
|
||
|
||
**51 remain**, and they cluster in `_l` (LOD) and `_dead` variants — `e001_l`,
|
||
`e010_bdy_01_l`, `e011_bdy_01_l`, `e016_l`, `e104_bdy_05_l`, `e106_eng_02_l`,
|
||
`e501_01_l`, `_rou_f301_base_dead`, `_rou_f302_base_dead`, `e303_base_dead`.
|
||
|
||
**A tempting explanation, tested and false.** The obvious reading is that a
|
||
variant shares its base's vertex and index counts, so the two are mutually
|
||
confusable *and* adjacent, defeating locality. Checked across every container:
|
||
**2 714 variant/base pairs, and exactly zero share identical counts.**
|
||
|
||
**What is actually happening: one region is a universal false positive.**
|
||
`e010_bdy_01_l` is 171 verts / 90 tris and **no other resource in its container
|
||
shares those counts** — yet in `Stage_S02` and `S26` it decodes to
|
||
**1600 × 2100 × 4800**, the *same* bounds `e303_wep_01` (172 verts / 110 tris)
|
||
produced before the fix. Differently-shaped resources are landing on the same
|
||
place. So the attractor is not "another mesh with my shape" but a region of
|
||
**round, axis-aligned box data** that validates for many different `(vtx, idx)`
|
||
shapes at once — every index lands in range and the triangles are coherent boxes.
|
||
|
||
That also explains why the neighbourhood fix helped so broadly: it steers
|
||
resources away from a single strong attractor rather than resolving many
|
||
pairwise confusions.
|
||
|
||
**Selection-based fixes are exhausted — tested.** The proposed tiebreak was
|
||
implemented as a *last resort* (accept the attractor only if nothing else
|
||
validates), first keyed on "all coordinates multiples of 50" and then on the
|
||
sharper **"all coordinates integral"** — the attractor reads `(42, −900, 2400)`,
|
||
`(−600, −500, −950)` while real geometry carries fractions like
|
||
`(24.55, 9.84, 4.46)`. **Neither changed anything: still 51.**
|
||
|
||
That null result is itself the answer. A mechanism that defers the attractor
|
||
whenever another candidate exists, changing nothing, means **no alternative
|
||
candidate validates for any of the 51** — the correct block is *not in the
|
||
candidate list at all*. Both attempts were reverted rather than kept.
|
||
|
||
So the residual is **not** a selection problem, and no reordering, scoring or
|
||
tiebreak will move it. The frontier is `vertex_run_starts` — the unit-normal
|
||
run scan that builds the candidate list — which does not emit a start for these
|
||
resources' real vertex buffers. That is where the remaining 51 live.
|
||
|
||
The ignored test
|
||
[`mesh_consistency_disc.rs`](../../crates/sylpheed-formats/tests/mesh_consistency_disc.rs)
|
||
still asserts the target state and now records 63 rather than 125; the remaining
|
||
cases are where the neighbourhood is itself wrong or absent.
|
||
|
||
### Where the mis-decode is *not*: the grouped-pool anchor
|
||
|
||
An attempt to fix it by making `anchor_grouped_meshes` choose the **best-scoring**
|
||
`vb0` (rather than the first candidate clearing the 0.85 gate) changed **nothing**
|
||
— still 5 480 of 6 294 decoded and still 125 inconsistent — and instrumenting the
|
||
pivot loop shows why: for `e303_wep_01` it never runs. **The resource has a single
|
||
sub-mesh, so it is decoded by the per-block adjacency path
|
||
(`anchor_pool_mesh`), not the grouped-pool anchor.**
|
||
|
||
So the silent mis-decode lives in the **per-block** anchor. That is worth knowing
|
||
before anyone else spends time on the grouped-pool pivot, which is the more
|
||
prominent and better-documented of the two and the natural first suspect.
|
||
|
||
The change was reverted: it was untargeted, unproven, and added a scoring path
|
||
with no demonstrated benefit. (Its one reusable idea — that several `vb0`
|
||
candidates can clear the gate and first-in-scan-order is an arbitrary tiebreak —
|
||
still applies to whichever anchor turns out to be at fault.)
|
||
|
||
## Two follow-ups on the anchor (2026-08-12)
|
||
|
||
**The descriptor does not address the geometry.** If it did, the whole
|
||
candidate-scan could be replaced by direct addressing. It cannot: across `e106`'s
|
||
resources in `Stage_S01`, `anchored_vb − entry.data_offset` ranges from
|
||
**1 199 052 to 4 173 988** with no constant or stride. `data_offset` locates the
|
||
*descriptor*, and nothing in the first six descriptor words tracks the vertex
|
||
pool. The scan is necessary.
|
||
|
||
**Cross-container agreement does not prove legitimate reuse.** The earlier
|
||
measurement — 1 241 of 1 242 related pairs identical in every container — was
|
||
read as "sharing is normal". It is weaker than that: a **systematic** error is
|
||
invisible to a consistency test, because it is consistent. The same dump shows
|
||
`e106_bdy_02` and `e106_bdy_03_m` anchoring to the identical offset
|
||
(`1 505 556`), and `e106_bdy_01_l` with `e106_brg_01_m` (`4 251 208`) — a hull
|
||
half and a *different* body's medium LOD, or a hull half and a bridge LOD. Those
|
||
are different parts; one of each pair must be wrong.
|
||
|
||
So the honest position is: **sharing is common (19 %), some of it is certainly
|
||
legitimate (a mirrored twin pair genuinely shares one geometry), and some is
|
||
certainly not** — and cross-container consistency cannot tell them apart. A test
|
||
that can: compare a shared pair against a runtime capture, which is ground truth
|
||
for what the engine actually draws.
|