Tested the tempting explanation for the last 51 -- that _l/_dead variants share their base's vertex and index counts, making them mutually confusable and adjacent so locality cannot help. It is FALSE: 2714 variant/base pairs across every container, and exactly zero share identical counts. What is actually happening is better news. e010_bdy_01_l is 171 verts / 90 tris and NO other resource in its container shares those counts, yet in Stage_S02 and S26 it decodes to 1600x2100x4800 -- the same bounds e303_wep_01 (172/110) produced before the fix. Differently-shaped resources land on the same place, so the attractor is a region of round axis-aligned box data that validates for many (vtx, idx) shapes at once, not "another mesh with my shape". That also explains why the neighbourhood fix helped so broadly: it steers resources away from one strong attractor rather than resolving many pairwise confusions. Candidate next step recorded with its risk: de-prefer candidates whose sampled positions are all multiples of 50 with a large span. It must be a TIEBREAK between validating candidates rather than a rejection -- an earlier sweep for that signature also flagged legitimate e_rou_* composite proxies -- and it needs the same gate (decoded must not fall from 5480). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
480 lines
30 KiB
Markdown
480 lines
30 KiB
Markdown
# XBG7 — mesh geometry (inside XPR2 model containers)
|
||
|
||
- **Confidence:** 🟡 `PROBABLE` for the single-stream layout (below); ❔ `HYPOTHESIS` / undecoded
|
||
for the complex multi-stream body layout.
|
||
- **Parser in:** `sylpheed-formats/src/mesh.rs` (`Xbg7Model::from_xpr2`), tests
|
||
`tests/mesh_disc.rs`. Container parsing reused from `src/texture.rs` (`Xpr2Header` /
|
||
`Xpr2ResourceEntry`).
|
||
- **Applies to:** ship / weapon / prop models in `hidden/resource3d/*.xpr` (166 files).
|
||
- **Method:** clean-room — static hex inspection of the retail disc + geometric validation of the
|
||
recovered triangles (non-degenerate area, indices in range, bbox matches the descriptor's stored
|
||
size). No game code decompiled or copied.
|
||
|
||
## Where XBG7 lives
|
||
|
||
Models are ordinary **`XPR2`** containers (see the XPR2 texture doc / `texture.rs`). The 16-byte
|
||
resource-directory entries (from file offset `0x10`) carry `TX2D` texture resources **and** one or
|
||
more `XBG7` geometry resources:
|
||
|
||
```
|
||
entry = [ tag:4 ][ data_offset:u32 ][ descriptor_size:u32 ][ name_offset:u32 ] (big-endian)
|
||
```
|
||
|
||
Offsets are relative to the directory base `0x10`. The `XBG7` *descriptor* (at `data_offset+0x10`,
|
||
`descriptor_size` bytes) is a **scene / material / node graph** — it holds node names
|
||
(`rou_f001_mnt1_root`, `Light`, …), a bounding value (`0x41F00000` = 30.0 ≈ the ship's ~30-unit
|
||
length), material names matching the `TX2D` channels (`_col` albedo, `_spc` specular, `_gls` gloss,
|
||
`_lum` luminance), and per-sub-mesh records. The **vertex / index buffers** live in the container's
|
||
shared **data section** (from `header_size`).
|
||
|
||
## Sub-mesh records (in the descriptor)
|
||
|
||
Read in file order by a sliding 4-byte scan; each is a big-endian tuple:
|
||
|
||
```
|
||
[ vtx_count:u32 ][ 0:u32 ][ idx_count:u32 ][ tail:u32 ]
|
||
3..=65535 ==0 mult. of 3 1..=64
|
||
```
|
||
|
||
(For `rou_f001_wep_00`: `vtx_count=215`, `idx_count=1092` — matches the recovered geometry exactly.)
|
||
|
||
## The single-stream data layout — 🟡 PROBABLE (decoded, GPU-cross-checked)
|
||
|
||
For 36 of the 166 models (weapons, simple props) the data section is a straight sequence of
|
||
sub-meshes, carved from `header_size` in record order:
|
||
|
||
```
|
||
per sub-mesh block:
|
||
[ 12-byte header (contents undecoded) ]
|
||
[ index buffer : idx_count × u16 BE ] triangle list (prim=4, GPU-confirmed)
|
||
[ vertex buffer : vtx_count × stride bytes ] ← declaration-driven
|
||
(pad to 16 bytes → next sub-mesh)
|
||
```
|
||
|
||
The 12-byte header precedes the **index** buffer (the same block shape as stage
|
||
resources — see below); the vertex buffer follows the indices with no further
|
||
gap. (Earlier this was mis-modelled as `[index][12-byte gap][vertex]`, which put
|
||
the vertex buffer at the identical offset but read the index buffer 12 bytes too
|
||
early — turning the 12 header bytes into 6 junk indices = **2 leading degenerate
|
||
triangles** (a stray-triangle artifact) and dropping the last 6 real indices.
|
||
Skipping the header fixes the triangle list with no change to vertex coverage.)
|
||
|
||
**Vertex declaration.** The layout is **not fixed-stride**. The descriptor holds a declaration table
|
||
(right after the `(index_bytes, index_count)` marker) of `{offset:u32, format-code:u32, usage<<16:u32}`
|
||
big-endian triples, terminated by `offset == 0x00FF0000` / `code == 0xFFFFFFFF`:
|
||
|
||
| usage | element | format code | format | size |
|
||
|-------|----------|-------------|--------|------|
|
||
| 0x00 | POSITION | `0x2A23B9` | f32×3 | 12 B |
|
||
| 0x03 | NORMAL | `0x1A2360` | f16×4 (use xyz) | 8 B |
|
||
| 0x05 | TEXCOORD | `0x2C235F` | f16×2 (u,v) | 4 B |
|
||
|
||
Stride = max element extent. Models **omit elements** → variable stride (20 = pos+normal, no UV;
|
||
24 = pos+normal+uv). Each element is read in **naive big-endian component order** (the raw file
|
||
bytes; see the endianness note below). Assuming a fixed stride-24 was why the old decoder mis-aligned
|
||
and declined the pos+normal-only models.
|
||
|
||
**Alignment pinned by the normals.** The vertex buffer starts at `index_end + 12` (a fixed 12-byte
|
||
header — NOT `align16`, which lands 4 bytes early on most models). The correct offset is the unique
|
||
one where recovered normals are exactly unit-length.
|
||
|
||
**Safety gate:** every index is validated `< vtx_count`, and when the declaration has a normal
|
||
element the mean recovered `|normal|` must be ≈1 (`[0.5, 2.0]`). A model failing either is rejected
|
||
(`MeshError::UnsupportedLayout`) rather than emitting garbage.
|
||
|
||
### Endianness — file bytes are naive-BE, `k8in32` is a red herring ✅
|
||
|
||
The Canary GPU capture reports every vertex stream with fetch `endian = 2` (`k8in32`, each 32-bit
|
||
word byte-reversed). This describes the **guest-memory** copy the GPU fetches — **not** the `.xpr`
|
||
file bytes. Reading the file with a `k8in32` transform breaks the normals (mean `|normal|` → 1.33);
|
||
the plain per-element big-endian read yields exactly unit normals. So the game **rearranges** the
|
||
vertex data between the on-disc `.xpr` and the uploaded buffer; the decoder reads the file directly
|
||
and must use naive BE.
|
||
|
||
## Stage containers — multi-resource, grouped pools ✅ (content-anchored)
|
||
|
||
`hidden/resource3d/Stage_S*.xpr` are not single models but **collections of
|
||
enemy / prop sub-models** — up to ~400 `XBG7` resources each (e.g. `Stage_S07` =
|
||
378). Their data layout differs from the weapon files:
|
||
|
||
- Each resource is a block `[12-byte header][index buffer][vertex buffer]`.
|
||
Unlike weapons there is **no 12-byte gap between index and vertex** — the
|
||
vertex buffer directly follows the indices.
|
||
- The **index count** is the descriptor's `(index_bytes, index_count)` marker
|
||
(the *total* for the resource — a resource may have several sub-meshes summing
|
||
to it), **not** the first sub-mesh record. The **vertex count** is a `u32`
|
||
stored **32 bytes before** the marker.
|
||
- The blocks are **scattered through the data section, interleaved with the
|
||
container's texture data**, in an allocation order that is **not** directory
|
||
order and is **not stored** in any descriptor field we could find (the
|
||
descriptor holds sizes — `rel 160 ≈ index_bytes+3`, `rel 164 =
|
||
0x1000_0000 | (vertex_bytes+2)` — but no data offset). Reconstructing that
|
||
allocation order is unsolved.
|
||
|
||
Because the offset is not stored, each resource's block is located by
|
||
**content**: parser `Xbg7Model::stage_models` does one `O(file)` pass per
|
||
distinct stride to find every **vertex-buffer start** (an offset whose NORMAL —
|
||
`f16×4` at vertex `+12` — is unit length while the *previous* stride slot's is
|
||
not, i.e. a run boundary; ~one candidate per block, not millions), then pins each
|
||
resource to the unique candidate where the `index_count` indices ending just
|
||
before it are all `< vertex_count`, reference nearly all vertices, and yield
|
||
non-degenerate triangles with real extent. This is fast (≤ ~2 s on a 70 MB
|
||
stage) and unambiguous (no two resources collide). Blocks that fail validation —
|
||
the few quantized hero bodies — are **skipped**, never emitted as garbage.
|
||
|
||
**Coverage:** 5662 sub-models decode across the 22 stage files (e.g. `Stage_S07`
|
||
366/378, `Stage_S10` 7/9 — including the main enemy bodies `e003`/`e005`, their
|
||
LODs, weapons, and props). The viewer (`spawn_stage_models`) lays the decoded
|
||
sub-models out as a side-by-side "cast sheet", skipping the few huge skybox-plane
|
||
resources (300 k-unit quads). Cross-checked: `e003` = 2383 v / 1436 t bbox
|
||
23.5×9.5×32.5; `e005` = 2566 v / 1507 t; both 0 degenerate.
|
||
|
||
## Not yet decoded — ❔ the complex body layout
|
||
|
||
The hero-ship *body* meshes (`DeltaSaber_A/_T/_W.xpr` resource `f004`, and ~100 other models) still
|
||
decline. Two open sub-problems: (1) **multi-sub-mesh models** whose *first* sub-mesh decodes but a
|
||
later one's inter-mesh offset isn't yet handled (the `align16` advance is a guess) — these are
|
||
declined whole; (2) the big body meshes, where the data section does not start with an index buffer
|
||
and the geometry sits at descriptor-addressed offsets. NB the GPU capture showed **every** rendered
|
||
mesh is *single-stream* (just wider strides, e.g. 44 bytes = pos + f32×3 + colour + 2×f16×4), so the
|
||
body is likely single-stream-with-a-richer-declaration rather than the "separate streams" first
|
||
guessed — it was simply not rendered in the captured session (menu only). A capture taken *in a
|
||
mission* (where `DeltaSaber` renders) would hand over its exact declaration directly. `DeltaSaber_A`'s
|
||
5 XBG7 blocks are `f004` (body) + `_rou_f004_mnv01_L/_R`, `_mnv02`, `_turn180` (maneuver / pose).
|
||
|
||
## Evidence log
|
||
|
||
- 2026-07-12 — `rou_f001_wep_00.xpr`: XPR2 dir = 1×XBG7 (`f001_wep_00`) + 3×TX2D
|
||
(`_col/_gls/_spc`). Data section starts with a u16-BE index run (max 214), then stride-24
|
||
vertices. Descriptor record `[215,0,1092,4]` at desc `+0x2B0`; index-buffer byte size `0x888`
|
||
(=2184=1092×2) at desc `+0x1A0`. Recovered mesh = 215 v / 364 t, 362 non-degenerate, median tri
|
||
area 0.015 — coherent. → single-stream layout **PROBABLE**.
|
||
- 2026-07-12 — descriptor-driven sequential carving over all 166 models: **42 carve** under the
|
||
index-only check. Real 3D extent confirmed on `rou_f001_wep_04` (bbox 3.7×1.2×3.5).
|
||
- 2026-07-12 (refine) — attribute ranges differed per model (`wep_00` UV≈attr0/2, `wep_04`
|
||
attr4/5, `wep_03` attr1≈±60000 = garbage) → **refuted the fixed "6 half attrs, UV=attr0/2"
|
||
reading.** Found the descriptor **vertex declaration** (offsets 0x0C/0x14, usages 0x03 NORMAL /
|
||
0x05 TEXCOORD). Solved the vertex-base offset with a **unit-normal validator**: `index_end + 12`
|
||
gives median `|normal| = 1.000` on every weapon model (`wep_00/02/03/04`), vs `align16` landing 4
|
||
bytes early. UVs then land in `[0,1]`. Adding the normal gate: **25 models decode clean +
|
||
normal-valid** (the rest — incl. `Stage_S*` degenerate blobs — correctly declined).
|
||
- 2026-07-12 (DYNAMIC) — added a cvar-gated draw logger to Canary
|
||
(`command_processor.cc::LogDrawForRE`, cvar `log_draws`), captured the Ready Room / Briefings.
|
||
**GPU ground truth confirmed the static layout exactly**: primitive `prim=4` = **triangle LIST**
|
||
(settles the list-vs-strip question), and a stream with `f32x3 @offset0` + `f16x4 @3dw` +
|
||
`f16x2 @5dw`, stride 6 dwords = 24 bytes — matching `POSITION@0, NORMAL@0x0C, TEXCOORD@0x14`.
|
||
Revealed **stride varies** (24, 20, 28, 44 …) and that all streams are **single-stream** →
|
||
parsed the declaration for variable stride: coverage **25 → 36** (e.g. `wep_05` is pos+normal,
|
||
stride 20, no UV — previously mis-aligned). Also confirmed the endianness note above: fetch
|
||
`endian=2` (k8in32) is the *guest* copy; file stays naive-BE. `Stage_S*` now decode (stride 20).
|
||
- 2026-07-12 (STAGE) — `Stage_S*.xpr` decoded as multi-resource containers. Found each geometry
|
||
block is `[12B hdr][index buffer][vertex buffer]`, index count = descriptor marker (total, e.g.
|
||
`e003` = 4308 spanning 2 sub-meshes), vertex count = `u32` at marker−32 (`e003` = 2383 → verified
|
||
by max-index 2382 and 0 degenerate tris, bbox 23.5×9.5×32.5). Blocks are scattered among texture
|
||
data with **no stored offset** (descriptor rel 160 = idx_bytes+3, rel 164 = `0x1000_0000 |
|
||
(vtx_bytes+2)` are sizes, not offsets; block starts e.g. `e003`@0x10230, `e003_l`@0x5d000,
|
||
`e005_l`@0x9b000 are not directory-ordered). Solved by **content anchoring**: a single per-stride
|
||
pass finds vertex-run starts (unit NORMAL at +12 whose previous slot isn't), then match each
|
||
resource by strict index+triangle validation. **5662 sub-models decode across 22 stages** (S07
|
||
366/378), incl. the previously-declined main bodies `e005` (2566 v) and weapons — ≤2 s on 70 MB.
|
||
Parser `Xbg7Model::stage_models`, tests `stage_models_{decode,sweep,quality_audit}`.
|
||
- 2026-07-17 — **triangle-LIST re-confirmed; a strip interlude refuted; winding-consistency gate
|
||
added.** A 2026-07 change had briefly re-read the index buffers as triangle *strips* (to "fill
|
||
holes"). Refuted objectively with a new `XVERIFY` diagnostic that compares both readings by
|
||
**stored-normal agreement** (each triangle's cross-product face normal vs the sum of its vertices'
|
||
stored normals): the LIST reading gives agreement **1.000** on every clean weapon (`wep_00/03/04/19`
|
||
= only possible with correct topology + winding), the STRIP reading **~0.49** (random). The strip
|
||
reading also over-generated ~2.5× the triangles (wep_00: 938 vs 364) — a hole-filling garbage soup.
|
||
Reverted to LIST in both paths (`from_xpr2`, `read_pool_mesh`), matching the `prim=4` GPU capture.
|
||
Added an objective **winding-consistency gate** `max(na, 1-na)`: a correct carve is internally
|
||
consistent (agreement ≈1.0, or ≈0.0 for inverted-but-consistent winding — a real single-sided
|
||
mesh), a mis-carve scatters to the ≈0.5 middle. `from_xpr2` declines sub-meshes below 0.90 (e.g.
|
||
`wep_23` na=0.398 → declined instead of a spike-mess); the single-model **content-anchor fallback**
|
||
gates at 0.85; the large multi-resource **stage** path stays ungated (its enemy meshes span a
|
||
continuous 0.5–1.0 consistency range — a hard gate there dropped ~48/314 legit S07 blocks).
|
||
**Routing fixed:** `decode_models` (CLI) and the viewer now route by `count_xbg7` (1 → validated
|
||
records-based list decode, fallback to strict-gated anchor; >1 → stage anchor) instead of the old
|
||
"whichever decoder yields more verts" rule — that rule let stage content-anchoring win on
|
||
single-model weapon files and fabricate **phantom** blocks (a `wep_00` clone appearing inside
|
||
`wep_19`), duplicates, and spike-mess anchors. Weapons now: 33 clean-decode / 26 declined (declined
|
||
= genuinely multi-stream or un-carvable, shown as nothing rather than garbage); stage coverage
|
||
unchanged (S07 314). `expand_triangle_strip` retained as an `XVERIFY`-only diagnostic.
|
||
- 2026-07-12 — `DeltaSaber_A.xpr` body: data does **not** begin with indices; plain-`f32×3` runs
|
||
with ship-scale extent (span ≈27–34, matching bbox 30.0) found only at high offsets
|
||
(`data+0x28634C`, …) → multi-stream, **undecoded**.
|
||
- 2026-07-18 — **GROUPED-POOL layout cracked → the hero ship (Delta Saber) fully decodes.** The
|
||
detailed models (`DeltaSaber_*.xpr` + ~100 others) were declined for **location**, not format —
|
||
their vertex format is the standard stride-24 triangle list. A resource's *several* sub-meshes
|
||
don't interleave `[idx][vtx]` per block; they share **two grouped pools**: an **index pool**
|
||
(buffers concatenated in descriptor-marker order, each **4-byte aligned**) followed by a **vertex
|
||
pool** (each sub-pool `vtx_count × stride`, same order), with the index pool ending **exactly**
|
||
where the vertex pool begins. So the whole resource pivots on one unknown, the first vertex-pool
|
||
start `vb0` (= index-pool end, found by the unit-normal vertex-run scan); everything else is
|
||
derived: `ib0 = vb0 − span`, `ib[i] = align4(ib[i-1] + idx_count[i-1]·2)`,
|
||
`vb[i] = vb[i-1] + vtx_count[i-1]·stride`. Reversed statically from `DeltaSaber_T.xpr` and
|
||
**cross-checked against a Canary GPU draw-log capture** (mission ship = `DeltaSaber_T.xpr`, found
|
||
via the `--log_file_io` kernel hook): `f001` = body (idx@`data+0xC` = 0x5500C, vtx@0x61ACC, 10891 v
|
||
/ 8187 t) + **7 detail parts** (fins/cockpit/wingtips, markers at descriptor 0x3BEC…0x58FC) =
|
||
**8650 tris**, and **every sub-mesh decodes at 0 degenerate / full coverage / winding-agreement
|
||
1.000**. This is the layout the per-block adjacency anchor (`ib = vb − idx_bytes`) rendered as a
|
||
**spiky phantom** (it read 24561 indices starting 2782 B too late, agree 0.64, 1277 degenerate).
|
||
Insight: a single index marker reduces the grouped model to `index_end = vb0`, i.e. the existing
|
||
adjacency `ib = vb − idx_bytes` — so grouped **generalises** the single-block anchor (n=1 is
|
||
identical). Implemented as `anchor_grouped_meshes` (mesh.rs): `anchor_models` routes resources with
|
||
>1 index marker to it (validated per sub-mesh; on failure falls back to the old first-marker
|
||
adjacency anchor so stage coverage never regresses); single-marker stages/props keep the exact
|
||
prior path. The shared acceptance test is factored into `validate_block` (the connectivity
|
||
heuristic is relaxed for *derived* grouped parts, which are pinned by in-range + consistency, so
|
||
small flat fins aren't mis-rejected). Render self-check: `sylpheed-cli mesh render DeltaSaber_T.xpr
|
||
--only f001` (exact-name match excludes the `_rou_f001_mnv*` animation poses) → clean complete
|
||
fighter. Test `hero_ship_grouped_pool_decodes`. Colours/UVs still pending the running-game oracle.
|
||
- 2026-07-18 (refinement) — **4-byte vertex-pool alignment + weapon recovery.** The grouped-pool
|
||
rule "index pool ends exactly where the vertex pool begins" is really "the vertex pool is **4-byte
|
||
aligned** after the index pool": `vb0 = align4(ib0 + span)`, so 0..=3 bytes of padding can sit
|
||
between them. DeltaSaber's index pool ended already-aligned (pad 0), which hid this; **19
|
||
weapon/`*_hangar` models** (single- and multi-marker: `wep_08/11/34/58/62/69/81/83…`) have pad 2
|
||
and so decoded to *nothing* — the viewer then showed them as a flat 2D texture instead of a model.
|
||
Fix: both anchors try `pad ∈ 0..=3` (`ib = vb − idx_bytes − pad` for the single-block adjacency
|
||
anchor; `ib0 = vb0 − span − pad` for the grouped pivot), validated — a wrong pad reads shifted
|
||
indices → agreement collapses < 0.85, so only the true pad passes. pad>0 in the ungated stage path
|
||
is gated at a strict 0.85 to avoid a false anchor; pad 0 keeps its exact prior behaviour (stages
|
||
unchanged). Result: all 19 now decode as clean models (e.g. `wep_34` 1243 v / 1233 t, a
|
||
long-barrelled gun-pod; `wep_08` 3 sub-meshes / 478 t). Viewer routing already falls through
|
||
`from_xpr2` → `anchor_models(0.85)` for single-XBG7 files, so the recovered grouped/padded weapons
|
||
now preview as meshes.
|
||
- 2026-07-18 (refinement 2) — **pivot on the largest sub-mesh; all 19 recovered.** Three weapons
|
||
(`wep_81`, `wep_81_hangar`, `wep_30_hangar`) still declined because the grouped pivot validated
|
||
`markers[0]`, which for these is a tiny *elongated* lead bracket that fails the connectivity gate
|
||
even when perfectly placed. Fixed by pivoting the alignment check on the **largest** marker (max
|
||
index count) — the sub-mesh whose triangle-quality/connectivity signature most reliably confirms
|
||
`(ib0, vb0)`. Once the pivot validates, markers up to it are read unconditionally (a legitimately
|
||
tiny/flat lead part may fail the quality gates yet still be real), and markers after it stay
|
||
validated so a stray trailing marker ends the chain. Result: **all 19 previously-declined weapons
|
||
decode** (`wep_81` 460 t missile w/ tail fins; `wep_30_hangar` 334 t). DeltaSaber unchanged (its
|
||
body IS the largest marker → same pivot). 7/7 disc tests green, stage quality audit unchanged.
|
||
|
||
## The declined set, measured (2026-08-11)
|
||
|
||
The module note said "a few multi-stream / quantized bodies remain" declined.
|
||
Measured across all 166 `hidden/resource3d/*.xpr`:
|
||
|
||
- **6 294 XBG7 resources, 5 480 decoded (87.1 %), 814 declined**, in **31 of 166**
|
||
containers. Worst: `Stage_S09` 64/380, `Stage_S06` 58/324, `ptc_pack` 57/136.
|
||
- The declined set is **not** "a few hero bodies". By name prefix it is **492
|
||
`e*`** (enemy craft), **142 `f*`**, **73 `n*`**, **23 `eff*`**, plus destroyed
|
||
variants (`_rou_f402_dead`, `_rou_f302_base_dead`) and one weapon
|
||
(`_rou_e011_wep04`).
|
||
|
||
### A shortcut that does not work
|
||
|
||
The resource descriptor's third word looked like a format/stream flag — decoded
|
||
`g001…g003` carry `0x00010001` while declined `t170`/`t180` carry `0x00020004`,
|
||
which reads temptingly as `(streams << 16) | format`. **It is not that.**
|
||
Histogramming it over the whole disc puts decoded *and* declined resources at
|
||
every value:
|
||
|
||
```
|
||
word[2] decoded declined
|
||
0x00010001 4479 328
|
||
0x00010002 126 27
|
||
0x00010003 120 112
|
||
0x00010004 142 84
|
||
… … …
|
||
```
|
||
|
||
Its low half runs 1…0x52 and tracks sub-mesh count, not vertex format. **So
|
||
decodability is not declared in the descriptor** — it is a property of whether
|
||
the unit-normal anchor scan can locate `vb0`, which is exactly what the current
|
||
code already tests. Anyone attacking this should not spend time on the
|
||
descriptor: 229 of the declined resources even carry the *most* common
|
||
`0x00010001` with under 1 KB of data, i.e. they are small meshes the scan has too
|
||
little signal to anchor, not exotic formats.
|
||
|
||
## Silent mis-decodes: a detector, and how many there are (2026-08-11)
|
||
|
||
The [declined set](#the-declined-set-measured-2026-08-11) is the *honest* failure
|
||
mode — 814 resources the decoder refuses. This is the other kind: geometry that
|
||
decodes without complaint and is wrong.
|
||
|
||
### The case that exposed it
|
||
|
||
`e303_wep_01` decodes from fourteen containers. In eleven it is a **49 × 23 × 42**
|
||
turret with organic vertices (`24.55, 0.00, 4.46` …). In `Stage_S02`, `S08` and
|
||
`S26` the *same* resource — **identical 172 vertices and 330 indices** — decodes
|
||
to **1600 × 2100 × 4800** of axis-aligned box corners:
|
||
|
||
```
|
||
Stage_S01 [ 24.55 0.00 4.46] [ 24.55 9.84 2.91] normals varied
|
||
Stage_S02 [ 42.00 -900.00 2400.00] [-600.00 -500.00 -500.00] normals (0,0,1)
|
||
[ -600.00 -500.00 -950.00] [-600.00 -950.00 -950.00] ← box face corners
|
||
```
|
||
|
||
The anchor scan located a **different buffer that happens to share the vertex and
|
||
index counts**, so every size-based check it makes passes. This is exactly the
|
||
"declined only for *location*, not format" risk the module notes describe — except
|
||
here it does not decline, it succeeds wrongly.
|
||
|
||
### The detector: cross-container bounds consistency
|
||
|
||
A resource shared by several containers must decode to the same bounds. That
|
||
needs no ground truth, and it measures the problem:
|
||
|
||
- **681 resources appear in ≥2 containers.**
|
||
- **125 of them decode to different bounds while reporting identical vertex and
|
||
triangle counts** — a lower bound on silent mis-decodes (a resource wrong in
|
||
*every* container is invisible to this test).
|
||
|
||
Examples: `_rou_f401` decodes as `62×25×10` in 16 containers and `4738×3147×4738`
|
||
in 2; `_rou_e011_wep05` produces **four** different spans across 8 containers.
|
||
|
||
### Repair candidate, and its limits
|
||
|
||
Taking the **majority span** across containers resolves **104 of the 125**; 14 are
|
||
exact 50/50 splits that a vote cannot decide. It agrees with the ground truth in
|
||
the one case that has independent evidence — `e303_wep_01`, where the 11-container
|
||
majority is the turret the render and the runtime capture both support.
|
||
|
||
🟡 **It is a heuristic and is otherwise unvalidated.** For
|
||
`_rou_e302_base_break` the majority is the *larger* span (`685×1206×1444`, 8 of
|
||
15) and nothing yet says which is right. Use the detector to flag; do not silently
|
||
rewrite geometry on a vote.
|
||
|
||
### ROOT CAUSE: the candidate list is container-global
|
||
|
||
Traced 2026-08-12. `anchor_pool_mesh` (the **per-block** path, which is the one
|
||
that handles single-sub-mesh resources like `e303_wep_01`) walks a candidate list
|
||
built by `vertex_run_starts(bytes, data_base, stride)` — **one scan of the whole
|
||
container per stride**, shared by every resource of that stride. It accepts the
|
||
**first** candidate that validates.
|
||
|
||
So a resource is anchored to *whatever block matches its signature first in file
|
||
order*, and nothing ties that block to the resource it belongs to. Two resources
|
||
sharing `(stride, vertex count, index count)` are interchangeable to this search.
|
||
|
||
**The wrong block is not distinguishable by quality.** Tracing the accept for
|
||
`e303_wep_01`:
|
||
|
||
```
|
||
Stage_S01 ACCEPT vb=4600480 pad=0 span= 49 × 23 × 42 passes 0.85 = true
|
||
Stage_S02 ACCEPT vb=18403456 pad=0 span=1600 × 2100 × 4800 passes 0.85 = true
|
||
```
|
||
|
||
Both clear the strict winding-consistency gate, because the wrongly-taken block
|
||
**is** real, coherent geometry — just another resource's. That rules out a whole
|
||
family of fixes: no threshold, no scoring, no "pick the best candidate" changes
|
||
this, and the earlier attempt to add best-of-N selection in the grouped-pool
|
||
anchor duly changed nothing.
|
||
|
||
**The search space has to be constrained instead — and the fix is now pinned
|
||
down.**
|
||
|
||
**The correct block is already in the candidate list.** Enumerating *every*
|
||
validating candidate for `e303_wep_01` in `Stage_S02` gives exactly two:
|
||
|
||
```
|
||
vb = 18 403 456 span 1600 × 2100 × 4800 ← what the decoder takes, only because it is first
|
||
vb = 52 257 440 span 49 × 23 × 42 ← correct: the same size all 11 good containers give
|
||
```
|
||
|
||
So nothing needs to be found that the scan is missing; the wrong one merely
|
||
appears earlier in file order.
|
||
|
||
**Locality picks the right one.** Recording each resource's accepted anchor in
|
||
descriptor order shows that global **monotonicity is refuted** — only 25 of 47
|
||
steps increase in `Stage_S01` and 130 of 248 in `Stage_S02`, i.e. no better than
|
||
chance. But *neighbourhood* holds strongly: in `Stage_S02` this resource's
|
||
descriptor neighbours anchor at **51 974 668** and **52 218 424**, its correct
|
||
candidate is **52 257 440**, and the block it wrongly takes is at **18 403 456** —
|
||
two thirds of the file away from its own family.
|
||
|
||
**Proposed rule:** among candidates that validate, prefer the one **nearest the
|
||
anchors of the neighbouring resources** (equivalently: decode in descriptor order
|
||
and prefer candidates close to the previous resource's anchor), falling back to
|
||
first-match when there is no neighbour yet. That needs no new format knowledge,
|
||
and it selects `52 257 440` here.
|
||
|
||
### ✅ Implemented (2026-08-12) — inconsistency halved, coverage unchanged
|
||
|
||
`anchor_pool_mesh_near` tries candidates in order of distance from a reference,
|
||
and `anchor_models_filtered` runs **two passes**: pass 1 anchors first-match to
|
||
learn where resources land, then pass 2 re-anchors each resource preferring its
|
||
**neighbourhood** — the median anchor of its ±2 descriptor neighbours. A resource
|
||
with too few anchored neighbours keeps pass 1's result, so nothing regresses to
|
||
guesswork.
|
||
|
||
| | decoded | shared | inconsistent |
|
||
|---|---|---|---|
|
||
| before | 5 480 / 6 294 | 681 | **125** |
|
||
| neighbourhood anchor | 5 480 / 6 294 | 681 | **63** |
|
||
| + refining the map | 5 480 / 6 294 | 681 | **51** |
|
||
|
||
**Refining matters** because pass 1's anchor map contains the very mistakes the
|
||
neighbourhood is meant to correct, so a resource beside a mis-anchored neighbour
|
||
inherits a bad reference. Re-anchoring against the improving map and repeating
|
||
converges quickly — two rounds, with a third changing nothing.
|
||
|
||
**Coverage is unchanged and inconsistency halves.** `e303_wep_01` now decodes to
|
||
49 × 23 × 42 in *all* containers, and `e106` renders as a destroyer instead of a
|
||
slab ([before](../captures/e106-static-assembly-volume-bug.png) ·
|
||
[after](../captures/e106-static-assembly-fixed.png)) — its two shared turrets sit
|
||
symmetrically at X[−203,−154] and X[154,203].
|
||
|
||
**The filtered path needed care.** `models_named` (what the viewer's ship
|
||
rendering uses) drops non-wanted resources, which would leave a filtered decode
|
||
with no neighbourhood at all — and silently keep the old behaviour. Resources are
|
||
now collected regardless of the filter, but only the asked-for ones and their ±2
|
||
neighbours are decoded in pass 1, so a filtered decode stays proportional to what
|
||
was asked for.
|
||
|
||
**51 remain**, and they cluster in `_l` (LOD) and `_dead` variants — `e001_l`,
|
||
`e010_bdy_01_l`, `e011_bdy_01_l`, `e016_l`, `e104_bdy_05_l`, `e106_eng_02_l`,
|
||
`e501_01_l`, `_rou_f301_base_dead`, `_rou_f302_base_dead`, `e303_base_dead`.
|
||
|
||
**A tempting explanation, tested and false.** The obvious reading is that a
|
||
variant shares its base's vertex and index counts, so the two are mutually
|
||
confusable *and* adjacent, defeating locality. Checked across every container:
|
||
**2 714 variant/base pairs, and exactly zero share identical counts.**
|
||
|
||
**What is actually happening: one region is a universal false positive.**
|
||
`e010_bdy_01_l` is 171 verts / 90 tris and **no other resource in its container
|
||
shares those counts** — yet in `Stage_S02` and `S26` it decodes to
|
||
**1600 × 2100 × 4800**, the *same* bounds `e303_wep_01` (172 verts / 110 tris)
|
||
produced before the fix. Differently-shaped resources are landing on the same
|
||
place. So the attractor is not "another mesh with my shape" but a region of
|
||
**round, axis-aligned box data** that validates for many different `(vtx, idx)`
|
||
shapes at once — every index lands in range and the triangles are coherent boxes.
|
||
|
||
That also explains why the neighbourhood fix helped so broadly: it steers
|
||
resources away from a single strong attractor rather than resolving many
|
||
pairwise confusions.
|
||
|
||
🟡 **Candidate next step, with its risk stated:** reject (or de-prefer) a
|
||
candidate whose sampled positions are *all* multiples of 50 with a large span —
|
||
the attractor's signature. The risk is real: an earlier sweep for exactly that
|
||
signature also flagged legitimate `e_rou_*` composite proxies, so it must be a
|
||
**tiebreak between validating candidates**, never an outright rejection, and it
|
||
needs the same before/after gate (decoded must not fall from 5 480).
|
||
|
||
The ignored test
|
||
[`mesh_consistency_disc.rs`](../../crates/sylpheed-formats/tests/mesh_consistency_disc.rs)
|
||
still asserts the target state and now records 63 rather than 125; the remaining
|
||
cases are where the neighbourhood is itself wrong or absent.
|
||
|
||
### Where the mis-decode is *not*: the grouped-pool anchor
|
||
|
||
An attempt to fix it by making `anchor_grouped_meshes` choose the **best-scoring**
|
||
`vb0` (rather than the first candidate clearing the 0.85 gate) changed **nothing**
|
||
— still 5 480 of 6 294 decoded and still 125 inconsistent — and instrumenting the
|
||
pivot loop shows why: for `e303_wep_01` it never runs. **The resource has a single
|
||
sub-mesh, so it is decoded by the per-block adjacency path
|
||
(`anchor_pool_mesh`), not the grouped-pool anchor.**
|
||
|
||
So the silent mis-decode lives in the **per-block** anchor. That is worth knowing
|
||
before anyone else spends time on the grouped-pool pivot, which is the more
|
||
prominent and better-documented of the two and the natural first suspect.
|
||
|
||
The change was reverted: it was untargeted, unproven, and added a scoring path
|
||
with no demonstrated benefit. (Its one reusable idea — that several `vb0`
|
||
candidates can clear the gate and first-in-scan-order is an arbitrary tiebreak —
|
||
still applies to whichever anchor turns out to be at fault.)
|