The raw Canary capture logs from the 2026-07 sessions are still on disc at
/sylph-home/re/shipcap/xenia_ship_capture_*.log, and their DRAW lines carry
vbase -- the GPU vertex base, i.e. ground truth for which buffer the engine draws
a part from. That is precisely the oracle the shared-decode question needs, and
it needs no new capture run.
Restricted to the ship-geometry stride 24, across the three logs: 6093 draws from
2291 distinct vbases, and only 77 vbases (3.4%) drawn more than once. So one
buffer serving several parts is the exception in the engine, while our decoder
has 19% of resources sharing geometry.
The measures differ (draws-per-buffer vs resources-per-geometry) so this is
evidence not proof, but it points the same way as e106_bdy_02 == e106_bdy_03_m:
much of that 19% is anchoring error rather than asset reuse.
Per-part use is already available: correlate_capture matches a draw to a resource
by vcount plus decoded positions, and the same match yields each part's vbase --
so two resources our decoder gives identical geometry can be checked directly,
different vbase meaning our shared decode is wrong.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two follow-ups, both negative, both worth recording so the next attempt does not
repeat them.
The descriptor cannot replace the candidate scan. If entry.data_offset located
the geometry, anchoring would be direct addressing. Across e106's resources in
Stage_S01, anchored_vb - data_offset ranges from 1199052 to 4173988 with no
constant or stride; data_offset locates the DESCRIPTOR and none of the first six
descriptor words tracks the vertex pool.
And yesterday's "1241 of 1242 pairs identical in every container, so sharing is
legitimate reuse" is weaker than it read. A SYSTEMATIC error is invisible to a
consistency test, because it is consistent. The same dump shows e106_bdy_02 and
e106_bdy_03_m anchoring to the identical offset (1505556), and e106_bdy_01_l with
e106_brg_01_m (4251208) -- a hull half and a different body's medium LOD, a hull
half and a bridge LOD. Different parts; one of each pair is wrong.
Honest position: sharing is common (19%), some is certainly legitimate (a
mirrored twin pair genuinely shares one geometry) and some certainly is not, and
cross-container consistency cannot separate them. What can: comparing a shared
pair against a runtime capture.
Instrumentation reverted; only docs land here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
I wrote that "two distinct resources sharing one decode is itself the bug" when
withdrawing the neighbourhood anchor. Measured, that is wrong.
1043 of 5480 decoded resources (19%) share geometry with another resource, and of
1242 related pairs that share a decode somewhere, 1241 are identical in EVERY
container they co-occur in. That is what legitimate asset reuse looks like, not a
defect. A mirrored pair like bdy_01/bdy_02 is SUPPOSED to share one geometry with
the reflection applied at placement -- which is precisely what apply_twin_mirrors
does.
So the twin-mirror regression was never about sharing. It is about WHICH of two
mirrored buffers is canonical: the disc holds an X+ and an X- version, the engine
treats one as the base, and the neighbourhood anchor moved these resources to the
nearer buffer, which is the other one. The fix is to pin the canonical buffer
with the capture as oracle, not to split the twins apart.
Exactly one pair is provably mis-anchored by this test: e105_bdy_02_l /
e105_brg_m, identical in 7 of the 15 containers holding both and different in the
rest -- two names cannot be the same geometry only sometimes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The neighbourhood anchor (f18d591) and its refinement (27a0701) took
cross-container inconsistency from 125 to 51 with coverage unchanged, and made
e106 render as a destroyer rather than a slab. Both are reverted.
ship::tests::static_assembly_matches_runtime_capture is gated on SYLPHEED_ISO, so
it SKIPS in an ordinary cargo test -- which is why the regression was invisible
in every suite run so far. With the ISO it fails:
e106_bdy_01: static M row0 [-1.0, 0.0, 0.0] != captured [1.0, 0.0, 0.0]
e106_bdy_01 and _02 are a mirrored pair whose two buffers hold the same geometry
reflected in X, and BOTH resources currently decode to the SAME buffer (identical
counts, span and mean_x). apply_twin_mirrors picks which instance to reflect from
the sign of that mean_x, so which buffer wins flips the decision:
before both twins mean_x = -66.83 -> mirror bdy_02 (matches the capture)
after both twins mean_x = +66.83 -> mirror bdy_01 (contradicts it)
Neither is right -- two resources sharing one decode is itself the bug and the
mirror heuristic has been compensating. The capture is ground truth, so a change
that contradicts it does not ship. The real fix must give each twin its own
buffer first.
Kept from the attempt: this test now also asserts the SET of static placements
against the capture (allow-list {e303_wep_01} for vbase dedup), so extra
placements can finally fail it -- the direction it could never fail in before.
Docs, backlog, INDEX and the ignored test's message all corrected to say
diagnosed-not-fixed rather than fixed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Implemented the proposed tiebreak as a last resort (take the attractor only if
nothing else validates), keyed first on "all coordinates multiples of 50" and
then on the sharper "all coordinates integral" -- the attractor reads
(42, -900, 2400), (-600, -500, -950) while real geometry carries fractions like
(24.55, 9.84, 4.46). Neither changed anything: still 51 inconsistent, 5480
decoded.
The null result is the finding. A mechanism that defers the attractor whenever
another candidate exists, and changes nothing, means no alternative candidate
validates for any of the 51: the correct block is not in the candidate list at
all. So the residual is not a selection problem and no reordering, scoring or
tiebreak will move it.
The frontier is vertex_run_starts -- the unit-normal run scan that builds the
candidate list -- which emits no start for these resources' real vertex buffers.
Both attempts reverted; only docs land here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Tested the tempting explanation for the last 51 -- that _l/_dead variants share
their base's vertex and index counts, making them mutually confusable and
adjacent so locality cannot help. It is FALSE: 2714 variant/base pairs across
every container, and exactly zero share identical counts.
What is actually happening is better news. e010_bdy_01_l is 171 verts / 90 tris
and NO other resource in its container shares those counts, yet in Stage_S02 and
S26 it decodes to 1600x2100x4800 -- the same bounds e303_wep_01 (172/110)
produced before the fix. Differently-shaped resources land on the same place, so
the attractor is a region of round axis-aligned box data that validates for many
(vtx, idx) shapes at once, not "another mesh with my shape".
That also explains why the neighbourhood fix helped so broadly: it steers
resources away from one strong attractor rather than resolving many pairwise
confusions.
Candidate next step recorded with its risk: de-prefer candidates whose sampled
positions are all multiples of 50 with a large span. It must be a TIEBREAK
between validating candidates rather than a rejection -- an earlier sweep for
that signature also flagged legitimate e_rou_* composite proxies -- and it needs
the same gate (decoded must not fall from 5480).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Pass 1's anchor map contains exactly the mistakes the neighbourhood is meant to
correct, so a resource sitting beside a mis-anchored neighbour inherits a bad
reference. Re-anchoring against the improving map and repeating converges
quickly: two rounds, and a third changes nothing (the loop exits early when a
round is a fixpoint).
before decoded 5480/6294 inconsistent 125
neighbourhood anchor decoded 5480/6294 inconsistent 63
+ refining the map decoded 5480/6294 inconsistent 51
Coverage still unchanged. The 51 that remain cluster in _l (LOD) and _dead
variants -- e001_l, e010_bdy_01_l, e106_eng_02_l, _rou_f302_base_dead,
e303_base_dead and friends. A plausible reading is that a variant shares its
base's vertex and index counts, making the two mutually confusable so that
locality cannot separate them; recorded as untested rather than asserted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
anchor_pool_mesh took the FIRST candidate in file order from a container-global
vertex-run scan, so a resource could be handed another resource's block whenever
both shared (stride, vertex count, index count). Both blocks are real geometry and
both pass every quality gate, so only position separates them.
anchor_pool_mesh_near now tries candidates in order of distance from a reference,
and anchor_models_filtered runs two passes: pass 1 anchors first-match to learn
where resources land, pass 2 re-anchors each resource preferring the median anchor
of its +/-2 descriptor neighbours. Too few anchored neighbours -> keep pass 1, so
nothing regresses to guesswork.
before decoded 5480/6294 shared 681 inconsistent 125
after decoded 5480/6294 shared 681 inconsistent 63
Coverage unchanged, inconsistency halved. e303_wep_01 decodes to 49x23x42 in ALL
containers now, and e106 renders as a destroyer instead of a slab -- its two
shared turrets symmetric at X[-203,-154] and X[154,203]. That resolves the
user-reported "capital ships assemble wrong" for this cause.
The filtered path needed care: models_named (what the viewer uses) dropped
non-wanted resources, which would have left filtered decodes with no
neighbourhood and silently kept the old behaviour. Resources are now collected
regardless of the filter, but only the asked-for ones and their +/-2 neighbours
are decoded in pass 1, so a filtered decode stays proportional to what was asked.
63 cases remain; mesh_consistency_disc stays ignored and now records 63, not 125.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Enumerating every validating candidate for e303_wep_01 in Stage_S02 gives exactly
two:
vb = 18403456 span 1600 x 2100 x 4800 <- taken, only because it is first
vb = 52257440 span 49 x 23 x 42 <- correct, the size all 11 good
containers give
So the scan is not missing anything; the wrong block merely appears earlier in
file order.
Monotonic anchoring is REFUTED as the constraint: recording each resource's
accepted anchor in descriptor order gives 25/47 increasing steps in Stage_S01 and
130/248 in Stage_S02, no better than chance. Locality holds instead -- in
Stage_S02 the descriptor neighbours anchor at 51974668 and 52218424, the correct
candidate is 52257440, and the wrongly-taken block sits at 18403456, two thirds
of the file from its own family.
Proposed rule, needing no new format knowledge: among validating candidates
prefer the one nearest the neighbouring resources' anchors, falling back to
first-match when there is no neighbour yet. It selects 52257440 here.
Not implemented: it moves the anchor for all 6294 resources, so it needs the
before/after measurement (decoded must not fall from 5480, inconsistent should
fall from 125) and then un-ignoring mesh_consistency_disc.
Instrumentation reverted; only docs land here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
anchor_pool_mesh walks candidates from vertex_run_starts(bytes, data_base,
stride): ONE scan of the whole container per stride, shared by every resource of
that stride, and it accepts the first candidate that validates. Nothing ties the
chosen block to the resource it belongs to, so two resources sharing (stride,
vertex count, index count) are interchangeable to this search.
The wrong block is not distinguishable by quality. Tracing the accept for
e303_wep_01:
Stage_S01 vb=4600480 span 49 x 23 x 42 passes 0.85 = true
Stage_S02 vb=18403456 span 1600 x 2100 x 4800 passes 0.85 = true
Both clear the strict winding gate, because the wrongly-taken block IS real
coherent geometry -- another resource's. That rules out a family of fixes: no
threshold, scoring, or best-of-N selection can help, which is exactly why the
previous iteration's attempt changed nothing.
The search space must be constrained instead. Two candidates recorded, ordered by
how much new format knowledge they need: per-resource data extents from the XPR2
resource table, or monotonic anchoring (resource k's vb after resource k-1's) if
data blocks follow descriptor order -- testable against the 556 resources that
already decode consistently.
Instrumentation reverted; only docs land here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Tried the obvious fix: make anchor_grouped_meshes pick the best-scoring vb0
instead of the first candidate clearing the 0.85 gate. It changed nothing --
still 5480 of 6294 decoded, still 125 inconsistent -- and instrumenting the pivot
loop shows why: for e303_wep_01 that function never runs. The resource has a
single sub-mesh, so it goes through the per-block adjacency path
(anchor_pool_mesh) instead.
Recorded because the grouped-pool anchor is the prominent, well-documented one
and the natural first suspect; the fault is in the other path.
The change is reverted -- untargeted, unproven, and it added a scoring path for
no measured benefit. The idea behind it (several vb0 candidates can clear the
gate, and first-in-scan-order is an arbitrary tiebreak) still applies to whichever
anchor is actually at fault.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Following the e106 slab to its cause. e303_wep_01 decodes from 14 containers: in
11 it is a 49x23x42 turret with organic vertices, in Stage_S02/S08/S26 the same
resource -- identical 172 verts and 330 indices -- decodes to 1600x2100x4800 of
axis-aligned box corners with (0,0,1) normals. The anchor scan found a different
buffer that happens to share both counts, so every size-based check passes. That
is the "declined for location, not format" risk succeeding wrongly instead of
declining.
The detector needs no ground truth: a resource shared by several containers must
decode to the same bounds. 681 resources appear in >=2 containers and 125 decode
to different bounds while reporting identical vertex/triangle counts -- a lower
bound, since a resource wrong everywhere is invisible to it. _rou_f401 is
62x25x10 in 16 containers and 4738x3147x4738 in 2; _rou_e011_wep05 gives four
different spans across 8.
Majority vote would resolve 104 of the 125 (14 are exact 50/50 splits) and agrees
with ground truth in the one case that has a render and a runtime capture behind
it. Recorded as a heuristic, not applied: for _rou_e302_base_break the majority
is the LARGER span and nothing yet says which is right. Flag, do not silently
rewrite geometry on a vote.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
"A few multi-stream / quantized bodies remain" declined understates it. Across
all 166 resource3d containers: 6294 XBG7 resources, 5480 decoded (87.1%), 814
declined, in 31 of 166 containers. By name the declined set is 492 e* (enemy
craft), 142 f*, 73 n*, 23 eff*, plus *_dead destroyed variants -- not hero
bodies.
Also recorded as a dead end, so nobody repeats it: the descriptor's third word
looks like (streams << 16) | format -- decoded g001..g003 carry 0x00010001 while
declined t170/t180 carry 0x00020004 -- but histogramming it disc-wide puts
decoded AND declined at every value (0x00010001 alone: 4479 decoded, 328
declined). Its low half tracks sub-mesh count. Decodability is not declared in
the descriptor; it is whether the unit-normal anchor scan can locate vb0, which
the code already tests. 229 declined resources carry the most common word[2] with
under 1 KB of data, i.e. small meshes with too little signal to anchor rather
than exotic formats.
No decode gained this iteration; the gap is now quantified and one attractive
wrong turn is closed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Revert the mis-guided "triangle STRIP" reading (a937779): XBG7 index
buffers are triangle LISTS (prim=4 in the GPU capture; stored-normal
agreement 1.000 as a list vs ~0.49 as a strip). Reinstate the
winding-consistency gate.
Crack the grouped-pool layout used by the hero ship and ~150 detailed
models. A resource's sub-meshes share one index pool (buffers 4-byte
aligned, in descriptor-marker order) followed by one vertex pool (each
vtx_count*stride, same order); the vertex pool is 4-byte aligned after
the index pool. The whole resource is derived from one anchored pivot:
ib0 = vb0 - span - pad (pad in 0..=3, alignment)
ib[i] = align4(ib[i-1] + idx_count[i-1]*2)
vb[i] = vb[i-1] + vtx_count[i-1]*stride
vb0 is found via the unit-normal vertex-run scan; the alignment is
confirmed by validating the LARGEST sub-mesh (most reliable), after
which the rest are read/validated. A single marker reduces this to the
existing adjacency anchor, so grouped generalises it.
Results (cross-checked against a Canary GPU draw-log capture of
DeltaSaber_T.xpr):
- DeltaSaber_T f001 = body + 7 detail parts = 8650 tris, every sub-mesh
0-degenerate / full-coverage / winding-agreement 1.000.
- All 19 previously-declined weapon models now decode (they hit pad 2
and/or lead with a tiny bracket that broke a markers[0] pivot). Corpus
audit: 0/146 geometry files fail (was 19 -> flat-texture in the viewer).
- Stages unchanged (single-marker path is byte-identical; grouped falls
back to the old anchor on failure). 7/7 disc tests green incl. the
strict stage quality audit; new hero_ship_grouped_pool_decodes test.
Viewer: route by count_xbg7; --only matches an exact model name (so the
neutral pose renders without the mnv*/turn180 animation poses). Fix the
albedo matcher: match a sub-model to its _col map by entity stem
(e007_bdy_01 -> e007_col) instead of a full-name prefix, lifting stage
sub-model texturing from ~25% to ~95% (the rest were flat grey). Colour
correctness (channel order/sRGB) remains a separate dynamic-RE item.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Stage containers (hidden/resource3d/Stage_S*.xpr) are collections of up to
~450 enemy/prop sub-models, not single meshes. Xbg7Model::stage_models decodes
them: each resource is a [12-byte header][index buffer][vertex buffer] block
(index count = descriptor marker, vertex count = u32 32 bytes before it) whose
on-disc offset is NOT stored, so it is located by content — one O(file) pass per
stride finds vertex-buffer starts (unit NORMAL at +12 whose previous slot isn't)
and each resource is pinned to the candidate whose indices validate and produce
non-degenerate, well-connected triangles. A connectivity gate (mean triangle
edge <= 0.28x the bbox diagonal) rejects spiky mis-anchors. ~4993 sub-models
decode across the 22 stages.
The same insight fixes the weapon single-model layout: it is [12-byte header]
[index][vertex] too, not [index][12-byte gap][vertex]. Reading indices from the
block start turned the 12 header bytes into 6 junk indices (2 leading degenerate
triangles — the recurring stray-triangle artifact) and dropped the last 6 real
indices. Skipping the header leaves vertex offsets identical (coverage unchanged
at 36/166) and corrects the triangle list. Verified on wep_00/03/04.
Adds `sylpheed-cli mesh {info,render}` — a headless software rasterizer that
writes a shaded PNG (orthographic, z-buffered, two-sided), so recovered geometry
can be verified without the GUI. Stages render as a normalised thumbnail grid;
--only filters sub-models.
Tests: stage_models_{decode,sweep,quality_audit}; docs/re updated (xbg7-mesh.md
evidence log + INDEX.md).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Dynamic-RE follow-up: capture Canary's GPU vertex-fetch + draw calls and
feed the ground truth back into the static decoder.
The GPU capture confirmed the reverse-engineered layout exactly — meshes
draw as triangle LISTs (prim=4) with pos f32x3 @0, normal f16x4 @0x0C,
uv f16x2 @0x14 — and revealed the XBG7 vertex format is NOT fixed-stride:
models omit elements (stride 20 = pos+normal, no UV; 24 = pos+normal+uv).
- mesh.rs: parse the descriptor's vertex declaration ({offset, format-code,
usage} triples; 0x2A23B9=pos f32x3, 0x1A2360=normal f16x4, 0x2C235F=uv
f16x2) to drive per-model stride + element offsets, instead of assuming
stride 24. Coverage 25 -> 36 fully-validated models (e.g. the Stage_S*
props, which are pos+normal only). Same index-range + unit-normal safety
gates; complex/mismatched layouts still declined.
- Endianness note (documented): the capture's fetch endian=k8in32 describes
the GPU's guest-memory copy, NOT the .xpr file bytes — reading the file
with k8in32 breaks the normals (|n|->1.33); naive big-endian per element
is correct (the game rearranges vertex data on load).
- tests: a coverage-regression test (>=35 models) + the existing weapon /
body-declined disc tests still pass.
Confirmed against a Canary draw-log capture; see docs/re/structures/
xbg7-mesh.md (evidence log + declaration table).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reverse-engineer the single-stream XBG7 geometry layout (clean-room: hex
inspection + geometric validation of the retail disc's
hidden/resource3d/*.xpr, no game code copied) and present models as
textured 3D meshes in the explorer.
Format (docs/re/structures/xbg7-mesh.md): XBG7 geometry resources sit
inside XPR2 containers alongside TX2D textures. For ~25 single-stream
models (weapons, simple props) the data section is a sequence of
sub-meshes, each an u16-BE triangle-list index buffer followed (after a
fixed 12-byte header) by a stride-24 vertex buffer whose declaration is
in the descriptor: POSITION f32x3 @0x00, NORMAL f16x4 @0x0C, TEXCOORD
f16x2 @0x14. Sub-mesh (vtx,idx) counts come from descriptor tuples.
The +12 vertex offset is pinned by the recovered normals being exactly
unit-length (align16 lands 4 bytes early and silently corrupts every
field). A safety gate rejects any model whose indices are out of range
or whose mean |normal| is not ~1, declining garbage (Stage_S*
placeholders, the complex multi-stream hero-ship body) rather than
mis-decoding it.
- mesh.rs: Xbg7Model::from_xpr2 -> GameMesh { positions, normals, uvs,
indices }; standalone f16->f32; unit + real-disc tests (weapon decodes
to 215v/364t with unit normals + in-range UVs; DeltaSaber body
declined).
- texture.rs: from_xpr2_index / texture_names so the viewer can pick a
model's _col albedo map.
- viewer: loose .xpr with decodable XBG7 spawns Bevy meshes (real normals,
double-sided) textured with the albedo, framed by the orbit camera; the
central egui panel goes transparent so the 3D scene shows through.
Complex multi-stream body meshes (DeltaSaber f004, other vertex layouts)
remain undecoded and are cleanly declined — next target is dynamic RE.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
