Filtering after the assignment made every subset query a full decode (~15s on a
50MB container). full_decode_cached memoises it per container -- fingerprint is
length plus three sampled 4KB windows, keyed with min_consistency, last four kept.
Decoding five ships from Stage_S02 in turn: 10.5s for the first, then 48us-1.4ms.
A stage now costs one decode rather than one per ship. Ten suites green, viewer
builds, and a spot-checked resource still lands on the same offset.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
models_named pruned to the wanted set BEFORE distinct assignment, so collision
resolution saw a different resource population and returned different offsets:
27 of 356 resources in Stage_S02 decoded differently when asked for alone,
including real geometry (f001_bdy_30, f106_sld_02_l/m/d, f101_wep_01_l). Both the
viewer and assemble_ship decode subsets, so both could disagree with the
container's own answer. This was a regression from distinct assignment itself.
Fixed by filtering the OUTPUT: the assignment always runs over the whole
container. One-name, three-name and full decodes now agree exactly. Cost: a
single-resource query on a 50MB container goes from near-instant to ~15s;
per-container caching is the follow-up. Ten suites green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
XBG7_MONOTONE=1 enforces descriptor order per (stride, vtx, idx) signature: 89
minority decodes, unchanged. Monotonicity constrains order WITHIN a container
while the disagreement is BETWEEN containers, which hold different numbers of
these boxes in different arrangements. The remaining 89 are bounding-box identity
ambiguities needing descriptor-level information, not another anchoring
heuristic. Knob kept, default off, measurement recorded; suites green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rendering assembled e106 from Stage_S02 showed the old slab: e106_bdy_03 spanning
600x1600x998 where three other containers give 276x236x941. Its anchor had slack
3, and the coverage gate tolerated up to three unreferenced tail vertices --
tolerance that was hiding a mis-anchor, since real blocks reach their last vertex
exactly (8580 of 8629). Requiring exact coverage moves it to the block the other
containers agree on and the slab disappears.
Costs 3 resources (6212 -> 6209), inconsistency 39 -> 38, capture oracle
unchanged at 46/46, suite green. Evidence: captures/e106-cover-slack-before-after.png
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sweeping XBG7_GROUPED_CONSISTENCY 0.85 -> 0.80 -> 0.75 decodes no additional
resources and leaves inconsistency at 39, while changing which geometry some
grouped models get -- strictly worse, so 0.85 stays. With the degeneracy/extent
fix already refuted, the remaining 82 misses are not a threshold away; they need
a structural answer (marker lists that do not map onto stored blocks), which
means a capture, not more tuning. Records the state of the four shipped changes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Under the winding gate, coverage is 6212/6294 (98.7%) with 82 misses left,
attributed 42 degenerate/extent, 31 winding, 9 coverage, 0 connectivity. The
biggest bucket turns out NOT to be the blocker: replacing the absolute area test
with a scale-free collinearity test decodes no more resources and raises
inconsistency 39 -> 44, and dropping the extent floor to 0.05 adds two. Both stay
as opt-in knobs (XBG7_REL_DEGEN, XBG7_MIN_EXTENT) rather than defaults. Also
fixed another stale default label in edge_cap_sweep.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Gate attribution put connectivity behind 153 of 225 misses -- the largest
blocker, and the gate already known to reject a capture-proven block. Testing it
against winding consistency (an objective topology signal: ~1.0 or ~0.0 for a
real mesh, ~0.5 for a mis-carve) shows a floor of 0.70 with connectivity inert
dominates the previous default on both axes: 6212 resources decoded (+143) with
39 shared inconsistencies (-17), capture oracle unchanged at 46/46, twin
invariant clean, coverage invariant intact. Suite green.
The 0.80 cliff (5770 resources, 0 inconsistent) is documented, not taken --
consistency is the weaker witness, and both points are one env var apart.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The last two blocks that under-covered their vertex pool were f102_break.dat and
f104_break.dat in ptc_pack.xpr, each reading a neighbouring block's index buffer
against the wrong declaration (414 verts indexed to 404; 160 indexed to 79).
Their marker lists do not map onto the stored blocks -- only 2 of 9 and 4 of 10
sub-meshes decoded at all. Requiring coverage (max_idx + 4 >= vtx_count) for
pre-pivot sub-meshes drops exactly the mismatched pieces.
Every decoded sub-mesh disc-wide now covers its pool: 8580 at slack 0, 49 within
tolerance, none beyond, none negative. Coverage 6069/6294, inconsistency 56,
truth table 46/46 -- all unchanged. Suite green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
coverage_audit measures index coverage per decoded sub-mesh. 8586 of them
reference their last vertex exactly, so the 'buffer not covered' gate is well
founded -- but 18 had NEGATIVE slack: indices up to 364 vertices past the end of
their own buffer, emitted because anchor_grouped_meshes reads pre-pivot
sub-meshes unconditionally. Quality gates stay relaxed there (a tiny flat lead
part is legitimately poor) but index range is now required.
Coverage 6069/6294 unchanged, inconsistency 56 unchanged, truth table still 46/46
claimed, suite green; the vertex total drops by exactly the 1546 garbage verts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
undecoded.rs supplies the denominator the coverage numbers never had; the disc
holds 6294 XBG7 resources, 6069 decode, 225 are searched and missed, 0 lack a
descriptor. gate_histogram.rs attributes each miss to the furthest gate its best
candidate reached: 120 connectivity, 74 grouped-pool (different path), 15
degenerate/extent, 9 winding, 7 buffer-not-covered. Recorded as a work-list, not
a verdict -- a wrong candidate can pass more gates than the true block.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
debug_grouped_report (why_rejected) shows n206_02's alternative pool is ACCEPTED
at pad 0 under production gates and is in the candidate list -- and the decoder
does take it: n206_02 now anchors at 0x342d984. The 'still collapsed' reading
came from the audit comparing decoded geometry, and that offset holds a direct
(unmirrored) copy, so a separated pair still looked identical. The audit now
requires a SHARED BUFFER to call it a collapse: disc-wide 18 exact mirrors, 16
related, 0 collapses, 0 unrelated -- and the regression test drops its exception.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
anchor_grouped_meshes takes the same taken set; a colliding grouped model is
re-placed whole past everything claimed, or keeps what it had. Runtime oracle
unchanged (45/46 exact, 0 unclaimed), coverage unchanged (6069), cross-container
inconsistency 62 -> 56, suite green.
It does not clear the n206 twin collapse: no alternative pool validates for the
loser, so that pair is a validator case (like eng_02_l before the cap move), not
a selection one. Also fixes a stale '0.28 (default)' label in edge_cap_sweep.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Swept against the 46 capture-named Stage_S02 buffers with distinct assignment in
place: 0.28 anchors 40 exactly and leaves 4 unclaimed; 0.42 anchors 45 and leaves
none; nothing above 0.42 improves further, so it is the least permissive value
that captures the whole gain. 589 more resources decode, nothing is lost. The
cross-container consistency proxy worsens (46 -> 62) -- it cannot see a
systematic mis-anchor, and where the two disagree the capture wins. Suite green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Selection was per-resource and greedy, so two resources could take one vertex
buffer while a valid one sat unused. A runtime capture proved that wrong for the
mirrored e106 hull twins: the container holds both halves and the engine draws
each from its own address. Now the first claimant keeps a buffer and later
resources re-anchor past everything already claimed (coverage can never regress;
grouped-pool models untouched).
Against the 46 capture-named Stage_S02 buffers: exact anchors 29 -> 40, unclaimed
12 -> 4. Disc-wide: 5480 resources decoded (unchanged), cross-container
inconsistency 125 -> 46.
The twins' mirror therefore lives in the DATA, not in the placement matrix: the
embedded e106_bdy_02 row and the two assertions encoding the old convention are
updated, each with the reason recorded. Full suite green incl. disc/ISO gates.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
XBG7_EDGE_CAP / XBG7_SMALL_TRIS make the threshold sweepable (defaults unchanged,
full suite green). Above 0.417 the capture-proven eng_02_l anchors exactly right
and no e106 part regresses, and nothing that decoded at 0.28 is lost -- but ~250
existing anchors move silently and 18 shared resources lose cross-container
consistency. Not changed: the movers have no oracle yet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
debug_find_index_buffer scans the container for an index buffer that validates
against a capture-proven vertex buffer. For eng_02_l nothing validates with the
connectivity test on; with it off the nearest hit is exact pad-0 adjacency
(vb-ib = 144 = 72*2). The block's mean_edge/diag is 0.417 against a 0.28 cap --
the documented false positive for coarse LODs, now caught with ground truth.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
debug_try_anchor asks validate_block directly at the capture-proven offsets. Both
119-vert twin buffers are accepted by both twins (so the correct block lost the
first-match race, and a distinct assignment fixes it); the drawn 51-vert bridge
buffer is accepted by both bridge resources; eng_02_l's proven offset is rejected
outright, even with the pad widened to 64 -- a validator gap, not a selection one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
debug_vertex_run_starts exposes the anchor scan's candidate list: Stage_S01 has
15710 stride-24 starts and all three capture-proven offsets (0x3c55d8 twin,
0x40e418 bridge, 0x44a32c eng_02_l) are among them. anchor_pool_mesh takes the
first that validates, so an earlier lookalike wins. Scoped to the current
decoder's e106 cases; does not overturn the residual-51 finding.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first read of the capture-truth table located our resources by searching the
container for their leading vertices, which reads much worse than reality: the
same leading run occurs at several offsets in one container. GameMesh now carries
the offset the anchor scan actually chose, so the comparison is exact -- 4 of the
ship's drawn buffers are anchored correctly, 2 are the twin collapse, and 2 are
real mis-anchors of a size we do decode (brg 51 verts, eng_02_l 44).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The neighbourhood anchor (f18d591) and its refinement (27a0701) took
cross-container inconsistency from 125 to 51 with coverage unchanged, and made
e106 render as a destroyer rather than a slab. Both are reverted.
ship::tests::static_assembly_matches_runtime_capture is gated on SYLPHEED_ISO, so
it SKIPS in an ordinary cargo test -- which is why the regression was invisible
in every suite run so far. With the ISO it fails:
e106_bdy_01: static M row0 [-1.0, 0.0, 0.0] != captured [1.0, 0.0, 0.0]
e106_bdy_01 and _02 are a mirrored pair whose two buffers hold the same geometry
reflected in X, and BOTH resources currently decode to the SAME buffer (identical
counts, span and mean_x). apply_twin_mirrors picks which instance to reflect from
the sign of that mean_x, so which buffer wins flips the decision:
before both twins mean_x = -66.83 -> mirror bdy_02 (matches the capture)
after both twins mean_x = +66.83 -> mirror bdy_01 (contradicts it)
Neither is right -- two resources sharing one decode is itself the bug and the
mirror heuristic has been compensating. The capture is ground truth, so a change
that contradicts it does not ship. The real fix must give each twin its own
buffer first.
Kept from the attempt: this test now also asserts the SET of static placements
against the capture (allow-list {e303_wep_01} for vbase dedup), so extra
placements can finally fail it -- the direction it could never fail in before.
Docs, backlog, INDEX and the ignored test's message all corrected to say
diagnosed-not-fixed rather than fixed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Pass 1's anchor map contains exactly the mistakes the neighbourhood is meant to
correct, so a resource sitting beside a mis-anchored neighbour inherits a bad
reference. Re-anchoring against the improving map and repeating converges
quickly: two rounds, and a third changes nothing (the loop exits early when a
round is a fixpoint).
before decoded 5480/6294 inconsistent 125
neighbourhood anchor decoded 5480/6294 inconsistent 63
+ refining the map decoded 5480/6294 inconsistent 51
Coverage still unchanged. The 51 that remain cluster in _l (LOD) and _dead
variants -- e001_l, e010_bdy_01_l, e106_eng_02_l, _rou_f302_base_dead,
e303_base_dead and friends. A plausible reading is that a variant shares its
base's vertex and index counts, making the two mutually confusable so that
locality cannot separate them; recorded as untested rather than asserted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
anchor_pool_mesh took the FIRST candidate in file order from a container-global
vertex-run scan, so a resource could be handed another resource's block whenever
both shared (stride, vertex count, index count). Both blocks are real geometry and
both pass every quality gate, so only position separates them.
anchor_pool_mesh_near now tries candidates in order of distance from a reference,
and anchor_models_filtered runs two passes: pass 1 anchors first-match to learn
where resources land, pass 2 re-anchors each resource preferring the median anchor
of its +/-2 descriptor neighbours. Too few anchored neighbours -> keep pass 1, so
nothing regresses to guesswork.
before decoded 5480/6294 shared 681 inconsistent 125
after decoded 5480/6294 shared 681 inconsistent 63
Coverage unchanged, inconsistency halved. e303_wep_01 decodes to 49x23x42 in ALL
containers now, and e106 renders as a destroyer instead of a slab -- its two
shared turrets symmetric at X[-203,-154] and X[154,203]. That resolves the
user-reported "capital ships assemble wrong" for this cause.
The filtered path needed care: models_named (what the viewer uses) dropped
non-wanted resources, which would have left filtered decodes with no
neighbourhood and silently kept the old behaviour. Resources are now collected
regardless of the filter, but only the asked-for ones and their +/-2 neighbours
are decoded in pass 1, so a filtered decode stays proportional to what was asked.
63 cases remain; mesh_consistency_disc stays ignored and now records 63, not 125.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The module said "a few entries disagree with the actual frame count, so we walk
by the T8aD magic instead". They do not disagree. An LSTA is a display list of
inline elements that are either T8aD sprites or PRMD primitives (the flat
coloured quad the UI bundles use to dim a scene), and the count at 0x04 counts
both: across all 64 lists on the disc, count == T8aD + PRMD, with no exceptions.
The six lists that looked wrong (GP_DEBRIEFING_PILOTLOG, GP_MISSION_SELECT, two
language builds each) each hold exactly one primitive, which is the whole of the
off-by-one.
Also measured after the T8aD rectangle-list fix: all 1281 sprite frames decode,
100%.
parse() still returns sprites and skips primitives -- that is the useful
behaviour -- but the docs now say so instead of blaming the header.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The "~15% deferred variants" were not variants. Auditing every T8aD on the disc
gave 19216 surfaces, 18442 decoding (96.0%) and 774 failing in two clusters:
GP_DIALOG strips declaring 524x63 with a "tile count" of 1 or 2 instead of 3, and
small textures in the six *2D language paks whose pixels ran past the end of the
file.
Both fall out of the per-tile header, which is not opaque flags: it is four BE
u32 -- dst X, dst Y, width, height. A 15x18 icon stores a 13x18 rectangle at
(1,0); pdmes010 stores (59,6,256,54) and (315,6,149,54), the second beginning
exactly 16 + 256*54*4 bytes after the first. So 0x1c is a RECTANGLE COUNT and the
256-grid reading was an accident of most surfaces being stored as full-width
bands.
Parser rewritten to that model, still refusing to guess: a rectangle must fit the
declared surface and its pixels must fit the file, else None. Disc decode is now
19216/19216 = 100.00%.
Two test fixtures were built to the old model and are corrected rather than
worked around. lsta's t8ad_frame wrote NO offset-table entry, so the decoder read
"pixels" from inside the header -- the test passed only because it checked
dimensions alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Tier 3 matched a part to its hardpoint by trailing index, so `e105_brg`
compared "01" == "" against GN_Bridge_01 and fell through silently. The runtime
capture is what exposed it: the game draws the bridge and places it at
[0, 70, -1850] rel e105_bdy_01, and assemble_ship emitted nothing there.
With no index to match on, take the lowest-numbered frame of the category.
Diffing assemble_ship part counts across every container: 34 (stage, ship)
entries gain parts — e102 +2 (bridge and engine), e104 +1, e105 +1, Stages
02-29. ship_audit is unchanged, so nothing regressed, and the capture now agrees
to dT 0.03 / dR 0.000.
Also fixes the diff itself: correlate_frames compared static against a rotation
sampled from the first block, which can belong to another INSTANCE of the class.
Scoped to the position-agreeing cluster, e105_eng_01 goes 1.711 -> 0.000 and
both e106 nacelles to 0.000. The one remaining rotation delta (e106_wep_02_01,
0.134) is a turret whose rotation varies by 0.182 between blocks that agree on
its position — the runtime disagrees with itself more than with the assembler.
The new rotVar column makes that distinction visible.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A capture at controlled range (ship_capture_close.sh: lock a capital ship, close
on it, F10 per range band) finally draws capital-ship hulls at full detail. Two
correctness fixes were needed before the numbers meant anything:
* one F10 log is ~14 frames with no delimiter, and WV_ref^-1 . WV_p only cancels
the camera within one frame — segment_frames splits on vertex-buffer
recurrence, and correlate_frames cross-checks the blocks against each other
instead of trusting a single shot;
* aggregate by consensus, not median: a stage holds several ships of one class
sharing vertex buffers, so a block can mix two instances.
Result: f105, e105 and e106 reproduce assemble_ship to <=0.43 units in
translation and 0.000 in rotation for every part that does not move. The e106
rules generalise, and the viewer bug report now points at the viewer. Narrow
leftovers: e105_brg is missing from assemble_ship, e105_eng_01 rotation differs
by 1.711.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
capture_verify example: clusters every part's ship-relative transform across
the new multi-snapshot, multi-instance F10 captures (5 snapshots, 43 e106
instances, all angles). Verdict: the dominant clusters match the static
assembly to ~1 unit on EVERY part — including BOTH engine nacelles at
(+-131, -133, -131) — so the "engines inside the hull" appearance is the
game's own placement: the engine geometry sits recessed in the aft hull, and
the visible "thrusters" in-game are exhaust FX drawn at the GN_Jet/GN_SJet
frames (Z ~ -570, past the stern).
To close that perception gap the viewer now draws simple exhaust cones at the
game's own jet frames (ship::exhaust_frames; part of the external-parts
toggle). The jet frames flip Z, so the cone apex is authored at +Z and lands
trailing aft — verified in the offline render.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fixes the two things the user saw in the Ships browser:
- "Can't find the thrusters": show_external defaulted OFF, hiding the engine
rig / bridge / turrets entirely. Externals are EXACT since the node-TRS fix,
so the toggle now defaults ON (relabelled; off = bare hull bodies).
- "Some parts very far away": families without a composite scene graph (fighter
morph sets like f002 whose detached parts are authored far from the origin,
shared turret/prop part families) fell into the stack-at-origin fallback and
rendered as piles with stray pieces. ShipModel now carries has_composite and
the catalog lists only real assembled ships.
Offline audit across ALL stages (ship_audit example): the only far-away
outliers were composite-less f002 morph parts (now filtered); every capital
ship assembles coherently (e108 places its 7 turret instances, f105 its
mirrored shield generators). All primary composites are single-key tracks —
multikey exists only in animation composites (e901 attacks, missile-open),
which assemble_ship does not select.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The runtime capture served its true purpose: as the answer key that exposed the
static encoding. The XBG7 node joint table (node+0x44) holds NINE keyframe
channels [TX TY TZ RY RX RZ SX SY SZ] behind EIGHT pointer slots shifted one
track forward: channel k+1 = f64@(ptr[k]+8), and channel 0 -- TX, which no
pointer names -- sits at ptr[0]-0x48 (tracks are 0x50-byte records, value at
+8). The old 8-slot read took TY as X, RY as Y (usually 0 -- why both hulls
stacked on the centreline) and never saw TX at all (the +-264 hull offsets).
Euler order is Ry(ch3)*Rx(ch4)*Rz(ch5) with angles applied directly, pinned by
the captured engine nacelle Rx(-15)*Rz(30) decomposition. mesh.rs gains
read_trs9/node_rotation; scene_world_nodes and node_transforms both fixed
(saber fin overrides unaffected).
assemble_ship rewritten fully static and exact:
- every composite-node instance placed (alias duplicates collapsed, real
instances kept) including cross-id turret mounts (rou_e303_wep_01_root x2)
- the e_rou_<id>_eng cluster rig mounts at GN_Engine_01 (two mirrored nacelles
+-131 with -+30-degree cant + centre engine) -- world = frame o rig_local
- brg/sld at their exact GN frames (frames were always exact)
- shared-geometry twin pairs at +-TX: X-reflect the instance whose offset
opposes the geometry's dominant side (viewer reverses winding on det<0)
- squeezed node names resolve (wep02 -> wep_02_01)
Ground-truth gate: ship::tests::static_assembly_matches_runtime_capture asserts
static == the baked e106 capture for all 8 parts (T<1.0, R<0.02) plus both
nacelles, both turrets, and the starboard-hull mirror. Viewer now uses pure
static assembly; ship_capture + the baked table remain as the verification
oracle. Renders show a coherent, bilaterally-symmetric destroyer.
81 formats-lib + all disc tests green; viewer builds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The second F10 capture (ship at _m LOD distance) closes e106 entirely: all 8
parts placed including the bridge both earlier captures missed, cross-validating
the doc's 7 known translations to 0.1 units and adding brg_01 at the exact
centreline (0, 153.3, -164.0).
Matching hardened by what the real capture taught us:
- LOD-aware: each part tries every variant vcount (base/_m/_l/_d) — a distant
ship draws its LOD copies, same local frame.
- Position-validated, SET-based, against the UNION of a part's variants: buffer
order != decode order, and one draw's vcount equalled the _m count while its
buffer held the FULL 1633-vert geometry. A coincidental vcount (foreign
51-vert mesh vs the bridge) is rejected by geometry.
- Runtime mirror handled: twins share one file geometry; the engine uploads the
starboard copy X-reflected. Mirror-validated draws bake diag(-1,1,1) and the
viewer reverses triangle winding for det<0 so front faces stay outward.
- Near-axis rotation entries snapped to exact 0/+-1 for a clean table; eng_01
keeps its genuine 30-degree nacelle rotation.
data/ship_placements.txt now ships e106 (viewer renders via the captured table);
embedded_e106_is_complete locks the baked data. part_pos + capture_match helper
examples added. 10 ship_capture tests; 80 formats-lib tests green; viewer builds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The correlator now accepts BOTH capture formats: the F10 ship-capture snapshot
and the draw-logger format (mission_draws.log / xenia_re_draws.log) via
parse_drawlog — group the ship shader's draws by vertex-buffer base, vcount =
size_words/stride_words, keep the first c0..c2 WVP. So a capital ship seen in a
normal instrumented run bakes without a dedicated F10 pass. correlate_capture
auto-detects the format. Shared normalize_wvp between both parsers.
Also add examples/ship_dump.rs (offline static-placement inspector) and record
the measured static-assembler gaps for e106 in the doc: bdy_01/bdy_02 overlap
(runtime port/starboard mirror at X=+-264), eng_02 and wep_02_01 never placed.
These confirm exact placement is runtime-only — nothing more to squeeze
statically; the capture-driven table is the only path. The draw logs present on
this box are the player fighter, so no capital-ship table can be baked offline.
7 ship_capture tests (adds draw-log parse + correlate); 77 formats-lib green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Promote the F10 ship-capture correlator from an example into a reusable, unit-
tested library module (ship_capture): parse_capture (log -> per-draw rigid
WorldView from the c0..c2 WVP constants), correlate (match parts to draws by
vertex count, express each in the reference part's frame), and a checked-in
placement-table format (serialize_table/parse_table, embedded via
embedded_placement from data/ship_placements.txt).
build_ship_model now prefers a ship's captured placement over the static
assemble_ship when a table entry exists (empty table -> unchanged fallback).
correlate_capture example refactored onto the module + gains --emit to print a
table block. Doc updated: bake plumbing done; remaining is running real F10
captures to populate the table (needs the emulator).
5 new ship_capture tests (parse/normalize/correlate/table round-trip); 76
formats-lib tests + viewer build green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two changes after user feedback that externals sit near-but-wrong (shield inset
into the hull, thruster floating close):
- Scale: the joint table's slots 5–7 are per-axis scale (a GN_ShieldG frame ships
at 0.5, GN_Jet FX at 2.4–6.5) and were being ignored, rendering scaled parts at
the wrong size. ScenePart now carries `s` and applies R·(S·v)+T.
- External placement is fundamentally approximate: a GN_* frame is the mount
PIVOT (f105's two GN_ShieldG frames are both on the centreline), while the
part's outboard/rotational offset lives in a detail sub-rig / runtime code this
static pass can't recover. So assemble_ship gains `include_external`: the hull
tier (rou_ nodes) is exact and always drawn; the external tier (bridge / shield
/ engine at GN_ frames) is opt-in via a "Show external parts" checkbox in the
Ships browser, off by default so the clean, correct hull is the default view.
The module doc is corrected to the scene-graph mechanism (the old "draw parts
untransformed" claim was wrong).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The sub-composites (`e_rou_<id>_eng`, `_wep_*`) are LOCAL detail rigs: their
nodes are posed in their own frame, not ship-space, so folding them in placed
the engines at ~mid-ship (Z≈99) instead of aft — the "floating in air" parts.
The PRIMARY composite (`e_rou_<id>`, the one with the most nodes) already carries
the full ship-space layout: every hull body AND every `GN_*` hardpoint frame
(GN_Bridge at the stern, GN_Engine aft, GN_ShieldG amidships…). Assemble from
that one composite only; engines/bridge/shield now resolve to their real aft/mid
frames. Robust to the odd single-composite case (e108's `e_rou_e108_Missile_open`
is still picked as its own primary).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The first cut drew every part at the origin, overlapping into a mess — the parts'
vertex buffers are authored around the origin, NOT pre-placed. The real world
placement lives in the composite scene-graph resource `e_rou_<id>`, exactly like
node_transforms poses the DeltaSaber's fins within one resource — but here the
nodes reference SEPARATE geometry resources.
Two tiers, matching how the game splits a ship (as the user described):
- Hull — `rou_<id>_*` scene nodes name the hull body resources and carry their
world transform (bodies that together form the hull, not individually
destructible). Composed parent∘child down the first-child/next-sibling tree.
- External hardpoints — `GN_*` frame nodes are the Vessel mounts (`GN_Bridge_01`,
`GN_ShieldG_01`, `GN_Engine_01`, matching the IDXD recipe Frame names). Each
bridge / shield-generator / engine part is placed at its frame by category+idx —
the individually-destructible parts.
- mesh: `scene_world_nodes` — walk a composite, emit every `rou_`/`GN_` node with
its composed world transform (same record layout + TRS convention as
node_transforms, cross-resource). ScenePart{resource,m,t}+apply.
- ship: `assemble_ship(bytes, id)` — resolve hull nodes to the ship's own parts
(shared cross-id turrets excluded — they need the per-mount recipe), match
external parts to GN frames, fall back to a raw draw when a prop has no
composite. is_base_part now also drops `_dNN`/`_lNN`/`_mNN` LOD copies.
- viewer: build_ship_model runs assemble_ship, decodes only referenced resources,
bakes each part's world pose into its vertices (one posed copy per placement),
then assembles via the shared prepare path. RequestShipRender carries the id.
Verified headless: parts now spread into coherent single-ship extents
(e108 185×174×683, f106 703×479×2079, f101 carrier 560×539×1712) instead of
piling at the origin. Turret placement (shared eNNN gun models on many GN_*Gun*
frames) still pending the per-mount recipe.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Project Sylpheed never stores a capital ship as a single mesh. Each ship is
modelled, then split into destructible parts (hull segments, bridge, engines,
integral turrets), each shipped as its own XBG7 resource inside a
hidden/resource3d/Stage_SNN.xpr container. The split lets the game hide/replace
a part when the player destroys it. Crucially each part keeps its ship-local
position, so reconstructing the whole ship is: draw every base part of one id
together, untransformed.
New `ship` module: Faction (from the e/f/n id prefix — ADAN/TCAF/Neutral),
ShipModel, ship_id_of, is_base_part (skips _l/_m/_d LODs, _bNN/_dead/_break
destruction geometry, e_rou_ scene nodes), and ships_in_container to group a
stage's resources into whole ships.
Supporting:
- mesh::xbg7_resource_names — list a container's XBG7 resource names (directory
walk only, no geometry decode).
- Xbg7Model::models_named — decode only the named resources (assemble a ship
from its ~8 parts instead of the whole ~300-resource stage).
- game_data::Vessel gains `model` (the rou_<id> hull stem) to link stats to the
3D part family.
Tests: id/part classification + a disc-gated reconstruction of the ADAN e108
frigate from Stage_S02.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Decode the player weapon arsenal from the UNITS tables
(GP_HANGAR_ARSENAL.pak): the selectable weapons per Delta Saber
hardpoint. Each `STANDARD_<hp>` header (Nose / Arm1-3) lists its options
up to the next header — extracted by shape, unioned + deduped across
configs.
The arsenal reads with the game's own names: NOSE guns (Stiletto,
Rapier, Broad Sword, Machine Cannon), ARM missiles/bombs (Falcon, Eagle,
White Shark, Piranha, Tomahawk Rail Gun, Maelstrom Bomb). Example
`arsenal` prints them. +1 test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The player's Delta Saber missile / special weapons (Weapon_DSaber_P_wep_*)
live in variant schemas (439b1f29, 28295901), not the main WEAPON schema,
so the schema-only filter missed 10 of them. Match all `Weapon`-shaped
records by table name instead (excluding the EnumWeapon name lists),
deduped by id: 121 → 131 weapons.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Decode the UNITS/ZUNITS player-unit tables (GP_HANGAR_ARSENAL.pak) into
flight assignments: which pilot flies each callsign (Rhino1→Katana,
Bird1→Sandra, …), plus the player craft. Assignments are `Callsign<N>-
Pilot` tokens, extracted by shape.
138 configs; the distinct line-ups trace the campaign's story — Raymond
leads Rhino flight early (Rhino1=Raymond, Rhino2=Katana), then Katana
takes the lead (Rhino1=Katana, Rhino2=Ellen, Rhino3=Gene, Rhino4=Yoji).
Example `flights` prints the line-ups. +1 test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Decode the EnumUnit tables (schema 35b8dc67) into mission rosters: the
distinct combatant `UN_*` ids for one battle, stage props (asteroids,
collision meshes) dropped. The tables aren't tagged with their stage on
disc, so `stage` is inferred from any `UN_S<NN>_…` prop id in the roster
and left None otherwise (honest — ~8 of 31 rosters self-identify).
Joined against load_units / load_vessels this gives each battle's
combatants with stats, e.g. S01 = ADAN Turret (100 HP), Attacker
(500 HP), Destroyer (10000 HP). Example `battle` prints them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Decode the message tables (schema b412e6d8) into dialogue lines: each
`Message_NNN` sub-record → a DemoMessage with the speaker, portrait
(face), voice-clip token, and caption page text-keys. Fields are found
by prefix (Character*/Face*/VOICE_*/id-prefixed pages), so the
positional / first-record-defines-schema layout doesn't matter.
10,263 lines parse; ~815 name an explicit speaker (the rest are system /
continuation lines — left unattributed rather than guessed). Paired with
localization::TextIndex the attributed ones read straight out — e.g.
KATANA [VOICE_RHIN_000] "Ellen, get back into formation!". The
voice-clip token ties each line to its audio bank, closing the loop with
the cutscene-voice work. +1 test; example `dialogue` prints
speaker + clip + resolved text.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Decode the per-stage Enumerate_Squadrons tables (UnitGroup_S<NN>.tbl)
into flight groups: formation shape, AI behaviour, side, and the member
craft. Definitions are delimited by the FormationID key; members are the
first `count` UN_ craft after the header (anything past them belongs to
the next squadron or the stage roster — the naive "all UN_ in span"
over-collected). e.g. the player's Rhino flight = a 2-craft TCAF
squadron of Delta Sabers flying Formation_2_Rhino. Schema hash varies
per stage so we match on the table name. Squadron id is positional and
left None when the id list and definitions don't line up (honest, not
guessed). +1 test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Project Sylpheed stores its UI/story text in IXUD entries as UTF-16LE,
interleaved `text, key` (a prose string immediately followed by the
identifier that names it). TextIndex::build(pak) scans a language pak's
IXUD entries and indexes every pair — 8457 English keys from
GP_MAIN_GAME_E: objectives, hints, lose conditions, character names,
and cutscene dialogue.
This makes the whole game read in English: the 16-mission campaign
(Repel the surprise attack → Shoot down Margras → the Prometheus Driver
finale), each mission's briefing, and the named cast (Katana, Raymond,
Ellen, Crichton, …) all resolve. API: get / objectives / lose_conditions
/ hints / character_name. 3 tests.
Examples: campaign / dossier print the readable campaign + cast.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Decode Project Sylpheed's IDXD data model into plain, cloneable structs
(from GP_MAIN_GAME_E.pak; the D/F/I/J/S paks are localized dups):
- Weapon (121), CraftUnit (89, with the AI flight model in .fields),
Vessel (23, structural component counts), PlayerConfig (24) — the
full combat balance.
- Character (68 — faction + portrait set), Stage (29 missions:
location, phases, and the per-stage resource tables it wires up).
- Generic Record / load_records(schema) reads any of the 105 IDXD
schemas without a bespoke struct.
Each blob = one entity; token[0] names the table, fields are
value-before-key. Named Option<> fields for the common stats + a full
`fields` map; defaulted fields stay None (they're code-resident, not on
disc — honest blanks, never guessed). 8 tests against the real disc.
Examples: iso_map / deep_map census the ISO's formats and IDXD schemas;
dm_extract / dm_rows / dm_roster / mission_map dump the decoded data.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Cracked how the game maps a movie to its voice track and reimplemented it,
fixing wrong/short voices for RT and hokyu cutscenes.
## Mechanism (RE'd from a Canary file-I/O trace, user-confirmed by ear)
The movie voices are ONE continuous XMA stream, physically chunked into
`<lang>\Movie\VOICE_*.slb` (and `\etc\VOICE_D_*.slb`) sound.pak TOC entries
whose boundaries do NOT align with the cutscene cues. A single cue routinely
spans two `.slb` chunks, so a `.slb` does not necessarily hold the track its
name claims (they are off-by-one vs the cues). Each cue ends at an inline
trailer descriptor `(sound_id:u32be, 0x11, …)` whose id repeats at +0x800.
Resolution chain (all derivable on-disc, no guest-code RE):
1. movie -> cue token (ADVERTISE_MOVIE manifest VOICETRACK)
2. cue token -> sound-id (master sound registry, tables.pak
a2c8c185=eng / b04238e0=jpn, schema 0x13cb84ba)
3. sound-id -> [start,end) scan the stream for trailer(id) and its
predecessor; cue audio = the bytes between,
read continuously across chunk boundaries.
Validated: decoded voice length matches each movie within ~0.4s, including
cross-boundary cues (ADV/RT01A/RT01B/RT01C/S00A/S01A + the 5 bound hokyu).
## Changes
- sylpheed-formats/src/movie_voice.rs (new): registry_voice_ids(),
find_descriptor(), find_descriptor_before() (gap-tolerant predecessor).
Unit-tested.
- viewer iso_loader.rs: resolve_movie_voice_region() + decode_voice_region()
(continuous read via existing read_segment_range + to_xma_riffs). Voice/
audio requests now region-first; a `\Movie\` token that fails region stays
SILENT (never plays the wrong off-by-one chunk). Removed the old
movie_is_demo_based suppress hack.
- Anchor tries subdirs Movie/etc/Voice so hokyu `\etc\VOICE_D_*` resolve too.
- Examples resolve_all/validate_cues/hokyu_final document + validate the pipeline.
## Status
- RT / S / ADV cutscene voices: CORRECT (user-confirmed).
- 5 manifest-bound hokyu (VOICE_D_450-454): CORRECT (user-confirmed).
## OPEN (handoff)
13 of 18 hokyu movies have NO manifest VOICETRACK; the game reuses the 5
recordings across stages via a code rule. `hokyu_fallback_token()` currently
guesses by category (ship LS/DS x source carrier-A/tanker-H: LS/A->450,
LS/H->453, DS/A->452, DS/H->454). USER REPORTS THIS IS SOMETIMES WRONG.
- LS/carrier has two takes (450 from s02A, 451 from s09A); the early-vs-late
split is unknown — unbound LS/A currently all default to 450.
- Definitive fix = Canary `--phase_a_fileio_only` file.read trace of an unbound
hokyu (e.g. hokyu_LS_s03A) to see which VOICE_D the game actually streams,
then encode the real rule. Same method that cracked the RT mapping.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The movie manifest (tables.pak, schema 0x067025b9) is the game's authoritative mission -> phase -> cutscene table. Previously we only scraped VOICE_/SUBTITLE_/.wmv strings by prefix; now we decode the real two-array (slot-keys, then value-records) structure.
MovieEntry gains slot, kind (System/Intro/Phase/PhaseEnd/Supply), mission, phase, subtitle, and telop (the previously-ignored .prt on-screen text overlay). Valueless slots (stages with no resupply video) are recovered without decoding the binary node region, via the fully-derivable MS<NN><X> -> S<NN><X> anchors.
Backward compatible: movie/voice_token and resolve_voice_entry/voice_token/is_manifest are unchanged. Validated end-to-end on the real disc: 101 entries (104 slots - 3 gaps), typed System=6/Intro=27/Phase=32/PhaseEnd=18/Supply=18. examples/manifest_map.rs dumps the full map.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
list_voice_clips now enumerates every spoken-line category so the standalone audio player covers them all — bound movie voices (\Movie\), in-mission radio (\Voice\, \etc\), and mission briefings (\Briefing\, whose BR<NN>_<MM> names lack "VOICE"); music (BGM_*) stays excluded.
Viewer voice-library fixes: invalidate the library when a new game source loads, and don't latch an empty result as "loaded" (sounds.tbl always has thousands of clips, so empty = a failed read) so re-opening the menu retries.
Voice decode reverted to the first sub-wave (base behaviour): a naive multi-sub-wave concat produced the wrong track for RT movies. The per-sub-wave splitter (to_xma_riffs) is kept + documented for the eventual real fix, which needs ground-truth playback instrumentation to resolve segments-vs-takes-vs-timecode placement.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The movie cutscene subtitle + voice pipeline, driven by the ADVERTISE_MOVIE
manifest (the authoritative movie -> subtitle -> voice index). Also flushes
several sessions of local WIP (async viewer loading, grouped-pool XBG7/hero-ship
decode, drawlog tooling). See docs/HANDOFF-movie-voice-subtitles-2026-07-19.md.
Subtitles (movie_subtitle.rs):
- Full movie->track->text chain; join multi-line captions sharing one timing
(fixes S13A dropped "Look at it father" line); overlap-safe active_cues();
Latin-1 accents preserved.
Voice (slb.rs): XACT .slb -> XMA1 RIFF; take the FIRST sub-wave bounded by its
declared data size (fixes S10-S16 alternate-take garble); list_voice_clips.
Manifest (movie_manifest.rs): parse ADVERTISE_MOVIE (0x5B983A08) for the real
movie->voice binding (not always VOICE_<movie>; e.g. hokyu -> VOICE_D_* in etc\).
Resolve the token's sound.pak path via sounds.tbl. DIRECT bindings only — the
demo-id shared-clip fallback for unbound hokyu movies was verified WRONG in-game
and reverted (unbound hokyu stay unvoiced; correct join key is an OPEN problem).
Viewer: manifest-driven voice (movie player toggle + solo button), standalone
"Voice Lines" browser, stacked caption overlay.
Tests: 46 formats-lib + 11 viewer-lib + movie_manifest/movie_subtitle/slb disc
tests; full workspace green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Revert the mis-guided "triangle STRIP" reading (a937779): XBG7 index
buffers are triangle LISTS (prim=4 in the GPU capture; stored-normal
agreement 1.000 as a list vs ~0.49 as a strip). Reinstate the
winding-consistency gate.
Crack the grouped-pool layout used by the hero ship and ~150 detailed
models. A resource's sub-meshes share one index pool (buffers 4-byte
aligned, in descriptor-marker order) followed by one vertex pool (each
vtx_count*stride, same order); the vertex pool is 4-byte aligned after
the index pool. The whole resource is derived from one anchored pivot:
ib0 = vb0 - span - pad (pad in 0..=3, alignment)
ib[i] = align4(ib[i-1] + idx_count[i-1]*2)
vb[i] = vb[i-1] + vtx_count[i-1]*stride
vb0 is found via the unit-normal vertex-run scan; the alignment is
confirmed by validating the LARGEST sub-mesh (most reliable), after
which the rest are read/validated. A single marker reduces this to the
existing adjacency anchor, so grouped generalises it.
Results (cross-checked against a Canary GPU draw-log capture of
DeltaSaber_T.xpr):
- DeltaSaber_T f001 = body + 7 detail parts = 8650 tris, every sub-mesh
0-degenerate / full-coverage / winding-agreement 1.000.
- All 19 previously-declined weapon models now decode (they hit pad 2
and/or lead with a tiny bracket that broke a markers[0] pivot). Corpus
audit: 0/146 geometry files fail (was 19 -> flat-texture in the viewer).
- Stages unchanged (single-marker path is byte-identical; grouped falls
back to the old anchor on failure). 7/7 disc tests green incl. the
strict stage quality audit; new hero_ship_grouped_pool_decodes test.
Viewer: route by count_xbg7; --only matches an exact model name (so the
neutral pose renders without the mnv*/turn180 animation poses). Fix the
albedo matcher: match a sub-model to its _col map by entity stem
(e007_bdy_01 -> e007_col) instead of a full-name prefix, lifting stage
sub-model texturing from ~25% to ~95% (the rest were flat grey). Colour
correctness (channel order/sRGB) remains a separate dynamic-RE item.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The weapon/prop models (rou_fxxx_wep_nn.xpr) rendered with triangular holes in
the hull and misplaced/extra spikes. Root cause: XBG7 sub-mesh index buffers are
triangle STRIPS, but the decoder read them as triangle LISTS — a strip of N
indices is N-2 triangles, a list is only N/3, so ~2/3 of every hull was missing
(the holes) and each list-triple of strip data spanned unrelated vertices (the
spikes). The tell: triangles-per-vertex was 0.5-1.0 with 0 unreferenced vertices
(a closed surface needs ~2.0).
- mesh.rs: expand_triangle_strip() converts strip -> list with alternating
winding, skipping degenerate triangles (repeated index = strip restart).
Applied in both from_xpr2 (weapons) and read_pool_mesh (stage sub-models).
All 71 weapon submeshes now have healthy ratios; hulls fill in (wep_03 cannon,
wep_04 pod, wep_11, wep_37 verified coherent).
- Module doc updated (strip, not list).
- sylpheed-cli: `mesh info` reports per-submesh degenerate/unref/spanning-triangle
counts; `mesh render` gains --dist (camera zoom). XMESHDBG env dumps the
descriptor index markers.
Known residual: an index buffer may concatenate several strips with no degenerate
bridge, leaving ~2 spanning "spike" triangles at each restart (index jump to a new
vertex region) — <1% of tris on most models. Decoding the restart mechanism is a
follow-up (a blanket spanning-filter is unsafe: clean models have legit elongated
tip triangles).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
