debug_grouped_report (why_rejected) shows n206_02's alternative pool is ACCEPTED
at pad 0 under production gates and is in the candidate list -- and the decoder
does take it: n206_02 now anchors at 0x342d984. The 'still collapsed' reading
came from the audit comparing decoded geometry, and that offset holds a direct
(unmirrored) copy, so a separated pair still looked identical. The audit now
requires a SHARED BUFFER to call it a collapse: disc-wide 18 exact mirrors, 16
related, 0 collapses, 0 unrelated -- and the regression test drops its exception.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
twin_mirror_audit applies the capture-derived rule to all 166 containers: of 34
equal-count twin pairs, 18 are exact X-mirrors, 15 related another way, 1
identical, 0 unrelated. Two calibration fixes were needed first (authored halves
need a tolerance, and a mirrored pair may be stored in another vertex order).
The one collapse, n206_01/_02, is a grouped-pool pair -- the path distinct
assignment excludes -- so it names the next target. Added a disc-gated
regression test; refreshed the stale ignore message on the consistency test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
texture_disc.rs and mesh_disc.rs fall back to "/home/fabi/RE Project Sylpheed/..."
when SYLPHEED_ISO / SYLPHEED_RES3D are unset. The directory is "RE - Project
Sylpheed"; the fallbacks were missing the " - ", so stage_models_decode,
stage_models_sweep and stage_models_quality_audit failed with NotFound for anyone
who did not happen to know the env var. They now pass unaided (10/10 in
mesh_disc).
Found while auditing what the suite actually exercises after withdrawing the
anchor change: an ISO-gated test skipping, and an ignored test failing on a typo,
are both "green" in a normal run.
Full state, with SYLPHEED_ISO/DISC/RES3D set and --include-ignored: 82 lib tests
and every disc/ISO suite pass; the single failure is
shared_resources_decode_identically_in_every_container, which is deliberately
written as the target state and #[ignore]d.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The neighbourhood anchor (f18d591) and its refinement (27a0701) took
cross-container inconsistency from 125 to 51 with coverage unchanged, and made
e106 render as a destroyer rather than a slab. Both are reverted.
ship::tests::static_assembly_matches_runtime_capture is gated on SYLPHEED_ISO, so
it SKIPS in an ordinary cargo test -- which is why the regression was invisible
in every suite run so far. With the ISO it fails:
e106_bdy_01: static M row0 [-1.0, 0.0, 0.0] != captured [1.0, 0.0, 0.0]
e106_bdy_01 and _02 are a mirrored pair whose two buffers hold the same geometry
reflected in X, and BOTH resources currently decode to the SAME buffer (identical
counts, span and mean_x). apply_twin_mirrors picks which instance to reflect from
the sign of that mean_x, so which buffer wins flips the decision:
before both twins mean_x = -66.83 -> mirror bdy_02 (matches the capture)
after both twins mean_x = +66.83 -> mirror bdy_01 (contradicts it)
Neither is right -- two resources sharing one decode is itself the bug and the
mirror heuristic has been compensating. The capture is ground truth, so a change
that contradicts it does not ship. The real fix must give each twin its own
buffer first.
Kept from the attempt: this test now also asserts the SET of static placements
against the capture (allow-list {e303_wep_01} for vbase dedup), so extra
placements can finally fail it -- the direction it could never fail in before.
Docs, backlog, INDEX and the ignored test's message all corrected to say
diagnosed-not-fixed rather than fixed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Pass 1's anchor map contains exactly the mistakes the neighbourhood is meant to
correct, so a resource sitting beside a mis-anchored neighbour inherits a bad
reference. Re-anchoring against the improving map and repeating converges
quickly: two rounds, and a third changes nothing (the loop exits early when a
round is a fixpoint).
before decoded 5480/6294 inconsistent 125
neighbourhood anchor decoded 5480/6294 inconsistent 63
+ refining the map decoded 5480/6294 inconsistent 51
Coverage still unchanged. The 51 that remain cluster in _l (LOD) and _dead
variants -- e001_l, e010_bdy_01_l, e106_eng_02_l, _rou_f302_base_dead,
e303_base_dead and friends. A plausible reading is that a variant shares its
base's vertex and index counts, making the two mutually confusable so that
locality cannot separate them; recorded as untested rather than asserted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
anchor_pool_mesh took the FIRST candidate in file order from a container-global
vertex-run scan, so a resource could be handed another resource's block whenever
both shared (stride, vertex count, index count). Both blocks are real geometry and
both pass every quality gate, so only position separates them.
anchor_pool_mesh_near now tries candidates in order of distance from a reference,
and anchor_models_filtered runs two passes: pass 1 anchors first-match to learn
where resources land, pass 2 re-anchors each resource preferring the median anchor
of its +/-2 descriptor neighbours. Too few anchored neighbours -> keep pass 1, so
nothing regresses to guesswork.
before decoded 5480/6294 shared 681 inconsistent 125
after decoded 5480/6294 shared 681 inconsistent 63
Coverage unchanged, inconsistency halved. e303_wep_01 decodes to 49x23x42 in ALL
containers now, and e106 renders as a destroyer instead of a slab -- its two
shared turrets symmetric at X[-203,-154] and X[154,203]. That resolves the
user-reported "capital ships assemble wrong" for this cause.
The filtered path needed care: models_named (what the viewer uses) dropped
non-wanted resources, which would have left filtered decodes with no
neighbourhood and silently kept the old behaviour. Resources are now collected
regardless of the filter, but only the asked-for ones and their +/-2 neighbours
are decoded in pass 1, so a filtered decode stays proportional to what was asked.
63 cases remain; mesh_consistency_disc stays ignored and now records 63, not 125.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Each finding today was originally missed by reasoning from a sample, so these
assert DISC-WIDE invariants rather than one hand-picked file.
ui_surfaces_disc.rs (3 passing)
every_t8ad_on_the_disc_decodes -- all ~19216 surfaces; the old
256-grid model looked like 96%
lsta_count_equals_sprites_plus_primitives -- header counts T8aD AND PRMD,
64/64, which is what made the
count look unreliable
ratc_nesting_is_exactly_one_level -- nested records are leaves; zero
grandchildren disc-wide
mesh_consistency_disc.rs (1 ignored, deliberately)
shared_resources_decode_identically_in_every_container
The mesh test is written as the TARGET state, not a snapshot of the bug: a
resource shared by several containers must decode to the same bounds, which today
fails for 125 of 681 shared resources. Fixing the anchor scan makes it pass;
un-ignoring it is then the last step rather than a rewrite. It only compares
decodes that agree on vertex/triangle counts, so "found different geometry" stays
a separate question from "placed the same geometry differently".
All suites green: 81 lib + the disc guards, with 2 ignored.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The movie cutscene subtitle + voice pipeline, driven by the ADVERTISE_MOVIE
manifest (the authoritative movie -> subtitle -> voice index). Also flushes
several sessions of local WIP (async viewer loading, grouped-pool XBG7/hero-ship
decode, drawlog tooling). See docs/HANDOFF-movie-voice-subtitles-2026-07-19.md.
Subtitles (movie_subtitle.rs):
- Full movie->track->text chain; join multi-line captions sharing one timing
(fixes S13A dropped "Look at it father" line); overlap-safe active_cues();
Latin-1 accents preserved.
Voice (slb.rs): XACT .slb -> XMA1 RIFF; take the FIRST sub-wave bounded by its
declared data size (fixes S10-S16 alternate-take garble); list_voice_clips.
Manifest (movie_manifest.rs): parse ADVERTISE_MOVIE (0x5B983A08) for the real
movie->voice binding (not always VOICE_<movie>; e.g. hokyu -> VOICE_D_* in etc\).
Resolve the token's sound.pak path via sounds.tbl. DIRECT bindings only — the
demo-id shared-clip fallback for unbound hokyu movies was verified WRONG in-game
and reverted (unbound hokyu stay unvoiced; correct join key is an OPEN problem).
Viewer: manifest-driven voice (movie player toggle + solo button), standalone
"Voice Lines" browser, stacked caption overlay.
Tests: 46 formats-lib + 11 viewer-lib + movie_manifest/movie_subtitle/slb disc
tests; full workspace green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Revert the mis-guided "triangle STRIP" reading (a937779): XBG7 index
buffers are triangle LISTS (prim=4 in the GPU capture; stored-normal
agreement 1.000 as a list vs ~0.49 as a strip). Reinstate the
winding-consistency gate.
Crack the grouped-pool layout used by the hero ship and ~150 detailed
models. A resource's sub-meshes share one index pool (buffers 4-byte
aligned, in descriptor-marker order) followed by one vertex pool (each
vtx_count*stride, same order); the vertex pool is 4-byte aligned after
the index pool. The whole resource is derived from one anchored pivot:
ib0 = vb0 - span - pad (pad in 0..=3, alignment)
ib[i] = align4(ib[i-1] + idx_count[i-1]*2)
vb[i] = vb[i-1] + vtx_count[i-1]*stride
vb0 is found via the unit-normal vertex-run scan; the alignment is
confirmed by validating the LARGEST sub-mesh (most reliable), after
which the rest are read/validated. A single marker reduces this to the
existing adjacency anchor, so grouped generalises it.
Results (cross-checked against a Canary GPU draw-log capture of
DeltaSaber_T.xpr):
- DeltaSaber_T f001 = body + 7 detail parts = 8650 tris, every sub-mesh
0-degenerate / full-coverage / winding-agreement 1.000.
- All 19 previously-declined weapon models now decode (they hit pad 2
and/or lead with a tiny bracket that broke a markers[0] pivot). Corpus
audit: 0/146 geometry files fail (was 19 -> flat-texture in the viewer).
- Stages unchanged (single-marker path is byte-identical; grouped falls
back to the old anchor on failure). 7/7 disc tests green incl. the
strict stage quality audit; new hero_ship_grouped_pool_decodes test.
Viewer: route by count_xbg7; --only matches an exact model name (so the
neutral pose renders without the mnv*/turn180 animation poses). Fix the
albedo matcher: match a sub-model to its _col map by entity stem
(e007_bdy_01 -> e007_col) instead of a full-name prefix, lifting stage
sub-model texturing from ~25% to ~95% (the rest were flat grey). Colour
correctness (channel order/sRGB) remains a separate dynamic-RE item.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Stage containers (hidden/resource3d/Stage_S*.xpr) are collections of up to
~450 enemy/prop sub-models, not single meshes. Xbg7Model::stage_models decodes
them: each resource is a [12-byte header][index buffer][vertex buffer] block
(index count = descriptor marker, vertex count = u32 32 bytes before it) whose
on-disc offset is NOT stored, so it is located by content — one O(file) pass per
stride finds vertex-buffer starts (unit NORMAL at +12 whose previous slot isn't)
and each resource is pinned to the candidate whose indices validate and produce
non-degenerate, well-connected triangles. A connectivity gate (mean triangle
edge <= 0.28x the bbox diagonal) rejects spiky mis-anchors. ~4993 sub-models
decode across the 22 stages.
The same insight fixes the weapon single-model layout: it is [12-byte header]
[index][vertex] too, not [index][12-byte gap][vertex]. Reading indices from the
block start turned the 12 header bytes into 6 junk indices (2 leading degenerate
triangles — the recurring stray-triangle artifact) and dropped the last 6 real
indices. Skipping the header leaves vertex offsets identical (coverage unchanged
at 36/166) and corrects the triangle list. Verified on wep_00/03/04.
Adds `sylpheed-cli mesh {info,render}` — a headless software rasterizer that
writes a shaded PNG (orthographic, z-buffered, two-sided), so recovered geometry
can be verified without the GUI. Stages render as a normalised thumbnail grid;
--only filters sub-models.
Tests: stage_models_{decode,sweep,quality_audit}; docs/re updated (xbg7-mesh.md
evidence log + INDEX.md).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Dynamic-RE follow-up: capture Canary's GPU vertex-fetch + draw calls and
feed the ground truth back into the static decoder.
The GPU capture confirmed the reverse-engineered layout exactly — meshes
draw as triangle LISTs (prim=4) with pos f32x3 @0, normal f16x4 @0x0C,
uv f16x2 @0x14 — and revealed the XBG7 vertex format is NOT fixed-stride:
models omit elements (stride 20 = pos+normal, no UV; 24 = pos+normal+uv).
- mesh.rs: parse the descriptor's vertex declaration ({offset, format-code,
usage} triples; 0x2A23B9=pos f32x3, 0x1A2360=normal f16x4, 0x2C235F=uv
f16x2) to drive per-model stride + element offsets, instead of assuming
stride 24. Coverage 25 -> 36 fully-validated models (e.g. the Stage_S*
props, which are pos+normal only). Same index-range + unit-normal safety
gates; complex/mismatched layouts still declined.
- Endianness note (documented): the capture's fetch endian=k8in32 describes
the GPU's guest-memory copy, NOT the .xpr file bytes — reading the file
with k8in32 breaks the normals (|n|->1.33); naive big-endian per element
is correct (the game rearranges vertex data on load).
- tests: a coverage-regression test (>=35 models) + the existing weapon /
body-declined disc tests still pass.
Confirmed against a Canary draw-log capture; see docs/re/structures/
xbg7-mesh.md (evidence log + declaration table).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reverse-engineer the single-stream XBG7 geometry layout (clean-room: hex
inspection + geometric validation of the retail disc's
hidden/resource3d/*.xpr, no game code copied) and present models as
textured 3D meshes in the explorer.
Format (docs/re/structures/xbg7-mesh.md): XBG7 geometry resources sit
inside XPR2 containers alongside TX2D textures. For ~25 single-stream
models (weapons, simple props) the data section is a sequence of
sub-meshes, each an u16-BE triangle-list index buffer followed (after a
fixed 12-byte header) by a stride-24 vertex buffer whose declaration is
in the descriptor: POSITION f32x3 @0x00, NORMAL f16x4 @0x0C, TEXCOORD
f16x2 @0x14. Sub-mesh (vtx,idx) counts come from descriptor tuples.
The +12 vertex offset is pinned by the recovered normals being exactly
unit-length (align16 lands 4 bytes early and silently corrupts every
field). A safety gate rejects any model whose indices are out of range
or whose mean |normal| is not ~1, declining garbage (Stage_S*
placeholders, the complex multi-stream hero-ship body) rather than
mis-decoding it.
- mesh.rs: Xbg7Model::from_xpr2 -> GameMesh { positions, normals, uvs,
indices }; standalone f16->f32; unit + real-disc tests (weapon decodes
to 215v/364t with unit normals + in-range UVs; DeltaSaber body
declined).
- texture.rs: from_xpr2_index / texture_names so the viewer can pick a
model's _col albedo map.
- viewer: loose .xpr with decodable XBG7 spawns Bevy meshes (real normals,
double-sided) textured with the albedo, framed by the orbit camera; the
central egui panel goes transparent so the 3D scene shows through.
Complex multi-stream body meshes (DeltaSaber f004, other vertex layouts)
remain undecoded and are cleanly declined — next target is dynamic RE.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add three static PAK-format decoders in the Bevy-free formats crate and
present them in the explorer:
- t8ad.rs: linear 32bpp A8R8G8B8 2D texture (UI/HUD art). Dims at
0x14/0x18, header size keyed by the type field @0x1c (container-safe).
Decodes the ~85% RGBA variants; defers the rest (likely DXT) rather
than misdecoding.
- ratc.rs: nested resource bundle — lists named children by magic scan.
- lsta.rs: sprite list — walks the inline T8aD frames.
Viewer: PakContent gains T8ad/Lsta/Ratc, decoded off-thread and bounded
by an 8M-texel per-entry cap; detail views show the texture, a sprite
grid, and a child list with thumbnails. Decoded images carry a
"colours unverified" note — channel-order/sRGB stays on the dynamic-RE
backlog.
Tests: 12 new unit + 3 real-disc (T8aD >=70% decode over the hangar
pack, LSTA frames, RATC named children incl. a decodable T8aD).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Decode and present four more PAK item types in the pack browser's detail
pane. All content is decoded off-thread in build_pak_rows and attached to
each PakRow, then rendered with egui's own texture facilities — the Bevy
texture pipeline and egui's global font system are untouched.
formats crate (Bevy-free, reusable by CLI):
- ixud: parse IXUD localized-string tables as subtitle cue lists (UTF-16BE
pool, SUBTITLE header + (text, timecode) pairs -> Vec<Cue{start,end,text}>).
- font: OTF/TTF/ttcf metadata (family / faces / glyphs) via ttf-parser.
viewer:
- PakRow gains `content: PakContent` (Subtitle | Font | Png | Text | None);
classify_content fills it for IXUD / fonts / PNG / xml+text within the
existing decode budget. Font samples are pre-rasterized off-thread with
ab_glyph into an image (ImageRgba) — ab_glyph returns Option/Result at every
step, so a bad font yields no sample instead of panicking (the earlier
egui set_fonts approach crashed the app on atlas rebuild).
- draw_pak_browser dispatches on content: subtitle cue table, font metadata +
sample image, PNG image, scrollable text; else the existing IDXD detail.
Borrow-split PakView instead of cloning the (now heavy) rows each frame;
one hash-keyed egui texture cache serves both PNG and font-sample images.
Subtitle<->movie auto-matching is deferred: movie filenames don't hash to the
IXUD keys and no cross-reference table was found (the link is an internal
MSG_DEMO id), so the movie player gets no subtitle track yet.
Tests: ixud unit tests; disc tests for the real English subtitle track
(cues + timecodes), eng/deu localization, font metadata, and off-thread
rasterization of the real font.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TXCM XPR2 resources are the game's world skyboxes (BG_Acheron, BG_Hargenteen,
…). The viewer showed only face 0 as a flat 2D image; now it decodes all 6 faces
and shows a labelled grid.
Formats (Bevy-free): X360Texture::cube_faces_from_xpr2() returns a Cubemap with 6
faces in D3D9 order (+X −X +Y −Y +Z −Z), or None for ordinary 2D textures. Face
layout derived from xenia's GetGuestTextureLayout — 6 back-to-back independently-
tiled surfaces, per-face stride = tiled-surface-size aligned to the 4 KiB
subresource boundary (kTextureSubresourceAlignmentBytes). Verified against real
BG_Acheron: data_size == 6 × 0x400000, and all 6 faces decode cleanly (own
Python decode + a disc test asserting 6×4 MiB faces and face 0 == the validated
from_xpr2 green-planet decode). Extracted a shared decode_surface() helper so the
2D and cube paths are byte-identical; from_xpr2 output unchanged (re-verified).
Viewer: new SkyboxPreview resource (6 egui face textures, reusing the existing
per-format x360_texture_to_bevy_image); populated in apply_loaded_texture's TXCM
branch; freed/reset alongside the other previews (factored free_texture/
free_skybox helpers). Central panel gains a skybox branch rendering a 3-column
labelled face grid.
DEFERRED (per "do not guess, else defer"): the interactive 3D skybox. Face data +
D3D9 order are validated, but wgpu cube-sampling handedness can't be confirmed
without eyeballing the GUI — a wrong-oriented skybox is worse than the correct
labelled grid. The grid is the reliable deliverable; the 3D look-around is a
follow-up once orientation is visually confirmed.
(Background agent did the investigation/validation but was blocked from writing
files; implemented here in the main tree from its findings, independently
re-verified.)
23 formats tests + disc cubemap test pass; viewer/CLI build; face-0 export
re-verified as the green planet.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reworks the XPR2 texture path after RE against the retail disc (viewer
previews were scrambled/failed).
- de-tile: replace the naive Morton-over-32×32 approximation with the
faithful Xenos Tiled2D bank/pipe/macro-tile address formula (ported
from xenia texture_address.h). Verified correct: the Acheron backdrop
decodes to a sharp planet with its spiral storm.
- endian: undo the X360 word byte-swap per the fetch-constant endianness
field (k8in16/k8in32/k16in32) — without it BCn/ARGB data is noise.
- cubemaps: parse TXCM resources (skybox/backdrops), decoding face 0,
instead of erroring "No TX2D found".
- A8R8G8B8: correct channel order after the k8in32 swap ([A,R,G,B]→RGBA)
— the Acheron backdrop is now correctly green, not red.
- XPR files are texture PACKS; add XPR_RES_INDEX to select a resource.
- CLI `texture export` now writes real PNGs (texpresso BCn decode +
image), replacing the stub. Adds texpresso + image deps.
- tests/texture_disc.rs: integration test running the pipeline over real
disc .xpr files (28/28 parse). RE debug knobs: XPR_NO_DETILE /
XPR_NO_ENDIAN / XPR_FORCE_ENDIAN.
KNOWN ISSUE: mipmapped DXT1 textures still decode to noise — mip 0 is
not at base_address in the multi-resource packs (localized to a mip
storage-offset quirk; uncompressed + de-tile + endian are all verified).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The pak TOC key was previously an unknown hash (crc32/fnv/djb2/… ruled
out), forcing content-only lookup. Recovered it by static RE of the
retail title:
sub_824609C8 (pak lookup-by-name) dups the requested path, lowercases
it (sub_825F4F90), hashes with sub_82455C78, then binary-searches the
sorted TOC. sub_82455C78 is a per-byte Barrett-reduced polynomial hash
(modulus 0x00FFF9D7, reciprocal 0x80031493): the low 24 bits are the
modular hash, the top byte an 8-bit additive checksum of the bytes.
New `hash` module reproduces it exactly (faithful op sequence, no
textbook %). Verified against the real disc: name_hash("files.tbl") ==
0x83421153 and name_hash("eng\\weapon.tbl") == 0x900C8DCD, both present
in retail TOCs. Add PakArchive::find_by_name / read_by_name and a disc
integration test resolving eng\weapon.tbl by path (→ valid IDXD).
Note: retail paths use backslash separators (eng\weapon.tbl).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add readers for Project Sylpheed's on-disc data format so the
reimplementation can load exact ship/weapon constants straight from
the game files.
- pak.rs: PakArchive — parse the IPFB `.pak` index + concatenated
`.pNN` segments, binary-search the TOC by name-hash, and transparently
inflate the per-entry "Z1" (zlib) container. Adds flate2.
- idxd.rs: IdxdObject — parse the IDXD reflective object serialization.
The string pool stores properties value-before-key with defaulted
fields omitted; typed getters (get_f32/get_i64/get_str/get_bool) read
explicit values reliably, get_raw exposes identifier fields, and
resolved_fields() enumerates every explicit (key,value). Defaulted
fields correctly return None rather than a neighbouring key.
- sylpheed-cli: `pak list` (inventory entries + identity) and
`pak dump <hash>` (full stat sheet for one object).
Verified against the real disc (auto-skipped without it): DeltaSaber
craft (HP=1000, Acceleration=600, velocity curve 100/700/1200, Turn=100,
RadarRange=500000) and the DSaber missile (LoadingCount=144, Interval=3.00,
Mass=0.77). 10 unit + 3 integration tests pass.
Not yet decoded: defaulted fields (many ratios, the *Count family) whose
values come from schema defaults or the hash-keyed binary node region.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
