Decode Project Sylpheed's IDXD data model into plain, cloneable structs
(from GP_MAIN_GAME_E.pak; the D/F/I/J/S paks are localized dups):
- Weapon (121), CraftUnit (89, with the AI flight model in .fields),
Vessel (23, structural component counts), PlayerConfig (24) — the
full combat balance.
- Character (68 — faction + portrait set), Stage (29 missions:
location, phases, and the per-stage resource tables it wires up).
- Generic Record / load_records(schema) reads any of the 105 IDXD
schemas without a bespoke struct.
Each blob = one entity; token[0] names the table, fields are
value-before-key. Named Option<> fields for the common stats + a full
`fields` map; defaulted fields stay None (they're code-resident, not on
disc — honest blanks, never guessed). 8 tests against the real disc.
Examples: iso_map / deep_map census the ISO's formats and IDXD schemas;
dm_extract / dm_rows / dm_roster / mission_map dump the decoded data.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 13 manifest-unbound hokyu (resupply) cutscenes reuse 5 recordings
(VOICE_D_450..454). The category guess (ship LS/DS × source carrier/
tanker) was wrong for some. The real selector is the cutscene's subtitle
demo-id: 600→450, 601→451, 602→452, 603→453, 604→454 — derived from the
5 BOUND hokyu (each carries both a demo-id and a VOICETRACK), so it
self-validates. Fixes hokyu_LS_s11A/s15A (→451, were wrongly 450) and
hokyu_LS_s24A/s27A (→silent, no voice cue). User-confirmed all correct.
hokyu_voice_token() builds the demo→token map from the bound hokyu and
looks up the target movie's demo-id; replaces the category fallback.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Cracked how the game maps a movie to its voice track and reimplemented it,
fixing wrong/short voices for RT and hokyu cutscenes.
## Mechanism (RE'd from a Canary file-I/O trace, user-confirmed by ear)
The movie voices are ONE continuous XMA stream, physically chunked into
`<lang>\Movie\VOICE_*.slb` (and `\etc\VOICE_D_*.slb`) sound.pak TOC entries
whose boundaries do NOT align with the cutscene cues. A single cue routinely
spans two `.slb` chunks, so a `.slb` does not necessarily hold the track its
name claims (they are off-by-one vs the cues). Each cue ends at an inline
trailer descriptor `(sound_id:u32be, 0x11, …)` whose id repeats at +0x800.
Resolution chain (all derivable on-disc, no guest-code RE):
1. movie -> cue token (ADVERTISE_MOVIE manifest VOICETRACK)
2. cue token -> sound-id (master sound registry, tables.pak
a2c8c185=eng / b04238e0=jpn, schema 0x13cb84ba)
3. sound-id -> [start,end) scan the stream for trailer(id) and its
predecessor; cue audio = the bytes between,
read continuously across chunk boundaries.
Validated: decoded voice length matches each movie within ~0.4s, including
cross-boundary cues (ADV/RT01A/RT01B/RT01C/S00A/S01A + the 5 bound hokyu).
## Changes
- sylpheed-formats/src/movie_voice.rs (new): registry_voice_ids(),
find_descriptor(), find_descriptor_before() (gap-tolerant predecessor).
Unit-tested.
- viewer iso_loader.rs: resolve_movie_voice_region() + decode_voice_region()
(continuous read via existing read_segment_range + to_xma_riffs). Voice/
audio requests now region-first; a `\Movie\` token that fails region stays
SILENT (never plays the wrong off-by-one chunk). Removed the old
movie_is_demo_based suppress hack.
- Anchor tries subdirs Movie/etc/Voice so hokyu `\etc\VOICE_D_*` resolve too.
- Examples resolve_all/validate_cues/hokyu_final document + validate the pipeline.
## Status
- RT / S / ADV cutscene voices: CORRECT (user-confirmed).
- 5 manifest-bound hokyu (VOICE_D_450-454): CORRECT (user-confirmed).
## OPEN (handoff)
13 of 18 hokyu movies have NO manifest VOICETRACK; the game reuses the 5
recordings across stages via a code rule. `hokyu_fallback_token()` currently
guesses by category (ship LS/DS x source carrier-A/tanker-H: LS/A->450,
LS/H->453, DS/A->452, DS/H->454). USER REPORTS THIS IS SOMETIMES WRONG.
- LS/carrier has two takes (450 from s02A, 451 from s09A); the early-vs-late
split is unknown — unbound LS/A currently all default to 450.
- Definitive fix = Canary `--phase_a_fileio_only` file.read trace of an unbound
hokyu (e.g. hokyu_LS_s03A) to see which VOICE_D the game actually streams,
then encode the real rule. Same method that cracked the RT mapping.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The movie manifest (tables.pak, schema 0x067025b9) is the game's authoritative mission -> phase -> cutscene table. Previously we only scraped VOICE_/SUBTITLE_/.wmv strings by prefix; now we decode the real two-array (slot-keys, then value-records) structure.
MovieEntry gains slot, kind (System/Intro/Phase/PhaseEnd/Supply), mission, phase, subtitle, and telop (the previously-ignored .prt on-screen text overlay). Valueless slots (stages with no resupply video) are recovered without decoding the binary node region, via the fully-derivable MS<NN><X> -> S<NN><X> anchors.
Backward compatible: movie/voice_token and resolve_voice_entry/voice_token/is_manifest are unchanged. Validated end-to-end on the real disc: 101 entries (104 slots - 3 gaps), typed System=6/Intro=27/Phase=32/PhaseEnd=18/Supply=18. examples/manifest_map.rs dumps the full map.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
list_voice_clips now enumerates every spoken-line category so the standalone audio player covers them all — bound movie voices (\Movie\), in-mission radio (\Voice\, \etc\), and mission briefings (\Briefing\, whose BR<NN>_<MM> names lack "VOICE"); music (BGM_*) stays excluded.
Viewer voice-library fixes: invalidate the library when a new game source loads, and don't latch an empty result as "loaded" (sounds.tbl always has thousands of clips, so empty = a failed read) so re-opening the menu retries.
Voice decode reverted to the first sub-wave (base behaviour): a naive multi-sub-wave concat produced the wrong track for RT movies. The per-sub-wave splitter (to_xma_riffs) is kept + documented for the eventual real fix, which needs ground-truth playback instrumentation to resolve segments-vs-takes-vs-timecode placement.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The movie cutscene subtitle + voice pipeline, driven by the ADVERTISE_MOVIE
manifest (the authoritative movie -> subtitle -> voice index). Also flushes
several sessions of local WIP (async viewer loading, grouped-pool XBG7/hero-ship
decode, drawlog tooling). See docs/HANDOFF-movie-voice-subtitles-2026-07-19.md.
Subtitles (movie_subtitle.rs):
- Full movie->track->text chain; join multi-line captions sharing one timing
(fixes S13A dropped "Look at it father" line); overlap-safe active_cues();
Latin-1 accents preserved.
Voice (slb.rs): XACT .slb -> XMA1 RIFF; take the FIRST sub-wave bounded by its
declared data size (fixes S10-S16 alternate-take garble); list_voice_clips.
Manifest (movie_manifest.rs): parse ADVERTISE_MOVIE (0x5B983A08) for the real
movie->voice binding (not always VOICE_<movie>; e.g. hokyu -> VOICE_D_* in etc\).
Resolve the token's sound.pak path via sounds.tbl. DIRECT bindings only — the
demo-id shared-clip fallback for unbound hokyu movies was verified WRONG in-game
and reverted (unbound hokyu stay unvoiced; correct join key is an OPEN problem).
Viewer: manifest-driven voice (movie player toggle + solo button), standalone
"Voice Lines" browser, stacked caption overlay.
Tests: 46 formats-lib + 11 viewer-lib + movie_manifest/movie_subtitle/slb disc
tests; full workspace green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Revert the mis-guided "triangle STRIP" reading (a937779): XBG7 index
buffers are triangle LISTS (prim=4 in the GPU capture; stored-normal
agreement 1.000 as a list vs ~0.49 as a strip). Reinstate the
winding-consistency gate.
Crack the grouped-pool layout used by the hero ship and ~150 detailed
models. A resource's sub-meshes share one index pool (buffers 4-byte
aligned, in descriptor-marker order) followed by one vertex pool (each
vtx_count*stride, same order); the vertex pool is 4-byte aligned after
the index pool. The whole resource is derived from one anchored pivot:
ib0 = vb0 - span - pad (pad in 0..=3, alignment)
ib[i] = align4(ib[i-1] + idx_count[i-1]*2)
vb[i] = vb[i-1] + vtx_count[i-1]*stride
vb0 is found via the unit-normal vertex-run scan; the alignment is
confirmed by validating the LARGEST sub-mesh (most reliable), after
which the rest are read/validated. A single marker reduces this to the
existing adjacency anchor, so grouped generalises it.
Results (cross-checked against a Canary GPU draw-log capture of
DeltaSaber_T.xpr):
- DeltaSaber_T f001 = body + 7 detail parts = 8650 tris, every sub-mesh
0-degenerate / full-coverage / winding-agreement 1.000.
- All 19 previously-declined weapon models now decode (they hit pad 2
and/or lead with a tiny bracket that broke a markers[0] pivot). Corpus
audit: 0/146 geometry files fail (was 19 -> flat-texture in the viewer).
- Stages unchanged (single-marker path is byte-identical; grouped falls
back to the old anchor on failure). 7/7 disc tests green incl. the
strict stage quality audit; new hero_ship_grouped_pool_decodes test.
Viewer: route by count_xbg7; --only matches an exact model name (so the
neutral pose renders without the mnv*/turn180 animation poses). Fix the
albedo matcher: match a sub-model to its _col map by entity stem
(e007_bdy_01 -> e007_col) instead of a full-name prefix, lifting stage
sub-model texturing from ~25% to ~95% (the rest were flat grey). Colour
correctness (channel order/sRGB) remains a separate dynamic-RE item.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The weapon/prop models (rou_fxxx_wep_nn.xpr) rendered with triangular holes in
the hull and misplaced/extra spikes. Root cause: XBG7 sub-mesh index buffers are
triangle STRIPS, but the decoder read them as triangle LISTS — a strip of N
indices is N-2 triangles, a list is only N/3, so ~2/3 of every hull was missing
(the holes) and each list-triple of strip data spanned unrelated vertices (the
spikes). The tell: triangles-per-vertex was 0.5-1.0 with 0 unreferenced vertices
(a closed surface needs ~2.0).
- mesh.rs: expand_triangle_strip() converts strip -> list with alternating
winding, skipping degenerate triangles (repeated index = strip restart).
Applied in both from_xpr2 (weapons) and read_pool_mesh (stage sub-models).
All 71 weapon submeshes now have healthy ratios; hulls fill in (wep_03 cannon,
wep_04 pod, wep_11, wep_37 verified coherent).
- Module doc updated (strip, not list).
- sylpheed-cli: `mesh info` reports per-submesh degenerate/unref/spanning-triangle
counts; `mesh render` gains --dist (camera zoom). XMESHDBG env dumps the
descriptor index markers.
Known residual: an index buffer may concatenate several strips with no degenerate
bridge, leaving ~2 spanning "spike" triangles at each restart (index jump to a new
vertex region) — <1% of tris on most models. Decoding the restart mechanism is a
follow-up (a blanket spanning-filter is unsafe: clean models have legit elongated
tip triangles).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-up to the 256×256-tiling reversal — the residual boundary seams on
≥2×2-tile textures are gone. Decoding the file header revealed the exact
layout (no oracle capture needed):
0x00 44 base header
0x1c 4 tile count = ceil(w/256) * ceil(h/256)
0x2c tiles*4 BE-u32 absolute offset of each row-major 256×256 tile
<off> 16 per-tile header, then tile_w*tile_h*4 A8R8G8B8 pixels
The old seams came from ignoring the offset table and the 16-byte per-tile
header (contiguous-packing drifted 16 bytes per tile). Small textures decoded
before only by luck: one tile puts pixels at 44+4+16 = 64, the old type-1
"header size". Now the 8AX title background, the prselect_win1 window frame,
and preff04 all decode pixel-perfect (verified against the title screen).
- t8ad.rs: parse() walks the offset table; dropped detile_256 + the
header-size-by-type table. Non-tilecount entries (field != ceil*ceil) return
None (likely DXT/other, deferred). New multi-tile round-trip test.
- sylpheed-cli: pak textures decodes via parse(); XDUMPHDR=1 dumps the base
header + offset table for RE. Removed the now-obsolete XTILE/XSCORE/XDESTRIP
experiment knobs and the reconstruct_tiles/TV-sweep helpers.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Verified against the running game (freeze-fixed Canary title screen) that
T8aD colours are correct (no R↔B swap) and that wide textures were garbled
by an unreversed tile layout, not a colour bug.
Reversed the layout: large T8aD are stored as 256×256 raster tiles in
row-major order, each tile raster internally, edge tiles clipped to the
image (payload is exactly w*h*4, no padding). Surfaces ≤256px wide are a
single tile column, identical to plain linear — which is why small UI
textures always decoded correctly.
- t8ad.rs: add detile_256() + apply in parse(). Single tile-row / single
tile-column textures now decode exactly (ptcopyright, ptbtn, ptlogo1 =
clean "PROJECT" logo, previously pure noise). Known residual: ≥2×2-tile
textures (>256 in both dims, e.g. the 8AX background) come out coherent
but with boundary seams — the multi-tile order is a subtle swizzle, TBD.
- sylpheed-cli: new `pak textures <pak> <out>` command — decodes every
T8aD (direct, RATC-nested, LSTA frames) to PNG for A/B against the game.
Env debug knobs for layout RE: XTILE/XDESTRIP/XDETILE_W/XREINTERPRET_PITCH
/XSCORE (TV-ranked tile sweep), plus --verbose size classification.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Route stage containers to stage_models (decode both single + stage paths, keep
whichever yields more geometry) — fixes stages showing the dummy-root "black
cube". Present a stage's sub-models as a thumbnail grid: each recentred and
uniformly scaled to a fixed cell so a 1-unit prop and a 3000-unit structure are
equally visible; skybox-plane / stray-anchor models (>20000u) are culled. Each
sub-model is textured by matching its name to the container's `_col` albedo.
Fix the orbit camera: zoom radius was hard-capped at 50u with fixed clip planes,
trapping the camera inside multi-thousand-unit stages. Zoom limits and near/far
planes now scale to the framed model.
Adds docs/HANDOFF-stage-mesh-2026-07-12.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stage containers (hidden/resource3d/Stage_S*.xpr) are collections of up to
~450 enemy/prop sub-models, not single meshes. Xbg7Model::stage_models decodes
them: each resource is a [12-byte header][index buffer][vertex buffer] block
(index count = descriptor marker, vertex count = u32 32 bytes before it) whose
on-disc offset is NOT stored, so it is located by content — one O(file) pass per
stride finds vertex-buffer starts (unit NORMAL at +12 whose previous slot isn't)
and each resource is pinned to the candidate whose indices validate and produce
non-degenerate, well-connected triangles. A connectivity gate (mean triangle
edge <= 0.28x the bbox diagonal) rejects spiky mis-anchors. ~4993 sub-models
decode across the 22 stages.
The same insight fixes the weapon single-model layout: it is [12-byte header]
[index][vertex] too, not [index][12-byte gap][vertex]. Reading indices from the
block start turned the 12 header bytes into 6 junk indices (2 leading degenerate
triangles — the recurring stray-triangle artifact) and dropped the last 6 real
indices. Skipping the header leaves vertex offsets identical (coverage unchanged
at 36/166) and corrects the triangle list. Verified on wep_00/03/04.
Adds `sylpheed-cli mesh {info,render}` — a headless software rasterizer that
writes a shaded PNG (orthographic, z-buffered, two-sided), so recovered geometry
can be verified without the GUI. Stages render as a normalised thumbnail grid;
--only filters sub-models.
Tests: stage_models_{decode,sweep,quality_audit}; docs/re updated (xbg7-mesh.md
evidence log + INDEX.md).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Dynamic-RE follow-up: capture Canary's GPU vertex-fetch + draw calls and
feed the ground truth back into the static decoder.
The GPU capture confirmed the reverse-engineered layout exactly — meshes
draw as triangle LISTs (prim=4) with pos f32x3 @0, normal f16x4 @0x0C,
uv f16x2 @0x14 — and revealed the XBG7 vertex format is NOT fixed-stride:
models omit elements (stride 20 = pos+normal, no UV; 24 = pos+normal+uv).
- mesh.rs: parse the descriptor's vertex declaration ({offset, format-code,
usage} triples; 0x2A23B9=pos f32x3, 0x1A2360=normal f16x4, 0x2C235F=uv
f16x2) to drive per-model stride + element offsets, instead of assuming
stride 24. Coverage 25 -> 36 fully-validated models (e.g. the Stage_S*
props, which are pos+normal only). Same index-range + unit-normal safety
gates; complex/mismatched layouts still declined.
- Endianness note (documented): the capture's fetch endian=k8in32 describes
the GPU's guest-memory copy, NOT the .xpr file bytes — reading the file
with k8in32 breaks the normals (|n|->1.33); naive big-endian per element
is correct (the game rearranges vertex data on load).
- tests: a coverage-regression test (>=35 models) + the existing weapon /
body-declined disc tests still pass.
Confirmed against a Canary draw-log capture; see docs/re/structures/
xbg7-mesh.md (evidence log + declaration table).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reverse-engineer the single-stream XBG7 geometry layout (clean-room: hex
inspection + geometric validation of the retail disc's
hidden/resource3d/*.xpr, no game code copied) and present models as
textured 3D meshes in the explorer.
Format (docs/re/structures/xbg7-mesh.md): XBG7 geometry resources sit
inside XPR2 containers alongside TX2D textures. For ~25 single-stream
models (weapons, simple props) the data section is a sequence of
sub-meshes, each an u16-BE triangle-list index buffer followed (after a
fixed 12-byte header) by a stride-24 vertex buffer whose declaration is
in the descriptor: POSITION f32x3 @0x00, NORMAL f16x4 @0x0C, TEXCOORD
f16x2 @0x14. Sub-mesh (vtx,idx) counts come from descriptor tuples.
The +12 vertex offset is pinned by the recovered normals being exactly
unit-length (align16 lands 4 bytes early and silently corrupts every
field). A safety gate rejects any model whose indices are out of range
or whose mean |normal| is not ~1, declining garbage (Stage_S*
placeholders, the complex multi-stream hero-ship body) rather than
mis-decoding it.
- mesh.rs: Xbg7Model::from_xpr2 -> GameMesh { positions, normals, uvs,
indices }; standalone f16->f32; unit + real-disc tests (weapon decodes
to 215v/364t with unit normals + in-range UVs; DeltaSaber body
declined).
- texture.rs: from_xpr2_index / texture_names so the viewer can pick a
model's _col albedo map.
- viewer: loose .xpr with decodable XBG7 spawns Bevy meshes (real normals,
double-sided) textured with the albedo, framed by the orbit camera; the
central egui panel goes transparent so the 3D scene shows through.
Complex multi-stream body meshes (DeltaSaber f004, other vertex layouts)
remain undecoded and are cleanly declined — next target is dynamic RE.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add three static PAK-format decoders in the Bevy-free formats crate and
present them in the explorer:
- t8ad.rs: linear 32bpp A8R8G8B8 2D texture (UI/HUD art). Dims at
0x14/0x18, header size keyed by the type field @0x1c (container-safe).
Decodes the ~85% RGBA variants; defers the rest (likely DXT) rather
than misdecoding.
- ratc.rs: nested resource bundle — lists named children by magic scan.
- lsta.rs: sprite list — walks the inline T8aD frames.
Viewer: PakContent gains T8ad/Lsta/Ratc, decoded off-thread and bounded
by an 8M-texel per-entry cap; detail views show the texture, a sprite
grid, and a child list with thumbnails. Decoded images carry a
"colours unverified" note — channel-order/sRGB stays on the dynamic-RE
backlog.
Tests: 12 new unit + 3 real-disc (T8aD >=70% decode over the hangar
pack, LSTA frames, RATC named children incl. a decodable T8aD).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
inner_format_label mapped only IDXD/xml/printable-tag magics; fonts and
images fell through to a bare hex label (00010000, 89504E47, …). Recognize
the known binary signatures and return friendly short tags so the pak
browser's format column reads them: sfnt/true/typ1 → "ttf", OTTO → "otf",
ttcf → "ttc", PNG → "png", plus RIFF/DDS. Still-unidentified magics keep the
hex fallback. Shared by the CLI `pak list` and the GUI browser.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Decode and present four more PAK item types in the pack browser's detail
pane. All content is decoded off-thread in build_pak_rows and attached to
each PakRow, then rendered with egui's own texture facilities — the Bevy
texture pipeline and egui's global font system are untouched.
formats crate (Bevy-free, reusable by CLI):
- ixud: parse IXUD localized-string tables as subtitle cue lists (UTF-16BE
pool, SUBTITLE header + (text, timecode) pairs -> Vec<Cue{start,end,text}>).
- font: OTF/TTF/ttcf metadata (family / faces / glyphs) via ttf-parser.
viewer:
- PakRow gains `content: PakContent` (Subtitle | Font | Png | Text | None);
classify_content fills it for IXUD / fonts / PNG / xml+text within the
existing decode budget. Font samples are pre-rasterized off-thread with
ab_glyph into an image (ImageRgba) — ab_glyph returns Option/Result at every
step, so a bad font yields no sample instead of panicking (the earlier
egui set_fonts approach crashed the app on atlas rebuild).
- draw_pak_browser dispatches on content: subtitle cue table, font metadata +
sample image, PNG image, scrollable text; else the existing IDXD detail.
Borrow-split PakView instead of cloning the (now heavy) rows each frame;
one hash-keyed egui texture cache serves both PNG and font-sample images.
Subtitle<->movie auto-matching is deferred: movie filenames don't hash to the
IXUD keys and no cross-reference table was found (the link is an internal
MSG_DEMO id), so the movie player gets no subtitle track yet.
Tests: ixud unit tests; disc tests for the real English subtitle track
(cues + timecodes), eng/deu localization, font metadata, and off-thread
rasterization of the real font.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add an in-explorer video player for the dat/movie/*.wmv cutscenes.
Decode via the ffmpeg/ffprobe CLI (no libav linking): video streams from an
`ffmpeg -f rawvideo -pix_fmt rgba` pipe read into fixed w*h*4 frame chunks over
a bounded channel (paused/behind => ffmpeg back-pressures on its pipe); one
reused Image is overwritten in place each tick. Audio is best-effort: the track
is pre-decoded to a temp stereo WAV and played through a rodio Sink (volume /
play / pause / seek-via-skip_duration for free). Native-only under
cfg(not(wasm32)); VideoPreview registers unconditionally so the UI compiles for
wasm. Mirrors the .pak load pipeline (FileSelected -> bg thread ->
IsoLoaderMsg::VideoLoaded -> apply_loaded_video -> advance_video_playback), with
a single free_video teardown wired into every other viewer's reset.
Controls:
- play/pause button, click-on-frame toggle, Space / arrow (+-10s) shortcuts
- seekable timeline with mm:ss labels; volume slider (greyed when no audio)
- live scrubbing: a coalescing one-shot `ffmpeg -ss -frames:v 1` grabber always
jumps to the knob's latest position, so dragging shows the frame under the
knob in real time; that grabbed frame also bridges the gap after a commit-seek
so there's no black flash while the streaming decoder catches up.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TXCM XPR2 resources are the game's world skyboxes (BG_Acheron, BG_Hargenteen,
…). The viewer showed only face 0 as a flat 2D image; now it decodes all 6 faces
and shows a labelled grid.
Formats (Bevy-free): X360Texture::cube_faces_from_xpr2() returns a Cubemap with 6
faces in D3D9 order (+X −X +Y −Y +Z −Z), or None for ordinary 2D textures. Face
layout derived from xenia's GetGuestTextureLayout — 6 back-to-back independently-
tiled surfaces, per-face stride = tiled-surface-size aligned to the 4 KiB
subresource boundary (kTextureSubresourceAlignmentBytes). Verified against real
BG_Acheron: data_size == 6 × 0x400000, and all 6 faces decode cleanly (own
Python decode + a disc test asserting 6×4 MiB faces and face 0 == the validated
from_xpr2 green-planet decode). Extracted a shared decode_surface() helper so the
2D and cube paths are byte-identical; from_xpr2 output unchanged (re-verified).
Viewer: new SkyboxPreview resource (6 egui face textures, reusing the existing
per-format x360_texture_to_bevy_image); populated in apply_loaded_texture's TXCM
branch; freed/reset alongside the other previews (factored free_texture/
free_skybox helpers). Central panel gains a skybox branch rendering a 3-column
labelled face grid.
DEFERRED (per "do not guess, else defer"): the interactive 3D skybox. Face data +
D3D9 order are validated, but wgpu cube-sampling handedness can't be confirmed
without eyeballing the GUI — a wrong-oriented skybox is worse than the correct
labelled grid. The grid is the reliable deliverable; the 3D look-around is a
follow-up once orientation is visually confirmed.
(Background agent did the investigation/validation but was blocked from writing
files; implemented here in the main tree from its findings, independently
re-verified.)
23 formats tests + disc cubemap test pass; viewer/CLI build; face-0 export
re-verified as the green planet.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Four small UX fixes to the asset explorer:
1. Remove the no-op "View" menu (Textures/Meshes/Audio toggles + Wireframe
checkbox) — none were wired to anything. Dropped ViewMode + the current_mode/
wireframe fields from ViewerState (nothing read them); the central panel
already dispatches on which preview resource is populated.
2. Loading indicator — clicking a drawer item now flips FileBrowserState.loading
and the central panel shows a centered spinner + "Loading <name>…" until the
background read/decode applies, instead of leaving the previous item on screen
with no feedback. Cleared in poll_loader_channel on FileLoaded/PakLoaded/
Error/Cancelled.
3. Drawer shows only the .pak index, not its .pNN data segments — the segments
are the pack's body (loaded when the .pak is opened), so listing them (with no
preview) was noise. Dropped the virtual "<stem>.pak" folder too; the .pak now
sits as a normal leaf in its real directory.
4. Bottom status bar no longer overflows — horizontal_wrapped, only the source's
final path component (full path on hover), file count, and a right-aligned,
shortened controls hint. Removed the mode label.
Viewer + formats build; all tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Roadmap #3. The explorer can now open IPFB data packs — the game's ship/weapon/
mission definition tables. Selecting a `.pak` shows a master-detail browser:
entry list (hash · inner-format · identity) on the left, the selected IDXD
object's schema + explicit-field property table on the right.
Load path: a `.pak` selection reads the index plus its sibling `.pNN` segments
(ISO reader or extracted dir, probing p00.. until the first gap), assembles via
PakArchive::from_parts, and builds owned PakRow/PakDetail entirely off-thread —
so the UI holds only plain data (no borrow of a PakArchive, WASM-safe). Two
decode caps (64 MiB cumulative, 16 MiB/entry) keep a huge sound.pak from hanging;
over-budget entries show as "(not decoded)".
Wiring mirrors the texture path: new IsoLoaderMsg::PakLoaded → PendingPak staging
→ apply_pak. A shared reset (free GPU handle + clear texture/text/pak previews)
runs in both apply systems, so exactly one viewer is active per selection and the
prior GPU texture is always freed when switching modes.
Reuse over duplication: the CLI's inner_label / idxd_identity move into the
formats crate as pak::inner_format_label and IdxdObject::identity (bodies
verbatim → CLI output unchanged), now shared by CLI and GUI.
PakView registered unconditionally (wasm-safe; population native-only). ViewMode
unchanged — the central panel dispatches on the populated resource. Workspace
builds; 22 formats tests + all crate tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Explorer drawer was a flat list of all 353 disc files and only textures had a
preview. Two additions:
1. Directory tree — the left drawer is now a collapsible `📁` tree, rebuilt each
frame from the path list (ui.rs: build_tree / split_for_tree / render_dir).
Filtering narrows leaves and auto-expands matches; `.p00…p04` data segments
nest under a virtual `<stem>.pak` node. Selection still keys off the original
file index, so the loader pipeline is untouched.
2. Plain-text viewer — config.ini and any file that sniffs as text now show
read-only, selectable, monospace content with a Wrap toggle and an encoding
label. Detection/decoding live in the Bevy-free formats crate:
vfs::is_probably_text + vfs::decode_text (BOM-aware UTF-8/UTF-16LE/BE, BOM-less
UTF-16LE heuristic, NUL-reject + 95%-printable fallback) — pure std, WASM-safe,
unit-tested (6 new tests), and reused by the CLI sniffer (new cyan `txt`).
New TextPreview resource (registered unconditionally so it exists on wasm);
populated in apply_loaded_texture with a 2 MiB char-boundary-safe cap. Also fixes
a latent bug: switching from a texture to a non-texture left stale preview state —
both texture and text state now reset on every selection.
ViewerState gains text_wrap. Formats tests: 22 pass; workspace compiles.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The dword-swap that fixes DXT1 is a property of the BC1 COLOUR block, not of
BCn in general. RE on a DXT5A (BC4) mask showed the alpha block is byte-indexed
and wants the endian swap ALONE — dword-swapping it makes it worse. The blanket
swap from the previous commit was therefore breaking the (common) DXT5A/DXN
alpha & normal masks.
Now targeted by format:
- DXT1 : swap dwords of each 8-byte colour block (verified).
- DXT2_3 / DXT4_5 : swap only the colour half (bytes 8..16) of each 16-byte
block; alpha half endian-only. TENTATIVE — these formats
are near-absent in the game assets, so unvalidated against
a clear image.
- DXT5A / DXN : no dword swap (endian only).
DXT1 export re-verified clean (weapon skins); 16 formats tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
DXT1 (108/137 textures in Base.xpr — the bulk of picture assets) decoded to
noise. Root cause found by RE: Xbox 360 BCn blocks store the two 32-bit words
of each 64-bit sub-block in the opposite order to the PC/DDS layout — colour
endpoints live in the HIGH dword, indices in the low one. The endian field
(k8in16) only fixes byte order within the 16-bit words; it does not reorder the
dwords, so without this every DXT texture transposed endpoints/indices → noise.
Isolation that led here:
- de-tile proven correct for bpb=8 (coherent per-block signature map; the
linear read is scrambled) — same faithful Xenos Tiled2D as the verified
bpb=4 ARGB path (green Acheron backdrop).
- inspecting a smooth region, coherent colour endpoints appeared only in
bytes[4..8], with the high-entropy indices in bytes[0..4].
Fix: swap_bc_block_dwords() swaps the two dwords within each 64-bit unit after
the byte-level endian swap, for every BCn format. Verified in the real Rust CLI:
weapon skins (rou_f001_wep_*) now decode to clean, recognisable images.
Viewer: DXT1 is fixed transparently (from_xpr2 feeds tex.data straight to the
GPU as Bc1). Also corrected the uncompressed path — post-swap k_8_8_8_8 is
[A,R,G,B]; reorder to [R,G,B,A] and upload as Rgba8UnormSrgb (was Bgra8, wrong).
Knob: XPR_NO_BC_DWORD_SWAP disables the swap for A/B validation.
KNOWN REMAINING: 16-byte blocks (BC2/DXT3, BC3/DXT5) and BC4/BC5 (DXT5A/DXN)
still need their alpha+colour half layout worked out — they decode to noise for
now. DXT1 + uncompressed + cubemaps are correct.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reworks the XPR2 texture path after RE against the retail disc (viewer
previews were scrambled/failed).
- de-tile: replace the naive Morton-over-32×32 approximation with the
faithful Xenos Tiled2D bank/pipe/macro-tile address formula (ported
from xenia texture_address.h). Verified correct: the Acheron backdrop
decodes to a sharp planet with its spiral storm.
- endian: undo the X360 word byte-swap per the fetch-constant endianness
field (k8in16/k8in32/k16in32) — without it BCn/ARGB data is noise.
- cubemaps: parse TXCM resources (skybox/backdrops), decoding face 0,
instead of erroring "No TX2D found".
- A8R8G8B8: correct channel order after the k8in32 swap ([A,R,G,B]→RGBA)
— the Acheron backdrop is now correctly green, not red.
- XPR files are texture PACKS; add XPR_RES_INDEX to select a resource.
- CLI `texture export` now writes real PNGs (texpresso BCn decode +
image), replacing the stub. Adds texpresso + image deps.
- tests/texture_disc.rs: integration test running the pipeline over real
disc .xpr files (28/28 parse). RE debug knobs: XPR_NO_DETILE /
XPR_NO_ENDIAN / XPR_FORCE_ENDIAN.
KNOWN ISSUE: mipmapped DXT1 textures still decode to noise — mip 0 is
not at base_address in the multi-resource packs (localized to a mip
storage-offset quirk; uncompressed + de-tile + endian are all verified).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add hash::recover_toc_name + TOC_NAME_SCHEMES (confirmed path schemes:
unit\<ID>.tbl, weapon\<ID>.tbl, message\<ID>.tbl, effect\<ID>.tbl,
<name>.tbl) that reproduce an entry's original backslash path from its
internal identity string via the recovered name-hash.
`pak list` now extracts identifier candidates from each IDXD entry
(ID/Name/Model + pool tokens) and prints the resolved path when a scheme
matches — e.g. unit\UN_f001_TCAF_DeltaSaber_T.tbl,
weapon\Weapon_…_Missile.tbl, message\CharacterCARL.tbl — plus a
name-resolved count. 308/1004 resolved on GP_MAIN_GAME_E; the remainder
use deeper cross-referenced paths (deferred).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The pak TOC key was previously an unknown hash (crc32/fnv/djb2/… ruled
out), forcing content-only lookup. Recovered it by static RE of the
retail title:
sub_824609C8 (pak lookup-by-name) dups the requested path, lowercases
it (sub_825F4F90), hashes with sub_82455C78, then binary-searches the
sorted TOC. sub_82455C78 is a per-byte Barrett-reduced polynomial hash
(modulus 0x00FFF9D7, reciprocal 0x80031493): the low 24 bits are the
modular hash, the top byte an 8-bit additive checksum of the bytes.
New `hash` module reproduces it exactly (faithful op sequence, no
textbook %). Verified against the real disc: name_hash("files.tbl") ==
0x83421153 and name_hash("eng\\weapon.tbl") == 0x900C8DCD, both present
in retail TOCs. Add PakArchive::find_by_name / read_by_name and a disc
integration test resolving eng\weapon.tbl by path (→ valid IDXD).
Note: retail paths use backslash separators (eng\weapon.tbl).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add readers for Project Sylpheed's on-disc data format so the
reimplementation can load exact ship/weapon constants straight from
the game files.
- pak.rs: PakArchive — parse the IPFB `.pak` index + concatenated
`.pNN` segments, binary-search the TOC by name-hash, and transparently
inflate the per-entry "Z1" (zlib) container. Adds flate2.
- idxd.rs: IdxdObject — parse the IDXD reflective object serialization.
The string pool stores properties value-before-key with defaulted
fields omitted; typed getters (get_f32/get_i64/get_str/get_bool) read
explicit values reliably, get_raw exposes identifier fields, and
resolved_fields() enumerates every explicit (key,value). Defaulted
fields correctly return None rather than a neighbouring key.
- sylpheed-cli: `pak list` (inventory entries + identity) and
`pak dump <hash>` (full stat sheet for one object).
Verified against the real disc (auto-skipped without it): DeltaSaber
craft (HP=1000, Acceleration=600, velocity curve 100/700/1200, Turn=100,
RadarRange=500000) and the DSaber missile (LoadingCount=144, Interval=3.00,
Mass=0.77). 10 unit + 3 integration tests pass.
Not yet decoded: defaulted fields (many ratios, the *Count family) whose
values come from schema defaults or the hash-keyed binary node region.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
