22 Commits

Author SHA1 Message Date
MechaCat02
8ed8c85f18 [WIP] Audio codec probe/detection + CLI audio-info + viewer audio panel
Snapshot of local WIP before rebasing onto the movie/voice remote work.
- audio.rs: AudioCodec enum, AudioInfo::probe (RIFF/WAVE parse, raw-XMA2
  Shannon-entropy heuristic looks_like_raw_xma2), from_wav, riff_data_span.
- cli/main.rs: `audio-info` subcommand (cmd_audio_info).
- viewer ui.rs/iso_loader.rs: draw_audio_detail panel wiring.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 20:42:48 +02:00
MechaCat02
3e1040b472 [texture] Present Canary GPUTEXTUREFORMAT names + metadata; handle XPR5
- Port xenia-canary's complete 64-entry texture-format table (xenos.h enum +
  texture_info_formats.inl) into GPU_FORMATS + gpu_format_name()/desc() and
  X360TextureFormat::gpu_name()/desc().
- Viewer + CLI `texture info` now show the canonical k_… name, bpp and
  compressed flag (e.g. "k_DXT1 (Dxt1) · 4 bpp, compressed"); UnsupportedFormat
  now names the format instead of a bare hex code.
- Detect non-XPR2 containers up front: the game ships one XPR5 file
  (Common.xpr) alongside 165 XPR2 — report UnsupportedContainer("XPR5")
  cleanly instead of a "bad magic" parse error.
- Add a table-integrity test (index==code, spot-checks, decodable-variant
  consistency).

Static-RE scan of the 166 resource3d XPRs: only k_DXT1 (145), k_8_8_8_8 (19),
k_DXT5A (1) are used — all already decoded, so this is presentation, not new
decoders.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 21:09:55 +02:00
MechaCat02
390c67cb61 [viewer] Surface recovered TOC names in the PAK browser + refresh RE panel
- Move the IDXD->TOC-path resolver into sylpheed-formats as
  IdxdObject::recover_toc_path() so the CLI `pak list` and the GUI pack
  browser share one implementation (behaviour-preserving; CLI still
  resolves 308/1004 on GP_MAIN_GAME_E).
- PAK browser now shows each entry's recovered original path (via the
  name-hash) in green, in both the entry list and the IDXD detail heading.
- Replace the stale welcome "RE Notes" table (mesh/audio "unknown") with an
  accurate reverse-engineered-formats status grid + correct authorship.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 20:42:40 +02:00
MechaCat02
d23339a3aa feat(viewer): stage thumbnail grid, per-model albedo, camera zoom fix
Route stage containers to stage_models (decode both single + stage paths, keep
whichever yields more geometry) — fixes stages showing the dummy-root "black
cube". Present a stage's sub-models as a thumbnail grid: each recentred and
uniformly scaled to a fixed cell so a 1-unit prop and a 3000-unit structure are
equally visible; skybox-plane / stray-anchor models (>20000u) are culled. Each
sub-model is textured by matching its name to the container's `_col` albedo.

Fix the orbit camera: zoom radius was hard-capped at 50u with fixed clip planes,
trapping the camera inside multi-thousand-unit stages. Zoom limits and near/far
planes now scale to the framed model.

Adds docs/HANDOFF-stage-mesh-2026-07-12.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 20:56:45 +02:00
MechaCat02
2053f31d17 feat(formats,cli): decode XBG7 stage containers + fix weapon layout
Stage containers (hidden/resource3d/Stage_S*.xpr) are collections of up to
~450 enemy/prop sub-models, not single meshes. Xbg7Model::stage_models decodes
them: each resource is a [12-byte header][index buffer][vertex buffer] block
(index count = descriptor marker, vertex count = u32 32 bytes before it) whose
on-disc offset is NOT stored, so it is located by content — one O(file) pass per
stride finds vertex-buffer starts (unit NORMAL at +12 whose previous slot isn't)
and each resource is pinned to the candidate whose indices validate and produce
non-degenerate, well-connected triangles. A connectivity gate (mean triangle
edge <= 0.28x the bbox diagonal) rejects spiky mis-anchors. ~4993 sub-models
decode across the 22 stages.

The same insight fixes the weapon single-model layout: it is [12-byte header]
[index][vertex] too, not [index][12-byte gap][vertex]. Reading indices from the
block start turned the 12 header bytes into 6 junk indices (2 leading degenerate
triangles — the recurring stray-triangle artifact) and dropped the last 6 real
indices. Skipping the header leaves vertex offsets identical (coverage unchanged
at 36/166) and corrects the triangle list. Verified on wep_00/03/04.

Adds `sylpheed-cli mesh {info,render}` — a headless software rasterizer that
writes a shaded PNG (orthographic, z-buffered, two-sided), so recovered geometry
can be verified without the GUI. Stages render as a normalised thumbnail grid;
--only filters sub-models.

Tests: stage_models_{decode,sweep,quality_audit}; docs/re updated (xbg7-mesh.md
evidence log + INDEX.md).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 20:56:35 +02:00
MechaCat02
4096b2d2a5 feat(formats): declaration-driven XBG7 decode (variable stride)
Dynamic-RE follow-up: capture Canary's GPU vertex-fetch + draw calls and
feed the ground truth back into the static decoder.

The GPU capture confirmed the reverse-engineered layout exactly — meshes
draw as triangle LISTs (prim=4) with pos f32x3 @0, normal f16x4 @0x0C,
uv f16x2 @0x14 — and revealed the XBG7 vertex format is NOT fixed-stride:
models omit elements (stride 20 = pos+normal, no UV; 24 = pos+normal+uv).

- mesh.rs: parse the descriptor's vertex declaration ({offset, format-code,
  usage} triples; 0x2A23B9=pos f32x3, 0x1A2360=normal f16x4, 0x2C235F=uv
  f16x2) to drive per-model stride + element offsets, instead of assuming
  stride 24. Coverage 25 -> 36 fully-validated models (e.g. the Stage_S*
  props, which are pos+normal only). Same index-range + unit-normal safety
  gates; complex/mismatched layouts still declined.
- Endianness note (documented): the capture's fetch endian=k8in32 describes
  the GPU's guest-memory copy, NOT the .xpr file bytes — reading the file
  with k8in32 breaks the normals (|n|->1.33); naive big-endian per element
  is correct (the game rearranges vertex data on load).
- tests: a coverage-regression test (>=35 models) + the existing weapon /
  body-declined disc tests still pass.

Confirmed against a Canary draw-log capture; see docs/re/structures/
xbg7-mesh.md (evidence log + declaration table).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 18:42:21 +02:00
MechaCat02
ef6e448268 feat(formats,viewer): decode XBG7 meshes + 3D model preview
Reverse-engineer the single-stream XBG7 geometry layout (clean-room: hex
inspection + geometric validation of the retail disc's
hidden/resource3d/*.xpr, no game code copied) and present models as
textured 3D meshes in the explorer.

Format (docs/re/structures/xbg7-mesh.md): XBG7 geometry resources sit
inside XPR2 containers alongside TX2D textures. For ~25 single-stream
models (weapons, simple props) the data section is a sequence of
sub-meshes, each an u16-BE triangle-list index buffer followed (after a
fixed 12-byte header) by a stride-24 vertex buffer whose declaration is
in the descriptor: POSITION f32x3 @0x00, NORMAL f16x4 @0x0C, TEXCOORD
f16x2 @0x14. Sub-mesh (vtx,idx) counts come from descriptor tuples.

The +12 vertex offset is pinned by the recovered normals being exactly
unit-length (align16 lands 4 bytes early and silently corrupts every
field). A safety gate rejects any model whose indices are out of range
or whose mean |normal| is not ~1, declining garbage (Stage_S*
placeholders, the complex multi-stream hero-ship body) rather than
mis-decoding it.

- mesh.rs: Xbg7Model::from_xpr2 -> GameMesh { positions, normals, uvs,
  indices }; standalone f16->f32; unit + real-disc tests (weapon decodes
  to 215v/364t with unit normals + in-range UVs; DeltaSaber body
  declined).
- texture.rs: from_xpr2_index / texture_names so the viewer can pick a
  model's _col albedo map.
- viewer: loose .xpr with decodable XBG7 spawns Bevy meshes (real normals,
  double-sided) textured with the albedo, framed by the orbit camera; the
  central egui panel goes transparent so the 3D scene shows through.

Complex multi-stream body meshes (DeltaSaber f004, other vertex layouts)
remain undecoded and are cleanly declined — next target is dynamic RE.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 17:47:06 +02:00
MechaCat02
e6da726f7b docs(re): CONFIRM T8aD colour interpretation (linear UNORM, ARGB order)
Resolved the "colours look odd" channel-order/sRGB question for k_8_8_8_8
(T8aD 2D UI textures) with no emulator run:

- sRGB: Canary's Vulkan host-format table maps plain k_8_8_8_8 to
  R8G8B8A8_UNORM and contains no R8G8B8A8_SRGB anywhere — gamma is a
  separate explicit EDRAM path. So these are raw linear bytes; apply no
  sRGB/gamma decode.
- Channel order: measured 789 real disc textures — byte0 is the opaque
  alpha (==0xFF-dominant) in 97% — so on-disk is ARGB. Cross-checked
  against xenia-rs decode_k8888_tiled, the Canary-validated M1/M2 path,
  which nets the same [A,R,G,B]->[R,G,B,A]. sylpheed-formats is correct.

Promotes the k8888 entry HYPOTHESIS -> CONFIRMED; INDEX updated. XPR2
skybox tiling + viewer display-gamma remain separate open items.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 17:11:17 +02:00
MechaCat02
b7028471e9 docs(re): add reverse-engineering knowledge base + conventions
Some checks failed
CI / Native — ubuntu-latest (push) Failing after 7m53s
CI / WASM — Web (push) Failing after 6m32s
CI / Formatting (push) Failing after 35s
CI / Native — macos-latest (push) Has been cancelled
CI / Native — windows-latest (push) Has been cancelled
Establish docs/re/ as the spec-side of the clean-room:
- README: confidence tiers (HYPOTHESIS/PROBABLE/CONFIRMED), promotion
  requires NEW independent evidence, append-only evidence logs,
  demote-on-contradiction, and a clean-room firewall (describe behaviour +
  cite addresses; never paste decompiled code). Documents the toolchain
  (sylpheed.db + zq.py, xenia-rs probes, Wine-Canary oracle) and the
  join-by-guest-PC method with the VA-equality caveat.
- Function/structure templates + a seeded INDEX (9 already-reversed formats).
- First entry: texture-color-k8888 — CONFIRMED plain k_8_8_8_8 is linear
  UNORM (no sRGB), channel order still to be measured.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 17:04:23 +02:00
MechaCat02
e1ca95c0af feat(formats,viewer): decode T8aD textures, RATC bundles, LSTA sprites
Add three static PAK-format decoders in the Bevy-free formats crate and
present them in the explorer:

- t8ad.rs: linear 32bpp A8R8G8B8 2D texture (UI/HUD art). Dims at
  0x14/0x18, header size keyed by the type field @0x1c (container-safe).
  Decodes the ~85% RGBA variants; defers the rest (likely DXT) rather
  than misdecoding.
- ratc.rs: nested resource bundle — lists named children by magic scan.
- lsta.rs: sprite list — walks the inline T8aD frames.

Viewer: PakContent gains T8ad/Lsta/Ratc, decoded off-thread and bounded
by an 8M-texel per-entry cap; detail views show the texture, a sprite
grid, and a child list with thumbnails. Decoded images carry a
"colours unverified" note — channel-order/sRGB stays on the dynamic-RE
backlog.

Tests: 12 new unit + 3 real-disc (T8aD >=70% decode over the hangar
pack, LSTA frames, RATC named children incl. a decodable T8aD).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 16:49:50 +02:00
MechaCat02
10425aba8c feat(formats): name font/image inner formats instead of hex
inner_format_label mapped only IDXD/xml/printable-tag magics; fonts and
images fell through to a bare hex label (00010000, 89504E47, …). Recognize
the known binary signatures and return friendly short tags so the pak
browser's format column reads them: sfnt/true/typ1 → "ttf", OTTO → "otf",
ttcf → "ttc", PNG → "png", plus RIFF/DDS. Still-unidentified magics keep the
hex fallback. Shared by the CLI `pak list` and the GUI browser.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 15:24:26 +02:00
MechaCat02
e17bc0216d feat(viewer): present subtitles, fonts, PNG + text in the PAK browser
Decode and present four more PAK item types in the pack browser's detail
pane. All content is decoded off-thread in build_pak_rows and attached to
each PakRow, then rendered with egui's own texture facilities — the Bevy
texture pipeline and egui's global font system are untouched.

formats crate (Bevy-free, reusable by CLI):
- ixud: parse IXUD localized-string tables as subtitle cue lists (UTF-16BE
  pool, SUBTITLE header + (text, timecode) pairs -> Vec<Cue{start,end,text}>).
- font: OTF/TTF/ttcf metadata (family / faces / glyphs) via ttf-parser.

viewer:
- PakRow gains `content: PakContent` (Subtitle | Font | Png | Text | None);
  classify_content fills it for IXUD / fonts / PNG / xml+text within the
  existing decode budget. Font samples are pre-rasterized off-thread with
  ab_glyph into an image (ImageRgba) — ab_glyph returns Option/Result at every
  step, so a bad font yields no sample instead of panicking (the earlier
  egui set_fonts approach crashed the app on atlas rebuild).
- draw_pak_browser dispatches on content: subtitle cue table, font metadata +
  sample image, PNG image, scrollable text; else the existing IDXD detail.
  Borrow-split PakView instead of cloning the (now heavy) rows each frame;
  one hash-keyed egui texture cache serves both PNG and font-sample images.

Subtitle<->movie auto-matching is deferred: movie filenames don't hash to the
IXUD keys and no cross-reference table was found (the link is an internal
MSG_DEMO id), so the movie player gets no subtitle track yet.

Tests: ixud unit tests; disc tests for the real English subtitle track
(cues + timecodes), eng/deu localization, font metadata, and off-thread
rasterization of the real font.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 15:19:31 +02:00
MechaCat02
91bd2543f4 feat(viewer): WMV cutscene player with scrub, keyboard + audio
Add an in-explorer video player for the dat/movie/*.wmv cutscenes.

Decode via the ffmpeg/ffprobe CLI (no libav linking): video streams from an
`ffmpeg -f rawvideo -pix_fmt rgba` pipe read into fixed w*h*4 frame chunks over
a bounded channel (paused/behind => ffmpeg back-pressures on its pipe); one
reused Image is overwritten in place each tick. Audio is best-effort: the track
is pre-decoded to a temp stereo WAV and played through a rodio Sink (volume /
play / pause / seek-via-skip_duration for free). Native-only under
cfg(not(wasm32)); VideoPreview registers unconditionally so the UI compiles for
wasm. Mirrors the .pak load pipeline (FileSelected -> bg thread ->
IsoLoaderMsg::VideoLoaded -> apply_loaded_video -> advance_video_playback), with
a single free_video teardown wired into every other viewer's reset.

Controls:
- play/pause button, click-on-frame toggle, Space / arrow (+-10s) shortcuts
- seekable timeline with mm:ss labels; volume slider (greyed when no audio)
- live scrubbing: a coalescing one-shot `ffmpeg -ss -frames:v 1` grabber always
  jumps to the knob's latest position, so dragging shows the frame under the
  knob in real time; that grabbed frame also bridges the gap after a commit-seek
  so there's no black flash while the streaming decoder catches up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 14:31:52 +02:00
MechaCat02
589659bec6 feat(viewer): world cubemap (skybox) viewer — 6-face grid
TXCM XPR2 resources are the game's world skyboxes (BG_Acheron, BG_Hargenteen,
…). The viewer showed only face 0 as a flat 2D image; now it decodes all 6 faces
and shows a labelled grid.

Formats (Bevy-free): X360Texture::cube_faces_from_xpr2() returns a Cubemap with 6
faces in D3D9 order (+X −X +Y −Y +Z −Z), or None for ordinary 2D textures. Face
layout derived from xenia's GetGuestTextureLayout — 6 back-to-back independently-
tiled surfaces, per-face stride = tiled-surface-size aligned to the 4 KiB
subresource boundary (kTextureSubresourceAlignmentBytes). Verified against real
BG_Acheron: data_size == 6 × 0x400000, and all 6 faces decode cleanly (own
Python decode + a disc test asserting 6×4 MiB faces and face 0 == the validated
from_xpr2 green-planet decode). Extracted a shared decode_surface() helper so the
2D and cube paths are byte-identical; from_xpr2 output unchanged (re-verified).

Viewer: new SkyboxPreview resource (6 egui face textures, reusing the existing
per-format x360_texture_to_bevy_image); populated in apply_loaded_texture's TXCM
branch; freed/reset alongside the other previews (factored free_texture/
free_skybox helpers). Central panel gains a skybox branch rendering a 3-column
labelled face grid.

DEFERRED (per "do not guess, else defer"): the interactive 3D skybox. Face data +
D3D9 order are validated, but wgpu cube-sampling handedness can't be confirmed
without eyeballing the GUI — a wrong-oriented skybox is worse than the correct
labelled grid. The grid is the reliable deliverable; the 3D look-around is a
follow-up once orientation is visually confirmed.

(Background agent did the investigation/validation but was blocked from writing
files; implemented here in the main tree from its findings, independently
re-verified.)

23 formats tests + disc cubemap test pass; viewer/CLI build; face-0 export
re-verified as the green planet.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 13:38:07 +02:00
MechaCat02
765e4a573b refactor(viewer): explorer UX cleanups (menu, loading, tree, status bar)
Four small UX fixes to the asset explorer:

1. Remove the no-op "View" menu (Textures/Meshes/Audio toggles + Wireframe
   checkbox) — none were wired to anything. Dropped ViewMode + the current_mode/
   wireframe fields from ViewerState (nothing read them); the central panel
   already dispatches on which preview resource is populated.

2. Loading indicator — clicking a drawer item now flips FileBrowserState.loading
   and the central panel shows a centered spinner + "Loading <name>…" until the
   background read/decode applies, instead of leaving the previous item on screen
   with no feedback. Cleared in poll_loader_channel on FileLoaded/PakLoaded/
   Error/Cancelled.

3. Drawer shows only the .pak index, not its .pNN data segments — the segments
   are the pack's body (loaded when the .pak is opened), so listing them (with no
   preview) was noise. Dropped the virtual "<stem>.pak" folder too; the .pak now
   sits as a normal leaf in its real directory.

4. Bottom status bar no longer overflows — horizontal_wrapped, only the source's
   final path component (full path on hover), file count, and a right-aligned,
   shortened controls hint. Removed the mode label.

Viewer + formats build; all tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 13:21:38 +02:00
MechaCat02
fbd4550d62 feat(viewer): data pack (.pak / IDXD) browser
Roadmap #3. The explorer can now open IPFB data packs — the game's ship/weapon/
mission definition tables. Selecting a `.pak` shows a master-detail browser:
entry list (hash · inner-format · identity) on the left, the selected IDXD
object's schema + explicit-field property table on the right.

Load path: a `.pak` selection reads the index plus its sibling `.pNN` segments
(ISO reader or extracted dir, probing p00.. until the first gap), assembles via
PakArchive::from_parts, and builds owned PakRow/PakDetail entirely off-thread —
so the UI holds only plain data (no borrow of a PakArchive, WASM-safe). Two
decode caps (64 MiB cumulative, 16 MiB/entry) keep a huge sound.pak from hanging;
over-budget entries show as "(not decoded)".

Wiring mirrors the texture path: new IsoLoaderMsg::PakLoaded → PendingPak staging
→ apply_pak. A shared reset (free GPU handle + clear texture/text/pak previews)
runs in both apply systems, so exactly one viewer is active per selection and the
prior GPU texture is always freed when switching modes.

Reuse over duplication: the CLI's inner_label / idxd_identity move into the
formats crate as pak::inner_format_label and IdxdObject::identity (bodies
verbatim → CLI output unchanged), now shared by CLI and GUI.

PakView registered unconditionally (wasm-safe; population native-only). ViewMode
unchanged — the central panel dispatches on the populated resource. Workspace
builds; 22 formats tests + all crate tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 23:17:10 +02:00
MechaCat02
2658dd20a6 feat(viewer): directory-tree browser + plain-text viewer
Explorer drawer was a flat list of all 353 disc files and only textures had a
preview. Two additions:

1. Directory tree — the left drawer is now a collapsible `📁` tree, rebuilt each
   frame from the path list (ui.rs: build_tree / split_for_tree / render_dir).
   Filtering narrows leaves and auto-expands matches; `.p00…p04` data segments
   nest under a virtual `<stem>.pak` node. Selection still keys off the original
   file index, so the loader pipeline is untouched.

2. Plain-text viewer — config.ini and any file that sniffs as text now show
   read-only, selectable, monospace content with a Wrap toggle and an encoding
   label. Detection/decoding live in the Bevy-free formats crate:
   vfs::is_probably_text + vfs::decode_text (BOM-aware UTF-8/UTF-16LE/BE, BOM-less
   UTF-16LE heuristic, NUL-reject + 95%-printable fallback) — pure std, WASM-safe,
   unit-tested (6 new tests), and reused by the CLI sniffer (new cyan `txt`).

New TextPreview resource (registered unconditionally so it exists on wasm);
populated in apply_loaded_texture with a 2 MiB char-boundary-safe cap. Also fixes
a latent bug: switching from a texture to a non-texture left stale preview state —
both texture and text state now reset on every selection.

ViewerState gains text_wrap. Formats tests: 22 pass; workspace compiles.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 23:05:58 +02:00
MechaCat02
854fd8dfd3 fix(texture): make BC dword-swap format-aware (don't mangle DXT5A)
The dword-swap that fixes DXT1 is a property of the BC1 COLOUR block, not of
BCn in general. RE on a DXT5A (BC4) mask showed the alpha block is byte-indexed
and wants the endian swap ALONE — dword-swapping it makes it worse. The blanket
swap from the previous commit was therefore breaking the (common) DXT5A/DXN
alpha & normal masks.

Now targeted by format:
 - DXT1            : swap dwords of each 8-byte colour block (verified).
 - DXT2_3 / DXT4_5 : swap only the colour half (bytes 8..16) of each 16-byte
                     block; alpha half endian-only. TENTATIVE — these formats
                     are near-absent in the game assets, so unvalidated against
                     a clear image.
 - DXT5A / DXN     : no dword swap (endian only).

DXT1 export re-verified clean (weapon skins); 16 formats tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 22:13:37 +02:00
MechaCat02
405233e84f fix(texture): crack X360 DXT1 dword-swap; wire correct decode into viewer
DXT1 (108/137 textures in Base.xpr — the bulk of picture assets) decoded to
noise. Root cause found by RE: Xbox 360 BCn blocks store the two 32-bit words
of each 64-bit sub-block in the opposite order to the PC/DDS layout — colour
endpoints live in the HIGH dword, indices in the low one. The endian field
(k8in16) only fixes byte order within the 16-bit words; it does not reorder the
dwords, so without this every DXT texture transposed endpoints/indices → noise.

Isolation that led here:
 - de-tile proven correct for bpb=8 (coherent per-block signature map; the
   linear read is scrambled) — same faithful Xenos Tiled2D as the verified
   bpb=4 ARGB path (green Acheron backdrop).
 - inspecting a smooth region, coherent colour endpoints appeared only in
   bytes[4..8], with the high-entropy indices in bytes[0..4].

Fix: swap_bc_block_dwords() swaps the two dwords within each 64-bit unit after
the byte-level endian swap, for every BCn format. Verified in the real Rust CLI:
weapon skins (rou_f001_wep_*) now decode to clean, recognisable images.

Viewer: DXT1 is fixed transparently (from_xpr2 feeds tex.data straight to the
GPU as Bc1). Also corrected the uncompressed path — post-swap k_8_8_8_8 is
[A,R,G,B]; reorder to [R,G,B,A] and upload as Rgba8UnormSrgb (was Bgra8, wrong).

Knob: XPR_NO_BC_DWORD_SWAP disables the swap for A/B validation.

KNOWN REMAINING: 16-byte blocks (BC2/DXT3, BC3/DXT5) and BC4/BC5 (DXT5A/DXN)
still need their alpha+colour half layout worked out — they decode to noise for
now. DXT1 + uncompressed + cubemaps are correct.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 22:08:23 +02:00
MechaCat02
840db9c549 fix(texture): correct Xenos de-tile + endian + cubemaps + PNG export
Reworks the XPR2 texture path after RE against the retail disc (viewer
previews were scrambled/failed).

- de-tile: replace the naive Morton-over-32×32 approximation with the
  faithful Xenos Tiled2D bank/pipe/macro-tile address formula (ported
  from xenia texture_address.h). Verified correct: the Acheron backdrop
  decodes to a sharp planet with its spiral storm.
- endian: undo the X360 word byte-swap per the fetch-constant endianness
  field (k8in16/k8in32/k16in32) — without it BCn/ARGB data is noise.
- cubemaps: parse TXCM resources (skybox/backdrops), decoding face 0,
  instead of erroring "No TX2D found".
- A8R8G8B8: correct channel order after the k8in32 swap ([A,R,G,B]→RGBA)
  — the Acheron backdrop is now correctly green, not red.
- XPR files are texture PACKS; add XPR_RES_INDEX to select a resource.
- CLI `texture export` now writes real PNGs (texpresso BCn decode +
  image), replacing the stub. Adds texpresso + image deps.
- tests/texture_disc.rs: integration test running the pipeline over real
  disc .xpr files (28/28 parse). RE debug knobs: XPR_NO_DETILE /
  XPR_NO_ENDIAN / XPR_FORCE_ENDIAN.

KNOWN ISSUE: mipmapped DXT1 textures still decode to noise — mip 0 is
not at base_address in the multi-resource packs (localized to a mip
storage-offset quirk; uncompressed + de-tile + endian are all verified).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 21:37:20 +02:00
MechaCat02
d92c7ddaea feat(viewer): ISO / extracted-dir loading pipeline + native file dialog
Some checks failed
CI / Native — ubuntu-latest (push) Failing after 7m14s
CI / WASM — Web (push) Failing after 6m35s
CI / Formatting (push) Failing after 35s
CI / Native — macos-latest (push) Has been cancelled
CI / Native — windows-latest (push) Has been cancelled
WIP: add iso_loader.rs — bridges the async XisoReader / synchronous
GameAssets to Bevy's ECS via background threads + mpsc channels polled
per frame (no main-thread blocking); UI open-iso/open-dir/file-select
events → texture preview. Register IsoLoaderPlugin; wire asset_loader
and the egui UI. Deps: rfd (native file dialog, workspace + viewer),
futures, and bevy tonemapping_luts feature.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 07:18:45 +02:00
MechaCat02
ce9fe08bec refactor(formats): rework X360 texture descriptor + xiso reader
WIP: restructure texture.rs (X360TextureDesc accessors / decode path) and
adjust the xiso reader. Builds clean (sylpheed-formats).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 07:18:37 +02:00
34 changed files with 8498 additions and 585 deletions

917
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -39,6 +39,12 @@ anyhow = "1"
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
# ── Native file dialog ────────────────────────────────────────────────────────
rfd = "0.14"
# ── Audio playback (native video player) ──────────────────────────────────────
rodio = "0.20"
# ── CLI ───────────────────────────────────────────────────────────────────────
clap = { version = "4", features = ["derive"] }

View File

@@ -19,3 +19,6 @@ tracing = { workspace = true }
tracing-subscriber = { workspace = true }
colored = "2"
indicatif = "0.17"
# Texture PNG export: BCn software decode + image encode.
texpresso = "2"
image = { version = "0.25", default-features = false, features = ["png"] }

View File

@@ -97,6 +97,27 @@ enum Commands {
#[command(subcommand)]
cmd: PakCommands,
},
/// XBG7 mesh tools (inspect / headless render to PNG)
Mesh {
#[command(subcommand)]
cmd: MeshCommands,
},
/// Audio tools (identify WAV/XMA/XMA2 + metadata)
Audio {
#[command(subcommand)]
cmd: AudioCommands,
},
}
#[derive(Subcommand)]
enum AudioCommands {
/// Identify an audio file/stream and print its metadata
Info {
/// Path to a WAV / XMA / raw audio stream
file: PathBuf,
},
}
#[derive(Subcommand)]
@@ -118,6 +139,37 @@ enum PakCommands {
},
}
#[derive(Subcommand)]
enum MeshCommands {
/// Print the decoded sub-models of an XBG7 container (`.xpr`)
Info {
/// Path to the `.xpr` model / stage container
file: PathBuf,
},
/// Headless-render the decoded mesh(es) to a shaded PNG (software rasterizer)
Render {
/// Path to the `.xpr` model / stage container
file: PathBuf,
/// Output PNG path
output: PathBuf,
/// Image size in pixels (square)
#[arg(long, default_value_t = 900)]
size: u32,
/// Camera yaw in degrees
#[arg(long, default_value_t = 35.0)]
yaw: f32,
/// Camera pitch in degrees
#[arg(long, default_value_t = 22.0)]
pitch: f32,
/// Force the stage grid layout even for single models
#[arg(long)]
row: bool,
/// Only render sub-models whose name contains this substring
#[arg(long)]
only: Option<String>,
},
}
#[derive(Subcommand)]
enum TextureCommands {
/// Print information about a texture file
@@ -155,13 +207,51 @@ async fn main() -> Result<()> {
TextureCommands::Info { file } => cmd_texture_info(&file),
TextureCommands::Export { file, output } => cmd_texture_export(&file, &output),
},
Commands::Mesh { cmd } => match cmd {
MeshCommands::Info { file } => cmd_mesh_info(&file),
MeshCommands::Render { file, output, size, yaw, pitch, row, only } => {
cmd_mesh_render(&file, &output, size, yaw, pitch, row, only)
}
},
Commands::Pak { cmd } => match cmd {
PakCommands::List { pak, idxd_only } => cmd_pak_list(&pak, idxd_only),
PakCommands::Dump { pak, hash } => cmd_pak_dump(&pak, &hash),
},
Commands::Audio { cmd } => match cmd {
AudioCommands::Info { file } => cmd_audio_info(&file),
},
}
}
// ── audio info ───────────────────────────────────────────────────────────────
fn cmd_audio_info(file: &Path) -> Result<()> {
use sylpheed_formats::AudioInfo;
let bytes = std::fs::read(file).with_context(|| format!("Cannot read {}", file.display()))?;
let info = AudioInfo::probe(&bytes);
println!("{} {}", "Audio:".green().bold(), file.display());
println!(" Codec : {}", info.codec.label().yellow());
let opt = |v: Option<String>| v.unwrap_or_else(|| "".dimmed().to_string());
println!(" Channels : {}", opt(info.channels.map(|c| c.to_string())));
println!(" Sample rate: {}", opt(info.sample_rate.map(|r| format!("{r} Hz"))));
println!(" Bit depth : {}", opt(info.bits_per_sample.map(|b| format!("{b}-bit"))));
if let Some(d) = info.duration_secs {
println!(" Duration : {d:.2} s");
}
if let Some(p) = info.xma_packets {
println!(" XMA packets: {} (2048 B each)", p.to_string().yellow());
}
println!(" Size : {} bytes", info.size_bytes.to_string().yellow());
if info.codec.needs_decoder() {
println!(
" {} decode not supported (needs an XMA2 decoder + the sound-bank descriptor)",
"note:".dimmed()
);
}
Ok(())
}
// ── extract ────────────────────────────────────────────────────────────────
async fn cmd_extract(iso_path: &Path, output_dir: &Path) -> Result<()> {
@@ -272,6 +362,7 @@ fn cmd_sniff(dir: &Path, unknown_only: bool) -> Result<()> {
"bin" => label.red().to_string(),
"xpr" => label.green().to_string(),
"dds" => label.green().to_string(),
"txt" => label.cyan().to_string(),
_ => label.yellow().to_string(),
};
@@ -315,9 +406,16 @@ fn cmd_texture_info(file: &Path) -> Result<()> {
let tex = X360Texture::from_xpr2(&bytes)
.with_context(|| format!("Failed to parse texture: {}", file.display()))?;
let d = tex.format.desc();
println!("{} {}", "Texture:".green().bold(), file.display());
println!(" Resolution : {}×{}", tex.width.to_string().yellow(), tex.height.to_string().yellow());
println!(" Format : {:?}", tex.format);
println!(
" Format : {} ({:?}) · {} bpp, {}",
tex.format.gpu_name().yellow(),
tex.format,
d.bpp,
if d.compressed { "compressed" } else { "uncompressed" },
);
println!(" Mip levels : {}", tex.mip_levels);
println!(" Data size : {} bytes", tex.data.len().to_string().yellow());
@@ -330,91 +428,338 @@ fn cmd_texture_export(file: &Path, output: &Path) -> Result<()> {
let bytes = std::fs::read(file)
.with_context(|| format!("Cannot read {}", file.display()))?;
use sylpheed_formats::texture::{X360Texture, X360TextureFormat};
use sylpheed_formats::texture::X360Texture;
let tex = X360Texture::from_xpr2(&bytes)?;
let rgba = decode_to_rgba8(&tex)
.with_context(|| format!("decoding {:?} texture", tex.format))?;
// For BCn compressed textures we need to software-decompress to RGBA8
// before saving to PNG. This uses the `texpresso` crate.
//
// TODO: add `texpresso = "2"` to Cargo.toml for BCn software decode.
// For now, print a helpful message.
match tex.format {
X360TextureFormat::A8R8G8B8 | X360TextureFormat::X8R8G8B8 => {
// Raw BGRA8 — can save directly (with channel swizzle)
println!(
"{} TODO: save raw BGRA8 to PNG (add `image` crate)",
"Note:".yellow()
);
}
compressed_format => {
println!(
"{} Format {:?} needs BCn decompression before PNG export.",
"Note:".yellow(), compressed_format
);
println!(
" Add `texpresso` crate and implement decompression in texture.rs"
);
println!(
" Alternatively, use the Bevy viewer to inspect textures visually."
);
}
}
image::save_buffer(
output,
&rgba,
tex.width,
tex.height,
image::ExtendedColorType::Rgba8,
)
.with_context(|| format!("writing PNG {}", output.display()))?;
println!(
" Texture parsed OK: {}×{} {:?}",
tex.width, tex.height, tex.format
"{} {}×{} {:?}{}{}",
"Exported".green().bold(),
tex.width,
tex.height,
tex.format,
if tex.is_cubemap { " (cubemap face 0)" } else { "" },
output.display().to_string().cyan(),
);
Ok(())
}
// ── mesh info / render ──────────────────────────────────────────────────────
/// Decode a container and return its sub-models the same way the viewer routes:
/// a single model (weapon / prop) OR a stage's many sub-models — whichever
/// yields more geometry.
fn decode_models(bytes: &[u8]) -> Vec<sylpheed_formats::mesh::Xbg7Model> {
use sylpheed_formats::mesh::Xbg7Model;
let single = Xbg7Model::from_xpr2(bytes)
.ok()
.filter(|m| !m.meshes.is_empty());
let single_verts = single.as_ref().map(|m| m.totals().0).unwrap_or(0);
let stage = Xbg7Model::stage_models(bytes);
let stage_verts: usize = stage.iter().map(|m| m.totals().0).sum();
if !stage.is_empty() && stage_verts > single_verts {
stage
} else {
single.into_iter().collect()
}
}
fn cmd_mesh_info(file: &Path) -> Result<()> {
let bytes = std::fs::read(file).with_context(|| format!("Cannot read {}", file.display()))?;
let models = decode_models(&bytes);
if models.is_empty() {
println!("{} no decodable XBG7 geometry", "Mesh:".yellow().bold());
return Ok(());
}
let (mut tv, mut tt) = (0usize, 0usize);
println!("{} {}", "Mesh:".green().bold(), file.display());
for m in &models {
let (v, t) = m.totals();
tv += v;
tt += t;
let mut lo = [f32::MAX; 3];
let mut hi = [f32::MIN; 3];
for sub in &m.meshes {
for p in &sub.positions {
for a in 0..3 {
lo[a] = lo[a].min(p[a]);
hi[a] = hi[a].max(p[a]);
}
}
}
println!(
" {:16} {:>6} v {:>6} t bbox [{:.1} {:.1} {:.1}]",
m.name,
v,
t,
hi[0] - lo[0],
hi[1] - lo[1],
hi[2] - lo[2]
);
}
println!(
" {} {} sub-models · {} verts · {} tris",
"TOTAL".bold(),
models.len(),
tv,
tt
);
Ok(())
}
#[allow(clippy::too_many_arguments)]
fn cmd_mesh_render(
file: &Path,
output: &Path,
size: u32,
yaw: f32,
pitch: f32,
force_row: bool,
only: Option<String>,
) -> Result<()> {
let bytes = std::fs::read(file).with_context(|| format!("Cannot read {}", file.display()))?;
let mut models = decode_models(&bytes);
if let Some(sub) = &only {
models.retain(|m| m.name.contains(sub.as_str()));
}
if models.is_empty() {
anyhow::bail!("no decodable XBG7 geometry in {}", file.display());
}
// ── Build a triangle soup. ──
// Single models render centred; multi-model containers (stages) get the
// viewer's normalised **thumbnail grid**: each sub-model recentred and
// uniformly scaled to a fixed cell, so all are equally visible regardless of
// native scale (mirrors `spawn_stage_models`).
let multi = models.len() > 1 || force_row;
let mut tris: Vec<[[f32; 3]; 3]> = Vec::new();
const CELL: f32 = 10.0;
const GAP: f32 = 4.0;
let grid_pitch = CELL + GAP;
let cols = (models.len() as f32).sqrt().ceil().max(1.0) as usize;
for (i, m) in models.iter().enumerate() {
let mut lo = [f32::MAX; 3];
let mut hi = [f32::MIN; 3];
for sub in &m.meshes {
for p in &sub.positions {
for a in 0..3 {
lo[a] = lo[a].min(p[a]);
hi[a] = hi[a].max(p[a]);
}
}
}
if lo[0] > hi[0] {
continue;
}
let center = [
(lo[0] + hi[0]) * 0.5,
(lo[1] + hi[1]) * 0.5,
(lo[2] + hi[2]) * 0.5,
];
let (scale, cell) = if multi {
let extent = (hi[0] - lo[0]).max(hi[1] - lo[1]).max(hi[2] - lo[2]).max(1e-3);
let col = i % cols;
let row = i / cols;
(CELL / extent, [col as f32 * grid_pitch, -(row as f32) * grid_pitch, 0.0])
} else {
(1.0, [0.0, 0.0, 0.0])
};
for sub in &m.meshes {
for tri in sub.indices.chunks_exact(3) {
let (a, b, c) = (tri[0] as usize, tri[1] as usize, tri[2] as usize);
if a >= sub.positions.len() || b >= sub.positions.len() || c >= sub.positions.len() {
continue;
}
let f = |i: usize| {
let p = sub.positions[i];
[
(p[0] - center[0]) * scale + cell[0],
(p[1] - center[1]) * scale + cell[1],
(p[2] - center[2]) * scale + cell[2],
]
};
tris.push([f(a), f(b), f(c)]);
}
}
}
if tris.is_empty() {
anyhow::bail!("no triangles to render");
}
let rgba = rasterize(&tris, size, yaw, pitch);
image::save_buffer(output, &rgba, size, size, image::ExtendedColorType::Rgba8)
.with_context(|| format!("writing PNG {}", output.display()))?;
println!(
"{} {} tris → {} ({}×{}, yaw {:.0}° pitch {:.0}°)",
"Rendered".green().bold(),
tris.len(),
output.display().to_string().cyan(),
size,
size,
yaw,
pitch,
);
Ok(())
}
/// Minimal software rasterizer: orthographic, z-buffered, two-sided Lambert +
/// headlight shading over a flat grey material on a dark background. Enough to
/// judge whether recovered geometry is coherent.
fn rasterize(tris: &[[[f32; 3]; 3]], size: u32, yaw_deg: f32, pitch_deg: f32) -> Vec<u8> {
let n = size as usize;
let (yaw, pitch) = (yaw_deg.to_radians(), pitch_deg.to_radians());
let (cy, sy) = (yaw.cos(), yaw.sin());
let (cp, sp) = (pitch.cos(), pitch.sin());
// Rotate a world point into view space (yaw about Y, then pitch about X).
let view = |p: [f32; 3]| -> [f32; 3] {
let x = p[0] * cy + p[2] * sy;
let z0 = -p[0] * sy + p[2] * cy;
let y = p[1] * cp - z0 * sp;
let z = p[1] * sp + z0 * cp;
[x, y, z]
};
// View-space bbox → orthographic fit.
let mut lo = [f32::MAX; 3];
let mut hi = [f32::MIN; 3];
for t in tris {
for v in t {
let q = view(*v);
for a in 0..3 {
lo[a] = lo[a].min(q[a]);
hi[a] = hi[a].max(q[a]);
}
}
}
let span = (hi[0] - lo[0]).max(hi[1] - lo[1]).max(1e-3);
let scale = (n as f32) * 0.9 / span;
let cx = (lo[0] + hi[0]) * 0.5;
let cyv = (lo[1] + hi[1]) * 0.5;
let to_screen = |q: [f32; 3]| -> (f32, f32, f32) {
let sx = (q[0] - cx) * scale + n as f32 * 0.5;
let sy = n as f32 * 0.5 - (q[1] - cyv) * scale;
(sx, sy, q[2])
};
let mut color = vec![18u8; n * n * 4];
for i in 0..n * n {
color[i * 4 + 3] = 255;
}
let mut depth = vec![f32::MAX; n * n];
// Light in view space (upper-left-front).
let light = {
let l = [-0.4f32, 0.6, 0.7];
let m = (l[0] * l[0] + l[1] * l[1] + l[2] * l[2]).sqrt();
[l[0] / m, l[1] / m, l[2] / m]
};
for t in tris {
let v0 = view(t[0]);
let v1 = view(t[1]);
let v2 = view(t[2]);
// Face normal in view space.
let e1 = [v1[0] - v0[0], v1[1] - v0[1], v1[2] - v0[2]];
let e2 = [v2[0] - v0[0], v2[1] - v0[1], v2[2] - v0[2]];
let mut nrm = [
e1[1] * e2[2] - e1[2] * e2[1],
e1[2] * e2[0] - e1[0] * e2[2],
e1[0] * e2[1] - e1[1] * e2[0],
];
let nl = (nrm[0] * nrm[0] + nrm[1] * nrm[1] + nrm[2] * nrm[2]).sqrt();
if nl < 1e-12 {
continue;
}
nrm = [nrm[0] / nl, nrm[1] / nl, nrm[2] / nl];
// Two-sided: diffuse from |n·L|, plus a headlight term from |n.z|.
let diff = (nrm[0] * light[0] + nrm[1] * light[1] + nrm[2] * light[2]).abs();
let head = nrm[2].abs();
let inten = (0.18 + 0.55 * diff + 0.3 * head).min(1.0);
let shade = (inten * 210.0) as u8;
let (ax, ay, az) = to_screen(v0);
let (bx, by, bz) = to_screen(v1);
let (ccx, ccy, ccz) = to_screen(v2);
let minx = ax.min(bx).min(ccx).floor().max(0.0) as usize;
let maxx = ax.max(bx).max(ccx).ceil().min(n as f32 - 1.0) as usize;
let miny = ay.min(by).min(ccy).floor().max(0.0) as usize;
let maxy = ay.max(by).max(ccy).ceil().min(n as f32 - 1.0) as usize;
let area = (bx - ax) * (ccy - ay) - (by - ay) * (ccx - ax);
if area.abs() < 1e-6 {
continue;
}
for py in miny..=maxy {
for px in minx..=maxx {
let fx = px as f32 + 0.5;
let fy = py as f32 + 0.5;
let w0 = ((bx - fx) * (ccy - fy) - (by - fy) * (ccx - fx)) / area;
let w1 = ((ccx - fx) * (ay - fy) - (ccy - fy) * (ax - fx)) / area;
let w2 = 1.0 - w0 - w1;
if w0 < 0.0 || w1 < 0.0 || w2 < 0.0 {
continue;
}
let z = w0 * az + w1 * bz + w2 * ccz;
let idx = py * n + px;
if z < depth[idx] {
depth[idx] = z;
color[idx * 4] = shade;
color[idx * 4 + 1] = shade;
color[idx * 4 + 2] = (shade as f32 * 1.02).min(255.0) as u8;
}
}
}
}
color
}
/// Software-decode a de-tiled `X360Texture` (mip 0) to tightly-packed RGBA8.
///
/// BCn blocks are decompressed with `texpresso`; uncompressed A8R8G8B8/X8R8G8B8
/// is byte-swizzled from the Xenos in-memory BGRA order.
fn decode_to_rgba8(tex: &sylpheed_formats::texture::X360Texture) -> Result<Vec<u8>> {
use sylpheed_formats::texture::X360TextureFormat as F;
let (w, h) = (tex.width as usize, tex.height as usize);
let mut rgba = vec![0u8; w * h * 4];
let bc = |fmt: texpresso::Format, rgba: &mut [u8]| {
fmt.decompress(&tex.data, w, h, rgba);
};
match tex.format {
F::Dxt1 => bc(texpresso::Format::Bc1, &mut rgba),
F::Dxt3 => bc(texpresso::Format::Bc2, &mut rgba),
F::Dxt5 => bc(texpresso::Format::Bc3, &mut rgba),
F::A8R8G8B8 | F::X8R8G8B8 => {
// After the k8in32 endian swap in from_xpr2, k_8_8_8_8 pixels are in
// [A,R,G,B] byte order (verified against the retail Acheron backdrop).
// Emit RGBA. X8 has no meaningful alpha.
let opaque = matches!(tex.format, F::X8R8G8B8);
for (px, out) in tex.data.chunks_exact(4).zip(rgba.chunks_exact_mut(4)) {
out[0] = px[1]; // R
out[1] = px[2]; // G
out[2] = px[3]; // B
out[3] = if opaque { 0xFF } else { px[0] };
}
}
other => {
anyhow::bail!("PNG export for {other:?} (BC4/BC5) not implemented yet");
}
}
Ok(rgba)
}
// ── pak list ────────────────────────────────────────────────────────────────
/// Best-effort human label for a decompressed entry's inner format.
fn inner_label(payload: &[u8]) -> String {
if payload.len() < 4 {
return "empty".into();
}
let m = &payload[0..4];
if m == b"IDXD" {
"IDXD".into()
} else if payload.starts_with(b"<?xml") {
"xml".into()
} else if m.iter().all(|&b| b.is_ascii_graphic()) {
// Many inner formats are 4-char tags (RATC, T8aD, IXUD, LSTA, ttcf, …).
String::from_utf8_lossy(m).into_owned()
} else {
format!("{:02X}{:02X}{:02X}{:02X}", m[0], m[1], m[2], m[3])
}
}
/// For an IDXD entry, a short identity string (ID / Name / Model if present).
fn idxd_identity(obj: &IdxdObject) -> String {
for key in ["ID", "Name", "Model"] {
if let Some(v) = obj.get_raw(key) {
return format!("{key}={v}");
}
}
format!("schema {:08x}", obj.schema_hash)
}
/// Try to recover an IDXD entry's original TOC path from its identity tokens.
/// Uses the entry's ID/Name/Model fields plus identifier-like pool tokens as
/// candidates for [`sylpheed_formats::hash::recover_toc_name`].
fn idxd_toc_name(obj: &IdxdObject, name_hash: u32) -> Option<String> {
let mut cands: Vec<&str> = Vec::new();
for key in ["ID", "Name", "Model"] {
if let Some(v) = obj.get_raw(key) {
cands.push(v);
}
}
for t in obj.tokens() {
if t.contains('_') || t.len() >= 5 {
cands.push(t.as_str());
}
}
sylpheed_formats::hash::recover_toc_name(name_hash, &cands)
}
use sylpheed_formats::pak::inner_format_label as inner_label;
fn cmd_pak_list(pak: &Path, idxd_only: bool) -> Result<()> {
let arc = PakArchive::open(pak).with_context(|| format!("opening {}", pak.display()))?;
@@ -443,7 +788,7 @@ fn cmd_pak_list(pak: &Path, idxd_only: bool) -> Result<()> {
}
let (detail, name) = if is_idxd {
match IdxdObject::parse(&payload) {
Ok(o) => (idxd_identity(&o), idxd_toc_name(&o, e.name_hash)),
Ok(o) => (o.identity(), o.recover_toc_path(e.name_hash)),
Err(_) => (String::new(), None),
}
} else {

View File

@@ -10,6 +10,7 @@ authors.workspace = true
xdvdfs = { workspace = true }
binrw = { workspace = true }
flate2 = "1" # zlib/DEFLATE for IPFB "Z1" entries (miniz_oxide backend, WASM-safe)
ttf-parser = { version = "0.24", default-features = false, features = ["std", "opentype-layout"] } # font metadata (OTF/TTF/ttcf), WASM-safe
tokio = { workspace = true }
futures = { workspace = true }
thiserror = { workspace = true }

View File

@@ -1,85 +1,405 @@
//! Audio format parsing — XMA and XWB handling.
//! Audio format parsing — WAV/PCM decode + Xbox 360 XMA/XMA2 recognition.
//!
//! Xbox 360 games use **XMA** (Xbox Media Audio) as their primary audio codec.
//! XMA is a proprietary Microsoft codec derived from WMA Pro.
//! ## What the game actually ships
//! Project Sylpheed's in-game audio lives in `dat/sound.pak` (IPFB): ~9500
//! entries of **raw, header-less XMA2** stream data — no per-entry `RIFF`/`fmt `
//! header, no compression wrapper (identical sound effects are byte-for-byte
//! duplicate entries). The per-stream format (channel count, sample rate, loop
//! points) is therefore **not** in the stream data; it lives in a separate
//! sound-bank descriptor that has not been reverse engineered yet. So this
//! module can *identify* those streams (and count their XMA packets) but cannot
//! yet decode them to PCM — that needs (a) the bank descriptor and (b) an XMA2
//! decoder.
//!
//! ## The Challenge
//! XMA decoding requires Microsoft's proprietary XMA decoder, which is only
//! available in the Windows DirectX runtime. There are two approaches:
//! ## What this module does today
//! - Fully parses **RIFF/WAVE PCM** (8/16/24-bit int, 32-bit float) → `GameAudio`
//! (interleaved `f32`), ready for playback/export of any converted audio.
//! - Reads metadata from **RIFF/WAVE XMA (`0x0165`) / XMA2 (`0x0166`)** headers
//! (channels, sample rate) — decode still unsupported.
//! - Recognizes **raw XMA2** stream blobs by entropy + packet alignment and
//! reports the 2048-byte packet count.
//!
//! ### Option A: Convert on extraction (recommended for Milestone 1)
//! Use `xma2encode.exe` (from Xbox 360 SDK) or `ffmpeg` (has partial XMA support)
//! to pre-convert all audio to OGG/WAV during the extraction step.
//!
//! ```bash
//! # Convert a single XMA file to WAV using ffmpeg
//! ffmpeg -i audio.xma output.wav
//!
//! # Or use VGMStream's test.exe for more accurate XMA decoding
//! ```
//!
//! ### Option B: XMA2 software decoder
//! The `xma2dec` project provides an open-source XMA2 decoder.
//! GitHub: https://github.com/koolkdev/xmalib (research-quality)
//!
//! ### XWB Wave Bank format
//! Xbox 360 games typically bundle audio into XWB (Xbox Wave Bank) files.
//! These are containers that hold multiple XMA streams.
//! Reference: https://github.com/microsoft/DirectXTK/tree/main/Audio
//! XMA framing constants are from xenia-canary `src/xenia/apu/xma_context.h`
//! (`kBytesPerPacket = 2048`, `kSamplesPerFrame = 512`, `kBytesPerSample = 2`).
use thiserror::Error;
// XMA framing (xenia-canary xma_context.h).
/// One XMA2 packet is 2048 bytes (4-byte header + 2044 bytes of frame data).
pub const XMA_BYTES_PER_PACKET: usize = 2048;
/// Decoded PCM samples produced per XMA frame, per channel.
pub const XMA_SAMPLES_PER_FRAME: u32 = 512;
// WAVE format tags.
const WAVE_FORMAT_PCM: u16 = 0x0001;
const WAVE_FORMAT_IEEE_FLOAT: u16 = 0x0003;
const WAVE_FORMAT_EXTENSIBLE: u16 = 0xFFFE;
const WAVE_FORMAT_XMA: u16 = 0x0165;
const WAVE_FORMAT_XMA2: u16 = 0x0166;
#[derive(Debug, Error)]
pub enum AudioError {
#[error("Unknown audio format magic: {0:?}")]
UnknownMagic([u8; 4]),
#[error("XMA decoding requires pre-conversion. See audio.rs for instructions.")]
XmaNotSupported,
#[error("Parse error: {0}")]
Parse(String),
#[error("not a recognized audio container")]
Unrecognized,
#[error("malformed audio: {0}")]
Malformed(&'static str),
#[error("codec needs a decoder this crate does not provide: {0:?}")]
NeedsDecoder(AudioCodec),
#[error("unsupported PCM sample format: tag {tag:#06x}, {bits} bits")]
UnsupportedPcm { tag: u16, bits: u16 },
}
/// An audio clip decoded to raw PCM, ready for Bevy's audio system.
/// Recognized audio container / codec.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum AudioCodec {
/// RIFF/WAVE integer PCM (`WAVE_FORMAT_PCM`).
Pcm,
/// RIFF/WAVE IEEE-float PCM (`WAVE_FORMAT_IEEE_FLOAT`).
PcmFloat,
/// RIFF/WAVE XMA (`0x0165`).
Xma,
/// RIFF/WAVE XMA2 (`0x0166`).
Xma2,
/// Header-less XMA2 stream (the game's `sound.pak` entries), identified
/// heuristically — no embedded channel/rate metadata.
RawXma2,
Unknown,
}
impl AudioCodec {
/// Does decoding this codec require support this crate does not (yet) have?
pub fn needs_decoder(&self) -> bool {
matches!(self, Self::Xma | Self::Xma2 | Self::RawXma2)
}
pub fn label(&self) -> &'static str {
match self {
Self::Pcm => "PCM",
Self::PcmFloat => "PCM float",
Self::Xma => "XMA (RIFF)",
Self::Xma2 => "XMA2 (RIFF)",
Self::RawXma2 => "XMA2 (raw stream)",
Self::Unknown => "unknown",
}
}
}
/// Non-decoding metadata describing an audio blob.
#[derive(Debug, Clone)]
pub struct AudioInfo {
pub codec: AudioCodec,
pub channels: Option<u16>,
pub sample_rate: Option<u32>,
pub bits_per_sample: Option<u16>,
/// PCM samples per channel, when derivable (WAV only).
pub samples_per_channel: Option<u64>,
pub duration_secs: Option<f32>,
pub size_bytes: usize,
/// 2048-byte XMA packet count, for XMA/raw-XMA streams.
pub xma_packets: Option<u32>,
}
impl AudioInfo {
fn empty(codec: AudioCodec, size: usize) -> Self {
Self {
codec,
channels: None,
sample_rate: None,
bits_per_sample: None,
samples_per_channel: None,
duration_secs: None,
size_bytes: size,
xma_packets: None,
}
}
/// Best-effort classification of an audio blob. Never fails: unrecognized
/// input yields [`AudioCodec::Unknown`].
///
/// Order matters: a `RIFF/WAVE` header is authoritative; only header-less,
/// high-entropy, packet-sized blobs fall through to the raw-XMA2 heuristic.
pub fn probe(bytes: &[u8]) -> Self {
if let Some(info) = parse_riff_wave(bytes) {
return info;
}
if looks_like_raw_xma2(bytes) {
let mut info = Self::empty(AudioCodec::RawXma2, bytes.len());
info.xma_packets = Some(bytes.len().div_ceil(XMA_BYTES_PER_PACKET) as u32);
return info;
}
Self::empty(AudioCodec::Unknown, bytes.len())
}
}
/// True when `bytes` is likely a raw, header-less XMA2 stream: no known audio
/// magic, but a large, near-incompressible (high-entropy) payload — the shape
/// of the game's `sound.pak` audio entries. Heuristic: not conclusive, but the
/// probe runs only after every structured format has been ruled out.
pub fn looks_like_raw_xma2(bytes: &[u8]) -> bool {
// XMA streams are always at least a couple of packets long.
if bytes.len() < XMA_BYTES_PER_PACKET * 2 {
return false;
}
if bytes.starts_with(b"RIFF") || bytes.starts_with(b"XMA2") {
return false; // handled by the RIFF path
}
shannon_entropy_bits(&bytes[..bytes.len().min(64 * 1024)]) >= 7.8
}
/// Shannon entropy in bits/byte over `data` (0.0..=8.0). Compressed audio sits
/// very close to 8; structured/text data is much lower.
fn shannon_entropy_bits(data: &[u8]) -> f64 {
if data.is_empty() {
return 0.0;
}
let mut counts = [0u32; 256];
for &b in data {
counts[b as usize] += 1;
}
let n = data.len() as f64;
counts
.iter()
.filter(|&&c| c > 0)
.map(|&c| {
let p = c as f64 / n;
-p * p.log2()
})
.sum()
}
// ── RIFF/WAVE ──────────────────────────────────────────────────────────────────
/// Parse a RIFF/WAVE header for metadata. Returns `None` if `bytes` is not a
/// `RIFF....WAVE` container. All RIFF fields are little-endian.
fn parse_riff_wave(bytes: &[u8]) -> Option<AudioInfo> {
if bytes.len() < 12 || &bytes[..4] != b"RIFF" || &bytes[8..12] != b"WAVE" {
return None;
}
let le16 = |o: usize| u16::from_le_bytes([bytes[o], bytes[o + 1]]);
let le32 =
|o: usize| u32::from_le_bytes([bytes[o], bytes[o + 1], bytes[o + 2], bytes[o + 3]]);
let mut pos = 12;
let (mut tag, mut channels, mut rate, mut bits) = (0u16, 0u16, 0u32, 0u16);
let mut data_bytes: Option<u64> = None;
let mut have_fmt = false;
while pos + 8 <= bytes.len() {
let id = &bytes[pos..pos + 4];
let size = le32(pos + 4) as usize;
let body = pos + 8;
match id {
b"fmt " if body + 16 <= bytes.len() => {
tag = le16(body);
channels = le16(body + 2);
rate = le32(body + 4);
bits = le16(body + 14);
// WAVE_FORMAT_EXTENSIBLE stores the real tag in the GUID's first
// two bytes, right after cbSize (+2) → +24 from the fmt body.
if tag == WAVE_FORMAT_EXTENSIBLE && body + 26 <= bytes.len() {
tag = le16(body + 24);
}
have_fmt = true;
}
b"data" => data_bytes = Some(size as u64),
_ => {}
}
// Chunks are word-aligned (pad byte when size is odd).
pos = body + size + (size & 1);
}
if !have_fmt {
return None;
}
let codec = match tag {
WAVE_FORMAT_PCM => AudioCodec::Pcm,
WAVE_FORMAT_IEEE_FLOAT => AudioCodec::PcmFloat,
WAVE_FORMAT_XMA => AudioCodec::Xma,
WAVE_FORMAT_XMA2 => AudioCodec::Xma2,
_ => AudioCodec::Unknown,
};
let mut info = AudioInfo::empty(codec, bytes.len());
info.channels = Some(channels).filter(|&c| c > 0);
info.sample_rate = Some(rate).filter(|&r| r > 0);
info.bits_per_sample = Some(bits).filter(|&b| b > 0);
match codec {
AudioCodec::Pcm | AudioCodec::PcmFloat => {
if let (Some(d), true) = (data_bytes, channels > 0 && bits > 0) {
let frame = channels as u64 * (bits as u64 / 8);
if frame > 0 {
let spc = d / frame;
info.samples_per_channel = Some(spc);
if rate > 0 {
info.duration_secs = Some(spc as f32 / rate as f32);
}
}
}
}
AudioCodec::Xma | AudioCodec::Xma2 => {
if let Some(d) = data_bytes {
info.xma_packets = Some((d / XMA_BYTES_PER_PACKET as u64) as u32);
}
}
_ => {}
}
Some(info)
}
// ── PCM decode → GameAudio ─────────────────────────────────────────────────────
/// An audio clip decoded to raw interleaved `f32` PCM (`L R L R …`).
#[derive(Debug, Clone)]
pub struct GameAudio {
/// Raw PCM samples (interleaved for stereo: L R L R ...)
pub samples: Vec<f32>,
pub channels: u16,
pub sample_rate: u32,
}
impl GameAudio {
/// Parse audio from raw bytes.
///
/// Currently only WAV/PCM is supported. For XMA files, pre-convert using:
/// `ffmpeg -i file.xma file.wav`
pub fn from_bytes(bytes: &[u8]) -> Result<Self, AudioError> {
// Check for WAV magic
if bytes.len() >= 4 && &bytes[..4] == b"RIFF" {
return Self::from_wav(bytes);
/// Decode a RIFF/WAVE **PCM** file (int 8/16/24-bit or 32-bit float) to
/// interleaved `f32`. XMA/XMA2 return [`AudioError::NeedsDecoder`].
pub fn from_wav(bytes: &[u8]) -> Result<Self, AudioError> {
let info = parse_riff_wave(bytes).ok_or(AudioError::Unrecognized)?;
if info.codec.needs_decoder() {
return Err(AudioError::NeedsDecoder(info.codec));
}
let channels = info.channels.ok_or(AudioError::Malformed("no channels"))?;
let rate = info.sample_rate.ok_or(AudioError::Malformed("no sample rate"))?;
let bits = info.bits_per_sample.ok_or(AudioError::Malformed("no bit depth"))?;
// XMA magic bytes
if bytes.len() >= 4 && (
&bytes[..4] == b"XWB\0" || // Wave bank
&bytes[..4] == b"XMA2" // Raw XMA2
) {
return Err(AudioError::XmaNotSupported);
}
// Locate the `data` chunk body.
let (off, len) = riff_data_span(bytes).ok_or(AudioError::Malformed("no data chunk"))?;
let data = &bytes[off..off + len];
Err(AudioError::UnknownMagic(
bytes[..4.min(bytes.len())].try_into().unwrap_or([0u8; 4])
))
}
fn from_wav(bytes: &[u8]) -> Result<Self, AudioError> {
// Minimal WAV parser for PCM files
// A proper implementation should use the `hound` crate:
// https://crates.io/crates/hound
//
// TODO: Add `hound = "3"` to Cargo.toml and implement properly
Err(AudioError::Parse(
"WAV parsing TODO: add `hound` crate and implement".to_string()
))
let samples: Vec<f32> = match (info.codec, bits) {
(AudioCodec::Pcm, 8) => data.iter().map(|&b| (b as f32 - 128.0) / 128.0).collect(),
(AudioCodec::Pcm, 16) => data
.chunks_exact(2)
.map(|c| i16::from_le_bytes([c[0], c[1]]) as f32 / 32768.0)
.collect(),
(AudioCodec::Pcm, 24) => data
.chunks_exact(3)
.map(|c| {
let v = ((c[2] as i32) << 16) | ((c[1] as i32) << 8) | c[0] as i32;
let v = (v << 8) >> 8; // sign-extend 24→32
v as f32 / 8_388_608.0
})
.collect(),
(AudioCodec::PcmFloat, 32) => data
.chunks_exact(4)
.map(|c| f32::from_le_bytes([c[0], c[1], c[2], c[3]]))
.collect(),
_ => return Err(AudioError::UnsupportedPcm { tag: 0, bits }),
};
Ok(Self { samples, channels, sample_rate: rate })
}
}
/// Byte span (offset, length) of the WAVE `data` chunk body, if present.
fn riff_data_span(bytes: &[u8]) -> Option<(usize, usize)> {
if bytes.len() < 12 || &bytes[..4] != b"RIFF" || &bytes[8..12] != b"WAVE" {
return None;
}
let le32 =
|o: usize| u32::from_le_bytes([bytes[o], bytes[o + 1], bytes[o + 2], bytes[o + 3]]) as usize;
let mut pos = 12;
while pos + 8 <= bytes.len() {
let size = le32(pos + 4);
let body = pos + 8;
if &bytes[pos..pos + 4] == b"data" {
let len = size.min(bytes.len().saturating_sub(body));
return Some((body, len));
}
pos = body + size + (size & 1);
}
None
}
#[cfg(test)]
mod tests {
use super::*;
/// Build a minimal 16-bit PCM WAV in memory (stereo, 2 frames).
fn tiny_wav() -> Vec<u8> {
let mut v = Vec::new();
let data: [i16; 4] = [1000, -1000, 32767, -32768]; // L R L R
let data_bytes: Vec<u8> = data.iter().flat_map(|s| s.to_le_bytes()).collect();
v.extend_from_slice(b"RIFF");
v.extend_from_slice(&(36 + data_bytes.len() as u32).to_le_bytes());
v.extend_from_slice(b"WAVE");
v.extend_from_slice(b"fmt ");
v.extend_from_slice(&16u32.to_le_bytes());
v.extend_from_slice(&WAVE_FORMAT_PCM.to_le_bytes());
v.extend_from_slice(&2u16.to_le_bytes()); // channels
v.extend_from_slice(&48000u32.to_le_bytes());
v.extend_from_slice(&(48000u32 * 2 * 2).to_le_bytes());
v.extend_from_slice(&4u16.to_le_bytes()); // block align
v.extend_from_slice(&16u16.to_le_bytes()); // bits
v.extend_from_slice(b"data");
v.extend_from_slice(&(data_bytes.len() as u32).to_le_bytes());
v.extend_from_slice(&data_bytes);
v
}
#[test]
fn probe_and_decode_pcm_wav() {
let wav = tiny_wav();
let info = AudioInfo::probe(&wav);
assert_eq!(info.codec, AudioCodec::Pcm);
assert_eq!(info.channels, Some(2));
assert_eq!(info.sample_rate, Some(48000));
assert_eq!(info.samples_per_channel, Some(2));
let audio = GameAudio::from_wav(&wav).unwrap();
assert_eq!(audio.channels, 2);
assert_eq!(audio.samples.len(), 4);
assert!((audio.samples[2] - 0.99997).abs() < 1e-3); // 32767/32768
}
#[test]
fn probe_xma2_riff_reports_metadata_not_decode() {
// Minimal RIFF/WAVE with an XMA2 fmt tag.
let mut v = Vec::new();
v.extend_from_slice(b"RIFF");
v.extend_from_slice(&200u32.to_le_bytes());
v.extend_from_slice(b"WAVE");
v.extend_from_slice(b"fmt ");
v.extend_from_slice(&16u32.to_le_bytes());
v.extend_from_slice(&WAVE_FORMAT_XMA2.to_le_bytes());
v.extend_from_slice(&2u16.to_le_bytes());
v.extend_from_slice(&44100u32.to_le_bytes());
v.extend_from_slice(&0u32.to_le_bytes());
v.extend_from_slice(&0u16.to_le_bytes());
v.extend_from_slice(&0u16.to_le_bytes());
let info = AudioInfo::probe(&v);
assert_eq!(info.codec, AudioCodec::Xma2);
assert_eq!(info.channels, Some(2));
assert!(info.codec.needs_decoder());
assert!(matches!(
GameAudio::from_wav(&v),
Err(AudioError::NeedsDecoder(AudioCodec::Xma2))
));
}
#[test]
fn raw_high_entropy_blob_reads_as_raw_xma2() {
// A pseudo-random 8 KB blob (no magic) → RawXma2 with a packet count.
let mut b = vec![0u8; 8192];
let mut x = 0x2545_F491u32;
for v in b.iter_mut() {
x ^= x << 13;
x ^= x >> 17;
x ^= x << 5;
*v = (x & 0xFF) as u8;
}
let info = AudioInfo::probe(&b);
assert_eq!(info.codec, AudioCodec::RawXma2);
assert_eq!(info.xma_packets, Some(4)); // 8192 / 2048
}
#[test]
fn structured_low_entropy_blob_is_unknown_not_audio() {
let b = b"IDXD............a bunch of readable ASCII text fields....".repeat(40);
assert_eq!(AudioInfo::probe(&b).codec, AudioCodec::Unknown);
}
}

View File

@@ -0,0 +1,64 @@
//! Font metadata for the embedded UI fonts.
//!
//! The packs carry sfnt fonts (`\x00\x01\x00\x00` TrueType, `OTTO` OpenType/CFF)
//! and TrueType Collections (`ttcf`). We only need light metadata to *present*
//! them — family name, face count, glyph count — which `ttf-parser` extracts
//! safely (malformed input yields `None`, never a panic). The raw bytes are
//! handed to egui separately for a live sample.
use ttf_parser::{fonts_in_collection, name_id, Face};
/// Light font metadata for the viewer.
#[derive(Debug, Clone, PartialEq)]
pub struct FontInfo {
/// "TrueType", "OpenType (CFF)", or "TrueType Collection".
pub kind: &'static str,
/// Family / full name (best-effort; empty if the name table is unreadable).
pub family: String,
/// Number of faces (1 for a plain sfnt, N for a `ttcf` collection).
pub faces: u32,
/// Glyph count of the first face.
pub glyphs: u16,
}
/// Whether `bytes` starts with a recognized font signature.
pub fn is_font(bytes: &[u8]) -> bool {
matches!(
bytes.get(0..4),
Some(b"\x00\x01\x00\x00") | Some(b"OTTO") | Some(b"true") | Some(b"ttcf")
)
}
/// Extract light metadata, or `None` if the font can't be parsed.
pub fn parse_info(bytes: &[u8]) -> Option<FontInfo> {
let is_collection = bytes.get(0..4) == Some(b"ttcf");
let faces = fonts_in_collection(bytes).unwrap_or(1).max(1);
// Face 0 gives the representative family + glyph count.
let face = Face::parse(bytes, 0).ok()?;
let glyphs = face.number_of_glyphs();
let kind = if is_collection {
"TrueType Collection"
} else if bytes.get(0..4) == Some(b"OTTO") {
"OpenType (CFF)"
} else {
"TrueType"
};
// Prefer the typographic/full family name; fall back to any readable name.
let family = face
.names()
.into_iter()
.filter(|n| n.name_id == name_id::FULL_NAME || n.name_id == name_id::FAMILY)
.find_map(|n| n.to_string())
.or_else(|| face.names().into_iter().find_map(|n| n.to_string()))
.unwrap_or_default();
Some(FontInfo {
kind,
family,
faces,
glyphs,
})
}

View File

@@ -171,6 +171,39 @@ impl IdxdObject {
}
out
}
/// A short identity string for the object: the first of `ID` / `Name` /
/// `Model` present, else the schema hash. Shared by the CLI `pak list` and
/// the GUI pack browser.
pub fn identity(&self) -> String {
for key in ["ID", "Name", "Model"] {
if let Some(v) = self.get_raw(key) {
return format!("{key}={v}");
}
}
format!("schema {:08x}", self.schema_hash)
}
/// Recover this entry's original TOC path (e.g. `unit\rou_f001.tbl`) from its
/// identity/pool tokens by re-hashing candidates against `name_hash` with the
/// known path schemes. Returns `None` when no scheme reproduces the hash.
///
/// Shared by the CLI `pak list` and the GUI pack browser so both surface the
/// same recovered names. See [`crate::hash::recover_toc_name`].
pub fn recover_toc_path(&self, name_hash: u32) -> Option<String> {
let mut cands: Vec<&str> = Vec::new();
for key in ["ID", "Name", "Model"] {
if let Some(v) = self.get_raw(key) {
cands.push(v);
}
}
for t in self.tokens() {
if t.contains('_') || t.len() >= 5 {
cands.push(t.as_str());
}
}
crate::hash::recover_toc_name(name_hash, &cands)
}
}
/// Extract the trailing string pool. Finds the smallest offset whose suffix is

View File

@@ -0,0 +1,187 @@
//! IXUD localized-string / subtitle table reader.
//!
//! `IXUD` entries in the language packs (`dat/movie/<lang>.pak`, the
//! `GP_MAIN_GAME_<lang>` tables, …) hold localized UTF-16**BE** strings. The
//! movie language packs use them as **subtitle cue lists**: a `SUBTITLE` header
//! followed by alternating `(text, timecode)` tokens, e.g.
//!
//! ```text
//! SUBTITLE
//! "Calm down!" 00:09.40-00:11.00
//! "Why are you taking Margras away?" 00:18.50-00:19.70
//! ```
//!
//! Reference-style tracks store a message key (`MSG_DEMO_240`) plus a single
//! start timecode instead of inline text; the actual string then lives in a
//! separate global table (not resolved here).
//!
//! ## Binary layout (as far as needed)
//! ```text
//! 0x00 4 Magic "IXUD"
//! 0x04 4 version (1)
//! 0x08 4 schema/type hash (constant 0x6CC83E70)
//! .. .. record directory { key_hash u32, offset u32, len u32 } × n
//! .. .. UTF-16BE string pool, NUL-terminated entries
//! ```
//! We don't need the directory to *present* the track — decoding the pool and
//! pairing tokens after the `SUBTITLE` header is enough and robust.
/// Magic at the start of every IXUD entry.
pub const IXUD_MAGIC: [u8; 4] = *b"IXUD";
/// One subtitle cue: a start time (seconds), optional end time, and the text
/// (or a `MSG_*` reference key for reference-style tracks).
#[derive(Debug, Clone, PartialEq)]
pub struct Cue {
pub start: f32,
pub end: Option<f32>,
pub text: String,
}
/// A decoded subtitle track.
#[derive(Debug, Clone, Default, PartialEq)]
pub struct Subtitle {
pub cues: Vec<Cue>,
}
impl Subtitle {
/// True when the track carries no inline text — every cue is a `MSG_*`
/// reference whose string lives in a separate global table.
pub fn is_reference_only(&self) -> bool {
!self.cues.is_empty() && self.cues.iter().all(|c| c.text.starts_with("MSG_"))
}
}
/// Whether `bytes` is an IXUD entry.
pub fn is_ixud(bytes: &[u8]) -> bool {
bytes.len() >= 4 && bytes[0..4] == IXUD_MAGIC
}
/// Parse an IXUD entry as a subtitle track. Returns `None` if not IXUD.
/// Malformed pools yield as many well-formed cues as can be recovered.
pub fn parse(bytes: &[u8]) -> Option<Subtitle> {
if !is_ixud(bytes) {
return None;
}
// Decode the whole payload as UTF-16BE and split into NUL-terminated,
// printable runs. The binary header/directory decodes to control/CJK-range
// junk that we skip by seeking to the `SUBTITLE` marker.
let tokens = utf16be_tokens(bytes);
let start = tokens
.iter()
.position(|t| t.ends_with("SUBTITLE"))
.map(|i| i + 1)
.unwrap_or(0);
let rest = &tokens[start..];
// Pair (text, timecode). A token that parses as a timecode closes the cue
// opened by the preceding text token; unpaired tokens are skipped so a
// single glitch doesn't desync the rest.
let mut cues = Vec::new();
let mut i = 0;
while i + 1 < rest.len() {
if let Some((s, e)) = parse_timing(&rest[i + 1]) {
cues.push(Cue {
start: s,
end: e,
text: rest[i].clone(),
});
i += 2;
} else {
i += 1;
}
}
Some(Subtitle { cues })
}
/// Split the payload (interpreted as UTF-16BE) into printable, NUL-delimited
/// tokens. Control chars terminate the current token.
fn utf16be_tokens(bytes: &[u8]) -> Vec<String> {
let mut tokens = Vec::new();
let mut cur = String::new();
for pair in bytes.chunks_exact(2) {
let u = u16::from_be_bytes([pair[0], pair[1]]);
match char::from_u32(u as u32) {
Some(c) if !c.is_control() => cur.push(c),
_ => {
if !cur.is_empty() {
tokens.push(std::mem::take(&mut cur));
}
}
}
}
if !cur.is_empty() {
tokens.push(cur);
}
tokens
}
/// Parse `MM:SS.ss` (`" 9:04.40"` style, minutes may be blank/space-padded) or
/// a `start-end` range into seconds.
fn parse_timing(s: &str) -> Option<(f32, Option<f32>)> {
let one = |p: &str| -> Option<f32> {
let (mm, ss) = p.trim().split_once(':')?;
let m: f32 = if mm.trim().is_empty() {
0.0
} else {
mm.trim().parse().ok()?
};
let sec: f32 = ss.trim().parse().ok()?;
Some(m * 60.0 + sec)
};
match s.split_once('-') {
Some((a, b)) => Some((one(a)?, Some(one(b)?))),
None => Some((one(s)?, None)),
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Build a synthetic IXUD payload: an 8-byte header stand-in + UTF-16BE
/// NUL-terminated tokens.
fn synth(tokens: &[&str]) -> Vec<u8> {
let mut b = IXUD_MAGIC.to_vec();
b.extend_from_slice(&[0, 0, 0, 1, 0x6C, 0xC8, 0x3E, 0x70]); // version + schema
for t in tokens {
for u in t.encode_utf16() {
b.extend_from_slice(&u.to_be_bytes());
}
b.extend_from_slice(&[0, 0]); // NUL
}
b
}
#[test]
fn inline_subtitle_track() {
let b = synth(&[
"SUBTITLE",
"Calm down!",
"00:09.40-00:11.00",
"Let go of me!",
"00:13.10-00:14.50",
]);
let sub = parse(&b).unwrap();
assert_eq!(sub.cues.len(), 2);
assert_eq!(sub.cues[0].text, "Calm down!");
assert_eq!(sub.cues[0].start, 9.4);
assert_eq!(sub.cues[0].end, Some(11.0));
assert!(!sub.is_reference_only());
}
#[test]
fn reference_track() {
let b = synth(&["SUBTITLE", "MSG_DEMO_240", "00:01.00", "MSG_DEMO_241", "00:03.20"]);
let sub = parse(&b).unwrap();
assert_eq!(sub.cues.len(), 2);
assert_eq!(sub.cues[0].end, None);
assert!(sub.is_reference_only());
}
#[test]
fn rejects_non_ixud() {
assert!(parse(b"IDXD\0\0\0\0").is_none());
}
}

View File

@@ -25,6 +25,21 @@ pub mod pak;
// IDXD reflective object (ship / weapon / effect / menu definitions) reader
pub mod idxd;
// IXUD localized string / subtitle table reader
pub mod ixud;
// Embedded font (OTF / TTF / ttcf) metadata
pub mod font;
// T8aD 2D UI/HUD texture
pub mod t8ad;
// RATC nested resource bundle
pub mod ratc;
// LSTA sprite list (inline T8aD frames)
pub mod lsta;
// XISO reading is not available in-browser
#[cfg(not(target_arch = "wasm32"))]
pub mod xiso;
@@ -36,7 +51,13 @@ pub mod mesh;
pub mod audio;
/// Re-export the most commonly used types at the crate root.
pub use audio::{AudioCodec, AudioInfo, GameAudio};
pub use font::FontInfo;
pub use idxd::{IdxdError, IdxdObject};
pub use ixud::{Cue, Subtitle};
pub use mesh::{GameMesh, Xbg7Model};
pub use ratc::RatcChild;
pub use t8ad::T8adImage;
pub use pak::{PakArchive, PakEntry, PakError};
pub use texture::{X360Texture, X360TextureFormat};
pub use vfs::{GameAssets, VfsError};

View File

@@ -0,0 +1,78 @@
//! `LSTA` — a sprite list: a header followed by N inline [`T8aD`](crate::t8ad)
//! frames concatenated back-to-back.
//!
//! A `count` lives at `@0x04`, but a few entries disagree with the actual frame
//! count, so we walk by the `T8aD` magic instead (robust) and decode each frame
//! from its slice up to the next frame (or end).
use crate::t8ad::{self, T8adImage, T8AD_MAGIC};
/// Magic at the start of an LSTA sprite list.
pub const LSTA_MAGIC: [u8; 4] = *b"LSTA";
/// Whether `bytes` is an LSTA list.
pub fn is_lsta(bytes: &[u8]) -> bool {
bytes.len() >= 4 && bytes[0..4] == LSTA_MAGIC
}
/// Decode all inline T8aD frames. Frames that don't decode (unsupported T8aD
/// variant) are skipped. Returns `None` only for non-LSTA input.
pub fn parse(bytes: &[u8]) -> Option<Vec<T8adImage>> {
if !is_lsta(bytes) {
return None;
}
// Offsets of each inline T8aD frame (search past the 4-byte magic).
let mut offs = Vec::new();
let mut i = 4;
while i + 4 <= bytes.len() {
if bytes[i..i + 4] == T8AD_MAGIC {
offs.push(i);
i += 4;
} else {
i += 1;
}
}
let mut frames = Vec::with_capacity(offs.len());
for (idx, &off) in offs.iter().enumerate() {
let next = offs.get(idx + 1).copied().unwrap_or(bytes.len());
if let Some(img) = t8ad::parse(&bytes[off..next]) {
frames.push(img);
}
}
Some(frames)
}
#[cfg(test)]
mod tests {
use super::*;
fn t8ad_frame(w: u32, h: u32) -> Vec<u8> {
let mut b = vec![0u8; 64];
b[0..4].copy_from_slice(&T8AD_MAGIC);
b[0x14..0x18].copy_from_slice(&w.to_be_bytes());
b[0x18..0x1c].copy_from_slice(&h.to_be_bytes());
b[0x1c..0x20].copy_from_slice(&1u32.to_be_bytes());
b.extend_from_slice(&vec![0x80u8; (w * h * 4) as usize]);
b
}
#[test]
fn walks_inline_frames() {
let mut b = LSTA_MAGIC.to_vec();
b.extend_from_slice(&2u32.to_be_bytes());
b.extend_from_slice(&[0u8; 15]); // header remainder
b.extend_from_slice(&t8ad_frame(4, 4));
b.extend_from_slice(&t8ad_frame(2, 3));
let frames = parse(&b).unwrap();
assert_eq!(frames.len(), 2);
assert_eq!((frames[0].width, frames[0].height), (4, 4));
assert_eq!((frames[1].width, frames[1].height), (2, 3));
}
#[test]
fn rejects_non_lsta() {
assert!(parse(b"T8aD....").is_none());
}
}

View File

@@ -1,71 +1,808 @@
//! Mesh format parsing — TO BE REVERSE ENGINEERED.
//! XBG7 mesh geometry decoder (geometry resources inside XPR2 containers).
//!
//! Project Sylpheed uses a completely custom engine with unknown mesh formats.
//! This module is a scaffold: populate these structs as you discover the
//! actual binary layout using a hex editor (010 Editor) and Ghidra.
//! ## Clean-room note
//!
//! ## RE Strategy for Meshes
//! This format was reverse-engineered **purely by static observation of the
//! retail disc's `hidden/resource3d/*.xpr` files** (hex inspection + geometric
//! validation of the recovered triangles). No game code was decompiled or
//! copied. See `docs/re/structures/xbg7-mesh.md` for the evidence log.
//!
//! 1. Extract the game files using xdvdfs
//! 2. Look for files with extensions like .mdl, .mesh, .geo, .obj, .pak
//! 3. Open them in a hex editor — look for:
//! - Repeating patterns of 12 bytes (XYZ float vertices)
//! - Groups of 3 uint16s (triangle indices)
//! - Header magic bytes
//! 4. Cross-reference with Ghidra's XEX analysis to find the load functions
//! ## Where XBG7 lives
//!
//! ## Useful tools
//! - 010 Editor with binary templates
//! - Noesis (can preview many console formats)
//! - binrw (this project) for writing the parser once the format is known
//! Ship / weapon / prop models are `XPR2` containers (see [`crate::texture`]).
//! Their resource directory holds `TX2D` texture resources **and** one or more
//! `XBG7` geometry resources. The `XBG7` *descriptor* (at the resource's
//! `data_offset`) is a scene/material graph; the actual vertex and index
//! buffers live in the container's shared data section (from `header_size`).
//!
//! ## The "simple" layout decoded here (CONFIRMED)
//!
//! For single-stream models (weapons, simple props — 36 of the 166 disc models)
//! the data section is a straight sequence of sub-meshes, each:
//!
//! ```text
//! [ index buffer : idx_count × u16 big-endian ] triangle list
//! [ 12-byte vertex-buffer header (contents undecoded) ]
//! [ vertex buffer : vtx_count × stride bytes ] (declaration-driven)
//! (pad to 16 bytes → next sub-mesh)
//! ```
//!
//! The vertex layout is **not fixed** — it comes from a **vertex declaration**
//! in the descriptor: a table of `{offset, format-code, usage}` triples (usage
//! `0x00` POSITION `f32×3`, `0x03` NORMAL `f16×4`, `0x05` TEXCOORD `f16×2`).
//! Models omit UV or use fewer elements, so stride varies (20 = pos+normal,
//! 24 = pos+normal+uv, …). Each element is read in naive big-endian component
//! order. Correct alignment is pinned by the recovered normals being exactly
//! unit-length. **Cross-checked against a Canary GPU vertex-fetch capture**,
//! which confirmed the primitive type (triangle list), formats, and offsets —
//! see `docs/re/structures/xbg7-mesh.md`.
//!
//! `vtx_count` / `idx_count` come from per-sub-mesh records in the descriptor:
//! a `[vtx_count:u32][0:u32][idx_count:u32][tail:u32]` tuple (big-endian), read
//! in file order. Every index is validated to be `< vtx_count`; if any
//! sub-mesh fails to carve cleanly the whole model is rejected
//! ([`MeshError::UnsupportedLayout`]) rather than emitting garbage.
//!
//! ## Not yet decoded
//!
//! The hero-ship *body* meshes (`DeltaSaber_*.xpr` `f004`, and ~100 other
//! models) use a more complex **multi-stream** layout — separate position /
//! attribute streams at descriptor-addressed offsets, quantized positions —
//! which is not handled here and is cleanly declined.
use crate::texture::{Xpr2Header, Xpr2ResourceEntry};
use binrw::BinRead;
use std::io::Cursor;
use thiserror::Error;
#[derive(Debug, Error)]
pub enum MeshError {
#[error("Unknown mesh magic: {0:?}")]
UnknownMagic([u8; 4]),
#[error("Unsupported mesh version: {0}")]
UnsupportedVersion(u32),
#[error("Not an XPR2 container")]
NotXpr2,
#[error("No XBG7 geometry resource in container")]
NoGeometry,
#[error("Mesh layout not supported (multi-stream / quantized body mesh)")]
UnsupportedLayout,
#[error("Parse error: {0}")]
Parse(String),
Parse(#[from] binrw::Error),
}
/// A decoded 3D mesh ready for Bevy.
/// Vertex positions, normals, UVs, and indices.
#[derive(Debug, Default, Clone)]
pub struct GameMesh {
/// Interleaved vertex positions [x, y, z, x, y, z, ...]
/// Vertex positions in model space `[x, y, z]`.
pub positions: Vec<[f32; 3]>,
/// Vertex normals [nx, ny, nz, ...]
/// Vertex normals `[nx, ny, nz]` — empty when not stored (compute smooth
/// normals from geometry instead).
pub normals: Vec<[f32; 3]>,
/// UV texture coordinates [u, v, ...]
/// Texture coordinates `[u, v]`. **Best-guess channel** (attr halves 0 & 2)
/// pending in-game visual confirmation — see the module doc.
pub uvs: Vec<[f32; 2]>,
/// Triangle list indices
/// Triangle-list indices (3 per triangle).
pub indices: Vec<u32>,
/// Name of this mesh (if available in the file)
/// Sub-mesh / node name from the descriptor, when available.
pub name: Option<String>,
}
impl GameMesh {
/// Parse a mesh from raw bytes.
/// A model = the set of sub-meshes recovered from one XPR2 container's first
/// XBG7 resource, plus the resource's name.
#[derive(Debug, Default, Clone)]
pub struct Xbg7Model {
pub name: String,
pub meshes: Vec<GameMesh>,
}
impl Xbg7Model {
/// Total vertex / triangle counts across all sub-meshes.
pub fn totals(&self) -> (usize, usize) {
let v = self.meshes.iter().map(|m| m.positions.len()).sum();
let t = self.meshes.iter().map(|m| m.indices.len() / 3).sum();
(v, t)
}
/// Decode the geometry of the first XBG7 resource in an XPR2 container.
///
/// TODO: implement once the actual file format is identified.
/// Currently returns an error until the format is reverse engineered.
pub fn from_bytes(_bytes: &[u8]) -> Result<Vec<Self>, MeshError> {
// ┌──────────────────────────────────────────────────────────────┐
// │ REVERSE ENGINEERING TODO │
// │ │
// │ Steps to implement this: │
// │ 1. Find mesh files in the extraction (look for .mdl etc.) │
// │ 2. Identify the file format using identify_format() in vfs │
// │ 3. Use 010 Editor to map the binary structure │
// │ 4. Add binrw #[derive(BinRead)] structs above │
// │ 5. Implement this function to parse and return meshes │
// └──────────────────────────────────────────────────────────────┘
Err(MeshError::Parse(
"Mesh format not yet reverse engineered. \
See RE TODO in src/mesh.rs".to_string()
))
/// Returns [`MeshError::UnsupportedLayout`] for models whose data section
/// does not carve cleanly under the simple single-stream layout (the
/// complex body meshes) — never partial / garbage geometry.
pub fn from_xpr2(bytes: &[u8]) -> Result<Self, MeshError> {
if bytes.len() < 16 || &bytes[..4] != b"XPR2" {
return Err(MeshError::NotXpr2);
}
let mut cur = Cursor::new(bytes);
let header = Xpr2Header::read(&mut cur)?;
let mut xbg: Option<Xpr2ResourceEntry> = None;
for _ in 0..header.num_resources {
let e = Xpr2ResourceEntry::read(&mut cur)?;
if &e.type_tag == b"XBG7" {
xbg = Some(e);
break;
}
}
let xbg = xbg.ok_or(MeshError::NoGeometry)?;
const DIR_BASE: usize = 0x10;
let desc = xbg.data_offset as usize + DIR_BASE;
let desc_end = (desc + xbg.descriptor_size as usize).min(bytes.len());
if desc >= bytes.len() {
return Err(MeshError::UnsupportedLayout);
}
// Resource name (for labelling).
let name = read_cstr(bytes, xbg.name_offset as usize + DIR_BASE)
.unwrap_or_else(|| "XBG7".to_string());
// Parse the vertex declaration (element offsets/formats + stride). The
// XBG7 layout is NOT fixed-stride — models omit UV or use fewer elements
// (stride 20 = pos+normal, stride 24 = pos+normal+uv, …). Confirmed
// against a Canary GPU vertex-fetch capture (see docs/re/xbg7-mesh.md).
let decl = parse_vertex_decl(&bytes[desc..desc_end])
.ok_or(MeshError::UnsupportedLayout)?;
// Extract the ordered list of sub-mesh (vtx_count, idx_count) records.
let records = submesh_records(&bytes[desc..desc_end]);
if records.is_empty() {
return Err(MeshError::UnsupportedLayout);
}
// Carve the data section sequentially.
let base = header.header_size as usize;
let mut off = 0usize; // relative to `base`
let mut meshes = Vec::new();
for (vtx_count, idx_count) in records {
// Each sub-mesh block is `[12-byte header][index buffer][vertex
// buffer]` — the SAME layout as stage resources (see
// `stage_models`). The header must be skipped: reading indices from
// the block start instead treats the 12 header bytes as 6 junk
// indices (2 leading degenerate triangles — the stray-triangle
// artifact) and drops the last 6 real indices. Skipping it leaves the
// vertex buffer at the identical offset (`+12 + idx_bytes`), so
// coverage is unchanged; only the triangle list is corrected.
let ib = base + off + VERTEX_BUFFER_GAP;
let ie = ib + idx_count * 2;
let vb = ie;
let ve = vb + vtx_count * decl.stride;
if ve > bytes.len() {
return Err(MeshError::UnsupportedLayout);
}
// Indices (u16 BE), validated against the sub-mesh vertex count.
let mut indices = Vec::with_capacity(idx_count);
for k in 0..idx_count {
let i = be16(bytes, ib + k * 2) as u32;
if i >= vtx_count as u32 {
return Err(MeshError::UnsupportedLayout);
}
indices.push(i);
}
// Vertices per the declaration. The .xpr stores each element in
// naive big-endian component order (f32 / f16 read at consecutive
// offsets) — the GPU's `k8in32` fetch endianness applies to the
// rearranged guest-memory copy, not to these file bytes.
let mut positions = Vec::with_capacity(vtx_count);
let mut normals = Vec::with_capacity(vtx_count);
let mut uvs = Vec::with_capacity(vtx_count);
let mut normal_len_sum = 0.0f32;
for v in 0..vtx_count {
let o = vb + v * decl.stride;
// POSITION: f32×3 big-endian.
let p = o + decl.pos_offset;
let x = bef(bytes, p);
let y = bef(bytes, p + 4);
let z = bef(bytes, p + 8);
if !(x.is_finite() && y.is_finite() && z.is_finite()) {
return Err(MeshError::UnsupportedLayout);
}
positions.push([x, y, z]);
// NORMAL: f16×4 (use xyz).
if let Some(no) = decl.normal_offset {
let nb = o + no;
let nx = half(bytes, nb);
let ny = half(bytes, nb + 2);
let nz = half(bytes, nb + 4);
normal_len_sum += (nx * nx + ny * ny + nz * nz).sqrt();
normals.push([nx, ny, nz]);
}
// TEXCOORD: f16×2.
if let Some(uo) = decl.uv_offset {
let ub = o + uo;
uvs.push([half(bytes, ub), half(bytes, ub + 2)]);
}
}
// Sanity gate: when the declaration has a normal element, a correctly
// aligned vertex buffer yields unit-length normals. A mean far from 1
// means the layout does not fit (wrong stride / offset) — decline
// rather than emit garbage.
if decl.normal_offset.is_some() {
let mean = normal_len_sum / vtx_count.max(1) as f32;
if !(0.5..=2.0).contains(&mean) {
return Err(MeshError::UnsupportedLayout);
}
}
meshes.push(GameMesh {
positions,
normals,
uvs,
indices,
name: None,
});
off = align16(ve) - base;
}
Ok(Xbg7Model { name, meshes })
}
/// Decode **every** locatable XBG7 geometry resource in a container.
///
/// "Stage" containers (`hidden/resource3d/Stage_*.xpr`) are collections of
/// many enemy / prop sub-models, each an independent XBG7 resource. Unlike
/// the single-stream weapon layout (index buffer immediately followed by its
/// vertex buffer), a stage's index buffers and vertex buffers live in
/// **separate grouped pools**, and the container stores each resource's
/// buffer *sizes* (index count via the marker, vertex count 32 bytes before
/// it) but **not** an explicit data offset — the on-disc block layout is a
/// separate allocation order we have not reversed.
///
/// Rather than guess that order, each resource's `[index buffer][vertex
/// buffer]` block is located by **content**: the unique offset in the data
/// section where (a) all `index_count` indices are `< vertex_count`, (b) the
/// stored normals are unit length, and (c) the resulting triangles are
/// non-degenerate with a real spatial extent. This signature is strong
/// enough to pin a block unambiguously in a multi-megabyte file. Resources
/// that cannot be located and validated this way are **skipped** (never
/// emitted as garbage) — including the biggest hero bodies, which use the
/// same quantized/complex layout that [`Xbg7Model::from_xpr2`] declines.
///
/// Returns one [`Xbg7Model`] per decoded resource (empty if none decode).
pub fn stage_models(bytes: &[u8]) -> Vec<Xbg7Model> {
let mut out = Vec::new();
if bytes.len() < 16 || &bytes[..4] != b"XPR2" {
return out;
}
let mut cur = Cursor::new(bytes);
let header = match Xpr2Header::read(&mut cur) {
Ok(h) => h,
Err(_) => return out,
};
let data_base = header.header_size as usize;
if data_base >= bytes.len() {
return out;
}
// ── Collect every XBG7 resource's parameters up front. ──
const DIR_BASE: usize = 0x10;
struct Res {
name: String,
index_count: usize,
vtx_count: usize,
decl: VertexDecl,
}
let mut resources: Vec<Res> = Vec::new();
for _ in 0..header.num_resources {
let e = match Xpr2ResourceEntry::read(&mut cur) {
Ok(e) => e,
Err(_) => break,
};
if &e.type_tag != b"XBG7" {
continue;
}
let desc = e.data_offset as usize + DIR_BASE;
let desc_end = (desc + e.descriptor_size as usize).min(bytes.len());
if desc >= bytes.len() || desc_end <= desc {
continue;
}
let d = &bytes[desc..desc_end];
let (mk_rel, index_count) = match find_index_marker(d) {
Some(m) => m,
None => continue,
};
let decl = match parse_vertex_decl(d) {
Some(v) => v,
None => continue,
};
// Total vertex count is stored 32 bytes before the index marker.
if mk_rel < 32 {
continue;
}
let vtx_count = be32(d, mk_rel - 32) as usize;
if !(3..=400_000).contains(&vtx_count) || index_count < 3 {
continue;
}
// Anchoring relies on the unit-normal signature; skip resources with
// no NORMAL element (too ambiguous to pin safely).
if decl.normal_offset.is_none() {
continue;
}
let name = read_cstr(bytes, e.name_offset as usize + DIR_BASE)
.unwrap_or_else(|| "XBG7".to_string());
resources.push(Res {
name,
index_count,
vtx_count,
decl,
});
}
if resources.is_empty() {
return out;
}
// ── One O(file) pass per distinct stride: find vertex-block *starts*. ──
//
// Each geometry block is `[12B header][index buffer][vertex buffer]`, and
// the blocks are scattered among texture data with no stored offset. But
// a vertex buffer is a run of stride-sized records whose NORMAL (f16×4 at
// +12) is unit length; a *block start* is the unique offset where that
// run begins — the previous stride slot is NOT a unit-normal vertex (it's
// index bytes / header). Collecting those starts turns the per-resource
// search from O(file) into a scan of a few hundred candidates.
let mut strides: Vec<usize> = resources.iter().map(|r| r.decl.stride).collect();
strides.sort_unstable();
strides.dedup();
let mut starts_by_stride: std::collections::BTreeMap<usize, Vec<usize>> =
std::collections::BTreeMap::new();
for &s in &strides {
starts_by_stride.insert(s, vertex_run_starts(bytes, data_base, s));
}
for r in &resources {
let starts = &starts_by_stride[&r.decl.stride];
if let Some(mesh) = anchor_pool_mesh(
bytes,
starts,
r.index_count,
r.vtx_count,
&r.decl,
) {
out.push(Xbg7Model {
name: r.name.clone(),
meshes: vec![mesh],
});
}
}
out
}
}
/// Scan the data section for offsets that begin a `stride`-sized unit-normal
/// vertex run (NORMAL is `f16×4` at vertex offset +12). A run *start* is an
/// offset whose normal is unit while the preceding stride slot's is not — i.e.
/// the first vertex of a buffer, not a mid-buffer position. Returns the sorted
/// candidate starts (block vertex-buffer offsets).
fn vertex_run_starts(bytes: &[u8], data_base: usize, stride: usize) -> Vec<usize> {
const NRM: usize = 12; // POSITION f32×3 occupies [0,12); NORMAL f16×4 follows
let mut starts = Vec::new();
if stride < NRM + 8 {
return starts;
}
let is_unit = |o: usize| -> bool {
if o + NRM + 6 > bytes.len() {
return false;
}
let nx = half(bytes, o + NRM);
let ny = half(bytes, o + NRM + 2);
let nz = half(bytes, o + NRM + 4);
let l = (nx * nx + ny * ny + nz * nz).sqrt();
(0.85..=1.15).contains(&l)
};
// Vertex buffers begin on 4-byte boundaries in practice; step 4.
let end = bytes.len().saturating_sub(NRM + 6);
let mut o = data_base;
while o <= end {
if is_unit(o) && (o < data_base + stride || !is_unit(o - stride)) {
starts.push(o);
}
o += 4;
}
starts
}
/// Locate a stage resource's `[index buffer][vertex buffer]` block among the
/// precomputed vertex-run `starts` (see [`vertex_run_starts`]) and decode it, or
/// return `None` if no candidate validates. A candidate `vb` is accepted when
/// the `index_count` indices ending just before it are all `< vtx_count`,
/// reference (nearly) all vertices, and produce non-degenerate triangles with a
/// real spatial extent — a signature strong enough to pin the block.
fn anchor_pool_mesh(
bytes: &[u8],
starts: &[usize],
index_count: usize,
vtx_count: usize,
decl: &VertexDecl,
) -> Option<GameMesh> {
let stride = decl.stride;
let idx_bytes = index_count * 2;
let vtx_bytes = vtx_count.checked_mul(stride)?;
for &vb in starts {
// The index buffer sits immediately before the vertex buffer.
if vb < idx_bytes {
continue;
}
let ib = vb - idx_bytes;
if vb + vtx_bytes > bytes.len() {
continue;
}
// ── Full index validation: every index in range, uses ~all vertices. ──
let mut max_idx = 0u32;
let mut ok = true;
for k in 0..index_count {
let i = be16(bytes, ib + k * 2) as u32;
if i >= vtx_count as u32 {
ok = false;
break;
}
max_idx = max_idx.max(i);
}
if !ok || (max_idx as usize) + 4 < vtx_count {
continue;
}
// ── Triangle quality: finite, non-degenerate, real spatial extent. ──
let pos = decl.pos_offset;
let mut lo = [f32::MAX; 3];
let mut hi = [f32::MIN; 3];
let mut degenerate = 0usize;
let mut sampled = 0usize;
let mut edge_sum = 0.0f32;
let tris = index_count / 3;
let tstep = (tris / 96).max(1);
let mut t = 0;
let mut bad = false;
while t < tris {
let mut p = [[0.0f32; 3]; 3];
for (c, pc) in p.iter_mut().enumerate() {
let vi = be16(bytes, ib + (3 * t + c) * 2) as usize;
let base = vb + vi * stride + pos;
for (a, slot) in pc.iter_mut().enumerate() {
let x = bef(bytes, base + a * 4);
if !x.is_finite() || x.abs() > 1.0e6 {
bad = true;
break;
}
*slot = x;
lo[a] = lo[a].min(x);
hi[a] = hi[a].max(x);
}
if bad {
break;
}
}
if bad {
break;
}
let u = [p[1][0] - p[0][0], p[1][1] - p[0][1], p[1][2] - p[0][2]];
let w = [p[2][0] - p[0][0], p[2][1] - p[0][1], p[2][2] - p[0][2]];
let cx = [
u[1] * w[2] - u[2] * w[1],
u[2] * w[0] - u[0] * w[2],
u[0] * w[1] - u[1] * w[0],
];
if 0.5 * (cx[0] * cx[0] + cx[1] * cx[1] + cx[2] * cx[2]).sqrt() < 1.0e-9 {
degenerate += 1;
}
// Sum the triangle's three edge lengths (for the connectivity check).
let e3 = [p[2][0] - p[1][0], p[2][1] - p[1][1], p[2][2] - p[1][2]];
edge_sum += (u[0] * u[0] + u[1] * u[1] + u[2] * u[2]).sqrt()
+ (w[0] * w[0] + w[1] * w[1] + w[2] * w[2]).sqrt()
+ (e3[0] * e3[0] + e3[1] * e3[1] + e3[2] * e3[2]).sqrt();
sampled += 1;
t += tstep;
}
if bad {
continue;
}
let extent = (hi[0] - lo[0]).max(hi[1] - lo[1]).max(hi[2] - lo[2]);
if extent < 0.5 || sampled == 0 || degenerate * 10 > sampled * 3 {
continue; // too flat, or >30% degenerate → not this block
}
// Connectivity check: a correctly-anchored mesh has triangle edges that
// are SMALL relative to its overall size (~0.050.15 of the bbox
// diagonal). A wrong anchor / cross-wired index buffer connects distant
// vertices, so its mean edge spans a large fraction of the model (a spiky
// mess). Reject those — try another candidate or decline.
let diag = ((hi[0] - lo[0]).powi(2) + (hi[1] - lo[1]).powi(2) + (hi[2] - lo[2]).powi(2))
.sqrt()
.max(1e-6);
let mean_edge = edge_sum / (sampled as f32 * 3.0);
if mean_edge / diag > 0.28 {
continue;
}
// ── Accepted: read the full mesh. ──
return Some(read_pool_mesh(bytes, ib, vb, index_count, vtx_count, decl));
}
None
}
/// Read positions / normals / uvs / indices for an anchored stage block.
fn read_pool_mesh(
bytes: &[u8],
ib: usize,
vb: usize,
index_count: usize,
vtx_count: usize,
decl: &VertexDecl,
) -> GameMesh {
let stride = decl.stride;
let indices: Vec<u32> = (0..index_count)
.map(|k| be16(bytes, ib + k * 2) as u32)
.collect();
let mut positions = Vec::with_capacity(vtx_count);
let mut normals = Vec::with_capacity(vtx_count);
let mut uvs = Vec::with_capacity(vtx_count);
for v in 0..vtx_count {
let o = vb + v * stride;
let p = o + decl.pos_offset;
positions.push([bef(bytes, p), bef(bytes, p + 4), bef(bytes, p + 8)]);
if let Some(no) = decl.normal_offset {
let nb = o + no;
normals.push([half(bytes, nb), half(bytes, nb + 2), half(bytes, nb + 4)]);
}
if let Some(uo) = decl.uv_offset {
let ub = o + uo;
uvs.push([half(bytes, ub), half(bytes, ub + 2)]);
}
}
GameMesh {
positions,
normals,
uvs,
indices,
name: None,
}
}
// ── Layout constants ────────────────────────────────────────────────────────
/// Size of the header that precedes each sub-mesh block's index buffer
/// (`[12-byte header][index buffer][vertex buffer]`). Contents not yet decoded.
const VERTEX_BUFFER_GAP: usize = 12;
// ── Vertex declaration ───────────────────────────────────────────────────────
/// The vertex layout for one XBG7 resource, parsed from the descriptor's
/// declaration table (shared by all its sub-meshes).
struct VertexDecl {
/// Bytes per vertex.
stride: usize,
/// Byte offset of the POSITION element (`f32×3`) within a vertex.
pos_offset: usize,
/// Byte offset of the NORMAL element (`f16×4`), if present.
normal_offset: Option<usize>,
/// Byte offset of the TEXCOORD element (`f16×2`), if present.
uv_offset: Option<usize>,
}
/// Known element format codes → element size in bytes (from the GPU capture:
/// POSITION `f32×3`, NORMAL `f16×4`, TEXCOORD `f16×2`).
fn decl_code_size(code: u32) -> Option<usize> {
match code {
0x2A_23B9 => Some(12), // f32×3 (POSITION)
0x1A_2360 => Some(8), // f16×4 (NORMAL)
0x2C_235F => Some(4), // f16×2 (TEXCOORD)
_ => None,
}
}
/// Parse the XBG7 vertex declaration: a table of `{offset:u32, code:u32,
/// usage<<16:u32}` big-endian triples that follows the `(index_bytes,
/// index_count)` marker, terminated by an `offset == 0x00FF0000` /
/// `code == 0xFFFFFFFF` sentinel. Usage codes: `0` POSITION, `3` NORMAL,
/// `5` TEXCOORD. Stride is the max element extent; unknown element sizes are
/// inferred from the next element's offset.
/// Locate the `(index_bytes, index_count)` marker in a descriptor: the first
/// big-endian pair where `index_bytes == index_count * 2` and `index_count` is a
/// positive multiple of 3. Returns `(rel_offset, index_count)`.
fn find_index_marker(desc: &[u8]) -> Option<(usize, usize)> {
let mut rel = 0usize;
while rel + 40 <= desc.len() {
let a = be32(desc, rel);
let c = be32(desc, rel + 4);
if c >= 3 && c % 3 == 0 && c < 400_000 && a == c * 2 {
return Some((rel, c as usize));
}
rel += 4;
}
None
}
fn parse_vertex_decl(desc: &[u8]) -> Option<VertexDecl> {
let mk = find_index_marker(desc)?.0;
// Read declaration triples.
let mut elems: Vec<(usize, u32, u32)> = Vec::new(); // (offset, code, usage)
let mut r = mk + 8;
for _ in 0..16 {
if r + 12 > desc.len() {
break;
}
let off = be32(desc, r);
let code = be32(desc, r + 4);
let usage = be32(desc, r + 8) >> 16;
if off == 0x00FF_0000 || code == 0xFFFF_FFFF {
break;
}
if off as usize > 0x1000 {
break; // out-of-range offset — not a real element
}
elems.push((off as usize, code & 0x00FF_FFFF, usage));
r += 12;
}
if elems.is_empty() {
return None;
}
let mut stride = 0usize;
for (i, &(off, code, _)) in elems.iter().enumerate() {
let size = decl_code_size(code).unwrap_or_else(|| {
if i + 1 < elems.len() {
elems[i + 1].0.saturating_sub(off)
} else {
4
}
});
stride = stride.max(off + size);
}
if stride == 0 || stride > 256 {
return None;
}
let pos_offset = elems
.iter()
.find(|&&(_, c, u)| c == 0x2A_23B9 || u == 0)
.map(|&(o, _, _)| o)
.unwrap_or(0);
let normal_offset = elems.iter().find(|&&(_, _, u)| u == 3).map(|&(o, _, _)| o);
let uv_offset = elems.iter().find(|&&(_, _, u)| u == 5).map(|&(o, _, _)| o);
Some(VertexDecl {
stride,
pos_offset,
normal_offset,
uv_offset,
})
}
// ── Descriptor sub-mesh record scan ─────────────────────────────────────────
/// Scan an XBG7 descriptor for the ordered list of per-sub-mesh
/// `(vtx_count, idx_count)` records.
///
/// The record is a big-endian tuple `[vtx:u32][0:u32][idx:u32][tail:u32]` with
/// `3 ≤ vtx ≤ 65535`, the second word zero, `idx` a positive multiple of 3, and
/// a small non-zero `tail`. Found by a sliding 4-byte scan (records are not on
/// a fixed stride in the scene graph).
fn submesh_records(desc: &[u8]) -> Vec<(usize, usize)> {
let mut out = Vec::new();
if desc.len() < 16 {
return out;
}
let mut rel = 0usize;
while rel + 16 <= desc.len() {
let a = be32(desc, rel);
let z = be32(desc, rel + 4);
let c = be32(desc, rel + 8);
let t = be32(desc, rel + 12);
if (3..=65535).contains(&a)
&& z == 0
&& c >= 3
&& c <= 200_000
&& c % 3 == 0
&& (1..=64).contains(&t)
{
out.push((a as usize, c as usize));
rel += 16; // consume the record
} else {
rel += 4;
}
}
out
}
// ── Little primitive readers ────────────────────────────────────────────────
#[inline]
fn align16(x: usize) -> usize {
(x + 15) & !15
}
#[inline]
fn be16(b: &[u8], o: usize) -> u16 {
u16::from_be_bytes([b[o], b[o + 1]])
}
#[inline]
fn be32(b: &[u8], o: usize) -> u32 {
u32::from_be_bytes([b[o], b[o + 1], b[o + 2], b[o + 3]])
}
#[inline]
fn bef(b: &[u8], o: usize) -> f32 {
f32::from_be_bytes([b[o], b[o + 1], b[o + 2], b[o + 3]])
}
/// Big-endian IEEE-754 half → f32.
#[inline]
fn half(b: &[u8], o: usize) -> f32 {
f16_to_f32(be16(b, o))
}
/// Minimal IEEE-754 binary16 → binary32 (no external dep).
fn f16_to_f32(h: u16) -> f32 {
let sign = (h >> 15) & 1;
let exp = (h >> 10) & 0x1F;
let mant = h & 0x3FF;
let bits: u32 = match exp {
0 if mant == 0 => (sign as u32) << 31, // ±0
0 => {
// subnormal → normalize
let mut e: i32 = -1;
let mut m = mant as u32;
loop {
e += 1;
m <<= 1;
if m & 0x400 != 0 {
break;
}
}
let exp32 = (127 - 15 - e) as u32;
((sign as u32) << 31) | (exp32 << 23) | ((m & 0x3FF) << 13)
}
0x1F => ((sign as u32) << 31) | (0xFF << 23) | ((mant as u32) << 13), // Inf/NaN
_ => {
let exp32 = (exp as i32 - 15 + 127) as u32;
((sign as u32) << 31) | (exp32 << 23) | ((mant as u32) << 13)
}
};
f32::from_bits(bits)
}
fn read_cstr(b: &[u8], o: usize) -> Option<String> {
if o >= b.len() {
return None;
}
let end = b[o..].iter().position(|&c| c == 0).map(|p| o + p)?;
if end == o {
return None;
}
Some(String::from_utf8_lossy(&b[o..end]).into_owned())
}
// ── Tests ───────────────────────────────────────────────────────────────────
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn half_roundtrip_known_values() {
assert_eq!(f16_to_f32(0x3C00), 1.0); // 1.0
assert_eq!(f16_to_f32(0x0000), 0.0); // +0
assert_eq!(f16_to_f32(0xBC00), -1.0); // -1.0
assert_eq!(f16_to_f32(0x4000), 2.0); // 2.0
assert!((f16_to_f32(0x3800) - 0.5).abs() < 1e-6); // 0.5
}
#[test]
fn submesh_record_scan_finds_tuple() {
// [vtx=215][0][idx=1092][tail=4]
let mut d = vec![0u8; 32];
d[0..4].copy_from_slice(&215u32.to_be_bytes());
d[8..12].copy_from_slice(&1092u32.to_be_bytes());
d[12..16].copy_from_slice(&4u32.to_be_bytes());
let recs = submesh_records(&d);
assert_eq!(recs, vec![(215, 1092)]);
}
#[test]
fn rejects_non_xpr2() {
assert!(matches!(
Xbg7Model::from_xpr2(b"NOPEnotacontainerXXXXXXXX"),
Err(MeshError::NotXpr2)
));
}
}

View File

@@ -262,6 +262,41 @@ fn be32(b: &[u8], o: usize) -> u32 {
u32::from_be_bytes([b[o], b[o + 1], b[o + 2], b[o + 3]])
}
/// Best-effort human label for a decompressed entry's inner format, from its
/// first bytes: a known signature (`IDXD`, fonts, `png`, …), a printable 4-char
/// tag (RATC, IXUD, LSTA, …), or a hex fallback for still-unidentified magics.
/// Shared by the CLI `pak list` and the GUI pack browser.
pub fn inner_format_label(payload: &[u8]) -> String {
if payload.len() < 4 {
return "empty".into();
}
let m = &payload[0..4];
// Known binary/text signatures → friendly short tags (so fonts/images don't
// show as bare hex like `00010000` / `89504E47`).
let known = match m {
b"IDXD" => Some("IDXD"),
b"\x89PNG" => Some("png"),
b"\x00\x01\x00\x00" | b"true" | b"typ1" => Some("ttf"), // sfnt TrueType
b"OTTO" => Some("otf"), // OpenType (CFF)
b"ttcf" => Some("ttc"), // TrueType Collection
b"RIFF" => Some("riff"),
b"DDS " => Some("dds"),
_ if payload.starts_with(b"<?xml") => Some("xml"),
_ => None,
};
if let Some(tag) = known {
return tag.into();
}
if m.iter().all(|&b| b.is_ascii_graphic()) {
// Many inner formats are 4-char tags (RATC, T8aD, IXUD, LSTA, …).
String::from_utf8_lossy(m).into_owned()
} else {
format!("{:02X}{:02X}{:02X}{:02X}", m[0], m[1], m[2], m[3])
}
}
#[cfg(test)]
mod tests {
use super::*;

View File

@@ -0,0 +1,131 @@
//! `RATC` — a nested resource bundle.
//!
//! After a small header, a RATC holds named children — `foo.t32` (T8aD
//! textures), `foo.rat` (nested RATC), fonts, PNG — each immediately preceded by
//! its ASCII name string. The children are self-locating by their 4-char magic,
//! so we list them by scanning for those magics and pairing each with the name
//! run that precedes it. This is reliable for *listing* (names / types / sizes
//! are literal bytes); decoding a child's pixels is delegated to that child's
//! own parser ([`crate::t8ad`]).
/// A child resource inside a RATC bundle.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct RatcChild {
/// Name string preceding the child (e.g. `prselectbtn_g04b.t32`), or empty.
pub name: String,
/// Child magic tag: `T8aD`, `RATC`, `ttcf`, `png`, …
pub kind: String,
/// Byte offset of the child (its magic) within the RATC payload.
pub offset: usize,
/// Byte length of the child, up to the next child (or end of payload).
pub size: usize,
}
/// Magic at the start of a RATC bundle.
pub const RATC_MAGIC: [u8; 4] = *b"RATC";
/// Whether `bytes` is a RATC bundle.
pub fn is_ratc(bytes: &[u8]) -> bool {
bytes.len() >= 4 && bytes[0..4] == RATC_MAGIC
}
/// Recognized child magics and their display tag.
fn child_kind(m: &[u8]) -> Option<&'static str> {
match m {
b"T8aD" => Some("T8aD"),
b"RATC" => Some("RATC"),
b"ttcf" => Some("ttc"),
b"\x89PNG" => Some("png"),
_ => None,
}
}
/// List the children of a RATC bundle (the self-magic at offset 0 is skipped).
pub fn parse(bytes: &[u8]) -> Option<Vec<RatcChild>> {
if !is_ratc(bytes) {
return None;
}
// Offsets of every recognized child magic (skip the self RATC at 0).
let mut offs: Vec<(usize, &'static str)> = Vec::new();
let mut i = 4;
while i + 4 <= bytes.len() {
if let Some(kind) = child_kind(&bytes[i..i + 4]) {
offs.push((i, kind));
i += 4;
} else {
i += 1;
}
}
let mut children = Vec::with_capacity(offs.len());
for (idx, &(off, kind)) in offs.iter().enumerate() {
let next = offs.get(idx + 1).map(|&(o, _)| o).unwrap_or(bytes.len());
children.push(RatcChild {
name: name_before(bytes, off),
kind: kind.to_string(),
offset: off,
size: next.saturating_sub(off),
});
}
Some(children)
}
/// The nearest name string preceding `off`: the *last* printable run (len ≥ 3)
/// in the 96 bytes before the child magic. A few record-header bytes usually sit
/// between the name and the magic, so an exact-adjacency scan isn't enough.
fn name_before(bytes: &[u8], off: usize) -> String {
let start = off.saturating_sub(96);
let window = &bytes[start..off];
let mut best = String::new();
let mut run_start: Option<usize> = None;
let flush = |from: usize, to: usize, best: &mut String| {
if to - from >= 3 {
*best = String::from_utf8_lossy(&window[from..to]).trim().to_string();
}
};
for (i, &c) in window.iter().enumerate() {
if (0x20..=0x7e).contains(&c) {
run_start.get_or_insert(i);
} else if let Some(s) = run_start.take() {
flush(s, i, &mut best);
}
}
if let Some(s) = run_start {
flush(s, window.len(), &mut best);
}
best
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn lists_named_children() {
let mut b = RATC_MAGIC.to_vec();
b.extend_from_slice(&[0u8; 28]); // header padding
b.extend_from_slice(b"logo.t32");
let t8_off = b.len();
b.extend_from_slice(b"T8aD");
b.extend_from_slice(&[0u8; 40]); // some child bytes
b.extend_from_slice(b"sub.rat");
let ratc_off = b.len();
b.extend_from_slice(b"RATC");
b.extend_from_slice(&[0u8; 8]);
let kids = parse(&b).unwrap();
assert_eq!(kids.len(), 2);
assert_eq!(kids[0].kind, "T8aD");
assert_eq!(kids[0].name, "logo.t32");
assert_eq!(kids[0].offset, t8_off);
assert_eq!(kids[0].size, ratc_off - t8_off);
assert_eq!(kids[1].kind, "RATC");
assert_eq!(kids[1].name, "sub.rat");
}
#[test]
fn rejects_non_ratc() {
assert!(parse(b"T8aD....").is_none());
}
}

View File

@@ -0,0 +1,134 @@
//! `T8aD` — the game's 2D UI/HUD texture format.
//!
//! A linear (untiled) 32bpp surface stored **A8R8G8B8** (Xbox byte order), with
//! a small fixed-per-variant header:
//!
//! ```text
//! 0x00 4 Magic "T8aD"
//! 0x14 4 width (BE u32)
//! 0x18 4 height (BE u32)
//! 0x1c 4 type field → header size: 1→64 2→84 3→104 4→124 15→344
//! <header> w*h*4 bytes of A8R8G8B8 pixel data, row-major
//! ```
//!
//! Static forensics over the disc showed ~85% of entries decode exactly with
//! this rule; the rest (unknown type field or a `w*h*4` that doesn't fit — most
//! likely DXT / palettized variants) return `None` rather than a wrong image.
//!
//! Keying the header size off the type field (not `len - w*h*4`) makes the
//! decoder **container-safe**: RATC/LSTA hand us a child slice that may have
//! trailing padding before the next child, so we must not infer the header from
//! the slice length.
//!
//! ⚠️ Colour correctness (channel order / endianness / sRGB) is unverified
//! against the running game — see the RE backlog.
/// Magic at the start of every T8aD surface.
pub const T8AD_MAGIC: [u8; 4] = *b"T8aD";
/// A decoded T8aD surface as tightly-packed RGBA8 (row-major, top-left origin).
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct T8adImage {
pub width: u32,
pub height: u32,
pub rgba: Vec<u8>,
}
/// Whether `bytes` starts with the T8aD magic.
pub fn is_t8ad(bytes: &[u8]) -> bool {
bytes.len() >= 4 && bytes[0..4] == T8AD_MAGIC
}
/// Header size in bytes for a given type field (`@0x1c`), or `None` for an
/// unrecognized variant.
fn header_size(type_field: u32) -> Option<usize> {
match type_field {
1 => Some(64),
2 => Some(84),
3 => Some(104),
4 => Some(124),
15 => Some(344),
_ => None,
}
}
#[inline]
fn be32(b: &[u8], off: usize) -> u32 {
u32::from_be_bytes([b[off], b[off + 1], b[off + 2], b[off + 3]])
}
/// Decode a T8aD surface from a slice whose first bytes ARE the magic. Returns
/// `None` for non-T8aD input or a variant we can't decode as RGBA (never guesses).
pub fn parse(bytes: &[u8]) -> Option<T8adImage> {
if !is_t8ad(bytes) || bytes.len() < 0x40 {
return None;
}
let width = be32(bytes, 0x14);
let height = be32(bytes, 0x18);
if !(1..=4096).contains(&width) || !(1..=4096).contains(&height) {
return None;
}
let header = header_size(be32(bytes, 0x1c))?;
let n = (width as usize) * (height as usize) * 4;
if bytes.len() < header + n {
return None; // DXT/palettized/short variant — defer, don't misdecode
}
// A8R8G8B8 → RGBA8.
let src = &bytes[header..header + n];
let mut rgba = vec![0u8; n];
for (px, out) in src.chunks_exact(4).zip(rgba.chunks_exact_mut(4)) {
let (a, r, g, b) = (px[0], px[1], px[2], px[3]);
out[0] = r;
out[1] = g;
out[2] = b;
out[3] = a;
}
Some(T8adImage {
width,
height,
rgba,
})
}
#[cfg(test)]
mod tests {
use super::*;
/// Build a synthetic type-1 (header 64) T8aD with a known A8R8G8B8 pattern.
fn synth(w: u32, h: u32) -> Vec<u8> {
let mut b = vec![0u8; 64];
b[0..4].copy_from_slice(&T8AD_MAGIC);
b[0x14..0x18].copy_from_slice(&w.to_be_bytes());
b[0x18..0x1c].copy_from_slice(&h.to_be_bytes());
b[0x1c..0x20].copy_from_slice(&1u32.to_be_bytes()); // type 1 → header 64
for i in 0..(w * h) {
b.extend_from_slice(&[(i & 0xff) as u8, 0x24, 0x63, 0xB2]); // A, R, G, B
}
b
}
#[test]
fn decodes_argb_to_rgba() {
let b = synth(4, 2);
let img = parse(&b).expect("decodes");
assert_eq!((img.width, img.height), (4, 2));
assert_eq!(img.rgba.len(), 4 * 2 * 4);
// pixel 0: A=0,R=0x24,G=0x63,B=0xB2 → RGBA = 24 63 B2 00
assert_eq!(&img.rgba[0..4], &[0x24, 0x63, 0xB2, 0x00]);
// pixel 1: A=1 → alpha byte
assert_eq!(&img.rgba[4..8], &[0x24, 0x63, 0xB2, 0x01]);
}
#[test]
fn rejects_unknown_variant_and_short() {
// type field 7 (unknown) → None
let mut b = synth(2, 2);
b[0x1c..0x20].copy_from_slice(&7u32.to_be_bytes());
assert!(parse(&b).is_none());
// truncated pixel data → None
let b = synth(64, 64);
assert!(parse(&b[..200]).is_none());
assert!(parse(b"IDXD\0\0\0\0").is_none());
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -108,14 +108,23 @@ impl GameAssets {
/// Use this to identify unknown file types during reverse engineering.
pub fn identify_format(bytes: &[u8]) -> FileFormat {
if bytes.len() < 4 {
return FileFormat::Unknown;
// Very short files can still be text (e.g. a one-line config).
return if is_probably_text(bytes) {
FileFormat::Text
} else {
FileFormat::Unknown
};
}
match &bytes[..4] {
b"XPR2" => FileFormat::Xpr2Texture,
b"RIFF" => FileFormat::Riff, // could be WAV or XWB
b"XWB\0" => FileFormat::XwbAudio, // Xbox Wave Bank
b"RIFF" => FileFormat::Riff, // could be WAV or XWB
b"XWB\0" => FileFormat::XwbAudio, // Xbox Wave Bank
[0x89, b'P', b'N', b'G'] => FileFormat::Png,
b"DDS " => FileFormat::Dds,
// No binary magic matched — fall back to a content heuristic so text
// configs (`config.ini`, etc.) are recognised even though they have no
// signature.
_ if is_probably_text(bytes) => FileFormat::Text,
_ => FileFormat::Unknown,
}
}
@@ -127,6 +136,8 @@ pub enum FileFormat {
XwbAudio,
Png,
Dds,
/// Plain text (ASCII / UTF-8 / UTF-16) — `.ini` and similar.
Text,
Unknown,
}
@@ -138,7 +149,162 @@ impl FileFormat {
Self::XwbAudio => "xwb",
Self::Png => "png",
Self::Dds => "dds",
Self::Text => "txt",
Self::Unknown => "bin",
}
}
}
// ── Text detection & decoding (pure std, WASM-safe) ───────────────────────────
/// True if `b` is a byte we'd expect inside a text file: common whitespace,
/// printable ASCII, or any high byte (UTF-8 continuation / Latin-1). Mirrors the
/// printable convention used by the IDXD string-pool extractor.
#[inline]
fn is_text_byte(b: u8) -> bool {
matches!(b, b'\t' | b'\n' | b'\r') || (0x20..0x7f).contains(&b) || b >= 0x80
}
/// Heuristic sniff for whether `bytes` is human-readable text.
///
/// Recognises UTF-8 / UTF-16 BOMs outright, BOM-less UTF-16LE by its
/// characteristic odd-position NUL bytes, and otherwise requires a byte sample
/// to be almost entirely text bytes with no embedded NUL (NUL is the strongest
/// binary signal). Decoding is left to [`decode_text`].
pub fn is_probably_text(bytes: &[u8]) -> bool {
if bytes.is_empty() {
return false;
}
// Byte-order marks are unambiguous.
if bytes.starts_with(&[0xEF, 0xBB, 0xBF]) // UTF-8
|| bytes.starts_with(&[0xFF, 0xFE]) // UTF-16LE
|| bytes.starts_with(&[0xFE, 0xFF])
// UTF-16BE
{
return true;
}
// Cap the scan so huge files stay cheap.
let sample = &bytes[..bytes.len().min(4096)];
// BOM-less UTF-16LE: ASCII text encodes as `<char> 0x00 <char> 0x00 …`, so
// roughly half the bytes (the odd positions) are NUL and the even bytes are
// printable.
if sample.len() >= 16 {
let nul_odd = sample.iter().skip(1).step_by(2).filter(|&&b| b == 0).count();
if nul_odd * 2 >= sample.len() / 2 {
let printable_even = sample.iter().step_by(2).filter(|&&b| is_text_byte(b)).count();
if printable_even * 2 >= sample.len() / 2 - 2 {
return true;
}
}
}
// Single-byte (ASCII / UTF-8): an embedded NUL means binary; otherwise
// require the overwhelming majority to be text bytes.
if sample.contains(&0) {
return false;
}
let text = sample.iter().filter(|&&b| is_text_byte(b)).count();
text * 100 >= sample.len() * 95
}
/// Decode `bytes` to a `String`, returning `(text, encoding_label)`.
///
/// Honours UTF-8 / UTF-16LE / UTF-16BE BOMs and BOM-less UTF-16LE, falling back
/// to lossy UTF-8. Never fails — undecodable sequences become U+FFFD. Pure std,
/// so both the viewer and CLI can share it.
pub fn decode_text(bytes: &[u8]) -> (String, &'static str) {
if let Some(rest) = bytes.strip_prefix(&[0xEF, 0xBB, 0xBF]) {
return (String::from_utf8_lossy(rest).into_owned(), "UTF-8 (BOM)");
}
if let Some(rest) = bytes.strip_prefix(&[0xFF, 0xFE]) {
return (decode_utf16(rest, false), "UTF-16LE (BOM)");
}
if let Some(rest) = bytes.strip_prefix(&[0xFE, 0xFF]) {
return (decode_utf16(rest, true), "UTF-16BE (BOM)");
}
// BOM-less UTF-16LE detection (same signal as `is_probably_text`).
let sample = &bytes[..bytes.len().min(4096)];
if sample.len() >= 16 {
let nul_odd = sample.iter().skip(1).step_by(2).filter(|&&b| b == 0).count();
if nul_odd * 2 >= sample.len() / 2 {
return (decode_utf16(bytes, false), "UTF-16LE");
}
}
(String::from_utf8_lossy(bytes).into_owned(), "UTF-8 / ASCII")
}
fn decode_utf16(bytes: &[u8], big_endian: bool) -> String {
let units: Vec<u16> = bytes
.chunks_exact(2)
.map(|c| {
if big_endian {
u16::from_be_bytes([c[0], c[1]])
} else {
u16::from_le_bytes([c[0], c[1]])
}
})
.collect();
String::from_utf16_lossy(&units)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn ascii_config_is_text() {
let ini = b"[Video]\r\nWidth=1280\r\nHeight=720\r\n";
assert!(is_probably_text(ini));
assert_eq!(identify_format(ini), FileFormat::Text);
let (text, enc) = decode_text(ini);
assert!(text.contains("Width=1280"));
assert_eq!(enc, "UTF-8 / ASCII");
}
#[test]
fn magic_takes_precedence_over_text() {
// An XPR2 header happens to start with printable bytes; the magic match
// must win so it's never misfiled as text.
let mut xpr = b"XPR2".to_vec();
xpr.extend_from_slice(&[0u8; 32]);
assert_eq!(identify_format(&xpr), FileFormat::Xpr2Texture);
}
#[test]
fn binary_with_nul_is_not_text() {
let bin = [0u8, 1, 2, 0xFF, 0x00, 0x89, 0x10, 0x00, 0x42];
assert!(!is_probably_text(&bin));
assert_eq!(identify_format(&bin), FileFormat::Unknown);
}
#[test]
fn utf16le_bom_is_text_and_decodes() {
let data = [0xFF, 0xFE, b'H', 0x00, b'i', 0x00];
assert!(is_probably_text(&data));
assert_eq!(identify_format(&data), FileFormat::Text);
let (text, enc) = decode_text(&data);
assert_eq!(text, "Hi");
assert_eq!(enc, "UTF-16LE (BOM)");
}
#[test]
fn bomless_utf16le_detected() {
// "Hello, world!!!!" as UTF-16LE, no BOM (>=16 bytes so the heuristic runs).
let mut data = Vec::new();
for &b in b"Hello, world!!!!" {
data.push(b);
data.push(0);
}
assert!(is_probably_text(&data));
let (text, enc) = decode_text(&data);
assert_eq!(text, "Hello, world!!!!");
assert_eq!(enc, "UTF-16LE");
}
#[test]
fn empty_is_not_text() {
assert!(!is_probably_text(&[]));
}
}

View File

@@ -14,25 +14,48 @@ use std::path::Path;
use anyhow::{Context, Result};
use tracing::{debug, info};
use xdvdfs::blockdev::OffsetWrapper;
use xdvdfs::layout::VolumeDescriptor;
/// A handle to an open XISO image.
pub struct XisoReader<F: Read + Seek> {
///
/// The inner file is wrapped in an [`OffsetWrapper`] which probes the four
/// known XGD partition offsets (raw XISO, XGD1, XGD2, XGD3) at open time,
/// so both single-layer raw dumps **and** full dual-layer disc images are
/// supported transparently.
pub struct XisoReader<F: Read + Seek + Send + Sync> {
volume: VolumeDescriptor,
file: F,
/// Offset-aware block device — all sector reads are shifted by the
/// detected partition start offset automatically.
file: OffsetWrapper<F, std::io::Error>,
}
impl<F: Read + Seek + Send + Sync + 'static> XisoReader<F> {
pub async fn open(mut file: F) -> Result<Self> {
let volume = xdvdfs::read::read_volume(&mut file)
pub async fn open(file: F) -> Result<Self> {
// OffsetWrapper::new probes four known partition offsets:
// 0x00000000 — raw XISO (trimmed, sector 0 = XDVDFS start)
// 0x183E0000 — XGD1 (original Xbox)
// 0x0FD90000 — XGD2 (Xbox 360, most retail titles)
// 0x02080000 — XGD3 (Xbox 360, later dual-layer titles)
let mut wrapper = OffsetWrapper::new(file).await.map_err(|e| {
anyhow::anyhow!(
"No valid XDVDFS partition found in this disc image. \
Tried raw XISO, XGD1, XGD2, and XGD3 offsets. \
Is this a valid Xbox 360 (or original Xbox) disc image? \
(internal error: {e:?})"
)
})?;
// Re-read the volume descriptor via the wrapper (now at the correct offset).
let volume = xdvdfs::read::read_volume(&mut wrapper)
.await
.context("Failed to read XDVDFS volume descriptor. Is this a valid Xbox 360 ISO?")?;
.context("Found XDVDFS partition but failed to parse volume descriptor")?;
info!(
"Opened XISO: root directory table at sector {}",
{ let s = volume.root_table.region.sector; s }
);
Ok(Self { volume, file })
Ok(Self { volume, file: wrapper })
}
/// List all files in the disc image, recursively (directories excluded).
@@ -135,10 +158,13 @@ impl<F: Read + Seek + Send + Sync + 'static> XisoReader<F> {
}
}
/// Open an XISO from a file path (the common case).
/// Open a disc image from a file path.
///
/// Accepts raw XISO dumps and full XGD1/XGD2/XGD3 disc images — the correct
/// partition offset is detected automatically.
pub async fn open_iso(path: &Path) -> Result<XisoReader<std::fs::File>> {
let file = std::fs::File::open(path)
.with_context(|| format!("Cannot open ISO: {}", path.display()))?;
.with_context(|| format!("Cannot open disc image: {}", path.display()))?;
XisoReader::open(file).await
}

View File

@@ -0,0 +1,234 @@
//! Integration test: decode XBG7 geometry from REAL `.xpr` model files.
//!
//! Uses loose files extracted from the retail disc (models live in
//! `hidden/resource3d/*.xpr`). Skipped unless the directory is found; point
//! `SYLPHEED_RES3D` at it to override.
//!
//! Run: `cargo test -p sylpheed-formats --test mesh_disc -- --ignored --nocapture`
use std::path::PathBuf;
use sylpheed_formats::mesh::Xbg7Model;
fn res3d_dir() -> Option<PathBuf> {
if let Ok(p) = std::env::var("SYLPHEED_RES3D") {
let p = PathBuf::from(p);
if p.is_dir() {
return Some(p);
}
}
let default =
PathBuf::from("/home/fabi/RE Project Sylpheed/sylph_extract/hidden/resource3d");
default.is_dir().then_some(default)
}
#[test]
#[ignore = "requires extracted disc models — set SYLPHEED_RES3D"]
fn weapon_model_decodes_to_expected_geometry() {
let Some(dir) = res3d_dir() else {
eprintln!("SKIP: resource3d dir not found (set SYLPHEED_RES3D)");
return;
};
// rou_f001_wep_00 = the player ship's first weapon: 1 sub-mesh,
// 215 vertices, 364 triangles (verified by hex analysis).
let bytes = std::fs::read(dir.join("rou_f001_wep_00.xpr")).unwrap();
let model = Xbg7Model::from_xpr2(&bytes).expect("weapon must decode");
assert_eq!(model.meshes.len(), 1);
let (v, t) = model.totals();
assert_eq!(v, 215, "vertex count");
assert_eq!(t, 364, "triangle count");
let m = &model.meshes[0];
assert_eq!(m.positions.len(), 215);
assert_eq!(m.uvs.len(), 215);
assert_eq!(m.normals.len(), 215);
assert_eq!(m.indices.len(), 1092);
// every index in range
assert!(m.indices.iter().all(|&i| (i as usize) < m.positions.len()));
// positions are real geometry within the model's ~2-unit bbox
let ys: Vec<f32> = m.positions.iter().map(|p| p[1]).collect();
let span = ys.iter().cloned().fold(f32::MIN, f32::max)
- ys.iter().cloned().fold(f32::MAX, f32::min);
assert!(span > 1.0 && span < 10.0, "y-span {span} out of range");
// Correct vertex alignment ⇒ normals are unit-length (the pin for the
// +12 vertex offset) and UVs land in a sane texture range.
let mean_nlen: f32 = m
.normals
.iter()
.map(|n| (n[0] * n[0] + n[1] * n[1] + n[2] * n[2]).sqrt())
.sum::<f32>()
/ m.normals.len() as f32;
assert!((mean_nlen - 1.0).abs() < 0.05, "mean |normal| {mean_nlen} ≠ 1");
assert!(
m.uvs.iter().all(|uv| uv[0] > -0.1 && uv[0] < 2.0 && uv[1] > -0.1 && uv[1] < 2.0),
"UVs out of expected [0,1]-ish range"
);
}
#[test]
#[ignore = "requires extracted disc models — set SYLPHEED_RES3D"]
fn declaration_driven_decode_covers_expected_model_count() {
let Some(dir) = res3d_dir() else {
return;
};
let mut ok = 0usize;
let mut total = 0usize;
for entry in std::fs::read_dir(&dir).unwrap() {
let path = entry.unwrap().path();
if path.extension().and_then(|e| e.to_str()) != Some("xpr") {
continue;
}
total += 1;
let bytes = std::fs::read(&path).unwrap();
if let Ok(model) = Xbg7Model::from_xpr2(&bytes) {
if !model.meshes.is_empty() {
// Every decoded model must be self-consistent (indices in range,
// and unit normals where present).
for m in &model.meshes {
assert!(
m.indices.iter().all(|&i| (i as usize) < m.positions.len()),
"{path:?}: index out of range"
);
}
ok += 1;
}
}
}
eprintln!("XBG7 declaration-driven decode: {ok}/{total} models");
// Variable-stride declaration parsing lifted coverage vs the old fixed
// stride-24 decoder (25 → 36 fully-validated models; multi-sub-mesh models
// whose later sub-mesh offset isn't yet handled are still declined whole).
assert!(ok >= 35, "coverage regressed: only {ok}/{total} decoded");
}
#[test]
#[ignore = "requires extracted disc models — set SYLPHEED_RES3D"]
fn complex_body_mesh_is_declined_not_garbage() {
let Some(dir) = res3d_dir() else {
return;
};
// The hero-ship body uses the multi-stream layout we do not decode; it must
// be cleanly rejected, never returned as partial geometry.
let bytes = std::fs::read(dir.join("DeltaSaber_A.xpr")).unwrap();
match Xbg7Model::from_xpr2(&bytes) {
Err(_) => {} // expected: declined
Ok(m) => {
// If it ever does decode, it must at least be self-consistent.
for mesh in &m.meshes {
assert!(mesh.indices.iter().all(|&i| (i as usize) < mesh.positions.len()));
}
}
}
}
/// Stage containers decode multiple enemy/prop sub-models via content anchoring.
/// Prints coverage; asserts the known-good Stage_S10 meshes decode with sane geometry.
#[test]
#[ignore]
fn stage_models_decode() {
use sylpheed_formats::mesh::Xbg7Model;
let dir = std::env::var("SYLPHEED_RES3D").unwrap_or_else(|_| {
"/home/fabi/RE Project Sylpheed/sylph_extract/hidden/resource3d".to_string()
});
let path = format!("{dir}/Stage_S10.xpr");
let bytes = std::fs::read(&path).expect("read Stage_S10");
let models = Xbg7Model::stage_models(&bytes);
for m in &models {
let (v, t) = m.totals();
let mut lo = [f32::MAX; 3];
let mut hi = [f32::MIN; 3];
for sub in &m.meshes {
for p in &sub.positions {
for a in 0..3 {
lo[a] = lo[a].min(p[a]);
hi[a] = hi[a].max(p[a]);
}
}
}
let ext = [hi[0] - lo[0], hi[1] - lo[1], hi[2] - lo[2]];
println!(
" {:16} v={v} t={t} bbox=[{:.1},{:.1},{:.1}]",
m.name, ext[0], ext[1], ext[2]
);
}
// Known: e003 (main enemy, ~1400 tris) must be among the decoded models.
let e003 = models.iter().find(|m| m.name == "e003").expect("e003 decoded");
let (v, t) = e003.totals();
assert_eq!(v, 2383, "e003 vertex count");
assert!(t > 1400, "e003 triangle count {t}");
assert!(models.len() >= 3, "at least 3 stage sub-models, got {}", models.len());
}
/// Timing/coverage sweep across all stage containers (manual; release recommended).
#[test]
#[ignore]
fn stage_models_sweep() {
use sylpheed_formats::mesh::Xbg7Model;
use std::time::Instant;
let dir = std::env::var("SYLPHEED_RES3D").unwrap_or_else(|_| {
"/home/fabi/RE Project Sylpheed/sylph_extract/hidden/resource3d".to_string()
});
let mut names: Vec<_> = std::fs::read_dir(&dir)
.unwrap()
.filter_map(|e| e.ok().map(|e| e.file_name().into_string().unwrap()))
.filter(|n| n.starts_with("Stage_") && n.ends_with(".xpr"))
.collect();
names.sort();
let mut tot = 0usize;
for n in &names {
let bytes = std::fs::read(format!("{dir}/{n}")).unwrap();
let t0 = Instant::now();
let models = Xbg7Model::stage_models(&bytes);
let dt = t0.elapsed().as_millis();
tot += models.len();
println!("{n:16} {:>4} MB {:>3} models {:>5} ms", bytes.len()/1_000_000, models.len(), dt);
}
println!("TOTAL stage sub-models decoded: {tot}");
}
/// Quality audit: for a big stage, verify decoded models are real geometry
/// (bounded bbox, low full-mesh degeneracy) rather than false anchors.
#[test]
#[ignore]
fn stage_models_quality_audit() {
use sylpheed_formats::mesh::Xbg7Model;
let dir = std::env::var("SYLPHEED_RES3D").unwrap_or_else(|_| {
"/home/fabi/RE Project Sylpheed/sylph_extract/hidden/resource3d".to_string()
});
let bytes = std::fs::read(format!("{dir}/Stage_S07.xpr")).unwrap();
let models = Xbg7Model::stage_models(&bytes);
let (mut small, mut mid, mut huge, mut dupnames) = (0, 0, 0, 0);
let mut seen = std::collections::HashSet::new();
let mut worst_deg = 0.0f32;
for m in &models {
if !seen.insert(m.name.clone()) { dupnames += 1; }
let mut lo = [f32::MAX; 3]; let mut hi = [f32::MIN; 3];
let mut deg = 0usize; let mut tot = 0usize;
for sub in &m.meshes {
for p in &sub.positions { for a in 0..3 { lo[a]=lo[a].min(p[a]); hi[a]=hi[a].max(p[a]); } }
for tri in sub.indices.chunks_exact(3) {
let (a,b,c)=(tri[0] as usize,tri[1] as usize,tri[2] as usize);
if a>=sub.positions.len()||b>=sub.positions.len()||c>=sub.positions.len(){continue;}
let pa=sub.positions[a]; let pb=sub.positions[b]; let pc=sub.positions[c];
let u=[pb[0]-pa[0],pb[1]-pa[1],pb[2]-pa[2]];
let v=[pc[0]-pa[0],pc[1]-pa[1],pc[2]-pa[2]];
let cx=[u[1]*v[2]-u[2]*v[1],u[2]*v[0]-u[0]*v[2],u[0]*v[1]-u[1]*v[0]];
if 0.5*(cx[0]*cx[0]+cx[1]*cx[1]+cx[2]*cx[2]).sqrt()<1e-9 { deg+=1; }
tot+=1;
}
}
let ext = (hi[0]-lo[0]).max(hi[1]-lo[1]).max(hi[2]-lo[2]);
let df = if tot>0 { deg as f32/tot as f32 } else {1.0};
worst_deg = worst_deg.max(df);
if ext < 200.0 { small += 1; } else if ext < 5000.0 { mid += 1; } else { huge += 1; }
}
println!("S07: {} models | small(<200u)={} mid={} huge(>5k)={} | dup-names={} | worst full-mesh degeneracy={:.1}%",
models.len(), small, mid, huge, dupnames, worst_deg*100.0);
// Each XBG7 resource must anchor to a *distinct* block (no collisions), the
// bulk must be bounded-scale geometry, and none may be mostly-degenerate.
assert_eq!(dupnames, 0, "no two resources should anchor to the same block");
assert!(small + mid > models.len() * 9 / 10, "≥90% bounded-scale geometry");
assert!(huge < models.len() / 20, "few huge (skybox-plane) models");
assert!(worst_deg < 0.35, "no model should be mostly-degenerate");
}

View File

@@ -6,6 +6,7 @@
use std::path::{Path, PathBuf};
use sylpheed_formats::texture::X360Texture;
use sylpheed_formats::{IdxdObject, PakArchive};
/// Locate the extracted disc root, or `None` to skip.
@@ -122,3 +123,139 @@ fn name_lookup_resolves_weapon_tbl() {
let obj = IdxdObject::parse(&bytes).expect("weapon.tbl parses as IDXD");
assert!(obj.count > 0);
}
/// Real T8aD entries decode to sane RGBA surfaces, and a good fraction of the
/// pack's T8aD entries are decodable (the RGBA-variant rule holds broadly).
#[test]
fn hangar_t8ad_textures_decode() {
skip_without_disc!(root);
let arc = PakArchive::open(root.join("dat/GP_HANGAR_ARSENAL.pak")).unwrap();
let (mut seen, mut decoded) = (0usize, 0usize);
for e in arc.entries() {
if e.comp_size > 4_000_000 {
continue;
}
let Ok(p) = arc.read(e) else { continue };
if !sylpheed_formats::t8ad::is_t8ad(&p) {
continue;
}
seen += 1;
if let Some(img) = sylpheed_formats::t8ad::parse(&p) {
decoded += 1;
assert!((1..=4096).contains(&img.width) && (1..=4096).contains(&img.height));
assert_eq!(img.rgba.len(), (img.width * img.height * 4) as usize);
}
}
assert!(seen > 100, "expected many T8aD, saw {seen}");
// The RGBA rule should cover the large majority (≈85% disc-wide).
assert!(decoded * 100 >= seen * 70, "decoded {decoded}/{seen}");
}
/// A real LSTA sprite list yields inline T8aD frames.
#[test]
fn lsta_sprite_list_decodes() {
skip_without_disc!(root);
let arc = PakArchive::open(root.join("dat/GP_HANGAR_ARSENAL.pak")).unwrap();
let mut found = false;
for e in arc.entries() {
let Ok(p) = arc.read(e) else { continue };
if !sylpheed_formats::lsta::is_lsta(&p) {
continue;
}
if let Some(frames) = sylpheed_formats::lsta::parse(&p) {
if !frames.is_empty() {
found = true;
assert!(frames.iter().all(|f| !f.rgba.is_empty()));
}
}
}
assert!(found, "no decodable LSTA found");
}
/// A real RATC bundle lists named children including a decodable T8aD.
#[test]
fn ratc_bundle_lists_children() {
skip_without_disc!(root);
let arc = PakArchive::open(root.join("dat/GP_HANGAR_ARSENAL.pak")).unwrap();
let mut ok = false;
for e in arc.entries() {
if e.comp_size > 4_000_000 {
continue;
}
let Ok(p) = arc.read(e) else { continue };
if !sylpheed_formats::ratc::is_ratc(&p) {
continue;
}
let Some(kids) = sylpheed_formats::ratc::parse(&p) else { continue };
let has_named = kids.iter().any(|c| c.name.contains('.'));
let has_t8ad = kids
.iter()
.any(|c| c.kind == "T8aD" && sylpheed_formats::t8ad::parse(&p[c.offset..c.offset + c.size]).is_some());
if !kids.is_empty() && has_named && has_t8ad {
ok = true;
break;
}
}
assert!(ok, "no RATC with named children + a decodable T8aD");
}
/// The English movie subtitle track (an `IXUD` entry) decodes to timed cues.
#[test]
fn eng_movie_subtitle_track() {
skip_without_disc!(root);
let arc = PakArchive::open(root.join("dat/movie/eng.pak")).unwrap();
// S04A's track — the longest inline English one (starts "Calm down!").
let bytes = arc.read_by_hash(0x73d2_2a3b).expect("entry present").unwrap();
let sub = sylpheed_formats::ixud::parse(&bytes).expect("IXUD parses as subtitle");
assert!(sub.cues.len() > 50, "cues={}", sub.cues.len());
assert!(!sub.is_reference_only());
assert_eq!(sub.cues[0].text, "Calm down!");
assert!((sub.cues[0].start - 9.4).abs() < 0.01);
assert_eq!(sub.cues[0].end, Some(11.0));
}
/// The subtitle tracks are genuinely localized: the same entry hash differs
/// between the English and German packs.
#[test]
fn subtitles_are_localized() {
skip_without_disc!(root);
let eng = PakArchive::open(root.join("dat/movie/eng.pak")).unwrap();
let deu = PakArchive::open(root.join("dat/movie/deu.pak")).unwrap();
let e = eng.read_by_hash(0x73d2_2a3b).unwrap().unwrap();
let d = deu.read_by_hash(0x73d2_2a3b).unwrap().unwrap();
assert_ne!(e, d, "eng/deu subtitle payloads should differ");
}
/// The English movie pack's embedded subtitle font parses to sane metadata.
#[test]
fn eng_movie_font_metadata() {
skip_without_disc!(root);
let arc = PakArchive::open(root.join("dat/movie/eng.pak")).unwrap();
let bytes = arc.read_by_hash(0x5cd0_fca6).expect("entry present").unwrap();
assert!(sylpheed_formats::font::is_font(&bytes));
let info = sylpheed_formats::font::parse_info(&bytes).expect("font parses");
assert!(info.glyphs > 0, "glyphs={}", info.glyphs);
assert!(!info.family.is_empty(), "family should be readable");
}
/// The Acheron backdrop is a TXCM world cubemap: 6 faces, and the cube path's
/// face 0 matches the (validated) 2D `from_xpr2` face-0 decode.
#[test]
fn acheron_world_cubemap_six_faces() {
skip_without_disc!(root);
let bytes = std::fs::read(root.join("hidden/resource3d/BG_Acheron.xpr")).unwrap();
let cube = X360Texture::cube_faces_from_xpr2(&bytes)
.expect("cube decode ok")
.expect("BG_Acheron is a TXCM cubemap");
assert_eq!((cube.width, cube.height), (1024, 1024));
assert_eq!(cube.faces.len(), 6);
// A8R8G8B8 → 4 bytes/texel, de-tiled to width×height.
for face in &cube.faces {
assert_eq!(face.len(), 1024 * 1024 * 4);
}
// Face 0 must equal what the existing 2D path decodes (the green planet).
let face0_2d = X360Texture::from_xpr2(&bytes).unwrap();
assert!(face0_2d.is_cubemap);
assert_eq!(cube.faces[0], face0_2d.data);
}

View File

@@ -0,0 +1,140 @@
//! Integration test: run the XPR2 texture pipeline against REAL `.xpr` files
//! read directly from the retail disc image, reproducing exactly what the
//! viewer's texture-preview path does (`identify_format` → `X360Texture::
//! from_xpr2`). Skipped unless `SYLPHEED_ISO` points at the disc (or the
//! default dev path exists).
//!
//! Run: `cargo test -p sylpheed-formats --test texture_disc -- --ignored --nocapture`
use std::path::PathBuf;
use sylpheed_formats::texture::X360Texture;
use sylpheed_formats::vfs::identify_format;
fn iso_path() -> Option<PathBuf> {
if let Ok(p) = std::env::var("SYLPHEED_ISO") {
let p = PathBuf::from(p);
if p.is_file() {
return Some(p);
}
}
let default = PathBuf::from(
"/home/fabi/RE Project Sylpheed/Project Sylpheed - Arc of Deception (USA, Europe) (En,Ja).iso",
);
default.is_file().then_some(default)
}
#[tokio::test]
#[ignore = "requires the retail ISO — set SYLPHEED_ISO"]
async fn xpr_pipeline_over_disc_sample() {
let Some(iso) = iso_path() else {
eprintln!("SKIP: ISO not found (set SYLPHEED_ISO)");
return;
};
let mut reader = sylpheed_formats::xiso::open_iso(&iso).await.unwrap();
let all = reader.list_all_files().await.unwrap();
let mut xprs: Vec<String> = all
.into_iter()
.filter(|f| f.to_lowercase().ends_with(".xpr"))
.collect();
xprs.sort();
println!("found {} .xpr files", xprs.len());
// Sample across the set so we hit different texture sizes/formats.
let sample: Vec<String> = xprs.iter().step_by((xprs.len() / 24).max(1)).cloned().collect();
let mut ok = 0usize;
let mut fail = 0usize;
let mut by_format: std::collections::BTreeMap<String, usize> = Default::default();
let mut nonxpr = 0usize;
// Optionally dump the sampled .xpr bytes so the CLI can export them to PNG
// for visual de-tiling validation: `DUMP_XPR_DIR=/tmp/xpr cargo test …`.
let dump_dir = std::env::var("DUMP_XPR_DIR").ok();
if let Some(d) = &dump_dir {
std::fs::create_dir_all(d).unwrap();
}
for path in &sample {
let bytes = reader.read_file(path).await.unwrap();
if let Some(d) = &dump_dir {
let base = path.rsplit('/').next().unwrap();
std::fs::write(format!("{d}/{base}"), &bytes).unwrap();
}
let fmt = identify_format(&bytes);
let magic: String = bytes
.iter()
.take(4)
.map(|b| if b.is_ascii_graphic() { *b as char } else { '.' })
.collect();
if fmt != sylpheed_formats::vfs::FileFormat::Xpr2Texture {
nonxpr += 1;
println!(" [{path}] NOT XPR2 (magic {magic:?}, {} bytes)", bytes.len());
continue;
}
match X360Texture::from_xpr2(&bytes) {
Ok(t) => {
ok += 1;
*by_format.entry(format!("{:?}", t.format)).or_default() += 1;
// Sanity: does the decoded data length match the descriptor
// dimensions (what the GPU upload will require)?
let bs = t.format.block_size() as u32;
let bw = ((t.width + bs - 1) / bs).max(1) as usize;
let bh = ((t.height + bs - 1) / bs).max(1) as usize;
let need = bw * bh * t.format.bytes_per_block();
let size_ok = if need == t.data.len() { "ok" } else { "MISMATCH" };
println!(
" [{path}] {:?} {}x{} mips={} data={} need={} {size_ok}",
t.format, t.width, t.height, t.mip_levels, t.data.len(), need
);
}
Err(e) => {
fail += 1;
println!(" [{path}] from_xpr2 FAILED: {e}");
dump_xpr2_structure(&bytes);
}
}
}
println!("\nSUMMARY: ok={ok} fail={fail} non-xpr2={nonxpr} formats={by_format:?}");
}
/// Dump the XPR2 header + resource directory the way `from_xpr2` reads it,
/// so we can see why a file's TX2D scan comes up empty.
fn dump_xpr2_structure(bytes: &[u8]) {
let be = |o: usize| u32::from_be_bytes(bytes[o..o + 4].try_into().unwrap());
let magic: String = bytes[..4].iter().map(|b| *b as char).collect();
println!(
" hdr magic={magic:?} header_size=0x{:X} data_size=0x{:X} num_resources={}",
be(0x04),
be(0x08),
be(0x0C)
);
let n = be(0x0C).min(16); // guard against garbage counts
for i in 0..n {
let base = 0x10 + i as usize * 16;
if base + 16 > bytes.len() {
break;
}
let tag: String = bytes[base..base + 4]
.iter()
.map(|b| if b.is_ascii_graphic() { *b as char } else { '.' })
.collect();
println!(
" res[{i}] tag={tag:?} data_off=0x{:X} desc_size=0x{:X} name_off=0x{:X}",
be(base + 4),
be(base + 8),
be(base + 12)
);
}
// First 48 bytes hex for orientation.
let hx: String = bytes
.iter()
.take(48)
.map(|b| format!("{b:02X} "))
.collect();
println!(" hex[0..48] {hx}");
}

View File

@@ -31,11 +31,14 @@ bevy = { workspace = true, features = [
"bevy_winit",
"multi_threaded",
"png",
# Required for TonyMcMapFace tonemapping (default Camera3d tonemapper)
"tonemapping_luts",
# Linux display backends
"x11",
"wayland",
] }
bevy_egui = { workspace = true }
futures = { workspace = true }
thiserror = { workspace = true }
tracing = { workspace = true }
@@ -46,3 +49,10 @@ dev = ["bevy/dynamic_linking"]
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
tracing-subscriber = { workspace = true }
rfd = { workspace = true }
# Audio for the WMV video player (pulls cpal → alsa on Linux).
rodio = { workspace = true }
# PNG decode for pak image entries (pure Rust; Bevy already pulls it in).
image = { version = "0.25", default-features = false, features = ["png"] }
# Off-thread rasterization of embedded-font samples (avoids egui's global fonts).
ab_glyph = "0.2"

View File

@@ -12,10 +12,11 @@
//! 4. Bevy uploads it to the GPU automatically
use bevy::asset::io::Reader;
use bevy::asset::{AssetLoader, AsyncReadExt, LoadContext};
use bevy::asset::{AssetLoader, LoadContext};
use bevy::image::ImageSampler;
use bevy::prelude::*;
use bevy::render::render_asset::RenderAssetUsages;
use bevy::render::render_resource::{Extent3d, TextureDimension, TextureFormat};
use bevy::render::render_resource::{Extent3d, TextureDescriptor, TextureDimension, TextureFormat, TextureUsages};
use sylpheed_formats::texture::{X360Texture, X360TextureFormat};
// ── Plugin ────────────────────────────────────────────────────────────────────
@@ -86,24 +87,52 @@ impl AssetLoader for Xpr2TextureLoader {
/// Convert a decoded `X360Texture` into a Bevy-compatible `Image`.
///
/// Maps Xbox 360 D3DFORMAT codes to `wgpu::TextureFormat` values.
/// Bevy uses wgpu internally, so this mapping is direct.
/// Constructs the `Image` struct directly rather than using `Image::new()`,
/// because `Image::new()` calls `pixel_size()` which panics for BCn
/// block-compressed formats.
pub fn x360_texture_to_bevy_image(tex: X360Texture) -> Result<Image, Xpr2LoadError> {
let bevy_format = x360_format_to_wgpu(&tex.format)?;
let format = x360_format_to_wgpu(&tex.format)?;
Ok(Image::new(
Extent3d {
width: tex.width,
height: tex.height,
depth_or_array_layers: 1,
// Uncompressed k_8_8_8_8: after `from_xpr2`'s k8in32 endian swap the bytes
// are in [A,R,G,B] order (verified against the retail Acheron backdrop).
// wgpu has no ARGB format, so reorder to [R,G,B,A] and upload as Rgba8
// (see `x360_format_to_wgpu`). BCn data is already GPU-ready.
let data = match tex.format {
X360TextureFormat::A8R8G8B8 | X360TextureFormat::X8R8G8B8 => {
let opaque = matches!(tex.format, X360TextureFormat::X8R8G8B8);
let mut out = tex.data;
for px in out.chunks_exact_mut(4) {
let (a, r, g, b) = (px[0], px[1], px[2], px[3]);
px[0] = r;
px[1] = g;
px[2] = b;
px[3] = if opaque { 0xFF } else { a };
}
out
}
_ => tex.data,
};
Ok(Image {
data,
texture_descriptor: TextureDescriptor {
label: None,
size: Extent3d {
width: tex.width,
height: tex.height,
depth_or_array_layers: 1,
},
mip_level_count: 1,
sample_count: 1,
dimension: TextureDimension::D2,
format,
usage: TextureUsages::TEXTURE_BINDING | TextureUsages::COPY_DST,
view_formats: &[],
},
TextureDimension::D2,
tex.data,
bevy_format,
// Make the texture available on both CPU and GPU
// (RenderOnly would be more efficient once RE is complete)
RenderAssetUsages::MAIN_WORLD | RenderAssetUsages::RENDER_WORLD,
))
sampler: ImageSampler::Default,
texture_view_descriptor: None,
asset_usage: RenderAssetUsages::MAIN_WORLD | RenderAssetUsages::RENDER_WORLD,
})
}
/// Map an `X360TextureFormat` to the corresponding `wgpu::TextureFormat`.
@@ -119,15 +148,13 @@ fn x360_format_to_wgpu(format: &X360TextureFormat) -> Result<TextureFormat, Xpr2
X360TextureFormat::Dxt3 => TextureFormat::Bc2RgbaUnormSrgb,
// DXT5 / BC3
X360TextureFormat::Dxt5 => TextureFormat::Bc3RgbaUnormSrgb,
// DXN / BC5 — normal maps (RG, not sRGB)
// DXN / BC5 — two-channel normal maps (RG, not sRGB)
X360TextureFormat::Dxn => TextureFormat::Bc5RgUnorm,
// Uncompressed ARGB — note Xbox 360 is BGRA order (big-endian)
// We may need to swizzle R and B channels
// DXT5A / BC4 — single-channel (gloss, specular, luminance maps)
X360TextureFormat::Dxt5A => TextureFormat::Bc4RUnorm,
// Uncompressed — reordered to [R,G,B,A] by `x360_texture_to_bevy_image`.
X360TextureFormat::A8R8G8B8 | X360TextureFormat::X8R8G8B8 => {
// Xbox 360 stores as ARGB big-endian (BGRA in memory)
// wgpu uses Rgba8UnormSrgb — swizzle may be needed
// TODO: verify channel order against actual game textures
TextureFormat::Bgra8UnormSrgb
TextureFormat::Rgba8UnormSrgb
}
})
}

View File

@@ -32,6 +32,10 @@ pub struct OrbitCamera {
pub orbit_sensitivity: f32,
pub zoom_sensitivity: f32,
pub pan_sensitivity: f32,
/// Zoom limits — set when framing a model so large scenes can be pulled back
/// far enough (the old fixed 0.5..50 cap trapped the camera inside big stages).
pub min_radius: f32,
pub max_radius: f32,
}
impl Default for OrbitCamera {
@@ -44,6 +48,8 @@ impl Default for OrbitCamera {
orbit_sensitivity: 0.005,
zoom_sensitivity: 0.3,
pan_sensitivity: 0.003,
min_radius: 0.05,
max_radius: 500.0,
}
}
}
@@ -54,19 +60,26 @@ fn spawn_camera(mut commands: Commands) {
commands.spawn((
Camera3d::default(),
// Wide near/far so both tiny weapons and multi-thousand-unit stages fit;
// the planes are re-scaled to the zoom distance each frame (below).
Projection::Perspective(PerspectiveProjection {
near: 0.05,
far: 100_000.0,
..default()
}),
transform,
orbit,
));
}
fn orbit_camera(
mut query: Query<(&mut OrbitCamera, &mut Transform)>,
mut query: Query<(&mut OrbitCamera, &mut Transform, &mut Projection)>,
mouse_buttons: Res<ButtonInput<MouseButton>>,
keys: Res<ButtonInput<KeyCode>>,
mut mouse_motion: EventReader<MouseMotion>,
mut scroll: EventReader<MouseWheel>,
) {
let Ok((mut cam, mut transform)) = query.get_single_mut() else { return };
let Ok((mut cam, mut transform, mut projection)) = query.get_single_mut() else { return };
let mut delta_motion = Vec2::ZERO;
for ev in mouse_motion.read() {
@@ -99,13 +112,20 @@ fn orbit_camera(
// Zoom (scroll)
cam.radius -= scroll_delta * cam.zoom_sensitivity * cam.radius;
cam.radius = cam.radius.clamp(0.5, 50.0);
cam.radius = cam.radius.clamp(cam.min_radius, cam.max_radius);
// Reset (R key)
if keys.just_pressed(KeyCode::KeyR) {
*cam = OrbitCamera::default();
}
// Keep the clip planes proportional to the zoom distance so depth precision
// stays usable across scales (tiny prop → whole stage grid).
if let Projection::Perspective(p) = projection.as_mut() {
p.near = (cam.radius * 0.02).clamp(0.02, 50.0);
p.far = (cam.radius * 50.0).max(2000.0);
}
*transform = orbit_transform(&cam);
}

File diff suppressed because it is too large Load Diff

View File

@@ -20,6 +20,7 @@ use bevy_egui::EguiPlugin;
pub mod asset_loader;
pub mod camera;
pub mod iso_loader;
pub mod ui;
// ── Application state ────────────────────────────────────────────────────────
@@ -27,33 +28,16 @@ pub mod ui;
/// Global viewer state, shared across all UI and rendering systems.
#[derive(Resource)]
pub struct ViewerState {
/// Which type of asset is currently being browsed / previewed.
pub current_mode: ViewMode,
/// Whether to render meshes in wireframe mode (toggle with F2).
pub wireframe: bool,
/// Whether the plain-text viewer wraps long lines.
pub text_wrap: bool,
}
impl Default for ViewerState {
fn default() -> Self {
Self {
current_mode: ViewMode::Textures,
wireframe: false,
}
Self { text_wrap: true }
}
}
/// The active view / asset type being inspected.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum ViewMode {
/// XPR2 texture preview (Milestone 1).
#[default]
Textures,
/// 3D mesh viewer (Milestone 2+).
Mesh,
/// Audio waveform / playback (Milestone 2+).
Audio,
}
// ── App builder ──────────────────────────────────────────────────────────────
/// Build and run the viewer application.
@@ -75,6 +59,7 @@ pub fn run() {
app.add_plugins(EguiPlugin);
app.add_plugins(asset_loader::SylpheedAssetPlugin);
app.add_plugins(iso_loader::IsoLoaderPlugin);
app.add_plugins(camera::OrbitCameraPlugin);
app.add_plugins(ui::ViewerUiPlugin);

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,94 @@
# Handoff — XBG7 stage-container meshes, unified layout, mesh renderer (2026-07-12)
## Summary
This session extended the XBG7 mesh work from single weapons/props to the big
**stage containers**, fixed a long-standing weapon decode artifact, added a
headless mesh renderer for self-verification, and reworked the viewer's stage
presentation. Companion Xenia-Canary GPU instrumentation (the `log_draws`
draw-logger used as the ground-truth oracle) is on the `instrument-current`
branch of the Canary fork.
## What landed (Reborn — `feature/ipfb-idxd-parser`)
### 1. Stage containers decode — `Xbg7Model::stage_models`
`hidden/resource3d/Stage_S*.xpr` are **collections of up to ~450 enemy/prop
sub-models**, not single meshes. Each resource is a block
`[12-byte header][index buffer][vertex buffer]`; the index count is the
descriptor's `(index_bytes, index_count)` marker and the vertex count is the
`u32` 32 bytes before it. The blocks are **scattered among the container's
texture data with no stored offset**, so each is located by **content**: one
`O(file)` pass per stride finds vertex-buffer starts (unit NORMAL at vertex +12
whose previous stride slot isn't — a block boundary), then each resource is
pinned to the candidate where its indices validate and produce non-degenerate,
well-connected triangles. Fast (≤ ~4 s on a 70 MB stage), unambiguous.
- **Coverage: ~4993 sub-models across the 22 stages** (content-anchored).
- A **connectivity gate** (mean triangle edge ≤ 0.28× bbox diagonal) rejects
mis-anchored blocks that would render as spiky messes.
### 2. Weapon layout fix (unified with stages)
Weapons use the **same** `[12-byte header][index][vertex]` block layout. The old
decoder read the index buffer 12 bytes too early → 2 leading degenerate
triangles (the recurring "stray white triangle" artifact) + a lost tail of 6
real indices. Now the header is skipped; vertex offsets are unchanged, so
coverage stays 36/166 and the triangle lists are correct. Verified on
wep_00/03/04 via the renderer.
### 3. Headless mesh renderer — `sylpheed-cli mesh {info,render}`
A software rasterizer (orthographic, z-buffered, two-sided shading) that writes a
PNG. Lets anyone (including the assistant) verify recovered geometry without the
GUI. `--only <substr>` filters sub-models; `--yaw/--pitch/--size/--row` control
the view. Stage containers render as a normalised thumbnail grid.
### 4. Viewer UX
- Stage files show a **thumbnail grid**: each sub-model recentred + uniformly
scaled to a fixed cell (a 1-unit prop and a 3000-unit structure are equally
visible), skybox-plane / stray-anchor models (>20000 u) culled.
- Stage sub-models are **textured** by matching name → `_col` albedo.
- **Camera fixed**: zoom was hard-capped at 50 u (camera trapped inside big
stages); now zoom limits + clip planes scale to the framed model.
- Dispatch decodes both single-model and stage paths and keeps whichever yields
more geometry (fixes stages previously showing the dummy-root "black cube").
## State / how to verify
```bash
# formats + disc tests (needs the extracted disc)
SYLPHEED_RES3D=/path/to/hidden/resource3d \
cargo test -p sylpheed-formats --test mesh_disc -- --ignored --nocapture
# render any model / stage to a PNG
cargo run --release -p sylpheed-cli -- mesh render \
.../resource3d/Stage_S10.xpr /tmp/s10.png --size 1000
cargo run --release -p sylpheed-cli -- mesh render \
.../resource3d/rou_f001_wep_00.xpr /tmp/wep.png
# viewer
cargo run -p sylpheed-viewer # File → Open Directory → the extracted disc
```
## Known limitations / next steps
- **Multi-submesh main bodies with separate vertex pools** still mis-decode
(e.g. `Stage_S10` `e005`: a coherent fighter overlaid with wrong spike
triangles). `e003` works only because its two submeshes *share* one pool. Same
unsolved class as the **DeltaSaber hero body** (`f004`, still declined). Fix
needs per-submesh vertex-base RE. The LODs of these bodies (e.g. `e005_l`)
decode cleanly.
- **The per-resource data offset is not stored** — blocks are found by content.
Reversing the container's allocation order would give exact offsets + 100%
coverage but is a larger effort.
- **Texture colour correctness** remains the known "unverified" dynamic-RE item
(channel order / sRGB) for both 2D textures and model albedos.
- The viewer's stage decode runs on the main thread (~10 s on a 70 MB stage in a
debug build); move off-thread if the load hitch matters.
## Canary oracle (`instrument-current` on the Xenia-Canary fork)
`command_processor.cc::LogDrawForRE` (cvar `log_draws`) dumps each distinct
draw's primitive type, index buffer, vertex declaration, and sample vertex
positions to `xenia_re_draws.log`. Used to confirm the XBG7 layout (triangle
list, f32×3 position + f16×4 normal + f16×2 uv). Build with the memory-safe
flags (`CC=clang-19 CXX=clang++-19 CMAKE_BUILD_PARALLEL_LEVEL=3 … -j 3`); run one
emulator process at a time.

30
docs/re/INDEX.md Normal file
View File

@@ -0,0 +1,30 @@
# RE knowledge index
Confidence: ✅ `CONFIRMED` · 🟡 `PROBABLE` · ❔ `HYPOTHESIS`. See [README](README.md).
Formats we've already reversed are, for now, **documented by their parser + disc round-trip
tests** (the executable spec) rather than a prose file — the "Spec" column points there.
Promote to a prose `structures/…md` file when a format needs behavioural notes beyond layout.
## Data structures / formats
| Format | Conf. | Spec (parser + tests) | Notes |
|--------|-------|-----------------------|-------|
| IPFB `.pak` archive | ✅ | `sylpheed-formats/src/pak.rs` + `tests/pak_idxd_disc.rs` | header + 12-byte TOC, Z1/zlib payloads |
| name-hash (TOC keys) | ✅ | `sylpheed-formats/src/hash.rs` | Barrett-reduction hash; recovers original paths |
| IDXD object/table | ✅ | `sylpheed-formats/src/idxd.rs` | self-describing; ship/weapon stats verified vs known values |
| XPR2 texture + cubemap | 🟡 | `sylpheed-formats/src/texture.rs` | de-tile + A8R8G8B8; **colours unverified** (dynamic item) |
| T8aD 2D texture | 🟡 | `sylpheed-formats/src/t8ad.rs` | ~85% decode; **colours ✅ CONFIRMED** ([k8888](structures/texture-color-k8888.md)); ~15% variants deferred |
| RATC bundle | 🟡 | `sylpheed-formats/src/ratc.rs` | child listing confirmed; one level deep |
| LSTA sprite list | 🟡 | `sylpheed-formats/src/lsta.rs` | inline T8aD frames |
| IXUD subtitle | 🟡 | `sylpheed-formats/src/ixud.rs` | timed cues; **movie↔track link unknown** (dynamic item) |
| Fonts (ttf/otf/ttc) | ✅ | `sylpheed-formats/src/font.rs` | standard OpenType, parsed via ttf-parser |
| XBG7 mesh | 🟡/❔ | `sylpheed-formats/src/mesh.rs` + `tests/mesh_disc.rs` ([xbg7](structures/xbg7-mesh.md)) | weapons/props: declaration-driven variable stride (36 models), GPU-confirmed. **Stage containers: 5662 sub-models across 22 stages** via content-anchored grouped pools (`stage_models`). Quantized hero bodies (DeltaSaber `f004`) still declined |
## Functions / code paths
_None documented yet — populated during the dynamic-RE phase._
| Function | Conf. | Reimpl. | Summary |
|----------|-------|---------|---------|
| — | — | — | — |

88
docs/re/README.md Normal file
View File

@@ -0,0 +1,88 @@
# Reverse-engineering knowledge base
This directory is the **spec-side** of the clean-room: it records what the original
*Project Sylpheed* binary **does** (behaviour) and how its **data is laid out**, so that
the Rust port can be implemented **from these specs** without re-deriving anything and
without ever copying original code.
It exists to answer one question fast: *"do we already know how X works, and how sure are we?"*
---
## The one rule that matters
> **Never document a claim more confidently than the evidence supports, and never
> paste original code here.**
A wrong-but-confident note is worse than no note: someone builds on it and the bug hides
for weeks. Every entry therefore carries an explicit **confidence** and its **evidence**.
This mirrors the project method — *measure the oracle, never infer; refute before believing.*
### Clean-room firewall
- ✅ Allowed: behaviour descriptions, field offsets/types, formulas, state machines,
observed input→output pairs, and **references** to the original by address
(`sub_821B68C0`) or to `xenia-rs/sylpheed.db`.
- ❌ Forbidden here and in `crates/`: pasted decompiled C/C++ or verbatim disassembled
function bodies presented as the thing to reimplement. Cite the address; describe the
behaviour in your own words. Disassembly is a tool for *understanding*, not a source to copy.
---
## Confidence levels
| Level | Meaning | Bar to reach it |
|-------|---------|-----------------|
| `CONFIRMED` | Behaviour verified against ground truth. | ≥2 independent observations **or** one observation cross-checked against an oracle (canary framebuffer, a known-correct value, a second code path). |
| `PROBABLE` | Strong single-source inference. | One clean observation, or an unambiguous static read of the disassembly. |
| `HYPOTHESIS` | Educated guess, not yet tested. | Anything else. Must say what would confirm/refute it. |
**Promotion requires new evidence, not re-reading the old evidence.** A `HYPOTHESIS` that
"looks right again" is still a `HYPOTHESIS`. Only an *independent* check promotes it.
If evidence later contradicts an entry, **demote it and record the contradiction** — do
not silently edit the conclusion.
---
## When to document
- **Right after** a function/structure crosses from `HYPOTHESIS` to at least `PROBABLE`
before moving to the next code path, so the knowledge isn't lost or re-derived.
- **Whenever confidence changes** (up or down) — append to the Evidence log, don't overwrite.
- **Not** while it's still a pure guess with no evidence — a one-liner in the relevant
backlog/plan is enough until there's something to stand on.
## What to document
- **Functions/code paths** → `docs/re/functions/<name>.md` (one file per function or tight cluster).
- **Data structures / formats** → `docs/re/structures/<name>.md`.
- Keep the index in [`INDEX.md`](INDEX.md) (one line each: name · confidence · one-line summary).
Use the templates: [`_TEMPLATE.function.md`](_TEMPLATE.function.md),
[`_TEMPLATE.structure.md`](_TEMPLATE.structure.md).
---
## How we find and confirm code paths (the toolchain)
Everything joins on the **guest virtual address (PC)** — code addresses are fixed by the
XEX load, identical across our emulator and canary.
- **Static (cheap, try first):** `xenia-rs/sylpheed.db` (DuckDB: 25 481 functions, xrefs,
strings, vtables, imports). Query with `xenia-rs/zq.py``zq.py grep <str>`,
`zq.py xref <addr>`, `zq.py dis <lo> <hi>`, `zq.py fn <pc>`. Entry points are usually a
**string** (`zq.py grep MSG_DEMO`) or an **import** (movie/XMA API) xref'd back to the loader.
- **Dynamic (when static is ambiguous):** run `xenia-rs` with its probe suite —
`--pc-probe` / `--audit-pc-probe-hex` (fires at block entry), `--mem-watch` (mid-block
reads/writes of a VA), `--lr-trace` (call/return chains), `--trace-instructions`,
`--dump-addr` (read guest memory). These already exist; prefer them over hacking canary.
- **Oracle (correctness ground truth):** canary — the **Wine cross-build**
`xenia-canary/build-cross/bin/Windows/Debug/xenia_canary.exe` (the native Linux ELF
**crashes / does not run** — do not use it). This is the only emulator that reaches the
in-game menu; our `xenia-rs` never got past the intro video. Use canary to *observe output*
(capture its framebuffer for texture colours), not usually to instrument code — though its
`build-cross` toolchain does compile, so small C++ probes + rebuild are possible when needed.
Run **muted, one emulator process at a time**, point it at the real ISO (not the symlink).
> ⚠️ **VA-equality caveat:** join **code** by PC (fixed), but **never** assume a data VA
> holds the same bytes across emulators — allocators differ. Compare data by content/layout.

View File

@@ -0,0 +1,23 @@
# `<function name / sub_XXXXXXXX>`
- **Address:** `0x________` (in `sylpheed.db`; `zq.py fn 0x________`)
- **Confidence:** `HYPOTHESIS | PROBABLE | CONFIRMED`
- **Reimplemented in:** `crates/…/…rs::fn` — or *not yet*
- **Related:** other RE entries, `[[memory-slug]]`
## What it does
One-paragraph behavioural summary, in your own words. No pasted disassembly.
## Signature / calling convention
Inputs (registers/args + meaning), outputs, side effects (memory it writes, globals, imports it calls).
## Behaviour
Step-by-step of the observable behaviour — the *spec* to implement from. Formulas, branches,
state transitions. Cite addresses for detail (`the loop at 0x…`), don't transcribe code.
## Evidence log (append-only; newest last)
- `YYYY-MM-DD`*how we know*: static disasm read / `--lr-trace` from 0x… / canary output /
N observations. What it established. → confidence set to `…`.
## Open questions / what would raise confidence
The specific check that would confirm or refute the current claim.

View File

@@ -0,0 +1,29 @@
# `<structure / format name>`
- **Confidence:** `HYPOTHESIS | PROBABLE | CONFIRMED`
- **Parser in:** `crates/sylpheed-formats/src/…rs` — or *not yet*
- **Seen in:** which pak / asset / memory region
- **Related:** other RE entries, `[[memory-slug]]`
## Layout
```text
offset size type name notes
0x00 4 magic "____"
0x__ 4 u32be …
```
Endianness, alignment, variable-length rules, how child/section sizes are derived.
## Field semantics
What each field *means* and its observed value range. Distinguish **confirmed** fields from
**guessed** ones explicitly (a guessed field with a plausible value is still a guess).
## Coverage / limits
What fraction of real instances this decodes; which variants are deferred and why.
## Evidence log (append-only; newest last)
- `YYYY-MM-DD` — how established (forensic scan over N disc entries / round-trip test /
oracle comparison). → confidence `…`.
## Open questions / what would raise confidence
e.g. "colours unverified until compared against canary's framebuffer."

View File

@@ -0,0 +1,50 @@
# Texture colour interpretation — `k_8_8_8_8` (32bpp UI/HUD textures)
- **Confidence:** mixed — see per-claim tags below.
- **Parser in:** `sylpheed-formats/src/t8ad.rs`, `src/texture.rs` (XPR2)
- **Applies to:** T8aD 2D UI textures, XPR2 32bpp surfaces — the "colours look a bit odd" concern.
- **Related:** [[reborn-dynamic-re-plan]], INDEX rows T8aD / XPR2.
## What this resolves
Whether our decoded 32bpp textures use the right **sRGB-vs-linear** interpretation and the right
**channel order**. Ground truth = Canary's Vulkan texture cache, which is the code that actually
produces correct on-screen colour.
## Findings
### ✅ CONFIRMED — plain `k_8_8_8_8` is LINEAR (UNORM), never sRGB
Canary's host-format table maps plain `k_8_8_8_8``VK_FORMAT_R8G8B8A8_UNORM` with a straight
32-bit copy load shader (`kLoadShaderIndex32bpb`) and an **identity** format swizzle
(`XE_GPU_TEXTURE_SWIZZLE_RGBA`). There is **no** `..._SRGB` host format anywhere in the table;
gamma is a separate explicit path (`k_8_8_8_8_GAMMA_EDRAM` only). Therefore a plain 32bpp texture
is sampled as **raw UNORM bytes — apply no sRGB/gamma decode.**
*Implication:* if the viewer/engine applies an sRGB→linear (or linear→sRGB) step to these, that is
wrong. Show the bytes as-is (raw RGBA8), gamma only where the game explicitly requests it.
### ✅ CONFIRMED — on-disk order is **ARGB** (alpha first); `[a,r,g,b]→[r,g,b,a]` is correct
Settled by two independent lines that agree, without an emulator run:
1. **Measured on the disc data** — across 789 real T8aD textures (GP_HANGAR_ARSENAL, GP_MAIN_GAME_E,
DefTables), the byte position that is most-often exactly `0xFF` (the opaque-alpha signature of UI
art) is **byte 0 in 767/789 (97%)** — tf1 385/0, tf2 382/12. So texel byte 0 = alpha ⇒ on-disk
layout is **A,R,G,B**. (An earlier bimodality test tied byte0/byte3 only because colour channels
are also `0x00`-heavy from black backgrounds; the `==0xFF` discriminator isolates alpha cleanly.)
2. **Cross-check vs our Canary-validated renderer**`xenia-rs`'s `decode_k8888_tiled` (the path
behind the user-confirmed M1 splash / M2 video colours) nets memory-`ARGB` → endian-swap →
`swap(0,2)` = `[A,R,G,B]→[R,G,B,A]`, identical to ours. Since that renderer was confirmed against
Canary's output, Canary's ground truth is transitively in this chain.
Net: our channel order and the linear/UNORM interpretation are both **correct** for the ~85% RGBA
variants. A live Canary framebuffer capture would be redundant confirmation, not a new signal.
## Remaining / adjacent
- **XPR2** shares the ARGB→RGBA reorder + the linear/UNORM finding, but its **tiling** (de-tile) is a
separate path; if XPR2 skyboxes still look odd it's tiling or viewer display-gamma, not channel order.
- **Viewer display gamma:** decode is correct; if egui shows these too dark/bright it's how egui
interprets the RGBA8 (sRGB-encoded vs linear) at display time — a rendering nit, not a decode bug.
## Evidence log
- `2026-07-11` — static read of Canary `vulkan_texture_cache.cc` host-format table; no `R8G8B8A8_SRGB`
entry exists → sRGB/UNORM claim `CONFIRMED`.
- `2026-07-11` — measured `==0xFF` alpha dominance over 789 disc T8aD textures (byte0 = alpha 97%) +
cross-checked vs `xenia-rs decode_k8888_tiled` (the Canary-validated M1/M2 path). → channel-order
claim promoted `HYPOTHESIS → CONFIRMED`.

View File

@@ -0,0 +1,183 @@
# XBG7 — mesh geometry (inside XPR2 model containers)
- **Confidence:** 🟡 `PROBABLE` for the single-stream layout (below); ❔ `HYPOTHESIS` / undecoded
for the complex multi-stream body layout.
- **Parser in:** `sylpheed-formats/src/mesh.rs` (`Xbg7Model::from_xpr2`), tests
`tests/mesh_disc.rs`. Container parsing reused from `src/texture.rs` (`Xpr2Header` /
`Xpr2ResourceEntry`).
- **Applies to:** ship / weapon / prop models in `hidden/resource3d/*.xpr` (166 files).
- **Method:** clean-room — static hex inspection of the retail disc + geometric validation of the
recovered triangles (non-degenerate area, indices in range, bbox matches the descriptor's stored
size). No game code decompiled or copied.
## Where XBG7 lives
Models are ordinary **`XPR2`** containers (see the XPR2 texture doc / `texture.rs`). The 16-byte
resource-directory entries (from file offset `0x10`) carry `TX2D` texture resources **and** one or
more `XBG7` geometry resources:
```
entry = [ tag:4 ][ data_offset:u32 ][ descriptor_size:u32 ][ name_offset:u32 ] (big-endian)
```
Offsets are relative to the directory base `0x10`. The `XBG7` *descriptor* (at `data_offset+0x10`,
`descriptor_size` bytes) is a **scene / material / node graph** — it holds node names
(`rou_f001_mnt1_root`, `Light`, …), a bounding value (`0x41F00000` = 30.0 ≈ the ship's ~30-unit
length), material names matching the `TX2D` channels (`_col` albedo, `_spc` specular, `_gls` gloss,
`_lum` luminance), and per-sub-mesh records. The **vertex / index buffers** live in the container's
shared **data section** (from `header_size`).
## Sub-mesh records (in the descriptor)
Read in file order by a sliding 4-byte scan; each is a big-endian tuple:
```
[ vtx_count:u32 ][ 0:u32 ][ idx_count:u32 ][ tail:u32 ]
3..=65535 ==0 mult. of 3 1..=64
```
(For `rou_f001_wep_00`: `vtx_count=215`, `idx_count=1092` — matches the recovered geometry exactly.)
## The single-stream data layout — 🟡 PROBABLE (decoded, GPU-cross-checked)
For 36 of the 166 models (weapons, simple props) the data section is a straight sequence of
sub-meshes, carved from `header_size` in record order:
```
per sub-mesh block:
[ 12-byte header (contents undecoded) ]
[ index buffer : idx_count × u16 BE ] triangle list (prim=4, GPU-confirmed)
[ vertex buffer : vtx_count × stride bytes ] ← declaration-driven
(pad to 16 bytes → next sub-mesh)
```
The 12-byte header precedes the **index** buffer (the same block shape as stage
resources — see below); the vertex buffer follows the indices with no further
gap. (Earlier this was mis-modelled as `[index][12-byte gap][vertex]`, which put
the vertex buffer at the identical offset but read the index buffer 12 bytes too
early — turning the 12 header bytes into 6 junk indices = **2 leading degenerate
triangles** (a stray-triangle artifact) and dropping the last 6 real indices.
Skipping the header fixes the triangle list with no change to vertex coverage.)
**Vertex declaration.** The layout is **not fixed-stride**. The descriptor holds a declaration table
(right after the `(index_bytes, index_count)` marker) of `{offset:u32, format-code:u32, usage<<16:u32}`
big-endian triples, terminated by `offset == 0x00FF0000` / `code == 0xFFFFFFFF`:
| usage | element | format code | format | size |
|-------|----------|-------------|--------|------|
| 0x00 | POSITION | `0x2A23B9` | f32×3 | 12 B |
| 0x03 | NORMAL | `0x1A2360` | f16×4 (use xyz) | 8 B |
| 0x05 | TEXCOORD | `0x2C235F` | f16×2 (u,v) | 4 B |
Stride = max element extent. Models **omit elements** → variable stride (20 = pos+normal, no UV;
24 = pos+normal+uv). Each element is read in **naive big-endian component order** (the raw file
bytes; see the endianness note below). Assuming a fixed stride-24 was why the old decoder mis-aligned
and declined the pos+normal-only models.
**Alignment pinned by the normals.** The vertex buffer starts at `index_end + 12` (a fixed 12-byte
header — NOT `align16`, which lands 4 bytes early on most models). The correct offset is the unique
one where recovered normals are exactly unit-length.
**Safety gate:** every index is validated `< vtx_count`, and when the declaration has a normal
element the mean recovered `|normal|` must be ≈1 (`[0.5, 2.0]`). A model failing either is rejected
(`MeshError::UnsupportedLayout`) rather than emitting garbage.
### Endianness — file bytes are naive-BE, `k8in32` is a red herring ✅
The Canary GPU capture reports every vertex stream with fetch `endian = 2` (`k8in32`, each 32-bit
word byte-reversed). This describes the **guest-memory** copy the GPU fetches — **not** the `.xpr`
file bytes. Reading the file with a `k8in32` transform breaks the normals (mean `|normal|` → 1.33);
the plain per-element big-endian read yields exactly unit normals. So the game **rearranges** the
vertex data between the on-disc `.xpr` and the uploaded buffer; the decoder reads the file directly
and must use naive BE.
## Stage containers — multi-resource, grouped pools ✅ (content-anchored)
`hidden/resource3d/Stage_S*.xpr` are not single models but **collections of
enemy / prop sub-models** — up to ~400 `XBG7` resources each (e.g. `Stage_S07` =
378). Their data layout differs from the weapon files:
- Each resource is a block `[12-byte header][index buffer][vertex buffer]`.
Unlike weapons there is **no 12-byte gap between index and vertex** — the
vertex buffer directly follows the indices.
- The **index count** is the descriptor's `(index_bytes, index_count)` marker
(the *total* for the resource — a resource may have several sub-meshes summing
to it), **not** the first sub-mesh record. The **vertex count** is a `u32`
stored **32 bytes before** the marker.
- The blocks are **scattered through the data section, interleaved with the
container's texture data**, in an allocation order that is **not** directory
order and is **not stored** in any descriptor field we could find (the
descriptor holds sizes — `rel 160 ≈ index_bytes+3`, `rel 164 =
0x1000_0000 | (vertex_bytes+2)` — but no data offset). Reconstructing that
allocation order is unsolved.
Because the offset is not stored, each resource's block is located by
**content**: parser `Xbg7Model::stage_models` does one `O(file)` pass per
distinct stride to find every **vertex-buffer start** (an offset whose NORMAL —
`f16×4` at vertex `+12` — is unit length while the *previous* stride slot's is
not, i.e. a run boundary; ~one candidate per block, not millions), then pins each
resource to the unique candidate where the `index_count` indices ending just
before it are all `< vertex_count`, reference nearly all vertices, and yield
non-degenerate triangles with real extent. This is fast (≤ ~2 s on a 70 MB
stage) and unambiguous (no two resources collide). Blocks that fail validation —
the few quantized hero bodies — are **skipped**, never emitted as garbage.
**Coverage:** 5662 sub-models decode across the 22 stage files (e.g. `Stage_S07`
366/378, `Stage_S10` 7/9 — including the main enemy bodies `e003`/`e005`, their
LODs, weapons, and props). The viewer (`spawn_stage_models`) lays the decoded
sub-models out as a side-by-side "cast sheet", skipping the few huge skybox-plane
resources (300 k-unit quads). Cross-checked: `e003` = 2383 v / 1436 t bbox
23.5×9.5×32.5; `e005` = 2566 v / 1507 t; both 0 degenerate.
## Not yet decoded — ❔ the complex body layout
The hero-ship *body* meshes (`DeltaSaber_A/_T/_W.xpr` resource `f004`, and ~100 other models) still
decline. Two open sub-problems: (1) **multi-sub-mesh models** whose *first* sub-mesh decodes but a
later one's inter-mesh offset isn't yet handled (the `align16` advance is a guess) — these are
declined whole; (2) the big body meshes, where the data section does not start with an index buffer
and the geometry sits at descriptor-addressed offsets. NB the GPU capture showed **every** rendered
mesh is *single-stream* (just wider strides, e.g. 44 bytes = pos + f32×3 + colour + 2×f16×4), so the
body is likely single-stream-with-a-richer-declaration rather than the "separate streams" first
guessed — it was simply not rendered in the captured session (menu only). A capture taken *in a
mission* (where `DeltaSaber` renders) would hand over its exact declaration directly. `DeltaSaber_A`'s
5 XBG7 blocks are `f004` (body) + `_rou_f004_mnv01_L/_R`, `_mnv02`, `_turn180` (maneuver / pose).
## Evidence log
- 2026-07-12 — `rou_f001_wep_00.xpr`: XPR2 dir = 1×XBG7 (`f001_wep_00`) + 3×TX2D
(`_col/_gls/_spc`). Data section starts with a u16-BE index run (max 214), then stride-24
vertices. Descriptor record `[215,0,1092,4]` at desc `+0x2B0`; index-buffer byte size `0x888`
(=2184=1092×2) at desc `+0x1A0`. Recovered mesh = 215 v / 364 t, 362 non-degenerate, median tri
area 0.015 — coherent. → single-stream layout **PROBABLE**.
- 2026-07-12 — descriptor-driven sequential carving over all 166 models: **42 carve** under the
index-only check. Real 3D extent confirmed on `rou_f001_wep_04` (bbox 3.7×1.2×3.5).
- 2026-07-12 (refine) — attribute ranges differed per model (`wep_00` UV≈attr0/2, `wep_04`
attr4/5, `wep_03` attr1≈±60000 = garbage) → **refuted the fixed "6 half attrs, UV=attr0/2"
reading.** Found the descriptor **vertex declaration** (offsets 0x0C/0x14, usages 0x03 NORMAL /
0x05 TEXCOORD). Solved the vertex-base offset with a **unit-normal validator**: `index_end + 12`
gives median `|normal| = 1.000` on every weapon model (`wep_00/02/03/04`), vs `align16` landing 4
bytes early. UVs then land in `[0,1]`. Adding the normal gate: **25 models decode clean +
normal-valid** (the rest — incl. `Stage_S*` degenerate blobs — correctly declined).
- 2026-07-12 (DYNAMIC) — added a cvar-gated draw logger to Canary
(`command_processor.cc::LogDrawForRE`, cvar `log_draws`), captured the Ready Room / Briefings.
**GPU ground truth confirmed the static layout exactly**: primitive `prim=4` = **triangle LIST**
(settles the list-vs-strip question), and a stream with `f32x3 @offset0` + `f16x4 @3dw` +
`f16x2 @5dw`, stride 6 dwords = 24 bytes — matching `POSITION@0, NORMAL@0x0C, TEXCOORD@0x14`.
Revealed **stride varies** (24, 20, 28, 44 …) and that all streams are **single-stream**
parsed the declaration for variable stride: coverage **25 → 36** (e.g. `wep_05` is pos+normal,
stride 20, no UV — previously mis-aligned). Also confirmed the endianness note above: fetch
`endian=2` (k8in32) is the *guest* copy; file stays naive-BE. `Stage_S*` now decode (stride 20).
- 2026-07-12 (STAGE) — `Stage_S*.xpr` decoded as multi-resource containers. Found each geometry
block is `[12B hdr][index buffer][vertex buffer]`, index count = descriptor marker (total, e.g.
`e003` = 4308 spanning 2 sub-meshes), vertex count = `u32` at marker32 (`e003` = 2383 → verified
by max-index 2382 and 0 degenerate tris, bbox 23.5×9.5×32.5). Blocks are scattered among texture
data with **no stored offset** (descriptor rel 160 = idx_bytes+3, rel 164 = `0x1000_0000 |
(vtx_bytes+2)` are sizes, not offsets; block starts e.g. `e003`@0x10230, `e003_l`@0x5d000,
`e005_l`@0x9b000 are not directory-ordered). Solved by **content anchoring**: a single per-stride
pass finds vertex-run starts (unit NORMAL at +12 whose previous slot isn't), then match each
resource by strict index+triangle validation. **5662 sub-models decode across 22 stages** (S07
366/378), incl. the previously-declined main bodies `e005` (2566 v) and weapons — ≤2 s on 70 MB.
Parser `Xbg7Model::stage_models`, tests `stage_models_{decode,sweep,quality_audit}`.
- 2026-07-12 — `DeltaSaber_A.xpr` body: data does **not** begin with indices; plain-`f32×3` runs
with ship-scale extent (span ≈2734, matching bbox 30.0) found only at high offsets
(`data+0x28634C`, …) → multi-stream, **undecoded**.