fabi/Syplheed-Reborn
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: fabi:d92c7ddaead654848c4d077b254b2718de220c47
Branches Tags
fabi:feature/ipfb-idxd-parser fabi:main
..
compare: fabi:feature/ipfb-idxd-parser
Branches Tags
fabi:feature/ipfb-idxd-parser fabi:main

41 Commits
d92c7ddaea ... feature/ip

Author SHA1 Message Date
MechaCat02
a68405216f re: runtime-capture pipeline for exact capital-ship part placement 
Static external-part placement (ship::assemble_ship) is only approximate — a GN_*
frame is the mount pivot, and the real offset lives in a runtime detail rig. So
capture ground truth from Canary instead.

Finding: the guest vertex buffer holds LOCAL coords (byte-identical to the .xpr),
so capital-ship parts are placed entirely in the vertex shader. The per-part
transform is in the VS float constants — c0..c2 are the WorldViewProjection rows;
normalising each by its norm (= projection x/y/z scales, sy/sx = 16:9) yields the
rigid WorldView (rows orthonormal, det +1). Expressing every part relative to a
reference part cancels the camera: rel_p = WV_ref⁻¹·WV_p, a pure ship-space rigid
transform.

- examples/correlate_capture.rs: parse xenia_ship_capture.log, decode the ship's
  parts, match by vertex count, recover each part's ship-relative placement.
- docs/re/ship-placement-runtime-capture.md: full method + the capture protocol
  (Canary branch capture-ship-placement, F10 hotkey) + validated e106 result.

Validated on e106 (ADAN Destroyer, Stage_S01): all 7 drawn parts matched by
vertex count; bdy_01/bdy_02 recovered as a PORT/STARBOARD pair (X = ∓264) that
static had overlapping; engines correctly aft; extent 872×670×2181. Only brg_01
(bridge) missing — culled at that camera angle, needs a bridge-facing capture.

HANDOFF: next = re-capture bridge, bake a per-ship placement table the viewer
prefers over assemble_ship, then capture the cruiser / ACROPOLIS / TCAF ships.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-24 10:50:26 +02:00
MechaCat02
79c78cc7e1 ship: apply scale + gate approximate external parts behind a toggle 
Two changes after user feedback that externals sit near-but-wrong (shield inset
into the hull, thruster floating close):

- Scale: the joint table's slots 5–7 are per-axis scale (a GN_ShieldG frame ships
  at 0.5, GN_Jet FX at 2.4–6.5) and were being ignored, rendering scaled parts at
  the wrong size. ScenePart now carries `s` and applies R·(S·v)+T.
- External placement is fundamentally approximate: a GN_* frame is the mount
  PIVOT (f105's two GN_ShieldG frames are both on the centreline), while the
  part's outboard/rotational offset lives in a detail sub-rig / runtime code this
  static pass can't recover. So assemble_ship gains `include_external`: the hull
  tier (rou_ nodes) is exact and always drawn; the external tier (bridge / shield
  / engine at GN_ frames) is opt-in via a "Show external parts" checkbox in the
  Ships browser, off by default so the clean, correct hull is the default view.

The module doc is corrected to the scene-graph mechanism (the old "draw parts
untransformed" claim was wrong).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-23 22:11:47 +02:00
MechaCat02
8c8e2dbeb4 ship: use only the primary composite — fixes parts floating mid-ship 
The sub-composites (`e_rou_<id>_eng`, `_wep_*`) are LOCAL detail rigs: their
nodes are posed in their own frame, not ship-space, so folding them in placed
the engines at ~mid-ship (Z≈99) instead of aft — the "floating in air" parts.

The PRIMARY composite (`e_rou_<id>`, the one with the most nodes) already carries
the full ship-space layout: every hull body AND every `GN_*` hardpoint frame
(GN_Bridge at the stern, GN_Engine aft, GN_ShieldG amidships…). Assemble from
that one composite only; engines/bridge/shield now resolve to their real aft/mid
frames. Robust to the odd single-composite case (e108's `e_rou_e108_Missile_open`
is still picked as its own primary).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-23 21:52:20 +02:00
MechaCat02
a8b5691d1c ship: assemble capital ships from the composite scene graph (real placement) 
The first cut drew every part at the origin, overlapping into a mess — the parts'
vertex buffers are authored around the origin, NOT pre-placed. The real world
placement lives in the composite scene-graph resource `e_rou_<id>`, exactly like
node_transforms poses the DeltaSaber's fins within one resource — but here the
nodes reference SEPARATE geometry resources.

Two tiers, matching how the game splits a ship (as the user described):
- Hull — `rou_<id>_*` scene nodes name the hull body resources and carry their
  world transform (bodies that together form the hull, not individually
  destructible). Composed parent∘child down the first-child/next-sibling tree.
- External hardpoints — `GN_*` frame nodes are the Vessel mounts (`GN_Bridge_01`,
  `GN_ShieldG_01`, `GN_Engine_01`, matching the IDXD recipe Frame names). Each
  bridge / shield-generator / engine part is placed at its frame by category+idx —
  the individually-destructible parts.

- mesh: `scene_world_nodes` — walk a composite, emit every `rou_`/`GN_` node with
  its composed world transform (same record layout + TRS convention as
  node_transforms, cross-resource). ScenePart{resource,m,t}+apply.
- ship: `assemble_ship(bytes, id)` — resolve hull nodes to the ship's own parts
  (shared cross-id turrets excluded — they need the per-mount recipe), match
  external parts to GN frames, fall back to a raw draw when a prop has no
  composite. is_base_part now also drops `_dNN`/`_lNN`/`_mNN` LOD copies.
- viewer: build_ship_model runs assemble_ship, decodes only referenced resources,
  bakes each part's world pose into its vertices (one posed copy per placement),
  then assembles via the shared prepare path. RequestShipRender carries the id.

Verified headless: parts now spread into coherent single-ship extents
(e108 185×174×683, f106 703×479×2079, f101 carrier 560×539×1712) instead of
piling at the origin. Turret placement (shared eNNN gun models on many GN_*Gun*
frames) still pending the per-mount recipe.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-23 21:44:25 +02:00
MechaCat02
720a5fec27 viewer: Ships browser (View ▸ Ships) — assemble & view whole capital ships 
Adds a Ships window that scans the stage containers, reconstructs each capital
ship from its XBG7 part family (via sylpheed_formats::ship), and renders the
whole assembled model in the 3D view on click.

- iso_loader: ShipBrowser resource + ShipRow (id, faction, name, parts, stage,
  and linked Vessel stats: HP/size/turrets/bridges/shield gens), RequestShipCatalog
  + RequestShipRender events, ShipCatalogLoaded message. build_ship_catalog scans
  Stage_S01..S30, groups resources into ships, joins Vessel recipes by id, and
  sorts capital ships (Vessel-matched) first. handle_ship_render_request decodes
  only the ship's parts (models_named) and assembles them off-thread, reusing the
  existing XprPrepared → apply_prepared_xpr render + camera-focus path.
- prepare_models gains an `assemble` mode (prepare_ship): parts share ONE recentre
  so each keeps its ship-local offset (bridge fore, engines aft) instead of the
  per-model recentre / thumbnail grid used for a stage browse.
- ui: draw_ships_ui (own system, keeps draw_viewer_ui under the 16-param limit) —
  faction filter chips, name/id search, collapsible per-ship stat cards with an
  "Assemble & view in 3D" button; colour-coded by faction. View ▸ Ships menu item.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-23 21:16:15 +02:00
MechaCat02
cacb576ecd formats: ship module — reconstruct capital ships from XBG7 part families 
Project Sylpheed never stores a capital ship as a single mesh. Each ship is
modelled, then split into destructible parts (hull segments, bridge, engines,
integral turrets), each shipped as its own XBG7 resource inside a
hidden/resource3d/Stage_SNN.xpr container. The split lets the game hide/replace
a part when the player destroys it. Crucially each part keeps its ship-local
position, so reconstructing the whole ship is: draw every base part of one id
together, untransformed.

New `ship` module: Faction (from the e/f/n id prefix — ADAN/TCAF/Neutral),
ShipModel, ship_id_of, is_base_part (skips _l/_m/_d LODs, _bNN/_dead/_break
destruction geometry, e_rou_ scene nodes), and ships_in_container to group a
stage's resources into whole ships.

Supporting:
- mesh::xbg7_resource_names — list a container's XBG7 resource names (directory
  walk only, no geometry decode).
- Xbg7Model::models_named — decode only the named resources (assemble a ship
  from its ~8 parts instead of the whole ~300-resource stage).
- game_data::Vessel gains `model` (the rou_<id> hull stem) to link stats to the
  3D part family.

Tests: id/part classification + a disc-gated reconstruction of the ADAN e108
frigate from Stage_S02.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-23 21:16:04 +02:00
MechaCat02
df724b6bcb viewer: Game Data browser (View ▸ Game Data) 
Wire the decoded game_data + localization tables into the viewer as a
browsable window. View ▸ Game Data opens a floating panel with category
tabs — Weapons, Craft, Capital Ships, Characters, Missions, Arsenal,
Flights — decoded off-thread from GP_MAIN_GAME_E + GP_HANGAR_ARSENAL.

- Weapons/Craft/Ships: sortable stat grids (power/velocity/range;
  HP/cruise/accel/radar/turrets; HP/length/turrets/bridges/shield-gen),
  with a live name filter.
- Characters: name (localized) + faction (colour-coded) + portrait count.
- Missions: collapsible per stage — location, resolved objectives, fail
  conditions, and the enemy roster with HP.
- Arsenal: the player's nose/arm weapon options in four columns.
- Flights: distinct wingman line-ups.

Its own egui system (keeps draw_viewer_ui under Bevy's 16-param limit);
a RequestGameData event from the menu opens it and lazily decodes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-23 20:44:40 +02:00
MechaCat02
d41b065813 game_data: Arsenal loader — player weapon options per hardpoint 
Decode the player weapon arsenal from the UNITS tables
(GP_HANGAR_ARSENAL.pak): the selectable weapons per Delta Saber
hardpoint. Each `STANDARD_<hp>` header (Nose / Arm1-3) lists its options
up to the next header — extracted by shape, unioned + deduped across
configs.

The arsenal reads with the game's own names: NOSE guns (Stiletto,
Rapier, Broad Sword, Machine Cannon), ARM missiles/bombs (Falcon, Eagle,
White Shark, Piranha, Tomahawk Rail Gun, Maelstrom Bomb). Example
`arsenal` prints them. +1 test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-23 20:30:13 +02:00
MechaCat02
f2991b61ab game_data: load_weapons catches variant-schema weapons 
The player's Delta Saber missile / special weapons (Weapon_DSaber_P_wep_*)
live in variant schemas (439b1f29, 28295901), not the main WEAPON schema,
so the schema-only filter missed 10 of them. Match all `Weapon`-shaped
records by table name instead (excluding the EnumWeapon name lists),
deduped by id: 121 → 131 weapons.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-23 20:26:35 +02:00
MechaCat02
f5182f9649 game_data: PilotRoster loader — player squadron flight assignments 
Decode the UNITS/ZUNITS player-unit tables (GP_HANGAR_ARSENAL.pak) into
flight assignments: which pilot flies each callsign (Rhino1→Katana,
Bird1→Sandra, …), plus the player craft. Assignments are `Callsign<N>-
Pilot` tokens, extracted by shape.

138 configs; the distinct line-ups trace the campaign's story — Raymond
leads Rhino flight early (Rhino1=Raymond, Rhino2=Katana), then Katana
takes the lead (Rhino1=Katana, Rhino2=Ellen, Rhino3=Gene, Rhino4=Yoji).
Example `flights` prints the line-ups. +1 test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-23 20:14:22 +02:00
MechaCat02
551291f869 game_data: UnitRoster loader — per-mission battle compositions 
Decode the EnumUnit tables (schema 35b8dc67) into mission rosters: the
distinct combatant `UN_*` ids for one battle, stage props (asteroids,
collision meshes) dropped. The tables aren't tagged with their stage on
disc, so `stage` is inferred from any `UN_S<NN>_…` prop id in the roster
and left None otherwise (honest — ~8 of 31 rosters self-identify).

Joined against load_units / load_vessels this gives each battle's
combatants with stats, e.g. S01 = ADAN Turret (100 HP), Attacker
(500 HP), Destroyer (10000 HP). Example `battle` prints them.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-23 20:11:32 +02:00
MechaCat02
6e704291be game_data: DemoMessage loader — the dialogue backbone 
Decode the message tables (schema b412e6d8) into dialogue lines: each
`Message_NNN` sub-record → a DemoMessage with the speaker, portrait
(face), voice-clip token, and caption page text-keys. Fields are found
by prefix (Character*/Face*/VOICE_*/id-prefixed pages), so the
positional / first-record-defines-schema layout doesn't matter.

10,263 lines parse; ~815 name an explicit speaker (the rest are system /
continuation lines — left unattributed rather than guessed). Paired with
localization::TextIndex the attributed ones read straight out — e.g.
KATANA [VOICE_RHIN_000] "Ellen, get back into formation!". The
voice-clip token ties each line to its audio bank, closing the loop with
the cutscene-voice work. +1 test; example `dialogue` prints
speaker + clip + resolved text.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-23 19:59:14 +02:00
MechaCat02
2930770060 game_data: Squadron loader — mission flight groups 
Decode the per-stage Enumerate_Squadrons tables (UnitGroup_S<NN>.tbl)
into flight groups: formation shape, AI behaviour, side, and the member
craft. Definitions are delimited by the FormationID key; members are the
first `count` UN_ craft after the header (anything past them belongs to
the next squadron or the stage roster — the naive "all UN_ in span"
over-collected). e.g. the player's Rhino flight = a 2-craft TCAF
squadron of Delta Sabers flying Formation_2_Rhino. Schema hash varies
per stage so we match on the table name. Squadron id is positional and
left None when the id list and definitions don't line up (honest, not
guessed). +1 test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-23 19:54:48 +02:00
MechaCat02
d73601fe45 localization: resolve the game's UTF-16 text keys to display strings 
Project Sylpheed stores its UI/story text in IXUD entries as UTF-16LE,
interleaved `text, key` (a prose string immediately followed by the
identifier that names it). TextIndex::build(pak) scans a language pak's
IXUD entries and indexes every pair — 8457 English keys from
GP_MAIN_GAME_E: objectives, hints, lose conditions, character names,
and cutscene dialogue.

This makes the whole game read in English: the 16-mission campaign
(Repel the surprise attack → Shoot down Margras → the Prometheus Driver
finale), each mission's briefing, and the named cast (Katana, Raymond,
Ellen, Crichton, …) all resolve. API: get / objectives / lose_conditions
/ hints / character_name. 3 tests.

Examples: campaign / dossier print the readable campaign + cast.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-23 19:49:42 +02:00
MechaCat02
99f2ef8871 game_data: typed loaders for the combat / character / mission tables 
Decode Project Sylpheed's IDXD data model into plain, cloneable structs
(from GP_MAIN_GAME_E.pak; the D/F/I/J/S paks are localized dups):

- Weapon (121), CraftUnit (89, with the AI flight model in .fields),
  Vessel (23, structural component counts), PlayerConfig (24) — the
  full combat balance.
- Character (68 — faction + portrait set), Stage (29 missions:
  location, phases, and the per-stage resource tables it wires up).
- Generic Record / load_records(schema) reads any of the 105 IDXD
  schemas without a bespoke struct.

Each blob = one entity; token[0] names the table, fields are
value-before-key. Named Option<> fields for the common stats + a full
`fields` map; defaulted fields stay None (they're code-resident, not on
disc — honest blanks, never guessed). 8 tests against the real disc.

Examples: iso_map / deep_map census the ISO's formats and IDXD schemas;
dm_extract / dm_rows / dm_roster / mission_map dump the decoded data.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-23 19:49:21 +02:00
MechaCat02
161359e151 Fix hokyu voice: resolve by subtitle demo-id, not ship category 
The 13 manifest-unbound hokyu (resupply) cutscenes reuse 5 recordings
(VOICE_D_450..454). The category guess (ship LS/DS × source carrier/
tanker) was wrong for some. The real selector is the cutscene's subtitle
demo-id: 600→450, 601→451, 602→452, 603→453, 604→454 — derived from the
5 BOUND hokyu (each carries both a demo-id and a VOICETRACK), so it
self-validates. Fixes hokyu_LS_s11A/s15A (→451, were wrongly 450) and
hokyu_LS_s24A/s27A (→silent, no voice cue). User-confirmed all correct.

hokyu_voice_token() builds the demo→token map from the bound hokyu and
looks up the target movie's demo-id; replaces the category fallback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-23 19:48:56 +02:00
MechaCat02
053aa81953 Cutscene voice: resolve movies via continuous-stream cue regions 
Cracked how the game maps a movie to its voice track and reimplemented it,
fixing wrong/short voices for RT and hokyu cutscenes.

## Mechanism (RE'd from a Canary file-I/O trace, user-confirmed by ear)

The movie voices are ONE continuous XMA stream, physically chunked into
`<lang>\Movie\VOICE_*.slb` (and `\etc\VOICE_D_*.slb`) sound.pak TOC entries
whose boundaries do NOT align with the cutscene cues. A single cue routinely
spans two `.slb` chunks, so a `.slb` does not necessarily hold the track its
name claims (they are off-by-one vs the cues). Each cue ends at an inline
trailer descriptor `(sound_id:u32be, 0x11, …)` whose id repeats at +0x800.

Resolution chain (all derivable on-disc, no guest-code RE):
  1. movie -> cue token         (ADVERTISE_MOVIE manifest VOICETRACK)
  2. cue token -> sound-id       (master sound registry, tables.pak
                                  a2c8c185=eng / b04238e0=jpn, schema 0x13cb84ba)
  3. sound-id -> [start,end)      scan the stream for trailer(id) and its
                                  predecessor; cue audio = the bytes between,
                                  read continuously across chunk boundaries.

Validated: decoded voice length matches each movie within ~0.4s, including
cross-boundary cues (ADV/RT01A/RT01B/RT01C/S00A/S01A + the 5 bound hokyu).

## Changes

- sylpheed-formats/src/movie_voice.rs (new): registry_voice_ids(),
  find_descriptor(), find_descriptor_before() (gap-tolerant predecessor).
  Unit-tested.
- viewer iso_loader.rs: resolve_movie_voice_region() + decode_voice_region()
  (continuous read via existing read_segment_range + to_xma_riffs). Voice/
  audio requests now region-first; a `\Movie\` token that fails region stays
  SILENT (never plays the wrong off-by-one chunk). Removed the old
  movie_is_demo_based suppress hack.
- Anchor tries subdirs Movie/etc/Voice so hokyu `\etc\VOICE_D_*` resolve too.
- Examples resolve_all/validate_cues/hokyu_final document + validate the pipeline.

## Status

- RT / S / ADV cutscene voices: CORRECT (user-confirmed).
- 5 manifest-bound hokyu (VOICE_D_450-454): CORRECT (user-confirmed).

## OPEN (handoff)

13 of 18 hokyu movies have NO manifest VOICETRACK; the game reuses the 5
recordings across stages via a code rule. `hokyu_fallback_token()` currently
guesses by category (ship LS/DS x source carrier-A/tanker-H: LS/A->450,
LS/H->453, DS/A->452, DS/H->454). USER REPORTS THIS IS SOMETIMES WRONG.
- LS/carrier has two takes (450 from s02A, 451 from s09A); the early-vs-late
  split is unknown — unbound LS/A currently all default to 450.
- Definitive fix = Canary `--phase_a_fileio_only` file.read trace of an unbound
  hokyu (e.g. hokyu_LS_s03A) to see which VOICE_D the game actually streams,
  then encode the real rule. Same method that cracked the RT mapping.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-23 06:53:55 +02:00
MechaCat02
ed5c2f24c3 feat(formats): decode the movie manifest's full mission/phase structure 
The movie manifest (tables.pak, schema 0x067025b9) is the game's authoritative mission -> phase -> cutscene table. Previously we only scraped VOICE_/SUBTITLE_/.wmv strings by prefix; now we decode the real two-array (slot-keys, then value-records) structure.

MovieEntry gains slot, kind (System/Intro/Phase/PhaseEnd/Supply), mission, phase, subtitle, and telop (the previously-ignored .prt on-screen text overlay). Valueless slots (stages with no resupply video) are recovered without decoding the binary node region, via the fully-derivable MS<NN><X> -> S<NN><X> anchors.

Backward compatible: movie/voice_token and resolve_voice_entry/voice_token/is_manifest are unchanged. Validated end-to-end on the real disc: 101 entries (104 slots - 3 gaps), typed System=6/Intro=27/Phase=32/PhaseEnd=18/Supply=18. examples/manifest_map.rs dumps the full map.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-21 22:49:19 +02:00
MechaCat02
eee1607e1b feat(formats,viewer): standalone voice player covers all line categories 
list_voice_clips now enumerates every spoken-line category so the standalone audio player covers them all — bound movie voices (\Movie\), in-mission radio (\Voice\, \etc\), and mission briefings (\Briefing\, whose BR<NN>_<MM> names lack "VOICE"); music (BGM_*) stays excluded.

Viewer voice-library fixes: invalidate the library when a new game source loads, and don't latch an empty result as "loaded" (sounds.tbl always has thousands of clips, so empty = a failed read) so re-opening the menu retries.

Voice decode reverted to the first sub-wave (base behaviour): a naive multi-sub-wave concat produced the wrong track for RT movies. The per-sub-wave splitter (to_xma_riffs) is kept + documented for the eventual real fix, which needs ground-truth playback instrumentation to resolve segments-vs-takes-vs-timecode placement.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-21 22:49:09 +02:00
MechaCat02
95de29f290 feat(formats,viewer): movie subtitles, voice decode, and the movie manifest 
The movie cutscene subtitle + voice pipeline, driven by the ADVERTISE_MOVIE
manifest (the authoritative movie -> subtitle -> voice index). Also flushes
several sessions of local WIP (async viewer loading, grouped-pool XBG7/hero-ship
decode, drawlog tooling). See docs/HANDOFF-movie-voice-subtitles-2026-07-19.md.

Subtitles (movie_subtitle.rs):
- Full movie->track->text chain; join multi-line captions sharing one timing
  (fixes S13A dropped "Look at it father" line); overlap-safe active_cues();
  Latin-1 accents preserved.

Voice (slb.rs): XACT .slb -> XMA1 RIFF; take the FIRST sub-wave bounded by its
declared data size (fixes S10-S16 alternate-take garble); list_voice_clips.

Manifest (movie_manifest.rs): parse ADVERTISE_MOVIE (0x5B983A08) for the real
movie->voice binding (not always VOICE_<movie>; e.g. hokyu -> VOICE_D_* in etc\).
Resolve the token's sound.pak path via sounds.tbl. DIRECT bindings only — the
demo-id shared-clip fallback for unbound hokyu movies was verified WRONG in-game
and reverted (unbound hokyu stay unvoiced; correct join key is an OPEN problem).

Viewer: manifest-driven voice (movie player toggle + solo button), standalone
"Voice Lines" browser, stacked caption overlay.

Tests: 46 formats-lib + 11 viewer-lib + movie_manifest/movie_subtitle/slb disc
tests; full workspace green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-19 20:15:41 +02:00
MechaCat02
578c71a1b9 [formats,viewer] Decode grouped-pool XBG7 models; fix mesh routing + albedo 
Revert the mis-guided "triangle STRIP" reading (a937779): XBG7 index
buffers are triangle LISTS (prim=4 in the GPU capture; stored-normal
agreement 1.000 as a list vs ~0.49 as a strip). Reinstate the
winding-consistency gate.

Crack the grouped-pool layout used by the hero ship and ~150 detailed
models. A resource's sub-meshes share one index pool (buffers 4-byte
aligned, in descriptor-marker order) followed by one vertex pool (each
vtx_count*stride, same order); the vertex pool is 4-byte aligned after
the index pool. The whole resource is derived from one anchored pivot:
  ib0 = vb0 - span - pad   (pad in 0..=3, alignment)
  ib[i] = align4(ib[i-1] + idx_count[i-1]*2)
  vb[i] = vb[i-1] + vtx_count[i-1]*stride
vb0 is found via the unit-normal vertex-run scan; the alignment is
confirmed by validating the LARGEST sub-mesh (most reliable), after
which the rest are read/validated. A single marker reduces this to the
existing adjacency anchor, so grouped generalises it.

Results (cross-checked against a Canary GPU draw-log capture of
DeltaSaber_T.xpr):
- DeltaSaber_T f001 = body + 7 detail parts = 8650 tris, every sub-mesh
  0-degenerate / full-coverage / winding-agreement 1.000.
- All 19 previously-declined weapon models now decode (they hit pad 2
  and/or lead with a tiny bracket that broke a markers[0] pivot). Corpus
  audit: 0/146 geometry files fail (was 19 -> flat-texture in the viewer).
- Stages unchanged (single-marker path is byte-identical; grouped falls
  back to the old anchor on failure). 7/7 disc tests green incl. the
  strict stage quality audit; new hero_ship_grouped_pool_decodes test.

Viewer: route by count_xbg7; --only matches an exact model name (so the
neutral pose renders without the mnv*/turn180 animation poses). Fix the
albedo matcher: match a sub-model to its _col map by entity stem
(e007_bdy_01 -> e007_col) instead of a full-name prefix, lifting stage
sub-model texturing from ~25% to ~95% (the rest were flat grey). Colour
correctness (channel order/sRGB) remains a separate dynamic-RE item.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-18 19:54:39 +02:00
MechaCat02
a937779c77 [formats] XBG7 meshes: index buffers are triangle STRIPS, not lists 
The weapon/prop models (rou_fxxx_wep_nn.xpr) rendered with triangular holes in
the hull and misplaced/extra spikes. Root cause: XBG7 sub-mesh index buffers are
triangle STRIPS, but the decoder read them as triangle LISTS — a strip of N
indices is N-2 triangles, a list is only N/3, so ~2/3 of every hull was missing
(the holes) and each list-triple of strip data spanned unrelated vertices (the
spikes). The tell: triangles-per-vertex was 0.5-1.0 with 0 unreferenced vertices
(a closed surface needs ~2.0).

- mesh.rs: expand_triangle_strip() converts strip -> list with alternating
  winding, skipping degenerate triangles (repeated index = strip restart).
  Applied in both from_xpr2 (weapons) and read_pool_mesh (stage sub-models).
  All 71 weapon submeshes now have healthy ratios; hulls fill in (wep_03 cannon,
  wep_04 pod, wep_11, wep_37 verified coherent).
- Module doc updated (strip, not list).
- sylpheed-cli: `mesh info` reports per-submesh degenerate/unref/spanning-triangle
  counts; `mesh render` gains --dist (camera zoom). XMESHDBG env dumps the
  descriptor index markers.

Known residual: an index buffer may concatenate several strips with no degenerate
bridge, leaving ~2 spanning "spike" triangles at each restart (index jump to a new
vertex region) — <1% of tris on most models. Decoding the restart mechanism is a
follow-up (a blanket spanning-filter is unsafe: clean models have legit elongated
tip triangles).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-18 13:04:12 +02:00
MechaCat02
7143ea18fe [formats] Fully crack T8aD: decode the header's per-tile offset table 
Follow-up to the 256×256-tiling reversal — the residual boundary seams on
≥2×2-tile textures are gone. Decoding the file header revealed the exact
layout (no oracle capture needed):

  0x00  44        base header
  0x1c  4         tile count = ceil(w/256) * ceil(h/256)
  0x2c  tiles*4   BE-u32 absolute offset of each row-major 256×256 tile
  <off> 16        per-tile header, then tile_w*tile_h*4 A8R8G8B8 pixels

The old seams came from ignoring the offset table and the 16-byte per-tile
header (contiguous-packing drifted 16 bytes per tile). Small textures decoded
before only by luck: one tile puts pixels at 44+4+16 = 64, the old type-1
"header size". Now the 8AX title background, the prselect_win1 window frame,
and preff04 all decode pixel-perfect (verified against the title screen).

- t8ad.rs: parse() walks the offset table; dropped detile_256 + the
  header-size-by-type table. Non-tilecount entries (field != ceil*ceil) return
  None (likely DXT/other, deferred). New multi-tile round-trip test.
- sylpheed-cli: pak textures decodes via parse(); XDUMPHDR=1 dumps the base
  header + offset table for RE. Removed the now-obsolete XTILE/XSCORE/XDESTRIP
  experiment knobs and the reconstruct_tiles/TV-sweep helpers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-18 11:55:25 +02:00
MechaCat02
4ba723b9a5 [formats] Crack large-T8aD tiling: 256×256 row-major raster tiles 
Verified against the running game (freeze-fixed Canary title screen) that
T8aD colours are correct (no R↔B swap) and that wide textures were garbled
by an unreversed tile layout, not a colour bug.

Reversed the layout: large T8aD are stored as 256×256 raster tiles in
row-major order, each tile raster internally, edge tiles clipped to the
image (payload is exactly w*h*4, no padding). Surfaces ≤256px wide are a
single tile column, identical to plain linear — which is why small UI
textures always decoded correctly.

- t8ad.rs: add detile_256() + apply in parse(). Single tile-row / single
  tile-column textures now decode exactly (ptcopyright, ptbtn, ptlogo1 =
  clean "PROJECT" logo, previously pure noise). Known residual: ≥2×2-tile
  textures (>256 in both dims, e.g. the 8AX background) come out coherent
  but with boundary seams — the multi-tile order is a subtle swizzle, TBD.
- sylpheed-cli: new `pak textures <pak> <out>` command — decodes every
  T8aD (direct, RATC-nested, LSTA frames) to PNG for A/B against the game.
  Env debug knobs for layout RE: XTILE/XDESTRIP/XDETILE_W/XREINTERPRET_PITCH
  /XSCORE (TV-ranked tile sweep), plus --verbose size classification.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-17 23:37:35 +02:00
MechaCat02
d23339a3aa feat(viewer): stage thumbnail grid, per-model albedo, camera zoom fix 
Route stage containers to stage_models (decode both single + stage paths, keep
whichever yields more geometry) — fixes stages showing the dummy-root "black
cube". Present a stage's sub-models as a thumbnail grid: each recentred and
uniformly scaled to a fixed cell so a 1-unit prop and a 3000-unit structure are
equally visible; skybox-plane / stray-anchor models (>20000u) are culled. Each
sub-model is textured by matching its name to the container's `_col` albedo.

Fix the orbit camera: zoom radius was hard-capped at 50u with fixed clip planes,
trapping the camera inside multi-thousand-unit stages. Zoom limits and near/far
planes now scale to the framed model.

Adds docs/HANDOFF-stage-mesh-2026-07-12.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-12 20:56:45 +02:00
MechaCat02
2053f31d17 feat(formats,cli): decode XBG7 stage containers + fix weapon layout 
Stage containers (hidden/resource3d/Stage_S*.xpr) are collections of up to
~450 enemy/prop sub-models, not single meshes. Xbg7Model::stage_models decodes
them: each resource is a [12-byte header][index buffer][vertex buffer] block
(index count = descriptor marker, vertex count = u32 32 bytes before it) whose
on-disc offset is NOT stored, so it is located by content — one O(file) pass per
stride finds vertex-buffer starts (unit NORMAL at +12 whose previous slot isn't)
and each resource is pinned to the candidate whose indices validate and produce
non-degenerate, well-connected triangles. A connectivity gate (mean triangle
edge <= 0.28x the bbox diagonal) rejects spiky mis-anchors. ~4993 sub-models
decode across the 22 stages.

The same insight fixes the weapon single-model layout: it is [12-byte header]
[index][vertex] too, not [index][12-byte gap][vertex]. Reading indices from the
block start turned the 12 header bytes into 6 junk indices (2 leading degenerate
triangles — the recurring stray-triangle artifact) and dropped the last 6 real
indices. Skipping the header leaves vertex offsets identical (coverage unchanged
at 36/166) and corrects the triangle list. Verified on wep_00/03/04.

Adds `sylpheed-cli mesh {info,render}` — a headless software rasterizer that
writes a shaded PNG (orthographic, z-buffered, two-sided), so recovered geometry
can be verified without the GUI. Stages render as a normalised thumbnail grid;
--only filters sub-models.

Tests: stage_models_{decode,sweep,quality_audit}; docs/re updated (xbg7-mesh.md
evidence log + INDEX.md).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-12 20:56:35 +02:00
MechaCat02
4096b2d2a5 feat(formats): declaration-driven XBG7 decode (variable stride) 
Dynamic-RE follow-up: capture Canary's GPU vertex-fetch + draw calls and
feed the ground truth back into the static decoder.

The GPU capture confirmed the reverse-engineered layout exactly — meshes
draw as triangle LISTs (prim=4) with pos f32x3 @0, normal f16x4 @0x0C,
uv f16x2 @0x14 — and revealed the XBG7 vertex format is NOT fixed-stride:
models omit elements (stride 20 = pos+normal, no UV; 24 = pos+normal+uv).

- mesh.rs: parse the descriptor's vertex declaration ({offset, format-code,
  usage} triples; 0x2A23B9=pos f32x3, 0x1A2360=normal f16x4, 0x2C235F=uv
  f16x2) to drive per-model stride + element offsets, instead of assuming
  stride 24. Coverage 25 -> 36 fully-validated models (e.g. the Stage_S*
  props, which are pos+normal only). Same index-range + unit-normal safety
  gates; complex/mismatched layouts still declined.
- Endianness note (documented): the capture's fetch endian=k8in32 describes
  the GPU's guest-memory copy, NOT the .xpr file bytes — reading the file
  with k8in32 breaks the normals (|n|->1.33); naive big-endian per element
  is correct (the game rearranges vertex data on load).
- tests: a coverage-regression test (>=35 models) + the existing weapon /
  body-declined disc tests still pass.

Confirmed against a Canary draw-log capture; see docs/re/structures/
xbg7-mesh.md (evidence log + declaration table).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-12 18:42:21 +02:00
MechaCat02
ef6e448268 feat(formats,viewer): decode XBG7 meshes + 3D model preview 
Reverse-engineer the single-stream XBG7 geometry layout (clean-room: hex
inspection + geometric validation of the retail disc's
hidden/resource3d/*.xpr, no game code copied) and present models as
textured 3D meshes in the explorer.

Format (docs/re/structures/xbg7-mesh.md): XBG7 geometry resources sit
inside XPR2 containers alongside TX2D textures. For ~25 single-stream
models (weapons, simple props) the data section is a sequence of
sub-meshes, each an u16-BE triangle-list index buffer followed (after a
fixed 12-byte header) by a stride-24 vertex buffer whose declaration is
in the descriptor: POSITION f32x3 @0x00, NORMAL f16x4 @0x0C, TEXCOORD
f16x2 @0x14. Sub-mesh (vtx,idx) counts come from descriptor tuples.

The +12 vertex offset is pinned by the recovered normals being exactly
unit-length (align16 lands 4 bytes early and silently corrupts every
field). A safety gate rejects any model whose indices are out of range
or whose mean |normal| is not ~1, declining garbage (Stage_S*
placeholders, the complex multi-stream hero-ship body) rather than
mis-decoding it.

- mesh.rs: Xbg7Model::from_xpr2 -> GameMesh { positions, normals, uvs,
  indices }; standalone f16->f32; unit + real-disc tests (weapon decodes
  to 215v/364t with unit normals + in-range UVs; DeltaSaber body
  declined).
- texture.rs: from_xpr2_index / texture_names so the viewer can pick a
  model's _col albedo map.
- viewer: loose .xpr with decodable XBG7 spawns Bevy meshes (real normals,
  double-sided) textured with the albedo, framed by the orbit camera; the
  central egui panel goes transparent so the 3D scene shows through.

Complex multi-stream body meshes (DeltaSaber f004, other vertex layouts)
remain undecoded and are cleanly declined — next target is dynamic RE.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-12 17:47:06 +02:00
MechaCat02
e6da726f7b docs(re): CONFIRM T8aD colour interpretation (linear UNORM, ARGB order) 
Resolved the "colours look odd" channel-order/sRGB question for k_8_8_8_8
(T8aD 2D UI textures) with no emulator run:

- sRGB: Canary's Vulkan host-format table maps plain k_8_8_8_8 to
  R8G8B8A8_UNORM and contains no R8G8B8A8_SRGB anywhere — gamma is a
  separate explicit EDRAM path. So these are raw linear bytes; apply no
  sRGB/gamma decode.
- Channel order: measured 789 real disc textures — byte0 is the opaque
  alpha (==0xFF-dominant) in 97% — so on-disk is ARGB. Cross-checked
  against xenia-rs decode_k8888_tiled, the Canary-validated M1/M2 path,
  which nets the same [A,R,G,B]->[R,G,B,A]. sylpheed-formats is correct.

Promotes the k8888 entry HYPOTHESIS -> CONFIRMED; INDEX updated. XPR2
skybox tiling + viewer display-gamma remain separate open items.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-11 17:11:17 +02:00
MechaCat02
b7028471e9 docs(re): add reverse-engineering knowledge base + conventions
Some checks failed
CI / Native — ubuntu-latest (push) Failing after 7m53s
Details
CI / WASM — Web (push) Failing after 6m32s
Details
CI / Formatting (push) Failing after 35s
Details
CI / Native — macos-latest (push) Has been cancelled
Details
CI / Native — windows-latest (push) Has been cancelled
Details 
Establish docs/re/ as the spec-side of the clean-room:
- README: confidence tiers (HYPOTHESIS/PROBABLE/CONFIRMED), promotion
  requires NEW independent evidence, append-only evidence logs,
  demote-on-contradiction, and a clean-room firewall (describe behaviour +
  cite addresses; never paste decompiled code). Documents the toolchain
  (sylpheed.db + zq.py, xenia-rs probes, Wine-Canary oracle) and the
  join-by-guest-PC method with the VA-equality caveat.
- Function/structure templates + a seeded INDEX (9 already-reversed formats).
- First entry: texture-color-k8888 — CONFIRMED plain k_8_8_8_8 is linear
  UNORM (no sRGB), channel order still to be measured.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-11 17:04:23 +02:00
MechaCat02
e1ca95c0af feat(formats,viewer): decode T8aD textures, RATC bundles, LSTA sprites 
Add three static PAK-format decoders in the Bevy-free formats crate and
present them in the explorer:

- t8ad.rs: linear 32bpp A8R8G8B8 2D texture (UI/HUD art). Dims at
  0x14/0x18, header size keyed by the type field @0x1c (container-safe).
  Decodes the ~85% RGBA variants; defers the rest (likely DXT) rather
  than misdecoding.
- ratc.rs: nested resource bundle — lists named children by magic scan.
- lsta.rs: sprite list — walks the inline T8aD frames.

Viewer: PakContent gains T8ad/Lsta/Ratc, decoded off-thread and bounded
by an 8M-texel per-entry cap; detail views show the texture, a sprite
grid, and a child list with thumbnails. Decoded images carry a
"colours unverified" note — channel-order/sRGB stays on the dynamic-RE
backlog.

Tests: 12 new unit + 3 real-disc (T8aD >=70% decode over the hangar
pack, LSTA frames, RATC named children incl. a decodable T8aD).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-11 16:49:50 +02:00
MechaCat02
10425aba8c feat(formats): name font/image inner formats instead of hex 
inner_format_label mapped only IDXD/xml/printable-tag magics; fonts and
images fell through to a bare hex label (00010000, 89504E47, …). Recognize
the known binary signatures and return friendly short tags so the pak
browser's format column reads them: sfnt/true/typ1 → "ttf", OTTO → "otf",
ttcf → "ttc", PNG → "png", plus RIFF/DDS. Still-unidentified magics keep the
hex fallback. Shared by the CLI `pak list` and the GUI browser.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-11 15:24:26 +02:00
MechaCat02
e17bc0216d feat(viewer): present subtitles, fonts, PNG + text in the PAK browser 
Decode and present four more PAK item types in the pack browser's detail
pane. All content is decoded off-thread in build_pak_rows and attached to
each PakRow, then rendered with egui's own texture facilities — the Bevy
texture pipeline and egui's global font system are untouched.

formats crate (Bevy-free, reusable by CLI):
- ixud: parse IXUD localized-string tables as subtitle cue lists (UTF-16BE
  pool, SUBTITLE header + (text, timecode) pairs -> Vec<Cue{start,end,text}>).
- font: OTF/TTF/ttcf metadata (family / faces / glyphs) via ttf-parser.

viewer:
- PakRow gains `content: PakContent` (Subtitle | Font | Png | Text | None);
  classify_content fills it for IXUD / fonts / PNG / xml+text within the
  existing decode budget. Font samples are pre-rasterized off-thread with
  ab_glyph into an image (ImageRgba) — ab_glyph returns Option/Result at every
  step, so a bad font yields no sample instead of panicking (the earlier
  egui set_fonts approach crashed the app on atlas rebuild).
- draw_pak_browser dispatches on content: subtitle cue table, font metadata +
  sample image, PNG image, scrollable text; else the existing IDXD detail.
  Borrow-split PakView instead of cloning the (now heavy) rows each frame;
  one hash-keyed egui texture cache serves both PNG and font-sample images.

Subtitle<->movie auto-matching is deferred: movie filenames don't hash to the
IXUD keys and no cross-reference table was found (the link is an internal
MSG_DEMO id), so the movie player gets no subtitle track yet.

Tests: ixud unit tests; disc tests for the real English subtitle track
(cues + timecodes), eng/deu localization, font metadata, and off-thread
rasterization of the real font.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-11 15:19:31 +02:00
MechaCat02
91bd2543f4 feat(viewer): WMV cutscene player with scrub, keyboard + audio 
Add an in-explorer video player for the dat/movie/*.wmv cutscenes.

Decode via the ffmpeg/ffprobe CLI (no libav linking): video streams from an
`ffmpeg -f rawvideo -pix_fmt rgba` pipe read into fixed w*h*4 frame chunks over
a bounded channel (paused/behind => ffmpeg back-pressures on its pipe); one
reused Image is overwritten in place each tick. Audio is best-effort: the track
is pre-decoded to a temp stereo WAV and played through a rodio Sink (volume /
play / pause / seek-via-skip_duration for free). Native-only under
cfg(not(wasm32)); VideoPreview registers unconditionally so the UI compiles for
wasm. Mirrors the .pak load pipeline (FileSelected -> bg thread ->
IsoLoaderMsg::VideoLoaded -> apply_loaded_video -> advance_video_playback), with
a single free_video teardown wired into every other viewer's reset.

Controls:
- play/pause button, click-on-frame toggle, Space / arrow (+-10s) shortcuts
- seekable timeline with mm:ss labels; volume slider (greyed when no audio)
- live scrubbing: a coalescing one-shot `ffmpeg -ss -frames:v 1` grabber always
  jumps to the knob's latest position, so dragging shows the frame under the
  knob in real time; that grabbed frame also bridges the gap after a commit-seek
  so there's no black flash while the streaming decoder catches up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-11 14:31:52 +02:00
MechaCat02
589659bec6 feat(viewer): world cubemap (skybox) viewer — 6-face grid 
TXCM XPR2 resources are the game's world skyboxes (BG_Acheron, BG_Hargenteen,
…). The viewer showed only face 0 as a flat 2D image; now it decodes all 6 faces
and shows a labelled grid.

Formats (Bevy-free): X360Texture::cube_faces_from_xpr2() returns a Cubemap with 6
faces in D3D9 order (+X −X +Y −Y +Z −Z), or None for ordinary 2D textures. Face
layout derived from xenia's GetGuestTextureLayout — 6 back-to-back independently-
tiled surfaces, per-face stride = tiled-surface-size aligned to the 4 KiB
subresource boundary (kTextureSubresourceAlignmentBytes). Verified against real
BG_Acheron: data_size == 6 × 0x400000, and all 6 faces decode cleanly (own
Python decode + a disc test asserting 6×4 MiB faces and face 0 == the validated
from_xpr2 green-planet decode). Extracted a shared decode_surface() helper so the
2D and cube paths are byte-identical; from_xpr2 output unchanged (re-verified).

Viewer: new SkyboxPreview resource (6 egui face textures, reusing the existing
per-format x360_texture_to_bevy_image); populated in apply_loaded_texture's TXCM
branch; freed/reset alongside the other previews (factored free_texture/
free_skybox helpers). Central panel gains a skybox branch rendering a 3-column
labelled face grid.

DEFERRED (per "do not guess, else defer"): the interactive 3D skybox. Face data +
D3D9 order are validated, but wgpu cube-sampling handedness can't be confirmed
without eyeballing the GUI — a wrong-oriented skybox is worse than the correct
labelled grid. The grid is the reliable deliverable; the 3D look-around is a
follow-up once orientation is visually confirmed.

(Background agent did the investigation/validation but was blocked from writing
files; implemented here in the main tree from its findings, independently
re-verified.)

23 formats tests + disc cubemap test pass; viewer/CLI build; face-0 export
re-verified as the green planet.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-11 13:38:07 +02:00
MechaCat02
765e4a573b refactor(viewer): explorer UX cleanups (menu, loading, tree, status bar) 
Four small UX fixes to the asset explorer:

1. Remove the no-op "View" menu (Textures/Meshes/Audio toggles + Wireframe
   checkbox) — none were wired to anything. Dropped ViewMode + the current_mode/
   wireframe fields from ViewerState (nothing read them); the central panel
   already dispatches on which preview resource is populated.

2. Loading indicator — clicking a drawer item now flips FileBrowserState.loading
   and the central panel shows a centered spinner + "Loading <name>…" until the
   background read/decode applies, instead of leaving the previous item on screen
   with no feedback. Cleared in poll_loader_channel on FileLoaded/PakLoaded/
   Error/Cancelled.

3. Drawer shows only the .pak index, not its .pNN data segments — the segments
   are the pack's body (loaded when the .pak is opened), so listing them (with no
   preview) was noise. Dropped the virtual "<stem>.pak" folder too; the .pak now
   sits as a normal leaf in its real directory.

4. Bottom status bar no longer overflows — horizontal_wrapped, only the source's
   final path component (full path on hover), file count, and a right-aligned,
   shortened controls hint. Removed the mode label.

Viewer + formats build; all tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-11 13:21:38 +02:00
MechaCat02
fbd4550d62 feat(viewer): data pack (.pak / IDXD) browser 
Roadmap #3. The explorer can now open IPFB data packs — the game's ship/weapon/
mission definition tables. Selecting a `.pak` shows a master-detail browser:
entry list (hash · inner-format · identity) on the left, the selected IDXD
object's schema + explicit-field property table on the right.

Load path: a `.pak` selection reads the index plus its sibling `.pNN` segments
(ISO reader or extracted dir, probing p00.. until the first gap), assembles via
PakArchive::from_parts, and builds owned PakRow/PakDetail entirely off-thread —
so the UI holds only plain data (no borrow of a PakArchive, WASM-safe). Two
decode caps (64 MiB cumulative, 16 MiB/entry) keep a huge sound.pak from hanging;
over-budget entries show as "(not decoded)".

Wiring mirrors the texture path: new IsoLoaderMsg::PakLoaded → PendingPak staging
→ apply_pak. A shared reset (free GPU handle + clear texture/text/pak previews)
runs in both apply systems, so exactly one viewer is active per selection and the
prior GPU texture is always freed when switching modes.

Reuse over duplication: the CLI's inner_label / idxd_identity move into the
formats crate as pak::inner_format_label and IdxdObject::identity (bodies
verbatim → CLI output unchanged), now shared by CLI and GUI.

PakView registered unconditionally (wasm-safe; population native-only). ViewMode
unchanged — the central panel dispatches on the populated resource. Workspace
builds; 22 formats tests + all crate tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-10 23:17:10 +02:00
MechaCat02
2658dd20a6 feat(viewer): directory-tree browser + plain-text viewer 
Explorer drawer was a flat list of all 353 disc files and only textures had a
preview. Two additions:

1. Directory tree — the left drawer is now a collapsible `📁` tree, rebuilt each
   frame from the path list (ui.rs: build_tree / split_for_tree / render_dir).
   Filtering narrows leaves and auto-expands matches; `.p00…p04` data segments
   nest under a virtual `<stem>.pak` node. Selection still keys off the original
   file index, so the loader pipeline is untouched.

2. Plain-text viewer — config.ini and any file that sniffs as text now show
   read-only, selectable, monospace content with a Wrap toggle and an encoding
   label. Detection/decoding live in the Bevy-free formats crate:
   vfs::is_probably_text + vfs::decode_text (BOM-aware UTF-8/UTF-16LE/BE, BOM-less
   UTF-16LE heuristic, NUL-reject + 95%-printable fallback) — pure std, WASM-safe,
   unit-tested (6 new tests), and reused by the CLI sniffer (new cyan `txt`).

New TextPreview resource (registered unconditionally so it exists on wasm);
populated in apply_loaded_texture with a 2 MiB char-boundary-safe cap. Also fixes
a latent bug: switching from a texture to a non-texture left stale preview state —
both texture and text state now reset on every selection.

ViewerState gains text_wrap. Formats tests: 22 pass; workspace compiles.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-10 23:05:58 +02:00
MechaCat02
854fd8dfd3 fix(texture): make BC dword-swap format-aware (don't mangle DXT5A) 
The dword-swap that fixes DXT1 is a property of the BC1 COLOUR block, not of
BCn in general. RE on a DXT5A (BC4) mask showed the alpha block is byte-indexed
and wants the endian swap ALONE — dword-swapping it makes it worse. The blanket
swap from the previous commit was therefore breaking the (common) DXT5A/DXN
alpha & normal masks.

Now targeted by format:
 - DXT1            : swap dwords of each 8-byte colour block (verified).
 - DXT2_3 / DXT4_5 : swap only the colour half (bytes 8..16) of each 16-byte
                     block; alpha half endian-only. TENTATIVE — these formats
                     are near-absent in the game assets, so unvalidated against
                     a clear image.
 - DXT5A / DXN     : no dword swap (endian only).

DXT1 export re-verified clean (weapon skins); 16 formats tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-10 22:13:37 +02:00
MechaCat02
405233e84f fix(texture): crack X360 DXT1 dword-swap; wire correct decode into viewer 
DXT1 (108/137 textures in Base.xpr — the bulk of picture assets) decoded to
noise. Root cause found by RE: Xbox 360 BCn blocks store the two 32-bit words
of each 64-bit sub-block in the opposite order to the PC/DDS layout — colour
endpoints live in the HIGH dword, indices in the low one. The endian field
(k8in16) only fixes byte order within the 16-bit words; it does not reorder the
dwords, so without this every DXT texture transposed endpoints/indices → noise.

Isolation that led here:
 - de-tile proven correct for bpb=8 (coherent per-block signature map; the
   linear read is scrambled) — same faithful Xenos Tiled2D as the verified
   bpb=4 ARGB path (green Acheron backdrop).
 - inspecting a smooth region, coherent colour endpoints appeared only in
   bytes[4..8], with the high-entropy indices in bytes[0..4].

Fix: swap_bc_block_dwords() swaps the two dwords within each 64-bit unit after
the byte-level endian swap, for every BCn format. Verified in the real Rust CLI:
weapon skins (rou_f001_wep_*) now decode to clean, recognisable images.

Viewer: DXT1 is fixed transparently (from_xpr2 feeds tex.data straight to the
GPU as Bc1). Also corrected the uncompressed path — post-swap k_8_8_8_8 is
[A,R,G,B]; reorder to [R,G,B,A] and upload as Rgba8UnormSrgb (was Bgra8, wrong).

Knob: XPR_NO_BC_DWORD_SWAP disables the swap for A/B validation.

KNOWN REMAINING: 16-byte blocks (BC2/DXT3, BC3/DXT5) and BC4/BC5 (DXT5A/DXN)
still need their alpha+colour half layout worked out — they decode to noise for
now. DXT1 + uncompressed + cubemaps are correct.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-10 22:08:23 +02:00
MechaCat02
840db9c549 fix(texture): correct Xenos de-tile + endian + cubemaps + PNG export 
Reworks the XPR2 texture path after RE against the retail disc (viewer
previews were scrambled/failed).

- de-tile: replace the naive Morton-over-32×32 approximation with the
  faithful Xenos Tiled2D bank/pipe/macro-tile address formula (ported
  from xenia texture_address.h). Verified correct: the Acheron backdrop
  decodes to a sharp planet with its spiral storm.
- endian: undo the X360 word byte-swap per the fetch-constant endianness
  field (k8in16/k8in32/k16in32) — without it BCn/ARGB data is noise.
- cubemaps: parse TXCM resources (skybox/backdrops), decoding face 0,
  instead of erroring "No TX2D found".
- A8R8G8B8: correct channel order after the k8in32 swap ([A,R,G,B]→RGBA)
  — the Acheron backdrop is now correctly green, not red.
- XPR files are texture PACKS; add XPR_RES_INDEX to select a resource.
- CLI `texture export` now writes real PNGs (texpresso BCn decode +
  image), replacing the stub. Adds texpresso + image deps.
- tests/texture_disc.rs: integration test running the pipeline over real
  disc .xpr files (28/28 parse). RE debug knobs: XPR_NO_DETILE /
  XPR_NO_ENDIAN / XPR_FORCE_ENDIAN.

KNOWN ISSUE: mipmapped DXT1 textures still decode to noise — mip 0 is
not at base_address in the multi-resource packs (localized to a mip
storage-offset quirk; uncompressed + de-tile + endian are all verified).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-10 21:37:20 +02:00
68 changed files with 16048 additions and 310 deletions
Expand all files Collapse all files

468
Cargo.lock generated
View File

@@ -60,8 +60,8 @@ dependencies = [
"hashbrown 0.15.5",
"paste",
"static_assertions",
"windows",
"windows-core",
"windows 0.58.0",
"windows-core 0.58.0",
]
[[package]]
@@ -112,6 +112,28 @@ version = "0.2.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923"
[[package]]
name = "alsa"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed7572b7ba83a31e20d1b48970ee402d2e3e0537dcfe0a3ff4d6eb7508617d43"
dependencies = [
"alsa-sys",
"bitflags 2.11.0",
"cfg-if",
"libc",
]
[[package]]
name = "alsa-sys"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db8fee663d06c4e303404ef5f40488a53e062f89ba8bfed81f42325aafad1527"
dependencies = [
"libc",
"pkg-config",
]
[[package]]
name = "android-activity"
version = "0.6.1"
@@ -121,10 +143,10 @@ dependencies = [
"android-properties",
"bitflags 2.11.0",
"cc",
"jni",
"jni 0.22.4",
"libc",
"log",
"ndk",
"ndk 0.9.0",
"ndk-context",
"ndk-sys 0.6.0+11769913",
"num_enum",
@@ -1248,7 +1270,25 @@ dependencies = [
"proc-macro2",
"quote",
"regex",
"rustc-hash",
"rustc-hash 1.1.0",
"shlex",
"syn 2.0.117",
]
[[package]]
name = "bindgen"
version = "0.72.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "993776b509cfb49c750f11b8f07a46fa23e0a1386ffc01fb1e7d343efc387895"
dependencies = [
"bitflags 2.11.0",
"cexpr",
"clang-sys",
"itertools 0.13.0",
"proc-macro2",
"quote",
"regex",
"rustc-hash 2.1.3",
"shlex",
"syn 2.0.117",
]
@@ -1420,6 +1460,12 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "byteorder"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fd0f2584146f6f2ef48085050886acf353beff7305ebd1ae69500e27c67f64b"
[[package]]
name = "byteorder-lite"
version = "0.1.0"
@@ -1470,6 +1516,12 @@ dependencies = [
"shlex",
]
[[package]]
name = "cesu8"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6d43a04d8753f35258c91f8ec639f792891f748a1edbd759cf1dcea3382ad83c"
[[package]]
name = "cexpr"
version = "0.6.0"
@@ -1561,6 +1613,12 @@ version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c8d4a3bb8b1e0c1050499d1815f5ab16d04f0959b233085fb31653fbfc9d98f9"
[[package]]
name = "claxon"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4bfbf56724aa9eca8afa4fcfadeb479e722935bb2a0900c2d37e0cc477af0688"
[[package]]
name = "clipboard-win"
version = "5.4.1"
@@ -1764,6 +1822,49 @@ dependencies = [
"libc",
]
[[package]]
name = "coreaudio-rs"
version = "0.11.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "321077172d79c662f64f5071a03120748d5bb652f5231570141be24cfcd2bace"
dependencies = [
"bitflags 1.3.2",
"core-foundation-sys",
"coreaudio-sys",
]
[[package]]
name = "coreaudio-sys"
version = "0.2.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9b4739a805a62757a83e5654fa3faabec0442666b263bb2287d5a8185bfd953"
dependencies = [
"bindgen 0.72.1",
]
[[package]]
name = "cpal"
version = "0.15.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "873dab07c8f743075e57f524c583985fbaf745602acbe916a01539364369a779"
dependencies = [
"alsa",
"core-foundation-sys",
"coreaudio-rs",
"dasp_sample",
"jni 0.21.1",
"js-sys",
"libc",
"mach2",
"ndk 0.8.0",
"ndk-context",
"oboe",
"wasm-bindgen",
"wasm-bindgen-futures",
"web-sys",
"windows 0.54.0",
]
[[package]]
name = "cpufeatures"
version = "0.2.17"
@@ -1791,6 +1892,25 @@ dependencies = [
"crossbeam-utils",
]
[[package]]
name = "crossbeam-deque"
version = "0.8.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5181e0de7b61eb03a81e347d6dd8797bae9da5146707b51077e2d71a54ec0ceb"
dependencies = [
"crossbeam-epoch",
"crossbeam-utils",
]
[[package]]
name = "crossbeam-epoch"
version = "0.9.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2d6914041f254d6e9176c01941b21115dcfb7089e55135a35411081bd106ef3f"
dependencies = [
"crossbeam-utils",
]
[[package]]
name = "crossbeam-utils"
version = "0.8.21"
@@ -1830,6 +1950,12 @@ version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f27ae1dd37df86211c42e150270f82743308803d90a6f6e6651cd730d5e1732f"
[[package]]
name = "dasp_sample"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c87e182de0887fd5361989c677c4e8f5000cd9491d6d563161a8f3a5519fc7f"
[[package]]
name = "data-encoding"
version = "2.10.0"
@@ -2453,7 +2579,7 @@ dependencies = [
"log",
"presser",
"thiserror 1.0.69",
"windows",
"windows 0.58.0",
]
[[package]]
@@ -2548,6 +2674,12 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dfa686283ad6dd069f105e5ab091b04c62850d3e4cf5d67debad1933f55023df"
[[package]]
name = "hound"
version = "3.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "62adaabb884c94955b19907d60019f4e145d091c75345379e70d1ee696f7854f"
[[package]]
name = "icu_collections"
version = "2.1.1"
@@ -2726,6 +2858,22 @@ version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "jni"
version = "0.21.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1a87aa2bb7d2af34197c04845522473242e1aa17c12f4935d5856491a7fb8c97"
dependencies = [
"cesu8",
"cfg-if",
"combine",
"jni-sys 0.3.1",
"log",
"thiserror 1.0.69",
"walkdir",
"windows-sys 0.45.0",
]
[[package]]
name = "jni"
version = "0.22.4"
@@ -2845,6 +2993,17 @@ version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe"
[[package]]
name = "lewton"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "777b48df9aaab155475a83a7df3070395ea1ac6902f5cd062b8f2b028075c030"
dependencies = [
"byteorder",
"ogg",
"tinyvec",
]
[[package]]
name = "libc"
version = "0.2.183"
@@ -2927,6 +3086,15 @@ dependencies = [
"twox-hash 2.1.2",
]
[[package]]
name = "mach2"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d640282b302c0bb0a2a8e0233ead9035e3bed871f0b7e81fe4a1ec829765db44"
dependencies = [
"libc",
]
[[package]]
name = "malloc_buf"
version = "0.0.6"
@@ -3047,7 +3215,7 @@ dependencies = [
"indexmap",
"log",
"pp-rs",
"rustc-hash",
"rustc-hash 1.1.0",
"spirv",
"termcolor",
"thiserror 1.0.69",
@@ -3068,12 +3236,26 @@ dependencies = [
"once_cell",
"regex",
"regex-syntax",
"rustc-hash",
"rustc-hash 1.1.0",
"thiserror 1.0.69",
"tracing",
"unicode-ident",
]
[[package]]
name = "ndk"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2076a31b7010b17a38c01907c45b945e8f11495ee4dd588309718901b1f7a5b7"
dependencies = [
"bitflags 2.11.0",
"jni-sys 0.3.1",
"log",
"ndk-sys 0.5.0+25.2.9519653",
"num_enum",
"thiserror 1.0.69",
]
[[package]]
name = "ndk"
version = "0.9.0"
@@ -3169,6 +3351,17 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "num-derive"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed3955f1a9c7c0c15e092f9c887db08b1fc683305fdf6eb6684f22555355e202"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "num-traits"
version = "0.2.19"
@@ -3506,6 +3699,29 @@ dependencies = [
"objc",
]
[[package]]
name = "oboe"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e8b61bebd49e5d43f5f8cc7ee2891c16e0f41ec7954d36bcb6c14c5e0de867fb"
dependencies = [
"jni 0.21.1",
"ndk 0.8.0",
"ndk-context",
"num-derive",
"num-traits",
"oboe-sys",
]
[[package]]
name = "oboe-sys"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6c8bb09a4a2b1d668170cfe0a7d5bc103f8999fb316c98099b6a9939c9f2e79d"
dependencies = [
"cc",
]
[[package]]
name = "offset-allocator"
version = "0.2.0"
@@ -3516,6 +3732,15 @@ dependencies = [
"nonmax",
]
[[package]]
name = "ogg"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6951b4e8bf21c8193da321bcce9c9dd2e13c858fe078bf9054a288b419ae5d6e"
dependencies = [
"byteorder",
]
[[package]]
name = "once_cell"
version = "1.21.4"
@@ -3554,7 +3779,7 @@ version = "0.25.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "36820e9051aca1014ddc75770aab4d68bc1e9e632f0f5627c4086bc216fb583b"
dependencies = [
"ttf-parser",
"ttf-parser 0.25.1",
]
[[package]]
@@ -3901,6 +4126,26 @@ version = "0.6.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "20675572f6f24e9e76ef639bc5552774ed45f1c30e2951e1e99c59888861c539"
[[package]]
name = "rayon"
version = "1.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fb39b166781f92d482534ef4b4b1b2568f42613b53e5b6c160e24cfbfa30926d"
dependencies = [
"either",
"rayon-core",
]
[[package]]
name = "rayon-core"
version = "1.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "22e18b0f0062d30d4230b2e85ff77fdfe4326feb054b9783a3460d8435c8ab91"
dependencies = [
"crossbeam-deque",
"crossbeam-utils",
]
[[package]]
name = "redox_syscall"
version = "0.4.1"
@@ -3986,6 +4231,19 @@ dependencies = [
"windows-sys 0.48.0",
]
[[package]]
name = "rodio"
version = "0.20.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e7ceb6607dd738c99bc8cb28eff249b7cd5c8ec88b9db96c0608c1480d140fb1"
dependencies = [
"claxon",
"cpal",
"hound",
"lewton",
"symphonia",
]
[[package]]
name = "ron"
version = "0.8.1"
@@ -4004,6 +4262,12 @@ version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08d43f7aa6b08d49f382cde6a7982047c3426db949b1424bc4b7ec9ae12c6ce2"
[[package]]
name = "rustc-hash"
version = "2.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6b1e7f9a428571be2dc5bc0505c13fb6bf936822b894ec87abf8a08a4e51742d"
[[package]]
name = "rustc_version"
version = "0.4.1"
@@ -4345,8 +4609,10 @@ dependencies = [
"anyhow",
"clap",
"colored",
"image",
"indicatif",
"sylpheed-formats",
"texpresso",
"tokio",
"tracing",
"tracing-subscriber",
@@ -4360,11 +4626,13 @@ dependencies = [
"binrw",
"flate2",
"futures",
"rayon",
"serde",
"serde_json",
"thiserror 2.0.18",
"tokio",
"tracing",
"ttf-parser 0.24.1",
"xdvdfs",
]
@@ -4372,16 +4640,68 @@ dependencies = [
name = "sylpheed-viewer"
version = "0.1.0"
dependencies = [
"ab_glyph",
"bevy",
"bevy_egui",
"futures",
"image",
"rfd",
"rodio",
"sylpheed-formats",
"thiserror 2.0.18",
"tracing",
"tracing-subscriber",
]
[[package]]
name = "symphonia"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5773a4c030a19d9bfaa090f49746ff35c75dfddfa700df7a5939d5e076a57039"
dependencies = [
"lazy_static",
"symphonia-bundle-mp3",
"symphonia-core",
"symphonia-metadata",
]
[[package]]
name = "symphonia-bundle-mp3"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4872dd6bb56bf5eac799e3e957aa1981086c3e613b27e0ac23b176054f7c57ed"
dependencies = [
"lazy_static",
"log",
"symphonia-core",
"symphonia-metadata",
]
[[package]]
name = "symphonia-core"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ea00cc4f79b7f6bb7ff87eddc065a1066f3a43fe1875979056672c9ef948c2af"
dependencies = [
"arrayvec",
"bitflags 1.3.2",
"bytemuck",
"lazy_static",
"log",
]
[[package]]
name = "symphonia-metadata"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "36306ff42b9ffe6e5afc99d49e121e0bd62fe79b9db7b9681d48e29fa19e6b16"
dependencies = [
"encoding_rs",
"lazy_static",
"log",
"symphonia-core",
]
[[package]]
name = "syn"
version = "1.0.109"
@@ -4437,6 +4757,15 @@ dependencies = [
"winapi-util",
]
[[package]]
name = "texpresso"
version = "2.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8550677e2259d675a7841cb1403db35f330cc9e58674c8c5caa12dd12c51dc71"
dependencies = [
"libm",
]
[[package]]
name = "thiserror"
version = "1.0.69"
@@ -4683,7 +5012,7 @@ version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "528bdd1f0e27b5dd9a4ededf154e824b0532731e4af73bb531de46276e0aab1e"
dependencies = [
"bindgen",
"bindgen 0.70.1",
"cc",
"cfg-if",
"once_cell",
@@ -4721,6 +5050,12 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "ttf-parser"
version = "0.24.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5be21190ff5d38e8b4a2d3b6a3ae57f612cc39c96e83cedeaf7abc338a8bac4a"
[[package]]
name = "ttf-parser"
version = "0.25.1"
@@ -5089,7 +5424,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fe985f41e291eecef5e5c0770a18d28390addb03331c043964d9e916453d6f16"
dependencies = [
"core-foundation 0.10.1",
"jni",
"jni 0.22.4",
"log",
"ndk-context",
"objc2 0.6.4",
@@ -5147,7 +5482,7 @@ dependencies = [
"parking_lot",
"profiling",
"raw-window-handle",
"rustc-hash",
"rustc-hash 1.1.0",
"smallvec",
"thiserror 1.0.69",
"wgpu-hal",
@@ -5189,14 +5524,14 @@ dependencies = [
"range-alloc",
"raw-window-handle",
"renderdoc-sys",
"rustc-hash",
"rustc-hash 1.1.0",
"smallvec",
"thiserror 1.0.69",
"wasm-bindgen",
"web-sys",
"wgpu-types",
"windows",
"windows-core",
"windows 0.58.0",
"windows-core 0.58.0",
]
[[package]]
@@ -5219,13 +5554,33 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "windows"
version = "0.54.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9252e5725dbed82865af151df558e754e4a3c2c30818359eb17465f1346a1b49"
dependencies = [
"windows-core 0.54.0",
"windows-targets 0.52.6",
]
[[package]]
name = "windows"
version = "0.58.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dd04d41d93c4992d421894c18c8b43496aa748dd4c081bac0dc93eb0489272b6"
dependencies = [
"windows-core",
"windows-core 0.58.0",
"windows-targets 0.52.6",
]
[[package]]
name = "windows-core"
version = "0.54.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "12661b9c89351d684a50a8a643ce5f608e20243b9fb84687800163429f161d65"
dependencies = [
"windows-result 0.1.2",
"windows-targets 0.52.6",
]
@@ -5237,7 +5592,7 @@ checksum = "6ba6d44ec8c2591c134257ce647b7ea6b20335bf6379a27dac5f1641fcf59f99"
dependencies = [
"windows-implement",
"windows-interface",
"windows-result",
"windows-result 0.2.0",
"windows-strings",
"windows-targets 0.52.6",
]
@@ -5270,6 +5625,15 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-result"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e383302e8ec8515204254685643de10811af0ed97ea37210dc26fb0032647f8"
dependencies = [
"windows-targets 0.52.6",
]
[[package]]
name = "windows-result"
version = "0.2.0"
@@ -5285,10 +5649,19 @@ version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4cd9b125c486025df0eabcb585e62173c6c9eddcec5d117d3b6e8c30e2ee4d10"
dependencies = [
"windows-result",
"windows-result 0.2.0",
"windows-targets 0.52.6",
]
[[package]]
name = "windows-sys"
version = "0.45.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "75283be5efb2831d37ea142365f009c02ec203cd29a3ebecbc093d52315b66d0"
dependencies = [
"windows-targets 0.42.2",
]
[[package]]
name = "windows-sys"
version = "0.48.0"
@@ -5334,6 +5707,21 @@ dependencies = [
"windows-link",
]
[[package]]
name = "windows-targets"
version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e5180c00cd44c9b1c88adb3693291f1cd93605ded80c250a75d472756b4d071"
dependencies = [
"windows_aarch64_gnullvm 0.42.2",
"windows_aarch64_msvc 0.42.2",
"windows_i686_gnu 0.42.2",
"windows_i686_msvc 0.42.2",
"windows_x86_64_gnu 0.42.2",
"windows_x86_64_gnullvm 0.42.2",
"windows_x86_64_msvc 0.42.2",
]
[[package]]
name = "windows-targets"
version = "0.48.5"
@@ -5382,6 +5770,12 @@ dependencies = [
"windows_x86_64_msvc 0.53.1",
]
[[package]]
name = "windows_aarch64_gnullvm"
version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "597a5118570b68bc08d8d59125332c54f1ba9d9adeedeef5b99b02ba2b0698f8"
[[package]]
name = "windows_aarch64_gnullvm"
version = "0.48.5"
@@ -5400,6 +5794,12 @@ version = "0.53.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a9d8416fa8b42f5c947f8482c43e7d89e73a173cead56d044f6a56104a6d1b53"
[[package]]
name = "windows_aarch64_msvc"
version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e08e8864a60f06ef0d0ff4ba04124db8b0fb3be5776a5cd47641e942e58c4d43"
[[package]]
name = "windows_aarch64_msvc"
version = "0.48.5"
@@ -5418,6 +5818,12 @@ version = "0.53.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9d782e804c2f632e395708e99a94275910eb9100b2114651e04744e9b125006"
[[package]]
name = "windows_i686_gnu"
version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c61d927d8da41da96a81f029489353e68739737d3beca43145c8afec9a31a84f"
[[package]]
name = "windows_i686_gnu"
version = "0.48.5"
@@ -5448,6 +5854,12 @@ version = "0.53.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fa7359d10048f68ab8b09fa71c3daccfb0e9b559aed648a8f95469c27057180c"
[[package]]
name = "windows_i686_msvc"
version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "44d840b6ec649f480a41c8d80f9c65108b92d89345dd94027bfe06ac444d1060"
[[package]]
name = "windows_i686_msvc"
version = "0.48.5"
@@ -5466,6 +5878,12 @@ version = "0.53.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e7ac75179f18232fe9c285163565a57ef8d3c89254a30685b57d83a38d326c2"
[[package]]
name = "windows_x86_64_gnu"
version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8de912b8b8feb55c064867cf047dda097f92d51efad5b491dfb98f6bbb70cb36"
[[package]]
name = "windows_x86_64_gnu"
version = "0.48.5"
@@ -5484,6 +5902,12 @@ version = "0.53.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9c3842cdd74a865a8066ab39c8a7a473c0778a3f29370b5fd6b4b9aa7df4a499"
[[package]]
name = "windows_x86_64_gnullvm"
version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "26d41b46a36d453748aedef1486d5c7a85db22e56aff34643984ea85514e94a3"
[[package]]
name = "windows_x86_64_gnullvm"
version = "0.48.5"
@@ -5502,6 +5926,12 @@ version = "0.53.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0ffa179e2d07eee8ad8f57493436566c7cc30ac536a3379fdf008f47f6bb7ae1"
[[package]]
name = "windows_x86_64_msvc"
version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9aec5da331524158c6d1a4ac0ab1541149c0b9505fde06423b02f5ef0106b9f0"
[[package]]
name = "windows_x86_64_msvc"
version = "0.48.5"
@@ -5542,7 +5972,7 @@ dependencies = [
"js-sys",
"libc",
"memmap2",
"ndk",
"ndk 0.9.0",
"objc2 0.5.2",
"objc2-app-kit 0.2.2",
"objc2-foundation 0.2.2",

3
Cargo.toml
View File

@@ -42,6 +42,9 @@ tracing-subscriber = { version = "0.3", features = ["env-filter"] }
# ── Native file dialog ────────────────────────────────────────────────────────
rfd = "0.14"
# ── Audio playback (native video player) ──────────────────────────────────────
rodio = "0.20"
# ── CLI ───────────────────────────────────────────────────────────────────────
clap = { version = "4", features = ["derive"] }

3
crates/sylpheed-cli/Cargo.toml
View File

@@ -19,3 +19,6 @@ tracing = { workspace = true }
tracing-subscriber = { workspace = true }
colored = "2"
indicatif = "0.17"
# Texture PNG export: BCn software decode + image encode.
texpresso = "2"
image = { version = "0.25", default-features = false, features = ["png"] }

811
crates/sylpheed-cli/src/main.rs
View File

@@ -97,6 +97,12 @@ enum Commands {
#[command(subcommand)]
cmd: PakCommands,
},
/// XBG7 mesh tools (inspect / headless render to PNG)
Mesh {
#[command(subcommand)]
cmd: MeshCommands,
},
}
#[derive(Subcommand)]
@@ -116,6 +122,51 @@ enum PakCommands {
/// Entry name-hash, e.g. `0x7c96296c`
hash: String,
},
/// Decode every T8aD texture in the pak (direct, RATC-nested, and LSTA
/// frames) to PNG — our decoder's output, for A/B against the running game.
Textures {
/// Path to the `.pak` index
pak: PathBuf,
/// Output directory for the PNGs (created if missing)
output: PathBuf,
/// Print per-texture dimensions + tile count.
#[arg(long)]
verbose: bool,
},
}
#[derive(Subcommand)]
enum MeshCommands {
/// Print the decoded sub-models of an XBG7 container (`.xpr`)
Info {
/// Path to the `.xpr` model / stage container
file: PathBuf,
},
/// Headless-render the decoded mesh(es) to a shaded PNG (software rasterizer)
Render {
/// Path to the `.xpr` model / stage container
file: PathBuf,
/// Output PNG path
output: PathBuf,
/// Image size in pixels (square)
#[arg(long, default_value_t = 900)]
size: u32,
/// Camera yaw in degrees
#[arg(long, default_value_t = 35.0)]
yaw: f32,
/// Camera pitch in degrees
#[arg(long, default_value_t = 22.0)]
pitch: f32,
/// Camera distance multiplier (1.0 = framed; <1 zooms in, >1 out)
#[arg(long, default_value_t = 1.0)]
dist: f32,
/// Force the stage grid layout even for single models
#[arg(long)]
row: bool,
/// Only render sub-models whose name contains this substring
#[arg(long)]
only: Option<String>,
},
}
#[derive(Subcommand)]
@@ -155,9 +206,18 @@ async fn main() -> Result<()> {
TextureCommands::Info { file } => cmd_texture_info(&file),
TextureCommands::Export { file, output } => cmd_texture_export(&file, &output),
},
Commands::Mesh { cmd } => match cmd {
MeshCommands::Info { file } => cmd_mesh_info(&file),
MeshCommands::Render { file, output, size, yaw, pitch, dist, row, only } => {
cmd_mesh_render(&file, &output, size, yaw, pitch, dist, row, only)
}
},
Commands::Pak { cmd } => match cmd {
PakCommands::List { pak, idxd_only } => cmd_pak_list(&pak, idxd_only),
PakCommands::Dump { pak, hash } => cmd_pak_dump(&pak, &hash),
PakCommands::List { pak, idxd_only } => cmd_pak_list(&pak, idxd_only),
PakCommands::Dump { pak, hash } => cmd_pak_dump(&pak, &hash),
PakCommands::Textures { pak, output, verbose } => {
cmd_pak_textures(&pak, &output, verbose)
}
},
}
}
@@ -272,6 +332,7 @@ fn cmd_sniff(dir: &Path, unknown_only: bool) -> Result<()> {
"bin" => label.red().to_string(),
"xpr" => label.green().to_string(),
"dds" => label.green().to_string(),
"txt" => label.cyan().to_string(),
_ => label.yellow().to_string(),
};
@@ -330,73 +391,527 @@ fn cmd_texture_export(file: &Path, output: &Path) -> Result<()> {
let bytes = std::fs::read(file)
.with_context(|| format!("Cannot read {}", file.display()))?;
use sylpheed_formats::texture::{X360Texture, X360TextureFormat};
use sylpheed_formats::texture::X360Texture;
let tex = X360Texture::from_xpr2(&bytes)?;
let rgba = decode_to_rgba8(&tex)
.with_context(|| format!("decoding {:?} texture", tex.format))?;
// For BCn compressed textures we need to software-decompress to RGBA8
// before saving to PNG. This uses the `texpresso` crate.
//
// TODO: add `texpresso = "2"` to Cargo.toml for BCn software decode.
// For now, print a helpful message.
match tex.format {
X360TextureFormat::A8R8G8B8 | X360TextureFormat::X8R8G8B8 => {
// Raw BGRA8 — can save directly (with channel swizzle)
println!(
"{} TODO: save raw BGRA8 to PNG (add `image` crate)",
"Note:".yellow()
);
}
compressed_format => {
println!(
"{} Format {:?} needs BCn decompression before PNG export.",
"Note:".yellow(), compressed_format
);
println!(
" Add `texpresso` crate and implement decompression in texture.rs"
);
println!(
" Alternatively, use the Bevy viewer to inspect textures visually."
);
}
}
image::save_buffer(
output,
&rgba,
tex.width,
tex.height,
image::ExtendedColorType::Rgba8,
)
.with_context(|| format!("writing PNG {}", output.display()))?;
println!(
" Texture parsed OK: {}×{} {:?}",
tex.width, tex.height, tex.format
"{} {}×{} {:?}{} → {}",
"Exported".green().bold(),
tex.width,
tex.height,
tex.format,
if tex.is_cubemap { " (cubemap face 0)" } else { "" },
output.display().to_string().cyan(),
);
Ok(())
}
// ── mesh info / render ──────────────────────────────────────────────────────
/// Decode a container and return its sub-models the same way the viewer routes:
/// a single model (weapon / prop) OR a stage's many sub-models — whichever
/// yields more geometry.
fn decode_models(bytes: &[u8]) -> Vec<sylpheed_formats::mesh::Xbg7Model> {
use sylpheed_formats::mesh::{count_xbg7, Xbg7Model};
// Route by container kind, NOT by whichever decoder yields more verts (that
// old heuristic let `stage_models`' content-anchoring win on single-model
// files, fabricating phantom / duplicate / mis-anchored blocks). A file with
// one XBG7 resource is a single model (weapon / prop) → use only the
// validated records-based list decode; content-anchoring a single-model file
// invents geometry. Many XBG7 resources → a Stage_* collection → anchor them.
if count_xbg7(bytes) > 1 {
// Multi-resource Stage_* collection → content-anchor every resource
// (each anchor now gated by stored-normal agreement).
Xbg7Model::stage_models(bytes)
} else if let Some(m) = Xbg7Model::from_xpr2(bytes)
.ok()
.filter(|m| !m.meshes.is_empty())
{
// Single model the records-based list decode carved (authoritative).
vec![m]
} else {
// Single model from_xpr2 couldn't locate (its sequential carve missed
// the block) — fall back to content-anchoring, which finds it by shape.
// Use a STRICT winding-consistency gate (0.85): on a single-model file a
// mis-anchor is an obvious phantom / spike-mess and must be declined,
// unlike the large stage corpus which keeps the ungated path.
Xbg7Model::anchor_models(bytes, 0.85)
}
}
fn cmd_mesh_info(file: &Path) -> Result<()> {
let bytes = std::fs::read(file).with_context(|| format!("Cannot read {}", file.display()))?;
let models = decode_models(&bytes);
if models.is_empty() {
println!("{} no decodable XBG7 geometry", "Mesh:".yellow().bold());
return Ok(());
}
let (mut tv, mut tt) = (0usize, 0usize);
println!("{} {}", "Mesh:".green().bold(), file.display());
for m in &models {
let (v, t) = m.totals();
tv += v;
tt += t;
let mut lo = [f32::MAX; 3];
let mut hi = [f32::MIN; 3];
for sub in &m.meshes {
for p in &sub.positions {
for a in 0..3 {
lo[a] = lo[a].min(p[a]);
hi[a] = hi[a].max(p[a]);
}
}
}
println!(
" {:16} {:>6} v {:>6} t bbox [{:.1} {:.1} {:.1}]",
m.name,
v,
t,
hi[0] - lo[0],
hi[1] - lo[1],
hi[2] - lo[2]
);
// Per-sub-mesh integrity diagnostics: degenerate triangles (a zero-area
// "hole"), vertices referenced by no triangle (dropped geometry), and the
// referenced index range vs the vertex count (short/over reads).
for (si, sub) in m.meshes.iter().enumerate() {
let nv = sub.positions.len();
let mut referenced = vec![false; nv];
let (mut degen, mut oob, mut imax) = (0usize, 0usize, 0u32);
for tri in sub.indices.chunks_exact(3) {
let (a, b, c) = (tri[0], tri[1], tri[2]);
imax = imax.max(a).max(b).max(c);
if a == b || b == c || a == c {
degen += 1;
}
for &i in tri {
if (i as usize) < nv {
referenced[i as usize] = true;
} else {
oob += 1;
}
}
}
let unref = referenced.iter().filter(|&&r| !r).count();
// Spanning triangles: longest edge ≫ the median (strip-junction spikes).
let edge = |a: u32, b: u32| {
let (p, q) = (sub.positions[a as usize], sub.positions[b as usize]);
((p[0] - q[0]).powi(2) + (p[1] - q[1]).powi(2) + (p[2] - q[2]).powi(2)).sqrt()
};
let mut maxedges: Vec<f32> = sub
.indices
.chunks_exact(3)
.map(|t| edge(t[0], t[1]).max(edge(t[1], t[2])).max(edge(t[0], t[2])))
.collect();
maxedges.sort_by(|a, b| a.partial_cmp(b).unwrap());
let median = maxedges.get(maxedges.len() / 2).copied().unwrap_or(1.0).max(1e-6);
let spanning = maxedges.iter().filter(|&&e| e > 6.0 * median).count();
println!(
" sub{si}: {nv} v, {} tris | degenerate {degen}, unref-verts {unref}, spanning>6×med {spanning}, idx_max {imax}/{}{}",
sub.indices.len() / 3,
nv.saturating_sub(1),
if oob > 0 { format!(", OOB {oob}") } else { String::new() },
);
// XDUMPVERT=1 → print the first few vertex positions per sub-mesh, for
// content-matching a decoded sub-mesh against the GPU draw log.
if std::env::var("XDUMPVERT").is_ok() {
let mut lo = [f32::MAX; 3];
let mut hi = [f32::MIN; 3];
for p in &sub.positions {
for a in 0..3 {
lo[a] = lo[a].min(p[a]);
hi[a] = hi[a].max(p[a]);
}
}
println!(
" bbox X[{:.2}..{:.2}] Y[{:.2}..{:.2}] Z[{:.2}..{:.2}] ctr({:.2},{:.2},{:.2})",
lo[0], hi[0], lo[1], hi[1], lo[2], hi[2],
(lo[0]+hi[0])/2.0, (lo[1]+hi[1])/2.0, (lo[2]+hi[2])/2.0
);
}
}
}
println!(
" {} {} sub-models · {} verts · {} tris",
"TOTAL".bold(),
models.len(),
tv,
tt
);
Ok(())
}
#[allow(clippy::too_many_arguments)]
fn cmd_mesh_render(
file: &Path,
output: &Path,
size: u32,
yaw: f32,
pitch: f32,
dist: f32,
force_row: bool,
only: Option<String>,
) -> Result<()> {
let bytes = std::fs::read(file).with_context(|| format!("Cannot read {}", file.display()))?;
let mut models = decode_models(&bytes);
if let Some(sub) = &only {
// Prefer an exact name match (e.g. `f001` for the neutral ship pose,
// excluding the `_rou_f001_mnv*` animation poses that also *contain*
// "f001"); fall back to substring when nothing matches exactly.
if models.iter().any(|m| m.name == *sub) {
models.retain(|m| m.name == *sub);
} else {
models.retain(|m| m.name.contains(sub.as_str()));
}
}
if models.is_empty() {
anyhow::bail!("no decodable XBG7 geometry in {}", file.display());
}
// ── Build a triangle soup. ──
// Single models render centred; multi-model containers (stages) get the
// viewer's normalised **thumbnail grid**: each sub-model recentred and
// uniformly scaled to a fixed cell, so all are equally visible regardless of
// native scale (mirrors `spawn_stage_models`).
let multi = models.len() > 1 || force_row;
// XMIRROR=x|y|z → negate that axis, to test an Xbox(LH)→Bevy(RH) handedness
// flip against reference screenshots.
let mirror: [f32; 3] = match std::env::var("XMIRROR").ok().as_deref() {
Some("x") => [-1.0, 1.0, 1.0],
Some("y") => [1.0, -1.0, 1.0],
Some("z") => [1.0, 1.0, -1.0],
_ => [1.0, 1.0, 1.0],
};
let mut tris: Vec<[[f32; 3]; 3]> = Vec::new();
// XCOLORSUB=1 tints each sub-mesh a distinct colour (to see which sub is
// which part / where the "extra fin" comes from). Parallel to `tris`.
let color_sub = std::env::var("XCOLORSUB").is_ok();
// XONLYSUB=N renders only the N-th global sub-mesh (to isolate one part).
let only_sub: Option<usize> = std::env::var("XONLYSUB").ok().and_then(|s| s.parse().ok());
let mut tints: Vec<[f32; 3]> = Vec::new();
const PALETTE: [[f32; 3]; 8] = [
[1.0, 1.0, 1.0], // sub0 body = white
[1.0, 0.35, 0.35], // sub1 red
[0.35, 1.0, 0.35], // sub2 green
[0.4, 0.55, 1.0], // sub3 blue
[1.0, 0.9, 0.3], // sub4 yellow
[1.0, 0.5, 1.0], // sub5 magenta
[0.3, 1.0, 1.0], // sub6 cyan
[1.0, 0.6, 0.2], // sub7 orange
];
let mut sub_gi = 0usize;
const CELL: f32 = 10.0;
const GAP: f32 = 4.0;
let grid_pitch = CELL + GAP;
let cols = (models.len() as f32).sqrt().ceil().max(1.0) as usize;
for (i, m) in models.iter().enumerate() {
let mut lo = [f32::MAX; 3];
let mut hi = [f32::MIN; 3];
for sub in &m.meshes {
for p in &sub.positions {
for a in 0..3 {
lo[a] = lo[a].min(p[a]);
hi[a] = hi[a].max(p[a]);
}
}
}
if lo[0] > hi[0] {
continue;
}
let center = [
(lo[0] + hi[0]) * 0.5,
(lo[1] + hi[1]) * 0.5,
(lo[2] + hi[2]) * 0.5,
];
let (scale, cell) = if multi {
let extent = (hi[0] - lo[0]).max(hi[1] - lo[1]).max(hi[2] - lo[2]).max(1e-3);
let col = i % cols;
let row = i / cols;
(CELL / extent, [col as f32 * grid_pitch, -(row as f32) * grid_pitch, 0.0])
} else {
(1.0, [0.0, 0.0, 0.0])
};
// XNODEXFORM=1 applies the XBG7 scene-graph node placement (fins move to
// the tail) — to verify the transforms recovered from the graph.
let placements = if std::env::var("XNODEXFORM").is_ok() {
sylpheed_formats::mesh::node_transforms(&bytes, &m.name)
} else {
Vec::new()
};
for (sub_local, sub) in m.meshes.iter().enumerate() {
// Every scene-graph instance that draws this sub-mesh (mirrored fin
// pair, L/R winglets…); `None` = no graph placement → identity.
let mine: Vec<Option<&sylpheed_formats::mesh::NodePlacement>> = {
let v: Vec<_> = placements
.iter()
.filter(|p| p.sub_index == sub_local)
.map(Some)
.collect();
if v.is_empty() {
vec![None]
} else {
v
}
};
// Sub-mesh indices are a triangle list (the decoder has already
// expanded the file's triangle strips).
let n = sub.positions.len();
// XSPANONLY=1 renders ONLY long-edge ("spanning") triangles; XSPANHIDE=1
// renders everything EXCEPT them — to see whether the flagged spanning
// triangles are real geometry or decode artifacts (phantom sheets).
let span_only = std::env::var("XSPANONLY").is_ok();
let span_hide = std::env::var("XSPANHIDE").is_ok();
let med = {
let mut e: Vec<f32> = sub
.indices
.chunks_exact(3)
.filter(|t| (t[0] as usize) < n && (t[1] as usize) < n && (t[2] as usize) < n)
.map(|t| {
let d = |a: u32, b: u32| {
let (p, q) = (sub.positions[a as usize], sub.positions[b as usize]);
((p[0] - q[0]).powi(2) + (p[1] - q[1]).powi(2) + (p[2] - q[2]).powi(2))
.sqrt()
};
d(t[0], t[1]).max(d(t[1], t[2])).max(d(t[0], t[2]))
})
.collect();
e.sort_by(|a, b| a.partial_cmp(b).unwrap());
e.get(e.len() / 2).copied().unwrap_or(1.0).max(1e-6)
};
if let Some(want) = only_sub {
if sub_gi != want {
sub_gi += 1;
continue;
}
}
let tint = if color_sub {
PALETTE[sub_gi % PALETTE.len()]
} else {
[1.0, 1.0, 1.0]
};
for place in &mine {
let f = |i: usize| {
let p = place.map(|pl| pl.apply(sub.positions[i])).unwrap_or(sub.positions[i]);
[
(p[0] - center[0]) * scale * mirror[0] + cell[0],
(p[1] - center[1]) * scale * mirror[1] + cell[1],
(p[2] - center[2]) * scale * mirror[2] + cell[2],
]
};
for tri in sub.indices.chunks_exact(3) {
let (a, b, c) = (tri[0] as usize, tri[1] as usize, tri[2] as usize);
if a < n && b < n && c < n {
if span_only || span_hide {
let d = |i: usize, j: usize| {
let (p, q) = (sub.positions[i], sub.positions[j]);
((p[0] - q[0]).powi(2)
+ (p[1] - q[1]).powi(2)
+ (p[2] - q[2]).powi(2))
.sqrt()
};
let spanning = d(a, b).max(d(b, c)).max(d(a, c)) > 6.0 * med;
if span_only && !spanning {
continue;
}
if span_hide && spanning {
continue;
}
}
tris.push([f(a), f(b), f(c)]);
tints.push(tint);
}
}
}
sub_gi += 1;
}
}
if tris.is_empty() {
anyhow::bail!("no triangles to render");
}
let rgba = rasterize(&tris, &tints, size, yaw, pitch, dist);
image::save_buffer(output, &rgba, size, size, image::ExtendedColorType::Rgba8)
.with_context(|| format!("writing PNG {}", output.display()))?;
println!(
"{} {} tris → {} ({}×{}, yaw {:.0}° pitch {:.0}°)",
"Rendered".green().bold(),
tris.len(),
output.display().to_string().cyan(),
size,
size,
yaw,
pitch,
);
Ok(())
}
/// Minimal software rasterizer: orthographic, z-buffered, two-sided Lambert +
/// headlight shading over a flat grey material on a dark background. Enough to
/// judge whether recovered geometry is coherent.
fn rasterize(
tris: &[[[f32; 3]; 3]],
tints: &[[f32; 3]],
size: u32,
yaw_deg: f32,
pitch_deg: f32,
dist: f32,
) -> Vec<u8> {
let n = size as usize;
let (yaw, pitch) = (yaw_deg.to_radians(), pitch_deg.to_radians());
let (cy, sy) = (yaw.cos(), yaw.sin());
let (cp, sp) = (pitch.cos(), pitch.sin());
// Rotate a world point into view space (yaw about Y, then pitch about X).
let view = |p: [f32; 3]| -> [f32; 3] {
let x = p[0] * cy + p[2] * sy;
let z0 = -p[0] * sy + p[2] * cy;
let y = p[1] * cp - z0 * sp;
let z = p[1] * sp + z0 * cp;
[x, y, z]
};
// View-space bbox → orthographic fit.
let mut lo = [f32::MAX; 3];
let mut hi = [f32::MIN; 3];
for t in tris {
for v in t {
let q = view(*v);
for a in 0..3 {
lo[a] = lo[a].min(q[a]);
hi[a] = hi[a].max(q[a]);
}
}
}
let span = (hi[0] - lo[0]).max(hi[1] - lo[1]).max(1e-3);
let scale = (n as f32) * 0.9 / (span * dist.max(1e-3));
let cx = (lo[0] + hi[0]) * 0.5;
let cyv = (lo[1] + hi[1]) * 0.5;
let to_screen = |q: [f32; 3]| -> (f32, f32, f32) {
let sx = (q[0] - cx) * scale + n as f32 * 0.5;
let sy = n as f32 * 0.5 - (q[1] - cyv) * scale;
(sx, sy, q[2])
};
let mut color = vec![18u8; n * n * 4];
for i in 0..n * n {
color[i * 4 + 3] = 255;
}
let mut depth = vec![f32::MAX; n * n];
// Light in view space (upper-left-front).
let light = {
let l = [-0.4f32, 0.6, 0.7];
let m = (l[0] * l[0] + l[1] * l[1] + l[2] * l[2]).sqrt();
[l[0] / m, l[1] / m, l[2] / m]
};
for (ti, t) in tris.iter().enumerate() {
let tint = tints.get(ti).copied().unwrap_or([1.0, 1.0, 1.0]);
let v0 = view(t[0]);
let v1 = view(t[1]);
let v2 = view(t[2]);
// Face normal in view space.
let e1 = [v1[0] - v0[0], v1[1] - v0[1], v1[2] - v0[2]];
let e2 = [v2[0] - v0[0], v2[1] - v0[1], v2[2] - v0[2]];
let mut nrm = [
e1[1] * e2[2] - e1[2] * e2[1],
e1[2] * e2[0] - e1[0] * e2[2],
e1[0] * e2[1] - e1[1] * e2[0],
];
let nl = (nrm[0] * nrm[0] + nrm[1] * nrm[1] + nrm[2] * nrm[2]).sqrt();
if nl < 1e-12 {
continue;
}
nrm = [nrm[0] / nl, nrm[1] / nl, nrm[2] / nl];
// Two-sided: diffuse from |n·L|, plus a headlight term from |n.z|.
let diff = (nrm[0] * light[0] + nrm[1] * light[1] + nrm[2] * light[2]).abs();
let head = nrm[2].abs();
let inten = (0.18 + 0.55 * diff + 0.3 * head).min(1.0);
let shade = (inten * 210.0) as u8;
let (ax, ay, az) = to_screen(v0);
let (bx, by, bz) = to_screen(v1);
let (ccx, ccy, ccz) = to_screen(v2);
let minx = ax.min(bx).min(ccx).floor().max(0.0) as usize;
let maxx = ax.max(bx).max(ccx).ceil().min(n as f32 - 1.0) as usize;
let miny = ay.min(by).min(ccy).floor().max(0.0) as usize;
let maxy = ay.max(by).max(ccy).ceil().min(n as f32 - 1.0) as usize;
let area = (bx - ax) * (ccy - ay) - (by - ay) * (ccx - ax);
if area.abs() < 1e-6 {
continue;
}
for py in miny..=maxy {
for px in minx..=maxx {
let fx = px as f32 + 0.5;
let fy = py as f32 + 0.5;
let w0 = ((bx - fx) * (ccy - fy) - (by - fy) * (ccx - fx)) / area;
let w1 = ((ccx - fx) * (ay - fy) - (ccy - fy) * (ax - fx)) / area;
let w2 = 1.0 - w0 - w1;
if w0 < 0.0 || w1 < 0.0 || w2 < 0.0 {
continue;
}
let z = w0 * az + w1 * bz + w2 * ccz;
let idx = py * n + px;
if z < depth[idx] {
depth[idx] = z;
color[idx * 4] = (shade as f32 * tint[0]).min(255.0) as u8;
color[idx * 4 + 1] = (shade as f32 * tint[1]).min(255.0) as u8;
color[idx * 4 + 2] = (shade as f32 * tint[2] * 1.02).min(255.0) as u8;
}
}
}
}
color
}
/// Software-decode a de-tiled `X360Texture` (mip 0) to tightly-packed RGBA8.
///
/// BCn blocks are decompressed with `texpresso`; uncompressed A8R8G8B8/X8R8G8B8
/// is byte-swizzled from the Xenos in-memory BGRA order.
fn decode_to_rgba8(tex: &sylpheed_formats::texture::X360Texture) -> Result<Vec<u8>> {
use sylpheed_formats::texture::X360TextureFormat as F;
let (w, h) = (tex.width as usize, tex.height as usize);
let mut rgba = vec![0u8; w * h * 4];
let bc = |fmt: texpresso::Format, rgba: &mut [u8]| {
fmt.decompress(&tex.data, w, h, rgba);
};
match tex.format {
F::Dxt1 => bc(texpresso::Format::Bc1, &mut rgba),
F::Dxt3 => bc(texpresso::Format::Bc2, &mut rgba),
F::Dxt5 => bc(texpresso::Format::Bc3, &mut rgba),
F::A8R8G8B8 | F::X8R8G8B8 => {
// After the k8in32 endian swap in from_xpr2, k_8_8_8_8 pixels are in
// [A,R,G,B] byte order (verified against the retail Acheron backdrop).
// Emit RGBA. X8 has no meaningful alpha.
let opaque = matches!(tex.format, F::X8R8G8B8);
for (px, out) in tex.data.chunks_exact(4).zip(rgba.chunks_exact_mut(4)) {
out[0] = px[1]; // R
out[1] = px[2]; // G
out[2] = px[3]; // B
out[3] = if opaque { 0xFF } else { px[0] };
}
}
other => {
anyhow::bail!("PNG export for {other:?} (BC4/BC5) not implemented yet");
}
}
Ok(rgba)
}
// ── pak list ────────────────────────────────────────────────────────────────
/// Best-effort human label for a decompressed entry's inner format.
fn inner_label(payload: &[u8]) -> String {
if payload.len() < 4 {
return "empty".into();
}
let m = &payload[0..4];
if m == b"IDXD" {
"IDXD".into()
} else if payload.starts_with(b"<?xml") {
"xml".into()
} else if m.iter().all(|&b| b.is_ascii_graphic()) {
// Many inner formats are 4-char tags (RATC, T8aD, IXUD, LSTA, ttcf, …).
String::from_utf8_lossy(m).into_owned()
} else {
format!("{:02X}{:02X}{:02X}{:02X}", m[0], m[1], m[2], m[3])
}
}
/// For an IDXD entry, a short identity string (ID / Name / Model if present).
fn idxd_identity(obj: &IdxdObject) -> String {
for key in ["ID", "Name", "Model"] {
if let Some(v) = obj.get_raw(key) {
return format!("{key}={v}");
}
}
format!("schema {:08x}", obj.schema_hash)
}
use sylpheed_formats::pak::inner_format_label as inner_label;
/// Try to recover an IDXD entry's original TOC path from its identity tokens.
/// Uses the entry's ID/Name/Model fields plus identifier-like pool tokens as
@@ -443,7 +958,7 @@ fn cmd_pak_list(pak: &Path, idxd_only: bool) -> Result<()> {
}
let (detail, name) = if is_idxd {
match IdxdObject::parse(&payload) {
Ok(o) => (idxd_identity(&o), idxd_toc_name(&o, e.name_hash)),
Ok(o) => (o.identity(), idxd_toc_name(&o, e.name_hash)),
Err(_) => (String::new(), None),
}
} else {
@@ -524,3 +1039,179 @@ fn cmd_pak_dump(pak: &Path, hash_str: &str) -> Result<()> {
}
Ok(())
}
// ── pak textures ─────────────────────────────────────────────────────────────
/// Turn a child name into a filesystem-safe fragment.
fn safe_name(s: &str) -> String {
s.chars()
.map(|c| {
if c.is_ascii_alphanumeric() || matches!(c, '.' | '_' | '-') {
c
} else {
'_'
}
})
.collect()
}
#[derive(Default)]
struct TexStats {
written: usize,
skipped: usize,
}
fn be32_at(b: &[u8], off: usize) -> u32 {
u32::from_be_bytes([b[off], b[off + 1], b[off + 2], b[off + 3]])
}
/// Decode one T8aD slice (whose first bytes are the magic) to PNG. With
/// `verbose`, prints its dimensions + tile count; the `XDUMPHDR` env var dumps
/// the raw header (base + offset table) for format RE.
fn emit_t8ad(
slice: &[u8],
hash: u32,
stem: &str,
output: &Path,
verbose: bool,
stats: &mut TexStats,
) -> Result<()> {
use sylpheed_formats::t8ad;
if !t8ad::is_t8ad(slice) || slice.len() < 0x40 {
return Ok(());
}
let (w, h, tiles) = (
be32_at(slice, 0x14),
be32_at(slice, 0x18),
be32_at(slice, 0x1c),
);
// Debug: dump the header — 44-byte base + the `tiles`-entry u32 offset table.
if std::env::var("XDUMPHDR").is_ok() {
println!("\n{stem} {w}x{h} tiles={tiles}");
print!(" base[0x00..0x2c]:");
for i in (0..44).step_by(4) {
print!(" {:08x}", be32_at(slice, i));
}
print!("\n offsets:");
for t in 0..(tiles as usize).min(64) {
if 0x2c + t * 4 + 4 <= slice.len() {
print!(" {}", be32_at(slice, 0x2c + t * 4));
}
}
println!();
return Ok(());
}
if verbose {
println!(" {hash:08x} {stem:<34} {w:>4}x{h:<4} tiles {tiles}");
}
match t8ad::parse(slice) {
Some(img) => {
let out =
output.join(format!("{hash:08x}_{stem}_{}x{}.png", img.width, img.height));
image::save_buffer(
&out,
&img.rgba,
img.width,
img.height,
image::ExtendedColorType::Rgba8,
)
.with_context(|| format!("writing PNG {}", out.display()))?;
stats.written += 1;
}
None => stats.skipped += 1,
}
Ok(())
}
/// Decode every T8aD in a pak (direct entries, RATC-nested children, LSTA
/// frames) to PNG — our decoder's exact output — for A/B against the game.
fn cmd_pak_textures(pak: &Path, output: &Path, verbose: bool) -> Result<()> {
use sylpheed_formats::{lsta, ratc, t8ad};
let arc = PakArchive::open(pak).with_context(|| format!("opening {}", pak.display()))?;
std::fs::create_dir_all(output)
.with_context(|| format!("creating {}", output.display()))?;
println!(
"{} {} → {}",
"Textures".green().bold(),
pak.display().to_string().cyan(),
output.display().to_string().cyan(),
);
let mut stats = TexStats::default();
for e in arc.entries() {
let payload = match arc.read(e) {
Ok(p) => p,
Err(_) => continue,
};
let hash = e.name_hash;
// Direct T8aD entry.
if t8ad::is_t8ad(&payload) {
emit_t8ad(&payload, hash, "direct", output, verbose, &mut stats)?;
continue;
}
// LSTA sprite list = N inline T8aD frames (walk by magic, emit each).
if lsta::is_lsta(&payload) {
let mut off = 0usize;
let mut idx = 0usize;
while let Some(pos) = payload[off..]
.windows(4)
.position(|w| w == &t8ad::T8AD_MAGIC)
{
let start = off + pos;
let next = payload[start + 4..]
.windows(4)
.position(|w| w == &t8ad::T8AD_MAGIC)
.map(|p| start + 4 + p)
.unwrap_or(payload.len());
emit_t8ad(
&payload[start..next],
hash,
&format!("lsta{idx:03}"),
output,
verbose,
&mut stats,
)?;
idx += 1;
off = next;
}
continue;
}
// RATC bundle: decode its T8aD children (named, e.g. `foo.t32`).
if ratc::is_ratc(&payload) {
if let Some(children) = ratc::parse(&payload) {
for (i, child) in children.iter().enumerate() {
if child.kind != "T8aD" {
continue;
}
let end = (child.offset + child.size).min(payload.len());
if child.offset >= end {
continue;
}
let stem = if child.name.is_empty() {
format!("child{i:03}")
} else {
safe_name(&child.name)
};
emit_t8ad(&payload[child.offset..end], hash, &stem, output, verbose, &mut stats)?;
}
}
continue;
}
}
println!(
"\n {} PNG(s) written, {} undecodable (non-tilecount variants — likely DXT)",
stats.written.to_string().green(),
stats.skipped.to_string().yellow(),
);
Ok(())
}

7
crates/sylpheed-formats/Cargo.toml
View File

@@ -10,6 +10,7 @@ authors.workspace = true
xdvdfs = { workspace = true }
binrw = { workspace = true }
flate2 = "1" # zlib/DEFLATE for IPFB "Z1" entries (miniz_oxide backend, WASM-safe)
ttf-parser = { version = "0.24", default-features = false, features = ["std", "opentype-layout"] } # font metadata (OTF/TTF/ttcf), WASM-safe
tokio = { workspace = true }
futures = { workspace = true }
thiserror = { workspace = true }
@@ -18,5 +19,11 @@ tracing = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
# Native-only: parallelise the per-resource XBG7 stage anchoring (hundreds of
# independent sub-models per container). rayon needs OS threads, so it is not
# pulled in for the wasm32 build, which falls back to a sequential decode.
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
rayon = "1"
[dev-dependencies]
tokio = { workspace = true }

9
crates/sylpheed-formats/examples/arsenal.rs Normal file
View File

@@ -0,0 +1,9 @@
use sylpheed_formats::{game_data as gd, PakArchive};
fn main(){
let disc=std::env::var("SYLPHEED_DISC").unwrap();
let pak=PakArchive::open(format!("{disc}/dat/GP_HANGAR_ARSENAL.pak")).unwrap();
let a=gd::load_arsenal(&pak);
for (hp,list) in [("NOSE",&a.nose),("ARM1",&a.arm1),("ARM2",&a.arm2),("ARM3",&a.arm3)]{
println!("{hp} ({}): {:?}", list.len(), list.iter().take(8).collect::<Vec<_>>());
}
}

19
crates/sylpheed-formats/examples/battle.rs Normal file
View File

@@ -0,0 +1,19 @@
use sylpheed_formats::{game_data as gd, PakArchive};
fn main(){
let disc=std::env::var("SYLPHEED_DISC").unwrap();
let pak=PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).unwrap();
let units=gd::load_units(&pak); let vessels=gd::load_vessels(&pak);
let hp=|id:&str|->String{
units.iter().find(|u|u.id.as_deref()==Some(id)).and_then(|u|u.hp).map(|h|format!("{h:.0}hp"))
.or_else(||vessels.iter().find(|v|v.id.as_deref()==Some(id)).and_then(|v|v.hp).map(|h|format!("{h:.0}HP")))
.unwrap_or("·".into())
};
let rosters=gd::load_unit_rosters(&pak);
for r in rosters.iter().filter(|r|r.stage.is_some()).take(4){
println!("\n{} — {} combatants:", r.stage.as_deref().unwrap(), r.units.len());
for u in r.units.iter().filter(|u|u.contains("ADAN")).take(6){
let short=u.trim_start_matches("UN_").split('_').skip(1).collect::<Vec<_>>().join("_");
println!(" {short:32} {}", hp(u));
}
}
}

24
crates/sylpheed-formats/examples/campaign.rs Normal file
View File

@@ -0,0 +1,24 @@
use sylpheed_formats::{game_data, localization::TextIndex, PakArchive};
fn main(){
let disc=std::env::var("SYLPHEED_DISC").unwrap();
let pak=PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).unwrap();
let text=TextIndex::build(&pak);
let mut stages=game_data::load_stages(&pak);
stages.retain(|s|s.id.starts_with('S') && s.id.len()==3 && s.id[1..].parse::<u32>().map(|n|n<=16).unwrap_or(false));
stages.sort_by(|a,b|a.id.cmp(&b.id));
println!("═══ CAMPAIGN (S01–S16) ═══");
for s in &stages{
let obj=text.objectives(&s.id,1);
println!("\n{} · {}", s.id, s.location.as_deref().unwrap_or("?"));
for o in obj.iter().take(2){ println!(" ▸ {o}"); }
}
// roster with real names
let mut chars=game_data::load_characters(&pak);
chars.retain(|c|c.faction.as_deref()==Some("TCAF") && c.unique==Some(true) && c.faces.len()>=4);
println!("\n═══ PRINCIPAL CAST (TCAF, named) ═══");
for c in &chars{
let id=c.id.as_deref().unwrap_or("");
let name=text.character_name(id.trim_start_matches("Character")).or_else(||c.name_key.as_deref().and_then(|k|text.get(k))).unwrap_or("?");
println!(" {name:12} ({} portraits) [{}]", c.faces.len(), id.trim_start_matches("Character"));
}
}

207
crates/sylpheed-formats/examples/correlate_capture.rs Normal file
View File

@@ -0,0 +1,207 @@
//! Recover exact capital-ship part placement from a Canary `xenia_ship_capture.log`
//! (F10 snapshot: per-draw guest vertex-buffer + up to 48 vertex-shader float
//! constants).
//!
//! FINDING (2026-07-24): the captured vertex BUFFER holds LOCAL coordinates —
//! byte-identical to the `.xpr` — so capital-ship parts are placed entirely in the
//! **vertex shader**. The per-part transform is in the VS constants: `c0..c2` are
//! the **WorldViewProjection** matrix rows. Their norms are `(sx, sy, 1)` = the
//! projection x/y scales (sy/sx = 16:9 aspect); dividing each row by its norm
//! yields the rigid **WorldView** (verified: rows orthonormal, det +1). The camera
//! View cancels when we express every part relative to a reference part:
//! rel_p = WV_ref⁻¹ · WV_p (a pure ship-space rigid transform)
//! Applying `rel_p` to part `p`'s local geometry assembles the ship exactly, in the
//! reference part's frame — ground truth, no static guessing.
//!
//! Usage:
//! SYLPHEED_ISO=... cargo run --release --example correlate_capture -- \
//! <capture.log> <Stage_SNN> <ship_id> [ref_part_substr]
//! e.g. `... /tmp/xenia_ship_capture.log Stage_S01 e106 bdy_04`
use sylpheed_formats::mesh::{xbg7_resource_names, Xbg7Model};
use sylpheed_formats::ship::{is_base_part, ship_id_of};
use sylpheed_formats::xiso::open_iso;
use std::collections::HashSet;
use std::path::Path;
type M3 = [[f64; 3]; 3];
struct Draw {
vbase: u32,
vcount: u32,
/// Rigid WorldView: R (rows) + T (view-space translation).
r: M3,
t: [f64; 3],
ok: bool,
}
fn parse(text: &str) -> Vec<Draw> {
let mut out = Vec::new();
let mut vbase = 0u32;
let mut vcount = 0u32;
let mut consts: Vec<(usize, [f64; 4])> = Vec::new();
let mut flush = |vbase: u32, vcount: u32, consts: &[(usize, [f64; 4])], out: &mut Vec<Draw>| {
if vbase == 0 {
return;
}
// c0..c2 = WVP rows; normalise each to unit → rigid WorldView.
let get = |i: usize| consts.iter().find(|(k, _)| *k == i).map(|(_, v)| *v);
let (Some(c0), Some(c1), Some(c2)) = (get(0), get(1), get(2)) else {
out.push(Draw { vbase, vcount, r: [[0.0; 3]; 3], t: [0.0; 3], ok: false });
return;
};
let rows = [c0, c1, c2];
let mut r = [[0.0; 3]; 3];
let mut t = [0.0; 3];
let mut ok = true;
for (i, row) in rows.iter().enumerate() {
let n = (row[0] * row[0] + row[1] * row[1] + row[2] * row[2]).sqrt();
if n < 1e-6 {
ok = false;
break;
}
r[i] = [row[0] / n, row[1] / n, row[2] / n];
t[i] = row[3] / n;
}
out.push(Draw { vbase, vcount, r, t, ok });
};
for line in text.lines() {
let l = line.trim();
if let Some(rest) = l.strip_prefix("DRAW ") {
flush(vbase, vcount, &consts, &mut out);
consts.clear();
let f = |k: &str| rest.split_whitespace().find_map(|t| t.strip_prefix(k));
vbase = f("vbase=0x").and_then(|s| u32::from_str_radix(s, 16).ok()).unwrap_or(0);
vcount = f("vcount=").and_then(|s| s.parse().ok()).unwrap_or(0);
} else if l.starts_with("vsconst") {
for cap in l.split("c").skip(1) {
// "<i>=(x,y,z,w) ..."
if let Some((idx, rest)) = cap.split_once('=') {
if let Ok(i) = idx.trim().parse::<usize>() {
let nums: Vec<f64> = rest
.trim_start_matches('(')
.split(')')
.next()
.unwrap_or("")
.split(',')
.filter_map(|x| x.trim().parse().ok())
.collect();
if nums.len() == 4 {
consts.push((i, [nums[0], nums[1], nums[2], nums[3]]));
}
}
}
}
}
}
flush(vbase, vcount, &consts, &mut out);
out
}
fn mt_apply(r: &M3, t: &[f64; 3], v: [f64; 3]) -> [f64; 3] {
[
r[0][0] * v[0] + r[0][1] * v[1] + r[0][2] * v[2] + t[0],
r[1][0] * v[0] + r[1][1] * v[1] + r[1][2] * v[2] + t[1],
r[2][0] * v[0] + r[2][1] * v[1] + r[2][2] * v[2] + t[2],
]
}
fn transpose(m: &M3) -> M3 {
[
[m[0][0], m[1][0], m[2][0]],
[m[0][1], m[1][1], m[2][1]],
[m[0][2], m[1][2], m[2][2]],
]
}
fn mmul(a: &M3, b: &M3) -> M3 {
let mut o = [[0.0; 3]; 3];
for i in 0..3 {
for j in 0..3 {
o[i][j] = (0..3).map(|k| a[i][k] * b[k][j]).sum();
}
}
o
}
fn main() {
let args: Vec<String> = std::env::args().collect();
if args.len() < 4 {
eprintln!("usage: correlate_capture <capture.log> <Stage_SNN> <ship_id> [ref_part_substr]");
std::process::exit(2);
}
let (log, stage, id) = (&args[1], &args[2], &args[3]);
let ref_sub = args.get(4).map(|s| s.as_str()).unwrap_or("bdy_04");
let iso = std::env::var("SYLPHEED_ISO").expect("SYLPHEED_ISO");
let draws = parse(&std::fs::read_to_string(log).expect("read log"));
eprintln!("parsed {} draws ({} with WorldView)", draws.len(), draws.iter().filter(|d| d.ok).count());
let bytes = {
let rt = tokio::runtime::Builder::new_current_thread().enable_all().build().unwrap();
rt.block_on(async {
let mut r = open_iso(Path::new(&iso)).await.unwrap();
r.read_file(&format!("hidden/resource3d/{stage}.xpr")).await.unwrap()
})
};
let names = xbg7_resource_names(&bytes);
let parts: Vec<String> =
names.iter().filter(|n| is_base_part(n) && ship_id_of(n) == Some(id.as_str())).cloned().collect();
let want: HashSet<String> = parts.iter().cloned().collect();
let models = Xbg7Model::models_named(&bytes, &want, &|| false);
// Match each part to a captured draw by vertex count; keep its WorldView.
struct Placed {
part: String,
r: M3,
t: [f64; 3],
local: Vec<[f64; 3]>,
}
let mut placed: Vec<Placed> = Vec::new();
let mut used: HashSet<u32> = HashSet::new();
for part in &parts {
let Some(m) = models.iter().find(|m| &m.name == part) else { continue };
let local: Vec<[f64; 3]> =
m.meshes.iter().flat_map(|s| s.positions.iter()).map(|p| [p[0] as f64, p[1] as f64, p[2] as f64]).collect();
let vc = local.len() as u32;
if let Some(d) = draws.iter().find(|d| d.ok && d.vcount == vc && !used.contains(&d.vbase)) {
used.insert(d.vbase);
placed.push(Placed { part: part.clone(), r: d.r, t: d.t, local });
} else {
println!("{part:20} (no matching captured draw — occluded/culled?)");
}
}
// Reference part → ship frame. rel_p = WV_ref⁻¹ · WV_p (camera cancels).
let Some(rf) = placed.iter().find(|p| p.part.contains(ref_sub)).or(placed.first()) else {
eprintln!("no parts placed");
return;
};
let rt_ref = transpose(&rf.r);
let ref_t = rf.t;
println!("\nreference = {} → ship-relative placement (M rows, T):", rf.part);
let mut lo = [f64::MAX; 3];
let mut hi = [f64::MIN; 3];
for p in &placed {
// rel R = Rᵀ_ref · R_p ; rel T = Rᵀ_ref · (T_p − T_ref)
let rel_r = mmul(&rt_ref, &p.r);
let dt = [p.t[0] - ref_t[0], p.t[1] - ref_t[1], p.t[2] - ref_t[2]];
let rel_t = mt_apply(&rt_ref, &[0.0; 3], dt);
// Assembled centroid (validation) + overall extent.
let mut c = [0.0; 3];
for v in &p.local {
let w = mt_apply(&rel_r, &rel_t, *v);
for k in 0..3 {
c[k] += w[k];
lo[k] = lo[k].min(w[k]);
hi[k] = hi[k].max(w[k]);
}
}
let n = p.local.len().max(1) as f64;
println!(
" {:18} T=[{:8.1}{:8.1}{:8.1}] centroid=[{:8.1}{:8.1}{:8.1}]",
p.part, rel_t[0], rel_t[1], rel_t[2], c[0] / n, c[1] / n, c[2] / n
);
}
println!(
"\nassembled extent = [{:.0} {:.0} {:.0}] ({} parts placed)",
hi[0] - lo[0], hi[1] - lo[1], hi[2] - lo[2], placed.len()
);
}

59
crates/sylpheed-formats/examples/deep_map.rs Normal file
View File

@@ -0,0 +1,59 @@
use sylpheed_formats::{ratc, idxd::IdxdObject, PakArchive};
use std::collections::BTreeMap;
fn main(){
let disc=std::env::var("SYLPHEED_DISC").unwrap();
// 1. RATC child-type census across the big RATC paks
let mut childtypes:BTreeMap<String,u64>=BTreeMap::new();
let mut ratc_unparsed=0u64;
for pk in ["GP_READY_ROOM","GP_MOVIE_THEATER","GP_DIALOG","GP_DEBRIEFING_PILOTLOG","GP_TITLE","GP_BUNK"]{
let Ok(arc)=PakArchive::open(format!("{disc}/dat/{pk}.pak")) else{continue};
for e in arc.entries(){
let Ok(b)=arc.read(e) else{continue};
if !ratc::is_ratc(&b){continue;}
match ratc::parse(&b){
Some(kids)=> for k in kids{
let ext=k.name.rsplit('.').next().unwrap_or("?").to_lowercase();
*childtypes.entry(ext).or_default()+=1;
},
None=>ratc_unparsed+=1,
}
}
}
println!("=== RATC child-type census (big RATC paks) ===");
let mut cv:Vec<_>=childtypes.into_iter().collect(); cv.sort_by_key(|x|std::cmp::Reverse(x.1));
for (e,c) in &cv{ println!(" .{e:8} ×{c}"); }
println!(" (RATC bundles that failed to parse: {ratc_unparsed})");
// 2. The 00000002 mystery format (GP_MAIN_GAME_E)
let arc=PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).unwrap();
for e in arc.entries(){
let Ok(b)=arc.read(e) else{continue};
if b.len()>=4 && b[0..4]==[0,0,0,2]{
let hx:String=b[..48.min(b.len())].iter().map(|x|format!("{x:02x}")).collect::<Vec<_>>().join(" ");
let asc:String=b[..64.min(b.len())].iter().map(|&x|if(0x20..0x7f).contains(&x){x as char}else{'.'}).collect();
println!("\n=== 00000002 format sample ({}B) ===\n{hx}\n{asc}",b.len());
break;
}
}
// 3. Sample the top undecoded IDXD schemas: first tokens (guess semantics)
println!("\n=== top undecoded IDXD schemas — sample tokens (semantic hints) ===");
let targets:[u32;6]=[0xb412e6d8,0x026379ab,0x43faa517,0x3c5b0549,0x6ab4825a,0x0426e81d];
for want in targets{
let mut shown=false;
for e in arc.entries(){
if shown{break;}
let Ok(b)=arc.read(e) else{continue};
if b.len()<12 || &b[0..4]!=b"IDXD"{continue;}
let s=u32::from_be_bytes([b[8],b[9],b[10],b[11]]);
if s!=want{continue;}
if let Ok(o)=IdxdObject::parse(&b){
let t=o.tokens();
let sample:Vec<String>=t.iter().take(14).map(|s|{let s=s.chars().take(18).collect::<String>();s}).collect();
println!(" {want:08x}: {sample:?}");
shown=true;
}
}
if !shown{ println!(" {want:08x}: (parse failed / not found)"); }
}
}

13
crates/sylpheed-formats/examples/dialogue.rs Normal file
View File

@@ -0,0 +1,13 @@
use sylpheed_formats::{game_data, localization::TextIndex, PakArchive};
fn main(){
let disc=std::env::var("SYLPHEED_DISC").unwrap();
let pak=PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).unwrap();
let text=TextIndex::build(&pak);
let msgs=game_data::load_demo_messages(&pak);
println!("{} dialogue lines total\n", msgs.len());
for m in msgs.iter().filter(|m|m.character.is_some()&&!m.page_keys.is_empty()).take(8){
let who=m.character.as_deref().unwrap_or("?").trim_start_matches("Character");
let line:String=m.page_keys.iter().filter_map(|k|text.get(k)).collect::<Vec<_>>().join(" ");
println!(" {who:10} [{}] “{}”", m.voice_clip.as_deref().unwrap_or("-"), line.chars().take(64).collect::<String>());
}
}

29
crates/sylpheed-formats/examples/dm_extract.rs Normal file
View File

@@ -0,0 +1,29 @@
use sylpheed_formats::{idxd::IdxdObject, PakArchive};
use std::collections::BTreeMap;
fn main(){
let disc=std::env::var("SYLPHEED_DISC").unwrap();
let arc=PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).unwrap();
let tables:[(u32,&str);4]=[(0x0426e81d,"Player / physics+scoring"),(0x6ab4825a,"Weapon"),(0x43faa517,"Unit / craft"),(0x3c5b0549,"Vessel / capital ship")];
for (want,label) in tables{
// collect all records of this schema
let mut recs:Vec<IdxdObject>=vec![];
for e in arc.entries(){ let Ok(b)=arc.read(e) else{continue}; let Ok(o)=IdxdObject::parse(&b) else{continue}; if o.schema_hash==want{recs.push(o);} }
// field-union (explicit-valued keys), with occurrence count
let mut cols:BTreeMap<String,u32>=BTreeMap::new();
for o in &recs{ for (k,_) in o.resolved_fields(){ *cols.entry(k.into()).or_default()+=1; } }
println!("\n╔══ {label} [{want:08x}] {} records ══", recs.len());
let mut cv:Vec<_>=cols.into_iter().collect(); cv.sort_by_key(|x|std::cmp::Reverse(x.1));
let colstr:String=cv.iter().take(28).map(|(k,c)|format!("{k}({c})")).collect::<Vec<_>>().join(" ");
println!("║ numeric/enum fields: {colstr}");
// dump 2 sample records: identity + explicit fields
for o in recs.iter().take(2){
let id=o.get_raw("ID").unwrap_or("?");
let name=o.get_raw("Name").unwrap_or("");
print!("║ • {id}");
if !name.is_empty(){print!(" «{name}»");}
println!();
let fields:Vec<String>=o.resolved_fields().iter().map(|(k,v)|format!("{k}={v}")).collect();
for chunk in fields.chunks(5){ println!("║ {}", chunk.join(" ")); }
}
}
}

11
crates/sylpheed-formats/examples/dm_roster.rs Normal file
View File

@@ -0,0 +1,11 @@
use sylpheed_formats::{game_data, PakArchive};
use std::collections::BTreeMap;
fn main(){
let disc=std::env::var("SYLPHEED_DISC").unwrap();
let pak=PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).unwrap();
let chars=game_data::load_characters(&pak);
let mut byfac:BTreeMap<String,Vec<String>>=Default::default();
for c in &chars{ byfac.entry(c.faction.clone().unwrap_or("·".into())).or_default().push(format!("{}({})",c.id.clone().unwrap_or_default().trim_start_matches("Character"),c.faces.len())); }
println!("{} characters across {} factions:",chars.len(),byfac.len());
for (f,mut v) in byfac{ v.sort(); println!(" [{f}] {}", v.join(" ")); }
}

14
crates/sylpheed-formats/examples/dm_rows.rs Normal file
View File

@@ -0,0 +1,14 @@
use sylpheed_formats::{idxd::IdxdObject, PakArchive};
fn short(s:&str)->String{ s.trim_start_matches("Weapon_").trim_start_matches("UN_").trim_start_matches("UnitName_UN_").into() }
fn g<'a>(o:&'a IdxdObject,k:&str)->String{ o.get_f32(k).map(|v|{if v==v.trunc(){format!("{}",v as i64)}else{format!("{v}")}}).or_else(||o.get_raw(k).filter(|x|x.chars().next().map(|c|c.is_ascii_digit()).unwrap_or(false)).map(|s|s.to_string())).unwrap_or("·".into()) }
fn main(){
let disc=std::env::var("SYLPHEED_DISC").unwrap();
let arc=PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).unwrap();
let recs=|want:u32|->Vec<IdxdObject>{ arc.entries().iter().filter_map(|e|arc.read(e).ok()).filter_map(|b|IdxdObject::parse(&b).ok()).filter(|o|o.schema_hash==want).collect() };
println!("### WEAPONS (name | target | load | int | power | vel | range | trig)");
for o in recs(0x6ab4825a).iter().take(16){ println!("{:<34}\t{}\t{}\t{}\t{}\t{}\t{}\t{}", short(o.get_raw("ID").unwrap_or("?")), o.get_raw("TargetType").unwrap_or("·"), g(o,"LoadingCount"),g(o,"Interval"),g(o,"Power"),g(o,"Velocity"),g(o,"MaximumRange"),g(o,"TriggerShotCount")); }
println!("\n### UNITS/CRAFT (name | HP | cruise | accel | radar | turrets | score)");
for o in recs(0x43faa517).iter().take(16){ println!("{:<38}\t{}\t{}\t{}\t{}\t{}\t{}", short(o.get_raw("ID").unwrap_or("?")), g(o,"HP"),g(o,"CruisingVelocity"),g(o,"Acceleration"),g(o,"RadarRange"),g(o,"TurretCount"),g(o,"ScorePoint")); }
println!("\n### VESSELS/CAPITAL SHIPS (name | HP | Sz_X | Sz_Z | radar | turrets | bridges | hatches | shieldgen | thrusters)");
for o in recs(0x3c5b0549).iter(){ println!("{:<30}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}", short(o.get_raw("ID").unwrap_or("?")), g(o,"HP"),g(o,"Size_X"),g(o,"Size_Z"),g(o,"RadarRange"),g(o,"TurretCount"),g(o,"BridgeCount"),g(o,"HatchCount"),g(o,"ShieldGeneratorCount"),g(o,"ThrusterCount")); }
}

15
crates/sylpheed-formats/examples/dossier.rs Normal file
View File

@@ -0,0 +1,15 @@
use sylpheed_formats::{game_data, localization::TextIndex, PakArchive};
fn main(){
let disc=std::env::var("SYLPHEED_DISC").unwrap();
let pak=PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).unwrap();
let text=TextIndex::build(&pak);
for n in 1..=16u32{
let sid=format!("S{n:02}");
// primary objective across phases (first that resolves)
let obj:Vec<String>=(1..=3).flat_map(|p|text.objectives(&sid,p)).map(|s|s.to_string()).collect();
let lose:Vec<String>=(1..=3).flat_map(|p|text.lose_conditions(&sid,p)).map(|s|s.to_string()).collect();
let full_obj=obj.join(" ");
let full_lose=lose.into_iter().take(2).collect::<Vec<_>>().join(" ");
println!("{sid}\t{full_obj}\t{full_lose}");
}
}

17
crates/sylpheed-formats/examples/flights.rs Normal file
View File

@@ -0,0 +1,17 @@
use sylpheed_formats::{game_data as gd, PakArchive};
fn main(){
let disc=std::env::var("SYLPHEED_DISC").unwrap();
let pak=PakArchive::open(format!("{disc}/dat/GP_HANGAR_ARSENAL.pak")).unwrap();
let rosters=gd::load_pilot_rosters(&pak);
// distinct rosters by their pilot set
let mut seen=std::collections::BTreeSet::new(); let mut shown=0;
println!("{} pilot-roster configs; distinct line-ups:", rosters.len());
for r in &rosters{
let key:String=r.pilots.iter().map(|(c,p)|format!("{c}:{p}")).collect::<Vec<_>>().join(",");
if seen.insert(key) && shown<8 {
shown+=1;
let flt:Vec<String>=r.pilots.iter().map(|(c,p)|format!("{c}={p}")).collect();
println!(" {}", flt.join(" "));
}
}
}

43
crates/sylpheed-formats/examples/hokyu_demo_resolve.rs Normal file
View File

@@ -0,0 +1,43 @@
use sylpheed_formats::{hash::name_hash, movie_manifest, movie_subtitle as ms, movie_voice, slb, PakArchive};
use std::fs;use std::io::{Read,Seek,SeekFrom};use std::process::Command;
fn rg(disc:&str,g:u64,n:usize)->Vec<u8>{let mut segs=vec![];let mut cum=0u64;for i in 0..5{let p=format!("{disc}/dat/sound.p{i:02}");if let Ok(m)=fs::metadata(&p){segs.push((cum,m.len(),p));cum+=m.len();}}let mut out=vec![];let(mut need,mut pos)=(n,g);for(base,len,path)in &segs{if need==0||pos>=base+len||pos<*base{continue;}let local=pos-base;let take=need.min((len-local)as usize);let mut f=fs::File::open(path).unwrap();f.seek(SeekFrom::Start(local)).unwrap();let mut b=vec![0u8;take];f.read_exact(&mut b).unwrap();out.extend_from_slice(&b);need-=take;pos+=take as u64;}out}
fn ct(h:&[u8],n:&[u8])->bool{h.windows(n.len()).any(|w|w==n)}
fn dur(w:&str)->String{let o=Command::new("ffprobe").args(["-v","error","-show_entries","format=duration","-of","default=nw=1:nk=1",w]).output().unwrap();String::from_utf8_lossy(&o.stdout).trim().to_string()}
fn main(){
let disc=std::env::var("SYLPHEED_DISC").unwrap();
let tpak=PakArchive::open(format!("{disc}/dat/tables.pak")).unwrap();
let man=tpak.entries().iter().find_map(|e|tpak.read(e).ok().filter(|b|movie_manifest::is_manifest(b))).unwrap();
let lpak=PakArchive::open(format!("{disc}/dat/movie/eng.pak")).unwrap();
let reg=tpak.entries().iter().find_map(|e|tpak.read(e).ok().filter(|b|ct(b,b"eng\\Movie\\VOICE_ADV.slb"))).unwrap();
let ids=movie_voice::registry_voice_ids(&reg);
let stoc=fs::read(format!("{disc}/dat/sound.pak")).unwrap();
let ents=PakArchive::parse_toc(&stoc).unwrap();
// demo->token from bound hokyu
let hok:Vec<_>=movie_manifest::parse(&man).into_iter().filter(|m|m.movie.starts_with("hokyu_")).collect();
let resolve=|movie:&str|->Option<String>{
movie_manifest::voice_token(&man,movie).or_else(||{
let want=ms::track_voice_cues(&lpak,movie).first().map(|&(d,_)|d)?;
hok.iter().find_map(|e|{let t=e.voice_token.clone().filter(|_|e.movie.starts_with("hokyu_"))?; ms::track_voice_cues(&lpak,&e.movie).iter().any(|&(d,_)|d==want).then_some(t)})
})
};
for e in &hok{
let mv=&e.movie;
let bound=e.voice_token.is_some();
let tok=resolve(mv);
let demo=ms::track_voice_cues(&lpak,mv).first().map(|&(d,_)|d);
let mut d="—".to_string();
if let Some(tok)=&tok{
if let Some(&id)=ids.get(tok){
if let Some(anchor)=["Movie","etc","Voice"].iter().find_map(|dir|{let h=name_hash(&format!("eng\\{dir}\\{tok}.slb"));ents.binary_search_by_key(&h,|x|x.name_hash).ok().map(|i|ents[i].offset as u64)}){
let ws=(anchor.saturating_sub(2*1024*1024))&!3; let win=rg(&disc,ws,8*1024*1024);
if let Some(el)=movie_voice::find_descriptor(&win,id){let end=ws+el as u64;
let start=movie_voice::find_descriptor(&win,id.wrapping_sub(1)).or_else(||movie_voice::find_descriptor_before(&win,el)).map(|o|ws+o as u64).filter(|&s|s<end&&end-s<1_500_000).unwrap_or(anchor);
let region=rg(&disc,start,(end-start)as usize); let mut rf=slb::to_xma_riffs(&region); if rf.is_empty(){rf=slb::to_xma_riff_best(&region).into_iter().collect();}
if let Some(r)=rf.first(){let xp=format!("/tmp/hd_{mv}.xma.wav");let wp=format!("/tmp/hd_{mv}.wav");fs::write(&xp,r).unwrap();let _=Command::new("ffmpeg").args(["-hide_banner","-v","error","-y","-i",&xp,&wp]).status();d=dur(&wp);}
}
}
}
}
println!("{mv:16} {:8} demo={:?} -> {:11} voice={d}s", if bound{"BOUND"}else{"unbound"}, demo, tok.unwrap_or("(silent)".into()));
}
}

36
crates/sylpheed-formats/examples/hokyu_final.rs Normal file
View File

@@ -0,0 +1,36 @@
use sylpheed_formats::{hash::name_hash, movie_manifest, movie_voice, slb, PakArchive};
use std::fs;use std::io::{Read,Seek,SeekFrom};use std::process::Command;
fn rg(disc:&str,g:u64,n:usize)->Vec<u8>{let mut segs=vec![];let mut cum=0u64;for i in 0..5{let p=format!("{disc}/dat/sound.p{i:02}");if let Ok(m)=fs::metadata(&p){segs.push((cum,m.len(),p));cum+=m.len();}}let mut out=vec![];let(mut need,mut pos)=(n,g);for(base,len,path)in &segs{if need==0||pos>=base+len||pos<*base{continue;}let local=pos-base;let take=need.min((len-local)as usize);let mut f=fs::File::open(path).unwrap();f.seek(SeekFrom::Start(local)).unwrap();let mut b=vec![0u8;take];f.read_exact(&mut b).unwrap();out.extend_from_slice(&b);need-=take;pos+=take as u64;}out}
fn contains(h:&[u8],n:&[u8])->bool{h.windows(n.len()).any(|w|w==n)}
fn dur(w:&str)->String{let o=Command::new("ffprobe").args(["-v","error","-show_entries","format=duration","-of","default=nw=1:nk=1",w]).output().unwrap();String::from_utf8_lossy(&o.stdout).trim().to_string()}
fn hokyu_fallback(m:&str)->Option<String>{if !m.starts_with("hokyu_"){return None}let ls=m.contains("_LS_");let ds=m.contains("_DS_");let a=m.ends_with('A');let h=m.ends_with('H');Some(match(ls,ds,a,h){(true,_,true,_)=>"VOICE_D_450",(true,_,_,true)=>"VOICE_D_453",(_,true,true,_)=>"VOICE_D_452",(_,true,_,true)=>"VOICE_D_454",_=>return None}.into())}
fn main(){
let disc=std::env::var("SYLPHEED_DISC").unwrap();
let tpak=PakArchive::open(format!("{disc}/dat/tables.pak")).unwrap();
let manifest=tpak.entries().iter().find_map(|e|tpak.read(e).ok().filter(|b|movie_manifest::is_manifest(b))).unwrap();
let code="eng";
let registry=tpak.entries().iter().find_map(|e|tpak.read(e).ok().filter(|b|contains(b,format!("{code}\\Movie\\VOICE_ADV.slb").as_bytes()))).unwrap();
let ids=movie_voice::registry_voice_ids(&registry);
let stoc=fs::read(format!("{disc}/dat/sound.pak")).unwrap();
let entries=PakArchive::parse_toc(&stoc).unwrap();
let hokyu:Vec<String>=movie_manifest::parse(&manifest).into_iter().filter(|m|m.movie.starts_with("hokyu_")).map(|m|m.movie).collect();
for movie in &hokyu{
let bound=movie_manifest::voice_token(&manifest,movie);
let token=bound.clone().or_else(||hokyu_fallback(movie));
let Some(token)=token else{println!("{movie:16} NO TOKEN"); continue};
let src = if bound.is_some(){"bound"}else{"fallback"};
let Some(&id)=ids.get(&token) else{println!("{movie:16} {token} not in registry");continue};
let Some(anchor)=["Movie","etc","Voice"].iter().find_map(|d|{let h=name_hash(&format!("{code}\\{d}\\{token}.slb"));entries.binary_search_by_key(&h,|e|e.name_hash).ok().map(|i|entries[i].offset as u64)}) else{println!("{movie:16} no anchor");continue};
let win_start=(anchor.saturating_sub(2*1024*1024))&!3;
let window=rg(&disc,win_start,8*1024*1024);
let Some(el)=movie_voice::find_descriptor(&window,id) else{println!("{movie:16} desc not found");continue};
let end=win_start+el as u64;
let start=movie_voice::find_descriptor(&window,id.wrapping_sub(1)).or_else(||movie_voice::find_descriptor_before(&window,el)).map(|o|win_start+o as u64).filter(|&s|s<end&&end-s<1_500_000).unwrap_or(anchor);
let region=rg(&disc,start,(end-start)as usize);
let mut riffs=slb::to_xma_riffs(&region); if riffs.is_empty(){riffs=slb::to_xma_riff_best(&region).into_iter().collect();}
let mut d="?".into();
if let Some(r)=riffs.first(){let xp=format!("/tmp/hf_{movie}.xma.wav");let wp=format!("/tmp/hf_{movie}.wav");fs::write(&xp,r).unwrap();let _=Command::new("ffmpeg").args(["-hide_banner","-v","error","-y","-i",&xp,&wp]).status();d=dur(&wp);}
let mv=dur(&format!("{disc}/dat/movie/{movie}.wmv"));
println!("{movie:16} {src:8} {token:12} id={id} voice={d}s movie={mv}s");
}
}

50
crates/sylpheed-formats/examples/iso_map.rs Normal file
View File

@@ -0,0 +1,50 @@
use sylpheed_formats::PakArchive;
use std::collections::BTreeMap;
fn be32(b:&[u8],o:usize)->u32{ if o+4<=b.len(){u32::from_be_bytes([b[o],b[o+1],b[o+2],b[o+3]])}else{0} }
fn magic(b:&[u8])->String{
if b.len()<4 {return "(<4B)".into();}
let m=&b[0..4];
if m.iter().all(|&c|(0x20..0x7f).contains(&c)){ String::from_utf8_lossy(m).into() }
else if m==b"\x89PNG"{"PNG".into()} else if m==[0,1,0,0]{"ttf".into()}
else { format!("{:02x}{:02x}{:02x}{:02x}",m[0],m[1],m[2],m[3]) }
}
fn main(){
let disc=std::env::var("SYLPHEED_DISC").unwrap();
// Known/understood IDXD schemas (semantic parsers we have)
let known_schema:BTreeMap<u32,&str>=[(0x067025b9,"ADVERTISE_MOVIE (movie_manifest)"),(0x13cb84ba,"sound registry / sounds.tbl")].into();
let mut fmt_count:BTreeMap<String,(u64,u64)>=BTreeMap::new(); // fmt -> (count, bytes)
let mut schema_census:BTreeMap<u32,(u64,u64,String)>=BTreeMap::new(); // schema -> (count,bytes,sample pak)
let mut unknown_magics:BTreeMap<String,(u64,Vec<String>)>=BTreeMap::new();
let paks:Vec<String>={let mut v=vec![]; for e in std::fs::read_dir(format!("{disc}/dat")).unwrap(){let p=e.unwrap().path(); if p.extension().map(|x|x=="pak").unwrap_or(false){v.push(p.file_stem().unwrap().to_string_lossy().into());}} v.sort(); v};
println!("pak entries decompressed formats");
for pk in &paks{
let Ok(arc)=PakArchive::open(format!("{disc}/dat/{pk}.pak")) else{continue};
let mut per:BTreeMap<String,u64>=BTreeMap::new();
let mut tot=0u64; let n=arc.entries().len();
for e in arc.entries(){
let Ok(b)=arc.read(e) else{continue};
tot+=b.len() as u64;
let mut f=magic(&b);
if f=="IDXD"{ let s=be32(&b,8); schema_census.entry(s).or_insert((0,0,pk.clone())).0+=1; schema_census.get_mut(&s).unwrap().1+=b.len() as u64; f=format!("IDXD:{s:08x}"); }
else if !["XPR2","IXUD","LSTA","RATC","RIFF","XBG7","OTTO","PNG","ttf","DDS ","T8AD"].contains(&f.as_str()){
let ent=unknown_magics.entry(f.clone()).or_insert((0,vec![])); ent.0+=1; if ent.1.len()<3 && !ent.1.contains(pk){ent.1.push(pk.clone());}
}
*per.entry(if f.starts_with("IDXD:"){"IDXD".into()}else{f.clone()}).or_default()+=1;
let g=fmt_count.entry(if f.starts_with("IDXD:"){"IDXD".into()}else{f}).or_insert((0,0)); g.0+=1; g.1+=b.len() as u64;
}
let mut fs:Vec<_>=per.into_iter().collect(); fs.sort_by_key(|x|std::cmp::Reverse(x.1));
let top:String=fs.iter().take(4).map(|(k,v)|format!("{k}×{v}")).collect::<Vec<_>>().join(" ");
println!("{pk:26} {n:>7} {:>10}KB {top}", tot/1024);
}
println!("\n=== FORMAT TOTALS (across all paks) ===");
let mut fv:Vec<_>=fmt_count.into_iter().collect(); fv.sort_by_key(|x|std::cmp::Reverse(x.1.1));
for (f,(c,b)) in &fv{ println!(" {f:12} {c:>6} entries {:>8}KB", b/1024); }
println!("\n=== IDXD SCHEMA CENSUS ({} distinct schemas) ===", schema_census.len());
let mut sv:Vec<_>=schema_census.into_iter().collect(); sv.sort_by_key(|x|std::cmp::Reverse(x.1.1));
for (s,(c,b,pk)) in &sv{
let tag=known_schema.get(s).copied().unwrap_or("??? UNDECODED");
println!(" {s:08x} {c:>5} ent {:>7}KB e.g. {pk:22} {tag}", b/1024);
}
println!("\n=== UNKNOWN / UNCLASSIFIED MAGICS ===");
for (m,(c,pks)) in &unknown_magics{ println!(" {m:12} ×{c:<5} in {pks:?}"); }
}

46
crates/sylpheed-formats/examples/manifest_map.rs Normal file
View File

@@ -0,0 +1,46 @@
//! Validation: dump the parsed mission/phase cutscene map from the real manifest.
use sylpheed_formats::movie_manifest::{self, MovieKind};
use sylpheed_formats::pak::PakArchive;
fn main() {
let disc = std::env::var("SYLPHEED_DISC").expect("set SYLPHEED_DISC");
let arc = PakArchive::open(format!("{disc}/dat/tables.pak")).unwrap();
let man = arc
.entries()
.iter()
.find_map(|e| arc.read(e).ok().filter(|b| movie_manifest::is_manifest(b)))
.expect("manifest");
let entries = movie_manifest::parse(&man);
let mut counts = [0usize; 5];
for e in &entries {
counts[match e.kind {
MovieKind::System => 0,
MovieKind::Intro => 1,
MovieKind::Phase => 2,
MovieKind::PhaseEnd => 3,
MovieKind::Supply => 4,
}] += 1;
let m = e.mission.map(|x| x.to_string()).unwrap_or_default();
let p = e.phase.map(|x| x.to_string()).unwrap_or_default();
println!(
"{:<22} {:<9} m={:<2} p={:<1} {:<16} {:<14} tel={:<14} sub={}",
e.slot,
format!("{:?}", e.kind),
m,
p,
e.movie,
e.voice_token.as_deref().unwrap_or("-"),
e.telop.as_deref().unwrap_or("-"),
e.subtitle.as_deref().unwrap_or("-"),
);
}
eprintln!(
"TOTAL {} entries — System={} Intro={} Phase={} PhaseEnd={} Supply={}",
entries.len(),
counts[0],
counts[1],
counts[2],
counts[3],
counts[4]
);
}

47
crates/sylpheed-formats/examples/mission_map.rs Normal file
View File

@@ -0,0 +1,47 @@
use sylpheed_formats::{idxd::IdxdObject, PakArchive};
fn main(){
let disc=std::env::var("SYLPHEED_DISC").unwrap();
let arc=PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).unwrap();
// 1. All stages: BackGroundID + phases + stage tag (from EnumUnit_SNN)
println!("=== STAGES (StageResource 3c9ae32e) ===");
let mut rows=vec![];
for e in arc.entries(){
let Ok(b)=arc.read(e) else{continue}; let Ok(o)=IdxdObject::parse(&b) else{continue};
if o.schema_hash!=0x3c9ae32e {continue;}
let t=o.tokens();
let bg=o.get_raw("BackGroundID").unwrap_or("?").to_string();
let stage=t.iter().find_map(|s|s.strip_prefix("EnumUnit_").map(|x|x.trim_end_matches(".tbl").to_string())).unwrap_or("?".into());
let phases=t.iter().filter(|s|s.starts_with("Phase_")).count();
let unitgrp=t.iter().any(|s|s.starts_with("UnitGroup_"));
let route=t.iter().any(|s|s.starts_with("Route_"));
let formation=t.iter().any(|s|s.starts_with("FormationSet_"));
rows.push((stage,bg,phases,unitgrp,route,formation));
}
rows.sort();
for (s,bg,p,ug,rt,fm) in &rows{ println!(" {s:8} location={bg:14} phases={p} [squadrons:{} route:{} formations:{}]", if *ug{"Y"}else{"·"},if *rt{"Y"}else{"·"},if *fm{"Y"}else{"·"}); }
// 2. Parse objectives table into (stage,phase) -> key counts
println!("\n=== OBJECTIVES (033b5b7e) — per stage/phase key groups ===");
for e in arc.entries(){
let Ok(b)=arc.read(e) else{continue}; let Ok(o)=IdxdObject::parse(&b) else{continue};
if o.schema_hash!=0x033b5b7e {continue;}
let t=o.tokens();
let mut groups:std::collections::BTreeMap<String,u32>=Default::default();
for tok in t{ if let Some(p)=tok.find("_Objective_").or_else(||tok.find("_Lose_")).or_else(||tok.find("_Hint_")){ let key=&tok[..p]; *groups.entry(key.trim_start_matches(|c:char|!c.is_ascii_alphabetic()).into()).or_default()+=1; } }
let stages:std::collections::BTreeSet<String>=groups.keys().filter_map(|k|k.get(..3).map(|s|s.to_string())).collect();
println!(" covers {} stages: {:?}", stages.len(), stages);
println!(" sample S01_P1 group counts: {:?}", groups.iter().filter(|(k,_)|k.starts_with("S01_P1")).collect::<Vec<_>>());
break;
}
// 3. Resolve objective TEXT: search all pak entries for the key "S01_P1_Objective_00" as an ID with English text
println!("\n=== OBJECTIVE TEXT resolution probe ===");
for e in arc.entries(){
let Ok(b)=arc.read(e) else{continue}; let Ok(o)=IdxdObject::parse(&b) else{continue};
let t=o.tokens();
if let Some(i)=t.iter().position(|s|s=="S01_P1_Objective_00"){
let lo=i.saturating_sub(1); let hi=(i+3).min(t.len());
println!(" schema {:08x}: ...{:?}...", o.schema_hash, &t[lo..hi]);
}
}
}

36
crates/sylpheed-formats/examples/resolve_all.rs Normal file
View File

@@ -0,0 +1,36 @@
use sylpheed_formats::{hash::name_hash, movie_manifest, movie_voice, slb, PakArchive};
use std::fs;use std::io::{Read,Seek,SeekFrom};use std::process::Command;
fn rg(disc:&str,g:u64,n:usize)->Vec<u8>{let mut segs=vec![];let mut cum=0u64;for i in 0..5{let p=format!("{disc}/dat/sound.p{i:02}");if let Ok(m)=fs::metadata(&p){segs.push((cum,m.len(),p));cum+=m.len();}}let mut out=vec![];let(mut need,mut pos)=(n,g);for(base,len,path)in &segs{if need==0||pos>=base+len||pos<*base{continue;}let local=pos-base;let take=need.min((len-local)as usize);let mut f=fs::File::open(path).unwrap();f.seek(SeekFrom::Start(local)).unwrap();let mut b=vec![0u8;take];f.read_exact(&mut b).unwrap();out.extend_from_slice(&b);need-=take;pos+=take as u64;}out}
fn contains(h:&[u8],n:&[u8])->bool{h.windows(n.len()).any(|w|w==n)}
fn dur(w:&str)->String{let o=Command::new("ffprobe").args(["-v","error","-show_entries","format=duration","-of","default=nw=1:nk=1",w]).output().unwrap();String::from_utf8_lossy(&o.stdout).trim().to_string()}
fn main(){
let disc=std::env::var("SYLPHEED_DISC").unwrap();
let tpak=PakArchive::open(format!("{disc}/dat/tables.pak")).unwrap();
let manifest=tpak.entries().iter().find_map(|e|tpak.read(e).ok().filter(|b|movie_manifest::is_manifest(b))).unwrap();
let code="eng";
let registry=tpak.entries().iter().find_map(|e|tpak.read(e).ok().filter(|b|contains(b,format!("{code}\\Movie\\VOICE_ADV.slb").as_bytes()))).unwrap();
let ids=movie_voice::registry_voice_ids(&registry);
let stoc=fs::read(format!("{disc}/dat/sound.pak")).unwrap();
let entries=PakArchive::parse_toc(&stoc).unwrap();
for movie in ["ADV","RT01A","RT01B","RT01C_1","S00A","S01A","hokyu_LS_s02A","hokyu_LS_s09A","hokyu_LS_s02H","hokyu_DS_s07H"]{
let Some(token)=movie_manifest::voice_token(&manifest,movie) else { println!("{movie:16} no token"); continue };
let Some(&id)=ids.get(&token) else { println!("{movie:16} token {token} not in registry"); continue };
let Some(anchor)=["Movie","etc","Voice"].iter().find_map(|d|{let h=name_hash(&format!("{code}\\{d}\\{token}.slb"));entries.binary_search_by_key(&h,|e|e.name_hash).ok().map(|i|entries[i].offset as u64)}) else { println!("{movie:16} no anchor for {token}"); continue };
let win_start=(anchor.saturating_sub(2*1024*1024))&!3;
let window=rg(&disc,win_start,8*1024*1024);
let Some(end_local)=movie_voice::find_descriptor(&window,id) else { println!("{movie:16} id {id}: desc NOT FOUND"); continue };
let end=win_start+end_local as u64;
let start=movie_voice::find_descriptor(&window,id.wrapping_sub(1))
.or_else(||movie_voice::find_descriptor_before(&window,end_local))
.map(|o|win_start+o as u64)
.filter(|&s|s<end && end-s<1_500_000)
.unwrap_or(anchor);
let region=rg(&disc,start,(end-start) as usize);
let mut riffs=slb::to_xma_riffs(&region);
if riffs.is_empty(){ riffs=slb::to_xma_riff_best(&region).into_iter().collect(); }
let mut d="?".into();
if let Some(r)=riffs.first(){ let xp=format!("/tmp/ra2_{movie}.xma.wav");let wp=format!("/tmp/ra2_{movie}.wav");fs::write(&xp,r).unwrap();let _=Command::new("ffmpeg").args(["-hide_banner","-v","error","-y","-i",&xp,&wp]).status();d=dur(&wp);}
let mv=dur(&format!("{disc}/dat/movie/{movie}.wmv"));
println!("{movie:16} id={id:5} region={}KB riffs={} voice={d}s movie={mv}s", (end-start)/1024, riffs.len());
}
}

33
crates/sylpheed-formats/examples/spawn_survey.rs Normal file
View File

@@ -0,0 +1,33 @@
use sylpheed_formats::{idxd::IdxdObject, PakArchive};
use std::collections::BTreeMap;
fn main(){
let disc=std::env::var("SYLPHEED_DISC").unwrap();
let arc=PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).unwrap();
// schemas whose type0/content is squadron/formation/route/group/position
let mut hit:BTreeMap<u32,(u32,String,String)>=BTreeMap::new();
for e in arc.entries(){
let Ok(b)=arc.read(e) else{continue}; let Ok(o)=IdxdObject::parse(&b) else{continue};
let t=o.tokens(); let t0=t.first().cloned().unwrap_or_default();
let joined:String=t.iter().take(20).map(|s|s.as_str()).collect::<Vec<_>>().join(" ");
if ["Squadron","Formation","Route","UnitGroup","Position","Spawn","Wave","Frame","NullFrame"].iter().any(|k|t0.contains(k)||joined.contains(k)){
let ent=hit.entry(o.schema_hash).or_insert((0,t0.clone(),joined.chars().take(90).collect()));
ent.0+=1;
}
}
let mut v:Vec<_>=hit.into_iter().collect(); v.sort_by_key(|x|std::cmp::Reverse(x.1.0));
for (s,(c,t0,sample)) in &v{ println!("[{s:08x}] ×{c:<4} type0={t0:<18.18} :: {sample}"); }
// dump a Squadron/UnitGroup blob fully to see if it has positions (binary floats) or refs
println!("\n─── sample squadron/formation blob (first matching) ───");
for e in arc.entries(){
let Ok(b)=arc.read(e) else{continue}; let Ok(o)=IdxdObject::parse(&b) else{continue};
let t=o.tokens(); let t0=t.first().cloned().unwrap_or_default();
if t0.contains("Squadron")||t0.contains("Formation")||t0.contains("UnitGroup"){
println!("schema {:08x} type0={t0} count={} tokens={}, blob {}B", o.schema_hash,o.count,t.len(),b.len());
for (i,tk) in t.iter().take(30).enumerate(){ print!("{i:>2}:{tk:<20.20}"); if i%4==3{println!();} } println!();
// scan blob for float-like values (reasonable coords) in the binary region
let nfloats=(0..b.len().saturating_sub(4)).step_by(4).filter(|&i|{let f=f32::from_be_bytes([b[i],b[i+1],b[i+2],b[i+3]]); f.is_finite()&&f.abs()>1.0&&f.abs()<1e6}).count();
println!(" ~{nfloats} plausible BE-float words in blob (positions live in binary region if high)");
break;
}
}
}

11
crates/sylpheed-formats/examples/squadrons.rs Normal file
View File

@@ -0,0 +1,11 @@
use sylpheed_formats::{game_data, PakArchive};
fn main(){
let disc=std::env::var("SYLPHEED_DISC").unwrap();
let pak=PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).unwrap();
let sq=game_data::load_squadrons(&pak);
println!("{} squadron definitions", sq.len());
for s in sq.iter().filter(|s|s.side.as_deref()==Some("TCAF")).take(6){
let mem:Vec<String>=s.members.iter().map(|m|m.trim_start_matches("UN_").chars().take(18).collect()).collect();
println!(" {:8} {:22} {:24} x{} {:?}", s.id.clone().unwrap_or("?".into()), s.formation_id.clone().unwrap_or_default(), s.ai_id.clone().unwrap_or_default(), s.count.unwrap_or(0), mem);
}
}

32
crates/sylpheed-formats/examples/validate_cues.rs Normal file
View File

@@ -0,0 +1,32 @@
use sylpheed_formats::slb;
use std::fs;use std::io::{Read,Seek,SeekFrom};use std::process::Command;
fn rg(disc:&str,g:u64,n:usize)->Vec<u8>{let mut segs=vec![];let mut cum=0u64;for i in 0..5{let p=format!("{disc}/dat/sound.p{i:02}");if let Ok(m)=fs::metadata(&p){segs.push((cum,m.len(),p));cum+=m.len();}}let mut out=vec![];let(mut need,mut pos)=(n,g);for(base,len,path)in &segs{if need==0||pos>=base+len||pos<*base{continue;}let local=pos-base;let take=need.min((len-local)as usize);let mut f=fs::File::open(path).unwrap();f.seek(SeekFrom::Start(local)).unwrap();let mut b=vec![0u8;take];f.read_exact(&mut b).unwrap();out.extend_from_slice(&b);need-=take;pos+=take as u64;}out}
fn dur(w:&str)->String{let o=Command::new("ffprobe").args(["-v","error","-show_entries","format=duration:stream=channels","-of","default=nw=1:nk=1",w]).output().unwrap();String::from_utf8_lossy(&o.stdout).replace('\n'," ")}
fn movdur(disc:&str,m:&str)->String{ dur(&format!("{disc}/dat/movie/{m}")) }
fn main(){
let disc=std::env::var("SYLPHEED_DISC").unwrap();
// descriptor trailer global offsets (from scan) => cue N data = [desc(N-1)..desc(N)]
// (id, end_off, movie)
let cues=[(1600u32,437044592u64,"ADV.wmv"),(1601,437345648,"RT01A.wmv"),(1602,437712240,"RT01B.wmv"),
(1603,438080880,"RT01C_1.wmv"),(1604,438451568,"RT01C_2.wmv"),(1605,438789488,"RT02A.wmv")];
for k in 1..cues.len(){
let (id,end,mov)=cues[k];
let start=cues[k-1].1;
let region=rg(&disc,start,(end-start) as usize + 12000); // +tail to include full data before next trailer
let riffs=slb::to_xma_riffs(&region);
let mut total=0.0f32; let mut parts=vec![];
for (j,r) in riffs.iter().enumerate(){
let xp=format!("/tmp/cue{id}_{j}.xma.wav"); let wp=format!("/tmp/cue{id}_{j}.wav");
fs::write(&xp,r).unwrap();
let _=Command::new("ffmpeg").args(["-hide_banner","-v","error","-y","-i",&xp,&wp]).status();
let d=dur(&wp); parts.push(d.clone());
total+=d.split_whitespace().next().unwrap_or("0").parse::<f32>().unwrap_or(0.0);
}
// spanning?
let span_adv_end=437547264u64;
let spans = start < span_adv_end && end > span_adv_end || (start/1_000 != end/1_000);
println!("cue {id} ({mov}): region[{start}..{end}] {} bytes, {} riff(s), Σ={:.1}s | movie={} parts={:?}",
end-start, riffs.len(), total, movdur(&disc,mov), parts);
}
println!("\nWAVs at /tmp/cue16XX_*.wav (listen: RT01B=1602, RT01C_1=1603, RT01C_2=1604)");
}

64
crates/sylpheed-formats/src/font.rs Normal file
View File

@@ -0,0 +1,64 @@
//! Font metadata for the embedded UI fonts.
//!
//! The packs carry sfnt fonts (`\x00\x01\x00\x00` TrueType, `OTTO` OpenType/CFF)
//! and TrueType Collections (`ttcf`). We only need light metadata to *present*
//! them — family name, face count, glyph count — which `ttf-parser` extracts
//! safely (malformed input yields `None`, never a panic). The raw bytes are
//! handed to egui separately for a live sample.
use ttf_parser::{fonts_in_collection, name_id, Face};
/// Light font metadata for the viewer.
#[derive(Debug, Clone, PartialEq)]
pub struct FontInfo {
/// "TrueType", "OpenType (CFF)", or "TrueType Collection".
pub kind: &'static str,
/// Family / full name (best-effort; empty if the name table is unreadable).
pub family: String,
/// Number of faces (1 for a plain sfnt, N for a `ttcf` collection).
pub faces: u32,
/// Glyph count of the first face.
pub glyphs: u16,
}
/// Whether `bytes` starts with a recognized font signature.
pub fn is_font(bytes: &[u8]) -> bool {
matches!(
bytes.get(0..4),
Some(b"\x00\x01\x00\x00") | Some(b"OTTO") | Some(b"true") | Some(b"ttcf")
)
}
/// Extract light metadata, or `None` if the font can't be parsed.
pub fn parse_info(bytes: &[u8]) -> Option<FontInfo> {
let is_collection = bytes.get(0..4) == Some(b"ttcf");
let faces = fonts_in_collection(bytes).unwrap_or(1).max(1);
// Face 0 gives the representative family + glyph count.
let face = Face::parse(bytes, 0).ok()?;
let glyphs = face.number_of_glyphs();
let kind = if is_collection {
"TrueType Collection"
} else if bytes.get(0..4) == Some(b"OTTO") {
"OpenType (CFF)"
} else {
"TrueType"
};
// Prefer the typographic/full family name; fall back to any readable name.
let family = face
.names()
.into_iter()
.filter(|n| n.name_id == name_id::FULL_NAME || n.name_id == name_id::FAMILY)
.find_map(|n| n.to_string())
.or_else(|| face.names().into_iter().find_map(|n| n.to_string()))
.unwrap_or_default();
Some(FontInfo {
kind,
family,
faces,
glyphs,
})
}

980
crates/sylpheed-formats/src/game_data.rs Normal file
View File

@@ -0,0 +1,980 @@
//! Typed loaders for Project Sylpheed's combat data tables.
//!
//! The game keeps its balance data in reflective [`crate::idxd`] tables — one
//! blob per entity, `token[0]` naming the table, fields laid out value-before-key.
//! This module maps the four combat schemas to plain, cloneable structs: the
//! common stats as named `Option<…>` fields, and **every** explicitly-set field in
//! a [`fields`](Weapon::fields) map so nothing is lost.
//!
//! Reverse-engineered 2026-07-23 from `GP_MAIN_GAME_E.pak` (English; the D/F/I/J/S
//! paks are localized duplicates). Fields left at their default value omit their
//! value on disc — those live in title code, not here — so a `None` means "not set
//! on disc", never a guess.
//!
//! ```no_run
//! use sylpheed_formats::{game_data, PakArchive};
//! let pak = PakArchive::open("dat/GP_MAIN_GAME_E.pak").unwrap();
//! for w in game_data::load_weapons(&pak) {
//! println!("{} — power {:?}, range {:?}", w.id.unwrap_or_default(), w.power, w.max_range);
//! }
//! ```
use crate::idxd::IdxdObject;
use crate::pak::PakArchive;
use std::collections::BTreeMap;
/// IDXD schema ids of the combat tables (the `schema_hash` field of each blob).
pub mod schema {
/// Player craft config: physics, cameras, scoring, difficulty, render pipeline.
pub const PLAYER: u32 = 0x0426_e81d;
/// Weapon definitions (guns, missiles, beams, bombs).
pub const WEAPON: u32 = 0x6ab4_825a;
/// Craft / units — fighters, bombers, turrets, with their AI flight model.
pub const UNIT: u32 = 0x43fa_a517;
/// Vessels — capital ships, with structural component counts.
pub const VESSEL: u32 = 0x3c5b_0549;
/// Characters — pilots / crew / comms, with faction and portrait set.
pub const CHARACTER: u32 = 0xbd86_d41c;
/// Stage resource manifest — per mission: location, phases, resource tables.
pub const STAGE: u32 = 0x3c9a_e32e;
/// Message / demo dialogue — per line: speaker, portrait, voice clip, pages.
pub const MESSAGE: u32 = 0xb412_e6d8;
}
/// Parse a numeric field value, tolerating a trailing `f`/`F` (e.g. `"3.0f"`).
fn as_f32(v: Option<&String>) -> Option<f32> {
let v = v?;
let v = v.strip_suffix(['f', 'F']).unwrap_or(v);
v.parse().ok()
}
fn as_i64(v: Option<&String>) -> Option<i64> {
v?.parse().ok()
}
/// Collect every explicitly-set field of an IDXD blob into a `key → value` map,
/// plus the identity strings. Identifier-valued fields (`Model`, `ShotType`, …)
/// that carry a bare name rather than a value are not in the map — read those from
/// [`IdxdObject::get_raw`] if needed.
fn field_map(o: &IdxdObject) -> BTreeMap<String, String> {
o.resolved_fields()
.into_iter()
.map(|(k, v)| (k.to_string(), v.to_string()))
.collect()
}
/// Read every record of `schema` from a pak, mapping each with `build`.
fn load_table<T>(pak: &PakArchive, schema: u32, build: impl Fn(&IdxdObject) -> Option<T>) -> Vec<T> {
pak.entries()
.iter()
.filter_map(|e| pak.read(e).ok())
.filter_map(|b| IdxdObject::parse(&b).ok())
.filter(|o| o.schema_hash == schema)
.filter_map(|o| build(&o))
.collect()
}
// ── Weapon ────────────────────────────────────────────────────────────────────
/// A weapon definition (schema [`schema::WEAPON`]).
#[derive(Debug, Clone)]
pub struct Weapon {
pub id: Option<String>,
pub name: Option<String>,
/// Target mask, e.g. `"Vessel,Craft,Structure"`.
pub target_type: Option<String>,
pub power: Option<f32>,
pub velocity: Option<f32>,
pub min_velocity: Option<f32>,
pub max_velocity: Option<f32>,
pub min_range: Option<f32>,
pub max_range: Option<f32>,
/// Ammo / reload budget (`LoadingCount`).
pub loading_count: Option<i64>,
/// Seconds between shots.
pub interval: Option<f32>,
/// Shots per trigger pull (volley / lock count).
pub trigger_shot_count: Option<i64>,
pub mass: Option<f32>,
pub heating: Option<f32>,
pub cooling: Option<f32>,
pub life_time: Option<f32>,
/// All explicitly-set fields (a superset of the named ones above).
pub fields: BTreeMap<String, String>,
}
impl Weapon {
fn from_idxd(o: &IdxdObject) -> Option<Self> {
let f = field_map(o);
Some(Weapon {
id: o.get_raw("ID").map(str::to_string),
name: o.get_raw("Name").map(str::to_string),
target_type: o.get_raw("TargetType").map(str::to_string),
power: as_f32(f.get("Power")),
velocity: as_f32(f.get("Velocity")),
min_velocity: as_f32(f.get("MinimumVelocity")),
max_velocity: as_f32(f.get("MaximumVelocity")),
min_range: as_f32(f.get("MinimumRange")),
max_range: as_f32(f.get("MaximumRange")),
loading_count: as_i64(f.get("LoadingCount")),
interval: as_f32(f.get("Interval")),
trigger_shot_count: as_i64(f.get("TriggerShotCount")),
mass: as_f32(f.get("Mass")),
heating: as_f32(f.get("Heating")),
cooling: as_f32(f.get("Cooling")),
life_time: as_f32(f.get("LifeTime")),
fields: f,
})
}
/// Any explicit field parsed as `f32` (for the long-tail stats not named above).
pub fn field_f32(&self, key: &str) -> Option<f32> {
as_f32(self.fields.get(key))
}
}
/// Load every weapon from a pak. Catches all `Weapon`-shaped records, not just the
/// main [`schema::WEAPON`] — player special/missile weapons live in variant schemas
/// (e.g. `Weapon_DSaber_P_wep_*` in `GP_HANGAR_ARSENAL.pak`). Deduplicated by id.
pub fn load_weapons(pak: &PakArchive) -> Vec<Weapon> {
let mut seen = std::collections::BTreeSet::new();
let mut out = Vec::new();
for e in pak.entries() {
let Ok(b) = pak.read(e) else { continue };
let Ok(o) = IdxdObject::parse(&b) else { continue };
// `Weapon`/`QWeapon`/… but not the `EnumWeapon` name lists.
let t0 = o.tokens().first().map(String::as_str).unwrap_or("");
if !t0.ends_with("Weapon") || t0.contains("Enum") {
continue;
}
if let Some(w) = Weapon::from_idxd(&o) {
if w.id.as_deref().is_some_and(|id| seen.insert(id.to_string())) {
out.push(w);
}
}
}
out
}
// ── Craft / unit ────────────────────────────────────────────────────────────────
/// A craft / unit — fighter, bomber, turret (schema [`schema::UNIT`]). Carries the
/// AI flight model in [`fields`](CraftUnit::fields) (`AV_Pitch*`, `BarrelRoll_*`,
/// `TurnAttack_*`, …).
#[derive(Debug, Clone)]
pub struct CraftUnit {
pub id: Option<String>,
pub name: Option<String>,
pub hp: Option<f32>,
pub is_destructible: Option<bool>,
pub size_x: Option<f32>,
pub size_y: Option<f32>,
pub size_z: Option<f32>,
pub size_radius: Option<f32>,
pub radar_range: Option<f32>,
pub fcs_range: Option<f32>,
pub cruising_velocity: Option<f32>,
pub maximum_velocity: Option<f32>,
pub acceleration: Option<f32>,
pub deceleration: Option<f32>,
pub shield_ratio: Option<f32>,
pub turret_count: Option<i64>,
pub score_point: Option<i64>,
pub fields: BTreeMap<String, String>,
}
impl CraftUnit {
fn from_idxd(o: &IdxdObject) -> Option<Self> {
let f = field_map(o);
Some(CraftUnit {
id: o.get_raw("ID").map(str::to_string),
name: o.get_raw("Name").map(str::to_string),
hp: as_f32(f.get("HP")),
is_destructible: o.get_bool("IsDestructible"),
size_x: as_f32(f.get("Size_X")),
size_y: as_f32(f.get("Size_Y")),
size_z: as_f32(f.get("Size_Z")),
size_radius: as_f32(f.get("Size_Radius")),
radar_range: as_f32(f.get("RadarRange")),
fcs_range: as_f32(f.get("FCSRange")),
cruising_velocity: as_f32(f.get("CruisingVelocity")),
maximum_velocity: as_f32(f.get("MaximumVelocity")),
acceleration: as_f32(f.get("Acceleration")),
deceleration: as_f32(f.get("Deceleration")),
shield_ratio: as_f32(f.get("ShieldRatio")),
turret_count: as_i64(f.get("TurretCount")),
score_point: as_i64(f.get("ScorePoint")),
fields: f,
})
}
pub fn field_f32(&self, key: &str) -> Option<f32> {
as_f32(self.fields.get(key))
}
}
/// Load every craft / unit from a pak.
pub fn load_units(pak: &PakArchive) -> Vec<CraftUnit> {
load_table(pak, schema::UNIT, CraftUnit::from_idxd)
}
// ── Vessel / capital ship ───────────────────────────────────────────────────────
/// A capital ship (schema [`schema::VESSEL`]). Component counts drive the
/// destructible hardpoints.
#[derive(Debug, Clone)]
pub struct Vessel {
pub id: Option<String>,
pub name: Option<String>,
/// The 3D hull resource stem, e.g. `rou_e105` — the `eNNN`/`fNNN` id links to
/// the XBG7 part family (`e105_bdy_*`, `e105_brg_*`, …) inside a `Stage_SNN.xpr`.
pub model: Option<String>,
pub hp: Option<f32>,
pub size_x: Option<f32>,
pub size_y: Option<f32>,
pub size_z: Option<f32>,
pub radar_range: Option<f32>,
pub fcs_range: Option<f32>,
pub maximum_velocity: Option<f32>,
pub shield_ratio: Option<f32>,
pub score_point: Option<i64>,
pub turret_count: Option<i64>,
pub bridge_count: Option<i64>,
pub hatch_count: Option<i64>,
pub shield_generator_count: Option<i64>,
pub thruster_count: Option<i64>,
pub fields: BTreeMap<String, String>,
}
impl Vessel {
fn from_idxd(o: &IdxdObject) -> Option<Self> {
let f = field_map(o);
Some(Vessel {
id: o.get_raw("ID").map(str::to_string),
name: o.get_raw("Name").map(str::to_string),
model: o.get_raw("Model").map(str::to_string),
hp: as_f32(f.get("HP")),
size_x: as_f32(f.get("Size_X")),
size_y: as_f32(f.get("Size_Y")),
size_z: as_f32(f.get("Size_Z")),
radar_range: as_f32(f.get("RadarRange")),
fcs_range: as_f32(f.get("FCSRange")),
maximum_velocity: as_f32(f.get("MaximumVelocity")),
shield_ratio: as_f32(f.get("ShieldRatio")),
score_point: as_i64(f.get("ScorePoint")),
turret_count: as_i64(f.get("TurretCount")),
bridge_count: as_i64(f.get("BridgeCount")),
hatch_count: as_i64(f.get("HatchCount")),
shield_generator_count: as_i64(f.get("ShieldGeneratorCount")),
thruster_count: as_i64(f.get("ThrusterCount")),
fields: f,
})
}
pub fn field_f32(&self, key: &str) -> Option<f32> {
as_f32(self.fields.get(key))
}
}
/// Load every capital ship from a pak.
pub fn load_vessels(pak: &PakArchive) -> Vec<Vessel> {
load_table(pak, schema::VESSEL, Vessel::from_idxd)
}
// ── Player config ───────────────────────────────────────────────────────────────
/// Player craft config (schema [`schema::PLAYER`]) — one per mission. Physics,
/// projectile caps and scoring here; cameras, difficulty and the render pipeline
/// live in [`fields`](PlayerConfig::fields).
#[derive(Debug, Clone)]
pub struct PlayerConfig {
pub air_drag_factor: Option<f32>,
pub gravity_factor: Option<f32>,
pub bullet_limit: Option<i64>,
pub laser_limit: Option<i64>,
pub homing_limit: Option<i64>,
pub space_size: Option<f32>,
pub supply_range: Option<f32>,
pub main_mission_bonus: Option<i64>,
pub rank_score_s: Option<i64>,
pub rank_score_a: Option<i64>,
pub rank_score_b: Option<i64>,
pub rank_score_c: Option<i64>,
pub rank_score_d: Option<i64>,
pub fields: BTreeMap<String, String>,
}
impl PlayerConfig {
fn from_idxd(o: &IdxdObject) -> Option<Self> {
let f = field_map(o);
Some(PlayerConfig {
air_drag_factor: as_f32(f.get("AirDragFactor")),
gravity_factor: as_f32(f.get("GravityFactor")),
bullet_limit: as_i64(f.get("BulletLimit")),
laser_limit: as_i64(f.get("LaserLimit")),
homing_limit: as_i64(f.get("HomingLimit")),
space_size: as_f32(f.get("SpaceSize")),
supply_range: as_f32(f.get("SupplyRange")),
main_mission_bonus: as_i64(f.get("MainMissionBonus")),
rank_score_s: as_i64(f.get("RankScore_S")),
rank_score_a: as_i64(f.get("RankScore_A")),
rank_score_b: as_i64(f.get("RankScore_B")),
rank_score_c: as_i64(f.get("RankScore_C")),
rank_score_d: as_i64(f.get("RankScore_D")),
fields: f,
})
}
pub fn field_f32(&self, key: &str) -> Option<f32> {
as_f32(self.fields.get(key))
}
}
/// Load every player config (one per mission) from a pak.
pub fn load_player_configs(pak: &PakArchive) -> Vec<PlayerConfig> {
load_table(pak, schema::PLAYER, PlayerConfig::from_idxd)
}
// ── Character ───────────────────────────────────────────────────────────────────
/// A pilot / crew portrait: an emotion id (`FaceRAYMOND_07`) and its texture
/// (`pjf003_C02.t32`, a T8aD tile inside a RATC bundle).
#[derive(Debug, Clone)]
pub struct Face {
pub id: String,
pub texture: String,
}
/// A character (schema [`CHARACTER`](schema::CHARACTER)) — pilots, crew, comms.
#[derive(Debug, Clone)]
pub struct Character {
pub id: Option<String>,
/// Localized display-name key (resolve against the game's string tables).
pub name_key: Option<String>,
/// Faction / side, e.g. `TCAF` (player) or `ADAN` (enemy).
pub faction: Option<String>,
pub unique: Option<bool>,
/// Portrait set: each emotion face and the texture that draws it.
pub faces: Vec<Face>,
pub fields: BTreeMap<String, String>,
}
impl Character {
fn from_idxd(o: &IdxdObject) -> Option<Self> {
let t = o.tokens();
// Faces are `(texture, FaceID)` pairs after the `Faces` marker (value-
// before-key: the FaceID names the emotion, the `.t32` before it is its art).
let mut faces = Vec::new();
if let Some(start) = t.iter().position(|s| s == "Faces") {
for i in (start + 1)..t.len() {
if t[i].starts_with("Face") && i > 0 {
faces.push(Face {
id: t[i].clone(),
texture: t[i - 1].clone(),
});
}
}
}
Some(Character {
id: o.get_raw("ID").map(str::to_string),
name_key: o.get_raw("Name").map(str::to_string),
faction: o.get_raw("SideID").map(str::to_string),
unique: o.get_bool("Unique"),
faces,
fields: field_map(o),
})
}
}
/// Load every character from a pak.
pub fn load_characters(pak: &PakArchive) -> Vec<Character> {
load_table(pak, schema::CHARACTER, Character::from_idxd)
}
// ── Stage / mission ─────────────────────────────────────────────────────────────
/// The raw token immediately before `key` in `tokens` (value-before-key), for
/// identifier-valued fields the generic resolver skips (filenames, locations).
fn raw_before<'a>(tokens: &'a [String], key: &str) -> Option<&'a str> {
let i = tokens.iter().position(|t| t == key)?;
(i > 0).then(|| tokens[i - 1].as_str())
}
/// A mission / stage (schema [`STAGE`](schema::STAGE)). The manifest that wires a
/// mission together: its location, phase count, and the per-stage tables that
/// populate it (unit roster, squadrons, formations, flight routes, AI, objectives).
/// Resolve the referenced `*_S<NN>.tbl` tables with [`load_records`] for the
/// spawn/formation detail.
#[derive(Debug, Clone)]
pub struct Stage {
/// Stage tag derived from the unit-table name, e.g. `"S01"`.
pub id: String,
/// In-world location (`BackGroundID`), e.g. `Lebendorf`, `Acheron`, `Earth`.
pub location: Option<String>,
/// Number of `Phase_N` sections.
pub phases: u32,
pub unit_table: Option<String>,
pub character_table: Option<String>,
pub squadron_table: Option<String>,
pub formation_table: Option<String>,
pub route_table: Option<String>,
pub ai_params_table: Option<String>,
pub subobjective_table: Option<String>,
pub message_table: Option<String>,
/// Every `resource-kind → table` reference in the manifest.
pub resources: BTreeMap<String, String>,
}
impl Stage {
fn from_idxd(o: &IdxdObject) -> Option<Self> {
let t = o.tokens();
// Each resource is `table-name → Enumerate<Kind>` (value-before-key).
let mut resources = BTreeMap::new();
for i in 1..t.len() {
let key = t[i].as_str();
if key.starts_with("Enumerate") || key == "MessageSet" || key == "NamePlate" {
resources.insert(key.to_string(), t[i - 1].clone());
}
}
let unit_table = raw_before(t, "EnumerateUnit").map(str::to_string);
let id = unit_table
.as_deref()
.and_then(|s| s.strip_prefix("EnumUnit_"))
.map(|s| s.trim_end_matches(".tbl").to_string())
.unwrap_or_default();
Some(Stage {
id,
location: o.get_raw("BackGroundID").map(str::to_string),
phases: t.iter().filter(|s| s.starts_with("Phase_")).count() as u32,
unit_table,
character_table: raw_before(t, "EnumerateCharacter").map(str::to_string),
squadron_table: raw_before(t, "EnumerateSquadron").map(str::to_string),
formation_table: raw_before(t, "EnumerateFormation").map(str::to_string),
route_table: raw_before(t, "EnumerateNullFrame").map(str::to_string),
ai_params_table: raw_before(t, "EnumerateAIParams").map(str::to_string),
subobjective_table: raw_before(t, "EnumerateSubobjective").map(str::to_string),
message_table: raw_before(t, "MessageSet").map(str::to_string),
resources,
})
}
}
/// Load every stage / mission manifest from a pak.
pub fn load_stages(pak: &PakArchive) -> Vec<Stage> {
load_table(pak, schema::STAGE, Stage::from_idxd)
}
// ── Squadron / flight group ─────────────────────────────────────────────────────
/// A squadron — a flight group that spawns together: its formation shape, AI
/// behaviour, side, and member craft (with their pilots). Parsed from the
/// per-stage `Enumerate_Squadrons` tables (`UnitGroup_S<NN>.tbl`). The player's
/// Rhino flight, for instance, is a 2-craft TCAF squadron flying `Formation_2_Rhino`.
#[derive(Debug, Clone)]
pub struct Squadron {
/// Squadron tag (`TCN001`, `ADT101`, …), matched positionally to its
/// definition; `None` if the id list and definitions don't line up.
pub id: Option<String>,
pub formation_id: Option<String>,
pub ai_id: Option<String>,
/// Faction — `TCAF` (allied) or `ADAN` (enemy).
pub side: Option<String>,
pub count: Option<i64>,
/// Member craft (`UN_*` unit ids).
pub members: Vec<String>,
}
/// Load every squadron from a pak. Scans all `Enumerate_Squadrons` tables (their
/// schema hash varies per stage, but the table name is stable).
pub fn load_squadrons(pak: &PakArchive) -> Vec<Squadron> {
let mut out = Vec::new();
for e in pak.entries() {
let Ok(b) = pak.read(e) else { continue };
let Ok(o) = IdxdObject::parse(&b) else { continue };
let t = o.tokens();
if !t.first().is_some_and(|s| s.contains("Enumerate_Squadron")) {
continue;
}
// Definitions are delimited by the `FormationID` key (its value, the
// formation, sits just before it). The id list precedes the first one.
let fpos: Vec<usize> = t
.iter()
.enumerate()
.filter(|(_, s)| *s == "FormationID")
.map(|(i, _)| i)
.collect();
let Some(&first) = fpos.first() else { continue };
// The squadron id list: tokens after the header up to the first formation.
let ids: Vec<&String> = t[1..first.saturating_sub(1)].iter().collect();
for (k, &fi) in fpos.iter().enumerate() {
let start = fi.saturating_sub(1);
let end = fpos.get(k + 1).map(|&n| n.saturating_sub(1)).unwrap_or(t.len());
let span = &t[start..end];
let before = |key: &str| -> Option<String> {
span.iter().position(|s| s == key).and_then(|i| (i > 0).then(|| span[i - 1].clone()))
};
let count = before("Count").and_then(|v| v.parse::<i64>().ok());
// Members are the first `count` `UN_` craft after the header block
// (which ends at DisableInterval, else Count); anything past them
// belongs to the next squadron or the stage roster.
let member_start = span
.iter()
.position(|s| s == "DisableInterval")
.or_else(|| span.iter().position(|s| s == "Count"))
.map(|i| i + 1)
.unwrap_or(0);
let take = count.unwrap_or(0).max(0) as usize;
out.push(Squadron {
id: (ids.len() == fpos.len()).then(|| ids[k].clone()),
formation_id: Some(t[fi - 1].clone()),
ai_id: before("AIID"),
side: before("SideID"),
count,
members: span[member_start..]
.iter()
.filter(|s| s.starts_with("UN_"))
.take(take)
.cloned()
.collect(),
});
}
}
out
}
// ── Demo message / dialogue line ────────────────────────────────────────────────
/// One dialogue line (schema [`MESSAGE`](schema::MESSAGE)): who speaks, with which
/// portrait and voice clip, and the text-key(s) of its caption pages. Resolve the
/// page keys against a [`crate::localization::TextIndex`] for the words, and the
/// voice clip against the movie-voice banks for the audio.
#[derive(Debug, Clone)]
pub struct DemoMessage {
/// Line id, e.g. `MSG_VOICE_A_007`.
pub id: String,
/// Speaker (`CharacterRAYMOND`) when the line names one; many system /
/// continuation lines are unattributed (`None`).
pub character: Option<String>,
/// Portrait / emotion (`FaceRAYMOND_07`).
pub face: Option<String>,
/// Voice clip token (`VOICE_A_007`).
pub voice_clip: Option<String>,
/// Caption page text keys (`MSG_VOICE_A_007_000_00`, …) — resolve for the text.
pub page_keys: Vec<String>,
}
/// Load every dialogue line from a pak. Each `Message_NNN` sub-record within the
/// message blobs becomes one [`DemoMessage`]; fields are found by prefix, so the
/// positional/defaulted layout doesn't matter.
pub fn load_demo_messages(pak: &PakArchive) -> Vec<DemoMessage> {
let is_marker = |s: &str| {
s.strip_prefix("Message_").is_some_and(|n| !n.is_empty() && n.bytes().all(|b| b.is_ascii_digit()))
};
let mut out = Vec::new();
for e in pak.entries() {
let Ok(b) = pak.read(e) else { continue };
let Ok(o) = IdxdObject::parse(&b) else { continue };
if o.schema_hash != schema::MESSAGE {
continue;
}
let t = o.tokens();
let marks: Vec<usize> = t.iter().enumerate().filter(|(_, s)| is_marker(s)).map(|(i, _)| i).collect();
for (k, &m) in marks.iter().enumerate() {
let end = marks.get(k + 1).copied().unwrap_or(t.len());
let slice = &t[m + 1..end];
let Some(id) = slice.first().cloned() else { continue };
out.push(DemoMessage {
character: slice.iter().find(|s| s.starts_with("Character")).cloned(),
face: slice.iter().find(|s| s.starts_with("Face")).cloned(),
voice_clip: slice.iter().find(|s| s.starts_with("VOICE_")).cloned(),
page_keys: slice
.iter()
.filter(|s| s.len() > id.len() && s.starts_with(&id) && s[id.len()..].starts_with('_'))
.cloned()
.collect(),
id,
});
}
}
out
}
// ── Unit roster (per-mission combatants) ────────────────────────────────────────
/// Schema of the per-stage `EnumUnit` roster tables.
const ENUM_UNIT_SCHEMA: u32 = 0x35b8_dc67;
/// A mission's unit roster — the craft, vessels and props that can appear in one
/// battle (an `EnumUnit_S<NN>` table). Join the ids against [`load_units`] /
/// [`load_vessels`] for stats. The table isn't tagged with its stage on disc, so
/// [`stage`](UnitRoster::stage) is inferred from any `S<NN>_`-prefixed prop id in
/// the roster (asteroid collision meshes etc.) and is `None` when none is present.
#[derive(Debug, Clone)]
pub struct UnitRoster {
pub stage: Option<String>,
/// Distinct combatant unit ids (`UN_*`), excluding stage props (asteroids,
/// collision meshes).
pub units: Vec<String>,
}
/// Load every mission unit roster from a pak.
pub fn load_unit_rosters(pak: &PakArchive) -> Vec<UnitRoster> {
let mut out = Vec::new();
for e in pak.entries() {
let Ok(b) = pak.read(e) else { continue };
let Ok(o) = IdxdObject::parse(&b) else { continue };
if o.schema_hash != ENUM_UNIT_SCHEMA {
continue;
}
let ids: Vec<&str> = o.tokens().iter().filter(|s| s.starts_with("UN_")).map(String::as_str).collect();
// Stage tag from an `UN_S<NN>_…` prop id (e.g. UN_S01_Asteroid_cmesh_…).
let stage = ids.iter().find_map(|s| {
let r = s.trim_start_matches("UN_");
let bytes = r.as_bytes();
(r.len() >= 3 && bytes[0] == b'S' && bytes[1].is_ascii_digit() && bytes[2].is_ascii_digit())
.then(|| r[..3].to_string())
});
// Combatants: drop the stage props, dedup preserving order.
let mut units = Vec::new();
for id in ids {
let prop = id.contains("Asteroid") || id.contains("cmesh") || id.contains("_Box");
if !prop && !units.iter().any(|u| u == id) {
units.push(id.to_string());
}
}
if !units.is_empty() {
out.push(UnitRoster { stage, units });
}
}
out
}
// ── Pilot roster (player squadron assignments) ──────────────────────────────────
/// The player squadron's flight assignments for a mission — which pilot flies each
/// callsign (`Rhino1` → `Katana`, `Bird1` → `Sandra`, …). Parsed from the
/// `UNITS`/`ZUNITS` player-unit tables (in `GP_HANGAR_ARSENAL.pak`); assignments
/// vary per mission (different missions field different wingmen).
#[derive(Debug, Clone)]
pub struct PilotRoster {
/// `(callsign, pilot)` pairs in flight order, e.g. `("Rhino1", "Katana")`.
pub pilots: Vec<(String, String)>,
/// The player craft unit id (`UN_f002_TCAF_DeltaSaber…`).
pub player_unit: Option<String>,
}
/// True for a `Callsign<N>-Pilot` token (a single hyphen, digit-suffixed callsign,
/// alphabetic pilot) — the flight-assignment shape.
fn is_assignment(s: &str) -> bool {
let Some((cs, pilot)) = s.split_once('-') else { return false };
!pilot.is_empty()
&& pilot.chars().all(|c| c.is_ascii_alphabetic())
&& cs.len() >= 2
&& cs.as_bytes()[cs.len() - 1].is_ascii_digit()
&& cs.chars().all(|c| c.is_ascii_alphanumeric())
}
/// Load every player pilot roster from a pak (scan `GP_HANGAR_ARSENAL.pak`). One
/// per player-unit config; rosters with no assignment tokens are skipped.
pub fn load_pilot_rosters(pak: &PakArchive) -> Vec<PilotRoster> {
let mut out = Vec::new();
for e in pak.entries() {
let Ok(b) = pak.read(e) else { continue };
let Ok(o) = IdxdObject::parse(&b) else { continue };
let t = o.tokens();
let pilots: Vec<(String, String)> = t
.iter()
.filter(|s| is_assignment(s))
.filter_map(|s| s.split_once('-').map(|(c, p)| (c.to_string(), p.to_string())))
.collect();
if pilots.len() >= 2 {
out.push(PilotRoster {
pilots,
player_unit: t.iter().find(|s| s.starts_with("UN_")).cloned(),
});
}
}
out
}
// ── Arsenal (player weapon options per hardpoint) ───────────────────────────────
/// The player's selectable weapons, by hardpoint (from the `UNITS` player-unit
/// tables in `GP_HANGAR_ARSENAL.pak`). These are the arsenal display ids — the
/// Delta Saber's nose guns (`Stiletto_BG1`, `Broad_Sword_SG1`, …) and arm missiles
/// (`Falcon_9AM`, `White_Shark_T53R`, …) the player equips in the hangar.
#[derive(Debug, Clone, Default)]
pub struct Arsenal {
/// Nose slot — the fixed forward gun options.
pub nose: Vec<String>,
/// The three arm hardpoints — missile / bomb / special options.
pub arm1: Vec<String>,
pub arm2: Vec<String>,
pub arm3: Vec<String>,
}
/// The weapon ids listed under a `STANDARD_<hp>` header (skipping its `Type` key),
/// up to the next `STANDARD_`/`PlayerSET_` header.
fn hardpoint_options(tokens: &[String], header: &str) -> Vec<String> {
let Some(start) = tokens.iter().position(|s| s == header) else { return Vec::new() };
tokens[start + 1..]
.iter()
.skip_while(|s| *s == "Type")
.take_while(|s| !s.starts_with("STANDARD_") && !s.starts_with("PlayerSET"))
.filter(|s| *s != "Type")
.cloned()
.collect()
}
/// Load the player arsenal from a pak (`GP_HANGAR_ARSENAL.pak`). Unions the weapon
/// options across every player-unit config, deduped in first-seen order.
pub fn load_arsenal(pak: &PakArchive) -> Arsenal {
let mut a = Arsenal::default();
let mut seen = [(); 4].map(|_| std::collections::BTreeSet::new());
let slots: [(&str, usize); 4] = [
("STANDARD_NOSE", 0),
("STANDARD_ARM1", 1),
("STANDARD_ARM2", 2),
("STANDARD_ARM3", 3),
];
for e in pak.entries() {
let Ok(b) = pak.read(e) else { continue };
let Ok(o) = IdxdObject::parse(&b) else { continue };
if !o.tokens().first().is_some_and(|s| s.ends_with("UNITS")) {
continue;
}
for (header, idx) in slots {
let dst = match idx {
0 => &mut a.nose,
1 => &mut a.arm1,
2 => &mut a.arm2,
_ => &mut a.arm3,
};
for w in hardpoint_options(o.tokens(), header) {
if seen[idx].insert(w.clone()) {
dst.push(w);
}
}
}
}
a
}
// ── Generic record access (any of the ~105 schemas) ─────────────────────────────
/// A generically-decoded IDXD record: its table name, schema id, identity, and
/// every explicitly-set field. Use this to read the ~99 schemas without a bespoke
/// struct (missions, UI layouts, effects, enums, …).
#[derive(Debug, Clone)]
pub struct Record {
/// The table name — `token[0]` (e.g. `Weapon`, `StageResource`, `Sperkers`).
/// May carry a stray leading byte from the string-pool boundary on some
/// records; group by [`schema`](Self::schema), not this, for reliability.
pub table: String,
pub schema: u32,
pub id: Option<String>,
pub name: Option<String>,
/// Every explicitly-set field (`key → value`).
pub fields: BTreeMap<String, String>,
}
impl Record {
fn from_idxd(o: &IdxdObject) -> Option<Self> {
Some(Record {
table: o.tokens().first().cloned().unwrap_or_default(),
schema: o.schema_hash,
id: o.get_raw("ID").map(str::to_string),
name: o.get_raw("Name").map(str::to_string),
fields: field_map(o),
})
}
pub fn f32(&self, key: &str) -> Option<f32> {
as_f32(self.fields.get(key))
}
pub fn i64(&self, key: &str) -> Option<i64> {
as_i64(self.fields.get(key))
}
}
/// Load every record of an arbitrary `schema` from a pak, generically. Pair with
/// the [`schema`] constants or a hash from the IDXD census.
pub fn load_records(pak: &PakArchive, schema: u32) -> Vec<Record> {
load_table(pak, schema, Record::from_idxd)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn as_f32_tolerates_trailing_f() {
assert_eq!(as_f32(Some(&"3.0f".to_string())), Some(3.0));
assert_eq!(as_f32(Some(&"-0.5".to_string())), Some(-0.5));
assert_eq!(as_f32(Some(&"Vessel,Craft".to_string())), None);
}
// Integration tests against a real disc — skipped unless SYLPHEED_DISC is set.
fn pak() -> Option<PakArchive> {
let disc = std::env::var("SYLPHEED_DISC").ok()?;
PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")).ok()
}
#[test]
fn loads_weapons() {
let Some(pak) = pak() else { return };
let ws = load_weapons(&pak);
assert!(ws.len() >= 100, "expected ≥100 weapons, got {}", ws.len());
let rocket = ws
.iter()
.find(|w| w.id.as_deref().is_some_and(|s| s.contains("DeltaSaber_Rocket")))
.expect("Delta Saber rocket present");
assert_eq!(rocket.velocity, Some(3000.0));
assert_eq!(rocket.max_range, Some(6000.0));
assert_eq!(rocket.loading_count, Some(1000));
assert_eq!(rocket.target_type.as_deref(), Some("Vessel,Craft,Structure"));
}
#[test]
fn loads_units_and_vessels() {
let Some(pak) = pak() else { return };
let units = load_units(&pak);
assert!(units.len() >= 80);
// Player craft: HP 1000, radar 500000.
let player = units
.iter()
.find(|u| u.id.as_deref().is_some_and(|s| s.contains("DeltaSaber_T")))
.expect("player craft present");
assert_eq!(player.hp, Some(1000.0));
assert_eq!(player.radar_range, Some(500000.0));
let vessels = load_vessels(&pak);
let flagship = vessels
.iter()
.find(|v| v.id.as_deref().is_some_and(|s| s.contains("SDBattleship")))
.expect("SD-Battleship present");
assert_eq!(flagship.hp, Some(100000.0));
assert_eq!(flagship.turret_count, Some(25));
}
#[test]
fn loads_player_configs() {
let Some(pak) = pak() else { return };
let cfgs = load_player_configs(&pak);
assert!(!cfgs.is_empty());
let c = &cfgs[0];
assert_eq!(c.bullet_limit, Some(512));
assert_eq!(c.laser_limit, Some(32));
assert_eq!(c.rank_score_s, Some(10000));
}
#[test]
fn loads_characters() {
let Some(pak) = pak() else { return };
let chars = load_characters(&pak);
assert!(chars.len() >= 20, "expected many characters, got {}", chars.len());
let raymond = chars
.iter()
.find(|c| c.id.as_deref() == Some("CharacterRAYMOND"))
.expect("Raymond present");
assert_eq!(raymond.faction.as_deref(), Some("TCAF"));
assert!(!raymond.faces.is_empty());
assert!(raymond.faces.iter().all(|f| f.texture.ends_with(".t32")));
}
#[test]
fn loads_stages() {
let Some(pak) = pak() else { return };
let stages = load_stages(&pak);
let s01 = stages.iter().find(|s| s.id == "S01").expect("stage S01 present");
assert_eq!(s01.location.as_deref(), Some("Lebendorf"));
assert_eq!(s01.phases, 3);
assert_eq!(s01.unit_table.as_deref(), Some("EnumUnit_S01.tbl"));
assert_eq!(s01.squadron_table.as_deref(), Some("UnitGroup_S01.tbl"));
// 16-mission main campaign is present.
for n in 1..=16 {
assert!(stages.iter().any(|s| s.id == format!("S{n:02}")), "missing S{n:02}");
}
}
#[test]
fn loads_squadrons() {
let Some(pak) = pak() else { return };
let sq = load_squadrons(&pak);
assert!(sq.len() >= 20, "expected many squadrons, got {}", sq.len());
// A TCAF Rhino flight of Delta Sabers, membership matching its count.
let rhino = sq
.iter()
.find(|s| {
s.side.as_deref() == Some("TCAF")
&& s.formation_id.as_deref().is_some_and(|f| f.contains("Rhino"))
&& s.members.iter().any(|m| m.contains("DeltaSaber"))
})
.expect("a TCAF Rhino Delta Saber squadron");
assert_eq!(rhino.members.len() as i64, rhino.count.unwrap_or(-1));
assert!(rhino.members.iter().all(|m| m.contains("DeltaSaber")));
}
#[test]
fn loads_demo_messages() {
let Some(pak) = pak() else { return };
let msgs = load_demo_messages(&pak);
assert!(msgs.len() > 200, "expected many dialogue lines, got {}", msgs.len());
// Most lines are `MSG_*` ids with a speaker; page keys extend the id.
let msg_ids = msgs.iter().filter(|m| m.id.starts_with("MSG_")).count();
assert!(msg_ids > msgs.len() * 9 / 10, "{msg_ids}/{}", msgs.len());
// Attributed dialogue (an explicit speaker) is a minority — most lines are
// unattributed system/continuation messages — but there are hundreds.
let attributed = msgs.iter().filter(|m| m.character.is_some()).count();
assert!(attributed > 500, "attributed lines: {attributed}");
let m = msgs.iter().find(|m| m.character.is_some() && !m.page_keys.is_empty()).unwrap();
assert!(m.page_keys.iter().all(|k| k.starts_with(&m.id)));
}
#[test]
fn loads_unit_rosters() {
let Some(pak) = pak() else { return };
let rosters = load_unit_rosters(&pak);
assert!(rosters.len() > 20, "expected many rosters, got {}", rosters.len());
// At least a few tag their stage; all rosters carry combatants, no props.
assert!(rosters.iter().filter(|r| r.stage.is_some()).count() >= 4);
assert!(rosters.iter().all(|r| {
!r.units.is_empty() && r.units.iter().all(|u| !u.contains("Asteroid"))
}));
// A roster tagged S01 exists and names the player's Delta Saber.
let s01 = rosters.iter().find(|r| r.stage.as_deref() == Some("S01"));
assert!(s01.is_some_and(|r| r.units.iter().any(|u| u.contains("DeltaSaber"))));
}
#[test]
fn loads_pilot_rosters() {
let Ok(disc) = std::env::var("SYLPHEED_DISC") else { return };
let Ok(pak) = PakArchive::open(format!("{disc}/dat/GP_HANGAR_ARSENAL.pak")) else { return };
let rosters = load_pilot_rosters(&pak);
assert!(!rosters.is_empty(), "expected player pilot rosters");
// Some config assigns Katana to a Rhino callsign (she leads Rhino flight).
assert!(rosters.iter().any(|r| {
r.pilots.iter().any(|(cs, p)| cs.starts_with("Rhino") && p == "Katana")
}));
// Bird flight exists too.
assert!(rosters.iter().any(|r| r.pilots.iter().any(|(cs, _)| cs.starts_with("Bird"))));
}
#[test]
fn loads_arsenal() {
let Ok(disc) = std::env::var("SYLPHEED_DISC") else { return };
let Ok(pak) = PakArchive::open(format!("{disc}/dat/GP_HANGAR_ARSENAL.pak")) else { return };
let a = load_arsenal(&pak);
assert!(a.nose.len() >= 5 && a.arm1.len() >= 5, "nose {} arm1 {}", a.nose.len(), a.arm1.len());
assert!(a.nose.iter().any(|w| w.starts_with("Stiletto")));
assert!(a.arm1.iter().any(|w| w.starts_with("Falcon")));
// Headers never leak into the option lists.
assert!([&a.nose, &a.arm1, &a.arm2, &a.arm3]
.iter()
.all(|v| v.iter().all(|w| !w.starts_with("STANDARD_") && *w != "Type")));
}
#[test]
fn generic_records_read_any_schema() {
let Some(pak) = pak() else { return };
// Stage-resource table via the generic reader (no bespoke struct).
let stages = load_records(&pak, 0x3c9a_e32e);
assert!(!stages.is_empty());
assert!(stages.iter().all(|r| r.table.ends_with("StageResource")));
}
}

12
crates/sylpheed-formats/src/idxd.rs
View File

@@ -171,6 +171,18 @@ impl IdxdObject {
}
out
}
/// A short identity string for the object: the first of `ID` / `Name` /
/// `Model` present, else the schema hash. Shared by the CLI `pak list` and
/// the GUI pack browser.
pub fn identity(&self) -> String {
for key in ["ID", "Name", "Model"] {
if let Some(v) = self.get_raw(key) {
return format!("{key}={v}");
}
}
format!("schema {:08x}", self.schema_hash)
}
}
/// Extract the trailing string pool. Finds the smallest offset whose suffix is

187
crates/sylpheed-formats/src/ixud.rs Normal file
View File

@@ -0,0 +1,187 @@
//! IXUD localized-string / subtitle table reader.
//!
//! `IXUD` entries in the language packs (`dat/movie/<lang>.pak`, the
//! `GP_MAIN_GAME_<lang>` tables, …) hold localized UTF-16**BE** strings. The
//! movie language packs use them as **subtitle cue lists**: a `SUBTITLE` header
//! followed by alternating `(text, timecode)` tokens, e.g.
//!
//! ```text
//! SUBTITLE
//! "Calm down!" 00:09.40-00:11.00
//! "Why are you taking Margras away?" 00:18.50-00:19.70
//! ```
//!
//! Reference-style tracks store a message key (`MSG_DEMO_240`) plus a single
//! start timecode instead of inline text; the actual string then lives in a
//! separate global table (not resolved here).
//!
//! ## Binary layout (as far as needed)
//! ```text
//! 0x00 4 Magic "IXUD"
//! 0x04 4 version (1)
//! 0x08 4 schema/type hash (constant 0x6CC83E70)
//! .. .. record directory { key_hash u32, offset u32, len u32 } × n
//! .. .. UTF-16BE string pool, NUL-terminated entries
//! ```
//! We don't need the directory to *present* the track — decoding the pool and
//! pairing tokens after the `SUBTITLE` header is enough and robust.
/// Magic at the start of every IXUD entry.
pub const IXUD_MAGIC: [u8; 4] = *b"IXUD";
/// One subtitle cue: a start time (seconds), optional end time, and the text
/// (or a `MSG_*` reference key for reference-style tracks).
#[derive(Debug, Clone, PartialEq)]
pub struct Cue {
pub start: f32,
pub end: Option<f32>,
pub text: String,
}
/// A decoded subtitle track.
#[derive(Debug, Clone, Default, PartialEq)]
pub struct Subtitle {
pub cues: Vec<Cue>,
}
impl Subtitle {
/// True when the track carries no inline text — every cue is a `MSG_*`
/// reference whose string lives in a separate global table.
pub fn is_reference_only(&self) -> bool {
!self.cues.is_empty() && self.cues.iter().all(|c| c.text.starts_with("MSG_"))
}
}
/// Whether `bytes` is an IXUD entry.
pub fn is_ixud(bytes: &[u8]) -> bool {
bytes.len() >= 4 && bytes[0..4] == IXUD_MAGIC
}
/// Parse an IXUD entry as a subtitle track. Returns `None` if not IXUD.
/// Malformed pools yield as many well-formed cues as can be recovered.
pub fn parse(bytes: &[u8]) -> Option<Subtitle> {
if !is_ixud(bytes) {
return None;
}
// Decode the whole payload as UTF-16BE and split into NUL-terminated,
// printable runs. The binary header/directory decodes to control/CJK-range
// junk that we skip by seeking to the `SUBTITLE` marker.
let tokens = utf16be_tokens(bytes);
let start = tokens
.iter()
.position(|t| t.ends_with("SUBTITLE"))
.map(|i| i + 1)
.unwrap_or(0);
let rest = &tokens[start..];
// Pair (text, timecode). A token that parses as a timecode closes the cue
// opened by the preceding text token; unpaired tokens are skipped so a
// single glitch doesn't desync the rest.
let mut cues = Vec::new();
let mut i = 0;
while i + 1 < rest.len() {
if let Some((s, e)) = parse_timing(&rest[i + 1]) {
cues.push(Cue {
start: s,
end: e,
text: rest[i].clone(),
});
i += 2;
} else {
i += 1;
}
}
Some(Subtitle { cues })
}
/// Split the payload (interpreted as UTF-16BE) into printable, NUL-delimited
/// tokens. Control chars terminate the current token.
fn utf16be_tokens(bytes: &[u8]) -> Vec<String> {
let mut tokens = Vec::new();
let mut cur = String::new();
for pair in bytes.chunks_exact(2) {
let u = u16::from_be_bytes([pair[0], pair[1]]);
match char::from_u32(u as u32) {
Some(c) if !c.is_control() => cur.push(c),
_ => {
if !cur.is_empty() {
tokens.push(std::mem::take(&mut cur));
}
}
}
}
if !cur.is_empty() {
tokens.push(cur);
}
tokens
}
/// Parse `MM:SS.ss` (`" 9:04.40"` style, minutes may be blank/space-padded) or
/// a `start-end` range into seconds.
fn parse_timing(s: &str) -> Option<(f32, Option<f32>)> {
let one = |p: &str| -> Option<f32> {
let (mm, ss) = p.trim().split_once(':')?;
let m: f32 = if mm.trim().is_empty() {
0.0
} else {
mm.trim().parse().ok()?
};
let sec: f32 = ss.trim().parse().ok()?;
Some(m * 60.0 + sec)
};
match s.split_once('-') {
Some((a, b)) => Some((one(a)?, Some(one(b)?))),
None => Some((one(s)?, None)),
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Build a synthetic IXUD payload: an 8-byte header stand-in + UTF-16BE
/// NUL-terminated tokens.
fn synth(tokens: &[&str]) -> Vec<u8> {
let mut b = IXUD_MAGIC.to_vec();
b.extend_from_slice(&[0, 0, 0, 1, 0x6C, 0xC8, 0x3E, 0x70]); // version + schema
for t in tokens {
for u in t.encode_utf16() {
b.extend_from_slice(&u.to_be_bytes());
}
b.extend_from_slice(&[0, 0]); // NUL
}
b
}
#[test]
fn inline_subtitle_track() {
let b = synth(&[
"SUBTITLE",
"Calm down!",
"00:09.40-00:11.00",
"Let go of me!",
"00:13.10-00:14.50",
]);
let sub = parse(&b).unwrap();
assert_eq!(sub.cues.len(), 2);
assert_eq!(sub.cues[0].text, "Calm down!");
assert_eq!(sub.cues[0].start, 9.4);
assert_eq!(sub.cues[0].end, Some(11.0));
assert!(!sub.is_reference_only());
}
#[test]
fn reference_track() {
let b = synth(&["SUBTITLE", "MSG_DEMO_240", "00:01.00", "MSG_DEMO_241", "00:03.20"]);
let sub = parse(&b).unwrap();
assert_eq!(sub.cues.len(), 2);
assert_eq!(sub.cues[0].end, None);
assert!(sub.is_reference_only());
}
#[test]
fn rejects_non_ixud() {
assert!(parse(b"IDXD\0\0\0\0").is_none());
}
}

39
crates/sylpheed-formats/src/lib.rs
View File

@@ -25,6 +25,21 @@ pub mod pak;
// IDXD reflective object (ship / weapon / effect / menu definitions) reader
pub mod idxd;
// IXUD localized string / subtitle table reader
pub mod ixud;
// Embedded font (OTF / TTF / ttcf) metadata
pub mod font;
// T8aD 2D UI/HUD texture
pub mod t8ad;
// RATC nested resource bundle
pub mod ratc;
// LSTA sprite list (inline T8aD frames)
pub mod lsta;
// XISO reading is not available in-browser
#[cfg(not(target_arch = "wasm32"))]
pub mod xiso;
@@ -35,8 +50,32 @@ pub mod mesh;
// Audio parsing — scaffold for XMA handling
pub mod audio;
// Movie cutscene subtitles — the movie → track → text chain.
pub mod movie_subtitle;
// XACT `.slb` sound banks (voice / music) → decodable XMA1 RIFF.
pub mod slb;
// The movie manifest: authoritative movie → subtitle → voice index.
pub mod movie_manifest;
pub mod movie_voice;
pub mod game_data;
pub mod localization;
// Whole-ship assembly from XBG7 part families (capital ships as split parts).
pub mod ship;
/// Re-export the most commonly used types at the crate root.
pub use font::FontInfo;
pub use idxd::{IdxdError, IdxdObject};
pub use ixud::{Cue, Subtitle};
pub use movie_subtitle::{SubCue, SubLang};
pub use mesh::{GameMesh, Xbg7Model};
pub use ratc::RatcChild;
pub use t8ad::T8adImage;
pub use pak::{PakArchive, PakEntry, PakError};
pub use texture::{X360Texture, X360TextureFormat};
pub use vfs::{GameAssets, VfsError};

179
crates/sylpheed-formats/src/localization.rs Normal file
View File

@@ -0,0 +1,179 @@
//! Localization — resolve the game's text keys to display strings.
//!
//! Project Sylpheed stores its UI/story text in `IXUD` entries as **UTF-16LE**,
//! interleaved `text, key, text, key, …`: a prose string immediately followed by
//! the identifier that names it. Objectives (`S01_P1_Objective_00`), hints, lose
//! conditions, character names (`CharacterName_CharacterRAYMOND`) and cutscene
//! dialogue (`MSG_*`) all resolve this way. One pak per language — pass
//! `GP_MAIN_GAME_E.pak` for English, `GP_MAIN_GAME_J.pak` for Japanese, etc.
//!
//! ```no_run
//! use sylpheed_formats::{localization::TextIndex, PakArchive};
//! let pak = PakArchive::open("dat/GP_MAIN_GAME_E.pak").unwrap();
//! let text = TextIndex::build(&pak);
//! assert_eq!(text.character_name("CharacterRAYMOND"), Some("Raymond"));
//! for line in text.objectives("S01", 1) { println!("• {line}"); }
//! ```
use crate::pak::PakArchive;
use std::collections::BTreeMap;
/// A UTF-16 unit that belongs to a printable text run (ASCII + Latin-1 + Latin
/// Extended-A). Control codes and the CJK planes split tokens.
fn is_text_unit(u: u16) -> bool {
matches!(u, 0x20..=0x7e | 0xa0..=0xff | 0x100..=0x17f)
}
/// Split a UTF-16LE blob into printable tokens (mirrors the subtitle tokenizer:
/// step 2 inside a run, 1 on a break to resync odd alignment).
fn utf16_tokens(bytes: &[u8]) -> Vec<String> {
let mut out = Vec::new();
let mut cur = String::new();
let mut i = 0;
while i + 1 < bytes.len() {
let u = u16::from_le_bytes([bytes[i], bytes[i + 1]]);
if is_text_unit(u) {
if let Some(c) = char::from_u32(u as u32) {
cur.push(c);
}
i += 2;
} else {
if cur.chars().count() >= 2 {
out.push(std::mem::take(&mut cur));
} else {
cur.clear();
}
i += 1;
}
}
if cur.chars().count() >= 2 {
out.push(cur);
}
out
}
/// An identifier token — the key half of a `text, key` pair.
fn is_key(s: &str) -> bool {
s.len() >= 5
&& s.contains('_')
&& s.chars().all(|c| c.is_ascii_alphanumeric() || c == '_')
&& s.chars().any(|c| c.is_ascii_uppercase())
}
/// A display-text token — anything with a lowercase letter that isn't itself a key.
fn is_prose(s: &str) -> bool {
!is_key(s) && s.chars().any(|c| c.is_ascii_lowercase())
}
/// Text-key → display-string index for one language, built from a `GP_MAIN_GAME_*`
/// pak.
#[derive(Debug, Clone, Default)]
pub struct TextIndex {
entries: BTreeMap<String, String>,
}
impl TextIndex {
/// Scan a language pak's `IXUD` entries and index every `text → key` pair.
pub fn build(pak: &PakArchive) -> Self {
let mut entries = BTreeMap::new();
for e in pak.entries() {
let Ok(b) = pak.read(e) else { continue };
if b.len() < 4 || &b[0..4] != b"IXUD" {
continue;
}
let toks = utf16_tokens(&b);
for w in toks.windows(2) {
if is_key(&w[1]) && is_prose(&w[0]) {
entries.entry(w[1].clone()).or_insert_with(|| w[0].clone());
}
}
}
TextIndex { entries }
}
/// The display string for a text key, if present.
pub fn get(&self, key: &str) -> Option<&str> {
self.entries.get(key).map(String::as_str)
}
/// Number of indexed keys.
pub fn len(&self) -> usize {
self.entries.len()
}
pub fn is_empty(&self) -> bool {
self.entries.is_empty()
}
/// The objective lines for a stage/phase, e.g. `("S01", 1)`. Empty if none
/// resolve (unused objective slots are simply absent on disc).
pub fn objectives(&self, stage: &str, phase: u32) -> Vec<&str> {
self.numbered(&format!("{stage}_P{phase}_Objective"))
}
/// The lose-condition lines for a stage/phase.
pub fn lose_conditions(&self, stage: &str, phase: u32) -> Vec<&str> {
self.numbered(&format!("{stage}_P{phase}_Lose"))
}
/// The hint lines for a stage/phase.
pub fn hints(&self, stage: &str, phase: u32) -> Vec<&str> {
self.numbered(&format!("{stage}_P{phase}_Hint"))
}
/// A character's display name from its id (`CharacterRAYMOND` → `Raymond`).
pub fn character_name(&self, character_id: &str) -> Option<&str> {
self.get(&format!("CharacterName_{character_id}"))
}
/// Collect a `<base>_NN` run of lines while they resolve.
fn numbered(&self, base: &str) -> Vec<&str> {
(0..16)
.filter_map(|i| self.get(&format!("{base}_{i:02}")))
.collect()
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn tokenizes_mixed_runs() {
// "Hi" NUL "K_1" as UTF-16LE → two tokens.
let mut b = Vec::new();
for s in ["Hi", "\0", "K_1x"] {
for c in s.chars() {
b.extend_from_slice(&(c as u16).to_le_bytes());
}
}
let t = utf16_tokens(&b);
assert!(t.contains(&"Hi".to_string()));
assert!(t.contains(&"K_1x".to_string()));
}
#[test]
fn key_and_prose_classify() {
assert!(is_key("S01_P1_Objective_00"));
assert!(is_key("CharacterName_CharacterRAYMOND"));
assert!(!is_key("Repel the attack."));
assert!(is_prose("Repel the attack."));
assert!(is_prose("Raymond"));
assert!(!is_prose("S01_P1_Objective_00"));
}
// Disc-backed — skipped unless SYLPHEED_DISC is set.
#[test]
fn resolves_real_text() {
let Ok(disc) = std::env::var("SYLPHEED_DISC") else { return };
let Ok(pak) = PakArchive::open(format!("{disc}/dat/GP_MAIN_GAME_E.pak")) else { return };
let t = TextIndex::build(&pak);
assert!(t.len() > 5000, "expected a large index, got {}", t.len());
assert_eq!(t.character_name("CharacterRAYMOND"), Some("Raymond"));
assert_eq!(
t.get("S01_P1_Objective_00"),
Some("Repel the enemy's surprise attack.")
);
assert!(!t.objectives("S01", 1).is_empty());
}
}

78
crates/sylpheed-formats/src/lsta.rs Normal file
View File

@@ -0,0 +1,78 @@
//! `LSTA` — a sprite list: a header followed by N inline [`T8aD`](crate::t8ad)
//! frames concatenated back-to-back.
//!
//! A `count` lives at `@0x04`, but a few entries disagree with the actual frame
//! count, so we walk by the `T8aD` magic instead (robust) and decode each frame
//! from its slice up to the next frame (or end).
use crate::t8ad::{self, T8adImage, T8AD_MAGIC};
/// Magic at the start of an LSTA sprite list.
pub const LSTA_MAGIC: [u8; 4] = *b"LSTA";
/// Whether `bytes` is an LSTA list.
pub fn is_lsta(bytes: &[u8]) -> bool {
bytes.len() >= 4 && bytes[0..4] == LSTA_MAGIC
}
/// Decode all inline T8aD frames. Frames that don't decode (unsupported T8aD
/// variant) are skipped. Returns `None` only for non-LSTA input.
pub fn parse(bytes: &[u8]) -> Option<Vec<T8adImage>> {
if !is_lsta(bytes) {
return None;
}
// Offsets of each inline T8aD frame (search past the 4-byte magic).
let mut offs = Vec::new();
let mut i = 4;
while i + 4 <= bytes.len() {
if bytes[i..i + 4] == T8AD_MAGIC {
offs.push(i);
i += 4;
} else {
i += 1;
}
}
let mut frames = Vec::with_capacity(offs.len());
for (idx, &off) in offs.iter().enumerate() {
let next = offs.get(idx + 1).copied().unwrap_or(bytes.len());
if let Some(img) = t8ad::parse(&bytes[off..next]) {
frames.push(img);
}
}
Some(frames)
}
#[cfg(test)]
mod tests {
use super::*;
fn t8ad_frame(w: u32, h: u32) -> Vec<u8> {
let mut b = vec![0u8; 64];
b[0..4].copy_from_slice(&T8AD_MAGIC);
b[0x14..0x18].copy_from_slice(&w.to_be_bytes());
b[0x18..0x1c].copy_from_slice(&h.to_be_bytes());
b[0x1c..0x20].copy_from_slice(&1u32.to_be_bytes());
b.extend_from_slice(&vec![0x80u8; (w * h * 4) as usize]);
b
}
#[test]
fn walks_inline_frames() {
let mut b = LSTA_MAGIC.to_vec();
b.extend_from_slice(&2u32.to_be_bytes());
b.extend_from_slice(&[0u8; 15]); // header remainder
b.extend_from_slice(&t8ad_frame(4, 4));
b.extend_from_slice(&t8ad_frame(2, 3));
let frames = parse(&b).unwrap();
assert_eq!(frames.len(), 2);
assert_eq!((frames[0].width, frames[0].height), (4, 4));
assert_eq!((frames[1].width, frames[1].height), (2, 3));
}
#[test]
fn rejects_non_lsta() {
assert!(parse(b"T8aD....").is_none());
}
}

2036
crates/sylpheed-formats/src/mesh.rs
View File

File diff suppressed because it is too large Load Diff

428
crates/sylpheed-formats/src/movie_manifest.rs Normal file
View File

@@ -0,0 +1,428 @@
//! The movie manifest: the game's authoritative **mission → phase → cutscene**
//! table (statically reversed). Beyond the movie→voice binding, it defines the
//! whole cutscene *structure*: which video (+ subtitle, + on-screen text overlay,
//! + voice) plays at each mission phase, and of what kind (story intro, in-mission
//! phase cutscene, resupply, …).
//!
//! ## On-disc structure (`dat/tables.pak`, entry hash `0x5b983a08`)
//!
//! One `IDXD` table (`Z1`+zlib block), schema `0x067025b9`. Unlike a flat IDXD
//! object, this one is an **array of records**, and its string pool is laid out
//! as **two consecutive arrays**:
//! 1. the **slot KEYS** — `LOGO1`, `ADVERTISE_MOVIE`, `STAFF_ROLL`, then one
//! per cutscene: `MS<NN><X>`, `STAGE<NN>_PHASE0<N>`, `STAGE<NN>_PHASE_END…`,
//! `S<NN>_SUPPLY_<ACROPOLIS|TANKER>`, in play order;
//! 2. the **value RECORDS**, in the same order — each a run led by `<movie>.wmv`
//! and (optionally) a `<pak>+…​.prt` overlay (field `TELOP`), a
//! `<pak>+SUBTITLE_<movie>.tbl` (field `SUBTITLE`), and a `VOICE_<token>`
//! (field `VOICETRACK`). A few slot keys are **valueless** (e.g. stages 5 and
//! 12 have no resupply video) — those slots have no record and are skipped.
//!
//! The **slot key is the semantic role** — this is how the game separates a
//! cutscene voice from in-mission radio chatter (structurally, by which slot/bank,
//! not one boolean): `MS*` = story intro (`VOICE_S*`), `STAGE*_PHASE*` = radio-box
//! cutscene (`VOICE_RT*`), `S*_SUPPLY_*` = resupply (`VOICE_D_45x` or none).
//! Resupply videos are **reused across stages** (`S04_SUPPLY_ACROPOLIS` →
//! `hokyu_DS_s02A`), so the mapping is genuinely a table, not derivable from names.
//!
//! The voice token is likewise **not** always `VOICE_<movie>`: most movies use
//! `VOICE_<movie>` (in `<lang>\Movie\`), but the resupply cutscenes bind to
//! in-mission radio clips like `VOICE_D_450` (in `<lang>\etc\`), and some have no
//! voice at all. This manifest is the ground truth.
//!
//! ## Alignment
//!
//! Slot keys and value records would be a 1:1 positional zip but for the valueless
//! slots. We recover the gaps without decoding the binary node/index region by
//! using the fully-derivable **`MS<NN><X>` → `S<NN><X>` anchors**: a non-`MS` slot
//! whose record would be an `MS` target actually belongs to the upcoming `MS`
//! slot, so the non-`MS` slot is a gap. (Decoding the node region would make this
//! exact and also unlock the defaulted numeric fields in other IDXD tables.)
use crate::slb::VoiceLang;
/// The role a cutscene slot plays in the mission flow, from its slot key.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum MovieKind {
/// `LOGO*`, `ADVERTISE_MOVIE`, `STAFF_ROLL` — boot/system movies.
System,
/// `MS<NN><X>` — story intro movie (camera on the speaker; stereo `VOICE_S*`).
Intro,
/// `STAGE<NN>_PHASE0<N>` — in-mission phase cutscene (radio box; `VOICE_RT*`).
Phase,
/// `STAGE<NN>_PHASE_END[_0N]` — phase-end cutscene.
PhaseEnd,
/// `S<NN>_SUPPLY_<ACROPOLIS|TANKER>` — resupply (hokyu) cutscene.
Supply,
}
/// One cutscene slot's manifest row: its semantic slot plus the bound assets.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct MovieEntry {
/// The slot key, e.g. `STAGE01_PHASE01`, `MS00A`, `S02_SUPPLY_ACROPOLIS`.
/// Empty when parsed from a blob without the slot-key array (fallback).
pub slot: String,
/// The slot's role in the mission flow (intro vs. phase cutscene vs. supply …).
pub kind: MovieKind,
/// Mission/stage number (`STAGE<NN>` / `MS<NN>` / `S<NN>_SUPPLY`), if any.
pub mission: Option<u8>,
/// Phase number for [`MovieKind::Phase`]/[`MovieKind::PhaseEnd`], if any.
pub phase: Option<u8>,
/// Movie basename without extension, e.g. `S13A`, `RT01A`, `hokyu_DS_s13A`.
pub movie: String,
/// Voice bank basename (`VOICE_S13A`, `VOICE_D_450`), or `None` for no voice.
pub voice_token: Option<String>,
/// Subtitle table basename (`SUBTITLE_S13A.tbl`), pak prefix stripped, if any.
pub subtitle: Option<String>,
/// On-screen text-overlay (`TELOP`) basename (`pwrt01.prt`), if any.
pub telop: Option<String>,
}
/// Does this blob look like the movie manifest? (`IDXD` whose string pool
/// carries movie + subtitle + voice references.) Used to locate it in
/// `tables.pak` without hardcoding a hash.
pub fn is_manifest(bytes: &[u8]) -> bool {
bytes.len() >= 4
&& &bytes[0..4] == b"IDXD"
&& contains(bytes, b".wmv")
&& contains(bytes, b"VOICE_")
&& contains(bytes, b"SUBTITLE_")
}
/// The field-type keys that appear once in the value region as a schema template.
const FIELD_KEYS: [&str; 4] = ["MOVIE", "VOICETRACK", "SUBTITLE", "TELOP"];
/// Parse the manifest into per-slot cutscene rows, in play order.
///
/// Each returned [`MovieEntry`] carries a movie; valueless slots (e.g. stage 5's
/// absent resupply) are omitted. When the two-array slot/record layout is present
/// (the real manifest) every row is tagged with its authoritative slot/kind/
/// mission/phase; otherwise (a blob without the key array) the kind is inferred
/// from the movie name and `slot` is empty.
pub fn parse(bytes: &[u8]) -> Vec<MovieEntry> {
let toks = ascii_runs(bytes, 3);
// Locate the slot-key array (`LOGO1` … first `<movie>.wmv`) and the value
// region (from the first `.wmv` onward). Fall back to a keyless scrape.
let first_wmv = toks.iter().position(|t| t.ends_with(".wmv"));
let key_start = toks.iter().position(|t| t == "LOGO1");
let (slot_keys, value_toks): (Vec<&str>, &[String]) = match (key_start, first_wmv) {
(Some(k), Some(w)) if k < w => (
toks[k..w].iter().map(String::as_str).collect(),
&toks[w..],
),
_ => (Vec::new(), toks.as_slice()),
};
// Build the value records (drop the schema-template field keys).
let mut records: Vec<MovieEntry> = Vec::new();
for t in value_toks.iter().filter(|t| !FIELD_KEYS.contains(&t.as_str())) {
if let Some(name) = t.strip_suffix(".wmv") {
records.push(MovieEntry {
slot: String::new(),
kind: kind_from_movie(name),
mission: None,
phase: None,
movie: name.to_string(),
voice_token: None,
subtitle: None,
telop: None,
});
} else if let Some(rec) = records.last_mut() {
if let Some(v) = t.strip_prefix("VOICE_") {
if rec.voice_token.is_none() {
rec.voice_token = Some(format!("VOICE_{v}"));
}
} else if t.ends_with(".tbl") {
rec.subtitle.get_or_insert_with(|| after_plus(t));
} else if t.ends_with(".prt") {
rec.telop.get_or_insert_with(|| after_plus(t));
}
}
}
// Attach slot keys to records, recovering valueless-slot gaps via MS anchors.
if !slot_keys.is_empty() {
let ms_targets: std::collections::HashSet<String> = slot_keys
.iter()
.filter_map(|k| ms_target(k))
.collect();
let mut j = 0usize;
for slot in &slot_keys {
if j >= records.len() {
break;
}
// A gap: this slot's "record" is actually the next MS slot's movie.
let would_be = &records[j].movie;
let is_gap = match ms_target(slot) {
// MS slot: only binds if its record is the expected `S<NN><X>`.
Some(expected) => *would_be != expected,
// Non-MS slot: a gap iff the record belongs to an upcoming MS slot.
None => ms_targets.contains(would_be),
};
if is_gap {
continue;
}
let (kind, mission, phase) = classify_slot(slot);
let rec = &mut records[j];
rec.slot = (*slot).to_string();
rec.kind = kind;
rec.mission = mission;
rec.phase = phase;
j += 1;
}
}
records
}
/// If `slot` is an `MS<NN><X>` intro slot, the movie it must bind to (`S<NN><X>`).
fn ms_target(slot: &str) -> Option<String> {
let rest = slot.strip_prefix("MS")?;
// `<NN><X>`: two digits then a letter suffix.
if rest.len() >= 3 && rest[..2].bytes().all(|b| b.is_ascii_digit()) {
Some(format!("S{rest}"))
} else {
None
}
}
/// Semantic role + mission/phase from a slot key.
fn classify_slot(slot: &str) -> (MovieKind, Option<u8>, Option<u8>) {
if let Some(rest) = slot.strip_prefix("MS") {
if rest.len() >= 2 {
return (MovieKind::Intro, rest[..2].parse().ok(), None);
}
}
if let Some(rest) = slot.strip_prefix("STAGE") {
let nn: String = rest.chars().take_while(|c| c.is_ascii_digit()).collect();
if let Ok(m) = nn.parse::<u8>() {
if slot.contains("PHASE_END") {
// trailing `_NN` if present (`STAGE12_PHASE_END_02`)
let phase = slot.rsplit('_').next().and_then(|x| x.parse().ok());
return (MovieKind::PhaseEnd, Some(m), phase);
}
// `STAGE<NN>_PHASE0<N>`
let phase = rest[nn.len()..]
.strip_prefix("_PHASE")
.and_then(|p| p.parse().ok());
return (MovieKind::Phase, Some(m), phase);
}
}
if slot.contains("_SUPPLY") {
if let Some(rest) = slot.strip_prefix('S') {
let nn: String = rest.chars().take_while(|c| c.is_ascii_digit()).collect();
return (MovieKind::Supply, nn.parse().ok(), None);
}
}
(MovieKind::System, None, None)
}
/// Fallback kind from a movie basename (used when no slot-key array is present).
fn kind_from_movie(movie: &str) -> MovieKind {
if movie.starts_with("hokyu_") {
MovieKind::Supply
} else if movie.starts_with("RT") {
MovieKind::Phase
} else if movie.len() >= 3
&& movie.starts_with('S')
&& movie[1..3].bytes().all(|b| b.is_ascii_digit())
{
MovieKind::Intro
} else {
MovieKind::System
}
}
/// The part of a `<pak>+<name>` reference after the `+` (or the whole string).
fn after_plus(s: &str) -> String {
s.rsplit('+').next().unwrap_or(s).to_string()
}
/// The voice bank basename bound to `movie`, if the manifest lists one.
///
/// Returns `None` both when the movie is absent and when it is present with no
/// voice — callers that need to distinguish should use [`parse`].
pub fn voice_token(bytes: &[u8], movie: &str) -> Option<String> {
parse(bytes)
.into_iter()
.find(|e| e.movie == movie)
.and_then(|e| e.voice_token)
}
/// Resolve a movie to its full `sound.pak` voice-entry name for `lang`, using the
/// manifest for the *token* and `sounds.tbl` for the token's *directory* (Movie /
/// etc / Voice differ per token). `None` = the movie has no voice-over.
pub fn resolve_voice_entry(
manifest: &[u8],
sounds_tbl: &[u8],
movie: &str,
lang: VoiceLang,
) -> Option<String> {
let token = voice_token(manifest, movie)?;
resolve_token_path(sounds_tbl, &token, lang)
}
/// Find the full `<lang>\…\<token>.slb` path for a voice `token` in a decompressed
/// `sounds.tbl` (the token's subdir — `Movie`, `etc`, `Voice` — is not fixed).
pub fn resolve_token_path(sounds_tbl: &[u8], token: &str, lang: VoiceLang) -> Option<String> {
let prefix = format!("{}\\", lang.code_pub());
let suffix = format!("\\{token}.slb");
ascii_runs(sounds_tbl, 6)
.into_iter()
.find(|r| r.starts_with(&prefix) && r.ends_with(&suffix))
}
fn contains(hay: &[u8], needle: &[u8]) -> bool {
hay.windows(needle.len()).any(|w| w == needle)
}
/// Collect printable-ASCII runs of at least `min` chars.
fn ascii_runs(bytes: &[u8], min: usize) -> Vec<String> {
let mut out = Vec::new();
let mut cur = String::new();
for &b in bytes {
if (0x20..=0x7e).contains(&b) {
cur.push(b as char);
} else {
if cur.len() >= min {
out.push(std::mem::take(&mut cur));
} else {
cur.clear();
}
}
}
if cur.len() >= min {
out.push(cur);
}
out
}
#[cfg(test)]
mod tests {
use super::*;
fn synth() -> Vec<u8> {
// Minimal IDXD string pool: three movies — a standard one, a hokyu bound
// to a VOICE_D radio clip, and a hokyu with no voice.
let mut b = b"IDXD".to_vec();
b.extend_from_slice(&[0, 0, 0, 0x69, 0, 0]); // binary header (as on disc)
for s in [
"S13A.wmv",
"eng.pak+SUBTITLE_S13A.tbl",
"VOICE_S13A",
"hokyu_LS_s02A.wmv",
"eng.pak+SUBTITLE_hokyu_LS_s02A.tbl",
"VOICE_D_450",
"hokyu_DS_s13A.wmv",
"eng.pak+SUBTITLE_hokyu_DS_s13A.tbl",
] {
b.extend_from_slice(s.as_bytes());
b.push(0);
}
b
}
#[test]
fn groups_movies_and_binds_voice() {
// Keyless fallback path: movie + voice + subtitle + inferred kind.
let m = parse(&synth());
assert_eq!(m.len(), 3);
assert_eq!(m[0].movie, "S13A");
assert_eq!(m[0].voice_token.as_deref(), Some("VOICE_S13A"));
assert_eq!(m[0].kind, MovieKind::Intro);
assert_eq!(m[0].subtitle.as_deref(), Some("SUBTITLE_S13A.tbl"));
assert_eq!(m[1].voice_token.as_deref(), Some("VOICE_D_450"));
assert_eq!(m[1].kind, MovieKind::Supply);
assert_eq!(m[2].voice_token, None, "hokyu_DS_s13A has no voice-over");
assert_eq!(m[2].kind, MovieKind::Supply);
}
/// Build a two-array (keys-then-records) manifest like the real one.
fn synth_two_array(keys: &[&str], records: &[&[&str]]) -> Vec<u8> {
let mut b = b"IDXD".to_vec();
b.extend_from_slice(&[0, 0, 0, 0x05, 0, 0]);
for k in keys {
b.extend_from_slice(k.as_bytes());
b.push(0);
}
for rec in records {
for s in *rec {
b.extend_from_slice(s.as_bytes());
b.push(0);
}
}
b
}
#[test]
fn two_array_slots_kind_mission_phase() {
let blob = synth_two_array(
&["LOGO1", "MS01A", "STAGE01_PHASE01", "STAGE01_PHASE_END_02"],
&[
&["logo1.wmv", "MOVIE"],
&["S01A.wmv", "eng.pak+SUBTITLE_S01A.tbl", "VOICE_S01A"],
&["RT01A.wmv", "eng.pak+pwrt01.prt", "VOICE_RT01A", "VOICETRACK"],
&["RT01C_2.wmv", "VOICE_RT01C_2"],
],
);
let m = parse(&blob);
assert_eq!(m.len(), 4);
assert_eq!((m[0].slot.as_str(), m[0].kind), ("LOGO1", MovieKind::System));
assert_eq!(
(m[1].slot.as_str(), m[1].kind, m[1].mission),
("MS01A", MovieKind::Intro, Some(1))
);
assert_eq!(
(m[2].slot.as_str(), m[2].kind, m[2].mission, m[2].phase),
("STAGE01_PHASE01", MovieKind::Phase, Some(1), Some(1))
);
assert_eq!(m[2].telop.as_deref(), Some("pwrt01.prt"));
assert_eq!(
(m[3].slot.as_str(), m[3].kind, m[3].phase),
("STAGE01_PHASE_END_02", MovieKind::PhaseEnd, Some(2))
);
}
#[test]
fn valueless_supply_slot_is_skipped_via_ms_anchor() {
// S05_SUPPLY has no record; its "record" (S06A) belongs to the next MS slot.
let blob = synth_two_array(
&["LOGO1", "MS05A", "S05_SUPPLY_ACROPOLIS", "MS06A"],
&[
&["logo1.wmv"],
&["S05A.wmv", "VOICE_S05A"],
&["S06A.wmv", "VOICE_S06A"],
],
);
let m = parse(&blob);
assert_eq!(m.len(), 3, "the valueless supply slot yields no entry");
assert_eq!((m[1].slot.as_str(), m[1].movie.as_str()), ("MS05A", "S05A"));
// S06A binds to MS06A, NOT to the intervening valueless supply slot.
assert_eq!((m[2].slot.as_str(), m[2].movie.as_str()), ("MS06A", "S06A"));
assert!(m.iter().all(|e| e.slot != "S05_SUPPLY_ACROPOLIS"));
}
#[test]
fn resolves_token_directory_from_sounds_tbl() {
// sounds.tbl places the two tokens in different subdirs.
let mut tbl = Vec::new();
for s in ["eng\\Movie\\VOICE_S13A.slb", "eng\\etc\\VOICE_D_450.slb"] {
tbl.extend_from_slice(s.as_bytes());
tbl.push(0);
}
let man = synth();
assert_eq!(
resolve_voice_entry(&man, &tbl, "S13A", VoiceLang::English).as_deref(),
Some("eng\\Movie\\VOICE_S13A.slb")
);
assert_eq!(
resolve_voice_entry(&man, &tbl, "hokyu_LS_s02A", VoiceLang::English).as_deref(),
Some("eng\\etc\\VOICE_D_450.slb")
);
assert_eq!(
resolve_voice_entry(&man, &tbl, "hokyu_DS_s13A", VoiceLang::English),
None
);
}
}

454
crates/sylpheed-formats/src/movie_subtitle.rs Normal file
View File

@@ -0,0 +1,454 @@
//! Movie cutscene subtitles: the movie → track → text chain.
//!
//! Reverse-engineered statically (see `docs/re/structures/movie-subtitles.md`).
//! A cutscene's on-screen captions are assembled from three places on the disc:
//!
//! 1. `dat/movie/<lang>.pak` holds one **timing track** per movie, keyed by
//! [`track_key`] = `name_hash("subtitle_<movie>.tbl")`. Each track is a
//! `Z1`+zlib block that decompresses to an **IXUD** container whose UTF-16**LE**
//! payload is `SUBTITLE MSG_DEMO_<demo> <mm:ss.cc> …` — i.e. *which*
//! demo-message shows *when* (radio / resupply movies), or inline text.
//! 2. `dat/GP_MAIN_GAME_<L>.pak` holds the **caption text**: more `Z1`+zlib IXUD
//! blocks where each line is stored as `text` immediately followed by its key
//! `MSG_DEMO_<demo>_<page>_<line>` (multi-line captions split across lines).
//!
//! [`load`] joins the two into timed [`SubCue`]s.
//!
//! Note: the IXUD payload here is UTF-16 **little-endian** (verified on the real
//! disc); this is deliberately separate from the older big-endian [`crate::ixud`]
//! presenter, which targets a different (uncompressed) variant.
use std::collections::BTreeMap;
use crate::hash::name_hash;
use crate::pak::PakArchive;
/// Subtitle language. `pak_code` selects `dat/movie/<code>.pak`; `game_code`
/// selects `dat/GP_MAIN_GAME_<code>.pak` (the caption text).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum SubLang {
English,
Japanese,
German,
French,
Spanish,
Italian,
}
impl SubLang {
/// All languages, in menu order.
pub const ALL: [SubLang; 6] = [
SubLang::English,
SubLang::Japanese,
SubLang::German,
SubLang::French,
SubLang::Spanish,
SubLang::Italian,
];
/// `dat/movie/<code>.pak` stem (the timing tracks + caption font).
pub fn pak_code(self) -> &'static str {
match self {
SubLang::English => "eng",
SubLang::Japanese => "jpn",
SubLang::German => "deu",
SubLang::French => "fra",
SubLang::Spanish => "esp",
SubLang::Italian => "ita",
}
}
/// `dat/GP_MAIN_GAME_<code>.pak` suffix (the caption text pack).
pub fn game_code(self) -> &'static str {
match self {
SubLang::English => "E",
SubLang::Japanese => "J",
SubLang::German => "D",
SubLang::French => "F",
SubLang::Spanish => "S",
SubLang::Italian => "I",
}
}
/// Human label for a UI selector. Kept to Latin script so it renders in a
/// default (non-CJK) UI font; Japanese is romanized for the same reason.
pub fn label(self) -> &'static str {
match self {
SubLang::English => "English",
SubLang::Japanese => "Japanese",
SubLang::German => "Deutsch",
SubLang::French => "Français",
SubLang::Spanish => "Español",
SubLang::Italian => "Italiano",
}
}
}
/// One timed caption line.
#[derive(Debug, Clone, PartialEq)]
pub struct SubCue {
pub start: f32,
pub end: Option<f32>,
pub text: String,
}
/// The `<lang>.pak` TOC key for a movie's subtitle timing track.
pub fn track_key(movie_basename: &str) -> u32 {
name_hash(&format!("subtitle_{movie_basename}.tbl"))
}
/// Load and resolve a movie's timed captions in `lang`.
///
/// `lang_pak` is `dat/movie/<lang>.pak` (+segments); `text_pak` is
/// `dat/GP_MAIN_GAME_<L>.pak` (+segments). Returns the cues in start order, or an
/// empty vec if the movie has no timed track (e.g. story movies whose text is a
/// pre-rendered title card).
pub fn load(movie_basename: &str, lang_pak: &PakArchive, text_pak: &PakArchive) -> Vec<SubCue> {
let Some(Ok(track)) = lang_pak.read_by_hash(track_key(movie_basename)) else {
return Vec::new();
};
if !is_ixud(&track) {
return Vec::new();
}
let raw = parse_track(&track);
if raw.is_empty() {
return Vec::new();
}
// Only build the (large) text table if the track actually references demos.
let needs_text = raw.iter().any(|(t, _, _)| demo_ref(t).is_some());
let text = if needs_text {
build_demo_text(text_pak)
} else {
BTreeMap::new()
};
let mut cues = Vec::new();
for (token, start, end) in raw {
let resolved = match demo_ref(&token) {
Some(demo) => match text.get(&demo) {
Some(lines) => lines.join("\n"),
None => continue, // demo id with no text on disc → skip
},
None => clean(&token), // already inline text
};
if resolved.trim().is_empty() {
continue;
}
cues.push(SubCue {
start,
end,
text: resolved,
});
}
cues.sort_by(|a, b| a.start.partial_cmp(&b.start).unwrap_or(std::cmp::Ordering::Equal));
cues
}
/// The `MSG_DEMO_<id>` ids a movie's timing track references, in track order
/// (empty for inline-text or absent tracks). Language-independent — the same
/// demo ids appear in every `<lang>.pak`. Used to share a voice clip between
/// movies that play the same demo line (the resupply cutscenes reuse one clip
/// per line while only the video differs).
pub fn track_demo_ids(lang_pak: &PakArchive, movie_basename: &str) -> Vec<u32> {
let Some(Ok(track)) = lang_pak.read_by_hash(track_key(movie_basename)) else {
return Vec::new();
};
if !is_ixud(&track) {
return Vec::new();
}
parse_track(&track)
.iter()
.filter_map(|(t, _, _)| demo_ref(t))
.collect()
}
/// Per-line `(demo id, start seconds)` for a movie's timing track — the radio-box
/// voice schedule. Each `MSG_DEMO_<id>` reference takes the start time of the
/// timing that follows it (a caption may list two demo lines under one timing;
/// both then share that start). Empty for inline-text / absent tracks (those are
/// story movies whose voice is the single manifest `VOICETRACK`, not per-line).
pub fn track_voice_cues(lang_pak: &PakArchive, movie_basename: &str) -> Vec<(u32, f32)> {
let Some(Ok(track)) = lang_pak.read_by_hash(track_key(movie_basename)) else {
return Vec::new();
};
if !is_ixud(&track) {
return Vec::new();
}
let toks = utf16le_tokens(&track);
let start = toks
.iter()
.position(|t| t.ends_with("SUBTITLE"))
.map(|i| i + 1)
.unwrap_or(0);
let mut out = Vec::new();
let mut pending: Vec<u32> = Vec::new();
for tok in &toks[start..] {
match parse_timing(tok) {
Some((s, _)) => {
for d in pending.drain(..) {
out.push((d, s));
}
}
None => {
if let Some(d) = demo_ref(tok) {
pending.push(d);
}
}
}
}
out
}
/// Build `demo id → ordered caption lines` from a `GP_MAIN_GAME_<L>` pack.
///
/// Scans every IXUD block, pairing each text token with the immediately
/// following `MSG_DEMO_<demo>_<page>_<line>` key, and groups the lines per demo
/// ordered by (page, line).
pub fn build_demo_text(text_pak: &PakArchive) -> BTreeMap<u32, Vec<String>> {
// demo -> (page,line) -> text
let mut by_demo: BTreeMap<u32, BTreeMap<(u32, u32), String>> = BTreeMap::new();
for entry in text_pak.entries() {
let Ok(bytes) = text_pak.read(entry) else {
continue;
};
if !is_ixud(&bytes) {
continue;
}
let toks = utf16le_tokens(&bytes);
for w in toks.windows(2) {
if let Some((demo, page, line)) = text_key(&w[1]) {
// Pair only when the preceding token is real text — not another
// MSG_DEMO key (bare keys are also serialized consecutively in the
// record directory, which would otherwise masquerade as text).
if demo_ref(&w[0]).is_none()
&& text_key(&w[0]).is_none()
&& !w[0].trim().is_empty()
{
by_demo
.entry(demo)
.or_default()
.insert((page, line), clean(&w[0]));
}
}
}
}
by_demo
.into_iter()
.map(|(d, m)| (d, m.into_values().collect()))
.collect()
}
/// Parse a timing track's IXUD payload into `(token, start, end)` triples, where
/// `token` is either a `MSG_DEMO_<d>` reference or an inline caption string.
///
/// A single on-screen caption may be stored as **several consecutive text
/// tokens** followed by one timing (the source hard-splits a multi-line caption
/// into one token per line — e.g. `"Look at it father"` + `"& beautiful isn't
/// it"` share the `01:14.80-01:17.60` timing). We therefore accumulate every
/// text token seen since the last timing and join them with a newline when the
/// timing arrives, instead of pairing strictly 1:1 (which silently dropped every
/// line but the last of a multi-line caption).
fn parse_track(ixud: &[u8]) -> Vec<(String, f32, Option<f32>)> {
let toks = utf16le_tokens(ixud);
let start = toks
.iter()
.position(|t| t.ends_with("SUBTITLE"))
.map(|i| i + 1)
.unwrap_or(0);
let rest = &toks[start..];
let mut out = Vec::new();
let mut pending: Vec<&str> = Vec::new();
for tok in rest {
match parse_timing(tok) {
Some((s, e)) => {
if !pending.is_empty() {
out.push((pending.join("\n"), s, e));
pending.clear();
}
}
None => pending.push(tok),
}
}
out
}
fn is_ixud(b: &[u8]) -> bool {
b.len() >= 4 && &b[0..4] == b"IXUD"
}
/// Normalize a caption string: the source stores line breaks as the literal
/// two-character escape `\n`; turn it into a real newline and trim.
fn clean(s: &str) -> String {
s.replace("\\n", "\n").trim().to_string()
}
/// `MSG_DEMO_<d>` → `Some(d)` (reference token, no page/line).
fn demo_ref(t: &str) -> Option<u32> {
let rest = t.strip_prefix("MSG_DEMO_")?;
if rest.contains('_') {
return None; // that's a text key (has page/line), not a bare reference
}
rest.parse().ok()
}
/// `MSG_DEMO_<d>_<page>_<line>` → `Some((d,page,line))` (text key).
fn text_key(t: &str) -> Option<(u32, u32, u32)> {
let rest = t.strip_prefix("MSG_DEMO_")?;
let mut it = rest.split('_');
let d = it.next()?.parse().ok()?;
let p = it.next()?.parse().ok()?;
let l = it.next()?.parse().ok()?;
if it.next().is_some() {
return None;
}
Some((d, p, l))
}
/// Extract UTF-16**LE** text runs from a blob, **independent of byte alignment**.
///
/// The IXUD string pool packs records at odd byte offsets, so a fixed 2-byte
/// stride from the block start misreads every character. Instead we scan a
/// sliding window, decoding each 16-bit LE unit and keeping runs of "text-like"
/// code points (ASCII, Latin-1 supplement — the German umlauts / accented
/// Latin — and Latin Extended-A). A non-text unit ends the run and we advance by
/// one byte to re-lock onto the correct parity of the next run. Accepting the
/// Latin ranges (not just ASCII) is what keeps `müsste`, `Français`, `español`
/// intact — earlier the accented char *and its predecessor* were dropped.
///
/// CJK (Japanese) is deliberately out of range: those units also occur as noise
/// at the wrong parity, and the UI font can't render them anyway.
fn utf16le_tokens(bytes: &[u8]) -> Vec<String> {
let mut tokens = Vec::new();
let mut cur = String::new();
let mut i = 0;
while i + 1 < bytes.len() {
let u = u16::from_le_bytes([bytes[i], bytes[i + 1]]);
if is_text_unit(u) {
if let Some(c) = char::from_u32(u as u32) {
cur.push(c);
}
i += 2;
} else {
if cur.chars().count() >= 2 {
tokens.push(std::mem::take(&mut cur));
} else {
cur.clear();
}
i += 1;
}
}
if cur.chars().count() >= 2 {
tokens.push(cur);
}
tokens
}
/// Whether a UTF-16 unit is a printable Latin text character we keep in a run.
/// Excludes the C0/C1 control blocks (incl. 0x7F–0x9F) so binary noise breaks
/// runs instead of joining them.
fn is_text_unit(u: u16) -> bool {
matches!(u, 0x20..=0x7e | 0xa0..=0xff | 0x100..=0x17f)
}
/// Parse `MM:SS.ss` or a `start-end` range into seconds.
fn parse_timing(s: &str) -> Option<(f32, Option<f32>)> {
let one = |p: &str| -> Option<f32> {
let (mm, ss) = p.trim().split_once(':')?;
let m: f32 = if mm.trim().is_empty() {
0.0
} else {
mm.trim().parse().ok()?
};
let sec: f32 = ss.trim().parse().ok()?;
Some(m * 60.0 + sec)
};
match s.split_once('-') {
Some((a, b)) => Some((one(a)?, Some(one(b)?))),
None => Some((one(s)?, None)),
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn track_key_matches_disc() {
// Verified against dat/movie/eng.pak.
assert_eq!(track_key("S00A"), 0x6F2D_9663);
assert_eq!(track_key("hokyu_DS_s02A"), 0x3662_B1F8);
assert_eq!(track_key("RT01C_1"), 0x756F_69FB);
}
#[test]
fn demo_and_text_keys() {
assert_eq!(demo_ref("MSG_DEMO_192"), Some(192));
assert_eq!(demo_ref("MSG_DEMO_604_000_00"), None);
assert_eq!(text_key("MSG_DEMO_604_000_01"), Some((604, 0, 1)));
assert_eq!(text_key("MSG_DEMO_192"), None);
}
#[test]
fn keeps_latin1_accents() {
// "müsste" must survive intact (umlaut + its neighbour), not collapse to
// "sste". Encode as UTF-16LE at an ODD start offset to exercise the
// alignment-independent scan.
let mut b = vec![0xAAu8]; // 1 junk byte → strings start at an odd offset
for s in ["müsste", "Français", "español"] {
for u in s.encode_utf16() {
b.extend_from_slice(&u.to_le_bytes());
}
b.extend_from_slice(&[0, 0]);
}
let toks = utf16le_tokens(&b);
assert!(toks.contains(&"müsste".to_string()), "got {toks:?}");
assert!(toks.contains(&"Français".to_string()), "got {toks:?}");
assert!(toks.contains(&"español".to_string()), "got {toks:?}");
}
/// Build a synthetic UTF-16LE IXUD and check the LE token walk + timing pair.
#[test]
fn parses_le_track() {
let mut b = b"IXUD".to_vec();
b.extend_from_slice(&[0, 1, 0, 0, 0x70, 0x3E, 0xC8, 0x6C]);
for t in ["SUBTITLE", "MSG_DEMO_5", "00:01.50", "Hello", "00:03.00"] {
for u in t.encode_utf16() {
b.extend_from_slice(&u.to_le_bytes());
}
b.extend_from_slice(&[0, 0]);
}
let raw = parse_track(&b);
assert_eq!(raw.len(), 2);
assert_eq!(raw[0], ("MSG_DEMO_5".to_string(), 1.5, None));
assert_eq!(raw[1], ("Hello".to_string(), 3.0, None));
}
/// Two text tokens before a single timing = one two-line caption; the first
/// line must NOT be dropped (the real S13A `Look at it father` regression).
#[test]
fn joins_multiline_caption() {
let mut b = b"IXUD".to_vec();
b.extend_from_slice(&[0, 1, 0, 0, 0x70, 0x3E, 0xC8, 0x6C]);
for t in [
"SUBTITLE",
"That is such magnificent power.",
"01:09.70-01:12.20",
"Look at it father",
"& beautiful isn't it",
"01:14.80-01:17.60",
] {
for u in t.encode_utf16() {
b.extend_from_slice(&u.to_le_bytes());
}
b.extend_from_slice(&[0, 0]);
}
let raw = parse_track(&b);
assert_eq!(raw.len(), 2);
assert_eq!(raw[0].0, "That is such magnificent power.");
assert_eq!(
raw[1].0, "Look at it father\n& beautiful isn't it",
"both lines of the caption must be kept"
);
assert_eq!((raw[1].1, raw[1].2), (74.8, Some(77.6)));
}
}

142
crates/sylpheed-formats/src/movie_voice.rs Normal file
View File

@@ -0,0 +1,142 @@
//! Cutscene-voice resolution: movie → sound-id → continuous-stream byte region.
//!
//! Reverse-engineered 2026-07-22 (Canary file-I/O trace, user-confirmed by ear).
//! The movie voices form **one continuous XMA stream**, physically chunked into
//! `<lang>\Movie\VOICE_*.slb` `sound.pak` TOC entries whose boundaries do **not**
//! align with the cutscene cues — a single cue routinely spans two `.slb` chunks,
//! so a `.slb` does not necessarily hold the track its name claims. Each cue's
//! audio ends at an inline **trailer descriptor** `(sound_id: u32be, 0x11, …)`
//! whose id repeats at +0x800. Cue N's audio = the stream bytes
//! `[descriptor(N-1) .. descriptor(N)]`, read continuously across chunk
//! boundaries. Verified: decoded voice length matches each movie within ~0.3 s,
//! including cross-boundary cues (RT01B/RT01C span `VOICE_ADV`→`VOICE_RT01A`→…).
//!
//! Resolution chain:
//! 1. movie → cue name (`ADVERTISE_MOVIE` manifest `VOICETRACK`, e.g. `VOICE_RT01A`)
//! 2. cue name → sound-id (master sound registry, [`registry_voice_ids`], → 1601)
//! 3. sound-id → `[start, end]` global offsets via [`find_descriptor`] over the
//! `sound.pNN` stream (the caller supplies the bytes + the physical anchor).
use std::collections::HashMap;
/// Parse the master sound registry — the large per-language `tables.pak` IDXD
/// entry (schema `0x13cb84ba`) that carries the `<lang>\Movie\VOICE_*.slb` paths —
/// into `cue-name → sound-id` from its `NAME\0 ID\0` string pool
/// (e.g. `VOICE_ADV`→1600, `VOICE_RT01A`→1601).
pub fn registry_voice_ids(registry: &[u8]) -> HashMap<String, u32> {
let mut toks: Vec<&[u8]> = Vec::new();
let mut start = 0usize;
for i in 0..registry.len() {
if registry[i] == 0 {
if i > start {
toks.push(&registry[start..i]);
}
start = i + 1;
}
}
let mut out = HashMap::new();
for w in toks.windows(2) {
let (name, num) = (w[0], w[1]);
if name.starts_with(b"VOICE_")
&& !num.is_empty()
&& num.iter().all(u8::is_ascii_digit)
{
if let (Ok(n), Ok(id)) = (
std::str::from_utf8(name),
std::str::from_utf8(num).unwrap().parse::<u32>(),
) {
out.entry(n.to_string()).or_insert(id);
}
}
}
out
}
/// The constant marker following a cue's sound-id in its inline trailer.
const DESC_MARK: u32 = 0x11;
/// The trailer repeats its id this many bytes later; used to reject a coincidental
/// `(id, 0x11)` pair inside XMA audio (probability of a false match is ~2⁻⁶⁴).
const DESC_REPEAT: usize = 0x800;
/// Byte offset (within `buf`, a slice of the continuous voice stream) of cue
/// `id`'s trailer descriptor: the first big-endian `(id, 0x11)` pair whose id
/// repeats at `+0x800`. `None` if absent in the slice.
pub fn find_descriptor(buf: &[u8], id: u32) -> Option<usize> {
if buf.len() < DESC_REPEAT + 4 {
return None;
}
let be = |o: usize| u32::from_be_bytes([buf[o], buf[o + 1], buf[o + 2], buf[o + 3]]);
let end = buf.len() - (DESC_REPEAT + 4);
let mut o = 0;
while o <= end {
if be(o) == id && be(o + 4) == DESC_MARK && be(o + DESC_REPEAT) == id {
return Some(o);
}
o += 4;
}
None
}
/// Plausible sound-id range for a trailer (all real cue ids are well under this).
const ID_MAX: u32 = 0x1_0000;
/// Offset of the nearest cue trailer strictly **before** `before` — any plausible
/// id. Used to bound a cue's START when its `id-1` predecessor doesn't exist
/// (the sound-id sequence has gaps, e.g. `VOICE_D_453`=7142 → `VOICE_D_454`=7188).
/// Scans downward and returns the first (largest-offset) valid `(id, 0x11)` with
/// the `+0x800` id-repeat. `None` if there is no trailer below `before`.
pub fn find_descriptor_before(buf: &[u8], before: usize) -> Option<usize> {
if before < 4 {
return None;
}
let cap = buf.len().checked_sub(DESC_REPEAT + 4)?;
let be = |o: usize| u32::from_be_bytes([buf[o], buf[o + 1], buf[o + 2], buf[o + 3]]);
// Start strictly below `before` (4-aligned) so the target's own trailer is
// never returned as its predecessor.
let mut o = (before - 1).min(cap) & !3;
loop {
let id = be(o);
if id >= 1 && id < ID_MAX && be(o + 4) == DESC_MARK && be(o + DESC_REPEAT) == id {
return Some(o);
}
if o < 4 {
return None;
}
o -= 4;
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn registry_parses_name_id_pairs() {
let mut b = Vec::new();
for tok in [
b"VOICE_ADV".as_slice(),
b"1600",
b"VOICE_RT01A",
b"1601",
b"NOT_A_VOICE",
b"9",
] {
b.extend_from_slice(tok);
b.push(0);
}
let ids = registry_voice_ids(&b);
assert_eq!(ids.get("VOICE_ADV"), Some(&1600));
assert_eq!(ids.get("VOICE_RT01A"), Some(&1601));
assert_eq!(ids.get("NOT_A_VOICE"), None);
}
#[test]
fn find_descriptor_matches_repeat() {
let mut b = vec![0u8; 0x900];
b[0..4].copy_from_slice(&1601u32.to_be_bytes());
b[4..8].copy_from_slice(&0x11u32.to_be_bytes());
b[DESC_REPEAT..DESC_REPEAT + 4].copy_from_slice(&1601u32.to_be_bytes());
assert_eq!(find_descriptor(&b, 1601), Some(0));
assert_eq!(find_descriptor(&b, 1600), None);
}
}

47
crates/sylpheed-formats/src/pak.rs
View File

@@ -178,6 +178,18 @@ impl PakArchive {
})
}
/// Parse **only** the TOC from a `.pak` index, without any segment data.
///
/// For huge archives (`sound.pak` is ~1 GB across `.p00..p04`) this lets a
/// caller locate one entry's `(offset, size)` and read just that byte range
/// from the segments, instead of loading the whole thing into memory. The
/// returned entries are sorted by `name_hash` (binary-searchable).
pub fn parse_toc(index: &[u8]) -> Result<Vec<PakEntry>, PakError> {
// Reuse from_parts' validation with an empty data blob; the entries are
// independent of the data.
Ok(Self::from_parts(index, Vec::new())?.entries)
}
/// All TOC entries, in stored order (ascending `name_hash`).
pub fn entries(&self) -> &[PakEntry] {
&self.entries
@@ -262,6 +274,41 @@ fn be32(b: &[u8], o: usize) -> u32 {
u32::from_be_bytes([b[o], b[o + 1], b[o + 2], b[o + 3]])
}
/// Best-effort human label for a decompressed entry's inner format, from its
/// first bytes: a known signature (`IDXD`, fonts, `png`, …), a printable 4-char
/// tag (RATC, IXUD, LSTA, …), or a hex fallback for still-unidentified magics.
/// Shared by the CLI `pak list` and the GUI pack browser.
pub fn inner_format_label(payload: &[u8]) -> String {
if payload.len() < 4 {
return "empty".into();
}
let m = &payload[0..4];
// Known binary/text signatures → friendly short tags (so fonts/images don't
// show as bare hex like `00010000` / `89504E47`).
let known = match m {
b"IDXD" => Some("IDXD"),
b"\x89PNG" => Some("png"),
b"\x00\x01\x00\x00" | b"true" | b"typ1" => Some("ttf"), // sfnt TrueType
b"OTTO" => Some("otf"), // OpenType (CFF)
b"ttcf" => Some("ttc"), // TrueType Collection
b"RIFF" => Some("riff"),
b"DDS " => Some("dds"),
_ if payload.starts_with(b"<?xml") => Some("xml"),
_ => None,
};
if let Some(tag) = known {
return tag.into();
}
if m.iter().all(|&b| b.is_ascii_graphic()) {
// Many inner formats are 4-char tags (RATC, T8aD, IXUD, LSTA, …).
String::from_utf8_lossy(m).into_owned()
} else {
format!("{:02X}{:02X}{:02X}{:02X}", m[0], m[1], m[2], m[3])
}
}
#[cfg(test)]
mod tests {
use super::*;

131
crates/sylpheed-formats/src/ratc.rs Normal file
View File

@@ -0,0 +1,131 @@
//! `RATC` — a nested resource bundle.
//!
//! After a small header, a RATC holds named children — `foo.t32` (T8aD
//! textures), `foo.rat` (nested RATC), fonts, PNG — each immediately preceded by
//! its ASCII name string. The children are self-locating by their 4-char magic,
//! so we list them by scanning for those magics and pairing each with the name
//! run that precedes it. This is reliable for *listing* (names / types / sizes
//! are literal bytes); decoding a child's pixels is delegated to that child's
//! own parser ([`crate::t8ad`]).
/// A child resource inside a RATC bundle.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct RatcChild {
/// Name string preceding the child (e.g. `prselectbtn_g04b.t32`), or empty.
pub name: String,
/// Child magic tag: `T8aD`, `RATC`, `ttcf`, `png`, …
pub kind: String,
/// Byte offset of the child (its magic) within the RATC payload.
pub offset: usize,
/// Byte length of the child, up to the next child (or end of payload).
pub size: usize,
}
/// Magic at the start of a RATC bundle.
pub const RATC_MAGIC: [u8; 4] = *b"RATC";
/// Whether `bytes` is a RATC bundle.
pub fn is_ratc(bytes: &[u8]) -> bool {
bytes.len() >= 4 && bytes[0..4] == RATC_MAGIC
}
/// Recognized child magics and their display tag.
fn child_kind(m: &[u8]) -> Option<&'static str> {
match m {
b"T8aD" => Some("T8aD"),
b"RATC" => Some("RATC"),
b"ttcf" => Some("ttc"),
b"\x89PNG" => Some("png"),
_ => None,
}
}
/// List the children of a RATC bundle (the self-magic at offset 0 is skipped).
pub fn parse(bytes: &[u8]) -> Option<Vec<RatcChild>> {
if !is_ratc(bytes) {
return None;
}
// Offsets of every recognized child magic (skip the self RATC at 0).
let mut offs: Vec<(usize, &'static str)> = Vec::new();
let mut i = 4;
while i + 4 <= bytes.len() {
if let Some(kind) = child_kind(&bytes[i..i + 4]) {
offs.push((i, kind));
i += 4;
} else {
i += 1;
}
}
let mut children = Vec::with_capacity(offs.len());
for (idx, &(off, kind)) in offs.iter().enumerate() {
let next = offs.get(idx + 1).map(|&(o, _)| o).unwrap_or(bytes.len());
children.push(RatcChild {
name: name_before(bytes, off),
kind: kind.to_string(),
offset: off,
size: next.saturating_sub(off),
});
}
Some(children)
}
/// The nearest name string preceding `off`: the *last* printable run (len ≥ 3)
/// in the 96 bytes before the child magic. A few record-header bytes usually sit
/// between the name and the magic, so an exact-adjacency scan isn't enough.
fn name_before(bytes: &[u8], off: usize) -> String {
let start = off.saturating_sub(96);
let window = &bytes[start..off];
let mut best = String::new();
let mut run_start: Option<usize> = None;
let flush = |from: usize, to: usize, best: &mut String| {
if to - from >= 3 {
*best = String::from_utf8_lossy(&window[from..to]).trim().to_string();
}
};
for (i, &c) in window.iter().enumerate() {
if (0x20..=0x7e).contains(&c) {
run_start.get_or_insert(i);
} else if let Some(s) = run_start.take() {
flush(s, i, &mut best);
}
}
if let Some(s) = run_start {
flush(s, window.len(), &mut best);
}
best
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn lists_named_children() {
let mut b = RATC_MAGIC.to_vec();
b.extend_from_slice(&[0u8; 28]); // header padding
b.extend_from_slice(b"logo.t32");
let t8_off = b.len();
b.extend_from_slice(b"T8aD");
b.extend_from_slice(&[0u8; 40]); // some child bytes
b.extend_from_slice(b"sub.rat");
let ratc_off = b.len();
b.extend_from_slice(b"RATC");
b.extend_from_slice(&[0u8; 8]);
let kids = parse(&b).unwrap();
assert_eq!(kids.len(), 2);
assert_eq!(kids[0].kind, "T8aD");
assert_eq!(kids[0].name, "logo.t32");
assert_eq!(kids[0].offset, t8_off);
assert_eq!(kids[0].size, ratc_off - t8_off);
assert_eq!(kids[1].kind, "RATC");
assert_eq!(kids[1].name, "sub.rat");
}
#[test]
fn rejects_non_ratc() {
assert!(parse(b"T8aD....").is_none());
}
}

394
crates/sylpheed-formats/src/ship.rs Normal file
View File

@@ -0,0 +1,394 @@
//! Whole-ship assembly from XBG7 part families.
//!
//! Project Sylpheed's capital ships are never stored as a single mesh. A ship is
//! modelled, then **split into destructible parts** — hull segments, bridge,
//! engines, integral turrets — each shipped as its own XBG7 resource inside a
//! `hidden/resource3d/Stage_SNN.xpr` container. The split lets the game hide a
//! part when the player destroys it (bridge, shield generator, thruster) and swap
//! in a `_break` variant.
//!
//! Each part's vertices are authored around the origin; the world placement lives
//! in the **primary composite scene graph** `e_rou_<id>` (see [`assemble_ship`]).
//! Its node hierarchy (`rou_<id>_root`→`…_front`/`…_rear`→`rou_<id>_bdy_NN`) names
//! the separate hull resources and carries their world TRS — the cross-resource
//! analogue of how [`mesh::node_transforms`] poses one model's sub-parts. The
//! `GN_*` nodes in the same composite are the Vessel hardpoint frames (bridge /
//! shield / engine / turret mounts, matching the IDXD recipe `Frame` names).
//!
//! Resource naming (learned from the retail stage containers):
//! - `<id>` is `[a-z]NNN` (`e106`, `f105`, `n050`). The leading letter is the
//! faction: `e` = ADAN (enemy), `f` = TCAF (player side), `n` = neutral /
//! structures / debris.
//! - A base part is `<id>_<name>` (`e106_bdy_01`, `e106_brg_01`, `f105_sld_02`).
//! - `_l` / `_m` / `_d`(`NN`) suffixes are lower level-of-detail copies; `_bNN` /
//! `_dead` / `_break` are destruction geometry; `_joint` / `_open` / `_Near` are
//! logic/animation nodes. All are skipped.
//! - `e_rou_<id>` is the primary composite; `e_rou_<id>_eng` / `_wep_*` are LOCAL
//! detail rigs (their nodes are in their own frame, not ship-space) and are NOT
//! used for placement.
use crate::mesh::{scene_world_nodes, xbg7_resource_names, ScenePart};
use std::collections::HashSet;
/// Which side a ship belongs to, from the first letter of its resource id.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Faction {
/// `e…` — ADAN (the enemy).
Adan,
/// `f…` — TCAF (the player's coalition).
Tcaf,
/// `n…` — neutral: stations, structures, asteroids, debris.
Neutral,
/// Anything else (`j…`, `s…`, …).
Other,
}
impl Faction {
fn from_id(id: &str) -> Faction {
match id.as_bytes().first() {
Some(b'e') => Faction::Adan,
Some(b'f') => Faction::Tcaf,
Some(b'n') => Faction::Neutral,
_ => Faction::Other,
}
}
/// Short human label.
pub fn label(self) -> &'static str {
match self {
Faction::Adan => "ADAN",
Faction::Tcaf => "TCAF",
Faction::Neutral => "Neutral",
Faction::Other => "Other",
}
}
}
/// A ship reconstructed from one XBG7 part family in a stage container.
#[derive(Debug, Clone)]
pub struct ShipModel {
/// The family id, e.g. `e106` — matches a [`crate::game_data::Vessel`] whose
/// `model` is `rou_e106`.
pub id: String,
/// Faction inferred from the id's leading letter.
pub faction: Faction,
/// Base geometry resource names to draw together, in directory order.
pub parts: Vec<String>,
}
/// The `[a-z]NNN` id prefix of a resource name, if it has one (`e106_bdy_01`
/// → `e106`). Returns `None` for scene/logic resources (`e_rou_e106`) and
/// anything not starting with a letter + three digits.
pub fn ship_id_of(resource: &str) -> Option<&str> {
let b = resource.as_bytes();
if b.len() >= 4
&& b[0].is_ascii_lowercase()
&& b[1].is_ascii_digit()
&& b[2].is_ascii_digit()
&& b[3].is_ascii_digit()
{
// Reject a longer alnum run (`e1234…`): the id is exactly letter+3 digits,
// followed by end-of-string or a non-alphanumeric (`_`).
if b.get(4).is_none_or(|c| !c.is_ascii_alphanumeric()) {
return Some(&resource[..4]);
}
}
None
}
/// Whether a resource is a base geometry part (not a LOD copy, destruction
/// variant, or scene/logic node) that should be drawn in a whole-ship render.
pub fn is_base_part(resource: &str) -> bool {
if ship_id_of(resource).is_none() {
return false;
}
// Level-of-detail copies of a base part: a trailing `_l`/`_m`/`_d` token,
// optionally with a number (`_d01`, `_l02`).
if let Some(last) = resource.rsplit('_').next() {
let mut cs = last.chars();
if matches!(cs.next(), Some('l' | 'm' | 'd')) && cs.all(|c| c.is_ascii_digit()) {
return false;
}
}
// Destruction / logic geometry.
if resource.contains("break")
|| resource.contains("dead")
|| resource.contains("_joint")
|| resource.contains("_open")
|| resource.contains("_Near")
{
return false;
}
// Break-piece geometry carries a bare `b` / `bNN` token (`e106_brg_01_b_02`,
// `e108_eng_01_b01`). `bdy` etc. are safe — only an exact `b`-then-digits token
// counts.
if resource.split('_').skip(1).any(|t| {
t == "b" || (t.starts_with('b') && t.len() >= 2 && t[1..].bytes().all(|c| c.is_ascii_digit()))
}) {
return false;
}
true
}
/// Group every base part in an XPR2 container into whole ships, keyed by id.
///
/// Ships are returned sorted by id. A stage container also holds shared turret /
/// prop models as their own ids — every group with at least one base part is
/// returned; the caller can filter by [`ShipModel::parts`] length or faction.
pub fn ships_in_container(bytes: &[u8]) -> Vec<ShipModel> {
let names = xbg7_resource_names(bytes);
let mut ids: Vec<String> = Vec::new();
let mut ships: std::collections::HashMap<String, ShipModel> = std::collections::HashMap::new();
for n in &names {
if !is_base_part(n) {
continue;
}
let id = ship_id_of(n).unwrap().to_string();
let entry = ships.entry(id.clone()).or_insert_with(|| {
ids.push(id.clone());
ShipModel { id: id.clone(), faction: Faction::from_id(&id), parts: Vec::new() }
});
entry.parts.push(n.clone());
}
ids.sort();
ids.into_iter().map(|id| ships.remove(&id).unwrap()).collect()
}
/// Map a composite scene-graph node name to the geometry resource it draws.
/// Hull nodes are `rou_<id>_<part>[_root|_mov|_NN]`; the geometry resource is the
/// `<id>_<part>` stem. Structural nodes (`rou_e106_front`, `…_root`) resolve to
/// nothing and only pose their children.
fn resolve_hull_resource(node: &str, resources: &HashSet<String>) -> Option<String> {
let stem = node.rsplit_once("rou_").map(|(_, s)| s).unwrap_or(node);
if resources.contains(stem) {
return Some(stem.to_string());
}
for suf in ["_root", "_mov"] {
if let Some(s) = stem.strip_suffix(suf) {
if resources.contains(s) {
return Some(s.to_string());
}
}
}
// Trailing `_NN` instance index (`eng_01_1` → `eng_01`).
if let Some(pos) = stem.rfind('_') {
if !stem[pos + 1..].is_empty() && stem[pos + 1..].bytes().all(|c| c.is_ascii_digit()) {
let s = &stem[..pos];
if resources.contains(s) {
return Some(s.to_string());
}
}
}
None
}
/// The trailing digit run of a name (`GN_Bridge_01` → `01`, `sld_02` → `02`).
fn trailing_index(name: &str) -> &str {
let bytes = name.as_bytes();
let mut start = bytes.len();
while start > 0 && bytes[start - 1].is_ascii_digit() {
start -= 1;
}
&name[start..]
}
/// Which stage-container resources are the composite scene graphs for ship `id`
/// (`e_rou_e106`, `e_rou_e106_eng`, …). Excludes destruction (`_break`) scenes.
fn composites_for<'a>(names: &'a [String], id: &str) -> Vec<&'a String> {
let needle = format!("rou_{id}");
names
.iter()
.filter(|n| {
n.contains(&needle)
&& ship_id_of(n).is_none()
&& !n.contains("break")
&& !n.contains("dead")
})
.collect()
}
/// Reconstruct a whole ship as a list of geometry-resource placements in a shared
/// ship-local frame — the placement the game builds at runtime.
///
/// Two tiers, matching how the game splits a ship:
/// 1. **Hull** — `rou_<id>_*` nodes in the primary composite scene graph name the
/// hull body resources and carry their world transform. This tier is exact: the
/// bodies (and any engine posed by a `rou_` node) land where the game puts them.
/// 2. **External hardpoints** — `GN_*` frame nodes are the Vessel mounts; each
/// unplaced bridge / shield-generator / engine part (`<id>_brg_*`, `_sld_*`,
/// `_eng_*`) is placed at its matching frame by category + index. This tier is
/// APPROXIMATE: a `GN_*` frame is the mount *pivot* (often on the centreline),
/// while the part's outboard/rotational offset lives in a detail sub-rig or in
/// runtime code that this static pass does not recover — so external parts land
/// near, not exactly, where they belong. Gated by `include_external`.
///
/// Turret placement (shared `e3NN`/`e4NN` gun models mounted at many `GN_*Gun*`
/// frames) needs the per-mount Vessel recipe and is not resolved at all.
pub fn assemble_ship(bytes: &[u8], id: &str, include_external: bool) -> Vec<ScenePart> {
let names = xbg7_resource_names(bytes);
let resources: HashSet<String> = names.iter().cloned().collect();
// The PRIMARY composite carries the real ship-space layout (hull bodies +
// every `GN_*` hardpoint frame). Sibling composites (`e_rou_<id>_eng`,
// `_wep_*`) are LOCAL detail rigs — their nodes are in their own frame, not
// ship-space, so folding them in floats parts mid-ship. Pick the composite
// with the most nodes; that is the assembled hull.
let scenes: Vec<(String, Vec<ScenePart>)> = composites_for(&names, id)
.into_iter()
.map(|c| (c.clone(), scene_world_nodes(bytes, c)))
.collect();
let Some((_, nodes)) = scenes.iter().max_by_key(|(_, n)| n.len()) else {
// No composite at all → raw fallback below.
let mut placed = Vec::new();
const ID: [[f32; 3]; 3] = [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]];
for part in names.iter().filter(|n| is_base_part(n) && ship_id_of(n) == Some(id)) {
placed.push(ScenePart { resource: part.clone(), m: ID, t: [0.0; 3], s: [1.0; 3] });
}
return placed;
};
let mut placed: Vec<ScenePart> = Vec::new();
let mut placed_res: HashSet<String> = HashSet::new();
// GN hardpoint frames, world-space, from the primary composite.
let mut frames: Vec<ScenePart> = Vec::new();
for node in nodes {
if node.resource.starts_with("GN_") {
frames.push(node.clone());
} else if let Some(res) = resolve_hull_resource(&node.resource, &resources) {
// Only the ship's own parts. A composite also mounts shared, cross-id
// turret models (`e303_wep_*` on an `e106` hull); those need the
// per-mount Vessel recipe and are placed separately.
if ship_id_of(&res) != Some(id) {
continue;
}
if placed_res.insert(res.clone()) {
placed.push(ScenePart { resource: res, m: node.m, t: node.t, s: node.s });
}
}
}
// Place unplaced external parts at their Vessel hardpoint frame (APPROXIMATE —
// see the doc comment). Skipped for a clean, exact hull-only render.
const CATS: [(&str, &str); 3] = [("brg", "Bridge"), ("sld", "ShieldG"), ("eng", "Engine")];
for part in names
.iter()
.filter(|_| include_external)
.filter(|n| is_base_part(n) && ship_id_of(n) == Some(id))
{
if placed_res.contains(part) {
continue;
}
let rest = &part[(id.len() + 1).min(part.len())..]; // "brg_01"
let mut toks = rest.split('_');
let cat = toks.next().unwrap_or("");
let idx = toks.next().unwrap_or("");
let Some((_, gncat)) = CATS.iter().find(|(c, _)| *c == cat) else {
continue;
};
if let Some(frame) =
frames.iter().find(|f| f.resource.contains(gncat) && trailing_index(&f.resource) == idx)
{
placed.push(ScenePart { resource: part.clone(), m: frame.m, t: frame.t, s: frame.s });
placed_res.insert(part.clone());
}
}
// Fallback for a ship with no composite scene graph (a simple prop): draw its
// base parts untransformed rather than nothing.
if placed.is_empty() {
const ID: [[f32; 3]; 3] = [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]];
for part in names.iter().filter(|n| is_base_part(n) && ship_id_of(n) == Some(id)) {
placed.push(ScenePart { resource: part.clone(), m: ID, t: [0.0; 3], s: [1.0; 3] });
}
}
placed
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn id_extraction() {
assert_eq!(ship_id_of("e106_bdy_01"), Some("e106"));
assert_eq!(ship_id_of("f105_sld_02_l"), Some("f105"));
assert_eq!(ship_id_of("e_rou_e106"), None);
assert_eq!(ship_id_of("_rou_e106_break"), None);
assert_eq!(ship_id_of("Base"), None);
}
#[test]
fn base_part_filter() {
assert!(is_base_part("e106_bdy_01"));
assert!(is_base_part("e106_brg_01"));
assert!(is_base_part("f105_wep_02_01"));
assert!(!is_base_part("e106_bdy_01_l")); // LOD
assert!(!is_base_part("e106_bdy_01_m")); // LOD
assert!(!is_base_part("e106_brg_01_b_02")); // break piece
assert!(!is_base_part("_rou_e106_break")); // logic
assert!(!is_base_part("e_rou_e106")); // scene node
}
#[test]
fn faction_from_id() {
assert_eq!(Faction::from_id("e106"), Faction::Adan);
assert_eq!(Faction::from_id("f105"), Faction::Tcaf);
assert_eq!(Faction::from_id("n050"), Faction::Neutral);
assert_eq!(Faction::from_id("j004"), Faction::Other);
}
// Disc-gated: reconstruct the ADAN cruiser family from a real stage container.
#[test]
fn ships_from_real_stage() {
let Ok(iso) = std::env::var("SYLPHEED_ISO") else {
eprintln!("SYLPHEED_ISO unset — skipping");
return;
};
let bytes = {
use crate::xiso::open_iso;
let rt = tokio::runtime::Builder::new_current_thread().build().unwrap();
rt.block_on(async {
let mut r = open_iso(std::path::Path::new(&iso)).await.unwrap();
r.read_file("hidden/resource3d/Stage_S02.xpr").await.unwrap()
})
};
let ships = ships_in_container(&bytes);
assert!(!ships.is_empty(), "stage should hold ships");
// The ADAN frigate e108 appears in S02 with hull + engine + weapon parts.
let e108 = ships.iter().find(|s| s.id == "e108").expect("e108 present");
assert_eq!(e108.faction, Faction::Adan);
assert!(e108.parts.iter().any(|p| p.contains("_bdy_")));
assert!(e108.parts.iter().all(|p| is_base_part(p)));
}
// Disc-gated: the scene graph must SPREAD a ship's parts, not stack them at
// the origin. Reconstruct the ADAN frigate e106 and assert its bridge sits
// clearly aft of its forward hull body (a real ship layout, not a pile).
#[test]
fn assemble_spreads_parts() {
let Ok(iso) = std::env::var("SYLPHEED_ISO") else {
eprintln!("SYLPHEED_ISO unset — skipping");
return;
};
let bytes = {
use crate::xiso::open_iso;
let rt = tokio::runtime::Builder::new_current_thread().build().unwrap();
rt.block_on(async {
let mut r = open_iso(std::path::Path::new(&iso)).await.unwrap();
r.read_file("hidden/resource3d/Stage_S02.xpr").await.unwrap()
})
};
let placed = assemble_ship(&bytes, "e106", true);
assert!(placed.len() >= 5, "e106 should place its hull + external parts");
// The forward hull body and the bridge must land at distinct fore/aft Z.
let z = |res: &str| placed.iter().find(|p| p.resource == res).map(|p| p.t[2]);
let bdy = z("e106_bdy_01").expect("bdy_01 placed");
let brg = z("e106_brg_01").expect("brg_01 placed at its GN_Bridge frame");
assert!(
(bdy - brg).abs() > 200.0,
"bridge ({brg}) and forward body ({bdy}) must be far apart, not stacked"
);
}
}

370
crates/sylpheed-formats/src/slb.rs Normal file
View File

@@ -0,0 +1,370 @@
//! `.slb` XACT sound banks → a decodable XMA1 `RIFF`.
//!
//! `dat/sound.pak` (9519 entries across `sound.p00..p04`) is the game's audio
//! bank. Each entry is an XACT `.slb` wrapping **XMA1** (`fmt ` tag `0x0165`,
//! 48 kHz). Files are named `<lang>\Voice\…`, `<lang>\Movie\VOICE_<movie>.slb`,
//! `BGM_###.slb`, etc. (see `docs/re/structures/sound-slb.md`); look them up by
//! [`crate::hash::name_hash`]. Movie cutscene voice = one continuous
//! `<eng|jpn>\Movie\VOICE_<movie>.slb` track meant to play from the video start.
//!
//! Two on-disc layouts, both reversed statically:
//! - **RIFF present** — a standard `RIFF/WAVE` sits inside the bank; its 32-byte
//! `fmt ` is the real `XMAWAVEFORMAT` and the XMA packets are everything after
//! that RIFF's `data` chunk header (the declared `data` size is unreliable, so
//! we take to end and let the caller clamp to the known media length).
//! - **Headerless** — no RIFF at all; a fixed **1392-byte** header precedes raw
//! XMA1 packets (48 kHz, 2 channels).
//!
//! [`to_xma_riff`] rebuilds a standalone `RIFF/WAVE` (XMA1) for either layout,
//! ready to hand to an XMA decoder (e.g. FFmpeg's `xma1`). The content is always
//! mono (some clips put it in the left channel only, others duplicate L=R), so
//! the decode step should downmix to mono (take the left channel).
/// Fixed offset of the raw XMA1 stream in a headerless `.slb` (no `RIFF`).
pub const HEADERLESS_DATA_OFFSET: usize = 1392;
/// Voice language for cutscene audio. Only English and Japanese voice exist on
/// the disc (subtitles cover more languages, voice does not).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum VoiceLang {
English,
Japanese,
}
impl VoiceLang {
fn code(self) -> &'static str {
match self {
VoiceLang::English => "eng",
VoiceLang::Japanese => "jpn",
}
}
/// `eng` / `jpn` — the `sound.pak` path prefix for this voice language.
pub fn code_pub(self) -> &'static str {
self.code()
}
}
/// The `sound.pak` entry name for a movie's continuous voice track, e.g.
/// `eng\Movie\VOICE_RT07A.slb`. Hash it with [`crate::hash::name_hash`] to get
/// the `sound.pak` TOC key.
pub fn movie_voice_name(movie_basename: &str, lang: VoiceLang) -> String {
format!("{}\\Movie\\VOICE_{}.slb", lang.code(), movie_basename)
}
/// One playable voice clip discovered in `sounds.tbl`.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct VoiceClip {
/// Full `sound.pak` entry name, e.g. `eng\Voice\VOICE_ADAN_010.slb`.
pub name: String,
/// Speaker/category code, e.g. `ADAN`, `TCAF`, `A`, `RT07A`.
pub speaker: String,
/// Short UI label, e.g. `ADAN 010`.
pub display: String,
}
/// Enumerate the voice/dialog clips named in a decompressed `sounds.tbl` (the
/// IDXD in `tables.pak`). Extracts every `<lang>\{Voice,etc,Movie,Briefing}\…`
/// path ending in `.slb` for `lang`, parsed into `(name, speaker, display)`.
pub fn list_voice_clips(sounds_tbl: &[u8], lang: VoiceLang) -> Vec<VoiceClip> {
let prefix = format!("{}\\", lang.code());
let mut seen = std::collections::BTreeSet::new();
let mut out = Vec::new();
// Scan for printable-ASCII runs; keep those that look like a voice path.
let mut i = 0;
while i < sounds_tbl.len() {
let start = i;
while i < sounds_tbl.len() && (0x20..=0x7e).contains(&sounds_tbl[i]) {
i += 1;
}
if i - start >= 6 {
if let Ok(s) = std::str::from_utf8(&sounds_tbl[start..i]) {
// Every spoken-line category, so the standalone player covers them
// all: in-mission radio (`\Voice\`, `\etc\`) and bound movie voices
// (`\Movie\`) all carry `VOICE_`; mission-briefing lines live in
// `\Briefing\` as `BR<NN>_<MM>.slb` (no `VOICE` in the name).
let is_voice = s.contains("VOICE") || s.contains("\\Briefing\\");
if s.starts_with(&prefix) && s.ends_with(".slb") && is_voice {
if seen.insert(s.to_string()) {
out.push(parse_voice_clip(s));
}
}
}
}
i += 1;
}
out
}
fn parse_voice_clip(name: &str) -> VoiceClip {
// `<lang>\<cat>\VOICE_<SPK>_<NNN>.slb` or `..\VOICE_<movie>.slb`.
let stem = name
.rsplit('\\')
.next()
.unwrap_or(name)
.strip_suffix(".slb")
.unwrap_or(name);
let body = stem.strip_prefix("VOICE_").unwrap_or(stem);
let (speaker, display) = match body.rsplit_once('_') {
Some((spk, num)) if num.chars().all(|c| c.is_ascii_digit()) => {
(spk.to_string(), format!("{spk} {num}"))
}
_ => (body.to_string(), body.to_string()),
};
VoiceClip {
name: name.to_string(),
speaker,
display,
}
}
/// Build one standalone XMA1 `RIFF/WAVE` **per sub-wave** in a `.slb` bank, in
/// on-disc order. A `.slb` is an XACT bank of one or more sub-waves; the sub-waves
/// are EITHER alternate takes (the first is the whole track — e.g. `VOICE_S01A`,
/// `VOICE_ADV`: sub0 ≈ the movie length) OR sequential segments that must be
/// concatenated (e.g. `VOICE_RT07A`: 24s + 14s + 11s ≈ the 50s movie). The caller
/// decodes each to PCM, concatenates in order, and clamps to the movie length —
/// that yields the full track for segment banks while the clamp drops the
/// duplicate takes for alternate-take banks. (Dynamic RE via Canary file-I/O
/// tracing confirmed the movie→voice binding; this fixes the *decode* of `RT*`.)
pub fn to_xma_riffs(slb: &[u8]) -> Vec<Vec<u8>> {
let mut out = Vec::new();
if find(slb, b"RIFF", 0).is_none() {
// Headerless single-stream bank.
if let Some(data) = slb.get(HEADERLESS_DATA_OFFSET..) {
if !data.is_empty() {
out.push(build_riff(&synth_xma1_fmt(2, 2, 48000), data));
}
}
return out;
}
let mut pos = 0usize;
while let Some(ri) = find(slb, b"RIFF", pos) {
// Parse this sub-wave's fmt + data (declared size is honest per sub-wave).
let Some(fi) = find(slb, b"fmt ", ri) else { break };
let Some(fsz) = le32(slb, fi + 4) else { break };
let Some(fmt_end) = fi.checked_add(8).and_then(|v| v.checked_add(fsz as usize)) else {
break;
};
if fmt_end > slb.len() {
break;
}
let Some(di) = find(slb, b"data", fi) else { break };
let Some(dsz) = le32(slb, di + 4) else { break };
let Some(ds) = di.checked_add(8) else { break };
let de = ds
.checked_add(dsz as usize)
.unwrap_or(slb.len())
.min(slb.len());
if let Some(data) = slb.get(ds..de) {
if !data.is_empty() {
out.push(build_riff(&slb[fi..fmt_end], data));
}
}
// Advance past this sub-wave's data to find the next RIFF.
pos = de.max(ri + 4);
}
out
}
/// Build a standalone XMA1 `RIFF/WAVE` from a `.slb` bank, decodable by FFmpeg's
/// `xma1`. Returns `None` if the bank is too small / malformed. This is the
/// FIRST sub-wave only; prefer [`to_xma_riffs`] for correct multi-segment banks.
pub fn to_xma_riff(slb: &[u8]) -> Option<Vec<u8>> {
if let Some(ri) = find(slb, b"RIFF", 0) {
// RIFF layout: a `.slb` is an XACT bank of one or more sub-waves, each
// `[seek][RIFF: fmt + Dmmy pad + data][declared_size XMA bytes]`. Take the
// FIRST sub-wave, bounded by its **declared `data` size** (which is
// honest per sub-wave). Decoding to end-of-file instead would append the
// later sub-waves — for multi-take story movies those are ALTERNATE takes,
// which is what made S10–S16 play the wrong audio.
let fi = find(slb, b"fmt ", ri)?;
let fsz = le32(slb, fi + 4)? as usize;
let fmt_end = fi.checked_add(8)?.checked_add(fsz)?;
if fmt_end > slb.len() {
return None;
}
let fmt_chunk = &slb[fi..fmt_end];
let di = find(slb, b"data", fi)?;
let dsz = le32(slb, di + 4)? as usize;
let end = di.checked_add(8)?.checked_add(dsz)?.min(slb.len());
let data = slb.get(di + 8..end)?;
Some(build_riff(fmt_chunk, data))
} else {
// Headerless: 1392-byte header, then raw XMA1 (48 kHz, 2 channels).
let data = slb.get(HEADERLESS_DATA_OFFSET..)?;
if data.is_empty() {
return None;
}
Some(build_riff(&synth_xma1_fmt(2, 2, 48000), data))
}
}
/// Rebuild a standalone XMA1 `RIFF/WAVE` from a single-stream `.slb`, robust to
/// the layout variants seen in `<lang>\etc\` radio clips. Unlike [`to_xma_riff`]
/// (which assumes `RIFF → fmt → data` in order), this picks the **largest `data`
/// chunk anywhere** in the bank — some radio banks store the audio *before* the
/// trailing `RIFF`/`fmt` metadata (an empty post-`RIFF` `data` chunk), which the
/// ordered scan misses. Pairs it with the first `fmt ` chunk; falls back to the
/// headerless layout. Returns `None` only when no usable audio can be found.
pub fn to_xma_riff_best(slb: &[u8]) -> Option<Vec<u8>> {
// Largest usable `data` chunk (bounded by its declared size and the buffer).
let mut best: Option<(usize, usize)> = None; // (data offset, usable payload len)
let mut i = 0;
while let Some(di) = find(slb, b"data", i) {
let declared = le32(slb, di + 4).unwrap_or(0) as usize;
let usable = declared.min(slb.len().saturating_sub(di + 8));
if best.map_or(true, |(_, b)| usable > b) {
best = Some((di, usable));
}
i = di + 4;
}
if let (Some(fi), Some((di, sz))) = (find(slb, b"fmt ", 0), best) {
if sz > 512 {
let fsz = le32(slb, fi + 4)? as usize;
let fmt_end = (fi + 8 + fsz).min(slb.len());
let data = slb.get(di + 8..di + 8 + sz)?;
return Some(build_riff(slb.get(fi..fmt_end)?, data));
}
}
// Headerless fallback: fixed data offset, synthesized XMA1 stereo/48k fmt.
let data = slb.get(HEADERLESS_DATA_OFFSET..)?;
(!data.is_empty()).then(|| build_riff(&synth_xma1_fmt(2, 2, 48000), data))
}
/// A minimal `fmt ` chunk carrying an XMA1 `XMAWAVEFORMAT` (one stream).
fn synth_xma1_fmt(channels: u8, channel_mask: u16, rate: u32) -> Vec<u8> {
let mut fmt = Vec::with_capacity(40);
fmt.extend_from_slice(b"fmt ");
fmt.extend_from_slice(&32u32.to_le_bytes());
// XMAWAVEFORMAT header
fmt.extend_from_slice(&0x0165u16.to_le_bytes()); // wFormatTag = XMA1
fmt.extend_from_slice(&16u16.to_le_bytes()); // BitsPerSample
fmt.extend_from_slice(&0u16.to_le_bytes()); // EncodeOptions
fmt.extend_from_slice(&0u16.to_le_bytes()); // LargestSkip
fmt.extend_from_slice(&1u16.to_le_bytes()); // NumStreams
fmt.push(0); // LoopCount
fmt.push(3); // Version
// XMASTREAMFORMAT[0]
fmt.extend_from_slice(&(rate * channels as u32 * 2).to_le_bytes()); // PsuedoBytesPerSec
fmt.extend_from_slice(&rate.to_le_bytes()); // SampleRate
fmt.extend_from_slice(&0u32.to_le_bytes()); // LoopStart
fmt.extend_from_slice(&0u32.to_le_bytes()); // LoopEnd
fmt.push(4); // SubframeData
fmt.push(channels); // Channels
fmt.extend_from_slice(&channel_mask.to_le_bytes()); // ChannelMask
fmt
}
fn build_riff(fmt_chunk: &[u8], data: &[u8]) -> Vec<u8> {
let mut body = Vec::with_capacity(4 + fmt_chunk.len() + 8 + data.len());
body.extend_from_slice(b"WAVE");
body.extend_from_slice(fmt_chunk);
body.extend_from_slice(b"data");
body.extend_from_slice(&(data.len() as u32).to_le_bytes());
body.extend_from_slice(data);
let mut out = Vec::with_capacity(8 + body.len());
out.extend_from_slice(b"RIFF");
out.extend_from_slice(&(body.len() as u32).to_le_bytes());
out.extend_from_slice(&body);
out
}
fn find(hay: &[u8], needle: &[u8], from: usize) -> Option<usize> {
if from >= hay.len() {
return None;
}
hay[from..]
.windows(needle.len())
.position(|w| w == needle)
.map(|p| p + from)
}
fn le32(b: &[u8], o: usize) -> Option<u32> {
let s = b.get(o..o + 4)?;
Some(u32::from_le_bytes([s[0], s[1], s[2], s[3]]))
}
#[cfg(test)]
mod tests {
use super::*;
use crate::hash::name_hash;
#[test]
fn movie_voice_names_and_hashes() {
assert_eq!(
movie_voice_name("RT07A", VoiceLang::English),
"eng\\Movie\\VOICE_RT07A.slb"
);
// Verified present in dat/sound.pak (VOICE_S00A == TOC entry 0x2A4F97D3).
assert_eq!(name_hash("eng\\Movie\\VOICE_RT07A.slb"), 0xA44A_EA1C);
assert_eq!(name_hash("eng\\Movie\\VOICE_S00A.slb"), 0x2A4F_97D3);
}
#[test]
fn takes_only_first_subwave() {
// Bank of TWO sub-waves; only the FIRST must be extracted (bounded by its
// declared `data` size), not everything to end-of-file.
let fmt = synth_xma1_fmt(2, 2, 48000);
let mut slb = vec![0xAB; 16];
slb.extend_from_slice(b"RIFF");
slb.extend_from_slice(&999u32.to_le_bytes()); // riff size (ignored)
slb.extend_from_slice(b"WAVE");
slb.extend_from_slice(&fmt);
slb.extend_from_slice(b"data");
slb.extend_from_slice(&4u32.to_le_bytes()); // declared: 4 bytes
slb.extend_from_slice(&[1, 2, 3, 4]); // sub-wave 1 XMA
slb.extend_from_slice(&[9, 9, 9, 9]); // a second sub-wave's bytes — excluded
let riff = to_xma_riff(&slb).unwrap();
assert_eq!(&riff[0..4], b"RIFF");
let dpos = find(&riff, b"data", 0).unwrap();
assert_eq!(le32(&riff, dpos + 4), Some(4));
assert_eq!(&riff[dpos + 8..], &[1, 2, 3, 4]);
}
#[test]
fn list_voice_clips_covers_movie_radio_and_briefing() {
// The standalone player must enumerate every spoken-line category: bound
// movie voices, in-mission radio (\etc\ + \Voice\), and briefing (\Briefing\,
// whose BR<NN>_<MM> names lack "VOICE"). Music (BGM_*) must stay excluded.
let mut tbl = Vec::new();
for s in [
"eng\\Movie\\VOICE_S13A.slb",
"eng\\etc\\VOICE_D_450.slb",
"eng\\Voice\\VOICE_ADAN_010.slb",
"eng\\Briefing\\BR01_01.slb",
"eng\\bgm\\BGM_001.slb",
] {
tbl.extend_from_slice(s.as_bytes());
tbl.push(0);
}
let names: Vec<String> = list_voice_clips(&tbl, VoiceLang::English)
.into_iter()
.map(|c| c.name)
.collect();
for want in [
"eng\\Movie\\VOICE_S13A.slb",
"eng\\etc\\VOICE_D_450.slb",
"eng\\Voice\\VOICE_ADAN_010.slb",
"eng\\Briefing\\BR01_01.slb",
] {
assert!(names.iter().any(|n| n == want), "missing {want}");
}
assert!(
!names.iter().any(|n| n.contains("BGM_")),
"music must not be listed as a voice clip"
);
}
#[test]
fn rebuilds_riff_from_headerless() {
let mut slb = vec![0u8; HEADERLESS_DATA_OFFSET];
slb.extend_from_slice(&[9, 8, 7, 6]);
let riff = to_xma_riff(&slb).unwrap();
let dpos = find(&riff, b"data", 0).unwrap();
assert_eq!(&riff[dpos + 8..], &[9, 8, 7, 6]);
// synthetic fmt advertises XMA1.
let fpos = find(&riff, b"fmt ", 0).unwrap();
assert_eq!(le32(&riff, fpos + 8).map(|v| v as u16), Some(0x0165));
}
}

179
crates/sylpheed-formats/src/t8ad.rs Normal file
View File

@@ -0,0 +1,179 @@
//! `T8aD` — the game's 2D UI/HUD texture format.
//!
//! A 32bpp **A8R8G8B8** (Xbox byte order) surface stored as **256×256 raster
//! tiles in row-major order** — each tile prefixed by a 16-byte tile header, edge
//! tiles clipped to the image bounds. Fully reversed 2026-07-17 from the file
//! header and verified against the running game (title screen).
//!
//! ```text
//! 0x00 4 Magic "T8aD"
//! 0x14 4 width (BE u32)
//! 0x18 4 height (BE u32)
//! 0x1c 4 tile count (BE u32) = ceil(w/256) * ceil(h/256)
//! 0x2c tiles*4 offset table: absolute byte offset of each row-major tile
//! <off> 16 per-tile header (flags + tile w/h), then:
//! <off+16> tile_w * tile_h * 4 bytes of A8R8G8B8 pixels, row-major
//! ```
//!
//! Surfaces ≤256px wide are a single tile column, so the first tile's pixels sit
//! at `44 + tiles*4 + 16 = 64` — which is why the old "type→header size 64/84/…"
//! rule (header = 44 + tiles*20) happened to decode small textures correctly: for
//! one tile it lands on the same pixel start. Wide textures were garbled because
//! the offset table + 16-byte per-tile headers weren't accounted for.
/// Magic at the start of every T8aD surface.
pub const T8AD_MAGIC: [u8; 4] = *b"T8aD";
/// A decoded T8aD surface as tightly-packed RGBA8 (row-major, top-left origin).
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct T8adImage {
pub width: u32,
pub height: u32,
pub rgba: Vec<u8>,
}
/// Whether `bytes` starts with the T8aD magic.
pub fn is_t8ad(bytes: &[u8]) -> bool {
bytes.len() >= 4 && bytes[0..4] == T8AD_MAGIC
}
#[inline]
fn be32(b: &[u8], off: usize) -> u32 {
u32::from_be_bytes([b[off], b[off + 1], b[off + 2], b[off + 3]])
}
/// Side of the square storage tile, in texels, and the per-tile header size.
const TILE: usize = 256;
const TILE_HDR: usize = 16;
/// Decode a T8aD surface from a slice whose first bytes ARE the magic. Returns
/// `None` for non-T8aD input or a variant we can't decode as RGBA (never guesses).
///
/// Layout (reversed from the header + verified against the running game):
/// a 44-byte base header, then a `tiles`-entry big-endian u32 **offset table** at
/// `0x2c`, where `tiles` = the field at `0x1c` = `ceil(w/256) * ceil(h/256)`.
/// Each entry is the absolute byte offset of a **row-major** 256×256 tile; every
/// tile is a 16-byte tile header followed by `tile_w*tile_h*4` A8R8G8B8 pixels,
/// edge tiles clipped to the image bounds.
pub fn parse(bytes: &[u8]) -> Option<T8adImage> {
if !is_t8ad(bytes) || bytes.len() < 0x40 {
return None;
}
let width = be32(bytes, 0x14) as usize;
let height = be32(bytes, 0x18) as usize;
if !(1..=4096).contains(&width) || !(1..=4096).contains(&height) {
return None;
}
let tiles = be32(bytes, 0x1c) as usize;
let cols = width.div_ceil(TILE);
let rows = height.div_ceil(TILE);
// The field at 0x1c must be the tile count; otherwise it's a variant we don't
// decode (e.g. DXT / palettized) — defer rather than misdecode.
if tiles == 0 || tiles != cols * rows {
return None;
}
const TABLE: usize = 0x2c;
if bytes.len() < TABLE + tiles * 4 {
return None;
}
let mut rgba = vec![0u8; width * height * 4];
for ty in 0..rows {
for tx in 0..cols {
let tile = ty * cols + tx;
let pixels = be32(bytes, TABLE + tile * 4) as usize + TILE_HDR;
let tw = TILE.min(width - tx * TILE);
let th = TILE.min(height - ty * TILE);
if pixels + tw * th * 4 > bytes.len() {
return None; // truncated / not the layout we expect
}
for row in 0..th {
let mut s = pixels + row * tw * 4;
let mut d = ((ty * TILE + row) * width + tx * TILE) * 4;
for _ in 0..tw {
// A8R8G8B8 → RGBA8.
rgba[d] = bytes[s + 1];
rgba[d + 1] = bytes[s + 2];
rgba[d + 2] = bytes[s + 3];
rgba[d + 3] = bytes[s];
s += 4;
d += 4;
}
}
}
}
Some(T8adImage {
width: width as u32,
height: height as u32,
rgba,
})
}
#[cfg(test)]
mod tests {
use super::*;
/// Build a synthetic single-tile T8aD (`w,h ≤ 256`) with a known A8R8G8B8
/// pattern: base header + 1-entry offset table + 16-byte tile header + pixels.
fn synth(w: u32, h: u32) -> Vec<u8> {
assert!(w <= 256 && h <= 256);
let mut b = vec![0u8; 0x2c];
b[0..4].copy_from_slice(&T8AD_MAGIC);
b[0x14..0x18].copy_from_slice(&w.to_be_bytes());
b[0x18..0x1c].copy_from_slice(&h.to_be_bytes());
b[0x1c..0x20].copy_from_slice(&1u32.to_be_bytes()); // 1 tile
b.extend_from_slice(&0x30u32.to_be_bytes()); // offset table: tile 0 @ 0x30
b.extend_from_slice(&[0u8; 16]); // 16-byte tile header → pixels at 0x40
for i in 0..(w * h) {
b.extend_from_slice(&[(i & 0xff) as u8, 0x24, 0x63, 0xB2]); // A, R, G, B
}
b
}
#[test]
fn decodes_argb_to_rgba() {
let b = synth(4, 2);
let img = parse(&b).expect("decodes");
assert_eq!((img.width, img.height), (4, 2));
assert_eq!(img.rgba.len(), 4 * 2 * 4);
// pixel 0: A=0,R=0x24,G=0x63,B=0xB2 → RGBA = 24 63 B2 00
assert_eq!(&img.rgba[0..4], &[0x24, 0x63, 0xB2, 0x00]);
// pixel 1: A=1 → alpha byte
assert_eq!(&img.rgba[4..8], &[0x24, 0x63, 0xB2, 0x01]);
}
#[test]
fn assembles_row_major_tiles_via_offset_table() {
// 300×1 → 2 tiles: (0,0)=256×1 red, (1,0)=44×1 blue, each +16-byte header.
let (w, h): (u32, u32) = (300, 1);
let mut b = vec![0u8; 0x2c];
b[0..4].copy_from_slice(&T8AD_MAGIC);
b[0x14..0x18].copy_from_slice(&w.to_be_bytes());
b[0x18..0x1c].copy_from_slice(&h.to_be_bytes());
b[0x1c..0x20].copy_from_slice(&2u32.to_be_bytes()); // 2 tiles
let off0 = 0x2c + 2 * 4; // after the 2-entry table
let off1 = off0 + 16 + 256 * 4; // tile-0 header + its 256 pixels
b.extend_from_slice(&(off0 as u32).to_be_bytes());
b.extend_from_slice(&(off1 as u32).to_be_bytes());
b.extend_from_slice(&[0u8; 16]);
b.extend_from_slice(&[0xFF, 0xFF, 0, 0].repeat(256)); // A,R,G,B red
b.extend_from_slice(&[0u8; 16]);
b.extend_from_slice(&[0xFF, 0, 0, 0xFF].repeat(44)); // A,R,G,B blue
let img = parse(&b).expect("decodes");
assert_eq!((img.width, img.height), (300, 1));
assert_eq!(&img.rgba[0..4], &[0xFF, 0, 0, 0xFF]); // tile 0 → red
assert_eq!(&img.rgba[256 * 4..256 * 4 + 4], &[0, 0, 0xFF, 0xFF]); // tile 1 → blue
}
#[test]
fn rejects_wrong_tilecount_and_short() {
// tile-count field that isn't ceil(w/256)*ceil(h/256) → None
let mut b = synth(2, 2);
b[0x1c..0x20].copy_from_slice(&7u32.to_be_bytes());
assert!(parse(&b).is_none());
// truncated pixel data → None
let b = synth(64, 64);
assert!(parse(&b[..200]).is_none());
assert!(parse(b"IDXD\0\0\0\0").is_none());
}
}

427
crates/sylpheed-formats/src/texture.rs
View File

@@ -164,6 +164,11 @@ impl Xpr2ResourceEntry {
pub fn is_texture(&self) -> bool {
&self.type_tag == b"TX2D"
}
/// Cubemap resource (`TXCM`) — 6 faces sharing one GPUFC descriptor.
pub fn is_cubemap(&self) -> bool {
&self.type_tag == b"TXCM"
}
}
// ── Decoded texture ───────────────────────────────────────────────────────────
@@ -179,12 +184,34 @@ pub struct X360Texture {
pub height: u32,
pub format: X360TextureFormat,
pub mip_levels: u32,
/// True when the source resource was a cubemap (`TXCM`). `data` then holds
/// only face 0 (the +X face) decoded as a 2D image — enough for a preview.
pub is_cubemap: bool,
/// De-tiled texture data in linear order.
/// BCn: standard packed block data (DDS layout).
/// Uncompressed: BGRA8 pixel data.
pub data: Vec<u8>,
}
/// A decoded Xbox 360 cubemap (`TXCM`) — a world skybox. The 6 faces are each a
/// de-tiled 2D surface in the same layout as [`X360Texture::data`], in D3D9
/// `D3DCUBEMAP_FACES` order.
#[derive(Debug, Clone)]
pub struct Cubemap {
pub width: u32,
pub height: u32,
pub format: X360TextureFormat,
/// Exactly 6 faces, D3D9 order: +X, -X, +Y, -Y, +Z, -Z.
pub faces: Vec<Vec<u8>>,
}
impl Cubemap {
/// The D3D9 cube-face name for slice `i` (0..6).
pub fn face_label(i: usize) -> &'static str {
["+X", "-X", "+Y", "-Y", "+Z", "-Z"].get(i).copied().unwrap_or("?")
}
}
impl X360Texture {
/// Parse the first TX2D texture from an XPR2 file's raw bytes.
///
@@ -194,6 +221,45 @@ impl X360Texture {
/// 3. Read its GPUTEXTURE_FETCH_CONSTANT (GPUFC) at descriptor +0x18
/// 4. De-tile the pixel data (if tiled) → linear layout
pub fn from_xpr2(bytes: &[u8]) -> Result<Self, TextureError> {
// XPR files are frequently PACKS of many textures; XPR_RES_INDEX picks
// the Nth texture resource (default 0) for RE/browse validation.
let want = std::env::var("XPR_RES_INDEX")
.ok()
.and_then(|v| v.trim().parse::<usize>().ok())
.unwrap_or(0);
Self::from_xpr2_index(bytes, want)
}
/// List the names of the texture resources (`TX2D` / `TXCM`) in an XPR2
/// container, in directory order — the same order [`from_xpr2_index`]
/// selects by. Non-texture resources (e.g. `XBG7`) are skipped.
pub fn texture_names(bytes: &[u8]) -> Vec<String> {
use std::io::Cursor;
let mut cur = Cursor::new(bytes);
let Ok(header) = Xpr2Header::read(&mut cur) else {
return Vec::new();
};
let mut names = Vec::new();
for _ in 0..header.num_resources {
let Ok(e) = Xpr2ResourceEntry::read(&mut cur) else {
break;
};
if e.is_texture() || e.is_cubemap() {
const DIR_BASE: usize = 0x10;
let no = e.name_offset as usize + DIR_BASE;
let name = bytes
.get(no..)
.and_then(|s| s.iter().position(|&c| c == 0).map(|p| &s[..p]))
.map(|s| String::from_utf8_lossy(s).into_owned())
.unwrap_or_default();
names.push(name);
}
}
names
}
/// Decode the `want`-th texture resource (`TX2D` / `TXCM`, directory order).
pub fn from_xpr2_index(bytes: &[u8], want: usize) -> Result<Self, TextureError> {
use std::io::Cursor;
let mut cur = Cursor::new(bytes);
@@ -206,10 +272,14 @@ impl X360Texture {
entries.push(Xpr2ResourceEntry::read(&mut cur)?);
}
// Find the first TX2D texture resource
// Select a texture resource. TX2D = 2D texture; TXCM = cubemap
// (skybox / backdrop) — same 52-byte descriptor + GPUFC layout, but the
// pixel section holds 6 faces. For a preview we decode face 0.
let tex_entry = entries.iter()
.find(|e| e.is_texture())
.filter(|e| e.is_texture() || e.is_cubemap())
.nth(want)
.ok_or(TextureError::NoTextureFound)?;
let is_cubemap = tex_entry.is_cubemap();
// The descriptor is at file offset = data_offset + 0x10 (directory base)
const DIR_BASE: usize = 0x10;
@@ -239,6 +309,11 @@ impl X360Texture {
let format = X360TextureFormat::from_u8(fmt_code)
.ok_or(TextureError::UnsupportedFormat(fmt_code))?;
// GPUFC[1] bits[7:6] = endianness. X360 stores texture words byte-
// swapped; without undoing this, BCn endpoints/indices and ARGB channels
// decode to noise. (fetch-constant `endianness`, xenos.h Endian.)
let endianness = ((gpufc1 >> 6) & 0x3) as u8;
// GPUFC[1] bits[31:12] = base_address (4KB-aligned byte offset into data section)
let base_address = (gpufc1 & 0xFFFFF000) as usize;
@@ -260,23 +335,74 @@ impl X360Texture {
have: bytes.len(),
});
}
let raw_data = &bytes[data_start..];
let linear_data = if is_tiled {
detile(raw_data, width, height, format)?
} else {
// Linear layout — copy only the mip-0 slice
let block_size = format.block_size() as u32;
let bw = ((width + block_size - 1) / block_size).max(1);
let bh = ((height + block_size - 1) / block_size).max(1);
let needed = bw as usize * bh as usize * format.bytes_per_block();
if raw_data.len() < needed {
return Err(TextureError::BufferTooSmall { needed, have: raw_data.len() });
}
raw_data[..needed].to_vec()
};
// De-tile + endian-correct the single (face-0) surface.
let data = decode_surface(&bytes[data_start..], width, height, format, is_tiled, endianness)?;
Ok(X360Texture { width, height, format, mip_levels: mip_count, data: linear_data })
Ok(X360Texture { width, height, format, mip_levels: mip_count, is_cubemap, data })
}
/// Decode all 6 faces of a cubemap (`TXCM`) resource, or `Ok(None)` if the
/// selected resource is an ordinary 2D texture.
///
/// X360 cubemaps store the 6 faces back-to-back in the pixel section, each a
/// full independently-tiled 2D surface whose stride is the tiled surface size
/// rounded up to a 4 KiB subresource boundary (`kTextureSubresourceAlignmentBytes`).
/// Faces are in D3D9 `D3DCUBEMAP_FACES` order: +X, -X, +Y, -Y, +Z, -Z.
/// (Derived from xenia `texture_util.cc::GetGuestTextureLayout`; verified on
/// `BG_Acheron`: `data_size == 6 × 0x400000` and all 6 faces decode cleanly.)
pub fn cube_faces_from_xpr2(bytes: &[u8]) -> Result<Option<Cubemap>, TextureError> {
use std::io::Cursor;
let mut cur = Cursor::new(bytes);
let header = Xpr2Header::read(&mut cur)?;
let mut entries = Vec::new();
for _ in 0..header.num_resources {
entries.push(Xpr2ResourceEntry::read(&mut cur)?);
}
let want = std::env::var("XPR_RES_INDEX")
.ok()
.and_then(|v| v.trim().parse::<usize>().ok())
.unwrap_or(0);
let tex_entry = entries
.iter()
.filter(|e| e.is_texture() || e.is_cubemap())
.nth(want)
.ok_or(TextureError::NoTextureFound)?;
if !tex_entry.is_cubemap() {
return Ok(None); // ordinary 2D texture — use `from_xpr2`
}
const DIR_BASE: usize = 0x10;
let gpufc_base = tex_entry.data_offset as usize + DIR_BASE + 0x18;
if bytes.len() < gpufc_base + 6 * 4 {
return Err(TextureError::BufferTooSmall { needed: gpufc_base + 24, have: bytes.len() });
}
let be_u32 = |o: usize| u32::from_be_bytes(bytes[o..o + 4].try_into().unwrap());
let gpufc0 = be_u32(gpufc_base);
let gpufc1 = be_u32(gpufc_base + 0x04);
let gpufc2 = be_u32(gpufc_base + 0x08);
let fmt_code = (gpufc1 & 0x3F) as u8;
let format = X360TextureFormat::from_u8(fmt_code)
.ok_or(TextureError::UnsupportedFormat(fmt_code))?;
let endianness = ((gpufc1 >> 6) & 0x3) as u8;
let base_address = (gpufc1 & 0xFFFFF000) as usize;
let width = (gpufc2 & 0x1FFF) + 1;
let height = ((gpufc2 >> 13) & 0x1FFF) + 1;
let is_tiled = (gpufc0 >> 31) != 0;
let data_start = header.header_size as usize + base_address;
let stride = tiled_face_stride(width, height, format);
let mut faces = Vec::with_capacity(6);
for f in 0..6 {
let start = data_start + f * stride;
let raw = bytes
.get(start..)
.ok_or(TextureError::BufferTooSmall { needed: start + 1, have: bytes.len() })?;
faces.push(decode_surface(raw, width, height, format, is_tiled, endianness)?);
}
Ok(Some(Cubemap { width, height, format, faces }))
}
/// Parse a texture from already-known parameters + raw tiled data.
@@ -290,19 +416,163 @@ impl X360Texture {
format: X360TextureFormat,
) -> Result<Self, TextureError> {
let linear_data = detile(tiled_data, width, height, format)?;
Ok(X360Texture { width, height, format, mip_levels: 1, data: linear_data })
Ok(X360Texture { width, height, format, mip_levels: 1, is_cubemap: false, data: linear_data })
}
}
/// Apply the Xenos texture endian swap in place, as byte permutations over the
/// data. Xbox 360 stores texture words big-endian; this converts them to the
/// PC-standard little-endian layout that BCn decoders and wgpu expect.
/// `endianness` is the fetch-constant `dword_1` bits[7:6]:
/// 0 = none, 1 = k8in16, 2 = k8in32, 3 = k16in32 (xenia `xenos.h` `Endian`).
pub fn apply_endian_swap(data: &mut [u8], endianness: u8) {
match endianness {
1 => {
// k8in16 — swap the two bytes of each 16-bit half.
for c in data.chunks_exact_mut(2) {
c.swap(0, 1);
}
}
2 => {
// k8in32 — reverse each 32-bit word.
for c in data.chunks_exact_mut(4) {
c.reverse();
}
}
3 => {
// k16in32 — swap the two 16-bit halves of each 32-bit word.
for c in data.chunks_exact_mut(4) {
c.swap(0, 2);
c.swap(1, 3);
}
}
_ => {} // kNone
}
}
/// De-tile (if tiled) + endian-correct one texture surface into linear PC
/// layout. Shared by `from_xpr2` (face 0) and `cube_faces_from_xpr2` (6 faces).
/// The `XPR_NO_DETILE` / `XPR_NO_ENDIAN` / `XPR_FORCE_ENDIAN` / `XPR_NO_BC_DWORD_SWAP`
/// debug knobs are honoured here so both paths behave identically.
fn decode_surface(
raw_data: &[u8],
width: u32,
height: u32,
format: X360TextureFormat,
is_tiled: bool,
endianness: u8,
) -> Result<Vec<u8>, TextureError> {
let force_linear = std::env::var("XPR_NO_DETILE").is_ok();
let mut linear_data = if is_tiled && !force_linear {
detile(raw_data, width, height, format)?
} else {
// Linear layout — copy only the mip-0 slice.
let block_size = format.block_size() as u32;
let bw = ((width + block_size - 1) / block_size).max(1);
let bh = ((height + block_size - 1) / block_size).max(1);
let needed = bw as usize * bh as usize * format.bytes_per_block();
if raw_data.len() < needed {
return Err(TextureError::BufferTooSmall { needed, have: raw_data.len() });
}
raw_data[..needed].to_vec()
};
// Undo the X360 word byte-swap so block/pixel data is PC little-endian.
let endianness = std::env::var("XPR_FORCE_ENDIAN")
.ok()
.and_then(|v| v.trim().parse::<u8>().ok())
.unwrap_or(endianness);
if std::env::var("XPR_NO_ENDIAN").is_err() {
apply_endian_swap(&mut linear_data, endianness);
}
// BC1 colour-block dword swap (see the long note where this was discovered):
// colour endpoints live in the high dword on X360; format-targeted so BC4/BC5
// (byte-indexed alpha/normal blocks) are left as endian-only.
if std::env::var("XPR_NO_BC_DWORD_SWAP").is_err() {
match format {
X360TextureFormat::Dxt1 => swap_bc_block_dwords(&mut linear_data),
X360TextureFormat::Dxt3 | X360TextureFormat::Dxt5 => {
for block in linear_data.chunks_exact_mut(16) {
swap_bc_block_dwords(&mut block[8..16]);
}
}
_ => {}
}
}
Ok(linear_data)
}
/// Byte stride between consecutive cubemap faces: the tiled surface size
/// (`pitch_aligned × height_aligned × bpb`, both padded to 32-block macro tiles)
/// rounded up to the 4 KiB subresource alignment (`kTextureSubresourceAlignmentBytes`).
fn tiled_face_stride(width: u32, height: u32, format: X360TextureFormat) -> usize {
let bs = format.block_size() as u32;
let bw = ((width + bs - 1) / bs).max(1);
let bh = ((height + bs - 1) / bs).max(1);
let pitch_aligned = align_up(bw, STORAGE_ALIGN_BLOCKS).max(MACRO_TILE_BLOCKS);
let height_aligned = align_up(bh, STORAGE_ALIGN_BLOCKS).max(MACRO_TILE_BLOCKS);
let surface = pitch_aligned as usize * height_aligned as usize * format.bytes_per_block();
(surface + 0xFFF) & !0xFFF
}
/// Swap the two 32-bit dwords within each 64-bit unit of `data`, in place.
///
/// Xbox 360 stores the BC1 *colour* block with its two 32-bit words in the
/// opposite order to the PC/DDS layout, putting the colour endpoints ahead of
/// the indices. Apply to a BC1 buffer (or the colour half of a BC2/BC3 block)
/// after the byte-level endian swap. Any trailing bytes that don't fill a full
/// 8-byte group are left untouched.
pub fn swap_bc_block_dwords(data: &mut [u8]) {
for unit in data.chunks_exact_mut(8) {
// [d0 d1 d2 d3 | d4 d5 d6 d7] → [d4 d5 d6 d7 | d0 d1 d2 d3]
let (lo, hi) = unit.split_at_mut(4);
lo.swap_with_slice(hi);
}
}
// ── Core de-tiling algorithm ──────────────────────────────────────────────────
/// De-tile an Xbox 360 GPU texture from tiled to linear (row-major) layout.
/// Macro-tile side in blocks (Xenos tiles are 32×32 *blocks*, where a "block"
/// is one BCn 4×4 group or one uncompressed texel). `texture_address.h`
/// `kTextureTileWidthHeight` / `kMacroTileWidth`.
const MACRO_TILE_BLOCKS: u32 = 32;
/// Storage pitch/height alignment, in blocks (`kStoragePitchHeightAlignmentBlocks`).
const STORAGE_ALIGN_BLOCKS: u32 = 32;
#[inline]
fn align_up(v: u32, a: u32) -> u32 {
(v + a - 1) & !(a - 1)
}
/// Byte offset of block (x, y) within an Xbox 360 2D tiled surface.
///
/// Xbox 360 stores textures in 32×32 texel macro-tiles. Within each
/// macro-tile the DXT blocks (or raw pixels) are in Morton (Z-order) order.
/// This function reverses that ordering for the mip-0 level.
/// Faithful port of xenia-canary `texture_address.h::Tiled2D` + `TiledCombine`
/// (documented Xenos hardware tiling — bank/pipe/macro-tile addressing, NOT a
/// plain Morton curve). `x`/`y` and `pitch_aligned` are in block units;
/// `bpb_log2` is log2(bytes-per-block). Returns a byte offset.
#[inline]
fn tiled_2d_offset(x: i32, y: i32, pitch_aligned: u32, bpb_log2: u32) -> i32 {
let outer_blocks = ((y >> 5) * (pitch_aligned >> 5) as i32 + (x >> 5)) << 6;
let inner_blocks = (((y >> 1) & 0b111) << 3) | (x & 0b111);
let outer_inner_bytes = (outer_blocks | inner_blocks) << bpb_log2;
let bank = (y >> 4) & 0b1;
let pipe = ((x >> 3) & 0b11) ^ (((y >> 3) & 0b1) << 1);
let y_lsb = y & 1;
// TiledCombine: splice bank/pipe/y_lsb bits into the byte address.
((y_lsb << 4) | (pipe << 6) | (bank << 11))
| (outer_inner_bytes & 0b1111)
| (((outer_inner_bytes >> 4) & 0b1) << 5)
| (((outer_inner_bytes >> 5) & 0b111) << 8)
| (outer_inner_bytes >> 8 << 12)
}
/// De-tile an Xbox 360 GPU texture (mip-0) from tiled to linear (row-major)
/// layout, using the exact Xenos address formula.
///
/// Algorithm based on Xenia's `texture_util.cc` `TileTexture()`.
/// `src` must hold the tiled mip-0 surface (its storage pitch/height are
/// rounded up to 32 blocks, so it may be larger than the visible image). The
/// returned buffer is tightly packed linear block data (DDS layout for BCn).
pub fn detile(
src: &[u8],
width: u32,
@@ -311,50 +581,30 @@ pub fn detile(
) -> Result<Vec<u8>, TextureError> {
let block_size = format.block_size() as u32;
let bpb = format.bytes_per_block();
let bpb_log2 = (bpb as u32).trailing_zeros();
// Texture dimensions in blocks
let blocks_wide = ((width + block_size - 1) / block_size).max(1);
// Visible dimensions in blocks, and the padded storage pitch/height.
let blocks_wide = ((width + block_size - 1) / block_size).max(1);
let blocks_tall = ((height + block_size - 1) / block_size).max(1);
let pitch_aligned = align_up(blocks_wide, STORAGE_ALIGN_BLOCKS).max(MACRO_TILE_BLOCKS);
let height_aligned = align_up(blocks_tall, STORAGE_ALIGN_BLOCKS).max(MACRO_TILE_BLOCKS);
let expected = blocks_wide as usize * blocks_tall as usize * bpb;
if src.len() < expected {
return Err(TextureError::BufferTooSmall { needed: expected, have: src.len() });
// The tiled surface occupies pitch_aligned × height_aligned blocks.
let src_needed = pitch_aligned as usize * height_aligned as usize * bpb;
if src.len() < src_needed {
return Err(TextureError::BufferTooSmall { needed: src_needed, have: src.len() });
}
let mut dst = vec![0u8; expected];
let dst_len = blocks_wide as usize * blocks_tall as usize * bpb;
let mut dst = vec![0u8; dst_len];
// Xbox 360 macro-tiles are always 32×32 texels → 8×8 blocks for BCn (4-texel blocks)
let macro_tile_blocks = 32 / block_size;
let macro_tiles_wide = (blocks_wide + macro_tile_blocks - 1) / macro_tile_blocks;
let macro_tiles_tall = (blocks_tall + macro_tile_blocks - 1) / macro_tile_blocks;
let blocks_per_macro_tile = (macro_tile_blocks * macro_tile_blocks) as usize;
for macro_y in 0..macro_tiles_tall {
for macro_x in 0..macro_tiles_wide {
let macro_base = ((macro_y * macro_tiles_wide + macro_x) as usize)
* blocks_per_macro_tile
* bpb;
for local in 0..blocks_per_macro_tile as u32 {
// Decode Morton (Z-order) index → (lx, ly) within macro-tile
let (lx, ly) = morton_decode(local);
let block_x = macro_x * macro_tile_blocks + lx;
let block_y = macro_y * macro_tile_blocks + ly;
// Skip blocks outside the actual texture
if block_x >= blocks_wide || block_y >= blocks_tall {
continue;
}
let src_offset = macro_base + local as usize * bpb;
let dst_offset = (block_y * blocks_wide + block_x) as usize * bpb;
if src_offset + bpb <= src.len() && dst_offset + bpb <= dst.len() {
dst[dst_offset..dst_offset + bpb]
.copy_from_slice(&src[src_offset..src_offset + bpb]);
}
for by in 0..blocks_tall {
for bx in 0..blocks_wide {
let src_offset = tiled_2d_offset(bx as i32, by as i32, pitch_aligned, bpb_log2) as usize;
let dst_offset = (by * blocks_wide + bx) as usize * bpb;
if src_offset + bpb <= src.len() && dst_offset + bpb <= dst.len() {
dst[dst_offset..dst_offset + bpb]
.copy_from_slice(&src[src_offset..src_offset + bpb]);
}
}
}
@@ -399,11 +649,58 @@ mod tests {
}
#[test]
fn detile_noop_for_1x1_block() {
// A 4×4 DXT1 texture = exactly 1 block = 8 bytes; de-tiling is identity
let src = vec![0xDE, 0xAD, 0xBE, 0xEF, 0x01, 0x02, 0x03, 0x04];
fn tiled_offset_origin_is_zero() {
// Block (0,0) always maps to byte offset 0 for any pitch / bpb.
assert_eq!(tiled_2d_offset(0, 0, 32, 3), 0);
assert_eq!(tiled_2d_offset(0, 0, 64, 4), 0);
}
#[test]
fn detile_single_block_reads_offset_zero() {
// A 4×4 DXT1 texture = 1 visible block, but Xenos pads the tiled
// surface to 32×32 blocks (8192 bytes). Block (0,0) sits at offset 0,
// so the de-tiled output equals the first 8 source bytes.
let mut src = vec![0u8; 32 * 32 * 8];
src[..8].copy_from_slice(&[0xDE, 0xAD, 0xBE, 0xEF, 0x01, 0x02, 0x03, 0x04]);
let result = detile(&src, 4, 4, X360TextureFormat::Dxt1).unwrap();
assert_eq!(result, src);
assert_eq!(result, &src[..8]);
}
#[test]
fn swap_bc_dwords_swaps_each_64bit_half() {
// One 8-byte BC1 block: X360 stores it as [indices][endpoints]; the swap
// must move the endpoint dword to the front so BC decoders find it.
let mut one = vec![0, 1, 2, 3, 4, 5, 6, 7];
swap_bc_block_dwords(&mut one);
assert_eq!(one, vec![4, 5, 6, 7, 0, 1, 2, 3]);
// A 16-byte BC3 block = two independent 8-byte halves; each is swapped.
let mut two: Vec<u8> = (0..16).collect();
swap_bc_block_dwords(&mut two);
assert_eq!(
two,
vec![4, 5, 6, 7, 0, 1, 2, 3, 12, 13, 14, 15, 8, 9, 10, 11]
);
// Applying it twice is the identity (it's its own inverse).
let mut back = two.clone();
swap_bc_block_dwords(&mut back);
assert_eq!(back, (0..16).collect::<Vec<u8>>());
}
#[test]
fn cubemap_face_stride_and_labels() {
// Acheron: 1024×1024 A8R8G8B8 → 1024*1024*4 = 4 MiB, already 4 KiB-aligned.
// Verified against the real file: data_size == 6 × 0x400000.
assert_eq!(
tiled_face_stride(1024, 1024, X360TextureFormat::A8R8G8B8),
0x400000
);
// A tiny surface still occupies a full 32×32-block tile, 4 KiB-aligned.
assert_eq!(tiled_face_stride(4, 4, X360TextureFormat::A8R8G8B8), 0x1000);
assert_eq!(Cubemap::face_label(0), "+X");
assert_eq!(Cubemap::face_label(2), "+Y");
assert_eq!(Cubemap::face_label(5), "-Z");
}
#[test]

172
crates/sylpheed-formats/src/vfs.rs
View File

@@ -108,14 +108,23 @@ impl GameAssets {
/// Use this to identify unknown file types during reverse engineering.
pub fn identify_format(bytes: &[u8]) -> FileFormat {
if bytes.len() < 4 {
return FileFormat::Unknown;
// Very short files can still be text (e.g. a one-line config).
return if is_probably_text(bytes) {
FileFormat::Text
} else {
FileFormat::Unknown
};
}
match &bytes[..4] {
b"XPR2" => FileFormat::Xpr2Texture,
b"RIFF" => FileFormat::Riff, // could be WAV or XWB
b"XWB\0" => FileFormat::XwbAudio, // Xbox Wave Bank
b"RIFF" => FileFormat::Riff, // could be WAV or XWB
b"XWB\0" => FileFormat::XwbAudio, // Xbox Wave Bank
[0x89, b'P', b'N', b'G'] => FileFormat::Png,
b"DDS " => FileFormat::Dds,
// No binary magic matched — fall back to a content heuristic so text
// configs (`config.ini`, etc.) are recognised even though they have no
// signature.
_ if is_probably_text(bytes) => FileFormat::Text,
_ => FileFormat::Unknown,
}
}
@@ -127,6 +136,8 @@ pub enum FileFormat {
XwbAudio,
Png,
Dds,
/// Plain text (ASCII / UTF-8 / UTF-16) — `.ini` and similar.
Text,
Unknown,
}
@@ -138,7 +149,162 @@ impl FileFormat {
Self::XwbAudio => "xwb",
Self::Png => "png",
Self::Dds => "dds",
Self::Text => "txt",
Self::Unknown => "bin",
}
}
}
// ── Text detection & decoding (pure std, WASM-safe) ───────────────────────────
/// True if `b` is a byte we'd expect inside a text file: common whitespace,
/// printable ASCII, or any high byte (UTF-8 continuation / Latin-1). Mirrors the
/// printable convention used by the IDXD string-pool extractor.
#[inline]
fn is_text_byte(b: u8) -> bool {
matches!(b, b'\t' | b'\n' | b'\r') || (0x20..0x7f).contains(&b) || b >= 0x80
}
/// Heuristic sniff for whether `bytes` is human-readable text.
///
/// Recognises UTF-8 / UTF-16 BOMs outright, BOM-less UTF-16LE by its
/// characteristic odd-position NUL bytes, and otherwise requires a byte sample
/// to be almost entirely text bytes with no embedded NUL (NUL is the strongest
/// binary signal). Decoding is left to [`decode_text`].
pub fn is_probably_text(bytes: &[u8]) -> bool {
if bytes.is_empty() {
return false;
}
// Byte-order marks are unambiguous.
if bytes.starts_with(&[0xEF, 0xBB, 0xBF]) // UTF-8
|| bytes.starts_with(&[0xFF, 0xFE]) // UTF-16LE
|| bytes.starts_with(&[0xFE, 0xFF])
// UTF-16BE
{
return true;
}
// Cap the scan so huge files stay cheap.
let sample = &bytes[..bytes.len().min(4096)];
// BOM-less UTF-16LE: ASCII text encodes as `<char> 0x00 <char> 0x00 …`, so
// roughly half the bytes (the odd positions) are NUL and the even bytes are
// printable.
if sample.len() >= 16 {
let nul_odd = sample.iter().skip(1).step_by(2).filter(|&&b| b == 0).count();
if nul_odd * 2 >= sample.len() / 2 {
let printable_even = sample.iter().step_by(2).filter(|&&b| is_text_byte(b)).count();
if printable_even * 2 >= sample.len() / 2 - 2 {
return true;
}
}
}
// Single-byte (ASCII / UTF-8): an embedded NUL means binary; otherwise
// require the overwhelming majority to be text bytes.
if sample.contains(&0) {
return false;
}
let text = sample.iter().filter(|&&b| is_text_byte(b)).count();
text * 100 >= sample.len() * 95
}
/// Decode `bytes` to a `String`, returning `(text, encoding_label)`.
///
/// Honours UTF-8 / UTF-16LE / UTF-16BE BOMs and BOM-less UTF-16LE, falling back
/// to lossy UTF-8. Never fails — undecodable sequences become U+FFFD. Pure std,
/// so both the viewer and CLI can share it.
pub fn decode_text(bytes: &[u8]) -> (String, &'static str) {
if let Some(rest) = bytes.strip_prefix(&[0xEF, 0xBB, 0xBF]) {
return (String::from_utf8_lossy(rest).into_owned(), "UTF-8 (BOM)");
}
if let Some(rest) = bytes.strip_prefix(&[0xFF, 0xFE]) {
return (decode_utf16(rest, false), "UTF-16LE (BOM)");
}
if let Some(rest) = bytes.strip_prefix(&[0xFE, 0xFF]) {
return (decode_utf16(rest, true), "UTF-16BE (BOM)");
}
// BOM-less UTF-16LE detection (same signal as `is_probably_text`).
let sample = &bytes[..bytes.len().min(4096)];
if sample.len() >= 16 {
let nul_odd = sample.iter().skip(1).step_by(2).filter(|&&b| b == 0).count();
if nul_odd * 2 >= sample.len() / 2 {
return (decode_utf16(bytes, false), "UTF-16LE");
}
}
(String::from_utf8_lossy(bytes).into_owned(), "UTF-8 / ASCII")
}
fn decode_utf16(bytes: &[u8], big_endian: bool) -> String {
let units: Vec<u16> = bytes
.chunks_exact(2)
.map(|c| {
if big_endian {
u16::from_be_bytes([c[0], c[1]])
} else {
u16::from_le_bytes([c[0], c[1]])
}
})
.collect();
String::from_utf16_lossy(&units)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn ascii_config_is_text() {
let ini = b"[Video]\r\nWidth=1280\r\nHeight=720\r\n";
assert!(is_probably_text(ini));
assert_eq!(identify_format(ini), FileFormat::Text);
let (text, enc) = decode_text(ini);
assert!(text.contains("Width=1280"));
assert_eq!(enc, "UTF-8 / ASCII");
}
#[test]
fn magic_takes_precedence_over_text() {
// An XPR2 header happens to start with printable bytes; the magic match
// must win so it's never misfiled as text.
let mut xpr = b"XPR2".to_vec();
xpr.extend_from_slice(&[0u8; 32]);
assert_eq!(identify_format(&xpr), FileFormat::Xpr2Texture);
}
#[test]
fn binary_with_nul_is_not_text() {
let bin = [0u8, 1, 2, 0xFF, 0x00, 0x89, 0x10, 0x00, 0x42];
assert!(!is_probably_text(&bin));
assert_eq!(identify_format(&bin), FileFormat::Unknown);
}
#[test]
fn utf16le_bom_is_text_and_decodes() {
let data = [0xFF, 0xFE, b'H', 0x00, b'i', 0x00];
assert!(is_probably_text(&data));
assert_eq!(identify_format(&data), FileFormat::Text);
let (text, enc) = decode_text(&data);
assert_eq!(text, "Hi");
assert_eq!(enc, "UTF-16LE (BOM)");
}
#[test]
fn bomless_utf16le_detected() {
// "Hello, world!!!!" as UTF-16LE, no BOM (>=16 bytes so the heuristic runs).
let mut data = Vec::new();
for &b in b"Hello, world!!!!" {
data.push(b);
data.push(0);
}
assert!(is_probably_text(&data));
let (text, enc) = decode_text(&data);
assert_eq!(text, "Hello, world!!!!");
assert_eq!(enc, "UTF-16LE");
}
#[test]
fn empty_is_not_text() {
assert!(!is_probably_text(&[]));
}
}

426
crates/sylpheed-formats/tests/mesh_disc.rs Normal file
View File

@@ -0,0 +1,426 @@
//! Integration test: decode XBG7 geometry from REAL `.xpr` model files.
//!
//! Uses loose files extracted from the retail disc (models live in
//! `hidden/resource3d/*.xpr`). Skipped unless the directory is found; point
//! `SYLPHEED_RES3D` at it to override.
//!
//! Run: `cargo test -p sylpheed-formats --test mesh_disc -- --ignored --nocapture`
use std::path::PathBuf;
use sylpheed_formats::mesh::{material_groups, node_transforms, submesh_albedos, Xbg7Model};
fn res3d_dir() -> Option<PathBuf> {
if let Ok(p) = std::env::var("SYLPHEED_RES3D") {
let p = PathBuf::from(p);
if p.is_dir() {
return Some(p);
}
}
let default =
PathBuf::from("/home/fabi/RE Project Sylpheed/sylph_extract/hidden/resource3d");
default.is_dir().then_some(default)
}
#[test]
#[ignore = "requires extracted disc models — set SYLPHEED_RES3D"]
fn hero_ship_submesh_material_graph() {
let Some(dir) = res3d_dir() else {
eprintln!("SKIP: resource3d dir not found (set SYLPHEED_RES3D)");
return;
};
// The XBG7 node/material graph tags each sub-mesh record with its own albedo
// (`rou_…_col` → TX2D name). DeltaSaber_T `f001`: the 7 detail parts use
// bdy_04/06/07 — NOT the body's bdy_01a — which is the per-part texturing the
// viewer needs so fins aren't painted with the hull map.
let bytes = std::fs::read(dir.join("DeltaSaber_T.xpr")).unwrap();
let mut map = std::collections::HashMap::new();
for (v, i, name) in submesh_albedos(&bytes, "f001") {
map.insert((v, i), name);
}
// (vtx, idx) → expected albedo (rou_ stripped = TX2D name).
assert_eq!(map.get(&(314, 645)).map(String::as_str), Some("f001_bdy_04_col"));
assert_eq!(map.get(&(48, 84)).map(String::as_str), Some("f001_bdy_04_col"));
assert_eq!(map.get(&(96, 180)).map(String::as_str), Some("f001_bdy_06_col"));
assert_eq!(map.get(&(60, 108)).map(String::as_str), Some("f001_bdy_07_col"));
// Names strip cleanly to real TX2D resources.
let tex = sylpheed_formats::texture::X360Texture::texture_names(&bytes);
for name in map.values() {
assert!(tex.iter().any(|t| t == name), "albedo {name} not a TX2D resource");
}
}
#[test]
#[ignore = "requires extracted disc models — set SYLPHEED_RES3D"]
fn hero_ship_body_splits_by_material() {
let Some(dir) = res3d_dir() else {
eprintln!("SKIP: resource3d dir not found (set SYLPHEED_RES3D)");
return;
};
// The merged body (sub0 = 24561 indices / 8187 tris) is drawn as 9 groups
// sharing one index buffer, each with its own albedo (bdy_01a hull, bdy_01b,
// bdy_02/03, and the `daiza` hangar stand). `material_groups` recovers the
// cumulative-offset chain so the viewer can texture each slice.
let bytes = std::fs::read(dir.join("DeltaSaber_T.xpr")).unwrap();
let groups = material_groups(&bytes, "f001", 24561);
assert_eq!(groups.len(), 9, "body must split into 9 material groups");
// Offsets chain from 0 and cover the whole index buffer exactly.
let mut cursor = 0usize;
for g in &groups {
assert_eq!(g.idx_offset, cursor, "group offsets must be contiguous");
cursor += g.idx_count;
}
assert_eq!(cursor, 24561, "groups must cover all body indices");
// Expected per-group albedo (verified against the GPU draw log).
let albedos: Vec<&str> = groups.iter().map(|g| g.albedo.as_str()).collect();
assert_eq!(
albedos,
[
"f001_bdy_01a_col",
"f001_bdy_01a_col",
"f001_bdy_01a_col",
"f001_bdy_01b_col",
"f001_bdy_01a_col",
"f001_bdy_01b_col",
"f001_bdy_02_col",
"f001_bdy_03_col",
"f001_bdy_daiza_col",
]
);
// A single-material detail part (sub1 = 645 indices) → one group, bdy_04.
let fin = material_groups(&bytes, "f001", 645);
assert_eq!(fin.len(), 1);
assert_eq!(fin[0].albedo, "f001_bdy_04_col");
assert_eq!(fin[0].idx_offset, 0);
}
#[test]
#[ignore = "requires extracted disc models — set SYLPHEED_RES3D"]
fn hero_ship_node_transforms_move_fins_to_tail() {
let Some(dir) = res3d_dir() else {
eprintln!("SKIP: resource3d dir not found (set SYLPHEED_RES3D)");
return;
};
// For the DeltaSaber, node_transforms returns the MEASURED runtime placements
// (Canary draw-log WVP; see mesh.rs::DELTASABER_MEASURED) — the fin assemblies
// are mounted OUT on the nacelles (world X ≈ −4.4..−6.6), an offset the static
// graph omits. Body identity; each part a left/right mirrored pair. Vertex
// space: X right, Y up, Z fore/aft (engines at −Z).
let bytes = std::fs::read(dir.join("DeltaSaber_T.xpr")).unwrap();
let places = node_transforms(&bytes, "f001");
assert!(!places.is_empty(), "expected node placements");
// Body (sub 0) identity, drawn once.
let body: Vec<_> = places.iter().filter(|p| p.sub_index == 0).collect();
assert_eq!(body.len(), 1, "body drawn once");
assert!(body[0].t.iter().all(|c| c.abs() < 0.1), "body must not translate");
assert!(!body[0].reflect, "body is not mirrored");
// Fin bdy_04 (sub 1): mirrored pair near the tail, mounted OUT on the nacelle
// (|X| ≈ 5.2, not the graph's inboard ≈1.1).
let fins: Vec<_> = places.iter().filter(|p| p.sub_index == 1).collect();
assert_eq!(fins.len(), 2, "V-tail is a mirrored pair");
assert!(fins.iter().all(|f| (f.t[2] + 9.72).abs() < 0.3), "fin fore/aft ≈ −9.7");
assert!(fins.iter().all(|f| f.t[0].abs() > 4.0), "fin mounted out on the nacelle");
assert!(fins[0].t[0] * fins[1].t[0] < 0.0, "the pair mirrors across X");
assert!(fins.iter().any(|f| f.reflect), "one of the pair is reflected");
// Winglet bdy_06 (sub 4): on the nacelle (|X| ≈ 6.6, Z ≈ −15.5).
let wings: Vec<_> = places.iter().filter(|p| p.sub_index == 4).collect();
assert_eq!(wings.len(), 2, "L/R winglet pair");
assert!(wings.iter().all(|p| p.t[0].abs() > 6.0 && (p.t[2] + 15.5).abs() < 0.3),
"winglets out on the nacelle");
assert!(wings[0].t[0] * wings[1].t[0] < 0.0, "winglets mirror across X");
// Small fin bdy_10 (sub 6): inboard nacelle stub (|X| ≈ 4.45, Z ≈ −17).
let sfins: Vec<_> = places.iter().filter(|p| p.sub_index == 6).collect();
assert_eq!(sfins.len(), 2, "L/R small-fin pair");
assert!(sfins.iter().all(|p| (p.t[0].abs() - 4.45).abs() < 0.3), "small fins on the stub");
assert!(sfins[0].t[0] * sfins[1].t[0] < 0.0, "small fins mirror across X");
}
#[test]
#[ignore = "requires extracted disc models — set SYLPHEED_RES3D"]
fn weapon_model_decodes_to_expected_geometry() {
let Some(dir) = res3d_dir() else {
eprintln!("SKIP: resource3d dir not found (set SYLPHEED_RES3D)");
return;
};
// rou_f001_wep_00 = the player ship's first weapon: 1 sub-mesh,
// 215 vertices, 364 triangles (verified by hex analysis).
let bytes = std::fs::read(dir.join("rou_f001_wep_00.xpr")).unwrap();
let model = Xbg7Model::from_xpr2(&bytes).expect("weapon must decode");
assert_eq!(model.meshes.len(), 1);
let (v, t) = model.totals();
assert_eq!(v, 215, "vertex count");
assert_eq!(t, 364, "triangle count");
let m = &model.meshes[0];
assert_eq!(m.positions.len(), 215);
assert_eq!(m.uvs.len(), 215);
assert_eq!(m.normals.len(), 215);
assert_eq!(m.indices.len(), 1092);
// every index in range
assert!(m.indices.iter().all(|&i| (i as usize) < m.positions.len()));
// positions are real geometry within the model's ~2-unit bbox
let ys: Vec<f32> = m.positions.iter().map(|p| p[1]).collect();
let span = ys.iter().cloned().fold(f32::MIN, f32::max)
- ys.iter().cloned().fold(f32::MAX, f32::min);
assert!(span > 1.0 && span < 10.0, "y-span {span} out of range");
// Correct vertex alignment ⇒ normals are unit-length (the pin for the
// +12 vertex offset) and UVs land in a sane texture range.
let mean_nlen: f32 = m
.normals
.iter()
.map(|n| (n[0] * n[0] + n[1] * n[1] + n[2] * n[2]).sqrt())
.sum::<f32>()
/ m.normals.len() as f32;
assert!((mean_nlen - 1.0).abs() < 0.05, "mean |normal| {mean_nlen} ≠ 1");
assert!(
m.uvs.iter().all(|uv| uv[0] > -0.1 && uv[0] < 2.0 && uv[1] > -0.1 && uv[1] < 2.0),
"UVs out of expected [0,1]-ish range"
);
}
#[test]
#[ignore = "requires extracted disc models — set SYLPHEED_RES3D"]
fn declaration_driven_decode_covers_expected_model_count() {
let Some(dir) = res3d_dir() else {
return;
};
let mut ok = 0usize;
let mut total = 0usize;
for entry in std::fs::read_dir(&dir).unwrap() {
let path = entry.unwrap().path();
if path.extension().and_then(|e| e.to_str()) != Some("xpr") {
continue;
}
total += 1;
let bytes = std::fs::read(&path).unwrap();
if let Ok(model) = Xbg7Model::from_xpr2(&bytes) {
if !model.meshes.is_empty() {
// Every decoded model must be self-consistent (indices in range,
// and unit normals where present).
for m in &model.meshes {
assert!(
m.indices.iter().all(|&i| (i as usize) < m.positions.len()),
"{path:?}: index out of range"
);
}
ok += 1;
}
}
}
eprintln!("XBG7 declaration-driven decode: {ok}/{total} models");
// Variable-stride declaration parsing lifted coverage vs the old fixed
// stride-24 decoder (25 → 36 fully-validated models; multi-sub-mesh models
// whose later sub-mesh offset isn't yet handled are still declined whole).
assert!(ok >= 35, "coverage regressed: only {ok}/{total} decoded");
}
#[test]
#[ignore = "requires extracted disc models — set SYLPHEED_RES3D"]
fn complex_body_mesh_is_declined_not_garbage() {
let Some(dir) = res3d_dir() else {
return;
};
// The hero-ship body uses the multi-stream layout we do not decode; it must
// be cleanly rejected, never returned as partial geometry.
let bytes = std::fs::read(dir.join("DeltaSaber_A.xpr")).unwrap();
match Xbg7Model::from_xpr2(&bytes) {
Err(_) => {} // expected: declined
Ok(m) => {
// If it ever does decode, it must at least be self-consistent.
for mesh in &m.meshes {
assert!(mesh.indices.iter().all(|&i| (i as usize) < mesh.positions.len()));
}
}
}
}
/// The hero ship's **grouped-pool** layout decodes fully: `DeltaSaber_T.xpr`'s
/// `f001` resource is one vertex+index pool shared by 8 sub-meshes (body + 7
/// detail parts). Reversed statically and cross-checked against a Canary GPU
/// draw-log capture — every sub-mesh's stored normals agree with its triangle
/// winding (0 degenerate, full vertex coverage). This is the layout the old
/// per-block adjacency anchor rendered as a spiky phantom.
#[test]
#[ignore = "requires extracted disc models — set SYLPHEED_RES3D"]
fn hero_ship_grouped_pool_decodes() {
let Some(dir) = res3d_dir() else {
eprintln!("SKIP: resource3d dir not found (set SYLPHEED_RES3D)");
return;
};
let bytes = std::fs::read(dir.join("DeltaSaber_T.xpr")).unwrap();
let models = Xbg7Model::stage_models(&bytes);
// The neutral pose `f001` (the mnv*/turn180 resources are animation poses).
let f001 = models.iter().find(|m| m.name == "f001").expect("f001 decoded");
assert_eq!(f001.meshes.len(), 8, "body + 7 detail sub-meshes");
let (v, t) = f001.totals();
assert_eq!(v, 11607, "vertex total across sub-meshes");
assert_eq!(t, 8650, "triangle total (body 8187 + 7 parts)");
// Sub-mesh 0 is the body: exactly the draw-log-verified geometry.
let body = &f001.meshes[0];
assert_eq!(body.positions.len(), 10891);
assert_eq!(body.indices.len(), 24561);
for (i, m) in f001.meshes.iter().enumerate() {
// Every index in range.
assert!(
m.indices.iter().all(|&ix| (ix as usize) < m.positions.len()),
"sub{i}: index out of range"
);
// Correct alignment ⇒ unit normals, and the winding agrees with them
// (the decisive correctness signal, ~1.0, not the ~0.5 of a mis-carve).
let n = m.normals.len();
assert_eq!(n, m.positions.len(), "sub{i}: a normal per vertex");
let mean_nlen: f32 = m
.normals
.iter()
.map(|nv| (nv[0] * nv[0] + nv[1] * nv[1] + nv[2] * nv[2]).sqrt())
.sum::<f32>()
/ n as f32;
assert!((mean_nlen - 1.0).abs() < 0.05, "sub{i}: mean |normal| {mean_nlen} ≠ 1");
let mut agree = 0usize;
let mut counted = 0usize;
for tri in m.indices.chunks_exact(3) {
let (a, b, c) = (tri[0] as usize, tri[1] as usize, tri[2] as usize);
let (pa, pb, pc) = (m.positions[a], m.positions[b], m.positions[c]);
let u = [pb[0] - pa[0], pb[1] - pa[1], pb[2] - pa[2]];
let w = [pc[0] - pa[0], pc[1] - pa[1], pc[2] - pa[2]];
let f = [
u[1] * w[2] - u[2] * w[1],
u[2] * w[0] - u[0] * w[2],
u[0] * w[1] - u[1] * w[0],
];
if f[0] * f[0] + f[1] * f[1] + f[2] * f[2] < 1e-12 {
continue;
}
let sn = [
m.normals[a][0] + m.normals[b][0] + m.normals[c][0],
m.normals[a][1] + m.normals[b][1] + m.normals[c][1],
m.normals[a][2] + m.normals[b][2] + m.normals[c][2],
];
if f[0] * sn[0] + f[1] * sn[1] + f[2] * sn[2] > 0.0 {
agree += 1;
}
counted += 1;
}
let na = agree as f32 / counted.max(1) as f32;
assert!(
na.max(1.0 - na) > 0.95,
"sub{i}: winding-vs-normal agreement {na} — mis-carved (should be ~1.0 or ~0.0)"
);
}
}
/// Stage containers decode multiple enemy/prop sub-models via content anchoring.
/// Prints coverage; asserts the known-good Stage_S10 meshes decode with sane geometry.
#[test]
#[ignore]
fn stage_models_decode() {
use sylpheed_formats::mesh::Xbg7Model;
let dir = std::env::var("SYLPHEED_RES3D").unwrap_or_else(|_| {
"/home/fabi/RE Project Sylpheed/sylph_extract/hidden/resource3d".to_string()
});
let path = format!("{dir}/Stage_S10.xpr");
let bytes = std::fs::read(&path).expect("read Stage_S10");
let models = Xbg7Model::stage_models(&bytes);
for m in &models {
let (v, t) = m.totals();
let mut lo = [f32::MAX; 3];
let mut hi = [f32::MIN; 3];
for sub in &m.meshes {
for p in &sub.positions {
for a in 0..3 {
lo[a] = lo[a].min(p[a]);
hi[a] = hi[a].max(p[a]);
}
}
}
let ext = [hi[0] - lo[0], hi[1] - lo[1], hi[2] - lo[2]];
println!(
" {:16} v={v} t={t} bbox=[{:.1},{:.1},{:.1}]",
m.name, ext[0], ext[1], ext[2]
);
}
// Known: e003 (main enemy, ~1400 tris) must be among the decoded models.
let e003 = models.iter().find(|m| m.name == "e003").expect("e003 decoded");
let (v, t) = e003.totals();
assert_eq!(v, 2383, "e003 vertex count");
assert!(t > 1400, "e003 triangle count {t}");
assert!(models.len() >= 3, "at least 3 stage sub-models, got {}", models.len());
}
/// Timing/coverage sweep across all stage containers (manual; release recommended).
#[test]
#[ignore]
fn stage_models_sweep() {
use sylpheed_formats::mesh::Xbg7Model;
use std::time::Instant;
let dir = std::env::var("SYLPHEED_RES3D").unwrap_or_else(|_| {
"/home/fabi/RE Project Sylpheed/sylph_extract/hidden/resource3d".to_string()
});
let mut names: Vec<_> = std::fs::read_dir(&dir)
.unwrap()
.filter_map(|e| e.ok().map(|e| e.file_name().into_string().unwrap()))
.filter(|n| n.starts_with("Stage_") && n.ends_with(".xpr"))
.collect();
names.sort();
let mut tot = 0usize;
for n in &names {
let bytes = std::fs::read(format!("{dir}/{n}")).unwrap();
let t0 = Instant::now();
let models = Xbg7Model::stage_models(&bytes);
let dt = t0.elapsed().as_millis();
tot += models.len();
println!("{n:16} {:>4} MB {:>3} models {:>5} ms", bytes.len()/1_000_000, models.len(), dt);
}
println!("TOTAL stage sub-models decoded: {tot}");
}
/// Quality audit: for a big stage, verify decoded models are real geometry
/// (bounded bbox, low full-mesh degeneracy) rather than false anchors.
#[test]
#[ignore]
fn stage_models_quality_audit() {
use sylpheed_formats::mesh::Xbg7Model;
let dir = std::env::var("SYLPHEED_RES3D").unwrap_or_else(|_| {
"/home/fabi/RE Project Sylpheed/sylph_extract/hidden/resource3d".to_string()
});
let bytes = std::fs::read(format!("{dir}/Stage_S07.xpr")).unwrap();
let models = Xbg7Model::stage_models(&bytes);
let (mut small, mut mid, mut huge, mut dupnames) = (0, 0, 0, 0);
let mut seen = std::collections::HashSet::new();
let mut worst_deg = 0.0f32;
for m in &models {
if !seen.insert(m.name.clone()) { dupnames += 1; }
let mut lo = [f32::MAX; 3]; let mut hi = [f32::MIN; 3];
let mut deg = 0usize; let mut tot = 0usize;
for sub in &m.meshes {
for p in &sub.positions { for a in 0..3 { lo[a]=lo[a].min(p[a]); hi[a]=hi[a].max(p[a]); } }
for tri in sub.indices.chunks_exact(3) {
let (a,b,c)=(tri[0] as usize,tri[1] as usize,tri[2] as usize);
if a>=sub.positions.len()||b>=sub.positions.len()||c>=sub.positions.len(){continue;}
let pa=sub.positions[a]; let pb=sub.positions[b]; let pc=sub.positions[c];
let u=[pb[0]-pa[0],pb[1]-pa[1],pb[2]-pa[2]];
let v=[pc[0]-pa[0],pc[1]-pa[1],pc[2]-pa[2]];
let cx=[u[1]*v[2]-u[2]*v[1],u[2]*v[0]-u[0]*v[2],u[0]*v[1]-u[1]*v[0]];
if 0.5*(cx[0]*cx[0]+cx[1]*cx[1]+cx[2]*cx[2]).sqrt()<1e-9 { deg+=1; }
tot+=1;
}
}
let ext = (hi[0]-lo[0]).max(hi[1]-lo[1]).max(hi[2]-lo[2]);
let df = if tot>0 { deg as f32/tot as f32 } else {1.0};
worst_deg = worst_deg.max(df);
if ext < 200.0 { small += 1; } else if ext < 5000.0 { mid += 1; } else { huge += 1; }
}
println!("S07: {} models | small(<200u)={} mid={} huge(>5k)={} | dup-names={} | worst full-mesh degeneracy={:.1}%",
models.len(), small, mid, huge, dupnames, worst_deg*100.0);
// Each XBG7 resource must anchor to a *distinct* block (no collisions), the
// bulk must be bounded-scale geometry, and none may be mostly-degenerate.
assert_eq!(dupnames, 0, "no two resources should anchor to the same block");
assert!(small + mid > models.len() * 9 / 10, "≥90% bounded-scale geometry");
assert!(huge < models.len() / 20, "few huge (skybox-plane) models");
assert!(worst_deg < 0.35, "no model should be mostly-degenerate");
}

102
crates/sylpheed-formats/tests/movie_manifest_disc.rs Normal file
View File

@@ -0,0 +1,102 @@
//! Real-disc test for the movie manifest → voice binding. Skipped without
//! `SYLPHEED_DISC`.
use std::path::PathBuf;
use sylpheed_formats::movie_manifest;
use sylpheed_formats::slb::VoiceLang;
use sylpheed_formats::PakArchive;
fn disc_root() -> Option<PathBuf> {
let p = PathBuf::from(std::env::var("SYLPHEED_DISC").ok()?);
p.join("dat").is_dir().then_some(p)
}
/// Read the manifest + `eng\sounds.tbl` out of `tables.pak`.
fn load_manifest_and_sounds(root: &PathBuf) -> (Vec<u8>, Vec<u8>) {
let pak = PakArchive::open(root.join("dat/tables.pak")).unwrap();
let manifest = pak
.entries()
.iter()
.find_map(|e| pak.read(e).ok().filter(|b| movie_manifest::is_manifest(b)))
.expect("movie manifest present in tables.pak");
let sounds = pak
.read_by_name("eng\\sounds.tbl")
.expect("sounds.tbl present")
.unwrap();
(manifest, sounds)
}
#[test]
fn binds_and_resolves_movie_voice() {
let Some(root) = disc_root() else {
eprintln!("SKIP: set SYLPHEED_DISC");
return;
};
let (manifest, sounds) = load_manifest_and_sounds(&root);
let entries = movie_manifest::parse(&manifest);
assert!(entries.len() > 90, "expected ~101 movies, got {}", entries.len());
// Standard story movie → VOICE_<movie> in <lang>\Movie\.
assert_eq!(
movie_manifest::voice_token(&manifest, "S13A").as_deref(),
Some("VOICE_S13A")
);
assert_eq!(
movie_manifest::resolve_voice_entry(&manifest, &sounds, "S13A", VoiceLang::English)
.as_deref(),
Some("eng\\Movie\\VOICE_S13A.slb")
);
// A resupply movie bound to an in-mission radio clip → VOICE_D_* in \etc\.
// (This is what a `VOICE_<movie>` guess would miss entirely.)
assert_eq!(
movie_manifest::voice_token(&manifest, "hokyu_LS_s02A").as_deref(),
Some("VOICE_D_450")
);
assert_eq!(
movie_manifest::resolve_voice_entry(
&manifest,
&sounds,
"hokyu_LS_s02A",
VoiceLang::English
)
.as_deref(),
Some("eng\\etc\\VOICE_D_450.slb")
);
// A resupply movie the game leaves unbound in the manifest → no DIRECT
// binding. (Extending to unbound movies by shared demo line was verified
// WRONG against the running game, so we do NOT resolve these.)
let e = entries
.iter()
.find(|e| e.movie == "hokyu_DS_s13A")
.expect("hokyu_DS_s13A present in manifest");
assert_eq!(e.voice_token, None, "hokyu_DS_s13A has no direct voice binding");
assert_eq!(
movie_manifest::resolve_voice_entry(&manifest, &sounds, "hokyu_DS_s13A", VoiceLang::English),
None
);
// Every resolved voice entry must actually exist in sound.pak.
let snd = std::fs::read(root.join("dat/sound.pak")).unwrap();
let keys: std::collections::HashSet<u32> = PakArchive::parse_toc(&snd)
.unwrap()
.iter()
.map(|e| e.name_hash)
.collect();
let mut resolved = 0;
let mut missing = Vec::new();
for e in &entries {
if let Some(full) =
movie_manifest::resolve_voice_entry(&manifest, &sounds, &e.movie, VoiceLang::English)
{
resolved += 1;
if !keys.contains(&sylpheed_formats::hash::name_hash(&full)) {
missing.push(full);
}
}
}
assert!(resolved > 80, "expected 80+ voiced movies, got {resolved}");
assert!(missing.is_empty(), "resolved but absent in sound.pak: {missing:?}");
}

63
crates/sylpheed-formats/tests/movie_subtitle_disc.rs Normal file
View File

@@ -0,0 +1,63 @@
//! Real-disc test for the movie subtitle chain. Skipped when the extracted disc
//! is absent (set `SYLPHEED_DISC` to the extract root to enable).
use std::path::PathBuf;
use sylpheed_formats::movie_subtitle::{self, SubLang};
use sylpheed_formats::PakArchive;
fn disc_root() -> Option<PathBuf> {
let p = PathBuf::from(std::env::var("SYLPHEED_DISC").ok()?);
p.join("dat").is_dir().then_some(p)
}
#[test]
fn resolves_english_radio_subtitles() {
let Some(root) = disc_root() else {
eprintln!("SKIP: set SYLPHEED_DISC");
return;
};
let lang = SubLang::English;
let lang_pak = PakArchive::open(root.join(format!("dat/movie/{}.pak", lang.pak_code()))).unwrap();
let text_pak =
PakArchive::open(root.join(format!("dat/GP_MAIN_GAME_{}.pak", lang.game_code()))).unwrap();
// A radio movie: real timed English dialogue.
let cues = movie_subtitle::load("RT01C_1", &lang_pak, &text_pak);
assert!(!cues.is_empty(), "RT01C_1 should have timed cues");
assert!(cues[0].start >= 0.0 && cues[0].start < 5.0);
assert!(
cues.iter().any(|c| c.text.contains("Rhino Leader")),
"expected recognizable dialogue, got: {:?}",
cues.iter().map(|c| &c.text).collect::<Vec<_>>()
);
// No MSG_DEMO keys should leak into resolved caption text.
assert!(
cues.iter().all(|c| !c.text.contains("MSG_DEMO")),
"unresolved key leaked: {:?}",
cues.iter().map(|c| &c.text).collect::<Vec<_>>()
);
// A resupply movie: the canonical resupply line.
let hokyu = movie_subtitle::load("hokyu_DS_s02A", &lang_pak, &text_pak);
assert!(hokyu.iter().any(|c| c.text.contains("Resupply complete")));
// S00A carries inline narration text (not MSG_DEMO refs) with timecodes.
let story = movie_subtitle::load("S00A", &lang_pak, &text_pak);
assert!(!story.is_empty() && story.iter().any(|c| c.text.contains("27th century")));
// S13A stores a two-line caption as two consecutive text tokens sharing one
// timing; both lines must survive (the "Look at it father" regression — the
// first line used to be dropped).
let s13 = movie_subtitle::load("S13A", &lang_pak, &text_pak);
let multiline = s13
.iter()
.find(|c| c.text.contains("Look at it father"))
.expect("S13A should contain the 'Look at it father' caption");
assert_eq!(
multiline.text, "Look at it father\n& beautiful isn't it",
"both lines of the caption must be present"
);
assert!((multiline.start - 74.8).abs() < 0.1, "start {}", multiline.start);
}

137
crates/sylpheed-formats/tests/pak_idxd_disc.rs
View File

@@ -6,6 +6,7 @@
use std::path::{Path, PathBuf};
use sylpheed_formats::texture::X360Texture;
use sylpheed_formats::{IdxdObject, PakArchive};
/// Locate the extracted disc root, or `None` to skip.
@@ -122,3 +123,139 @@ fn name_lookup_resolves_weapon_tbl() {
let obj = IdxdObject::parse(&bytes).expect("weapon.tbl parses as IDXD");
assert!(obj.count > 0);
}
/// Real T8aD entries decode to sane RGBA surfaces, and a good fraction of the
/// pack's T8aD entries are decodable (the RGBA-variant rule holds broadly).
#[test]
fn hangar_t8ad_textures_decode() {
skip_without_disc!(root);
let arc = PakArchive::open(root.join("dat/GP_HANGAR_ARSENAL.pak")).unwrap();
let (mut seen, mut decoded) = (0usize, 0usize);
for e in arc.entries() {
if e.comp_size > 4_000_000 {
continue;
}
let Ok(p) = arc.read(e) else { continue };
if !sylpheed_formats::t8ad::is_t8ad(&p) {
continue;
}
seen += 1;
if let Some(img) = sylpheed_formats::t8ad::parse(&p) {
decoded += 1;
assert!((1..=4096).contains(&img.width) && (1..=4096).contains(&img.height));
assert_eq!(img.rgba.len(), (img.width * img.height * 4) as usize);
}
}
assert!(seen > 100, "expected many T8aD, saw {seen}");
// The RGBA rule should cover the large majority (≈85% disc-wide).
assert!(decoded * 100 >= seen * 70, "decoded {decoded}/{seen}");
}
/// A real LSTA sprite list yields inline T8aD frames.
#[test]
fn lsta_sprite_list_decodes() {
skip_without_disc!(root);
let arc = PakArchive::open(root.join("dat/GP_HANGAR_ARSENAL.pak")).unwrap();
let mut found = false;
for e in arc.entries() {
let Ok(p) = arc.read(e) else { continue };
if !sylpheed_formats::lsta::is_lsta(&p) {
continue;
}
if let Some(frames) = sylpheed_formats::lsta::parse(&p) {
if !frames.is_empty() {
found = true;
assert!(frames.iter().all(|f| !f.rgba.is_empty()));
}
}
}
assert!(found, "no decodable LSTA found");
}
/// A real RATC bundle lists named children including a decodable T8aD.
#[test]
fn ratc_bundle_lists_children() {
skip_without_disc!(root);
let arc = PakArchive::open(root.join("dat/GP_HANGAR_ARSENAL.pak")).unwrap();
let mut ok = false;
for e in arc.entries() {
if e.comp_size > 4_000_000 {
continue;
}
let Ok(p) = arc.read(e) else { continue };
if !sylpheed_formats::ratc::is_ratc(&p) {
continue;
}
let Some(kids) = sylpheed_formats::ratc::parse(&p) else { continue };
let has_named = kids.iter().any(|c| c.name.contains('.'));
let has_t8ad = kids
.iter()
.any(|c| c.kind == "T8aD" && sylpheed_formats::t8ad::parse(&p[c.offset..c.offset + c.size]).is_some());
if !kids.is_empty() && has_named && has_t8ad {
ok = true;
break;
}
}
assert!(ok, "no RATC with named children + a decodable T8aD");
}
/// The English movie subtitle track (an `IXUD` entry) decodes to timed cues.
#[test]
fn eng_movie_subtitle_track() {
skip_without_disc!(root);
let arc = PakArchive::open(root.join("dat/movie/eng.pak")).unwrap();
// S04A's track — the longest inline English one (starts "Calm down!").
let bytes = arc.read_by_hash(0x73d2_2a3b).expect("entry present").unwrap();
let sub = sylpheed_formats::ixud::parse(&bytes).expect("IXUD parses as subtitle");
assert!(sub.cues.len() > 50, "cues={}", sub.cues.len());
assert!(!sub.is_reference_only());
assert_eq!(sub.cues[0].text, "Calm down!");
assert!((sub.cues[0].start - 9.4).abs() < 0.01);
assert_eq!(sub.cues[0].end, Some(11.0));
}
/// The subtitle tracks are genuinely localized: the same entry hash differs
/// between the English and German packs.
#[test]
fn subtitles_are_localized() {
skip_without_disc!(root);
let eng = PakArchive::open(root.join("dat/movie/eng.pak")).unwrap();
let deu = PakArchive::open(root.join("dat/movie/deu.pak")).unwrap();
let e = eng.read_by_hash(0x73d2_2a3b).unwrap().unwrap();
let d = deu.read_by_hash(0x73d2_2a3b).unwrap().unwrap();
assert_ne!(e, d, "eng/deu subtitle payloads should differ");
}
/// The English movie pack's embedded subtitle font parses to sane metadata.
#[test]
fn eng_movie_font_metadata() {
skip_without_disc!(root);
let arc = PakArchive::open(root.join("dat/movie/eng.pak")).unwrap();
let bytes = arc.read_by_hash(0x5cd0_fca6).expect("entry present").unwrap();
assert!(sylpheed_formats::font::is_font(&bytes));
let info = sylpheed_formats::font::parse_info(&bytes).expect("font parses");
assert!(info.glyphs > 0, "glyphs={}", info.glyphs);
assert!(!info.family.is_empty(), "family should be readable");
}
/// The Acheron backdrop is a TXCM world cubemap: 6 faces, and the cube path's
/// face 0 matches the (validated) 2D `from_xpr2` face-0 decode.
#[test]
fn acheron_world_cubemap_six_faces() {
skip_without_disc!(root);
let bytes = std::fs::read(root.join("hidden/resource3d/BG_Acheron.xpr")).unwrap();
let cube = X360Texture::cube_faces_from_xpr2(&bytes)
.expect("cube decode ok")
.expect("BG_Acheron is a TXCM cubemap");
assert_eq!((cube.width, cube.height), (1024, 1024));
assert_eq!(cube.faces.len(), 6);
// A8R8G8B8 → 4 bytes/texel, de-tiled to width×height.
for face in &cube.faces {
assert_eq!(face.len(), 1024 * 1024 * 4);
}
// Face 0 must equal what the existing 2D path decodes (the green planet).
let face0_2d = X360Texture::from_xpr2(&bytes).unwrap();
assert!(face0_2d.is_cubemap);
assert_eq!(cube.faces[0], face0_2d.data);
}

104
crates/sylpheed-formats/tests/slb_disc.rs Normal file
View File

@@ -0,0 +1,104 @@
//! Real-disc test: read a movie voice `.slb` out of the segmented `sound.pak`
//! and rebuild a decodable XMA1 RIFF. Skipped without `SYLPHEED_DISC`.
use std::fs::File;
use std::io::{Read, Seek, SeekFrom};
use std::path::PathBuf;
use sylpheed_formats::hash::name_hash;
use sylpheed_formats::slb::{self, VoiceLang};
use sylpheed_formats::PakArchive;
fn disc_root() -> Option<PathBuf> {
let p = PathBuf::from(std::env::var("SYLPHEED_DISC").ok()?);
p.join("dat").is_dir().then_some(p)
}
/// Read `[off, off+size)` from `dat/sound.p00..` (segments concatenated).
fn read_range(root: &PathBuf, mut off: u64, size: usize) -> Vec<u8> {
let mut out = Vec::with_capacity(size);
let mut need = size;
for i in 0..100u32 {
if need == 0 {
break;
}
let seg = root.join(format!("dat/sound.p{i:02}"));
let Ok(meta) = std::fs::metadata(&seg) else { break };
let len = meta.len();
if off >= len {
off -= len;
continue;
}
let mut f = File::open(&seg).unwrap();
f.seek(SeekFrom::Start(off)).unwrap();
let take = need.min((len - off) as usize);
let start = out.len();
out.resize(start + take, 0);
f.read_exact(&mut out[start..]).unwrap();
need -= take;
off = 0;
}
out
}
#[test]
fn reads_and_rebuilds_movie_voice_riff() {
let Some(root) = disc_root() else {
eprintln!("SKIP: set SYLPHEED_DISC");
return;
};
let toc = std::fs::read(root.join("dat/sound.pak")).unwrap();
let entries = PakArchive::parse_toc(&toc).unwrap();
assert_eq!(entries.len(), 9519);
for movie in ["S00A", "RT07A", "RT02D_1"] {
// RT02D_1 is a headerless variant; S00A/RT07A embed a RIFF.
let key = name_hash(&slb::movie_voice_name(movie, VoiceLang::English));
let idx = entries
.binary_search_by_key(&key, |e| e.name_hash)
.unwrap_or_else(|_| panic!("{movie} voice missing"));
let e = &entries[idx];
let bank = read_range(&root, e.offset as u64, e.comp_size as usize);
let riff = slb::to_xma_riff(&bank).expect("rebuild RIFF");
assert_eq!(&riff[0..4], b"RIFF", "{movie}");
assert_eq!(&riff[8..12], b"WAVE", "{movie}");
// fmt chunk advertises XMA1 (tag 0x0165).
let fpos = riff.windows(4).position(|w| w == b"fmt ").unwrap();
let tag = u16::from_le_bytes([riff[fpos + 8], riff[fpos + 9]]);
assert_eq!(tag, 0x0165, "{movie} should be XMA1");
// data chunk carries the XMA payload.
let dpos = riff.windows(4).position(|w| w == b"data").unwrap();
let dsz = u32::from_le_bytes([
riff[dpos + 4],
riff[dpos + 5],
riff[dpos + 6],
riff[dpos + 7],
]) as usize;
assert!(dsz > 10_000, "{movie} data too small: {dsz}");
assert_eq!(riff.len(), dpos + 8 + dsz, "{movie} data length consistent");
}
}
#[test]
fn enumerates_voice_clips_from_sounds_tbl() {
let Some(root) = disc_root() else {
eprintln!("SKIP: set SYLPHEED_DISC");
return;
};
let pak = PakArchive::open(root.join("dat/tables.pak")).unwrap();
let tbl = pak.read_by_name("eng\\sounds.tbl").unwrap().unwrap();
let clips = slb::list_voice_clips(&tbl, VoiceLang::English);
assert!(clips.len() > 1000, "expected many voice clips, got {}", clips.len());
// Every clip is an eng voice path present in sound.pak.
let sound = std::fs::read(root.join("dat/sound.pak")).unwrap();
let keys: std::collections::HashSet<u32> =
PakArchive::parse_toc(&sound).unwrap().iter().map(|e| e.name_hash).collect();
let present = clips.iter().filter(|c| keys.contains(&name_hash(&c.name))).count();
assert!(
present as f32 / clips.len() as f32 > 0.95,
"{present}/{} clips resolve in sound.pak",
clips.len()
);
assert!(clips.iter().any(|c| c.speaker == "ADAN"));
}

140
crates/sylpheed-formats/tests/texture_disc.rs Normal file
View File

@@ -0,0 +1,140 @@
//! Integration test: run the XPR2 texture pipeline against REAL `.xpr` files
//! read directly from the retail disc image, reproducing exactly what the
//! viewer's texture-preview path does (`identify_format` → `X360Texture::
//! from_xpr2`). Skipped unless `SYLPHEED_ISO` points at the disc (or the
//! default dev path exists).
//!
//! Run: `cargo test -p sylpheed-formats --test texture_disc -- --ignored --nocapture`
use std::path::PathBuf;
use sylpheed_formats::texture::X360Texture;
use sylpheed_formats::vfs::identify_format;
fn iso_path() -> Option<PathBuf> {
if let Ok(p) = std::env::var("SYLPHEED_ISO") {
let p = PathBuf::from(p);
if p.is_file() {
return Some(p);
}
}
let default = PathBuf::from(
"/home/fabi/RE Project Sylpheed/Project Sylpheed - Arc of Deception (USA, Europe) (En,Ja).iso",
);
default.is_file().then_some(default)
}
#[tokio::test]
#[ignore = "requires the retail ISO — set SYLPHEED_ISO"]
async fn xpr_pipeline_over_disc_sample() {
let Some(iso) = iso_path() else {
eprintln!("SKIP: ISO not found (set SYLPHEED_ISO)");
return;
};
let mut reader = sylpheed_formats::xiso::open_iso(&iso).await.unwrap();
let all = reader.list_all_files().await.unwrap();
let mut xprs: Vec<String> = all
.into_iter()
.filter(|f| f.to_lowercase().ends_with(".xpr"))
.collect();
xprs.sort();
println!("found {} .xpr files", xprs.len());
// Sample across the set so we hit different texture sizes/formats.
let sample: Vec<String> = xprs.iter().step_by((xprs.len() / 24).max(1)).cloned().collect();
let mut ok = 0usize;
let mut fail = 0usize;
let mut by_format: std::collections::BTreeMap<String, usize> = Default::default();
let mut nonxpr = 0usize;
// Optionally dump the sampled .xpr bytes so the CLI can export them to PNG
// for visual de-tiling validation: `DUMP_XPR_DIR=/tmp/xpr cargo test …`.
let dump_dir = std::env::var("DUMP_XPR_DIR").ok();
if let Some(d) = &dump_dir {
std::fs::create_dir_all(d).unwrap();
}
for path in &sample {
let bytes = reader.read_file(path).await.unwrap();
if let Some(d) = &dump_dir {
let base = path.rsplit('/').next().unwrap();
std::fs::write(format!("{d}/{base}"), &bytes).unwrap();
}
let fmt = identify_format(&bytes);
let magic: String = bytes
.iter()
.take(4)
.map(|b| if b.is_ascii_graphic() { *b as char } else { '.' })
.collect();
if fmt != sylpheed_formats::vfs::FileFormat::Xpr2Texture {
nonxpr += 1;
println!(" [{path}] NOT XPR2 (magic {magic:?}, {} bytes)", bytes.len());
continue;
}
match X360Texture::from_xpr2(&bytes) {
Ok(t) => {
ok += 1;
*by_format.entry(format!("{:?}", t.format)).or_default() += 1;
// Sanity: does the decoded data length match the descriptor
// dimensions (what the GPU upload will require)?
let bs = t.format.block_size() as u32;
let bw = ((t.width + bs - 1) / bs).max(1) as usize;
let bh = ((t.height + bs - 1) / bs).max(1) as usize;
let need = bw * bh * t.format.bytes_per_block();
let size_ok = if need == t.data.len() { "ok" } else { "MISMATCH" };
println!(
" [{path}] {:?} {}x{} mips={} data={} need={} {size_ok}",
t.format, t.width, t.height, t.mip_levels, t.data.len(), need
);
}
Err(e) => {
fail += 1;
println!(" [{path}] from_xpr2 FAILED: {e}");
dump_xpr2_structure(&bytes);
}
}
}
println!("\nSUMMARY: ok={ok} fail={fail} non-xpr2={nonxpr} formats={by_format:?}");
}
/// Dump the XPR2 header + resource directory the way `from_xpr2` reads it,
/// so we can see why a file's TX2D scan comes up empty.
fn dump_xpr2_structure(bytes: &[u8]) {
let be = |o: usize| u32::from_be_bytes(bytes[o..o + 4].try_into().unwrap());
let magic: String = bytes[..4].iter().map(|b| *b as char).collect();
println!(
" hdr magic={magic:?} header_size=0x{:X} data_size=0x{:X} num_resources={}",
be(0x04),
be(0x08),
be(0x0C)
);
let n = be(0x0C).min(16); // guard against garbage counts
for i in 0..n {
let base = 0x10 + i as usize * 16;
if base + 16 > bytes.len() {
break;
}
let tag: String = bytes[base..base + 4]
.iter()
.map(|b| if b.is_ascii_graphic() { *b as char } else { '.' })
.collect();
println!(
" res[{i}] tag={tag:?} data_off=0x{:X} desc_size=0x{:X} name_off=0x{:X}",
be(base + 4),
be(base + 8),
be(base + 12)
);
}
// First 48 bytes hex for orientation.
let hx: String = bytes
.iter()
.take(48)
.map(|b| format!("{b:02X} "))
.collect();
println!(" hex[0..48] {hx}");
}

6
crates/sylpheed-viewer/Cargo.toml
View File

@@ -50,3 +50,9 @@ dev = ["bevy/dynamic_linking"]
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
tracing-subscriber = { workspace = true }
rfd = { workspace = true }
# Audio for the WMV video player (pulls cpal → alsa on Linux).
rodio = { workspace = true }
# PNG decode for pak image entries (pure Rust; Bevy already pulls it in).
image = { version = "0.25", default-features = false, features = ["png"] }
# Off-thread rasterization of embedded-font samples (avoids egui's global fonts).
ab_glyph = "0.2"

26
crates/sylpheed-viewer/src/asset_loader.rs
View File

@@ -93,8 +93,28 @@ impl AssetLoader for Xpr2TextureLoader {
pub fn x360_texture_to_bevy_image(tex: X360Texture) -> Result<Image, Xpr2LoadError> {
let format = x360_format_to_wgpu(&tex.format)?;
// Uncompressed k_8_8_8_8: after `from_xpr2`'s k8in32 endian swap the bytes
// are in [A,R,G,B] order (verified against the retail Acheron backdrop).
// wgpu has no ARGB format, so reorder to [R,G,B,A] and upload as Rgba8
// (see `x360_format_to_wgpu`). BCn data is already GPU-ready.
let data = match tex.format {
X360TextureFormat::A8R8G8B8 | X360TextureFormat::X8R8G8B8 => {
let opaque = matches!(tex.format, X360TextureFormat::X8R8G8B8);
let mut out = tex.data;
for px in out.chunks_exact_mut(4) {
let (a, r, g, b) = (px[0], px[1], px[2], px[3]);
px[0] = r;
px[1] = g;
px[2] = b;
px[3] = if opaque { 0xFF } else { a };
}
out
}
_ => tex.data,
};
Ok(Image {
data: tex.data,
data,
texture_descriptor: TextureDescriptor {
label: None,
size: Extent3d {
@@ -132,9 +152,9 @@ fn x360_format_to_wgpu(format: &X360TextureFormat) -> Result<TextureFormat, Xpr2
X360TextureFormat::Dxn => TextureFormat::Bc5RgUnorm,
// DXT5A / BC4 — single-channel (gloss, specular, luminance maps)
X360TextureFormat::Dxt5A => TextureFormat::Bc4RUnorm,
// Uncompressed ARGB — Xbox 360 stores as BGRA in memory
// Uncompressed — reordered to [R,G,B,A] by `x360_texture_to_bevy_image`.
X360TextureFormat::A8R8G8B8 | X360TextureFormat::X8R8G8B8 => {
TextureFormat::Bgra8UnormSrgb
TextureFormat::Rgba8UnormSrgb
}
})
}

108
crates/sylpheed-viewer/src/camera.rs
View File

@@ -8,6 +8,10 @@
use bevy::prelude::*;
use bevy::input::mouse::{MouseMotion, MouseWheel};
use bevy::render::render_asset::RenderAssetUsages;
use bevy::render::render_resource::{
Extent3d, TextureDimension, TextureFormat, TextureViewDescriptor, TextureViewDimension,
};
pub struct OrbitCameraPlugin;
@@ -32,6 +36,10 @@ pub struct OrbitCamera {
pub orbit_sensitivity: f32,
pub zoom_sensitivity: f32,
pub pan_sensitivity: f32,
/// Zoom limits — set when framing a model so large scenes can be pulled back
/// far enough (the old fixed 0.5..50 cap trapped the camera inside big stages).
pub min_radius: f32,
pub max_radius: f32,
}
impl Default for OrbitCamera {
@@ -44,29 +52,114 @@ impl Default for OrbitCamera {
orbit_sensitivity: 0.005,
zoom_sensitivity: 0.3,
pan_sensitivity: 0.003,
min_radius: 0.05,
max_radius: 500.0,
}
}
}
fn spawn_camera(mut commands: Commands) {
fn spawn_camera(mut commands: Commands, mut images: ResMut<Assets<Image>>) {
let orbit = OrbitCamera::default();
let transform = orbit_transform(&orbit);
// The game's ship shader reflects a shared HDR environment cubemap off the
// hull (three cube samples — see the shader RE). Reproduce that "look" with a
// procedural sky cube feeding Bevy's image-based lighting, so metallic
// surfaces reflect an environment instead of reading flat. Procedural (not a
// bundled KTX2) so it also works on WASM with no extra assets.
let env = make_env_cubemap(&mut images);
commands.spawn((
Camera3d::default(),
// Wide near/far so both tiny weapons and multi-thousand-unit stages fit;
// the planes are re-scaled to the zoom distance each frame (below).
Projection::Perspective(PerspectiveProjection {
near: 0.05,
far: 100_000.0,
..default()
}),
transform,
orbit,
EnvironmentMapLight {
diffuse_map: env.clone(),
specular_map: env,
intensity: 900.0,
..default()
},
));
}
/// Build a small procedural sky cubemap (6 faces) for image-based lighting.
///
/// A vertical gradient (bright zenith → warm horizon → dark ground) plus a warm
/// "sun" highlight roughly where the game's key light sits — enough for a
/// metallic hull to read as reflective rather than flat. Stored as an sRGB cube
/// (filterable, no half-float encoding needed); a single mip means reflections
/// stay sharp (fine for a shiny ship).
fn make_env_cubemap(images: &mut Assets<Image>) -> Handle<Image> {
let size: i32 = 64;
// Approx key-light direction from the RE (PS c32 ≈ (0,-0.76,-0.65), i.e. light
// arriving from top-front); place the bright spot there.
let sun = Vec3::new(0.0, 0.85, 0.5).normalize();
let zenith = Vec3::new(0.70, 0.80, 0.95);
let horizon = Vec3::new(0.42, 0.45, 0.50);
let ground = Vec3::new(0.09, 0.09, 0.11);
let srgb = |c: f32| (c.clamp(0.0, 1.0).powf(1.0 / 2.2) * 255.0).round() as u8;
let mut data: Vec<u8> = Vec::with_capacity((size * size * 6 * 4) as usize);
for face in 0..6 {
for y in 0..size {
for x in 0..size {
let u = (x as f32 + 0.5) / size as f32 * 2.0 - 1.0;
let v = (y as f32 + 0.5) / size as f32 * 2.0 - 1.0;
// Standard cube-face → direction mapping.
let d = match face {
0 => Vec3::new(1.0, -v, -u),
1 => Vec3::new(-1.0, -v, u),
2 => Vec3::new(u, 1.0, v),
3 => Vec3::new(u, -1.0, -v),
4 => Vec3::new(u, -v, 1.0),
_ => Vec3::new(-u, -v, -1.0),
}
.normalize();
let mut col = if d.y >= 0.0 {
horizon.lerp(zenith, (d.y).powf(0.6))
} else {
horizon.lerp(ground, (-d.y).powf(0.5))
};
let s = d.dot(sun).max(0.0).powf(60.0);
col += Vec3::new(1.0, 0.85, 0.6) * s * 1.5;
data.extend_from_slice(&[srgb(col.x), srgb(col.y), srgb(col.z), 255]);
}
}
}
let mut image = Image::new(
Extent3d {
width: size as u32,
height: size as u32,
depth_or_array_layers: 6,
},
TextureDimension::D2,
data,
TextureFormat::Rgba8UnormSrgb,
RenderAssetUsages::RENDER_WORLD,
);
image.texture_view_descriptor = Some(TextureViewDescriptor {
dimension: Some(TextureViewDimension::Cube),
..default()
});
images.add(image)
}
fn orbit_camera(
mut query: Query<(&mut OrbitCamera, &mut Transform)>,
mut query: Query<(&mut OrbitCamera, &mut Transform, &mut Projection)>,
mouse_buttons: Res<ButtonInput<MouseButton>>,
keys: Res<ButtonInput<KeyCode>>,
mut mouse_motion: EventReader<MouseMotion>,
mut scroll: EventReader<MouseWheel>,
) {
let Ok((mut cam, mut transform)) = query.get_single_mut() else { return };
let Ok((mut cam, mut transform, mut projection)) = query.get_single_mut() else { return };
let mut delta_motion = Vec2::ZERO;
for ev in mouse_motion.read() {
@@ -99,13 +192,20 @@ fn orbit_camera(
// Zoom (scroll)
cam.radius -= scroll_delta * cam.zoom_sensitivity * cam.radius;
cam.radius = cam.radius.clamp(0.5, 50.0);
cam.radius = cam.radius.clamp(cam.min_radius, cam.max_radius);
// Reset (R key)
if keys.just_pressed(KeyCode::KeyR) {
*cam = OrbitCamera::default();
}
// Keep the clip planes proportional to the zoom distance so depth precision
// stays usable across scales (tiny prop → whole stage grid).
if let Projection::Perspective(p) = projection.as_mut() {
p.near = (cam.radius * 0.02).clamp(0.02, 50.0);
p.far = (cam.radius * 50.0).max(2000.0);
}
*transform = orbit_transform(&cam);
}

4088
crates/sylpheed-viewer/src/iso_loader.rs
View File

File diff suppressed because it is too large Load Diff

50
crates/sylpheed-viewer/src/lib.rs
View File

@@ -28,33 +28,16 @@ pub mod ui;
/// Global viewer state, shared across all UI and rendering systems.
#[derive(Resource)]
pub struct ViewerState {
/// Which type of asset is currently being browsed / previewed.
pub current_mode: ViewMode,
/// Whether to render meshes in wireframe mode (toggle with F2).
pub wireframe: bool,
/// Whether the plain-text viewer wraps long lines.
pub text_wrap: bool,
}
impl Default for ViewerState {
fn default() -> Self {
Self {
current_mode: ViewMode::Textures,
wireframe: false,
}
Self { text_wrap: true }
}
}
/// The active view / asset type being inspected.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum ViewMode {
/// XPR2 texture preview (Milestone 1).
#[default]
Textures,
/// 3D mesh viewer (Milestone 2+).
Mesh,
/// Audio waveform / playback (Milestone 2+).
Audio,
}
// ── App builder ──────────────────────────────────────────────────────────────
/// Build and run the viewer application.
@@ -89,9 +72,28 @@ pub fn run() {
}
fn setup_scene(mut commands: Commands) {
commands.spawn(DirectionalLight {
illuminance: 10_000.0,
shadows_enabled: true,
..default()
// Bright ambient so surfaces facing away from every light aren't pure black.
commands.insert_resource(AmbientLight {
color: Color::srgb(0.9, 0.93, 1.0),
brightness: 900.0,
});
// A multi-directional rig (key + fills + rim + underside) so an orbiting
// camera always has some light on the visible side — the single front light
// left the backside unreadable.
let lights = [
(Vec3::new(1.0, 2.0, 1.5), 10_000.0, true), // key: front-top-right
(Vec3::new(-2.0, 1.0, 0.5), 4_500.0, false), // fill: left
(Vec3::new(0.5, 0.8, -2.0), 6_000.0, false), // rim: behind
(Vec3::new(0.0, -1.5, 0.5), 2_500.0, false), // underside fill
];
for (from, lux, shadows) in lights {
commands.spawn((
DirectionalLight {
illuminance: lux,
shadows_enabled: shadows,
..default()
},
Transform::from_translation(from).looking_at(Vec3::ZERO, Vec3::Y),
));
}
}

1527
crates/sylpheed-viewer/src/ui.rs
View File

File diff suppressed because it is too large Load Diff

95
docs/HANDOFF-movie-voice-subtitles-2026-07-19.md Normal file
View File

@@ -0,0 +1,95 @@
# Handoff — movie subtitles, voice, and the movie manifest (2026-07-19)
Branch `feature/ipfb-idxd-parser`. This commit flushes several sessions of local
WIP; the **new, finished** work is the movie subtitle + voice + audio pipeline
and the movie manifest. One item is deliberately **left open** (see §4).
## 1. What's DONE and verified (static RE + tests)
### Movie subtitles (`crates/sylpheed-formats/src/movie_subtitle.rs`)
- Full movie→track→text chain (see `docs/re/structures/movie-subtitles.md`).
- **Multi-line caption fix**: a caption stored as several consecutive text tokens
sharing one timing (S13A: `"Look at it father"` + `"& beautiful isn't it"` @
`01:14.80`) — `parse_track` now joins them with `\n`. Previously all but the
last line were dropped. Disc test asserts both lines survive.
- **Overlap rendering**: `MovieSubtitles::active_cues(t)` returns every cue active
at `t`; the viewer stacks them (was: only the first cue shown).
- Umlauts / Latin-1 accents preserved (`utf16le_tokens`); Japanese label romanized
(CJK font still a known gap — see §5).
### Voice decode (`crates/sylpheed-formats/src/slb.rs`)
- `sound.pak` entries are XACT `.slb` banks wrapping **XMA1** (fmt tag `0x0165`,
48 kHz, mono content). `to_xma_riff` rebuilds a decodable RIFF; FFmpeg `xma1`
decodes it. Downmix mono via `-af pan=mono|c0=c0`.
- **Multi-subwave fix**: take the FIRST sub-wave bounded by its declared `data`
size — NOT `data..EOF` (which appended later takes = the S10–S16 garble).
Verified: S13A→83.78s == video.
- `list_voice_clips` enumerates the voice library from `sounds.tbl`.
### Movie manifest (`crates/sylpheed-formats/src/movie_manifest.rs`) — the index
- `tables.pak` entry `ADVERTISE_MOVIE` (IDXD, hash `0x5B983A08`) is the
authoritative **movie → subtitle → voice** map. Located by shape (`is_manifest`),
parsed by grouping the string pool on `.wmv`.
- **Authoritative voice binding** replaces the old `VOICE_<movie>` guess:
101 movies, 83 with a `VOICE_` token, 18 without. The token is NOT always
`VOICE_<movie>` — 5 `hokyu_*` movies bind to `VOICE_D_450..454` in `\etc\`,
which a guess would miss. Token's subdir varies (Movie/etc/Voice) → resolve via
`sounds.tbl`. Disc test: all 83 resolved entries exist in `sound.pak`.
### Viewer wiring (`iso_loader.rs`, `ui.rs`)
- `resolve_movie_voice_clip` reads the manifest + `sounds.tbl` (DIRECT bindings
only) → `handle_voice_request` (movie player 🗣 toggle) and the 🎧 solo button
(`RequestAudio.movie`) use it. Unbound movies correctly post no audio.
- Standalone **View → 🎙 Voice Lines** browser (filter + ▶ play any `sound.pak`
clip) via `VoiceLibrary` + `AudioPreview`.
Tests: 46 formats-lib + 11 viewer-lib + disc (`movie_manifest_disc`,
`movie_subtitle_disc`, `slb_disc`) all green. Full `cargo test --workspace` green.
## 2. Also bundled in this commit (prior local WIP, not this session's focus)
- Viewer **async stage/model loading** (off-thread `prepare_xpr`, cancellable) —
`iso_loader.rs`/`ui.rs`/`camera.rs`. See memory `reborn-viewer-async-loading`.
- **Grouped-pool XBG7 / hero-ship** decode refinements in `mesh.rs`, `cli/main.rs`,
`mesh_disc.rs`. See memory `reborn-ship-drawlog` / `reborn-saber-texture-investigation`.
- `tools/analyze_drawlog_wvp.py`, `tools/extract_movie_subtitles.py`.
These build and test green but were not re-verified for behaviour this session.
## 3. USER TO VERIFY IN GUI (couldn't be tested from CLI)
1. S13A cutscene shows BOTH lines of the "Look at it father" caption.
2. Story-movie voice (S13A etc.) lines up with the video after the subwave fix.
3. The 5 directly-bound `hokyu_*` movies play voice; other hokyu are silent (see §4).
4. View → 🎙 Voice Lines browser lists and plays clips.
## 4. OPEN PROBLEM — unbound-hokyu resupply voice (do NOT re-guess)
The resupply cutscenes clearly SHARE voice recordings (video-only varies per
mission), but the correct join key for the **13 unbound `hokyu_*`** movies is
**unknown**:
- Manifest DIRECTLY binds only 5: `LS_s02A→450, LS_s09A→451, DS_s02A→452,
LS_s02H→453, DS_s07H→454` (grouping verified against the raw layout).
- Keying unbound movies by subtitle **demo id** (600→450…604→454, so
`hokyu_DS_s13A` demo602→`VOICE_D_452`) was implemented, tested against the
running game by the user, and is **WRONG** (wrong line). It has been **reverted**
— unbound hokyu now stay unvoiced rather than play wrong audio.
- Red flag: only `VOICE_D_450..454` exist; decoded durations `450=2.8s 451=1.6s
452=2.2s` but `453=0.14s 454=0.43s` — far too short for the spoken line, so
these `.slb` are likely **multi-subwave / not cleanly sliced** (same class as
the deferred B/C banks below).
**Next-box options:** (a) get the real join key from mission-event data (mission
scripts/IDXD tables that trigger resupply), or (b) build a proper multi-subwave
`.slb` decoder and verify audio by ear. Needs a concrete calibration clue from the
user (e.g. "s13A should sound like <movie X>" or the actual spoken words).
## 5. Other deferred items (unchanged)
- ~49 "layout-B/C" movie/voice banks (most RT + S07B/S11A/S12B/S13B) + some
individual clips need an XMA1 packet/seek-table reassembler.
- CJK/Japanese text rendering in the viewer (needs a CJK TTF via egui FontDefinitions).
- Texture colours (channel order / sRGB) still on the dynamic-RE backlog.
## 6. Resume checklist (next box)
1. `git pull` on `feature/ipfb-idxd-parser`; set `SYLPHEED_DISC=<extract root>`
(had `.../sylph_extract`).
2. Build guardrail: `CARGO_BUILD_JOBS=4` always (15 GB box; full `-j` OOM'd before).
3. `cargo test --workspace` (with `SYLPHEED_DISC`) should be green.
4. Pick up §4 (unbound-hokyu voice) — the only open thread from this session.

94
docs/HANDOFF-stage-mesh-2026-07-12.md Normal file
View File

@@ -0,0 +1,94 @@
# Handoff — XBG7 stage-container meshes, unified layout, mesh renderer (2026-07-12)
## Summary
This session extended the XBG7 mesh work from single weapons/props to the big
**stage containers**, fixed a long-standing weapon decode artifact, added a
headless mesh renderer for self-verification, and reworked the viewer's stage
presentation. Companion Xenia-Canary GPU instrumentation (the `log_draws`
draw-logger used as the ground-truth oracle) is on the `instrument-current`
branch of the Canary fork.
## What landed (Reborn — `feature/ipfb-idxd-parser`)
### 1. Stage containers decode — `Xbg7Model::stage_models`
`hidden/resource3d/Stage_S*.xpr` are **collections of up to ~450 enemy/prop
sub-models**, not single meshes. Each resource is a block
`[12-byte header][index buffer][vertex buffer]`; the index count is the
descriptor's `(index_bytes, index_count)` marker and the vertex count is the
`u32` 32 bytes before it. The blocks are **scattered among the container's
texture data with no stored offset**, so each is located by **content**: one
`O(file)` pass per stride finds vertex-buffer starts (unit NORMAL at vertex +12
whose previous stride slot isn't — a block boundary), then each resource is
pinned to the candidate where its indices validate and produce non-degenerate,
well-connected triangles. Fast (≤ ~4 s on a 70 MB stage), unambiguous.
- **Coverage: ~4993 sub-models across the 22 stages** (content-anchored).
- A **connectivity gate** (mean triangle edge ≤ 0.28× bbox diagonal) rejects
mis-anchored blocks that would render as spiky messes.
### 2. Weapon layout fix (unified with stages)
Weapons use the **same** `[12-byte header][index][vertex]` block layout. The old
decoder read the index buffer 12 bytes too early → 2 leading degenerate
triangles (the recurring "stray white triangle" artifact) + a lost tail of 6
real indices. Now the header is skipped; vertex offsets are unchanged, so
coverage stays 36/166 and the triangle lists are correct. Verified on
wep_00/03/04 via the renderer.
### 3. Headless mesh renderer — `sylpheed-cli mesh {info,render}`
A software rasterizer (orthographic, z-buffered, two-sided shading) that writes a
PNG. Lets anyone (including the assistant) verify recovered geometry without the
GUI. `--only <substr>` filters sub-models; `--yaw/--pitch/--size/--row` control
the view. Stage containers render as a normalised thumbnail grid.
### 4. Viewer UX
- Stage files show a **thumbnail grid**: each sub-model recentred + uniformly
scaled to a fixed cell (a 1-unit prop and a 3000-unit structure are equally
visible), skybox-plane / stray-anchor models (>20000 u) culled.
- Stage sub-models are **textured** by matching name → `_col` albedo.
- **Camera fixed**: zoom was hard-capped at 50 u (camera trapped inside big
stages); now zoom limits + clip planes scale to the framed model.
- Dispatch decodes both single-model and stage paths and keeps whichever yields
more geometry (fixes stages previously showing the dummy-root "black cube").
## State / how to verify
```bash
# formats + disc tests (needs the extracted disc)
SYLPHEED_RES3D=/path/to/hidden/resource3d \
cargo test -p sylpheed-formats --test mesh_disc -- --ignored --nocapture
# render any model / stage to a PNG
cargo run --release -p sylpheed-cli -- mesh render \
.../resource3d/Stage_S10.xpr /tmp/s10.png --size 1000
cargo run --release -p sylpheed-cli -- mesh render \
.../resource3d/rou_f001_wep_00.xpr /tmp/wep.png
# viewer
cargo run -p sylpheed-viewer # File → Open Directory → the extracted disc
```
## Known limitations / next steps
- **Multi-submesh main bodies with separate vertex pools** still mis-decode
(e.g. `Stage_S10` `e005`: a coherent fighter overlaid with wrong spike
triangles). `e003` works only because its two submeshes *share* one pool. Same
unsolved class as the **DeltaSaber hero body** (`f004`, still declined). Fix
needs per-submesh vertex-base RE. The LODs of these bodies (e.g. `e005_l`)
decode cleanly.
- **The per-resource data offset is not stored** — blocks are found by content.
Reversing the container's allocation order would give exact offsets + 100%
coverage but is a larger effort.
- **Texture colour correctness** remains the known "unverified" dynamic-RE item
(channel order / sRGB) for both 2D textures and model albedos.
- The viewer's stage decode runs on the main thread (~10 s on a 70 MB stage in a
debug build); move off-thread if the load hitch matters.
## Canary oracle (`instrument-current` on the Xenia-Canary fork)
`command_processor.cc::LogDrawForRE` (cvar `log_draws`) dumps each distinct
draw's primitive type, index buffer, vertex declaration, and sample vertex
positions to `xenia_re_draws.log`. Used to confirm the XBG7 layout (triangle
list, f32×3 position + f16×4 normal + f16×2 uv). Build with the memory-safe
flags (`CC=clang-19 CXX=clang++-19 CMAKE_BUILD_PARALLEL_LEVEL=3 … -j 3`); run one
emulator process at a time.

31
docs/re/INDEX.md Normal file
View File

@@ -0,0 +1,31 @@
# RE knowledge index
Confidence: ✅ `CONFIRMED` · 🟡 `PROBABLE` · ❔ `HYPOTHESIS`. See [README](README.md).
Formats we've already reversed are, for now, **documented by their parser + disc round-trip
tests** (the executable spec) rather than a prose file — the "Spec" column points there.
Promote to a prose `structures/…md` file when a format needs behavioural notes beyond layout.
## Data structures / formats
| Format | Conf. | Spec (parser + tests) | Notes |
|--------|-------|-----------------------|-------|
| IPFB `.pak` archive | ✅ | `sylpheed-formats/src/pak.rs` + `tests/pak_idxd_disc.rs` | header + 12-byte TOC, Z1/zlib payloads |
| name-hash (TOC keys) | ✅ | `sylpheed-formats/src/hash.rs` | Barrett-reduction hash; recovers original paths |
| IDXD object/table | ✅ | `sylpheed-formats/src/idxd.rs` | self-describing; ship/weapon stats verified vs known values |
| XPR2 texture + cubemap | 🟡 | `sylpheed-formats/src/texture.rs` | de-tile + A8R8G8B8; **colours unverified** (dynamic item) |
| T8aD 2D texture | 🟡 | `sylpheed-formats/src/t8ad.rs` | ~85% decode; **colours ✅ CONFIRMED** ([k8888](structures/texture-color-k8888.md)); ~15% variants deferred |
| RATC bundle | 🟡 | `sylpheed-formats/src/ratc.rs` | child listing confirmed; one level deep |
| LSTA sprite list | 🟡 | `sylpheed-formats/src/lsta.rs` | inline T8aD frames |
| IXUD subtitle | 🟡 | `sylpheed-formats/src/ixud.rs` | timed cues; **movie↔track link unknown** (dynamic item) |
| Fonts (ttf/otf/ttc) | ✅ | `sylpheed-formats/src/font.rs` | standard OpenType, parsed via ttf-parser |
| XBG7 mesh | 🟡/❔ | `sylpheed-formats/src/mesh.rs` + `tests/mesh_disc.rs` ([xbg7](structures/xbg7-mesh.md)) | weapons/props: declaration-driven variable stride (36 models), GPU-confirmed. **Stage containers: 5662 sub-models across 22 stages** via content-anchored grouped pools (`stage_models`). Quantized hero bodies (DeltaSaber `f004`) still declined |
| Capital-ship part placement | 🟡 | `sylpheed-formats/src/ship.rs` (static) + [runtime capture](ship-placement-runtime-capture.md) | hull placement static-exact; external parts approximate statically. **Runtime capture** (Canary F10 → VS-constant WorldView) gives ground truth — validated on `e106` destroyer; not yet baked into the viewer |
## Functions / code paths
_None documented yet — populated during the dynamic-RE phase._
| Function | Conf. | Reimpl. | Summary |
|----------|-------|---------|---------|
| — | — | — | — |

88
docs/re/README.md Normal file
View File

@@ -0,0 +1,88 @@
# Reverse-engineering knowledge base
This directory is the **spec-side** of the clean-room: it records what the original
*Project Sylpheed* binary **does** (behaviour) and how its **data is laid out**, so that
the Rust port can be implemented **from these specs** without re-deriving anything and
without ever copying original code.
It exists to answer one question fast: *"do we already know how X works, and how sure are we?"*
---
## The one rule that matters
> **Never document a claim more confidently than the evidence supports, and never
> paste original code here.**
A wrong-but-confident note is worse than no note: someone builds on it and the bug hides
for weeks. Every entry therefore carries an explicit **confidence** and its **evidence**.
This mirrors the project method — *measure the oracle, never infer; refute before believing.*
### Clean-room firewall
- ✅ Allowed: behaviour descriptions, field offsets/types, formulas, state machines,
observed input→output pairs, and **references** to the original by address
(`sub_821B68C0`) or to `xenia-rs/sylpheed.db`.
- ❌ Forbidden here and in `crates/`: pasted decompiled C/C++ or verbatim disassembled
function bodies presented as the thing to reimplement. Cite the address; describe the
behaviour in your own words. Disassembly is a tool for *understanding*, not a source to copy.
---
## Confidence levels
| Level | Meaning | Bar to reach it |
|-------|---------|-----------------|
| `CONFIRMED` | Behaviour verified against ground truth. | ≥2 independent observations **or** one observation cross-checked against an oracle (canary framebuffer, a known-correct value, a second code path). |
| `PROBABLE` | Strong single-source inference. | One clean observation, or an unambiguous static read of the disassembly. |
| `HYPOTHESIS` | Educated guess, not yet tested. | Anything else. Must say what would confirm/refute it. |
**Promotion requires new evidence, not re-reading the old evidence.** A `HYPOTHESIS` that
"looks right again" is still a `HYPOTHESIS`. Only an *independent* check promotes it.
If evidence later contradicts an entry, **demote it and record the contradiction** — do
not silently edit the conclusion.
---
## When to document
- **Right after** a function/structure crosses from `HYPOTHESIS` to at least `PROBABLE` —
before moving to the next code path, so the knowledge isn't lost or re-derived.
- **Whenever confidence changes** (up or down) — append to the Evidence log, don't overwrite.
- **Not** while it's still a pure guess with no evidence — a one-liner in the relevant
backlog/plan is enough until there's something to stand on.
## What to document
- **Functions/code paths** → `docs/re/functions/<name>.md` (one file per function or tight cluster).
- **Data structures / formats** → `docs/re/structures/<name>.md`.
- Keep the index in [`INDEX.md`](INDEX.md) (one line each: name · confidence · one-line summary).
Use the templates: [`_TEMPLATE.function.md`](_TEMPLATE.function.md),
[`_TEMPLATE.structure.md`](_TEMPLATE.structure.md).
---
## How we find and confirm code paths (the toolchain)
Everything joins on the **guest virtual address (PC)** — code addresses are fixed by the
XEX load, identical across our emulator and canary.
- **Static (cheap, try first):** `xenia-rs/sylpheed.db` (DuckDB: 25 481 functions, xrefs,
strings, vtables, imports). Query with `xenia-rs/zq.py` — `zq.py grep <str>`,
`zq.py xref <addr>`, `zq.py dis <lo> <hi>`, `zq.py fn <pc>`. Entry points are usually a
**string** (`zq.py grep MSG_DEMO`) or an **import** (movie/XMA API) xref'd back to the loader.
- **Dynamic (when static is ambiguous):** run `xenia-rs` with its probe suite —
`--pc-probe` / `--audit-pc-probe-hex` (fires at block entry), `--mem-watch` (mid-block
reads/writes of a VA), `--lr-trace` (call/return chains), `--trace-instructions`,
`--dump-addr` (read guest memory). These already exist; prefer them over hacking canary.
- **Oracle (correctness ground truth):** canary — the **Wine cross-build**
`xenia-canary/build-cross/bin/Windows/Debug/xenia_canary.exe` (the native Linux ELF
**crashes / does not run** — do not use it). This is the only emulator that reaches the
in-game menu; our `xenia-rs` never got past the intro video. Use canary to *observe output*
(capture its framebuffer for texture colours), not usually to instrument code — though its
`build-cross` toolchain does compile, so small C++ probes + rebuild are possible when needed.
Run **muted, one emulator process at a time**, point it at the real ISO (not the symlink).
> ⚠️ **VA-equality caveat:** join **code** by PC (fixed), but **never** assume a data VA
> holds the same bytes across emulators — allocators differ. Compare data by content/layout.

23
docs/re/_TEMPLATE.function.md Normal file
View File

@@ -0,0 +1,23 @@
# `<function name / sub_XXXXXXXX>`
- **Address:** `0x________` (in `sylpheed.db`; `zq.py fn 0x________`)
- **Confidence:** `HYPOTHESIS | PROBABLE | CONFIRMED`
- **Reimplemented in:** `crates/…/…rs::fn` — or *not yet*
- **Related:** other RE entries, `[[memory-slug]]`
## What it does
One-paragraph behavioural summary, in your own words. No pasted disassembly.
## Signature / calling convention
Inputs (registers/args + meaning), outputs, side effects (memory it writes, globals, imports it calls).
## Behaviour
Step-by-step of the observable behaviour — the *spec* to implement from. Formulas, branches,
state transitions. Cite addresses for detail (`the loop at 0x…`), don't transcribe code.
## Evidence log (append-only; newest last)
- `YYYY-MM-DD` — *how we know*: static disasm read / `--lr-trace` from 0x… / canary output /
N observations. What it established. → confidence set to `…`.
## Open questions / what would raise confidence
The specific check that would confirm or refute the current claim.

29
docs/re/_TEMPLATE.structure.md Normal file
View File

@@ -0,0 +1,29 @@
# `<structure / format name>`
- **Confidence:** `HYPOTHESIS | PROBABLE | CONFIRMED`
- **Parser in:** `crates/sylpheed-formats/src/…rs` — or *not yet*
- **Seen in:** which pak / asset / memory region
- **Related:** other RE entries, `[[memory-slug]]`
## Layout
```text
offset size type name notes
0x00 4 magic "____"
0x__ 4 u32be …
…
```
Endianness, alignment, variable-length rules, how child/section sizes are derived.
## Field semantics
What each field *means* and its observed value range. Distinguish **confirmed** fields from
**guessed** ones explicitly (a guessed field with a plausible value is still a guess).
## Coverage / limits
What fraction of real instances this decodes; which variants are deferred and why.
## Evidence log (append-only; newest last)
- `YYYY-MM-DD` — how established (forensic scan over N disc entries / round-trip test /
oracle comparison). → confidence `…`.
## Open questions / what would raise confidence
e.g. "colours unverified until compared against canary's framebuffer."

105
docs/re/ship-placement-runtime-capture.md Normal file
View File

@@ -0,0 +1,105 @@
# Capital-ship part placement — runtime capture (ground truth)
**Status:** pipeline working end-to-end for one ship (`e106` ADAN Destroyer);
correlator recovers exact ship-relative transforms. Not yet baked into the viewer.
## Problem
Capital ships ship as **split XBG7 parts** (hull bodies, bridge, engines, turrets)
in `hidden/resource3d/Stage_SNN.xpr`. Reconstructing the whole ship needs each
part's placement. Static reverse-engineering of the composite scene graph
(`e_rou_<id>`, `GN_*` frames) got the **hull right but external parts only
approximately** — a `GN_*` frame is the mount *pivot*, and the real outboard /
rotational offset lives in a detail sub-rig or in runtime code not recoverable
statically. See `sylpheed-formats::ship::assemble_ship` (the static best-effort).
## Finding
The **guest vertex buffer holds LOCAL coordinates** — byte-identical to the
`.xpr` (verified: `e106_bdy_04`'s first vertex `(-20.0000,153.1539,45.9997)`
matches the decode exactly). So parts are placed **entirely in the vertex
shader**; the buffer carries no placement. (Contrast: static *stage* geometry is
pre-transformed into world space and byte-matches the `.xpr` at file offsets —
that's a different case.)
The per-part transform is in the **vertex-shader float constants**. For the ship
shader (`vs=0xC7F781F4C1D58054`), constants `c0..c2` are the **WorldViewProjection**
matrix rows:
- Their norms are `(sx, sy, 1)` = the projection x/y scales; `sy/sx ≈ 1.78 = 16:9`
aspect. Dividing each row by its norm gives the rigid **WorldView** (verified:
normalized rows orthonormal to 1e-4, `det = +1`).
- The camera View cancels when expressing every part relative to a reference part:
`rel_p = WV_ref⁻¹ · WV_p` = `(Rᵀ_ref·R_p , Rᵀ_ref·(T_p − T_ref))` — a pure
ship-space rigid transform. Applying `rel_p` to part `p`'s local geometry
assembles the ship exactly, in the reference part's frame.
(`c4..c14` also vary per part — likely a normal matrix / a second WVP variant /
WorldView split; not needed, `c0..c2` suffice.)
## Capture instrumentation
Branch **`capture-ship-placement`** in the `xenia-canary-native` worktree (off the
`instrument-current`-descended native branch, which already has GPU `log_draws`).
- **F10** in the emulator window → `xe::gpu::RequestShipCaptureFrame()`.
- Dumps the next ≤8000 draws (de-duped by vertex-buffer address) to
**`xenia_ship_capture.log`** in the binary's dir
(`build/bin/Linux/Release/`). Per draw:
```
DRAW vbase=0x… stride=… vcount=… indices=… prim=… vs=0x…
pos: (x,y,z) … up to 64 LOCAL vertex positions (== .xpr)
vsconst base=N: c0=(x,y,z,w) c1=… … first 48 VS float4 constants
```
- Files: `src/xenia/gpu/command_processor.{cc,h}` (`CaptureShipDrawForRE`,
`RequestShipCaptureFrame`), `src/xenia/app/emulator_window.cc` (F10 key).
- Run: `run-canary-native.sh` (HW Vulkan, interactive). Play into the mission,
frame the ship side-on, press F10.
## Correlator
`sylpheed-formats/examples/correlate_capture.rs`:
```
SYLPHEED_ISO=… cargo run --release --example correlate_capture -- \
<capture.log> <Stage_SNN> <ship_id> [ref_part_substr]
```
Parses the log, decodes the ship's base parts, matches each part to a draw by
**vertex count** (unique per part), recovers WorldView from `c0..c2`, and prints
each part's ship-relative transform (reference-part frame) + assembled extent.
## Validated result — `e106` ADAN Destroyer (Mission 1 / Stage_S01, side view)
Vertex-count → part map (all matched exactly): bdy_01/02 = 1261, bdy_03 = 815,
bdy_04 = 1633, eng_01 = 583, eng_02 = 491, wep_02_01 = 1002, brg_01 = 202.
Ship-relative placement (reference = `bdy_04`, aft hull):
| part | T (ship-relative) | note |
|---|---|---|
| bdy_04 | (0, 0, 0) | reference, aft hull |
| eng_01 | (131, −133, −132) | **aft** (−Z), starboard |
| eng_02 | (0, −49, −140) | aft, centre |
| bdy_01 | (**−264**, −150, 1165) | **port** hull, forward |
| bdy_02 | (**+264**, −150, 1165) | **starboard** hull, forward |
| bdy_03 | (0, −35, 1076) | forward keel/spine |
| wep_02_01 | (0, 49, 1009) | forward weapon |
Assembled extent **872 × 670 × 2181** (a coherent ~2181-long destroyer).
**Key wins vs static:** `bdy_01`/`bdy_02` are a **port/starboard pair** (X = ∓264),
which static had overlapping at one point; engines land correctly aft; the ship is
not the 1894-tall tower the static Y put out.
**Missing:** `brg_01` (bridge, vcount 202) was **culled/occluded** at that camera
angle (0 draws) — needs a second capture framing the superstructure.
## Next steps
1. **Bridge:** re-capture with the bridge visible → fills `brg_01`.
2. **Bake:** promote the correlator to a module; emit a per-ship placement table
(`ship_id → [(part, R, T)]`) as a checked-in data file; have the viewer's
`build_ship_model` prefer the captured table over `assemble_ship` when present.
3. **Other ships:** same capture for the cruiser (`e105`, Stage_S02), ACROPOLIS
(`f101`), TCAF cruiser/destroyer (`f105`/`f106`). One side-on F10 each.
4. **Turrets:** the shared `e3NN`/`e4NN` gun models appear as their own draws in the
capture too — correlate them to place turrets (static never resolved these).

174
docs/re/structures/movie-subtitles.md Normal file
View File

@@ -0,0 +1,174 @@
# Movie subtitles & the movie ↔ mission ↔ text chain
Reverse-engineered 2026-07-19 (static, from the extracted disc). The full chain
that links a cutscene movie to its on-screen subtitle text is now closed.
## Files involved
- `dat/movie/*.wmv` — the cutscene videos. **Named by mission** (see below).
- `dat/movie/<lang>.pak` + `<lang>.p00` — per-language **subtitle timing tracks**
(`eng`, `jpn`, `deu`, `fra`, `esp`, `ita`) plus the caption font.
- `dat/GP_MAIN_GAME_<L>.pak` + `.p00` — per-language **caption TEXT**
(`E`=Eng, `J`=Jpn, `D`=Deu, `F`=Fra, `I`=Ita, `S`=Esp).
- `dat/tables.pak` entry `ADVERTISE_MOVIE` (hash `0x5B983A08`) — the master
**movie manifest**: all 101 `.wmv` names in mission-progression order, each
bound to its subtitle track and (optionally) its **voice bank** (see below).
## Movie filename → mission
Purely from the filename:
| Pattern | Meaning |
|---------|---------|
| `S<NN><P>.wmv` | Stage `NN` **story** cutscene, part `P` (A/B/C…) — e.g. `S02C` = stage 2, 3rd story scene |
| `RT<NN><P>.wmv` | Stage `NN` **radio / briefing** transmission, part `P` (`_1`/`_2` = split clips) |
| `hokyu_<LS\|DS>_s<NN><P>.wmv` | Stage `NN` **resupply** scene (`hokyu` = 補給). `LS`/`DS` = the two resupply-ship variants |
| `ADV.wmv` | Intro / title movie |
97 movies total: 27 story, 50 radio, 19 resupply, 1 intro. (Some referenced
stages — s24, s27, RT16 — exist as keys but the .wmv isn't in this extract.)
## Subtitle timing track — `<lang>.pak`
IPFB archive (`IPFB`, BE-u32 count, 16-byte header; TOC of
`[name_hash u32][offset u32][size u32]` triples, sorted by hash, into `.p00`).
- **Track key = `name_hash("subtitle_<movie_basename>.tbl")`** (the hash
lowercases internally, so basename case is irrelevant). This is the movie→track
link. Verified: `subtitle_S00A.tbl` → `0x6F2D9663`, `subtitle_hokyu_DS_s02A.tbl`
→ `0x3662B1F8`, `subtitle_RT01C_1.tbl` → `0x756F69FB`.
- The archive also holds **RATC** pre-rendered title-card / number textures
(`pwterop_s01a1.t32`, `pwrt_rt01_str.t32`, `pwnum0-9.t32`) + one TrueType font.
- **Each track data block is `Z1`+zlib**: bytes `5A 31` ("Z1"), a small header,
then a raw zlib stream (`78 DA`/`78 9C`). `zlib.decompress(blob[blob.find(b"\x78\xda"):])`.
- Decompressed = an **IXUD** container. Payload (UTF-16LE) is the timing sheet:
`SUBTITLE MSG_DEMO_<demo> <mm:ss.cc> MSG_DEMO_<demo> <mm:ss.cc> …`.
So the track says *which* demo-message shows *when*, not the text itself.
## Caption text — `GP_MAIN_GAME_<L>.pak`
Same IPFB+`.p00`. Among its ~1119 entries, **32 blocks are `Z1`+zlib → IXUD**
string containers holding the movie caption text. Layout: `IXUD`, u32 version,
hash@0x08, count@0x14, then `(recordhash,offset,len)` triples, then a UTF-16LE
string region where **each line is stored as `text` immediately followed by its
key** `MSG_DEMO_<demo>_<page>_<line>` (captions wrap across `_00`,`_01`, …).
537 English lines recovered. Entries 1 & 19 are the IDXD *schema* records
(`ID`, `PageCount`, `Character`=speaker e.g. `TCAFSUPPLY`, `Face`, line refs) —
no text, just structure.
## Movie → voice track: the manifest binding (`ADVERTISE_MOVIE`)
The `ADVERTISE_MOVIE` manifest is **also the authoritative movie→voice index**.
Its string pool emits, per movie, a run led by `<movie>.wmv` optionally followed
by `<pak>+….prt` (overlay art), `<pak>+SUBTITLE_<movie>.tbl`, and a bare
`VOICE_<token>`. Grouping the pool on `.wmv` (records are emitted in order)
recovers `movie → Option<voice_token>` without decoding the IDXD record binary
(`crate::movie_manifest`).
The voice token is **not** always `VOICE_<movie>`, so the manifest is required —
guessing both misses real bindings and invents tracks for silent movies:
- **83 / 101 movies have a voice token.** Story/radio movies use `VOICE_<movie>`
in `<lang>\Movie\`.
- **5 `hokyu_*` resupply movies bind to in-mission radio clips** — e.g.
`hokyu_LS_s02A → VOICE_D_450`, which lives in `<lang>\etc\`, *not* `Movie`.
A `VOICE_<movie>` guess would never find these.
- **18 movies have no direct voice token** = 4 boot logos + 1 HD test pattern +
**13 `hokyu_*` movies** (incl. `hokyu_DS_s13A`). Only the manifest's **direct**
bindings are trusted for playback.
### Shared resupply voice — UNRESOLVED for unbound movies
The manifest directly binds only **5** resupply movies, each to a shared
`VOICE_D_45x` clip in `<lang>\etc\`:
| bound movie | subtitle demo | clip |
|-------------|---------------|------|
| `hokyu_LS_s02A` | 600 | `VOICE_D_450` |
| `hokyu_LS_s09A` | 601 | `VOICE_D_451` |
| `hokyu_DS_s02A` | 602 | `VOICE_D_452` |
| `hokyu_LS_s02H` | 603 | `VOICE_D_453` |
| `hokyu_DS_s07H` | 604 | `VOICE_D_454` |
The resupply cutscenes clearly **share** voice recordings (only the video varies
per mission), so the 13 unbound `hokyu_*` movies must reuse one of these — but
the **correct join key is not yet known**:
- Keying by subtitle **demo id** (so `hokyu_DS_s13A`, demo 602 → `VOICE_D_452`)
was tried and is **WRONG** — it plays the wrong recording in-game. Do not use.
- Only `VOICE_D_450..454` exist (no 44x/45x neighbours). Decoded durations are
suspicious — `450`=2.8s, `451`=1.6s, `452`=2.2s, but `453`=**0.14s**,
`454`=**0.43s** — far too short for the spoken line, so these `.slb` banks are
likely **multi-subwave / not cleanly sliced** by the current extractor (same
class as the deferred B/C banks).
⇒ The unbound-hokyu voice mapping is **open** (needs either the real join key
from mission data, or a proper multi-subwave `.slb` decode + audio verification).
`hokyu_LS_s24A`/`s27A` have no subtitle track at all (stages absent from this
extract).
The token's `sound.pak` **subdirectory is not fixed** (`Movie` / `etc` / `Voice`),
so resolve it via `sounds.tbl` (which lists the full `<lang>\…\<token>.slb` path)
rather than assuming a directory. `movie_manifest::resolve_voice_entry` does the
full chain. Verified: all 83 resolved entries exist in `sound.pak`.
## Caption packing quirks (parser must handle)
- **Multi-line captions are split into consecutive text tokens** that share one
trailing timing, e.g. S13A stores `"Look at it father"` + `"& beautiful isn't
it"` before `01:14.80-01:17.60`. Accumulate every text token since the last
timing and join with `\n`; pairing strictly 1:1 silently drops all but the last
line.
- **Overlapping spans**: some tracks show two captions at once (an open-ended
radio line still up when the next range line starts). The viewer stacks every
cue active at `t` (`MovieSubtitles::active_cues`) instead of showing only the
first.
## The full join
```
tables.pak / ADVERTISE_MOVIE → list of movies (mission order)
<movie> ─ nh("subtitle_<movie>.tbl") ─→ <lang>.pak track
track (IXUD) → [ (MSG_DEMO_<d>, timecode), … ]
MSG_DEMO_<d> → GP_MAIN_GAME_<L> → "the localized caption line(s)"
```
## Coverage
- 92 / 97 movies have a subtitle track.
- **66 movies carry timed `MSG_DEMO` captions** — the **radio (`RT*`)** and
**resupply (`hokyu_*`)** movies. These fully decode to timed text.
- The **27 story (`S*`) movies have a track but 0 timed captions** — their text
is delivered as the **pre-rendered title-card textures** (`pwterop_*`, burned
styling), not MSG_DEMO lines.
## Worked examples (English)
```
RT01C_1.wmv (Stage 1 radio, part C):
00:00.50 [14] We did it! Okay, all pilots follow my lead!
00:06.80 [15] Rhino Leader to ACROPOLIS. We made it through and we're coming
home. Roger. It's good to see you're all safe.
00:19.30 [17] Yeah, but Brandon ... Damn. There's only seven of us. …
hokyu_DS_s02A.wmv (Stage 2 resupply):
00:00.00 [602] Resupply complete. You are cleared for take-off!
```
## Reusable extractor
`tools/extract_movie_subtitles.py` — parses `<lang>.pak`, resolves each movie's
track, cross-references `GP_MAIN_GAME_<L>` text, and prints per-movie timed
transcripts + the movie→mission table.
## In-mission dialogue (future work)
`GP_MAIN_GAME_<L>.pak` is the **global** message store, not just movie captions:
its `MSG_DEMO_*` table also holds the in-mission radio/dialogue lines (same demo
id space). So the *text* of gameplay dialogue is already decodable with
[`crate::movie_subtitle::build_demo_text`]. What's missing is the **trigger** —
which demo id fires at which mission event — and that lives in the **mission
data** (mission scripts / `GP_MAIN_GAME` IDXD tables), not in the text pack. When
reversing mission data, look for demo-id references there to bind dialogue to
events; the IDXD "Message" schema records also carry `Character` (speaker) and
`Face` (portrait) per line.

62
docs/re/structures/sound-slb.md Normal file
View File

@@ -0,0 +1,62 @@
# Sound bank audio — `sound.pak` / `.slb` / XMA1
Reverse-engineered 2026-07-19 (static). `dat/sound.pak` (+ `sound.p00..p04`,
~1.08 GB) is the game's entire audio bank: **9519** IPFB entries, each an XACT
`.slb` sound bank wrapping **XMA1** audio.
## Names
Entry keys are `name_hash(<string>)` of filenames stored verbatim in the IDXD
string pools of `tables.pak` → `eng\sounds.tbl` (entry 39) and `jpn\sounds.tbl`
(entry 45). All 9519 recovered. Families:
- `<lang>\Voice\VOICE_<SPK>_<NNN>.slb` — per-line character voice (SPK = ADAN,
TCAF, RHIN, ADPL, BIRD, ACRO, ZZZZ…). Only `eng`/`jpn` voice exists.
- `<lang>\etc\VOICE_{A,B,C,D}_<NNN>.slb`, `<lang>\Briefing\BR<n>_<m>.slb`.
- **`<lang>\Movie\VOICE_<movie>.slb`** — the continuous voice track for cutscene
`<movie>.wmv` (e.g. `eng\Movie\VOICE_RT07A.slb`). Played from the video start.
- `BGM_###.slb` (32) — music. `Static.slb`, `JNGL_###` — SFX/ambience.
Only English and Japanese have voice; subtitles cover more languages, voice does
not. See [`crate::slb::movie_voice_name`].
## Codec
XMA1: RIFF `fmt ` tag **0x0165**, a 32-byte `XMAWAVEFORMAT`, **48000 Hz**. The
content is **mono** — some clips carry it in the **left channel only** (right is
silence), others are **dual-mono** (L = R). Downmix to mono by taking the left
channel (it always holds the full signal).
`XMAWAVEFORMAT` (one stream, 32-byte `fmt `): SampleRate @ fmt+16 (u32 LE),
Channels @ fmt+29 (u8), ChannelMask @ fmt+30 (u16). Movie voices report 2
channels / mask 0x2 despite the mono content.
## `.slb` layouts → decodable RIFF
Two on-disc layouts (`sound.pak` movie voices split ≈ 63 RIFF / 15 headerless):
- **RIFF present** — a `RIFF/WAVE` sits inside the bank; its 32-byte `fmt ` is
the real header. The XMA packets are **everything after that RIFF's `data`
chunk header** (`slb[data_pos+8..]`); the declared `data` size is unreliable
(sometimes half the real length), so take to end and clamp to the media length
downstream. Some banks have trailing bank data past the real audio — always
beyond the video length, so clamping to the video duration drops it.
- **Headerless** — no `RIFF` anywhere; a fixed **1392-byte** header precedes raw
XMA1 packets (48 kHz, 2 channels). Data = `slb[1392..]`.
[`crate::slb::to_xma_riff`] rebuilds a standalone XMA1 `RIFF/WAVE` for either
layout, ready for FFmpeg's `xma1` decoder.
## Decode (FFmpeg)
`ffmpeg -i rebuilt.riff -af "pan=mono|c0=c0" -t <media_len> out.wav` — the RIFF
is fed to the `xma1` decoder, downmixed to the left channel, clamped to length.
Verified: `VOICE_S00A` → 93.9 s == `S00A.wmv` 93.87 s; `VOICE_ADV` → 137.3 s ==
137.7 s; `VOICE_RT07A` → 49 s ≈ 50 s.
## Reading one entry cheaply
`sound.pak` is ~1 GB. Use [`crate::PakArchive::parse_toc`] on the small `.pak`
index, binary-search the name-hash, then read just `[offset, offset+size)` from
the `.pNN` segments (the viewer's `read_segment_range` seeks into the right
segment) instead of loading the whole archive.

50
docs/re/structures/texture-color-k8888.md Normal file
View File

@@ -0,0 +1,50 @@
# Texture colour interpretation — `k_8_8_8_8` (32bpp UI/HUD textures)
- **Confidence:** mixed — see per-claim tags below.
- **Parser in:** `sylpheed-formats/src/t8ad.rs`, `src/texture.rs` (XPR2)
- **Applies to:** T8aD 2D UI textures, XPR2 32bpp surfaces — the "colours look a bit odd" concern.
- **Related:** [[reborn-dynamic-re-plan]], INDEX rows T8aD / XPR2.
## What this resolves
Whether our decoded 32bpp textures use the right **sRGB-vs-linear** interpretation and the right
**channel order**. Ground truth = Canary's Vulkan texture cache, which is the code that actually
produces correct on-screen colour.
## Findings
### ✅ CONFIRMED — plain `k_8_8_8_8` is LINEAR (UNORM), never sRGB
Canary's host-format table maps plain `k_8_8_8_8` → `VK_FORMAT_R8G8B8A8_UNORM` with a straight
32-bit copy load shader (`kLoadShaderIndex32bpb`) and an **identity** format swizzle
(`XE_GPU_TEXTURE_SWIZZLE_RGBA`). There is **no** `..._SRGB` host format anywhere in the table;
gamma is a separate explicit path (`k_8_8_8_8_GAMMA_EDRAM` only). Therefore a plain 32bpp texture
is sampled as **raw UNORM bytes — apply no sRGB/gamma decode.**
→ *Implication:* if the viewer/engine applies an sRGB→linear (or linear→sRGB) step to these, that is
wrong. Show the bytes as-is (raw RGBA8), gamma only where the game explicitly requests it.
### ✅ CONFIRMED — on-disk order is **ARGB** (alpha first); `[a,r,g,b]→[r,g,b,a]` is correct
Settled by two independent lines that agree, without an emulator run:
1. **Measured on the disc data** — across 789 real T8aD textures (GP_HANGAR_ARSENAL, GP_MAIN_GAME_E,
DefTables), the byte position that is most-often exactly `0xFF` (the opaque-alpha signature of UI
art) is **byte 0 in 767/789 (97%)** — tf1 385/0, tf2 382/12. So texel byte 0 = alpha ⇒ on-disk
layout is **A,R,G,B**. (An earlier bimodality test tied byte0/byte3 only because colour channels
are also `0x00`-heavy from black backgrounds; the `==0xFF` discriminator isolates alpha cleanly.)
2. **Cross-check vs our Canary-validated renderer** — `xenia-rs`'s `decode_k8888_tiled` (the path
behind the user-confirmed M1 splash / M2 video colours) nets memory-`ARGB` → endian-swap →
`swap(0,2)` = `[A,R,G,B]→[R,G,B,A]`, identical to ours. Since that renderer was confirmed against
Canary's output, Canary's ground truth is transitively in this chain.
Net: our channel order and the linear/UNORM interpretation are both **correct** for the ~85% RGBA
variants. A live Canary framebuffer capture would be redundant confirmation, not a new signal.
## Remaining / adjacent
- **XPR2** shares the ARGB→RGBA reorder + the linear/UNORM finding, but its **tiling** (de-tile) is a
separate path; if XPR2 skyboxes still look odd it's tiling or viewer display-gamma, not channel order.
- **Viewer display gamma:** decode is correct; if egui shows these too dark/bright it's how egui
interprets the RGBA8 (sRGB-encoded vs linear) at display time — a rendering nit, not a decode bug.
## Evidence log
- `2026-07-11` — static read of Canary `vulkan_texture_cache.cc` host-format table; no `R8G8B8A8_SRGB`
entry exists → sRGB/UNORM claim `CONFIRMED`.
- `2026-07-11` — measured `==0xFF` alpha dominance over 789 disc T8aD textures (byte0 = alpha 97%) +
cross-checked vs `xenia-rs decode_k8888_tiled` (the Canary-validated M1/M2 path). → channel-order
claim promoted `HYPOTHESIS → CONFIRMED`.

254
docs/re/structures/xbg7-mesh.md Normal file
View File

@@ -0,0 +1,254 @@
# XBG7 — mesh geometry (inside XPR2 model containers)
- **Confidence:** 🟡 `PROBABLE` for the single-stream layout (below); ❔ `HYPOTHESIS` / undecoded
for the complex multi-stream body layout.
- **Parser in:** `sylpheed-formats/src/mesh.rs` (`Xbg7Model::from_xpr2`), tests
`tests/mesh_disc.rs`. Container parsing reused from `src/texture.rs` (`Xpr2Header` /
`Xpr2ResourceEntry`).
- **Applies to:** ship / weapon / prop models in `hidden/resource3d/*.xpr` (166 files).
- **Method:** clean-room — static hex inspection of the retail disc + geometric validation of the
recovered triangles (non-degenerate area, indices in range, bbox matches the descriptor's stored
size). No game code decompiled or copied.
## Where XBG7 lives
Models are ordinary **`XPR2`** containers (see the XPR2 texture doc / `texture.rs`). The 16-byte
resource-directory entries (from file offset `0x10`) carry `TX2D` texture resources **and** one or
more `XBG7` geometry resources:
```
entry = [ tag:4 ][ data_offset:u32 ][ descriptor_size:u32 ][ name_offset:u32 ] (big-endian)
```
Offsets are relative to the directory base `0x10`. The `XBG7` *descriptor* (at `data_offset+0x10`,
`descriptor_size` bytes) is a **scene / material / node graph** — it holds node names
(`rou_f001_mnt1_root`, `Light`, …), a bounding value (`0x41F00000` = 30.0 ≈ the ship's ~30-unit
length), material names matching the `TX2D` channels (`_col` albedo, `_spc` specular, `_gls` gloss,
`_lum` luminance), and per-sub-mesh records. The **vertex / index buffers** live in the container's
shared **data section** (from `header_size`).
## Sub-mesh records (in the descriptor)
Read in file order by a sliding 4-byte scan; each is a big-endian tuple:
```
[ vtx_count:u32 ][ 0:u32 ][ idx_count:u32 ][ tail:u32 ]
3..=65535 ==0 mult. of 3 1..=64
```
(For `rou_f001_wep_00`: `vtx_count=215`, `idx_count=1092` — matches the recovered geometry exactly.)
## The single-stream data layout — 🟡 PROBABLE (decoded, GPU-cross-checked)
For 36 of the 166 models (weapons, simple props) the data section is a straight sequence of
sub-meshes, carved from `header_size` in record order:
```
per sub-mesh block:
[ 12-byte header (contents undecoded) ]
[ index buffer : idx_count × u16 BE ] triangle list (prim=4, GPU-confirmed)
[ vertex buffer : vtx_count × stride bytes ] ← declaration-driven
(pad to 16 bytes → next sub-mesh)
```
The 12-byte header precedes the **index** buffer (the same block shape as stage
resources — see below); the vertex buffer follows the indices with no further
gap. (Earlier this was mis-modelled as `[index][12-byte gap][vertex]`, which put
the vertex buffer at the identical offset but read the index buffer 12 bytes too
early — turning the 12 header bytes into 6 junk indices = **2 leading degenerate
triangles** (a stray-triangle artifact) and dropping the last 6 real indices.
Skipping the header fixes the triangle list with no change to vertex coverage.)
**Vertex declaration.** The layout is **not fixed-stride**. The descriptor holds a declaration table
(right after the `(index_bytes, index_count)` marker) of `{offset:u32, format-code:u32, usage<<16:u32}`
big-endian triples, terminated by `offset == 0x00FF0000` / `code == 0xFFFFFFFF`:
| usage | element | format code | format | size |
|-------|----------|-------------|--------|------|
| 0x00 | POSITION | `0x2A23B9` | f32×3 | 12 B |
| 0x03 | NORMAL | `0x1A2360` | f16×4 (use xyz) | 8 B |
| 0x05 | TEXCOORD | `0x2C235F` | f16×2 (u,v) | 4 B |
Stride = max element extent. Models **omit elements** → variable stride (20 = pos+normal, no UV;
24 = pos+normal+uv). Each element is read in **naive big-endian component order** (the raw file
bytes; see the endianness note below). Assuming a fixed stride-24 was why the old decoder mis-aligned
and declined the pos+normal-only models.
**Alignment pinned by the normals.** The vertex buffer starts at `index_end + 12` (a fixed 12-byte
header — NOT `align16`, which lands 4 bytes early on most models). The correct offset is the unique
one where recovered normals are exactly unit-length.
**Safety gate:** every index is validated `< vtx_count`, and when the declaration has a normal
element the mean recovered `|normal|` must be ≈1 (`[0.5, 2.0]`). A model failing either is rejected
(`MeshError::UnsupportedLayout`) rather than emitting garbage.
### Endianness — file bytes are naive-BE, `k8in32` is a red herring ✅
The Canary GPU capture reports every vertex stream with fetch `endian = 2` (`k8in32`, each 32-bit
word byte-reversed). This describes the **guest-memory** copy the GPU fetches — **not** the `.xpr`
file bytes. Reading the file with a `k8in32` transform breaks the normals (mean `|normal|` → 1.33);
the plain per-element big-endian read yields exactly unit normals. So the game **rearranges** the
vertex data between the on-disc `.xpr` and the uploaded buffer; the decoder reads the file directly
and must use naive BE.
## Stage containers — multi-resource, grouped pools ✅ (content-anchored)
`hidden/resource3d/Stage_S*.xpr` are not single models but **collections of
enemy / prop sub-models** — up to ~400 `XBG7` resources each (e.g. `Stage_S07` =
378). Their data layout differs from the weapon files:
- Each resource is a block `[12-byte header][index buffer][vertex buffer]`.
Unlike weapons there is **no 12-byte gap between index and vertex** — the
vertex buffer directly follows the indices.
- The **index count** is the descriptor's `(index_bytes, index_count)` marker
(the *total* for the resource — a resource may have several sub-meshes summing
to it), **not** the first sub-mesh record. The **vertex count** is a `u32`
stored **32 bytes before** the marker.
- The blocks are **scattered through the data section, interleaved with the
container's texture data**, in an allocation order that is **not** directory
order and is **not stored** in any descriptor field we could find (the
descriptor holds sizes — `rel 160 ≈ index_bytes+3`, `rel 164 =
0x1000_0000 | (vertex_bytes+2)` — but no data offset). Reconstructing that
allocation order is unsolved.
Because the offset is not stored, each resource's block is located by
**content**: parser `Xbg7Model::stage_models` does one `O(file)` pass per
distinct stride to find every **vertex-buffer start** (an offset whose NORMAL —
`f16×4` at vertex `+12` — is unit length while the *previous* stride slot's is
not, i.e. a run boundary; ~one candidate per block, not millions), then pins each
resource to the unique candidate where the `index_count` indices ending just
before it are all `< vertex_count`, reference nearly all vertices, and yield
non-degenerate triangles with real extent. This is fast (≤ ~2 s on a 70 MB
stage) and unambiguous (no two resources collide). Blocks that fail validation —
the few quantized hero bodies — are **skipped**, never emitted as garbage.
**Coverage:** 5662 sub-models decode across the 22 stage files (e.g. `Stage_S07`
366/378, `Stage_S10` 7/9 — including the main enemy bodies `e003`/`e005`, their
LODs, weapons, and props). The viewer (`spawn_stage_models`) lays the decoded
sub-models out as a side-by-side "cast sheet", skipping the few huge skybox-plane
resources (300 k-unit quads). Cross-checked: `e003` = 2383 v / 1436 t bbox
23.5×9.5×32.5; `e005` = 2566 v / 1507 t; both 0 degenerate.
## Not yet decoded — ❔ the complex body layout
The hero-ship *body* meshes (`DeltaSaber_A/_T/_W.xpr` resource `f004`, and ~100 other models) still
decline. Two open sub-problems: (1) **multi-sub-mesh models** whose *first* sub-mesh decodes but a
later one's inter-mesh offset isn't yet handled (the `align16` advance is a guess) — these are
declined whole; (2) the big body meshes, where the data section does not start with an index buffer
and the geometry sits at descriptor-addressed offsets. NB the GPU capture showed **every** rendered
mesh is *single-stream* (just wider strides, e.g. 44 bytes = pos + f32×3 + colour + 2×f16×4), so the
body is likely single-stream-with-a-richer-declaration rather than the "separate streams" first
guessed — it was simply not rendered in the captured session (menu only). A capture taken *in a
mission* (where `DeltaSaber` renders) would hand over its exact declaration directly. `DeltaSaber_A`'s
5 XBG7 blocks are `f004` (body) + `_rou_f004_mnv01_L/_R`, `_mnv02`, `_turn180` (maneuver / pose).
## Evidence log
- 2026-07-12 — `rou_f001_wep_00.xpr`: XPR2 dir = 1×XBG7 (`f001_wep_00`) + 3×TX2D
(`_col/_gls/_spc`). Data section starts with a u16-BE index run (max 214), then stride-24
vertices. Descriptor record `[215,0,1092,4]` at desc `+0x2B0`; index-buffer byte size `0x888`
(=2184=1092×2) at desc `+0x1A0`. Recovered mesh = 215 v / 364 t, 362 non-degenerate, median tri
area 0.015 — coherent. → single-stream layout **PROBABLE**.
- 2026-07-12 — descriptor-driven sequential carving over all 166 models: **42 carve** under the
index-only check. Real 3D extent confirmed on `rou_f001_wep_04` (bbox 3.7×1.2×3.5).
- 2026-07-12 (refine) — attribute ranges differed per model (`wep_00` UV≈attr0/2, `wep_04`
attr4/5, `wep_03` attr1≈±60000 = garbage) → **refuted the fixed "6 half attrs, UV=attr0/2"
reading.** Found the descriptor **vertex declaration** (offsets 0x0C/0x14, usages 0x03 NORMAL /
0x05 TEXCOORD). Solved the vertex-base offset with a **unit-normal validator**: `index_end + 12`
gives median `|normal| = 1.000` on every weapon model (`wep_00/02/03/04`), vs `align16` landing 4
bytes early. UVs then land in `[0,1]`. Adding the normal gate: **25 models decode clean +
normal-valid** (the rest — incl. `Stage_S*` degenerate blobs — correctly declined).
- 2026-07-12 (DYNAMIC) — added a cvar-gated draw logger to Canary
(`command_processor.cc::LogDrawForRE`, cvar `log_draws`), captured the Ready Room / Briefings.
**GPU ground truth confirmed the static layout exactly**: primitive `prim=4` = **triangle LIST**
(settles the list-vs-strip question), and a stream with `f32x3 @offset0` + `f16x4 @3dw` +
`f16x2 @5dw`, stride 6 dwords = 24 bytes — matching `POSITION@0, NORMAL@0x0C, TEXCOORD@0x14`.
Revealed **stride varies** (24, 20, 28, 44 …) and that all streams are **single-stream** →
parsed the declaration for variable stride: coverage **25 → 36** (e.g. `wep_05` is pos+normal,
stride 20, no UV — previously mis-aligned). Also confirmed the endianness note above: fetch
`endian=2` (k8in32) is the *guest* copy; file stays naive-BE. `Stage_S*` now decode (stride 20).
- 2026-07-12 (STAGE) — `Stage_S*.xpr` decoded as multi-resource containers. Found each geometry
block is `[12B hdr][index buffer][vertex buffer]`, index count = descriptor marker (total, e.g.
`e003` = 4308 spanning 2 sub-meshes), vertex count = `u32` at marker−32 (`e003` = 2383 → verified
by max-index 2382 and 0 degenerate tris, bbox 23.5×9.5×32.5). Blocks are scattered among texture
data with **no stored offset** (descriptor rel 160 = idx_bytes+3, rel 164 = `0x1000_0000 |
(vtx_bytes+2)` are sizes, not offsets; block starts e.g. `e003`@0x10230, `e003_l`@0x5d000,
`e005_l`@0x9b000 are not directory-ordered). Solved by **content anchoring**: a single per-stride
pass finds vertex-run starts (unit NORMAL at +12 whose previous slot isn't), then match each
resource by strict index+triangle validation. **5662 sub-models decode across 22 stages** (S07
366/378), incl. the previously-declined main bodies `e005` (2566 v) and weapons — ≤2 s on 70 MB.
Parser `Xbg7Model::stage_models`, tests `stage_models_{decode,sweep,quality_audit}`.
- 2026-07-17 — **triangle-LIST re-confirmed; a strip interlude refuted; winding-consistency gate
added.** A 2026-07 change had briefly re-read the index buffers as triangle *strips* (to "fill
holes"). Refuted objectively with a new `XVERIFY` diagnostic that compares both readings by
**stored-normal agreement** (each triangle's cross-product face normal vs the sum of its vertices'
stored normals): the LIST reading gives agreement **1.000** on every clean weapon (`wep_00/03/04/19`
= only possible with correct topology + winding), the STRIP reading **~0.49** (random). The strip
reading also over-generated ~2.5× the triangles (wep_00: 938 vs 364) — a hole-filling garbage soup.
Reverted to LIST in both paths (`from_xpr2`, `read_pool_mesh`), matching the `prim=4` GPU capture.
Added an objective **winding-consistency gate** `max(na, 1-na)`: a correct carve is internally
consistent (agreement ≈1.0, or ≈0.0 for inverted-but-consistent winding — a real single-sided
mesh), a mis-carve scatters to the ≈0.5 middle. `from_xpr2` declines sub-meshes below 0.90 (e.g.
`wep_23` na=0.398 → declined instead of a spike-mess); the single-model **content-anchor fallback**
gates at 0.85; the large multi-resource **stage** path stays ungated (its enemy meshes span a
continuous 0.5–1.0 consistency range — a hard gate there dropped ~48/314 legit S07 blocks).
**Routing fixed:** `decode_models` (CLI) and the viewer now route by `count_xbg7` (1 → validated
records-based list decode, fallback to strict-gated anchor; >1 → stage anchor) instead of the old
"whichever decoder yields more verts" rule — that rule let stage content-anchoring win on
single-model weapon files and fabricate **phantom** blocks (a `wep_00` clone appearing inside
`wep_19`), duplicates, and spike-mess anchors. Weapons now: 33 clean-decode / 26 declined (declined
= genuinely multi-stream or un-carvable, shown as nothing rather than garbage); stage coverage
unchanged (S07 314). `expand_triangle_strip` retained as an `XVERIFY`-only diagnostic.
- 2026-07-12 — `DeltaSaber_A.xpr` body: data does **not** begin with indices; plain-`f32×3` runs
with ship-scale extent (span ≈27–34, matching bbox 30.0) found only at high offsets
(`data+0x28634C`, …) → multi-stream, **undecoded**.
- 2026-07-18 — **GROUPED-POOL layout cracked → the hero ship (Delta Saber) fully decodes.** The
detailed models (`DeltaSaber_*.xpr` + ~100 others) were declined for **location**, not format —
their vertex format is the standard stride-24 triangle list. A resource's *several* sub-meshes
don't interleave `[idx][vtx]` per block; they share **two grouped pools**: an **index pool**
(buffers concatenated in descriptor-marker order, each **4-byte aligned**) followed by a **vertex
pool** (each sub-pool `vtx_count × stride`, same order), with the index pool ending **exactly**
where the vertex pool begins. So the whole resource pivots on one unknown, the first vertex-pool
start `vb0` (= index-pool end, found by the unit-normal vertex-run scan); everything else is
derived: `ib0 = vb0 − span`, `ib[i] = align4(ib[i-1] + idx_count[i-1]·2)`,
`vb[i] = vb[i-1] + vtx_count[i-1]·stride`. Reversed statically from `DeltaSaber_T.xpr` and
**cross-checked against a Canary GPU draw-log capture** (mission ship = `DeltaSaber_T.xpr`, found
via the `--log_file_io` kernel hook): `f001` = body (idx@`data+0xC` = 0x5500C, vtx@0x61ACC, 10891 v
/ 8187 t) + **7 detail parts** (fins/cockpit/wingtips, markers at descriptor 0x3BEC…0x58FC) =
**8650 tris**, and **every sub-mesh decodes at 0 degenerate / full coverage / winding-agreement
1.000**. This is the layout the per-block adjacency anchor (`ib = vb − idx_bytes`) rendered as a
**spiky phantom** (it read 24561 indices starting 2782 B too late, agree 0.64, 1277 degenerate).
Insight: a single index marker reduces the grouped model to `index_end = vb0`, i.e. the existing
adjacency `ib = vb − idx_bytes` — so grouped **generalises** the single-block anchor (n=1 is
identical). Implemented as `anchor_grouped_meshes` (mesh.rs): `anchor_models` routes resources with
>1 index marker to it (validated per sub-mesh; on failure falls back to the old first-marker
adjacency anchor so stage coverage never regresses); single-marker stages/props keep the exact
prior path. The shared acceptance test is factored into `validate_block` (the connectivity
heuristic is relaxed for *derived* grouped parts, which are pinned by in-range + consistency, so
small flat fins aren't mis-rejected). Render self-check: `sylpheed-cli mesh render DeltaSaber_T.xpr
--only f001` (exact-name match excludes the `_rou_f001_mnv*` animation poses) → clean complete
fighter. Test `hero_ship_grouped_pool_decodes`. Colours/UVs still pending the running-game oracle.
- 2026-07-18 (refinement) — **4-byte vertex-pool alignment + weapon recovery.** The grouped-pool
rule "index pool ends exactly where the vertex pool begins" is really "the vertex pool is **4-byte
aligned** after the index pool": `vb0 = align4(ib0 + span)`, so 0..=3 bytes of padding can sit
between them. DeltaSaber's index pool ended already-aligned (pad 0), which hid this; **19
weapon/`*_hangar` models** (single- and multi-marker: `wep_08/11/34/58/62/69/81/83…`) have pad 2
and so decoded to *nothing* — the viewer then showed them as a flat 2D texture instead of a model.
Fix: both anchors try `pad ∈ 0..=3` (`ib = vb − idx_bytes − pad` for the single-block adjacency
anchor; `ib0 = vb0 − span − pad` for the grouped pivot), validated — a wrong pad reads shifted
indices → agreement collapses < 0.85, so only the true pad passes. pad>0 in the ungated stage path
is gated at a strict 0.85 to avoid a false anchor; pad 0 keeps its exact prior behaviour (stages
unchanged). Result: all 19 now decode as clean models (e.g. `wep_34` 1243 v / 1233 t, a
long-barrelled gun-pod; `wep_08` 3 sub-meshes / 478 t). Viewer routing already falls through
`from_xpr2` → `anchor_models(0.85)` for single-XBG7 files, so the recovered grouped/padded weapons
now preview as meshes.
- 2026-07-18 (refinement 2) — **pivot on the largest sub-mesh; all 19 recovered.** Three weapons
(`wep_81`, `wep_81_hangar`, `wep_30_hangar`) still declined because the grouped pivot validated
`markers[0]`, which for these is a tiny *elongated* lead bracket that fails the connectivity gate
even when perfectly placed. Fixed by pivoting the alignment check on the **largest** marker (max
index count) — the sub-mesh whose triangle-quality/connectivity signature most reliably confirms
`(ib0, vb0)`. Once the pivot validates, markers up to it are read unconditionally (a legitimately
tiny/flat lead part may fail the quality gates yet still be real), and markers after it stay
validated so a stray trailing marker ends the chain. Result: **all 19 previously-declined weapons
decode** (`wep_81` 460 t missile w/ tail fins; `wep_30_hangar` 334 t). DeltaSaber unchanged (its
body IS the largest marker → same pivot). 7/7 disc tests green, stage quality audit unchanged.

126
tools/analyze_drawlog_wvp.py Normal file
View File

@@ -0,0 +1,126 @@
#!/usr/bin/env python3
"""Recover each ship part's runtime WORLD transform from a Canary draw log.
The draw logger (branch capture-drawlog) now dumps, per distinct draw:
- the first ~8 object-space vertex positions (pre-transform), and
- the vertex-shader float constants c0..c31 (which hold the transform matrices).
The game bakes each part's WORLD matrix into the constants, so the shader's
World*View*Proj (WVP) for a part is VP * part_World. The body (bdy_01) is drawn
with an identity world, so body_WVP = VP. Hence for any part drawn in the SAME
frame (the ship's parts always are):
part_World = inv(body_WVP) * part_WVP (View/Proj cancel)
We don't know a-priori which 4 consecutive constants form the WVP, so we try every
block c[i..i+3] and keep the one for which inv(body_WVP)*part_WVP is a RIGID
transform (orthonormal 3x3 + translation) for the parts — that uniquely identifies
the matrix block. The translation column of each part's world matrix is the
placement we want (esp. the ±X nacelle offset the static XBG7 graph lacks).
Usage: analyze_drawlog_wvp.py xenia_re_draws.log
Identify the body draw via --body-indices (the draw with the most indices) or by
matching object-space positions to our decoded sub-meshes.
"""
import sys, re
import numpy as np
def parse(path):
draws = []
cur = None
consts = {}
mode = None
for line in open(path):
if line.startswith("DRAW "):
if cur is not None:
cur["consts"] = consts
draws.append(cur)
cur = {"header": line.strip(), "positions": [], "textures": [],
"size_words": 0}
consts = {}
mode = None
m = re.search(r"index\[base=0x([0-9A-Fa-f]+) count=(\d+)", line)
cur["idx_base"] = int(m.group(1), 16) if m else None
cur["idx_count"] = int(m.group(2)) if m else None
m = re.search(r"indices=(\d+)", line)
cur["indices"] = int(m.group(1)) if m else None
elif cur is not None and line.strip().startswith("stream "):
# Largest vertex buffer (size_words) = the hull (body, identity world).
m = re.search(r"size_words=(\d+)", line)
if m:
cur["size_words"] = max(cur["size_words"], int(m.group(1)))
elif cur is None:
continue
elif line.strip().startswith("positions:"):
cur["positions"] = [tuple(map(float, t.split(",")))
for t in re.findall(r"\(([^)]+)\)", line)]
elif line.strip().startswith("textures:"):
cur["textures"] = re.findall(r"base=0x([0-9A-Fa-f]+)", line)
elif line.strip().startswith("vsconst"):
mode = "vsconst"
elif mode == "vsconst":
m = re.match(r"\s*c(\d+)\s+(-?[\d.]+)\s+(-?[\d.]+)\s+(-?[\d.]+)\s+(-?[\d.]+)", line)
if m:
consts[int(m.group(1))] = np.array([float(m.group(i)) for i in range(2, 6)])
else:
mode = None
if cur is not None:
cur["consts"] = consts
draws.append(cur)
return draws
def wvp_from(consts, i):
"""4x4 from constants c[i..i+3] as rows."""
if not all(k in consts for k in (i, i+1, i+2, i+3)):
return None
return np.array([consts[i], consts[i+1], consts[i+2], consts[i+3]])
def is_rigid(M):
"""3x3 upper-left orthonormal (rotation/reflection), within tolerance."""
R = M[:3, :3]
if not np.all(np.isfinite(R)):
return False
G = R @ R.T
return np.allclose(G, np.eye(3), atol=1e-2) or np.allclose(G / (np.trace(G)/3), np.eye(3), atol=1e-2)
def main():
if len(sys.argv) < 2:
print(__doc__); return
draws = parse(sys.argv[1])
print(f"parsed {len(draws)} draws")
# Body = the draw with the largest vertex buffer (the ~10891-vert hull, drawn
# with an identity world). idx_count is unreliable (the hull's 9 material
# groups dedup to the first group's partial count).
body = max(draws, key=lambda d: d.get("size_words") or 0)
print(f"body draw: {body['header'][:90]} size_words={body['size_words']}")
# Try every constant block; report the one giving rigid part transforms.
for i in range(0, 29):
bwvp = wvp_from(body["consts"], i)
if bwvp is None:
continue
try:
binv = np.linalg.inv(bwvp)
except np.linalg.LinAlgError:
continue
worlds = []
rigid = 0
for d in draws:
pw = wvp_from(d["consts"], i)
if pw is None:
continue
W = binv @ pw # column-vector convention; may need transpose
worlds.append((d, W))
if is_rigid(W) or is_rigid(W.T):
rigid += 1
if rigid >= max(2, len(worlds)//2):
print(f"\n=== WVP block c{i}..c{i+3}: {rigid}/{len(worlds)} rigid ===")
for d, W in worlds:
# translation is the last column (or last row, convention-dependent)
tcol = W[:3, 3]
trow = W[3, :3]
print(f" idx={d['idx_count']:>6} base={d['idx_base'] and hex(d['idx_base'])}"
f" t_col=({tcol[0]:+.2f},{tcol[1]:+.2f},{tcol[2]:+.2f})"
f" t_row=({trow[0]:+.2f},{trow[1]:+.2f},{trow[2]:+.2f})")
if __name__ == "__main__":
main()

88
tools/extract_movie_subtitles.py Normal file
View File

@@ -0,0 +1,88 @@
#!/usr/bin/env python3
import struct, zlib, re, os
MOD=0x00FFF9D7; RECIP=0x80031493
def rol(v,n,b=32):
v&=(1<<b)-1; return ((v<<n)|(v>>(b-n)))&((1<<b)-1)
def nh(name):
bs=bytearray(name.encode('latin-1'))
for i,b in enumerate(bs):
if 0x41<=b<=0x5A: bs[i]=b+0x20
a=0;bb=0
for byte in bs:
c=(byte if byte<0x80 else byte-0x100)&0xFFFFFFFF
a=((rol(a,8)&0xFFFFFF00)+c)&0xFFFFFFFF
bb=(bb+c)&0xFFFFFFFF
hi=((a*RECIP)>>32)&0xFFFFFFFF; q=rol(hi,9)&0x1FF
a=(a-(q*MOD))&0xFFFFFFFF
return (((bb<<24)&0xFF000000)|(a&0xFFFFFF))&0xFFFFFFFF
EX="/home/fabi/RE Project Sylpheed/sylph_extract"
def load_pak(base):
"""base like dat/movie/eng ; returns list of (hash,off,size), data bytes"""
pak=open(f"{EX}/{base}.pak","rb").read()
n=struct.unpack(">I",pak[4:8])[0]
es=[struct.unpack(">III",pak[16+12*i:28+12*i]) for i in range(n)]
p00=f"{EX}/{base}.p00"
data=open(p00,"rb").read() if os.path.exists(p00) else pak
return es, data
def unz(blob):
if blob[:2]==b'Z1':
zi=blob.find(b'\x78\xda');
if zi<0: zi=blob.find(b'\x78\x9c')
return zlib.decompress(blob[zi:])
return blob
# --- verify the track key scheme ---
es,data=load_pak("dat/movie/eng")
keys={h for h,_,_ in es}
for mv,exp in [("S00A",0x6F2D9663),("hokyu_DS_s02A",0x3662B1F8),("RT01C_1",0x756F69FB),("S16A",0xEEB85377)]:
k=nh(f"subtitle_{mv}.tbl")
print(f"subtitle_{mv}.tbl -> {k:08X} expect {exp:08X} inpak={k in keys} {'OK' if k==exp else 'MISMATCH'}")
# --- build MSG_DEMO text table from GP_MAIN_GAME_E ---
es2,data2=load_pak("dat/GP_MAIN_GAME_E")
text={} # 'MSG_DEMO_ddd_ppp_ll' -> string
keyrx=re.compile(r'^MSG_DEMO_\d+(_\d+)*$')
for h,o,s in es2:
dec=unz(data2[o:o+s])
if dec[:4]!=b'IXUD': continue
# walk null-terminated utf-16le strings in the whole block
toks=re.findall(rb'(?:[\x20-\x7e]\x00){2,}', dec)
toks=[t.decode('utf-16-le') for t in toks]
# pair: text followed by its key
for i in range(len(toks)-1):
if keyrx.match(toks[i+1]) and not keyrx.match(toks[i]):
text[toks[i+1]]=toks[i]
print(f"\nMSG_DEMO text lines loaded: {len(text)}")
# --- render a movie's subtitle transcript ---
def track_for(mv):
k=nh(f"subtitle_{mv}.tbl")
for h,o,s in es:
if h==k: return unz(data[o:o+s])
return None
# index text by integer demo id -> ordered list of lines
bydemo={}
for k,v in text.items():
m=re.match(r'MSG_DEMO_(\d+)_',k)
if m: bydemo.setdefault(int(m.group(1)),[]).append((k,v))
for d in bydemo: bydemo[d].sort()
def transcript(mv):
dec=track_for(mv)
if not dec: return f"[no track for {mv}]"
toks=re.findall(rb'(?:[\x20-\x7e]\x00){2,}', dec)
toks=[t.decode('utf-16-le') for t in toks]
out=[f"=== {mv}.wmv ==="]
# collect all MSG_DEMO ids and timecodes in order; pair positionally
seq=[('id',int(re.match(r'MSG_DEMO_(\d+)$',t).group(1))) if re.match(r'MSG_DEMO_(\d+)$',t)
else ('tc',t) if re.match(r'\d\d:\d\d\.\d\d$',t) else ('x',t) for t in toks]
ids=[v for k,v in seq if k=='id']; tcs=[v for k,v in seq if k=='tc']
for j,demo in enumerate(ids):
tc=tcs[j] if j<len(tcs) else "--:--.--"
lines=[v for _,v in bydemo.get(demo,[])]
out.append(f" {tc} [{demo:3d}] "+(" ".join(lines) if lines else "(no text on disc)"))
return "\n".join(out)
for mv in ["S00A","S16A","hokyu_DS_s02A"]:
print(); print(transcript(mv))