feat(formats,viewer): movie subtitles, voice decode, and the movie manifest

The movie cutscene subtitle + voice pipeline, driven by the ADVERTISE_MOVIE
manifest (the authoritative movie -> subtitle -> voice index). Also flushes
several sessions of local WIP (async viewer loading, grouped-pool XBG7/hero-ship
decode, drawlog tooling). See docs/HANDOFF-movie-voice-subtitles-2026-07-19.md.

Subtitles (movie_subtitle.rs):
- Full movie->track->text chain; join multi-line captions sharing one timing
  (fixes S13A dropped "Look at it father" line); overlap-safe active_cues();
  Latin-1 accents preserved.

Voice (slb.rs): XACT .slb -> XMA1 RIFF; take the FIRST sub-wave bounded by its
declared data size (fixes S10-S16 alternate-take garble); list_voice_clips.

Manifest (movie_manifest.rs): parse ADVERTISE_MOVIE (0x5B983A08) for the real
movie->voice binding (not always VOICE_<movie>; e.g. hokyu -> VOICE_D_* in etc\).
Resolve the token's sound.pak path via sounds.tbl. DIRECT bindings only — the
demo-id shared-clip fallback for unbound hokyu movies was verified WRONG in-game
and reverted (unbound hokyu stay unvoiced; correct join key is an OPEN problem).

Viewer: manifest-driven voice (movie player toggle + solo button), standalone
"Voice Lines" browser, stacked caption overlay.

Tests: 46 formats-lib + 11 viewer-lib + movie_manifest/movie_subtitle/slb disc
tests; full workspace green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-07-19 20:15:41 +02:00
parent 578c71a1b9
commit 95de29f290
22 changed files with 4684 additions and 493 deletions

View File

@@ -0,0 +1,95 @@
# Handoff — movie subtitles, voice, and the movie manifest (2026-07-19)
Branch `feature/ipfb-idxd-parser`. This commit flushes several sessions of local
WIP; the **new, finished** work is the movie subtitle + voice + audio pipeline
and the movie manifest. One item is deliberately **left open** (see §4).
## 1. What's DONE and verified (static RE + tests)
### Movie subtitles (`crates/sylpheed-formats/src/movie_subtitle.rs`)
- Full movie→track→text chain (see `docs/re/structures/movie-subtitles.md`).
- **Multi-line caption fix**: a caption stored as several consecutive text tokens
sharing one timing (S13A: `"Look at it father"` + `"& beautiful isn't it"` @
`01:14.80`) — `parse_track` now joins them with `\n`. Previously all but the
last line were dropped. Disc test asserts both lines survive.
- **Overlap rendering**: `MovieSubtitles::active_cues(t)` returns every cue active
at `t`; the viewer stacks them (was: only the first cue shown).
- Umlauts / Latin-1 accents preserved (`utf16le_tokens`); Japanese label romanized
(CJK font still a known gap — see §5).
### Voice decode (`crates/sylpheed-formats/src/slb.rs`)
- `sound.pak` entries are XACT `.slb` banks wrapping **XMA1** (fmt tag `0x0165`,
48 kHz, mono content). `to_xma_riff` rebuilds a decodable RIFF; FFmpeg `xma1`
decodes it. Downmix mono via `-af pan=mono|c0=c0`.
- **Multi-subwave fix**: take the FIRST sub-wave bounded by its declared `data`
size — NOT `data..EOF` (which appended later takes = the S10S16 garble).
Verified: S13A→83.78s == video.
- `list_voice_clips` enumerates the voice library from `sounds.tbl`.
### Movie manifest (`crates/sylpheed-formats/src/movie_manifest.rs`) — the index
- `tables.pak` entry `ADVERTISE_MOVIE` (IDXD, hash `0x5B983A08`) is the
authoritative **movie → subtitle → voice** map. Located by shape (`is_manifest`),
parsed by grouping the string pool on `.wmv`.
- **Authoritative voice binding** replaces the old `VOICE_<movie>` guess:
101 movies, 83 with a `VOICE_` token, 18 without. The token is NOT always
`VOICE_<movie>` — 5 `hokyu_*` movies bind to `VOICE_D_450..454` in `\etc\`,
which a guess would miss. Token's subdir varies (Movie/etc/Voice) → resolve via
`sounds.tbl`. Disc test: all 83 resolved entries exist in `sound.pak`.
### Viewer wiring (`iso_loader.rs`, `ui.rs`)
- `resolve_movie_voice_clip` reads the manifest + `sounds.tbl` (DIRECT bindings
only) → `handle_voice_request` (movie player 🗣 toggle) and the 🎧 solo button
(`RequestAudio.movie`) use it. Unbound movies correctly post no audio.
- Standalone **View → 🎙 Voice Lines** browser (filter + ▶ play any `sound.pak`
clip) via `VoiceLibrary` + `AudioPreview`.
Tests: 46 formats-lib + 11 viewer-lib + disc (`movie_manifest_disc`,
`movie_subtitle_disc`, `slb_disc`) all green. Full `cargo test --workspace` green.
## 2. Also bundled in this commit (prior local WIP, not this session's focus)
- Viewer **async stage/model loading** (off-thread `prepare_xpr`, cancellable) —
`iso_loader.rs`/`ui.rs`/`camera.rs`. See memory `reborn-viewer-async-loading`.
- **Grouped-pool XBG7 / hero-ship** decode refinements in `mesh.rs`, `cli/main.rs`,
`mesh_disc.rs`. See memory `reborn-ship-drawlog` / `reborn-saber-texture-investigation`.
- `tools/analyze_drawlog_wvp.py`, `tools/extract_movie_subtitles.py`.
These build and test green but were not re-verified for behaviour this session.
## 3. USER TO VERIFY IN GUI (couldn't be tested from CLI)
1. S13A cutscene shows BOTH lines of the "Look at it father" caption.
2. Story-movie voice (S13A etc.) lines up with the video after the subwave fix.
3. The 5 directly-bound `hokyu_*` movies play voice; other hokyu are silent (see §4).
4. View → 🎙 Voice Lines browser lists and plays clips.
## 4. OPEN PROBLEM — unbound-hokyu resupply voice (do NOT re-guess)
The resupply cutscenes clearly SHARE voice recordings (video-only varies per
mission), but the correct join key for the **13 unbound `hokyu_*`** movies is
**unknown**:
- Manifest DIRECTLY binds only 5: `LS_s02A→450, LS_s09A→451, DS_s02A→452,
LS_s02H→453, DS_s07H→454` (grouping verified against the raw layout).
- Keying unbound movies by subtitle **demo id** (600→450…604→454, so
`hokyu_DS_s13A` demo602→`VOICE_D_452`) was implemented, tested against the
running game by the user, and is **WRONG** (wrong line). It has been **reverted**
— unbound hokyu now stay unvoiced rather than play wrong audio.
- Red flag: only `VOICE_D_450..454` exist; decoded durations `450=2.8s 451=1.6s
452=2.2s` but `453=0.14s 454=0.43s` — far too short for the spoken line, so
these `.slb` are likely **multi-subwave / not cleanly sliced** (same class as
the deferred B/C banks below).
**Next-box options:** (a) get the real join key from mission-event data (mission
scripts/IDXD tables that trigger resupply), or (b) build a proper multi-subwave
`.slb` decoder and verify audio by ear. Needs a concrete calibration clue from the
user (e.g. "s13A should sound like <movie X>" or the actual spoken words).
## 5. Other deferred items (unchanged)
- ~49 "layout-B/C" movie/voice banks (most RT + S07B/S11A/S12B/S13B) + some
individual clips need an XMA1 packet/seek-table reassembler.
- CJK/Japanese text rendering in the viewer (needs a CJK TTF via egui FontDefinitions).
- Texture colours (channel order / sRGB) still on the dynamic-RE backlog.
## 6. Resume checklist (next box)
1. `git pull` on `feature/ipfb-idxd-parser`; set `SYLPHEED_DISC=<extract root>`
(had `.../sylph_extract`).
2. Build guardrail: `CARGO_BUILD_JOBS=4` always (15 GB box; full `-j` OOM'd before).
3. `cargo test --workspace` (with `SYLPHEED_DISC`) should be green.
4. Pick up §4 (unbound-hokyu voice) — the only open thread from this session.

View File

@@ -0,0 +1,174 @@
# Movie subtitles & the movie ↔ mission ↔ text chain
Reverse-engineered 2026-07-19 (static, from the extracted disc). The full chain
that links a cutscene movie to its on-screen subtitle text is now closed.
## Files involved
- `dat/movie/*.wmv` — the cutscene videos. **Named by mission** (see below).
- `dat/movie/<lang>.pak` + `<lang>.p00` — per-language **subtitle timing tracks**
(`eng`, `jpn`, `deu`, `fra`, `esp`, `ita`) plus the caption font.
- `dat/GP_MAIN_GAME_<L>.pak` + `.p00` — per-language **caption TEXT**
(`E`=Eng, `J`=Jpn, `D`=Deu, `F`=Fra, `I`=Ita, `S`=Esp).
- `dat/tables.pak` entry `ADVERTISE_MOVIE` (hash `0x5B983A08`) — the master
**movie manifest**: all 101 `.wmv` names in mission-progression order, each
bound to its subtitle track and (optionally) its **voice bank** (see below).
## Movie filename → mission
Purely from the filename:
| Pattern | Meaning |
|---------|---------|
| `S<NN><P>.wmv` | Stage `NN` **story** cutscene, part `P` (A/B/C…) — e.g. `S02C` = stage 2, 3rd story scene |
| `RT<NN><P>.wmv` | Stage `NN` **radio / briefing** transmission, part `P` (`_1`/`_2` = split clips) |
| `hokyu_<LS\|DS>_s<NN><P>.wmv` | Stage `NN` **resupply** scene (`hokyu` = 補給). `LS`/`DS` = the two resupply-ship variants |
| `ADV.wmv` | Intro / title movie |
97 movies total: 27 story, 50 radio, 19 resupply, 1 intro. (Some referenced
stages — s24, s27, RT16 — exist as keys but the .wmv isn't in this extract.)
## Subtitle timing track — `<lang>.pak`
IPFB archive (`IPFB`, BE-u32 count, 16-byte header; TOC of
`[name_hash u32][offset u32][size u32]` triples, sorted by hash, into `.p00`).
- **Track key = `name_hash("subtitle_<movie_basename>.tbl")`** (the hash
lowercases internally, so basename case is irrelevant). This is the movie→track
link. Verified: `subtitle_S00A.tbl``0x6F2D9663`, `subtitle_hokyu_DS_s02A.tbl`
`0x3662B1F8`, `subtitle_RT01C_1.tbl``0x756F69FB`.
- The archive also holds **RATC** pre-rendered title-card / number textures
(`pwterop_s01a1.t32`, `pwrt_rt01_str.t32`, `pwnum0-9.t32`) + one TrueType font.
- **Each track data block is `Z1`+zlib**: bytes `5A 31` ("Z1"), a small header,
then a raw zlib stream (`78 DA`/`78 9C`). `zlib.decompress(blob[blob.find(b"\x78\xda"):])`.
- Decompressed = an **IXUD** container. Payload (UTF-16LE) is the timing sheet:
`SUBTITLE MSG_DEMO_<demo> <mm:ss.cc> MSG_DEMO_<demo> <mm:ss.cc> …`.
So the track says *which* demo-message shows *when*, not the text itself.
## Caption text — `GP_MAIN_GAME_<L>.pak`
Same IPFB+`.p00`. Among its ~1119 entries, **32 blocks are `Z1`+zlib → IXUD**
string containers holding the movie caption text. Layout: `IXUD`, u32 version,
hash@0x08, count@0x14, then `(recordhash,offset,len)` triples, then a UTF-16LE
string region where **each line is stored as `text` immediately followed by its
key** `MSG_DEMO_<demo>_<page>_<line>` (captions wrap across `_00`,`_01`, …).
537 English lines recovered. Entries 1 & 19 are the IDXD *schema* records
(`ID`, `PageCount`, `Character`=speaker e.g. `TCAFSUPPLY`, `Face`, line refs) —
no text, just structure.
## Movie → voice track: the manifest binding (`ADVERTISE_MOVIE`)
The `ADVERTISE_MOVIE` manifest is **also the authoritative movie→voice index**.
Its string pool emits, per movie, a run led by `<movie>.wmv` optionally followed
by `<pak>+….prt` (overlay art), `<pak>+SUBTITLE_<movie>.tbl`, and a bare
`VOICE_<token>`. Grouping the pool on `.wmv` (records are emitted in order)
recovers `movie → Option<voice_token>` without decoding the IDXD record binary
(`crate::movie_manifest`).
The voice token is **not** always `VOICE_<movie>`, so the manifest is required —
guessing both misses real bindings and invents tracks for silent movies:
- **83 / 101 movies have a voice token.** Story/radio movies use `VOICE_<movie>`
in `<lang>\Movie\`.
- **5 `hokyu_*` resupply movies bind to in-mission radio clips** — e.g.
`hokyu_LS_s02A → VOICE_D_450`, which lives in `<lang>\etc\`, *not* `Movie`.
A `VOICE_<movie>` guess would never find these.
- **18 movies have no direct voice token** = 4 boot logos + 1 HD test pattern +
**13 `hokyu_*` movies** (incl. `hokyu_DS_s13A`). Only the manifest's **direct**
bindings are trusted for playback.
### Shared resupply voice — UNRESOLVED for unbound movies
The manifest directly binds only **5** resupply movies, each to a shared
`VOICE_D_45x` clip in `<lang>\etc\`:
| bound movie | subtitle demo | clip |
|-------------|---------------|------|
| `hokyu_LS_s02A` | 600 | `VOICE_D_450` |
| `hokyu_LS_s09A` | 601 | `VOICE_D_451` |
| `hokyu_DS_s02A` | 602 | `VOICE_D_452` |
| `hokyu_LS_s02H` | 603 | `VOICE_D_453` |
| `hokyu_DS_s07H` | 604 | `VOICE_D_454` |
The resupply cutscenes clearly **share** voice recordings (only the video varies
per mission), so the 13 unbound `hokyu_*` movies must reuse one of these — but
the **correct join key is not yet known**:
- Keying by subtitle **demo id** (so `hokyu_DS_s13A`, demo 602 → `VOICE_D_452`)
was tried and is **WRONG** — it plays the wrong recording in-game. Do not use.
- Only `VOICE_D_450..454` exist (no 44x/45x neighbours). Decoded durations are
suspicious — `450`=2.8s, `451`=1.6s, `452`=2.2s, but `453`=**0.14s**,
`454`=**0.43s** — far too short for the spoken line, so these `.slb` banks are
likely **multi-subwave / not cleanly sliced** by the current extractor (same
class as the deferred B/C banks).
⇒ The unbound-hokyu voice mapping is **open** (needs either the real join key
from mission data, or a proper multi-subwave `.slb` decode + audio verification).
`hokyu_LS_s24A`/`s27A` have no subtitle track at all (stages absent from this
extract).
The token's `sound.pak` **subdirectory is not fixed** (`Movie` / `etc` / `Voice`),
so resolve it via `sounds.tbl` (which lists the full `<lang>\…\<token>.slb` path)
rather than assuming a directory. `movie_manifest::resolve_voice_entry` does the
full chain. Verified: all 83 resolved entries exist in `sound.pak`.
## Caption packing quirks (parser must handle)
- **Multi-line captions are split into consecutive text tokens** that share one
trailing timing, e.g. S13A stores `"Look at it father"` + `"& beautiful isn't
it"` before `01:14.80-01:17.60`. Accumulate every text token since the last
timing and join with `\n`; pairing strictly 1:1 silently drops all but the last
line.
- **Overlapping spans**: some tracks show two captions at once (an open-ended
radio line still up when the next range line starts). The viewer stacks every
cue active at `t` (`MovieSubtitles::active_cues`) instead of showing only the
first.
## The full join
```
tables.pak / ADVERTISE_MOVIE → list of movies (mission order)
<movie> ─ nh("subtitle_<movie>.tbl") ─→ <lang>.pak track
track (IXUD) → [ (MSG_DEMO_<d>, timecode), … ]
MSG_DEMO_<d> → GP_MAIN_GAME_<L> → "the localized caption line(s)"
```
## Coverage
- 92 / 97 movies have a subtitle track.
- **66 movies carry timed `MSG_DEMO` captions** — the **radio (`RT*`)** and
**resupply (`hokyu_*`)** movies. These fully decode to timed text.
- The **27 story (`S*`) movies have a track but 0 timed captions** — their text
is delivered as the **pre-rendered title-card textures** (`pwterop_*`, burned
styling), not MSG_DEMO lines.
## Worked examples (English)
```
RT01C_1.wmv (Stage 1 radio, part C):
00:00.50 [14] We did it! Okay, all pilots follow my lead!
00:06.80 [15] Rhino Leader to ACROPOLIS. We made it through and we're coming
home. Roger. It's good to see you're all safe.
00:19.30 [17] Yeah, but Brandon ... Damn. There's only seven of us. …
hokyu_DS_s02A.wmv (Stage 2 resupply):
00:00.00 [602] Resupply complete. You are cleared for take-off!
```
## Reusable extractor
`tools/extract_movie_subtitles.py` — parses `<lang>.pak`, resolves each movie's
track, cross-references `GP_MAIN_GAME_<L>` text, and prints per-movie timed
transcripts + the movie→mission table.
## In-mission dialogue (future work)
`GP_MAIN_GAME_<L>.pak` is the **global** message store, not just movie captions:
its `MSG_DEMO_*` table also holds the in-mission radio/dialogue lines (same demo
id space). So the *text* of gameplay dialogue is already decodable with
[`crate::movie_subtitle::build_demo_text`]. What's missing is the **trigger**
which demo id fires at which mission event — and that lives in the **mission
data** (mission scripts / `GP_MAIN_GAME` IDXD tables), not in the text pack. When
reversing mission data, look for demo-id references there to bind dialogue to
events; the IDXD "Message" schema records also carry `Character` (speaker) and
`Face` (portrait) per line.

View File

@@ -0,0 +1,62 @@
# Sound bank audio — `sound.pak` / `.slb` / XMA1
Reverse-engineered 2026-07-19 (static). `dat/sound.pak` (+ `sound.p00..p04`,
~1.08 GB) is the game's entire audio bank: **9519** IPFB entries, each an XACT
`.slb` sound bank wrapping **XMA1** audio.
## Names
Entry keys are `name_hash(<string>)` of filenames stored verbatim in the IDXD
string pools of `tables.pak``eng\sounds.tbl` (entry 39) and `jpn\sounds.tbl`
(entry 45). All 9519 recovered. Families:
- `<lang>\Voice\VOICE_<SPK>_<NNN>.slb` — per-line character voice (SPK = ADAN,
TCAF, RHIN, ADPL, BIRD, ACRO, ZZZZ…). Only `eng`/`jpn` voice exists.
- `<lang>\etc\VOICE_{A,B,C,D}_<NNN>.slb`, `<lang>\Briefing\BR<n>_<m>.slb`.
- **`<lang>\Movie\VOICE_<movie>.slb`** — the continuous voice track for cutscene
`<movie>.wmv` (e.g. `eng\Movie\VOICE_RT07A.slb`). Played from the video start.
- `BGM_###.slb` (32) — music. `Static.slb`, `JNGL_###` — SFX/ambience.
Only English and Japanese have voice; subtitles cover more languages, voice does
not. See [`crate::slb::movie_voice_name`].
## Codec
XMA1: RIFF `fmt ` tag **0x0165**, a 32-byte `XMAWAVEFORMAT`, **48000 Hz**. The
content is **mono** — some clips carry it in the **left channel only** (right is
silence), others are **dual-mono** (L = R). Downmix to mono by taking the left
channel (it always holds the full signal).
`XMAWAVEFORMAT` (one stream, 32-byte `fmt `): SampleRate @ fmt+16 (u32 LE),
Channels @ fmt+29 (u8), ChannelMask @ fmt+30 (u16). Movie voices report 2
channels / mask 0x2 despite the mono content.
## `.slb` layouts → decodable RIFF
Two on-disc layouts (`sound.pak` movie voices split ≈ 63 RIFF / 15 headerless):
- **RIFF present** — a `RIFF/WAVE` sits inside the bank; its 32-byte `fmt ` is
the real header. The XMA packets are **everything after that RIFF's `data`
chunk header** (`slb[data_pos+8..]`); the declared `data` size is unreliable
(sometimes half the real length), so take to end and clamp to the media length
downstream. Some banks have trailing bank data past the real audio — always
beyond the video length, so clamping to the video duration drops it.
- **Headerless** — no `RIFF` anywhere; a fixed **1392-byte** header precedes raw
XMA1 packets (48 kHz, 2 channels). Data = `slb[1392..]`.
[`crate::slb::to_xma_riff`] rebuilds a standalone XMA1 `RIFF/WAVE` for either
layout, ready for FFmpeg's `xma1` decoder.
## Decode (FFmpeg)
`ffmpeg -i rebuilt.riff -af "pan=mono|c0=c0" -t <media_len> out.wav` — the RIFF
is fed to the `xma1` decoder, downmixed to the left channel, clamped to length.
Verified: `VOICE_S00A` → 93.9 s == `S00A.wmv` 93.87 s; `VOICE_ADV` → 137.3 s ==
137.7 s; `VOICE_RT07A` → 49 s ≈ 50 s.
## Reading one entry cheaply
`sound.pak` is ~1 GB. Use [`crate::PakArchive::parse_toc`] on the small `.pak`
index, binary-search the name-hash, then read just `[offset, offset+size)` from
the `.pNN` segments (the viewer's `read_segment_range` seeks into the right
segment) instead of loading the whole archive.