Syplheed-Reborn/docs/HANDOFF-movie-voice-subtitles-2026-07-19.md

# Handoff — movie subtitles, voice, and the movie manifest (2026-07-19)

Branch `feature/ipfb-idxd-parser`. This commit flushes several sessions of local
WIP; the **new, finished** work is the movie subtitle + voice + audio pipeline
and the movie manifest. One item is deliberately **left open** (see §4).

## 1. What's DONE and verified (static RE + tests)

### Movie subtitles (`crates/sylpheed-formats/src/movie_subtitle.rs`)
- Full movie→track→text chain (see `docs/re/structures/movie-subtitles.md`).
- **Multi-line caption fix**: a caption stored as several consecutive text tokens
  sharing one timing (S13A: `"Look at it father"` + `"& beautiful isn't it"` @
  `01:14.80`) — `parse_track` now joins them with `\n`. Previously all but the
  last line were dropped. Disc test asserts both lines survive.
- **Overlap rendering**: `MovieSubtitles::active_cues(t)` returns every cue active
  at `t`; the viewer stacks them (was: only the first cue shown).
- Umlauts / Latin-1 accents preserved (`utf16le_tokens`); Japanese label romanized
  (CJK font still a known gap — see §5).

### Voice decode (`crates/sylpheed-formats/src/slb.rs`)
- `sound.pak` entries are XACT `.slb` banks wrapping **XMA1** (fmt tag `0x0165`,
  48 kHz, mono content). `to_xma_riff` rebuilds a decodable RIFF; FFmpeg `xma1`
  decodes it. Downmix mono via `-af pan=mono|c0=c0`.
- **Multi-subwave fix**: take the FIRST sub-wave bounded by its declared `data`
  size — NOT `data..EOF` (which appended later takes = the S10–S16 garble).
  Verified: S13A→83.78s == video.
- `list_voice_clips` enumerates the voice library from `sounds.tbl`.

### Movie manifest (`crates/sylpheed-formats/src/movie_manifest.rs`) — the index
- `tables.pak` entry `ADVERTISE_MOVIE` (IDXD, hash `0x5B983A08`) is the
  authoritative **movie → subtitle → voice** map. Located by shape (`is_manifest`),
  parsed by grouping the string pool on `.wmv`.
- **Authoritative voice binding** replaces the old `VOICE_<movie>` guess:
  101 movies, 83 with a `VOICE_` token, 18 without. The token is NOT always
  `VOICE_<movie>` — 5 `hokyu_*` movies bind to `VOICE_D_450..454` in `\etc\`,
  which a guess would miss. Token's subdir varies (Movie/etc/Voice) → resolve via
  `sounds.tbl`. Disc test: all 83 resolved entries exist in `sound.pak`.

### Viewer wiring (`iso_loader.rs`, `ui.rs`)
- `resolve_movie_voice_clip` reads the manifest + `sounds.tbl` (DIRECT bindings
  only) → `handle_voice_request` (movie player 🗣 toggle) and the 🎧 solo button
  (`RequestAudio.movie`) use it. Unbound movies correctly post no audio.
- Standalone **View → 🎙 Voice Lines** browser (filter + ▶ play any `sound.pak`
  clip) via `VoiceLibrary` + `AudioPreview`.

Tests: 46 formats-lib + 11 viewer-lib + disc (`movie_manifest_disc`,
`movie_subtitle_disc`, `slb_disc`) all green. Full `cargo test --workspace` green.

## 2. Also bundled in this commit (prior local WIP, not this session's focus)
- Viewer **async stage/model loading** (off-thread `prepare_xpr`, cancellable) —
  `iso_loader.rs`/`ui.rs`/`camera.rs`. See memory `reborn-viewer-async-loading`.
- **Grouped-pool XBG7 / hero-ship** decode refinements in `mesh.rs`, `cli/main.rs`,
  `mesh_disc.rs`. See memory `reborn-ship-drawlog` / `reborn-saber-texture-investigation`.
- `tools/analyze_drawlog_wvp.py`, `tools/extract_movie_subtitles.py`.

These build and test green but were not re-verified for behaviour this session.

## 3. USER TO VERIFY IN GUI (couldn't be tested from CLI)
1. S13A cutscene shows BOTH lines of the "Look at it father" caption.
2. Story-movie voice (S13A etc.) lines up with the video after the subwave fix.
3. The 5 directly-bound `hokyu_*` movies play voice; other hokyu are silent (see §4).
4. View → 🎙 Voice Lines browser lists and plays clips.

## 4. OPEN PROBLEM — unbound-hokyu resupply voice (do NOT re-guess)
The resupply cutscenes clearly SHARE voice recordings (video-only varies per
mission), but the correct join key for the **13 unbound `hokyu_*`** movies is
**unknown**:
- Manifest DIRECTLY binds only 5: `LS_s02A→450, LS_s09A→451, DS_s02A→452,
  LS_s02H→453, DS_s07H→454` (grouping verified against the raw layout).
- Keying unbound movies by subtitle **demo id** (600→450…604→454, so
  `hokyu_DS_s13A` demo602→`VOICE_D_452`) was implemented, tested against the
  running game by the user, and is **WRONG** (wrong line). It has been **reverted**
  — unbound hokyu now stay unvoiced rather than play wrong audio.
- Red flag: only `VOICE_D_450..454` exist; decoded durations `450=2.8s 451=1.6s
  452=2.2s` but `453=0.14s 454=0.43s` — far too short for the spoken line, so
  these `.slb` are likely **multi-subwave / not cleanly sliced** (same class as
  the deferred B/C banks below).

**Next-box options:** (a) get the real join key from mission-event data (mission
scripts/IDXD tables that trigger resupply), or (b) build a proper multi-subwave
`.slb` decoder and verify audio by ear. Needs a concrete calibration clue from the
user (e.g. "s13A should sound like <movie X>" or the actual spoken words).

## 5. Other deferred items (unchanged)
- ~49 "layout-B/C" movie/voice banks (most RT + S07B/S11A/S12B/S13B) + some
  individual clips need an XMA1 packet/seek-table reassembler.
- CJK/Japanese text rendering in the viewer (needs a CJK TTF via egui FontDefinitions).
- Texture colours (channel order / sRGB) still on the dynamic-RE backlog.

## 6. Resume checklist (next box)
1. `git pull` on `feature/ipfb-idxd-parser`; set `SYLPHEED_DISC=<extract root>`
   (had `.../sylph_extract`).
2. Build guardrail: `CARGO_BUILD_JOBS=4` always (15 GB box; full `-j` OOM'd before).
3. `cargo test --workspace` (with `SYLPHEED_DISC`) should be green.
4. Pick up §4 (unbound-hokyu voice) — the only open thread from this session.