Syplheed-Reborn/docs/HANDOFF-movie-voice-subtitles-2026-07-19.md

Handoff — movie subtitles, voice, and the movie manifest (2026-07-19)

Branch feature/ipfb-idxd-parser. This commit flushes several sessions of local WIP; the new, finished work is the movie subtitle + voice + audio pipeline and the movie manifest. One item is deliberately left open (see §4).

1. What's DONE and verified (static RE + tests)

Movie subtitles (crates/sylpheed-formats/src/movie_subtitle.rs)

• Full movie→track→text chain (see docs/re/structures/movie-subtitles.md).
• Multi-line caption fix: a caption stored as several consecutive text tokens sharing one timing (S13A: "Look at it father" + "& beautiful isn't it" @ 01:14.80) — parse_track now joins them with \n. Previously all but the last line were dropped. Disc test asserts both lines survive.
• Overlap rendering: MovieSubtitles::active_cues(t) returns every cue active at t; the viewer stacks them (was: only the first cue shown).
• Umlauts / Latin-1 accents preserved (utf16le_tokens); Japanese label romanized (CJK font still a known gap — see §5).

Voice decode (crates/sylpheed-formats/src/slb.rs)

• sound.pak entries are XACT .slb banks wrapping XMA1 (fmt tag 0x0165, 48 kHz, mono content). to_xma_riff rebuilds a decodable RIFF; FFmpeg xma1 decodes it. Downmix mono via -af pan=mono|c0=c0.
• Multi-subwave fix: take the FIRST sub-wave bounded by its declared data size — NOT data..EOF (which appended later takes = the S10–S16 garble). Verified: S13A→83.78s == video.
• list_voice_clips enumerates the voice library from sounds.tbl.

Movie manifest (crates/sylpheed-formats/src/movie_manifest.rs) — the index

• tables.pak entry ADVERTISE_MOVIE (IDXD, hash 0x5B983A08) is the authoritative movie → subtitle → voice map. Located by shape (is_manifest), parsed by grouping the string pool on .wmv.
• Authoritative voice binding replaces the old VOICE_<movie> guess: 101 movies, 83 with a VOICE_ token, 18 without. The token is NOT always VOICE_<movie> — 5 hokyu_* movies bind to VOICE_D_450..454 in \etc\, which a guess would miss. Token's subdir varies (Movie/etc/Voice) → resolve via sounds.tbl. Disc test: all 83 resolved entries exist in sound.pak.

Viewer wiring (iso_loader.rs, ui.rs)

• resolve_movie_voice_clip reads the manifest + sounds.tbl (DIRECT bindings only) → handle_voice_request (movie player 🗣 toggle) and the 🎧 solo button (RequestAudio.movie) use it. Unbound movies correctly post no audio.
• Standalone View → 🎙 Voice Lines browser (filter + ▶ play any sound.pak clip) via VoiceLibrary + AudioPreview.

Tests: 46 formats-lib + 11 viewer-lib + disc (movie_manifest_disc, movie_subtitle_disc, slb_disc) all green. Full cargo test --workspace green.

2. Also bundled in this commit (prior local WIP, not this session's focus)

• Viewer async stage/model loading (off-thread prepare_xpr, cancellable) — iso_loader.rs/ui.rs/camera.rs. See memory reborn-viewer-async-loading.
• Grouped-pool XBG7 / hero-ship decode refinements in mesh.rs, cli/main.rs, mesh_disc.rs. See memory reborn-ship-drawlog / reborn-saber-texture-investigation.
• tools/analyze_drawlog_wvp.py, tools/extract_movie_subtitles.py.

These build and test green but were not re-verified for behaviour this session.

3. USER TO VERIFY IN GUI (couldn't be tested from CLI)

1. S13A cutscene shows BOTH lines of the "Look at it father" caption.
2. Story-movie voice (S13A etc.) lines up with the video after the subwave fix.
3. The 5 directly-bound hokyu_* movies play voice; other hokyu are silent (see §4).
4. View → 🎙 Voice Lines browser lists and plays clips.

4. OPEN PROBLEM — unbound-hokyu resupply voice (do NOT re-guess)

The resupply cutscenes clearly SHARE voice recordings (video-only varies per mission), but the correct join key for the 13 unbound hokyu_* movies is unknown:

• Manifest DIRECTLY binds only 5: LS_s02A→450, LS_s09A→451, DS_s02A→452, LS_s02H→453, DS_s07H→454 (grouping verified against the raw layout).
• Keying unbound movies by subtitle demo id (600→450…604→454, so hokyu_DS_s13A demo602→VOICE_D_452) was implemented, tested against the running game by the user, and is WRONG (wrong line). It has been reverted — unbound hokyu now stay unvoiced rather than play wrong audio.
• Red flag: only VOICE_D_450..454 exist; decoded durations 450=2.8s 451=1.6s 452=2.2s but 453=0.14s 454=0.43s — far too short for the spoken line, so these .slb are likely multi-subwave / not cleanly sliced (same class as the deferred B/C banks below).

Next-box options: (a) get the real join key from mission-event data (mission scripts/IDXD tables that trigger resupply), or (b) build a proper multi-subwave .slb decoder and verify audio by ear. Needs a concrete calibration clue from the user (e.g. "s13A should sound like " or the actual spoken words).

5. Other deferred items (unchanged)

• ~49 "layout-B/C" movie/voice banks (most RT + S07B/S11A/S12B/S13B) + some individual clips need an XMA1 packet/seek-table reassembler.
• CJK/Japanese text rendering in the viewer (needs a CJK TTF via egui FontDefinitions).
• Texture colours (channel order / sRGB) still on the dynamic-RE backlog.

6. Resume checklist (next box)

1. git pull on feature/ipfb-idxd-parser; set SYLPHEED_DISC=<extract root> (had .../sylph_extract).
2. Build guardrail: CARGO_BUILD_JOBS=4 always (15 GB box; full -j OOM'd before).
3. cargo test --workspace (with SYLPHEED_DISC) should be green.
4. Pick up §4 (unbound-hokyu voice) — the only open thread from this session.