fabi/Syplheed-Reborn
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Cutscene voice: resolve movies via continuous-stream cue regions

Browse Source 
Cracked how the game maps a movie to its voice track and reimplemented it,
fixing wrong/short voices for RT and hokyu cutscenes.

## Mechanism (RE'd from a Canary file-I/O trace, user-confirmed by ear)

The movie voices are ONE continuous XMA stream, physically chunked into
`<lang>\Movie\VOICE_*.slb` (and `\etc\VOICE_D_*.slb`) sound.pak TOC entries
whose boundaries do NOT align with the cutscene cues. A single cue routinely
spans two `.slb` chunks, so a `.slb` does not necessarily hold the track its
name claims (they are off-by-one vs the cues). Each cue ends at an inline
trailer descriptor `(sound_id:u32be, 0x11, …)` whose id repeats at +0x800.

Resolution chain (all derivable on-disc, no guest-code RE):
  1. movie -> cue token         (ADVERTISE_MOVIE manifest VOICETRACK)
  2. cue token -> sound-id       (master sound registry, tables.pak
                                  a2c8c185=eng / b04238e0=jpn, schema 0x13cb84ba)
  3. sound-id -> [start,end)      scan the stream for trailer(id) and its
                                  predecessor; cue audio = the bytes between,
                                  read continuously across chunk boundaries.

Validated: decoded voice length matches each movie within ~0.4s, including
cross-boundary cues (ADV/RT01A/RT01B/RT01C/S00A/S01A + the 5 bound hokyu).

## Changes

- sylpheed-formats/src/movie_voice.rs (new): registry_voice_ids(),
  find_descriptor(), find_descriptor_before() (gap-tolerant predecessor).
  Unit-tested.
- viewer iso_loader.rs: resolve_movie_voice_region() + decode_voice_region()
  (continuous read via existing read_segment_range + to_xma_riffs). Voice/
  audio requests now region-first; a `\Movie\` token that fails region stays
  SILENT (never plays the wrong off-by-one chunk). Removed the old
  movie_is_demo_based suppress hack.
- Anchor tries subdirs Movie/etc/Voice so hokyu `\etc\VOICE_D_*` resolve too.
- Examples resolve_all/validate_cues/hokyu_final document + validate the pipeline.

## Status

- RT / S / ADV cutscene voices: CORRECT (user-confirmed).
- 5 manifest-bound hokyu (VOICE_D_450-454): CORRECT (user-confirmed).

## OPEN (handoff)

13 of 18 hokyu movies have NO manifest VOICETRACK; the game reuses the 5
recordings across stages via a code rule. `hokyu_fallback_token()` currently
guesses by category (ship LS/DS x source carrier-A/tanker-H: LS/A->450,
LS/H->453, DS/A->452, DS/H->454). USER REPORTS THIS IS SOMETIMES WRONG.
- LS/carrier has two takes (450 from s02A, 451 from s09A); the early-vs-late
  split is unknown — unbound LS/A currently all default to 450.
- Definitive fix = Canary `--phase_a_fileio_only` file.read trace of an unbound
  hokyu (e.g. hokyu_LS_s03A) to see which VOICE_D the game actually streams,
  then encode the real rule. Same method that cracked the RT mapping.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-07-23 06:53:55 +02:00
parent ed5c2f24c3
commit 053aa81953
9 changed files with 577 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files

228
crates/sylpheed-viewer/src/iso_loader.rs
View File

@@ -1752,9 +1752,13 @@ fn poll_loader_channel(
generation,
wav,
}) => {
if generation == mvoice.generation
&& mvoice.movie.as_deref() == Some(movie.as_str())
{
let accept = generation == mvoice.generation
&& mvoice.movie.as_deref() == Some(movie.as_str());
info!(
"[voice] loaded movie={movie:?} gen={generation} (cur movie={:?} gen={}) accept={accept} wav={wav:?}",
mvoice.movie, mvoice.generation
);
if accept {
mvoice.loading = false;
mvoice.available = wav.is_some();
mvoice.pending_wav = wav;
@@ -3140,30 +3144,63 @@ fn decode_sound_clip(
use sylpheed_formats::{hash::name_hash, slb};
let key = name_hash(clip_name);
let bytes = read_sound_entry(source, key)?;
// Decode the first sub-wave (base behaviour). NOTE: multi-sub-wave assembly
// (segments vs alternate takes vs timecode-placed lines) is UNRESOLVED — a
// naive concat produced the wrong track for RT movies. Pending ground-truth
// voice-playback instrumentation (Canary APU/XMA) before a real fix.
let riff = slb::to_xma_riff(&bytes).ok_or("not a decodable .slb")?;
// A `.slb` is an XACT bank of one or more sub-waves. Decode EVERY sub-wave and
// concatenate them, then clamp to the movie length. This is robust to both bank
// shapes we see: segment banks (e.g. VOICE_RT07A = 3 sub-waves 24+14+11s ≈ the
// 50s movie) yield the full dialogue, while the clamp drops the extra ALTERNATE
// takes of full-length banks (e.g. VOICE_S00A, whose sub-wave 0 already spans
// the 94s movie). Playing only sub-wave 0 (the old behaviour) dropped 2/3 of the
// dialogue for segment banks — the "RT voice wrong" bug. Verified vs real durations.
let mut riffs = slb::to_xma_riffs(&bytes);
if riffs.is_empty() {
// Some banks (data-before-header `\etc\` radio clips) confuse the
// multi-sub-wave scanner; the robust single-stream decoder handles them,
// so the standalone browser can still play them.
riffs = slb::to_xma_riff_best(&bytes).into_iter().collect();
}
decode_riffs_to_wav(riffs, &format!("{key:08x}"), duration)
}
/// Decode a set of XMA sub-wave RIFFs into a single mono WAV: concatenate them,
/// downmix to mono (left channel holds the signal for dual-mono sources), and
/// clamp to `duration` seconds. Shared by the per-`.slb` decoder and the
/// continuous movie-voice decoder.
#[cfg(not(target_arch = "wasm32"))]
fn decode_riffs_to_wav(riffs: Vec<Vec<u8>>, tag: &str, duration: f32) -> Result<PathBuf, String> {
if riffs.is_empty() {
return Err("no decodable audio".into());
}
let dir = std::env::temp_dir();
let xma_path = dir.join(format!("sylph_voice_{key:08x}.xma.wav"));
let out_path = dir.join(format!("sylph_voice_{key:08x}.wav"));
std::fs::write(&xma_path, &riff).map_err(|e| e.to_string())?;
let out_path = dir.join(format!("sylph_voice_{tag}.wav"));
let mut inputs = Vec::with_capacity(riffs.len());
for (i, riff) in riffs.iter().enumerate() {
let p = dir.join(format!("sylph_voice_{tag}_{i}.xma.wav"));
std::fs::write(&p, riff).map_err(|e| e.to_string())?;
inputs.push(p);
}
let dur = if duration.is_finite() && duration > 0.0 {
duration
} else {
600.0
};
let status = Command::new("ffmpeg")
.args(["-hide_banner", "-y", "-i"])
.arg(&xma_path)
.args(["-af", "pan=mono|c0=c0", "-t"])
let filter = format!(
"{}concat=n={}:v=0:a=1,pan=mono|c0=c0[a]",
(0..inputs.len()).map(|i| format!("[{i}:a]")).collect::<String>(),
inputs.len(),
);
let mut cmd = Command::new("ffmpeg");
cmd.args(["-hide_banner", "-y"]);
for p in &inputs {
cmd.arg("-i").arg(p);
}
cmd.args(["-filter_complex", &filter, "-map", "[a]", "-t"])
.arg(format!("{dur}"))
.arg(&out_path)
.output();
let _ = std::fs::remove_file(&xma_path);
.arg(&out_path);
let status = cmd.output();
for p in &inputs {
let _ = std::fs::remove_file(p);
}
match status {
Ok(o) if o.status.success() && out_path.metadata().map(|m| m.len() > 44).unwrap_or(false) => {
Ok(out_path)
@@ -3176,6 +3213,115 @@ fn decode_sound_clip(
}
}
/// Categorical fallback voice token for a hokyu (resupply) cutscene the manifest
/// leaves unbound. Only 5 of the 18 hokyu movies carry an explicit `VOICETRACK`;
/// the game reuses those 5 generic recordings across stages, keyed by
/// ship (`LS`/`DS`) × resupply source (carrier `…A` / tanker `…H`). The four
/// category→recording pairs are read off the bound examples; `LS`/carrier has two
/// takes (450 from s02A, 451 from s09A) — we default unbound LS/carrier to 450.
/// Returns `None` for non-hokyu movies (they resolve via the manifest only).
fn hokyu_fallback_token(movie: &str) -> Option<String> {
if !movie.starts_with("hokyu_") {
return None;
}
let ls = movie.contains("_LS_");
let ds = movie.contains("_DS_");
let carrier = movie.ends_with('A');
let tanker = movie.ends_with('H');
let token = match (ls, ds, carrier, tanker) {
(true, _, true, _) => "VOICE_D_450", // light ship, carrier resupply
(true, _, _, true) => "VOICE_D_453", // light ship, tanker resupply
(_, true, true, _) => "VOICE_D_452", // Delta Saber, carrier resupply
(_, true, _, true) => "VOICE_D_454", // Delta Saber, tanker resupply
_ => return None,
};
Some(token.to_string())
}
/// Resolve a movie's cutscene voice to a **continuous byte region** of the
/// `sound.pNN` voice stream, in `[start, end)` global offsets.
///
/// The movie voices are one continuous XMA stream chunked into `VOICE_*.slb` TOC
/// entries whose boundaries do NOT match the cutscene cues (a cue routinely spans
/// two `.slb` chunks — so a `.slb` need not hold the track its name claims). Each
/// cue ends at an inline `(sound_id, 0x11, …)` trailer; cue N = the bytes between
/// trailer N-1 and trailer N. Chain: movie → cue name (manifest VOICETRACK) →
/// sound-id (master registry) → region (scan the stream for the two trailers).
/// Returns `None` for movies whose voice is not a `\Movie\` bank (e.g. hokyu
/// `\etc\` clips), which the caller then resolves the legacy per-clip way.
#[cfg(not(target_arch = "wasm32"))]
fn resolve_movie_voice_region(
source: &SourceKind,
movie: &str,
lang: sylpheed_formats::slb::VoiceLang,
) -> Option<(u64, u64)> {
use sylpheed_formats::{hash::name_hash, movie_manifest, movie_voice, PakArchive};
let code = lang.code_pub();
let tpak = read_pak_archive_blocking(source, "dat/tables.pak").ok()?;
// movie → cue token (e.g. "VOICE_RT01A")
let manifest = tpak
.entries()
.iter()
.find_map(|e| tpak.read(e).ok().filter(|b| movie_manifest::is_manifest(b)))?;
let token = movie_manifest::voice_token(&manifest, movie)
.or_else(|| hokyu_fallback_token(movie))?;
// token → sound-id via the master registry (the large per-language IDXD entry
// carrying the `<lang>\Movie\VOICE_*.slb` paths).
let marker = format!("{code}\\Movie\\VOICE_ADV.slb");
let registry = tpak.entries().iter().find_map(|e| {
tpak.read(e)
.ok()
.filter(|b| b.windows(marker.len()).any(|w| w == marker.as_bytes()))
})?;
let id = *movie_voice::registry_voice_ids(&registry).get(&token)?;
// physical anchor: the TOC offset of this token's own `.slb` chunk — a start
// point near the cue's trailers (the cue itself may sit before/after it). The
// token's subdir varies: `Movie` (ADV/RT/S cutscenes) or `etc` (hokyu VOICE_D).
let stoc = read_source_file(source, "dat/sound.pak").ok()?;
let entries = PakArchive::parse_toc(&stoc).ok()?;
let anchor = ["Movie", "etc", "Voice"].iter().find_map(|dir| {
let h = name_hash(&format!("{code}\\{dir}\\{token}.slb"));
entries
.binary_search_by_key(&h, |e| e.name_hash)
.ok()
.map(|i| entries[i].offset as u64)
})?;
// Scan a window spanning both directions from the anchor for this cue's trailer
// and its predecessor. The window must cover the largest bank (ADV ≈ 3.6 MB).
let win_start = anchor.saturating_sub(2 * 1024 * 1024) & !3;
let window = read_segment_range(source, "dat/sound", win_start, 8 * 1024 * 1024).ok()?;
let end_local = movie_voice::find_descriptor(&window, id)?;
let end = win_start + end_local as u64;
// Cue start = its predecessor trailer. Prefer the exact `id-1` trailer; if the
// id sequence has a gap (e.g. VOICE_D_453→454), fall back to the nearest trailer
// below this one — but only if it's within one bank (~1.5 MB), else this is the
// first cue in its block and the audio starts at the bank anchor itself.
let start = movie_voice::find_descriptor(&window, id.wrapping_sub(1))
.or_else(|| movie_voice::find_descriptor_before(&window, end_local))
.map(|o| win_start + o as u64)
.filter(|&s| s < end && end - s < 1_500_000)
.unwrap_or(anchor);
Some((start, end))
}
/// Decode a continuous movie-voice byte region (from [`resolve_movie_voice_region`])
/// into a mono WAV, clamped to `duration`.
#[cfg(not(target_arch = "wasm32"))]
fn decode_voice_region(
source: &SourceKind,
start: u64,
end: u64,
duration: f32,
) -> Result<PathBuf, String> {
use sylpheed_formats::slb;
let bytes = read_segment_range(source, "dat/sound", start, (end - start) as usize)?;
let mut riffs = slb::to_xma_riffs(&bytes);
if riffs.is_empty() {
riffs = slb::to_xma_riff_best(&bytes).into_iter().collect();
}
decode_riffs_to_wav(riffs, &format!("mv_{start:x}"), duration)
}
/// Resolve a movie's `sound.pak` voice-entry name via the **movie manifest**
/// (`tables.pak`), which authoritatively binds each movie to its voice bank —
/// several `hokyu_*` resupply movies point at in-mission `VOICE_D_*` clips in
@@ -3221,8 +3367,24 @@ fn handle_voice_request(
let (movie, lang, duration, generation) =
(req.movie.clone(), req.lang, req.duration, req.generation);
std::thread::spawn(move || {
let wav = resolve_movie_voice_clip(&source, &movie, lang)
.and_then(|clip| decode_sound_clip(&source, &clip, duration).ok());
// The cutscene voices live in one continuous XMA stream, addressed by cue
// byte-region (movie → sound-id → [trailer(N-1)..trailer(N)]); this gives
// the CORRECT, complete voice for ADV / S / RT movies, including the many
// cues that span two `.slb` chunks. Movies whose voice is not a `\Movie\`
// bank (hokyu resupply → `\etc\` demo clips) resolve to no region and fall
// through to the legacy per-clip decoder. A `\Movie\` token that failed
// region resolution must NOT play its raw `.slb` — that off-by-one chunk is
// the wrong track — so it stays silent rather than wrong.
let wav = resolve_movie_voice_region(&source, &movie, lang)
.and_then(|(s, e)| decode_voice_region(&source, s, e, duration).ok())
.or_else(|| {
let clip = resolve_movie_voice_clip(&source, &movie, lang)?;
if clip.contains("\\Movie\\") {
return None;
}
decode_sound_clip(&source, &clip, duration).ok()
});
info!("[voice] movie={movie:?} gen={generation} -> wav={wav:?}");
let _ = sender.send(IsoLoaderMsg::VoiceLoaded {
movie,
generation,
@@ -3265,15 +3427,22 @@ fn handle_audio_request(
req.generation,
);
std::thread::spawn(move || {
// Resolve the movie voice bank from the manifest when asked, else decode
// the named clip directly.
let entry = match movie {
Some((m, lang)) => resolve_movie_voice_clip(&source, &m, lang),
None => Some(clip),
};
let wav = entry
.and_then(|c| decode_sound_clip(&source, &c, f32::INFINITY).ok())
.and_then(|p| wav_duration(&p).map(|d| (p, d)));
// For a movie, decode its full-length cutscene voice via the continuous
// stream (same resolution as playback), falling back to a non-`\Movie\`
// clip (hokyu `\etc\`); for a named library clip, decode it directly.
let wav = match movie {
Some((m, lang)) => resolve_movie_voice_region(&source, &m, lang)
.and_then(|(s, e)| decode_voice_region(&source, s, e, f32::INFINITY).ok())
.or_else(|| {
let c = resolve_movie_voice_clip(&source, &m, lang)?;
if c.contains("\\Movie\\") {
return None;
}
decode_sound_clip(&source, &c, f32::INFINITY).ok()
}),
None => decode_sound_clip(&source, &clip, f32::INFINITY).ok(),
}
.and_then(|p| wav_duration(&p).map(|d| (p, d)));
let _ = sender.send(IsoLoaderMsg::AudioLoaded {
name: display,
generation,
@@ -3414,6 +3583,7 @@ fn advance_video_playback(
if let Some(wav) = voice.pending_wav.take() {
if let Some(handle) = &inner.stream_handle {
let vol = if voice.enabled { video.volume } else { 0.0 };
info!("[voice] APPLY sink movie={:?} wav={wav:?} pos={}", voice.movie, video.position);
inner.voice_sink =
build_audio_sink(handle, &wav, video.position, vol, video.playing && !video.scrubbing);
inner.voice_applied_volume = vol;

86
crates/sylpheed-viewer/src/ui.rs
View File

@@ -227,37 +227,73 @@ fn draw_viewer_ui(
ui.label("Filter:");
ui.text_edit_singleline(&mut voice_lib.filter);
});
let f = voice_lib.filter.to_lowercase();
// Group the thousands of entries as directory → speaker so the
// list is navigable (e.g. browse `Voice` by character to find a
// cutscene's radio line). `name` is `<lang>\<dir>\<file>.slb`.
use std::collections::BTreeMap;
let mut groups: BTreeMap<&str, BTreeMap<&str, Vec<&sylpheed_formats::slb::VoiceClip>>> =
BTreeMap::new();
for c in &voice_lib.clips {
if !f.is_empty() && !c.name.to_lowercase().contains(&f) {
continue;
}
let dir = c.name.rsplit('\\').nth(1).unwrap_or("?");
groups
.entry(dir)
.or_default()
.entry(c.speaker.as_str())
.or_default()
.push(c);
}
let shown: usize = groups.values().flat_map(|s| s.values()).map(Vec::len).sum();
ui.label(
egui::RichText::new(format!("{} clips", voice_lib.clips.len()))
egui::RichText::new(format!("{shown} / {} clips", voice_lib.clips.len()))
.weak()
.small(),
);
ui.separator();
let f = voice_lib.filter.to_lowercase();
let clips: Vec<_> = voice_lib
.clips
.iter()
.filter(|c| f.is_empty() || c.name.to_lowercase().contains(&f))
.take(2000)
.cloned()
.collect();
let filtering = !f.is_empty();
egui::ScrollArea::vertical().show(ui, |ui| {
for c in &clips {
ui.horizontal(|ui| {
if ui.button("▶").on_hover_text(&c.name).clicked() {
audio.generation = audio.generation.wrapping_add(1);
audio.loading = true;
audio.active = true;
audio.name = c.display.clone();
events.audio.send(RequestAudio {
clip: c.name.clone(),
display: c.display.clone(),
movie: None,
generation: audio.generation,
});
}
ui.label(&c.display);
});
for (dir, speakers) in &groups {
let dtotal: usize = speakers.values().map(Vec::len).sum();
egui::CollapsingHeader::new(format!("📁 {dir} ({dtotal})"))
.id_salt(("vdir", *dir))
.default_open(filtering)
.show(ui, |ui| {
for (speaker, clips) in speakers {
egui::CollapsingHeader::new(format!(
"{speaker} ({})",
clips.len()
))
.id_salt(("vspk", *dir, *speaker))
.default_open(filtering || clips.len() <= 6)
.show(ui, |ui| {
for c in clips {
ui.horizontal(|ui| {
if ui
.button("▶")
.on_hover_text(&c.name)
.clicked()
{
audio.generation =
audio.generation.wrapping_add(1);
audio.loading = true;
audio.active = true;
audio.name = c.display.clone();
events.audio.send(RequestAudio {
clip: c.name.clone(),
display: c.display.clone(),
movie: None,
generation: audio.generation,
});
}
ui.label(&c.display);
});
}
});
}
});
}
});
}