I argued `highest_rate` had no case because ADV's higher-rate presentation is dual-mono while its louder one is mono-in-stereo, so the extra bytes buy a duplicated channel rather than fidelity. The Decoder tested that disc-wide over the 28 three-stream cues: the stream-3/stream-2 size ratio runs min 0.0778, median 1.2565, max 2.9163, sd 0.5057, with only 12 of 28 within 15% of 1.0, and declared rates scatter with them. A 37x spread is not a duplicated channel. The CHANNEL MEASUREMENT STANDS -- ADV chunk 1 is mono-in-stereo and chunk 2 is dual-mono at -8.318574, this port's own decode, which the Decoder could not re-run and did not dispute. What fails is the step from one asset to the format. NOTHING IN THE EXPORT CHANGES. `loudest` is a per-asset content rule -- it reads the peak of the streams in front of it -- so a scattering structural ratio cannot undermine it. What changes is the REASON, in four places: authored/audio.json's presentation_why, the selector comment in audio.rs, BLOCKED.md's row, and DECISIONS.md. The honest statement is narrower: `highest_rate` was never refuted, it was never argued for, and neither is `loudest`. That is why the entry is marked CHOSEN rather than measured, and why one capture deletes it. Recorded on the pattern rather than just the instance: this is the third claim of mine in two iterations that generalised a single-asset observation, after "the chunks are two stems" and "everything the sequencer paces off rest.t is late". All three were true of the thing I looked at. The failure is reaching for the rule a measurement would imply if it held everywhere and writing that down in the same breath as the measurement. Also noted, not mine and not affecting export_voice: S12B's three streams are byte-size identical, and BIRD_224 is three-stream while not being a movie cue. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N7FiFFFwbvG2uxdcEh8HyF
899 lines
39 KiB
Rust
899 lines
39 KiB
Rust
//! Menu audio: disc XMA → Ogg Vorbis, because Godot 4 plays Vorbis natively and
|
||
//! will never be taught to read XMA or to open `sound.pak`.
|
||
//!
|
||
//! Two kinds of thing come out of here and they are not symmetric:
|
||
//!
|
||
//! * **SE cues** — three short mono waves out of `Static.slb`, one per menu
|
||
//! event. Where each one lives was *measured off the running game* (HANDOFF
|
||
//! Q8) and is **not on the disc in any findable form**.
|
||
//! * **BGM** — one bank of `sound.pak`, which is **two stems of one
|
||
//! performance** (HANDOFF Q10). They are summed here into a single file.
|
||
//!
|
||
//! ## Neither table lives in this file
|
||
//!
|
||
//! Both come from `authored/audio.json`, and that is the point of the module
|
||
//! rather than an accident of configuration. MISSION §3: a value that is
|
||
//! *measured* rather than *decoded* lives in `authored/`, carries a `why`, and
|
||
//! is deleted the day the disc states it. A measured offset compiled into a Rust
|
||
//! `const` is a measurement wearing the costume of a decoded field — it reads
|
||
//! like the exporter derived it, and nobody deletes it, because nobody can see
|
||
//! it. Contrast [`crate::video::MOVIES`], which *is* a `const` here: Q9 decoded
|
||
//! that mapping from the movie manifest on the disc.
|
||
//!
|
||
//! ## The assembly is not reimplemented here
|
||
//!
|
||
//! `sylpheed_formats::media` owns every question of the form *"which bytes
|
||
//! belong together"* — segment-spanning reads, multi-sub-wave banks, and the
|
||
//! delimiter-less `Static.slb` where a wave is only `(offset, packet_count)`.
|
||
//! This module asks it for `RIFF`s and converts them. The seam is deliberate:
|
||
//! everything before it is disc knowledge, everything after it is a codec
|
||
//! choice, and re-deriving the first half here is exactly the mistake the
|
||
//! mission names.
|
||
|
||
use anyhow::{bail, Context, Result};
|
||
use serde::Deserialize;
|
||
use std::collections::BTreeMap;
|
||
use std::path::{Path, PathBuf};
|
||
use std::process::Command;
|
||
use sylpheed_formats::media::{self, DiscSource};
|
||
|
||
/// One menu sound effect, and where its wave sits in a delimiter-less bank.
|
||
///
|
||
/// `offset` is a string so the file can carry `"0x1ec0"` — the form the RE
|
||
/// finding is written in. A reader comparing the two should not have to convert
|
||
/// 125 632 in their head to believe they match.
|
||
#[derive(Deserialize)]
|
||
pub struct CueSpec {
|
||
pub bank: String,
|
||
pub offset: String,
|
||
pub packets: usize,
|
||
pub channels: u8,
|
||
pub rate: u32,
|
||
/// The game's own cue identifier, where one has been *guessed by name*.
|
||
/// Absent means nobody claimed one — never that the binding is unknown.
|
||
#[serde(default)]
|
||
pub name_match: Option<String>,
|
||
pub why: String,
|
||
}
|
||
|
||
impl CueSpec {
|
||
fn offset(&self, event: &str) -> Result<usize> {
|
||
let s = self.offset.trim();
|
||
let parsed = match s.strip_prefix("0x").or_else(|| s.strip_prefix("0X")) {
|
||
Some(hex) => usize::from_str_radix(&hex.replace('_', ""), 16),
|
||
None => s.replace('_', "").parse(),
|
||
};
|
||
parsed.with_context(|| format!("cue `{event}`: `{s}` is not an offset"))
|
||
}
|
||
}
|
||
|
||
/// One music bank and what to do with it.
|
||
#[derive(Deserialize)]
|
||
pub struct BgmSpec {
|
||
pub bank: String,
|
||
/// What happens at the end of the file. Carried through to the manifest so
|
||
/// the runtime does not have to reach into `authored/` to find out, and so
|
||
/// that the *why* travels with the decision.
|
||
#[serde(default)]
|
||
pub r#loop: Option<String>,
|
||
pub why: String,
|
||
#[serde(default)]
|
||
pub loop_why: Option<String>,
|
||
#[serde(default)]
|
||
pub stems_why: Option<String>,
|
||
}
|
||
|
||
/// Which of a voice region's full-length presentations to export.
|
||
///
|
||
/// A region carries three presentations of one take and **nothing on the disc
|
||
/// ranks them** — `wEncodeOptions`, channel count and channel mask are
|
||
/// byte-identical across them. So this is a CHOICE, it lives in
|
||
/// `authored/audio.json` with its `why`, and it is deleted the day a capture
|
||
/// says which one the game plays.
|
||
#[derive(Deserialize, Default, Clone, Copy, PartialEq)]
|
||
#[serde(rename_all = "snake_case")]
|
||
pub enum Presentation {
|
||
/// Peak nearest full scale.
|
||
#[default]
|
||
Loudest,
|
||
/// Most bytes per second.
|
||
HighestRate,
|
||
}
|
||
|
||
/// `authored/audio.json`, with the documentation keys dropped.
|
||
pub struct Config {
|
||
pub se: Vec<(String, CueSpec)>,
|
||
pub bgm: Vec<(String, BgmSpec)>,
|
||
pub voice: Presentation,
|
||
}
|
||
|
||
/// Read `authored/audio.json`, or `None` when there is no such file.
|
||
///
|
||
/// Absent is not an error: an export with no audio is what every milestone
|
||
/// before P6 produced, and it should stay possible to take one.
|
||
pub fn load(authored: &Path) -> Result<Option<Config>> {
|
||
let path = authored.join("audio.json");
|
||
if !path.exists() {
|
||
return Ok(None);
|
||
}
|
||
#[derive(Deserialize)]
|
||
struct File {
|
||
#[serde(default)]
|
||
se: BTreeMap<String, serde_json::Value>,
|
||
#[serde(default)]
|
||
bgm: BTreeMap<String, serde_json::Value>,
|
||
#[serde(default)]
|
||
voice: BTreeMap<String, serde_json::Value>,
|
||
}
|
||
let raw = std::fs::read_to_string(&path)
|
||
.with_context(|| format!("read {}", path.display()))?;
|
||
let file: File = serde_json::from_str(&raw)
|
||
.with_context(|| format!("parse {}", path.display()))?;
|
||
|
||
// `_` is the house convention for a prose block explaining the section it
|
||
// sits in -- see `authored/flow.json` and `authored/timing.json`. It is
|
||
// documentation, not an entry, and the schema must skip it rather than
|
||
// force the reasoning out of the file that holds the decision.
|
||
fn entries<T: serde::de::DeserializeOwned>(
|
||
m: BTreeMap<String, serde_json::Value>,
|
||
what: &str,
|
||
) -> Result<Vec<(String, T)>> {
|
||
m.into_iter()
|
||
.filter(|(k, _)| k != "_")
|
||
.map(|(k, v)| {
|
||
let parsed = serde_json::from_value(v)
|
||
.with_context(|| format!("authored/audio.json: {what}.{k}"))?;
|
||
Ok((k, parsed))
|
||
})
|
||
.collect()
|
||
}
|
||
|
||
// Absent means `loudest`, which is what the file says today. A default here
|
||
// is safe in a way a default matrix is not: the manifest records which
|
||
// presentation was taken and why, on every entry.
|
||
let voice = match file.voice.get("presentation") {
|
||
Some(v) => serde_json::from_value(v.clone())
|
||
.with_context(|| format!("authored/audio.json: voice.presentation {v}"))?,
|
||
None => Presentation::default(),
|
||
};
|
||
Ok(Some(Config {
|
||
se: entries(file.se, "se")?,
|
||
bgm: entries(file.bgm, "bgm")?,
|
||
voice,
|
||
}))
|
||
}
|
||
|
||
/// Vorbis quality. `-q:a 5` is ffmpeg's usual transparent-ish setting and is
|
||
/// what [`crate::video`] already uses for the movies' audio; using one value
|
||
/// across the export means a level difference between a cue and a movie cannot
|
||
/// be a codec artefact.
|
||
const VORBIS_Q: &str = "5";
|
||
|
||
pub struct Exported {
|
||
pub name: String,
|
||
pub file: String,
|
||
pub command: String,
|
||
pub why: String,
|
||
/// dBFS peak of the decoded result. Recorded because the BGM sum can clip
|
||
/// and a clipped file is not visibly different from a correct one.
|
||
pub peak_dbfs: Option<f32>,
|
||
/// Seconds, as ffprobe reads them back off the finished file. Recorded
|
||
/// because the cue durations are the one thing about the SE export that an
|
||
/// outside finding predicts, so they are the one thing that can be checked.
|
||
pub duration_s: Option<f32>,
|
||
pub kind: &'static str,
|
||
/// The game's own identifier where it is a name match, never a measurement.
|
||
pub name_match: Option<String>,
|
||
/// What the runtime should do at the end of the file, where that was
|
||
/// authored. `None` on a cue: a cue ends.
|
||
pub loop_mode: Option<String>,
|
||
/// How many sub-waves `media` returned for a bank. Carried out of here so
|
||
/// the caller can warn when it contradicts HANDOFF's census -- this module
|
||
/// does not get to decide that one of them is not a stem.
|
||
pub sub_waves: usize,
|
||
}
|
||
|
||
/// Run ffmpeg, writing to a temp name and renaming on success.
|
||
///
|
||
/// The rename is not tidiness: the filesystem is shared with another agent, and
|
||
/// a reader that catches a half-written Ogg gets a confident wrong duration
|
||
/// rather than an error. `docs/port/AUDIO-VERIFICATION.md` records that this
|
||
/// already happened once on a video.
|
||
fn run_ffmpeg(argv: &[String], out: &Path) -> Result<()> {
|
||
// The extension goes LAST, not the `.partial`. ffmpeg picks its muxer from
|
||
// the output filename, so `.back.ogg.partial` is not a slightly uglier
|
||
// temp name -- it is a hard failure before a byte is written: "Unable to
|
||
// choose an output format". `video.rs` already had this shape; this
|
||
// function was written from scratch and did not.
|
||
let stem = out.file_stem().unwrap_or_default().to_string_lossy().into_owned();
|
||
let ext = out.extension().unwrap_or_default().to_string_lossy().into_owned();
|
||
let partial = out.with_file_name(format!(".{stem}.partial.{ext}"));
|
||
let mut argv = argv.to_vec();
|
||
let last = argv.len() - 1;
|
||
argv[last] = partial.display().to_string();
|
||
let res = Command::new("ffmpeg")
|
||
.args(&argv)
|
||
.output()
|
||
.context("run ffmpeg -- is it on PATH?")?;
|
||
if !res.status.success() {
|
||
let _ = std::fs::remove_file(&partial);
|
||
bail!(
|
||
"ffmpeg failed writing {}:\n{}",
|
||
out.display(),
|
||
String::from_utf8_lossy(&res.stderr)
|
||
);
|
||
}
|
||
std::fs::rename(&partial, out)?;
|
||
Ok(())
|
||
}
|
||
|
||
/// Peak level and duration of a finished file.
|
||
///
|
||
/// Measured rather than assumed because the BGM is a **sum of two stems** and a
|
||
/// sum can clip, and because silence is the audio failure that looks like
|
||
/// success: a file of exactly the right duration, full of zeroes. Both numbers
|
||
/// pass every check that is not looking for them, so the export looks for them.
|
||
fn measure(path: &Path) -> (Option<f32>, Option<f32>) {
|
||
let out = Command::new("ffmpeg")
|
||
.args(["-hide_banner", "-v", "info", "-i"])
|
||
.arg(path)
|
||
.args(["-af", "astats=measure_perchannel=none", "-f", "null", "-"])
|
||
.output();
|
||
let Ok(out) = out else { return (None, None) };
|
||
let text = String::from_utf8_lossy(&out.stderr).into_owned();
|
||
// `astats` writes through the filter log, so every line carries a
|
||
// `[Parsed_astats_0 @ 0x…] ` prefix. Matching on the line START silently
|
||
// finds nothing and reports "peak unmeasured", which is the failure this
|
||
// measurement exists to catch -- so it is found as a SUBSTRING.
|
||
const KEY: &str = "Peak level dB:";
|
||
let peak = text
|
||
.lines()
|
||
.find_map(|l| l.split_once(KEY)?.1.trim().parse().ok());
|
||
let dur = Command::new("ffprobe")
|
||
.args([
|
||
"-v", "error", "-show_entries", "format=duration",
|
||
"-of", "csv=p=0",
|
||
])
|
||
.arg(path)
|
||
.output()
|
||
.ok()
|
||
.and_then(|o| String::from_utf8_lossy(&o.stdout).trim().parse().ok());
|
||
(peak, dur)
|
||
}
|
||
|
||
/// Write one XMA `RIFF` beside the output so ffmpeg has something to open.
|
||
///
|
||
/// Kept next to the result rather than in `/tmp` so a failed export leaves the
|
||
/// intermediate where the person debugging it will look, and removed on success
|
||
/// so the tree holds only formats a modder can open (MODDING rule 3).
|
||
fn stage_riff(dir: &Path, stem: &str, riff: &[u8]) -> Result<PathBuf> {
|
||
let path = dir.join(format!(".{stem}.xma.wav"));
|
||
std::fs::write(&path, riff).with_context(|| format!("write {}", path.display()))?;
|
||
Ok(path)
|
||
}
|
||
|
||
/// The menu cues, as `audio/se/<event>.ogg`.
|
||
pub fn export_cues<S: DiscSource + ?Sized>(
|
||
source: &S,
|
||
out: &Path,
|
||
cues: &[(String, CueSpec)],
|
||
) -> Result<Vec<Exported>> {
|
||
if cues.is_empty() {
|
||
return Ok(Vec::new());
|
||
}
|
||
let dir = out.join("audio/se");
|
||
std::fs::create_dir_all(&dir)?;
|
||
let mut done = Vec::new();
|
||
for (event, cue) in cues {
|
||
let offset = cue.offset(event)?;
|
||
// `media` owns the assembly. Asking it for the RIFF rather than reading
|
||
// `Static.slb` here is the whole point of the seam -- and it REFUSES a
|
||
// short read rather than returning a truncated stream, because a
|
||
// truncated XMA decodes to plausible-sounding garbage.
|
||
let riff = media::se_wave_riff(
|
||
source, &cue.bank, offset, cue.packets, cue.channels, cue.rate,
|
||
)
|
||
.map_err(anyhow::Error::msg)
|
||
.with_context(|| format!("assemble the {event} cue"))?;
|
||
|
||
let staged = stage_riff(&dir, event, &riff)?;
|
||
let ogg = dir.join(format!("{event}.ogg"));
|
||
let argv: Vec<String> = [
|
||
"-hide_banner", "-loglevel", "error", "-y",
|
||
"-i", &staged.display().to_string(),
|
||
"-c:a", "libvorbis", "-q:a", VORBIS_Q,
|
||
&ogg.display().to_string(),
|
||
]
|
||
.iter()
|
||
.map(|s| s.to_string())
|
||
.collect();
|
||
let command = format!("ffmpeg {}", argv.join(" "));
|
||
run_ffmpeg(&argv, &ogg)?;
|
||
let (peak, dur) = measure(&ogg);
|
||
std::fs::remove_file(&staged).ok();
|
||
|
||
done.push(Exported {
|
||
name: event.clone(),
|
||
file: format!("audio/se/{event}.ogg"),
|
||
command,
|
||
why: format!(
|
||
"{} AUTHORED because it is measured, not decoded: authored/audio.json \
|
||
se.{event}. Located in {} at {:#x} for {} packet(s); the ASSEMBLY is \
|
||
sylpheed_formats::media::se_wave_riff, which refuses a short read.",
|
||
cue.why, cue.bank, offset, cue.packets
|
||
),
|
||
peak_dbfs: peak,
|
||
duration_s: dur,
|
||
kind: "se",
|
||
name_match: cue.name_match.clone(),
|
||
loop_mode: None,
|
||
sub_waves: 1,
|
||
});
|
||
}
|
||
Ok(done)
|
||
}
|
||
|
||
/// One music bank, as a single `audio/bgm/<name>.ogg`.
|
||
///
|
||
/// **The two stems are summed, not concatenated and not split into two files.**
|
||
///
|
||
/// HANDOFF Q10: a bank's sub-waves are two stems of one performance, played
|
||
/// together — sample-synchronous, equal duration, on all 32 banks.
|
||
/// Concatenating them is explicitly wrong. Emitting two files would be wrong
|
||
/// here for a different reason: MODDING rule 1 is *one logical asset, one
|
||
/// file*, and a modder who had to line two stems up by hand would be
|
||
/// reassembling exactly what the exporter exists to resolve.
|
||
///
|
||
/// **The sum is scaled by 1/n, and an earlier version of this comment argued the
|
||
/// opposite.** It said `normalize=0` sums at unity "because halving is a mix
|
||
/// decision nobody made". That was wrong twice over. Unity summing IS a decision
|
||
/// — and it is the one that can clip, which it duly did: `BGM_103` came out at
|
||
/// **+1.8 dBFS**. And 1/n is not a taste call but the smallest constant that
|
||
/// makes an n-input sum of unity-scale signals provably clip-free, which is the
|
||
/// same reasoning `video.rs` already uses for its 0.4142-normalised 5.1 downmix.
|
||
/// It is written out as an explicit `volume=` rather than left to `amix`'s
|
||
/// `normalize=1` default so the coefficient appears in the manifest's command
|
||
/// line: a default is a decision nobody made, and it can move under an ffmpeg
|
||
/// upgrade.
|
||
///
|
||
/// It preserves the stems' relative balance exactly, which is the only thing
|
||
/// about the sum that HANDOFF Q10 actually settles. The peak is still measured
|
||
/// and reported.
|
||
///
|
||
/// The file is named for the **role** (`main_menu`), not for the bank
|
||
/// (`BGM_001`). Which bank plays here is authored and expected to change; the
|
||
/// role is what the runtime asks for, and a rename of the disc asset should not
|
||
/// be a change to the Godot project.
|
||
pub fn export_bgm<S: DiscSource + ?Sized>(
|
||
source: &S,
|
||
out: &Path,
|
||
role: &str,
|
||
spec: &BgmSpec,
|
||
) -> Result<Option<Exported>> {
|
||
let riffs = match media::sound_bank_riffs(source, &spec.bank) {
|
||
Ok(r) if !r.is_empty() => r,
|
||
Ok(_) => return Ok(None),
|
||
// "This disc does not have that bank" is a MISSING ASSET, not a broken
|
||
// exporter: the manifest carries a warning and everything else still
|
||
// exports. Any other failure -- a short read, a malformed bank -- is a
|
||
// real error and stops the run, because a partly-read bank produces a
|
||
// file that plays.
|
||
Err(e) if e.contains("not present in sound.pak") => return Ok(None),
|
||
Err(e) => bail!("{}: {e}", spec.bank),
|
||
};
|
||
let dir = out.join("audio/bgm");
|
||
std::fs::create_dir_all(&dir)?;
|
||
|
||
let mut staged = Vec::new();
|
||
for (i, riff) in riffs.iter().enumerate() {
|
||
staged.push(stage_riff(&dir, &format!("{role}.{i}"), riff)?);
|
||
}
|
||
|
||
let ogg = dir.join(format!("{role}.ogg"));
|
||
let mut argv: Vec<String> = ["-hide_banner", "-loglevel", "error", "-y"]
|
||
.iter()
|
||
.map(|s| s.to_string())
|
||
.collect();
|
||
for s in &staged {
|
||
argv.push("-i".into());
|
||
argv.push(s.display().to_string());
|
||
}
|
||
if staged.len() > 1 {
|
||
argv.push("-filter_complex".into());
|
||
argv.push(format!(
|
||
"amix=inputs={n}:normalize=0,volume={:.6}",
|
||
1.0 / staged.len() as f64,
|
||
n = staged.len()
|
||
));
|
||
}
|
||
argv.extend(
|
||
["-c:a", "libvorbis", "-q:a", VORBIS_Q, &ogg.display().to_string()]
|
||
.iter()
|
||
.map(|s| s.to_string()),
|
||
);
|
||
let command = format!("ffmpeg {}", argv.join(" "));
|
||
run_ffmpeg(&argv, &ogg)?;
|
||
let (peak, dur) = measure(&ogg);
|
||
for s in &staged {
|
||
std::fs::remove_file(s).ok();
|
||
}
|
||
|
||
let mut why = format!(
|
||
"{} authored/audio.json bgm.{role} names bank {}. Its {} \
|
||
sub-wave(s) are SUMMED into one file and the sum is scaled by 1/{}, \
|
||
which is the smallest constant that cannot clip.",
|
||
spec.why,
|
||
spec.bank,
|
||
riffs.len(),
|
||
riffs.len()
|
||
);
|
||
if let Some(s) = &spec.stems_why {
|
||
why.push(' ');
|
||
why.push_str(s);
|
||
}
|
||
if let Some(s) = &spec.loop_why {
|
||
why.push(' ');
|
||
why.push_str(s);
|
||
}
|
||
|
||
Ok(Some(Exported {
|
||
name: role.to_string(),
|
||
file: format!("audio/bgm/{role}.ogg"),
|
||
command,
|
||
why,
|
||
peak_dbfs: peak,
|
||
duration_s: dur,
|
||
kind: "bgm",
|
||
name_match: None,
|
||
loop_mode: spec.r#loop.clone(),
|
||
sub_waves: riffs.len(),
|
||
}))
|
||
}
|
||
|
||
/// One cutscene's voice-over, as `audio/voice/<movie>.ogg`.
|
||
///
|
||
/// ## Why this is a separate file from the movie at all
|
||
///
|
||
/// A human play-test heard music under the intro and no dialogue, and the
|
||
/// obvious reading — "the transcode dropped a channel" — is wrong. `ADV.wmv`
|
||
/// genuinely carries **music and effects only**. On this disc a cutscene's voice
|
||
/// is a *different asset*: one continuous XMA stream in `sound.pak`, bound to
|
||
/// the movie by the movie manifest in `tables.pak` (`ADV` → `VOICE_ADV`). It was
|
||
/// not dropped by [`crate::video`]; it was never exported, because nothing here
|
||
/// asked for it.
|
||
///
|
||
/// ## The binding is resolved, never matched by name
|
||
///
|
||
/// `sylpheed_formats::media::resolve_movie_voice_region` walks
|
||
/// movie → cue token → sound id → byte region. It is the only route taken here,
|
||
/// and the reason is that the cheap route looks correct on exactly the movies a
|
||
/// person would check first. Measured on the retail disc:
|
||
///
|
||
/// | movie | region | inside the bank named after it? |
|
||
/// |---|---|---|
|
||
/// | `ADV` | 433 930 240…437 044 592 | yes |
|
||
/// | `S00A` | 452 798 464…455 499 120 | yes |
|
||
/// | `RT01A` | 437 044 592…437 345 648 | **no — it is inside `VOICE_ADV.slb`** |
|
||
///
|
||
/// So `VOICE_<movie>.slb` is a name that happens to hold the right audio twice
|
||
/// out of three, and the two it gets right are the two in this port's scope.
|
||
/// Reading the bank by name would have shipped, verified clean, and been wrong
|
||
/// for the radio cutscenes the moment anybody extended the export.
|
||
///
|
||
/// ## What a `None` means, and what it must not become
|
||
///
|
||
/// A movie whose region does not resolve is **genuinely unvoiced** — that is a
|
||
/// real answer for most `hokyu_*` resupply cutscenes, and the corpus already
|
||
/// paid for the alternative: resolving unbound movies through a shared demo line
|
||
/// played the *wrong recording*. Nothing is substituted. A `\Movie\` token that
|
||
/// resolved a clip but not a region stays silent for the same reason
|
||
/// `sylpheed-viewer` keeps it silent: its raw `.slb` is off by one chunk, so it
|
||
/// is not this movie's dialogue.
|
||
///
|
||
/// ## Three choices made here, and the reason each is not a guess
|
||
///
|
||
/// * **One file per movie** (MODDING rule 1), and **exactly one region chunk is
|
||
/// kept**. Not concatenated, not summed. This function got that wrong twice
|
||
/// before it got it right, and the history is kept below because each wrong
|
||
/// reading was ended by a measurement, not by an argument.
|
||
///
|
||
/// ## A region holds THREE PRESENTATIONS OF ONE TAKE — decoded, and not by me
|
||
///
|
||
/// A resolved region decodes to several chunks. Decoded and timed against the
|
||
/// movies' own lengths:
|
||
///
|
||
/// | movie | movie | chunk 0 | chunk 1 | chunk 2 |
|
||
/// |---|---|---|---|---|
|
||
/// | `ADV` | 137.437 s | 84.553 | **137.324** | **137.324** |
|
||
/// | `S00A` | 93.779 s | 68.072 | **93.694** | **93.694** |
|
||
/// | `RT01A` | — | 0.009 | **34.034** | — |
|
||
///
|
||
/// **Reading 1, concatenate:** 359 s of dialogue for a 137 s movie. Dead.
|
||
///
|
||
/// **Reading 2, sum them as HANDOFF Q10's two stems** — equal duration, each
|
||
/// spanning the movie, which is exactly Q10's *music* shape. ❌ **Refuted here,
|
||
/// and the claim had already been adopted into the RE corpus before I tested
|
||
/// it**: `S00A`'s second full-length chunk is **digital silence** (peak −inf)
|
||
/// and `ADV`'s is **0.60 × the first** with 26.8 dB of residual. Equal duration
|
||
/// was a shape match and carrying a music census across on it was my error.
|
||
///
|
||
/// **Reading 3, one stream. ✅ Decoded disc-wide by the Decoder**, by counting
|
||
/// stream starts inside every inter-descriptor span: **258 spans hold one
|
||
/// stream, 28 hold three, and nothing holds two or any other number.** The 95
|
||
/// movie-voice regions decompose 70 + 8 + 17. So a region is three presentations
|
||
/// of one take, and `359 = 84.55 + 137.32 + 137.32`. Summing a take with a
|
||
/// scaled copy of itself adds ~4 dB and colours it.
|
||
///
|
||
/// 🟡 **Which presentation to keep is a recommendation, not a field.** The
|
||
/// selector is the **highest byte rate** among the equal-duration survivors, on
|
||
/// the Decoder's advice. Nothing on the disc says which one the game plays, and
|
||
/// on `ADV` this picks the **quieter** of the two — −8.3 dBFS against 0.0. It is
|
||
/// stated in the manifest with that consequence so the choice is reversible; a
|
||
/// capture of the intro with dialogue audible settles it.
|
||
///
|
||
/// ## Chunk 0 is dropped, and it is a DUPLICATE rather than a truncation
|
||
///
|
||
/// This comment first guessed it was `BGM_103`'s third sub-wave — a bank header
|
||
/// — and a census over all 95 regions showed that is a *different structure*:
|
||
/// 78 open with a 10 240 B bank header, 17 with a leading headerless stream at
|
||
/// the disc's own `1392 mod 2048` data offset, and **the chunk count
|
||
/// discriminates neither**. The byte-span test then found it is **this movie's
|
||
/// own dialogue, 17 of 17** — not an in-mission line, which was the standing
|
||
/// hypothesis.
|
||
///
|
||
/// Which raised the real question: is dropping it a truncation? **No.** Measured
|
||
/// here with a decoder the RE container does not have — sliding envelope
|
||
/// correlation, overhang allowed, normalised over the overlap: **r = 0.998**
|
||
/// (`ADV`) and **0.932** (`S00A`), against controls of 1.000 (self) and 0.289 (a
|
||
/// different movie), with both lags placing chunk 0 **flush against the end** of
|
||
/// the kept stream. Confirmed in the sample domain at 16.7 / 23.2 dB of
|
||
/// residual. It is the tail of the take, presented again.
|
||
///
|
||
/// So the selection rule is stated in terms of what was measured — *the longest
|
||
/// duration, ties broken by byte rate, minus anything digitally silent* — and
|
||
/// every dropped chunk is named in the manifest with its length and peak.
|
||
/// * **Mono**, with the fold chosen from the stream's own declared channel
|
||
/// count rather than by passing `-ac 1` and hoping. A voice track that is
|
||
/// already mono is passed through untouched.
|
||
/// * **No sync offset.** The voice plays from the video's first frame, so the
|
||
/// runtime needs no delay and none is authored. The decoded length is
|
||
/// recorded beside the movie's own length in the manifest so a disagreement
|
||
/// is visible rather than absorbed.
|
||
pub fn export_voice<S: DiscSource + ?Sized>(
|
||
source: &S,
|
||
out: &Path,
|
||
movie: &str,
|
||
video_duration_s: Option<f32>,
|
||
presentation: Presentation,
|
||
) -> Result<Option<Exported>> {
|
||
use sylpheed_formats::slb::VoiceLang;
|
||
|
||
// English only: MISSION §7 puts localisation beyond English out of scope.
|
||
// The language is a parameter of the resolution, not of the file layout, so
|
||
// adding Japanese later is a second call and a second file, not a re-think.
|
||
let lang = VoiceLang::English;
|
||
let region = media::resolve_movie_voice_region(source, movie, lang);
|
||
let Some((start, end)) = region else {
|
||
return Ok(None);
|
||
};
|
||
let riffs = media::voice_region_riffs(source, start, end)
|
||
.map_err(anyhow::Error::msg)
|
||
.with_context(|| format!("decode the {movie} voice region"))?;
|
||
if riffs.is_empty() {
|
||
return Ok(None);
|
||
}
|
||
|
||
let dir = out.join("audio/voice");
|
||
std::fs::create_dir_all(&dir)?;
|
||
let mut all = Vec::new();
|
||
for (i, riff) in riffs.iter().enumerate() {
|
||
all.push(stage_riff(&dir, &format!("{movie}.{i}"), riff)?);
|
||
}
|
||
|
||
// Classify before mixing. XMA declares no duration, so each chunk is decoded
|
||
// and timed -- the only way to tell a stem from the leading region, and the
|
||
// measurement that showed concatenation to be wrong here.
|
||
let probed: Vec<(f32, f32)> = all.iter().map(|p| decoded_chunk(p)).collect();
|
||
let lengths: Vec<f32> = probed.iter().map(|&(d, _)| d).collect();
|
||
// A DIGITALLY SILENT chunk is dropped before anything else, and that is
|
||
// arithmetic rather than a judgement about content: it contributes nothing
|
||
// to a mix, and counting it in the 1/n normalisation costs 6.02 dB for
|
||
// nothing. `S00A`'s second full-length chunk is exactly this -- 4 497 300
|
||
// samples of zeroes, peak -inf -- and summing it is why that movie's voice
|
||
// came out at -16.2 dBFS against a source peaking at -4.2.
|
||
let silent: Vec<usize> = (0..all.len()).filter(|&i| probed[i].1 <= -90.0).collect();
|
||
let longest = (0..all.len())
|
||
.filter(|i| !silent.contains(i))
|
||
.map(|i| lengths[i])
|
||
.fold(0.0f32, f32::max);
|
||
// A tie at 1 ms. The two stems agree to six decimals and the chunk that is
|
||
// not one of them misses by tens of seconds, so nothing sits near this
|
||
// bound: it separates the measured cases without being a tuned threshold.
|
||
// ONE STREAM, NOT A SUM -- and this is the third reading of these chunks, each
|
||
// one refuted by a measurement rather than by an argument.
|
||
//
|
||
// They were concatenated (359 s for a 137 s movie), then summed as HANDOFF
|
||
// Q10's two stems (refuted here: `S00A`'s second is silence, `ADV`'s is
|
||
// 0.60x the first). The Decoder then decoded the shape disc-wide -- counting
|
||
// stream starts inside every inter-descriptor span gives 258 spans with ONE
|
||
// stream and 28 with THREE, and nothing with two or any other number, so a
|
||
// region carries **three presentations of one take**, not a mix. Summing a
|
||
// take with a scaled copy of itself adds ~4 dB and colours it.
|
||
//
|
||
// WHICH of the equal-duration survivors is a CHOICE, and it lives in
|
||
// `authored/audio.json` rather than here -- see [`Presentation`]. It was
|
||
// `highest_rate` on the Decoder's recommendation until that was withdrawn as
|
||
// self-contradictory. `loudest` is a PER-ASSET CONTENT choice and nothing
|
||
// more: the disc masters its other audio near full scale, and it puts the
|
||
// two cutscenes' dialogue at comparable levels.
|
||
//
|
||
// ⚠️ A structural argument for it — `ADV`'s higher-rate stream is dual-mono,
|
||
// so its extra bytes encode a duplicated channel rather than fidelity — was
|
||
// offered here and **does not generalise**. The channel measurement is
|
||
// `ADV`'s and stands; the inference was tested disc-wide over the 28
|
||
// three-stream cues and the size ratio runs 0.0778 to 2.9163. Neither rule
|
||
// has a structural argument behind it, which is exactly why the choice is
|
||
// authored rather than derived.
|
||
let tied: Vec<usize> = (0..all.len())
|
||
.filter(|&i| !silent.contains(&i) && (longest - lengths[i]).abs() < 0.001)
|
||
.collect();
|
||
let chosen = match presentation {
|
||
Presentation::HighestRate => tied.iter().copied().max_by_key(|&i| riffs[i].len()),
|
||
Presentation::Loudest => tied
|
||
.iter()
|
||
.copied()
|
||
.max_by(|&a, &b| probed[a].1.total_cmp(&probed[b].1)),
|
||
};
|
||
let keep: Vec<usize> = chosen.into_iter().collect();
|
||
let dropped: Vec<String> = (0..all.len())
|
||
.filter(|i| !keep.contains(i))
|
||
.map(|i| {
|
||
format!(
|
||
"chunk {i} ({:.3} s, {} B, peak {})",
|
||
lengths[i],
|
||
riffs[i].len(),
|
||
if silent.contains(&i) {
|
||
"SILENT".to_string()
|
||
} else {
|
||
format!("{:.1} dBFS", probed[i].1)
|
||
}
|
||
)
|
||
})
|
||
.collect();
|
||
if keep.is_empty() {
|
||
return Ok(None);
|
||
}
|
||
let staged: Vec<PathBuf> = keep.iter().map(|&i| all[i].clone()).collect();
|
||
|
||
// The fold averages the channels that CARRY SIGNAL, not the channels the
|
||
// stream declares.
|
||
//
|
||
// This function's first version averaged all declared channels, and the doc
|
||
// comment above it warned in as many words that "a stereo matrix applied to
|
||
// a mono voice track is not an error, it is a -6 dB attenuation that nothing
|
||
// reports". It then did exactly that: **channel 2 of both voice streams is
|
||
// digitally silent** -- peak -inf over the whole file, on `ADV` and on
|
||
// `S00A` -- so this is a mono recording carried in a nominally stereo
|
||
// stream, and averaging it with silence cost 5.94 dB. Checking the declared
|
||
// count is not checking the content, and only the content is the fold.
|
||
//
|
||
// Same principle as the silent-chunk drop above, one level down: a silent
|
||
// input contributes nothing to an average and counting it in the divisor is
|
||
// arithmetic, not a mixing decision. `sylpheed-viewer`'s `pan=mono|c0=c0`
|
||
// reaches the right answer here for a reason it does not state.
|
||
let live = live_channels(&staged[0]);
|
||
let channels = probe_channels(&staged[0]).unwrap_or(1);
|
||
let fold = if live.len() <= 1 && channels <= 1 {
|
||
String::new()
|
||
} else if live.len() == 1 {
|
||
format!(",pan=mono|c0=c{}", live[0])
|
||
} else {
|
||
let g = 1.0 / live.len() as f64;
|
||
let terms: Vec<String> = live.iter().map(|c| format!("{g:.6}*c{c}")).collect();
|
||
format!(",pan=mono|c0={}", terms.join("+"))
|
||
};
|
||
|
||
let ogg = dir.join(format!("{movie}.ogg"));
|
||
let mut argv: Vec<String> = ["-hide_banner", "-loglevel", "error", "-y"]
|
||
.iter()
|
||
.map(|s| s.to_string())
|
||
.collect();
|
||
for s in &staged {
|
||
argv.push("-i".into());
|
||
argv.push(s.display().to_string());
|
||
}
|
||
// One input, so no mix and no normalising coefficient: the stream reaches the
|
||
// Ogg at the level the disc has it, and the only filter is the mono fold.
|
||
let filter = format!("[0:a]anull{fold}[a]");
|
||
argv.push("-filter_complex".into());
|
||
argv.push(filter);
|
||
argv.push("-map".into());
|
||
argv.push("[a]".into());
|
||
argv.extend(
|
||
["-c:a", "libvorbis", "-q:a", VORBIS_Q, &ogg.display().to_string()]
|
||
.iter()
|
||
.map(|s| s.to_string()),
|
||
);
|
||
let command = format!("ffmpeg {}", argv.join(" "));
|
||
run_ffmpeg(&argv, &ogg)?;
|
||
let (peak, dur) = measure(&ogg);
|
||
for s in &all {
|
||
std::fs::remove_file(s).ok();
|
||
}
|
||
|
||
let against = match (dur, video_duration_s) {
|
||
(Some(d), Some(v)) => format!(
|
||
" Decoded length {d:.3} s against the movie's {v:.3} s (delta {:+.3} s); \
|
||
NOT trimmed to fit -- a clamp would hide a resolution error, and the \
|
||
runtime stops the voice when the video ends.",
|
||
d - v
|
||
),
|
||
_ => String::new(),
|
||
};
|
||
|
||
Ok(Some(Exported {
|
||
name: movie.to_string(),
|
||
file: format!("audio/voice/{movie}.ogg"),
|
||
command,
|
||
why: format!(
|
||
"DECODED, not authored: the movie manifest in tables.pak binds {movie} to a \
|
||
voice cue, and sylpheed_formats::media::resolve_movie_voice_region walks \
|
||
movie -> token -> sound id -> byte region [{start}, {end}) of the continuous \
|
||
voice stream. NOT matched by filename: RT01A's voice lives inside \
|
||
VOICE_ADV.slb, so the name is right for this movie by luck and wrong for \
|
||
others. Of {} region chunk(s), exactly ONE is kept -- not summed. A region \
|
||
carries THREE PRESENTATIONS OF ONE TAKE, decoded disc-wide by counting stream \
|
||
starts inside every inter-descriptor span: 258 spans hold one stream and 28 hold \
|
||
three, and nothing holds two. This exporter read these chunks wrongly twice \
|
||
before that landed -- first concatenating them (359 s for a 137 s movie), then \
|
||
summing them as HANDOFF Q10's two stems, which its own measurements refuted: \
|
||
S00A's second full-length chunk is DIGITAL SILENCE and ADV's is 0.60x the first \
|
||
with 26.8 dB of residual. Summing a take with a scaled copy of itself adds ~4 dB \
|
||
and colours it. 🟡 WHICH of the equal-duration survivors is kept is a CHOICE, \
|
||
not a decoded field -- authored/audio.json voice.presentation = {:?}, with its \
|
||
why. Nothing on the disc ranks the presentations: wEncodeOptions, channel count \
|
||
and channel mask are byte-identical across them. One capture of the movie with \
|
||
dialogue audible deletes that entry.{} Folded to mono from the {} of {channels} declared \
|
||
channel(s) that carry signal -- channel 2 of both voice streams is digitally \
|
||
silent, and averaging it in cost 5.94 dB until this was measured rather than \
|
||
read off the declared count.{against}",
|
||
riffs.len(),
|
||
match presentation {
|
||
Presentation::Loudest => "loudest",
|
||
Presentation::HighestRate => "highest_rate",
|
||
},
|
||
if dropped.is_empty() {
|
||
String::new()
|
||
} else {
|
||
format!(
|
||
" DROPPED, and NOT as junk -- the leading chunk is DECODED to be this \
|
||
movie's OWN dialogue, 17 of 17 regions (docs/re/structures/\
|
||
voice-region-leading-chunk.md; an earlier note here wrongly equated it \
|
||
with BGM_103's third sub-wave, which a disc-wide census showed is a \
|
||
different structure). This port then measured it to be the TAIL of the \
|
||
kept stream -- sliding envelope correlation r=0.998 (ADV) and 0.932 \
|
||
(S00A), the lag placing it flush against that stream's end, against \
|
||
controls of 1.000 self and 0.289 for a different movie -- so dropping it \
|
||
removes a DUPLICATE, not dialogue. Dropped: {}.",
|
||
dropped.join(", ")
|
||
)
|
||
},
|
||
live.len()
|
||
),
|
||
peak_dbfs: peak,
|
||
duration_s: dur,
|
||
kind: "voice",
|
||
name_match: None,
|
||
loop_mode: None,
|
||
sub_waves: riffs.len(),
|
||
}))
|
||
}
|
||
|
||
/// How many channels a staged `RIFF` declares, per ffprobe.
|
||
fn probe_channels(path: &Path) -> Option<u8> {
|
||
let out = Command::new("ffprobe")
|
||
.args([
|
||
"-v", "error", "-select_streams", "a:0",
|
||
"-show_entries", "stream=channels", "-of", "csv=p=0",
|
||
])
|
||
.arg(path)
|
||
.output()
|
||
.ok()?;
|
||
String::from_utf8_lossy(&out.stdout).trim().parse().ok()
|
||
}
|
||
|
||
/// Seconds a finished media file runs, per ffprobe.
|
||
///
|
||
/// Exposed so the caller can hand [`export_voice`] the movie's own length: the
|
||
/// voice is a separate asset with no shared container to agree with, so the only
|
||
/// way a resolution error shows up is a length that does not match the picture.
|
||
pub fn probe_duration(path: &Path) -> Option<f32> {
|
||
let out = Command::new("ffprobe")
|
||
.args([
|
||
"-v", "error", "-show_entries", "format=duration",
|
||
"-of", "csv=p=0",
|
||
])
|
||
.arg(path)
|
||
.output()
|
||
.ok()?;
|
||
String::from_utf8_lossy(&out.stdout).trim().parse().ok()
|
||
}
|
||
|
||
/// Seconds and peak dBFS that one staged XMA `RIFF` decodes to.
|
||
///
|
||
/// XMA carries no duration in its header, so the chunk is decoded to PCM and the
|
||
/// result measured. That is expensive and it is the only instrument that can
|
||
/// separate these chunks at all: `ffprobe` on the `RIFF` itself returns `N/A`
|
||
/// for duration, which a caller that trusted it would read as zero, and the
|
||
/// corpus records `sylpheed-cli audio info` mis-reading the same headers as
|
||
/// 16 channels at 4 310 Hz.
|
||
///
|
||
/// A silent chunk returns `-inf`, which the caller drops.
|
||
fn decoded_chunk(riff: &Path) -> (f32, f32) {
|
||
let wav = riff.with_extension("probe.wav");
|
||
let ok = Command::new("ffmpeg")
|
||
.args(["-hide_banner", "-loglevel", "error", "-y", "-i"])
|
||
.arg(riff)
|
||
.arg(&wav)
|
||
.output()
|
||
.map(|o| o.status.success())
|
||
.unwrap_or(false);
|
||
let out = if ok {
|
||
let (peak, dur) = measure(&wav);
|
||
(dur.unwrap_or(0.0), peak.unwrap_or(f32::NEG_INFINITY))
|
||
} else {
|
||
(0.0, f32::NEG_INFINITY)
|
||
};
|
||
let _ = std::fs::remove_file(&wav);
|
||
out
|
||
}
|
||
|
||
|
||
/// Which channel indices of a decoded stream are not digitally silent.
|
||
///
|
||
/// `astats` reports per-channel blocks: a `Channel: N` line followed by that
|
||
/// channel's own `Peak level dB`. A channel whose peak is `-inf` carries
|
||
/// nothing, and folding it into an average is a pure loss.
|
||
///
|
||
/// Falls back to "every declared channel is live" if the parse finds nothing,
|
||
/// because the failure to prefer is the one that changes no level.
|
||
fn live_channels(riff: &Path) -> Vec<usize> {
|
||
let wav = riff.with_extension("chan.wav");
|
||
let ok = Command::new("ffmpeg")
|
||
.args(["-hide_banner", "-loglevel", "error", "-y", "-i"])
|
||
.arg(riff)
|
||
.arg(&wav)
|
||
.output()
|
||
.map(|o| o.status.success())
|
||
.unwrap_or(false);
|
||
let mut live = Vec::new();
|
||
if ok {
|
||
if let Ok(out) = Command::new("ffmpeg")
|
||
.args(["-hide_banner", "-v", "info", "-i"])
|
||
.arg(&wav)
|
||
.args(["-af", "astats", "-f", "null", "-"])
|
||
.output()
|
||
{
|
||
let text = String::from_utf8_lossy(&out.stderr).into_owned();
|
||
let mut current: Option<usize> = None;
|
||
for line in text.lines() {
|
||
if let Some((_, n)) = line.split_once("Channel: ") {
|
||
// astats numbers channels from 1; `pan` addresses c0 upward.
|
||
current = n.trim().parse::<usize>().ok().map(|n| n.saturating_sub(1));
|
||
} else if let Some((_, v)) = line.split_once("Peak level dB: ") {
|
||
if let Some(c) = current.take() {
|
||
if v.trim().parse::<f32>().map(|p| p > -90.0).unwrap_or(false) {
|
||
live.push(c);
|
||
}
|
||
}
|
||
}
|
||
}
|
||
}
|
||
}
|
||
let _ = std::fs::remove_file(&wav);
|
||
if live.is_empty() {
|
||
live = (0..probe_channels(riff).unwrap_or(1) as usize).collect();
|
||
}
|
||
live
|
||
}
|