Files
Sylpheed/crates/sylpheed-export/src/audio.rs
Sylpheed port agent 92f1436836 port: withdraw the dual-mono generalisation -- the measurement stands, the rule does not
I argued `highest_rate` had no case because ADV's higher-rate presentation is
dual-mono while its louder one is mono-in-stereo, so the extra bytes buy a
duplicated channel rather than fidelity. The Decoder tested that disc-wide over
the 28 three-stream cues: the stream-3/stream-2 size ratio runs min 0.0778,
median 1.2565, max 2.9163, sd 0.5057, with only 12 of 28 within 15% of 1.0, and
declared rates scatter with them. A 37x spread is not a duplicated channel.

The CHANNEL MEASUREMENT STANDS -- ADV chunk 1 is mono-in-stereo and chunk 2 is
dual-mono at -8.318574, this port's own decode, which the Decoder could not
re-run and did not dispute. What fails is the step from one asset to the format.

NOTHING IN THE EXPORT CHANGES. `loudest` is a per-asset content rule -- it reads
the peak of the streams in front of it -- so a scattering structural ratio cannot
undermine it. What changes is the REASON, in four places: authored/audio.json's
presentation_why, the selector comment in audio.rs, BLOCKED.md's row, and
DECISIONS.md. The honest statement is narrower: `highest_rate` was never refuted,
it was never argued for, and neither is `loudest`. That is why the entry is
marked CHOSEN rather than measured, and why one capture deletes it.

Recorded on the pattern rather than just the instance: this is the third claim of
mine in two iterations that generalised a single-asset observation, after "the
chunks are two stems" and "everything the sequencer paces off rest.t is late".
All three were true of the thing I looked at. The failure is reaching for the
rule a measurement would imply if it held everywhere and writing that down in the
same breath as the measurement.

Also noted, not mine and not affecting export_voice: S12B's three streams are
byte-size identical, and BIRD_224 is three-stream while not being a movie cue.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N7FiFFFwbvG2uxdcEh8HyF
2026-08-29 15:39:22 +00:00

899 lines
39 KiB
Rust
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
//! Menu audio: disc XMA → Ogg Vorbis, because Godot 4 plays Vorbis natively and
//! will never be taught to read XMA or to open `sound.pak`.
//!
//! Two kinds of thing come out of here and they are not symmetric:
//!
//! * **SE cues** — three short mono waves out of `Static.slb`, one per menu
//! event. Where each one lives was *measured off the running game* (HANDOFF
//! Q8) and is **not on the disc in any findable form**.
//! * **BGM** — one bank of `sound.pak`, which is **two stems of one
//! performance** (HANDOFF Q10). They are summed here into a single file.
//!
//! ## Neither table lives in this file
//!
//! Both come from `authored/audio.json`, and that is the point of the module
//! rather than an accident of configuration. MISSION §3: a value that is
//! *measured* rather than *decoded* lives in `authored/`, carries a `why`, and
//! is deleted the day the disc states it. A measured offset compiled into a Rust
//! `const` is a measurement wearing the costume of a decoded field — it reads
//! like the exporter derived it, and nobody deletes it, because nobody can see
//! it. Contrast [`crate::video::MOVIES`], which *is* a `const` here: Q9 decoded
//! that mapping from the movie manifest on the disc.
//!
//! ## The assembly is not reimplemented here
//!
//! `sylpheed_formats::media` owns every question of the form *"which bytes
//! belong together"* — segment-spanning reads, multi-sub-wave banks, and the
//! delimiter-less `Static.slb` where a wave is only `(offset, packet_count)`.
//! This module asks it for `RIFF`s and converts them. The seam is deliberate:
//! everything before it is disc knowledge, everything after it is a codec
//! choice, and re-deriving the first half here is exactly the mistake the
//! mission names.
use anyhow::{bail, Context, Result};
use serde::Deserialize;
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use std::process::Command;
use sylpheed_formats::media::{self, DiscSource};
/// One menu sound effect, and where its wave sits in a delimiter-less bank.
///
/// `offset` is a string so the file can carry `"0x1ec0"` — the form the RE
/// finding is written in. A reader comparing the two should not have to convert
/// 125 632 in their head to believe they match.
#[derive(Deserialize)]
pub struct CueSpec {
pub bank: String,
pub offset: String,
pub packets: usize,
pub channels: u8,
pub rate: u32,
/// The game's own cue identifier, where one has been *guessed by name*.
/// Absent means nobody claimed one — never that the binding is unknown.
#[serde(default)]
pub name_match: Option<String>,
pub why: String,
}
impl CueSpec {
fn offset(&self, event: &str) -> Result<usize> {
let s = self.offset.trim();
let parsed = match s.strip_prefix("0x").or_else(|| s.strip_prefix("0X")) {
Some(hex) => usize::from_str_radix(&hex.replace('_', ""), 16),
None => s.replace('_', "").parse(),
};
parsed.with_context(|| format!("cue `{event}`: `{s}` is not an offset"))
}
}
/// One music bank and what to do with it.
#[derive(Deserialize)]
pub struct BgmSpec {
pub bank: String,
/// What happens at the end of the file. Carried through to the manifest so
/// the runtime does not have to reach into `authored/` to find out, and so
/// that the *why* travels with the decision.
#[serde(default)]
pub r#loop: Option<String>,
pub why: String,
#[serde(default)]
pub loop_why: Option<String>,
#[serde(default)]
pub stems_why: Option<String>,
}
/// Which of a voice region's full-length presentations to export.
///
/// A region carries three presentations of one take and **nothing on the disc
/// ranks them** — `wEncodeOptions`, channel count and channel mask are
/// byte-identical across them. So this is a CHOICE, it lives in
/// `authored/audio.json` with its `why`, and it is deleted the day a capture
/// says which one the game plays.
#[derive(Deserialize, Default, Clone, Copy, PartialEq)]
#[serde(rename_all = "snake_case")]
pub enum Presentation {
/// Peak nearest full scale.
#[default]
Loudest,
/// Most bytes per second.
HighestRate,
}
/// `authored/audio.json`, with the documentation keys dropped.
pub struct Config {
pub se: Vec<(String, CueSpec)>,
pub bgm: Vec<(String, BgmSpec)>,
pub voice: Presentation,
}
/// Read `authored/audio.json`, or `None` when there is no such file.
///
/// Absent is not an error: an export with no audio is what every milestone
/// before P6 produced, and it should stay possible to take one.
pub fn load(authored: &Path) -> Result<Option<Config>> {
let path = authored.join("audio.json");
if !path.exists() {
return Ok(None);
}
#[derive(Deserialize)]
struct File {
#[serde(default)]
se: BTreeMap<String, serde_json::Value>,
#[serde(default)]
bgm: BTreeMap<String, serde_json::Value>,
#[serde(default)]
voice: BTreeMap<String, serde_json::Value>,
}
let raw = std::fs::read_to_string(&path)
.with_context(|| format!("read {}", path.display()))?;
let file: File = serde_json::from_str(&raw)
.with_context(|| format!("parse {}", path.display()))?;
// `_` is the house convention for a prose block explaining the section it
// sits in -- see `authored/flow.json` and `authored/timing.json`. It is
// documentation, not an entry, and the schema must skip it rather than
// force the reasoning out of the file that holds the decision.
fn entries<T: serde::de::DeserializeOwned>(
m: BTreeMap<String, serde_json::Value>,
what: &str,
) -> Result<Vec<(String, T)>> {
m.into_iter()
.filter(|(k, _)| k != "_")
.map(|(k, v)| {
let parsed = serde_json::from_value(v)
.with_context(|| format!("authored/audio.json: {what}.{k}"))?;
Ok((k, parsed))
})
.collect()
}
// Absent means `loudest`, which is what the file says today. A default here
// is safe in a way a default matrix is not: the manifest records which
// presentation was taken and why, on every entry.
let voice = match file.voice.get("presentation") {
Some(v) => serde_json::from_value(v.clone())
.with_context(|| format!("authored/audio.json: voice.presentation {v}"))?,
None => Presentation::default(),
};
Ok(Some(Config {
se: entries(file.se, "se")?,
bgm: entries(file.bgm, "bgm")?,
voice,
}))
}
/// Vorbis quality. `-q:a 5` is ffmpeg's usual transparent-ish setting and is
/// what [`crate::video`] already uses for the movies' audio; using one value
/// across the export means a level difference between a cue and a movie cannot
/// be a codec artefact.
const VORBIS_Q: &str = "5";
pub struct Exported {
pub name: String,
pub file: String,
pub command: String,
pub why: String,
/// dBFS peak of the decoded result. Recorded because the BGM sum can clip
/// and a clipped file is not visibly different from a correct one.
pub peak_dbfs: Option<f32>,
/// Seconds, as ffprobe reads them back off the finished file. Recorded
/// because the cue durations are the one thing about the SE export that an
/// outside finding predicts, so they are the one thing that can be checked.
pub duration_s: Option<f32>,
pub kind: &'static str,
/// The game's own identifier where it is a name match, never a measurement.
pub name_match: Option<String>,
/// What the runtime should do at the end of the file, where that was
/// authored. `None` on a cue: a cue ends.
pub loop_mode: Option<String>,
/// How many sub-waves `media` returned for a bank. Carried out of here so
/// the caller can warn when it contradicts HANDOFF's census -- this module
/// does not get to decide that one of them is not a stem.
pub sub_waves: usize,
}
/// Run ffmpeg, writing to a temp name and renaming on success.
///
/// The rename is not tidiness: the filesystem is shared with another agent, and
/// a reader that catches a half-written Ogg gets a confident wrong duration
/// rather than an error. `docs/port/AUDIO-VERIFICATION.md` records that this
/// already happened once on a video.
fn run_ffmpeg(argv: &[String], out: &Path) -> Result<()> {
// The extension goes LAST, not the `.partial`. ffmpeg picks its muxer from
// the output filename, so `.back.ogg.partial` is not a slightly uglier
// temp name -- it is a hard failure before a byte is written: "Unable to
// choose an output format". `video.rs` already had this shape; this
// function was written from scratch and did not.
let stem = out.file_stem().unwrap_or_default().to_string_lossy().into_owned();
let ext = out.extension().unwrap_or_default().to_string_lossy().into_owned();
let partial = out.with_file_name(format!(".{stem}.partial.{ext}"));
let mut argv = argv.to_vec();
let last = argv.len() - 1;
argv[last] = partial.display().to_string();
let res = Command::new("ffmpeg")
.args(&argv)
.output()
.context("run ffmpeg -- is it on PATH?")?;
if !res.status.success() {
let _ = std::fs::remove_file(&partial);
bail!(
"ffmpeg failed writing {}:\n{}",
out.display(),
String::from_utf8_lossy(&res.stderr)
);
}
std::fs::rename(&partial, out)?;
Ok(())
}
/// Peak level and duration of a finished file.
///
/// Measured rather than assumed because the BGM is a **sum of two stems** and a
/// sum can clip, and because silence is the audio failure that looks like
/// success: a file of exactly the right duration, full of zeroes. Both numbers
/// pass every check that is not looking for them, so the export looks for them.
fn measure(path: &Path) -> (Option<f32>, Option<f32>) {
let out = Command::new("ffmpeg")
.args(["-hide_banner", "-v", "info", "-i"])
.arg(path)
.args(["-af", "astats=measure_perchannel=none", "-f", "null", "-"])
.output();
let Ok(out) = out else { return (None, None) };
let text = String::from_utf8_lossy(&out.stderr).into_owned();
// `astats` writes through the filter log, so every line carries a
// `[Parsed_astats_0 @ 0x…] ` prefix. Matching on the line START silently
// finds nothing and reports "peak unmeasured", which is the failure this
// measurement exists to catch -- so it is found as a SUBSTRING.
const KEY: &str = "Peak level dB:";
let peak = text
.lines()
.find_map(|l| l.split_once(KEY)?.1.trim().parse().ok());
let dur = Command::new("ffprobe")
.args([
"-v", "error", "-show_entries", "format=duration",
"-of", "csv=p=0",
])
.arg(path)
.output()
.ok()
.and_then(|o| String::from_utf8_lossy(&o.stdout).trim().parse().ok());
(peak, dur)
}
/// Write one XMA `RIFF` beside the output so ffmpeg has something to open.
///
/// Kept next to the result rather than in `/tmp` so a failed export leaves the
/// intermediate where the person debugging it will look, and removed on success
/// so the tree holds only formats a modder can open (MODDING rule 3).
fn stage_riff(dir: &Path, stem: &str, riff: &[u8]) -> Result<PathBuf> {
let path = dir.join(format!(".{stem}.xma.wav"));
std::fs::write(&path, riff).with_context(|| format!("write {}", path.display()))?;
Ok(path)
}
/// The menu cues, as `audio/se/<event>.ogg`.
pub fn export_cues<S: DiscSource + ?Sized>(
source: &S,
out: &Path,
cues: &[(String, CueSpec)],
) -> Result<Vec<Exported>> {
if cues.is_empty() {
return Ok(Vec::new());
}
let dir = out.join("audio/se");
std::fs::create_dir_all(&dir)?;
let mut done = Vec::new();
for (event, cue) in cues {
let offset = cue.offset(event)?;
// `media` owns the assembly. Asking it for the RIFF rather than reading
// `Static.slb` here is the whole point of the seam -- and it REFUSES a
// short read rather than returning a truncated stream, because a
// truncated XMA decodes to plausible-sounding garbage.
let riff = media::se_wave_riff(
source, &cue.bank, offset, cue.packets, cue.channels, cue.rate,
)
.map_err(anyhow::Error::msg)
.with_context(|| format!("assemble the {event} cue"))?;
let staged = stage_riff(&dir, event, &riff)?;
let ogg = dir.join(format!("{event}.ogg"));
let argv: Vec<String> = [
"-hide_banner", "-loglevel", "error", "-y",
"-i", &staged.display().to_string(),
"-c:a", "libvorbis", "-q:a", VORBIS_Q,
&ogg.display().to_string(),
]
.iter()
.map(|s| s.to_string())
.collect();
let command = format!("ffmpeg {}", argv.join(" "));
run_ffmpeg(&argv, &ogg)?;
let (peak, dur) = measure(&ogg);
std::fs::remove_file(&staged).ok();
done.push(Exported {
name: event.clone(),
file: format!("audio/se/{event}.ogg"),
command,
why: format!(
"{} AUTHORED because it is measured, not decoded: authored/audio.json \
se.{event}. Located in {} at {:#x} for {} packet(s); the ASSEMBLY is \
sylpheed_formats::media::se_wave_riff, which refuses a short read.",
cue.why, cue.bank, offset, cue.packets
),
peak_dbfs: peak,
duration_s: dur,
kind: "se",
name_match: cue.name_match.clone(),
loop_mode: None,
sub_waves: 1,
});
}
Ok(done)
}
/// One music bank, as a single `audio/bgm/<name>.ogg`.
///
/// **The two stems are summed, not concatenated and not split into two files.**
///
/// HANDOFF Q10: a bank's sub-waves are two stems of one performance, played
/// together — sample-synchronous, equal duration, on all 32 banks.
/// Concatenating them is explicitly wrong. Emitting two files would be wrong
/// here for a different reason: MODDING rule 1 is *one logical asset, one
/// file*, and a modder who had to line two stems up by hand would be
/// reassembling exactly what the exporter exists to resolve.
///
/// **The sum is scaled by 1/n, and an earlier version of this comment argued the
/// opposite.** It said `normalize=0` sums at unity "because halving is a mix
/// decision nobody made". That was wrong twice over. Unity summing IS a decision
/// — and it is the one that can clip, which it duly did: `BGM_103` came out at
/// **+1.8 dBFS**. And 1/n is not a taste call but the smallest constant that
/// makes an n-input sum of unity-scale signals provably clip-free, which is the
/// same reasoning `video.rs` already uses for its 0.4142-normalised 5.1 downmix.
/// It is written out as an explicit `volume=` rather than left to `amix`'s
/// `normalize=1` default so the coefficient appears in the manifest's command
/// line: a default is a decision nobody made, and it can move under an ffmpeg
/// upgrade.
///
/// It preserves the stems' relative balance exactly, which is the only thing
/// about the sum that HANDOFF Q10 actually settles. The peak is still measured
/// and reported.
///
/// The file is named for the **role** (`main_menu`), not for the bank
/// (`BGM_001`). Which bank plays here is authored and expected to change; the
/// role is what the runtime asks for, and a rename of the disc asset should not
/// be a change to the Godot project.
pub fn export_bgm<S: DiscSource + ?Sized>(
source: &S,
out: &Path,
role: &str,
spec: &BgmSpec,
) -> Result<Option<Exported>> {
let riffs = match media::sound_bank_riffs(source, &spec.bank) {
Ok(r) if !r.is_empty() => r,
Ok(_) => return Ok(None),
// "This disc does not have that bank" is a MISSING ASSET, not a broken
// exporter: the manifest carries a warning and everything else still
// exports. Any other failure -- a short read, a malformed bank -- is a
// real error and stops the run, because a partly-read bank produces a
// file that plays.
Err(e) if e.contains("not present in sound.pak") => return Ok(None),
Err(e) => bail!("{}: {e}", spec.bank),
};
let dir = out.join("audio/bgm");
std::fs::create_dir_all(&dir)?;
let mut staged = Vec::new();
for (i, riff) in riffs.iter().enumerate() {
staged.push(stage_riff(&dir, &format!("{role}.{i}"), riff)?);
}
let ogg = dir.join(format!("{role}.ogg"));
let mut argv: Vec<String> = ["-hide_banner", "-loglevel", "error", "-y"]
.iter()
.map(|s| s.to_string())
.collect();
for s in &staged {
argv.push("-i".into());
argv.push(s.display().to_string());
}
if staged.len() > 1 {
argv.push("-filter_complex".into());
argv.push(format!(
"amix=inputs={n}:normalize=0,volume={:.6}",
1.0 / staged.len() as f64,
n = staged.len()
));
}
argv.extend(
["-c:a", "libvorbis", "-q:a", VORBIS_Q, &ogg.display().to_string()]
.iter()
.map(|s| s.to_string()),
);
let command = format!("ffmpeg {}", argv.join(" "));
run_ffmpeg(&argv, &ogg)?;
let (peak, dur) = measure(&ogg);
for s in &staged {
std::fs::remove_file(s).ok();
}
let mut why = format!(
"{} authored/audio.json bgm.{role} names bank {}. Its {} \
sub-wave(s) are SUMMED into one file and the sum is scaled by 1/{}, \
which is the smallest constant that cannot clip.",
spec.why,
spec.bank,
riffs.len(),
riffs.len()
);
if let Some(s) = &spec.stems_why {
why.push(' ');
why.push_str(s);
}
if let Some(s) = &spec.loop_why {
why.push(' ');
why.push_str(s);
}
Ok(Some(Exported {
name: role.to_string(),
file: format!("audio/bgm/{role}.ogg"),
command,
why,
peak_dbfs: peak,
duration_s: dur,
kind: "bgm",
name_match: None,
loop_mode: spec.r#loop.clone(),
sub_waves: riffs.len(),
}))
}
/// One cutscene's voice-over, as `audio/voice/<movie>.ogg`.
///
/// ## Why this is a separate file from the movie at all
///
/// A human play-test heard music under the intro and no dialogue, and the
/// obvious reading — "the transcode dropped a channel" — is wrong. `ADV.wmv`
/// genuinely carries **music and effects only**. On this disc a cutscene's voice
/// is a *different asset*: one continuous XMA stream in `sound.pak`, bound to
/// the movie by the movie manifest in `tables.pak` (`ADV` → `VOICE_ADV`). It was
/// not dropped by [`crate::video`]; it was never exported, because nothing here
/// asked for it.
///
/// ## The binding is resolved, never matched by name
///
/// `sylpheed_formats::media::resolve_movie_voice_region` walks
/// movie → cue token → sound id → byte region. It is the only route taken here,
/// and the reason is that the cheap route looks correct on exactly the movies a
/// person would check first. Measured on the retail disc:
///
/// | movie | region | inside the bank named after it? |
/// |---|---|---|
/// | `ADV` | 433 930 240…437 044 592 | yes |
/// | `S00A` | 452 798 464…455 499 120 | yes |
/// | `RT01A` | 437 044 592…437 345 648 | **no — it is inside `VOICE_ADV.slb`** |
///
/// So `VOICE_<movie>.slb` is a name that happens to hold the right audio twice
/// out of three, and the two it gets right are the two in this port's scope.
/// Reading the bank by name would have shipped, verified clean, and been wrong
/// for the radio cutscenes the moment anybody extended the export.
///
/// ## What a `None` means, and what it must not become
///
/// A movie whose region does not resolve is **genuinely unvoiced** — that is a
/// real answer for most `hokyu_*` resupply cutscenes, and the corpus already
/// paid for the alternative: resolving unbound movies through a shared demo line
/// played the *wrong recording*. Nothing is substituted. A `\Movie\` token that
/// resolved a clip but not a region stays silent for the same reason
/// `sylpheed-viewer` keeps it silent: its raw `.slb` is off by one chunk, so it
/// is not this movie's dialogue.
///
/// ## Three choices made here, and the reason each is not a guess
///
/// * **One file per movie** (MODDING rule 1), and **exactly one region chunk is
/// kept**. Not concatenated, not summed. This function got that wrong twice
/// before it got it right, and the history is kept below because each wrong
/// reading was ended by a measurement, not by an argument.
///
/// ## A region holds THREE PRESENTATIONS OF ONE TAKE — decoded, and not by me
///
/// A resolved region decodes to several chunks. Decoded and timed against the
/// movies' own lengths:
///
/// | movie | movie | chunk 0 | chunk 1 | chunk 2 |
/// |---|---|---|---|---|
/// | `ADV` | 137.437 s | 84.553 | **137.324** | **137.324** |
/// | `S00A` | 93.779 s | 68.072 | **93.694** | **93.694** |
/// | `RT01A` | — | 0.009 | **34.034** | — |
///
/// **Reading 1, concatenate:** 359 s of dialogue for a 137 s movie. Dead.
///
/// **Reading 2, sum them as HANDOFF Q10's two stems** — equal duration, each
/// spanning the movie, which is exactly Q10's *music* shape. ❌ **Refuted here,
/// and the claim had already been adopted into the RE corpus before I tested
/// it**: `S00A`'s second full-length chunk is **digital silence** (peak inf)
/// and `ADV`'s is **0.60 × the first** with 26.8 dB of residual. Equal duration
/// was a shape match and carrying a music census across on it was my error.
///
/// **Reading 3, one stream. ✅ Decoded disc-wide by the Decoder**, by counting
/// stream starts inside every inter-descriptor span: **258 spans hold one
/// stream, 28 hold three, and nothing holds two or any other number.** The 95
/// movie-voice regions decompose 70 + 8 + 17. So a region is three presentations
/// of one take, and `359 = 84.55 + 137.32 + 137.32`. Summing a take with a
/// scaled copy of itself adds ~4 dB and colours it.
///
/// 🟡 **Which presentation to keep is a recommendation, not a field.** The
/// selector is the **highest byte rate** among the equal-duration survivors, on
/// the Decoder's advice. Nothing on the disc says which one the game plays, and
/// on `ADV` this picks the **quieter** of the two — 8.3 dBFS against 0.0. It is
/// stated in the manifest with that consequence so the choice is reversible; a
/// capture of the intro with dialogue audible settles it.
///
/// ## Chunk 0 is dropped, and it is a DUPLICATE rather than a truncation
///
/// This comment first guessed it was `BGM_103`'s third sub-wave — a bank header
/// — and a census over all 95 regions showed that is a *different structure*:
/// 78 open with a 10 240 B bank header, 17 with a leading headerless stream at
/// the disc's own `1392 mod 2048` data offset, and **the chunk count
/// discriminates neither**. The byte-span test then found it is **this movie's
/// own dialogue, 17 of 17** — not an in-mission line, which was the standing
/// hypothesis.
///
/// Which raised the real question: is dropping it a truncation? **No.** Measured
/// here with a decoder the RE container does not have — sliding envelope
/// correlation, overhang allowed, normalised over the overlap: **r = 0.998**
/// (`ADV`) and **0.932** (`S00A`), against controls of 1.000 (self) and 0.289 (a
/// different movie), with both lags placing chunk 0 **flush against the end** of
/// the kept stream. Confirmed in the sample domain at 16.7 / 23.2 dB of
/// residual. It is the tail of the take, presented again.
///
/// So the selection rule is stated in terms of what was measured — *the longest
/// duration, ties broken by byte rate, minus anything digitally silent* — and
/// every dropped chunk is named in the manifest with its length and peak.
/// * **Mono**, with the fold chosen from the stream's own declared channel
/// count rather than by passing `-ac 1` and hoping. A voice track that is
/// already mono is passed through untouched.
/// * **No sync offset.** The voice plays from the video's first frame, so the
/// runtime needs no delay and none is authored. The decoded length is
/// recorded beside the movie's own length in the manifest so a disagreement
/// is visible rather than absorbed.
pub fn export_voice<S: DiscSource + ?Sized>(
source: &S,
out: &Path,
movie: &str,
video_duration_s: Option<f32>,
presentation: Presentation,
) -> Result<Option<Exported>> {
use sylpheed_formats::slb::VoiceLang;
// English only: MISSION §7 puts localisation beyond English out of scope.
// The language is a parameter of the resolution, not of the file layout, so
// adding Japanese later is a second call and a second file, not a re-think.
let lang = VoiceLang::English;
let region = media::resolve_movie_voice_region(source, movie, lang);
let Some((start, end)) = region else {
return Ok(None);
};
let riffs = media::voice_region_riffs(source, start, end)
.map_err(anyhow::Error::msg)
.with_context(|| format!("decode the {movie} voice region"))?;
if riffs.is_empty() {
return Ok(None);
}
let dir = out.join("audio/voice");
std::fs::create_dir_all(&dir)?;
let mut all = Vec::new();
for (i, riff) in riffs.iter().enumerate() {
all.push(stage_riff(&dir, &format!("{movie}.{i}"), riff)?);
}
// Classify before mixing. XMA declares no duration, so each chunk is decoded
// and timed -- the only way to tell a stem from the leading region, and the
// measurement that showed concatenation to be wrong here.
let probed: Vec<(f32, f32)> = all.iter().map(|p| decoded_chunk(p)).collect();
let lengths: Vec<f32> = probed.iter().map(|&(d, _)| d).collect();
// A DIGITALLY SILENT chunk is dropped before anything else, and that is
// arithmetic rather than a judgement about content: it contributes nothing
// to a mix, and counting it in the 1/n normalisation costs 6.02 dB for
// nothing. `S00A`'s second full-length chunk is exactly this -- 4 497 300
// samples of zeroes, peak -inf -- and summing it is why that movie's voice
// came out at -16.2 dBFS against a source peaking at -4.2.
let silent: Vec<usize> = (0..all.len()).filter(|&i| probed[i].1 <= -90.0).collect();
let longest = (0..all.len())
.filter(|i| !silent.contains(i))
.map(|i| lengths[i])
.fold(0.0f32, f32::max);
// A tie at 1 ms. The two stems agree to six decimals and the chunk that is
// not one of them misses by tens of seconds, so nothing sits near this
// bound: it separates the measured cases without being a tuned threshold.
// ONE STREAM, NOT A SUM -- and this is the third reading of these chunks, each
// one refuted by a measurement rather than by an argument.
//
// They were concatenated (359 s for a 137 s movie), then summed as HANDOFF
// Q10's two stems (refuted here: `S00A`'s second is silence, `ADV`'s is
// 0.60x the first). The Decoder then decoded the shape disc-wide -- counting
// stream starts inside every inter-descriptor span gives 258 spans with ONE
// stream and 28 with THREE, and nothing with two or any other number, so a
// region carries **three presentations of one take**, not a mix. Summing a
// take with a scaled copy of itself adds ~4 dB and colours it.
//
// WHICH of the equal-duration survivors is a CHOICE, and it lives in
// `authored/audio.json` rather than here -- see [`Presentation`]. It was
// `highest_rate` on the Decoder's recommendation until that was withdrawn as
// self-contradictory. `loudest` is a PER-ASSET CONTENT choice and nothing
// more: the disc masters its other audio near full scale, and it puts the
// two cutscenes' dialogue at comparable levels.
//
// ⚠️ A structural argument for it — `ADV`'s higher-rate stream is dual-mono,
// so its extra bytes encode a duplicated channel rather than fidelity — was
// offered here and **does not generalise**. The channel measurement is
// `ADV`'s and stands; the inference was tested disc-wide over the 28
// three-stream cues and the size ratio runs 0.0778 to 2.9163. Neither rule
// has a structural argument behind it, which is exactly why the choice is
// authored rather than derived.
let tied: Vec<usize> = (0..all.len())
.filter(|&i| !silent.contains(&i) && (longest - lengths[i]).abs() < 0.001)
.collect();
let chosen = match presentation {
Presentation::HighestRate => tied.iter().copied().max_by_key(|&i| riffs[i].len()),
Presentation::Loudest => tied
.iter()
.copied()
.max_by(|&a, &b| probed[a].1.total_cmp(&probed[b].1)),
};
let keep: Vec<usize> = chosen.into_iter().collect();
let dropped: Vec<String> = (0..all.len())
.filter(|i| !keep.contains(i))
.map(|i| {
format!(
"chunk {i} ({:.3} s, {} B, peak {})",
lengths[i],
riffs[i].len(),
if silent.contains(&i) {
"SILENT".to_string()
} else {
format!("{:.1} dBFS", probed[i].1)
}
)
})
.collect();
if keep.is_empty() {
return Ok(None);
}
let staged: Vec<PathBuf> = keep.iter().map(|&i| all[i].clone()).collect();
// The fold averages the channels that CARRY SIGNAL, not the channels the
// stream declares.
//
// This function's first version averaged all declared channels, and the doc
// comment above it warned in as many words that "a stereo matrix applied to
// a mono voice track is not an error, it is a -6 dB attenuation that nothing
// reports". It then did exactly that: **channel 2 of both voice streams is
// digitally silent** -- peak -inf over the whole file, on `ADV` and on
// `S00A` -- so this is a mono recording carried in a nominally stereo
// stream, and averaging it with silence cost 5.94 dB. Checking the declared
// count is not checking the content, and only the content is the fold.
//
// Same principle as the silent-chunk drop above, one level down: a silent
// input contributes nothing to an average and counting it in the divisor is
// arithmetic, not a mixing decision. `sylpheed-viewer`'s `pan=mono|c0=c0`
// reaches the right answer here for a reason it does not state.
let live = live_channels(&staged[0]);
let channels = probe_channels(&staged[0]).unwrap_or(1);
let fold = if live.len() <= 1 && channels <= 1 {
String::new()
} else if live.len() == 1 {
format!(",pan=mono|c0=c{}", live[0])
} else {
let g = 1.0 / live.len() as f64;
let terms: Vec<String> = live.iter().map(|c| format!("{g:.6}*c{c}")).collect();
format!(",pan=mono|c0={}", terms.join("+"))
};
let ogg = dir.join(format!("{movie}.ogg"));
let mut argv: Vec<String> = ["-hide_banner", "-loglevel", "error", "-y"]
.iter()
.map(|s| s.to_string())
.collect();
for s in &staged {
argv.push("-i".into());
argv.push(s.display().to_string());
}
// One input, so no mix and no normalising coefficient: the stream reaches the
// Ogg at the level the disc has it, and the only filter is the mono fold.
let filter = format!("[0:a]anull{fold}[a]");
argv.push("-filter_complex".into());
argv.push(filter);
argv.push("-map".into());
argv.push("[a]".into());
argv.extend(
["-c:a", "libvorbis", "-q:a", VORBIS_Q, &ogg.display().to_string()]
.iter()
.map(|s| s.to_string()),
);
let command = format!("ffmpeg {}", argv.join(" "));
run_ffmpeg(&argv, &ogg)?;
let (peak, dur) = measure(&ogg);
for s in &all {
std::fs::remove_file(s).ok();
}
let against = match (dur, video_duration_s) {
(Some(d), Some(v)) => format!(
" Decoded length {d:.3} s against the movie's {v:.3} s (delta {:+.3} s); \
NOT trimmed to fit -- a clamp would hide a resolution error, and the \
runtime stops the voice when the video ends.",
d - v
),
_ => String::new(),
};
Ok(Some(Exported {
name: movie.to_string(),
file: format!("audio/voice/{movie}.ogg"),
command,
why: format!(
"DECODED, not authored: the movie manifest in tables.pak binds {movie} to a \
voice cue, and sylpheed_formats::media::resolve_movie_voice_region walks \
movie -> token -> sound id -> byte region [{start}, {end}) of the continuous \
voice stream. NOT matched by filename: RT01A's voice lives inside \
VOICE_ADV.slb, so the name is right for this movie by luck and wrong for \
others. Of {} region chunk(s), exactly ONE is kept -- not summed. A region \
carries THREE PRESENTATIONS OF ONE TAKE, decoded disc-wide by counting stream \
starts inside every inter-descriptor span: 258 spans hold one stream and 28 hold \
three, and nothing holds two. This exporter read these chunks wrongly twice \
before that landed -- first concatenating them (359 s for a 137 s movie), then \
summing them as HANDOFF Q10's two stems, which its own measurements refuted: \
S00A's second full-length chunk is DIGITAL SILENCE and ADV's is 0.60x the first \
with 26.8 dB of residual. Summing a take with a scaled copy of itself adds ~4 dB \
and colours it. 🟡 WHICH of the equal-duration survivors is kept is a CHOICE, \
not a decoded field -- authored/audio.json voice.presentation = {:?}, with its \
why. Nothing on the disc ranks the presentations: wEncodeOptions, channel count \
and channel mask are byte-identical across them. One capture of the movie with \
dialogue audible deletes that entry.{} Folded to mono from the {} of {channels} declared \
channel(s) that carry signal -- channel 2 of both voice streams is digitally \
silent, and averaging it in cost 5.94 dB until this was measured rather than \
read off the declared count.{against}",
riffs.len(),
match presentation {
Presentation::Loudest => "loudest",
Presentation::HighestRate => "highest_rate",
},
if dropped.is_empty() {
String::new()
} else {
format!(
" DROPPED, and NOT as junk -- the leading chunk is DECODED to be this \
movie's OWN dialogue, 17 of 17 regions (docs/re/structures/\
voice-region-leading-chunk.md; an earlier note here wrongly equated it \
with BGM_103's third sub-wave, which a disc-wide census showed is a \
different structure). This port then measured it to be the TAIL of the \
kept stream -- sliding envelope correlation r=0.998 (ADV) and 0.932 \
(S00A), the lag placing it flush against that stream's end, against \
controls of 1.000 self and 0.289 for a different movie -- so dropping it \
removes a DUPLICATE, not dialogue. Dropped: {}.",
dropped.join(", ")
)
},
live.len()
),
peak_dbfs: peak,
duration_s: dur,
kind: "voice",
name_match: None,
loop_mode: None,
sub_waves: riffs.len(),
}))
}
/// How many channels a staged `RIFF` declares, per ffprobe.
fn probe_channels(path: &Path) -> Option<u8> {
let out = Command::new("ffprobe")
.args([
"-v", "error", "-select_streams", "a:0",
"-show_entries", "stream=channels", "-of", "csv=p=0",
])
.arg(path)
.output()
.ok()?;
String::from_utf8_lossy(&out.stdout).trim().parse().ok()
}
/// Seconds a finished media file runs, per ffprobe.
///
/// Exposed so the caller can hand [`export_voice`] the movie's own length: the
/// voice is a separate asset with no shared container to agree with, so the only
/// way a resolution error shows up is a length that does not match the picture.
pub fn probe_duration(path: &Path) -> Option<f32> {
let out = Command::new("ffprobe")
.args([
"-v", "error", "-show_entries", "format=duration",
"-of", "csv=p=0",
])
.arg(path)
.output()
.ok()?;
String::from_utf8_lossy(&out.stdout).trim().parse().ok()
}
/// Seconds and peak dBFS that one staged XMA `RIFF` decodes to.
///
/// XMA carries no duration in its header, so the chunk is decoded to PCM and the
/// result measured. That is expensive and it is the only instrument that can
/// separate these chunks at all: `ffprobe` on the `RIFF` itself returns `N/A`
/// for duration, which a caller that trusted it would read as zero, and the
/// corpus records `sylpheed-cli audio info` mis-reading the same headers as
/// 16 channels at 4 310 Hz.
///
/// A silent chunk returns `-inf`, which the caller drops.
fn decoded_chunk(riff: &Path) -> (f32, f32) {
let wav = riff.with_extension("probe.wav");
let ok = Command::new("ffmpeg")
.args(["-hide_banner", "-loglevel", "error", "-y", "-i"])
.arg(riff)
.arg(&wav)
.output()
.map(|o| o.status.success())
.unwrap_or(false);
let out = if ok {
let (peak, dur) = measure(&wav);
(dur.unwrap_or(0.0), peak.unwrap_or(f32::NEG_INFINITY))
} else {
(0.0, f32::NEG_INFINITY)
};
let _ = std::fs::remove_file(&wav);
out
}
/// Which channel indices of a decoded stream are not digitally silent.
///
/// `astats` reports per-channel blocks: a `Channel: N` line followed by that
/// channel's own `Peak level dB`. A channel whose peak is `-inf` carries
/// nothing, and folding it into an average is a pure loss.
///
/// Falls back to "every declared channel is live" if the parse finds nothing,
/// because the failure to prefer is the one that changes no level.
fn live_channels(riff: &Path) -> Vec<usize> {
let wav = riff.with_extension("chan.wav");
let ok = Command::new("ffmpeg")
.args(["-hide_banner", "-loglevel", "error", "-y", "-i"])
.arg(riff)
.arg(&wav)
.output()
.map(|o| o.status.success())
.unwrap_or(false);
let mut live = Vec::new();
if ok {
if let Ok(out) = Command::new("ffmpeg")
.args(["-hide_banner", "-v", "info", "-i"])
.arg(&wav)
.args(["-af", "astats", "-f", "null", "-"])
.output()
{
let text = String::from_utf8_lossy(&out.stderr).into_owned();
let mut current: Option<usize> = None;
for line in text.lines() {
if let Some((_, n)) = line.split_once("Channel: ") {
// astats numbers channels from 1; `pan` addresses c0 upward.
current = n.trim().parse::<usize>().ok().map(|n| n.saturating_sub(1));
} else if let Some((_, v)) = line.split_once("Peak level dB: ") {
if let Some(c) = current.take() {
if v.trim().parse::<f32>().map(|p| p > -90.0).unwrap_or(false) {
live.push(c);
}
}
}
}
}
}
let _ = std::fs::remove_file(&wav);
if live.is_empty() {
live = (0..probe_channels(riff).unwrap_or(1) as usize).collect();
}
live
}