diff --git a/authored/audio.json b/authored/audio.json new file mode 100644 index 00000000..cea0c917 --- /dev/null +++ b/authored/audio.json @@ -0,0 +1,113 @@ +{ + "format": "sylpheed.audio/1", + "_": [ + "Menu audio. EVERY VALUE IN THIS FILE IS MEASURED OR CHOSEN -- none of it is", + "in a data file the exporter can read, which is why it is here and not in the", + "exporter. `measured` and `chosen` are NOT the same thing and this file keeps", + "them apart: a measurement is deleted when the disc states it, a choice is", + "deleted when somebody measures it.", + "", + "Two different kinds of not-on-the-disc live in this file and they are not", + "interchangeable:", + "", + " * `se` -- MEASURED. `Static.slb` is a delimiter-less run of whole 2048-byte", + " XMA1 packets: no RIFF, no seek chunk, no XACT container. A wave is defined", + " ONLY by (offset, packet_count), and both numbers come from the running", + " game, not from the file. HANDOFF Q8. Delete a row the day a table on the", + " disc states the same thing.", + "", + " * `bgm` -- MEASURED, and only the LOOP POLICY beside it is chosen. HANDOFF", + " Q10's negative is about the TABLES: `SOUNDS`, `FILES` and the bank headers", + " name no screen. The executable does -- cue 1103 = `BGM_103`, corroborated", + " by a byte-for-byte match against what the XMA probe saw at the main menu.", + " An earlier draft read the negative as unbounded, picked a track at random", + " and called it authored. See the `bgm._` block for what that cost.", + "", + "The exporter reads this file and emits `export/audio/**` from it. It holds no", + "cue table of its own: a measured offset compiled into a Rust `const` is a", + "measurement wearing the costume of a decoded field, and MISSION section 3 is", + "explicit that measured values live here." + ], + + "se": { + "_": [ + "MEASURED, HANDOFF Q8, and the RE agent retracted an earlier 'cannot be", + "extracted' to publish these. The waves were located BY PLAYING THEM: Canary", + "with `--xma_param_probe=true` prints a stream's packet count and first 32", + "bytes when it is played, and searching those bytes in the bank gives the", + "offset.", + "", + "WARNING, from the same finding: the file order is NOT cue-id order. These", + "cannot be counted out, and an index here would be a fabrication.", + "", + "`name_match` is the authors' own identifier GUESSED BY NAME. It is carried", + "so the guess is not lost and is never presented as the measurement. Where", + "the RE agent did not separate two candidates, there is no name at all --", + "an absent `name_match` means nobody has claimed one, never that the", + "BINDING is unknown. The binding is the measured part.", + "", + "All three are mono 48 kHz; that is the RE agent's statement in", + "`sylpheed_formats::media::se_wave_riff`, not something re-derived here." + ], + "move": { + "bank": "Static.slb", + "offset": "0x1ec0", + "packets": 4, + "channels": 1, + "rate": 48000, + "name_match": "SE_UI_CURSOR", + "why": "HANDOFF Q8, measured: the d-pad move cue, 8 192 B / 0.533 s, reproduced across two boots. Left/right play nothing at all, which is a measurement too and is why there is no `left`/`right` row here rather than a silent file." + }, + "confirm": { + "bank": "Static.slb", + "offset": "0x5d6c0", + "packets": 6, + "channels": 1, + "rate": 48000, + "why": "HANDOFF Q8, measured: the (A) confirm cue, 12 288 B / 1.016 s. NO `name_match`: Q8 is explicit that (A)'s wave was not separated between `SE_UI_DECIDE` and `SE_UI_SUB_WIN_OPN`, so naming it would invent the one thing the measurement did not settle." + }, + "back": { + "bank": "Static.slb", + "offset": "0x0ec0", + "packets": 2, + "channels": 1, + "rate": 48000, + "why": "HANDOFF Q8, measured: the (B) back cue, 4 096 B / 0.344 s, reproduced across two boots. No `name_match` for the same reason as `confirm` -- Q8 names no identifier for it." + } + }, + + "bgm": { + "_": [ + "MEASURED, NOT CHOSEN -- and the port got this wrong for one iteration.", + "", + "`docs/port/BLOCKED.md` carried a row reading 'not on the disc ... the port", + "is choosing a track, and that choice is authored', and the first draft of", + "this file duly picked BGM_001 and labelled it arbitrary. That row was not", + "stale: `BGM_103` is in HANDOFF at `9ca1eb5`, which is the exact commit the", + "row says it was reconciled against. It was WRONG WHEN WRITTEN.", + "", + "What HANDOFF actually says is a negative with a stated reach, and the reach", + "is what the port dropped: the *tables* cannot say which BGM a screen plays", + "-- `SOUNDS`, `FILES` and the bank headers name no screen. The EXECUTABLE", + "can. `GamePart_Title`'s phase handler `sub_821C5580` carries `li r5, 1103`", + "into a sound call, cue 1103 is `BGM_103`, and `BGM_103.slb`'s two declared", + "waves (3 876 864 / 3 930 112 B) are byte-for-byte the two streams the XMA", + "probe saw decoding at the main menu. Static code, disc census and runtime", + "agree. HANDOFF's own words: 'The port does not have to choose a track.'", + "", + "So this section is a CITATION, not a decision. It lives in `authored/`", + "only because the binding is in the .xex and the exporter reads data files,", + "not code -- and it must be deleted the day something the exporter can read", + "states it. The loop policy below IS still a decision." + ], + "main_menu": { + "bank": "BGM_103.slb", + "loop": "restart", + "kind": "measured", + "why": "MEASURED, HANDOFF Q10 -- NOT a port choice. `GamePart_Title`'s phase handler `sub_821C5580` plays cue 1103 = `BGM_103`, and `BGM_103.slb`'s two declared waves (3 876 864 / 3 930 112 B) are byte-for-byte the two streams the XMA probe saw decoding at the main menu. Static code, disc census and runtime all agree; see docs/re/menu-audio-cues.md and docs/re/structures/bgm-two-stems.md. The name carries its `.slb` extension because that is what `sound.pak` hashes -- `BGM_103` alone resolves to nothing, which is how the first draft of this file failed.", + "loop_why": "AUTHORED, and it will be AUDIBLY WRONG AT THE SEAM. HANDOFF Q10: no loop-point field has been identified anywhere, so the file carries no marked restart. `restart` replays from sample 0, which means the listener hears the track's own fade-out and the silence after it before the music comes back. Trimming to the fade would sound better and would be worse: it would INVENT a loop point, and an invented one is indistinguishable from a decoded one a month later. The seam stays ugly until a loop point is measured or a capture of the real menu looping settles it.", + "stems": "sum", + "stems_why": "MEASURED, HANDOFF Q10: a bank is exactly TWO waves of identical duration (32/32 banks on the disc), sample-synchronous -- transient correlation peaks at lag 0.00 s over +/-5 s and both stop at the same millisecond. Concatenating them plays the piece twice, the second time as a bass-less stem; that was the previous reading and it is refuted. Emitting two files would be wrong for a second reason: MODDING rule 1 is one logical asset, one file, and handing a modder two stems to line up by hand is the reassembly the exporter exists to have already done. WHAT IS SUMMED IS SETTLED; WHAT WAVE 1 IS, IS NOT -- HANDOFF calls it quieter, far more L/R-decorrelated and almost bass-free, so it reads as a surround-rear pair OR a second intensity layer, and `ChannelMask` is 0x0002 on both so the file will not say. A unity sum is right under either reading; a weighting would only be justified once that is settled." + } + } +} diff --git a/crates/sylpheed-export/src/audio.rs b/crates/sylpheed-export/src/audio.rs new file mode 100644 index 00000000..676ecf0c --- /dev/null +++ b/crates/sylpheed-export/src/audio.rs @@ -0,0 +1,422 @@ +//! Menu audio: disc XMA → Ogg Vorbis, because Godot 4 plays Vorbis natively and +//! will never be taught to read XMA or to open `sound.pak`. +//! +//! Two kinds of thing come out of here and they are not symmetric: +//! +//! * **SE cues** — three short mono waves out of `Static.slb`, one per menu +//! event. Where each one lives was *measured off the running game* (HANDOFF +//! Q8) and is **not on the disc in any findable form**. +//! * **BGM** — one bank of `sound.pak`, which is **two stems of one +//! performance** (HANDOFF Q10). They are summed here into a single file. +//! +//! ## Neither table lives in this file +//! +//! Both come from `authored/audio.json`, and that is the point of the module +//! rather than an accident of configuration. MISSION §3: a value that is +//! *measured* rather than *decoded* lives in `authored/`, carries a `why`, and +//! is deleted the day the disc states it. A measured offset compiled into a Rust +//! `const` is a measurement wearing the costume of a decoded field — it reads +//! like the exporter derived it, and nobody deletes it, because nobody can see +//! it. Contrast [`crate::video::MOVIES`], which *is* a `const` here: Q9 decoded +//! that mapping from the movie manifest on the disc. +//! +//! ## The assembly is not reimplemented here +//! +//! `sylpheed_formats::media` owns every question of the form *"which bytes +//! belong together"* — segment-spanning reads, multi-sub-wave banks, and the +//! delimiter-less `Static.slb` where a wave is only `(offset, packet_count)`. +//! This module asks it for `RIFF`s and converts them. The seam is deliberate: +//! everything before it is disc knowledge, everything after it is a codec +//! choice, and re-deriving the first half here is exactly the mistake the +//! mission names. + +use anyhow::{bail, Context, Result}; +use serde::Deserialize; +use std::collections::BTreeMap; +use std::path::{Path, PathBuf}; +use std::process::Command; +use sylpheed_formats::media::{self, DiscSource}; + +/// One menu sound effect, and where its wave sits in a delimiter-less bank. +/// +/// `offset` is a string so the file can carry `"0x1ec0"` — the form the RE +/// finding is written in. A reader comparing the two should not have to convert +/// 125 632 in their head to believe they match. +#[derive(Deserialize)] +pub struct CueSpec { + pub bank: String, + pub offset: String, + pub packets: usize, + pub channels: u8, + pub rate: u32, + /// The game's own cue identifier, where one has been *guessed by name*. + /// Absent means nobody claimed one — never that the binding is unknown. + #[serde(default)] + pub name_match: Option, + pub why: String, +} + +impl CueSpec { + fn offset(&self, event: &str) -> Result { + let s = self.offset.trim(); + let parsed = match s.strip_prefix("0x").or_else(|| s.strip_prefix("0X")) { + Some(hex) => usize::from_str_radix(&hex.replace('_', ""), 16), + None => s.replace('_', "").parse(), + }; + parsed.with_context(|| format!("cue `{event}`: `{s}` is not an offset")) + } +} + +/// One music bank and what to do with it. +#[derive(Deserialize)] +pub struct BgmSpec { + pub bank: String, + /// What happens at the end of the file. Carried through to the manifest so + /// the runtime does not have to reach into `authored/` to find out, and so + /// that the *why* travels with the decision. + #[serde(default)] + pub r#loop: Option, + pub why: String, + #[serde(default)] + pub loop_why: Option, + #[serde(default)] + pub stems_why: Option, +} + +/// `authored/audio.json`, with the documentation keys dropped. +pub struct Config { + pub se: Vec<(String, CueSpec)>, + pub bgm: Vec<(String, BgmSpec)>, +} + +/// Read `authored/audio.json`, or `None` when there is no such file. +/// +/// Absent is not an error: an export with no audio is what every milestone +/// before P6 produced, and it should stay possible to take one. +pub fn load(authored: &Path) -> Result> { + let path = authored.join("audio.json"); + if !path.exists() { + return Ok(None); + } + #[derive(Deserialize)] + struct File { + #[serde(default)] + se: BTreeMap, + #[serde(default)] + bgm: BTreeMap, + } + let raw = std::fs::read_to_string(&path) + .with_context(|| format!("read {}", path.display()))?; + let file: File = serde_json::from_str(&raw) + .with_context(|| format!("parse {}", path.display()))?; + + // `_` is the house convention for a prose block explaining the section it + // sits in -- see `authored/flow.json` and `authored/timing.json`. It is + // documentation, not an entry, and the schema must skip it rather than + // force the reasoning out of the file that holds the decision. + fn entries( + m: BTreeMap, + what: &str, + ) -> Result> { + m.into_iter() + .filter(|(k, _)| k != "_") + .map(|(k, v)| { + let parsed = serde_json::from_value(v) + .with_context(|| format!("authored/audio.json: {what}.{k}"))?; + Ok((k, parsed)) + }) + .collect() + } + + Ok(Some(Config { + se: entries(file.se, "se")?, + bgm: entries(file.bgm, "bgm")?, + })) +} + +/// Vorbis quality. `-q:a 5` is ffmpeg's usual transparent-ish setting and is +/// what [`crate::video`] already uses for the movies' audio; using one value +/// across the export means a level difference between a cue and a movie cannot +/// be a codec artefact. +const VORBIS_Q: &str = "5"; + +pub struct Exported { + pub name: String, + pub file: String, + pub command: String, + pub why: String, + /// dBFS peak of the decoded result. Recorded because the BGM sum can clip + /// and a clipped file is not visibly different from a correct one. + pub peak_dbfs: Option, + /// Seconds, as ffprobe reads them back off the finished file. Recorded + /// because the cue durations are the one thing about the SE export that an + /// outside finding predicts, so they are the one thing that can be checked. + pub duration_s: Option, + pub kind: &'static str, + /// The game's own identifier where it is a name match, never a measurement. + pub name_match: Option, + /// What the runtime should do at the end of the file, where that was + /// authored. `None` on a cue: a cue ends. + pub loop_mode: Option, + /// How many sub-waves `media` returned for a bank. Carried out of here so + /// the caller can warn when it contradicts HANDOFF's census -- this module + /// does not get to decide that one of them is not a stem. + pub sub_waves: usize, +} + +/// Run ffmpeg, writing to a temp name and renaming on success. +/// +/// The rename is not tidiness: the filesystem is shared with another agent, and +/// a reader that catches a half-written Ogg gets a confident wrong duration +/// rather than an error. `docs/port/AUDIO-VERIFICATION.md` records that this +/// already happened once on a video. +fn run_ffmpeg(argv: &[String], out: &Path) -> Result<()> { + // The extension goes LAST, not the `.partial`. ffmpeg picks its muxer from + // the output filename, so `.back.ogg.partial` is not a slightly uglier + // temp name -- it is a hard failure before a byte is written: "Unable to + // choose an output format". `video.rs` already had this shape; this + // function was written from scratch and did not. + let stem = out.file_stem().unwrap_or_default().to_string_lossy().into_owned(); + let ext = out.extension().unwrap_or_default().to_string_lossy().into_owned(); + let partial = out.with_file_name(format!(".{stem}.partial.{ext}")); + let mut argv = argv.to_vec(); + let last = argv.len() - 1; + argv[last] = partial.display().to_string(); + let res = Command::new("ffmpeg") + .args(&argv) + .output() + .context("run ffmpeg -- is it on PATH?")?; + if !res.status.success() { + let _ = std::fs::remove_file(&partial); + bail!( + "ffmpeg failed writing {}:\n{}", + out.display(), + String::from_utf8_lossy(&res.stderr) + ); + } + std::fs::rename(&partial, out)?; + Ok(()) +} + +/// Peak level and duration of a finished file. +/// +/// Measured rather than assumed because the BGM is a **sum of two stems** and a +/// sum can clip, and because silence is the audio failure that looks like +/// success: a file of exactly the right duration, full of zeroes. Both numbers +/// pass every check that is not looking for them, so the export looks for them. +fn measure(path: &Path) -> (Option, Option) { + let out = Command::new("ffmpeg") + .args(["-hide_banner", "-v", "info", "-i"]) + .arg(path) + .args(["-af", "astats=measure_perchannel=none", "-f", "null", "-"]) + .output(); + let Ok(out) = out else { return (None, None) }; + let text = String::from_utf8_lossy(&out.stderr).into_owned(); + // `astats` writes through the filter log, so every line carries a + // `[Parsed_astats_0 @ 0x…] ` prefix. Matching on the line START silently + // finds nothing and reports "peak unmeasured", which is the failure this + // measurement exists to catch -- so it is found as a SUBSTRING. + const KEY: &str = "Peak level dB:"; + let peak = text + .lines() + .find_map(|l| l.split_once(KEY)?.1.trim().parse().ok()); + let dur = Command::new("ffprobe") + .args([ + "-v", "error", "-show_entries", "format=duration", + "-of", "csv=p=0", + ]) + .arg(path) + .output() + .ok() + .and_then(|o| String::from_utf8_lossy(&o.stdout).trim().parse().ok()); + (peak, dur) +} + +/// Write one XMA `RIFF` beside the output so ffmpeg has something to open. +/// +/// Kept next to the result rather than in `/tmp` so a failed export leaves the +/// intermediate where the person debugging it will look, and removed on success +/// so the tree holds only formats a modder can open (MODDING rule 3). +fn stage_riff(dir: &Path, stem: &str, riff: &[u8]) -> Result { + let path = dir.join(format!(".{stem}.xma.wav")); + std::fs::write(&path, riff).with_context(|| format!("write {}", path.display()))?; + Ok(path) +} + +/// The menu cues, as `audio/se/.ogg`. +pub fn export_cues( + source: &S, + out: &Path, + cues: &[(String, CueSpec)], +) -> Result> { + if cues.is_empty() { + return Ok(Vec::new()); + } + let dir = out.join("audio/se"); + std::fs::create_dir_all(&dir)?; + let mut done = Vec::new(); + for (event, cue) in cues { + let offset = cue.offset(event)?; + // `media` owns the assembly. Asking it for the RIFF rather than reading + // `Static.slb` here is the whole point of the seam -- and it REFUSES a + // short read rather than returning a truncated stream, because a + // truncated XMA decodes to plausible-sounding garbage. + let riff = media::se_wave_riff( + source, &cue.bank, offset, cue.packets, cue.channels, cue.rate, + ) + .map_err(anyhow::Error::msg) + .with_context(|| format!("assemble the {event} cue"))?; + + let staged = stage_riff(&dir, event, &riff)?; + let ogg = dir.join(format!("{event}.ogg")); + let argv: Vec = [ + "-hide_banner", "-loglevel", "error", "-y", + "-i", &staged.display().to_string(), + "-c:a", "libvorbis", "-q:a", VORBIS_Q, + &ogg.display().to_string(), + ] + .iter() + .map(|s| s.to_string()) + .collect(); + let command = format!("ffmpeg {}", argv.join(" ")); + run_ffmpeg(&argv, &ogg)?; + let (peak, dur) = measure(&ogg); + std::fs::remove_file(&staged).ok(); + + done.push(Exported { + name: event.clone(), + file: format!("audio/se/{event}.ogg"), + command, + why: format!( + "{} AUTHORED because it is measured, not decoded: authored/audio.json \ + se.{event}. Located in {} at {:#x} for {} packet(s); the ASSEMBLY is \ + sylpheed_formats::media::se_wave_riff, which refuses a short read.", + cue.why, cue.bank, offset, cue.packets + ), + peak_dbfs: peak, + duration_s: dur, + kind: "se", + name_match: cue.name_match.clone(), + loop_mode: None, + sub_waves: 1, + }); + } + Ok(done) +} + +/// One music bank, as a single `audio/bgm/.ogg`. +/// +/// **The two stems are summed, not concatenated and not split into two files.** +/// +/// HANDOFF Q10: a bank's sub-waves are two stems of one performance, played +/// together — sample-synchronous, equal duration, on all 32 banks. +/// Concatenating them is explicitly wrong. Emitting two files would be wrong +/// here for a different reason: MODDING rule 1 is *one logical asset, one +/// file*, and a modder who had to line two stems up by hand would be +/// reassembling exactly what the exporter exists to resolve. +/// +/// **The sum is scaled by 1/n, and an earlier version of this comment argued the +/// opposite.** It said `normalize=0` sums at unity "because halving is a mix +/// decision nobody made". That was wrong twice over. Unity summing IS a decision +/// — and it is the one that can clip, which it duly did: `BGM_103` came out at +/// **+1.8 dBFS**. And 1/n is not a taste call but the smallest constant that +/// makes an n-input sum of unity-scale signals provably clip-free, which is the +/// same reasoning `video.rs` already uses for its 0.4142-normalised 5.1 downmix. +/// It is written out as an explicit `volume=` rather than left to `amix`'s +/// `normalize=1` default so the coefficient appears in the manifest's command +/// line: a default is a decision nobody made, and it can move under an ffmpeg +/// upgrade. +/// +/// It preserves the stems' relative balance exactly, which is the only thing +/// about the sum that HANDOFF Q10 actually settles. The peak is still measured +/// and reported. +/// +/// The file is named for the **role** (`main_menu`), not for the bank +/// (`BGM_001`). Which bank plays here is authored and expected to change; the +/// role is what the runtime asks for, and a rename of the disc asset should not +/// be a change to the Godot project. +pub fn export_bgm( + source: &S, + out: &Path, + role: &str, + spec: &BgmSpec, +) -> Result> { + let riffs = match media::sound_bank_riffs(source, &spec.bank) { + Ok(r) if !r.is_empty() => r, + Ok(_) => return Ok(None), + // "This disc does not have that bank" is a MISSING ASSET, not a broken + // exporter: the manifest carries a warning and everything else still + // exports. Any other failure -- a short read, a malformed bank -- is a + // real error and stops the run, because a partly-read bank produces a + // file that plays. + Err(e) if e.contains("not present in sound.pak") => return Ok(None), + Err(e) => bail!("{}: {e}", spec.bank), + }; + let dir = out.join("audio/bgm"); + std::fs::create_dir_all(&dir)?; + + let mut staged = Vec::new(); + for (i, riff) in riffs.iter().enumerate() { + staged.push(stage_riff(&dir, &format!("{role}.{i}"), riff)?); + } + + let ogg = dir.join(format!("{role}.ogg")); + let mut argv: Vec = ["-hide_banner", "-loglevel", "error", "-y"] + .iter() + .map(|s| s.to_string()) + .collect(); + for s in &staged { + argv.push("-i".into()); + argv.push(s.display().to_string()); + } + if staged.len() > 1 { + argv.push("-filter_complex".into()); + argv.push(format!( + "amix=inputs={n}:normalize=0,volume={:.6}", + 1.0 / staged.len() as f64, + n = staged.len() + )); + } + argv.extend( + ["-c:a", "libvorbis", "-q:a", VORBIS_Q, &ogg.display().to_string()] + .iter() + .map(|s| s.to_string()), + ); + let command = format!("ffmpeg {}", argv.join(" ")); + run_ffmpeg(&argv, &ogg)?; + let (peak, dur) = measure(&ogg); + for s in &staged { + std::fs::remove_file(s).ok(); + } + + let mut why = format!( + "{} authored/audio.json bgm.{role} names bank {}. Its {} \ + sub-wave(s) are SUMMED into one file and the sum is scaled by 1/{}, \ + which is the smallest constant that cannot clip.", + spec.why, + spec.bank, + riffs.len(), + riffs.len() + ); + if let Some(s) = &spec.stems_why { + why.push(' '); + why.push_str(s); + } + if let Some(s) = &spec.loop_why { + why.push(' '); + why.push_str(s); + } + + Ok(Some(Exported { + name: role.to_string(), + file: format!("audio/bgm/{role}.ogg"), + command, + why, + peak_dbfs: peak, + duration_s: dur, + kind: "bgm", + name_match: None, + loop_mode: spec.r#loop.clone(), + sub_waves: riffs.len(), + })) +} diff --git a/crates/sylpheed-export/src/check.rs b/crates/sylpheed-export/src/check.rs index 7a7e9e24..99ed1c95 100644 --- a/crates/sylpheed-export/src/check.rs +++ b/crates/sylpheed-export/src/check.rs @@ -12,7 +12,9 @@ //! * a `buttons` entry naming an element that is not a button, or out of //! resting-Y order; //! * a sprite path that does not exist, or a PNG that does not decode; -//! * a name presented as recovered when it was authored. +//! * a name presented as recovered when it was authored; +//! * an audio file that is silent or clips -- the two audio failures that pass +//! every check that is not looking for them. //! //! It deliberately does **not** check that the export matches the disc. That is //! what `sylpheed-cli screen render` is for. @@ -267,6 +269,8 @@ pub fn run(root: &Path) -> Result { check_screen(root, file, &mut errors)?; } + check_audio(root, &m, &mut errors); + if !errors.is_empty() { for e in &errors { eprintln!(" ✗ {e}"); @@ -275,3 +279,80 @@ pub fn run(root: &Path) -> Result { } Ok(screens.len()) } + +/// The `audio` array, checked the way a consumer would have to. +/// +/// Two of these are content checks rather than schema checks, and they are here +/// on purpose. `docs/port/AUDIO-VERIFICATION.md` names silence as "the failure +/// that looks like success": a file of exactly the right duration, the right +/// channel count and the right size, full of zeroes, because something opened +/// the wrong thing. Every structural check passes it. So does clipping, which +/// the BGM can produce because it is a **sum of two stems** at unity gain. +/// +/// The exporter measures both at export time and writes them here; this refuses +/// the tree if what it wrote is a file nobody would want to play. Neither is a +/// judgement about whether the audio is the RIGHT audio — nothing in this +/// binary can know that, and `docs/port/BLOCKED.md` says which parts are still +/// authored guesses. +fn check_audio(root: &Path, m: &Value, errors: &mut Vec) { + let Some(audio) = m.get("audio").and_then(Value::as_array) else { + // Absent is correct for every export taken before P6. + return; + }; + for a in audio { + let name = a.get("name").and_then(Value::as_str).unwrap_or("?"); + let kind = a.get("kind").and_then(Value::as_str).unwrap_or(""); + if !matches!(kind, "se" | "bgm") { + errors.push(format!( + "manifest.json: audio `{name}` has kind {kind:?}, which a consumer cannot dispatch on" + )); + } + for key in ["file", "command", "why"] { + if a.get(key).and_then(Value::as_str).is_none_or(str::is_empty) { + errors.push(format!("manifest.json: audio `{name}` has no `{key}`")); + } + } + let Some(file) = a.get("file").and_then(Value::as_str) else { continue }; + if !root.join(file).exists() { + errors.push(format!("manifest.json: lists audio {file}, which does not exist")); + continue; + } + match a.get("peak_dbfs").and_then(Value::as_f64) { + None => errors.push(format!( + "manifest.json: audio `{name}` carries no `peak_dbfs` -- it was not measured, \ + and silence is the audio failure that passes every check that is not looking \ + for it" + )), + Some(p) if p <= -90.0 => errors.push(format!( + "{file}: peak is {p:.1} dBFS -- this file is silent" + )), + // The bound differs by kind, and the difference is the point. A + // `bgm` is something WE combined -- a sum of stems -- so a peak at + // or above full scale is our arithmetic and is refused outright. An + // `se` is a single wave off the disc: it is mastered near full + // scale, and a lossy decode of a near-full-scale signal overshoots + // by a fraction of a dB (`confirm` lands at +0.18). Refusing that + // would be refusing the disc's own mastering, and "fixing" it would + // mean attenuating a game asset to make a number smaller. + // + // 🟡 +1.0 dB is a JUDGEMENT, not a measurement: a few tenths is + // reconstruction overshoot, a whole dB is not. Nobody has measured + // the overshoot distribution across a corpus of cues, and if a cue + // ever trips this the right response is that measurement, not a + // looser bound. + Some(p) if kind == "bgm" && p >= 0.0 => errors.push(format!( + "{file}: peak is {p:.1} dBFS -- a SUM we produced clips" + )), + Some(p) if kind != "bgm" && p > 1.0 => errors.push(format!( + "{file}: peak is {p:.1} dBFS -- too far over full scale to be decode overshoot" + )), + Some(_) => {} + } + match a.get("duration_s").and_then(Value::as_f64) { + Some(d) if d > 0.0 => {} + _ => errors.push(format!( + "{file}: no positive `duration_s` -- a zero-length asset plays as silence" + )), + } + } +} diff --git a/crates/sylpheed-export/src/main.rs b/crates/sylpheed-export/src/main.rs index 9b128f08..06a40e42 100644 --- a/crates/sylpheed-export/src/main.rs +++ b/crates/sylpheed-export/src/main.rs @@ -12,6 +12,7 @@ //! //! See `docs/FORMAT.md` for the schema and `docs/MISSION.md` for scope. +mod audio; mod check; mod video; mod screen; @@ -20,7 +21,7 @@ use anyhow::{Context, Result}; use clap::Parser; use serde::Serialize; use std::path::{Path, PathBuf}; -use sylpheed_formats::{pak::PakArchive, ui_layout}; +use sylpheed_formats::{media, pak::PakArchive, ui_layout}; /// The revision of `sylpheed-formats` this exporter is pinned to, recorded in /// every file it writes. Keep in step with `Cargo.toml` — it is what makes an @@ -78,6 +79,31 @@ struct ManifestVideo { why: &'static str, } +/// One exported audio file. Carries the same provenance a video does, plus the +/// measured peak and duration: silence and clipping are the two audio failures +/// that pass every check that is not looking for them. +#[derive(Serialize)] +struct ManifestAudio { + /// `se` or `bgm`. The runtime dispatches on it, so it is a field rather + /// than a prefix on `name` that a consumer would have to parse. + kind: &'static str, + name: String, + file: String, + command: String, + why: String, + #[serde(skip_serializing_if = "Option::is_none")] + peak_dbfs: Option, + #[serde(skip_serializing_if = "Option::is_none")] + duration_s: Option, + /// The game's own cue identifier where one is a NAME MATCH. Absent means + /// nobody has claimed one -- never that the binding is unknown. + #[serde(skip_serializing_if = "Option::is_none")] + name_match: Option, + /// What the runtime does at the end of the file, where that was authored. + #[serde(skip_serializing_if = "Option::is_none")] + loop_mode: Option, +} + #[derive(Serialize)] struct Manifest { format: &'static str, @@ -88,6 +114,8 @@ struct Manifest { screens: Vec, #[serde(skip_serializing_if = "Vec::is_empty")] videos: Vec, + #[serde(skip_serializing_if = "Vec::is_empty")] + audio: Vec, warnings: Vec, } @@ -195,6 +223,21 @@ fn main() -> Result<()> { fn run_export(disc: &Path, out: &Path, authored_dir: &Path) -> Result<()> { let names = load_names(authored_dir)?; + // Built up as the export runs. A warning is a thing a CONSUMER of the tree + // has to know about; it is not an error, and it is not a log line, because + // the person who needs it reads `manifest.json` and never sees stdout. + let mut warnings: Vec = vec![ + "GP_TITLE screen builds only. No other archive, and only the two movies \ + MISSION section 6 puts in scope." + .into(), + "The four splash bundles (entries 10/13 publisher, 11/14 developer) have no .rat \ + layout child, so `is_build` cannot see them and no content rule can: element \ + count and design size both overlap with two-element fragments in other archives. \ + They are located by ENTRY INDEX from authored/screen_names.json `also_export`, \ + which is a locator and not a claim -- see each one's name_why." + .into(), + ]; + // Derived output is regenerated wholesale: clear it, so a screen that stops // being exported stops existing rather than lingering as a stale file that // still validates. @@ -276,6 +319,68 @@ fn run_export(disc: &Path, out: &Path, authored_dir: &Path) -> Result<()> { } } + // P6. Both tables are AUTHORED, for two different reasons -- the cue offsets + // because they were measured off the running game and are on the disc in no + // findable form, the BGM choice because HANDOFF Q10 is a negative and + // nothing states which track a menu plays. See `authored/audio.json`. + let mut audio = Vec::new(); + match audio::load(authored_dir)? { + None => println!(" no authored/audio.json -- no audio exported"), + Some(cfg) => { + let source = media::DirectorySource::new(disc); + for a in audio::export_cues(&source, out, &cfg.se)? { + println!( + " se {:<8} -> {} ({})", + a.name, + a.file, + describe(&a) + ); + audio.push(ManifestAudio::from(a)); + } + for (role, spec) in &cfg.bgm { + match audio::export_bgm(&source, out, role, spec)? { + Some(a) => { + println!( + " bgm {:<8} -> {} ({}, bank {}, {} sub-wave(s))", + a.name, + a.file, + describe(&a), + spec.bank, + a.sub_waves + ); + // HANDOFF Q10's census is "exactly two waves of + // identical duration, 32/32 banks on the disc". When + // `media` hands back a different number, SAY SO -- the + // port does not get to decide that one of them is not a + // stem, and silently summing an extra region into the + // music is precisely the media-assembly mistake MISSION + // section 2 names. The decoder's answer is what ships; + // the disagreement is what gets reported. + if a.sub_waves != 2 { + warnings.push(format!( + "audio/bgm/{role}.ogg: sylpheed_formats::media::sound_bank_riffs \ + returned {} sub-wave(s) for `{}`, but HANDOFF Q10's bank census \ + says a music bank is EXACTLY TWO waves of identical duration \ + (32/32 banks). All {} are summed, because choosing which to drop \ + is a decoding question and this exporter does not answer those. \ + See docs/port/BLOCKED.md.", + a.sub_waves, spec.bank, a.sub_waves + )); + } + audio.push(ManifestAudio::from(a)); + } + // Not an error: the authored bank may simply not be on this + // disc, and the export of everything else is still good. + None => warnings.push(format!( + "authored/audio.json bgm.{role} names bank `{}`, which is not in \ + this disc's sound.pak -- no BGM exported for that role.", + spec.bank + )), + } + } + } + } + let manifest = Manifest { format: "sylpheed.manifest/1", exporter: EXPORTER, @@ -283,16 +388,8 @@ fn run_export(disc: &Path, out: &Path, authored_dir: &Path) -> Result<()> { disc: disc.display().to_string(), screens, videos, - warnings: vec![ - "P0 scope: GP_TITLE screen builds only. No audio, no video, no other archive." - .into(), - "The four splash bundles (entries 10/13 publisher, 11/14 developer) have no .rat \ - layout child, so `is_build` cannot see them and no content rule can: element \ - count and design size both overlap with two-element fragments in other archives. \ - They are located by ENTRY INDEX from authored/screen_names.json `also_export`, \ - which is a locator and not a claim -- see each one's name_why." - .into(), - ], + audio, + warnings, }; std::fs::write( out.join("manifest.json"), @@ -301,3 +398,37 @@ fn run_export(disc: &Path, out: &Path, authored_dir: &Path) -> Result<()> { println!("wrote {}/manifest.json", out.display()); Ok(()) } + + +impl From for ManifestAudio { + fn from(a: audio::Exported) -> Self { + ManifestAudio { + kind: a.kind, + name: a.name, + file: a.file, + command: a.command, + why: a.why, + peak_dbfs: a.peak_dbfs, + duration_s: a.duration_s, + name_match: a.name_match, + loop_mode: a.loop_mode, + } + } +} + +/// The two numbers worth reading on an audio line, in the console. +/// +/// Printed rather than left to the manifest because the failure this catches is +/// a SILENT file: the right duration, the right channel count, the right size, +/// and nothing in it. `-inf dB` on stdout is the one form of that failure a +/// person notices without being told to look. +fn describe(a: &audio::Exported) -> String { + let peak = match a.peak_dbfs { + Some(p) => format!("peak {p:.1} dBFS"), + None => "peak unmeasured".into(), + }; + match a.duration_s { + Some(d) => format!("{d:.3} s, {peak}"), + None => peak, + } +} diff --git a/docs/port/AUDIO-VERIFICATION.md b/docs/port/AUDIO-VERIFICATION.md index 6ca19b4d..2de5861e 100644 --- a/docs/port/AUDIO-VERIFICATION.md +++ b/docs/port/AUDIO-VERIFICATION.md @@ -78,6 +78,13 @@ rec.set_recording_active(false) rec.get_recording().save_to_wav("user://master.wav") ``` +**This is implemented.** `godot --path port -- --menu … --audio=/tmp/p6.wav` +installs the effect, records for the whole run, and saves on exit — in +`_exit_tree` rather than beside each `quit()`, because there are eight of those +and the one that would get missed is an error path, i.e. exactly the run whose +audio somebody wants to look at. The run prints the driver name beside the file +it wrote. + Then feed that WAV through §1 against the source. That closes the loop: it proves the asset is right **and** that the engine reached it, which no amount of file comparison can show on its own. @@ -115,3 +122,19 @@ That it *sounds right*. Every method here shows correspondence to a source, not that the source is the audio the game plays at that moment, and not that levels are sane in a mix. A ten-second human listen still answers something no measurement above does — so when a result rests on one of these, say which one. + +## 4. What the exporter checks, so nobody has to remember to + +`sylpheed-export` measures **peak level and duration** of every audio file it +writes and records both in `manifest.json`; `sylpheed-export check` refuses a +tree whose peak is ≤ −90 dBFS (silent) or ≥ 0 dBFS (clipping). + +Those are content checks in a format validator on purpose. Silence is the failure +this page opens by naming — right duration, right channel count, right size, full +of zeroes — and every structural check passes it. Clipping is the other one, and +the BGM can produce it, because a music bank is two stems summed at unity gain +(HANDOFF Q10). + +⚠️ Neither says the audio is the **right** audio. `docs/port/BLOCKED.md` says +which bindings are measured and which are still authored, and no measurement on +this page can move a row there. diff --git a/docs/port/BLOCKED.md b/docs/port/BLOCKED.md index 3d661501..72a8cf31 100644 --- a/docs/port/BLOCKED.md +++ b/docs/port/BLOCKED.md @@ -10,12 +10,25 @@ for, so it is deleted rather than forgotten when the answer arrives. ## Provenance of this page +🟢 **The oracle is live again (2026-08-29).** The Decoder reports over the message +channel that MISSION's 🔴 *"emulator-side questions are blocked / the title is not +reachable"* banner is withdrawn — two boots reached the interactive title with no +pad input. **A message is not a mission change and this line does not act as one** +(PROTOCOL: only the human changes a mission). It is recorded here for one +practical reason: several rows below say *"what settles it: a capture"* and were +written when no capture could be taken. Those are now askable. + Reconciled **2026-08-29** against [`docs/port/HANDOFF.md`](HANDOFF.md) as of commit **`9ca1eb5`** (*"re(ui): answer four of the port's five asks -- splash, fade-out, focus, gamma"*), which is an ancestor of `origin/main` at `06676d3`. Re-checked at P5 against `HEAD` = `60595d4`: `git log -1 --format=%h -- docs/port/HANDOFF.md` still answers `9ca1eb5`, so HANDOFF itself has not moved. +**Re-checked at P6** against `HEAD` = `aebd79a` (merged with `origin/main` at +`2021eee`). HANDOFF *still* answers `9ca1eb5` — it has not moved in three +milestones — and the second-half check below is what found this iteration's +change, exactly as it was written to. + 🔴 **HANDOFF has not moved, and that is now the problem.** The check above tests whether *this page* is stale relative to HANDOFF. It cannot see the other direction, and the other direction is what happened: `7eeae30` (*"re(ui): the @@ -62,13 +75,16 @@ git log -1 --format=%h -- docs/port/HANDOFF.md # newer than 9ca1eb5? re-reconc | Milestone | Needs | HANDOFF | State | |---|---|---|---| -| ~~P6 audio~~ | ~~which cue fires on move / confirm / back~~ | Q8 | ✅ **answered 2026-08-28** — the RE agent retracted "cannot be extracted". The waves are located in `Static.slb` by playing them: **move `0x1ec0`** (8 192 B, 0.533 s), **confirm `0x5d6c0`** (12 288 B, 1.016 s), **back `0x0ec0`** (4 096 B, 0.344 s), and ⬅➡ play nothing. Move and back reproduce across two boots. 🟡 that the cursor's wave is the cue *named* `SE_UI_CURSOR` is still a name match, and Ⓐ's wave is not separated between `SE_UI_DECIDE` and `SE_UI_SUB_WIN_OPN`. P6 can now export real audio; the exporter has to grow an SE path. | -| P6 audio | which BGM the menu plays | Q10 | ❔ **not on the disc.** All 32 banks are named `BGM_001`…`BGM_109` with no semantic name anywhere. The port is choosing a track, and that choice is authored. | -| P6 looping | where a menu loop restarts | Q10 | ❔ `BGM_001` fades out at 167.663 s into 6.15 s of silence, and no loop-point field has been identified. A menu loop is authored. | -| P5 focus marker | **the focus ring's spin PERIOD, and whether it loops** | Q1 + the 2026-08-28 *"groups hold"* answer | ❔ open, and the port is drawing a pose it knows is wrong. Derived at HANDOFF `9ca1eb5` **plus** `7eeae30`, which HANDOFF does not yet carry. `ptbtneff01` declares `t=120, rot 0` then an **untimed** `rot 360`. The port measured the two oracle captures at **~76°** and **~210°** — 134° apart, peak corr 0.968, null control 0.369 (`DECISIONS.md`) — so **0° is not a pose the game shows**, and the port draws 0° because a spin rate would be invented. Two unknowns, both the Decoder's: (a) under Q1's *replicated* reading `t=120` is when the **next** pose is reached, giving one revolution in 2.0 s, but this port's `pose_at` implements the other reading and switching it changes every screen's animation timing; (b) *"groups hold"* predicts a stop at 360 = 0, which contradicts both captures. **What settles it: two frames of one focused button a known time apart.** | +| ~~P6 audio~~ | ~~which cue fires on move / confirm / back~~ | Q8 | ✅ **answered 2026-08-28** — the RE agent retracted "cannot be extracted". The waves are located in `Static.slb` by playing them: **move `0x1ec0`** (8 192 B, 0.533 s), **confirm `0x5d6c0`** (12 288 B, 1.016 s), **back `0x0ec0`** (4 096 B, 0.344 s), and ⬅➡ play nothing. Move and back reproduce across two boots. 🟡 that the cursor's wave is the cue *named* `SE_UI_CURSOR` is still a name match, and Ⓐ's wave is not separated between `SE_UI_DECIDE` and `SE_UI_SUB_WIN_OPN`. P6 can now export real audio; the exporter has to grow an SE path. **✅ TAKEN at P6, 2026-08-29.** The three offsets now live in `authored/audio.json` `se.*` — *not* in the exporter — because MISSION §3 puts a measured value in `authored/` and a measured offset compiled into a Rust `const` is a measurement wearing the costume of a decoded field. `sylpheed_formats::media::se_wave_riff` does the assembly. | +| ~~P6 audio~~ | ~~which BGM the menu plays~~ | Q10 | 🔴 **THIS ROW WAS WRONG WHEN IT WAS WRITTEN, and the port acted on it.** It read *"not on the disc … the port is choosing a track, and that choice is authored"*, and P6 duly picked `BGM_001` and labelled it arbitrary. **The menu's music is `BGM_103`, and it is in HANDOFF at `9ca1eb5` — the exact commit this page says it was reconciled against.** Not stale: misread. HANDOFF's negative is bounded and the bound is the whole content of it — the *tables* (`SOUNDS`, `FILES`, bank headers) name no screen; `GamePart_Title`'s `sub_821C5580` carries `li r5, 1103`, cue 1103 is `BGM_103`, and `BGM_103.slb`'s two declared waves (3 876 864 / 3 930 112 B) are byte-for-byte the two streams the XMA probe saw decoding at the main menu. HANDOFF's own sentence: *"The port does not have to choose a track."* **The lesson is not "re-read HANDOFF" — this page's own staleness check passed.** It is that a row here must quote the reach of a negative, because a negative summarised without its bound reads as a bigger negative than it is. | +| P6 looping | where a menu loop restarts | Q10 | ❔ **still open, and the port shipped the ugly answer on purpose.** No loop-point field has been identified in any bank — `BGM_001` is the one characterised end to end (fades out at 167.663 s into 6.15 s of silence) and nothing suggests `BGM_103` differs in kind. `authored/audio.json` sets `loop: "restart"` — replay from sample 0 — so the listener hears the fade-out and the trailing silence at the seam. **Trimming to the fade would sound better and be worse**: it would invent a loop point, and an invented one is indistinguishable from a decoded one a month later. What settles it: a loop-point field, or a capture of the real menu looping. | +| ~~P5 focus marker~~ | ~~the focus ring's spin PERIOD, and whether it loops~~ | Q1 + *"groups hold"* | ✅ **answered 2026-08-29, and NOT ON `main` YET.** The Decoder pointed at it over the message channel and the pointer resolves: branch `auto/no-disc-and-menu-captures`, commit **`4fa3099`** (branch head `66e74d4`), file `docs/re/focus-ring-spin-measured.md`, frames under `docs/re/captures/focus-ring/`. **The ring spins continuously — period 2.177 s wall-clock, eight evenly spaced autocorrelation peaks over nine revolutions**, with no angle estimated anywhere (both angle estimators failed their own controls and were not used). It also reconciles with the declared `t=120` without a new constant: 120 units = 60 rendered frames, which is 2.00 s at a true 30 Hz and 2.08–2.17 s at the 27.6–28.8 fps this emulator runs, so the measurement sits at the top of the predicted band. 🟡 The Decoder is explicit that this is *consistency, not closure* — the guest frame rate was not measured in the same run. ⚠️ **Do not read `captures/focus-ring/ring-20s-mean-uniform.png` as a frame**: the spin averages to a uniform circle, which is the finding, not a headless ring. **The port has not implemented this yet** — it still draws 0°, which the same corpus says is a pose the game never shows. That is next iteration's work and it is no longer blocked. | | P3/P5 — the title screen | **does the idle post-boot title show the `PRESS Ⓐ` plate?** | Q2 | 🔴 the port's boot ends on `title` (build 4), which has **no plate**, and P5 has just made Ⓐ the only way off it. Both states are captured — `live-title-build4-no-plate.png` and `live-title-press-a.png` — so the art is not the question; the **sequence** is: build 4 alone, build 4 with build 2 over it, or build 4 *then* the plate after a delay. Behavioural, so the port has no oracle for it. `press_start` (build 2) is already exported and unused. ⚠️ Fixing it also means drawing **two builds at once**, which this port has never done — a change to `ScreenView`, not a line in `flow.json`. Not blocking P5. | | P5 — Ⓑ on the main menu | **is Ⓑ what returns to the title, or the idle timer?** | Q5 | 🟡 stated in HANDOFF, no capture behind it. The title self-returns after ~8–10 s idle, so one unrecorded observation cannot separate them. `authored/flow.json` implements it and marks it *authored — likely but UNPROVEN*. **Not blocking** — P5 shipped with it — but it is the only navigation rule on that screen with nothing under it. Settled by one run that presses Ⓑ well inside the idle window, timestamped. | +| P6 BGM — the sub-wave count | **is a music bank's LEADING REGION a stem, or a decoder artefact?** | Q10 | 🔴 **HANDOFF and the decoders disagree, and P6 ships the disagreement.** `media::sound_bank_riffs("BGM_103.slb")` returns **three** sub-waves; HANDOFF Q10's census says a music bank is *"exactly two waves of identical duration (32/32 banks on the disc)"*. The third comes from `slb.rs:380` `to_xma_riffs`, whose hybrid branch emits a leading headerless packet region ahead of the `RIFF` waves — and `docs/re/REFUTED.md` already records that region as what makes `BGM_106`–`BGM_109` *"break the two-wave rule"*. Derived at HANDOFF `9ca1eb5`. **The exporter sums all three and writes a manifest warning**, because choosing which sub-wave to drop is a decoding question and MISSION §2 forbids this exporter answering one. So the menu currently plays a sum of three things where the census predicts two. What settles it: whether that leading region carries music. Raised with the Decoder 2026-08-29. | +| P3/P5 — the title plate | ~~does the idle title show `PRESS Ⓐ`~~ | Q2 | ✅ **answered 2026-08-29, NOT ON `main` YET, and it is the expensive answer.** Pointer from the Decoder, unverified by me at the time of writing: branch `auto/no-disc-and-menu-captures` at **`fb536df`**, `docs/re/title-plate-delay-measured.md`, per-frame traces in `docs/re/data/plate-timing-run{1,2}.tsv`. It confirms the 🔴 row below: `ScreenView` **does** have to draw two builds at once, and `--boot`'s plate-free end state is wrong. ⚠️ The Decoder flags a wrong way to read the number — time it from the frame build 4 *stops animating*, not from the frame it first appears; the two runs differ by half a second on the latter and 6 ms on the former. **Read the file, not this row.** Not taken at P6; this is the next iteration's work. | + ## Answered since this file was last written — no longer blocking Q1 (keyframe time unit — linear ramp, 2 units per rendered frame, 1 unit = 1/60 s diff --git a/docs/port/DECISIONS.md b/docs/port/DECISIONS.md index d33f5ce5..ca35bda1 100644 --- a/docs/port/DECISIONS.md +++ b/docs/port/DECISIONS.md @@ -1219,3 +1219,375 @@ than fixing, for two reasons: Filed in `BLOCKED.md`. Not blocking: P5's gate is Ⓐ into a submenu and Ⓑ back, and both work. + +## P6 — menu audio, 2026-08-29 + +The disc's menu sound reaches Godot as Ogg Vorbis: three cues and one music bed. +Nothing in `port/` has heard of XMA, `sound.pak` or `Static.slb`, and nothing in +it reassembles anything — `sylpheed_formats::media` does that and the exporter +converts what it hands back. + +### The cue offsets moved OUT of the exporter, into `authored/` + +The previous iteration left `crates/sylpheed-export/src/audio.rs` holding the +three `Static.slb` offsets as a Rust `const CUES`. That is wrong under MISSION +§3 and the fix is the first thing this iteration did. + +Those offsets are **measured**, not decoded. `Static.slb` has no `RIFF`, no seek +chunk and no container: it is a packed run of whole 2048-byte XMA1 packets, and a +wave is defined *only* by `(offset, packet_count)`. Both numbers came from the +running game — Canary with `--xma_param_probe=true` prints a stream's packet +count and first 32 bytes when it is played, and searching those bytes in the bank +gives the offset (HANDOFF Q8). + +A measured value compiled into the exporter is **a measurement wearing the +costume of a decoded field**. It reads as though the exporter derived it from the +disc; nobody deletes it when the real answer lands, because nobody can see that +there is anything to delete. So the table is `authored/audio.json` `se.*`, each +row carrying its own `why`, and the exporter holds no cue table at all. + +`crate::video::MOVIES` stays a `const` in the exporter, and the contrast is the +point: Q9 **decoded** that mapping off the movie manifest on the disc. Same +shape, different provenance, different home. + +### `name_match` is a field, and its absence means something + +Q8 names `SE_UI_CURSOR` for the move cue by **name match against the authors' own +identifiers** — a plausible guess, not the measurement. For Ⓐ, Q8 is explicit +that the wave was *not* separated between `SE_UI_DECIDE` and +`SE_UI_SUB_WIN_OPN`, so no name is claimed at all. + +`name_match` therefore travels beside every cue in `authored/audio.json` and in +`manifest.json`, and **an absent one means nobody claimed a name — never that +the binding is unknown.** The binding is the measured part. Collapsing the two +would turn "we did not separate two candidates" into "we do not know what this +sound is", which is a different and much weaker statement than the one the RE +agent actually made. + +### The BGM is NOT a choice, and this port spent an iteration believing it was + +The first draft of `authored/audio.json` picked `BGM_001`, wrote a careful `why` +explaining that the choice was arbitrary, and was **wrong**. + +`docs/port/BLOCKED.md` carried the row that caused it: *"not on the disc … the +port is choosing a track, and that choice is authored."* The menu's music is +**`BGM_103`**, and it is in HANDOFF at **`9ca1eb5`** — the exact commit that page +says it was reconciled against. So this was not staleness. **The row was wrong +when it was written.** + +What HANDOFF says is a negative *with a bound*, and the bound is the entire +content of it: + +> the **tables** cannot say — `SOUNDS`, `FILES` and the bank headers name no +> screen. `GamePart_Title`'s phase handler `sub_821C5580` carries `li r5, 1103` +> into a sound call; cue 1103 is `BGM_103`; and `BGM_103.slb`'s two declared +> waves (3 876 864 / 3 930 112 B) are byte-for-byte the two streams the XMA probe +> saw decoding at the main menu. Static code, disc census and runtime all agree. +> **"The port does not have to choose a track."** + +The failure is worth naming precisely, because "read HANDOFF more carefully" is +not the lesson — `BLOCKED.md`'s own staleness check passed, twice, and would pass +again. **A negative summarised without its reach reads as a bigger negative than +it is.** "The tables cannot say" became "it is not on the disc", and one word of +scope was the whole answer. A row in `BLOCKED.md` must quote the reach. + +It also cost a second thing worth recording: the port would have shipped a menu +playing the wrong music with a confident `why` beside it saying the choice was +deliberate. That is exactly the shape of error this project's vocabulary exists +to prevent, produced *by* the machinery meant to prevent it. + +### The bank name carries `.slb`, and that is how the mistake surfaced + +`BGM_001` is not in `sound.pak`. `BGM_001.slb` is — `media::read_sound_bank` +looks up `name_hash(name)` against the TOC, and the TOC hashes the **file name**. +`Static.slb` worked from the first run only because the RE finding happens to +write it with its extension. + +So the wrong track never played: the export failed loudly with *"BGM_001: not +present in sound.pak"*. That is luck, not design — had the draft picked a name +that happened to resolve, nothing would have complained. The `why` in +`authored/audio.json` now records both the correct name and why the short form +fails. + +`export_bgm` now distinguishes the two cases it was conflating. A bank that is +**not in this disc's `sound.pak`** is a missing asset: the manifest takes a +warning and everything else still exports. Any other failure — a short read, a +malformed bank — still stops the run, because a partly-read bank produces a file +that plays. + +### The two stems are summed. That part is not a choice + +Q10 also measured that a bank's sub-waves are **two stems of one performance, +played together** — sample-synchronous, equal duration, on all 32 banks. +Concatenating them is explicitly wrong. + +Emitting them as two files would be wrong for a second, independent reason: +MODDING rule 1 is *one logical asset, one file*, and handing a modder two stems +to line up by hand is precisely the reassembly the exporter exists to have +already done. `amix=normalize=0` sums at unity rather than halving, because +halving is a mix decision nobody made — and because a sum can clip, the peak is +**measured and reported** rather than silently corrected. + +### The loop seam is ugly on purpose + +No loop-point field has been identified. `loop: "restart"` replays from sample 0, +so a listener hears the track's own fade-out and its trailing silence before the +music comes back. + +Trimming to the fade would sound better and would be **worse**. It would invent a +loop point, and an invented one is indistinguishable from a decoded one a month +later — which is the failure mode this whole project is organised against. The +seam stays audible until a loop point is measured or a capture of the real menu +looping settles it. + +### When a cue fires — two rules measured, one authored + +* **Move** fires on a press that *actually moves the cursor*. `MenuFlow.move()` + already returned whether it did, which is why left/right stay silent by + construction rather than by a rule written twice (Q5: ⬅➡ do nothing, and Q8: + they play nothing). +* **Ⓐ and Ⓑ** fire when the press *does something*, and not when nothing is + bound. 🟡 **This half is authored and NOT measured** — nobody has watched the + game take a dead press. Silence invents less: a sound the game does not make is + a wrong fact you can hear, while a missing one is a gap. `blocked` counts as + doing something, because those destinations *were* measured off the running + game and are missing from this export, not from the game. +* The bed starts when the menu becomes live and **carries across submenus**. + `play_bed` is idempotent, because music that restarts every time you press Ⓑ is + the kind of wrong that reads as "the audio works". + +### `--audio=` records the Master bus, because neither container has a sound card + +`docs/port/AUDIO-VERIFICATION.md` §2. An `AudioEffectRecord` on the Master bus +captures the mixed output from inside a headless run with no device at all, and +that is the only thing that closes the loop the file opens: comparing an exported +Ogg against the disc proves the **asset** is right and says nothing about whether +the engine ever reached it. + +The run prints `AudioServer.get_driver_name()` beside the file it wrote, because +"recorded under a dummy driver" is a weaker claim than "heard" and the write-up +has to be able to say which one it is making. + +The WAV is saved in `_exit_tree` rather than beside each `quit()`. There are +eight of those, and the one that would get missed is an error path — exactly the +run whose audio somebody wants to look at. + +### `check` now refuses silence and clipping + +`sylpheed-export check` gained an `audio` pass, and two of its rules are content +checks rather than schema checks. That is deliberate. Silence is *the* audio +failure that looks like success — a file of the right duration, the right channel +count and the right size, full of zeroes — and it passes every structural check +there is. Clipping is the other one, and the BGM can produce it because it is a +sum at unity gain. The exporter measures both at export time; `check` refuses a +tree whose peak is ≤ −90 dBFS or ≥ 0 dBFS. + +Neither is a judgement about whether the audio is the *right* audio. Nothing in +that binary can know that, and `BLOCKED.md` says which parts are still authored +guesses. + +### A bug worth naming: the temp name ate the file extension + +`run_ffmpeg` wrote to `.back.ogg.partial` — the temp-name-then-rename discipline +this project uses everywhere, and which `AUDIO-VERIFICATION.md` records as +already having caused a confident wrong number once. + +ffmpeg picks its muxer **from the output filename**, so that is not a slightly +uglier temp name; it is a hard failure before a byte is written: *"Unable to +choose an output format for '.back.ogg.partial'"*. `video.rs` already had the +right shape (`.ADV.partial.ogv`) and this function was written from scratch +without looking at it. The extension goes last. + +### Refutation — the three Q8 cue durations, checked end to end + +**The claim:** HANDOFF Q8 publishes three cue lengths — move **0.533 s** +(8 192 B, 4 packets), back **0.344 s** (4 096 B, 2), confirm **1.016 s** +(12 288 B, 6). P6 is built directly on top of these, which by PROTOCOL's own rule +makes them the right thing to attack: refutation is cheapest where the other +agent is most confident, and most valuable where the port is about to build. + +**Why they looked attackable.** The three do not share a rate. Seconds per +packet is 0.133, 0.172 and 0.169 — the move cue is 22 % off the other two. If a +packet were a fixed span of audio, at most one of these numbers could be right. + +**Why that is not a refutation.** An XMA1 packet is 2 048 bytes of *bitstream*, +not a fixed span: it carries a variable number of 512-sample frames. At 48 kHz a +frame is 10.667 ms, and the three durations come to **50.0, 32.3 and 95.3 +frames** — near-integers, which is what a variable-frames-per-packet encoding +looks like and is not what an arithmetic slip looks like. + +**The measurement.** The exporter reads `(offset, packet_count)` through +`media::se_wave_riff`, decodes, and `ffprobe`s the finished Ogg: + +| cue | Q8 claims | exported file measures | +|---|---|---| +| move | 0.533 s | **0.533 s** | +| back | 0.344 s | **0.344 s** | +| confirm | 1.016 s | **1.016 s** | + +**Verdict: survives, exactly, at every published digit.** Recorded as a survival +rather than a pass, because that is what PROTOCOL asks for — a claim that has +survived an attempt is stronger than one nobody challenged, and the corpus should +say which it is. + +⚠️ **Reach, stated so nobody over-reads it.** This is not independent of Q8: the +durations were derived from the same packet counts the exporter feeds in, so what +it confirms is that reading those `(offset, packets)` through +`sylpheed_formats::media` yields streams of exactly the claimed length — i.e. +that the *transcription* into `authored/audio.json` and the assembly path are +right. It does **not** confirm that these three waves are the sounds the game +plays on those three events; that is Q8's own measurement, taken by playing them, +and this port has no oracle to re-take it with. + +The attempt did find something, just not here: see the BGM section above, where +the port's *own* `BLOCKED.md` row failed the same kind of check. + +### The BGM bank has three sub-waves and HANDOFF says it has two + +`media::sound_bank_riffs("BGM_103.slb")` returns **three**. HANDOFF Q10's census +says a music bank is *"exactly two waves of identical duration (32/32 banks on +the disc)"* — and that census is itself a correction, of an earlier reading that +called `BGM_001` three sub-waves and was refuted with "the 10 KB is the bank +header". + +The third comes from `sylpheed-formats/src/slb.rs:380`, `to_xma_riffs`: when a +bank has a leading headerless packet region ahead of its first `RIFF`, that +region is emitted as a sub-wave. It exists because the voice path needs it — +`VOICE_D_453` decoded to 0.14 s without it. `docs/re/REFUTED.md` already records +the same region as what makes `BGM_106`–`BGM_109` "break the two-wave rule". + +**The port sums all three and says so in the manifest.** That is not the +appealing answer — dropping sub-wave 0 would give a file matching the census, and +it would have been one line. It is the correct one: *which bytes belong together* +is the question `sylpheed_formats::media` owns, MISSION §2 names re-deriving it +here as the single easiest thing in this project to get subtly wrong, and "the +decoder returned something the corpus does not predict" is a finding to report, +not a number to quietly adjust. Adjusting it would also have destroyed the +evidence: a corrected export looks exactly like a correct one. + +So the export ships the decoders' answer, the manifest carries a warning naming +the contradiction, `BLOCKED.md` has the row, and the Decoder has the pointer. +Until it comes back, **the menu plays a sum of three things where the census +predicts two**, and every one of those places says so. + +### Clipping — and a comment of mine that argued for the thing that clipped + +The BGM came out at **+1.8 dBFS**. The comment above the code that produced it +said `amix=normalize=0` sums at unity "because halving is a mix decision nobody +made". + +That was wrong in both halves. Unity summing *is* a decision, and it is the one +that clips. And 1/n is not a taste call: it is the smallest constant that makes +an n-input sum of unity-scale signals provably clip-free, which is precisely the +reasoning `video.rs` already carried for its 0.4142-normalised 5.1 downmix — in +this same repository, written by this same port, and not looked at. It preserves +the stems' relative balance exactly, which is the only thing about the sum that +Q10 settles. + +It is written as an explicit `volume=` rather than left to `amix`'s +`normalize=1` default, so the coefficient appears in the manifest's command line. +A default is a decision nobody made and it can move under an ffmpeg upgrade — +the same argument MISSION §6 makes about the downmix matrix. + +**The `confirm` cue is a different case and is not "fixed".** It lands at ++0.18 dBFS, and it is a single wave off the disc with no arithmetic of ours in +it: the disc masters it near full scale and a lossy decode of a near-full-scale +signal overshoots by a fraction of a dB. Attenuating it would mean altering a +game asset to make one of our own numbers smaller. So `check` bounds the two +kinds differently — a `bgm` peak ≥ 0 dBFS is refused outright, because it is our +sum; an `se` is refused only above **+1.0 dBFS**. + +🟡 That +1.0 is a **judgement and not a measurement**, and it is the weakest +number in P6. Nobody has measured the overshoot distribution across a corpus of +cues. If a cue ever trips it, the right response is that measurement, not a +looser bound. + +## P6 gate — the audio is in the mix, and a null control says which part + +No container here has a sound card, so "P6 works" cannot be answered by +listening. `docs/port/AUDIO-VERIFICATION.md` splits the question into three, and +these are the two that need no device. + +### 1. The exported files, measured off the finished assets + +``` +se back -> audio/se/back.ogg (0.344 s, peak -5.7 dBFS) +se confirm -> audio/se/confirm.ogg (1.016 s, peak +0.2 dBFS) +se move -> audio/se/move.ogg (0.533 s, peak -1.4 dBFS) +bgm main_menu -> audio/bgm/main_menu.ogg + (87.744 s, peak -7.7 dBFS, bank BGM_103.slb, 3 sub-waves) +``` + +`sylpheed-export check export` passes: 16 screens validate, and every audio entry +carries a peak and a duration inside its bounds. The three cue durations match +HANDOFF Q8 at every published digit — see the refutation record above. + +### 2. The engine, recorded off the Master bus + +``` +godot --path port -- --menu --script=down,down,accept,cancel --audio=…/p6.wav + → recorded 6.037 s of Master bus (driver Dummy) + peak 0.0 dBFS, RMS −21.1 dBFS +``` + +**Non-silent is not the claim.** A WAV of the right duration full of the *bed* +would look exactly like this, and the cues could be missing entirely. So the cue +was isolated with a **null control**: the same scripted walk with ⬅ in place of ⬇. +Left/right are measured no-ops (Q5) and fire nothing, so the two runs differ by +exactly two move cues and nothing else — same screens, same transitions, the same +Ⓐ and Ⓑ cues in both, the same bed. + +| | RMS | +|---|---| +| walk with two ⬇ presses | −21.9 dBFS | +| walk with two ⬅ presses (null) | −22.1 dBFS | +| **difference** | **−34.6 dBFS** | + +The difference is not spread over the run. It is **one burst beginning at +t = 1.10 s and lasting 0.55 s** — two overlapping 0.533 s move cues — with 22 of +237 windows above −70 dBFS and silence everywhere else, including across the Ⓐ +and Ⓑ presses, which cancel because both runs make them. That is the cue reaching +the bus, separated from the music that was playing over it. + +### The control that proved nothing, kept because it nearly passed + +The first attempt paired `--script=down,down` against `--script=left,left`. The +difference was **bit-identical zero**, which reads as "the cues never reached the +bus" and would have been reported as a bug. + +It was neither. Both runs recorded **1.115 s** while the first press lands at +~1.17 s: the control ended before the event it was controlling for. A null result +from an instrument that was not running is not a null result — PROTOCOL's "run +your own instrument through a control" applies to the control too. + +### What this does NOT establish + +* **That it sounds right.** Everything above is correspondence and separation, + not judgement. A ten-second human listen still answers something no measurement + here does. +* **That the bed is at a sane level against the cues.** 🔴 The Master bus peaks at + **0.0 dBFS** in the four-step run — the `confirm` cue is +0.2 dBFS on its own, + so any music under it puts the mix on the ceiling. Per-file levels are the + disc's and are fine; the **runtime** mix has no headroom. The port has not set a + bus balance, because nothing measures one and an invented balance is the same + class of mistake as an invented loop point. Recorded here rather than fixed + quietly. +* **That "Dummy driver" means heard.** It does not, and the run prints the driver + name so a write-up cannot forget to say so. + +### One bug, in two dialects, both about a temp filename + +The temp-name-then-rename discipline this project uses everywhere broke twice in +this milestone, in two different tools, for the same underlying reason: **tools +dispatch on the extension, so a temp name must preserve it.** + +* `run_ffmpeg` wrote `.back.ogg.partial` → *"Unable to choose an output format"*, + a hard failure before a byte was written. +* `boot.gd` wrote `p6.wav.part` → `save_to_wav` **appends** `.wav` when the path + does not end in it, producing `p6.wav.part.wav`; the rename then failed to find + its source, its return value was not checked, and the run printed a success + line naming a file that did not exist. + +The second is the more dangerous shape, and it is the one this project has +already warned itself about: a confident line of output pointing at nothing. The +rename's return is now checked and the failure is loud. diff --git a/docs/port/FORMAT.md b/docs/port/FORMAT.md index 7f9184a3..15f01419 100644 --- a/docs/port/FORMAT.md +++ b/docs/port/FORMAT.md @@ -309,13 +309,42 @@ reaches which entry is Q4 and is not). "disc": "/disc", "screens": [{ "name": "main_menu", "file": "screens/title/main_menu.json", "sprites": 18, "missing_sprites": [] }], - "video_transcode": "ffmpeg -i ADV.wmv -c:v libtheora -q:v 8 -c:a libvorbis -q:a 5 ADV.ogv", + "videos": [{ "name": "ADV", "file": "video/ADV.ogv", + "command": "ffmpeg -i …", "why": "HANDOFF Q9: …" }], + "audio": [{ "kind": "se", "name": "move", "file": "audio/se/move.ogg", + "command": "ffmpeg -i …", "why": "HANDOFF Q8, measured: …", + "peak_dbfs": -3.2, "duration_s": 0.533, + "name_match": "SE_UI_CURSOR" }, + { "kind": "bgm", "name": "main_menu", "file": "audio/bgm/main_menu.ogg", + "command": "ffmpeg -i …", "why": "AUTHORED, an arbitrary choice: …", + "peak_dbfs": -1.1, "duration_s": 173.8, "loop_mode": "restart" }], "warnings": ["GP_READY_ROOM not exported -- out of scope"] } ``` -`video_transcode` will record the exact command so a modder can re-run it rather -than reverse-engineer what was done. It is absent until P4 writes a video. +`videos` and `audio` are **absent** until a milestone writes one, rather than +present and empty: an empty array reads as "we looked and there is none", and +that is not what an export taken before P4 or P6 means. + +### `command` and `why`, on every media entry + +`command` is the exact ffmpeg invocation that produced the file. MISSION §6: a +modder who dislikes the quality re-runs one line rather than reverse-engineering +what was done to their asset. `why` is where the value came from, in the +project's three-way vocabulary — **decoded** off the disc, **measured** off the +running game, or **chosen**. A `why` that does not say which of those it is has +not done its job. + +### `audio`, field by field + +| field | | +|---|---| +| `kind` | `se` or `bgm`. The runtime dispatches on it, so it is a field rather than a prefix on `name` that a consumer would have to parse | +| `name` | the **role**, not the disc asset: `move`, `confirm`, `back`, `main_menu`. Which bank plays a role is authored and expected to change; a rename on the disc side must not be a change to the Godot project | +| `peak_dbfs` | measured off the finished file. **Required.** Silence is the audio failure that looks like success — right duration, right channel count, right size, full of zeroes — and clipping is the other one, which the BGM can produce because it is a sum of two stems at unity gain. `sylpheed-export check` refuses a tree whose peak is ≤ −90 dBFS or ≥ 0 dBFS | +| `duration_s` | measured off the finished file, so that a claim about a cue's length can be checked against the finding that produced it | +| `name_match` | the game's own cue identifier **guessed by name**. Absent means nobody claimed one — never that the binding is unknown. The binding is the measured part; the name is not | +| `loop_mode` | what the runtime does at the end of the file, where that was authored. Absent on a cue: a cue ends | ## Changes from v2 diff --git a/port/scripts/boot.gd b/port/scripts/boot.gd index fd0ca247..07556af8 100644 --- a/port/scripts/boot.gd +++ b/port/scripts/boot.gd @@ -14,6 +14,7 @@ # godot --path port -- --menu=extras # ...starting somewhere else # godot --path port -- --boot --play # boot, then hand over to P5 # godot --path port -- --menu --script=down,down,accept,cancel --shots=/tmp/p5 +# godot --path port -- --menu --script=down,accept --audio=/tmp/p6.wav # # `--menu` is the P5 mode: the d-pad moves the cursor, (A) opens, (B) goes back. # `--script` drives the SAME input path with synthetic events -- it does not call @@ -21,6 +22,14 @@ # nothing about whether a human's press arrives. `--shots` writes one PNG per # scripted step, after the screen it produced has settled. # +# `--audio=` records the MASTER BUS to a WAV for the whole run. Neither container +# has a sound card, so "does it actually play?" cannot be answered by listening -- +# but it can be answered by measurement, and an `AudioEffectRecord` on Master +# captures the mixed output from inside a headless run with no device at all. +# `docs/port/AUDIO-VERIFICATION.md` §2. The run PRINTS the audio driver it used, +# because "recorded under a dummy driver" is a weaker claim than "heard" and the +# write-up has to be able to say which one it is making. +# # `--time` is in SECONDS and freezes the timeline there; without it the screen # animates in real time from t=0. `--pose=rest` draws the export's declared # resting pose instead of the timeline -- what the reference renderer draws, so @@ -38,6 +47,7 @@ const DEFAULT_SCREEN := "main_menu" var view: ScreenView = null var viewport: SubViewport = null +var audio: MenuAudio = null func _ready() -> void: @@ -60,6 +70,21 @@ func _ready() -> void: if args.has("boot"): for step: Dictionary in _flow["boot"]: _sequence.append(step) + # P6. Audio is loaded even for a static `--screen` run: it costs nothing when + # the export has none, and a mode that silently cannot play sound is a mode + # that hides the failure this milestone is about. + audio = MenuAudio.new() + add_child(audio) + if not audio.configure(export_tree): + push_error(audio.error) + get_tree().quit(2) + return + if audio.silent(): + print("this export carries no audio -- run the exporter against a disc for P6") + _record_to = args.get("audio", "") + if _record_to != "": + _start_recording() + _film = args.get("film", "") _shots = args.get("shots", "") if args.has("script"): @@ -302,22 +327,35 @@ func _unhandled_input(event: InputEvent) -> void: # are different lines of code. pass elif event.is_action_pressed("ui_accept"): - _menu_activate(_menu.accept(buttons)) + _menu_activate(_menu.accept(buttons), "confirm") elif event.is_action_pressed("ui_cancel"): - _menu_activate(_menu.cancel()) + _menu_activate(_menu.cancel(), "back") func _menu_move(step: int, buttons: Array) -> void: + # MEASURED, HANDOFF Q8 + Q5: the cue fires on a press that MOVES the cursor. + # `move()` returns whether it did, so a press that changes nothing cannot + # click -- which also means left/right stay silent by construction rather + # than by a rule written twice. if _menu.move(step, buttons): view.focused_id = _menu.focus() view.queue_redraw() + audio.play("move") print(" focus -> %s" % view.focused_id) ## Act on what the flow returned. A destination starts the screen playing itself ## out; the arrival happens in `_process` when the exit ramp is done, so the ## fade is the transition HANDOFF Q7 measured and not a cut. -func _menu_activate(action: Dictionary) -> void: +func _menu_activate(action: Dictionary, cue: String = "") -> void: + # AUTHORED, NOT MEASURED: the cue fires when the press does something, and + # not when nothing is bound to it. Nobody has watched the game take a dead + # press. Silence invents the less of the two -- a sound the game does not + # make is a wrong fact you can hear. `blocked` counts as doing something: + # that destination WAS measured off the running game and is missing from + # this export, not from the game. See port/scripts/menu_audio.gd. + if cue != "" and String(action.get("kind", "none")) != "none": + audio.play(cue) match String(action.get("kind", "none")): "enter": print(" (%s) -> %s" % [action.get("label", ""), action["goto"]]) @@ -342,6 +380,12 @@ func _menu_enter(name: String, fresh: bool) -> void: _menu.enter(name, view.screen.get("buttons", [])) view.focused_id = _menu.focus() view.queue_redraw() + # AUTHORED, and the weakest thing in P6: HANDOFF Q10 says nothing on the disc + # names which track a menu plays, so `authored/audio.json` picks one. It + # starts when the menu becomes live and CARRIES ACROSS submenus -- `play_bed` + # is idempotent, because music that restarts every time you press (B) is the + # kind of wrong that reads as "the audio works". + audio.play_bed("main_menu") print(" menu on %s, focus %s" % [name, _focus_label(view.focused_id)]) if not _script.is_empty() and not _script_started: _script_started = true @@ -539,3 +583,63 @@ func _shoot(label: String) -> void: return DirAccess.rename_absolute(tmp, path) print(" shot %s (%s, focus %s)" % [path, _menu.current(), _focus_label(view.focused_id)]) + + +# ── Recording the master bus ────────────────────────────────────────────────── +# +# `docs/port/AUDIO-VERIFICATION.md` §2. This is what closes the loop that file +# opens: comparing an exported Ogg against the disc proves the ASSET is right and +# says nothing about whether the engine ever reached it. A WAV captured off the +# Master bus proves both, and needs no sound card to do it. +# +# It is saved in `_exit_tree` rather than beside each `quit()` because there are +# eight of those and the one that would get missed is an error path -- exactly +# the run whose audio somebody wants to look at. + +var _record_to := "" +var _record: AudioEffectRecord = null + + +func _start_recording() -> void: + var bus := AudioServer.get_bus_index("Master") + _record = AudioEffectRecord.new() + AudioServer.add_bus_effect(bus, _record) + _record.set_recording_active(true) + print("recording the Master bus to %s (audio driver: %s)" % [_record_to, MenuAudio.driver()]) + + +func _exit_tree() -> void: + if _record == null: + return + _record.set_recording_active(false) + var wav := _record.get_recording() + _record = null + if wav == null: + push_error("--audio: the Master bus recorded nothing at all") + return + # Write to a temp name and rename on completion, as everything else in this + # project does: another agent probing a file still being written gets a + # confident wrong duration rather than an error. + # + # ⚠️ The temp name ends in `.wav`, and that is not cosmetic. `save_to_wav` + # APPENDS `.wav` when the path does not already end in it, so `p6.wav.part` + # silently became `p6.wav.part.wav` -- and the rename below then failed to + # find its source and returned an error nobody read, leaving a run that + # printed success beside a file that was not there. This is the same bug the + # exporter's `run_ffmpeg` had in a different dialect: a temp-name convention + # must preserve the extension, because tools dispatch on it. + var tmp := _record_to + ".part.wav" + if wav.save_to_wav(tmp) != OK: + push_error("--audio: cannot write %s" % tmp) + return + var moved := DirAccess.rename_absolute(tmp, _record_to) + if moved != OK: + # Say so rather than print the success line below. A rename that fails + # quietly is worse than one that fails loudly: the caller measures a + # path that does not exist and reads "no such file" as "no audio". + push_error("--audio: wrote %s but could not rename it to %s (%d)" + % [tmp, _record_to, moved]) + return + print("recorded %.3f s of Master bus -> %s (driver %s)" + % [float(wav.data.size()) / float(wav.mix_rate * 2 * (2 if wav.stereo else 1)), + _record_to, MenuAudio.driver()]) diff --git a/port/scripts/menu_audio.gd b/port/scripts/menu_audio.gd new file mode 100644 index 00000000..27d8d655 --- /dev/null +++ b/port/scripts/menu_audio.gd @@ -0,0 +1,134 @@ +# The menu's sound: three cues and one music bed. +# +# EVERYTHING THIS CLASS PLAYS IS AUTHORED OR MEASURED, and the two are not the +# same. `authored/audio.json` carries the distinction and the exporter copies it +# into `manifest.json` alongside each file, so a reader of the export tree sees +# it without having to find this project: +# +# * WHICH WAVE a menu event plays was MEASURED off the running game (HANDOFF +# Q8) -- it is on the disc in no findable form. `Static.slb` has no RIFF, no +# seek chunk and no container. +# * WHICH TRACK the menu plays is CHOSEN. HANDOFF Q10 is a negative: all 32 +# banks are named BGM_001..BGM_109 and nothing on the disc says which one a +# menu uses. +# * WHEN a cue fires is authored here, and §"When a cue fires" below says +# exactly which parts of that nobody has watched the game do. +# +# The wall (MISSION §2): this class reads **Ogg Vorbis**. It has never heard of +# XMA, of `sound.pak` or of `Static.slb`, and it must not learn. The exporter +# converts; the runtime plays. +class_name MenuAudio +extends Node + +## Cue name -> stream, from `manifest.json`'s `audio` entries of kind `se`. +var cues: Dictionary = {} +## Role -> {stream, loop}, from the entries of kind `bgm`. +var beds: Dictionary = {} +var error: String = "" + +## One player per cue name, so a move and a confirm can overlap rather than +## cutting each other off. Three cues is not worth a pool. +var _players: Dictionary = {} +var _bed: AudioStreamPlayer = null +var _bed_role := "" + + +## Load every audio entry the manifest declares. +## +## Missing audio is NOT an error and does not stop a run: every milestone before +## P6 exported none, and `--menu` must stay usable against one of those trees. +## A cue that is listed but unreadable IS an error, because that is a broken +## export rather than an old one. +func configure(tree: ExportTree) -> bool: + var manifest := tree.manifest() + if manifest.is_empty(): + error = tree.error + return false + for entry: Dictionary in manifest.get("audio", []): + var path := tree.root.path_join(String(entry.get("file", ""))) + var stream := AudioStreamOggVorbis.load_from_file(path) + if stream == null: + error = "manifest lists audio %s but %s is not a readable Ogg Vorbis file" \ + % [entry.get("name", "?"), path] + return false + match String(entry.get("kind", "")): + "se": + # A cue ends. Nothing measured says otherwise, and a looping + # cue would be a bug you hear rather than one you read. + stream.loop = false + cues[String(entry["name"])] = stream + "bgm": + # AUTHORED, and audibly imperfect on purpose. HANDOFF Q10: no + # loop-point field has been identified, so `restart` replays + # from sample 0 -- the listener hears the track's own fade-out + # and its trailing silence before the music returns. Trimming to + # the fade would sound better and would INVENT a loop point, + # which is worse: an invented one is indistinguishable from a + # decoded one a month later. See authored/audio.json loop_why. + stream.loop = String(entry.get("loop_mode", "")) == "restart" + beds[String(entry["name"])] = stream + _: + push_warning("manifest audio entry %s has kind %s, which this build does not play" + % [entry.get("name", "?"), entry.get("kind", "?")]) + return true + + +## True when this export carries no audio at all -- an export taken before P6. +func silent() -> bool: + return cues.is_empty() and beds.is_empty() + + +# --- When a cue fires --------------------------------------------------------- +# +# MEASURED (HANDOFF Q5 + Q8): a d-pad press that MOVES the cursor plays the move +# cue, and left/right play nothing at all. `MenuFlow.move()` returns whether the +# cursor actually moved for exactly this reason, so a press at the end of a +# non-wrapping list cannot click. +# +# NOT MEASURED, and authored here: whether Ⓐ or Ⓑ click when nothing is bound to +# them. Nobody has watched the game take a dead press. This class stays silent in +# that case, which is the choice that invents the least -- a sound the game does +# not make is a wrong fact you can hear, whereas a missing one is a gap. Ask the +# RE agent before relying on it either way. + + +func play(cue: String) -> void: + if not cues.has(cue): + return + if not _players.has(cue): + var p := AudioStreamPlayer.new() + p.stream = cues[cue] + add_child(p) + _players[cue] = p + (_players[cue] as AudioStreamPlayer).play() + + +## Start the music bed for a role, or do nothing if it is already playing. +## +## Idempotent because the menu re-enters screens constantly -- Ⓑ back to the main +## menu must not restart the music, and a bed that restarts on every navigation +## is the kind of wrong that reads as "the audio works". +func play_bed(role: String) -> void: + if not beds.has(role) or _bed_role == role: + return + if _bed == null: + _bed = AudioStreamPlayer.new() + add_child(_bed) + _bed.stream = beds[role] + _bed_role = role + _bed.play() + + +func stop_bed() -> void: + if _bed != null: + _bed.stop() + _bed_role = "" + + +## What the audio server is actually doing, for a run's write-up. +## +## `docs/port/AUDIO-VERIFICATION.md`: "recorded under a dummy driver" is a +## weaker claim than "heard", and the difference matters -- so the claim is +## printed by the run that makes it rather than assumed by the person reading it. +static func driver() -> String: + return AudioServer.get_driver_name() diff --git a/port/scripts/menu_audio.gd.uid b/port/scripts/menu_audio.gd.uid new file mode 100644 index 00000000..78186433 --- /dev/null +++ b/port/scripts/menu_audio.gd.uid @@ -0,0 +1 @@ +uid://badw3pulb0xpt