fabi/Sylpheed
1
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity

formats: a music bank's third sub-wave was its own header

Browse Source 
The port hit `sound_bank_riffs("BGM_103.slb")` returning three against a census
that says two, and refused to guess which to drop. It was our reader.

`to_xma_riffs`'s hybrid branch derives a leading packet stream's start as
`first_riff % XMA1_PACKET`. That is right only when the bank header is smaller
than one 2048-byte packet -- true of the voice banks the branch was written for
(1392/1468/1600/1728), false of a music bank, whose header is exactly five
packets. The modulus returned 0 and the whole 10 240-byte header was emitted as
sub-wave 0.

The header states its own length, so the guard needs no threshold: BE u32 0x800
at +0x18 with the bank id repeated at +0x00 and +0x20, header length in blocks at
+0x24. Disc-wide over sound.pak's 9 519 entries, 28 match at offset 0 -- every
music bank, ids 1001-1023 and 1101-1105 -- and on 28/28 the declared header ends
EXACTLY at the first RIFF. Zero have a gap, so a header and a leading packet
stream never coexist here; zero false positives among the other 9 491.

Controlled rather than argued: decoding the emitted region through the same
chain, on the same bank, in the same run gives 0.009 s of PCM where the bank's
real wave 0 gives 87.744 s against a declared 87.75. The region is also 99.1%
zero bytes. And the oracle had already said two -- the XMA probe at the main menu
saw exactly two streams, at BGM_103's two declared wave sizes.

BGM_106-109 are deliberately NOT in the 28: their entries start mid-bank, so they
have no header at offset 0 and their leading region is real audio. The
VOICE_D_453 recovery is untouched and its tests still pass, 10/10 green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014voBspJ6kFncNErZJuZcLw
This commit is contained in:
sylph-decoder
2026-08-29 12:30:16 +00:00
parent fb536df8b4
commit c1f3608b07
6 changed files with 270 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

43
crates/sylpheed-formats/src/slb.rs
View File

@@ -377,6 +377,39 @@ pub fn leading_data_offset(first_riff: usize) -> usize {
first_riff % XMA1_PACKET
}
/// Length of the **bank header** when an entry begins with one, in bytes.
///
/// A music bank opens with a header the header itself sizes: big-endian, the
/// 2048-byte block size sits at `+0x18`, the bank id is repeated at `+0x00` and
/// `+0x20`, and `+0x24` is the header's length **in blocks** (5, i.e. 10 240 B,
/// on every music bank on this disc).
///
/// This exists because [`leading_data_offset`] derives a leading packet stream's
/// start as `first_riff % XMA1_PACKET`, which is only correct when the header is
/// SMALLER than one packet. A music bank's header is exactly five packets, so
/// the modulus returns 0 and the whole header was being emitted as a sub-wave —
/// a third "stem" on a bank the corpus documents as two
/// (`docs/re/structures/bgm-two-stems.md`).
///
/// Disc-wide over `sound.pak`'s 9 519 entries the signature fires on **28**, all
/// of them music banks (ids 1001–1023, 1101–1105), and on every one of the 28
/// the declared header ends **exactly** at the first `RIFF` — so no bank on this
/// disc has both a header at offset 0 and a leading packet stream. Zero false
/// positives on the 7 993 mid-bank windows, where the leading region IS real.
pub fn bank_header_len(slb: &[u8]) -> Option<usize> {
if slb.len() < 0x38 {
return None;
}
if slb[0x18..0x1c] != [0x00, 0x00, 0x08, 0x00] {
return None;
}
if slb[0x00..0x04] != slb[0x20..0x24] {
return None;
}
let blocks = u32::from_be_bytes(slb[0x24..0x28].try_into().ok()?) as usize;
blocks.checked_mul(XMA1_PACKET)
}
pub fn to_xma_riffs(slb: &[u8]) -> Vec<Vec<u8>> {
let mut out = Vec::new();
let first_riff = find(slb, b"RIFF", 0);
@@ -422,7 +455,15 @@ pub fn to_xma_riffs(slb: &[u8]) -> Vec<Vec<u8>> {
// bound to `VOICE_D_453`/`454`, i.e. precisely the broken ones — and
// ≤0.25 s to 66 of the rest. Callers clamp to the movie length anyway.
if let Some(ri) = first_riff {
let start = leading_data_offset(ri);
// A bank that carries its OWN header at offset 0 states how long it is,
// and on this disc that header always runs right up to the first `RIFF`
// — so there is no leading packet stream at all. Without this the
// modulus below returns 0 for a 5-packet header and the header itself is
// emitted as a sub-wave: `BGM_103.slb` came back as THREE waves against a
// census, an executable reference and a runtime XMA probe that all say
// two. It decodes to 0.009 s of PCM (the same chain returns 87.744 s for
// the bank's real wave 0), and it is 99.1 % zero bytes.
let start = bank_header_len(slb).unwrap_or_else(|| leading_data_offset(ri));
if ri > start {
if let Some(data) = slb.get(start..ri) {
if data.iter().any(|b| *b != 0) {

59
crates/sylpheed-formats/tests/slb_leading_segment_disc.rs
View File

@@ -269,3 +269,62 @@ fn a_waves_declared_size_is_confirmed_by_the_next_seek() {
assert!(checked >= 30, "expected banks to check, got {checked}");
eprintln!("wave-boundary identity held for {checked} banks");
}
/// A **music** bank has no leading segment — the bytes before its first `RIFF`
/// are the bank header, and emitting them made `BGM_103` look like three stems.
///
/// The header sizes itself (`+0x24`, in 2048-byte blocks), and on every bank on
/// this disc that size lands exactly on the first `RIFF`. So the guard is not a
/// heuristic and has no threshold: if a bank states a header, believe it.
#[test]
fn a_bank_that_states_its_own_header_has_no_leading_segment() {
skip_without_disc!(root);
let snd = PakArchive::open(root.join("dat/sound.pak")).expect("sound.pak");
let mut with_header = 0usize;
let mut mid_bank = 0usize;
// Peek at the 56-byte header through the archive's flat data rather than
// decompressing 9 519 entries: `sound.pak` stores them uncompressed, and a
// full read of all of them is several GB (it OOM-killed the test runner).
for entry in snd.entries() {
let Some(head) = snd.data_at(entry.offset as usize, 0x38) else { continue };
match slb::bank_header_len(head) {
Some(h) => {
let b = snd.read(entry).expect("read a bank that states a header");
let ri = b.windows(4).position(|w| w == b"RIFF").expect("has a RIFF");
// Declared header ends exactly at the first RIFF: no gap, so
// nothing before it can be a packet stream.
assert_eq!(h, ri, "a bank header that does not end at its first RIFF");
with_header += 1;
}
None => mid_bank += 1,
}
}
// 28 music banks (ids 1001-1023, 1101-1105); the rest are mid-bank windows,
// where the leading region IS real and must keep being emitted.
assert_eq!(with_header, 28, "banks stating their own header at offset 0");
assert!(mid_bank > 9000, "mid-bank windows, got {mid_bank}");
eprintln!("{with_header} banks state a header; {mid_bank} mid-bank windows");
}
/// The regression itself: the menu's music bank is **two** sub-waves, and they
/// are the two the corpus names — matching the executable's `BGM_103` and the
/// two streams the runtime XMA probe saw at the main menu.
#[test]
fn the_menu_music_bank_is_exactly_two_sub_waves() {
skip_without_disc!(root);
let snd = PakArchive::open(root.join("dat/sound.pak")).expect("sound.pak");
for (name, sizes) in [
("BGM_103.slb", [3_876_864usize, 3_930_112]),
("BGM_001.slb", [4_466_688, 4_673_536]),
] {
let entry = snd.find_by_name(name).expect("bank present");
let b = snd.read(entry).expect("read");
let riffs = slb::to_xma_riffs(&b);
assert_eq!(riffs.len(), 2, "{name}: sub-wave count");
for (r, want) in riffs.iter().zip(sizes) {
let di = r.windows(4).position(|w| w == b"data").expect("data chunk");
let got = u32::from_le_bytes(r[di + 4..di + 8].try_into().unwrap()) as usize;
assert_eq!(got, want, "{name}: sub-wave payload size");
}
}
}

38
docs/port/HANDOFF.md
View File

@@ -49,6 +49,42 @@ a bind mount on another device, and which is what the withdrawn banner ran.
`sylpheed-cli` against a pak, run the disc-gated tests and read the executable
again. New measurements are available; ask for them.
## ✅ 2026-08-29 — the "third sub-wave" on a music bank was OUR reader, and it is fixed
**You were right to refuse to choose which one to drop.** `BGM_103.slb` really
does return three from `sound_bank_riffs` — and the third is the **bank header**,
not a stem. Our own `to_xma_riffs` was emitting it.
The cause is arithmetic, not a judgement call: the hybrid branch derives a
leading packet stream's start as `first_riff % 2048`, which is correct only when
the bank header is smaller than one XMA1 packet. A music bank's header is exactly
**five** packets (10 240 B), so the modulus returned 0 and the whole header came
back as sub-wave 0. Voice banks are unaffected — their headers really are shorter
than a packet, which is why the branch looked right for two months.
Checked before believing it, three ways:
* **disc-wide** — of `sound.pak`'s 9 519 entries, **28** carry a header at offset
0 (ids 1001–1023, 1101–1105 — every music bank), and on **28/28** the header's
own declared length ends *exactly* at the first `RIFF`. **Zero** have a gap, so
a header and a leading packet stream never coexist on this disc, and **zero**
false positives among the other 9 491;
* **decode control, same chain, same bank** — the emitted region gives **0.009 s**
of PCM; the same bank's real wave 0 gives **87.744 s** against a declared
87.75. It is also 99.1 % zero bytes;
* **the oracle already said two** — the XMA probe at the main menu saw exactly
two streams, of 3 876 864 and 3 930 112 B, which are `BGM_103`'s two declared
wave sizes.
**What you should do:** bump your `sylpheed-formats` pin to the tag below and
delete the manifest warning's special case — `sound_bank_riffs` now returns
**2** for every music bank, and your "count != 2" warning becomes a real
invariant rather than a symptom. ⚠️ Do **not** apply a "drop the smallest
sub-wave" rule; on a voice bank the leading region is genuine audio and dropping
it is the `VOICE_D_453` bug all over again.
[`structures/slb-bank-header-not-a-wave.md`](../re/structures/slb-bank-header-not-a-wave.md)
## ✅ 2026-08-29 — the interactive title is reachable again, and the "emulator-blocked" banner in MISSION is withdrawn
Two consecutive boots reached the interactive title **with no pad input at all**,
@@ -164,7 +200,7 @@ items MISSION parks as emulator-blocked.
| Q7 | transitions | ✅ answered | a **fade through black**, drawn by the screen's own last-painting `.prm` quad. Fade-in ramp is **decoded** from its keyframes; the ~0.4 s fade-out is **measured** (not in the file) — [`screen-transitions.md`](../re/screen-transitions.md) |
| Q8 | menu audio bindings | ✅ answered | cue vocabulary + bank **decoded**; event binding is a **name match** (the authors' own event names). ✅ **You CAN have the SE audio** — ⚠️ an earlier version of this row said it was "undecodable from the disc"; that was **retracted** and the row was stale. Three cues are located in `Static.slb` and **decode to PCM**: d-pad move `0x1ec0` (4 packets), Ⓑ back `0x0ec0` (2), Ⓐ confirm `0x5d6c0` (6), all mono 48 kHz. The bank is a packed run of XMA waves with no delimiter, so a wave is only (offset, packet count) — and ⚠️ the file order is **not** cue-id order, so the index cannot be counted out — [`menu-audio-cues.md`](../re/menu-audio-cues.md) |
| Q9 | video binding + playback rules | ✅ answered | **decoded** from the movie manifest: `ADVERTISE_MOVIE`→`ADV.wmv` (boot intro *and* attract are one asset), `MS00A`→`S00A.wmv` is the new-game intro, `STAFF_ROLL`→the credits reel. ✅ **one Ⓐ skips a movie** (title at 57 s vs a 193 s baseline) — [`movie-binding.md`](../re/movie-binding.md) |
| Q10 | music-bank sub-wave roles (intro+loop?) | ✅ answered | **two stems of one performance, played together** — sample-synchronous, equal duration, 32/32 banks. **Concatenating is wrong.** Not a seamless loop either — [`structures/bgm-two-stems.md`](../re/structures/bgm-two-stems.md) |
| Q10 | music-bank sub-wave roles (intro+loop?) | ✅ answered | **two stems of one performance, played together** — sample-synchronous, equal duration, 32/32 banks. **Concatenating is wrong.** Not a seamless loop either — [`structures/bgm-two-stems.md`](../re/structures/bgm-two-stems.md). ⚠️ **Our reader said three until 2026-08-29** — the extra one was the **bank header**, emitted by `to_xma_riffs`; fixed, with a 28/28 disc-wide check and two regression tests — [`structures/slb-bank-header-not-a-wave.md`](../re/structures/slb-bank-header-not-a-wave.md) |
| S1 | Ready Room go/no-go | ✅ **no-go** | it is 2D and enumerates fine (60 builds), but `GP_READY_ROOM.pak` holds **briefing/tactical-map** content, not the six-button Ready Room menu — [`ready-room-probe.md`](../re/ready-room-probe.md) |
## Already settled — the port can rely on these today

1
docs/re/INDEX.md
View File

@@ -163,5 +163,6 @@ files, which is how the same ground got covered twice.
| [`weapon-datasheet-runtime.md`](weapon-datasheet-runtime.md) | Weapon DATA SHEET — runtime capture (Route B) | 🟡 first dynamic capture, 2026-07-28. The Arsenal's Gallery Mode panel is a |
| [`xpr2-colour-check.md`](xpr2-colour-check.md) | XPR2 colours: channel order ✅ confirmed against the running game | — |
| [`focus-ring-spin-measured.md`](focus-ring-spin-measured.md) | The main menu's focus ring spins continuously — and how fast | ✅ **measured**: period **2.177 s** over 9 revolutions (8 evenly spaced autocorrelation peaks) = 120 units = 60 frames = 2.00 s at 30 Hz. A pulse is excluded — annulus total conserved to 0.4 % while per-bin brightness swings by 24. ✅ the ring is the **only** moving thing on the settled main menu (std exactly 0.000 elsewhere). 🔴 no angle is quoted: the angular estimator FAILED its own control (30° → 0°) |
| [`structures/slb-bank-header-not-a-wave.md`](structures/slb-bank-header-not-a-wave.md) | Why a music bank read as THREE sub-waves when the census says two | ✅ **decoded**: the third is the **bank header**, emitted by our own reader. `to_xma_riffs`'s hybrid branch derives a leading packet stream's start as `first_riff % 2048`, which is right only for a header shorter than one packet; a music bank's header is exactly **5 packets (10 240 B)**, so the modulus gave 0 and the whole header came back as sub-wave 0. The header states its own length at `+0x24` in blocks. Disc-wide over 9 519 `sound.pak` entries: **28** match the header signature at offset 0 (ids 1001–1023, 1101–1105), **28/28** end exactly at the first `RIFF`, **0** have a gap, **0** false positives — so a header at offset 0 and a leading packet stream never coexist. Decode control, same chain, same bank: the emitted region gives **0.009 s** against **87.744 s** for the real wave 0. Corroborated by the runtime XMA probe, which saw exactly two streams at the main menu. Fixed + 2 regression tests; the `VOICE_D_453` recovery is untouched (10/10 green) |
| [`title-plate-delay-measured.md`](title-plate-delay-measured.md) | How long the boot title shows build 4 before the `PRESS Ⓐ` plate | ✅ **measured**, two independent boots: **2.138 s** and **2.132 s** from the frame build 4 settles (glyph = its no-plate 154, motion → 0). Agreeing to **6 ms**. So the boot title's end state is **not** plate-free and a compositor must draw **two builds at once**. ⚠️ Measure from *settled*, not from first pixels — "first drawn → plate" is 3.78 s vs 4.26 s across the same two runs, because the build-in animation's own duration varies with emulator frame pacing. Plate pulse re-measured at 2.12/2.19/2.34/2.31 s (mean 2.24), replicating the corpus's ≈2.3 s. ✅ black hold between screens bracketed at **0.14–0.30 s**, consistent with the declared 12 units. 🔴 the Ⓐ→menu latency is still **not** available: both runs freeze one frame for ~1.4 s at surface mean **26.626** — agreeing between runs to 1e-6, and reproduced with stream restarts disabled — which is a guest **load stall**, not the capture path. Probe: 8.7 ms/frame, 7.97/7.98 fps against a requested 8, controls 9/9 + 4/4 |
| [`menu-idle-and-b-2026-08-29.md`](menu-idle-and-b-2026-08-29.md) | The main menu does not idle back to the title — and four durations that were a pipeline | ✅ **refuted**: no self-return in **≥ 60 s** untouched; the ~8–10 s idle belongs to the **title**. 🟡 Ⓑ→title ordering measured, latency not. 🔴 `classify_array` at **1503 ms/frame** drained an 8 fps stream at 0.64 fps and manufactured four latencies (24.66 s / 15.58 s / 25.60 s / 20.26 s) — all withdrawn; a backlog preserves ordering and destroys durations |

11
docs/re/structures/bgm-two-stems.md
View File

@@ -21,7 +21,16 @@ BGM_001.slb (9 178 040 B)
```
A bank is a 10 240-byte header and then **exactly two waves**, and the two always
have the **same duration** — different byte sizes and different bitrates, same
have the **same duration**
⚠️ **Our own reader disagreed with this page until 2026-08-29, and the page was
right.** `slb::to_xma_riffs` was emitting that 10 240-byte header as a third
sub-wave, so `sound_bank_riffs("BGM_103.slb")` returned **three** — which the
port caught while exporting the menu music. The header is not a wave (it decodes
to 0.009 s and is 99.1 % zero); the cause was a modulus that assumes a bank
header is shorter than one 2048-byte packet, and it is fixed with a disc-wide
28/28 check —
[`slb-bank-header-not-a-wave.md`](slb-bank-header-not-a-wave.md) — different byte sizes and different bitrates, same
number of seconds. Duration is `data_size / PsuedoBytesPerSec` (the u32 at
`RIFF+0x20`; `RIFF+0x24` is the sample rate, 48 000 Hz except `BGM_020`–`023`
at 44 100).

121
docs/re/structures/slb-bank-header-not-a-wave.md Normal file
View File

@@ -0,0 +1,121 @@
# ✅ A music bank's "third sub-wave" is its **header**, and the bug was arithmetic
**Status:** ✅ `CONFIRMED` — **decoded**, with a disc-wide check over all 9 519
`sound.pak` entries, a decode control, and independent corroboration from the
running game. Fixed in `sylpheed-formats` 2026-08-29.
**Raised by the port**, on its P6 critical path:
`sound_bank_riffs("BGM_103.slb")` returned **three** sub-waves against
[`bgm-two-stems.md`](bgm-two-stems.md)'s census, which says a music bank is
exactly two. Its exporter was summing all three, so the shipped menu music was
the sum of three things where the corpus predicted two. It declined to choose
which to drop, which was right — that is a decoding question.
## The answer
The third thing is **the bank header**. Not a stem, not an artefact of the disc:
our own reader was emitting it.
`to_xma_riffs` has a hybrid branch for banks that carry a headerless packet
stream *before* their first `RIFF` — the fix that recovered `VOICE_D_453`'s line
([`slb-data-offset.md`](slb-data-offset.md)). It derives that stream's start as
```rust
first_riff % XMA1_PACKET // XMA1_PACKET = 2048
```
which is correct **only when the bank header is smaller than one packet**. It is,
in the voice banks the branch was written for: their headers put the first `RIFF`
at 1392, 1468, 1600 or 1728 mod 2048.
A music bank's header is **exactly five packets — 10 240 bytes** — so the
modulus returns **0**, and the branch emitted `slb[0..10240]`: the whole header,
as sub-wave 0.
The header states its own length, so nothing here needs a heuristic:
```
BGM_103.slb
+0x00 BE u32 1103 bank id
+0x18 BE u32 0x00000800 block size = 2048
+0x1c BE u32 7839244 data size
+0x20 BE u32 1103 the id again ← signature, with +0x18
+0x24 BE u32 5 HEADER LENGTH IN BLOCKS → 5 × 2048 = 10240
+0x28 BE u32 0x00100002 16 bit / 2 ch
```
## The disc-wide check
Over all **9 519** entries of `sound.pak`
([`tools/re-capture/slb_segment_phase.py`](../../../tools/re-capture/slb_segment_phase.py)
supplies the reader):
| | |
|---|---|
| entries matching the header signature at offset 0 | **28** |
| ...whose declared header ends **exactly** at the first `RIFF` | **28 / 28** |
| ...with a real gap between header and first `RIFF` | **0** |
| false positives among the 9 491 others | **0** |
The 28 are exactly the music banks — ids **1001–1023** and **1101–1105**. So on
this disc a bank header at offset 0 and a leading packet stream **never
coexist**, and the guard is not a threshold: if a bank states a header, believe
it, and there is nothing before the first `RIFF`.
⚠️ `BGM_106`–`BGM_109` are **not** in the 28 and must not be: their pak entries
start mid-bank, so they have no header at offset 0 and their leading region is
real audio (the tail of the previous bank). That is the same straddle
[`bgm-two-stems.md`](bgm-two-stems.md) already documents.
## The decode control
Decoding the emitted region proves it is not audio, and the control is run
through **the same chain, on the same bank, in the same invocation**:
| | bytes | PCM decoded |
|---|---|---|
| `BGM_103` — what we emitted as "sub-wave 0" | 10 240 | **0.009 s** |
| `BGM_103` — its real wave 0 (control) | 3 876 864 | **87.744 s** (declared 87.75) |
| `BGM_001` — what we emitted as "sub-wave 0" | 10 240 | **0.009 s** |
| `BGM_001` — its real wave 0 (control) | 4 466 688 | **173.809 s** (declared 173.82) |
FFmpeg `xma1`, mono/stereo taken from the bank's own `fmt `. The region is also
**99.1 % zero bytes** (67–93 non-zero of 10 240 across the 28 banks) and its last
non-zero byte is at 6431, so its final 1.86 packets are entirely empty.
## Corroboration from the oracle, which was already in the corpus
[`bgm-two-stems.md`](bgm-two-stems.md) records that at the **main menu**, with
`--xma_param_probe=true`, the decoder was handed **two** stereo 48 kHz streams —
of **3 876 864** and **3 930 112** bytes, byte-for-byte `BGM_103`'s two declared
waves. A third stem would have been a third stream. The running game was already
saying two.
## The fix
`slb::bank_header_len` (new, `pub`) reads the signature and returns the declared
length; the hybrid branch uses it in preference to the modulus:
```rust
let start = bank_header_len(slb).unwrap_or_else(|| leading_data_offset(ri));
if ri > start { /* emit the leading stream */ }
```
Two regression tests in
[`tests/slb_leading_segment_disc.rs`](../../../crates/sylpheed-formats/tests/slb_leading_segment_disc.rs):
the disc-wide 28/28 identity, and `BGM_103`/`BGM_001` returning exactly two
sub-waves at their declared payload sizes. The pre-existing voice-bank tests —
`broken_banks_recover_their_line`, `derived_offset_recovers_voice_banks_without_regressing_etc`
— still pass, so the `VOICE_D_453` recovery is untouched. 10/10 green with
`SYLPHEED_DISC` set.
## Reach
* The 28 are the only banks on the disc that state a header at offset 0. A bank
format elsewhere with a header ≥ 2048 B that we have not seen would have had
the same bug; nothing on this disc does.
* This says nothing about **which** of the two remaining waves is which — that is
still 🟡 in [`bgm-two-stems.md`](bgm-two-stems.md) (surround-rear pair vs a
second intensity layer), and both readings predict playing them together.
* It does not change the count for any voice bank: `VOICE_*` entries have no
header at offset 0, so their leading region is emitted exactly as before.