From 0abe908edd2cea6a7a37f4b89af38541dd93b923 Mon Sep 17 00:00:00 2001 From: Sylpheed RE agent Date: Wed, 26 Aug 2026 04:11:45 +0000 Subject: [PATCH] re: 69.8% of banks declare more data than they store -- and that explains TCAF_608 Of the 7586 banks with a RIFF and a data chunk after it, 5296 declare a data size larger than the pak entry holds; 2290 declare less (the ordinary multi-sub-wave case); NONE declare exactly what they hold. This contradicts the decoder comment claiming the declared size 'is honest per sub-wave'. The code clamps, so it is a documentation defect, not a crash. It also closes the loose end from the offset work: eng\Voice\VOICE_TCAF_608, the single bank where neither offset decoded, is 99% short -- there is nothing there to decode. Method note recorded: my first pass searched for 'data' from offset 0, which can match by chance inside the leading audio region. Anchoring the search after the first RIFF moved the count 5038 -> 5296. Separately, the 55 'early RIFF' English banks are not an anomaly: all 55 sit at exactly 1392 behind a zero-filled header -- a zero-length leading region, which both the old and new code already handle correctly. --- docs/re/BACKLOG.md | 9 +++-- docs/re/structures/slb-data-offset.md | 49 +++++++++++++++++++++++---- 2 files changed, 49 insertions(+), 9 deletions(-) diff --git a/docs/re/BACKLOG.md b/docs/re/BACKLOG.md index d41768a1..007e0c9c 100644 --- a/docs/re/BACKLOG.md +++ b/docs/re/BACKLOG.md @@ -31,8 +31,13 @@ mistaken a constant derived from `eng\etc\` for a property of the format. See `Pj_Silph.xgs`, named by the same table's `BANK_SE` and `SETTINGS` records — `sound.pak` is now 9 519 of 9 519 accounted for. -**Still open**: 55 English banks have their first `RIFF` at or before offset -1392, and the 28 offset-scan ties. +**Closed**: the 55 "early `RIFF`" English banks are not an anomaly — all 55 sit +at *exactly* 1392 behind a zero-filled header, i.e. a zero-length leading +region, which both the old code and the new derivation already handle. + +**Still open**: the 28 offset-scan ties, and ❔ why 69.8 % of banks declare more +`data` than they store (see +[`structures/slb-data-offset.md`](structures/slb-data-offset.md)). --- diff --git a/docs/re/structures/slb-data-offset.md b/docs/re/structures/slb-data-offset.md index 2c087ce9..f62cfcfe 100644 --- a/docs/re/structures/slb-data-offset.md +++ b/docs/re/structures/slb-data-offset.md @@ -51,15 +51,17 @@ must and does reproduce the old behaviour exactly. Individual cases: jpn\Voice\VOICE_TCAF_592.slb 2 910 -> 127 178 bytes (44x) eng\etc\VOICE_D_452.slb 30 154 -> 30 154 bytes (unchanged, control) -## The one counterexample +## The one counterexample — ✅ explained `eng\Voice\VOICE_TCAF_608.slb` decodes 2 840 bytes at 1392 and 896 at 1468. -It is worth being precise about what that is and is not. It is **not** a bank -where the old constant works and the derived offset fails: its leading region is -38 988 bytes, and *both* offsets yield well under a tenth of a second, i.e. both -fail. 1392 merely produces marginally more garbage. Whatever is wrong with this -bank is a separate defect and is **not settled** here. +It is **not** a bank where the old constant works and the derived offset fails: +both offsets yield well under a tenth of a second from a 38 988-byte region, +i.e. both fail, and 1392 merely produces marginally more garbage. + +**The reason is that the bank is truncated.** Its `data` chunk declares 759 808 +bytes and the pak entry holds 8 864 — **99 % short**. There is almost nothing +there to decode at any offset. See the section below. ## ❌ This withdraws my own claim from earlier the same day @@ -123,10 +125,43 @@ derived from it exactly, never scanned. * **Why the offset takes those four values**, and what the bytes before it are. There is no length field in the first 64 bytes — banks open on high-entropy data — so the offset is derived, not read. -* **`eng\Voice\VOICE_TCAF_608.slb`**, above. + * **The 28 ties.** The scan cannot separate them and falls back to 1392, which is right for roughly a third of that population and wrong for the rest. * **Why the offset takes exactly these four values by directory** is still unexplained — see above. * Nothing here was run **in the game** — this is a decoder-side result measured with FFmpeg as the oracle. + + +## 🟡 Most banks declare more `data` than they store + +**Measured 2026-08-26.** Of the 7 586 banks that carry both a `RIFF` and a +`data` chunk after it, **5 296 (69.8 %)** declare a `data` size larger than the +bytes actually present in the pak entry. The remaining 2 290 declare *less*, +which is the ordinary multi-sub-wave case. **Not one declares exactly what it +holds.** + +Worst cases run to 99 % short: + + eng\Movie\VOICE_RT16C.slb declared 1 810 432 available 489 392 -73 % + jpn\etc\VOICE_D_589.slb declared 1 177 600 available 6 708 -99 % + eng\Voice\VOICE_TCAF_608.slb declared 759 808 available 8 864 -99 % + +This **contradicts a claim in the decoder's own comment**, which says the +declared size "is honest per sub-wave". It is not, for about seven banks in ten. +The code is nonetheless safe — it clamps the range with `.min(slb.len())` — so +this is a documentation defect and an integrity observation, not a crash. + +⚠️ **Method note on this measurement.** My first pass searched for `data` from +offset 0, which can hit those four bytes by chance inside the leading audio +region and read a garbage length. Re-running it anchored *after* the first +`RIFF` changed the count from 5 038 to 5 296 — the flaw was slightly +*under*-counting, but it could as easily have gone the other way, and an +unanchored chunk search over binary audio is not a safe way to ask this +question. + +❔ **Why** the declared sizes are too large is **not settled**. Plausible +readings — an authoring-time allocation that was never trimmed, or deliberate +truncation of unused tails — are guesses; nothing here distinguishes them, and +the game has not been observed reading one of these banks.