diff --git a/docs/port/AUDIO-VERIFICATION.md b/docs/port/AUDIO-VERIFICATION.md index 59394df8..39da8956 100644 --- a/docs/port/AUDIO-VERIFICATION.md +++ b/docs/port/AUDIO-VERIFICATION.md @@ -181,6 +181,75 @@ And the failure this page already warns about, in a second costume: `--mute=true`. Fix only the first and Canary attaches a healthy 6-channel stream at 100 % volume, reports `Corked: no`, and emits a 19 MB WAV of zeroes. +## 7. A capture can be starved — right duration, holes punched through it + +`check-capture` tests this too, and it is the second way a recording looks +perfect and carries nothing. + +**A monitor sink advances at wall-clock rate and substitutes silence whenever the +producer is late.** An emulator running below real time therefore yields a file +of exactly the right duration, the right channel count, no duplicated channels — +chopped into fragments with holes between them, thousands of times over. + +Measured independently on the capture that prompted this (the Decoder's numbers +on the untruncated original in brackets): + +| | | +|---|---| +| frames silent on **all six** channels | **35.6 %** [39.3 %] | +| alternating runs | **10 482** [10 595] | +| median burst / gap | **13.5 ms / 3.9 ms** [13.6 / 3.9] | +| period | **17.4 ms → 57 Hz** [≈17.5 ms → 57 Hz] | + +⚠️ **This destroys envelope correlation by construction.** What dominates the +envelope of such a file is the dropout schedule, not the content — so §6's method +was working correctly on a file that could not carry the signal, and the negative +it produced said nothing about the game. + +### Two thresholds I invented were wrong, and the controls caught both + +1. **Counting exact-zero frames.** Real audio crosses zero constantly, so a clean + voice track scored **5 947 "gaps" of median 0.0 ms** and was called starved. A + gap is a **run**, not a sample: only runs of ≥ 1 ms count. +2. **Gap count and median length.** A genuine music-and-effects bed has **454 + gaps at a median of 1.4 ms** — quiet 16-bit passages really are zero for + milliseconds — so neither statistic separates it from a starved file. + +**What does separate them is the RATE**, by an order of magnitude in both +directions: + +| | gaps/s | median gap | all-channel silence | +|---|---|---|---| +| the starved capture | **32.9** | 3.9 ms | 35.6 % | +| a real music+SFX bed | **3.3** | 1.4 ms | 1.1 % | +| a voice track, 53 % pauses | **0.03** | — | — | + +The bar is **20 gaps/s** — 1.6× below the bad case, 6× above the worst good one, +and set from those controls rather than chosen and then justified. + +### 🔴 The monitor-sink route cannot be fixed by configuration + +`parec` reads a monitor that advances at wall-clock rate and substitutes silence, +so **every moment the emulator runs below real time is a hole**, and the timebase +is warped non-uniformly — deleting the silences compresses time unevenly rather +than repairing it. The route that would work is an **internal tap at +`SDLAudioDriver::SubmitFrame`**, which sees every frame the guest produces in +guest order with no wall clock in the loop. + +⚠️ That needs a Canary rebuild, and the Decoder has costed it: `build-canary` +targets a source root that does not exist in that container, the warm build tree +is configured against the same missing path, so any change is a full reconfigure +plus a full compile on a box with ~700 MB free and a history of parallel builds +OOM-killing the host. **A whole session for one probe** — the human's call, not +an agent's. + +### And a header that never got patched + +A streaming writer leaves `data` declaring **0 bytes**. `check-capture` says so +and tells you the duration is unverified — which is not pedantry: the file shared +here was **copied while it was still being written**, and the provenance claim +that came with it was wrong about both its length and what it contained. + ## 6. Finding one component inside a mix — and why §1's method cannot 🔴 **This section begins with a retraction.** Two captures of the game's own diff --git a/docs/port/DECISIONS.md b/docs/port/DECISIONS.md index fde58c1b..45299003 100644 --- a/docs/port/DECISIONS.md +++ b/docs/port/DECISIONS.md @@ -3370,3 +3370,66 @@ to its declared end. So declared and decoded agree to 12 ms and the trailing silence is *inside* the decode, not the difference between two methods. The cross-check stands — better than stated, since it is now three banks rather than a coincidence — and the sentence explaining it should go. + +## Take 2 was starved, my correlator was fine, and `check-capture` was incomplete + +The Decoder diagnosed take 2: a **starved** capture. Verified here independently +rather than taken on trust — 35.6 % of frames silent on all six channels, 10 482 +alternating runs, median burst 13.5 ms and gap 3.9 ms, a 17.4 ms period at 57 Hz. +Their untruncated original reads 39.3 % and 10 595 runs; the difference is +exactly the truncation, and every other number agrees. + +**So my rebuilt correlator was working correctly on a file that could not carry +the signal.** ✅ And the alarming reading it produced — *"the game may not play +the `.wmv`'s WMA track, so `ADV.ogv`'s audio has been wrong since P4"* — **is not +supported by this capture, and is not refuted either.** It is withdrawn as a +concern arising from evidence, and nothing is changed on account of it in either +direction. That matters more than it looks: it was the most expensive-to-act-on +hypothesis in the port, and it came from a file that could not speak to it. + +### The real deliverable: my own checker passed the starved file + +`check-capture` tested only for duplicated channels, so it cleared a recording +that was 36 % holes. A provenance check that passes the artefact it was built in +response to is not a check. + +It now measures starvation too. ⚠️ **Two thresholds I invented were both wrong, +and the controls caught both** — which is the part worth recording: + +1. **Counting exact-zero frames.** Real audio crosses zero constantly; a clean + voice track scored 5 947 "gaps" of median 0.0 ms and was called starved. **A + gap is a run, not a sample.** Only runs ≥ 1 ms count. +2. **Gap count and median length.** A genuine music bed shows **454 gaps at a + median of 1.4 ms** — quiet 16-bit passages really are zero for milliseconds — + so neither statistic separates it from a starved file. + +What separates them is the **rate**: + +| | gaps/s | median gap | all-channel silence | +|---|---|---|---| +| the starved capture | **32.9** | 3.9 ms | 35.6 % | +| a real music+SFX bed | **3.3** | 1.4 ms | 1.1 % | +| a voice track, 53 % pauses | **0.03** | — | — | + +Bar set at **20 gaps/s** — 1.6× below the bad case, 6× above the worst good one, +**derived from the controls rather than chosen and then justified.** Controlled +in both directions: real stereo bed PASS, six distinct tones PASS, starved +capture FAIL. It also now reports a `data` chunk declaring 0 bytes, which is what +a file copied while still being written looks like. + +### 🔴 The voice channel roles are not obtainable this session + +Both capture routes are closed and the Decoder has said so plainly. The monitor +sink is starved **by construction** — it advances at wall-clock rate and +substitutes silence, so every moment the emulator runs slow is a hole, and +deleting the holes warps the timebase rather than repairing it. The route that +works is an internal tap at `SDLAudioDriver::SubmitFrame`, and that needs a +Canary rebuild whose cost they have measured: the build root `build-canary` +targets does not exist in that container, the warm tree is configured against the +same missing path, so it is a full reconfigure and compile on a box with ~700 MB +free and a history of parallel builds OOM-killing the host. + +**A whole session for one probe.** That is a human's call and neither agent +should start it mid-loop. Until then the port keeps authoring with the known +recorded: one stream of three, 🔴 in the manifest, the console line and +`authored/audio.json`. diff --git a/tools/port/check-capture b/tools/port/check-capture index c6893c54..8e626aa3 100755 --- a/tools/port/check-capture +++ b/tools/port/check-capture @@ -67,6 +67,102 @@ for i in $(seq 0 $((ch-1))); do done done +# STARVATION: the second way a capture looks perfect and carries nothing. +# +# A monitor sink advances at WALL-CLOCK rate and substitutes silence whenever the +# producer is late. An emulator running below real time therefore yields a file +# of exactly the right duration, right channel count, no duplicated channels -- +# and chopped into fragments with holes punched between them, thousands of times +# over. Envelope correlation against such a file is destroyed by construction: +# what dominates the envelope is the dropout schedule, not the content. +# +# Measured on the capture that prompted this: 35.6 % of frames silent on all six +# channels, 10 482 alternating runs, median burst 13.5 ms and median gap 3.9 ms +# -- a 17.4 ms period, 57 Hz. The Decoder measured the untruncated original at +# 39.3 % and 10 595 runs; the two agree. +# +# THE DISCRIMINATOR IS THE RUN STRUCTURE, NOT THE SILENCE FRACTION. Real audio is +# full of silence -- a voice track is more than half gaps -- but those are TENS of +# runs of HUNDREDS of milliseconds. Dropout chop is THOUSANDS of runs of a few +# milliseconds. So the test is: many short all-channel gaps. +set +e +python3 - "$f" <<'PYEOF' +import array, struct, sys +d = open(sys.argv[1], 'rb').read() +i, fmt, off = 12, None, None +while i + 8 <= len(d): + cid = d[i:i+4]; sz = struct.unpack(' avail: + # A streaming writer that never patched its header. The file may also be a + # copy taken while it was still being written -- which happened, and made a + # provenance claim wrong. + print(" ⚠️ data chunk declares %d bytes, %d present -- header never patched;" + % (declared, avail)) + print(" treat the duration as unverified and check the file is complete.") +n = avail // (2 * ch) +a = array.array('h'); a.frombytes(d[off:off + n * 2 * ch]) +sil = bytearray(n) +for f_ in range(n): + b = f_ * ch + if not any(a[b+c] for c in range(ch)): sil[f_] = 1 +tot = sum(sil) +# A GAP IS A RUN, NOT A SAMPLE. The first version of this counted every frame +# whose channels were all exactly zero, and real audio crosses zero constantly -- +# it scored a clean voice track at 5 947 "gaps" of median 0.0 ms and called it +# starved. The known-good control caught it. Only runs of at least 1 ms (48 +# frames at 48 kHz) count: a zero-crossing is one sample, a dropout is hundreds. +MINGAP = max(1, rate // 1000) +runs_s, runs_n = [], [] +cur, ln = sil[0], 0 +for v in sil: + if v == cur: ln += 1 + else: + (runs_s if cur else runs_n).append(ln); cur = v; ln = 1 +(runs_s if cur else runs_n).append(ln) +runs_s = [r for r in runs_s if r >= MINGAP] +if not runs_s: + print(" all-channel silence 0.0% -- no gaps at all"); raise SystemExit(0) +rs = sorted(runs_s); med = 1000.0 * rs[len(rs)//2] / rate +secs = n / float(rate) +rate_per_s = len(runs_s) / secs +print(" all-channel silence %.1f%%, %d gap(s) over 1 ms (%.1f/s), median gap %.1f ms" + % (100.0*tot/n, len(runs_s), rate_per_s, med)) +# THE THRESHOLD IS SET FROM CONTROLS, and the first two I invented were both +# wrong -- they failed real audio. Measured: +# +# the starved capture 32.9 gaps/s, median 3.9 ms, 35.6 % silent +# a real music+SFX bed 3.3 gaps/s, median 1.4 ms, 1.1 % silent +# a voice track, 53 % pauses 0.03 gaps/s +# +# Real audio does contain short all-zero runs -- a quiet passage in 16-bit is +# genuinely zero for milliseconds -- so neither the gap COUNT nor the median +# length separates them. The RATE does, by an order of magnitude in both +# directions, and 20/s sits between with a 1.6x margin below the bad case and +# 6x above the worst good one. +if rate_per_s >= 20.0 and med < 50.0: + print(" 🔴 STARVED: %.1f gaps per second at a median of %.1f ms." + % (rate_per_s, med)) + print(" The producer was not keeping the sink fed.") + raise SystemExit(3) +PYEOF +starved=$? +set -e +if [ "$starved" = 3 ]; then + echo "FAIL: the recording is starved. A monitor sink advances at wall-clock rate" + echo " and substitutes silence when the producer is late, so this file has" + echo " the right duration and holes punched through the content. Correlation" + echo " against it is meaningless. See docs/port/AUDIO-VERIFICATION.md §7." + exit 1 +fi + if [ "$dupes" = 1 ]; then echo "FAIL: duplicated channels. A surround remap drops and duplicates silently;" echo " channels are missing from this file. Do not analyse it -- fix the"