port: make the documented control sweep executable; it was prose
AUDIO-VERIFICATION.md calls its six-file sweep 'the tool's real specification' and nothing ran it -- in a tool whose own history is two invented thresholds caught only by controls. The same document states the principle it was breaking: a control that does not execute is not a control. tools/port/check-capture-controls rebuilds five of the six and asserts their verdicts. The starved capture is gone and is reported MISSING rather than omitted, and deliberately not synthesised from its published statistics -- a control fitted to the answer it must give is not a control. Two things the sweep had to learn to be honest about. check-capture emits TWO verdicts and the doc's table compresses them; the voice control is PASS on channels and UNJUDGED on starvation by design, so the sweep asserts the pair. And a starved file short-circuits before the channel check, recorded as n/a rather than FAIL -- the check did not run and the check failed are different facts. My first 'real music bed' control was -ac 6 from a stereo source and FAILED correctly: an upmix leaves channels silent and byte-identical, which is what the provenance check exists to catch. The control was wrong, not the tool. Rebuilt from six non-overlapping spans of real audio. A second attempt used aloop=-1 and hung ffmpeg indefinitely. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N7FiFFFwbvG2uxdcEh8HyF
This commit is contained in:
@@ -260,6 +260,12 @@ actually decides the sample layout; a float tee is 32-bit and is still caught.
|
|||||||
|
|
||||||
### The control sweep, which is the tool's real specification
|
### The control sweep, which is the tool's real specification
|
||||||
|
|
||||||
|
**Run it: `tools/port/check-capture-controls`.** 🔴 Until 2026-08-30 this table was prose — the specification existed and nothing executed it, so a regression in `check-capture` or a drifting threshold would have gone unremarked in a tool whose own history is *two invented thresholds that were both wrong and were caught only by controls*. This document states the principle it was breaking: **"a control that does not execute is not a control."**
|
||||||
|
|
||||||
|
⚠️ The verdicts below are **compressed**. `check-capture` emits two — one for channel provenance, one for starvation — and the sweep asserts the pair, because the voice control is `PASS` on channels and `UNJUDGED` on starvation *by design* and a single word cannot say that. A starved file **short-circuits** before the channel check, which the sweep records as `n/a` rather than as a failure: *the check did not run* and *the check failed* are different facts.
|
||||||
|
|
||||||
|
⚠️ The **starved capture cannot be rebuilt** — that artifact was transient and is gone. The sweep reports it `MISSING` rather than omitting it, and deliberately does not synthesise one from the statistics published above: a control fitted to the answer it must give is not a control either.
|
||||||
|
|
||||||
| file | verdict |
|
| file | verdict |
|
||||||
|---|---|
|
|---|---|
|
||||||
| real music+SFX bed | `PASS` |
|
| real music+SFX bed | `PASS` |
|
||||||
|
|||||||
103
tools/port/check-capture-controls
Executable file
103
tools/port/check-capture-controls
Executable file
@@ -0,0 +1,103 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Run `check-capture` against its own documented control sweep.
|
||||||
|
#
|
||||||
|
# tools/port/check-capture-controls
|
||||||
|
#
|
||||||
|
# `AUDIO-VERIFICATION.md` calls that sweep **"the tool's real specification"**
|
||||||
|
# and prints it as a table. Nothing executed it. So the specification was prose:
|
||||||
|
# if `check-capture` regressed, or if a threshold drifted, no run would have said
|
||||||
|
# so -- and this is a tool whose own history is two invented thresholds that were
|
||||||
|
# both wrong and were caught only by controls.
|
||||||
|
#
|
||||||
|
# 🔴 The same document states the principle this violates: **"A control that does
|
||||||
|
# not execute is not a control."** It was written about a mono file that skipped
|
||||||
|
# its own check. The sweep as a whole was in exactly that condition.
|
||||||
|
#
|
||||||
|
# ⚠️ One control CANNOT be rebuilt: the starved capture itself was a transient
|
||||||
|
# artifact and is gone. It is reported as MISSING rather than omitted, because a
|
||||||
|
# sweep that quietly drops a control is the defect it exists to catch.
|
||||||
|
set -euo pipefail
|
||||||
|
cd "${PROJECT_DIR:-/work}"
|
||||||
|
W="${TMPDIR:-/tmp}/capture-controls"; mkdir -p "$W"
|
||||||
|
CC=tools/port/check-capture
|
||||||
|
fail=0
|
||||||
|
|
||||||
|
# `check-capture` emits TWO verdicts -- one for channel provenance, one for
|
||||||
|
# starvation -- and `AUDIO-VERIFICATION.md`'s table compresses them into a word.
|
||||||
|
# That is fine for a summary and wrong for an assertion: the voice control is
|
||||||
|
# `PASS` on channels and `UNJUDGED` on starvation *by design*, and a sweep that
|
||||||
|
# collapsed those could not tell "passed" from "declined to judge". So both are
|
||||||
|
# reported, and a control names the pair it expects.
|
||||||
|
verdict() { # file -> "<channels>/<starvation>"
|
||||||
|
local out ch st
|
||||||
|
out=$("$CC" "$1" 2>&1 || true)
|
||||||
|
if grep -q '^PARTIAL' <<<"$out"; then echo "PARTIAL/PARTIAL"; return; fi
|
||||||
|
# A starved file SHORT-CIRCUITS: the tool reports the starvation and never
|
||||||
|
# reaches the channel check, which is right -- channel provenance is moot in a
|
||||||
|
# recording with holes punched through it. Reported as `n/a`, not as a failure:
|
||||||
|
# "the check did not run" and "the check failed" are different facts, and
|
||||||
|
# collapsing them is how a sweep starts asserting things it never observed.
|
||||||
|
if grep -qi 'no duplicated channels' <<<"$out"; then ch=PASS
|
||||||
|
elif grep -qi 'BYTE-IDENTICAL' <<<"$out"; then ch=FAIL
|
||||||
|
else ch=n/a; fi
|
||||||
|
if grep -q 'UNJUDGED' <<<"$out"; then st=UNJUDGED
|
||||||
|
elif grep -qi 'starv\|holes\|FAIL' <<<"$out"; then st=FAIL
|
||||||
|
else st=PASS; fi
|
||||||
|
echo "$ch/$st"
|
||||||
|
}
|
||||||
|
expect() { # name, file, wanted
|
||||||
|
local got; got=$(verdict "$2")
|
||||||
|
if [ "$got" = "$3" ]; then printf ' %-42s %-8s ok\n' "$1" "$got"
|
||||||
|
else printf ' %-42s %-8s 🔴 EXPECTED %s\n' "$1" "$got" "$3"; fail=1; fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# Six distinct tones -- the duplicate-channel control. Frequencies chosen so no
|
||||||
|
# two channels share one, which is what the provenance check looks for.
|
||||||
|
ffmpeg -v error -y -f lavfi -i "sine=frequency=400:duration=6" \
|
||||||
|
-f lavfi -i "sine=frequency=800:duration=6" -f lavfi -i "sine=frequency=200:duration=6" \
|
||||||
|
-f lavfi -i "sine=frequency=1600:duration=6" -f lavfi -i "sine=frequency=3200:duration=6" \
|
||||||
|
-f lavfi -i "sine=frequency=6400:duration=6" \
|
||||||
|
-filter_complex "[0:a][1:a][2:a][3:a][4:a][5:a]join=inputs=6:channel_layout=5.1[a]" \
|
||||||
|
-map "[a]" -c:a pcm_s16le "$W/tones.wav"
|
||||||
|
ffmpeg -v error -y -i "$W/tones.wav" -c:a pcm_f32le "$W/tones_f32.wav"
|
||||||
|
|
||||||
|
# A real music+SFX bed: six channels of REAL material, one per channel.
|
||||||
|
#
|
||||||
|
# 🔴 The first version of this control was `-ac 6` from the stereo bed, and it
|
||||||
|
# FAILED -- correctly. An upmix leaves channels 2-5 silent and byte-identical,
|
||||||
|
# which is exactly what the provenance check exists to catch, so the control was
|
||||||
|
# a broken capture wearing a control's name. The tool was right and the control
|
||||||
|
# was wrong, which is the outcome a sweep must be able to tell from its opposite.
|
||||||
|
#
|
||||||
|
# Six NON-OVERLAPPING spans of real audio, one per channel, all continuous. A
|
||||||
|
# second attempt used `aloop=-1` to stretch the short UI cues into full-length
|
||||||
|
# channels and hung ffmpeg indefinitely; spans of the long assets need no looping.
|
||||||
|
ffmpeg -v error -y -i export/audio/bgm/main_menu.ogg \
|
||||||
|
-i export/audio/voice/ADV.ogg -i export/audio/voice/S00A.ogg \
|
||||||
|
-filter_complex "[0:a]atrim=2:10,asetpts=N/SR/TB,aformat=channel_layouts=mono[a0]; \
|
||||||
|
[0:a]atrim=20:28,asetpts=N/SR/TB,aformat=channel_layouts=mono[a1]; \
|
||||||
|
[1:a]atrim=15:23,asetpts=N/SR/TB,aformat=channel_layouts=mono[a2]; \
|
||||||
|
[1:a]atrim=40:48,asetpts=N/SR/TB,aformat=channel_layouts=mono[a3]; \
|
||||||
|
[2:a]atrim=12:20,asetpts=N/SR/TB,aformat=channel_layouts=mono[a4]; \
|
||||||
|
[2:a]atrim=35:43,asetpts=N/SR/TB,aformat=channel_layouts=mono[a5]; \
|
||||||
|
[a0][a1][a2][a3][a4][a5]join=inputs=6:channel_layout=5.1[a]" \
|
||||||
|
-map "[a]" -c:a pcm_s16le "$W/bed.wav"
|
||||||
|
|
||||||
|
# The same bed with 350 ms holes punched through it, every second.
|
||||||
|
ffmpeg -v error -y -i "$W/bed.wav" \
|
||||||
|
-af "volume=enable='lt(mod(t,1),0.35)':volume=0" -c:a pcm_s16le "$W/holes.wav"
|
||||||
|
# A voice track: mono, with the real pauses of speech.
|
||||||
|
ffmpeg -v error -y -i export/audio/voice/ADV.ogg -t 60 -ac 1 -c:a pcm_s16le "$W/voice.wav"
|
||||||
|
|
||||||
|
echo "check-capture against its documented control sweep:"
|
||||||
|
expect "six distinct tones (PCM)" "$W/tones.wav" PASS/PASS
|
||||||
|
expect "the same tones as float32" "$W/tones_f32.wav" PARTIAL/PARTIAL
|
||||||
|
expect "real music bed" "$W/bed.wav" PASS/PASS
|
||||||
|
expect "bed with 350 ms holes punched in" "$W/holes.wav" n/a/FAIL
|
||||||
|
expect "voice track, mono, real pauses" "$W/voice.wav" PASS/UNJUDGED
|
||||||
|
printf ' %-42s %-8s the artifact is gone; not synthesised, because\n' "the starved capture" "MISSING"
|
||||||
|
printf ' %-42s %-8s fitting one to its published statistics would be\n' "" ""
|
||||||
|
printf ' %-42s %-8s a control shaped to the answer it must give\n' "" ""
|
||||||
|
echo
|
||||||
|
[ $fail -eq 0 ] && echo "the sweep matches the specification" || echo "🔴 check-capture no longer matches AUDIO-VERIFICATION.md"
|
||||||
|
exit $fail
|
||||||
Reference in New Issue
Block a user