diff --git a/docs/port/BLOCKED.md b/docs/port/BLOCKED.md index c2809ef1..afaf8d92 100644 --- a/docs/port/BLOCKED.md +++ b/docs/port/BLOCKED.md @@ -147,6 +147,12 @@ HANDOFF. | ~~P1–P7 — the keyframe record layout~~ | ~~adopt the corrected pose/time pairing~~ | — | ✅ **ADOPTED 2026-08-29 by pinning `formats-pin-2026-08-29c`.** This row was wrong twice: it said the change *"cannot be taken yet"* and that it *"reaches the port only when that branch lands on `main`"*. **It arrives when the tag is pinned**, which is what MISSION §2's tagging rule exists for. ⚠️ And the knob I tested first, `SYLPHEED_KF_TIME_SHIFT`, is a **retired partial fix** that left pose 0 untimed — the real correction is the tagged crate's default, with the old reading behind `SYLPHEED_KF_TIME_LEGACY=1`. **The blast radius was far smaller than this row predicted**: under the correction *every pose is timed* (866 keyframes, 0 untimed), so `pose_at`'s synthetic-exit branch became dead code rather than wrong code and nothing needed re-deriving. Oracle: `publisher_logo` 1.00 %→**0.75 %**, `developer_logos` 0.39 %→**0.33 %**, `extras`' differing region collapsing from 736×525 to **398×295 at the sweep position**. 🔴 Open cost: `sylpheed-cli` builds from the workspace crate, so `verify-screen` compares two decoder eras until the tag reaches `main`. Revert to the path dependency then. | | ~~P7 / naming — the four unnamed builds~~ | ~~which locale and variant is each of entries 0, 1, 12, 15?~~ | — | ✅ **answered 2026-08-29** (`docs/re/ui-title-build-map.md`): all four are the loading screen, two variants — plain (7 elements) and dressed (10) — decoded from their own `pgloading_*` element names. ⚠️ **Not adopted as names yet, for two reasons the RE agent gave and one the port found.** Theirs: the executable names exactly two, and *which* bundle takes which name is 🟡 undecided, so `LOADING`/`LOADING2` must not go in an asset path; and locale is 🟡 — the English member of a pair is the one in the first half of `GP_TITLE.p00`, 8/8 structurally but only 3/3 where a capture can check, and the three pairs that matter are the three no capture can check. Mine: **the message gives the bundles as "0/1 and 10/11", which is the `is_build` ordinal, and `authored/screen_names.json` is keyed by PAK ENTRY** — in entry space 10 and 11 are `palogo_sqex` and `palogo_gamearts`, the splashes. See the refutation section in `DECISIONS.md`. | +## Coverage hole in my own check, 2026-08-31 — derived from HANDOFF `0159527` + +| Milestone | Needs | HANDOFF | State | +|---|---|---|---| +| P4/P7 — band check sensitivity | **nothing from anybody; my check does not cover `S00A`'s top end** | `0159527` | 🔴 **MEASURED GAP, reported per asset rather than hidden or asserted.** A 6 kHz-lowpassed source — a transcode that lost its whole top end — deviates **4.27 dB on `ADV` (covered, 2.8×)** and **1.28 dB on `S00A`, under the 1.5 dB pass threshold**, because `S00A`'s own 6–16 kHz content sits at −67 dB. **So that failure would pass on `S00A`.** Found by building the *hard* negative after the Decoder measured that unrelated music banks separate by 5.28 dB where an unrelated movie gave me 19–20 — a movie is an easy negative. Splitting the top band raised `ADV` from 2.58 to 4.27 dB; the pass threshold is unchanged. ⚠️ Not asserted, because a permanently red suite on a gap I cannot close today helps nobody; printed as **COVERED / NOT COVERED** per asset so it cannot become scenery. | + ## Open on my own side, 2026-08-30 — derived from HANDOFF `0159527` | Milestone | Needs | HANDOFF | State | diff --git a/docs/port/DECISIONS.md b/docs/port/DECISIONS.md index 455c3bc9..c0d23793 100644 --- a/docs/port/DECISIONS.md +++ b/docs/port/DECISIONS.md @@ -9,7 +9,7 @@ dies, which is what this file is for. -261 sections. Search this before re-deriving anything. +263 sections. Search this before re-deriving anything. * [P0 — the exporter, 2026-08-28](#p0--the-exporter-2026-08-28) * [P1 — Godot draws the screen, 2026-08-28](#p1--godot-draws-the-screen-2026-08-28) @@ -272,6 +272,8 @@ dies, which is what this file is for. * [🔴 My seek trap was over-general — the Decoder narrowed it](#my-seek-trap-was-over-general--the-decoder-narrowed-it) * [A capital letter hid a refuted claim in the file whose job is to say what is open](#a-capital-letter-hid-a-refuted-claim-in-the-file-whose-job-is-to-say-what-is-open) * [The difference path cannot verify a lossless encode — so nothing it says counts](#the-difference-path-cannot-verify-a-lossless-encode--so-nothing-it-says-counts) +* [The identity rule, turned back on my own newest tool — and it was biased](#the-identity-rule-turned-back-on-my-own-newest-tool--and-it-was-biased) +* [Their refutation attempt on my band check found a coverage hole and two defects](#their-refutation-attempt-on-my-band-check-found-a-coverage-hole-and-two-defects) ## P0 — the exporter, 2026-08-28 @@ -13440,3 +13442,100 @@ is unaffected and still asserts. 📌 What this changes about the milestone: **P4's waveform question is not merely open, it is open with a disqualified instrument.** That is worse than it looked yesterday and better than believing 8.73 dB meant something. + +## The identity rule, turned back on my own newest tool — and it was biased + +The Decoder ran my identity rule against their coherence estimator, it passed, and +they returned a sharper form of it: **a positive control that is merely "high" +hides the difference between an exact instrument and a lossy one.** Theirs read +0.94 for two reasons at once — a correct estimator *plus* a windowed delay — and +only the identity case could separate them. + +📌 **That lands on the band check I shipped yesterday and asserted in +`check-all`.** Its positive control was **0.29 and 0.66 dB** — small, and *small +is not zero*. A systematic bias would sit inside 0.66 dB looking like a pass. + +Adding source-against-itself: **7.656 dB.** Larger than the number the check calls +faithful. + +### The bias was in the control's construction, not the measurement + +`bands()` applies the fold to the **left side only**, which is correct for the +real comparison — a 5.1 source needs folding, an already-stereo transcode does +not. Applied to source-against-itself, that same asymmetry compares a folded +signal against a raw six-channel average. The fold is now per-side, and identity +reads **0.000 dB, exact**. + +✅ **The published 0.66 dB is unchanged** — that comparison was always +asymmetric-by-design and remains correct. What changed is that the instrument is +now **known unbiased** rather than assumed to be, and the check has three +separated populations instead of two: + +| | | +|---|---| +| identity | **0.000 dB — exact** | +| source vs its own transcode | 0.29 / 0.66 dB | +| source vs an unrelated movie | 19.10 / 20.02 dB | + +**The bottom of that scale is now anchored rather than inferred**, which is the +whole difference between "0.66 is small" and "0.66 is small *compared with zero, +measured*". + +📌 The general rule, now stated in the form that catches both our cases: **a +control that establishes only an upper bound on error cannot distinguish an exact +instrument from a slightly wrong one — and "slightly wrong" is the interesting +failure, because it passes.** Mine was one function argument. Theirs was one line. +Both were available from the day the tool was written. + +## Their refutation attempt on my band check found a coverage hole and two defects + +They tried to refute *"band energies need no alignment"*. **It survives** — 1 s of +misalignment costs 0.16 dB, well inside the pass band — but they narrowed it +correctly: at **10 s the cost reaches 1.00 dB**, because a fixed analysis window +covers different material once the shift is large. *"Needs no alignment"* was my +wording and it was too strong; the tool now says **robust to misalignment, not +free of it**. + +Their second point is the one that mattered: **the separation margin is +material-dependent.** Two unrelated music banks separate by only 5.28 dB where an +unrelated movie gave me 19–20. **A movie is a very easy negative.** So I built the +*hard* one — the failure this check exists to catch — and it failed. + +### 🔴 A 6 kHz-lowpassed source: `ADV` caught it, `S00A` does not + +| | worst band deviation | +|---|---| +| real transcodes | 0.29 / 0.67 dB | +| **6 kHz lowpass, `ADV`** | **4.27 dB — covered, 2.8×** | +| **6 kHz lowpass, `S00A`** | **1.28 dB — NOT COVERED, under the 1.5 dB threshold** | +| unrelated movie | 21.78 / 22.55 dB | + +**A transcode that lost everything above 6 kHz would pass this check on `S00A`**, +because `S00A`'s own 6–16 kHz content sits at −67 dB — removing it changes almost +nothing. The check's sensitivity is a property of the *material*, which is the +Decoder's negative-separation finding arriving on the positive side. + +📌 On the way, splitting the top band raised `ADV`'s detection from 2.58 to +4.27 dB. **That is changing the instrument's resolution so it can see a failure it +must see, driven by a control it failed — not loosening the pass threshold**, +which is unchanged. The distinction is the whole difference between fixing an +instrument and fitting one. + +⚠️ Reported per asset as **COVERED / NOT COVERED** rather than asserted: making +the suite permanently red on a gap I cannot close today helps nobody, and hiding +it turns a coverage hole into scenery. Tracked in `BLOCKED.md`. + +### 🔴 And repairing it exposed two defects that had been hiding each other + +* **`return 0` was unconditional.** Making the difference path report-only + swallowed the band verdict with it, so `check-all`'s `transcode-bands + must-pass` step **could not fail** — an asserting step that asserts nothing, + shipped by me, one day after I wrote up the same shape in someone else's work. + Band failures were being printed and discarded. +* **The disqualified difference path was still voting on the exit code.** Fixing + the return turned the run red for that reason rather than the real one. + +📌 **Two defects hiding each other**: with the return broken, the voting bug was +invisible; with the voting bug present, fixing the return produced a red run for +the wrong cause. Neither would have surfaced without building a control the tool +could fail — which is the argument for hard negatives in one line. diff --git a/tools/port/verify-capture b/tools/port/verify-capture index 2e1dbe3e..3d11a038 100755 --- a/tools/port/verify-capture +++ b/tools/port/verify-capture @@ -138,6 +138,38 @@ echo "RMSE is reported and is NOT a target: the capture carries the game's own" echo "tone ramp, so it has a floor. What finds a real defect is the DIFFERING" echo "REGION -- a missing or misplaced element is a large connected blob." echo +# 🔴 THE METRIC'S OWN ZERO, asserted before any row is printed. +# +# Every number below is "small is good", and this file already says the RMSE has +# a floor from the game's tone ramp. What was never established is the floor of +# the COMPARISON ITSELF. A control that only bounds error from above cannot tell +# an exact instrument from a slightly wrong one -- and slightly wrong is the +# failure that passes. The Decoder reached that form of it after their coherence +# estimator's positive control read 0.94 for two reasons at once. +# +# Measured here rather than assumed: a capture against itself, and against a PNG +# round-trip of itself, must both be EXACTLY 0. If they are not, the metric has a +# bias and no row below means what it says. +_ctl="" +for row in "${MAP[@]}"; do + IFS=: read -r _n _c _rest <<<"$row"; [ -f "$_c" ] && { _ctl="$_c"; break; } +done +if [ -n "$_ctl" ]; then + _rt="${TMPDIR:-/tmp}/verify-capture-rt.png"; convert "$_ctl" -quality 100 "$_rt" + for _pair in "$_ctl|$_ctl|identity" "$_ctl|$_rt|PNG round-trip"; do + IFS='|' read -r _a _b _lab <<<"$_pair" + _d=$(convert "$_a" "$_b" -metric RMSE -compare -format "%[distortion]" info: 2>&1 | tail -1) + _v=$(python3 -c "print('%.4f' % (float('$_d')*255))" 2>/dev/null || echo "?") + if [ "$_v" = "0.0000" ]; then + printf ' metric control, %-16s RMSE %s -- exact\n' "$_lab:" "$_v" + else + printf ' 🔴 metric control, %-13s RMSE %s -- NOT ZERO. The comparison is\n' "$_lab:" "$_v" + echo " biased and every row below is unreadable. Refusing." + exit 3 + fi + done + echo +fi printf '%-17s %-9s %-7s %-22s %s\n' screen raw-rmse diff region note for row in "${MAP[@]}"; do IFS=: read -r name cap pose forced capcrop <<<"$row" diff --git a/tools/port/verify-transcode-fidelity b/tools/port/verify-transcode-fidelity index 1d79415c..e7033422 100755 --- a/tools/port/verify-transcode-fidelity +++ b/tools/port/verify-transcode-fidelity @@ -153,7 +153,17 @@ def align(src, dst, af): return int(round(coarse * RATE)), c -BANDS = [(0, 500), (500, 2000), (2000, 6000), (6000, 16000)] +# 🔴 THE TOP BAND IS SPLIT BECAUSE THE NEAR-MISS CONTROL FAILED. With a single +# 6-16 kHz band, a 6 kHz-lowpassed source -- a transcode that lost its whole top +# end, the failure this check exists to catch -- deviated by only 2.58 dB and +# would have PASSED. The band was wide enough to average the loss away against +# the filter's transition region. +# +# ⚠️ This is changing the instrument's RESOLUTION so it can see a failure it must +# see, driven by a control it failed. It is NOT loosening the pass threshold for +# the real comparison, which is unchanged -- that would be tuning until the +# answer came out right, which is the thing this project keeps catching. +BANDS = [(0, 500), (500, 2000), (2000, 6000), (6000, 10000), (10000, 16000)] def band_db(path, af, lo, hi, seconds=25.0, skip=2.0): @@ -188,13 +198,28 @@ def band_db(path, af, lo, hi, seconds=25.0, skip=2.0): return float(m.group(1)) -def bands(src, dst, af, label): - """Per-band level, source against transcode. No alignment involved.""" +def bands(src, dst, af, label, af_dst=None): + """Per-band level, source against transcode. ROBUST to misalignment, not free of it. + + ⚠️ CLAIM NARROWED 2026-08-31 after the Decoder tried to refute it. It survives + -- **1 s of misalignment costs 0.16 dB**, well inside the 1.5 dB pass band -- + but it is **not literally alignment-free**: at **10 s the cost reaches 1.00 dB**, + because a fixed analysis window covers different material once the shift is + large relative to it. "Needs no alignment" was my wording and it was too + strong; the honest claim is robustness up to a few seconds. + + 🔴 THE FOLD IS PER-SIDE, and the identity control is what made that + necessary. `af` applies to the LEFT side only, which is correct for the real + comparison -- a 5.1 source needs folding, an already-stereo transcode does + not. Applying that same asymmetry to source-against-itself compares a folded + signal with a raw six-channel average and reports **7.656 dB on an + identity**, larger than the 0.66 dB this check calls a pass. + """ print(f" {label}") worst = 0.0 for lo, hi in BANDS: a = band_db(src, af, lo, hi) - b = band_db(dst, None, lo, hi) + b = band_db(dst, af_dst, lo, hi) if a is None or b is None: print(f" {lo:>5}-{hi:<5} Hz one side silent -- no comparison") continue @@ -283,7 +308,7 @@ def main(): fail += 1 continue af = downmix_of(man, name) - worst = bands(src, dst, af, f"{name} -- band energies (needs no alignment)") + worst = bands(src, dst, af, f"{name} -- band energies (robust to misalignment, not free of it)") verdict = "ok" if worst <= PASS_BAND_DB else "🔴 OUT OF TOLERANCE" print(f" worst band deviation {worst:.2f} dB {verdict}") if worst > PASS_BAND_DB: @@ -292,6 +317,59 @@ def main(): # that has only ever seen a faithful pair cannot be told from one that # compares a file with itself by accident -- and this tool has already # produced four confident wrong numbers on the other quantity. + # 🔴 THE IDENTITY CONTROL, added 2026-08-31 after the Decoder generalised + # my own rule back at me: **a positive control that is merely "high" + # hides the difference between an exact instrument and a lossy one.** + # This check's positive side was 0.29 and 0.66 dB -- small, and small is + # not zero. A systematic bias (the fold applied to one side only, a + # different window, a resampler difference) would sit inside 0.66 dB + # while looking like a pass. Source against itself must be EXACTLY 0.00 + # in every band, and anything else is the instrument, not the transcode. + ident = bands(src, src, af, " control: source vs ITSELF, must be exact", af_dst=af) + idv = "ok" if ident == 0.0 else f"🔴 {ident:.3f} dB on an identity -- the instrument is biased" + print(f" worst band deviation {ident:.3f} dB {idv}") + if ident != 0.0: + fail += 1 + # 🔴 A NEAR-MISS NEGATIVE, because an unrelated movie is an EASY one. + # The Decoder measured two unrelated music BANKS separating by just + # 5.28 dB where an unrelated movie gave me 19-20, so the margin against a + # hard negative is 8x, not 30x. The negative that matters is the failure + # this check exists to catch: a transcode that lost its top end. A 6 kHz + # lowpass of the source is that failure, constructed. + low = bands(src, src, af, " control: 6 kHz-lowpassed source, must be caught", + af_dst=(af + "," if af else "") + "lowpass=f=6000") + # Judged against THE CHECK'S OWN pass threshold, not an invented 3x. + # + # With the top band split this lands at 4.27 dB: it fails the 1.5 dB pass + # test, so the check does catch it -- but by 2.8x, against the 6.4x it + # has over the worst real transcode (0.67 dB). ⚠️ NOT COMFORTABLE, and + # said out loud rather than smoothed: a loss milder than a 6 kHz brick + # wall could sit between 0.67 and 1.5 and pass. The honest statement is + # that this check catches a SEVERE top-end loss and is not characterised + # for a mild one. + # + # The 3x bar it used to be judged against was mine and stricter than the + # check itself; using the check's own threshold is the principled + # criterion, and lowering the 3x to make a failing control pass would + # have been tuning. + # 🔴 REPORTED PER ASSET, NOT ASSERTED, and the reason is a measured gap + # rather than convenience. `ADV` catches the lowpass by 2.8x. **`S00A` + # does not catch it at all** -- 1.28 dB against a 1.5 dB threshold -- + # because its own 6-16 kHz content sits at -67 dB, so removing it changes + # almost nothing. The check's sensitivity is MATERIAL-DEPENDENT, which is + # the Decoder's finding about negative-separation arriving on the + # positive side. + # + # Asserting it would make the suite permanently red on a gap I cannot + # close today; hiding it would make a coverage hole into scenery. So it + # prints COVERED / NOT COVERED per asset and the gap is tracked in + # BLOCKED.md. The identity and unrelated-movie controls still assert. + if low > PASS_BAND_DB: + print(f" worst band deviation {low:.2f} dB COVERED, caught by" + f" {low / PASS_BAND_DB:.1f}x") + else: + print(f" worst band deviation {low:.2f} dB 🔴 NOT COVERED --" + f" a 6 kHz top-end loss on {name} would PASS this check") other = [v for v in man["videos"] if v["name"] != name] if other: osrc = os.path.join("export", other[0]["file"]) @@ -304,9 +382,11 @@ def main(): if af is None: print(f" {name:<28} ⚠️ no `-af` in the recorded command: the source" " is stereo, comparing without a fold") - down = compare(src, dst, af, name) - if down is None or down < PASS_DB: - fail += 1 + # Report-only: a disqualified path must not vote on the exit code. It + # did, which is why the run went red for the wrong reason the moment the + # return was fixed -- two defects hiding each other, and repairing one + # exposed the other rather than the run going quietly green. + compare(src, dst, af, name) if control: print(f" known negatives for {name}:") same = compare(src, src, af, " source vs itself") @@ -337,6 +417,15 @@ def main(): print(" What this run DOES establish is the trap list below, each reproduced") print(" here rather than reasoned about. See docs/port/DECISIONS.md.") print(" 🔴 It measures AUDIO only; `-q:v 8` was chosen on SSIM separately.") + # 🔴 THIS RETURN WAS UNCONDITIONAL `return 0` FOR A DAY. Making the difference + # path report-only swallowed the band verdict with it, so `check-all`'s + # `transcode-bands must-pass` step COULD NOT FAIL -- an asserting step that + # asserts nothing, which is the exact shape this project keeps finding in + # other people's work and had now shipped in mine. The band failures were + # being printed and discarded. + if fail: + print(f"\n🔴 {fail} band control failure(s)") + return 1 return 0