From 75be660fb4ea1cd534a7581b56f05932ad1ed059 Mon Sep 17 00:00:00 2001 From: Sylpheed port agent Date: Mon, 31 Aug 2026 01:10:30 +0000 Subject: [PATCH] port: the control harness now asserts itself, and it caught me twice doing it The gap I named and the Decoder prioritised: every --control run asserts that each check fails on a perturbed contract, and none asserted that a broken control reports broken. That is printing a verdict without asserting it, one level up. A harness that silently approves a dead check is exactly as useless as a check that silently approves a dead value. contract-check --selftest feeds the machinery a stub that cannot fail -- a function that prints 'everything is fine' and asserts nothing, which is precisely the defect I shipped in verify-transcode-fidelity's unconditional return 0 -- and requires the machinery to flag it. Exit codes follow the Decoder's convention: 0 all good, 1 a real check failed, 2 the HARNESS is broken and nothing it reported can be trusted. Asserting in check-all. It caught two defects while being written. The first version checked that the stub left the failure counter at zero and then REASONED that control() would therefore flag it -- arguing where a measurement was available, the error this whole thread has been about, committed inside the tool built to prevent it. Rewritten to push the stub through the real control() loop and read its verdict. It then returned 2 immediately: the stub was flagged, but as 'the control's own anchor is gone' rather than as a dead check, because the src selection anchored anything not in one specific list at the walk document instead of HANDOFF. A real failure for a fabricated reason, which is the confusion ANCHOR SPLIT exists to separate. Not covered and filed rather than left looking finished: check-claims, audit-kinds and verify-transcode-fidelity have controls and no harness self-test. The shape is known and the fix is cheap. Every asserting check passes. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01N7FiFFFwbvG2uxdcEh8HyF --- docs/port/BLOCKED.md | 6 ++++ docs/port/DECISIONS.md | 44 +++++++++++++++++++++++++++- tools/port/check-all | 5 ++++ tools/port/contract-check | 61 +++++++++++++++++++++++++++++++++++++-- 4 files changed, 112 insertions(+), 4 deletions(-) diff --git a/docs/port/BLOCKED.md b/docs/port/BLOCKED.md index b28bdfa8..2876e5b5 100644 --- a/docs/port/BLOCKED.md +++ b/docs/port/BLOCKED.md @@ -147,6 +147,12 @@ HANDOFF. | ~~P1–P7 — the keyframe record layout~~ | ~~adopt the corrected pose/time pairing~~ | — | ✅ **ADOPTED 2026-08-29 by pinning `formats-pin-2026-08-29c`.** This row was wrong twice: it said the change *"cannot be taken yet"* and that it *"reaches the port only when that branch lands on `main`"*. **It arrives when the tag is pinned**, which is what MISSION §2's tagging rule exists for. ⚠️ And the knob I tested first, `SYLPHEED_KF_TIME_SHIFT`, is a **retired partial fix** that left pose 0 untimed — the real correction is the tagged crate's default, with the old reading behind `SYLPHEED_KF_TIME_LEGACY=1`. **The blast radius was far smaller than this row predicted**: under the correction *every pose is timed* (866 keyframes, 0 untimed), so `pose_at`'s synthetic-exit branch became dead code rather than wrong code and nothing needed re-deriving. Oracle: `publisher_logo` 1.00 %→**0.75 %**, `developer_logos` 0.39 %→**0.33 %**, `extras`' differing region collapsing from 736×525 to **398×295 at the sweep position**. 🔴 Open cost: `sylpheed-cli` builds from the workspace crate, so `verify-screen` compares two decoder eras until the tag reaches `main`. Revert to the path dependency then. | | ~~P7 / naming — the four unnamed builds~~ | ~~which locale and variant is each of entries 0, 1, 12, 15?~~ | — | ✅ **answered 2026-08-29** (`docs/re/ui-title-build-map.md`): all four are the loading screen, two variants — plain (7 elements) and dressed (10) — decoded from their own `pgloading_*` element names. ⚠️ **Not adopted as names yet, for two reasons the RE agent gave and one the port found.** Theirs: the executable names exactly two, and *which* bundle takes which name is 🟡 undecided, so `LOADING`/`LOADING2` must not go in an asset path; and locale is 🟡 — the English member of a pair is the one in the first half of `GP_TITLE.p00`, 8/8 structurally but only 3/3 where a capture can check, and the three pairs that matter are the three no capture can check. Mine: **the message gives the bundles as "0/1 and 10/11", which is the `is_build` ordinal, and `authored/screen_names.json` is keyed by PAK ENTRY** — in entry space 10 and 11 are `palogo_sqex` and `palogo_gamearts`, the splashes. See the refutation section in `DECISIONS.md`. | +## Open on my own tooling, 2026-08-31 — derived from HANDOFF `d38adcf` + +| Milestone | Needs | HANDOFF | State | +|---|---|---|---| +| all — control harnesses that assert themselves | **nothing from anybody; three tools still lack it** | `d38adcf` | 🟡 **DONE FOR `contract-check`, NOT for the rest.** `--selftest` feeds the machinery a stub that cannot fail and requires it to be flagged; exit codes separate **0** all good / **1** a real check failed / **2** the harness is broken. Asserting in `check-all`. ⚠️ `check-claims`, `audit-kinds` and `verify-transcode-fidelity` have controls and **no harness self-test** — the shape is known and the fix is cheap, and this row exists so the gap does not read as finished. 🔴 The self-test caught two defects while being written: a first version that *argued* the harness would flag the stub instead of measuring it, and a `src` selection that anchored anything outside one list at the wrong document, flagging the stub for a fabricated reason. | + ## Coverage hole in my own check, 2026-08-31 — derived from HANDOFF `0159527` | Milestone | Needs | HANDOFF | State | diff --git a/docs/port/DECISIONS.md b/docs/port/DECISIONS.md index b342e67c..7c40b1b9 100644 --- a/docs/port/DECISIONS.md +++ b/docs/port/DECISIONS.md @@ -9,7 +9,7 @@ dies, which is what this file is for. -265 sections. Search this before re-deriving anything. +266 sections. Search this before re-deriving anything. * [P0 — the exporter, 2026-08-28](#p0--the-exporter-2026-08-28) * [P1 — Godot draws the screen, 2026-08-28](#p1--godot-draws-the-screen-2026-08-28) @@ -276,6 +276,7 @@ dies, which is what this file is for. * [Their refutation attempt on my band check found a coverage hole and two defects](#their-refutation-attempt-on-my-band-check-found-a-coverage-hole-and-two-defects) * [🔴 RETRACTED: the `S00A` coverage hole was my control's filter, not the check](#retracted-the-s00a-coverage-hole-was-my-controls-filter-not-the-check) * [Their two tools had the shape I shipped, and the general form is sharper now](#their-two-tools-had-the-shape-i-shipped-and-the-general-form-is-sharper-now) +* [Closing the two-directional gap: the control harness now asserts itself](#closing-the-two-directional-gap-the-control-harness-now-asserts-itself) ## P0 — the exporter, 2026-08-28 @@ -13598,3 +13599,44 @@ close here. 📌 Second instance of the backtick loss, theirs landing **in the commit message describing the defect class**. Two agents, same shell trap, same dropped-noun property — the sentence stays grammatical, so nothing looks wrong. + +## Closing the two-directional gap: the control harness now asserts itself + +The gap I named and the Decoder prioritised: **every `--control` run asserts that +each check fails on a perturbed contract; none asserted that a broken control +reports broken.** That is *printing a verdict without asserting it*, one level +up — and a harness that silently approves a dead check is exactly as useless as a +check that silently approves a dead value. + +`contract-check --selftest` feeds the machinery a **stub that cannot fail** — a +function that prints "everything is fine" and asserts nothing, which is precisely +the defect I shipped in `verify-transcode-fidelity`'s unconditional `return 0` — +and requires the machinery to flag it. Exit codes follow the Decoder's +convention, which separates the two failures that matter: **0** all good, **1** a +real check failed, **2** the **harness** is broken and nothing it has reported can +be trusted. Now an asserting step in `check-all`. + +### 🔴 It caught me twice while being written + +* **The first version argued instead of measuring.** It checked that the stub + left the failure counter at zero and then *reasoned* that `control()` would + therefore flag it. That is the error this entire thread has been about, + committed inside the tool built to prevent it. Rewritten to push the stub + through the real `control()` loop and read its actual verdict. +* **Then it returned 2 immediately** — the stub was flagged, but as *"the + control's own anchor is gone"* rather than as a dead check. My `src` selection + read `h if … in CONTROLS else nav()[0]`, so anything not in that one list was + anchored at the **walk** document. **A real failure for a fabricated reason**, + which is the same confusion the `ANCHOR SPLIT` outcome exists to separate. + Inverted to test membership in `NAV_CONTROLS` instead. + +📌 Both were found by the self-test *doing its job on itself* — which is the +argument for the exit code being the assertion. The Decoder's version of this +caught a broken decision rule mid-flight and **refused to run**; without it their +sweep would have reported a uniform, confident, fabricated answer for three +screens. + +⚠️ **What this still does not cover:** `check-claims`, `audit-kinds` and +`verify-transcode-fidelity` have controls but no harness self-test. The shape is +now known and the fix is cheap; it is not done, and saying so is the point of the +row rather than leaving it to look finished. diff --git a/tools/port/check-all b/tools/port/check-all index 821920ca..6f055654 100755 --- a/tools/port/check-all +++ b/tools/port/check-all @@ -44,6 +44,11 @@ step format-validator must-pass "$BIN" check # have been reduced to a check; the rest are still read by eye, or not at all. step contract-values must-pass tools/port/contract-check step contract-control must-pass tools/port/contract-check --control +# 🔴 The control harness itself is asserted. Every --control run says "each check +# fails on a perturbed contract"; none of them said "a broken control reports +# broken". A harness that silently approves a dead check is exactly as useless as +# a check that silently approves a dead value. +step control-harness must-pass tools/port/contract-check --selftest step modding-rules must-pass tools/port/check-modding # Every `kind` in authored/ is a claim about where a value came from, and until # 2026-08-30 nothing checked what any of them rested on -- seven were resting on diff --git a/tools/port/contract-check b/tools/port/contract-check index 82d925e3..615bd2f4 100755 --- a/tools/port/contract-check +++ b/tools/port/contract-check @@ -219,13 +219,66 @@ def fn_nav_perturbed(fn, old, new): nav = real -def control(h): +def selftest(h): + """Does the CONTROL MACHINERY notice a check that cannot fail? + + 🔴 THE GAP THIS CLOSES, named by me and prioritised by the Decoder: every + `--control` run asserts that each check FAILS on a perturbed contract. None + of them asserted that a **broken control reports broken**. That is the same + shape as printing a verdict without asserting it, one level up — and a + control harness that silently approves a dead check is exactly as useless as + a check that silently approves a dead value. + + So a stub check that can never fail is fed to the machinery, and the + machinery must flag it. If the stub comes back "✅ fails as it must", the + harness is broken and says so with its own exit code. + + Exit codes follow the Decoder's convention, which distinguishes the two + failures that matter: **0** all good, **1** a real check failed, **2** the + HARNESS is broken and nothing it reported can be trusted. + """ + import io, contextlib + + def always_ok(_h): + # Prints a verdict and asserts nothing -- the exact defect shipped in + # `verify-transcode-fidelity`'s unconditional `return 0`. + print(" stub: everything is fine") + + # 🔴 RUN THE REAL MACHINERY OVER THE STUB. A first version of this checked + # that the stub left FAIL at zero and then ARGUED that `control` would + # therefore flag it. That is reasoning where a measurement was available -- + # the error this whole thread has been about -- so the stub goes through the + # same `control()` loop the real checks do, and its verdict is read. + with contextlib.redirect_stdout(io.StringIO()) as buf: + verdict = control(h, extra=[(always_ok, "120", "121")]) + out = buf.getvalue() + stub_line = [l for l in out.splitlines() if "always_ok" in l] + if verdict is not False or not stub_line: + print(" 🔴 HARNESS BROKEN: the control machinery did not flag a check that") + print(" cannot fail. Nothing any `--control` run has reported is trustworthy.") + print(f" stub verdict: {verdict!r}; line: {stub_line}") + return 2 + if "PASSES A WRONG CONTRACT" not in stub_line[0]: + print(f" 🔴 HARNESS BROKEN: stub flagged, but not as a dead check: {stub_line[0].strip()}") + return 2 + print(" harness self-test: a check that cannot fail is flagged by the machinery ✅") + print(f" {stub_line[0].strip()}") + print(" Exit codes: 0 all good, 1 a real check failed, 2 the HARNESS is broken.") + return 0 + + +def control(h, extra=None): global FAIL import io, contextlib ok = True print(" known negatives -- every check must notice a perturbed contract:\n") - for fn, old, new in CONTROLS + [(f, o, n) for f, o, n in NAV_CONTROLS]: - src = h if (fn, old, new) in CONTROLS else nav()[0] + for fn, old, new in CONTROLS + [(f, o, n) for f, o, n in NAV_CONTROLS] + (extra or []): + # Membership tested against NAV_CONTROLS, not CONTROLS: anything else -- + # including a self-test stub passed in via `extra` -- is anchored on + # HANDOFF. Written the other way round, the stub was routed at the walk + # and flagged "the control's own anchor is gone", a real failure for a + # fabricated reason. + src = nav()[0] if (fn, old, new) in NAV_CONTROLS else h if old not in src: print(f" {fn.__name__:<22} 🔴 the control's own anchor is gone") ok = False @@ -426,6 +479,8 @@ def main(): sys.exit("no export/ -- run the exporter first; this check reads what is shipped") h = contract() print() + if "--selftest" in sys.argv: + return selftest(h) if "--control" in sys.argv: return 0 if control(h) else 1 for fn in (check_fade_quads, check_fade_out, check_plate_period,