fabi/Sylpheed
1
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity

port: the control harness now asserts itself, and it caught me twice doing it

Browse Source 
The gap I named and the Decoder prioritised: every --control run asserts that each
check fails on a perturbed contract, and none asserted that a broken control
reports broken. That is printing a verdict without asserting it, one level up. A
harness that silently approves a dead check is exactly as useless as a check that
silently approves a dead value.

contract-check --selftest feeds the machinery a stub that cannot fail -- a
function that prints 'everything is fine' and asserts nothing, which is precisely
the defect I shipped in verify-transcode-fidelity's unconditional return 0 -- and
requires the machinery to flag it. Exit codes follow the Decoder's convention: 0
all good, 1 a real check failed, 2 the HARNESS is broken and nothing it reported
can be trusted. Asserting in check-all.

It caught two defects while being written. The first version checked that the stub
left the failure counter at zero and then REASONED that control() would therefore
flag it -- arguing where a measurement was available, the error this whole thread
has been about, committed inside the tool built to prevent it. Rewritten to push
the stub through the real control() loop and read its verdict. It then returned 2
immediately: the stub was flagged, but as 'the control's own anchor is gone'
rather than as a dead check, because the src selection anchored anything not in
one specific list at the walk document instead of HANDOFF. A real failure for a
fabricated reason, which is the confusion ANCHOR SPLIT exists to separate.

Not covered and filed rather than left looking finished: check-claims, audit-kinds
and verify-transcode-fidelity have controls and no harness self-test. The shape is
known and the fix is cheap.

Every asserting check passes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N7FiFFFwbvG2uxdcEh8HyF
This commit is contained in:
Sylpheed port agent
2026-08-31 01:10:30 +00:00
parent 5e07346abf
commit 75be660fb4
4 changed files with 112 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
tools/port/check-all
View File

@@ -44,6 +44,11 @@ step format-validator must-pass "$BIN" check
# have been reduced to a check; the rest are still read by eye, or not at all.
step contract-values must-pass tools/port/contract-check
step contract-control must-pass tools/port/contract-check --control
# 🔴 The control harness itself is asserted. Every --control run says "each check
# fails on a perturbed contract"; none of them said "a broken control reports
# broken". A harness that silently approves a dead check is exactly as useless as
# a check that silently approves a dead value.
step control-harness must-pass tools/port/contract-check --selftest
step modding-rules must-pass tools/port/check-modding
# Every `kind` in authored/ is a claim about where a value came from, and until
# 2026-08-30 nothing checked what any of them rested on -- seven were resting on

61
tools/port/contract-check
View File

@@ -219,13 +219,66 @@ def fn_nav_perturbed(fn, old, new):
nav = real
def control(h):
def selftest(h):
"""Does the CONTROL MACHINERY notice a check that cannot fail?
🔴 THE GAP THIS CLOSES, named by me and prioritised by the Decoder: every
`--control` run asserts that each check FAILS on a perturbed contract. None
of them asserted that a **broken control reports broken**. That is the same
shape as printing a verdict without asserting it, one level up — and a
control harness that silently approves a dead check is exactly as useless as
a check that silently approves a dead value.
So a stub check that can never fail is fed to the machinery, and the
machinery must flag it. If the stub comes back "✅ fails as it must", the
harness is broken and says so with its own exit code.
Exit codes follow the Decoder's convention, which distinguishes the two
failures that matter: **0** all good, **1** a real check failed, **2** the
HARNESS is broken and nothing it reported can be trusted.
"""
import io, contextlib
def always_ok(_h):
# Prints a verdict and asserts nothing -- the exact defect shipped in
# `verify-transcode-fidelity`'s unconditional `return 0`.
print(" stub: everything is fine")
# 🔴 RUN THE REAL MACHINERY OVER THE STUB. A first version of this checked
# that the stub left FAIL at zero and then ARGUED that `control` would
# therefore flag it. That is reasoning where a measurement was available --
# the error this whole thread has been about -- so the stub goes through the
# same `control()` loop the real checks do, and its verdict is read.
with contextlib.redirect_stdout(io.StringIO()) as buf:
verdict = control(h, extra=[(always_ok, "120", "121")])
out = buf.getvalue()
stub_line = [l for l in out.splitlines() if "always_ok" in l]
if verdict is not False or not stub_line:
print(" 🔴 HARNESS BROKEN: the control machinery did not flag a check that")
print(" cannot fail. Nothing any `--control` run has reported is trustworthy.")
print(f" stub verdict: {verdict!r}; line: {stub_line}")
return 2
if "PASSES A WRONG CONTRACT" not in stub_line[0]:
print(f" 🔴 HARNESS BROKEN: stub flagged, but not as a dead check: {stub_line[0].strip()}")
return 2
print(" harness self-test: a check that cannot fail is flagged by the machinery ✅")
print(f" {stub_line[0].strip()}")
print(" Exit codes: 0 all good, 1 a real check failed, 2 the HARNESS is broken.")
return 0
def control(h, extra=None):
global FAIL
import io, contextlib
ok = True
print(" known negatives -- every check must notice a perturbed contract:\n")
for fn, old, new in CONTROLS + [(f, o, n) for f, o, n in NAV_CONTROLS]:
src = h if (fn, old, new) in CONTROLS else nav()[0]
for fn, old, new in CONTROLS + [(f, o, n) for f, o, n in NAV_CONTROLS] + (extra or []):
# Membership tested against NAV_CONTROLS, not CONTROLS: anything else --
# including a self-test stub passed in via `extra` -- is anchored on
# HANDOFF. Written the other way round, the stub was routed at the walk
# and flagged "the control's own anchor is gone", a real failure for a
# fabricated reason.
src = nav()[0] if (fn, old, new) in NAV_CONTROLS else h
if old not in src:
print(f" {fn.__name__:<22} 🔴 the control's own anchor is gone")
ok = False
@@ -426,6 +479,8 @@ def main():
sys.exit("no export/ -- run the exporter first; this check reads what is shipped")
h = contract()
print()
if "--selftest" in sys.argv:
return selftest(h)
if "--control" in sys.argv:
return 0 if control(h) else 1
for fn in (check_fade_quads, check_fade_out, check_plate_period,