method: a harness self-test, and the empty-register hole it found on its first run

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wuu56cE8vJGTBtn1ppsk8v
This commit is contained in:
sylph-decoder
2026-08-31 01:12:09 +00:00
parent 244648ae97
commit 481265895d

View File

@@ -2776,3 +2776,27 @@ failure was process rather than analysis:
📌 **An uncommitted tool is one timed-out command away from being unrecoverable,
and a tool edited during its own run is not the tool that ran.** Commit first, then
run. Cheap, and it would have prevented all four.
## A harness self-test — and the hole it found in mine on the first run
`sylpheed-port` closed this gap first, and named it precisely: their controls
asserted **failure-on-perturbation**, but nothing asserted that a **broken harness
reports broken**. Their test feeds the machinery a stub that cannot fail — prints
"everything is fine", asserts nothing — and requires it to be flagged.
✅ **Built the equivalent for `check_refuted.py`, and it found a real hole on its
first run**: a register that parses **no claims** reported **clean**, forever. That
is the same shape as their stub, sitting in the tool that guards the shared
register and whose clean runs both agents lean on. Fixed — it now refuses rather
than passing.
📌 Two details worth copying. **The self-test drives the real machinery as a
subprocess and reads its actual exit code**; their first version *reasoned* that
the control would flag the stub — the error this whole thread is about, committed
inside the tool built to prevent it. And **their exit convention separates the two
failures that matter**: `0` fine · `1` a real check failed · `2` the harness is
broken and nothing it reported can be trusted. A single non-zero cannot say which.
⚠️ **Filed, not fixed**: `impossibility_scope.py`, `bgm_stem_coherence.py` and
`ring_row.py` have controls and **no harness self-test**. Shape known, fix cheap,
not done — recorded so their absence is a stated gap rather than scenery.