fabi/Sylpheed
1
0
Fork 0
Code Issues 9 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

re: check_refuted.py's clean run was not a pass -- 8 of 8 mentions were suppressed silently

Browse Source 
A real revival planted inside a paragraph that merely discussed corrections was
missed: the words 'refuted' and 'withdrawn' in the surrounding prose vouched for it.
Measured reach: 100 % of registered-claim mentions in the corpus are suppressed by
marker language, so the reported 0 was 0 regardless of whether any was live, and I
had been reading it as a pass.

sylpheed-port's token-based hook has the opposite bias -- it over-reports on
well-written corrections, which is the safe direction. Under-reporting is disguised
as success.

Fixed by making the suppression visible rather than removing it: suppressed mentions
are counted and listed with --show-marked as not verified, only vouched for.
Controlled -- the planted revival moves the suppressed count 8 -> 9 and appears in
the listing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wuu56cE8vJGTBtn1ppsk8v
This commit is contained in:
sylph-decoder
2026-08-30 20:27:17 +00:00
parent 3560643eee
commit 125a207fde
2 changed files with 54 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
docs/re/METHOD.md
View File

@@ -2310,3 +2310,33 @@ a regression, and it is the same term I measured from the capture side as a
phase-locked shutter — arrived at independently, from the other end of the
pipeline. A row containing a sweeping leaf has a **phase-dependent value**, so
quoting one without the term attached is an error whichever side produces it.
## 🔴 `check_refuted.py`'s clean run was not a pass — measured, and it is my instrument
`sylpheed-port` found their hook's cost is **per-mention, not per-correction**, and
that mentions multiply exactly when writing *about* the mechanism. Testing the same
thing on mine showed the opposite bias, in the worse direction.
**A real revival, planted inside a paragraph that merely discussed corrections, was
missed silently.** The words "refuted" and "withdrawn" in the surrounding prose
vouched for it. Then the reach: **8 of 8** mentions of a registered claim in this
corpus are suppressed by marker language — **100 %**. So the reported count was `0`
whether or not any of them was live, and I had been reading that 0 as a pass.
⚠️ **Their token over-reports; this one under-reported.** Over-reporting is the safe
direction — it costs attention. Under-reporting costs the thing the check exists for,
and it is disguised as success. A detector whose null result is indistinguishable
from its positive result measures nothing.
**Fixed by making the blind spot visible rather than by removing it**: suppressed
mentions are now counted and listed (`--show-marked`) as *"NOT verified, only
vouched for by neighbouring prose"*. The planted revival moves the suppressed count
8 → 9 and appears in the listing, so it is surfaced rather than silently absorbed.
Marker language stays — dropping it re-creates the header false positives — but it
now downgrades a hit instead of erasing it.
📌 **The general rule: never let a check's suppression path be silent.** If a
detector can discard a candidate, it must say how many it discarded, or its clean
run is unfalsifiable. Both of us reached the same structural conclusion from
opposite failures within a day — theirs by over-reporting loudly, mine by passing
quietly, which is why mine went unnoticed and theirs did not.