fabi/Sylpheed
1
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity

port: add check-all; verify-screen ignored its own statistic; 'six expected DIFFERS' was wrong

Browse Source 
Eleven tools and nothing ran them together -- the ninth instance of correct,
documented and unexercised, one level up. check-all runs the four that assert,
reports the oracle table, and gives verify-screen an allowance that EXPIRES when
the pin lands rather than standing forever. All eleven exercised first; none had
rotted.

verify-screen computed over3 because 'a single max cannot tell 2 pixels from
25 444' and then decided the verdict on max alone: main_menu (max 4, over3 0)
read DIFFERS while extras (max 3, over3 0) read OK. The bar is unchanged; a frame
with no pixel over it now gets its own ROUNDING verdict.

And corrects a claim I have given the Decoder more than once. The real count was
ten, now eight: six forced-backdrop, two rounding, and TWO UNEXPLAINED -- title
at 790 px and title_jp at 20498, neither carrying a forced element. My leaf
hypothesis is refuted: emptying draw_leaf_for changes the numbers not at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N7FiFFFwbvG2uxdcEh8HyF
This commit is contained in:
Sylpheed port agent
2026-08-30 04:55:48 +00:00
parent 8b4ec4414d
commit fd6d9b6d47
4 changed files with 183 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

28
docs/port/BLOCKED.md
View File

@@ -602,3 +602,31 @@ tree. So the candidate fixes are about the *message*, not the order:
⚠️ Deliberately not chosen here: both change failure semantics across every tool,
and neither is measured against anything. It goes to whoever owns that call.
---
## `title` differs from `sylpheed-cli` on 790 pixels, and the leaf theory is refuted
*Derived from HANDOFF `9ca1eb5`. Raised 2026-08-30 by the port. Not urgent — the
port agrees with the **oracle** on this screen at 0.21 %, and that is the check
that counts.*
`verify-screen` has `title` at **790 pixels** over the bar and `title_jp` at
**20 498**, and neither carries a forced-backdrop element — so they are not
covered by the pinned-tag allowance the other six sit under. I had been reporting
"six expected DIFFERS"; the real count was ten.
🔴 **The obvious explanation is wrong.** `authored/rendering.json` records that the
consistency harness compares against a renderer drawing no `.rat` leaves, so the
`ptloop` sweeps were the candidate. Emptying `draw_leaf_for` and
`loop_leaf_on_screens` changes the figures **not at all** — the harness poses at
`rest`, where the leaves do not draw.
What would help: **which elements `sylpheed-cli` draws on build 4 at `rest`**, as a
list. The port's own list is in any `--screen=title` log (`drew N: …`). A set
difference answers it immediately, and it is a question about our own tool rather
than about the game — no oracle run, no emulator.
⚠️ Do not read this as the port being wrong. Against the **capture**, `title` is at
0.21 % and `title_plate` at 0.00093 %. This is two of our renderers disagreeing,
and the one with an oracle behind it is not the one under suspicion.

64
docs/port/DECISIONS.md
View File

@@ -6338,3 +6338,67 @@ claim about a tree, and a manifest for a tree that was never finished would be
worse. So this is **filed rather than fixed**: the behaviour is defensible and the
message is not, since "is that an export tree?" describes the symptom and hides
the cause. What a stranger needs to be told is *the last export failed; re-run it*.
## `check-all`, a verdict that ignored its own statistic, and a claim of mine that was wrong
Eleven tools under `tools/port/` and **nothing ran them together**, so each had to
be remembered individually. That is the ninth instance of this port's recurring
shape — correct, documented, unexercised — one level up: the checks were the thing
nobody was running.
`tools/port/check-all` runs the four that assert (`check`, `check-modding`,
`check-capture-controls`, `verify-menu-audio`), prints the oracle table, and
handles `verify-screen` specially. All eleven were exercised first and **none had
rotted**; `which-focus` independently picks NEW_GAME at a **93.8× margin**, which
is a second instrument agreeing with the capture fit's 10×.
Two things it is careful about:
* the six exploratory tools are **not** listed as passes. They produce artifacts
for a person to look at and have no verdict; counting them would invent six.
* `verify-capture` is **reported, not asserted** — it always exits 0. Its header
is right that the numbers are not a target, but *not a target* is not *not a
regression detector*, and nothing would notice `title_plate` moving off 0.00 %.
Named as a gap rather than papered over; a real fix needs stored baselines, and
what a baseline means when the pose is fitted is a decision, not a chore.
* the `verify-screen` allowance **expires on its own condition**. It is allowed to
fail only while `formats-pin-2026-08-29d` is not an ancestor of `origin/main`;
the day it lands, `check-all` fails instead. A suppression with no expiry is
just a hidden failure.
### 🔴 The verdict ignored the statistic added to inform it
`verify-screen` computes `over3` — how many pixels exceed the bar — because *"a
single `max` cannot tell 2 pixels from 25 444"*, its own words. **The verdict was
then decided on `max` alone.** So `main_menu` (max 4, `over3` **0**) read DIFFERS
while `extras` (max 3, `over3` 0) read OK: one unit on one pixel separating two
frames that are equivalent at the bar.
⚠️ Not fixed by raising the bar, which this file rightly forbids. The bar is still
3. A frame with **no** pixel over it now gets its own verdict, `ROUNDING`, instead
of being lumped in with a real disagreement. Tenth instance: the fix was
implemented, documented, and never wired to the thing it was for.
### 🔴 And "six expected DIFFERS" — which I have told the Decoder more than once — was wrong
The true count was **ten**, now **eight** after the rounding fix:
| screens | count | explained |
|---|---|---|
| the forced-backdrop six | 6 | ✅ the pin: two decoder eras |
| `main_menu`, `main_menu_jp` | 2 | ✅ now `ROUNDING`, not a disagreement |
| **`title`, `title_jp`** | **2** | 🔴 **not explained** |
`title` differs on **790** pixels and `title_jp` on **20 498**, and neither is the
forced-backdrop rule — those screens have no forced element. I had a blanket
allowance covering two disagreements I had never accounted for.
**My hypothesis for them is refuted.** `authored/rendering.json` notes that the
consistency harness compares against a renderer that draws no `.rat` leaves, so
the port's `ptloop` sweeps looked like the obvious cause. Emptying `draw_leaf_for`
and `loop_leaf_on_screens` changes the numbers **not at all** — 790 and 20 498
either way. `verify-screen` poses at `rest`, where the leaves evidently do not
draw. Filed as open.
⚠️ `title_jp` is a localisation screen and out of scope (MISSION §7). `title` is on
the boot path and is not.

74
tools/port/check-all Executable file
View File

@@ -0,0 +1,74 @@
#!/usr/bin/env bash
# Run every check this port has, and say which ones assert.
#
# tools/port/check-all
#
# There are eleven tools under `tools/port/` and nothing ran them together, so
# each had to be remembered individually. That is the ninth instance of this
# port's recurring shape -- something correct, documented and unexercised -- one
# level up: the checks themselves were the thing nobody was running.
#
# ⚠️ It runs the tools that ASSERT. The exploratory ones -- `screen-strip`,
# `which-focus`, `strip-padding`, `verify-dwell`, `check-capture`,
# `verify-video-audio` -- produce artifacts for a person to look at and have no
# verdict to collect. Listing them here as passes would be inventing six.
set -euo pipefail
cd "${PROJECT_DIR:-/work}"
export DISPLAY="${DISPLAY:-:97}"
OUT="${OUT:-${TMPDIR:-/tmp}/check-all}"; mkdir -p "$OUT"
BIN="${CARGO_TARGET_DIR:-/sylph-home/port/target-container}/debug/sylpheed-export"
fail=0
step() { # name, expectation, command...
local name="$1" expect="$2"; shift 2
local log="$OUT/${name}.log" rc=0
"$@" >"$log" 2>&1 || rc=$?
case "$expect" in
must-pass)
[ $rc -eq 0 ] && printf ' %-24s ok\n' "$name" \
|| { printf ' %-24s 🔴 FAILED (rc=%d) -- %s\n' "$name" "$rc" "$log"; fail=1; }
;;
report-only)
printf ' %-24s ran (no verdict -- see below)\n' "$name"
;;
esac
}
echo "asserting checks:"
step format-validator must-pass "$BIN" check
step modding-rules must-pass tools/port/check-modding
step capture-controls must-pass tools/port/check-capture-controls
step menu-audio must-pass env OUT="$OUT/audio" tools/port/verify-menu-audio
echo
echo "reported, not asserted:"
step oracle-captures report-only env OUT="$OUT/oracle" tools/port/verify-capture
sed -n '/^screen /,$p' "$OUT/oracle-captures.log" | sed 's/^/ /'
# 🔴 `verify-capture` prints and always exits 0. Its own header is right that the
# numbers are not a target -- the captures carry the game's tone ramp, so RMSE has
# a floor and driving it lower is fitting the ramp. But "not a target" is not the
# same as "not a regression detector", and nothing here would notice `title_plate`
# moving off 0.00 %. Asserting it needs a stored baseline per row, which is a real
# design decision about what a baseline means when the pose is fitted. NAMED, not
# quietly skipped.
echo
echo "consistency (expected to differ, for a stated reason):"
rc=0; env OUT="$OUT/screens" tools/port/verify-screen >"$OUT/verify-screen.log" 2>&1 || rc=$?
differs=$(grep -c DIFFERS "$OUT/verify-screen.log" || true)
# The allowance EXPIRES on its own condition rather than standing forever.
# `verify-screen` builds `sylpheed-cli` from the workspace crate while the
# exporter is pinned to a tag, so the two read different decoder eras and the six
# forced-backdrop screens must disagree. The day the tag is an ancestor of
# `origin/main` that stops being true, and this becomes a real failure again --
# which is what a suppressed check must never be allowed to do silently.
if git merge-base --is-ancestor formats-pin-2026-08-29d origin/main 2>/dev/null; then
[ $rc -eq 0 ] && printf ' %-24s ok (the pin has landed; no allowance left)\n' verify-screen \
|| { printf ' %-24s 🔴 FAILED and the pin HAS landed -- the allowance has expired\n' verify-screen; fail=1; }
else
printf ' %-24s %d DIFFERS, allowed: the pin is not on main, so this compares\n' verify-screen "$differs"
printf ' %-24s two decoder eras. Revert to the path dep when it lands.\n' ""
fi
echo
[ $fail -eq 0 ] && echo "every asserting check passes" || echo "🔴 a check failed"
exit $fail

18
tools/port/verify-screen
View File

@@ -158,7 +158,23 @@ print(json.load(open("export/"+f))["source"]["build"])' "$name")
if awk "BEGIN{exit !($ink <= 0)}"; then
verdict="BLANK -- both renderers drew nothing; this row proves nothing"
else
awk "BEGIN{exit !($max > 3)}" && { verdict=DIFFERS; status=1; }
# 🔴 THE VERDICT USES `over3`, NOT `max` ALONE, AND FOR YEARS IT DID NOT.
#
# This script computed `over3` precisely because "a single `max` cannot tell
# 2 pixels from 25 444" -- its own words, a few lines up -- and then decided
# the verdict on `max` regardless. So `main_menu` (max 4, over3 **0**) read
# DIFFERS while `extras` (max 3, over3 0) read OK: one unit on one pixel,
# separating two frames that are pixel-for-pixel equivalent at the bar.
#
# ⚠️ This is NOT raising the bar, which this file rightly warns against. The
# bar is still 3. What changes is that a frame with NO pixel over it gets a
# verdict of its own instead of being lumped in with a real disagreement --
# the distinction the statistic was added to make and was never given.
if awk "BEGIN{exit !($over > 0)}"; then
verdict=DIFFERS; status=1
elif awk "BEGIN{exit !($max > 3)}"; then
verdict="ROUNDING -- max $max but NO pixel over the bar"
fi
fi
printf '%-17s build %-3s max %-5s mean %-8s over3 %-7s %s\n' \
"$name" "$build" "$max" "${mean:0:6}" "$over" "$verdict"