ed14722996
The suite reported '2 DIFFERS, allowed: the pin is not on main, so this compares two decoder eras', and I had quoted that for several iterations without testing it. Built sylpheed-cli at formats-pin-2026-08-30 and at workspace HEAD and rendered through both: title, title_jp and main_menu come out 0 pixels different, despite 508 lines of difference in ui_layout.rs. The eras are not the cause, and the allowance was excusing a real signal with a wrong explanation. A second defect in the same eight lines: the expiry tested formats-pin-2026-08-29d while Cargo.toml pins formats-pin-2026-08-30, so it would have expired on a tag this tree does not use. The real reasons are per-screen and already documented: title is the ptloop sweep phase residual, title_jp is the --pose=rest sparkle handling -- where the port's shipped pose scores +0.9994 against the game to the reference's +0.8727, so the port is closer to the game on the row the script calls a disagreement. Replaced with a named set: title and title_jp by name, any other DIFFERS fails. A count cannot notice a different screen drifting while the total stays at two. Controlled both directions -- passes on the known pair, fails on main_menu or extras. The pin reminder now reads the tag out of Cargo.toml so it cannot drift. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N7FiFFFwbvG2uxdcEh8HyF
110 lines
5.4 KiB
Bash
Executable File
110 lines
5.4 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Run every check this port has, and say which ones assert.
|
|
#
|
|
# tools/port/check-all
|
|
#
|
|
# There are eleven tools under `tools/port/` and nothing ran them together, so
|
|
# each had to be remembered individually. That is the ninth instance of this
|
|
# port's recurring shape -- something correct, documented and unexercised -- one
|
|
# level up: the checks themselves were the thing nobody was running.
|
|
#
|
|
# ⚠️ It runs the tools that ASSERT. The exploratory ones -- `screen-strip`,
|
|
# `which-focus`, `strip-padding`, `verify-dwell`, `check-capture`,
|
|
# `verify-video-audio` -- produce artifacts for a person to look at and have no
|
|
# verdict to collect. Listing them here as passes would be inventing six.
|
|
set -euo pipefail
|
|
cd "${PROJECT_DIR:-/work}"
|
|
export DISPLAY="${DISPLAY:-:97}"
|
|
OUT="${OUT:-${TMPDIR:-/tmp}/check-all}"; mkdir -p "$OUT"
|
|
BIN="${CARGO_TARGET_DIR:-/sylph-home/port/target-container}/debug/sylpheed-export"
|
|
fail=0
|
|
|
|
step() { # name, expectation, command...
|
|
local name="$1" expect="$2"; shift 2
|
|
local log="$OUT/${name}.log" rc=0
|
|
"$@" >"$log" 2>&1 || rc=$?
|
|
case "$expect" in
|
|
must-pass)
|
|
[ $rc -eq 0 ] && printf ' %-24s ok\n' "$name" \
|
|
|| { printf ' %-24s 🔴 FAILED (rc=%d) -- %s\n' "$name" "$rc" "$log"; fail=1; }
|
|
;;
|
|
report-only)
|
|
printf ' %-24s ran (no verdict -- see below)\n' "$name"
|
|
;;
|
|
esac
|
|
}
|
|
|
|
echo "asserting checks:"
|
|
step format-validator must-pass "$BIN" check
|
|
step modding-rules must-pass tools/port/check-modding
|
|
step capture-controls must-pass tools/port/check-capture-controls
|
|
step menu-audio must-pass env OUT="$OUT/audio" tools/port/verify-menu-audio
|
|
# A stale index is worse than none: it answers "is this already decided?" with a
|
|
# confident no. That is not hypothetical -- see the entry it was built after.
|
|
step decisions-index must-pass tools/port/index-decisions --check
|
|
# A refuted claim asserted outside its correction is a lie the corpus tells a
|
|
# reader who greps for it. Registered claims must carry an explicit `[refuted]`.
|
|
step refuted-claims must-pass tools/port/check-claims
|
|
echo
|
|
echo "reported, not asserted:"
|
|
step oracle-captures report-only env OUT="$OUT/oracle" tools/port/verify-capture
|
|
sed -n '/^screen /,$p' "$OUT/oracle-captures.log" | sed 's/^/ /'
|
|
# 🔴 `verify-capture` prints and always exits 0. Its own header is right that the
|
|
# numbers are not a target -- the captures carry the game's tone ramp, so RMSE has
|
|
# a floor and driving it lower is fitting the ramp. But "not a target" is not the
|
|
# same as "not a regression detector", and nothing here would notice `title_plate`
|
|
# moving off 0.00 %. Asserting it needs a stored baseline per row, which is a real
|
|
# design decision about what a baseline means when the pose is fitted. NAMED, not
|
|
# quietly skipped.
|
|
|
|
echo
|
|
echo "consistency (expected to differ, for a stated reason):"
|
|
rc=0; env OUT="$OUT/screens" tools/port/verify-screen >"$OUT/verify-screen.log" 2>&1 || rc=$?
|
|
differs=$(grep -c DIFFERS "$OUT/verify-screen.log" || true)
|
|
unexpected=$(grep DIFFERS "$OUT/verify-screen.log" | awk '{print $1}' \
|
|
| grep -vx -e title -e title_jp || true)
|
|
|
|
# 🔴 THE OLD ALLOWANCE WAS FALSE, AND IT EXCUSED TWO ROWS WITH A REASON THAT IS
|
|
# NOT THE REASON.
|
|
#
|
|
# It said: "the pin is not on main, so this compares two decoder eras". Measured
|
|
# 2026-08-30 by building `sylpheed-cli` at `formats-pin-2026-08-30` and at the
|
|
# workspace HEAD and rendering both: `title`, `title_jp` and `main_menu` come out
|
|
# **0 pixels different**, despite 508 lines of difference in `ui_layout.rs`. The
|
|
# eras are not the cause of anything here.
|
|
#
|
|
# 🔴 And its expiry tested `formats-pin-2026-08-29d` while `Cargo.toml` pins
|
|
# `formats-pin-2026-08-30` -- so the allowance would have expired on a tag this
|
|
# tree does not use, in either direction, silently.
|
|
#
|
|
# The real reasons are per-screen and already documented:
|
|
# title -- the ptloop SWEEP PHASE residual, max 6 / over3 790, unchanged
|
|
# across every renderer change since P1 (DECISIONS.md).
|
|
# title_jp -- the `--pose=rest` sparkle handling. Adjudicated against the
|
|
# oracle: the port's SHIPPED pose scores r +0.9994 against the
|
|
# game where the reference scores +0.8727, and `--pose=rest` is
|
|
# what this script compares. The port is closer to the game on the
|
|
# row this script calls a disagreement.
|
|
#
|
|
# So the allowance is now a NAMED SET, not a count with an excuse. A DIFFERS on
|
|
# any other screen fails the run, which a count never could.
|
|
if [ -n "$unexpected" ]; then
|
|
printf ' %-24s 🔴 DIFFERS on %s -- not in the allowed set\n' verify-screen "$(echo $unexpected | tr '\n' ' ')"
|
|
fail=1
|
|
else
|
|
printf ' %-24s %d DIFFERS, both named and explained per screen:\n' verify-screen "$differs"
|
|
printf ' %-24s title = sweep phase; title_jp = rest-pose sparkles (the port is\n' ""
|
|
printf ' %-24s closer to the GAME there than the reference is).\n' ""
|
|
fi
|
|
|
|
# Separately, and unrelated to the rows above: revert to the path dependency when
|
|
# the pin lands. Read from Cargo.toml so it cannot drift out of step again.
|
|
pin=$(sed -n 's/.*tag = "\([^"]*\)".*/\1/p' crates/sylpheed-export/Cargo.toml | head -1)
|
|
if [ -n "$pin" ] && git merge-base --is-ancestor "$pin" origin/main 2>/dev/null; then
|
|
printf ' %-24s ⚠️ %s has landed on main -- revert Cargo.toml to the path dep\n' pin "$pin"
|
|
fi
|
|
|
|
echo
|
|
[ $fail -eq 0 ] && echo "every asserting check passes" || echo "🔴 a check failed"
|
|
exit $fail
|