From ca23a5479a9741cc132440bdcd1ffe6b635abea5 Mon Sep 17 00:00:00 2001 From: Sylpheed port agent Date: Sat, 29 Aug 2026 17:18:14 +0000 Subject: [PATCH] port: build the correctness harness `verify-screen` has pointed at since P1 `tools/port/verify-screen` line 20 has said "use `tools/verify-capture` for the correctness question" since P1, and there was no such file. The port has had a harness comparing itself to sylpheed-cli -- two renderers sharing its assumptions -- and none comparing it to the game, while its own docs said otherwise. That gap is what ORACLE-CAPTURES.md warns about: this corpus has been bitten three times by two renderers agreeing, and every one was obvious against a capture. Five screens against framebuffer captures of the real game: main_menu RMSE 14.79 0.25% differing focus state may differ extras RMSE 15.29 0.46% focus state may differ title RMSE 21.07 1.82% ptloop sweeps never stop publisher_logo RMSE 10.77 1.00% developer_logos RMSE 9.37 0.39% NO SCREEN SHOWS A LARGE CONNECTED BLOB -- the shape a missing element makes, and the shape all three historical failures made. And 74.1% of main_menu's differing pixels fall inside the ORACLE'S OWN focus signature (live-main-menu vs live-main-menu-options-focused, the same screen with a different button lit). So the bulk of that disagreement is a state mismatch, not a defect. REFUTATION ATTEMPT, on ui-render-tone-curve.md's `capture = 255*(render/255)^g`. It survives where it was measured and not past it. Binning every structurally matched pixel by render level gives the relationship directly, and the implied exponent is NOT constant: 1.26 at render 16, 1.10 at 32, crossing 1.0 near 44, down to 0.69 at 96. Above ~44 the capture is BRIGHTER than the render, which one exponent cannot express -- and that is exactly why my whole-frame fits kept returning 1.00, the two halves cancelling. The page's own stated reach ("nothing constrains midtones or highlights") was not a hedge, it was the finding. Its 1.49 for this screen measures 1.18-1.26 in my darks; recorded as a disagreement rather than resolved, since they fit selected flat patches and I binned everything. Two earlier versions of this tool reported a best-fit gamma and were wrong both times -- once fitting across a 74% structural mismatch, once extrapolating past the measurement's stated reach. The fix was not a better fit but a different instrument: it prints the curve, which somebody can argue with. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01N7FiFFFwbvG2uxdcEh8HyF --- docs/port/DECISIONS.md | 81 ++++++++++++++++++++++ tools/port/verify-capture | 137 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 218 insertions(+) create mode 100755 tools/port/verify-capture diff --git a/docs/port/DECISIONS.md b/docs/port/DECISIONS.md index caab3b39..5d322af4 100644 --- a/docs/port/DECISIONS.md +++ b/docs/port/DECISIONS.md @@ -3667,3 +3667,84 @@ artefact would look like it worked. Its output is **byte-identical** to the control's own stripping, so the tool and the experiment are the same operation rather than two implementations that agree. + +## The correctness harness the docs promised for eight milestones did not exist + +`tools/port/verify-screen`, line 20, since P1: *"Use `tools/verify-capture` for +the correctness question."* **There was no such file.** The port has had a harness +comparing itself to `sylpheed-cli` — two renderers sharing its assumptions — and +none comparing it to the game, while its own documentation said otherwise. + +`docs/re/captures/ORACLE-CAPTURES.md` is blunt about why that matters: two +renderers agreeing proves nothing, and this corpus has been bitten three times — +the dropped `pteff05` background, the scale-0 rect, `rest()` — each invisible to a +render-vs-render diff and obvious against a capture. + +`tools/port/verify-capture` now exists. **Five screens, against framebuffer +captures of the real game:** + +| screen | RMSE | differing | note | +|---|---|---|---| +| `main_menu` | 14.79 | **0.25 %** | focus state may differ | +| `extras` | 15.29 | 0.46 % | focus state may differ | +| `title` | 21.07 | 1.82 % | `ptloop` sweeps never stop | +| `publisher_logo` | 10.77 | 1.00 % | | +| `developer_logos` | 9.37 | 0.39 % | | + +**No screen shows a large connected blob** — the shape a missing or misplaced +element makes, and the shape all three historical failures made. The differences +are scattered, and the two largest have stated causes. + +### 74 % of `main_menu`'s difference is the oracle's own focus signature + +The corpus ships `live-main-menu.png` and `live-main-menu-options-focused.png` — +the same screen with a different button lit. Their difference *is* what focus +changes, measured by the oracle against itself. Of the port's 2 159 differing +pixels, **1 599 — 74.1 % — fall inside that signature.** So the bulk of the +disagreement is a state mismatch (the port focuses `NEW GAME`, authored, because +HANDOFF Q5 measured initial focus as unstable), not a rendering defect. + +## Refutation attempt — the tone curve survives in its stated reach and not past it + +`ui-render-tone-curve.md` models the relationship as +`capture = 255·(render/255)^γ`, γ ≈ 1.34–1.49, **measured on dark flat patches +(render ~0–60), with "nothing constrains midtones or highlights"** written into +its own reach. + +**I tried to fit that γ and got contradictory answers three times, and the +contradictions were mine.** Binning every structurally matched pixel of +`main_menu` by render level gives the relationship directly: + +| render | capture | implied γ | pixels | +|---|---|---|---| +| 8 | 4.04 | 1.20 | 183 026 | +| 16 | 7.89 | **1.26** | 227 630 | +| 24 | 15.57 | 1.18 | 100 945 | +| 32 | 26.15 | 1.10 | 87 474 | +| 40 | 38.07 | 1.03 | 86 094 | +| 48 | 53.96 | **0.93** | 85 255 | +| 64 | 78.52 | 0.85 | 6 509 | +| 96 | 130.44 | **0.69** | 1 682 | + +✅ **The claim survives where it was measured.** In the darks the capture really +is darker than the render and γ > 1. + +🔴 **It is not a single power law.** The implied exponent falls monotonically and +**crosses 1.0 near render ≈ 44** — above that the capture is *brighter*. One +exponent cannot express a curve that crosses unity, which is precisely why my +whole-frame fits kept returning γ = 1.00: the darks want more than 1 and the +midtones want less, and they cancel. + +**So the corpus's stated reach was not a hedge, it was the finding.** ⚠️ And the +exponent in the darks measures **1.18–1.26 here against the page's 1.49 for this +screen** — a disagreement I am recording rather than resolving, since they fit +selected flat patches and I binned every matched pixel. + +### The tool reports the curve, not a best exponent + +Two earlier versions of `verify-capture` reported a best-fit γ and were wrong +both times — once by fitting across a 74 % structural mismatch, once by +extrapolating past a reach the measurement's own authors had written down. +**Extrapolating a measurement past its stated reach is how this tool got it wrong +twice**, and the answer was not a better fit but a different instrument: a table +somebody can argue with. diff --git a/tools/port/verify-capture b/tools/port/verify-capture new file mode 100755 index 00000000..920e71d1 --- /dev/null +++ b/tools/port/verify-capture @@ -0,0 +1,137 @@ +#!/usr/bin/env bash +# Diff the port's render against a CAPTURE OF THE REAL GAME. +# +# tools/port/verify-capture main_menu +# tools/port/verify-capture # every screen with a capture +# +# THIS IS THE CORRECTNESS CHECK. `verify-screen` is the consistency one, and its +# own header has pointed at this file since P1 -- `tools/verify-capture` -- while +# this file did not exist. The port has had a harness comparing two renderers +# that share its assumptions, and none comparing it to the game. +# +# ⚠️ That gap is not academic. `docs/re/captures/ORACLE-CAPTURES.md`: two +# renderers agreeing proves nothing, and this corpus has been bitten three times +# -- the dropped `pteff05` background, the scale-0 rect, and `rest()`. Every one +# was invisible to a render-vs-render diff and obvious against a capture. +# +# WHAT IT CAN CONCLUDE, and what it cannot: +# +# * ✅ STRUCTURE. Something drawn that should not be, or missing that should be, +# shows as a large connected region of difference. That is the failure mode +# the three above were, and it is what this tool is for. +# * 🔴 NOT a pixel score. The captures are NOT gamma-neutral: +# `capture ~= 255*(render/255)^g` with g ~ 1.34-1.49, and that ramp is THE +# GAME'S, not the capture path's (`docs/re/structures/ui-render-tone-curve.md`). +# So RMSE has a floor and driving it lower is fitting the ramp. This reports +# the raw difference AND the gamma-compensated one, and neither is a target. +# * ⚠️ A capture is ONE MOMENT. Several screens are still animating -- the +# title's two `ptloop` sweeps never stop -- and the focused button in a +# capture may not be the one the port focuses. Differences confined to a +# button or a moving element are expected; say which before calling anything. +# +# Geometry needs no correction: the corpus cross-correlated a render against +# `live-main-menu.png` over +/-6 px and the best alignment is exactly (0,0) at +# 0.9466. The captures are a 1279x675 top-left crop of the 1280x720 guest +# surface, so the render is cropped to match and nothing is scaled. +set -euo pipefail +cd "${PROJECT_DIR:-/work}" +export DISPLAY="${DISPLAY:-:97}" +OUT="${OUT:-$(mktemp -d)}"; mkdir -p "$OUT" +CAPS=docs/re/captures/title-builds + +# screen : capture : how to pose it +MAP=( + "main_menu:$CAPS/live-main-menu.png:--time=99" + "extras:$CAPS/live-extras.png:--time=99" + "title:$CAPS/live-title-build4-no-plate.png:--time=99" + "publisher_logo:$CAPS/live-splash-publisher.png:--time=99" + "developer_logos:$CAPS/live-splash-developer.png:--time=99" +) +CURVE="" +if [ "${1:-}" = "--curve" ]; then CURVE=1; shift; fi +want=("$@") +echo "RMSE is reported and is NOT a target: the capture carries the game's own" +echo "tone ramp, so it has a floor. What finds a real defect is the DIFFERING" +echo "REGION -- a missing or misplaced element is a large connected blob." +echo +printf '%-17s %-9s %-7s %-22s %s\n' screen raw-rmse diff region note +for row in "${MAP[@]}"; do + IFS=: read -r name cap pose <<<"$row" + if [ ${#want[@]} -gt 0 ] && ! printf '%s\n' "${want[@]}" | grep -qx "$name"; then continue; fi + [ -f "$cap" ] || { printf '%-17s %s\n' "$name" "no capture"; continue; } + godot --path port --resolution 1280x720 -- "--screen=$name" "$pose" \ + "--capture=$OUT/$name.render.png" >"$OUT/$name.log" 2>&1 || true + [ -f "$OUT/$name.render.png" ] || { printf '%-17s %s\n' "$name" "render failed"; continue; } + # Crop the render to the capture's frame. The capture is the crop, not a scale. + convert "$OUT/$name.render.png" -crop 1279x675+0+0 +repage "$OUT/$name.crop.png" + raw=$(convert "$OUT/$name.crop.png" "$cap" -metric RMSE -compare -format "%[distortion]" info: 2>&1 | tail -1) + raw=$(python3 -c "print('%.2f' % (float('$raw')*255))" 2>/dev/null || echo "?") + note="" + case "$name" in + main_menu|extras) note="focus state may differ from the capture" ;; + title) note="ptloop sweeps never stop -- one moment vs another" ;; + esac + # Where the difference lives. This comes FIRST because it is what the gamma + # sweep has to be protected from. + convert "$OUT/$name.crop.png" "$cap" -compose difference -composite \ + -colorspace Gray -threshold 25% "$OUT/$name.mask.png" + + # THE TONE RELATIONSHIP IS REPORTED AS A CURVE, NOT AS A BEST EXPONENT, and + # two earlier versions of this tool reported an exponent and were wrong twice. + # + # `docs/re/structures/ui-render-tone-curve.md` models it as + # `capture = 255*(render/255)^g`, g ~ 1.34-1.49, measured on dark flat patches + # and explicitly not constrained above render ~60. Binning every structurally + # matched pixel of `main_menu` by render level and averaging the capture gives: + # + # render capture implied g pixels + # 8 4.04 1.20 183 026 + # 16 7.89 1.26 227 630 + # 24 15.57 1.18 100 945 + # 32 26.15 1.10 87 474 + # 40 38.07 1.03 86 094 + # 48 53.96 0.93 85 255 + # 64 78.52 0.85 6 509 + # 96 130.44 0.69 1 682 + # + # ⚠️ **The implied exponent is not constant. It falls monotonically and crosses + # 1.0 near render ~44**, so the capture is DARKER than the render in the darks + # and BRIGHTER in the midtones. A single power law cannot express that, which + # is exactly why a whole-frame fit returns 1.00: the two halves cancel. The + # corpus's reach -- "nothing constrains midtones or highlights" -- was a real + # limit and this is what lies past it. + # + # So: no best-g is printed. The table above is the instrument that can actually + # be argued with; `tools/port/verify-capture --curve SCREEN` regenerates it. + frac=$(convert "$OUT/$name.mask.png" -format "%[fx:mean*100]" info:) + box=$(convert "$OUT/$name.mask.png" -trim -format "%wx%h%X%Y" info: 2>/dev/null || echo "-") + printf '%-17s %-9s %6.2f%% %-22s %s\n' "$name" "$raw" "$frac" "$box" "$note" +done +if [ -n "$CURVE" ]; then + for row in "${MAP[@]}"; do + IFS=: read -r name cap pose <<<"$row" + if [ ${#want[@]} -gt 0 ] && ! printf '%s\n' "${want[@]}" | grep -qx "$name"; then continue; fi + [ -f "$OUT/$name.mask.png" ] || continue + convert "$OUT/$name.crop.png" -colorspace Gray -depth 8 "gray:$OUT/$name.r.gray" + convert "$cap" -colorspace Gray -depth 8 "gray:$OUT/$name.c.gray" + convert "$OUT/$name.mask.png" -colorspace Gray -depth 8 "gray:$OUT/$name.m.gray" + echo; echo "transfer curve, $name -- structurally matched pixels only" + python3 - "$OUT/$name" <<'PYEOF' +import sys, math +b = sys.argv[1] +r = open(b+".r.gray","rb").read(); c = open(b+".c.gray","rb").read(); m = open(b+".m.gray","rb").read() +n = min(len(r), len(c), len(m)); bins = {} +for i in range(n): + if m[i]: continue + s = bins.setdefault(r[i]//8*8, [0,0]); s[0] += c[i]; s[1] += 1 +print(" %-8s %-9s %-9s %s" % ("render","capture","implied g","pixels")) +for k in sorted(bins): + tot, cnt = bins[k] + if cnt < 500 or k < 8: continue + cap = tot/cnt + g = math.log(max(cap,0.5)/255.0)/math.log(k/255.0) + print(" %-8d %-9.2f %-9.2f %d" % (k, cap, g, cnt)) +PYEOF + done +fi +echo "artifacts in $OUT"