diff --git a/README.md b/README.md index 0e27e5e4..9f847044 100644 --- a/README.md +++ b/README.md @@ -47,18 +47,25 @@ assuming a value is on the disc. ## Verifying -`sylpheed-cli screen render` -- built from the same `sylpheed-formats` revision -the exporter is pinned to -- is the reference renderer. `tools/verify-screen` -draws every exported screen both ways and reports the largest per-channel -difference in the frame: +**The oracle is the Xenia Canary capture and the game**, not either renderer. +`sylpheed-cli screen render` is an explorer and extraction CLI for verifying +decodes, and it can be wrong -- three times both it and the port agreed and both +were wrong, each caught only by a capture. + +So `tools/verify-screen` is a **consistency check and a regression detector**, +not a grade. It draws every exported screen both ways -- built from the same +`sylpheed-formats` revision the exporter is pinned to -- and reports the largest +per-channel difference in the frame: ``` tools/verify-screen # every screen in the manifest tools/verify-screen main_menu # one of them ``` -Where the two disagree, one of them is wrong; `docs/DECISIONS.md` says which and -why, rather than tuning the port until the number goes down. +A difference means the two moved apart; `docs/DECISIONS.md` says which one moved +and why, rather than tuning the port until the number goes down. Correctness is +checked against the captures indexed at `docs/re/captures/ORACLE-CAPTURES.md` -- +mind that they are not gamma-neutral, so RMSE against them has a floor. ## Status diff --git a/docs/DECISIONS.md b/docs/DECISIONS.md index 720114c9..60f0a13c 100644 --- a/docs/DECISIONS.md +++ b/docs/DECISIONS.md @@ -750,3 +750,62 @@ transcode, is 2-channel, and decodes. Whether Godot emits it audibly is unconfirmed and is stated as unconfirmed rather than assumed from the stream's presence. It is a cheap check for anyone with a sound device and an impossible one here. + +--- + +## RETRACTION — `sylpheed-cli` is not the oracle, 2026-08-29 + +**This corrects a framing that runs through everything above, so it is a +retraction rather than an edit.** Every place this file called +`sylpheed-cli screen render` *"the reference renderer"* — and it does so +repeatedly, starting at P1 — overstated what it is. + +The correction comes from the human, via the RE agent, in their words: Reborn +"was/is just a GUI explorer and extraction CLI for verifying the decoding of the +various files. It may very well be wrong." **The oracle is the Xenia Canary +capture and the game.** + +So `tools/verify-screen` is a **consistency check between two decoders that +share their assumptions**, and a regression detector. It is not a correctness +check, and agreement in it is not evidence of correctness. + +### The embarrassing part is that this file already knew + +After the `ptframe1` case, P2's write-up says: *"Two renderers reading one field +through one decoder agreeing is not evidence that the field is right."* Then P1's +numbers kept being quoted as though 3/255 against `sylpheed-cli` meant the port +was right. Having the principle written down did not stop me leaning on the +agreement — which is worth recording, because that is the failure mode, not +ignorance of the principle. + +**Three times** both renderers agreed and both were wrong, all three caught by a +capture and catchable by nothing else: + +| | what both got wrong | how it surfaced | +|---|---|---| +| `pteff05` | the menu screens had **no background** | the RE agent decoded the RATC child name | +| scale 0 | drawn at full size instead of collapsed | RE agent's control run | +| `rest()` | `ptframe1`/`ptframe2` invisible; the menu bracket missing | `main-menu-oracle.png` | + +### What changes + +* `tools/verify-screen` says all of this in its own header, calls the CLI the + **comparison** renderer, and a `DIFFERS` row now means "we moved apart, find + out which of us moved" rather than "the port is wrong". +* The correctness question moves to the captures. The RE agent has committed + nine of them with an index at `docs/re/captures/ORACLE-CAPTURES.md`, covering + all five screens in scope — including a **main menu with `OPTIONS` focused**, + whose difference from the unfocused menu isolates exactly what focus changes. +* Three cautions travel with any capture comparison, and they are the RE agent's: + the captures are **not gamma-neutral** (γ ≈ 1.49 menu, 1.34 title — there is a + floor, do not chase it); **geometry is sound** (best alignment 0,0 at corr + 0.9466, so a positional disagreement is real); and each is **one moment of a + still-animating screen**, so compare settled poses or regions known to be at + rest. + +### What does not change + +The port keeps running `verify-screen` over all 16 screens every iteration. A +consistency check is still worth having — it is total, it is cheap, and it is +what catches a divergence the RE agent introduces on their side. It is simply +not a grade, and this file will stop quoting it as one. diff --git a/tools/verify-screen b/tools/verify-screen index fc98aa6a..f1ef2ada 100755 --- a/tools/verify-screen +++ b/tools/verify-screen @@ -1,6 +1,23 @@ #!/usr/bin/env bash # Diff Godot's drawing of an exported screen against `sylpheed-cli screen -# render` of the same build -- the P1 gate. +# render` of the same build. +# +# WHAT THIS IS, AND WHAT IT IS NOT. +# +# It is a CONSISTENCY check between two decoders that share their assumptions, +# and a REGRESSION detector: "did anything move since last commit". It is NOT a +# correctness check and agreement here is NOT evidence of correctness. +# +# `sylpheed-cli` is not the oracle. The oracle is the Xenia Canary capture and +# the game. Reborn is an explorer and extraction CLI for verifying decodes, and +# it can be wrong -- this corpus has been bitten three times by both renderers +# agreeing and both being wrong: pteff05 (the menu background, missing from +# both), scale-0, and rest(). Each time the capture caught it and neither +# renderer could have. +# +# So: a DIFFERS row means "we moved apart, go find out which of us moved". It +# does not mean the port is wrong. Where a capture and this tool disagree, the +# capture wins. Use `tools/verify-capture` for the correctness question. # # tools/verify-screen # every screen in the manifest # tools/verify-screen main_menu title # named screens @@ -11,7 +28,7 @@ # # The two renderers are held to the same inputs on purpose: # -# * the REFERENCE CLI is the one built by `build-reference-cli`, from the same +# * the COMPARISON CLI is the one built by `build-reference-cli`, from the same # `sylpheed-formats` revision the exporter is pinned to. /reborn's own # target/ is a live mount of the other agent's checkout and moves mid-run; a # pixel disagreement against a moving decoder proves nothing. @@ -30,8 +47,9 @@ # PORT against the reference. It is not the test of whether `rest` is right; # that one is the oracle capture, and the port already departs from it. # -# A difference here is not automatically the port's fault. Say which renderer is -# wrong and why -- do not tune until they match. +# A difference here is not automatically the port's fault, and it is not +# automatically a fault at all. Say which renderer moved and why -- do not tune +# until they match. set -euo pipefail cd "${PROJECT_DIR:-/work}"