docs: retract "reference renderer" -- sylpheed-cli is not the oracle

A framing correction from the human, and it runs through everything I have
written, so it is a retraction rather than a silent edit.

Reborn "was/is just a GUI explorer and extraction CLI for verifying the decoding
of the various files. It may very well be wrong." The oracle is the Xenia Canary
capture and the game.

So verify-screen is a CONSISTENCY check between two decoders that share their
assumptions, plus a regression detector -- not a correctness check, and
agreement in it is not evidence of correctness. Its header now says so, it calls
the CLI the COMPARISON renderer, and DIFFERS means "we moved apart, find out
which of us moved".

The uncomfortable part, recorded because it is the actual failure mode: this
file already contained the sentence "two renderers reading one field through one
decoder agreeing is not evidence that the field is right", written after the
ptframe1 case -- and I then went on quoting 3/255 against sylpheed-cli as though
it meant the port was right. Having the principle written down did not stop me
leaning on the agreement.

Three times both renderers agreed and both were wrong, each caught only by a
capture: pteff05 (menu screens had no background), scale-0 (drawn full size
instead of collapsed), rest() (the menu bracket missing).

Correctness moves to the captures -- nine of them, indexed at
docs/re/captures/ORACLE-CAPTURES.md, covering all five screens in scope. Three
cautions travel with them: not gamma-neutral (there is a floor, don't chase it),
geometry IS sound (a positional disagreement is real), and each is one moment of
a still-animating screen.

verify-screen keeps running over all 16 screens every iteration. It is still
worth having -- total, cheap, and it catches a divergence introduced on the RE
side. It is just not a grade.
This commit is contained in:
Sylpheed port agent
2026-08-29 08:45:48 +00:00
parent 579c8096c1
commit 2fbcd59920
3 changed files with 94 additions and 10 deletions

View File

@@ -47,18 +47,25 @@ assuming a value is on the disc.
## Verifying
`sylpheed-cli screen render` -- built from the same `sylpheed-formats` revision
the exporter is pinned to -- is the reference renderer. `tools/verify-screen`
draws every exported screen both ways and reports the largest per-channel
difference in the frame:
**The oracle is the Xenia Canary capture and the game**, not either renderer.
`sylpheed-cli screen render` is an explorer and extraction CLI for verifying
decodes, and it can be wrong -- three times both it and the port agreed and both
were wrong, each caught only by a capture.
So `tools/verify-screen` is a **consistency check and a regression detector**,
not a grade. It draws every exported screen both ways -- built from the same
`sylpheed-formats` revision the exporter is pinned to -- and reports the largest
per-channel difference in the frame:
```
tools/verify-screen # every screen in the manifest
tools/verify-screen main_menu # one of them
```
Where the two disagree, one of them is wrong; `docs/DECISIONS.md` says which and
why, rather than tuning the port until the number goes down.
A difference means the two moved apart; `docs/DECISIONS.md` says which one moved
and why, rather than tuning the port until the number goes down. Correctness is
checked against the captures indexed at `docs/re/captures/ORACLE-CAPTURES.md` --
mind that they are not gamma-neutral, so RMSE against them has a floor.
## Status

View File

@@ -750,3 +750,62 @@ transcode, is 2-channel, and decodes. Whether Godot emits it audibly is
unconfirmed and is stated as unconfirmed rather than assumed from the stream's
presence. It is a cheap check for anyone with a sound device and an impossible
one here.
---
## RETRACTION — `sylpheed-cli` is not the oracle, 2026-08-29
**This corrects a framing that runs through everything above, so it is a
retraction rather than an edit.** Every place this file called
`sylpheed-cli screen render` *"the reference renderer"* — and it does so
repeatedly, starting at P1 — overstated what it is.
The correction comes from the human, via the RE agent, in their words: Reborn
"was/is just a GUI explorer and extraction CLI for verifying the decoding of the
various files. It may very well be wrong." **The oracle is the Xenia Canary
capture and the game.**
So `tools/verify-screen` is a **consistency check between two decoders that
share their assumptions**, and a regression detector. It is not a correctness
check, and agreement in it is not evidence of correctness.
### The embarrassing part is that this file already knew
After the `ptframe1` case, P2's write-up says: *"Two renderers reading one field
through one decoder agreeing is not evidence that the field is right."* Then P1's
numbers kept being quoted as though 3/255 against `sylpheed-cli` meant the port
was right. Having the principle written down did not stop me leaning on the
agreement — which is worth recording, because that is the failure mode, not
ignorance of the principle.
**Three times** both renderers agreed and both were wrong, all three caught by a
capture and catchable by nothing else:
| | what both got wrong | how it surfaced |
|---|---|---|
| `pteff05` | the menu screens had **no background** | the RE agent decoded the RATC child name |
| scale 0 | drawn at full size instead of collapsed | RE agent's control run |
| `rest()` | `ptframe1`/`ptframe2` invisible; the menu bracket missing | `main-menu-oracle.png` |
### What changes
* `tools/verify-screen` says all of this in its own header, calls the CLI the
**comparison** renderer, and a `DIFFERS` row now means "we moved apart, find
out which of us moved" rather than "the port is wrong".
* The correctness question moves to the captures. The RE agent has committed
nine of them with an index at `docs/re/captures/ORACLE-CAPTURES.md`, covering
all five screens in scope — including a **main menu with `OPTIONS` focused**,
whose difference from the unfocused menu isolates exactly what focus changes.
* Three cautions travel with any capture comparison, and they are the RE agent's:
the captures are **not gamma-neutral** (γ ≈ 1.49 menu, 1.34 title — there is a
floor, do not chase it); **geometry is sound** (best alignment 0,0 at corr
0.9466, so a positional disagreement is real); and each is **one moment of a
still-animating screen**, so compare settled poses or regions known to be at
rest.
### What does not change
The port keeps running `verify-screen` over all 16 screens every iteration. A
consistency check is still worth having — it is total, it is cheap, and it is
what catches a divergence the RE agent introduces on their side. It is simply
not a grade, and this file will stop quoting it as one.

View File

@@ -1,6 +1,23 @@
#!/usr/bin/env bash
# Diff Godot's drawing of an exported screen against `sylpheed-cli screen
# render` of the same build -- the P1 gate.
# render` of the same build.
#
# WHAT THIS IS, AND WHAT IT IS NOT.
#
# It is a CONSISTENCY check between two decoders that share their assumptions,
# and a REGRESSION detector: "did anything move since last commit". It is NOT a
# correctness check and agreement here is NOT evidence of correctness.
#
# `sylpheed-cli` is not the oracle. The oracle is the Xenia Canary capture and
# the game. Reborn is an explorer and extraction CLI for verifying decodes, and
# it can be wrong -- this corpus has been bitten three times by both renderers
# agreeing and both being wrong: pteff05 (the menu background, missing from
# both), scale-0, and rest(). Each time the capture caught it and neither
# renderer could have.
#
# So: a DIFFERS row means "we moved apart, go find out which of us moved". It
# does not mean the port is wrong. Where a capture and this tool disagree, the
# capture wins. Use `tools/verify-capture` for the correctness question.
#
# tools/verify-screen # every screen in the manifest
# tools/verify-screen main_menu title # named screens
@@ -11,7 +28,7 @@
#
# The two renderers are held to the same inputs on purpose:
#
# * the REFERENCE CLI is the one built by `build-reference-cli`, from the same
# * the COMPARISON CLI is the one built by `build-reference-cli`, from the same
# `sylpheed-formats` revision the exporter is pinned to. /reborn's own
# target/ is a live mount of the other agent's checkout and moves mid-run; a
# pixel disagreement against a moving decoder proves nothing.
@@ -30,8 +47,9 @@
# PORT against the reference. It is not the test of whether `rest` is right;
# that one is the oracle capture, and the port already departs from it.
#
# A difference here is not automatically the port's fault. Say which renderer is
# wrong and why -- do not tune until they match.
# A difference here is not automatically the port's fault, and it is not
# automatically a fault at all. Say which renderer moved and why -- do not tune
# until they match.
set -euo pipefail
cd "${PROJECT_DIR:-/work}"