#!/usr/bin/env bash # Diff Godot's drawing of an exported screen against `sylpheed-cli screen # render` of the same build. # # WHAT THIS IS, AND WHAT IT IS NOT. # # It is a CONSISTENCY check between two decoders that share their assumptions, # and a REGRESSION detector: "did anything move since last commit". It is NOT a # correctness check and agreement here is NOT evidence of correctness. # # `sylpheed-cli` is not the oracle. The oracle is the Xenia Canary capture and # the game. Reborn is an explorer and extraction CLI for verifying decodes, and # it can be wrong -- this corpus has been bitten three times by both renderers # agreeing and both being wrong: pteff05 (the menu background, missing from # both), scale-0, and rest(). Each time the capture caught it and neither # renderer could have. # # 🔴 AND ITS FRAMES MUST NEVER BE SCORED AGAINST A CAPTURE. This script poses # `--pose=rest`, deliberately -- both renderers read `rest` through the same # decoder, which is what makes it a test of the PORT against the REFERENCE. It # is NOT the pose the port ships, and on some screens the two are very far # apart: `rest` for each `ptlogo_back2eff*` sparkle is the peak of its own # 4-unit flash, so `--pose=rest` lights all of them at once, a frame the game # never shows. # # I scored this script's `title_jp` frame against the oracle capture and # concluded the port had drifted away from the game -- r +0.7462 against the # reference's +0.8727. Posed as it SHIPS, the same block scores **+0.9994**. # The conclusion was an artefact of the pose, and it was written up as a finding. # Correctness questions go to `tools/port/verify-capture`, which poses as shipped. # # So: a DIFFERS row means "we moved apart, go find out which of us moved". It # does not mean the port is wrong. Where a capture and this tool disagree, the # capture wins. Use `tools/port/verify-capture` for the correctness question. # # tools/port/verify-screen # every screen in the manifest # tools/port/verify-screen main_menu title # named screens # # Writes .godot.png, .ref.png and .diff.png into # $OUT (default: a directory under /tmp) and prints, per screen, the largest # per-channel difference anywhere in the frame. # # The two renderers are held to the same inputs on purpose: # # * the COMPARISON CLI is the one built by `build-reference-cli`, from the same # `sylpheed-formats` revision the exporter is pinned to. /reborn's own # target/ is a live mount of the other agent's checkout and moves mid-run; a # pixel disagreement against a moving decoder proves nothing. # * `--black` because Godot clears to black and the screen carries its own # background. The CLI's default dim slate stands in for a 3D scene behind an # in-mission screen, which is not this screen. # # ⚠️ THAT PREMISE IS DECLARED ON 12 OF 16 SCREENS AND ASSUMED ON 4. Audited # 2026-08-30: a screen "carries its own background" when it declares a # full-screen untextured primitive at `t=0` with `fade_argb 0xff000000` -- # opaque black. Twelve do (`pteff00`, `palogo_eff0`, `pgloading_eff00`). # Four do NOT. I first called those four "composited rather than standalone" [refuted]; # that reading is REFUTED disc-wide (see below) and what they share is only # that they do not begin from black: # # press_start / press_start_jp -- one element, the plate, drawn OVER the # title; its own `name_why` says so. The game never shows it on black. # build_00 / build_01 -- loading variants carrying the `pgloading_*` set # WITHOUT the `pgloading_eff00` backdrop that build_12/15 declare. # # ✅ Harmless HERE, because both renderers are given `--black` and the # assumption cancels in a consistency check. It would NOT be harmless in an # oracle comparison, and `verify-capture` already avoids it: the plate is # scored as `--screen=title --overlay=press_start`, over the title, not on # black. # # 📌 The audit is a rule worth having WITHIN THIS ARCHIVE, and its first # reading was wrong. I called it "standalone versus composited"; the Decoder # ran it disc-wide and it does not carry: **76 of 965 builds, 7.9 %**, with # `GP_HANGAR_ARSENAL` **0 of 390**, `GP_OPTIONS` 0/14, `GP_PAUSE_MENU` 0/6 -- # screens a player plainly sees AS screens. Read as "composited", the rule # makes 92 % of the game composited, which the archives do not support. # # ✅ What survives is narrower: it separates **screens that begin from black** # from everything else. The negative class is heterogeneous -- a pause menu # over gameplay, a hangar over a 3D scene and a plate over a title are not the # same kind of thing -- which is exactly what a two-way rule cannot express. # # ⚠️ Within `GP_TITLE` it is exact and independently reproduced from the disc # (12/4, the four being entries 0-3). That is the only archive it is claimed # for. Do NOT carry it into `GP_READY_ROOM`, `GP_HANGAR_ARSENAL`, # `GP_MISSION_SELECT` or `GP_OPTIONS`: in three of them it classifies every # screen alike, so it would look like a clean answer and say nothing. # * `--primitives --animated` because those are what make the CLI draw the same # element set. `--focus` is NOT passed: nothing is focused at rest (HANDOFF # Q5 measured initial focus as unstable boot to boot, so choosing one is # P5's decision). # * `--pose=rest` on the Godot side. Since P2 the port's DEFAULT is to play the # timeline, and the settled timeline is deliberately NOT what `rest` says -- # the export's `rest` misses `ptframe1`/`ptframe2` on the main menu, and the # running game shows them (docs/DECISIONS.md). Both renderers read `rest` # through the same decoder, so asking for it here keeps this a test of the # PORT against the reference. It is not the test of whether `rest` is right; # that one is the oracle capture, and the port already departs from it. # # A difference here is not automatically the port's fault, and it is not # automatically a fault at all. Say which renderer moved and why -- do not tune # until they match. set -euo pipefail cd "${PROJECT_DIR:-/work}" # THE REFERENCE IS THE WORKSPACE'S OWN `sylpheed-cli`, and that is a change. # # It used to be a binary built per PINNED REVISION into `reference-cli//`, # because `sylpheed-formats` was a git dependency and /reborn's target/ was a # live mount of the other agent's checkout that moved mid-run. A pixel # disagreement against a moving decoder has a free variable in it. # # The monorepo merge (`65cefa7`) removed that problem by construction: # `crates/sylpheed-export/Cargo.toml` now says # `sylpheed-formats = { path = "../sylpheed-formats" }`, so the exporter, this # reference and the port all read ONE decoder -- the working tree's. # # 🔴 It also silently broke the old machinery, and this script did not notice. # `build-reference-cli` greps Cargo.toml for `Syplheed-Reborn.git", rev = "..."`; # that line no longer exists, so the script exits 1 and the binary at # `reference-cli/sylpheed-cli` is whatever was last built before the merge -- # here, three hours older than the sources and from a revision nothing points # at any more. Running the diff against it would have compared the port to a # decoder from another era and called the result a regression check. This # corpus has already been bitten by a stale reference renderer three times. # # So: build it from the workspace. `SYLPHEED_CLI` still overrides, for anyone # who does want to pin one deliberately. CLI="${SYLPHEED_CLI:-}" if [ -z "$CLI" ]; then CLI="${CARGO_TARGET_DIR:-/sylph-home/port/target-container}/release/sylpheed-cli" cargo build --release -p sylpheed-cli >/dev/null 2>&1 || true fi DISC="${SYLPHEED_DISC:-/disc}" OUT="${OUT:-${TMPDIR:-/tmp}/verify-screen}" export DISPLAY="${DISPLAY:-:97}" [ -x "$CLI" ] || { echo "no reference CLI at $CLI -- \`cargo build --release -p sylpheed-cli\` failed?" >&2; exit 2; } [ -f export/manifest.json ] || { echo "no export/manifest.json -- run build-export --run" >&2; exit 2; } # 🔴 THE REFERENCE BINARY IS NOT NECESSARILY THE ONE THIS SCRIPT BUILT. # # `CARGO_TARGET_DIR` is a SHARED `/sylph-home/port/target-container`. Two source # trees -- this workspace and any worktree built with the same variable set -- # write one `release/sylpheed-cli`, and cargo fingerprints per source path, so # each build reports "Finished" while the binary on disk belongs to whichever # tree wrote last. `cargo build` here returns in 0.15 s and changes nothing. # # That is the hazard the header above says the monorepo removed. It did not; the # shared target dir reintroduced it by another route. Measured 2026-08-30: a CLI # built from this workspace is `rest t=70` (the stale record layout) while the # binary actually sitting in the target dir was `rest t=12` (fixed) -- so this # script was comparing the port against a decoder from a tree nobody had named. # # ⚠️ It happened to be the RIGHT era, which is worse than wrong: it agreed with # the exporter's pin by luck, and one successful rebuild would have flipped it # silently. `title_jp` differs by 74 507 px between the two eras. # # So the era is CHECKED, against the export the port actually reads, rather than # assumed from having run `cargo build`. ref_rest=$("$CLI" screen info "$DISC/dat/GP_TITLE.pak" --build 5 --all 2>/dev/null \ | grep -i 'pteff00' | head -1 | sed -n 's/.*rest (0,0) t=\([0-9]*\).*/\1/p') exp_rest=$(python3 -c ' import json m=json.load(open("export/manifest.json")) f=next(s["file"] for s in m["screens"] if s["name"]=="main_menu") d=json.load(open("export/"+f)) print(int(next(e for e in d["elements"] if e.get("id")=="pteff00")["rest"]["t"]))') if [ -n "$ref_rest" ] && [ "$ref_rest" != "$exp_rest" ]; then echo "🔴 the reference CLI and the export disagree on the decoder era:" >&2 echo " reference $CLI says pteff00 rest t=$ref_rest" >&2 echo " export/ (built by the pinned exporter) says rest t=$exp_rest" >&2 echo " Every row below would compare two decoder eras. Refusing." >&2 echo "" >&2 echo " REMEDY, verified both directions 2026-08-30: this workspace's" >&2 echo " ui_layout.rs is the STALE era and still carries the retired" >&2 echo " SYLPHEED_KF_TIME_SHIFT knob, which converts it to the corrected" >&2 echo " reading. Re-run with SYLPHEED_KF_TIME_SHIFT=1 and the reference" >&2 echo " reports rest t=12, matching the pinned exporter; without it, t=70." >&2 echo " The knob is absent from the pinned tag, so it cannot affect export/." >&2 exit 2 fi mkdir -p "$OUT" # Godot needs one scan to register the `class_name` globals; without it every # script fails to parse and the run dies with no frame drawn. [ -d port/.godot ] || godot --headless --path port --import >/dev/null 2>&1 screens=("$@") if [ ${#screens[@]} -eq 0 ]; then mapfile -t screens < <(python3 -c ' import json; print("\n".join(s["name"] for s in json.load(open("export/manifest.json"))["screens"]))') fi status=0 for name in "${screens[@]}"; do build=$(python3 -c ' import json,sys m=json.load(open("export/manifest.json")) f=next(s["file"] for s in m["screens"] if s["name"]==sys.argv[1]) print(json.load(open("export/"+f))["source"]["build"])' "$name") # `--all` because the exporter now addresses by PAK ENTRY INDEX, which is the # numbering `--all` uses; without it the CLI enumerates only the 12 bundles # `is_build` accepts and `--build 10` would land on entry 12. `--all` widens # the list, it does not change how any one bundle composites. "$CLI" screen render "$DISC/dat/GP_TITLE.pak" "$OUT/$name.ref.png" \ --build "$build" --all --black --primitives --animated >/dev/null # 🔴 THE REFERENCE RENDERER SILENTLY OMITS A `.tbm` BACKGROUND. # # The Decoder reached and captured the TUTORIAL screen and found that # `screen render` draws every OTHER element of a `.tbm`-bearing build and # leaves the background out, with no diagnostic: their render of GP_TUTORIAL # build 0 is the correct layout on pure black, 6.0-6.4 % inked against the # game's 99.7 %. `docs/re/structures/tbm-submenu-not-reached.md`, their branch. # # I confirmed the shape of it here with both controls: `screen info` reports # `pubase.tbm` on GP_TUTORIAL build 0 and no `.tbm` on any of the 16 builds in # my manifest. So this trap CANNOT fire today. # # ⚠️ That is a fact about today's manifest, not a property of this script, and # the failure it would cause is the expensive kind: the port draws a # background the reference does not, the row reads DIFFERS, and the header # above tells the reader to go find out which renderer moved. Neither did. # The row would be a real disagreement caused by a KNOWN omission on the # reference side, and nothing on screen would say so. # # So the row says so. This does not change the verdict or the bar -- it # attaches the provenance to the one row that would otherwise mislead. tbm=$("$CLI" screen info "$DISC/dat/GP_TITLE.pak" --build "$build" --all 2>/dev/null \ | grep -ioc '\.tbm' || true) # 🔴 `--loop-phase=0` PINS THE PULSE, AND WITHOUT IT THIS SCRIPT WAS # NONDETERMINISTIC. `press_start` returned `over3` **5021, 8919, 5021** on # three identical runs: the plate's looping focus record rides `time_units`, # so the captured frame lands wherever the grab fell, while the reference # renderer cannot pulse at all. # # ⚠️ The port is NOT the thing that is wrong. A thing that pulses does not # stop because the screen has arrived, and the pulse is measured. What was # wrong is comparing a moving frame against a static one and calling the # difference a regression -- a detector that answers differently each run # teaches its reader to ignore it, which is worse than one that fails. # # So the phase is pinned HERE, in the harness, and nothing about playback # changes: `loop_phase_units` defaults to free-running everywhere else. # ⚠️ It is usually stable -- 3 of 4 control runs agreed -- which is exactly # why this survived: it looks deterministic most of the time. godot --path port --resolution 1280x720 -- \ "--screen=$name" --pose=rest --loop-phase=0 "--capture=$OUT/$name.godot.png" >"$OUT/$name.log" 2>&1 convert "$OUT/$name.godot.png" "$OUT/$name.ref.png" \ -compose difference -composite -colorspace Gray -auto-level "$OUT/$name.diff.png" read -r max mean <<<"$(convert "$OUT/$name.godot.png" "$OUT/$name.ref.png" \ -compose difference -composite -format "%[fx:maxima*255] %[fx:mean*255]" info:)" # HOW MANY pixels are over the bar, not just how far the worst one is. A # single `max` cannot tell 2 pixels from 25 444, and this run produced both: # `main_menu` trips the threshold on TWO pixels out of 921 600 while # `title_jp` trips it on 2.8 % of the frame. Reporting only the max made those # the same verdict, which is how a real disagreement hides behind a rounding # one. The bar itself is NOT raised -- tuning a threshold until things match # is the failure this script's own header warns about. over=$(convert "$OUT/$name.godot.png" "$OUT/$name.ref.png" \ -compose difference -composite -colorspace Gray -threshold $((3*65535/255)) \ -format "%[fx:int(mean*w*h)]" info:) # BOTH FRAMES BLANK IS NOT AGREEMENT, AND THIS SCRIPT USED TO SAY IT WAS. # # `build_12` and `build_15` -- the two dressed loading screens -- render as # pure black in BOTH renderers, mean 0 and max 0, so the difference is 0 and # the row read `max 0 over3 0 OK`. Two of the sixteen rows in the committed # baseline were comparing nothing against nothing and reporting the strongest # verdict this script has. # # That is worse than a missing test: it is a test that reports a pass. The # screens are black because `pgloading_eff00` is a full-frame opaque black # quad whose `rest.t` (38) sits inside its own opening black hold, and # `--pose=rest` freezes it there -- see docs/port/DECISIONS.md. Whether that # is the port's bug or the decoders' reading of `rest` is open; what is not # open is that a blank pair may not be scored. # # ✅ RESOLVED 2026-08-30, AND THE PARAGRAPH ABOVE IS NOW HISTORY. It was the # PAINT ORDER, not `rest`. `pgloading_eff00` carries `layer: null`, # `layer_source: none` -- the only elements in the export with neither a read # nor an implied key -- so without the forced-backdrop pass the first element # becomes `pgloading_loop5` and the opaque quad paints over everything. With # the pass, both screens render at max 214.5 in BOTH renderers (mean 1.949 # port, 1.918 reference) and the rows read `OK` on a real comparison. # # ⚠️ The guard STAYS. It is not firing today, which is exactly when a guard # quietly rots -- and it was right when it was written: two of sixteen rows # were comparing nothing against nothing and reporting this script's # strongest verdict. Leaving the reasoning above intact is deliberate; a # reader who hits a blank pair tomorrow needs it. # # So blankness is checked FIRST and reported as its own verdict. It is not a # failure -- the port may legitimately have nothing to draw -- but it is not a # pass either, and `status` is left alone so an unrelated screen's DIFFERS is # still what fails the run. ink=$(convert "$OUT/$name.godot.png" "$OUT/$name.ref.png" \ -evaluate-sequence max -colorspace Gray -format "%[fx:maxima*255]" info:) verdict=OK if awk "BEGIN{exit !($ink <= 0)}"; then verdict="BLANK -- both renderers drew nothing; this row proves nothing" else # 🔴 THE VERDICT USES `over3`, NOT `max` ALONE, AND FOR YEARS IT DID NOT. # # This script computed `over3` precisely because "a single `max` cannot tell # 2 pixels from 25 444" -- its own words, a few lines up -- and then decided # the verdict on `max` regardless. So `main_menu` (max 4, over3 **0**) read # DIFFERS while `extras` (max 3, over3 0) read OK: one unit on one pixel, # separating two frames that are pixel-for-pixel equivalent at the bar. # # ⚠️ This is NOT raising the bar, which this file rightly warns against. The # bar is still 3. What changes is that a frame with NO pixel over it gets a # verdict of its own instead of being lumped in with a real disagreement -- # the distinction the statistic was added to make and was never given. if awk "BEGIN{exit !($over > 0)}"; then verdict=DIFFERS; status=1 elif awk "BEGIN{exit !($max > 3)}"; then verdict="ROUNDING -- max $max but NO pixel over the bar" fi fi if [ "${tbm:-0}" -gt 0 ]; then verdict="$verdict [build carries a .tbm: the REFERENCE omits that background, so a DIFFERS here is likely theirs]" fi printf '%-17s build %-3s max %-5s mean %-8s over3 %-7s %s\n' \ "$name" "$build" "$max" "${mean:0:6}" "$over" "$verdict" done echo "artifacts in $OUT" exit $status