aa51e5af67
The previous entry scored verify-screen's title_jp frame against the oracle and concluded the port had moved away from the game. That frame is posed --pose=rest, which the port does not ship. Posed as it runs, the disputed block scores +0.9994 against the reference's +0.8727, and the whole surface +0.9652 against +0.9200 -- holding under gamma compensation and on the English control (+0.9946 vs +0.9560). The port is closer to the game than the reference on both title screens. Mechanism: ptlogo_back2eff1 is (0,0)(98,0)(100,255)(102,255)(104,0) -- a 4-unit sparkle whose rest.t is the peak of its own flash. Six of them stagger across the logo, so --pose=rest fires every sparkle at once. The 25.6% excess light was real and was in a frame nobody sees. verify-screen is not at fault: it poses rest deliberately, so that both renderers read one decoder and the run is a consistency check. I used a consistency-check frame for a correctness question. Its header now says its frames must never be scored against a capture. A second claim in that entry was also wrong -- both screens draw those layers under pose=rest; I had compared a --menu timeline log against a verify-screen rest log and read a mode difference as a screen difference. verify-capture takes a fifth per-row field, a capture crop, because this capture is a full display frame with the surface at +0+45 while the others are pre-cropped. With it title_jp reads RMSE 20.91 / 1.04%, beside title's 14.16 / 0.21%. The row prints 'no capture' until their branch merges. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N7FiFFFwbvG2uxdcEh8HyF
286 lines
16 KiB
Bash
Executable File
286 lines
16 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Diff the port's render against a CAPTURE OF THE REAL GAME.
|
|
#
|
|
# tools/port/verify-capture main_menu
|
|
# tools/port/verify-capture # every screen with a capture
|
|
#
|
|
# THIS IS THE CORRECTNESS CHECK. `verify-screen` is the consistency one, and its
|
|
# own header has pointed at this file since P1 -- `tools/verify-capture` -- while
|
|
# this file did not exist. The port has had a harness comparing two renderers
|
|
# that share its assumptions, and none comparing it to the game.
|
|
#
|
|
# ⚠️ That gap is not academic. `docs/re/captures/ORACLE-CAPTURES.md`: two
|
|
# renderers agreeing proves nothing, and this corpus has been bitten three times
|
|
# -- the dropped `pteff05` background, the scale-0 rect, and `rest()`. Every one
|
|
# was invisible to a render-vs-render diff and obvious against a capture.
|
|
#
|
|
# WHAT IT CAN CONCLUDE, and what it cannot:
|
|
#
|
|
# * ✅ STRUCTURE. Something drawn that should not be, or missing that should be,
|
|
# shows as a large connected region of difference. That is the failure mode
|
|
# the three above were, and it is what this tool is for.
|
|
# * 🔴 NOT a pixel score. The captures are NOT gamma-neutral:
|
|
# `capture ~= 255*(render/255)^g` with g ~ 1.34-1.49, and that ramp is THE
|
|
# GAME'S, not the capture path's (`docs/re/structures/ui-render-tone-curve.md`).
|
|
# So RMSE has a floor and driving it lower is fitting the ramp. This reports
|
|
# the raw difference AND the gamma-compensated one, and neither is a target.
|
|
# * ⚠️ A capture is ONE MOMENT. Several screens are still animating -- the
|
|
# title's two `ptloop` sweeps never stop -- and the focused button in a
|
|
# capture may not be the one the port focuses. Differences confined to a
|
|
# button or a moving element are expected; say which before calling anything.
|
|
#
|
|
# Geometry needs no correction: the corpus cross-correlated a render against
|
|
# `live-main-menu.png` over +/-6 px and the best alignment is exactly (0,0) at
|
|
# 0.9466. The captures are a 1279x675 top-left crop of the 1280x720 guest
|
|
# surface, so the render is cropped to match and nothing is scaled.
|
|
set -euo pipefail
|
|
cd "${PROJECT_DIR:-/work}"
|
|
export DISPLAY="${DISPLAY:-:97}"
|
|
OUT="${OUT:-$(mktemp -d)}"; mkdir -p "$OUT"
|
|
CAPS=docs/re/captures/title-builds
|
|
|
|
# screen : capture : how to pose it
|
|
# 🔴 A MENU CAPTURE HAS A BUTTON FOCUSED, AND THE FIRST VERSION OF THIS TOOL
|
|
# RENDERED WITH NONE. `--screen=` draws no focus record at all, so `main_menu`
|
|
# was being compared to the oracle in a state the oracle was never in: 2 159
|
|
# differing pixels, of which 74 % sat inside the focus signature. Rendered with
|
|
# focus it is 531 -- 0.06 % of the frame, a 4x improvement that was entirely my
|
|
# harness posing the port wrong.
|
|
#
|
|
# `--menu=` applies `authored/flow.json`'s initial focus and `--script=wait`
|
|
# shoots one settled frame and exits.
|
|
MAP=(
|
|
"main_menu:$CAPS/live-main-menu.png:menu"
|
|
"extras:$CAPS/live-extras.png:menu"
|
|
# The only capture of a MEASURED focus state, and it was unusable until
|
|
# `--focus=` was made to work on the `--menu` path (it parsed, was stored, and
|
|
# was overwritten by the authored initial focus on every `_menu_enter`).
|
|
#
|
|
# It discriminates: rendering each of the five buttons focused against this
|
|
# capture gives 0.1355 % for `ptbtn04` and 0.70-0.82 % for the other four. The
|
|
# port's focus rendering identifies the right button by a factor of five.
|
|
"main_menu_options:$CAPS/live-main-menu-options-focused.png:focus:ptbtn04"
|
|
# ⚠️ THE TITLE IS POSED AT t=357.7 UNITS, NOT AT ITS SETTLE, and the time is
|
|
# MEASURED rather than chosen. The two `ptloop` sweeps are a continuous
|
|
# animation whose leaf group ends at t=600 with the quads parked off-screen at
|
|
# x=1521, so posing at the settle compares a still frame against a capture
|
|
# taken mid-sweep and simply omits them.
|
|
#
|
|
# t=357.7 is the Decoder's REFINED fit, and the refinement is worth knowing.
|
|
# Its first value, 355, came from the two per-draw alpha bytes alone and left
|
|
# an 11.5 px residual that looked like a pivot problem. Solving the same
|
|
# instant on the vertex POSITIONS instead gives t=357.88 and 357.58 to
|
|
# +/-0.12 units, against +/-1.54 and +/-1.89 from the alphas -- alpha moves
|
|
# only 0.27-0.33 levels per unit, so one byte of quantisation is worth 6-8 px
|
|
# of sweep. At 357.7 the centres land within 0.70 px and both alphas inside one
|
|
# level. THE 11.5 px WAS THE FIT'S RESOLUTION, NOT GEOMETRY.
|
|
#
|
|
# ⚠️ And there is no pivot correction: the leaf pivot is (200, 90) on a 399x180
|
|
# sprite, so it is the centre to within half a pixel -- checked here against
|
|
# the export rather than taken.
|
|
#
|
|
# ⚠️ It is NOT the time that minimises the difference: t=390 measures 1.65 %
|
|
# against 1.82 % here. Picking that one would be fitting the pose to the
|
|
# score, which is the thing this harness exists not to do.
|
|
"title:$CAPS/live-title-build4-no-plate.png:t357"
|
|
# The Japanese title at rest, Decoder 310bf86. Settled pose by omission --
|
|
# the capture is demonstrated at rest (five frames over 6 s, 0 px change in
|
|
# the logo block while 5-8 % of the frame moves).
|
|
#
|
|
# 🔴 THIS ROW EXISTS BECAUSE SCORING THE WRONG FRAME COST A WRONG CONCLUSION.
|
|
# `verify-screen` poses at `--pose=rest`, which for this screen lights every
|
|
# `ptlogo_back2eff*` sparkle at its own peak simultaneously -- `rest` for those
|
|
# elements IS the peak of a 4-unit flash. That frame is fine for the
|
|
# consistency check it was built for and must never be scored against a
|
|
# capture: doing so put the port at r +0.7462 against the reference's +0.8727
|
|
# and I wrote up that the port had moved away from the game. Posed as it
|
|
# SHIPS, the same block scores **+0.9994**.
|
|
"title_jp:$CAPS/live-title-jp-at-rest.png:settled::1279x675+1+45"
|
|
# A SECOND title comparison, and the most sensitive row this tool has.
|
|
#
|
|
# `live-title-press-a.png` is the title WITH the plate. Posed at t=237 -- inside
|
|
# the plate's own 8-unit opaque window, t=236-238 -- the port matches it at
|
|
# **0.00093 %**, two orders below every other row. That makes it the best
|
|
# regression detector here: anything structural that moves will show.
|
|
#
|
|
# ⚠️ The instant is FITTED, not measured: 237 is where this capture's content
|
|
# places it, found by sweeping. That is legitimate for choosing which frame to
|
|
# compare against -- every row does it -- but it is not a claim about the game,
|
|
# and the 0.00093 % is therefore a floor for THIS pose, not a general accuracy.
|
|
#
|
|
# It also closes the systematic-error question the leaf sweep left open. Two
|
|
# independent captures fit at two DIFFERENT phases -- this one at 237, the
|
|
# no-plate one at ~400 -- and both to 0.01 % or better. A geometry error in how
|
|
# the sweeps are drawn would leave a floor in both. Neither has one.
|
|
"title_plate:$CAPS/live-title-press-a.png:plate"
|
|
# A BANDED row -- the capture is 1279x120, not a full frame, and the harness
|
|
# could not compare one until now. That was the only reason this capture sat
|
|
# unused; nothing about it was unusable.
|
|
#
|
|
# Its y offset is MEASURED, not guessed: sliding it down the render, the
|
|
# structural difference is 0.354 % at y=520 against 8.9-9.1 % five pixels
|
|
# either side and 17-52 % further out. A 25x drop over five pixels.
|
|
#
|
|
# ⚠️ Its residual is NOT the port's error. The port reproduces the same band of
|
|
# `live-title-press-a.png` EXACTLY (0.000 %), and the two captures differ from
|
|
# each other by 0.301 % -- two thin horizontal strips, 248x5 px and 206x1 px,
|
|
# the shape of a sub-pixel edge difference rather than a state difference. So
|
|
# 0.354 % is very nearly the oracle-to-oracle gap and this row's job is to stay
|
|
# near it, not to reach zero.
|
|
"title_band:$CAPS/live-attract-title-press-a-band.png:band"
|
|
"publisher_logo:$CAPS/live-splash-publisher.png:screen"
|
|
"developer_logos:$CAPS/live-splash-developer.png:screen"
|
|
)
|
|
CURVE=""
|
|
if [ "${1:-}" = "--curve" ]; then CURVE=1; shift; fi
|
|
want=("$@")
|
|
echo "RMSE is reported and is NOT a target: the capture carries the game's own"
|
|
echo "tone ramp, so it has a floor. What finds a real defect is the DIFFERING"
|
|
echo "REGION -- a missing or misplaced element is a large connected blob."
|
|
echo
|
|
printf '%-17s %-9s %-7s %-22s %s\n' screen raw-rmse diff region note
|
|
for row in "${MAP[@]}"; do
|
|
IFS=: read -r name cap pose forced capcrop <<<"$row"
|
|
if [ ${#want[@]} -gt 0 ] && ! printf '%s\n' "${want[@]}" | grep -qx "$name"; then continue; fi
|
|
[ -f "$cap" ] || { printf '%-17s %s\n' "$name" "no capture"; continue; }
|
|
if [ "$pose" = band ]; then
|
|
godot --path port --resolution 1280x720 -- "--screen=title" --overlay=press_start \
|
|
--time=3.95 "--capture=$OUT/$name.full.png" >"$OUT/$name.log" 2>&1 || true
|
|
[ -f "$OUT/$name.full.png" ] && convert "$OUT/$name.full.png" \
|
|
-crop 1279x120+0+520 +repage "$OUT/$name.render.png"
|
|
elif [ "$pose" = focus ]; then
|
|
godot --path port --resolution 1280x720 -- "--menu=main_menu" "--focus=$forced" \
|
|
--script=wait "--shots=$OUT/$name" >"$OUT/$name.log" 2>&1 || true
|
|
[ -f "$OUT/${name}_00_start.png" ] && cp "$OUT/${name}_00_start.png" "$OUT/$name.render.png"
|
|
elif [ "$pose" = menu ]; then
|
|
godot --path port --resolution 1280x720 -- "--menu=$name" --script=wait \
|
|
"--shots=$OUT/$name" >"$OUT/$name.log" 2>&1 || true
|
|
[ -f "$OUT/${name}_00_start.png" ] && cp "$OUT/${name}_00_start.png" "$OUT/$name.render.png"
|
|
elif [ "$pose" = plate ]; then
|
|
godot --path port --resolution 1280x720 -- "--screen=title" --overlay=press_start \
|
|
--time=3.95 "--capture=$OUT/$name.render.png" >"$OUT/$name.log" 2>&1 || true
|
|
elif [ "$pose" = t357 ]; then
|
|
# 🔴 NO `--time` HERE EITHER, and the row's note used to claim otherwise.
|
|
#
|
|
# It passed `--time=5.9617` (t=357.7 units, the Decoder's refined sweep fit)
|
|
# and that value was NEVER APPLIED: `pose_at` replaced it with the screen's
|
|
# settle instant, t=198, on every run. Every title figure this tool has ever
|
|
# printed -- including the 0.26 % the port has quoted repeatedly -- was
|
|
# measured at the SETTLE, under a note saying t=357.7.
|
|
#
|
|
# Honouring it now makes that visible: t=357.7 is PAST the title's own group,
|
|
# which ends at t=269, so the whole screen poses at its faded-out final
|
|
# keyframes and the disagreement goes to 30.97 %. The instant was only ever
|
|
# meant for the `ptloop` LEAF, which runs to t=600 and is looped separately
|
|
# by `loop_leaf` (authored/rendering.json). Applying it to the whole screen
|
|
# was always wrong; it was harmless only while it was ignored.
|
|
#
|
|
# So: pose at the settle, which is what was actually being measured, and let
|
|
# the leaf loop carry the sweeps' phase.
|
|
godot --path port --resolution 1280x720 -- "--screen=$name" \
|
|
"--capture=$OUT/$name.render.png" >"$OUT/$name.log" 2>&1 || true
|
|
else
|
|
# NO `--time`. It used to pass `--time=99` as an idiom for "settled", and
|
|
# that worked only because `--time` was SILENTLY IGNORED on a screen with a
|
|
# settle window: `pose_at` overwrote the requested instant with
|
|
# `settle_instant` whenever `holding` was true. The tool asked for t=5940
|
|
# units and was handed the settle instant, which is the pose it actually
|
|
# wants -- and the 0.01 % agreements on both splashes were measured through
|
|
# that accident. Now that `--time` is honoured, asking for it explicitly
|
|
# would pose past the end of every group, so the request is simply dropped
|
|
# and the settled pose asked for by omission.
|
|
godot --path port --resolution 1280x720 -- "--screen=$name" \
|
|
"--capture=$OUT/$name.render.png" >"$OUT/$name.log" 2>&1 || true
|
|
fi
|
|
[ -f "$OUT/$name.render.png" ] || { printf '%-17s %s\n' "$name" "render failed"; continue; }
|
|
# Crop the render to the capture's frame. The capture is the crop, not a scale.
|
|
if [ "$pose" = band ]; then
|
|
cp "$OUT/$name.render.png" "$OUT/$name.crop.png"
|
|
else
|
|
convert "$OUT/$name.render.png" -crop 1279x675+0+0 +repage "$OUT/$name.crop.png"
|
|
fi
|
|
# ⚠️ A FIFTH FIELD, because not every capture is pre-cropped to the game
|
|
# surface. Every capture in `$CAPS` until now was already 1279x675, so the
|
|
# render was cropped and the capture used as-is. The JP title capture is a
|
|
# full 1280x720 DISPLAY frame with the surface at +0+45 -- comparing it whole
|
|
# would score the port against a 45px shift and report a catastrophe.
|
|
#
|
|
# The offset is MEASURED, not inherited from the earlier submenu capture:
|
|
# row/column profile correlation against the port, with the English pair as a
|
|
# control, gives (0,0) for the control at r 0.994 and dy=-45 for this frame.
|
|
if [ -n "$capcrop" ]; then
|
|
convert "$cap" -crop "$capcrop" +repage "$OUT/$name.cap.png"
|
|
cap="$OUT/$name.cap.png"
|
|
fi
|
|
raw=$(convert "$OUT/$name.crop.png" "$cap" -metric RMSE -compare -format "%[distortion]" info: 2>&1 | tail -1)
|
|
raw=$(python3 -c "print('%.2f' % (float('$raw')*255))" 2>/dev/null || echo "?")
|
|
note=""
|
|
case "$name" in
|
|
main_menu|extras) note="rendered with authored initial focus" ;;
|
|
title) note="settle t=198; residual is the SWEEP PHASE -- see DECISIONS.md" ;;
|
|
esac
|
|
# Where the difference lives. This comes FIRST because it is what the gamma
|
|
# sweep has to be protected from.
|
|
convert "$OUT/$name.crop.png" "$cap" -compose difference -composite \
|
|
-colorspace Gray -threshold 25% "$OUT/$name.mask.png"
|
|
|
|
# THE TONE RELATIONSHIP IS REPORTED AS A CURVE, NOT AS A BEST EXPONENT, and
|
|
# two earlier versions of this tool reported an exponent and were wrong twice.
|
|
#
|
|
# `docs/re/structures/ui-render-tone-curve.md` models it as
|
|
# `capture = 255*(render/255)^g`, g ~ 1.34-1.49, measured on dark flat patches
|
|
# and explicitly not constrained above render ~60. Binning every structurally
|
|
# matched pixel of `main_menu` by render level and averaging the capture gives:
|
|
#
|
|
# render capture implied g pixels
|
|
# 8 4.04 1.20 183 026
|
|
# 16 7.89 1.26 227 630
|
|
# 24 15.57 1.18 100 945
|
|
# 32 26.15 1.10 87 474
|
|
# 40 38.07 1.03 86 094
|
|
# 48 53.96 0.93 85 255
|
|
# 64 78.52 0.85 6 509
|
|
# 96 130.44 0.69 1 682
|
|
#
|
|
# ⚠️ **The implied exponent is not constant. It falls monotonically and crosses
|
|
# 1.0 near render ~44**, so the capture is DARKER than the render in the darks
|
|
# and BRIGHTER in the midtones. A single power law cannot express that, which
|
|
# is exactly why a whole-frame fit returns 1.00: the two halves cancel. The
|
|
# corpus's reach -- "nothing constrains midtones or highlights" -- was a real
|
|
# limit and this is what lies past it.
|
|
#
|
|
# So: no best-g is printed. The table above is the instrument that can actually
|
|
# be argued with; `tools/port/verify-capture --curve SCREEN` regenerates it.
|
|
frac=$(convert "$OUT/$name.mask.png" -format "%[fx:mean*100]" info:)
|
|
box=$(convert "$OUT/$name.mask.png" -trim -format "%wx%h%X%Y" info: 2>/dev/null || echo "-")
|
|
printf '%-17s %-9s %6.2f%% %-22s %s\n' "$name" "$raw" "$frac" "$box" "$note"
|
|
done
|
|
if [ -n "$CURVE" ]; then
|
|
for row in "${MAP[@]}"; do
|
|
IFS=: read -r name cap pose <<<"$row"
|
|
if [ ${#want[@]} -gt 0 ] && ! printf '%s\n' "${want[@]}" | grep -qx "$name"; then continue; fi
|
|
[ -f "$OUT/$name.mask.png" ] || continue
|
|
convert "$OUT/$name.crop.png" -colorspace Gray -depth 8 "gray:$OUT/$name.r.gray"
|
|
convert "$cap" -colorspace Gray -depth 8 "gray:$OUT/$name.c.gray"
|
|
convert "$OUT/$name.mask.png" -colorspace Gray -depth 8 "gray:$OUT/$name.m.gray"
|
|
echo; echo "transfer curve, $name -- structurally matched pixels only"
|
|
python3 - "$OUT/$name" <<'PYEOF'
|
|
import sys, math
|
|
b = sys.argv[1]
|
|
r = open(b+".r.gray","rb").read(); c = open(b+".c.gray","rb").read(); m = open(b+".m.gray","rb").read()
|
|
n = min(len(r), len(c), len(m)); bins = {}
|
|
for i in range(n):
|
|
if m[i]: continue
|
|
s = bins.setdefault(r[i]//8*8, [0,0]); s[0] += c[i]; s[1] += 1
|
|
print(" %-8s %-9s %-9s %s" % ("render","capture","implied g","pixels"))
|
|
for k in sorted(bins):
|
|
tot, cnt = bins[k]
|
|
if cnt < 500 or k < 8: continue
|
|
cap = tot/cnt
|
|
g = math.log(max(cap,0.5)/255.0)/math.log(k/255.0)
|
|
print(" %-8d %-9.2f %-9.2f %d" % (k, cap, g, cnt))
|
|
PYEOF
|
|
done
|
|
fi
|
|
echo "artifacts in $OUT"
|