re(challenge): MISSION SELECT renders the cleared-stage mask, and a control run

With word A = 2 the screen lists Stage01 SELECTABLE, carrying a High Score and a Best Time, and Stage02-Stage08 GREYED OUT. One cleared stage, one selectable entry, at the bit index that names it -- the mask's meaning is now visible on screen rather than inferred from disassembly. Two runs, identical navigation, fresh boot each: control word A 0x00000002 -> MISSION SELECT opens normally poked word A 0xFFFFFFFF -> MmAllocatePhysicalMemoryEx fails on 128 MB, guest throws, Xenia shows "Disc Read Error" So last commit's heap failure was caused by the poke, and by a careless one: 0xFFFFFFFF claims stages that do not exist (0, 17, 24-31 in word A). Poking only real story ids (0x0001FFFE = stages 1-16) does not blow the heap. That the list screen changes behaviour with the mask is itself evidence word A feeds it. Getting a trustworthy control took three tries, and every failure produced a plausible wrong answer rather than an error: 1. the title-glyph oracle fired during the ATTRACT MOVIE, so A was pressed at nothing and the run "reported 0 failures" for a screen it never reached; 2. Xvfb keeps the previous instance's framebuffer until the new one draws, so a screenshot seconds after launch showed the OLD run -- "MAIN MENU reached after 1s", against a process that no longer existed; 3. a single-pixel "is NEW GAME white?" test matched a white LOADING FLASH. Fixes, all in challenge_probe.sh: blank the root and refuse screen oracles for the first 40 s; identify a screen by a PATTERN of sampled points (white text AND the dark panel behind it), not one pixel; require two consecutive samples. The new oracle was validated offline against all six saved screenshots and accepts exactly the one real main menu. General lesson worth the words: a navigation oracle that can only fail by returning the wrong screen will happily produce a whole run of confident, meaningless results.
2026-08-13 20:44:27 +00:00
parent 0bc790de52
commit 3ef2c438ae
3 changed files with 111 additions and 43 deletions
--- a/tools/re-capture/challenge_probe.sh
+++ b/tools/re-capture/challenge_probe.sh
@@ -6,13 +6,17 @@
 #   word A 0x828F40C0  bit = stage id, for ids < 24  (story 1-16, tutorial 18-23)
 #   word B 0x828F4814  bit = stage id - 24           (challenge 24-29)
 # so setting every bit should make all six challenge missions available without
-# playing the campaign. This boots, reaches the title, pokes both words, and
-# screenshots the menus so the result can be seen.
+# playing the campaign.
+#
+# POKE=1 (default) sets both words; POKE=0 runs the identical navigation without
+# touching them. Run it BOTH ways: the first poked run hit a Xenia heap failure on
+# the way into MISSION SELECT, and only the control says whether that failure has
+# anything to do with the poke.
 #
 # Runs as ONE blocking foreground call on purpose: setsid'd processes are reaped
 # at turn boundaries, so a session split across calls loses its emulator.
 #
-# Usage: challenge_probe.sh [boot_timeout_s]
+# Usage: [POKE=0|1] [TAG=name] challenge_probe.sh [boot_timeout_s]
 set -u
 export HOME=/sylph-home/re
 export DISPLAY=:99
@@ -23,65 +27,111 @@ HERE="$(cd "$(dirname "$0")" && pwd)"
 pad()  { python3 "$HERE/pad.py"   "$@"; }
 poke() { python3 "$HERE/gpoke.py" "$@"; }
 SHOTS="$HOME/shots"
-BOOT_TIMEOUT="${1:-420}"
+BOOT_TIMEOUT="${1:-400}"
+POKE="${POKE:-1}"
+POKE_A="${POKE_A:-0x0001FFFE}"   # stages 1-16 cleared
+POKE_B="${POKE_B:-0x0000003F}"   # challenge stages 24-29 cleared
+TAG="${TAG:-$([ "$POKE" = 1 ] && echo poked || echo control)}"
 mkdir -p "$SHOTS"

 say() { echo "[$(date +%H:%M:%S)] $*"; }
+shot() { screenshot "$SHOTS/chal-$TAG-$1.png" >/dev/null 2>&1; }

 # --- clean slate -------------------------------------------------------------
 pkill -9 -x xenia_canary 2>/dev/null
 sleep 1
 rm -f /dev/shm/xenia_* 2>/dev/null
 : > "$XENIA_PAD_FILE"
+LOG="$HOME/canary.stdout"

 # --- launch ------------------------------------------------------------------
-say "launching canary (lavapipe, file pad)"
+say "launching canary (lavapipe, file pad) — POKE=$POKE tag=$TAG"
 run-canary --audio --apu=sdl --log_mask=13 \
  --logged_profile_slot_0_xuid=E0300000EFBEA3D4 \
  --hid=file --pad_file="$XENIA_PAD_FILE" &
-CANARY_PID=$!
-trap 'pkill -9 -x xenia_canary 2>/dev/null' EXIT

-# --- wait for the title ------------------------------------------------------
-# Oracle: the green "PRESS (A) BUTTON" glyph at (625,618).
-say "waiting for the title (up to ${BOOT_TIMEOUT}s)"
-TITLE=0
-for _ in $(seq 1 "$BOOT_TIMEOUT"); do
-  if screenshot /tmp/title-probe.png >/dev/null 2>&1; then
-    read -r r g b < <(convert /tmp/title-probe.png -format \
-      "%[fx:int(255*p{625,618}.r)] %[fx:int(255*p{625,618}.g)] %[fx:int(255*p{625,618}.b)]" info: 2>/dev/null)
-    if [ -n "${g:-}" ] && [ "$g" -gt 130 ] && [ $((g - r)) -gt 45 ] && [ $((g - b)) -gt 45 ]; then
-      say "TITLE detected (rgb $r,$g,$b)"
-      TITLE=1
-      break
-    fi
-  fi
+# Xvfb keeps the LAST instance's framebuffer until the new one draws, so a
+# screenshot taken seconds after launch shows the PREVIOUS run's screen. That is
+# how a control run once reported "MAIN MENU reached after 1s" against a menu
+# belonging to a process that no longer existed. Blank the root, and refuse to
+# believe any screen oracle until the emulator has had time to draw its own.
+xsetroot -solid black 2>/dev/null || true
+LAUNCH_GRACE=40
+
+# --- reach the MAIN MENU, verifying instead of pressing blind ----------------
+# Two oracles, both sampled from real screenshots:
+#   title screen : green "PRESS (A) BUTTON" glyph at (625,618)
+#   main menu    : the "NEW GAME" text at (648,221) is pure white (254,254,254),
+#                  where the title has the yellow planet (208,189,88)
+# The first control run pressed A while the ATTRACT MOVIE happened to show a
+# greenish pixel at the title-glyph spot, then navigated a menu that was never
+# open and reported "0 failures" for a screen it never reached. Verify the menu.
+px() { convert /tmp/nav-probe.png -format \
+       "%[fx:int(255*p{$1}.r)] %[fx:int(255*p{$1}.g)] %[fx:int(255*p{$1}.b)]" info: 2>/dev/null; }
+
+# A screen is identified by a PATTERN of sampled points, never by one pixel. A
+# single "is (648,221) white?" test matched a white loading flash, and the run
+# then navigated a menu that was not on screen -- the same class of mistake as
+# trusting the stale framebuffer. Require the menu's contrast: white "NEW GAME"
+# text AND the dark blue panel behind it.
+at_menu() {
+  screenshot /tmp/nav-probe.png >/dev/null 2>&1 || return 1
+  read -r r g b < <(px "648,221")     # NEW GAME text: white
+  [ -n "${r:-}" ] || return 1
+  [ "$r" -gt 230 ] && [ "$g" -gt 230 ] && [ "$b" -gt 230 ] || return 1
+  read -r r2 g2 b2 < <(px "560,300")  # panel left of LOAD GAME: dark blue
+  [ -n "${r2:-}" ] || return 1
+  [ "$r2" -lt 120 ] && [ "$b2" -gt "$r2" ]
+}
+at_title() {
+  screenshot /tmp/nav-probe.png >/dev/null 2>&1 || return 1
+  read -r r g b < <(px "625,618")
+  [ -n "${g:-}" ] && [ "$g" -gt 130 ] && [ $((g - r)) -gt 45 ] && [ $((g - b)) -gt 45 ]
+}
+
+say "waiting for the main menu (up to ${BOOT_TIMEOUT}s)"
+MENU=0
+for i in $(seq 1 "$BOOT_TIMEOUT"); do
+  if [ "$i" -lt "$LAUNCH_GRACE" ]; then sleep 1; continue; fi
+  if at_menu && sleep 1 && at_menu; then say "MAIN MENU reached after ${i}s"; MENU=1; break; fi
+  if at_title; then say "  title visible — tapping A"; pad tap A 0.25; sleep 2; fi
  sleep 1
 done
-[ "$TITLE" = 1 ] || { say "TIMEOUT: no title"; screenshot "$SHOTS/chal-00-timeout.png"; exit 1; }
+[ "$MENU" = 1 ] || { say "TIMEOUT: never reached the main menu"; shot 00-timeout; exit 1; }
+shot 01-mainmenu

-# --- prove the file pad works before trusting anything else -------------------
-say "pad check: tapping A at the title"
-pad tap A 0.25
-sleep 3
-screenshot "$SHOTS/chal-01-after-A.png" >/dev/null
-say "file-pad log lines so far:"
-grep -c 'file-pad' "$HOME/canary.stdout" 2>/dev/null || true
-grep 'file-pad' "$HOME/canary.stdout" 2>/dev/null | tail -3
-
-# --- read the gate words BEFORE poking ---------------------------------------
-say "gate words before poke:"
+say "gate words:"
 poke r32 0x828F40C0 1
 poke r32 0x828F4814 1

-# --- poke --------------------------------------------------------------------
-say "poking word A = 0xFFFFFFFF, word B = 0x3F"
-poke w32 0x828F40C0 0xFFFFFFFF
-poke w32 0x828F4814 0x0000003F
+if [ "$POKE" = 1 ]; then
+  # Default to REAL stage ids only. 0xFFFFFFFF claims stages that do not exist
+  # (0, 17, and 24-31 in word A), and that run blew the guest heap:
+  # MmAllocatePhysicalMemoryEx could not satisfy 128 MB and the guest threw.
+  # Word A bits 1..16 = the story campaign; 18..23 would be the tutorials.
+  say "poking word A = $POKE_A, word B = $POKE_B"
+  poke w32 0x828F40C0 "$POKE_A"
+  poke w32 0x828F4814 "$POKE_B"
+else
+  say "control run — leaving the words untouched"
+fi

-# --- look at the menu --------------------------------------------------------
-sleep 2
-screenshot "$SHOTS/chal-02-mainmenu.png" >/dev/null
-say "screenshots in $SHOTS: chal-01-after-A.png chal-02-mainmenu.png"
-say "done — leaving the emulator running for follow-up"
-trap - EXIT
+# --- main menu -> EXTRAS -----------------------------------------------------
+# NEW GAME / LOAD GAME / TUTORIAL / OPTIONS / EXTRAS
+for _ in 1 2 3 4; do pad dpad down 0.06; sleep 0.35; done
+sleep 0.5
+pad tap A 0.15
+sleep 3
+shot 02-extras
+
+# --- EXTRAS -> MISSION SELECT ------------------------------------------------
+ALLOC_BEFORE=$(grep -c 'MmAllocatePhysicalMemoryEx: Allocation failed' "$LOG" 2>/dev/null | head -1)
+pad tap A 0.15
+sleep 4
+shot 03-missionselect
+ALLOC_AFTER=$(grep -c 'MmAllocatePhysicalMemoryEx: Allocation failed' "$LOG" 2>/dev/null | head -1)
+
+say "MmAllocatePhysicalMemoryEx failures: before=$ALLOC_BEFORE after=$ALLOC_AFTER"
+say "guest C++ exceptions: $(grep -c 'Guest attempted to throw a C++ exception' "$LOG" 2>/dev/null | head -1)"
+say "shots: $SHOTS/chal-$TAG-*.png"
+say "done — emulator left running"