re(challenge): MISSION SELECT renders the cleared-stage mask, and a control run

With word A = 2 the screen lists Stage01 SELECTABLE, carrying a High Score and a
Best Time, and Stage02-Stage08 GREYED OUT. One cleared stage, one selectable
entry, at the bit index that names it -- the mask's meaning is now visible on
screen rather than inferred from disassembly.

Two runs, identical navigation, fresh boot each:
  control  word A 0x00000002  -> MISSION SELECT opens normally
  poked    word A 0xFFFFFFFF  -> MmAllocatePhysicalMemoryEx fails on 128 MB,
                                 guest throws, Xenia shows "Disc Read Error"
So last commit's heap failure was caused by the poke, and by a careless one:
0xFFFFFFFF claims stages that do not exist (0, 17, 24-31 in word A). Poking only
real story ids (0x0001FFFE = stages 1-16) does not blow the heap. That the list
screen changes behaviour with the mask is itself evidence word A feeds it.

Getting a trustworthy control took three tries, and every failure produced a
plausible wrong answer rather than an error:
  1. the title-glyph oracle fired during the ATTRACT MOVIE, so A was pressed at
     nothing and the run "reported 0 failures" for a screen it never reached;
  2. Xvfb keeps the previous instance's framebuffer until the new one draws, so
     a screenshot seconds after launch showed the OLD run -- "MAIN MENU reached
     after 1s", against a process that no longer existed;
  3. a single-pixel "is NEW GAME white?" test matched a white LOADING FLASH.
Fixes, all in challenge_probe.sh: blank the root and refuse screen oracles for
the first 40 s; identify a screen by a PATTERN of sampled points (white text AND
the dark panel behind it), not one pixel; require two consecutive samples. The
new oracle was validated offline against all six saved screenshots and accepts
exactly the one real main menu.

General lesson worth the words: a navigation oracle that can only fail by
returning the wrong screen will happily produce a whole run of confident,
meaningless results.

docs/re/challenge-mission-gate.md

@@ -292,6 +292,24 @@ the running game and against a known progress state:
 
 Both words were then poked (`0xFFFFFFFF` / `0x3F`) and read back OK.
 
+**MISSION SELECT shows the mask directly** ([capture](captures/mission-select-stage01-only.png)).
+With word A = 2 the screen lists `Stage01` **selectable, with a High Score and Best
+Time**, and `Stage02`–`Stage08` **greyed out**. One cleared stage, one selectable
+entry, at the bit index that names it — the semantics are visible on screen, not
+inferred.
+
+**And the mask drives that screen.** Two runs, identical navigation, fresh boot each:
+
+| run | word A | MISSION SELECT |
+|---|---|---|
+| control | `0x00000002` (untouched) | opens; Stage01 selectable, rest greyed |
+| poked | `0xFFFFFFFF` | `MmAllocatePhysicalMemoryEx` fails on a 128 MB request, guest throws, Xenia shows "Disc Read Error" |
+
+So the earlier failure was **caused by the poke**, and by a careless one: `0xFFFFFFFF`
+claims stages that do not exist (`0`, `17`, `24`–`31` in word A). Poking only real
+story ids (`0x0001FFFE` = stages 1–16) does **not** blow the heap. That the list
+screen changes behaviour with the mask is itself confirmation that word A feeds it.
+
 **What the poke did not do (yet):** `EXTRAS` still shows only `MISSION SELECT /
 MOVIE THEATER / BACK` — no challenge entry — although the menu was built 26 s
 *after* the poke, so this is not staleness. Entering `MISSION SELECT` then failed,

tools/re-capture/challenge_probe.sh

@@ -6,13 +6,17 @@
 #   word A 0x828F40C0  bit = stage id, for ids < 24  (story 1-16, tutorial 18-23)
 #   word B 0x828F4814  bit = stage id - 24           (challenge 24-29)
 # so setting every bit should make all six challenge missions available without
-# playing the campaign. This boots, reaches the title, pokes both words, and
-# screenshots the menus so the result can be seen.
+# playing the campaign.
+#
+# POKE=1 (default) sets both words; POKE=0 runs the identical navigation without
+# touching them. Run it BOTH ways: the first poked run hit a Xenia heap failure on
+# the way into MISSION SELECT, and only the control says whether that failure has
+# anything to do with the poke.
 #
 # Runs as ONE blocking foreground call on purpose: setsid'd processes are reaped
 # at turn boundaries, so a session split across calls loses its emulator.
 #
-# Usage: challenge_probe.sh [boot_timeout_s]
+# Usage: [POKE=0|1] [TAG=name] challenge_probe.sh [boot_timeout_s]
 set -u
 export HOME=/sylph-home/re
 export DISPLAY=:99
@@ -23,65 +27,111 @@ HERE="$(cd "$(dirname "$0")" && pwd)"
 pad()  { python3 "$HERE/pad.py"   "$@"; }
 poke() { python3 "$HERE/gpoke.py" "$@"; }
 SHOTS="$HOME/shots"
-BOOT_TIMEOUT="${1:-420}"
+BOOT_TIMEOUT="${1:-400}"
+POKE="${POKE:-1}"
+POKE_A="${POKE_A:-0x0001FFFE}"   # stages 1-16 cleared
+POKE_B="${POKE_B:-0x0000003F}"   # challenge stages 24-29 cleared
+TAG="${TAG:-$([ "$POKE" = 1 ] && echo poked || echo control)}"
 mkdir -p "$SHOTS"
 
 say() { echo "[$(date +%H:%M:%S)] $*"; }
+shot() { screenshot "$SHOTS/chal-$TAG-$1.png" >/dev/null 2>&1; }
 
 # --- clean slate -------------------------------------------------------------
 pkill -9 -x xenia_canary 2>/dev/null
 sleep 1
 rm -f /dev/shm/xenia_* 2>/dev/null
 : > "$XENIA_PAD_FILE"
+LOG="$HOME/canary.stdout"
 
 # --- launch ------------------------------------------------------------------
-say "launching canary (lavapipe, file pad)"
+say "launching canary (lavapipe, file pad) — POKE=$POKE tag=$TAG"
 run-canary --audio --apu=sdl --log_mask=13 \
   --logged_profile_slot_0_xuid=E0300000EFBEA3D4 \
   --hid=file --pad_file="$XENIA_PAD_FILE" &
-CANARY_PID=$!
-trap 'pkill -9 -x xenia_canary 2>/dev/null' EXIT
 
-# --- wait for the title ------------------------------------------------------
-# Oracle: the green "PRESS (A) BUTTON" glyph at (625,618).
-say "waiting for the title (up to ${BOOT_TIMEOUT}s)"
-TITLE=0
-for _ in $(seq 1 "$BOOT_TIMEOUT"); do
-  if screenshot /tmp/title-probe.png >/dev/null 2>&1; then
-    read -r r g b < <(convert /tmp/title-probe.png -format \
-      "%[fx:int(255*p{625,618}.r)] %[fx:int(255*p{625,618}.g)] %[fx:int(255*p{625,618}.b)]" info: 2>/dev/null)
-    if [ -n "${g:-}" ] && [ "$g" -gt 130 ] && [ $((g - r)) -gt 45 ] && [ $((g - b)) -gt 45 ]; then
-      say "TITLE detected (rgb $r,$g,$b)"
-      TITLE=1
-      break
-    fi
-  fi
+# Xvfb keeps the LAST instance's framebuffer until the new one draws, so a
+# screenshot taken seconds after launch shows the PREVIOUS run's screen. That is
+# how a control run once reported "MAIN MENU reached after 1s" against a menu
+# belonging to a process that no longer existed. Blank the root, and refuse to
+# believe any screen oracle until the emulator has had time to draw its own.
+xsetroot -solid black 2>/dev/null || true
+LAUNCH_GRACE=40
+
+# --- reach the MAIN MENU, verifying instead of pressing blind ----------------
+# Two oracles, both sampled from real screenshots:
+#   title screen : green "PRESS (A) BUTTON" glyph at (625,618)
+#   main menu    : the "NEW GAME" text at (648,221) is pure white (254,254,254),
+#                  where the title has the yellow planet (208,189,88)
+# The first control run pressed A while the ATTRACT MOVIE happened to show a
+# greenish pixel at the title-glyph spot, then navigated a menu that was never
+# open and reported "0 failures" for a screen it never reached. Verify the menu.
+px() { convert /tmp/nav-probe.png -format \
+       "%[fx:int(255*p{$1}.r)] %[fx:int(255*p{$1}.g)] %[fx:int(255*p{$1}.b)]" info: 2>/dev/null; }
+
+# A screen is identified by a PATTERN of sampled points, never by one pixel. A
+# single "is (648,221) white?" test matched a white loading flash, and the run
+# then navigated a menu that was not on screen -- the same class of mistake as
+# trusting the stale framebuffer. Require the menu's contrast: white "NEW GAME"
+# text AND the dark blue panel behind it.
+at_menu() {
+  screenshot /tmp/nav-probe.png >/dev/null 2>&1 || return 1
+  read -r r g b < <(px "648,221")     # NEW GAME text: white
+  [ -n "${r:-}" ] || return 1
+  [ "$r" -gt 230 ] && [ "$g" -gt 230 ] && [ "$b" -gt 230 ] || return 1
+  read -r r2 g2 b2 < <(px "560,300")  # panel left of LOAD GAME: dark blue
+  [ -n "${r2:-}" ] || return 1
+  [ "$r2" -lt 120 ] && [ "$b2" -gt "$r2" ]
+}
+at_title() {
+  screenshot /tmp/nav-probe.png >/dev/null 2>&1 || return 1
+  read -r r g b < <(px "625,618")
+  [ -n "${g:-}" ] && [ "$g" -gt 130 ] && [ $((g - r)) -gt 45 ] && [ $((g - b)) -gt 45 ]
+}
+
+say "waiting for the main menu (up to ${BOOT_TIMEOUT}s)"
+MENU=0
+for i in $(seq 1 "$BOOT_TIMEOUT"); do
+  if [ "$i" -lt "$LAUNCH_GRACE" ]; then sleep 1; continue; fi
+  if at_menu && sleep 1 && at_menu; then say "MAIN MENU reached after ${i}s"; MENU=1; break; fi
+  if at_title; then say "  title visible — tapping A"; pad tap A 0.25; sleep 2; fi
   sleep 1
 done
-[ "$TITLE" = 1 ] || { say "TIMEOUT: no title"; screenshot "$SHOTS/chal-00-timeout.png"; exit 1; }
+[ "$MENU" = 1 ] || { say "TIMEOUT: never reached the main menu"; shot 00-timeout; exit 1; }
+shot 01-mainmenu
 
-# --- prove the file pad works before trusting anything else -------------------
-say "pad check: tapping A at the title"
-pad tap A 0.25
-sleep 3
-screenshot "$SHOTS/chal-01-after-A.png" >/dev/null
-say "file-pad log lines so far:"
-grep -c 'file-pad' "$HOME/canary.stdout" 2>/dev/null || true
-grep 'file-pad' "$HOME/canary.stdout" 2>/dev/null | tail -3
-
-# --- read the gate words BEFORE poking ---------------------------------------
-say "gate words before poke:"
+say "gate words:"
 poke r32 0x828F40C0 1
 poke r32 0x828F4814 1
 
-# --- poke --------------------------------------------------------------------
-say "poking word A = 0xFFFFFFFF, word B = 0x3F"
-poke w32 0x828F40C0 0xFFFFFFFF
-poke w32 0x828F4814 0x0000003F
+if [ "$POKE" = 1 ]; then
+  # Default to REAL stage ids only. 0xFFFFFFFF claims stages that do not exist
+  # (0, 17, and 24-31 in word A), and that run blew the guest heap:
+  # MmAllocatePhysicalMemoryEx could not satisfy 128 MB and the guest threw.
+  # Word A bits 1..16 = the story campaign; 18..23 would be the tutorials.
+  say "poking word A = $POKE_A, word B = $POKE_B"
+  poke w32 0x828F40C0 "$POKE_A"
+  poke w32 0x828F4814 "$POKE_B"
+else
+  say "control run — leaving the words untouched"
+fi
 
-# --- look at the menu --------------------------------------------------------
-sleep 2
-screenshot "$SHOTS/chal-02-mainmenu.png" >/dev/null
-say "screenshots in $SHOTS: chal-01-after-A.png chal-02-mainmenu.png"
-say "done — leaving the emulator running for follow-up"
-trap - EXIT
+# --- main menu -> EXTRAS -----------------------------------------------------
+# NEW GAME / LOAD GAME / TUTORIAL / OPTIONS / EXTRAS
+for _ in 1 2 3 4; do pad dpad down 0.06; sleep 0.35; done
+sleep 0.5
+pad tap A 0.15
+sleep 3
+shot 02-extras
+
+# --- EXTRAS -> MISSION SELECT ------------------------------------------------
+ALLOC_BEFORE=$(grep -c 'MmAllocatePhysicalMemoryEx: Allocation failed' "$LOG" 2>/dev/null | head -1)
+pad tap A 0.15
+sleep 4
+shot 03-missionselect
+ALLOC_AFTER=$(grep -c 'MmAllocatePhysicalMemoryEx: Allocation failed' "$LOG" 2>/dev/null | head -1)
+
+say "MmAllocatePhysicalMemoryEx failures: before=$ALLOC_BEFORE after=$ALLOC_AFTER"
+say "guest C++ exceptions: $(grep -c 'Guest attempted to throw a C++ exception' "$LOG" 2>/dev/null | head -1)"
+say "shots: $SHOTS/chal-$TAG-*.png"
+say "done — emulator left running"