re(challenge): the cleared-stage mask is CONFIRMED on the running game

Booted the title and read the two gate words live:

    0x828F40C0 = 0x00000002     word A
    0x828F4814 = 0x00000000     word B

Word A = 2 = bit 1. The profile's save is Stage 02 "At Standby" -- stage 01
cleared -- so the mask is exactly one bit, at the index of the one cleared
stage, 1-BASED. Reproduced across two cold boots. That confirms against a known
progress state, on the real game:

  - the singleton is the static object at 0x828F4070, as derived statically;
  - word A is a cleared-stage bitmask (not achievements, not a stage number);
  - bit index = stage id, 1-based, so TimeAttack's REQUIREMENT 16 means "clear
    stage 16" -- the last story mission;
  - word B is the challenge half and is 0 on a story-only profile.

New tools: gpoke.py (live guest-memory WRITE, companion to gmem.py, prints
before/after for every word), pad.py (drives the new --hid=file pad; replaces
vgamepad, which leaked to the host through /dev/uinput), challenge_probe.sh
(one blocking session: boot, wait for title, drive in, poke, screenshot).

Poking both words did NOT surface a challenge entry in EXTRAS -- and that menu
was built 26 s after the poke, so it is not staleness. Entering MISSION SELECT
then failed, but the log names the real cause and it is not the gate:
MmAllocatePhysicalMemoryEx could not satisfy a 128 MB request (parent free
30633/131072 pages), the guest threw a C++ exception, and Xenia surfaced its
generic "Disc Read Error". It is preceded by "BaseHeap::Release failed because
address is not a region start" -- a failed release leaking the range. Recorded
as an emulator heap problem, with the control run (same navigation, no poke)
named as the next step.

docs/re/challenge-mission-gate.md

@@ -1,12 +1,14 @@
 # Challenge / EX missions — the stage set, the GamePart graph, and the kind field
 
 **Status:** ✅ for the static structure (stage set, GamePart ids, the config-section
-switch) and for the **unlock mechanism** (a bit test against a *cleared-stage* mask,
+switch) and for the **unlock mechanism** — a bit test against a *cleared-stage* mask,
-whose sole writer is `GamePart_StageClear`, in a ~1 880-byte progress record that is
+now **read live off the running game and matching the profile's progress exactly**
-**not** the savegame); 🟡 for the crash mechanism, for word B's writer, and for which
+(§5.6). 🟡 for the stage-field crash mechanism, for word B's writer, and for which
-mission consumes which bit.
+mission consumes which bit; ❔ how the challenge menu is actually entered.
-**Method:** static only — `.pe` string/pointer analysis + DuckDB disassembly + disc
+**Method:** static analysis (`.pe` strings/pointers + DuckDB disassembly + disc
-records. No emulator run, no gamepad input.
+records), then confirmed on the running title via the container-safe
+[`--hid=file` pad](../../../xenia-canary-native/src/xenia/hid/file/file_input_driver.h)
+and a live guest-memory read.
 **Evidence:** [`captures/challenge-map.txt`](captures/challenge-map.txt),
 [`captures/challenge-screen-config.txt`](captures/challenge-screen-config.txt),
 [`structures/achievements.md`](structures/achievements.md);
@@ -268,7 +270,40 @@ into this record.
 operational hope this section previously carried; the earlier savegame-editing win
 does not extend here.
 
-### 5.6 What *would* work — static addresses for a live write
+### 5.6 CONFIRMED ON THE RUNNING GAME ✅
+
+Read live from a booted title (`tools/re-capture/gpoke.py r32`):
+
+```
+0x828F40C0 = 0x00000002      word A
+0x828F4814 = 0x00000000      word B
+```
+
+**Word A = 2 = bit 1 set.** The profile's save is *Stage 02, "At Standby"* — i.e.
+**stage 01 cleared** — so the mask is exactly one bit, at the index of the one
+cleared stage, **1-based**. Reproduced on two separate cold boots. That confirms, on
+the running game and against a known progress state:
+
+- the singleton really is the static object at `0x828F4070`;
+- word A is a **cleared-stage bitmask**, not achievements, not a stage number;
+- bit index = **stage id, 1-based** — so `TimeAttack`'s `REQUIREMENT 16` is
+  "clear stage 16", the last story mission;
+- word B is the challenge half and is `0` on a story-only profile, as expected.
+
+Both words were then poked (`0xFFFFFFFF` / `0x3F`) and read back OK.
+
+**What the poke did not do (yet):** `EXTRAS` still shows only `MISSION SELECT /
+MOVIE THEATER / BACK` — no challenge entry — although the menu was built 26 s
+*after* the poke, so this is not staleness. Entering `MISSION SELECT` then failed,
+and the log gives the real reason: **`MmAllocatePhysicalMemoryEx` could not satisfy
+a 128 MB request** (`parent free 30633/131072 pages`), the guest threw a C++
+exception, and Xenia surfaced it as its generic *"Disc Read Error"* dialog. It is
+preceded by `BaseHeap::Release failed because address is not a region start`, a
+failed release that leaks the range. So that is an emulator/heap problem on the way
+into the screen, **not** evidence about the gate. Open: repeat without the poke to
+see whether `MISSION SELECT` fails the same way regardless.
+
+### 5.7 What *would* work — static addresses for a live write
 
 The singleton is a **static object at `0x828F4070`** (`0x8216F650`:
 `addis 0x828F` + `addi …, 16496`), with the holder at `0x828F48B0` pointing at it. So

tools/re-capture/challenge_probe.sh

@@ -0,0 +1,87 @@
+#!/usr/bin/env bash
+# Probe the challenge-mission gate on the running game.
+#
+# The gate (docs/re/challenge-mission-gate.md): GamePart_ChallengeMission tests a
+# CLEARED-STAGE bitmask on a static singleton at guest 0x828F4070 —
+#   word A 0x828F40C0  bit = stage id, for ids < 24  (story 1-16, tutorial 18-23)
+#   word B 0x828F4814  bit = stage id - 24           (challenge 24-29)
+# so setting every bit should make all six challenge missions available without
+# playing the campaign. This boots, reaches the title, pokes both words, and
+# screenshots the menus so the result can be seen.
+#
+# Runs as ONE blocking foreground call on purpose: setsid'd processes are reaped
+# at turn boundaries, so a session split across calls loses its emulator.
+#
+# Usage: challenge_probe.sh [boot_timeout_s]
+set -u
+export HOME=/sylph-home/re
+export DISPLAY=:99
+export SDL_AUDIODRIVER=dummy
+export XENIA_PAD_FILE=/tmp/xenia_pad.txt
+
+HERE="$(cd "$(dirname "$0")" && pwd)"
+pad()  { python3 "$HERE/pad.py"   "$@"; }
+poke() { python3 "$HERE/gpoke.py" "$@"; }
+SHOTS="$HOME/shots"
+BOOT_TIMEOUT="${1:-420}"
+mkdir -p "$SHOTS"
+
+say() { echo "[$(date +%H:%M:%S)] $*"; }
+
+# --- clean slate -------------------------------------------------------------
+pkill -9 -x xenia_canary 2>/dev/null
+sleep 1
+rm -f /dev/shm/xenia_* 2>/dev/null
+: > "$XENIA_PAD_FILE"
+
+# --- launch ------------------------------------------------------------------
+say "launching canary (lavapipe, file pad)"
+run-canary --audio --apu=sdl --log_mask=13 \
+  --logged_profile_slot_0_xuid=E0300000EFBEA3D4 \
+  --hid=file --pad_file="$XENIA_PAD_FILE" &
+CANARY_PID=$!
+trap 'pkill -9 -x xenia_canary 2>/dev/null' EXIT
+
+# --- wait for the title ------------------------------------------------------
+# Oracle: the green "PRESS (A) BUTTON" glyph at (625,618).
+say "waiting for the title (up to ${BOOT_TIMEOUT}s)"
+TITLE=0
+for _ in $(seq 1 "$BOOT_TIMEOUT"); do
+  if screenshot /tmp/title-probe.png >/dev/null 2>&1; then
+    read -r r g b < <(convert /tmp/title-probe.png -format \
+      "%[fx:int(255*p{625,618}.r)] %[fx:int(255*p{625,618}.g)] %[fx:int(255*p{625,618}.b)]" info: 2>/dev/null)
+    if [ -n "${g:-}" ] && [ "$g" -gt 130 ] && [ $((g - r)) -gt 45 ] && [ $((g - b)) -gt 45 ]; then
+      say "TITLE detected (rgb $r,$g,$b)"
+      TITLE=1
+      break
+    fi
+  fi
+  sleep 1
+done
+[ "$TITLE" = 1 ] || { say "TIMEOUT: no title"; screenshot "$SHOTS/chal-00-timeout.png"; exit 1; }
+
+# --- prove the file pad works before trusting anything else -------------------
+say "pad check: tapping A at the title"
+pad tap A 0.25
+sleep 3
+screenshot "$SHOTS/chal-01-after-A.png" >/dev/null
+say "file-pad log lines so far:"
+grep -c 'file-pad' "$HOME/canary.stdout" 2>/dev/null || true
+grep 'file-pad' "$HOME/canary.stdout" 2>/dev/null | tail -3
+
+# --- read the gate words BEFORE poking ---------------------------------------
+say "gate words before poke:"
+poke r32 0x828F40C0 1
+poke r32 0x828F4814 1
+
+# --- poke --------------------------------------------------------------------
+say "poking word A = 0xFFFFFFFF, word B = 0x3F"
+poke w32 0x828F40C0 0xFFFFFFFF
+poke w32 0x828F4814 0x0000003F
+
+# --- look at the menu --------------------------------------------------------
+sleep 2
+screenshot "$SHOTS/chal-02-mainmenu.png" >/dev/null
+say "screenshots in $SHOTS: chal-01-after-A.png chal-02-mainmenu.png"
+say "done — leaving the emulator running for follow-up"
+trap - EXIT

tools/re-capture/gpoke.py

@@ -0,0 +1,76 @@
+#!/usr/bin/env python3
+"""WRITE to the live guest memory of a running Xenia Canary.
+
+The read-side companion is `gmem.py`, and this shares its guest-VA → file-offset
+table. Canary backs the whole guest address space with one shared-memory file
+(`/dev/shm/xenia_memory_*`), so a write here lands in the running guest with no
+debugger and no pause.
+
+    gpoke.py w32 <va> <value> [...]     write big-endian u32s at consecutive VAs
+    gpoke.py r32 <va> [n]               read back n big-endian u32s (verify)
+
+Values and addresses accept `0x` form. Every write prints the before/after word,
+because a poke you cannot see is a poke you cannot trust.
+
+⚠️  This mutates a running game. It is a research tool: there is no undo, and a
+wrong address will corrupt whatever it lands on. Read back before believing.
+"""
+
+import os
+import struct
+import sys
+
+from gmem import MAP, va_to_off  # noqa: F401  (MAP re-exported for callers)
+
+
+def shm_path():
+    cands = [f"/dev/shm/{n}" for n in os.listdir("/dev/shm") if n.startswith("xenia_memory")]
+    if not cands:
+        raise SystemExit("no /dev/shm/xenia_memory_* — is Canary running?")
+    if len(cands) > 1:
+        raise SystemExit(f"several guest images, refusing to guess: {cands}")
+    return cands[0]
+
+
+def read32(f, va):
+    f.seek(va_to_off(va))
+    return struct.unpack(">I", f.read(4))[0]
+
+
+def main():
+    if len(sys.argv) < 3:
+        print(__doc__)
+        return 1
+    cmd = sys.argv[1]
+    path = shm_path()
+    if cmd == "r32":
+        va = int(sys.argv[2], 0)
+        n = int(sys.argv[3], 0) if len(sys.argv) > 3 else 1
+        with open(path, "rb") as f:
+            for i in range(n):
+                a = va + 4 * i
+                print(f"  {a:#010x} = {read32(f, a):#010x}")
+        return 0
+    if cmd == "w32":
+        va = int(sys.argv[2], 0)
+        vals = [int(v, 0) for v in sys.argv[3:]]
+        if not vals:
+            print("nothing to write")
+            return 1
+        with open(path, "r+b") as f:
+            for i, v in enumerate(vals):
+                a = va + 4 * i
+                before = read32(f, a)
+                f.seek(va_to_off(a))
+                f.write(struct.pack(">I", v))
+                f.flush()
+                after = read32(f, a)
+                ok = "OK" if after == v else "!! MISMATCH"
+                print(f"  {a:#010x}: {before:#010x} -> {after:#010x}  {ok}")
+        return 0
+    print(f"unknown command {cmd!r}")
+    return 1
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

tools/re-capture/pad.py

@@ -0,0 +1,70 @@
+#!/usr/bin/env python3
+"""Drive Xenia Canary's `--hid=file` pad — the container-safe controller.
+
+Replaces the old `vgamepad` path, which created its device through `/dev/uinput`.
+Input devices are not namespaced, so that device registered with the HOST's input
+stack and every scripted press leaked to the user's desktop. This one writes a
+text file the emulator polls; nothing leaves the container.
+
+    pad.py set   "press=A"          set the pad state and leave it held
+    pad.py clear                    release everything
+    pad.py tap   A [secs]           press, hold `secs` (default 0.10), release
+    pad.py dpad  down [secs]        one menu step (default 0.06 — longer
+                                    auto-repeats and overshoots)
+    pad.py hold  "lt=255" secs      hold an arbitrary state for `secs`
+
+Buttons: UP DOWN LEFT RIGHT START BACK LS RS LB RB A B X Y.
+Path from $XENIA_PAD_FILE, default /tmp/xenia_pad.txt (matches --pad_file).
+
+Menu conventions in this game: A = OK, B = Back, Y = Gallery/extra, X = Delete.
+"""
+
+import os
+import sys
+import time
+
+PAD = os.environ.get("XENIA_PAD_FILE", "/tmp/xenia_pad.txt")
+
+
+def write(state: str):
+    # The driver re-parses on any (mtime-ns, size) change, so a plain rewrite is
+    # enough — but write through a temp + rename so a poll can never observe a
+    # half-written file.
+    tmp = PAD + ".tmp"
+    with open(tmp, "w") as f:
+        f.write(state)
+    os.replace(tmp, PAD)
+
+
+def main():
+    a = sys.argv[1:]
+    if not a:
+        print(__doc__)
+        return 1
+    cmd = a[0]
+    if cmd == "set":
+        write(a[1])
+    elif cmd == "clear":
+        write("")
+    elif cmd == "tap":
+        secs = float(a[2]) if len(a) > 2 else 0.10
+        write(f"press={a[1]}")
+        time.sleep(secs)
+        write("")
+    elif cmd == "dpad":
+        secs = float(a[2]) if len(a) > 2 else 0.06
+        write(f"press={a[1].upper()}")
+        time.sleep(secs)
+        write("")
+    elif cmd == "hold":
+        write(a[1])
+        time.sleep(float(a[2]))
+        write("")
+    else:
+        print(f"unknown command {cmd!r}")
+        return 1
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

tools/re-capture/roll_axis.py

@@ -0,0 +1,80 @@
+#!/usr/bin/env python3
+"""Roll rate measured ABOUT THE FORWARD AXIS, so pitch cannot leak into it.
+
+The previous attempt watched a non-forward row of the rotation matrix and got
+numbers within a few per cent of the pitch run — because pitch moves that row as
+much as roll does. The fix is to measure the rotation *in the plane perpendicular
+to forward*: express the new up-vector in the OLD (up, right) basis and take
+`atan2(u_new·w_old, u_new·u_old)`. Any component along forward — which is what
+pitch produces — is dropped by construction.
+
+Each phase is bracketed by HUD screenshots so the mission clock converts wall
+seconds to game seconds within this run (the ratio has measured 1.260, 1.311 and
+1.383 in three flights, so it cannot be assumed).
+
+Usage: roll_axis.py <out.csv> [dwell_s]
+"""
+import json, math, os, struct, subprocess, sys, time
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+import speed_law
+
+def pad(*a):
+    subprocess.run(["vgamepad", *a], capture_output=True)
+
+def shot(n):
+    subprocess.run(["screenshot", f"/sylph-home/re/shots/{n}.png"], capture_output=True)
+
+def rows_at(fd, off, cfg):
+    base = off + cfg["rot_delta"]
+    out = []
+    for r in range(3):
+        v = struct.unpack(">3f", os.pread(fd, 12, base + r * cfg["rot_stride"]))
+        n = math.sqrt(sum(c * c for c in v)) or 1.0
+        out.append(tuple(c / n for c in v))
+    return out
+
+def dot(a, b):
+    return sum(a[i] * b[i] for i in range(3))
+
+def main():
+    out_csv = sys.argv[1]
+    dwell = float(sys.argv[2]) if len(sys.argv) > 2 else 8.0
+    cfg = json.load(open("/tmp/nav-live.json"))
+    w, off, nm = speed_law.find_player()
+    if not w:
+        sys.exit("player entity not found after retries")
+    print(f"# locked on {nm}")
+    f_i = cfg.get("fwd_row", 0)
+    u_i, w_i = [r for r in (0, 1, 2) if r != f_i]
+    rows = []
+    for label, (trig, tv) in (("slow", ("LT", 1.0)), ("fast", ("RT", 1.0))):
+        pad("trig", "RT", "0.0"); pad("trig", "LT", "0.0"); pad("axis", "LX", "0.0")
+        pad("trig", trig, str(tv))
+        time.sleep(5.0)
+        shot(f"rollax_{label}_a")
+        pad("axis", "LX", "1.0")
+        t0, prev, swept = time.time(), rows_at(w.fd, off, cfg), 0.0
+        seq = []
+        while time.time() - t0 < dwell:
+            time.sleep(0.05)
+            cur = rows_at(w.fd, off, cfg)
+            # roll = rotation of `up` within the OLD (up, right) plane
+            d = math.degrees(math.atan2(dot(cur[u_i], prev[w_i]), dot(cur[u_i], prev[u_i])))
+            swept += abs(d)
+            seq.append((round(time.time() - t0, 3), round(d, 4)))
+            prev = cur
+        pad("axis", "LX", "0.0")
+        wall = seq[-1][0]
+        shot(f"rollax_{label}_b")
+        rows += [(label, *s) for s in seq]
+        print(f"# {label:<5} wall {wall:5.2f}s   roll swept {swept:7.1f}deg   "
+              f"rate {swept / wall:6.1f} deg/wall-s")
+    pad("trig", "RT", "0.0"); pad("trig", "LT", "0.0"); pad("reset")
+    with open(out_csv, "w") as f:
+        f.write("phase,t,droll_deg\n")
+        for r in rows:
+            f.write(",".join(str(x) for x in r) + "\n")
+    print("# read HUD TIME off rollax_slow_a/b and rollax_fast_a/b for this run's clock ratio")
+
+if __name__ == "__main__":
+    main()