re(challenge): the cleared-stage mask is CONFIRMED on the running game

Booted the title and read the two gate words live:

    0x828F40C0 = 0x00000002     word A
    0x828F4814 = 0x00000000     word B

Word A = 2 = bit 1. The profile's save is Stage 02 "At Standby" -- stage 01
cleared -- so the mask is exactly one bit, at the index of the one cleared
stage, 1-BASED. Reproduced across two cold boots. That confirms against a known
progress state, on the real game:

  - the singleton is the static object at 0x828F4070, as derived statically;
  - word A is a cleared-stage bitmask (not achievements, not a stage number);
  - bit index = stage id, 1-based, so TimeAttack's REQUIREMENT 16 means "clear
    stage 16" -- the last story mission;
  - word B is the challenge half and is 0 on a story-only profile.

New tools: gpoke.py (live guest-memory WRITE, companion to gmem.py, prints
before/after for every word), pad.py (drives the new --hid=file pad; replaces
vgamepad, which leaked to the host through /dev/uinput), challenge_probe.sh
(one blocking session: boot, wait for title, drive in, poke, screenshot).

Poking both words did NOT surface a challenge entry in EXTRAS -- and that menu
was built 26 s after the poke, so it is not staleness. Entering MISSION SELECT
then failed, but the log names the real cause and it is not the gate:
MmAllocatePhysicalMemoryEx could not satisfy a 128 MB request (parent free
30633/131072 pages), the guest threw a C++ exception, and Xenia surfaced its
generic "Disc Read Error". It is preceded by "BaseHeap::Release failed because
address is not a region start" -- a failed release leaking the range. Recorded
as an emulator heap problem, with the control run (same navigation, no poke)
named as the next step.
This commit is contained in:
2026-08-13 20:28:19 +00:00
parent f490fecefb
commit 023bb71cfd
7 changed files with 355 additions and 7 deletions

View File

@@ -0,0 +1,87 @@
#!/usr/bin/env bash
# Probe the challenge-mission gate on the running game.
#
# The gate (docs/re/challenge-mission-gate.md): GamePart_ChallengeMission tests a
# CLEARED-STAGE bitmask on a static singleton at guest 0x828F4070 —
# word A 0x828F40C0 bit = stage id, for ids < 24 (story 1-16, tutorial 18-23)
# word B 0x828F4814 bit = stage id - 24 (challenge 24-29)
# so setting every bit should make all six challenge missions available without
# playing the campaign. This boots, reaches the title, pokes both words, and
# screenshots the menus so the result can be seen.
#
# Runs as ONE blocking foreground call on purpose: setsid'd processes are reaped
# at turn boundaries, so a session split across calls loses its emulator.
#
# Usage: challenge_probe.sh [boot_timeout_s]
set -u
export HOME=/sylph-home/re
export DISPLAY=:99
export SDL_AUDIODRIVER=dummy
export XENIA_PAD_FILE=/tmp/xenia_pad.txt
HERE="$(cd "$(dirname "$0")" && pwd)"
pad() { python3 "$HERE/pad.py" "$@"; }
poke() { python3 "$HERE/gpoke.py" "$@"; }
SHOTS="$HOME/shots"
BOOT_TIMEOUT="${1:-420}"
mkdir -p "$SHOTS"
say() { echo "[$(date +%H:%M:%S)] $*"; }
# --- clean slate -------------------------------------------------------------
pkill -9 -x xenia_canary 2>/dev/null
sleep 1
rm -f /dev/shm/xenia_* 2>/dev/null
: > "$XENIA_PAD_FILE"
# --- launch ------------------------------------------------------------------
say "launching canary (lavapipe, file pad)"
run-canary --audio --apu=sdl --log_mask=13 \
--logged_profile_slot_0_xuid=E0300000EFBEA3D4 \
--hid=file --pad_file="$XENIA_PAD_FILE" &
CANARY_PID=$!
trap 'pkill -9 -x xenia_canary 2>/dev/null' EXIT
# --- wait for the title ------------------------------------------------------
# Oracle: the green "PRESS (A) BUTTON" glyph at (625,618).
say "waiting for the title (up to ${BOOT_TIMEOUT}s)"
TITLE=0
for _ in $(seq 1 "$BOOT_TIMEOUT"); do
if screenshot /tmp/title-probe.png >/dev/null 2>&1; then
read -r r g b < <(convert /tmp/title-probe.png -format \
"%[fx:int(255*p{625,618}.r)] %[fx:int(255*p{625,618}.g)] %[fx:int(255*p{625,618}.b)]" info: 2>/dev/null)
if [ -n "${g:-}" ] && [ "$g" -gt 130 ] && [ $((g - r)) -gt 45 ] && [ $((g - b)) -gt 45 ]; then
say "TITLE detected (rgb $r,$g,$b)"
TITLE=1
break
fi
fi
sleep 1
done
[ "$TITLE" = 1 ] || { say "TIMEOUT: no title"; screenshot "$SHOTS/chal-00-timeout.png"; exit 1; }
# --- prove the file pad works before trusting anything else -------------------
say "pad check: tapping A at the title"
pad tap A 0.25
sleep 3
screenshot "$SHOTS/chal-01-after-A.png" >/dev/null
say "file-pad log lines so far:"
grep -c 'file-pad' "$HOME/canary.stdout" 2>/dev/null || true
grep 'file-pad' "$HOME/canary.stdout" 2>/dev/null | tail -3
# --- read the gate words BEFORE poking ---------------------------------------
say "gate words before poke:"
poke r32 0x828F40C0 1
poke r32 0x828F4814 1
# --- poke --------------------------------------------------------------------
say "poking word A = 0xFFFFFFFF, word B = 0x3F"
poke w32 0x828F40C0 0xFFFFFFFF
poke w32 0x828F4814 0x0000003F
# --- look at the menu --------------------------------------------------------
sleep 2
screenshot "$SHOTS/chal-02-mainmenu.png" >/dev/null
say "screenshots in $SHOTS: chal-01-after-A.png chal-02-mainmenu.png"
say "done — leaving the emulator running for follow-up"
trap - EXIT

76
tools/re-capture/gpoke.py Executable file
View File

@@ -0,0 +1,76 @@
#!/usr/bin/env python3
"""WRITE to the live guest memory of a running Xenia Canary.
The read-side companion is `gmem.py`, and this shares its guest-VA → file-offset
table. Canary backs the whole guest address space with one shared-memory file
(`/dev/shm/xenia_memory_*`), so a write here lands in the running guest with no
debugger and no pause.
gpoke.py w32 <va> <value> [...] write big-endian u32s at consecutive VAs
gpoke.py r32 <va> [n] read back n big-endian u32s (verify)
Values and addresses accept `0x` form. Every write prints the before/after word,
because a poke you cannot see is a poke you cannot trust.
⚠️ This mutates a running game. It is a research tool: there is no undo, and a
wrong address will corrupt whatever it lands on. Read back before believing.
"""
import os
import struct
import sys
from gmem import MAP, va_to_off # noqa: F401 (MAP re-exported for callers)
def shm_path():
cands = [f"/dev/shm/{n}" for n in os.listdir("/dev/shm") if n.startswith("xenia_memory")]
if not cands:
raise SystemExit("no /dev/shm/xenia_memory_* — is Canary running?")
if len(cands) > 1:
raise SystemExit(f"several guest images, refusing to guess: {cands}")
return cands[0]
def read32(f, va):
f.seek(va_to_off(va))
return struct.unpack(">I", f.read(4))[0]
def main():
if len(sys.argv) < 3:
print(__doc__)
return 1
cmd = sys.argv[1]
path = shm_path()
if cmd == "r32":
va = int(sys.argv[2], 0)
n = int(sys.argv[3], 0) if len(sys.argv) > 3 else 1
with open(path, "rb") as f:
for i in range(n):
a = va + 4 * i
print(f" {a:#010x} = {read32(f, a):#010x}")
return 0
if cmd == "w32":
va = int(sys.argv[2], 0)
vals = [int(v, 0) for v in sys.argv[3:]]
if not vals:
print("nothing to write")
return 1
with open(path, "r+b") as f:
for i, v in enumerate(vals):
a = va + 4 * i
before = read32(f, a)
f.seek(va_to_off(a))
f.write(struct.pack(">I", v))
f.flush()
after = read32(f, a)
ok = "OK" if after == v else "!! MISMATCH"
print(f" {a:#010x}: {before:#010x} -> {after:#010x} {ok}")
return 0
print(f"unknown command {cmd!r}")
return 1
if __name__ == "__main__":
raise SystemExit(main())

70
tools/re-capture/pad.py Executable file
View File

@@ -0,0 +1,70 @@
#!/usr/bin/env python3
"""Drive Xenia Canary's `--hid=file` pad — the container-safe controller.
Replaces the old `vgamepad` path, which created its device through `/dev/uinput`.
Input devices are not namespaced, so that device registered with the HOST's input
stack and every scripted press leaked to the user's desktop. This one writes a
text file the emulator polls; nothing leaves the container.
pad.py set "press=A" set the pad state and leave it held
pad.py clear release everything
pad.py tap A [secs] press, hold `secs` (default 0.10), release
pad.py dpad down [secs] one menu step (default 0.06 — longer
auto-repeats and overshoots)
pad.py hold "lt=255" secs hold an arbitrary state for `secs`
Buttons: UP DOWN LEFT RIGHT START BACK LS RS LB RB A B X Y.
Path from $XENIA_PAD_FILE, default /tmp/xenia_pad.txt (matches --pad_file).
Menu conventions in this game: A = OK, B = Back, Y = Gallery/extra, X = Delete.
"""
import os
import sys
import time
PAD = os.environ.get("XENIA_PAD_FILE", "/tmp/xenia_pad.txt")
def write(state: str):
# The driver re-parses on any (mtime-ns, size) change, so a plain rewrite is
# enough — but write through a temp + rename so a poll can never observe a
# half-written file.
tmp = PAD + ".tmp"
with open(tmp, "w") as f:
f.write(state)
os.replace(tmp, PAD)
def main():
a = sys.argv[1:]
if not a:
print(__doc__)
return 1
cmd = a[0]
if cmd == "set":
write(a[1])
elif cmd == "clear":
write("")
elif cmd == "tap":
secs = float(a[2]) if len(a) > 2 else 0.10
write(f"press={a[1]}")
time.sleep(secs)
write("")
elif cmd == "dpad":
secs = float(a[2]) if len(a) > 2 else 0.06
write(f"press={a[1].upper()}")
time.sleep(secs)
write("")
elif cmd == "hold":
write(a[1])
time.sleep(float(a[2]))
write("")
else:
print(f"unknown command {cmd!r}")
return 1
return 0
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -0,0 +1,80 @@
#!/usr/bin/env python3
"""Roll rate measured ABOUT THE FORWARD AXIS, so pitch cannot leak into it.
The previous attempt watched a non-forward row of the rotation matrix and got
numbers within a few per cent of the pitch run — because pitch moves that row as
much as roll does. The fix is to measure the rotation *in the plane perpendicular
to forward*: express the new up-vector in the OLD (up, right) basis and take
`atan2(u_new·w_old, u_new·u_old)`. Any component along forward — which is what
pitch produces — is dropped by construction.
Each phase is bracketed by HUD screenshots so the mission clock converts wall
seconds to game seconds within this run (the ratio has measured 1.260, 1.311 and
1.383 in three flights, so it cannot be assumed).
Usage: roll_axis.py <out.csv> [dwell_s]
"""
import json, math, os, struct, subprocess, sys, time
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import speed_law
def pad(*a):
subprocess.run(["vgamepad", *a], capture_output=True)
def shot(n):
subprocess.run(["screenshot", f"/sylph-home/re/shots/{n}.png"], capture_output=True)
def rows_at(fd, off, cfg):
base = off + cfg["rot_delta"]
out = []
for r in range(3):
v = struct.unpack(">3f", os.pread(fd, 12, base + r * cfg["rot_stride"]))
n = math.sqrt(sum(c * c for c in v)) or 1.0
out.append(tuple(c / n for c in v))
return out
def dot(a, b):
return sum(a[i] * b[i] for i in range(3))
def main():
out_csv = sys.argv[1]
dwell = float(sys.argv[2]) if len(sys.argv) > 2 else 8.0
cfg = json.load(open("/tmp/nav-live.json"))
w, off, nm = speed_law.find_player()
if not w:
sys.exit("player entity not found after retries")
print(f"# locked on {nm}")
f_i = cfg.get("fwd_row", 0)
u_i, w_i = [r for r in (0, 1, 2) if r != f_i]
rows = []
for label, (trig, tv) in (("slow", ("LT", 1.0)), ("fast", ("RT", 1.0))):
pad("trig", "RT", "0.0"); pad("trig", "LT", "0.0"); pad("axis", "LX", "0.0")
pad("trig", trig, str(tv))
time.sleep(5.0)
shot(f"rollax_{label}_a")
pad("axis", "LX", "1.0")
t0, prev, swept = time.time(), rows_at(w.fd, off, cfg), 0.0
seq = []
while time.time() - t0 < dwell:
time.sleep(0.05)
cur = rows_at(w.fd, off, cfg)
# roll = rotation of `up` within the OLD (up, right) plane
d = math.degrees(math.atan2(dot(cur[u_i], prev[w_i]), dot(cur[u_i], prev[u_i])))
swept += abs(d)
seq.append((round(time.time() - t0, 3), round(d, 4)))
prev = cur
pad("axis", "LX", "0.0")
wall = seq[-1][0]
shot(f"rollax_{label}_b")
rows += [(label, *s) for s in seq]
print(f"# {label:<5} wall {wall:5.2f}s roll swept {swept:7.1f}deg "
f"rate {swept / wall:6.1f} deg/wall-s")
pad("trig", "RT", "0.0"); pad("trig", "LT", "0.0"); pad("reset")
with open(out_csv, "w") as f:
f.write("phase,t,droll_deg\n")
for r in rows:
f.write(",".join(str(x) for x in r) + "\n")
print("# read HUD TIME off rollax_slow_a/b and rollax_fast_a/b for this run's clock ratio")
if __name__ == "__main__":
main()