re(challenge): the part id is never persisted -- differential search says stack only

No literal 26 exists anywhere, so the GamePart id is computed. That does not stop it being found: the id is KNOWN at each screen from the GamePart table (EXTRAS = 5, MISSION SELECT = 7), so snapshot both and intersect. New tool diff_words.py does the classic differential search over the sparse guest image, and find_partslot.sh drives the two screens and runs it. Result: 171 MB scanned, exactly 4 addresses read 5 then 7 -- 0x708FFBEC, 0x708FFCBC, 0x708FFDAC, 0x708FFE20 -- and all four are guest STACK (the same run's log puts thread stacks at 0x709...). So the requested part id exists only as a stack argument in flight; there is no persistent field, which is consistent with finding no literal store, and means there is nothing stable to poke. That closes the last memory-and-menu route to the challenge missions. Reaching them needs either the genuine in-game unlock (an in-mission attainment, per AVSCRIPT_COMMAND_ATTAINMENT_CHALLENGE_MISSION_CARGO_SCORE) or an emulator-side hook that forces the transition -- a code change, not a poke. diff_words.py is worth keeping well beyond this question: it locates any field whose address is unknown but whose value is known at two moments.
2026-08-13 21:31:47 +00:00
parent a79a1da183
commit faa7b3d611
3 changed files with 167 additions and 0 deletions
--- a/docs/re/challenge-mission-gate.md
+++ b/docs/re/challenge-mission-gate.md
@@ -385,6 +385,30 @@ against the profile route. `+80` at least is handled as a small struct by addres
 (`addi r4, obj, 80` → copy helper `0x82175110`, written back via `0x8216FF70`), which
 is what a serialised value object looks like.

+### 5.8 The part id is never persisted — differential search ✅ (negative)
+
+`sub_821749C0` creates a part from `slot+12`, and no literal 26 exists anywhere, so
+the id is computed. It can still be found by **differential search**, because the id
+is *known* at each screen from §3: `GP_EXTRAS` = 5, `GP_MISSION_SELECT` = 7. Snapshot
+both screens and intersect (`tools/re-capture/diff_words.py`):
+
+```
+scanned 171 MB of allocated guest memory
+addresses reading 5 on EXTRAS and 7 on MISSION SELECT: 4
+   0x708FFBEC  0x708FFCBC  0x708FFDAC  0x708FFE20
+```
+
+**All four are guest stack** (thread stacks sit at `0x709…` in the same run's log).
+So the requested part id exists only as a **stack argument in flight** — there is no
+persistent field holding it, which is consistent with finding no literal store and
+means **there is nothing stable to poke**. Forcing a transition to GamePart 26 needs
+the caller's context, i.e. an emulator-side hook rather than a memory write.
+
+**Every memory-and-menu route to the challenge missions is now closed** (§5.4, §5.6,
+§5.7, this section). What remains is the genuine in-game unlock — an in-mission
+attainment, per `AVSCRIPT_COMMAND_ATTAINMENT_CHALLENGE_MISSION_CARGO_SCORE` — or a
+Canary patch that forces the transition in emulator code.
+
 ## 6. The unlock condition, from the strings ❔

 Three strings say challenge missions are *announced*, not menu-browsed:
--- a/tools/re-capture/diff_words.py
+++ b/tools/re-capture/diff_words.py
@@ -0,0 +1,74 @@
+#!/usr/bin/env python3
+"""Find guest addresses holding one value in snapshot A and another in snapshot B.
+
+The classic differential search, which is how you locate a field whose ADDRESS is
+unknown but whose VALUE is known at two moments. Used here for the GamePart slot:
+`GP_EXTRAS` = 5 on the extras screen, `GP_MISSION_SELECT` = 7 on the next one, so
+the slot is a word that reads 5 then 7.
+
+Both snapshots are sparse copies of `/dev/shm/xenia_memory_*`, so the ~4.6 GB of
+holes cost nothing: walk A's allocated extents with SEEK_DATA and only look at B
+where A already matched.
+
+    diff_words.py <snapA> <valA> <snapB> <valB> [--near <va> <span>]
+"""
+import os
+import struct
+import sys
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from gmem import off_to_vas  # noqa: E402
+
+
+def main():
+    if len(sys.argv) < 5:
+        print(__doc__)
+        return 1
+    pa, va_, pb, vb_ = sys.argv[1], int(sys.argv[2], 0), sys.argv[3], int(sys.argv[4], 0)
+    pat_a = struct.pack(">I", va_)
+    fa, fb = open(pa, "rb"), open(pb, "rb")
+    size = min(os.path.getsize(pa), os.path.getsize(pb))
+
+    hits, scanned = [], 0
+    off = 0
+    while off < size:
+        try:
+            off = os.lseek(fa.fileno(), off, os.SEEK_DATA)
+        except OSError:
+            break
+        if off >= size:
+            break
+        try:
+            end = min(os.lseek(fa.fileno(), off, os.SEEK_HOLE), size)
+        except OSError:
+            end = size
+        while off < end:
+            n = min(1 << 22, end - off)
+            fa.seek(off)
+            buf = fa.read(n)
+            scanned += len(buf)
+            i = 0
+            while True:
+                i = buf.find(pat_a, i)
+                if i < 0:
+                    break
+                p = off + i
+                if p % 4 == 0:
+                    fb.seek(p)
+                    if fb.read(4) == struct.pack(">I", vb_):
+                        hits.append(p)
+                i += 1
+            off += n
+
+    print(f"scanned {scanned/1e6:.0f} MB of allocated guest memory")
+    print(f"addresses reading {va_} in A and {vb_} in B: {len(hits)}")
+    for h in hits[:40]:
+        vas = off_to_vas(h)
+        print(f"   file off {h:#012x}   guest VA {[hex(v) for v in vas][:2]}")
+    if len(hits) > 40:
+        print(f"   … and {len(hits)-40} more")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())
--- a/tools/re-capture/find_partslot.sh
+++ b/tools/re-capture/find_partslot.sh
@@ -0,0 +1,69 @@
+#!/usr/bin/env bash
+# Locate the GamePart "slot" in guest memory by DIFFERENTIAL SEARCH.
+#
+# sub_821749C0 creates a part from slot+12 (the requested part id) and stores the
+# result at *(slot+16). The id is computed from a menu selection -- no literal 26
+# exists anywhere -- so it cannot be found statically. But the id is KNOWN at each
+# screen from the GamePart table (docs/re/challenge-mission-gate.md section 3):
+#
+#     GP_EXTRAS = 5      GP_MISSION_SELECT = 7
+#
+# so the slot is simply the word that reads 5 on the EXTRAS screen and 7 on the
+# MISSION SELECT screen. Snapshot both, intersect, and the candidates are few.
+#
+# Usage: find_partslot.sh [boot_timeout_s]
+set -u
+export HOME=/sylph-home/re DISPLAY=:99 SDL_AUDIODRIVER=dummy
+export XENIA_PAD_FILE=/tmp/xenia_pad.txt
+HERE="$(cd "$(dirname "$0")" && pwd)"
+pad()  { python3 "$HERE/pad.py"   "$@"; }
+poke() { python3 "$HERE/gpoke.py" "$@"; }
+BOOT_TIMEOUT="${1:-400}"
+say()  { echo "[$(date +%H:%M:%S)] $*"; }
+px()   { convert /tmp/nav-probe.png -format \
+         "%[fx:int(255*p{$1}.r)] %[fx:int(255*p{$1}.g)] %[fx:int(255*p{$1}.b)]" info: 2>/dev/null; }
+at_menu() {
+  screenshot /tmp/nav-probe.png >/dev/null 2>&1 || return 1
+  read -r r g b < <(px "648,221"); [ -n "${r:-}" ] || return 1
+  [ "$r" -gt 230 ] && [ "$g" -gt 230 ] && [ "$b" -gt 230 ] || return 1
+  read -r r2 g2 b2 < <(px "560,300"); [ -n "${r2:-}" ] || return 1
+  [ "$r2" -lt 120 ] && [ "$b2" -gt "$r2" ]
+}
+at_title() {
+  screenshot /tmp/nav-probe.png >/dev/null 2>&1 || return 1
+  read -r r g b < <(px "625,618"); [ -n "${g:-}" ] || return 1
+  [ "$g" -gt 130 ] && [ $((g - r)) -gt 45 ] && [ $((g - b)) -gt 45 ]
+}
+snap() { SHM=$(ls /dev/shm/xenia_memory_* 2>/dev/null | head -1); cp --sparse=always "$SHM" "$1"; }
+
+pkill -9 -x xenia_canary 2>/dev/null; sleep 1
+rm -f /dev/shm/xenia_* 2>/dev/null; : > "$XENIA_PAD_FILE"
+say "launching"
+run-canary --audio --apu=sdl --log_mask=13 \
+  --logged_profile_slot_0_xuid=E0300000EFBEA3D4 \
+  --hid=file --pad_file="$XENIA_PAD_FILE" &
+xsetroot -solid black 2>/dev/null || true
+
+MENU=0
+for i in $(seq 1 "$BOOT_TIMEOUT"); do
+  [ "$i" -lt 40 ] && { sleep 1; continue; }
+  if at_menu && sleep 1 && at_menu; then say "MAIN MENU after ${i}s"; MENU=1; break; fi
+  at_title && { say "  title — tapping A"; pad tap A 0.25; sleep 2; }
+  sleep 1
+done
+[ "$MENU" = 1 ] || { say "TIMEOUT"; exit 1; }
+
+say "-> EXTRAS (GamePart 5)"
+for _ in 1 2 3 4; do pad dpad down 0.06; sleep 0.35; done
+sleep 0.5; pad tap A 0.15; sleep 4
+screenshot "$HOME/shots/slot-01-extras.png" >/dev/null
+snap /sylph-home/re/snap-extras.bin; say "snapshot A (EXTRAS) taken"
+
+say "-> MISSION SELECT (GamePart 7)"
+pad tap A 0.15; sleep 5
+screenshot "$HOME/shots/slot-02-missionselect.png" >/dev/null
+snap /sylph-home/re/snap-msel.bin; say "snapshot B (MISSION SELECT) taken"
+
+say "intersecting: word == 5 in A and == 7 in B"
+python3 "$HERE/diff_words.py" /sylph-home/re/snap-extras.bin 5 /sylph-home/re/snap-msel.bin 7
+say "done — emulator left running"