re: widen unit coverage to all six tutorials, and prove the fields are run-invariant

Captures the five remaining tutorials (grab_tutorial.sh, one cold boot each)
and re-solves over seven snapshots from seven separate emulator runs. Coverage
18 -> 21 units, confirmed fields 22 -> 27 (Size_Y, MassScore, MinimumVelocity,
AV_PitchPlus_Min, AV_PitchMinus_Min join the zero-contradiction set).

The multi-run union exposed something the single-run check could not: the same
unit's object is NOT byte-identical between runs. unit_runtime.py --crosscheck
pins down why -- exactly 15 words differ, 13 of them holding guest heap
pointers, and none of them is a solved or interpolated field offset. So every
value reported is run-invariant, which is a stronger statement than the
within-run identity check that came before it.

The two non-pointer stragglers are a real caveat, now documented rather than
smoothed over: +0x2c8 reads 8000.0 for UN_f001_TCAF_DeltaSaber_T_Ttrl in two
tutorials and 10000.0 in a third, with a 0/1 flag at +0x2d0. A couple of words
in the object are set per stage, so it is mostly but not entirely the parsed
table. Unidentified, marked NEEDS-HUMAN.

Coverage honesty: all six tutorials together add only 3 units the missions do
not already have -- they reuse one training box, one drone and the player
craft. Further coverage needs story progress, not more tutorials.

Two navigation facts encoded in grab_tutorial.sh, both learned by breaking
them: the main menu is not input-ready for ~10 s after the title tap and early
d-pad presses are dropped (which sends the A to NEW GAME); and NEW GAME is not
a shortcut to Stage 01 -- it gates on DIFFICULTY then plays the prologue. A
NEW GAME excursion to the READY ROOM leaves game01/savedata byte-identical,
verified by diff against a backup.
This commit is contained in:
2026-07-29 18:22:47 +00:00
parent 2196ac112d
commit 1ee2d5e406
6 changed files with 577 additions and 130 deletions

View File

@@ -29,6 +29,7 @@ Findings produced with these: [`docs/re/weapon-datasheet-runtime.md`](../../docs
| `unit_runtime.py` | Same solver for the `unit\UN_*.tbl` definition objects (vtable `0x820af844`). Unions several snapshots — unit definitions are per-stage. |
| `schema_order.py` | Merge a sub-record's field-declaration order across all tables (topological sort); the layout check that pins fields no table ever values. |
| `order_check.py` | Test `offset = base + 4*index` for one table's sub-record against solver output. |
| `grab_tutorial.sh` | Cold-boot Canary, walk to the Nth TUTORIAL entry, wait for the stage load, snapshot guest RAM. One emulator per capture — backing out of a loaded mission wedges it. |
Snapshot first — `cp --sparse=always /dev/shm/xenia_memory_* snap.bin` (~2 s) — and
point `$GMEM_FILE` at the copy; the running emulator pegs every core under lavapipe.

View File

@@ -0,0 +1,66 @@
#!/usr/bin/env bash
# Capture one tutorial's unit definitions: cold-boot Canary, walk
# title -> TUTORIAL -> the Nth entry, wait for the stage to load, snapshot RAM.
#
# One emulator per tutorial on purpose: backing out of a loaded mission via
# PAUSE -> BACK TO MENU wedged the emulator (log stops, CPU spins), while a
# cold boot here is ~15 s. The unit definitions are all parsed at stage load,
# so a single snapshot right after the load is everything this needs.
set -u
N="${1:?usage: grab_tutorial.sh <index 0..5> <outfile>}"
OUT="${2:?}"
export HOME=/sylph-home/re SDL_AUDIODRIVER=dummy DISPLAY=:98
SD="$(cd "$(dirname "$0")" && pwd)"
RC="/home/fabi/RE - Project Sylpheed/sylpheed-reborn/tools/re-capture"
# The container entrypoint's Xvfb/openbox do NOT restart on their own, and a
# dead Xvfb leaves a <defunct> entry that still matches `pgrep` -- so test the
# display with xdpyinfo, never by process name.
ensure_display(){
if ! xdpyinfo -display "$DISPLAY" >/dev/null 2>&1; then
echo "[grab] $DISPLAY is down -> restarting Xvfb + openbox"
setsid nohup Xvfb "$DISPLAY" -screen 0 1280x720x24 -ac -nolisten tcp \
+extension GLX +extension RANDR </dev/null >/tmp/xvfb98.log 2>&1 &
for _ in $(seq 1 50); do xdpyinfo -display "$DISPLAY" >/dev/null 2>&1 && break; sleep 0.2; done
setsid nohup env DISPLAY="$DISPLAY" HOME=/sylph-home openbox </dev/null >/tmp/openbox98.log 2>&1 &
sleep 1
fi
xdpyinfo -display "$DISPLAY" >/dev/null 2>&1 || { echo "DISPLAY UNAVAILABLE"; exit 1; }
}
step(){ vgamepad dpad "$1"; sleep 0.2; vgamepad dpad center; sleep 0.6; }
shot(){ screenshot "$SD/$1" >/dev/null 2>&1; }
pkill -x xenia_canary 2>/dev/null; sleep 2
pgrep -x xenia_canary >/dev/null && { pkill -9 -x xenia_canary; sleep 2; }
rm -f /dev/shm/xenia_memory_* /dev/shm/xenia_code_cache_* 2>/dev/null
ensure_display
cd /sylph-home/re
setsid nohup run-canary --audio --apu=sdl --log_mask=13 \
--logged_profile_slot_0_xuid=E0300000EFBEA3D4 </dev/null >/dev/null 2>&1 &
sleep 5
"$RC/skip_intro.sh" 600 || { echo "BOOT FAILED"; exit 1; }
# The main menu is not input-ready for ~10 s after the title tap under lavapipe;
# d-pad presses before that are silently dropped, and the A then lands on NEW
# GAME instead of TUTORIAL. This wait is the whole fix.
sleep 14
shot "nav-$N-menu.png"
step down; step down # NEW GAME -> TUTORIAL
vgamepad tap A 250; sleep 8
shot "nav-$N-list.png"
i=0; while [ "$i" -lt "$N" ]; do step down; i=$((i+1)); done
shot "nav-$N-pick.png"
vgamepad tap A 250
# the stage load takes ~25-45 s under lavapipe; wait for the screen to settle
sleep 50
shot "nav-$N-loaded.png"
MEM=$(ls /dev/shm/xenia_memory_* 2>/dev/null | head -1)
[ -n "$MEM" ] || { echo "NO SHM"; exit 1; }
cp --sparse=always "$MEM" "$SD/$OUT" || exit 1
echo "SNAPSHOT $OUT ($(du -h "$SD/$OUT" | cut -f1))"

View File

@@ -195,6 +195,47 @@ def solve(disc, run):
return {f: v for f, v in solved.items() if f in winners}, numeric
def crosscheck(paths, solved):
"""Which words of a unit's object vary between snapshots, and are any of
them fields we claim to have solved?
Within one run the objects are byte-identical, but across runs the same
unit differs -- pointer words hold heap addresses, which move. This
separates "the emulator relocated it" from "the value is not definition
data", and it is the check that keeps the solved set honest.
"""
per = {}
for p in paths:
for k, v in runtime_objects([p])[0].items():
per.setdefault(k, {})[os.path.basename(p)] = v
diff = set()
shared = 0
for k, d in per.items():
if len(d) < 2:
continue
shared += 1
vs = list(d.values())
for v in vs[1:]:
diff |= {i // 4 * 4 for i in range(min(len(v), len(vs[0]))) if v[i] != vs[0][i]}
def looks_ptr(off):
vals = [struct.unpack_from(">I", v, off)[0]
for d in per.values() for v in d.values() if len(v) > off + 3]
return sum(1 for x in vals if 0x80000000 <= x < 0xC0000000), len(vals)
print(f"\n# cross-run check: {shared} unit(s) appear in more than one snapshot")
print(f"# words that differ between runs: {len(diff)}")
for off in sorted(diff):
p, n = looks_ptr(off)
kind = "guest pointer" if p == n else ("mixed" if p else "non-pointer")
print(f"# +{off:#05x} {kind} ({p}/{n} look like pointers)")
offs = {o for o, _, _, _, _ in solved.values()} | {o for o, _, _ in INTERPOLATED}
clash = sorted(offs & diff)
print("# solved/interpolated offsets that vary between runs: "
+ (", ".join(f"{o:#05x}" for o in clash) if clash
else "NONE — every reported field is run-invariant"))
def main():
tokens = sys.argv[1]
snaps = [a for a in sys.argv[2:] if not a.startswith("--")]
@@ -214,6 +255,9 @@ def main():
for field, (off, enc, agree, bad, distinct) in sorted(solved.items(), key=lambda kv: kv[1][0]):
print(f"{off:#8x} {enc:>4} {field:<32} {agree:5d} {distinct:4d} {len(bad):3d}")
if "--crosscheck" in sys.argv:
crosscheck(snaps, solved)
if "--csv" in sys.argv:
import csv