diff --git a/docs/re/BACKLOG.md b/docs/re/BACKLOG.md index 3913b4ef..b711b52b 100644 --- a/docs/re/BACKLOG.md +++ b/docs/re/BACKLOG.md @@ -642,6 +642,21 @@ search cannot find a *schedule*. watch in ~40 lines vs 150, first live run clean — 116 roster, 32 witnesses at 10/s, **0 stalled**, 7 losses, TSV written incrementally. ⚠️ Existing probes deliberately **not** ported — they work and other docs cite their results. +* ✅✅ **(2026-08-25) `REMAINING OB` FOUND AND VERIFIED: big-endian u32 at + `0xbdb59668`.** On the new harness, the HUD changed **4→8** and the intersection + collapsed in one step to a single `u32be` survivor (with `u16be`/`u8` hits being + the low half/byte of the same word). **Verified live**: HUD `012` vs + `mem@0xbdb59668 = 12`. Independently rediscovers the address the earliest + sessions found by digit-transition hunting. + 🔴 **Corrects the earlier "u32be refuted"** — that came from intersecting on a + HUD reading of "11" which was almost certainly a misread, since `ob_digits.png` + only has templates for **0 1 2 4 8** and misreads rather than rejects other + digits. **A single bad input permanently poisons an intersection**; the reader's + confidence scores were printed but never gated on. + 🟡 **The counter INCREASES: 4 → 8 → 12** in ~5 min, measured in memory. Marked + objective targets are being **added during the mission** — the arrival question + again, now with a one-word signal instead of a 32 MB scan. **Next: watch this + address across a whole mission.** * ~~🚧 BLOCKER: t=210/240 unreachable in one turn~~ — **superseded, see above**; it rested on an untested assumption that a turn is one shell call. 595 s shell cap − ~220 s boot (a ~190 s title movie that cannot be tapped through) − ~25 s startup = **~350 s observation ≈ 193 game-seconds**. diff --git a/docs/re/remaining-ob-hunt.md b/docs/re/remaining-ob-hunt.md index a1ce4de8..9f344de4 100644 --- a/docs/re/remaining-ob-hunt.md +++ b/docs/re/remaining-ob-hunt.md @@ -258,3 +258,73 @@ Two HUD readings at **different** values, in non-stalled samples. The counter changes on kills, so this lands back on the combat limit — unless a phase change or another event moves it. The earlier 4 → 11 observation shows it does move, which is what makes the approach worth continuing. + +--- + +# ✅ FOUND AND VERIFIED: `REMAINING OB` is a big-endian u32 at `0xbdb59668` + +## The run + +Ported onto the shared harness (`ob_hunt2.py`), the HUD changed **4 → 8** and the +intersection collapsed in one step: + +``` +t=126s HUD=4 u32be:8717 u32le:163 u16be:13844 u16be@1:3319 u16le:832 u8:36635 +t=191s HUD=8 u32be:1 u32le:0 u16be:1 u16be@1:2 u16le:0 u8:3 +t=254s HUD=8 (unchanged) +``` + +Resolving the survivor: + +| encoding | address | +|---|---| +| **`u32be`** | **`0xbdb59668`** | +| `u16be` | `0xbdb5966a` — the low half of the same word | +| `u8` | `0xbdb5966b` — the low byte of the same word | + +All three are the same location. The `u16be@1` pair and the other `u8` hits are +elsewhere and did not survive as a coherent field. + +## ✅ Verified live against the HUD + +Read three times, twenty seconds apart, screenshot and memory together: + +``` +HUD='0?2' mem@0xbdb59668 = 12 (HUD frame unreadable) +HUD='012' mem@0xbdb59668 = 12 MATCH +HUD='???' mem@0xbdb59668 = 12 (HUD frame unreadable) +``` + +The one legible frame agrees exactly. **`REMAINING OB` is a big-endian `u32` at +`0xbdb59668`** — independently rediscovered by value-intersection, and landing on +the same address the earliest sessions found by digit-transition hunting. + +## 🔴 Correction: the earlier "big-endian u32 refuted" was wrong + +An earlier run intersected HUD readings of 4 and then **11** and got zero +survivors in every encoding, which was written up as eliminating `u32be` for the +whole region. This run shows `u32be` holds the counter. + +The likely cause is the reading itself: `ob_digits.png` only has templates for +**0 1 2 4 8**, so any value containing 3/5/6/7/9 is misread rather than rejected, +and "11" was probably one of those. A single bad reading poisons an intersection +permanently — it removes the true address and nothing later can bring it back. + +**Lesson: an intersection method needs its inputs to be individually verifiable.** +The HUD reader's confidence scores were printed but never gated on; a minimum +score would have dropped the bad frame instead of trusting it. + +## 🟡 The counter *increases*: 4 → 8 → 12 + +Across roughly five minutes the value went 4, then 8, then 12 — measured in +memory, so not a digit misread. A counter of *remaining* marked targets that +rises means **targets are being added during the mission**. + +That is the arrival question again, and now with a cheap, reliable, one-word +signal instead of a 32 MB scan. It does not contradict +[mission-phase-deployment.md](mission-phase-deployment.md) — the roster's +deployment is fixed at load — but it does mean the game *marks* new objective +targets as the mission proceeds. + +Watching this one address across a mission is now the obvious next experiment, +and it costs almost nothing. diff --git a/tools/re-capture/ob_hunt2.py b/tools/re-capture/ob_hunt2.py new file mode 100755 index 00000000..3f02c92f --- /dev/null +++ b/tools/re-capture/ob_hunt2.py @@ -0,0 +1,96 @@ +#!/usr/bin/env python3 +"""REMAINING OB by HUD value, on the shared harness. + +Same idea as ob_by_hud.py -- read the counter off screen, keep heap positions +equal to it, intersect across readings -- but the witness, the incremental save +and the baseline discard now come from probeharness.Probe rather than being +re-implemented (and, twice, forgotten). + +Candidates persist to /tmp/ob_candidates.json after EVERY reading, so a chained +attach continues the same intersection on the same mission. Offsets are only +valid within one emulator instance; the session clears the file at launch. +""" +import json, os, subprocess, sys +sys.path.insert(0, __file__.rsplit('/', 1)[0]) +from probeharness import Probe +import gmem +import numpy as np +import ob_read + +SHOT = '/tmp/ob_hud.png' +CAND = '/tmp/ob_candidates.json' +ENCODINGS = [('u32be', '>u4', 4, 0), ('u32le', 'u2', 2, 0), ('u16be@1', '>u2', 2, 1), + ('u16le', ' (1 << (8 * w)) - 1: + continue + n = (len(buf) - skew) // w * w + a = np.frombuffer(buf[skew:skew + n], dtype=dt) + out[name] = (np.nonzero(a == value)[0].astype(np.int64) * w + skew) + return out + +def main(): + secs = int(sys.argv[1]) if len(sys.argv) > 1 else 300 + every = int(sys.argv[2]) if len(sys.argv) > 2 else 25 + p = Probe(baseline=116) + if not p.ok: + print(p.why); return 3 + print(p.summary(), flush=True) + p.log('t\thud\tencoding\tcandidates\tstalled', '/tmp/ob_hunt2.tsv') + + cand, base, seen = None, None, [] + if os.environ.get('SYLPH_OB_RESUME') == '1' and os.path.exists(CAND): + raw = json.load(open(CAND)) + cand = {k: np.array(v, dtype=np.int64) for k, v in raw['cand'].items()} + seen = raw['seen'] + print('resumed: %s' % {k: len(v) for k, v in cand.items()}, flush=True) + + while True: + v, note = hud() + buf, base = p.heap() + st = p.status() + if v is None: + print('t=%4ds HUD unreadable (%s)%s' % (p.elapsed, note, st), flush=True) + else: + hit = matches(buf, v) + cand = hit if cand is None else { + k: np.intersect1d(cand[k], hit[k], assume_unique=True) + for k in cand if k in hit} + seen.append(v) + json.dump({'cand': {k: v2.tolist() for k, v2 in cand.items()}, + 'seen': seen}, open(CAND, 'w')) + desc = ' '.join('%s:%d' % (k, len(cand[k])) for k in cand) + print('t=%4ds HUD=%-4d %s%s' % (p.elapsed, v, desc, st), flush=True) + for k in cand: + p.emit('%d\t%d\t%s\t%d\t%s' % (p.elapsed, v, k, len(cand[k]), st.strip())) + if not p.tick(every, secs): + break + + print('\n%s' % p.summary()) + print('HUD values seen: %s' % sorted(set(seen))) + if cand: + for k in sorted(cand, key=lambda k: len(cand[k])): + c = cand[k] + print(' %-8s %d' % (k, len(c))) + if 0 < len(c) <= 8 and len(set(seen)) >= 2: + for off in c: + va = gmem.primary_va(base + int(off)) + print(' va %s' % (('%#010x' % va) if va else '?')) + return 0 + +if __name__ == '__main__': + sys.exit(main()) diff --git a/tools/re-capture/obhunt2_attach.sh b/tools/re-capture/obhunt2_attach.sh new file mode 100755 index 00000000..3ff92c0f --- /dev/null +++ b/tools/re-capture/obhunt2_attach.sh @@ -0,0 +1,19 @@ +#!/usr/bin/env bash +set -u +export HOME=/sylph-home/re SDL_AUDIODRIVER=dummy DISPLAY=:98 +export PYTHONPATH=/sylph-home/.local/lib/python3.12/site-packages +SD="$(cd "$(dirname "$0")" && pwd)" +pgrep -x xenia_canary >/dev/null || { echo "NO EMULATOR"; exit 1; } +CFG=/tmp/nav-obhud2.json +for try in 1 2 3; do + python3 "$SD/pad.py" set "rt=1" >/dev/null 2>&1 || true; sleep 3 + python3 "$SD/pad.py" clear >/dev/null 2>&1 || true + if python3 "$SD/entities2.py" self 0x130 "$CFG" >/dev/null 2>&1; then + SYLPH_HUNT=1 SYLPH_KILL_TURRETS=1 SYLPH_KEEPOUT=1400 SYLPH_PREFER=e010 \ + nohup python3 "$SD/pilot.py" "$CFG" "${1:-400}" /tmp/obhud2-pilot.log 2>&1 & + P=$!; echo "--- pilot re-attached"; break + fi +done +SYLPH_OB_RESUME=1 python3 "$SD/ob_hunt2.py" "${1:-400}" "${2:-25}" +[ -n "${P:-}" ] && kill "$P" 2>/dev/null +echo "OBHUD ATTACH DONE" diff --git a/tools/re-capture/obhunt2_session.sh b/tools/re-capture/obhunt2_session.sh new file mode 100755 index 00000000..b035062e --- /dev/null +++ b/tools/re-capture/obhunt2_session.sh @@ -0,0 +1,20 @@ +#!/usr/bin/env bash +set -u +export HOME=/sylph-home/re SDL_AUDIODRIVER=dummy DISPLAY=:98 +export PYTHONPATH=/sylph-home/.local/lib/python3.12/site-packages +SD="$(cd "$(dirname "$0")" && pwd)" +"$SD/launch_mission.sh" fly || { echo "BOOT FAILED"; exit 1; } +CFG=/tmp/nav-obhud.json +for try in 1 2 3; do + python3 "$SD/pad.py" set "rt=1" >/dev/null 2>&1 || true; sleep 3 + python3 "$SD/pad.py" clear >/dev/null 2>&1 || true + if python3 "$SD/entities2.py" self 0x130 "$CFG" >/dev/null 2>&1; then + SYLPH_HUNT=1 SYLPH_KILL_TURRETS=1 SYLPH_KEEPOUT=1400 SYLPH_PREFER=e010 \ + nohup python3 "$SD/pilot.py" "$CFG" "${1:-240}" /tmp/obhud-pilot.log 2>&1 & + P=$!; echo "--- pilot flying"; break + fi +done +rm -f /tmp/ob_candidates.json +python3 "$SD/ob_hunt2.py" "${1:-240}" "${2:-25}" +[ -n "${P:-}" ] && kill "$P" 2>/dev/null +echo "OBHUD DONE"