bbc167425f
Two runs, neither answering whether a .tbm draws pixels. Run 1 TIMED the title->menu transition and was still on the title 8 s later (glyph 714, the plate's pulse trough), so the second tap did the transition and the 'submenu' capture is the menu. Void. Run 2 DETECTED the menu instead -- glyph 327, matching live-main-menu.png exactly -- tapped 0.8 s later, and 12 s after that was still on the menu. The log says why: 2 file-pad vk=5800 lines, i.e. ONE press, and one RE-INPUT delivery. The second tap was never delivered, with zero swallow lines so it is not the sign-in path. A 0.12 s press issued while the guest is still loading a screen is missed outright. So 'the press did nothing' and 'there was no press' look identical from the screen, and only the log separates them. Worth more than the run: this is the third time in one iteration that timing was used where detection was required -- the title->menu wait, the menu->submenu wait, and the press itself. Each fix is the same substitution, and each was written only after the timed version had produced a confident wrong answer. Also records that no focus detector is needed for this question, since every main-menu destination except EXTRAS carries a .tbm decider. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wuu56cE8vJGTBtn1ppsk8v
96 lines
4.0 KiB
Python
Executable File
96 lines
4.0 KiB
Python
Executable File
#!/usr/bin/env python3
|
|
"""Reach a submenu that carries a `.tbm` and capture it.
|
|
|
|
`ui-forced-backdrop.md` leaves 24 of its 62 deciding verdicts on `.tbm` elements
|
|
whose pixels this corpus cannot locate: not in the bundle, not a file, not a pak
|
|
entry, and **not in our composite** — `compose` skips an element with no resolvable
|
|
sprite, so our renderer draws *nothing* for a `.tbm`. The open question is whether
|
|
the game draws anything either. If it does not, those verdicts are inert rather
|
|
than correct.
|
|
|
|
⚠️ **No focus detector is needed, and that is deliberate.**
|
|
`s00a-drive-blocked-by-focus.md` records that a per-row brightness statistic
|
|
**failed its own control**, and that wrap-around makes counting presses useless.
|
|
But every main-menu destination except `EXTRAS` lands on an archive holding a
|
|
`.tbm` decider — `GP_SYSTEM` (`pqbase`), `GP_TUTORIAL` (`pubase`),
|
|
`GP_SAVE_LOAD` (`px_replay_base`), `GP_DIALOG` (`pcbase`). So pressing Ⓐ on
|
|
whatever happens to be focused is very likely to land somewhere useful, and the
|
|
screen is identified **afterwards, from the capture**, rather than chosen in
|
|
advance.
|
|
|
|
tbm_screen_capture.py OUTDIR [wait_s]
|
|
"""
|
|
import os
|
|
import subprocess
|
|
import sys
|
|
import time
|
|
|
|
import numpy as np
|
|
from PIL import Image
|
|
|
|
OUT = sys.argv[1]
|
|
WAIT = float(sys.argv[2]) if len(sys.argv) > 2 else 420
|
|
W, H = 1280, 720
|
|
NEED, CEIL, HOLD = 500, 2500, 12
|
|
# 🔴 The first version TIMED the title->menu transition (tap, wait 8 s, assume).
|
|
# It was still on the title 8 s later -- glyph 714, the plate's pulse trough --
|
|
# so the second tap performed the transition and no submenu was ever reached.
|
|
# The menu has its own signature: 327 (`live-main-menu.png`), far below the
|
|
# plate's 714..1520. Detect it, the way the title is detected.
|
|
MENU_LO, MENU_HI, MENU_HOLD = 250, 420, 6
|
|
PAD = os.path.join(os.path.dirname(os.path.abspath(__file__)), "pad.py")
|
|
|
|
|
|
def _open():
|
|
return subprocess.Popen(
|
|
["ffmpeg", "-loglevel", "error", "-f", "x11grab", "-draw_mouse", "0",
|
|
"-video_size", f"{W}x{H}", "-i", ":98", "-r", "4",
|
|
"-f", "rawvideo", "-pix_fmt", "rgb24", "-"],
|
|
stdout=subprocess.PIPE, bufsize=W * H * 3 * 2)
|
|
|
|
|
|
def tap(btn="A"):
|
|
subprocess.run([sys.executable, PAD, "tap", btn, "0.12"], check=False)
|
|
print(f"[{time.time()-T0:7.1f}s] tapped {btn}", flush=True)
|
|
|
|
|
|
def glyph(a):
|
|
r, g, b = a[:, :, 0], a[:, :, 1], a[:, :, 2]
|
|
return int(((g > 130) & (g - r > 45) & (g - b > 45)).sum())
|
|
|
|
|
|
T0 = time.time()
|
|
p, n, seg = _open(), W * H * 3, time.time()
|
|
log = open(f"{OUT}/series.tsv", "w"); log.write("# t_s\tglyph\tmean\tphase\n")
|
|
phase, streak, mark = "wait", 0, None
|
|
while True:
|
|
el = time.time() - T0
|
|
if phase == "wait" and el > WAIT:
|
|
print("TITLE NEVER APPEARED", flush=True); break
|
|
if time.time() - seg > 30:
|
|
p.kill(); p = _open(); seg = time.time()
|
|
buf = p.stdout.read(n)
|
|
if len(buf) < n:
|
|
p.kill(); p = _open(); seg = time.time(); continue
|
|
a = np.frombuffer(buf, np.uint8).reshape(H, W, 3).astype(int)
|
|
c = glyph(a)
|
|
log.write(f"{el:.3f}\t{c}\t{a.mean():.3f}\t{phase}\n"); log.flush()
|
|
if phase == "wait":
|
|
streak = streak + 1 if NEED <= c <= CEIL else 0
|
|
if streak >= HOLD:
|
|
tap(); mark = time.time(); phase = "menu"
|
|
elif phase == "menu":
|
|
streak = streak + 1 if MENU_LO <= c <= MENU_HI else 0
|
|
if streak >= MENU_HOLD:
|
|
Image.fromarray(a.astype(np.uint8)).save(f"{OUT}/menu.png")
|
|
print(f"[{el:7.1f}s] MENU detected (glyph {c}) — pressing A into a submenu", flush=True)
|
|
tap(); mark = time.time(); phase = "submenu"; streak = 0
|
|
elif time.time() - mark > 60:
|
|
print(f"[{el:7.1f}s] menu never detected (glyph {c}) — retapping", flush=True)
|
|
tap(); mark = time.time()
|
|
elif phase == "submenu" and time.time() - mark > 12:
|
|
Image.fromarray(a.astype(np.uint8)).save(f"{OUT}/submenu.png")
|
|
print(f"[{el:7.1f}s] submenu captured (glyph {c}, mean {a.mean():.1f})", flush=True)
|
|
break
|
|
p.kill(); log.close()
|