re: the .tbm submenu is not reached -- and the second tap was never delivered
Two runs, neither answering whether a .tbm draws pixels. Run 1 TIMED the title->menu transition and was still on the title 8 s later (glyph 714, the plate's pulse trough), so the second tap did the transition and the 'submenu' capture is the menu. Void. Run 2 DETECTED the menu instead -- glyph 327, matching live-main-menu.png exactly -- tapped 0.8 s later, and 12 s after that was still on the menu. The log says why: 2 file-pad vk=5800 lines, i.e. ONE press, and one RE-INPUT delivery. The second tap was never delivered, with zero swallow lines so it is not the sign-in path. A 0.12 s press issued while the guest is still loading a screen is missed outright. So 'the press did nothing' and 'there was no press' look identical from the screen, and only the log separates them. Worth more than the run: this is the third time in one iteration that timing was used where detection was required -- the title->menu wait, the menu->submenu wait, and the press itself. Each fix is the same substitution, and each was written only after the timed version had produced a confident wrong answer. Also records that no focus detector is needed for this question, since every main-menu destination except EXTRAS carries a .tbm decider. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wuu56cE8vJGTBtn1ppsk8v
This commit is contained in:
35
docs/re/data/tbm-submenu-attempt.txt
Normal file
35
docs/re/data/tbm-submenu-attempt.txt
Normal file
@@ -0,0 +1,35 @@
|
||||
# Attempt to reach a .tbm-bearing submenu -- NOT REACHED, and why.
|
||||
#
|
||||
# 2026-08-30. The open question is ui-forced-backdrop.md's 24 .tbm deciders:
|
||||
# our compose draws NOTHING for a .tbm, so their verdicts are 'correct or
|
||||
# inert' and indistinguishable. Every main-menu destination except EXTRAS
|
||||
# lands on an archive holding one, so no focus detector is needed -- press A
|
||||
# on whatever is focused and identify the screen from the capture.
|
||||
#
|
||||
# RUN 1 -- the title->menu transition was TIMED (tap, wait 8 s, assume).
|
||||
# 8 s later the screen was still the TITLE: glyph 714, the plate pulse's
|
||||
# trough. So the second tap performed the transition and the 'submenu'
|
||||
# capture is the main menu. Void for the question asked.
|
||||
#
|
||||
# RUN 2 -- menu DETECTED instead of timed (glyph in 250..420 for 6
|
||||
# consecutive samples; the main menu is 327 and the plate is 714..1520).
|
||||
# [351.3s] tapped A
|
||||
# [357.2s] MENU detected (glyph 327) -- pressing A into a submenu
|
||||
# [358.0s] tapped A
|
||||
# [369.8s] submenu captured (glyph 327, mean 44.4)
|
||||
# -> still the main menu. 2.4 % of pixels differ from the menu capture.
|
||||
#
|
||||
# 🔴 WHY: the second tap was NEVER DELIVERED.
|
||||
# [file-pad] vk=5800 lines: 2 -- that is ONE press (down + up)
|
||||
# [RE-INPUT] -> user=0 vk=5800: one down, one up
|
||||
# swallowed by IsUIActive: 0 (so NOT the sign-in dialog path)
|
||||
#
|
||||
# The tap at 358.0 s came 0.8 s after the menu appeared, while the guest was
|
||||
# still loading it. A 0.12 s press is missed entirely if the guest does not
|
||||
# poll during that window -- the pad driver reports what it emitted, and the
|
||||
# guest simply never asked.
|
||||
#
|
||||
# WHAT WORKS AND IS NOW VALIDATED:
|
||||
# * the plate-pulse title detector (three runs)
|
||||
# * the MENU detector: glyph 327, matching live-main-menu.png exactly
|
||||
# * delivery is checkable in-log: [RE-INPUT] ... -> user=0 vk=5800
|
||||
67
docs/re/structures/tbm-submenu-not-reached.md
Normal file
67
docs/re/structures/tbm-submenu-not-reached.md
Normal file
@@ -0,0 +1,67 @@
|
||||
# 🟡 Does a `.tbm` draw pixels? Still open — two runs, and the third timing bug in a row
|
||||
|
||||
**Classification: not an answer.** Recorded per *"do not improvise around a
|
||||
blocker"*, and because the *reason* both runs failed is the same mistake three
|
||||
times over.
|
||||
|
||||
## The question
|
||||
|
||||
[`ui-forced-backdrop.md`](ui-forced-backdrop.md) has **24** of its 62 deciding
|
||||
verdicts on `.tbm` elements. `compose` draws **nothing** for a `.tbm` — no
|
||||
resolvable sprite — so those verdicts are *"correct or inert"* and the two cannot
|
||||
be separated. A capture of a screen carrying one would separate them.
|
||||
|
||||
✅ **And no focus detector is needed**, which is what makes this cheap now.
|
||||
[`s00a-drive-blocked-by-focus.md`](../s00a-drive-blocked-by-focus.md) records that a
|
||||
per-row brightness statistic **failed its own control**, and that wrap-around makes
|
||||
counting presses useless. But *every* main-menu destination except `EXTRAS` lands on
|
||||
an archive holding a `.tbm` decider — `GP_SYSTEM` (`pqbase`), `GP_TUTORIAL`
|
||||
(`pubase`), `GP_SAVE_LOAD` (`px_replay_base`), `GP_DIALOG` (`pcbase`). So press Ⓐ on
|
||||
whatever is focused and identify the screen **from the capture**.
|
||||
|
||||
## What happened
|
||||
|
||||
| run | approach | result |
|
||||
|---|---|---|
|
||||
| 1 | **timed** the title→menu transition (tap, wait 8 s) | 8 s later still the **title** — glyph 714, the plate's pulse trough. The second tap did the transition; the "submenu" capture is the menu. **Void.** |
|
||||
| 2 | **detected** the menu (glyph 250–420 for 6 samples; menu is 327, plate is 714–1520) | menu found at 357.2 s, glyph **327** exactly. Tapped at 358.0 s. 12 s later: **still the menu.** |
|
||||
|
||||
## 🔴 The second tap was never delivered
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `[file-pad] vk=5800` lines | **2** — that is *one* press, down and up |
|
||||
| `[RE-INPUT] -> user=0 vk=5800` | one down, one up |
|
||||
| `swallowed by IsUIActive` | **0** — not the sign-in path |
|
||||
|
||||
The tap came **0.8 s after the menu appeared**, while the guest was still loading
|
||||
it. A **0.12 s** press is missed outright if the guest does not poll during that
|
||||
window. The pad driver reports what *it* emitted; the guest never asked.
|
||||
|
||||
⚠️ **So "the press did nothing" and "there was no press" look identical from the
|
||||
screen**, and only the log separates them. Any driven run that presses and waits
|
||||
must **confirm delivery in the log** before interpreting the result — otherwise a
|
||||
missed press reads as a screen that did not respond.
|
||||
|
||||
## The pattern worth more than the run
|
||||
|
||||
This is the **third** time in one iteration that timing was used where detection was
|
||||
required — the title→menu wait, the menu→submenu wait, and the press itself
|
||||
(assumed delivered rather than confirmed). The first two were caught because the
|
||||
screen was recognisable; the third only because the log records deliveries.
|
||||
|
||||
📌 Each fix has the same shape: **replace "wait long enough" with "watch for the
|
||||
thing".** The title detector, the menu detector and the delivery check are all that
|
||||
substitution, and each was written only after the timed version had already produced
|
||||
a confident wrong answer.
|
||||
|
||||
## What is now validated, and what the next run needs
|
||||
|
||||
✅ Working: the plate-pulse title detector (three runs); the **menu detector**, glyph
|
||||
**327**, matching `live-main-menu.png` exactly; and delivery confirmation from
|
||||
`[RE-INPUT]`.
|
||||
|
||||
**Next run:** hold Ⓐ longer than 0.12 s, **confirm the `[RE-INPUT]` line appears**,
|
||||
retry if it does not, then detect the submenu **by change** rather than by timer —
|
||||
a submenu load may also pass through a `pgloading_*` screen, so "different from the
|
||||
menu" is the signal, not a fixed wait.
|
||||
@@ -32,6 +32,12 @@ OUT = sys.argv[1]
|
||||
WAIT = float(sys.argv[2]) if len(sys.argv) > 2 else 420
|
||||
W, H = 1280, 720
|
||||
NEED, CEIL, HOLD = 500, 2500, 12
|
||||
# 🔴 The first version TIMED the title->menu transition (tap, wait 8 s, assume).
|
||||
# It was still on the title 8 s later -- glyph 714, the plate's pulse trough --
|
||||
# so the second tap performed the transition and no submenu was ever reached.
|
||||
# The menu has its own signature: 327 (`live-main-menu.png`), far below the
|
||||
# plate's 714..1520. Detect it, the way the title is detected.
|
||||
MENU_LO, MENU_HI, MENU_HOLD = 250, 420, 6
|
||||
PAD = os.path.join(os.path.dirname(os.path.abspath(__file__)), "pad.py")
|
||||
|
||||
|
||||
@@ -73,11 +79,16 @@ while True:
|
||||
streak = streak + 1 if NEED <= c <= CEIL else 0
|
||||
if streak >= HOLD:
|
||||
tap(); mark = time.time(); phase = "menu"
|
||||
elif phase == "menu" and time.time() - mark > 8:
|
||||
elif phase == "menu":
|
||||
streak = streak + 1 if MENU_LO <= c <= MENU_HI else 0
|
||||
if streak >= MENU_HOLD:
|
||||
Image.fromarray(a.astype(np.uint8)).save(f"{OUT}/menu.png")
|
||||
print(f"[{el:7.1f}s] menu captured (glyph {c}) — pressing A into a submenu", flush=True)
|
||||
tap(); mark = time.time(); phase = "submenu"
|
||||
elif phase == "submenu" and time.time() - mark > 10:
|
||||
print(f"[{el:7.1f}s] MENU detected (glyph {c}) — pressing A into a submenu", flush=True)
|
||||
tap(); mark = time.time(); phase = "submenu"; streak = 0
|
||||
elif time.time() - mark > 60:
|
||||
print(f"[{el:7.1f}s] menu never detected (glyph {c}) — retapping", flush=True)
|
||||
tap(); mark = time.time()
|
||||
elif phase == "submenu" and time.time() - mark > 12:
|
||||
Image.fromarray(a.astype(np.uint8)).save(f"{OUT}/submenu.png")
|
||||
print(f"[{el:7.1f}s] submenu captured (glyph {c}, mean {a.mean():.1f})", flush=True)
|
||||
break
|
||||
|
||||
Reference in New Issue
Block a user