tools: launch_mission reaches Stage 02 flight unattended again - wait for screens, not seconds
Two separate reasons the scripted route stopped, both measured rather than guessed: * LOAD -> READY ROOM is not 28 s. Both runs on 2026-08-23 overran it, so the next press was eaten by the transition and the run ended up in OPTIONS once and BRIEFINGS once. wait_screen.sh now waits for the screen, with an optional --tap that clears a dialog the caller cannot know about (a freshly restored profile inserts "Auto-Save is active. OK?" here). * The READY ROOM is DRAWN long before it is USABLE: it comes up with a "Preparing to Sortie" spinner and TAKE OFF greyed out. The two states differ by 1.7 units of blue whole-image, so screen_id.py cannot separate them and should not try. take_off_armed.py tests the label instead: 0.0000 bright pixels while preparing, 0.1633 once armed, on three captures from two runs. It carries its own position check - the always-enabled BRIEFINGS label below reads 0.1027 in all three, to four decimals, so if that reference is dark the boxes are off the labels and the answer is "unknown", not a confident wrong one. screen_id.py gains a "readyroom" class from the same measurements; nothing else reclassifies. Verified end to end and unattended: boot -> title -> LOAD GAME -> slot 01 -> READY ROOM -> TAKE OFF -> "IN FLIGHT at 34s", pilot bound and engaging. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
This commit is contained in:
BIN
docs/re/captures/readyroom-take-off-armed.png
Normal file
BIN
docs/re/captures/readyroom-take-off-armed.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 1.0 MiB |
BIN
docs/re/captures/stage02-inflight-unattended-launch.png
Normal file
BIN
docs/re/captures/stage02-inflight-unattended-launch.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 446 KiB |
@@ -69,8 +69,33 @@ tap A; sleep 8 # save list, slot 01 preselected
|
||||
tap A; sleep 4 # "Load game?" -- cursor starts on NO
|
||||
step up
|
||||
tap A
|
||||
sleep 28 # -> READY ROOM
|
||||
shot "lm-readyroom.png"
|
||||
# NOT a fixed sleep. LOAD -> READY ROOM took longer than 28 s in both runs on
|
||||
# 2026-08-23, so the next press was eaten by the transition and the run ended up
|
||||
# in OPTIONS (once) and BRIEFINGS (once); and a freshly restored profile inserts
|
||||
# an extra "Auto-Save is active. OK?" dialog here, which the `--tap A` clears
|
||||
# without the script having to know whether it is there.
|
||||
# KNOWN LIMIT, stated rather than hidden: those blind A taps are safe only
|
||||
# because everything before this point still lands. If the save-list presses
|
||||
# ever drift and the run is still sitting on the MAIN MENU here, a blind A would
|
||||
# start a NEW GAME. The real fix is to wait for each screen on the route rather
|
||||
# than only for this one; only this transition has actually been measured to
|
||||
# overrun, so only this one is waited for.
|
||||
"$SD/wait_screen.sh" readyroom 300 --tap A \
|
||||
|| { echo "NEVER REACHED READY ROOM"; exit 1; }
|
||||
# ...and the READY ROOM is DRAWN before it is usable: it comes up with a
|
||||
# "Preparing to Sortie" spinner and TAKE OFF greyed out, for tens of seconds.
|
||||
# Pressing during that does nothing, and the press after it then lands on the
|
||||
# wrong item. The two states are the same image to within 1.7 units of blue, so
|
||||
# they need the label test rather than screen_id.py.
|
||||
armed=0
|
||||
for _ in $(seq 1 60); do
|
||||
shot "lm-readyroom.png"
|
||||
if python3 "$SD/take_off_armed.py" "$SHOTS/lm-readyroom.png" >/dev/null; then
|
||||
armed=1; break
|
||||
fi
|
||||
sleep 3
|
||||
done
|
||||
[ $armed -eq 1 ] || { echo "TAKE OFF NEVER ARMED"; exit 1; }
|
||||
|
||||
if [ "$MODE" = "--hangar" ]; then
|
||||
step down; step down # BRIEFINGS -> HANGAR
|
||||
|
||||
@@ -23,7 +23,7 @@ Measured on known-good captures of each screen (1280x720, no menu bar):
|
||||
movie green 0% white varies no green at all
|
||||
|
||||
Usage: screen_id.py <png> [--json]
|
||||
Prints one of: title | menu | flight | other, plus the features.
|
||||
Prints one of: title | menu | readyroom | flight | other, plus the features.
|
||||
"""
|
||||
import json
|
||||
import subprocess
|
||||
@@ -59,8 +59,20 @@ def classify(f):
|
||||
if f is None:
|
||||
return "none"
|
||||
# Flight first: the HUD paints far more green than any menu.
|
||||
# (READY ROOM is checked before the menu rule because it is also dark-ish
|
||||
# and strongly blue, and would otherwise be swallowed by it.)
|
||||
if f["green"] > 0.004:
|
||||
return "flight"
|
||||
# The READY ROOM is far the brightest blue of any 2D screen: measured
|
||||
# (12.7, 36.5, 133.5) on two independent captures, byte-identical because
|
||||
# the screen is static. The next-bluest thing on the load path is the save
|
||||
# list at (16.7, 41.7, 81.0), and the mission briefing is cyan rather than
|
||||
# blue at (4.6, 47.7, 54.5), so a floor of b > 110 separates all three with
|
||||
# room to spare. Worth having as its own class: `launch_mission.sh` used a
|
||||
# fixed `sleep 28` for this transition and its next press was swallowed,
|
||||
# which put one run in OPTIONS and the next in BRIEFINGS.
|
||||
if f["b"] > 110 and f["r"] < 40 and f["b"] - f["g"] > 60:
|
||||
return "readyroom"
|
||||
# The main menu is dark and strongly blue. TWO measured signatures, not one:
|
||||
#
|
||||
# white 3.5% mean (25, 38, 70) the 2026-07 reference at the top
|
||||
|
||||
53
tools/re-capture/take_off_armed.py
Executable file
53
tools/re-capture/take_off_armed.py
Executable file
@@ -0,0 +1,53 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Is the READY ROOM's TAKE OFF item ARMED, or still `Preparing to Sortie`?
|
||||
|
||||
`screen_id.py` answers "which screen", by whole-image statistics, and that is the
|
||||
right contract for it — but it cannot answer this, because the two states of the
|
||||
READY ROOM differ by a spinner and a greyed word and are otherwise the same
|
||||
image: measured (13.4, 38.2, 135.2) while preparing against (12.7, 36.5, 133.5)
|
||||
when armed. Waiting on that 1.7-unit difference would be superstition.
|
||||
|
||||
The disabled state is unambiguous where it actually shows: the TAKE OFF label has
|
||||
**no** pixel above 170 while preparing, and **16.3 %** of its pixels above 170
|
||||
once armed. Measured on three captures from two independent runs.
|
||||
|
||||
This does use fixed boxes, which `screen_id.py`'s header warns about — so it
|
||||
carries its own check: the BRIEFINGS label below it is always enabled, and reads
|
||||
**0.1027 bright in all three captures**, to four decimal places, across those two
|
||||
runs. If that reference is not bright, the boxes are not on the labels and the
|
||||
answer is `unknown` rather than `not armed` — which is the failure mode the
|
||||
warning is about.
|
||||
|
||||
Only meaningful ON the READY ROOM — ask `screen_id.py` first. The MAIN MENU has
|
||||
its own text in these boxes and reads `armed` (0.1342), which is not a bug in
|
||||
this test but the reason it is not a screen classifier.
|
||||
|
||||
Usage: take_off_armed.py <png> -> prints armed|preparing|unknown
|
||||
exit 0 only when armed
|
||||
"""
|
||||
import sys
|
||||
from PIL import Image
|
||||
|
||||
TAKE_OFF = (548, 140, 745, 182)
|
||||
BRIEFINGS = (548, 196, 760, 238) # always enabled — the position self-check
|
||||
BRIGHT = 170
|
||||
|
||||
|
||||
def bright_fraction(im, box):
|
||||
px = list(im.crop(box).getdata())
|
||||
return sum(1 for v in px if v > BRIGHT) / len(px)
|
||||
|
||||
|
||||
def state(path):
|
||||
im = Image.open(path).convert("L")
|
||||
ref = bright_fraction(im, BRIEFINGS)
|
||||
if ref < 0.04:
|
||||
return "unknown", 0.0, ref
|
||||
tgt = bright_fraction(im, TAKE_OFF)
|
||||
return ("armed" if tgt > 0.04 else "preparing"), tgt, ref
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
st, tgt, ref = state(sys.argv[1])
|
||||
print(f"{st} take_off={tgt:.4f} briefings_ref={ref:.4f}")
|
||||
sys.exit(0 if st == "armed" else 1)
|
||||
57
tools/re-capture/wait_screen.sh
Executable file
57
tools/re-capture/wait_screen.sh
Executable file
@@ -0,0 +1,57 @@
|
||||
#!/usr/bin/env bash
|
||||
# Wait until screen_id.py reports <class>, instead of sleeping a guessed number
|
||||
# of seconds.
|
||||
#
|
||||
# Written because `launch_mission.sh` allowed a fixed 28 s for LOAD -> READY
|
||||
# ROOM and neither run on 2026-08-23 was ready in time: the next press was
|
||||
# swallowed by the transition, which left one run in OPTIONS and the next in
|
||||
# BRIEFINGS, and `wait_flight.sh` then tapped A into the wrong screen for five
|
||||
# minutes. See docs/re/dynamic-re-state-restore.md.
|
||||
#
|
||||
# `--tap BTN` presses BTN every 6 s *until the class is first seen* — that is
|
||||
# what clears a dialog the caller cannot know about, such as the
|
||||
# "Auto-Save is active. OK?" that a freshly restored profile puts up. Tapping
|
||||
# STOPS at the first match, because on most screens the button that dismisses a
|
||||
# dialog also leaves the screen you are waiting for.
|
||||
#
|
||||
# Two consecutive matches are required, so a single frame caught mid-fade does
|
||||
# not count as arrival.
|
||||
#
|
||||
# Usage: wait_screen.sh <class> [timeout_s] [--tap BTN]
|
||||
set -u
|
||||
CLASS="${1:?usage: wait_screen.sh <class> [timeout_s] [--tap BTN]}"
|
||||
TIMEOUT="${2:-180}"
|
||||
TAP=""
|
||||
[ "${3:-}" = "--tap" ] && TAP="${4:-A}"
|
||||
export HOME=/sylph-home/re
|
||||
DISP="${DISPLAY:-:98}"
|
||||
SD="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
# Same liveness rule as skip_intro.sh/wait_flight.sh: a failed screenshot must
|
||||
# not be indistinguishable from "the screen is not there yet".
|
||||
alive(){ ps -o pid=,stat= -C xenia_canary 2>/dev/null | awk '$2 !~ /^Z/ {print $1}'; }
|
||||
|
||||
DEADLINE=$(( SECONDS + TIMEOUT ))
|
||||
hits=0; seen=0; last_tap=0
|
||||
while [ $SECONDS -lt $DEADLINE ]; do
|
||||
rm -f /tmp/ws.png
|
||||
if ! screenshot /tmp/ws.png >/dev/null 2>&1 || [ ! -s /tmp/ws.png ]; then
|
||||
xdpyinfo -display "$DISP" >/dev/null 2>&1 \
|
||||
|| { echo "DISPLAY LOST at ${SECONDS}s"; exit 3; }
|
||||
sleep 2; continue
|
||||
fi
|
||||
[ -n "$(alive)" ] || { echo "EMULATOR GONE at ${SECONDS}s"; exit 4; }
|
||||
got="$(python3 "$SD/screen_id.py" /tmp/ws.png | awk '{print $1}')"
|
||||
if [ "$got" = "$CLASS" ]; then
|
||||
seen=1; hits=$(( hits + 1 ))
|
||||
[ $hits -ge 2 ] && { echo "$CLASS at ${SECONDS}s"; exit 0; }
|
||||
else
|
||||
hits=0
|
||||
if [ -n "$TAP" ] && [ $seen -eq 0 ] && [ $(( SECONDS - last_tap )) -ge 6 ]; then
|
||||
python3 "$SD/pad.py" tap "$TAP" 0.3
|
||||
last_tap=$SECONDS
|
||||
fi
|
||||
fi
|
||||
sleep 2
|
||||
done
|
||||
echo "NO $CLASS within ${TIMEOUT}s (last seen: ${got:-none})"; exit 1
|
||||
Reference in New Issue
Block a user