Following the real stage record (not the _Test template dumped earlier) reaches
the whole mission-parameter layer, all of it in the same self-describing IDXD
container as the squadron roster.
The big one: a stage is divided into Phase_N blocks -- three for Stage 02, each
with its own map path, map mesh, asteroid definition and background -- and
Route_S<NN>.tbl holds the arrival paths, with records named
Route_<squadron>_p<phase><kind>
tying a UnitGroup squadron id to a phase and to a time-stamped keyframed path of
(time, quat x4, pos x3). Route_ADN101_p1F is 3 frames at t = 0, 20, 30. The
identity len(fields) == FrameCount * 8 + 1 holds for 1449 of 1449 route records
across the 28 stages that have one, and 16/16 for FormationSet_S02.
Also decoded: SUBObjectiveSettings (per-objective bonus points by difficulty,
unlock item id, HUD strings) and AIParams (34 profiles, firing/guard/muster/
counter ranges plus 14 manoeuvre weights for Squad-type AI). The AIParams
numbers are exact original values from static RE and are portable as they are.
Adds tools/re-capture/stagetbl.py, which resolves a stage record by content and
can --follow every table it names, and commits two dumps as evidence.
Refuted and kept: the eight-value keyframe is the common case, not universal.
Formation_Fleet_01 has FrameCount=1 with 136 positional fields and
Formation_Fleet_02 has FrameCount=8 with 32, so a parser must not assume the
stride.
Corrects stage-definition-table.md, which was written from the _Test template
and is missing EnumerateSubobjective, EnumerateAIParams, BackGroundID and the
WingmanIconID fields the real record carries.
Not settled: what advances a phase -- the stage declares Phase_1..3 and routes
are phase-tagged, but nothing static says what ends one. That is a question for
the oracle, not for more static reading. Also open: the route-name kind letters
F/S/A/M/B, what activates a sub-objective, and StageMessageSet_S<NN>.tbl, which
does not resolve in GP_MAIN_GAME_E.pak.
stage\UnitGroup_S<NN>.tbl is now fully readable. A squadron record is Count
member tuples -- (unit model, message set, n, identity/nameplate) -- followed by
five named fields: Count, SideID, AIID, FormationID, DisableInterval. The
property entries carry their own field names inline, so the tag hash never has
to be inverted.
Two independent self-checks validate it corpus-wide, 1160/1160 each across all
28 stage tables on the disc:
- the length identity len(fields) == Count * 4 + 5, which is what pins the
member-tuple width at 4 and the named-field count at 5;
- agreement with the file's own Enumerate_Squadrons roster, which maps record
key to squadron id independently of the per-record string offset.
Adds tools/re-capture/unitgroup.py (pure static, runs no emulator) with a
--all --check self-check mode, and commits the Stage 02 dump as evidence.
Corrections to the container layout written yesterday, all three wrong:
- the 20-byte "(tag, 0, 0, count, size) section header" does not exist. It
was the file's last 16-byte record followed by a plain npool word. The
corrected layout is uniform across all 28 files; the old one failed on 9.
- squadron ids do not use a separate string base. Every offset in the file is
relative to the one string pool. The earlier "109 of 111" score was an
artefact of the uniform 7-byte id stride and had silently shifted every
name by three entries, which is why 17 TC*-named squadrons came out as
SideID=ADAN. The roster record refuted it outright.
- the roster is not always the last record; 9 stages put it elsewhere, so it
is found by its missing Count.
Refuted and kept: the 4-byte record key is not the squadron id's name hash
(0 of 112).
Not settled: what the key encodes, the member tuple's third field n, and where
the arrival interval values live. DisableInterval is only a per-squadron flag
(Yes for 31 of 1160); the durations, triggers and arrival positions are not in
this file. Formation_*.tbl and EnumSquadron_Test.tbl are next.
With the per-entity searches refuted at word and bit level, the question became
which object owns the counter. Sampling +-0x200 around it across a 4->8
transition: the control interval moved 0 of 256 words, and the step moved nine -
the counter plus four words holding ASCII '4' -> '8' NUL-padded, and four
pointers into 0xbcad2xxx that swap with them. Read live at HUD 008, all four
character slots hold '8'.
So the neighbourhood is the HUD's rendered text for this counter, which reframes
the address: it is the HUD widget's value rather than "the mission's own
objective counter" as this file called it.
Recorded against that, because it is already measured: there is no separate
mission-side copy moving on the same step. ob_hunt scans all of guest memory and
requires a match across two transitions, and it left exactly one address.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Run 8 died 54 s into its boot and this time bash named it: "176880 Killed" on the
run-canary line, which is SIGKILL. So the third failure mode is not an internal
fault - something outside the process is killing it.
And it is still not the OOM killer. Checked immediately after: oom_kill remained
0 and the allocation-stall counter did not move from 4421, so during run 8 the
cgroup never reached its limit (5.35 GB of 7 GiB), and the host had 13.8 GB
available. Two kills, no OOM evidence either time.
Rather than keep guessing after the fact, freeze_watch.sh now samples host
MemAvailable, cgroup memory.current and oom_kill on every poll and dumps the last
five samples when the process disappears - so the next occurrence carries its own
contemporaneous reading.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
An agent's commands run through a wrapper shell whose command line contains the
pattern being searched for, so `pgrep -f 'pilot\.py'` matches the shell running
that very pgrep, and `pkill -f 'fly_session|pilot\.py'` kills it mid-script. In
one session this killed two running scripts and twice made an "is it already
running?" guard answer yes because it had found itself. The [p]ilot bracket trick
does not help, because the real invocation appears on the wrapper's command line
too.
Recorded with what works instead: pgrep -x on the exact name, or ps -eo args
filtered against the snapshot-bash path that every wrapper carries, then kill by
explicit pid.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
v2 counts every KeWaitForSingleObject call per thread per second, tracks how many
distinct objects it saw, and logs the last result. Its HEALTHY baseline is a
result on its own: over a full 22-minute run the main thread cleared 500 calls/s
in 224 separate windows, peaking at 1235 calls/s over up to THIRTEEN distinct
objects, and the last result was X_STATUS_SUCCESS in all 314 windows. Not one
timeout.
So the game normally does hundreds of successful waits a second across many
objects - exactly the blind spot v1 could not see, and the reason a timeout-streak
counter reported the same single poller whether the game was frozen or healthy.
Stated as a consequence rather than a triumph: 500/s is NOT self-selecting, since
the main thread clears it constantly, so the freeze signal has to be a different
shape - a thread far above 1235/s, a new thread, or a window whose result is not
SUCCESS. That still needs a frozen sample; run 5 ended in GAME OVER at ~22 min
without freezing.
freeze_watch.sh now summarises the rate probe per thread when it captures.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Catching the in-mission freeze by hand costs a tool call every 25 s, and it
arrives anywhere between ten seconds and never - roughly half the runs that reach
flight freeze and the other half do not. freeze_watch.sh polls frozen.py, checks
the flight HUD is still on screen (the GAME OVER screen ANIMATES, so a run that
ended there is not a freeze and must not be scored as one), and on detection
snapshots what the stuck-wait probe has said.
Taking the probe snapshot at the moment of detection matters: the comparison
against the healthy-run baseline - one thread polling one Event at BE56BB5C - is
then made from the same instant rather than reconstructed afterwards from a log
that kept growing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Both exist for one diagnostic. KeWaitForSingleObject is declared kHighFrequency,
so it is silent even at LOG_LEVEL=3 unless
--log_high_frequency_kernel_calls=true is passed - and that flag writes ~157 MB
in ten minutes and slows the emulator so far that the title had still not
appeared when skip_intro's fixed 600 s budget ran out. The run was then scored
"BOOT FAILED" when it was only slow.
EXTRA_FLAGS passes one-off diagnostic cvars without making them defaults;
SKIP_INTRO_TIMEOUT lets a deliberately-slowed run have the time it needs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The gdb route works, and the recipe is written down: run-canary execs $XENIA_BIN,
so a wrapper that execs "gdb --args <real binary>" keeps the lockfile, the flags
and the process name (gdb forks and execs the real binary, so ps -C xenia_canary
still finds the inferior) while satisfying ptrace_scope=1 by being the parent.
The Release binary is not stripped - 26595 symtab entries - so frames have names.
The handle SIGSEGV/SIGBUS/SIG32-35 lines are mandatory: xenia uses SIGSEGV for
guest memory watches and the RT signals for thread suspend.
A run froze after ~4 minutes of flight, screen still "flight" rather than GAME
OVER, and all 79 threads had backtraces. EVERY ONE is in a wait - guest threads
in KeWaitForSingleObject / NtWaitForSingleObjectEx / SelfSuspend, the GPU command
processor parked idle, the main thread in poll().
It is nevertheless burning 1253 ticks per 10 s: 403 in the TimerQueue thread
(nanosleep inside TimerThreadMain) and 290 + 274 in two guest threads that the
backtrace shows blocked in KeWaitForSingleObject. A thread genuinely blocked
cannot burn 28% of a core, so those two are CYCLING - a timed wait that expires
and is re-entered - with the timer thread servicing them hot. Three samples
minutes apart show identical frames.
That refines the earlier "the guest is spinning, not deadlocked": the CPU burn is
real but it is in the WAIT PATH inside the kernel layer, not in guest code. The
shape is an event that never gets signalled.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The freeze's named next step is more expensive than it looked, and the reasons
are recorded before anyone starts it: xenia's stack walker is a TWENTY-LINE STUB
on POSIX (Create logs "unimplemented" and returns nullptr), so
ThreadDebugInfo::guest_pc is never filled here; PPCContext carries no live PC
either. A reverse host->guest map is buildable - the code cache already learns
the mapping in OnCodePlaced - but that plus a way to sample another thread's RIP
is a real emulator feature, not a patch.
The cheap route that does exist: gdb is installed and ptrace_scope is 1, so
attaching to a running emulator is refused but launching it UNDER gdb is not -
run-canary execs $XENIA_BIN, so a wrapper that execs "gdb --args <real binary>"
keeps the lockfile and flags and makes gdb the parent. That would separate
"spinning in guest JIT code" from "spinning in a xenia loop", which is the fork
this is stuck on. Not attempted yet.
ob_bitflag now locates the counter itself from the three addresses measured so
far, instead of refusing when the default one is wrong. That costs no
transitions, and with roughly half of all runs ending early, transitions are the
scarce resource.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
ob_bitflag.py is the follow-on the word-level refutation named: for every 4-byte
offset in the window and every one of its 32 bits, count how many entities have
it set, keep the pairs whose count is exactly the counter, and require them to
match again after a transition. Both polarities, since an objective could be
marked by a bit that is CLEAR on it.
Three runs, no verification, and the reasons are recorded: run 1 gave 187 + 33
candidates at counter 4 and then reported "the counter never moved" for 700 s -
about a mission that had ENDED in GAME OVER partway through; run 2 hit the same
dead mission; run 3 had the counter at a different address (the guard refused,
correctly) and then froze after one filter.
The hole is closed. frozen() asks whether the guest is ANIMATING, and the GAME
OVER screen animates happily - mean colour (114,22,63) - so every liveness check
passed while the mission was over. frozen.in_flight() classifies the screen with
screen_id, and ob_hunt/ob_flag/ob_bitflag now abort with NO LONGER IN FLIGHT.
That is the second confident negative in this investigation that was really about
a dead world, so the rule is written down: before believing "X never happened",
show that the thing that would produce X was still running.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
With the pilot finally shooting, the experiment completed both halves. Sample A
at counter 12 over 120 entities gave 2 offsets where exactly 12 entities share a
value; the counter then went 12 -> 11 and NEITHER survived. So within +-0x400 of
an entity's position triple there is no 4-byte word whose shared-value population
tracks REMAINING OB.
The limits are recorded as part of the result, because they bound it: the test
asks which entities share an EXACT 32-bit value, so a single bit ORed into a word
that also carries health or a timer would never show up - a bit-level version of
the same differential is the follow-on. Anything outside the window, or on
entities that entities2 cannot see (it types by position CHANGING, so stationary
objectives are invisible), is untested too, and the populations differed a lot
between samples - 120 against 194.
Separately: REMAINING OB went 12 -> 11, the first decrement of this whole
investigation, while pilot.py logged 411 fire=1 samples and the HUD reached YOU
KILLED WARPLANES 0003. Stated carefully - it does NOT show the counter counts
kills, since an earlier run had the hostile population fall by a third with no
movement; it shows some kills close something the counter tracks.
Two robustness fixes: ob_flag retries an empty entity sample (one void run was
caused by exactly that), and Pad releases everything on interpreter exit - a
file-backed pad PERSISTS after its writer dies, so a tool killed mid-press would
leave a button held and the game would walk through menus on its own.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Measured, not argued: command a 45-degree-off-the-nose error and watch whether it
shrinks, both axes, both sides, two pulse widths, with the opposite sign as a
control. Yaw's sign is correct (45 -> 11.3/32.7 at 0.6 s, 41.6/33.8 at 1.2 s).
Pitch's is inverted - the pilot's own sign GREW the error every time
(48.3/55.6/70.7/91.7) and the opposite shrank it every time (30.7/41.5/15.7/9.6).
A method artefact is recorded because it gave the opposite answer first: a 3 s
full-deflection pulse overshoots a 45-degree error so far that BOTH signs look
wrong (45 -> 164 and 45 -> 178). A long pulse cannot answer a sign question.
Verified against the game rather than by inspection. Before: fire=1 in 0 of 13521
samples, |aim yaw| pinned at 90.0, target 36-43 km away. After: 43 of 1732, aim
down to 2.3 degrees, range median 6.3 km, and the HUD's own ammunition counters
moving - NOSE BM 06000 -> 05723, MAIN MPM 00300 -> 00298.
Also fixed a leftover of the same FIFO era: pilot.py called pad.f.write("tap A
90") for the target-select double tap, which raised AttributeError once Pad
stopped having an `f`. Pad gained tap()/dpad(); ctrl_probe.py and target_probe.py
still use pad.f and now say so in place.
Still open: YOU KILLED is 0000 after 250 s of firing and REMAINING OB is still
012. The craft shoots, closes and selects; whether it destroys anything is next.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
flight_probe.Pad opened /tmp/sylph-vgamepad.fifo, "the vgamepad server's FIFO".
That server was removed when the uinput pad was replaced by Canary's --hid=file
driver, because a uinput device is not namespaced and scripted presses leaked to
the host's desktop. The FIFO is now an ordinary 91-byte file nothing reads, and
/tmp/xenia_pad.txt - the file the emulator polls - was 0 bytes while the
"autopilot" was flying. So every axis, trigger and button from pilot.py,
autopilot3.py, aim_probe.py and flight_probe.py went nowhere, silently.
The corpus already carried this trap for the SHELL scripts
(canary-scripted-input-traps.md, "every call here failed silently"). This class
was missed, and every flight tool imports it.
Verified against the oracle rather than by inspection, on runs confirmed
animating at both ends of every phase: before, full stick produced 0.00 degrees
of heading change over 4 s while the ship travelled 350-735 units, and the
attitude matrix at pos-0x70 was byte-identical; after, LX=-1 turns 12.72 degrees
and LX=+1 swings the flight direction from [1,0,0] to [0.13,-0.14,-0.98], with
the matrix moving 0.44/0.31 under stick and 0.0000 at neutral.
That also REFUTES the "stale attitude matrix" suspicion from the earlier pass -
pos-0x70 is live and tracks the ship; it only looked dead because nothing was
turning the ship.
Still open and said plainly: with the pad fixed a 150 s pilot run still fires 0
times, with |aim yaw| still exactly 90.0 and the target 36-43 km away. Steering
works now, so what is left is the stick SIGN against the pilot's error convention
or a target selection that commits to something too far to close. Both are
finally testable.
findrot_global/findself/findspeed/selfstate still write to the dead FIFO and now
say so in place; they are not repaired because none has been re-run since.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
fire=1 appears in 0 of 13521 samples. The gate that closes is measured rather
than guessed: over the 3004 samples that had a target, |aim yaw| is EXACTLY 90.0
degrees every time, which is the `if ez < 0` branch in sticks() - the target is
astern - with pitch near 180 and a range that grows 22km -> 49km and plateaus.
The craft flies away from what it is chasing for fifteen minutes and the turn
never completes.
What is NOT established is why, and the attempt is withdrawn rather than kept:
aim_probe.py reported the forward vector pinned at [-1,0,0] with 0.00 deg/s under
neutral, full-left and full-right stick, which looks like a stale attitude
matrix - but the guest had FROZEN partway through the probe, confirmed after the
fact by frozen.py and by the player position being identical across 3 s. A dead
world holds every matrix still. The probe is committed because it is the right
experiment; its numbers are not evidence.
One confusion resolved: today's entities2.py "0 moving triples" bind failures are
the freeze, not a tool defect - moving() types entities by position CHANGING, so
a frozen world yields nothing by construction.
live_delta.py gains a per-1MB-region summary; a flat list is useless at 236000
hits.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
live_delta.py exists for the freeze: a frozen run is guest code spinning, and
spinning code waits on guest memory, so the words still changing while frozen
should be the loop's own bookkeeping and whatever it polls. Measured control on a
RUNNING game, so the frozen number has something to be compared against:
1863296 words change in 4 s, 2.08% of the 357.7 MB of data extents. The sparse
4.6 GB costs nothing - the walk is over SEEK_DATA extents.
The freeze did not reproduce this run, so the frozen half of that measurement is
still owed.
ob_flag.py: an unreadable HUD frame is now retried rather than counted as a
mismatch. ob_read returns None on a frame it cannot read - an explosion across
the plate, a flash - and the sanity check treated that as "the address is wrong"
and aborted a perfectly good run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The control this file never had: a run with the Kernel channel on, analysed WHILE
STILL FLYING, has 2738 refused resumes on one pair - more than the frozen run's
1171. The target's own lines show why. The game runs a self-suspending worker
(NtSuspendThread on itself, a manager thread resumes it, thousands of times), and
a self-suspended thread is not host-suspended, so the host Resume legitimately
returns false EVERY cycle: 3115 refusals against 3115 resumes.
The error is named rather than buried: the warning's commit says "~7 times in a
normal boot" and this file generalised that from boot to gameplay, where the
number is thousands. The 150x anomaly was an artefact of the baseline. The
zero-CPU threads go with it - the healthy run has four of those too.
What the instrumented reproduction DOES establish is sharper than the lead was.
The last kernel event in 690000 lines is "Thread F8000204 self-suspending", with
self-suspends 3116 against resumes 3115 - but the resumer never issues another
NtResumeThread at all, so nothing was dropped in flight; every thread stopped
together. And the guest is SPINNING, not deadlocked: over 10 s while frozen the
main thread is in state R gaining 409 ticks and guest threads gain ~680 in total
while making not one kernel call. So it is guest code waiting on something in
guest memory, and the next question is which guest PC.
Two corrections fall out: "the log stopped growing" is not a freeze detector (it
goes quiet for 25 s in normal flight), and 0xbdb59668 held the counter again -
4 of 6 runs now.
freeze_report.py makes the analysis repeatable, and refuses to answer "did this
thread ever run" when the Kernel channel was off rather than reporting a false NO.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
A Stage 02 run froze at TIME 01:02 with a radio line caught mid-word. The
emulator was alive at ~200% CPU with its main thread in state R, two screenshots
six seconds apart were byte-identical, and the log ended in a spin: 1171 of the
run's 1200 "host resume was refused" lines are the single pair F80002AC ->
F8000240, starting at the line immediately after F80002AC is created, and the log
never grows again. The commit that added that warning records what normal looks
like - about 7 in a whole boot - so this is a 150x anomaly on one pair rather
than noise. F8000240 itself appears exactly once outside the spin, at creation,
and calls nothing.
Eleven of the frozen process's 79 host threads have zero CPU, four of them
consecutive late-created guest threads - the same signature as the lost resume
that c1b57f93b fixed for the title screen. That fix IS in this build, so either
there is a second window in that race or this only looks alike.
The inference is named as one: nothing here maps a guest handle to a host tid, so
"the zero-CPU threads are the ones being resumed" is a reading of two consistent
observations. And the refusals could equally be the game's reaction to a worker
stuck for another reason - log_mask=13 has the Kernel channel disabled, so not
one of F8000240's waits is visible. The next experiment is written down
concretely: reproduce with LOG_MASK=12 LOG_LEVEL=3 and map the handle to a tid.
Also caps ob_hunt's survivor listing at 40 - an aborted run printed all 21482 and
buried the line that mattered.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Three things from one Stage 02 run.
The address recurs a third time: HUD=4 RAM=4 at 0xbdb59668, so 3 of the 5 runs
measured put the counter exactly there.
The counter is NOT a live class head-count. With the counter at 4 the typed
entity list was 8 attackers, 7 friendly Delta Sabers, 7 turrets and the player -
no class has 4 members and no pair of them sums to 4. That sharpens the corpus's
existing "012 against 118 live ADAN" note from "not the hostile count" to "not
the count of any class this enumeration can see".
The flag experiment itself proves nothing, and why is the useful part. It found
20 offsets where exactly 4 of 23 entities agree, then reported "the counter never
moved" for 600 s. The guest had stopped advancing ten seconds into flight:
pilot.py logged 724 s of identical speed/yaw/pitch, and two screenshots six
seconds apart were byte-identical, max delta 0 over 863325 pixels - while
screen_id said "flight", the emulator burned 212% CPU and every liveness check
passed. So that was a fact about a dead world. Withdrawn along with it: the claim
in ob_session.sh that the counter climbs on its own in the first minutes, which
one advancing run supports and this one cannot.
frozen.py makes it a single call, checked in both directions (0 on the frozen
pair, 254 on two frames of a live run), and ob_hunt/ob_flag now say GUEST FROZEN
rather than waiting out their timeouts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
One filter to select (004 -> 008, 35897 -> 7), a second on a transition it was
NOT selected by (008 -> 012, 7 -> 1), leaving exactly one address; then three
live paired readings, RAM/screenshot/RAM, all agreeing with the HUD.
The address is the same one run 1 reported. That does NOT reverse yesterday's
refutation and the entry says so explicitly: runs 2 and 3 read a hard 0 there on
an allocated page while the HUD counted, so "it is there in every run" stays
refuted. What is withdrawn is the stronger claim that the number was meaningless
- it recurs exactly, in 2 of the 4 runs measured, and run 3's amber candidate
0xbdb49668 sits one 64 KB page below it at the identical page offset 0x9668. The
practical rule is therefore: try 0xbdb59668, check it against the HUD, re-scan
when it reads 0.
Why run 3 failed and run 4 did not is also recorded, because it is a method
lesson rather than luck: the evidence was always in the first four minutes of the
stage, and the earlier runs simply could not look often enough - every HUD
reading cost a human round trip, so the 008 step went by between two of them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Both hand attempts at this address managed exactly one filter and then lost the
counter, because every reading of the HUD costs a human round trip. ob_read.py
does it by template correlation over three fixed digit cells, and ob_hunt.py
uses that to run the whole method unattended: confirm the HUD on both sides of
the 0.9 s scan, then filter on each following transition, labelling the first as
the one it selected on and the rest as verification.
Two things measured rather than assumed:
* Normalisation is the point, not a nicety. The plate is translucent, so an
explosion turns it orange - on the 008 reference frame a cyan-stroke mask finds
6 of ~70 stroke columns and would silently read nothing. Per-cell mean/std
normalisation reads it correctly at 0.843.
* The accept rule is two-sided because the margin is narrow: over 12 cells from
four frames of known value the correct digit scores 0.843..1.000 and the best
WRONG digit reaches 0.789 (0 and 8 are similar outlines). So a floor of 0.80
AND a 0.05 margin over the runner-up, against a smallest observed correct
margin of 0.093. A bare threshold fitted between those two numbers would be
fitted to twelve samples.
Templates exist for 0 1 2 4 8 - the digits actually seen. Anything else reads as
"?" and callers must treat a "?" as unknown, never as a value. Rejects both
negatives tested: GAME OVER scores 0.05-0.18, the main menu 0.07-0.14.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
It tested one named pixel, (450,640), described as "inside the SHIELD bar". It is
not inside anything. `screenshot` crops to the GAME SURFACE, 1279x675, while
those constants were chosen for a 1280x720 window: measured on a frame that was
unmistakably in flight (TIME 01:18.05, REMAINING OB 004), the SHIELD bar's green
rows are 590-605 and the ARMOR bar's are 650-670, and (450,640) sits in the gap
between them reading (64,66,116) - blue.
That is why the corpus already carried "wait_flight.sh reported NEVER REACHED
FLIGHT while the game was plainly in flight" as an unexplained note, and it is
exactly the failure screen_id.py's own header warns about. Reproduced live here:
the script sat for five minutes reporting nothing while tapping A into the
cockpit every six seconds.
Now classified by whole-image statistics - screen_id.py's "flight" class, green
fraction 1.3-1.5% against <0.1% on every menu - which no crop or scale can move.
Two consecutive frames are required so an explosion in a cutscene cannot pass,
and the A tapping (which dismisses the objective card) STOPS once flight is seen,
because in the cockpit A is a weapon press and not a "continue".
Tested against the running game before being believed: 0 with "IN FLIGHT at 19s"
on the same session the old test had been failing on, and then in the unattended
launch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Two separate reasons the scripted route stopped, both measured rather than guessed:
* LOAD -> READY ROOM is not 28 s. Both runs on 2026-08-23 overran it, so the next
press was eaten by the transition and the run ended up in OPTIONS once and
BRIEFINGS once. wait_screen.sh now waits for the screen, with an optional
--tap that clears a dialog the caller cannot know about (a freshly restored
profile inserts "Auto-Save is active. OK?" here).
* The READY ROOM is DRAWN long before it is USABLE: it comes up with a
"Preparing to Sortie" spinner and TAKE OFF greyed out. The two states differ by
1.7 units of blue whole-image, so screen_id.py cannot separate them and should
not try. take_off_armed.py tests the label instead: 0.0000 bright pixels while
preparing, 0.1633 once armed, on three captures from two runs. It carries its
own position check - the always-enabled BRIEFINGS label below reads 0.1027 in
all three, to four decimals, so if that reference is dark the boxes are off the
labels and the answer is "unknown", not a confident wrong one.
screen_id.py gains a "readyroom" class from the same measurements; nothing else
reclassifies.
Verified end to end and unattended: boot -> title -> LOAD GAME -> slot 01 ->
READY ROOM -> TAKE OFF -> "IN FLIGHT at 34s", pilot bound and engaging.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
ob_sample.py pairs a live read of a guest VA with a screenshot and a crop of the
HUD counter, and reads the word again AFTER the shot so a sample whose two reads
disagree can be thrown away instead of believed -- that race is what left the
first three-snapshot filter with zero survivors.
ob_session.sh is fly_session.sh plus that sampler, because the evidence this
needs is a transition, and an unattended craft is dead in about a minute. Its
header records that launch_mission.sh does not yet finish unattended on restored
state, so nobody discovers that halfway through a boot.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The mission's objective counter is a big-endian u32 at guest VA 0xbdb59668 in a
Stage 02 run on the upstream baseline. It was selected on an 18->24 transition
and then tracked 24->23->22 against the HUD on its own - four readings, two
changes it was not filtered on.
That last point is the whole discipline here, because the first attempt failed it.
An earlier differential over 19->18 also produced exactly one candidate,
0xbc22e83c, which matched the transition it was selected on and was still wrong:
read live it held 26 while the HUD showed 017. One matching transition is not
evidence.
A second trap is recorded too: a three-snapshot filter requiring 19 -> 19 -> 18
left ZERO survivors, because the value moves between the memory copy and the
screenshot that reads it. Filtering on the next DISTINCT value instead found the
counter on the first try.
ob_scan.py carries the method: scan one snapshot, then filter the candidate set
against live /dev/shm/xenia_memory_* at each new value, so only the first pass
needs a 4.8 GB copy.
Stated plainly as unsettled: the ADDRESS is from one run and cross-run stability
is untested, so the durable result is the method rather than the number. And what
the counter counts - whether every OB-badged entity is one of them, and whether
that badge is a flag in the entity object - is the follow-on the pilot actually
needs to CHOOSE targets rather than just know how many remain.
Finding REMAINING OB in RAM means getting to the escort stage, and
launch_mission.sh exists for exactly that. It carried three of the dead patterns
this session has been clearing: six vgamepad calls (the command no longer exists,
so the entire scripted route pressed nothing), the XUID E0300000EFBEA3D4 which
has no profile behind it and therefore opens the sign-in dialog that swallows
every keystroke, and the emulator's stdout going to /dev/null. wait_flight.sh had
a fourth, in the tap that skips the launch cinematic.
Fixed, the route works far past where it used to stop: boot -> title -> LOAD GAME
-> slot 01 -> READY ROOM -> TAKE OFF -> the Stage 02 briefing map, with ACROPOLIS
labelled on it.
And it stops there: 384 dumps at PC 0x82307128 and no flight HUD within 300s. The
briefing is still animating (two shots a minute apart differ by RMSE 2986), so
the emulator is not frozen - flight simply never starts.
One discriminator worth recording: the TUTORIAL reaches flight and has run
completely crash-free, while this path crashes during the briefing, which is
where the stage content loads. That is consistent with the cache-flush shape the
crash has always had, and points at WHAT is being loaded rather than at how far
the navigation gets.
REMAINING OB is therefore still not found - the mission was never entered.
ship_capture_window.sh polls for the flight screen and presses F10 the moment it
appears rather than after a fixed sleep. One run gave three results.
First, the mission ran with ZERO crashes through t+152s - the first clean mission
run, where the three before it ended at 13243 / 11898 / 11497 - and it renders
and plays: player ship, starfield, full HUD, no dialog.
Second, the capture armed and wrote its file, so the mechanism works in-mission.
Third, and against expectation, the file holds NO ship geometry. 329KB, 181
deduped draws: 180 of them share a single vertex shader, all stride=28 vcount=3
prim=8 at full-screen coordinates, plus one full-screen quad, and not one draw
has positions outside the 1280x720 rectangle. The budget is not the limit -
kShipCaptureBudget is 8000 and only 181 distinct (vbase, WVP) pairs were seen -
and the scene was definitely drawing. The known-good capture from an earlier
session is 2.9 MB.
Fourth, a correction to the previous commit. It said the cache is "REFUTED as the
cure". Too strong: this run used the IDENTICAL complete cache as tut4 and
produced 0 crashes against tut4's 11497. What four runs support is that a
complete cache is not SUFFICIENT to prevent the storm and that run-to-run
variance dominates a 3-run comparison - not that the cache does nothing.
Next step is static: compare this capture's shape against the known-good one to
find why the 3D draws never reach CaptureShipDrawForRE.
resume_reliability.sh counts, per boot, whether the title's loader thread became
the CALLER of a kernel call. Counting boots where the press reached the title:
before the fix 1 of 5 ran the loader; after it, 5 of 5. Three of the post-fix
boots were driven far enough to classify the screen and all three reached the
MAIN MENU, each with the same shape - title at 216-252s, 40 kernel calls by the
loader, 6 ResolvePath reads. The failing boots before had zero of both, every
time.
Caveat kept in the doc: pre-fix runs were interleaved with compiles and a race is
load-sensitive, so the counts corroborate the mechanism rather than carry it.
Also fixes a self-inflicted confound. Two runs came back 'title_at_s=none' with
the emulator log ending in 'Killed', which I first read as the title detector
being too strict. It was not: an earlier copy of the script was still running
because 'pkill -f resume_reliability.sh' had matched the shell running it rather
than the script, and each of that copy's iterations kills the emulator BY NAME -
outside run-canary's lockfile. The harness now takes its own lockfile and refuses
a second copy, and the note records that pkill -f matches your own shell.
resume_reliability.sh boots N times and records, per boot, whether the title's
loader thread was created and whether it ever became the CALLER of a kernel call.
That is the thing the threading fix targets, and it is binary; "did the menu
appear" is a worse test because it needs the game to get further and the attract
loop confuses it.
Two defects fixed on the way, both found by the harness disagreeing with itself:
* is_title.py's glyph count alone still fired at 113s and 173s, during the
intro movie, and the single press was wasted there. Measured, not guessed:
on those runs NO loader thread was created at all, and the title handler
always creates one - even on the boots where it then fails to run. The title
check now also requires the frame to be STATIC, reusing the RMSE between the
two grabs 0.6s apart that the movie branch already computes, and raises the
glyph threshold to 800 (a real title measures ~1450-1520, the SQUARE ENIX
logo 0).
* `grep -ac ... || echo 0` corrupted every CSV field it guarded: grep exits 1
on zero matches, so the fallback APPENDED a second value and the field came
out "0 0".
Retrospective before/after from the logs already on disc, using ResolvePath after
the resume (logged at every verbosity, so it is comparable across all runs):
before the fix 1 of 5 boots that created a loader thread went on to load assets;
after it, 2 of 2. n is small and a proper count is running.
The stalled loader thread is a lost wakeup in Xenia's POSIX threading, fixed on
the canary branch as a60fe7d11 and written up here. A thread created suspended
publishes state_ and suspend_count_ in two separate lock scopes, and Resume()
waits only for state_ before testing suspend_count_ == 0 - so a resumer in the
gap drops the resume and the thread waits forever. The Linux XThread::Resume
discards that false, which is why the guest saw success.
On the first clean boot after the fix the loader thread is the CALLER on 20
kernel-call lines and issues 4 ResolvePath reads. Every failed boot before it had
exactly zero of both.
Stated plainly as not shown: that boots now reach the menu RELIABLY. One post-fix
boot, and it is confounded by the harness.
Which is the second half. skip_intro.sh's title test has now been wrong twice in
opposite directions: originally one absolute pixel (625,618) - a 1280x720
coordinate against the 1279x675 game surface, so it read the copyright line and
timed out with the title on screen - and then my replacement, screen_id.py, which
is too loose and called the SQUARE ENIX publisher logo "title" 151s into a boot,
spending the script's single press there. is_title.py now counts the green (A)
glyph over the whole frame: geometry-independent and specific, measured at 0
pixels on the logo and 1520 on a real title.
Two findings, the second only visible because of the first.
1. The harness has always had kernel logging switched off. log_mask DISABLES
categories (Kernel=1, Apu=2, Cpu=4, Gpu=8), so the long-standing --log_mask=13
meant Kernel+Cpu+Gpu off; kernel calls also log at Debug while log_level
defaults to Info. Seeing one needs BOTH LOG_MASK=12 and LOG_LEVEL=3, and no
boot log this project has taken ever contained a kernel call. boot_menu.sh now
takes LOG_MASK / LOG_LEVEL / EXTRA_FLAGS. A whole boot at Debug with Kernel on
is 23 MB, so the default was costing far more than it saved.
2. With that on, a captured failure shows the (A) handler doing everything right:
XamUserGetXUID(0, 7, ...)
NtCreateEvent(...)
ExCreateThread(..., entry=821748F0, ..., 00000001)
ExCreateThread Active: Thread Initially Suspended,
XThreadF80000CC (1F) Stack: 70880000-70900000
NtResumeThread(F80000CC, ...)
and the thread then never executing. Measured two independent ways: it makes
ZERO kernel calls - it appears 13 times in the log and every one is as an
ARGUMENT, never as the calling thread, while five other threads make 31905
calls after the resume - and its host thread has 00:00:00 CPU time while the
process sits at 546% and has burned 37 minutes of CPU in 6:46 wall.
A spinning thread burns CPU. This one has not run at all.
So the chain is: press delivered -> handler runs -> thread created suspended ->
resumed -> never scheduled. Input, the cache-flush crash and the game's own logic
are all excluded. A lost resume is a race, which is the first explanation that
fits the ~1-in-3 success rate.
Not settled: where the resume is lost, and no successful boot has been captured
with kernel logging to compare against.
Chasing why (A) does not advance the title turned up four harness bugs, one real
root cause, and a refutation of that root cause as the explanation.
FIXED, each verified:
* boot_menu.sh sent the emulator stdout to /dev/null, which is why none of the
rest was visible. It now keeps a log and prints its path.
* skip_intro.sh probed pixel (625,618) for the green (A) glyph - a 1280x720
coordinate. screenshot returns the 1279x675 game surface, where that point
is on the copyright line and reads (8,17,31). Symptom: a 600s TIMEOUT with
the title on screen. Now classified with screen_id.py; next boot printed
"TITLE at 239s -> A".
* skip_intro.sh no longer taps through the movies. Making that press real (the
vgamepad fix) was a REGRESSION: the boot then reached the title in 90s and
that title accepted nothing. menu_draw_capture.sh records the same finding
independently.
* boot_menu.sh signs in whichever profile exists instead of a hard-coded XUID.
ROOT CAUSE FOUND: naming a XUID with no profile behind it opens a sign-in
dialog, xam_dialogs_shown_ goes to 1, and IsUIActive() then discards every
XamInputGetKeystrokeEx. Measured 8,388,601 swallowed calls in one boot with the
pad log showing presses arriving normally throughout. Only B13EBABEBABEBABE
exists on disc; the script asked for E0300000EFBEA3D4. Signing in the real
profile takes the swallow count 8,388,601 -> 0.
AND REFUTED: that is not why the title is stuck. With a valid profile and zero
swallowed keystrokes, a single (A) on the title that ends the boot still does
nothing. The dialog bug was real and worth fixing; it is not the explanation.
Next probe is specific: --log_mask=13 records no kernel calls, so it is not even
known whether the game polls at the title. A log on the SUCCESSFUL
XamInputGetKeystrokeEx return, mirroring the one already on the swallow path,
answers it in one boot.
Trying to read a third paint order off the running game turned up two bugs in
the capture harness, both of which fail in ways that look like the game
misbehaving rather than the script being wrong.
1. `--audio` is not a cvar in this tree, and eight boot scripts passed it.
Xenia calls ShowSimpleMessageBox from ParseLaunchArguments, BEFORE logging is
initialised, so the symptom is a 10x10 window, no log, no guest memory and a
dialog that blocks on XIfEvent forever - i.e. a hang deep in the emulator.
run-canary`s own header documents this exact trap; the scripts predate it.
Removed from all eight.
2. `vgamepad` no longer exists - the uinput pad was replaced by the --hid=file
driver and pad.py - but skip_intro.sh still called it. The script runs
without `set -e`, so the call failed silently and the title branch pressed
nothing while still exiting 0. A caller was told "TITLE -> A" with the game
sitting on the title screen. It now presses through pad.py and exits 6 if
that fails.
The first bug is fixed and verified: the boot now reaches the title screen with
PRESS (A) BUTTON. The second is fixed but does NOT unblock the title - see the
next commit.
Element::rest() picked the keyframe with the largest gap to the next keyframe's
time. That reads a keyframe as a value held until the next one; it is the start
of a ramp toward it. A long gap after keyframe k means the screen spends that
time arriving at k+1, so the settled pose is at the far end of the gap.
The title wordmark zooms in over five frames and holds at (184,193) at 100% from
t=251 to t=264. The old rule picked the frame before the long gap: (179,186) at
101%, still mid-zoom.
Measured against the framebuffer capture of the running title screen, which is a
1:1 crop so frame coordinates map directly (confirmed: the copyright line lands
on row 669 in the capture and in both composites). Edge-correlated over the
wordmark box:
plateau (landed) best 0.4597 at shift (0,0)
longest dwell (old) best 0.1511 at shift (+3,+8), 0.1268 at (0,0)
The old composite scores 3x lower and only peaks after being moved, by about the
(-5,-7) that picking kf4 instead of kf5 predicts.
It also fixes six title elements the old rule rested at alpha 0x00 where the
capture plainly shows them, and pteff00.prm - the full-screen fade quad painted
last - which rested at opaque black. That was the blocker on .prm compositing.
Adds tools/re-capture/align_to_capture.py, which is how this was scored, and
turns the .prm test that deliberately asserted the old defect into a guard on
the fix.
Not settled and now the next item: compose ignores the keyframe fade alpha
entirely (blit modulates by tint only), which is why choosing the wrong keyframe
was invisible until now.
The layer-key order was adopted from two measured screens and then applied to
every build on the disc, so it owed a regression check against the screens the
corpus had already validated against the running game.
Rendered the tutorial PAUSE menu and the title main menu both ways and diffed:
3.8 % and 1.1 % of pixels differ, max delta 45/255 and 34/255, and the two
renders are indistinguishable in layout — the change is confined to blends where
translucent sprites overlap. No regression, but which order is more faithful on
those two screens is unsettled and recorded as such.
Adds a corpus-wide test asserting every composite's draw list is strictly
increasing in (layer key, declaration index), streaming one pak at a time so it
does not OOM alongside the other whole-disc tests. It reports the rule's reach:
341 of 965 builds are reordered, and it fails if that share collapses.
Driving menu -> NEW GAME -> DIFFICULTY -> SELECT DATA -> slot 01 with plain flags
gets further than any run so far: SELECT DATA is reached with ZERO crashes (the
screen is alive — a log_ui_draws probe there records 140 draws over 8 frames),
and then choosing a slot lets the game proceed into a cinematic or load, where it
crashes at 0x82307128 — the same cache-flush std::map erase.
So the crash is intermittent in WHERE it fires, not whether: boot, SELECT DATA,
and now after slot selection. There is no safe path through the menus to be found
by choosing different options.
And it is the blocker for every mission-side experiment — the second capital-ship
capture included. Navigation is not the obstacle any more; it is scripted and
works.
Not settled and said so: how to get past it. --mem_watch=false does not (twice).
--eh_dispatch is still untested because no run with it on has reached a throw.
The black-screen hang is a separate intermittent failure with no diagnosis. A fix
is emulator guest-race work, not RE.
Drove NEW GAME deliberately (the menu's first item, no d-pad, which is what the
run that first hit the crash actually did). Three runs with --eh_dispatch on: no
crash, no throw, black screen before the save-slot screen. It is tempting to read
that as the flag working.
The run with --cache_throw_diag as well disproves it. That cvar logs the throw
BEFORE any dispatch is attempted, so a throw would appear whatever dispatch then
did — and there were zero. No guest exception happened, so neither flag ran any
code, so neither can explain the difference. What took those runs out is the
intermittent content-load hang, before the crash path was reached.
eh_dispatch therefore stays untested, and the entry says so with the reasoning
rather than banking a false pass.
Confirmed on the way: the screen after NEW GAME is DIFFICULTY — its whole-image
signature matches the earlier capture exactly — so the menu path is understood
even though the runs die after it.
BACKLOG said the per-class ship generalisation needs "the capture-ship-placement
build of Canary". Checked: the ship capture is in the CURRENT build —
RequestShipCaptureFrame and CaptureShipDrawForRE are in command_processor.cc on
auto/re-ui-draw-order, and F10 wrote a 2.9 MB ship capture from this session's
binary. No separate build is needed.
What actually blocks it is navigation: a mission is behind the main menu, and (A)
at the title is accepted on about half of boots with nothing observable
predicting which. tutorial_launch.sh drives boot -> title -> menu -> TUTORIAL and
retries the WHOLE BOOT on refusal, because re-pressing the same title never
works — measured, not assumed.
First run of it: (A) refused, so it rebooted. That is the expected shape rather
than a failure of the harness.
Two routes closed, both cheaply.
The loading screen was the best discriminator reachable WITHOUT navigation — its
full-screen element is declared 7th of 10, so the two candidate orders would look
completely different. It does not appear during the boot at all: 23 captures
armed across an entire boot are all intro-movie frames, two full-screen quads
apiece. It belongs to a content load, which is behind the main menu, which is
behind the (A) that works half the time.
And a loophole this document had left open is shut: the title's sprites were
matched to build 4 BY SIZE, so the game might have been running some other build
whose table lists them in the captured order. Only two bundles in GP_TITLE.pak
contain all seven sprites, and neither is ascending in the captured order.
Recorded rather than glossed: one sub-order is genuinely ambiguous, because
ptlogo_back2eff and ptlogo_back2eff5 decode to the same 1133x280 and sit at
declaration 20 and 18, either side of ptlogo_back2 at 19. The UI quad carries a
k_8_8_8_8 colour attribute the hook does not log yet, and the two rest at
different fade alphas — logging it would name them apart.
OPTIONS is the cheapest reachable screen whose full-screen background is
declared LATE (a static sweep of all 30 UI paks ranks GP_DEBRIEFING_PILOTLOG
26-of-29 and GP_BUNK 15-of-17 above it, but both need a mission). Only such a
screen can tell "declaration order" from "background first", which is the open
question in the paint-order item.
It launches without any capture cvar now that F10 arms unconditionally, so the
navigation is not carrying the one variable that correlates with the title
refusing (A).
screen_id.py called the main menu "other". Its menu rule required a near-white
fraction above 1.5%, measured in 2026-07; the menu reached from the boot title
measures 0.03% (mean 13,26,59 — dark, strongly blue, essentially green-free).
That is worse than a cosmetic miss: a script that waits for "menu" and never
sees it reports the navigation as failed while the menu is on screen, which is
exactly what happened here. Both measured signatures are now documented in the
code and both classify.
menu_draw_capture.sh now taps ONCE on the first title rather than up to 40 times:
repeating was measured to be useless (the attract title accepts nothing) and the
first title accepts a single press.
title_states_capture.sh is new — it captures the draw list in both title states
in one run, which is what refuted the "the attract title omits the button plate"
theory.
Boot -> title -> main menu -> arm the UI draw-order capture there. What it
encodes is what cost time to learn:
* it does NOT tap (A) during the boot. A version that did delivered 88 presses
over the intro and left the guest on a black screen that never came back —
no crash, no throw, emulator healthy. The intro is ~3.5 minutes and reaches
the title on its own.
* it taps at the title until the screen stops being the title, rather than
pressing once and concluding, because acceptance is intermittent.
* after F10 it clicks the game surface, because F10 also opens the emulator's
menu bar and any Xenia UI makes IsUIActive() true, which swallows every later
guest keystroke.
It does not currently reach the menu — see docs/re/canary-scripted-input-traps.md
for the measurements. Committed because the traps are worth keeping encoded, and
because the next experiment (capture the draws in each title state) needs the
same scaffolding.
`ui_draw_capture.sh` grows three knobs the second iteration needed:
* ARM=early presses F10 before the title exists, so a long window contains the
frames in which a screen is BUILT (it turns out none are — the title screen
submits the same 11 draws every frame and never rebuilds);
* TARGET=menu taps A once on the title and arms on the main menu, skipping
attract movies on the way — it does not get there, but the blocker it hits is
documented rather than worked around;
* EXTRA_FLAGS passes emulator cvars through (--create_profile_if_none,
--mem_watch=false, --log_level).
`ui_draw_order.py` follows the capture's new vertex format (x, y, z) and reports
the Z it now has.
`ui_draw_capture.sh` boots to the title screen and arms Canary's new
`log_ui_draws` there — deliberately WITHOUT tapping A on the title, which is the
subject and which sends the guest into a save-data probe. ARM=early presses F10
before the title exists, for the frames in which a screen is built. A grab that
is not full-width is a hard error rather than something to classify.
`ui_draw_order.py` turns the capture into a named paint order: the UI shader
emits NDC, so a quad's pixel rect is exact, and the disc's sprites have
near-unique decoded sizes, so the rect's SIZE names the sprite. Matching is
nearest-within-6px with the distance printed, because a quad comes back a few
pixels under its sprite for a reason that is not yet measured.
The wrapper takes the xenia window's geometry from `xwininfo -root -children`
and crops the game surface out of it. Two things are wrong with that here, and
neither errors:
* `-children` lists only DIRECT children of the root, and openbox reparents the
game window — so the real one is not in that list at all;
* the app owns more than one window of class "xenia_canary" — an off-screen 10x10
helper as well as the 1280x745 game window — so `head -1` picked the 10x10 one.
Every grab came back a 10-pixel sliver, and nothing failed: `screen_id.py`
happily classified the sliver, the movie-skip heuristic fired on its noise, and
a whole session's worth of screen ids were meaningless. It also drove a stray
tap into the title screen's save-data probe, which crashed the guest.
Walk the full tree and take the largest xenia window by area, using its absolute
geometry (a reparented window's own +X+Y is relative to its frame).
WIP toward the residual flight-speed-law question (does a 1 s burst reach the
steady angular rate, or is there a per-axis multiplier?). The write-up already
concluded that host-side polling cannot answer it and named a Canary-side hook
as the tool required; that hook now exists (--frame_probe_log, committed as
auto/re-frame-probe in xenia-canary-native) and this is the harness for it.
- `rebuild_canary.sh` -- the surgical rebuild the box can actually do, kept in
the repo this time instead of in /tmp: compile only the changed objects, `ar`
them into their archive, and re-run the link command lifted out of the
generated ninja. A full `ninja` is impossible here (several TUs need dev
headers the image lacks) and the build cache cannot be re-configured. 31 s.
- `frame_burst.py` -- points the probe at the player craft's transform block
(pos-112, the three 16-byte-strided rows plus the position) and drives full
stick holds, recording each hold's start and end in the same clock the probe
stamps its lines with.
- `frame_session.sh` -- the whole run as ONE blocking foreground call, per the
session-lifetime rule; REUSE=1 drives a Canary that is already up.
- `nav_to_flight.sh` -- fly_stage.sh's navigation, split out so a live emulator
can be re-used. A boot to the title costs minutes under lavapipe and a run
that only failed to NAVIGATE should not pay for it twice.
The navigation change is the one worth reading. Every screen oracle here tested
named pixels ("648,221 is white"), which is only valid while the game image sits
at a known place on the root window -- and it does not: xenia's GTK window has a
menu bar, so on this display the image is ~25 px lower and every constant reads
the wrong row. Nothing errors. One run sat 300 s in front of a plainly visible
MAIN MENU reporting "no main menu"; the next missed the title screen entirely
and let the attract movie loop for ten minutes.
So `screen_id.py` identifies screens by WHOLE-IMAGE statistics instead -- the
fraction of green UI-text pixels, the fraction of near-white pixels, and the
per-channel means -- which no vertical shift, scale or letterbox can move. It is
calibrated against known-good captures and classifies all of them correctly:
title, three different menu screens, in-flight, and four movie frames as
"other". `bin/screenshot` additionally crops the menu bar off saved evidence
shots, deriving the offset from the window's own height rather than a constant.
Not yet a finding: the run has not reached flight, so no rate has been measured.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
To separate "a 1 s burst never reaches the steady rate" from "a per-axis
multiplier", measure inside ONE hold: successive 0.25 s windows of a single 3 s
press, holding speed, attitude and starting conditions constant by construction.
rep0 rate 16 59 325 209 130 322 238 110 310 151 183
rep1 rate 52 181 231 246 236 169 368 195 181 255 182
Not usable. At 0.25 s the windows do not contain enough guest updates to average,
so rate and speed both swing 3x window to window -- the same aliasing that once
manufactured a rate-vs-speed curve, reappearing at finer resolution. The first
window is lowest in BOTH repeats, which is what a ramp would look like, but the
sequence never plateaus, so the signal cannot be separated from the sampling. No
claim either way.
Widening the window does not rescue it: 0.5 s averages well enough, but a hold
long enough to contain several 0.5 s windows bleeds speed -- and speed is the
variable under test. The two effects are entangled at this observation rate.
So the residual needs a different INSTRUMENT, not another script. Live-RAM polling
samples an unsynchronised snapshot; the question wants the craft's angular
velocity as the guest computes it, once per frame. That is a Canary-side hook --
the same shape as the existing F10 ship-capture patch -- and the rebuild toolchain
already makes it cheap. Recorded as the recommendation rather than attempted as a
seventh variation of the same measurement.
The design the sweep could not provide: settle the throttle, measure the settled
speed, pitch for ONE second so speed barely moves inside the burst. Three
throttles, two repeats, row pin CONFIDENT, fresh flight.
LT min burst speed ~105 rate 113.6, 109.5 deg/wall-s
cruise ~383 100.2, 88.4
RT max ~1483 52.2, 70.5
Rate falls monotonically with speed -- 111.5 -> 94.3 -> 61.4 -- at three KNOWN,
SETTLED speeds instead of smeared across a bleeding one.
The decisive comparison needs no clock. Absolute rates depend on the run's clock
ratio, but the min:max RATIO cancels it:
measured min:max = 1.82
AV_PitchMinus_Min/Max 75/40 = 1.88 -> 3.0% apart
AV_PitchPlus_Min/Max 150/70 = 2.14 -> 15.1% apart
Two conclusions, neither resting on a clock measurement:
- _Min/_Max really do mean "at minimum / at maximum speed", with the rate
interpolating between them: shape confirmed to 3%.
- ly+ drives pitch-MINUS, not plus. The craft has asymmetric pitch authority
(75/40 down vs 150/70 up) and the ratio picks the pair cleanly.
Absolute magnitudes remain open: this run did not bracket the HUD clock, so
deg/GAME-second cannot be computed from it, and picking a ratio that makes the
numbers fit would be circular. The probe now screenshots the clock at both ends.
Also: fly_stage.sh now waits for the TAKE-OFF load too. Guarding only the stage
load left a run pressing A into a black screen and then reporting "player entity
not found" from a game that never reached flight.
Fresh flight, row pinning CONFIDENT (margin 0.413), one sweep and nothing before
it. axis_probe now REFUSES to measure on a WEAK pin (ALLOW_WEAK_PIN=1 overrides)
since it is a precondition, not a warning: roll is immune to the up/right
labelling but pitch and yaw are not.
Clock x1.26. Binned by speed, both in game units, against the linear
interpolation of AV_PitchPlus_Min 150 (at MinimumVelocity 100) to _Max 70 (at
MaximumVelocity 1200):
speed ~435 measured 100.8 predicted 125.6
speed ~572 113.8 115.7
speed ~709 126.3 105.7
speed ~846 83.1 95.7
speed ~983 72.7 85.8
Supported: the magnitudes (73-126 measured vs 86-126 predicted) and a falling
high-speed end. NOT supported: the interpolation law. Scatter is +-25%, the two
fastest bins hold 1 and 2 windows (the first moments before the speed bled), and
the slowest bin misses in the wrong direction.
The flaw is structural, not statistical: a sweep DRIVEN by the speed bleeding
cannot dwell at either extreme, which is exactly where the law is most testable.
What would settle it: hold a settled throttle and pitch for ~1 SECOND, so speed
barely moves inside the burst and each burst gives one honest (speed, rate) point;
repeat at LT / neutral / RT for three clean points at known speeds. Recorded as
the next design rather than attempted as a fifth variation of the same sweep.
