Syplheed-Reborn/docs/re/canary-scripted-input-traps.md

Getting past the title screen in the container — three traps and one blocker

Status: ✅ CONFIRMED for the three traps (each reproduced, and two of them fixed in the toolkit). 🔴 blocked on reaching the main menu: the guest reads Ⓐ at the title, and the step behind Ⓐ either crashes it or stalls it.

Written down because each of these presents as "the pad is dead" or "the game ignores input", and each has a completely different cause.

1. F10 opens the emulator's menu bar, and that swallows every guest keystroke

F10 is the RE hotkey (ship capture, and now log_ui_draws). It is also the toolkit key that opens the application menu bar, and it does both. With that menu open, XamState::IsUIActive() is true, so XamInputGetKeystrokeEx returns X_ERROR_SUCCESS with a zeroed keystroke before the driver is ever asked (xam_input.cc). The guest polls, gets an empty event, and does nothing — forever, and with no error anywhere.

This cost most of an iteration. Symptom: a run presses F10 to arm a capture, then every later Ⓐ does nothing; the title sits there while the pad log happily prints [file-pad] keystroke vk=5800 down. Screenshot captures/title-with-emulator-menu-open.png shows the giveaway — Open… / Open Recent / Install Content… / Exit over the title art. It is easy to miss because the menu is at the far left of the window and the game is still animating behind it.

Fix: after any F10, click once on the game surface to dismiss the menu. Escape does not close it. ui_draw_capture.sh now does this.

2. The pad is fine — the driver was never the problem

Worth stating because the file-pad's own history (--hid=file, GetKeystroke implemented specifically so "PRESS Ⓐ BUTTON" would work) makes it the natural suspect. It delivers: the log shows vk=5800 down/up per tap, and Ⓐ during a movie skips the movie, every time. When the title ignores Ⓐ, the pad is not why.

3. Ⓐ at the title needs a signed-in profile — and this container cannot type one in

With no profile, Ⓐ is handled: the guest calls XamShowSigninUI and Xenia pops its Sign In dialog (captures/title-signin-dialog.png), ProfileManager: Found 0 Profiles. That dialog is the proof the guest reads the button.

The dialog cannot be completed from here: "Create Profile" opens an ImGui text field for a gamertag, and synthetic X key events do not reach it. Tried, all with the window focused (xdotool getwindowfocus = the game window): xdotool type --window, xdotool type via XTEST, single xdotool key presses, clicking the field first. The field stays empty; Create with an empty gamertag is refused. Mouse clicks do work (they opened the dialog and the sub-panel), so it is text entry specifically.

Xenia's own Profile ▸ Show Profile Menu panel does have a working Create Profile button, and mouse clicks reach it — but the gamertag it then asks for is the same ImGui field, so the UI route dead-ends in the same place.

Fix, landed in Canary (auto/re-ui-draw-order): a --create_profile_if_none=<gamertag> cvar that calls ProfileManager::CreateProfile(tag, autologin, default_xuid) at startup when no profile exists. default_xuid is fixed at B13EBABEBABEBABE, so a script can then pass --logged_profile_slot_0_xuid=B13EBABEBABEBABE and be deterministic. Verified: ProfileManager: RE bootstrap profile 'SylphRE' -> created, then on the next boot Found 1 Profiles / Loaded SylphRE (GUID: B13EBABEBABEBABE) to slot 0, and the Profiles Menu panel shows the user (captures/xenia-profile-panel-signed-in.png). The profile persists on disk, so later runs need only --logged_profile_slot_0_xuid=B13EBABEBABEBABE.

Withdrawn: Ⓐ is not blocked — it works, intermittently

The main menu has been reached (captures/main-menu-reached.png — NEW GAME / LOAD GAME / TUTORIAL / OPTIONS / EXTRAS). So the previous version of this entry, which called the title's Ⓐ a hard blocker, is withdrawn. What is true is narrower and more annoying: Ⓐ advances the title only sometimes — one success in roughly four attempts, with the press verifiably delivered every time ([file-pad] keystroke vk=5800 down/up in the log) and no Xenia UI active.

The successful press landed on the first title after boot, before any attract cycle; the failures were on a title that had come round again. That is a hypothesis, not a rule — a later attempt on a first title also failed (the screen flickered to something else for one sample and came back), so timing is involved and the window has not been pinned down.

What was eliminated on the way, so nobody re-walks it

• IsUIActive() is now observable. XamInputGetKeystrokeEx returns X_ERROR_SUCCESS with a zeroed keystroke before consulting any driver when a Xenia dialog is up — an invisible input blackout. Canary now logs [RE-INPUT] XamInputGetKeystrokeEx swallowed by IsUIActive (dialogs=N …) when that happens. On the runs where Ⓐ failed, it never fires, so the blackout is not the cause there. (It is the cause after F10 — trap 1 above.)
• The driver filter is fine. The game polls with flags = 3, and InputSystem::FilterDrivers keeps a driver when flags & InputType is non-zero; the file pad reports Controller = 1, so it is selected.
• The game does not query sign-in on the press. Tracing every Xam* call around a press shows only XamInputGetCapabilities, XamInputGetKeystrokeEx and XamInputGetState — no content, user or signin call at all.

Two traps in the measuring rig itself, which cost more than the bug

• A FIFO trace consumer that exits stalls the emulator. --phase_a_event_log_path writes 1.5 GB a minute, so it was pointed at a named pipe with a grep on the other end. When that reader goes away, the emulator blocks on the write — and the symptom is exactly "the pad is dead": the guest stops polling, the driver never refreshes, nothing is logged. Two runs' worth of "the driver never reacts" was this, not the game.
• The trace's return_value is a placeholder. phase_a_bridge::EmitReturn is called as EmitReturn(name, 0) — literally zero — so every kernel.return event reads "status": "0x00000000". "Every keystroke call returns SUCCESS" looked like hard evidence for the IsUIActive theory and means nothing. The theory had to be tested with a real log line instead.

The scripted reproduction was tried, and it failed

"Tap Ⓐ once per second and see whether the success rate goes to 1" was the next step this entry named. It was run (tools/re-capture/menu_draw_capture.sh), and the answer is no:

run	what was pressed	result
E	one Ⓐ on the first title after boot	main menu
F	40 × Ⓐ at 1/s on the title, then one each of START, B, BACK, X, Y	title, unchanged
G	Ⓐ every 4 s through the whole boot (88 presses)	permanent black screen, no crash, never recovers

In run F every press was delivered — the driver logs keystroke vk=5800 down/up for each — and the new [RE-INPUT] line never fired, so nothing was swallowed. The guest received them and ignored them. No button does anything on that title: START, B, BACK, X and Y are as inert as Ⓐ (captures/title-unresponsive-attract.png).

Run G is worth its own warning: hammering Ⓐ through the boot is not merely useless, it breaks the run — 88 presses left a black screen that never came back (no crash, no throw, the emulator healthy). The script no longer taps during boot; the intro takes ~3.5 minutes and gets to the title by itself.

So the difference is not the number of presses or the button — it is which title you are looking at, and that is now measured rather than guessed:

• the title that ends the boot sequence accepts a single Ⓐ and opens the main menu — 2 of 2, in independent runs, one of which never pressed F10;
• the title the attract loop returns to accepts nothing — Ⓐ, START, B, BACK, X, Y, dozens of presses, every one delivered, none swallowed.

The recipe, then: launch, wait for the first title classification, tap Ⓐ once. Do not tap during the boot, and do not wait for a second title.

The tell is not visible in what the game draws — a log_ui_draws capture in each state gives 13 quads at identical rects, ptbtn00 (the PRESS Ⓐ BUTTON plate) included in both. See ui-title-paint-order-capture.md.

Still open

Why the attract title stops accepting input, given that it draws exactly the same thing. Nothing in the composition, the input path or the sign-in state differs; the difference is in the guest's own state machine, and reading it means reading that code.

Not open any more, and worth stating because it was the thing blocking the RE: reaching the main menu is now routine (first title, one Ⓐ), and the second screen's paint order has been captured with it.

2026-08-19 — the "first title accepts Ⓐ" rule is weaker than stated, and there is a confound

The entry above says the boot title accepts a single Ⓐ, 2 of 2. Continuing to use that recipe, it failed four times running. The full record, every run with the press verifiably delivered:

run	--log_ui_draws	title appeared at	Ⓐ
crash-dir	off	~83 s	menu
boottitle	off	147 s	menu
A/B "off"	off	—	menu
menucap 1	on	292 s	no (40 taps)
menucap 2	on	177 s	no
optcap 1	on	241 s	no
optcap 2	on	219 s	no
A/B "on" 1	on	never (black screen)	—
A/B "on" 2	on	186 s	no

3 of 3 without the flag, 0 of 5 with it. That is a strong correlation, and it points at my own instrumentation cvar — which would be a serious finding, since --log_ui_draws is what captures the paint order in the first place.

It is also completely confounded, and the table shows it. Every success had its title inside 147 s; every failure had it at 177 s or later. Flag and boot-time are perfectly correlated across these nine runs, so this evidence cannot separate "the capture cvar breaks input" from "a title that arrives late is already in the attract state and does not accept input". Both stories fit all nine rows.

A mechanism is hard to see for the flag: log_ui_draws is read only when F10 arms a capture; the per-draw hook is one relaxed atomic load, and the swap hook another. Nothing there should reach the guest. That argues for the boot-time reading — but "I cannot see how" is not evidence, and the correlation is what was measured.

What would separate them, cheaply and next: run the flag ON several times and record (boot time → outcome). A single ON run whose title arrives inside 147 s and which then accepts Ⓐ exonerates the flag outright.

A defect I introduced, recorded rather than left to bite

Widening the menu signature (previous commit) made screen_id.py classify some boot/attract frames as menu — twice in one run, at t=54 s and t=146 s, long before any menu exists. The menu and a dark-blue movie frame are not separable from a single grab. Scripts that navigate on this must require the classification to hold across two grabs a second apart; the classifier itself cannot fix it.

The confound was broken, and it did not go the way the table suggested

An interleaved series (same script, arms alternating, so cache warming cannot masquerade as either variable) settles the boot-time question and leaves the other one open:

arm	title at	Ⓐ
--log_ui_draws on	252 s	stuck
off	268 s	menu
on	254 s	stuck
off	232 s	stuck

Boot time is refuted. The latest title of every run recorded — 268 s — accepted Ⓐ, and a 232 s title refused. Whatever gates the button, it is not "the title arrived late".

The flag still correlates and still has no mechanism. Over all twelve runs it is 0 of 7 with the flag and 4 of 5 without. Against that:

• log_ui_draws is read in exactly one place — when F10 arms a capture — and F10 was never pressed in any of those runs;
• the per-draw hook is one relaxed atomic load, the swap hook another;
• the two arms' startup config dumps are byte-identical (246 settings, zero differences).

A correlation with no mechanism, and an arm that failed once, is not a finding. It is a variable to remove, which is what was done: F10 now arms the UI capture unconditionally, like the ship capture beside it, so no run needs the flag at launch. Verified by artifact — F10 with no capture flags writes the log.

A third failure mode, recorded because it is now frequent

Three boots have ended on a permanent black screen: one after 88 scripted Ⓐ presses, and two with no input at all. No crash, no guest throw, the emulator healthy and still resolving cache paths. It is not caused by the pad, and it is not the cache-flush crash. Roughly one boot in five in this session.

Still open, restated honestly

What gates Ⓐ at the title. Four candidate explanations have now been eliminated or refuted — Xenia UI swallowing the keystroke, the driver filter, a sign-in or content call on the press, and the age of the title — and the fifth (the capture flag) has a strong correlation and no possible mechanism. The next honest step is not another guess from outside: it is the guest's own state machine, and that means reading the title code in sylpheed.db.

2026-08-19 (later) — the flag hypothesis loses its own test

With the capture cvar removed from the launch line entirely (F10 arms unconditionally now), a fresh navigation run reached the boot title and Ⓐ did nothing. So the flag is not necessary for the failure, and the correlation — 0 of 7 with it, 4 of 6 without it — does not survive as an explanation.

What is left is the plain reading, and it is worth stating as the conclusion rather than as a stepping stone: Ⓐ at the title succeeds about half the time, and nothing measurable from outside predicts which. Eliminated so far: Xenia UI swallowing the keystroke (diagnostic never fires), the driver filter (flags 3 vs Controller = 1), a sign-in or content call on the press (none is made), the age of the title (the latest title of all accepted; an earlier one refused), and now the capture cvar.

The input path, mapped statically

Since the outside is exhausted, the guest side was traced in sylpheed.db:

entry_point 824AB748
  └ sub_8216EA68   (968 B)   main loop
      └ sub_822F1AA8 (996 B) per-frame input update
          └ sub_82457038 (628 B)  THE PAD POLL
              └ sub_824AA870 → XamInputGetKeystrokeEx   (flags = 3)

XamInputGetKeystrokeEx has exactly one caller in the whole title (sub_824AA870, a wrapper that maps user index 0xFF to 0 and ORs in 0x4000_0000 = ANY_USER), and that has exactly one caller, the poller. The poller's shape is readable and matches the runtime trace:

• +0 flags word — bit 31 "disabled", bit 30 "copy current→previous", bit 3 "also poll device state";
• a keystroke queue at +68 with a count at +72: it drains GetKeystrokeEx with flags = 3 into 8-byte slots (the same flags = 3 the phase-A trace recorded, so this is the code the trace was watching);
• the current pad state at +36, the previous at +52, compared field by field at the end (buttons +40/+56, triggers +42/+43, sticks +44…+50) to return "changed" (3) or "unchanged" (0), and 2 if the device vanished.

Nothing in that path is state-gated — it polls unconditionally and reports changes. Whatever ignores Ⓐ at an attract title is a consumer further up, and finding it means identifying the object the poller fills (passed in as r3 from the per-frame update) and then its readers. That is a real RE job, not a one-iteration probe, and it is where this thread now stands.

A confound in all of the above: the box was not always idle

Worth recording against every timing number in this entry. The container runs the emulator, the Canary build, and Reborn's disc test suite on the same 12 cores, and Xenia alone uses about 5.5. Several of the Ⓐ experiments — and most of the boot-duration measurements — were taken while a cargo test or a build-canary was running. One boot then failed to reach the title at all within 400 s, where an unloaded boot reaches it in 130–290 s.

That does not explain the Ⓐ result on its own: the interleaved A/B alternated arms under whatever load was present, and the latest title of all (268 s) accepted while a 232 s title refused. But a starved guest missing an input window is a live hypothesis that the data collected so far cannot exclude, and the honest next measurement is the one now being taken — the same navigation on an idle box, with nothing else running.

The black screen has a trigger: loading content (2026-08-19)

Running the same navigation with --eh_dispatch=true (to test whether dispatching the guest's C++ exception to its catch handler survives the SELECT DATA crash) produced zero crashes and zero throws — and that is not the good news it looks like, because the crash path was never reached.

What happened instead, with screenshots at every step:

main menu → TUTORIAL → the LESSON LIST (BASIC CONTROLS / HEADS-UP DISPLAY /
            RADAR / SUPPLY AND SPECIAL MOVES / RADIO ORDERS / ADVANCED
            CONTROLS / BACK)     captures/tutorial-lesson-list.png
          → Ⓐ on BASIC CONTROLS
          → BLACK SCREEN, permanently, for the remaining 160 s

No crash dump, no GUEST-THROW, the emulator healthy and the guest still running. So --eh_dispatch is untested — the experiment it was set up for did not occur.

What it did establish: the black screen this session has hit four times is not random. It follows a content load — here, selecting a tutorial lesson. Earlier black screens fit that too: one came after 88 scripted Ⓐ presses through the boot (which would have selected something), and one after a boot that was never touched but had reached the attract loop. The two failure modes on the way to a mission are therefore distinct:

• the SELECT DATA crash — the cache-flush std::map erase, when the path goes through save-slot enumeration;
• the black-screen hang — no crash, no throw, on loading content.

Neither has been beaten, and this run tested neither: it took a different branch through the menus than the run before it (TUTORIAL leads to a lesson list, and the DIFFICULTY screen captured earlier came from a different selection). The --eh_dispatch test needs a run that actually reaches SELECT DATA.

--eh_dispatch is still untested, and here is why that is a logical claim, not an excuse

Three more runs, driving NEW GAME deliberately (the menu's first item, no d-pad — which is what the run that first hit the crash actually did, though it believed it had chosen TUTORIAL):

flags	reached	crash dumps	GUEST-THROW
(none) — earlier run	menu → DIFFICULTY → SELECT DATA	537	yes
--eh_dispatch=true ×2	menu → black	0	0
--eh_dispatch=true --cache_throw_diag=true	menu → DIFFICULTY → black	0	0

The third row is the informative one. cache_throw_diag runs LogGuestThrow before any dispatch is attempted, so a throw would have been logged whatever the dispatch then did — and there were none. So no guest exception occurred in that run at all, and neither flag can have changed anything: both are inert until a throw happens.

Therefore the difference between the rows is not the flags. It is the intermittent content-load hang, which took the run out before it reached the save-slot screen. The flag remains untested for the reason the runs record, and the temptation to read "0 crashes with --eh_dispatch" as a fix is exactly what the log disproves.

Also confirmed on the way: the screen after NEW GAME is DIFFICULTY — its whole-image signature (9.7, 19.5, 53.9) matches the earlier capture of that screen exactly — so the menu path is understood even though the run dies after it.

✅ The title does act on Ⓐ — what stalls is the loader thread (2026-08-19)

Status: ✅ CONFIRMED by first-divergence across three boots of the same binary. 🔴 The stall itself is unexplained. ⚠️ "the title screen ignores Ⓐ" is withdrawn — it was the wrong description of the symptom, and it sent two iterations looking at the input path, which is fine.

Advancing past the title is intermittent — 1 success in 3 on 2026-08-19 with the same binary, profile and procedure — and the input path had already been excluded three ways (the press is logged as delivered to the guest). Comparing the successful boot against the failed ones line by line says what actually differs.

A slot-(1F) guest thread is spawned by the press. It starts exactly once per run, immediately after the Ⓐ keydown, at the same stack base (70880000-70900000) in both runs that got one:

run	Ⓐ delivered at line	(1F) thread starts	ResolvePath calls after
prm6 — reached the menu	6498	6500	6
opt2 — stuck on the title	1287	1288	0
opt — stalled before the title	never	never starts	—

The run that never accepted a press never creates a (1F) thread at all, which rules out "a periodic worker that happens to start around then".

So the game does act on the button: it spawns its loader. In the successful run that loader immediately reads six paths out of the on-disc cache —

HostPathDevice::ResolvePath(\aab216c3\a\c7e701e)
HostPathDevice::ResolvePath(\d5faa9db\e\b80b1a0)
HostPathDevice::ResolvePath(\d5faa9db\c\9f11f8b)

— and the menu appears. In the failed run the same thread starts and performs no file I/O ever again (log). Total ResolvePath counts for the three boots are 90 / 84 / 78: the successful run's extra six are exactly the ones after the press, so the boots are otherwise identical in I/O.

Refuted: it is not the cache-flush crash

The obvious suspect was the std::out_of_range in the cache flush (title-crash-stl-tree.md), which that page shows an incomplete on-disc cache triggers ~100 s into a boot. All four of today's runs have zero GUEST-THROW, zero CRASH DUMP and zero Access Violation. The guest is alive throughout — RSS steady, the keystroke-poll counter climbing past 15 000 — it simply stops loading.

Where the next probe goes

Not at input, and not at the crash. At the (1F) thread: it is created, it is scheduled (the guest keeps running), and it issues no I/O. Whatever it waits on first is the thing to find — a KeSetEvent / NtWaitForSingleObject trace on that thread, or the file-system call it never reaches.

Worth noting for anyone scripting boots meanwhile: retry whole boots. Nothing about the failed state recovers — four further presses over four minutes changed nothing, which is consistent with a loader that is stuck rather than a screen that is not listening.

✅ The loader thread is created and never runs (2026-08-19)

Status: ✅ CONFIRMED on a captured failure, two independent ways (zero kernel calls, zero host CPU time). 🔴 Where the resume is lost is not identified.

First: the harness had kernel logging switched off

log_mask disables categories — Kernel=1, Apu=2, Cpu=4, Gpu=8 — so the long-standing --log_mask=13 in every boot script meant Kernel + Cpu + Gpu off. Kernel calls also log at Debug, and log_level defaults to Info. So seeing a single kernel call needs both LOG_MASK=12 LOG_LEVEL=3, and no boot log this project has ever taken contained one. boot_menu.sh now accepts LOG_MASK / LOG_LEVEL / EXTRA_FLAGS. The cost is small: a whole boot at Debug with Kernel on is 23 MB.

What the press actually does

With that on, one boot reproduced the failure and the handler is fully visible:

XamUserGetXUID(00000000, 00000007, …)
NtCreateEvent(701CED80(701CEDE0), 0, 0, 0)
ExCreateThread(701CEDC0(701CEE50), 0, 0, 824AFF88, 821748F0, BC369B20, 00000001)
ExCreateThread Active: Thread Initially Suspended,
XThreadF80000CC (1F) Stack: 70880000-70900000
NtResumeThread(F80000CC, 701CEE40(00000000))

The game checks the signed-in user, creates an event, creates a thread with CREATE_SUSPENDED at guest entry 0x821748F0, and resumes it. Nothing is wrong on the guest side.

And then the thread never executes

Two independent measurements, on the stalled process:

• Zero kernel calls. F80000CC appears in the log 13 times and every one is as an argument — Added handle, NtDuplicateObject, NtResumeThread. Not one line has it as the calling thread. Meanwhile 31 905 kernel calls are made after the resume by five other threads (KeReleaseSemaphore ×15 991, XamInputGetKeystrokeEx ×2 500, …).
• Zero host CPU time. ps -L on the emulator shows 71 threads; the only one created after the press, XThread47FFF6C0, has 00:00:00 — while the process as a whole is at 546 % CPU and has burned 37 minutes of CPU in 6:46 of wall time.

A thread that is spinning burns CPU. This one has not run at all.

(evidence)

What that settles

The chain is now: press delivered → handler runs → thread created suspended → resumed → never scheduled. Every earlier suspect is excluded — input delivery (logged as delivered, three ways), the cache-flush crash (zero GUEST-THROW / CRASH DUMP / Access Violation in four runs), and the game's own logic (the handler does everything it should).

It also fits the intermittency, which nothing else did: a lost resume is a race, and this boot path succeeds about 1 time in 3.

What is not settled

• 🔴 Where the resume is lost. ExCreateThread with ThreadInitiallySuspended and NtResumeThread are both in xboxkrnl_threading.cc; the next step is Xenia's XThread create/resume pair and whether a resume can land before the host thread has started waiting.
• ❔ No successful boot has been captured with kernel logging on. The comparison would say whether the difference is in the guest (a different call sequence) or purely in the host (the same sequence, a resume that took). One more boot at LOG_MASK=12 LOG_LEVEL=3 is all it needs — the failure rate makes that cheap.
• ❔ Guest entry 0x821748F0 is now a named RE target: whatever the menu loader is.