Before deriving a new vtable I checked where gworld.py's constants came from. They cite structures/unit-struct-runtime.md, which states its provenance: "Captured 2026-07-29 ... all six tutorials and Stage 02 'Declaration of War' loaded from save slot 01." Loaded from a save slot -- not through MISSION SELECT, which is the route every run this session has taken, and which needs the cleared-stage mask poke to offer a stage at all. So the constants may not be stale; they may just need the state that route produces. That is a cheaper question than writing a new vtable finder, and it should be answered first. First attempt inconclusive: driving the main menu's first item blind, five presses deep, advanced the progress counter every time (3 -> 5 -> 6 -> 8 -> 10 -> 12, so the game responds) but left DEF_VTABLE / INST_VTABLE at 0/0 throughout. Without a screen identity this is dead reckoning, and the first item may not be the load-game entry -- newgame_path.sh documents it as NEW GAME with SELECT DATA two screens further in. Next: reach the save-slot screen deliberately rather than by counting presses, and load slot 01.
320 lines
15 KiB
Markdown
320 lines
15 KiB
Markdown
# ✅✅ The freeze is `XamShowSigninUI` — a modal dialog nobody can dismiss
|
||
|
||
**Root-caused 2026-08-26**, after a long chain of wrong answers recorded in
|
||
[`mission-freeze-heap-exhaustion.md`](mission-freeze-heap-exhaustion.md).
|
||
|
||
## The mechanism, from Xenia's source
|
||
|
||
Every Xam UI dialog goes through `xeXamDispatchDialog` (`xam_ui.cc:60`), and the
|
||
calling **guest thread blocks**:
|
||
|
||
```cpp
|
||
xe::threading::Fence fence;
|
||
if (app_context.CallInUIThreadSynchronous(
|
||
[&dialog, &fence]() { dialog->Then(&fence); })) {
|
||
fence.Wait(); // <- the guest thread stops here until dismissed
|
||
```
|
||
|
||
With kernel logging enabled (`--log_mask=0`; note **`log_mask` disables**
|
||
categories, so the `--log_mask=13` used throughout this corpus hides exactly
|
||
this), the last kernel call before the freeze is:
|
||
|
||
d> F8000008 XamShowSigninUI(00000001, 00000001)
|
||
|
||
The game asks for a signed-in profile, Xenia opens a modal sign-in dialog, and
|
||
the guest waits on the fence **forever** — nothing in a scripted run dismisses an
|
||
ImGui dialog.
|
||
|
||
That accounts for every symptom at once: `Main XThread` **futex-blocked at 0 ms
|
||
CPU** rather than spinning; the emulator alive and healthy; no guest progress; no
|
||
faults; and independence from both the GPU backend and the allocation outcome.
|
||
|
||
## ✅ The fix, and the before/after
|
||
|
||
`run-canary` accepts `--logged_profile_slot_0_xuid=B13EBABEBABEBABE`. Same route,
|
||
same inputs, one variable changed:
|
||
|
||
| | without a profile | **with `--logged_profile_slot_0_xuid`** |
|
||
|---|---|---|
|
||
| screen id reached | **4, forever** | **4 → 5 → 6 → 8 → 9 → 10** |
|
||
| `XamShowSigninUI` | **called** | not called |
|
||
| allocation failures | 1 | **0** |
|
||
| guest throws | 1 | **0** |
|
||
| guest churn | 0.000 % (19 B of 8 MB) | **1.006 %** (81 KB of 8 MB) |
|
||
|
||
The game walks through six further screens where it previously sat at one
|
||
forever. `XamShowDeviceSelectorUI` is then requested, but
|
||
`storage_selection_dialog` defaults to **false**, so that dialog is not shown and
|
||
does not block.
|
||
|
||
## 🔴 Why this took so long — and what it invalidates
|
||
|
||
The `--log_mask=13` used by every script in this corpus **disables kernel
|
||
logging**, which is the one category that names the dialog. Without it the freeze
|
||
presents as a bare futex block with no explanation, which is why it was
|
||
attributed in turn to a heap leak, allocation rounding, `MmQueryStatistics`, a
|
||
build regression, the navigation route, the savegame, shader compilation, an
|
||
infinite guest loop, and finally the software rasterizer — each refuted in its
|
||
own section.
|
||
|
||
⚠️ The nearest miss is worth recording: `challenge-mission-gate.md` reported
|
||
Xenia showing a *"Disc Read Error"* on this failure. That is a **different**
|
||
dialog (`XamShowDirtyDiscErrorUI`), and it calls `exit(1)` — "This is death, and
|
||
should never return" — so it cannot be our freeze, whose process stays alive.
|
||
Checking that distinction is what led to the dialog mechanism.
|
||
|
||
❔ Still open: a mission has not been reached yet — `DEF_VTABLE` / `INST_VTABLE`
|
||
scans are still 0 at screen 10, so the remaining screens are menus (mission
|
||
select, briefing, ready room). That is now ordinary navigation work, not a
|
||
blocker.
|
||
|
||
## ⚠️ The fix must be applied at LAUNCH — `nav_to_flight.sh` cannot do it
|
||
|
||
`nav_to_flight.sh` contains **no** `run-canary` invocation: it drives an
|
||
*already-running* emulator. So whether a profile is signed in depends entirely on
|
||
how the caller launched it, and every `nav_to_flight` run in this session was
|
||
launched with a bare `run-canary --apu=sdl …`.
|
||
|
||
**That is why those runs froze**, and it is worth stating plainly because the
|
||
script prints `signing in profile B13EBABEBABEBABE` — which comes from the
|
||
emulator's own startup, not from the script arranging it.
|
||
|
||
run-canary --apu=sdl --logged_profile_slot_0_xuid=B13EBABEBABEBABE # then nav_to_flight.sh
|
||
|
||
## 🔴 Refuted: "each screen loads its own `GP_*.pak`"
|
||
|
||
A plausible way to get a *real* screen identity — watch which pak the game opens
|
||
— does not work. With full file logging, the only names that appear are at boot
|
||
(`sound.p0x`, `tables.p00`, `hidden`), and nothing new is opened when the UI
|
||
moves between screens. The paks are opened **once** and read by offset
|
||
thereafter, so file I/O carries no per-screen signal.
|
||
|
||
## ⚠️ Boots are flaky, independently of this bug
|
||
|
||
One launch in this iteration never reached the title at all: progress counter
|
||
stuck at `0`, screen black (mean `0,0,0`), guest churn **0.001 %**. That is a
|
||
*different* failure from the sign-in freeze — which stalls *after* the main menu
|
||
with the counters already advanced — and it matches the flakiness
|
||
`canary-scripted-input-traps.md` already documents ("(A) at the title is accepted
|
||
on roughly half of boots and nothing observable predicts which"). **Distinguish
|
||
them by the counters**: sign-in freeze stalls with counters advanced past the
|
||
title; a bad boot never leaves 0.
|
||
|
||
## ✅ The fix holds end-to-end — `nav_to_flight.sh` now runs to completion
|
||
|
||
With the emulator launched correctly, and the **boot verified by the progress
|
||
counter before spending time on navigation** (counter > 0 means the title was
|
||
reached; a flaky boot never leaves 0), `nav_to_flight.sh 2` ran the whole
|
||
sequence — main menu at 20 s, mask poke, stage select, briefing skip, ready room,
|
||
take-off — where every previous attempt stalled.
|
||
|
||
And the game is **alive at the end of it**:
|
||
|
||
| | frozen (before the fix) | now |
|
||
|---|---|---|
|
||
| screen liveness | **0.00 %** | **2.13 %** |
|
||
| guest churn | 0.000 % (19 B of 8 MB) | **0.288 – 2.932 %** |
|
||
|
||
An animating screen and a busy guest, at the point that used to be stone dead.
|
||
|
||
## ❔ But no units are loaded, and that is measured rather than assumed
|
||
|
||
`DEF_VTABLE` (`0x820AF844`) has **no hits** on a direct byte search of guest
|
||
memory, with a working control in the same run — searching for the poked
|
||
cleared-stage mask `0x0001FFFE` finds it at **`0x828F40C0`**, exactly where
|
||
`nav_to_flight` wrote it. So the search is sound and the vtable is genuinely
|
||
absent. Both constants were also checked against `sylpheed.db`'s `vtables` table
|
||
and are real vtables (`ANON_Class_C329B3BE`, `ANON_Class_503A3BED`), so they are
|
||
not stale.
|
||
|
||
Waiting does not change it: three minutes at counter 17 with `DEF/INST = 0 / 0`
|
||
throughout.
|
||
|
||
So the run reaches a **live, animating state that is not a mission with spawned
|
||
units** — most likely the launch cinematic or a post-take-off screen. The freeze
|
||
is gone; identifying where the sequence actually lands, and what still separates
|
||
it from unit spawn, is the remaining work.
|
||
|
||
## ✅ Where the take-off sequence lands: the **cutscene**, not a hang
|
||
|
||
Logging file and kernel activity through the whole navigation
|
||
(`--log_level=3 --log_mask=0`) and counting the names that appear:
|
||
|
||
12 \dat\movie <- heavy, repeated movie access
|
||
3 \dat\sound
|
||
2 Stage_S01 <- the stage resource IS referenced
|
||
1 XamShowDeviceSelectorUI
|
||
|
||
So after take-off the game is **playing the launch cutscene**. Under software
|
||
rendering a 720p movie decode is very slow — this corpus already had to raise
|
||
`INSTRUCTIONS_PER_MS` 100× because the movie handler has a 2 000 ms
|
||
software-decode deadline — which is exactly why this state persists for minutes.
|
||
|
||
The state is unambiguously **live**, and by the widest margin measured so far:
|
||
|
||
| | frozen | menus | **this state** |
|
||
|---|---|---|---|
|
||
| screen liveness | 0.00 % | 0.22 % | **11.00 %** |
|
||
| guest churn | 0.000 % | 0.190 % | **0.671 %** |
|
||
|
||
and the progress counter advances on input — `17 → 101` across the skip attempts
|
||
— so the pad is reaching the game.
|
||
|
||
## ❔ But the cutscene does not skip, and units never spawn
|
||
|
||
Four rounds of START / Ⓐ / Ⓑ advanced the counter every time yet left
|
||
`DEF_VTABLE` / `INST_VTABLE` at **0 / 0**. The input registers; it just does not
|
||
end the sequence.
|
||
|
||
So the remaining gap is narrow and well-defined: **the take-off cutscene runs and
|
||
is not skippable by the obvious buttons.** Either it must be played out (long,
|
||
under lavapipe), or the right skip input has not been found, or it is waiting on
|
||
something the movie path never delivers. That is the next question, and it is now
|
||
a question about *one* identified sequence rather than about a mysterious hang.
|
||
|
||
## 🟡 The post-take-off state is measurably neither a menu nor flight
|
||
|
||
`wait_flight.sh` carries the best discriminator in the corpus, and it is a
|
||
whole-image statistic rather than a named pixel: the flight HUD paints green over
|
||
the frame at **1.3–1.5 %** green fraction, against **< 0.1 %** on every menu.
|
||
|
||
Measured on the post-take-off state:
|
||
|
||
menu green=0.0022 white=0.0001 mean=(1.6, 45.6, 50.1)
|
||
|
||
**0.22 %** — an order of magnitude above a menu, an order of magnitude below
|
||
flight, and a dark blue-green frame. `screen_id.py` files that under `menu`,
|
||
which is simply the closest bucket it has; the state is neither.
|
||
|
||
`wait_flight.sh` then tapped Ⓐ for **280 seconds** and never saw the HUD:
|
||
`NO FLIGHT HUD within 280s (last: menu)`.
|
||
|
||
Taken with the 12 `\dat\movie` accesses, the reading is consistent: this is the
|
||
**cutscene**, it does not end on Ⓐ, and it does not reach the flight HUD within
|
||
five minutes of trying.
|
||
|
||
❔ What that leaves, in order of cheapness: (1) how the movie path signals
|
||
completion — the corpus already decoded the movie handler for the intro video and
|
||
found a software-decode deadline, so the same machinery may say what "done" looks
|
||
like; (2) whether a mission can be started by a route that skips the launch
|
||
cinematic; (3) whether the cutscene simply needs far longer than five minutes
|
||
under `lavapipe`.
|
||
|
||
**Status of the oracle overall: boot, menus and take-off are now reliable and
|
||
reproducible; only the cutscene→mission step is unsolved.** That is a much
|
||
smaller and better-defined gap than the freeze it replaced.
|
||
|
||
---
|
||
|
||
# ✅✅ The mission **does** load — and `DEF_VTABLE = 0` was a stale constant, not an empty game
|
||
|
||
**2026-08-26.** Three iterations concluded "no units are loaded" from
|
||
`gworld.scan_vtable(DEF_VTABLE)` returning 0. That inference is **withdrawn**.
|
||
The scan was measuring the wrong thing.
|
||
|
||
Counting unit-name strings in guest memory, with the **title screen as a
|
||
control** (same boot recipe, no mission loaded):
|
||
|
||
| string | title (control) | after take-off |
|
||
|---|---|---|
|
||
| **`DeltaSaber`** | **0** | **81** |
|
||
| `rou_` | 1 981 | **7 680** |
|
||
| `UN_` | 19 | **167** |
|
||
| `e010` | 67 | 134 |
|
||
| `ADAN` | 1 450 | 1 921 |
|
||
|
||
`DeltaSaber` is the **player's craft**, and it is **absent** at the title and
|
||
present 81 times after take-off. Every other count rises several-fold. The stage
|
||
content is unambiguously in memory.
|
||
|
||
⚠️ The control is what makes this a result. Raw counts alone prove nothing —
|
||
`rou_` and `ADAN` are numerous at the title too, because the unit *tables* load at
|
||
boot. Only the title-vs-mission **difference**, and `DeltaSaber`'s 0 → 81 in
|
||
particular, separates "the tables are loaded" from "the mission is loaded". My
|
||
first reading of these counts skipped the control and over-claimed; the control
|
||
was run before publishing.
|
||
|
||
## 🔴 What this withdraws
|
||
|
||
* **"No units are loaded"** — wrong. The mission loads.
|
||
* **"The take-off sequence lands in a cutscene that never ends"** — the movie
|
||
accesses are real, but the conclusion drawn from `DEF_VTABLE = 0` alongside them
|
||
does not follow. The game may well be *in* the mission.
|
||
* By extension, `gworld.py`'s `DEF_VTABLE = 0x820AF844` / `INST_VTABLE =
|
||
0x820AF030` do **not** locate entities in this build/state, even though both are
|
||
genuine vtables in `sylpheed.db`. They are the wrong ones, or entities are
|
||
reached another way.
|
||
|
||
## ❔ Next, and it is now concrete
|
||
|
||
Derive the **correct** entity vtable rather than trusting the stored constant:
|
||
the `UN_` strings reachable this way turned out to be `UN_NOSE` / `UN_MOUNT`
|
||
attachment names living in the **XEX image** (`0x820A…`), not runtime definition
|
||
records — so the route in is a name that only exists at mission time.
|
||
**`DeltaSaber` is exactly that name**: find its occurrences in the heap, find what
|
||
points at them, and read the vtable pointer of the referencing object.
|
||
|
||
## 🔴 The "follow the name pointer" route is structurally dead for pak strings
|
||
|
||
Locating `DeltaSaber` at mission time works: **53 occurrences, every one in the
|
||
heap** (`0xBC66…`–`0xBC6C…`), none in the XEX image —
|
||
`DeltaSaber_T.xpr`, `_Special`, `_NoseGun`, `_Missile`, `_TwinGun`.
|
||
|
||
But searching all of guest memory for a 32-bit big-endian pointer to those
|
||
addresses returns **0 references** for every one tried.
|
||
|
||
That is not a search bug, it is the format. Those strings live in the mission
|
||
pak's **IDXD string pool**, and this corpus decoded that container long ago:
|
||
records reference names by **offset into the pool**, never by absolute pointer
|
||
([`structures/idxd-container.md`](structures/idxd-container.md)). So there are no
|
||
pointers to find, and "find the name, follow what points at it" cannot work on
|
||
pak data by construction.
|
||
|
||
## 🟡 Which narrows what the string counts actually proved
|
||
|
||
The title-vs-mission control stands — `DeltaSaber` really is 0 at the title and 81
|
||
after take-off, so **mission-specific data is loaded**, which is more than "the
|
||
unit tables load at boot". But these are **asset-table strings**, not live entity
|
||
objects, so they do **not** show that entities have been *spawned*. The honest
|
||
statement is:
|
||
|
||
* ✅ the mission's pak data is loaded;
|
||
* ❔ whether entity objects exist is still unmeasured, because the only two
|
||
probes tried — `gworld`'s vtable constants, and name-pointer following — are
|
||
respectively stale and structurally inapplicable.
|
||
|
||
❔ Remaining routes, none tried: derive the entity vtable from the **code** (find
|
||
the constructor that writes a vtable pointer, via `sylpheed.db`'s `vptr_writes`
|
||
table, which exists precisely for this); or find the entity list from the mission
|
||
update function rather than from data.
|
||
|
||
## ✅ The corpus records how the working entity captures were made — and it is a different route
|
||
|
||
Before deriving a new vtable, I checked where `gworld.py`'s constants came from.
|
||
They cite [`structures/unit-struct-runtime.md`](structures/unit-struct-runtime.md),
|
||
which states its provenance plainly:
|
||
|
||
> Captured 2026-07-29 from Xenia Canary running the retail disc in the sylph-re
|
||
> container: **all six tutorials** and **Stage 02 "Declaration of War" loaded from
|
||
> save slot 01.**
|
||
|
||
**Loaded from a save slot** — not through `MISSION SELECT`, which is the route
|
||
every run in this session has taken (and which needs the cleared-stage mask poke
|
||
to offer a stage at all). So the constants are not necessarily stale: they may
|
||
simply require the state that route produces.
|
||
|
||
That reframes the entity hunt. Before writing a new vtable finder, the cheaper
|
||
question is whether the *save* route reaches a state the *mission-select* route
|
||
does not.
|
||
|
||
🟡 First attempt inconclusive: driving the main menu's **first** item blind, five
|
||
presses deep, advanced the progress counter every time (`3 → 5 → 6 → 8 → 10 →
|
||
12`, so the game is responding) but left `DEF_VTABLE` / `INST_VTABLE` at **0 / 0**
|
||
throughout. Without a screen identity the route is dead reckoning, and the first
|
||
item may not be the load-game entry at all — `newgame_path.sh` documents it as
|
||
NEW GAME, with `SELECT DATA` two screens further in.
|
||
|
||
❔ Next: reach the save-slot screen deliberately rather than by counting presses,
|
||
and load slot 01. `newgame_path.sh` already encodes NEW GAME → DIFFICULTY →
|
||
SELECT DATA and is the closest existing script to that path.
|