Sylpheed/docs/re/mission-freeze-heap-exhaustion.md

The mission freeze is a failed 128 MB guest allocation

✅ Root-caused 2026-08-26 on a Stage 02 run. The freeze this corpus lists as "THE blocker" is not a hang in the emulator: it is the guest asking for 128 MB of physical memory, being refused, throwing, and stopping.

The freeze, measured rather than assumed

Two independent witnesses, both taken while screen_id.py was calling the screen flight:

three screenshots over 8 s	rmse 0.00, 0.00 % of pixels changed
a 4 MB slab of guest RAM re-read after 2 s	0 bytes changed

The emulator process is not deadlocked — 399 % CPU across 15 running threads. It is spinning while the guest does nothing.

The cause is in the emulator log, and it is specific

The log stops dead mid-stage-load, last four lines:

BaseHeap::Alloc failed to find contiguous range
PhysicalHeap::AllocRange unable to alloc physical memory in parent heap
    (requested 134217728 bytes, parent free 28969/131072 pages)
MmAllocatePhysicalMemoryEx: Allocation failed: 00000000  Size: 08000000
Guest attempted to throw a C++ exception!

0x08000000 is 128 MB. The parent heap holds 131 072 pages and had 28 969 free — about 113 MB. So this is not only fragmentation, as failed to find contiguous range suggests on its own: there was less free memory in that heap than the request needed, contiguous or not.

Immediately before it, repeatedly:

BaseHeap::Release failed because address is not a region start
PhysicalHeap::Release failed due to parent heap failure

challenge-mission-gate.md already noted that this failed release "leaks the range". A leak that repeats through a session gives the mechanism the freeze's variable onset (27, 45, 83, 183 s across runs) needs: free pages drain until some 128 MB request loses, and when that happens depends on how much has leaked, not on the clock.

🔴 This refutes "poking only real story ids does not blow the heap"

challenge-mission-gate.md §5.6 records that poking 0xFFFFFFFF into the cleared-stage mask blows the heap on a 128 MB request, and concludes:

Poking only real story ids (0x0001FFFE = stages 1–16) does not blow the heap.

This run poked 0x0001FFFE — the value called safe — and hit the same 128 MB failure, with parent free 28969/131072 against that page's 30633/131072.

✅ And the control run settles it: the poke is irrelevant

That page ends with the right experiment — "repeat without the poke to see whether it fails the same way regardless" — so I ran it. nav_to_flight.sh gained SYLPH_NO_POKE=1, which skips the write; only Stage 1 is selectable without it, so the control also changes stage, which makes the agreement stronger rather than weaker.

run	mask	stage	outcome
poked	0x0001FFFE	Stage 02	frozen; 128 MB request refused
control	untouched, reads 0x0	Stage 01	frozen; 128 MB request refused

Both logs carry the identical line — not merely the same error, the same numbers:

requested 134217728 bytes, parent free 28969/131072 pages

28969 in both, across two stages and two boots. The control was verified three ways: the mask read back 0x0 from the live mapping, the log was this run's, and the liveness test was repeated with the stale process killed so exactly one emulator was running (three frames, rmse 0.00; an 8 MB slab, 0 bytes changed in 3 s).

🔴 So the poke does not cause it, and neither does the stage. The guest reproducibly arrives at a 128 MB physical request with ~113 MB free and throws. The identical free-page count across independent runs says the allocation pattern up to that point is deterministic — this is not a race and not memory pressure from the host.

⚠️ A frozen game passes the screen classifier

AGENT.md says to classify screens by whole-image statistics rather than named pixels, and that is right. But a single-frame statistic cannot tell flight from frozen in flight — screen_id.py called every frozen frame flight, which is exactly why the entity probes returned 0 definitions, 0 movers and 0 vtable hits with no indication anything was wrong.

A second frame costs nothing and is decisive. Any probe that assumes the game is live should diff two frames a few seconds apart, or re-read a slab of guest RAM, before believing an empty result means "the thing is not there".

❔ Still open: why the guest needs a single 128 MB block, whether the heap can be given more, and whether the failure is reachable without the mask poke at all.

✅ It is the emulated console's 512 MB, and the host is not short

Worth ruling out explicitly, because "allocation failed" invites blaming the box:

resource	measured at the freeze	binding?
/dev/shm (backs guest memory)	2.0 G, 17 % used, 1.7 G free	no
host RAM	15.9 G total, 12.3 G available	no
container cgroup memory.max	2.9 G of 7.0 G	no
emulated console RAM	113 MB free, 128 MB wanted	yes

The guest memory file is 4.5 GB apparent but sparse — only 319 MB of real blocks — so the 2 GB tmpfs was never close to full either.

The decisive arithmetic is in the log's own numbers:

131072 pages x 4 KB = 512 MB      <- the Xbox 360's unified memory, exactly
 28969 pages x 4 KB = 113 MB free
          0x08000000 = 128 MB requested

So parent heap is the emulated console's physical memory, not anything of the host's. The game runs a real 512 MB console down to 113 MB free and then asks for 128 MB. Nothing about the container is involved.

✅ The mechanism, confirmed in Xenia's source — the failed release leaks, provably

The adjacency argument below is no longer the evidence. src/xenia/memory.cc closes the loop by itself.

1. The number in the error is unreserved_page_count_. The message is emitted at memory.cc:1807:

XELOGE("PhysicalHeap::Alloc unable to alloc physical memory in parent heap "
       "(requested {} bytes, parent free {}/{} pages)",
       size, parent_heap_->unreserved_page_count(),
       parent_heap_->total_page_count());

So the 28969/131072 in the log is that counter, on the parent heap.

2. That counter is incremented in exactly one place — memory.cc:1445, inside BaseHeap::Release's page-table loop:

for (uint32_t page_number = base_page_number; page_number <= end_page_number;
     ++page_number) {
  auto& page_entry = page_table_[page_number];
  page_entry.qword = 0;
  unreserved_page_count_++;        // <- the only increment
}
InsertFreeBlock(base_page_number, base_page_entry.region_page_count);

3. The failing path returns before reaching it — memory.cc:1399, the very top of the same function:

if (base_page_entry.base_address != base_page_number) {
  XELOGE("BaseHeap::Release failed because address is not a region start");
  return false;                    // <- page table untouched, no free block
}

4. And it is the same heap. PhysicalHeap::Release delegates to parent_heap_->Release(...), so the release that fails and the allocation that later comes up short are against the same 512 MB parent heap.

Therefore every address is not a region start is a release that returns zero pages to the counter the allocator later consults. The pages stay reserved for the life of the process. This is not an inference from adjacency — it is the control flow.

🟡 What is still not measured: the magnitude

Mechanism confirmed, quantity not. 512 − 113 = 399 MB is unaccounted for, and there are only 22–23 failed releases per run, which would need an implausible ~18 MB average each to explain the whole deficit alone. So leaked releases are a contributor and possibly not the dominant one; the game may simply hold a lot of live memory at stage load.

The experiment that would settle it is an allocation ledger — log every MmAllocatePhysicalMemoryEx / MmFreePhysicalMemory with its size and balance the books — rather than more freeze reproductions.

🔴 There is no memory-size knob to work around it

Worth stating so nobody looks: Xenia has no cvar for guest memory size. The only memory cvars in memory.cc are protect_zero, protect_on_release, scribble_heap and the MMIO ones, and xboxkrnl_memory.cc says outright "We don't support separate devkit memory, so just ignore this flag" when a game asks for the devkit's extra RAM. The 512 MB is hardcoded to the retail console. So the freeze cannot be dodged by giving the emulator more — the fix has to be the release path, in a canary build.

🟡 The likely mechanism: a release path that leaks

Immediately before the failed allocation, in both runs:

run	BaseHeap::Release failed because address is not a region start
poked, Stage 02	23
control, Stage 01	22

and they are adjacent to it — in the control run they sit at lines 1115–1176 with the allocation failure at line 1179. A release that cannot find its region start returns without freeing, so each one leaves guest physical pages permanently claimed. Twenty-odd of those over a session is a plausible route from 512 MB down to 113 MB.

🟡 Plausible, not shown: I have not measured free pages over time, only at the moment of failure, so "the leak causes the exhaustion" is still an inference from adjacency and count. The test that would settle it is to sample the heap's free page count through a run and see whether it falls in steps that coincide with the failed releases.

❔ And the deeper question is why the release fails at all — address is not a region start says the guest is freeing something at an address Xenia's heap does not recognise as a region base, which is an emulator-side bug rather than a game one.

---

The allocation ledger — and 🔴 my leak hypothesis is wrong

2026-08-26, same day. The section above proposed that failed releases leak guest pages and drain the console. I built the ledger to test it, and it refutes that.

--log_level=3 --log_mask=0 makes MmAllocatePhysicalMemoryEx log its request, its granted base and its rounded size, and puts MmFreePhysicalMemory in the export trace, so allocations can be matched to frees. (log_mask disables categories — Kernel=1, Apu=2, Cpu=4, Gpu=8 — so the --log_mask=13 used everywhere in this corpus had the kernel log switched off.) Raw ledger: data/heap-ledger-stage01.txt.

🔴 There is no leak — the books balance

live guest physical allocations at the failure	379.5 MB in 84 blocks
free, per the failing allocator	113.2 MB
sum	492.7 MB of the console's 512 MB

The ~19 MB remainder is allocation paths this filter did not capture (image, stacks, kernel structures). Nothing is missing that a leak would explain.

🔴 And the 22 "leaked" releases are correct refusals

Every one of the 22 failing frees is an interior pointer into the same allocation — the 58.5 MB block at BC220000:

freed at	region base	region size	offset into region
BE078D80	BC220000	58.50 MB	30.35 MB
BDB18CC0	BC220000	58.50 MB	24.97 MB
BCA44980	BC220000	58.50 MB	8.14 MB
…

22 of 22 fall inside a known allocation, all in that one block. So the game sub-allocates out of one big physical pool and calls MmFreePhysicalMemory on the sub-blocks. Xenia is right to refuse them: BaseHeap::Release frees a whole region, so honouring an interior pointer would free 58.5 MB — and would do it twenty-two times. My proposed "fix" would have been a catastrophe.

🔴 Rounding is not it either

Requested 480.67 MB, granted 484.50 MB — 3.83 MB (0.80 %) of 64 KB-page rounding, against a shortfall of 14.84 MB. Not the cause.

✅ What actually happens: a doubling grow that holds both buffers

The last six ledger lines before the freeze:

= A7AC0000 Size: 02000000          allocate 32 MB
MmFreePhysicalMemory(A9AC0000)     free the previous buffer
= A3AC0000 Size: 04000000          allocate 64 MB
MmFreePhysicalMemory(A7AC0000)     free the 32 MB
MmAllocatePhysicalMemoryEx(08000000)   ask for 128 MB  -> FAILS

The game grows a buffer by doubling and allocates the new block before freeing the old, so the 128 MB step needs 64 + 128 = 192 MB live at once for that buffer alone, on top of ~315 MB held elsewhere. It comes up 14.84 MB short.

🔴 And there is no configuration that fixes it

• No guest memory-size cvar. memory.cc has only protect_zero, protect_on_release, scribble_heap and the MMIO ones.
• The 512 MB is baked into the address map, not a constant. The parent heap is Initialize(..., 0x00000000, 0x20000000, 4096) and its aliases sit at 0xA0000000 and 0xC0000000 — spaced exactly 0x20000000 apart. Growing the heap would run the 0xA0000000 window straight into 0xC0000000. It is not a one-line change.
• No eh_dispatch cvar in this tree. title-crash-stl-tree.md mentions one; this build has none. RtlRaiseException_entry routes 0xE06D7363 to HandleCppException, which logs and returns — so the guest's out-of-memory throw does not unwind, execution falls through into code that assumed it would not run, and the guest spins. That is the freeze.

❔ So the oracle is still broken, and here is what is left

Not fixed. The remaining candidates, in the order I would try them:

1. Find the ~19 MB the ledger does not see. If some of it is emulator-side rather than guest-side, recovering 15 MB of it clears the shortfall exactly. Log NtAllocateVirtualMemory and the image load as well.
2. Make the throw unwind. Implementing guest EH dispatch would let the game handle its own allocation failure — it evidently expects to, or it would not be probing sizes by doubling.
3. Reach flight another way. The tutorial loads less; two attempts timed out before reaching it, and that is unfinished rather than refuted.

---

Two patched builds, and 🔴 the allocation failure is not the freeze

2026-08-26. Both hypotheses about why the request is 128 MB, and the one fix that removed the failure, were tested against a rebuilt canary. Result: the oracle is still broken, and the headline above is now known to be incomplete.

🔴 The game does not size anything from MmQueryStatistics

xeMmQueryStatistics reports kernel_pages = 0x100 — 1 MB — under a comment that admits "These numbers are mostly guessed". A real console reserves far more, so Xenia over-reports title.available_pages. The game does import and call it: the only xref is sub_82612440, inside sub_82612420, which converts total_physical_pages and title.available_pages to bytes and stores them in its own struct; its caller keeps available-bytes in r23 and then creates 1280×800 and 640×400 surfaces. That looked like over-reporting driving over-allocation.

Patched kernel_pages 0x100 → 0x2000 (1 MB → 32 MB, twice the shortfall) and rebuilt:

	allocs	ever allocated	live at failure	failure line
stock (1 MB)	122	484.5 MB	379.5 MB / 84 blocks	free 28969/131072
patched (32 MB)	122	484.5 MB	379.5 MB / 84 blocks	free 28969/131072

Byte-identical. A 32× change in reported kernel memory changed nothing. Refuted, and the patch was reverted.

✅ The missing ~19 MB is Xenia's fixed startup reservations

memory.cc:240 reserves 16 MB of the parent heap before the guest runs:

heaps_.vC0000000.AllocFixed(0xC0000000, 0x01000000, 32, ...);   // GPU writeback

vC0000000's parent is the 512 MB physical heap, so this comes straight off the guest's budget — and 16 MB is larger than the 14.84 MB shortfall. That accounts for nearly all of the 19.3 MB the ledger could not see.

🔴 Shrinking it removes the allocation failure — and the game still freezes

Patched that reservation 0x01000000 → 0x00100000 (16 MB → 1 MB) and rebuilt. The stage load then completes with zero allocation failures and zero Guest attempted to throw — the 128 MB request is gone.

And the game is still frozen. Verified with one emulator running (the stale one killed first): three frames at rmse 0.00, an 8 MB guest slab with 0 bytes changed over 3 s, 389 % CPU across 3 running threads.

Worse, the ledger says it froze earlier: the doubling sequence now stops at 32 MB (A7AC0000 Size: 02000000), where the stock build reached 64 MB before failing at 128. Shrinking a live GPU region evidently hangs the GPU instead. The probe was reverted. data/heap-ledger-gpu-writeback-probe.txt.

🔴 What this costs the earlier conclusion

The section above says the freeze is the refused 128 MB allocation. That is now too strong. Removing the refusal does not remove the freeze. The honest statement is:

• the 128 MB refusal is real and reproducible, and Xenia's fixed 16 MB reservation is the swing factor that decides it;
• but the guest also hangs without it, at an earlier point, so the refusal is at best one of the ways this stage load dies, not the single cause.

❔ Still open, and the oracle remains unreliable: what hangs the guest when the allocation succeeds. The stock build's Guest attempted to throw a C++ exception! gave a mechanism for one path; the patched build reaches no exception at all, so the other path needs its own diagnosis — the log_stuck_waits instrumentation already on the canary branch is the obvious next tool.

🔴 Three flags in this corpus's scripts do not exist in this build

Checked with the 209-vs-67 000 byte log signature for a rejected argument: --eh_dispatch (named in newgame_path.sh and title-crash-stl-tree.md), --mem_watch (passed by newgame_path.sh), and --audio. All three are silently rejected, which blocks boot entirely rather than warning.

🔴 It is not a kernel-object wait — log_stuck_waits is silent

The canary branch already carries log_stuck_waits, written for exactly this question: it names the object a guest thread keeps timing out on, and its own help records that it is "silent on a healthy run — a wait that is being satisfied never builds a streak".

Turned on (via the config file — a --log_stuck_waits=true on the command line is accepted but the config value wins, and the startup dump shows which you got, so check the dump rather than trusting the flag) and run through to the freeze on the stock build:

log_stuck_waits = true
!> PhysicalHeap::AllocRange unable to alloc ... (requested 134217728 bytes, parent free 28969/131072)
!> Guest attempted to throw a C++ exception!

Not one stuck-wait line. So no guest thread is parked on a kernel object that never gets signalled. Combined with 389 % CPU across running threads, the guest is spinning in its own code, which is what the throw-that-returns predicts: execution resumes after the throw and runs on into code that assumed it would not.

That leaves the two paths clearly separated:

build	allocation	exception	stuck wait	outcome
stock	fails at 128 MB	thrown, does not unwind	none	frozen
GPU-writeback shrunk to 1 MB	succeeds	none	none	frozen earlier

The flag is cheap and silent, so it has been left enabled in the container's xenia-canary.config.toml.

🔴 Not a regression — an older binary reproduces it exactly

The obvious suspicion was that recent canary work broke it: this corpus has a 500-second Stage 02 flight from 2026-08-10, and the tree gained the file-pad driver, the UI-draw capture, a threading_posix resume fix and log_stuck_waits between then and now, with the running binary rebuilt on 08-24.

The container keeps older builds (/sylph-home/re/bin/*/xenia_canary, and a host build from 08-17 predating all of the 08-19 changes), and run-canary takes $XENIA_BIN — so this needed no rebuild. Running the 08-17 binary through the same route to the same point:

alloc failures: 1    guest throws: 1

Identical. So the failure is not new, and bisecting the emulator is not the way in. ⚠️ That also means the 08-10 run, which flew for 500 s, did not differ by binary — whatever let it reach flight is in the route or the game state, not the build.

❔ That is the open question now, and it is sharper than "why does it freeze": what did the working runs hold in memory that these do not? Both of the two largest live blocks — 112.88 MB at B50C0000 and 58.50 MB at BC220000 — are allocated at boot, before any menu, so 171 MB of the 379.5 MB is fixed regardless of route. The remaining ~208 MB is where a route difference could live, and the ledger can measure it: capture live-bytes at the moment TAKE OFF is pressed, for two different navigation paths, and compare.

---

🔴🔴 It is not a stage-load freeze at all — it freezes at MISSION SELECT

2026-08-26. Everything above calls this a stage-load or take-off freeze. That is wrong, and the evidence was in the run's own screenshots the whole time.

nav_to_flight.sh saves a shot at each step. Comparing them consecutively (not all against the first — see the mistake below):

transition	rmse	pixels changed
01-extras → 02-missionselect	51.07	99.42 %
02-missionselect → 03-selected	0.00	0.00 %
03 → 04 → 05 → 06 → 07 → 08-flight	0.00	0.00 %

Shots 02 through 08 are pixel-identical. The screen stops at MISSION SELECT and never changes again. Every step the script then reports — selected, after-A, readyroom, takeoff-hl, flight — is it pressing buttons at a frozen image, and screen_id.py classifying a static frame.

captures/freeze-at-missionselect.png is the frame it dies on.

🔴 So several claims on this page are withdrawn

• "MISSION SELECT and the stage list worked, and the failure came later, during the take-off load" — withdrawn. It froze entering MISSION SELECT.
• challenge-mission-gate.md §5.6 was right and I contradicted it. It says entering MISSION SELECT fails on a 128 MB request and Xenia surfaces a "Disc Read Error". That is exactly this. I read my own runs as getting further because the navigation script said so.
• Every "reached flight" in this session's runs is false, including the ones where screen_id.py returned flight. A single-frame classifier cannot tell a live screen from a frozen one — already noted on this page, and it still fooled me because I trusted the script's progress log instead of the frames.

⚠️ The comparison mistake, recorded because it nearly buried the result

The first pass compared every shot against shot 1, and got rmse=51.07, changed=99.42% for all seven — identical to two decimals across seven supposedly different images. That is not a result, it is a tell, and I briefly took it as "all frozen", then talked myself out of it when md5sum showed eight distinct hashes. The hashes differ only in PNG metadata. Compare consecutive frames, and treat a suspiciously constant statistic as a bug in the measurement, not a finding.

❔ What this changes about the oracle

The allocation failure happens while the display is already frozen, so it is downstream of whatever wedges the game, not the cause — consistent with the three runs that froze with the allocation succeeding, stopping at 16 MB, 32 MB and 128 MB respectively.

The actionable consequence: MISSION SELECT is the thing to avoid. Every route this session has used goes EXTRAS → MISSION SELECT. The 2026-08-10 run that flew for 500 s did not necessarily; newgame_path.sh and tutorial_launch.sh reach a mission by other menus. Testing a non-MISSION-SELECT route is the next experiment, and it is now the most promising one available.

---

🔴 MISSION SELECT is not special — corrected the same day

The section above concluded the freeze is "entering MISSION SELECT" and made avoiding that screen the next experiment. Ran it, and that conclusion is too narrow.

A liveness metric that actually separates the two states

tools/re-capture/route_liveness_probe.sh. Two frames five seconds apart, report the percentage of pixels that changed. The menu animates, so the separation is not marginal:

state	pixels changed in 5 s
main menu, healthy	99.80 – 99.97 %
frozen	0.00 %

This is the measurement that should have been used from the first run. It needs no navigation script to be honest about where it is.

The NEW GAME route freezes too

Walking the menu's first item — NEW GAME, which never touches MISSION SELECT:

step	liveness	screen_id says
main menu	99.80 %	menu
after Ⓐ on NEW GAME	7.33 %	title
after the next Ⓐ	0.00 %	"flight"

Frozen two presses off the main menu, with the same AllocRange unable ... 134217728 bytes in the log.

So the correct statement is broader

The game freezes on the first content load after the main menu, whichever menu item is taken. MISSION SELECT was simply the route every earlier run happened to use. The previous section's "avoid MISSION SELECT" recommendation is withdrawn — there is nothing to avoid.

That puts the memory account back at the centre: if ~379 MB is already live and a content load wants 128 MB, any content load fails, which is exactly what two different routes now show.

⚠️ Note also how badly screen_id.py misleads here: it called a frozen frame flight on a run that never left the menus. Liveness first, classification second — a classifier's answer is meaningless until the frame is known to be moving.

❔ Still open: whether the ~171 MB allocated at boot (112.88 MB at B50C0000, 58.50 MB at BC220000, both before any menu) is avoidable, since that plus a 128 MB content load is most of the console. That is the last untested lever, and it is a question about what those two blocks are.

---

✅ The 171 MB of boot allocations are the game's own pools, ~90 % empty

Read out of a live guest (main menu, liveness 98.59 %) rather than a frozen one, sampling a page every megabyte across each block:

block	requested	zero fraction	first bytes
B50C0000	112.82 MB	89.7 %	all zero
BC220000	58.45 MB	90.8 %	bc220010 00000000 bc79c9c0 bcd24c00 bc220040 bd17cda0 bc79ccc0 00000006

Neither holds loaded content at the main menu. Neither matches any disc file's size (nearest are the Stage_S*.xpr at 48–67 MB, none equal). They are pre-reserved pools, claimed at boot and mostly untouched.

✅ And BC220000 is the suballocator behind the "failed releases"

Its first words are pointers into itself — bc220010, bc79c9c0, bcd24c00, bc220040, bd17cda0, bc79ccc0 are all inside [BC220000, BFCA0000) — interleaved with small ordinals 6, 0, 4, 5, 0, 1, 2, 3. That is an intrusive free-list or node table.

This closes a loop from earlier on this page: the 22 BaseHeap::Release failed because address is not a region start messages were all frees of interior pointers into this exact block. Now it is clear why — the game suballocates from its own pool here, and hands those sub-pointers to MmFreePhysicalMemory. Confirmation that Xenia refusing them is correct, from the other direction.

❔ Which sharpens the contradiction rather than resolving it

So the budget is: the game reserves ~171 MB of pools at boot, holds ~379 MB by the main menu, and wants 128 MB more for a content load — about 495 MB of a 512 MB console, against Xenia's fixed 16 MB GPU-writeback reservation. That accounts for the ~15 MB shortfall, and the GPU-writeback probe confirmed the arithmetic by removing the failure when that 16 MB was freed.

But the same emulator, with the same 16 MB reservation, ran a mission for 500 seconds on 2026-08-10 — and that run's evidence is entity hull values sampled from guest memory at 2 Hz, not screenshots, so it cannot be the frozen-frame illusion that fooled this session twice. Something about that boot reserved less, and it is not the binary and not the route.

The next thing to vary is therefore the boot inputs: profile, save data and xenia-canary.config.toml. B50C0000's request is 0x070D28B0 — an odd, computed size, not a round reservation — so it plausibly depends on something configurable.

---

🔴 The allocation failure is definitively not the cause

Four runs now freeze at the same point — the first content load after the main menu — with three different allocation outcomes:

run	change	allocation outcome	froze?
stock	—	fails at 128 MB, guest throws	yes
GPU writeback 16 MB → 1 MB	emulator patch	none, stops at 32 MB	yes
heavy XMA debug logging	logging	none, stops at 16 MB	yes
no game save (--content_root to a profile-only tree)	boot input	none, no throw	yes

The memory account on this page is accurate and still worth having, but it describes a symptom. Everything above that treats the 128 MB refusal as the freeze is superseded by this table.

🔴 The savegame is not the variable either

--content_root=/tmp/content-nosave-a, containing a copy of the profile and no game save (the real content tree untouched), boots to the identical pools: BC220000 Size: 03A80000 and B50C0000 Size: 070E0000, same addresses, same sizes, same 43 allocations by the menu. Then freezes at the same press, with zero allocation failures and zero throws.

✅ It is not slow shader compilation either

Every earlier run called it frozen within ~30 s. Under lavapipe a content load could plausibly be compiling shaders, so this one was left for six minutes:

t+1min  liveness 0.00%   allocs 121   cpu 479%
t+2min  liveness 0.00%   allocs 122   cpu 461%
...
t+6min  liveness 0.00%   allocs 124   cpu 399%

No recovery, and no sign of one.

⚠️ And a correction I made mid-measurement

The allocation counter creeping (119 → 124) briefly looked like "the guest is running, only the display is stuck". It is not. A broad guest-memory sample — 400 extents, 8.2 MB — changed 18 bytes in 3 seconds. The guest is stuck; one thread still allocates roughly once a minute at 400 % CPU, which is a spin, not progress.

The earlier "8 MB slab at 0x40000000 unchanged" reading was also weak evidence for the opposite reason — one arbitrary region proves little either way. Sample broadly before calling a guest alive or dead, the same way the screen needed two frames rather than one.

❔ So the oracle is still broken and the cause is unidentified. What is now excluded: the heap failure, the leak, allocation rounding, MmQueryStatistics, a heap-size knob, the exception cvar, a kernel-object wait, a build regression, the navigation route, the savegame, and slow shader compilation.

---

✅ The stuck thread is located: a grow-and-copy in sub_82457780

2026-08-26. Ran the freeze under the corpus's own gdb wrapper (/sylph-home/re/bin/gdb-wrap/xenia_canary — ptrace_scope is 1, so a debugger must launch the process, and the wrapper's handle lines are required because Xenia uses SIGSEGV for guest memory watches).

One thread of 80 is in guest code. Every other thread sits in a futex or clock_nanosleep; thread 50, Main XThread, is at rip = 0xa05be939, inside JIT-generated code rather than libc.

Where, in guest terms

Xenia's x64 backend keeps the PPCContext in rsi (X64Emitter::GetContextReg() { return rsi; }), with r[32] at +0x20. Reading the guest GPRs there:

reg	value
r8	0xa3ac0000	the 64 MB buffer from the doubling sequence
r12	0xa3ac0a18	inside that buffer
r13	0x82457864	guest code — sub_82457780

And 0x82457864 sits in a grow-and-copy: a size computed as count × 8 (slwi r3, r27, 3) clamped against 0x1FFFFFFF, a call to 0x824F7240, then a loop that copies halfwords (lhz/sth) eight bytes at a time.

The host instruction it is stopped on is exactly that copy's store:

=> 0xa05be939:  mov %r12w,(%rdi,%rax,1)     rdi = 0x100000000 (membase)
                                            rax = 0x701d0000 (guest address)

a 16-bit store — the JIT's rendering of the loop's sth.

And it is not making progress

Sampled three times, seconds apart, with continue in between:

rip=a05be939 r13=82457864 r11=0 r31=701cf898
rip=a05be939 r13=82457864 r11=0 r31=701cf898
rip=a05be939 r13=82457864 r11=0 r31=701cf898

Identical every time. 🟡 Worth noting the loop's exit test is beq — it terminates only when r11 becomes exactly equal to r31, not >=. A start or end pointer that is inconsistent (or not 8-aligned relative to the other) would never satisfy it. That is a reading of the disassembly, not a demonstration, and r11 = 0 against r31 = 0x701cf898 is at least consistent with it.

🔴 Two of my own readings corrected

• "The guest spins at ~400 % CPU" — wrong. That was ps's cumulative average since process start, not an instantaneous rate. Per-thread sampling puts Main XThread nowhere near the top; the busiest are another guest thread at 24 % and the llvmpipe software rasterizers at ~7 % each.
• A SIGSEGV fault storm (Xenia's memory watches retrying forever) fitted the constant rip nicely and is refuted: 1 500 minor faults in 5 s, ~300/s, and zero major faults.

⚠️ Caveat on the method

These observations are under gdb, which intercepts every SIGSEGV even with nostop noprint pass, so absolute timings here are not the ungoverned ones. The freeze itself is not a gdb artefact — it reproduces in every non-gdb run — but "how slow" should be re-measured without it.

❔ Next: confirm or refute the beq reading by watching r11/r31 across a longer window, and identify what 0x824F7240 returns — if that allocator hands back a buffer whose end is not start + count×8, the loop's equality test is the freeze.

---

⚠️ Auditing the gdb session — what survives and what does not

2026-08-26. Went back to verify the previous section's instrumentation before building on it. Some of it does not hold.

✅ The context layout is now verified, not assumed

The trailing comments in ppc_context.h (uint64_t r[32]; // 0x20) are not struct offsets — the struct opens with eight cr unions and fpscr. So the previous section's rsi+0x20 was a guess that happened to be right. Confirmed empirically from a dump: the f[32] doubles (0x3ff0000000000000 = 1.0 and friends) begin at +0x140, and 32 × 8 bytes back from there puts r[32] at +0x20 … +0x11F. Good.

🔴 But the context is stale mid-function, so its GPRs are not evidence

In the same dump, r1 = 0. The guest stack pointer cannot be zero inside a function. Xenia's JIT keeps live guest registers in host registers and spills only at certain points, so a mid-function PPCContext read is a snapshot of whatever was last written back.

Therefore r13 = 0x82457864 does not show where the guest is executing, and the previous section's identification of sub_82457780 is withdrawn as unproven. The value may be a leftover from an earlier spill. What the code at that address does (a grow-and-copy with a beq exit) is accurately described — it just is not established that the guest is in it.

🔴 Three readings retracted for broken instrumentation

• "rip identical across samples ⇒ no progress". gdb stops the process between samples, and the JIT lays code out differently per run — rip was a05be939 in one run and a05bc839 in the next for the same instruction. Constant rip across samples inside one gdb session does not establish a hang.
• "Zero minor faults on the stuck thread" and "utime=0" — both parsed /proc/<pid>/task/<tid>/stat with awk positional fields. The comm field is parenthesised and contains spaces (Main XThread (F), so every field index after it was wrong. The state I printed as state=XThread should have been the giveaway. Both numbers are void.
• The earlier SIGSEGV fault-storm refutation rested on the same per-thread parse and is downgraded to unproven — though the process-wide rate (~300/s) still argues against a tight fault loop.

✅ What does survive

• The freeze reproduces without gdb, in every run, so it is not a debugger artefact.
• With the process stopped, exactly one thread is in userspace JIT code; a wchan census puts every other thread in futex_do_wait (55) or hrtimer_nanosleep (11). The active thread is Main XThread.
• Its host instruction is a 16-bit store through the membase, mov %r12w,(%rdi,%rax,1) with rdi = 0x100000000 and rax = 0x701d0000 — a guest stack-region address — and it is the first of a four-store group at +0, +2, +4, +6.
• Guest memory at 0x701d0000 and the surrounding 4 KB were unchanged over several seconds of running time.

❔ So the thread is identified and the instruction is identified; which guest function it belongs to is not. The way to get that honestly is a host→guest code mapping from Xenia itself rather than inference from a stale context — its code cache knows the answer, and dumping that mapping is the next step.

---

✅✅ The guest PC, proven — an infinite copy loop in sub_82457780

2026-08-26. The previous section withdrew this localisation because it rested on a stale PPCContext. It is now re-established by a sound method, and the withdrawal was right to demand one.

The method: Xenia's own JIT annotations

emit_source_annotations (a CPU cvar, off by default) makes MarkSourceOffset emit, at every guest instruction boundary:

nop; nop;  mov eax, <guest_address>;  nop; nop      -> 90 90 B8 xx xx xx xx 90 90

So the guest PC is readable straight out of the code bytes around rip — no context, no offsets, no inference. Dumping backwards from rip gives a clean run of them:

... B8 74 78 45 82 ...   -> guest 0x82457874
... B8 78 78 45 82 ...   -> guest 0x82457878
... B8 7C 78 45 82 ...   -> guest 0x8245787C
... B8 80 78 45 82 ...   -> guest 0x82457880
... B8 84 78 45 82 ...   -> guest 0x82457884

and the annotation immediately before rip is 0x824578A0.

Guest PC = 0x824578A0 = sth r6, 0(r9), inside sub_82457780 — the exact instruction predicted from the disassembly, reached independently.

The loop, and why it cannot end

82457868  mr    r11, r7           ; r11 = start
8245786c  cmplw cr6, r11, r31
82457870  beq   cr6, 0x824578B4   ; skip entirely if start == end
8245787c  mr    r10, r11          ; <== back-edge target
82457884  addi  r11, r11, 8       ; advance by 8
8245788c  cmplw cr6, r11, r31
82457890  lhz   r6, 0(r10)        ; copy four halfwords
824578a0  sth   r6, 0(r9)         ; <== the guest PC
824578b0  bne   cr6, 0x8245787C   ; continue while r11 != r31

The termination test is bne — exact inequality, on a pointer advancing in steps of 8. If r31 − r7 is not a positive multiple of 8, the comparison never becomes equal and the loop never exits.

✅ And the thread really is running

Measured with a safe /proc parse (see the trap below): over 4 seconds,

thread	CPU
Main XThread	890 ms
xenia_canary	860 ms
XThreadD55FF6C0	690 ms
llvmpipe-4	500 ms

Main XThread is the top consumer, and the process burns ~238 % CPU overall. So this is a spin, not a block — which is what an unterminatable loop looks like.

⚠️ A bad metric on the way: I tried to show the loop "marching" through memory by watching the last non-zero byte above 0x70200000. It never moved — but that statistic saturates in a region that is already written, so it could not have moved. It is not evidence either way, and it briefly looked like a refutation.

❔ Still open, and now narrow: what makes r31 − r7 non-congruent to 0 mod 8. r7 is loaded from 0(r30) and r31 is the container's end pointer, with a fresh buffer from 0x824F7240 in r3 — so the next step is to read those three values at the moment of the freeze, from host registers rather than the context.

---

🔴🔴 The infinite-loop conclusion is WRONG — gdb was the confound

2026-08-26. The section above concludes the guest spins forever in sub_82457780. Measured without gdb, that is false. Every observation supporting it came from a gdb-hosted run, and gdb intercepts every SIGSEGV — which Xenia uses for guest memory watches — so it perturbs exactly the thing being measured.

The same measurement, gdb removed

Six seconds of per-thread sampling (/proc parsed after the last )), on a run that reached the freeze with no debugger attached:

thread	CPU / 3 s	minor faults	state	wchan
Main XThread	0 ms	0	S	futex_do_wait
GPU Commands	10 ms	0	S	futex_do_wait
WSI swapchain queue	130 ms	0	S	futex_do_wait
llvmpipe-0 … llvmpipe-9	~1030 ms each / 6 s	0	—	—

Process total 265 % of one core, essentially all of it in the software rasterizer.

The guest thread is blocked on a futex and consuming zero CPU. It is not spinning, and it is not in sub_82457780 doing anything.

What was actually being measured under gdb

Under gdb the same thread appeared as the top CPU consumer (890 ms / 4 s) and appeared to sit in guest JIT code, later in xe::ExceptionHandlerCallback. That is the debugger's signal interception showing through, plus gdb repeatedly re-stopping the thread (state: t, wchan: ptrace_stop) between samples.

🟡 What survives is narrower but real: the emit_source_annotations technique is sound, and it did prove that when the guest thread was running under gdb its PC was 0x824578A0. That says the guest passes through that copy loop; it does not show the loop never terminates. The bne-exact-equality reading of the loop is still an accurate description of the disassembly and still a plausible hazard — but it is not this freeze.

🔴 Withdrawn: "an infinite copy loop in sub_82457780 is the freeze", and with it "Main XThread is the top CPU consumer, so it is a spin, not a block". The opposite is true.

✅ What the freeze actually looks like

• the guest is blocked, waiting on a host futex, using no CPU;
• the software rasterizer is saturated — nine llvmpipe threads at ~1 CPU- second per 6 seconds each — and has been for six minutes in an earlier run without ever producing a new frame;
• nothing faults, nothing allocates meaningfully, the screen never changes.

That is the signature of the guest waiting on a GPU operation that never completes, with lavapipe spinning on it — a host rendering problem, not a game-logic or guest-memory one. Which also explains why every game-side hypothesis this session was refuted in turn.

❔ Next: find what the guest is waiting on and what llvmpipe is rendering. GPU Commands idle at 10 ms/3 s while the rasterizer is pinned suggests the command processor has already handed off and is itself waiting.

⚠️ Method rule earned the hard way: do not diagnose a performance or liveness question under a debugger. Use it to read state at a known-good stopping point, then re-measure timing and CPU with it detached.

---

✅✅ The freeze REQUIRES the rasterizer — --gpu=null does not freeze

2026-08-26. The cleanest control available, and it settles which side the bug is on. Xenia has a null graphics backend (--gpu=null, xenia_main.cc:415). Same ISO, same route, same presses:

	lavapipe (default)	--gpu=null
Main XThread CPU	0 ms / 3 s	3 860 ms / 4 s
Main XThread state	S	R (running)
Main XThread wchan	futex_do_wait	0 (not blocked)
guest memory churn	~0 (18 bytes / 8.2 MB)	22.6 % of 6 MB in 4 s
allocations across the content load	stall	114 → 120, zero failures
process CPU	265 % (all llvmpipe)	330 % (mostly the guest)

With no rasterizer the guest does not freeze. It runs at close to a full core and walks through the content load — the exact point that hangs every rendered run — without a single allocation failure.

So the fault is in the host rendering path, and every game-side hypothesis this page worked through was looking in the wrong place. That is now explained rather than merely observed.

🔴 Blocked: telling "stuck" from "pathologically slow" needs hardware Vulkan

The remaining question about the rasterizer itself — whether lavapipe is deadlocked or merely taking hours over one pathological draw — cannot be answered here. This container has software rendering only, and that is one of the limits AGENT.md names explicitly. Six minutes of nine saturated llvmpipe threads with no frame produced is suggestive, not conclusive, and no amount of further waiting settles it. Recorded as a blocker rather than improvised around.

🟡 --gpu=null is a promising oracle for memory-based work — with one gap

Everything this project actually needs from a running game is read out of guest memory: entity positions, the flight model, the world unit. None of it needs pixels. --gpu=null supplies a live guest for exactly that.

The gap is navigation. Menu-walking is currently driven by screenshots, and with no rasterizer there is nothing to look at. Pressing Ⓐ blind got the guest through several screens — allocations advanced 114 → 120 — but never into a mission: a DEF_VTABLE / INST_VTABLE scan afterwards returns 0 / 0, so no units were ever loaded.

❔ Next, and it is a tractable engineering task rather than a mystery: navigate by guest memory instead of by screen. The menu state must be readable — the cleared-stage mask at 0x828F40C0 is already known and pokeable, so the menu layer keeps its state somewhere findable. A memory-driven nav_to_flight would make --gpu=null a complete oracle for every measurement on the backlog.

---

🔴🔴 "The freeze requires the rasterizer" is WITHDRAWN — the control was not controlled

2026-08-26. The section above concludes --gpu=null does not freeze. It is wrong, and the flaw is in the comparison rather than the data.

Driving the null backend blind, with the memory screen-id as the guide, to the same screen the rendered run froze on (screen = 4):

guest churn:   19 of 8,089,600 bytes in 4 s   (0.000%)
Main XThread:  0 ms CPU / 4 s, state S, wchan futex_do_wait
log:           AllocRange unable ... 134217728 bytes
               Guest attempted to throw a C++ exception!

Frozen, identically, with no rasterizer at all — same futex block, same 128 MB refusal, same throw.

What went wrong with the comparison

The earlier "control" measured the null-GPU run while it was still in the menus, and the lavapipe run at the freeze, then reported the difference as a backend effect. The two arms were at different points in the game. The allocations I cited as proof it had "walked through the content load" (114 → 120) were menu-time allocations; the content load had not happened yet.

A control needs both arms verified to be at the same state. The memory screen-id built this same session is exactly what makes that checkable, and I did not use it on the arm that mattered.

What this costs, and what survives

🔴 Withdrawn: "the freeze requires the rasterizer"; "the fault is in the host rendering path"; and the ⚠️ blocker on hardware Vulkan that followed from it — the rendering question is no longer load-bearing, so it is not what is blocking this.

✅ Survives: the freeze happens on the first content load after the main menu, on every route, on both GPU backends, with Main XThread futex-blocked at zero CPU rather than spinning. And the earlier observation stands that four runs froze with three different allocation outcomes — so the 128 MB refusal is correlated but not necessary.

✅ Also survives, and is independently verified: the memory screen-id navigation (menu-state-in-memory.md), now reproduced on a third run. It is what exposed this error.

❔ So the open question returns to: what is Main XThread waiting on? It is blocked on a host futex with the GPU excluded as the cause. The next lever is the one not yet tried — identify the futex and its holder — and that needs care, because gdb perturbs this measurement.