Syplheed-Reborn/docs/re/title-crash-stl-tree.md

The title-screen crash is an STL map/set erase on a bad iterator

Status: ✅ CONFIRMED — the guest throws std::out_of_range from an STL map/set erase during its cache flush, and an incomplete on-disc cache triggers it about 100 s into a boot (4 runs, 2 each way). 🟡 PROBABLE that this is the same defect as the Ready-Room crash — same exception type, same subsystem, same TOCTOU shape — but ❌ the mem_watch probe that handoff ranks as suspect #1 is eliminated here: with the cache cold, the throw happens with the probe off. ⚠️ two readings in the first version of this note are withdrawn (see the correction below): the fault address is not a corrupt pointer, and the access violation is not the bug.

Found while trying to get past the title screen for a second UI screen's paint order (canary-scripted-input-traps.md). It is worth a page of its own because it turns a crash that cost a whole mission to reproduce into one that costs a mv and 100 seconds.

The reproduction

Move the game's cache directory aside and boot with --cache_throw_diag=true:

mv ~/.local/share/Xenia/cache/aab216c3 /tmp/           # reversible; the game rebuilds it
run-canary --mem_watch=true --cache_throw_diag=true \
           --logged_profile_slot_0_xuid=B13EBABEBABEBABE
grep GUEST-THROW  <log>                                # ~100 s in

With the cache complete, the same boot produces no throw at all. The original symptom — what this note was opened for — looks like this:

Access Violation: read at 0x000000010000000C
PC: 0x82307128   guest thread 9
… preceded by  HostPathDevice::ResolvePath(\aab216c3\5\c10eae6)
   and         RtlRaiseException(702DF7F0(E06D7363), ContextArg)
               "Guest attempted to throw a C++ exception!"

Xenia pauses itself and stacks crash dialogs — 991 in one run, 2 437 in another.

What the code is

0x82307128 is inside sub_823070B0 (0x823070B0..0x823074D0, has EH), and the function identifies itself: it references the string 'invalid map/set<T> iterator' at 0x82062A8C, builds it with the string helpers at 0x8216E7E8 / 0x8216E5C8, and throws it through 0x825F23D8.

The node layout in the prologue is MSVC's std::_Tree_node exactly:

823070C8  lbz  r10, 25(r5)     ; iterator->_Ptr->_Isnil      (offset 25)
…                              ; if set -> build the string and THROW
82307124  bl   0x8244E2A8      ; (iterator helper)
82307128  lwz  r11, 0(r25)     ; node->_Left                 (offset 0)   <-- FAULT
8230713C  lwz  r27, 8(r25)     ; node->_Right                (offset 8)

_Left 0, _Parent 4, _Right 8, _Color 24, _Isnil 25 — that is the MSVC red-black tree node, so this is a std::map/std::set erase (it validates the iterator, then walks the node). The crash is the very first dereference after the validation.

Correction: the fault address is not a corrupt pointer, and the AV is not the bug

The first version of this note read 0x00000001_0000000C as "a 32-bit pointer with a stale high word" and offered an emulator register bug as one of two readings. Both readings were wrong, and the crash dump itself says so — it prints the registers, and

 r25  = 000000000000000C

is clean. Xenia maps the 4 GiB guest address space at host 0x1_00000000, so "read at 0x1_0000000C" is the host address of guest address 0x0000000C. The guest simply dereferenced the small integer 12.

And the access violation is a consequence of the throw, not an independent fault. sub_823070B0 validates the iterator, and on failure builds the string and calls the throw at 0x82307118; a throw does not return — except here. RtlRaiseException_entry in this build handles 0xE06D7363 by logging "Guest attempted to throw a C++ exception!" and returning (guest EH is only dispatched under --eh_dispatch, which its own cvar help records as disproven for this crash). So execution falls out of the throw into the code that assumed it would never run:

82307118  bl   0x825F23D8     ; throw std::out_of_range("invalid map/set<T> iterator")
8230711C  addi r3, r31, 276   ; <- execution RESUMES here, in this build
82307120  or   r25, r5, r5    ;    r5 has been clobbered by the throw call: 0x0C
82307124  bl   0x8244E2A8
82307128  lwz  r11, 0(r25)    ; <- AV, on 12

That means the 1 895 stacked crash dialogs are noise from one guest throw, and the event to study is the throw.

The throw, with the guest's own diagnosis

Run with --cache_throw_diag=true (a cvar this fork already carries) and the guest says everything — the full record is committed as captures/cache-flush-throw-cold-cache.log:

GUEST-THROW type=.?AVout_of_range@std@@ object=702DF950 lr=82612B50
GUEST-THROW guest stack: 825F2444 8230711C 8245A1BC 8245A86C 824AFFC4
CACHE-DUMP deque_count=38 list_count=37 map_size=24
CACHE-DUMP deque hashpairs: [0]D4EA4615:E46EE8CA [1]69D8E45C:E534FFEA … [37]AAB216C3:A2C8C185
CACHE-DUMP flush_sp=702DF9D0 snapshot_keys=1 deque_NOT_in_snapshot=38
   (these are what map::at throws on; in_live_map=1 => added during the flush's
    UNLOCKED window = the TOCTOU race)

• The thrown type is std::out_of_range — the same type the Ready-Room crash handoff names, from the same subsystem.
• The stack is throw ← sub_823070B0 ← sub_8245A098+0x124 ← sub_8245A5E0+0x28C ← sub_824AFF88+0x3C.
• The keys are <container hash>:<entry hash> pairs, and they are the on-disc cache files: AAB216C3:5C10EAE6 is \aab216c3\5\c10eae6, the very path the log resolves one line before the crash.
• The deque holds a duplicate: 38 entries, 37 distinct, with AAB216C3:A2C8C185 twice, against a map of 37 keys.

The mechanism itself was already worked out by whoever wrote this logger — the flush snapshots the map, then walks a deque every entry of which is missing from the snapshot but present in the live map, i.e. added inside the flush's unlocked window. This note adds no new theory there.

What is new: a 100-second, on-demand trigger

The throw is not random. It follows the state of the game's on-disc cache (~/.local/share/Xenia/cache/<container>/…, the \aab216c3\… device):

run	cache state	GUEST-THROW	crash dumps
A	complete (6 files)	0	0
B	directory moved aside — cold	1	1 895
C	partially rebuilt (1 file + a .tmp)	1	0
D	the original 6-file cache restored	0	0
E	cold and --mem_watch=false	1	2 437

All four reached the same boot stage (each resolves the same 87719002 / aab216c3 cache entries; the warm runs went further, so this is not "warm runs stopped early"). Runs B and C throw at the same point in the boot, around 100 s in, long before anything a player would call gameplay.

Run E is the important one, and it withdraws a claim this note made yesterday. The earlier version said "with --mem_watch=false the crash does not happen at all", which named the handoff's suspect #1 as measured. That comparison was confounded: every --mem_watch=false run so far had also had a warm cache. Holding the cache cold and turning the probe off, the guest throws anyway — so mem_watch is not the trigger for this crash, and the variable that is, is the cache.

So a suspect in the handoff's bisection plan no longer costs "one build plus one Ready-Room run": move the cache directory aside and boot. Note run C — the throw happened with no access violation behind it, which is why counting crash dialogs is the wrong signal and grep GUEST-THROW is the right one.

Stated as not settled: n = 2 on the warm side, n = 3 on the incomplete side, one box, one build; and the elimination of mem_watch is for this throw, not necessarily for the Ready-Room one, which nothing here has re-run. And this says nothing about why the deque grows during the flush — whether that race is reachable on hardware or is an artifact of emulator timing (the handoff's position) is exactly the open question, and a cheap trigger is only a tool for answering it.

The other crash PC, for completeness

The one unreproduced crash after an Ⓐ press was at 0x824578A0, in sub_82457780 (0x82457780..0x82457958, one caller, sub_82457038). It is an unrolled 4 × 16-bit copy loop:

82457890  lhz  r6, 0(r10)      82457898  lhz  r4, 4(r10)
82457894  lhz  r5, 2(r10)      8245789C  lhz  r10, 6(r10)
824578A0  sth  r6, 0(r9)   <-- FAULT (a STORE, not a load)

so a bad destination, in what looks like a small block copy — a different failure from the tree erase above, and with one observation it stays at that.

Why this matters beyond the blocker

The Ready-Room crash costs a full mission to reproduce, which is why its bisection plan in the handoff is written as "one build + one Ready-Room run" per suspect. If this title-screen crash is the same defect, each suspect costs 40 seconds instead, and suspect #1 (--mem_watch=false) is already measured here: it removes the crash.

It also fires on the way into a mission — which is what blocks the ship capture

2026-08-19. Driving the game toward a mission for the second capital-ship capture (tutorial_launch.sh: boot → title → Ⓐ → main menu → two d-pad steps → Ⓐ) reaches two screens nobody had captured, and then dies:

main menu → DIFFICULTY (EASY / NORMAL / HARD / BACK)   captures/difficulty-screen.png
          → SELECT DATA (save slots, "Current Storage: Dummy HDD")
          → CRASH: PC 0x82307128, guest thread 9, 537 stacked dumps
                                                   captures/select-data-crash.png

0x82307128 is the same std::map/set erase as the boot-time throw. So the cache-flush defect is not a boot curiosity: it fires again when the game enumerates save data on the way into every mission, and it paused the emulator for good — dismissing one dialog only reveals the next of 537.

Three things this pins:

• The blocker for the second capital-ship capture is this crash, not navigation. Navigation works; the game gets as far as the save-slot screen and dies there. BACKLOG's ship item should be read that way.
• mem_watch stays eliminated: this run had --mem_watch=false.
• The save/cache path is the common factor across both firings — the boot-time one followed HostPathDevice::ResolvePath(\aab216c3\…), and this one follows the save-slot enumeration.

Not settled: whether a warm cache prevents this firing the way it prevents the boot-time one. The cache was warm here (the 6-file aab216c3 restored earlier), so the answer looks like no — but that is one run, and the cold/warm A/B was only ever run against the boot-time throw.

Standing blocker: the mission path, measured end to end (2026-08-19)

Driving menu → NEW GAME → DIFFICULTY → SELECT DATA → pick slot 01 with plain flags (--mem_watch=false, no EH knobs) gets further than any run so far and still ends the same way:

step	outcome
main menu → NEW GAME	DIFFICULTY
DIFFICULTY → Ⓐ	SELECT DATA, and this time with 0 crashes (capture) — the screen is alive, a log_ui_draws probe there records 140 draws over 8 frames
slot 01 → Ⓐ	the game proceeds — several changing frames, a cinematic or load — and then crashes at 0x82307128, the same cache-flush std::map erase

Two things this settles, and one it does not.

Settled: the crash is intermittent in where it fires, not whether. It has now been seen at boot, at SELECT DATA, and after the save slot is chosen. The same run reached SELECT DATA cleanly and died one screen later. So there is no "safe path" through the menus to be found by picking different options — the flush throws whenever it next runs.

Settled: this is the blocker for every mission-side experiment. The second capital-ship capture, in-flight probes, mission-outcome work: all of them are behind this, and navigation is no longer the obstacle — that part is scripted and works (tools/re-capture/newgame_path.sh, blackscreen_probe.sh).

Not settled: how to get past it. --mem_watch=false does not (measured twice). --eh_dispatch remains untested because no run with it on has reached a throw. And the black-screen hang is a separate intermittent failure that takes some runs out earlier — it is not the crash, and it has no diagnosis yet.

The cheap trigger from the top of this note (an incomplete on-disc cache) still stands as the fastest way to reproduce the throw for bisection; what is missing is a fix, and that is guest-race work in the emulator, not RE.

The mission is now REACHABLE — and freezes there instead (2026-08-19)

Status: ✅ the blocker has moved, measured. 🔴 the crash is unchanged and still blocks a capture. 🔬 one new lead, deliberately not claimed as a cause.

This entry has said a second capital-ship capture "needs that crash dealt with first", because the run died at SELECT DATA before any mission existed. That is no longer where it stops.

With the Canary threading fix making the menu dependable (canary-scripted-input-traps.md), tutorial_launch.sh now drives boot → title → menu → TUTORIAL and the mission loads and renders: the flight HUD, the "Go to the box on your screen" prompt, the warship counters, the controller diagram (capture).

What still happens

• 13 243 crash dumps, every one at PC: 0x82307128 — the same address this page is about. Guest thread 9, Access Violation: read at 0x000000010000000C.
• Preceded by exactly one RtlRaiseException(E06D7363) / "Guest attempted to throw a C++ exception!", immediately after a HostPathDevice::ResolvePath() with empty arguments — the cache-flush shape this page documents.
• The game then freezes: two screenshots 6 s apart are identical (RMSE 0), no new dumps accumulate, and the process still burns 400 % CPU. Xenia suspends the crashing thread and the mission stops advancing.

So the honest status is: reachable, renders, unusable for a capture.

🔴 That lead is REFUTED (same day)

Withdrawn. The 1 663 refused resumes on F80001D8 are not evidence of a second lost resume, and the reason is a property of this codebase's logging that is worth knowing on its own.

F80001D8 did execute. The log has exactly one line from it —

K> F80001D8 XThread::Execute thid 41 (handle=F80001D8, 'XThread82FFE6C0 (F80001D8)', native=82FFE6C0)

— and then 137 000 lines of silence. That silence proves nothing: KeWaitForSingleObject and NtWaitForSingleObjectEx are declared kBlocking, kHighFrequency, and PrintKernelCall skips every kHighFrequency export unless --log_high_frequency_kernel_calls=true, which defaults to false. A thread parked in a wait is completely invisible here.

So the consistent reading is the boring one: the guest is kicking a worker that is blocked on an object, and Resume returns false because the thread is not suspended — a different mechanism entirely. Refused resumes are the expected result, exactly as first suspected before the count made it look interesting.

⚠️ Method note worth keeping. The title-loader conclusion did not rest on log silence — it rested on 00:00:00 host CPU time from ps -L, plus the fix changing behaviour 5/5 against 1/5. That is why it survives and this one does not. In this codebase, log silence alone is never sufficient evidence that a thread is idle.

(A grep of mine briefly said "zero kernel calls, ever" because the pattern ^[dikwF!]> missed the K> kernel prefix. Corrected before it was written down, and the conclusion changed as a result.)

To settle what F80001D8 actually waits on, one boot with --log_high_frequency_kernel_calls=true would show it.

🔬 The lead as originally recorded, kept for the reasoning

The XThread::Resume: host resume was refused diagnostic added with the threading fix fires 1 671 times on this path — and 1 663 of them are the same thread, F80001D8, with 152 before the first crash. On the menu path it fires about 7 times.

That is not asserted to be a bug. Resuming a thread that is not suspended legitimately returns false, and "call Resume to kick a worker" is a normal guest idiom that would produce exactly this. But 1 663 refusals on one thread, concentrated where the crash is, is specific enough to test: check whether F80001D8 makes any kernel calls between refusals. If it does, it is the kick idiom; if it does not, it is a second lost resume.

The cheapest next test is the cache

This page already establishes that an incomplete on-disc cache triggers the throw and that "with the cache complete, the same boot produces no throw at all". The cache directory is currently 40 MB, and the tree still carries aab216c3.partial-2131 and aab216c3.cold-rebuilt-2145 from earlier surgery — so it is quite possibly incomplete. Letting the game build a complete cache and re-running the tutorial is one boot to warm and one to test, and it would say whether the mission path is blocked by the crash at all or only by cache state.

🔴 Refuted: completing the on-disc cache does not stop the mission crash (2026-08-19)

This page's own reproduction — move the cache aside, get a throw ~100 s in; restore it, get none — made "the cache is incomplete" the obvious explanation for the mission-path crash. It is not the cure. Three tutorial runs, measured:

run	cache state	crashes early	storm begins	total dumps
tut2	subdir 6 missing	641 by t+24 s	~t+24 s	13 243
tut3	missing → gained 6 during the run	2 through t+80 s	~t+96–112 s	11 898
tut4	6 present from the start	2 through t+56 s	~t+56–88 s	11 497

The missing entry was real: \aab216c3\6 was requested and absent, and the game wrote it during tut3 (11 → 12 files). tut4 then ran with a complete cache and stormed anyway. Every dump in all three runs is PC: 0x82307128 — one address, no others.

So the cache is not what gates this. It may be what gated the boot-time throw this page originally documented; it does not gate the mission one.

✅ What the runs did give: a usable window

Both post-cache runs show the same shape — exactly 2 crashes, then nothing, for 56–80 seconds after the mission starts — where the first run was already at 641 by t+24 s. The mission renders and advances during it.

That is the practical opening for the thing this crash has been blocking: the second capital-ship capture needs F10 armed inside that window, not after it. It is not a guarantee — the storm began at t+24 s, ~t+96 s and ~t+56 s across the three runs, so the window varies and a run can lose it entirely — but two of three runs offered most of a minute of clean mission time.

What is still not settled

• 🔴 What actually triggers the storm. Not the cache; not input; the crash PC is invariant.
• ❔ What the 2 early crashes are, as distinct from the storm.
• ❔ Whether a capture taken inside the window is complete enough to be useful — untried.

✅ A crash-free mission run — and a capture that caught nothing (2026-08-19)

Status: ✅ the mission can run with zero crashes. ✅ the ship capture fires in-mission. 🔴 what it captured contains no ship geometry. ⚠️ the previous section's "the cache is refuted" is overstated and corrected below.

tools/re-capture/ship_capture_window.sh polls for the flight screen and presses F10 the moment it appears, rather than after a fixed sleep. One run:

• crashes=0 for the whole run, through t+152 s — the first mission run with no crash at all, where the three before it ended at 13 243 / 11 898 / 11 497.
• The mission renders and plays: player ship, starfield, full HUD, the "Go to the box on your screen" prompt, no dialog (capture).
• The capture armed and wrote its file — [SHIP-CAP] capture armed → xenia_ship_capture_01.log.

⚠️ Correcting the section above

That section says the cache is "REFUTED as the cure". That was too strong. This run used the identical complete cache as tut4 and produced 0 crashes against tut4's 11 497. Four runs now:

tut2  cache missing `6`     13 243
tut3  cache gains `6`       11 898
tut4  cache complete        11 497
ship  cache complete             0

What the evidence supports is narrower than what was written: a complete cache is not sufficient to prevent the storm, and run-to-run variance is large enough to dominate a 3-run comparison. It does not support "the cache does nothing".

🔴 The capture holds no ship geometry

329 KB, 181 deduped draws, and not one of them is 3D (log):

• 180 draws share a single vertex shader (0x0A6D1DD7767FDF27), all stride=28 vcount=3 prim=8, all at full-screen coordinates;
• one stride=20 vcount=6 prim=4 full-screen quad;
• zero draws whose positions are outside the 1280×720 screen rectangle.

The budget is not the limit — kShipCaptureBudget is 8 000 draws and only 181 distinct (vbase, WVP) pairs were seen. And the scene was definitely being drawn: the screenshot taken seconds later shows the player ship and starfield.

So the hook fires, the frame has 3D in it, and none of the 3D reaches the capture. For comparison, the capture this project already has from an earlier session is 2.9 MB.

🔴 Correction: nothing is broken — there was no ship to capture

The section above frames the empty capture as "the 3D draws never reach CaptureShipDrawForRE". That framing is wrong, and the static comparison it proposed is what showed it.

The discriminator for ship geometry is stride=24 prim=4 with a large vcount — xbg7-mesh.md records a real one as stride=24 vcount=10891 indices=18 prim=4. Not "positions outside the screen rectangle", which is what was used above and is a bad test: a UI sprite placed at (137,308) passes it.

By the correct test:

capture	prim=4 draws	verdict
this run, 329 KB	one, vcount=6 (a full-screen quad)	no ship geometry
uicap, 2.9 MB	none at all	no ship geometry either

So the 2–3 MB files this was being compared against are UI captures, not ship captures: 1 303 of their 1 582 draws are stride=24 prim=13, which ui-quad-class-foothold.md and ui-title-paint-order-capture.md both identify as the UI sprite shader. The "1 300 3D draws" they appeared to contain were UI sprite coordinates counted by the bad test.

The capture recorded exactly what was on screen. The tutorial's opening is an empty starfield, the player's own ship and a HUD — there is no capital ship in it. ship-placement-runtime-capture.md states the procedure plainly and always did: "Play into the mission, frame the ship side-on, press F10."

Next

Not a code question. A second capital-ship capture needs a capital ship on screen, which means driving real gameplay in a real mission (STAGE01 via LOAD GAME) far enough to frame one — a much larger ask than a menu step, and the original capture was taken interactively on HW Vulkan where this container has lavapipe. That is the honest size of what remains.