Syplheed-Reborn/docs/re/title-crash-stl-tree.md

# The title-screen crash is an STL `map`/`set` erase on a bad iterator

**Status:** ✅ `CONFIRMED` — the guest throws `std::out_of_range` from an STL
`map`/`set` erase during its cache flush, and an **incomplete on-disc cache**
triggers it about 100 s into a boot (4 runs, 2 each way). 🟡 `PROBABLE` that this
is the same defect as the Ready-Room crash — same exception type, same
subsystem, same TOCTOU shape — but ❌ **the `mem_watch` probe that handoff ranks
as suspect #1 is eliminated here**: with the cache cold, the throw happens with
the probe off. ⚠️
**two readings in the first version of this note are withdrawn** (see the
correction below): the fault address is not a corrupt pointer, and the access
violation is not the bug.

Found while trying to get past the title screen for a second UI screen's paint
order ([`canary-scripted-input-traps.md`](canary-scripted-input-traps.md)). It is
worth a page of its own because it turns a crash that cost a whole mission to
reproduce into one that costs a `mv` and 100 seconds.

## The reproduction

Move the game's cache directory aside and boot with `--cache_throw_diag=true`:

```bash
mv ~/.local/share/Xenia/cache/aab216c3 /tmp/           # reversible; the game rebuilds it
run-canary --mem_watch=true --cache_throw_diag=true \
           --logged_profile_slot_0_xuid=B13EBABEBABEBABE
grep GUEST-THROW  <log>                                # ~100 s in
```

With the cache complete, the same boot produces no throw at all. The original
symptom — what this note was opened for — looks like this:

```
Access Violation: read at 0x000000010000000C
PC: 0x82307128   guest thread 9
… preceded by  HostPathDevice::ResolvePath(\aab216c3\5\c10eae6)
   and         RtlRaiseException(702DF7F0(E06D7363), ContextArg)
               "Guest attempted to throw a C++ exception!"
```

Xenia pauses itself and stacks crash dialogs — 991 in one run, 2 437 in another.

## What the code is

`0x82307128` is inside `sub_823070B0` (`0x823070B0..0x823074D0`, has EH), and the
function identifies itself: it references the string
`'invalid map/set<T> iterator'` at `0x82062A8C`, builds it with the string
helpers at `0x8216E7E8` / `0x8216E5C8`, and throws it through `0x825F23D8`.

The node layout in the prologue is MSVC's `std::_Tree_node` exactly:

```
823070C8  lbz  r10, 25(r5)     ; iterator->_Ptr->_Isnil      (offset 25)
…                              ; if set -> build the string and THROW
82307124  bl   0x8244E2A8      ; (iterator helper)
82307128  lwz  r11, 0(r25)     ; node->_Left                 (offset 0)   <-- FAULT
8230713C  lwz  r27, 8(r25)     ; node->_Right                (offset 8)
```

`_Left` 0, `_Parent` 4, `_Right` 8, `_Color` 24, `_Isnil` 25 — that is the MSVC
red-black tree node, so this is a `std::map`/`std::set` **erase** (it validates
the iterator, then walks the node). The crash is the very first dereference
after the validation.

## Correction: the fault address is not a corrupt pointer, and the AV is not the bug

The first version of this note read `0x00000001_0000000C` as "a 32-bit pointer
with a stale high word" and offered an emulator register bug as one of two
readings. **Both readings were wrong, and the crash dump itself says so** — it
prints the registers, and

```
 r25  = 000000000000000C
```

is clean. Xenia maps the 4 GiB guest address space at host `0x1_00000000`, so
"read at `0x1_0000000C`" is the *host* address of guest address `0x0000000C`.
The guest simply dereferenced the small integer **12**.

**And the access violation is a consequence of the throw, not an independent
fault.** `sub_823070B0` validates the iterator, and on failure builds the string
and calls the throw at `0x82307118`; a throw does not return — except here.
`RtlRaiseException_entry` in this build handles `0xE06D7363` by logging
"Guest attempted to throw a C++ exception!" and **returning** (guest EH is only
dispatched under `--eh_dispatch`, which its own cvar help records as disproven
for this crash). So execution falls out of the throw into the code that assumed
it would never run:

```
82307118  bl   0x825F23D8     ; throw std::out_of_range("invalid map/set<T> iterator")
8230711C  addi r3, r31, 276   ; <- execution RESUMES here, in this build
82307120  or   r25, r5, r5    ;    r5 has been clobbered by the throw call: 0x0C
82307124  bl   0x8244E2A8
82307128  lwz  r11, 0(r25)    ; <- AV, on 12
```

That means the 1 895 stacked crash dialogs are noise from one guest throw, and
**the event to study is the throw**.

## The throw, with the guest's own diagnosis

Run with `--cache_throw_diag=true` (a cvar this fork already carries) and the
guest says everything — the full record is committed as
[`captures/cache-flush-throw-cold-cache.log`](captures/cache-flush-throw-cold-cache.log):

```
GUEST-THROW type=.?AVout_of_range@std@@ object=702DF950 lr=82612B50
GUEST-THROW guest stack: 825F2444 8230711C 8245A1BC 8245A86C 824AFFC4
CACHE-DUMP deque_count=38 list_count=37 map_size=24
CACHE-DUMP deque hashpairs: [0]D4EA4615:E46EE8CA [1]69D8E45C:E534FFEA … [37]AAB216C3:A2C8C185
CACHE-DUMP flush_sp=702DF9D0 snapshot_keys=1 deque_NOT_in_snapshot=38
   (these are what map::at throws on; in_live_map=1 => added during the flush's
    UNLOCKED window = the TOCTOU race)
```

* The thrown type is **`std::out_of_range`** — the same type the Ready-Room crash
  handoff names, from the same subsystem.
* The stack is `throw ← sub_823070B0 ← sub_8245A098+0x124 ← sub_8245A5E0+0x28C ←
  sub_824AFF88+0x3C`.
* The keys are `<container hash>:<entry hash>` pairs, and they are the on-disc
  cache files: `AAB216C3:5C10EAE6` is `\aab216c3\5\c10eae6`, the very path the
  log resolves one line before the crash.
* **The deque holds a duplicate**: 38 entries, 37 distinct, with
  `AAB216C3:A2C8C185` twice, against a map of 37 keys.

The mechanism itself was already worked out by whoever wrote this logger — the
flush snapshots the map, then walks a deque every entry of which is missing from
the snapshot but present in the live map, i.e. added inside the flush's unlocked
window. This note adds no new theory there.

## What is new: a 100-second, on-demand trigger

The throw is not random. It follows the state of the game's **on-disc cache**
(`~/.local/share/Xenia/cache/<container>/…`, the `\aab216c3\…` device):

| run | cache state | `GUEST-THROW` | crash dumps |
|---|---|---|---|
| A | complete (6 files) | **0** | 0 |
| B | directory moved aside — cold | **1** | 1 895 |
| C | partially rebuilt (1 file + a `.tmp`) | **1** | 0 |
| D | the original 6-file cache restored | **0** | 0 |
| E | cold **and** `--mem_watch=false` | **1** | 2 437 |

All four reached the same boot stage (each resolves the same `87719002` /
`aab216c3` cache entries; the warm runs went *further*, so this is not "warm runs
stopped early"). Runs B and C throw at the same point in the boot, around 100 s
in, long before anything a player would call gameplay.

Run E is the important one, and it **withdraws a claim this note made yesterday**.
The earlier version said "with `--mem_watch=false` the crash does not happen at
all", which named the handoff's suspect #1 as measured. That comparison was
confounded: every `--mem_watch=false` run so far had also had a *warm* cache.
Holding the cache cold and turning the probe off, the guest throws anyway — so
**`mem_watch` is not the trigger for this crash**, and the variable that is, is
the cache.

So a suspect in the handoff's bisection plan no longer costs "one build plus one
Ready-Room run": move the cache directory aside and boot. Note run C — the throw
happened with **no** access violation behind it, which is why counting crash
dialogs is the wrong signal and `grep GUEST-THROW` is the right one.

**Stated as not settled:** n = 2 on the warm side, n = 3 on the incomplete side,
one box, one build; and the elimination of `mem_watch` is for *this* throw, not
necessarily for the Ready-Room one, which nothing here has re-run. And this says
nothing about *why* the deque grows during the flush — whether that race is
reachable on hardware or is an artifact of emulator timing (the handoff's
position) is exactly the open question, and a cheap trigger is only a tool for
answering it.

## The other crash PC, for completeness

The one unreproduced crash after an Ⓐ press was at `0x824578A0`, in
`sub_82457780` (`0x82457780..0x82457958`, one caller, `sub_82457038`). It is an
unrolled **4 × 16-bit copy loop**:

```
82457890  lhz  r6, 0(r10)      82457898  lhz  r4, 4(r10)
82457894  lhz  r5, 2(r10)      8245789C  lhz  r10, 6(r10)
824578A0  sth  r6, 0(r9)   <-- FAULT (a STORE, not a load)
```

so a bad *destination*, in what looks like a small block copy — a different
failure from the tree erase above, and with one observation it stays at that.

## Why this matters beyond the blocker

The Ready-Room crash costs a full mission to reproduce, which is why its
bisection plan in the handoff is written as "one build + one Ready-Room run" per
suspect. If this title-screen crash is the same defect, each suspect costs **40
seconds** instead, and suspect #1 (`--mem_watch=false`) is already measured here:
it removes the crash.

## It also fires on the way into a mission — which is what blocks the ship capture

**2026-08-19.** Driving the game toward a mission for the second capital-ship
capture (`tutorial_launch.sh`: boot → title → Ⓐ → main menu → two d-pad steps →
Ⓐ) reaches two screens nobody had captured, and then dies:

```
main menu → DIFFICULTY (EASY / NORMAL / HARD / BACK)   captures/difficulty-screen.png
          → SELECT DATA (save slots, "Current Storage: Dummy HDD")
          → CRASH: PC 0x82307128, guest thread 9, 537 stacked dumps
                                                   captures/select-data-crash.png
```

`0x82307128` is the **same** `std::map`/`set` erase as the boot-time throw. So
the cache-flush defect is not a boot curiosity: it fires again when the game
enumerates save data on the way into every mission, and it paused the emulator
for good — dismissing one dialog only reveals the next of 537.

Three things this pins:

* **The blocker for the second capital-ship capture is this crash, not
  navigation.** Navigation works; the game gets as far as the save-slot screen
  and dies there. `BACKLOG`'s ship item should be read that way.
* **`mem_watch` stays eliminated**: this run had `--mem_watch=false`.
* **The save/cache path is the common factor** across both firings — the
  boot-time one followed `HostPathDevice::ResolvePath(\aab216c3\…)`, and this one
  follows the save-slot enumeration.

Not settled: whether a warm cache prevents *this* firing the way it prevents the
boot-time one. The cache was warm here (the 6-file `aab216c3` restored earlier),
so the answer looks like **no** — but that is one run, and the cold/warm A/B was
only ever run against the boot-time throw.

## Standing blocker: the mission path, measured end to end (2026-08-19)

Driving `menu → NEW GAME → DIFFICULTY → SELECT DATA → pick slot 01` with plain
flags (`--mem_watch=false`, no EH knobs) gets **further than any run so far** and
still ends the same way:

| step | outcome |
|---|---|
| main menu → NEW GAME | **DIFFICULTY** |
| DIFFICULTY → Ⓐ | **SELECT DATA**, and this time with **0 crashes** ([capture](captures/select-data-reached-no-crash.png)) — the screen is alive, a `log_ui_draws` probe there records **140 draws over 8 frames** |
| slot 01 → Ⓐ | the game proceeds — several changing frames, a cinematic or load — and then **crashes at `0x82307128`**, the same cache-flush `std::map` erase |

Two things this settles, and one it does not.

**Settled: the crash is intermittent in *where* it fires, not whether.** It has
now been seen at boot, at `SELECT DATA`, and after the save slot is chosen. The
same run reached `SELECT DATA` cleanly and died one screen later. So there is no
"safe path" through the menus to be found by picking different options — the
flush throws whenever it next runs.

**Settled: this is the blocker for every mission-side experiment.** The second
capital-ship capture, in-flight probes, mission-outcome work: all of them are
behind this, and navigation is no longer the obstacle — that part is scripted and
works (`tools/re-capture/newgame_path.sh`, `blackscreen_probe.sh`).

**Not settled: how to get past it.** `--mem_watch=false` does not (measured
twice). `--eh_dispatch` remains untested because no run with it on has reached a
throw. And the black-screen hang is a *separate* intermittent failure that takes
some runs out earlier — it is not the crash, and it has no diagnosis yet.

The cheap trigger from the top of this note (an incomplete on-disc cache) still
stands as the fastest way to reproduce the throw for bisection; what is missing
is a fix, and that is guest-race work in the emulator, not RE.


## The mission is now REACHABLE — and freezes there instead (2026-08-19)

**Status:** ✅ the blocker has **moved**, measured. 🔴 the crash is unchanged and
still blocks a capture. 🔬 one new lead, deliberately not claimed as a cause.

This entry has said a second capital-ship capture "needs that crash dealt with
first", because the run died at **SELECT DATA** before any mission existed. That
is no longer where it stops.

With the Canary threading fix making the menu dependable
([`canary-scripted-input-traps.md`](canary-scripted-input-traps.md)),
`tutorial_launch.sh` now drives boot → title → menu → TUTORIAL and **the mission
loads and renders**: the flight HUD, the "Go to the box on your screen" prompt,
the warship counters, the controller diagram
([capture](captures/tutorial-mission-reached-then-crash.png)).

### What still happens

* **13 243 crash dumps, every one at `PC: 0x82307128`** — the same address this
  page is about. Guest thread 9, `Access Violation: read at 0x000000010000000C`.
* Preceded by exactly **one** `RtlRaiseException(E06D7363)` / "Guest attempted to
  throw a C++ exception!", immediately after a `HostPathDevice::ResolvePath()`
  with **empty arguments** — the cache-flush shape this page documents.
* The game then **freezes**: two screenshots 6 s apart are identical (RMSE 0), no
  new dumps accumulate, and the process still burns 400 % CPU. Xenia suspends the
  crashing thread and the mission stops advancing.

So the honest status is: reachable, renders, unusable for a capture.

### 🔴 That lead is REFUTED (same day)

**Withdrawn.** The 1 663 refused resumes on `F80001D8` are not evidence of a
second lost resume, and the reason is a property of this codebase's logging that
is worth knowing on its own.

`F80001D8` **did** execute. The log has exactly one line from it —

```
K> F80001D8 XThread::Execute thid 41 (handle=F80001D8, 'XThread82FFE6C0 (F80001D8)', native=82FFE6C0)
```

— and then 137 000 lines of silence. That silence proves nothing:
`KeWaitForSingleObject` and `NtWaitForSingleObjectEx` are declared
`kBlocking, kHighFrequency`, and `PrintKernelCall` skips every `kHighFrequency`
export unless `--log_high_frequency_kernel_calls=true`, which defaults to
**false**. A thread parked in a wait is completely invisible here.

So the consistent reading is the boring one: the guest is kicking a worker that
is blocked **on an object**, and `Resume` returns false because the thread is not
*suspended* — a different mechanism entirely. Refused resumes are the expected
result, exactly as first suspected before the count made it look interesting.

⚠️ **Method note worth keeping.** The title-loader conclusion did *not* rest on
log silence — it rested on **`00:00:00` host CPU time** from `ps -L`, plus the
fix changing behaviour 5/5 against 1/5. That is why it survives and this one does
not. In this codebase, log silence alone is never sufficient evidence that a
thread is idle.

(A grep of mine briefly said "zero kernel calls, ever" because the pattern
`^[dikwF!]>` missed the `K>` kernel prefix. Corrected before it was written down,
and the conclusion changed as a result.)

To settle what `F80001D8` actually waits on, one boot with
`--log_high_frequency_kernel_calls=true` would show it.

### 🔬 The lead as originally recorded, kept for the reasoning

The `XThread::Resume: host resume was refused` diagnostic added with the
threading fix fires **1 671 times** on this path — and **1 663 of them are the
same thread, `F80001D8`**, with 152 before the first crash. On the menu path it
fires about 7 times.

That is *not* asserted to be a bug. Resuming a thread that is not suspended
legitimately returns false, and "call Resume to kick a worker" is a normal guest
idiom that would produce exactly this. But 1 663 refusals on one thread,
concentrated where the crash is, is specific enough to test: check whether
`F80001D8` makes any kernel calls between refusals. If it does, it is the kick
idiom; if it does not, it is a second lost resume.

### The cheapest next test is the cache

This page already establishes that an **incomplete on-disc cache** triggers the
throw and that "with the cache complete, the same boot produces no throw at all".
The cache directory is currently **40 MB**, and the tree still carries
`aab216c3.partial-2131` and `aab216c3.cold-rebuilt-2145` from earlier surgery —
so it is quite possibly incomplete. Letting the game build a complete cache and
re-running the tutorial is one boot to warm and one to test, and it would say
whether the mission path is blocked by the crash at all or only by cache state.


## 🔴 Refuted: completing the on-disc cache does not stop the mission crash (2026-08-19)

This page's own reproduction — move the cache aside, get a throw ~100 s in;
restore it, get none — made "the cache is incomplete" the obvious explanation for
the mission-path crash. It is not the cure. Three tutorial runs, measured:

| run | cache state | crashes early | storm begins | total dumps |
|---|---|---|---|---|
| `tut2` | subdir `6` **missing** | 641 by t+24 s | ~t+24 s | 13 243 |
| `tut3` | missing → **gained `6`** during the run | **2** through t+80 s | ~t+96–112 s | 11 898 |
| `tut4` | `6` present from the start | **2** through t+56 s | ~t+56–88 s | 11 497 |

The missing entry was real: `\aab216c3\6` was requested and absent, and the game
wrote it during `tut3` (11 → 12 files). `tut4` then ran with a complete cache
**and stormed anyway**. Every dump in all three runs is `PC: 0x82307128` — one
address, no others.

So the cache is not what gates this. It may be what gated the *boot-time* throw
this page originally documented; it does not gate the mission one.

### ✅ What the runs did give: a usable window

Both post-cache runs show the same shape — **exactly 2 crashes, then nothing, for
56–80 seconds after the mission starts** — where the first run was already at 641
by t+24 s. The mission renders and advances during it.

That is the practical opening for the thing this crash has been blocking: the
**second capital-ship capture** needs `F10` armed *inside* that window, not after
it. It is not a guarantee — the storm began at t+24 s, ~t+96 s and ~t+56 s across
the three runs, so the window varies and a run can lose it entirely — but two of
three runs offered most of a minute of clean mission time.

### What is still not settled

* 🔴 What actually triggers the storm. Not the cache; not input; the crash PC is
  invariant.
* ❔ What the 2 early crashes are, as distinct from the storm.
* ❔ Whether a capture taken inside the window is complete enough to be useful —
  untried.


## ✅ A crash-free mission run — and a capture that caught nothing (2026-08-19)

**Status:** ✅ the mission can run with **zero** crashes. ✅ the ship capture
fires in-mission. 🔴 what it captured contains no ship geometry. ⚠️ the previous
section's "the cache is refuted" is **overstated** and corrected below.

`tools/re-capture/ship_capture_window.sh` polls for the flight screen and presses
F10 the moment it appears, rather than after a fixed sleep. One run:

* **`crashes=0` for the whole run**, through t+152 s — the first mission run with
  no crash at all, where the three before it ended at 13 243 / 11 898 / 11 497.
* The mission renders and plays: player ship, starfield, full HUD, the "Go to the
  box on your screen" prompt, no dialog
  ([capture](captures/tutorial-mission-clean-run.png)).
* The capture armed and wrote its file —
  `[SHIP-CAP] capture armed → xenia_ship_capture_01.log`.

### ⚠️ Correcting the section above

That section says the cache is "REFUTED as the cure". That was too strong. This
run used the **identical complete cache** as `tut4` and produced **0** crashes
against `tut4`'s 11 497. Four runs now:

```
tut2  cache missing `6`     13 243
tut3  cache gains `6`       11 898
tut4  cache complete        11 497
ship  cache complete             0
```

What the evidence supports is narrower than what was written: a complete cache is
**not sufficient** to prevent the storm, and run-to-run variance is large enough
to dominate a 3-run comparison. It does not support "the cache does nothing".

### 🔴 The capture holds no ship geometry

329 KB, 181 deduped draws, and not one of them is 3D
([log](captures/ship-capture-no-geometry.log)):

* **180 draws share a single vertex shader** (`0x0A6D1DD7767FDF27`), all
  `stride=28 vcount=3 prim=8`, all at full-screen coordinates;
* one `stride=20 vcount=6 prim=4` full-screen quad;
* **zero** draws whose positions are outside the 1280×720 screen rectangle.

The budget is not the limit — `kShipCaptureBudget` is **8 000** draws and only
181 distinct `(vbase, WVP)` pairs were seen. And the scene was definitely being
drawn: the screenshot taken seconds later shows the player ship and starfield.

So the hook fires, the frame has 3D in it, and none of the 3D reaches the
capture. For comparison, the capture this project already has from an earlier
session is **2.9 MB**.

### 🔴 Correction: nothing is broken — there was no ship to capture

The section above frames the empty capture as "the 3D draws never reach
`CaptureShipDrawForRE`". **That framing is wrong**, and the static comparison it
proposed is what showed it.

**The discriminator for ship geometry is `stride=24 prim=4` with a large
vcount** — `xbg7-mesh.md` records a real one as
`stride=24 vcount=10891 indices=18 prim=4`. Not "positions outside the screen
rectangle", which is what was used above and is a bad test: a UI sprite placed at
(137,308) passes it.

By the correct test:

| capture | `prim=4` draws | verdict |
|---|---|---|
| this run, 329 KB | one, `vcount=6` (a full-screen quad) | no ship geometry |
| `uicap`, **2.9 MB** | **none at all** | no ship geometry either |

So the 2–3 MB files this was being compared against are **UI captures**, not ship
captures: 1 303 of their 1 582 draws are `stride=24 prim=13`, which
[`ui-quad-class-foothold.md`](ui-quad-class-foothold.md) and
[`ui-title-paint-order-capture.md`](ui-title-paint-order-capture.md) both identify
as the UI sprite shader. The "1 300 3D draws" they appeared to contain were UI
sprite coordinates counted by the bad test.

The capture recorded exactly what was on screen. The tutorial's opening is an
empty starfield, the player's own ship and a HUD — **there is no capital ship in
it**. [`ship-placement-runtime-capture.md`](ship-placement-runtime-capture.md)
states the procedure plainly and always did: *"Play into the mission, frame the
ship side-on, press F10."*

### Next

Not a code question. A second capital-ship capture needs a **capital ship on
screen**, which means driving real gameplay in a real mission (`STAGE01` via LOAD
GAME) far enough to frame one — a much larger ask than a menu step, and the
original capture was taken interactively on **HW Vulkan** where this container
has lavapipe. That is the honest size of what remains.