diff --git a/docs/re/captures/stage02-freeze-stuck-wait-probe.txt b/docs/re/captures/stage02-freeze-stuck-wait-probe.txt new file mode 100644 index 0000000..f800fe3 --- /dev/null +++ b/docs/re/captures/stage02-freeze-stuck-wait-probe.txt @@ -0,0 +1,29 @@ +FROZEN IN FLIGHT at 9s + +=== stuck-wait probe: (thread, object) pairs === + 25 thread F8000048 on object BE56BB5C (type 2) + +=== last 12 probe lines === +w> F8000048 [wait-probe] thread F8000048 has timed out 6500 times in a row on object BE56BB5C (type 2), timeout=18446744073709251616 +w> F8000048 [wait-probe] thread F8000048 has timed out 7000 times in a row on object BE56BB5C (type 2), timeout=18446744073709251616 +w> F8000048 [wait-probe] thread F8000048 has timed out 7500 times in a row on object BE56BB5C (type 2), timeout=18446744073709251616 +w> F8000048 [wait-probe] thread F8000048 has timed out 8000 times in a row on object BE56BB5C (type 2), timeout=18446744073709251616 +w> F8000048 [wait-probe] thread F8000048 has timed out 8500 times in a row on object BE56BB5C (type 2), timeout=18446744073709251616 +w> F8000048 [wait-probe] thread F8000048 has timed out 9000 times in a row on object BE56BB5C (type 2), timeout=18446744073709251616 +w> F8000048 [wait-probe] thread F8000048 has timed out 9500 times in a row on object BE56BB5C (type 2), timeout=18446744073709251616 +w> F8000048 [wait-probe] thread F8000048 has timed out 10000 times in a row on object BE56BB5C (type 2), timeout=18446744073709251616 +w> F8000048 [wait-probe] thread F8000048 has timed out 10500 times in a row on object BE56BB5C (type 2), timeout=18446744073709251616 +w> F8000048 [wait-probe] thread F8000048 has timed out 100 times in a row on object BE56BB5C (type 2), timeout=18446744073709251616 +w> F8000048 [wait-probe] thread F8000048 has timed out 100 times in a row on object BE56BB5C (type 2), timeout=18446744073709251616 +w> F8000048 [wait-probe] thread F8000048 has timed out 500 times in a row on object BE56BB5C (type 2), timeout=18446744073709251616 + +=== CPU over 10 s while FROZEN (ticks, state, tid, comm) === + 401 R 154342 xenia_canary + 292 S 154409 XThreadA91FF6C0 + 280 S 154410 XThreadA81FE6C0 + 60 S 154406 XThreadB1FFC6C0 + 58 R 154405 XThreadB3FFF6C0 + 48 S 154376 XMA Decoder (01 + 33 S 154344 Logging Writer + 18 S 154418 XThreadA59FB6C0 + total 1255 over 79 threads diff --git a/docs/re/mission-freeze-resume-spin.md b/docs/re/mission-freeze-resume-spin.md index bf6e8a4..0b8a01f 100644 --- a/docs/re/mission-freeze-resume-spin.md +++ b/docs/re/mission-freeze-resume-spin.md @@ -388,3 +388,50 @@ roughly half the runs that reached flight ended early (freeze or GAME OVER), but they are **not evenly distributed**, so "wait for a freeze" is a ~30-minute lottery ticket per run rather than a reliable step. The instrument, the watcher and the control are all in place; what is missing is one frozen sample. + +## 🔴 2026-08-24 — a freeze WAS caught, and the stuck-wait probe says nothing + +The fourth run froze **9 seconds** into the watcher's window, in flight +(`freeze_watch.sh` confirmed the HUD was still on screen), and the probe built +for exactly this moment reported **the healthy-run baseline and nothing else** +([`captures/stage02-freeze-stuck-wait-probe.txt`](../captures/stage02-freeze-stuck-wait-probe.txt)): + +``` +FROZEN IN FLIGHT at 9s +=== stuck-wait probe: (thread, object) pairs === + 25 thread F8000048 on object BE56BB5C (type 2) +``` + +One pair — the same poller, on the **same object VA** as every healthy run (only +the thread handle differs, handles being per-run). **No new (thread, object) pair +appeared.** The hypothesis the probe was built to catch is refuted: the freeze is +not a guest thread looping on `KeWaitForSingleObject` **timeouts against one +object**. + +And the CPU signature is unchanged from the gdb run, so the burn is real: + +``` +CPU over 10 s while FROZEN: 1 255 ticks over 79 threads + 401 xenia_canary (the TimerQueue thread), state R + 292 XThreadA91FF6C0 + 280 XThreadA81FE6C0 +``` + +### What the probe's blind spots leave + +Two readings survive, and each is a specific blind spot of the instrument rather +than a vague "something else": + +* **The waits cycle over DIFFERENT objects.** The counter only advances while the + object is the same, so a thread rotating over two or more handles never builds + a streak and is invisible. This is the likelier of the two. +* **The waits SUCCEED.** A signal/wait ping-pong returns `X_STATUS_SUCCESS`, not + `X_STATUS_TIMEOUT`, so there is nothing for a timeout counter to count — which + would also fit the log evidence from the kernel-channel run, where the + self-suspending worker cycled thousands of times *successfully*. + +**Next:** a second version that counts **calls per thread per second regardless +of object or result**, and logs a thread whose rate is absurd along with the +object and the return value. That is a small edit to the same hook plus another +build — and, as ever, another run of the freeze lottery, which this time paid out +on the first attempt.