# AUDIT-069 Session 4 — writer report v4 Date: 2026-05-20 xenia-rs HEAD: `e6d43a23ac393004d2e5adf2f0395fd0b5e6448b` (UNCHANGED from S1/S2/S3) `git diff HEAD | sha256sum`: `ed30fd526643918f67311caff0a10d1346d73fd0c0323e02477883cf5ff20357` (UNCHANGED at start AND end of S4) No ours source modifications. No canary instrumentation added. Canary `audit_61_branch_probe_pcs` cvar used (pre-existing from S1). Canary cache restored from `/tmp/canary-cache-bak-audit-068`. ## Headline (HIGH confidence — direct per-iteration measurement) S3's "producer-loop underrun" framing pointed in the right direction but mis-located the divergence. **Neither engine ever takes the exit-branch in `sub_82450A68` (PC=0x82450B50, the LR=epilog path)**. Both engines's dispatch threads stay in the loop indefinitely (no deadlock; just waiting). The actual divergence is in the **return value of the `NtWaitForMultipleObjectsEx` call at PC=0x82450B44**: - **Ours: r3 = 0x00000001 in 91/91 captures (100%)** — semaphore acquired. - **Canary: r3 = 0x00000102 in 3/4 captures (75%)** — WAIT_TIMEOUT. The two handles being waited on are: - **handle[0] = NtCreateEvent** at `[r31+88]` — the STOP event (signal → exit). - **handle[1] = NtCreateSemaphore(InitialCount=0, MaximumCount=0x7FFFFFFF)** at `[r31+92]` — the WORK semaphore (signal → process work). Both created by `sub_8244FF50` (spawn helper) BEFORE `ExCreateThread`. mem-watch confirms handle slots in ours: `0x104C` (event) / `0x1050` (semaphore) at run-1; absolute IDs drift across runs but the slot layout is invariant. This is **NOT an exit-branch divergence, NOT loop-underrun in the literal sense — it is a SEMAPHORE-STATE divergence**. In ours the work-semaphore count is non-zero at every wait entry (so the wait always returns immediately with success); in canary the count is zero at most wait entries (so the wait times out per the 16ms relative timeout). ## Method (READ-ONLY, no source mod) 1. Disassembled `sub_82450A68` body (80 instructions) via `xenia-rs disasm --at 0x82450A68 -n 200`. Saved to `s4/sub_82450A68-disasm.txt`. 2. Identified loop topology: prolog → wait-#1 → body (with inner search over 5-slot table at [r31+112..212]) → dispatch (bl 0x82450B68 → γ-signaler family) → re-wait → back-edge OR exit. 3. Ran ours-cold with `--branch-probe=` on 14 BB-entry PCs covering all loop-body paths. Captured 696 records over ~80s wallclock / 91 loop iterations. 4. Ran canary-cold (cache wiped → restored from `/tmp/canary-cache-bak-audit-068`) with same `audit_61_branch_probe_pcs` cvar set. Canary process faulted in vkd3d-proton at ~10s wallclock; captured 35 records / 4 loop iterations. Sufficient to surface the r3 distribution. 5. Used `--mem-watch=0x828F3BC0,0x828F3BC4` to identify which ours handle IDs live in slots `[r31+88]` and `[r31+92]`. Then disassembled `sub_8244FF50` to confirm event-vs-semaphore types via the import jumps (`NtCreateEvent` at 0x824A9F18, `NtCreateSemaphore` at 0x824AB0C0). 6. Cross-checked ours's kernel handlers (`nt_wait_for_multiple_objects_ex`, `do_wait_multiple`, `handle_consume`, `nt_release_semaphore`, `try_release_semaphore`, `wake_eligible_waiters`) — code looks correct in isolation; the divergence is NOT in these handlers directly. ## Per-PC iteration counts | PC | path | ours fires | canary fires | note | |---|---|---:|---:|---| | 0x82450AA4 | first-iter entry | 1 | 1 | both entered once | | 0x82450AAC | back-edge target | 91 | 4 | canary crashed early | | 0x82450AC0 | flag@212==0 → r4=5 | 2 | 0 | rare path | | 0x82450AC8 | flag@212!=0 → search | 90 | 4 | dominant | | 0x82450AE4 | inner-search continue | 72 | 17 | | | 0x82450AF4 | search-exhausted | 8 | 3 | no candidate found | | 0x82450AF8 | candidate-found | 82 | 1 | | | 0x82450B04 | budget skip | 81 | 0 | | | 0x82450B10 | budget refresh | 8 | 0 | | | 0x82450B28 | dispatch entry | 74 | 1 | bl 0x82450B68 | | 0x82450B34 | re-wait entry | 92 | 4 | | | **0x82450B50** | **EXIT (epilog)** | **0** | **0** | **never reached** | ## r3 at back-edge (the divergence signal) | | ours | canary | |---|---|---| | r3=0x1 | 91/91 (100%) | 1/4 (25%) | | r3=0x102 (TIMEOUT) | 0/91 (0%) | 3/4 (75%) | | r3=0x0 (handle[0] signaled) | 0/91 | 0/4 | | r3=other | 0/91 | 0/4 | This is the **per-iteration measurement** the user's framing predicted. The matching iterations show different r3 values at the SAME PC. The "load feeding the predicate" is, however, NOT a guest-memory load — it is the kernel-side return of `NtWaitForMultipleObjectsEx`. The divergent KERNEL STATE is the work-semaphore count. ## Wait wrapper chain (disasm-derived) ``` sub_824AB240: li r7, 0 ; alertable = 0 b 0x824AB190 ; tail-jump sub_824AB190(r3=numObj, r4=&handles, r5=WaitMode, r6=Timeout(=16 ms), r7=Alertable): ... bl 0x824ACA88 ; converts r4=16 ms → LARGE_INTEGER -160000 (relative 100-ns ticks) ... bl 0x8284E08C ; NtWaitForMultipleObjectsEx (ord 254, import @ VA 0x8284E08C) ; returns NTSTATUS in r3: ; 0 = WAIT_OBJECT_0 = handle[0] (stop event) signaled ; 1 = WAIT_OBJECT_0+1 = handle[1] (work semaphore) acquired (atomically decrements count by 1) ; 0x102 = WAIT_TIMEOUT = 16 ms elapsed with no signal ``` `sub_82450A68` branches on this: - `cmplwi cr6, r3, 0; beq cr6, 0xB50` → r3 == 0 → EXIT (stop event signaled) - `cmplwi cr6, r3, 0; bne cr6, 0xAAC` → r3 != 0 (including 0x102) → CONTINUE - r3 == 1 → at least one work-item is available → run the inner table search - r3 == 0x102 → just a 16ms timer wake; inner search will likely find no candidate and the loop just re-waits In canary's brief 4-iteration captured window, only iteration-0 had real work (`r3=1`); iterations 1-3 were timer-wakes (`r3=0x102`). In ours's 91-iteration window, all back-edges saw `r3=1`: someone has released the semaphore at least once between each consume. ## Handle slot identification (HIGH confidence) Via `--mem-watch=0x828F3BC0,0x828F3BC4`: ``` MEM-WATCH addr=0x828f3bc0 old=0x00000000 new=0x0000104c store_addr=0x828f3bc0 store_len=4 tid=1 pc=0x8244ffb0 lr=0x8244ffb0 MEM-WATCH addr=0x828f3bc4 old=0x00000000 new=0x00001050 store_addr=0x828f3bc4 store_len=4 tid=1 pc=0x8244ffcc lr=0x8244ffcc ``` Static disasm of writer PCs: ``` 0x8244FFAC: bl 0x824A9F18 ; NtCreateEvent wrapper 0x8244FFB0: stw r3, 88(r30) ; handle[0] = event = ours 0x104C 0x8244FFC8: bl 0x824AB0C0 ; NtCreateSemaphore wrapper (r4=0=Initial, r5=0x7FFFFFFF=Max) 0x8244FFCC: stw r3, 92(r30) ; handle[1] = semaphore = ours 0x1050 ``` The semaphore is created with **InitialCount=0**. So if no one ever calls `NtReleaseSemaphore` on it, the wait will only ever return `STATUS_TIMEOUT`. Canary's behavior (mostly 0x102, occasionally 0x1) matches this: producers release the semaphore ~1× per ~16ms. Ours's behavior (always 0x1) means **producers release the semaphore FASTER THAN the consumer drains it**. ## NtReleaseSemaphore call graph (xrefs to wrapper sub_824AB158) Wrapper sub_824AB158 calls NtReleaseSemaphore (ord 243, import @ VA 0x8284E07C). Called from 22 sites across 18 functions: ``` 0x822c6770 fn=0x822c6748 0x822c6848 fn=0x822c6808 0x822c95c4 .. 0x822c9718 fn=0x822c8b50 (×6 inline call sites) 0x822f23e8 fn=0x822f2328 0x823dd7f8 fn=0x823dd770 0x823dda3c fn=0x823dd838 0x823df008..1b4 fn=0x823de4b8 (×3) 0x823df604 fn=0x823df320 0x82450310 fn=0x82450218 ← dispatcher-module enqueuer (callers: sub_82452DC0 ×2) 0x824504c4 fn=0x824503A0 ← dispatcher-module enqueuer (callers: sub_82452690, sub_8245E1D8) 0x82450cdc fn=0x82450b68 ← THE DISPATCH FUNCTION itself (self-release) 0x82450d28 fn=0x82450b68 ← THE DISPATCH FUNCTION itself (self-release) 0x82456b48 fn=0x824569c0 (jump form) 0x82458020 fn=0x82457fe0 0x824584c8 fn=0x82458468 0x82459424 fn=0x824591c0 0x8245ab6c fn=0x8245aaf0 0x8245ac6c fn=0x8245abd8 0x8245ade0 fn=0x8245ad00 ``` **Critical observation**: the dispatch function `sub_82450B68` contains TWO release sites (at offsets 0xCDC, 0xD28). Each successful dispatch run can release the semaphore again. If both branches release +1 token, and the wait consumes only -1 per iteration, the count would drift up. This is consistent with the "ours over-released" hypothesis. Some sub_82450B68 branches release the semaphore via `lwz r3, 92(r27)` which is `handle[1]` of the dispatcher itself. So the dispatch function re-fills its own pipe. ## Hypothesis (MEDIUM-HIGH confidence) The semaphore is being over-released in ours due to a divergent **dispatch-loop control flow inside `sub_82450B68`** that differentially decides whether to fire the self-release. Either: (a) ours takes a sub_82450B68 branch that releases when canary's doesn't (this is the dual of S3's question: which sub-branches differ?), OR (b) ours's parse_timeout scales the 16 ms relative timeout by /100 (exports.rs:4495 — `magnitude.max(1) / 100`), turning a 16 ms wall-clock timeout into 1,600 emulator-ticks. This may differentially interact with how often the semaphore gets a release between wait entries. The exit-branch-at-matching-iteration framing from the user's task spec does NOT apply here: there IS no exit-branch divergence (both never exit). The divergence is in the wait return value, which has no proximate guest-memory load. The "load feeding the predicate" is a kernel-state read (the semaphore count) performed inside the kernel import handler itself. ## Most-recent kernel calls (tid=5 in ours, from S3 lr-trace data + S4 cross-check) Most-recent kernel calls before each wait at PC=0x82450B44 (re-wait site), on ours tid=5: - `NtReleaseSemaphore(handle=0x1050, count=1)` via wrapper sub_824AB158, lr=0x82450CDC OR lr=0x82450D28 (both inside sub_82450B68 dispatch body) — self-release in the dispatch tail. - `KeSetEvent(handle=0x10xx)` via wrapper sub_824AA2F0 OR sub_824AAF50 — γ-signaler family fires (the audit's original signaler PCs from S1/S3). - `KeQueryPerformanceCounter`-like via sub_824AA830 — used in budget refresh path. In **canary**, the equivalent sequence per S1's signal-probe-correlated.log (180s window) is similar (γ-signalers fire 492× on tid=10), but the SELF-RELEASE rate matters more — that determines whether the consumer keeps seeing a non-zero semaphore. ## S5 recommendation (refined) The right next step is **NOT** to walk further upstream in the γ-signaler chain (S3's lead). It is to **measure the per-branch flow inside `sub_82450B68` itself** — find which of its many branches release the semaphore and how that branch is selected. ### Path A (RECOMMENDED, ~0 LOC, read-only) `--branch-probe` covering `sub_82450B68` body (PCs 0x82450B68 .. 0x82451238, the dispatch body). Want to capture: 1. Frequency at the two release sites `0x82450CDC` and `0x82450D28` (per-call cumulative count on tid=5). 2. Frequency at the OTHER exit sites in sub_82450B68 (e.g. the early return at `0x82450EE8` which does NOT release). If ours's release-rate at CDC/D28 is significantly higher than canary's, that confirms (a). If similar, then (b) becomes the next theory. ### Path B (~80 LOC ours-side probe, no source mod) Use `--branch-probe` on PCs inside `xenia_kernel::exports::parse_timeout` to confirm the magnitude/100 scaling actually causes the divergence. Actually this requires source instrumentation since parse_timeout is Rust, not guest code. Mid-priority. ### Path C (~30 LOC canary diagnostic) Add canary cvar `audit_69_semaphore_count_probe = VA` that emits the post-Set count for the semaphore at native VA matching ours's [r31+92]'s underlying X_KSEMAPHORE. Compare per-iteration count progression canary-vs-ours. LOC budget for S5: Path A = 0, Path B = ~80, Path C = ~30. **Path A first** — narrows the divergence to specific sub_82450B68 sub-branch behavior at zero LOC cost. ## Cascade - **A** (disasm sub_82450A68): PASS (HIGH) — 80-instruction body, 3 BB-paths, 12 BB-entries identified. - **B** (ours per-iteration loop-branch trace): PASS (HIGH) — 91 back-edge captures, all r3=0x1. - **C** (canary same trace): PARTIAL (MEDIUM) — canary crashed at 4 iterations in vkd3d-proton on exit; 4 captures sufficient to surface r3=0x102 dominance, but not a long-window comparison. - **D** (identify divergent load): PARTIAL (MEDIUM) — no guest-memory load is the proximate cause; the divergence is in the kernel-side semaphore-count state. The "load" is conceptually inside `do_wait_multiple`'s read of `KernelObject::Semaphore.count`. Net 2/4 PASS-HIGH, 2/4 PARTIAL-MEDIUM. Methodology learned: when both engines stay in a loop, "which branch did ours take differently" is the WRONG question — ask "what's different at the SAME branch." ## Confidence flags (summary) | finding | confidence | |---|---| | Both engines never take exit-branch (B50) | HIGH | | ours back-edge r3=1 always (91/91) | HIGH | | canary back-edge r3=0x102 mostly (3/4) | HIGH | | handle[1] is NtCreateSemaphore w/ InitialCount=0 | HIGH | | handle[0] is NtCreateEvent | HIGH | | Divergence is kernel-side semaphore-count state | MEDIUM-HIGH | | sub_82450B68 self-release over-fires in ours | MEDIUM | | parse_timeout /100 scaling is contributing | LOW-MEDIUM | ## Discipline - xenia-rs HEAD `e6d43a23ac393004d2e5adf2f0395fd0b5e6448b` UNCHANGED (sha256 of `git diff HEAD` matches S1/S2/S3 end at session start AND end). - READ-ONLY ours. No source mod. `--branch-probe` / `--lr-trace` / `--mem-watch` / `--trace-handles-focus` are runtime read-only flags documented as "lockstep digest unaffected" (state.rs comments). - Canary `audit_61_branch_probe_pcs` cvar enabled with our PC set; set back to "" at session end. Verified. - Canary `mute = true` set during run, restored to `false` at session end. - Canary cache wiped before cold canary run, restored from `/tmp/canary-cache-bak-audit-068` at session end. ## Artifacts ``` audit-runs/audit-069-wait-signal-producer/s4/ sub_82450A68-disasm.txt (80 ins disasm: sub_82450A28 entry + body) ours-loop-branch-trace.stdout (696 BRANCH-PROBE records, ours-cold) ours-loop-branch-trace.stderr (empty under --quiet) canary-loop-branch-trace.stdout (1074 lines, 35 AUDIT-061-BR records) canary-loop-branch-trace.stderr (89 lines, wine/vkd3d setup + final fault) ours-mem-watch.stderr (2 MEM-WATCH records identifying handle slots) ours-mem-watch.stdout (empty) ours-signaler.jsonl (95 lr-trace records on wrapper PCs) ours-handles.{stdout,stderr} (probe for handle dump; --halt-on-deadlock didn't trigger) ours-trace-handles-summary.log (21 lines: focus startup + 8 ExCreateThread spawns) divergence-analysis.md (per-iter table, hypothesis, S5 leads) writer-report-v4.md (this file) ``` No canary instrumentation diff this session. No `fix-canary-s4.diff`. ## Summary of S1 → S2 → S3 → S4 arc - **S1** (2026-05-20 AM): identified canary tid=10 as the signaler; claimed ours lacks this thread (FALSIFIED by S2). - **S2** (2026-05-20 noon): spawn-chain runs identically on ours tid=5; refined to "wrong-handle selection" downstream (REFINED by S3). - **S3** (2026-05-20 PM): ours runs identical PC/LR chain but with ~5× fewer iterations. Producer-loop underrun classification. Wedge handle never even created in ours's truncated boot. - **S4** (2026-05-20 evening): per-iteration branch-probe shows **NEITHER engine ever exits the loop**. Divergence is in `NtWaitForMultipleObjectsEx` return: ours r3=1 always (semaphore acquired), canary r3=0x102 mostly (timeout). Root cause is **semaphore-count state divergence** — ours's work-semaphore is over-released relative to consume rate, OR ours's timeout never fires before signal. Hypothesis: divergence inside `sub_82450B68` dispatch body's self-release logic. The S5 question is no longer "which earlier kernel call differs" — it is "which sub-branch of `sub_82450B68` releases the semaphore in ours that canary's doesn't release in." Read-only branch-probe on sub_82450B68 body PCs.