xenia-rs/audit-runs/audit-069-wait-signal-producer/writer-report-v4.md

# AUDIT-069 Session 4 — writer report v4

Date: 2026-05-20
xenia-rs HEAD: `e6d43a23ac393004d2e5adf2f0395fd0b5e6448b` (UNCHANGED from S1/S2/S3)
`git diff HEAD | sha256sum`: `ed30fd526643918f67311caff0a10d1346d73fd0c0323e02477883cf5ff20357`
   (UNCHANGED at start AND end of S4)
No ours source modifications. No canary instrumentation added.
Canary `audit_61_branch_probe_pcs` cvar used (pre-existing from S1).
Canary cache restored from `/tmp/canary-cache-bak-audit-068`.

## Headline (HIGH confidence — direct per-iteration measurement)

S3's "producer-loop underrun" framing pointed in the right direction
but mis-located the divergence. **Neither engine ever takes the
exit-branch in `sub_82450A68` (PC=0x82450B50, the LR=epilog path)**.
Both engines's dispatch threads stay in the loop indefinitely (no
deadlock; just waiting).

The actual divergence is in the **return value of the
`NtWaitForMultipleObjectsEx` call at PC=0x82450B44**:

- **Ours: r3 = 0x00000001 in 91/91 captures (100%)** — semaphore acquired.
- **Canary: r3 = 0x00000102 in 3/4 captures (75%)** — WAIT_TIMEOUT.

The two handles being waited on are:
- **handle[0] = NtCreateEvent** at `[r31+88]` — the STOP event (signal → exit).
- **handle[1] = NtCreateSemaphore(InitialCount=0, MaximumCount=0x7FFFFFFF)**
  at `[r31+92]` — the WORK semaphore (signal → process work).

Both created by `sub_8244FF50` (spawn helper) BEFORE `ExCreateThread`.
mem-watch confirms handle slots in ours: `0x104C` (event) / `0x1050`
(semaphore) at run-1; absolute IDs drift across runs but the slot
layout is invariant.

This is **NOT an exit-branch divergence, NOT loop-underrun in the
literal sense — it is a SEMAPHORE-STATE divergence**. In ours the
work-semaphore count is non-zero at every wait entry (so the wait
always returns immediately with success); in canary the count is zero
at most wait entries (so the wait times out per the 16ms relative
timeout).

## Method (READ-ONLY, no source mod)

1. Disassembled `sub_82450A68` body (80 instructions) via
   `xenia-rs disasm --at 0x82450A68 -n 200`. Saved to
   `s4/sub_82450A68-disasm.txt`.
2. Identified loop topology: prolog → wait-#1 → body (with inner search
   over 5-slot table at [r31+112..212]) → dispatch (bl 0x82450B68 →
   γ-signaler family) → re-wait → back-edge OR exit.
3. Ran ours-cold with `--branch-probe=` on 14 BB-entry PCs covering all
   loop-body paths. Captured 696 records over ~80s wallclock /
   91 loop iterations.
4. Ran canary-cold (cache wiped → restored from
   `/tmp/canary-cache-bak-audit-068`) with same `audit_61_branch_probe_pcs`
   cvar set. Canary process faulted in vkd3d-proton at ~10s wallclock;
   captured 35 records / 4 loop iterations. Sufficient to surface the
   r3 distribution.
5. Used `--mem-watch=0x828F3BC0,0x828F3BC4` to identify which ours
   handle IDs live in slots `[r31+88]` and `[r31+92]`. Then
   disassembled `sub_8244FF50` to confirm event-vs-semaphore types via
   the import jumps (`NtCreateEvent` at 0x824A9F18, `NtCreateSemaphore`
   at 0x824AB0C0).
6. Cross-checked ours's kernel handlers (`nt_wait_for_multiple_objects_ex`,
   `do_wait_multiple`, `handle_consume`, `nt_release_semaphore`,
   `try_release_semaphore`, `wake_eligible_waiters`) — code looks
   correct in isolation; the divergence is NOT in these handlers
   directly.

## Per-PC iteration counts

| PC | path | ours fires | canary fires | note |
|---|---|---:|---:|---|
| 0x82450AA4 | first-iter entry | 1 | 1 | both entered once |
| 0x82450AAC | back-edge target | 91 | 4 | canary crashed early |
| 0x82450AC0 | flag@212==0 → r4=5 | 2 | 0 | rare path |
| 0x82450AC8 | flag@212!=0 → search | 90 | 4 | dominant |
| 0x82450AE4 | inner-search continue | 72 | 17 | |
| 0x82450AF4 | search-exhausted | 8 | 3 | no candidate found |
| 0x82450AF8 | candidate-found | 82 | 1 | |
| 0x82450B04 | budget skip | 81 | 0 | |
| 0x82450B10 | budget refresh | 8 | 0 | |
| 0x82450B28 | dispatch entry | 74 | 1 | bl 0x82450B68 |
| 0x82450B34 | re-wait entry | 92 | 4 | |
| **0x82450B50** | **EXIT (epilog)** | **0** | **0** | **never reached** |

## r3 at back-edge (the divergence signal)

| | ours | canary |
|---|---|---|
| r3=0x1 | 91/91 (100%) | 1/4 (25%) |
| r3=0x102 (TIMEOUT) | 0/91 (0%) | 3/4 (75%) |
| r3=0x0 (handle[0] signaled) | 0/91 | 0/4 |
| r3=other | 0/91 | 0/4 |

This is the **per-iteration measurement** the user's framing predicted.
The matching iterations show different r3 values at the SAME PC. The
"load feeding the predicate" is, however, NOT a guest-memory load — it
is the kernel-side return of `NtWaitForMultipleObjectsEx`. The
divergent KERNEL STATE is the work-semaphore count.

## Wait wrapper chain (disasm-derived)

```
sub_824AB240:
  li r7, 0          ; alertable = 0
  b 0x824AB190      ; tail-jump

sub_824AB190(r3=numObj, r4=&handles, r5=WaitMode, r6=Timeout(=16 ms), r7=Alertable):
  ...
  bl 0x824ACA88     ; converts r4=16 ms → LARGE_INTEGER -160000 (relative 100-ns ticks)
  ...
  bl 0x8284E08C     ; NtWaitForMultipleObjectsEx (ord 254, import @ VA 0x8284E08C)
  ; returns NTSTATUS in r3:
  ;   0      = WAIT_OBJECT_0   = handle[0] (stop event) signaled
  ;   1      = WAIT_OBJECT_0+1 = handle[1] (work semaphore) acquired (atomically decrements count by 1)
  ;   0x102  = WAIT_TIMEOUT    = 16 ms elapsed with no signal
```

`sub_82450A68` branches on this:
- `cmplwi cr6, r3, 0; beq cr6, 0xB50` → r3 == 0 → EXIT (stop event signaled)
- `cmplwi cr6, r3, 0; bne cr6, 0xAAC` → r3 != 0 (including 0x102) → CONTINUE
  - r3 == 1 → at least one work-item is available → run the inner table search
  - r3 == 0x102 → just a 16ms timer wake; inner search will likely find no candidate
    and the loop just re-waits

In canary's brief 4-iteration captured window, only iteration-0 had real
work (`r3=1`); iterations 1-3 were timer-wakes (`r3=0x102`). In ours's
91-iteration window, all back-edges saw `r3=1`: someone has released
the semaphore at least once between each consume.

## Handle slot identification (HIGH confidence)

Via `--mem-watch=0x828F3BC0,0x828F3BC4`:

```
MEM-WATCH addr=0x828f3bc0 old=0x00000000 new=0x0000104c
   store_addr=0x828f3bc0 store_len=4 tid=1 pc=0x8244ffb0 lr=0x8244ffb0
MEM-WATCH addr=0x828f3bc4 old=0x00000000 new=0x00001050
   store_addr=0x828f3bc4 store_len=4 tid=1 pc=0x8244ffcc lr=0x8244ffcc
```

Static disasm of writer PCs:
```
0x8244FFAC: bl 0x824A9F18    ; NtCreateEvent wrapper
0x8244FFB0: stw r3, 88(r30)  ; handle[0] = event = ours 0x104C
0x8244FFC8: bl 0x824AB0C0    ; NtCreateSemaphore wrapper (r4=0=Initial, r5=0x7FFFFFFF=Max)
0x8244FFCC: stw r3, 92(r30)  ; handle[1] = semaphore = ours 0x1050
```

The semaphore is created with **InitialCount=0**. So if no one ever
calls `NtReleaseSemaphore` on it, the wait will only ever return
`STATUS_TIMEOUT`. Canary's behavior (mostly 0x102, occasionally 0x1)
matches this: producers release the semaphore ~1× per ~16ms.

Ours's behavior (always 0x1) means **producers release the semaphore
FASTER THAN the consumer drains it**.

## NtReleaseSemaphore call graph (xrefs to wrapper sub_824AB158)

Wrapper sub_824AB158 calls NtReleaseSemaphore (ord 243, import @
VA 0x8284E07C). Called from 22 sites across 18 functions:

```
0x822c6770 fn=0x822c6748
0x822c6848 fn=0x822c6808
0x822c95c4 .. 0x822c9718 fn=0x822c8b50 (×6 inline call sites)
0x822f23e8 fn=0x822f2328
0x823dd7f8 fn=0x823dd770
0x823dda3c fn=0x823dd838
0x823df008..1b4 fn=0x823de4b8 (×3)
0x823df604 fn=0x823df320
0x82450310 fn=0x82450218   ← dispatcher-module enqueuer (callers: sub_82452DC0 ×2)
0x824504c4 fn=0x824503A0   ← dispatcher-module enqueuer (callers: sub_82452690, sub_8245E1D8)
0x82450cdc fn=0x82450b68   ← THE DISPATCH FUNCTION itself (self-release)
0x82450d28 fn=0x82450b68   ← THE DISPATCH FUNCTION itself (self-release)
0x82456b48 fn=0x824569c0 (jump form)
0x82458020 fn=0x82457fe0
0x824584c8 fn=0x82458468
0x82459424 fn=0x824591c0
0x8245ab6c fn=0x8245aaf0
0x8245ac6c fn=0x8245abd8
0x8245ade0 fn=0x8245ad00
```

**Critical observation**: the dispatch function `sub_82450B68`
contains TWO release sites (at offsets 0xCDC, 0xD28). Each successful
dispatch run can release the semaphore again. If both branches release
+1 token, and the wait consumes only -1 per iteration, the count would
drift up. This is consistent with the "ours over-released" hypothesis.

Some sub_82450B68 branches release the semaphore via `lwz r3, 92(r27)`
which is `handle[1]` of the dispatcher itself. So the dispatch function
re-fills its own pipe.

## Hypothesis (MEDIUM-HIGH confidence)

The semaphore is being over-released in ours due to a divergent
**dispatch-loop control flow inside `sub_82450B68`** that
differentially decides whether to fire the self-release. Either:
(a) ours takes a sub_82450B68 branch that releases when canary's doesn't
(this is the dual of S3's question: which sub-branches differ?), OR
(b) ours's parse_timeout scales the 16 ms relative timeout by /100
    (exports.rs:4495 — `magnitude.max(1) / 100`), turning a 16 ms wall-clock
    timeout into 1,600 emulator-ticks. This may differentially interact
    with how often the semaphore gets a release between wait entries.

The exit-branch-at-matching-iteration framing from the user's task spec
does NOT apply here: there IS no exit-branch divergence (both never
exit). The divergence is in the wait return value, which has no
proximate guest-memory load. The "load feeding the predicate" is a
kernel-state read (the semaphore count) performed inside the kernel
import handler itself.

## Most-recent kernel calls (tid=5 in ours, from S3 lr-trace
data + S4 cross-check)

Most-recent kernel calls before each wait at PC=0x82450B44 (re-wait
site), on ours tid=5:

- `NtReleaseSemaphore(handle=0x1050, count=1)` via wrapper
  sub_824AB158, lr=0x82450CDC OR lr=0x82450D28 (both inside sub_82450B68
  dispatch body) — self-release in the dispatch tail.
- `KeSetEvent(handle=0x10xx)` via wrapper sub_824AA2F0 OR sub_824AAF50 —
  γ-signaler family fires (the audit's original signaler PCs from S1/S3).
- `KeQueryPerformanceCounter`-like via sub_824AA830 — used in budget
  refresh path.

In **canary**, the equivalent sequence per S1's signal-probe-correlated.log
(180s window) is similar (γ-signalers fire 492× on tid=10), but the
SELF-RELEASE rate matters more — that determines whether the consumer
keeps seeing a non-zero semaphore.

## S5 recommendation (refined)

The right next step is **NOT** to walk further upstream in the
γ-signaler chain (S3's lead). It is to **measure the per-branch flow
inside `sub_82450B68` itself** — find which of its many branches
release the semaphore and how that branch is selected.

### Path A (RECOMMENDED, ~0 LOC, read-only)

`--branch-probe` covering `sub_82450B68` body (PCs 0x82450B68 ..
0x82451238, the dispatch body). Want to capture:

1. Frequency at the two release sites `0x82450CDC` and `0x82450D28`
   (per-call cumulative count on tid=5).
2. Frequency at the OTHER exit sites in sub_82450B68 (e.g. the early
   return at `0x82450EE8` which does NOT release).

If ours's release-rate at CDC/D28 is significantly higher than canary's,
that confirms (a). If similar, then (b) becomes the next theory.

### Path B (~80 LOC ours-side probe, no source mod)

Use `--branch-probe` on PCs inside `xenia_kernel::exports::parse_timeout`
to confirm the magnitude/100 scaling actually causes the divergence.
Actually this requires source instrumentation since parse_timeout is
Rust, not guest code. Mid-priority.

### Path C (~30 LOC canary diagnostic)

Add canary cvar `audit_69_semaphore_count_probe = VA` that emits the
post-Set count for the semaphore at native VA matching ours's
[r31+92]'s underlying X_KSEMAPHORE. Compare per-iteration count
progression canary-vs-ours.

LOC budget for S5: Path A = 0, Path B = ~80, Path C = ~30.

**Path A first** — narrows the divergence to specific sub_82450B68
sub-branch behavior at zero LOC cost.

## Cascade

- **A** (disasm sub_82450A68): PASS (HIGH) — 80-instruction body,
  3 BB-paths, 12 BB-entries identified.
- **B** (ours per-iteration loop-branch trace): PASS (HIGH) —
  91 back-edge captures, all r3=0x1.
- **C** (canary same trace): PARTIAL (MEDIUM) — canary crashed at
  4 iterations in vkd3d-proton on exit; 4 captures sufficient to surface
  r3=0x102 dominance, but not a long-window comparison.
- **D** (identify divergent load): PARTIAL (MEDIUM) — no guest-memory
  load is the proximate cause; the divergence is in the kernel-side
  semaphore-count state. The "load" is conceptually inside
  `do_wait_multiple`'s read of `KernelObject::Semaphore.count`.

Net 2/4 PASS-HIGH, 2/4 PARTIAL-MEDIUM. Methodology learned: when both
engines stay in a loop, "which branch did ours take differently" is the
WRONG question — ask "what's different at the SAME branch."

## Confidence flags (summary)

| finding | confidence |
|---|---|
| Both engines never take exit-branch (B50) | HIGH |
| ours back-edge r3=1 always (91/91) | HIGH |
| canary back-edge r3=0x102 mostly (3/4) | HIGH |
| handle[1] is NtCreateSemaphore w/ InitialCount=0 | HIGH |
| handle[0] is NtCreateEvent | HIGH |
| Divergence is kernel-side semaphore-count state | MEDIUM-HIGH |
| sub_82450B68 self-release over-fires in ours | MEDIUM |
| parse_timeout /100 scaling is contributing | LOW-MEDIUM |

## Discipline

- xenia-rs HEAD `e6d43a23ac393004d2e5adf2f0395fd0b5e6448b` UNCHANGED
  (sha256 of `git diff HEAD` matches S1/S2/S3 end at session start AND end).
- READ-ONLY ours. No source mod. `--branch-probe` / `--lr-trace` /
  `--mem-watch` / `--trace-handles-focus` are runtime read-only flags
  documented as "lockstep digest unaffected" (state.rs comments).
- Canary `audit_61_branch_probe_pcs` cvar enabled with our PC set; set
  back to "" at session end. Verified.
- Canary `mute = true` set during run, restored to `false` at session end.
- Canary cache wiped before cold canary run, restored from
  `/tmp/canary-cache-bak-audit-068` at session end.

## Artifacts

```
audit-runs/audit-069-wait-signal-producer/s4/
  sub_82450A68-disasm.txt          (80 ins disasm: sub_82450A28 entry + body)
  ours-loop-branch-trace.stdout    (696 BRANCH-PROBE records, ours-cold)
  ours-loop-branch-trace.stderr    (empty under --quiet)
  canary-loop-branch-trace.stdout  (1074 lines, 35 AUDIT-061-BR records)
  canary-loop-branch-trace.stderr  (89 lines, wine/vkd3d setup + final fault)
  ours-mem-watch.stderr            (2 MEM-WATCH records identifying handle slots)
  ours-mem-watch.stdout            (empty)
  ours-signaler.jsonl              (95 lr-trace records on wrapper PCs)
  ours-handles.{stdout,stderr}     (probe for handle dump; --halt-on-deadlock didn't trigger)
  ours-trace-handles-summary.log   (21 lines: focus startup + 8 ExCreateThread spawns)
  divergence-analysis.md           (per-iter table, hypothesis, S5 leads)
  writer-report-v4.md              (this file)
```

No canary instrumentation diff this session. No `fix-canary-s4.diff`.

## Summary of S1 → S2 → S3 → S4 arc

- **S1** (2026-05-20 AM): identified canary tid=10 as the signaler;
  claimed ours lacks this thread (FALSIFIED by S2).
- **S2** (2026-05-20 noon): spawn-chain runs identically on ours tid=5;
  refined to "wrong-handle selection" downstream (REFINED by S3).
- **S3** (2026-05-20 PM): ours runs identical PC/LR chain but with
  ~5× fewer iterations. Producer-loop underrun classification.
  Wedge handle never even created in ours's truncated boot.
- **S4** (2026-05-20 evening): per-iteration branch-probe shows
  **NEITHER engine ever exits the loop**. Divergence is in
  `NtWaitForMultipleObjectsEx` return: ours r3=1 always (semaphore
  acquired), canary r3=0x102 mostly (timeout). Root cause is
  **semaphore-count state divergence** — ours's work-semaphore is
  over-released relative to consume rate, OR ours's timeout never
  fires before signal. Hypothesis: divergence inside `sub_82450B68`
  dispatch body's self-release logic.

The S5 question is no longer "which earlier kernel call differs" —
it is "which sub-branch of `sub_82450B68` releases the semaphore in
ours that canary's doesn't release in." Read-only branch-probe on
sub_82450B68 body PCs.