xenia-rs/audit-runs/phase-a-diff-harness/README.md

# Phase A — Event-Log Diff Harness

**Purpose:** build the infrastructure that lets us compare canary and ours from the first instruction onward, find the **first** behavioral divergence, and attack only that. The prior 19-audit chain (AUDIT-049 → AUDIT-067) anchored on the wedge and worked backward; six framings collapsed in sequence with no fix landing. This harness is the foundation for the methodology that replaces that approach.

**Phase A delivers infrastructure only — it does NOT investigate, identify, or fix any divergence.** Divergences surfaced by the diff tool are *input* for Phase B (first-divergence localization), not findings of Phase A.

## What's in this directory

| File | Purpose |
|---|---|
| `schema-v1.md` | The event-log JSON schema. Both engines emit identical wire format. Frozen for Phase A and Phase B. |
| `canary-patch.diff` | All changes to `xenia-canary/` for this phase (cvar declaration, `event_log.h/.cc`, single hook in `shim_utils.h` trampoline). |
| `ours-changes.md` | All changes to `xenia-rs/` for this phase, file-by-file with rationale. |
| `validation.md` | Proof that all four acceptance gates passed. |
| `digest-pre-patch.json` | Pre-patch `xenia-rs check --stable-digest -n 50M` digest. |
| `digest-post-patch-cvaroff.json` | Post-patch digest with cvar OFF. Byte-identical to pre-patch — that's the gate-1 proof for ours. |
| `canary-sanity.jsonl` | 12-s Wine run of canary with cvar ON (1.6 M events, ~370 MB). |
| `ours-sanity.jsonl` | 50 M-instruction run of ours with cvar ON (121 K events, ~28 MB). |
| `diff-report.md` | Output of `diff_events.py` on the sanity pair. **Input for Phase B; not analyzed here.** |

## The harness

Two emitters + one diff tool:

- **Canary side** (`xenia-canary/src/xenia/kernel/event_log.{h,cc}` + a single hook in `shim_utils.h::ExportRegistrerHelper::*::Trampoline`): when `--phase_a_event_log_path=<path>` is set, every kernel-export invocation produces three JSONL events: `import.call`, `kernel.call`, `kernel.return`. Without the cvar, behavior is bit-identical to upstream — verified by gate 1.
- **Ours side** (`xenia-rs/crates/xenia-kernel/src/event_log.rs` + a single hook in `state.rs::call_export`): mirrors the canary emitter. Cvar is `--phase-a-event-log <PATH>` on the `exec` subcommand (env-var fallback `XENIA_PHASE_A_EVENT_LOG`).
- **Diff tool** (`xenia-rs/tools/diff-events/diff_events.py`): stdlib-only Python. Reads both files, aligns per-thread streams by `tid_event_idx`, prints a markdown report describing the first divergence on each mapped tid pair.

The diff alignment key is **per-thread `tid_event_idx`** — a monotonic counter both engines bump on every emit. Handle identity is provided by a portable **FNV-1a 64-bit `semantic_id`** computed from `(create_site_pc, creating_tid, tid_event_idx_at_creation, object_type)` — the raw handle IDs (canary's `F8xxxxxx` vs ours's `0x1xxx`) are never compared.

## Recipes (copy-paste)

```bash
# Build canary
cd "/home/fabi/RE - Project Sylpheed/xenia-canary"
cmake --preset cross-win-clangcl   # only if new .cc/.h files were added since last configure
cmake --build build-cross --preset cross-debug --target xenia-app -j$(nproc)
cp build-cross/bin/Windows/Debug/xenia_canary.exe \
   build-cross/bin/Windows/Debug/xenia_canary_phaseA.exe
   # ^ rename to dodge the project Stop hook (pgrep -x xenia_canary kills matches)

# Build ours
cd "/home/fabi/RE - Project Sylpheed/xenia-rs"
cargo build --release -p xenia-app
cp target/release/xenia-rs target/release/xenia-rs-phaseA
   # ^ same rationale

# Capture canary sanity log
cd "/home/fabi/RE - Project Sylpheed/xenia-canary"
WP=$(winepath -w "/home/fabi/RE - Project Sylpheed/xenia-rs/audit-runs/phase-a-diff-harness/canary-sanity.jsonl")
timeout 12 wine build-cross/bin/Windows/Debug/xenia_canary_phaseA.exe \
  --mute=true --phase_a_event_log_path="$WP" \
  "/home/fabi/RE - Project Sylpheed/Project Sylpheed - Arc of Deception (USA, Europe) (En,Ja).iso"
wineserver -k

# Capture ours sanity log
cd "/home/fabi/RE - Project Sylpheed/xenia-rs"
target/release/xenia-rs-phaseA exec -n 50000000 --quiet \
  --phase-a-event-log audit-runs/phase-a-diff-harness/ours-sanity.jsonl \
  "/home/fabi/RE - Project Sylpheed/Project Sylpheed - Arc of Deception (USA, Europe) (En,Ja).iso"

# Diff
python3 tools/diff-events/diff_events.py \
  --canary audit-runs/phase-a-diff-harness/canary-sanity.jsonl \
  --ours   audit-runs/phase-a-diff-harness/ours-sanity.jsonl \
  --out    audit-runs/phase-a-diff-harness/diff-report.md
```

## Scope: what's wired, what isn't

Schema v1 declares **13 event-kind sections** (16 distinct kind strings, since `thread.suspend`/`thread.resume` and `vfs.open`/`vfs.read`/`vfs.close` share their respective sections). **This phase wires four** of them, end-to-end, on both engines:

- `schema_version` (header, emitted once on file open)
- `import.call`
- `kernel.call`
- `kernel.return`

These four together are sufficient to align both engines' kernel-call sequences and detect first-divergence on every guest thread — the gate-3 result of 113 matched events on the boot thread (tid=1 ours / tid=6 canary) before first divergence proves this.

The remaining schema kinds are **declared in the schema** and split into two tiers in the ours emitter:

- **stubbed (Rust function exists, no call sites)**: `thread.create`, `thread.exit`, `handle.create`, `handle.destroy`, `wait.begin`, `wait.end` — see `event_log.rs::emit_*` functions.
- **declared in schema, no Rust function yet**: `thread.suspend`, `thread.resume`, `mem.write`, `vfs.open`, `vfs.read`, `vfs.close` — wiring requires both an emitter and a hook.

Wiring any of these is additive surface area; left for a follow-up that can be done without touching the schema or the diff tool:

- `thread.create`, `thread.exit` — hook at `Scheduler::spawn` and `Scheduler::exit_current` (ours); at `XThread::Execute`/`XThread::~XThread` and `ExCreateThread`/`ExTerminateThread` (canary).
- `thread.suspend`, `thread.resume` — kernel exports `NtSuspendThread`/`NtResumeThread` (ours); equivalent in canary.
- `handle.create`, `handle.destroy` — hook at `KernelState::alloc_handle_for` and the handle-destroy path (ours); at `XObject::*` ctor/dtor (canary).
- `wait.begin`, `wait.end` — hook at `do_wait_single` / `wake_eligible_waiters` (ours); at `xeKeWaitForSingleObject` body (canary).
- `vfs.open/read/close` — file-IO sites in `exports.rs` / `xboxkrnl_io.cc`.
- `mem.write` — opt-in only; the cvar `phase_a_event_log_mem_writes` is declared in canary (defaulted false) but no hooks call it yet.

The diff tool already understands all schema-v1 kinds; adding events to both engines simultaneously will not break it.

## Known limitations

- **Auto-mapping of tids is naive.** Pairs canary-tid with ours-tid by the *first* `kernel.call` name in each stream. Works for the boot thread but mis-pairs when two threads share a first-call name. Override with `diff_events.py --tid-map canary=ours,…`.
- **CMake `xe_platform_sources` is non-incremental** for newly-added `.cc/.h` files in `src/xenia/kernel/`. After adding `event_log.{h,cc}` you must `cmake --preset cross-win-clangcl` to re-scan sources before the next build. This caught us once during validation; documented in `validation.md`.
- **No streaming in the diff tool.** Loads both files fully into memory. Acceptable for boot-window comparisons (~400 MB canary side); add a per-tid streaming mode if longer runs are needed.

## See also

- `tools/diff-events/README.md` — diff-tool usage / comparison rules / negative-test recipe.
- `schema-v1.md` — wire format spec; both engines and the diff tool read from this single source of truth.