fabi/xenia-rs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: fabi:cc9ebbb5e770fbec853df4045ad35eeb1b402bf9
Branches Tags
fabi:master fabi:iterate-4A/apu-xma-stage1 fabi:iterate-3O/real-render-slice fabi:iterate-3M/logo-item-texture fabi:iterate-2Z/idx2-logo-load fabi:iterate-2X/texture-logo fabi:iterate-2W/per-present-interrupt fabi:iterate-2V/vdswap-no-wptr-bump fabi:iterate-2U/vdglobaldevice-cell fabi:iterate-2T/vdswap-present-interrupt fabi:iterate-2S/swap-callback-arm fabi:iterate-2O/gpu-syscmdbuf-drain fabi:iterate-2P/texture-decode-resolve fabi:iterate-2Q/vblank-swap-producer fabi:iterate-2M/pcr-current-cpu fabi:iterate-2K/physical-mirror-aliasing fabi:iterate-2J/timestamp-bundle-tick fabi:iterate-2I/worker-0x109c fabi:iterate-2BG/slab-recycle-handles fabi:iterate-2BF/synthetic-silph-spawn fabi:audit-2BF/sub82172BA0-bctrl-probe fabi:iterate-2BE/host-driven-isr-delivery fabi:handoff/2026-06-05-continue-elsewhere fabi:iterate-2AT-deref fabi:iterate-2AU-xaudio fabi:iterate-2AZ-vsync fabi:chore/portable-snapshot fabi:worktree-agent-a0848e51cc0d72503
fabi:intro-video-done
...
compare: fabi:5950dc86d50b9b20e3ae7a9648141dd46be2fd02
Branches Tags
fabi:iterate-4A/apu-xma-stage1 fabi:iterate-3O/real-render-slice fabi:iterate-3M/logo-item-texture fabi:iterate-2Z/idx2-logo-load fabi:master fabi:iterate-2X/texture-logo fabi:iterate-2W/per-present-interrupt fabi:iterate-2V/vdswap-no-wptr-bump fabi:iterate-2U/vdglobaldevice-cell fabi:iterate-2T/vdswap-present-interrupt fabi:iterate-2S/swap-callback-arm fabi:iterate-2O/gpu-syscmdbuf-drain fabi:iterate-2P/texture-decode-resolve fabi:iterate-2Q/vblank-swap-producer fabi:iterate-2M/pcr-current-cpu fabi:iterate-2K/physical-mirror-aliasing fabi:iterate-2J/timestamp-bundle-tick fabi:iterate-2I/worker-0x109c fabi:iterate-2BG/slab-recycle-handles fabi:iterate-2BF/synthetic-silph-spawn fabi:audit-2BF/sub82172BA0-bctrl-probe fabi:iterate-2BE/host-driven-isr-delivery fabi:handoff/2026-06-05-continue-elsewhere fabi:iterate-2AT-deref fabi:iterate-2AU-xaudio fabi:iterate-2AZ-vsync fabi:chore/portable-snapshot fabi:worktree-agent-a0848e51cc0d72503
fabi:intro-video-done

16 Commits
cc9ebbb5e7 ... 5950dc86d5

Author SHA1 Message Date
MechaCat02
5950dc86d5 [iterate-4A] jit: coverage — update-form loads/stores + mfspr/mtspr(LR/CTR) → 89.6% native 
Second coverage batch, data-driven from the histogram. Diff harness validates
every op against the interpreter (MISMATCHES=0 over 47.96M blocks).

Update-form load/stores (24 ops: lbzu..stdux, lfsu..stfdux and their x-forms):
same access as the base form but with EA = gpr[ra] + offset (ra used directly —
update forms are illegal with ra==0) and rA := EA written back (zero-extended)
after the access. New ea_d_update/ea_x_update/write_ea_back helpers; loads write
rD then rA (so rA wins if rd==ra, matching the interpreter). stwu especially is
the standard stack-frame push in every function prologue.

mfspr/mtspr for LR and CTR only (64-bit field copies): the ubiquitous mflr/mtlr
prologue-epilogue pair and mtctr. covered() gates on the compile-time SPR number
so the offset always resolves; XER (packs CA/OV/SO) and the modelled SPRs stay
uncovered. This is the biggest single jump — mflr/mtlr gate almost every
non-leaf function's prologue and epilogue blocks.

Coverage (single-block, diff mode): 80.6% -> 82.8% (update forms) -> 89.6%
(mfspr/mtspr). 7/7 jit unit tests; foreground-validated per the bg-SIGTERM
finding. Carry-setting shifts (sraw*/srad*), rotate-double (rldic*), and XER
mfspr/mtspr remain for later batches; VMX128 is the long pole.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-08 07:06:42 +02:00
MechaCat02
0becc9185a [iterate-4A] jit: coverage — cheap integer ALU ops (+5pt native, chain -14% vs interp) 
Data-driven from XENIA_JIT_HIST: after register caching the JIT was 73% native;
the frequent uncovered opcodes are VMX128 (the long pole), mtspr/mfspr, update-
form loads/stores, and a batch of cheap pure-register integer ops. This does the
cheap integer batch (each unblocks any block that only lacked it, and — since a
covered block can now chain through where an uncovered op used to break the
chain — the gain compounds: more native execution AND longer superblocks).

Added to covered() + emit_op, all validated against the interpreter by the diff
harness (none are sync_sensitive, so fully covered):
 - extsbx/extshx/extswx  — sign-extend byte/half/word (extsb/extsh write the
   i32-view zero-extended per the 32-bit ABI, CR0 i32; extsw sign-extends into
   the full 64, CR0 i64).
 - cntlzwx/cntlzdx        — count leading zeros (Cranelift clz).
 - negx (OE=0)           — rD = 0 - rA (ineg), full 64-bit, CR0 on low 32.
 - slwx/srwx             — word shifts; explicit `sh<32 ? shift : 0` select
   because the count is rB[58:63] (6 bits) and Cranelift's ishl/ushr mask to 5.
 - sldx/srdx             — doubleword shifts, `sh<64 ? shift : 0` (count 7 bits).
New emit_cr0_signed64 for the doubleword-result CR0 (a value with low32==0 but
high bits set is eq in the 32-bit view but not the 64-bit one). The carry-setting
shifts (sraw*/srad*, set XER-CA) and rotate-double (rldic*) are deferred to later
batches.

Validated (foreground per the bg-SIGTERM finding): 7/7 jit unit tests; diff
MISMATCHES=0 over 47.9M blocks; coverage 75.4% -> 80.6% (single-block, diff
mode). Perf (RUST_LOG=warn, -n 3e9 to amortize compiles): chain wall 57952ms ->
54748ms (-5.5% from coverage alone; run_block calls 110M -> 98M = more chaining),
chain -14.3% vs interp (63.9s) this session. -n 2e9 understates it (more blocks
compiled = more compile cost inside the STEP timer at under-amortized -n).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-08 06:33:01 +02:00
MechaCat02
e3cc6d11d0 [iterate-4A] jit: per-block GPR register caching — bodies now beat interp 
Guest GPRs were reloaded from PpcContext memory on every use and stored back
after every write, forcing memory traffic that held the JIT bodies at mere
interpreter parity. Now each GPR is loaded ONCE per block into a Cranelift SSA
value (kept in a host register by the register allocator); writes update an
in-EmitCtx cache (RegCache: [Option<Value>;32] + dirty bits) without storing;
dirty GPRs are flushed to memory only at the block boundary (cache_flush at the
end of emit_node_body).

Sound because nothing a block calls back into touches guest GPRs: the memory
trampolines take addr/val as args (I pass the cached SSA values), and the
FP-punt runs interpreter FP ops that touch fpr/fpscr/cr only. So the cache never
goes stale mid-block; flushing at the boundary makes gpr[] interpreter-identical
for the runner and the next block. FPR/CR still go through memory (the FP-punt
writes them, which would need flush-around-punt — deferred).

emit_op / ea_d / ea_x / gpr32 / store_gpr32z rewired to cache_read_gpr /
cache_write_gpr; cache_read_gpr copies the Option out before re-borrowing (a
RefCell double-borrow would panic).

Validated: 7/7 jit unit tests; diff MISMATCHES=0 over 47.9M blocks (foreground).
Perf (foreground, RUST_LOG=warn, -n 3e9 to amortize compile cost — -n<=1.5e9 is
compile-dominated and misleads): interp 95.8 MIPS -> jit-no-chain 100.2 MIPS
(bodies now BEAT interp, were parity) -> chain 100.4 MIPS / 57.95s = -15.6% wall
vs interp (was ~-6% at B2), dispatch 18.9%, 110M calls (1.95x merge). Same-batch
-n 2e9: interp 44.8s -> chain 41.0s = -8.6%. Register caching is what makes the
bodies fast enough that chaining's dispatch cut converts to real wall time
(B2's bodies had slowed to ~82 MIPS and cancelled the win).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-08 06:15:16 +02:00
MechaCat02
4b78c67605 [iterate-4A] jit: superblock chaining B2 — bcx two-way + loop back-edges 
Extends B1's forward-linear superblock into a full same-page CFG: chains
through conditional bcx (BOTH directions) and loop back-edges (a successor
already in the chain jumps to its existing IR block), not just unconditional
bx + fall-through. This roughly doubles the block-merge ratio (run_block calls
145.7M interp -> 116.2M B1 (1.25x) -> 74.7M B2 (1.95x)) and halves the
run_superblock dispatch bucket (30.4% interp -> 26.2% B1 -> 16.2% B2).

Mechanism:
 - enumerate_chain is now a two-pass CFG builder: BFS the reachable, chainable
   (same-page / covered / non-thunk) blocks deduped by start PC (so a back-edge
   becomes a loop, not a new node), capped at MAX_CHAIN_NODES=64; then resolve
   each terminator to Succ edges (node indices or Exit). bx/fall -> One; bcx ->
   Two{taken,fall} (each side Node or Exit); bclrx/bcctrx/uncovered -> Exit.
 - emit_bcx extracted from emit_op's bcx arm: computes the taken predicate ONCE
   (decrementing CTR at most once) and returns it; emit_node_body captures it so
   the two-way boundary branches on the SAME value (recomputing would double the
   CTR decrement). Single-block compile is unchanged (ignores the return).
 - No Cranelift Variables/phi needed for the loop CFG: the budget/MMIO state
   lives in memory (cycle_count, *mmio_count), so emit_stop reloads it and
   compares against entry-sampled constants (cycle_entry/mmio_entry), which the
   entry block dominates across back-edges. Budget = (cycle_count - cycle_entry)
   >= remaining, correct across loop iterations; a native loop spins until the
   budget is spent then hands back — same bound as the runner. MMIO check still
   skipped for load/store-free blocks. seal_all_blocks handles the arbitrary CFG.
 - Degenerate guard fixed: compile single-block only when the ENTRY has no
   chainable successor (Succ::Exit) — a one-node self-loop is NOT degenerate.

Validation: 3 new unit tests (bcx always-taken chains to target; bcx not-taken
falls through; self-loop runs natively until budget then stops — all bit-equal
to the interpreter over the same PCs) -> 7/7 jit tests pass. Diff MISMATCHES=0
over 47.95M blocks (the emit_bcx refactor didn't perturb op bodies).
XENIA_JIT_CHAIN=1 movie plays: tid25 resumes (decode handoff / interleaving
preserved under the deeper chaining).

Perf finding (honest): despite ~2x merge and half the dispatch, wall is ~parity
with B1 (both ~-6% vs interp same-batch). step_block MIPS dropped (~82 vs B1
~102) because the deeper/bigger native superblocks cost more per boundary (B1's
compile-time-constant budget became a runtime cycle_count reload, needed for
loops) and Cranelift's default regalloc spills more in larger functions. The
dispatch savings are real but offset by native-code overhead that grows with
chain length. Converting the extra chaining into wall time now needs
host-register allocation (tighter code) — the clear next lever. Env gate,
diff-off-under-chain, and probe/mem-watch exclusivity unchanged from B1.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-07 21:25:21 +02:00
MechaCat02
2c17a511f7 [iterate-4A] jit: native superblock chaining B1 (bx + fall-through), −6.4% wall 
Increment B1 of block-linking: compile a chain of same-page, fully-covered
blocks into ONE Cranelift function whose internal block-to-block edges are
direct machine jumps, eliminating the per-block return to run_block and the
run_superblock dispatch tax for the chain. Env-gated XENIA_JIT_CHAIN (off by
default), and additionally disabled under the diff harness or with diagnostic
probes / mem-watch armed (chaining runs several blocks natively without the
per-block-entry observation those need, so they are mutually exclusive).

Mechanism (cranelift-jit can't patch finalized code, so no QEMU-style TB
chaining): SUPERBLOCK COMPILATION. compile_chain enumerates the forward-linear
chain from the entry — following static bx targets and fall-throughs, all
same-page (so the entry page_version is a sound cache key), outside the thunk
band, and covered — and emits one node per block plus a shared exit. At every
internal boundary it re-checks the two runtime stop-conditions exactly as the
Rust runner does: an MMIO touch (*mmio_count advanced vs the entry sample) and
the instruction budget (cumulative >= remaining, passed as a runtime param);
either stops the chain with pc/cycle/timebase already correct, and the runner
re-dispatches from the clean boundary. B1 scope is forward-linear: a bx or
fall-through continues; a conditional bcx, dynamic bclrx/bcctrx, off-page /
thunk / uncovered / already-visited successor ends the chain (B2 adds bcx
two-way + loop back-edges). Being *more* conservative (stopping earlier) is
always safe.

Correctness rests on composition: each node body is the same emit_node_body IR
the single-block path emits (already diff-validated bit-exact), so only the
chain glue is new — validated by two new unit tests (multi-block chain vs
interpreter over the same PCs; budget cut at the exact boundary) plus e2e.
Boundary MMIO check is skipped for load/store-free blocks (a pure-ALU block
can't advance mmio_count), dropping a memory load+compare per ALU boundary.

Plumbing: build_block exposed (xenia-cpu); KernelState::thunk_band getter;
MemEnv mmio_count null-safety (points at a static zero when no flat mapping);
CompiledFn gains a remaining-budget param (single-block ignores it); all
compiled entries share the (ctx, mem_env, remaining) ABI.

Validation: diff MISMATCHES=0 over 25.0M blocks (single-block op bodies
unbroken by the emit_node_body refactor); XENIA_JIT_CHAIN=1 movie plays,
tid25 resumes, source-read=12; fair perf (RUST_LOG=warn) interp 44.1s vs
chain 41.3s = −6.4% wall — run_superblock "other" 30.4%->26.2%, run_block
calls 145.7M->116.2M (1.25x block merge), step_block at parity (100.6->102.4
MIPS; the earlier "slower" reading was cranelift IR-logging inside the STEP
timer). Uncommitted probe knobs unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-07 20:55:12 +02:00
MechaCat02
5d5dc5dd89 [iterate-4A] jit: cache MemEnv fast_mem + expose mmio_count (prep for block chaining) 
Behavior-neutral prep on top of the inline-load parity milestone:

  * JitCache caches mem.fast_mem() (resolved once — the mapping is invariant for
    a run) so compiled blocks build their MemEnv without a virtual call per
    execution. MemEnv::from_fast(mem, Option<FastMem>) is the non-virtual
    constructor. Measured impact is within noise (96.5 -> 96.4 MIPS) but it
    removes redundant per-block work.
  * FastMem/MemEnv gain mmio_count: a pointer to GuestMemory's monotonic MMIO
    access counter. Unused for now; the upcoming superblock-chaining JIT will
    sample it across a block boundary to stop chaining on an MMIO touch
    (preserving the interpreter's fine-grained MMIO ordering).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-07 20:02:05 +02:00
MechaCat02
da0509b4ac [iterate-4A] jit: inline load fast-path (skip trampoline) — JIT reaches interpreter parity 
Loads no longer call a trampoline on the common path. When memory exposes a flat
mapping and the address is proven non-MMIO and committed, the compiled block
loads directly from `membase + ea` (byte-swapped) with zero calls; otherwise it
falls back to the read trampoline.

Mechanism:
  * MemoryAccess::fast_mem() -> Option<FastMem { membase, page_table, mmio_mask,
    mmio_value }>. Default None; GuestMemory returns Some (flat 4 GiB mapping,
    page-table pointer, MMIO aperture pair). Wrappers that intercept accesses
    (the recompiler's speculative OverlayMemory) inherit None, so the fast path
    is disabled under the diff harness and their interception is preserved.
  * MemEnv carries membase/page_table/mmio_mask/mmio_value; MemEnv::new(mem)
    fills them (or nulls when fast_mem() is None).
  * emit_inline_load emits a 5-block diamond: membase==0 -> slow (disabled);
    (ea & mmio_mask) == mmio_value -> slow (maybe MMIO); page_table[ea>>12]
    COMMIT bit (49) clear -> slow (unmapped; the mapped check is mandatory —
    unmapped pages are PROT_NONE and a raw load would fault); else fast load
    membase+ea, bswap, zero-extend. Wired all integer loads (lbz/lhz/lwz/ld +
    x-forms) and FP loads (lfs/lfsx reuse the diamond then bitcast/fpromote;
    lfd/lfdx bit-copy). Stores and FP-punt still trampoline.

Validation (three ways, since the diff harness disables the fast path):
  * unit test jit_inline_load_matches_interpreter — real GuestMemory, both the
    mapped (fast) and unmapped (slow) address bit-match the interpreter.
  * e2e XENIA_JIT=1 boot+movie plays, clean exit.
  * diff regression: checked 149.2M blocks, MISMATCHES=0 (validates the
    unchanged slow/ALU/branch/FP logic).

Measured (2e9 instr, block-exec MIPS): interpreter 98.2 / JIT trampoline-loads
93.8 / JIT inline-loads 96.5 — inline loads recover the trampoline overhead,
bringing the JIT to parity (~1.7% slower, within noise). Beating the interpreter
needs the levers it can't do: block-linking (30% dispatch), host registers.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-07 19:44:56 +02:00
MechaCat02
d91311c486 [iterate-4A] jit: FP arithmetic via interpreter-punt (diff-clean, 73.48% native) 
Solve the FP-arithmetic fpscr problem without lowering fpscr into IR. Add a
generic single-instruction shim, xj_interp_op(ctx, env, raw, addr), that decodes
and runs one instruction through the interpreter's execute() on the live
context/memory — bit-exact by construction. This lets a block containing FP math
still be JIT-compiled: the surrounding integer/memory/branch ops run as machine
code and only the FP op calls back into Rust (the technique production JITs use
for complex ops). Validated on a full boot+movie run (movie plays, clean exit):

  checked 147.2M blocks, 73.48% native (108.2M, up from 39.82%!), MISMATCHES=0

The fpscr bookkeeping (rounding-mode-dependent rounding, sticky exception bits,
FPRF classification) was too intricate to emit correctly in IR; punting sidesteps
it entirely while still capturing the coverage.

is_fp_punt allowlist (all verified always-Continue, non-branch):
  faddx/faddsx, fsubx/fsubsx, fmulx/fmulsx, fdivx/fdivsx, fmaddx/fmaddsx,
  fmsubx/fmsubsx, fnmaddx/fnmaddsx, fnmsubx/fnmsubsx, frspx, fsqrtx, fresx,
  frsqrtex, fselx, fnabsx, fcmpu, fcmpo.
covered() returns true for them; emit_op emits `call interp(ctx, env, raw,
addr)`. execute() advances pc by 4 (harmless — the block epilogue stamps end_pc
for non-branch blocks; cycle_count is bumped once per block, not by execute).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-07 19:18:06 +02:00
MechaCat02
704e87a2a4 [iterate-4A] jit: FP-move tier — lfs/stfs/lfd/stfd + fmr/fabs/fneg (diff-clean, 39.82% native) 
Cover the fpscr-free FP moves. Validated bit-exact via the in-process
differential harness on a full boot+movie run (movie plays, clean exit):

  checked 146.2M blocks, 39.82% native (up from 38.78%), MISMATCHES=0

Coverage (all fpscr-free — pure data movement, no rounding flags):
  * FP loads: lfs/lfsx (load single, IEEE-widen to the f64 FPR via
    read32 -> bitcast F32 -> fpromote F64), lfd/lfdx (pure 64-bit bit copy).
  * FP stores: stfs/stfsx (fdemote f64 -> f32 -> bitcast i32 -> write32),
    stfd/stfdx (pure 64-bit bit copy). Stores reuse the reservation-kicking
    write trampolines.
  * FP reg moves (Rc=0 only — the `.` forms update CR1 from fpscr): fmr (bit
    copy), fabs (band_imm i64::MAX, clear sign), fneg (bxor_imm i64::MIN, flip
    sign) — bit ops that exactly match Rust f64 copy/abs/neg.

Note: the +1% lift is small because the movie's FP-heavy blocks almost always
mix an FP load with FP *arithmetic* (fadds/fmuls/fmadds), which stays
uncovered — it updates fpscr (FPRF/FI/FR, invalid-op flags), not yet lowered.
The moves that landed are the pure data-copy blocks.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-07 19:09:15 +02:00
MechaCat02
6475f4ba97 [iterate-4A] jit: memory tier — integer loads/stores via trampolines (diff-clean, 38.78% native) 
Add the memory tier: compiled loads/stores call `extern "C"` trampolines that
dispatch through the `MemoryAccess` trait, so MMIO dispatch, mem-watch,
page_version, and mmio_access_count stay bit-identical to the interpreter.
Validated bit-exact via the in-process differential harness on a full
boot+movie run (movie plays, clean exit):

  checked 148.2M blocks, 38.78% native (57.5M, up from 22.02%), MISMATCHES=0

Mechanism:
  * 8 trampolines: xj_read8/16/32/64(env, addr) and
    xj_write8/16/32/64(ctx, env, addr, val). Registered with the JITBuilder via
    symbol(), declared Linkage::Import in Jit::new (FuncIds in TrampIds), and
    re-referenced into each compiled function via declare_func_in_func.
  * emit_op now takes an EmitCtx { ctxp, memenv, trampoline FuncRefs }.
  * Store trampolines replicate the interpreter's pre-store reservation
    invalidation (store_reservation_kick) — an ordinary store to a reserved
    line must be observed by stwcx peers. They receive the PpcContext pointer so
    they can read ctx.reservation_table; the diff clone clears it (None), so
    speculation never touches shared reservation state.

Coverage added (all mirroring execute() exactly):
  * loads (zero-extended, non-update): lbz/lbzx, lhz/lhzx, lwz/lwzx, ld/ldx.
  * stores (non-update): stb/stbx, sth/sthx, stw/stwx, std/stdx.
  EA via ea_d (D-form disp) / ea_x (X-form indexed): ra==0 => 0 base, then
  truncate to 32 bits. Update (u) forms and algebraic sign-extending loads
  (lha/lwa) are not lowered yet.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-07 18:57:19 +02:00
MechaCat02
15d0d7e0bd [iterate-4A] jit: cover integer ALU + compares w/ CR emission (diff-clean, 22% native) 
Grow the covered opcode set from branches-only to the hot integer core.
Validated bit-exact vs the interpreter via the in-process differential harness
on a full boot+movie run (movie plays, clean exit):

  checked 147.9M blocks, 22.02% native (32.6M, up from 6.57%), MISMATCHES=0

New coverage (all mirroring interpreter::execute exactly):
  * add/sub: addx, subfx (OE=0 only — the overflow path is not lowered yet).
    Full 64-bit result; CR0 (when Rc) from the low-32 signed value.
  * reg-reg logical, 64-bit-preserving: orx (excluding the 0x7FFFFB78 db16cyc
    spin hint, which yields), andx, xorx.
  * reg-reg logical, u32-truncating (zeroes the upper 32): norx, nandx, andcx,
    orcx — stored zero-extended via store_gpr32z.
  * immediate logical: ori/oris/xori/xoris (64-bit, no CR); andi./andis.
    (always update CR0).
  * rlwinmx: rotate-left-word + mask (mask computed at emit time from the mb/me
    immediates); zeroes upper 32; CR0 when Rc.
  * compares: cmp/cmpi (signed, 64- or 32-bit per L), cmpl/cmpli (unsigned);
    write the crfd() field. Immediates sign- or zero-extended per form.

CR-write emission:
  * emit_store_cr(field, lt, gt, eq): stores the four CrField bytes
    {lt@0,gt@1,eq@2,so@3}; so = (xer_so != 0).
  * emit_cr0_signed32(val32): the Rc-form CR0 update — signed compare of the
    32-bit result against zero. Reuses emit_store_cr.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-07 07:36:49 +02:00
MechaCat02
56dbf52a5f [iterate-4A] jit: wire compiled-block cache + cover bx/bcx/bclrx (diff-clean, 6.57% native) 
Wire the Cranelift block-JIT into execution and add branch coverage. Both
increments validated bit-exact against the interpreter via the in-process
differential harness on a full boot+movie run (movie plays, clean exit):

  wiring only (addi/addis): checked 146.8M blocks, 0.01% native, MISMATCHES=0
  + branches (bx/bcx/bclrx): checked 148.5M blocks, 6.57% native (9.76M), MISMATCHES=0

jit.rs
  * JitCache: direct-mapped 64K-slot compiled-block cache keyed (start_pc,
    page_version) identically to BlockCache, so self-modifying / DMA'd code
    invalidates native code the same way. Caches the None ("uncovered") verdict
    so an uncovered block is compile-attempted at most once per (pc,version).
    Owns the Jit/JITModule (keeps every CompiledFn valid for its lifetime).
  * covered(): addi/addis + bx/bcx/bclrx. bcctrx excluded (indirect target +
    dispatch_rec diagnostic hook the native path would skip).
  * pc-handling refactor: a branch terminator writes pc itself (writes_pc());
    the block epilogue stores end_pc only for straight-line (max-len / page-
    boundary) blocks. cycle/timebase still += N (covered ops never fault/yield;
    a branch is always the last instruction).
  * Branch lowering uses immediate targets — the interpreter's ctx.pc equals the
    instruction address at emit time, so bx/bcx relative targets are constants.
    emit_branch_taken mirrors the interpreter: optional CTR decrement, ctr_ok =
    (ctr as u32 vs 0) inverted by BO3, cond_ok = CR-bit BI byte == BO1 (both BO
    sub-cases const-fold), combined with select. bclrx reads lr&!3 before the LK
    link overwrites lr.

recompiler.rs
  * run_block / diff_step take &mut JitCache. run_block runs the native fn when
    get_or_compile returns one (else interpreter fallback); diff_step runs it on
    the speculative clone/OverlayMemory so every native block is diff-checked.
  * report_jit_summary(): native-vs-interpreted block counts (XENIA_JIT / _DIFF).

main.rs
  * WorkerCtx owns a JitCache; run_superblock routing passes it to
    diff_step/run_block; report_jit_summary() at clean exit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-07 07:28:26 +02:00
MechaCat02
cc7ff58cb0 [iterate-4A] jit: Cranelift block-JIT scaffolding + addi/addis PoC (pipeline proven) 
Stage-2 foundation. Adds cranelift-jit/frontend/module/codegen 0.128.4 (the
1.90-compatible line; 0.133 needs Rust 1.94) and a new crates/xenia-cpu/src/jit.rs.

Jit owns a JITModule (and thus all compiled code memory). compile(&DecodedBlock)
lowers a block to native code ONLY if every opcode is covered() — otherwise
returns None and the caller interprets the whole block (coverage grows
opcode-by-opcode). ABI: extern "C" fn(*mut PpcContext, *const MemEnv) -> u32
(StepResult discriminant); guest registers are loaded/stored directly from the
#[repr(C)] PpcContext at offset_of! offsets. A covered block is straight-line and
always runs to completion, so pc advances to end_pc and cycle_count/timebase bump
by the instruction count once — matching the interpreter's per-instruction bump.

Covered set so far: addi, addis. Unit test jit_matches_interpreter_addi_block
compiles a 32x addi block and asserts the JIT's r3/pc/cycle_count/timebase match
the interpreter exactly — proves module setup, offset_of register access, IR
emission, the extern "C" calling convention, and cycle/pc accounting end-to-end.

Not yet wired into run_superblock (needs a compiled-block cache + routing); every
future opcode will be validated against the interpreter via the M0 XENIA_JIT_DIFF
harness before it counts as covered.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-06 20:45:31 +02:00
MechaCat02
07cd272412 [iterate-4A] jit: opcode-frequency histogram (XENIA_JIT_HIST) for coverage targeting 
Gated 1-in-16 sampled opcode histogram in the recompiler (tallied in run_block,
printed sorted with cumulative % at clean exit). Drives which opcodes the JIT
lowers first. Boot+movie result: workload is FP-heavy — addi 15%, lwz 12%,
rlwinm 10%, lfs 9%, stfs 7.5%, lfsx 5%, fmaddsx 4.6%, bc 3.7%, fmulsx 3.6%,
stw 3.4% ... top-20 = 90%, ~33% floating-point. This is why we go straight to a
Cranelift machine-code JIT (which attacks the FP op bodies) rather than a
closure-threaded stage (which only removes dispatch).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-06 20:36:53 +02:00
MechaCat02
ac289bca5e [iterate-4A] jit M0: recompiler seam + in-process differential harness 
Foundation for a staged PPC block-recompiler (plan: closure-threaded ->
Cranelift -> block-linking). No codegen yet — proves the integration seam and
the correctness harness before any lowering exists.

New crates/xenia-cpu/src/recompiler.rs:
- run_block: executes a DecodedBlock; M0 falls back to the interpreter's
  execute() for every opcode, so it is bit-identical to step_block. Later
  stages dispatch lowered ops here and fall back only for uncompiled opcodes.
- gates jit_enabled()/diff_enabled() (XENIA_JIT / XENIA_JIT_DIFF, cached).
- In-process differential harness (diff_step): the interpreter is AUTHORITATIVE
  (drives real ctx+mem, so a JIT bug can never corrupt a run); the JIT runs
  SPECULATIVELY on a ctx clone against OverlayMemory (writes buffered in a byte
  HashMap, reads fall through to real pre-block memory), then registers are
  compared. Blocks touching MMIO or sync_sensitive (reservation/barrier) are
  skipped — a device callback can't be run twice and reservation state is
  shared cross-thread. report_diff_summary() prints checked/skipped/mismatch.

Why in-process: the guest is only COARSELY deterministic — coord_idle_advance
ticks vsync from wall-clock when idle, so two separate runs are not bit-exact
and a cross-run signature compare would measure jitter, not JIT divergence.

Supporting changes: PpcContext #[derive(Clone)] (harness clears the speculative
clone's reservation_table Arc); is_mmio() on the MemoryAccess trait (default
false) + GuestMemory impl via find_mmio; execute() made pub(crate); run_superblock
routes the block body (DIFF->diff_step, JIT->run_block, else step_block).

Validated: full boot+movie XENIA_JIT_DIFF=1 run = checked 218.4M blocks,
skipped(mmio/sync) 511.6K (0.23%), MISMATCHES=0 CLEAN; movie plays (ADVreads=30,
tid25 resumes); diff-mode ~2x slower (test-only path).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-06 20:25:18 +02:00
MechaCat02
e07f93ed0a [iterate-4A] perf: superblock budget 128->192 (movie-validated ~7% wall) + profiler attribution 
Speed frontier, increment 1 (measure-first, then bank the cheap win).

Profiler (crates/xenia-gpu/src/prof.rs): extend XENIA_PROFILE with hierarchical
attribution of the single-thread lockstep loop — ROUND (per-round tax) /
PROLOGUE (worker_prologue) / RUNSB (run_superblock) top level, plus the
STEP/EPILOGUE/KERNEL/BUILD subsets. Gated zero-cost via is_on() (off-check emits
0 lines). Movie-time split: interp step_block 42%, run_superblock chain-loop
body ~30%, worker_prologue+epilogue ~22%, kernel HLE 6%, per-round tax 0.6%,
texture 0.4%. => overhead-bound; the ceiling is a JIT (attacks the ~72% interp
+ dispatch). The per-round tax is a non-lever (0.6%).

Budget (crates/xenia-app/src/main.rs SUPERBLOCK_INSTR_BUDGET 128->192): the
per-slot-visit tax (prologue+epilogue ~22%) scales with slot-visit count, and
chains are NOT break-limited here — 128->192 cuts visits 27.4M->18.9M (-31%),
128->256 -44%. Banked 192 (not 256): the movie decode pipeline is more
timing-sensitive than boot. At 256 the decode worker tid25 (0x82506588)
intermittently fails to resume (1-of-2 runs) with a weakened feeder loop
(source-read 27->5-11) — a scheduling race the coarser interleaving exposes.
192 is deterministic and safe: 3/3 runs byte-identical (source-read=21,
tid25-resume=1, ADVreads=30) at both env-override and compiled-default, ~-7%
movie wall, well below the 384 boot cliff. XENIA_SUPERBLOCK_BUDGET still
overrides for A/B.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-06 19:45:23 +02:00
14 changed files with 3456 additions and 55 deletions
Expand all files Collapse all files

241
Cargo.lock generated
View File

@@ -87,6 +87,12 @@ dependencies = [
"equator",
]
[[package]]
name = "allocator-api2"
version = "0.2.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923"
[[package]]
name = "android-activity"
version = "0.6.1"
@@ -521,6 +527,9 @@ name = "bumpalo"
version = "3.20.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5d20789868f4b01b2f2caec9f5c4e0213b41e3e5702a50157d699ae31ced2fcb"
dependencies = [
"allocator-api2",
]
[[package]]
name = "bytecheck"
@@ -874,6 +883,167 @@ dependencies = [
"libc",
]
[[package]]
name = "cranelift-assembler-x64"
version = "0.128.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "50a04121a197fde2fe896f8e7cac9812fc41ed6ee9c63e1906090f9f497845f6"
dependencies = [
"cranelift-assembler-x64-meta",
]
[[package]]
name = "cranelift-assembler-x64-meta"
version = "0.128.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a09e699a94f477303820fb2167024f091543d6240783a2d3b01a3f21c42bc744"
dependencies = [
"cranelift-srcgen",
]
[[package]]
name = "cranelift-bforest"
version = "0.128.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f07732c662a9755529e332d86f8c5842171f6e98ba4d5976a178043dad838654"
dependencies = [
"cranelift-entity",
]
[[package]]
name = "cranelift-bitset"
version = "0.128.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "18391da761cf362a06def7a7cf11474d79e55801dd34c2e9ba105b33dc0aef88"
[[package]]
name = "cranelift-codegen"
version = "0.128.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b3a09b3042c69810d255aef59ddc3b3e4c0644d1d90ecfd6e3837798cc88a3c"
dependencies = [
"bumpalo",
"cranelift-assembler-x64",
"cranelift-bforest",
"cranelift-bitset",
"cranelift-codegen-meta",
"cranelift-codegen-shared",
"cranelift-control",
"cranelift-entity",
"cranelift-isle",
"gimli",
"hashbrown 0.15.5",
"log",
"regalloc2",
"rustc-hash 2.1.2",
"serde",
"smallvec",
"target-lexicon",
"wasmtime-internal-math",
]
[[package]]
name = "cranelift-codegen-meta"
version = "0.128.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "75817926ec812241889208d1b190cadb7fedded4592a4bb01b8524babb9e4849"
dependencies = [
"cranelift-assembler-x64-meta",
"cranelift-codegen-shared",
"cranelift-srcgen",
"heck",
]
[[package]]
name = "cranelift-codegen-shared"
version = "0.128.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "859158f87a59476476eda3884d883c32e08a143cf3d315095533b362a3250a63"
[[package]]
name = "cranelift-control"
version = "0.128.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "03b65a9aec442d715cbf54d14548b8f395476c09cef7abe03e104a378291ab88"
dependencies = [
"arbitrary",
]
[[package]]
name = "cranelift-entity"
version = "0.128.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8334c99a7e86060c24028732efd23bac84585770dcb752329c69f135d64f2fc1"
dependencies = [
"cranelift-bitset",
]
[[package]]
name = "cranelift-frontend"
version = "0.128.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "43ac6c095aa5b3e845d7ca3461e67e2b65249eb5401477a5ff9100369b745111"
dependencies = [
"cranelift-codegen",
"log",
"smallvec",
"target-lexicon",
]
[[package]]
name = "cranelift-isle"
version = "0.128.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "69d3d992870ed4f0f2e82e2175275cb3a123a46e9660c6558c46417b822c91fa"
[[package]]
name = "cranelift-jit"
version = "0.128.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "807781e9097feba24c31fc8d85a217c2fbfc6c8b72a87da8717c056dc8b24a87"
dependencies = [
"anyhow",
"cranelift-codegen",
"cranelift-control",
"cranelift-entity",
"cranelift-module",
"cranelift-native",
"libc",
"log",
"region",
"target-lexicon",
"wasmtime-internal-jit-icache-coherence",
"windows-sys 0.61.2",
]
[[package]]
name = "cranelift-module"
version = "0.128.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6a588624367124596cb07c324fa865a4894200aede7fe933e817816188ace52"
dependencies = [
"anyhow",
"cranelift-codegen",
"cranelift-control",
]
[[package]]
name = "cranelift-native"
version = "0.128.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ee32e36beaf80f309edb535274cfe0349e1c5cf5799ba2d9f42e828285c6b52e"
dependencies = [
"cranelift-codegen",
"libc",
"target-lexicon",
]
[[package]]
name = "cranelift-srcgen"
version = "0.128.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "903adeaf4938e60209a97b53a2e4326cd2d356aab9764a1934630204bae381c9"
[[package]]
name = "crc32fast"
version = "1.5.0"
@@ -1359,6 +1529,11 @@ name = "gimli"
version = "0.32.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e629b9b98ef3dd8afe6ca2bd0f89306cec16d43d907889945bc5d6687f2f13c7"
dependencies = [
"fallible-iterator",
"indexmap",
"stable_deref_trait",
]
[[package]]
name = "gl_generator"
@@ -2127,6 +2302,15 @@ version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "112b39cec0b298b6c1999fee3e31427f74f676e4cb9879ed1a121b43661a4154"
[[package]]
name = "mach2"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d640282b302c0bb0a2a8e0233ead9035e3bed871f0b7e81fe4a1ec829765db44"
dependencies = [
"libc",
]
[[package]]
name = "malloc_buf"
version = "0.0.6"
@@ -3183,6 +3367,20 @@ dependencies = [
"bitflags 2.11.0",
]
[[package]]
name = "regalloc2"
version = "0.13.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08effbc1fa53aaebff69521a5c05640523fab037b34a4a2c109506bc938246fa"
dependencies = [
"allocator-api2",
"bumpalo",
"hashbrown 0.15.5",
"log",
"rustc-hash 2.1.2",
"smallvec",
]
[[package]]
name = "regex"
version = "1.12.3"
@@ -3212,6 +3410,18 @@ version = "0.8.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc897dd8d9e8bd1ed8cdad82b5966c3e0ecae09fb1907d58efaa013543185d0a"
[[package]]
name = "region"
version = "3.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6b6ebd13bc009aef9cd476c1310d49ac354d36e240cf1bd753290f3dc7199a7"
dependencies = [
"bitflags 1.3.2",
"libc",
"mach2",
"windows-sys 0.52.0",
]
[[package]]
name = "rend"
version = "0.4.2"
@@ -3808,6 +4018,12 @@ dependencies = [
"xattr",
]
[[package]]
name = "target-lexicon"
version = "0.13.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "adb6935a6f5c20170eeceb1a3835a49e12e19d792f6dd344ccc76a985ca5a6ca"
[[package]]
name = "tempfile"
version = "3.27.0"
@@ -4378,6 +4594,27 @@ dependencies = [
"unicode-ident",
]
[[package]]
name = "wasmtime-internal-jit-icache-coherence"
version = "41.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4b96df23179ae16d54fb3a420f84ffe4383ec9dd06fad3e5bc782f85f66e8e08"
dependencies = [
"anyhow",
"cfg-if",
"libc",
"windows-sys 0.61.2",
]
[[package]]
name = "wasmtime-internal-math"
version = "41.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "86d1380926682b44c383e9a67f47e7a95e60c6d3fa8c072294dab2c7de6168a0"
dependencies = [
"libm",
]
[[package]]
name = "wayland-backend"
version = "0.3.15"
@@ -5071,6 +5308,10 @@ name = "xenia-cpu"
version = "0.1.0"
dependencies = [
"bitflags 2.11.0",
"cranelift-codegen",
"cranelift-frontend",
"cranelift-jit",
"cranelift-module",
"serde",
"serde_json",
"thiserror 2.0.18",

99
crates/xenia-app/src/main.rs
View File

@@ -2479,6 +2479,10 @@ struct WorkerCtx {
hw_id: u8,
block_cache: xenia_cpu::block_cache::BlockCache,
decode_cache: xenia_cpu::decoder::DecodeCache,
/// Compiled-block cache (Cranelift JIT). Keyed identically to `block_cache`
/// on `(start_pc, page_version)`; owns the JIT module and thus the native
/// code. Only consulted on the `XENIA_JIT` / `XENIA_JIT_DIFF` paths.
jit_cache: xenia_cpu::jit::JitCache,
force_per_instr: bool,
}
@@ -2488,6 +2492,7 @@ impl WorkerCtx {
hw_id,
block_cache: xenia_cpu::block_cache::BlockCache::new(),
decode_cache: xenia_cpu::decoder::DecodeCache::new(),
jit_cache: xenia_cpu::jit::JitCache::new(),
force_per_instr,
}
}
@@ -3001,10 +3006,20 @@ fn worker_epilogue(
/// swaps / packets track the one-block baseline), then a sharp cliff at
/// ~384 collapses the present loop (a producer/consumer boot handoff
/// starves when one slot runs too long without returning to the round).
/// 128 sits 3× below that cliff with ~1.65× boot-to-splash speedup — a
/// deliberately conservative pick (correctness over the last few %). The
/// `XENIA_SUPERBLOCK_BUDGET` env var overrides it for further tuning.
const SUPERBLOCK_INSTR_BUDGET: u64 = 128;
///
/// Re-tuned against the INTRO-VIDEO workload (2026-07-06, XENIA_PROFILE
/// attribution): the per-slot-visit tax (worker_prologue + worker_epilogue)
/// was ~22% of movie wall, and raising the budget cuts slot-visits ~in
/// proportion (chains are NOT break-limited here — 128→192 dropped visits
/// 27.4M→18.9M, −31%; 128→256 −44%). But the movie's decode pipeline is more
/// timing-sensitive than boot: at 256 the decode worker tid25 (0x82506588)
/// INTERMITTENTLY fails to resume (1-of-2 runs) and the feeder loop weakens
/// (source-read 27→5-11) — a scheduling race the coarser interleaving exposes.
/// 192 is the sweet spot: 3/3 runs byte-identical (source-read=21,
/// tid25-resume=1, ADVreads=30) with ~−7% movie wall, and it stays well below
/// the 384 boot cliff. Do NOT raise past 192 without re-validating tid25
/// resume across several runs. `XENIA_SUPERBLOCK_BUDGET` overrides for A/B.
const SUPERBLOCK_INSTR_BUDGET: u64 = 192;
/// Effective superblock budget. Defaults to [`SUPERBLOCK_INSTR_BUDGET`];
/// `XENIA_SUPERBLOCK_BUDGET` overrides it (A/B tuning without a rebuild).
@@ -3068,6 +3083,26 @@ fn run_superblock(
let budget = superblock_budget();
// Install the JIT superblock-chaining config once per run (idempotent). The
// gate is deliberately conservative and stable for a run: chaining requires
// `XENIA_JIT_CHAIN`, the diff harness OFF (a multi-block superblock can't be
// compared against one interpreter step), and NO diagnostic probes / mem-watch
// armed (chaining runs several blocks natively without the per-block-entry
// observation the Heisenbug fix fires inside this loop — so chaining and
// probes are mutually exclusive). Probes/mem-watch are env-armed at startup,
// so evaluating this once is sound; a cached block never has to change shape.
if !wc.jit_cache.is_configured() {
let chain_enabled = xenia_cpu::recompiler::chain_requested()
&& xenia_cpu::recompiler::jit_enabled()
&& !xenia_cpu::recompiler::diff_enabled()
&& !kernel.any_probe_active()
&& !mem.has_mem_watch();
wc.jit_cache.set_chain_config(xenia_cpu::jit::ChainConfig {
enabled: chain_enabled,
thunk_band: kernel.thunk_band(),
});
}
// Heisenbug fix (toolkit audit, 2026-06-21): probes and mem-watch are
// OBSERVE-ONLY diagnostics and must NOT change guest scheduling. The
// previous implementation disabled superblock chaining whenever any
@@ -3129,7 +3164,22 @@ fn run_superblock(
let _prof_t0 = xenia_gpu::prof::is_on().then(std::time::Instant::now);
let result = {
let ctx = kernel.scheduler.ctx_mut_ref(thread_ref);
step_block(ctx, mem, block)
// JIT seam (M0). run_superblock's chain-loop + stop-conditions are
// unchanged; only the block body is routed:
// - XENIA_JIT_DIFF: interpreter authoritative + JIT checked in-process,
// - XENIA_JIT: JIT authoritative (M0 = interpreter fallback),
// - else: interpreter directly.
if xenia_cpu::recompiler::diff_enabled() {
xenia_cpu::recompiler::diff_step(ctx, mem, block, &mut wc.jit_cache)
} else if xenia_cpu::recompiler::jit_enabled() {
// Hand the JIT the remaining budget so a superblock stops
// chaining at exactly the same instruction count the runner
// would (`total_executed >= budget`).
let remaining = budget.saturating_sub(total_executed);
xenia_cpu::recompiler::run_block(ctx, mem, block, &mut wc.jit_cache, remaining)
} else {
step_block(ctx, mem, block)
}
};
let executed = kernel
.scheduler
@@ -3194,6 +3244,12 @@ fn run_superblock(
}
};
let _ep_pt = xenia_gpu::prof::is_on().then(|| {
xenia_gpu::prof::ScopeTimer::new(
&xenia_gpu::prof::EPILOGUE_NS,
&xenia_gpu::prof::EPILOGUE_CALLS,
)
});
worker_epilogue(
wc,
kernel,
@@ -3302,6 +3358,8 @@ fn run_execution(
// then drain any pending auto-signals whose deadline has passed.
// Both calls are no-ops when `XENIA_SILPH_UI_AUTOSIGNAL_DELAY`
// is unset (the pending queue stays empty).
let _round_pt = xenia_gpu::prof::is_on()
.then(|| xenia_gpu::prof::ScopeTimer::new(&xenia_gpu::prof::ROUND_NS, &xenia_gpu::prof::ROUND_CALLS));
kernel.set_now_cycle_hint(stats.instruction_count);
// Drive the coherent monotonic "now" the kernel deadline-arithmetic
// reads (`KernelState::now_basis_at` -> `Scheduler::global_clock`)
@@ -3336,6 +3394,7 @@ fn run_execution(
// a reusable stack array instead of allocating a fresh Vec per round.
kernel.scheduler.begin_round();
let order_n = kernel.scheduler.round_schedule_into(&mut order_buf);
drop(_round_pt); // end per-round tax measurement at the schedule boundary
let order = &order_buf[..order_n];
if order.is_empty() {
@@ -3360,7 +3419,13 @@ fn run_execution(
for &hw_id in order {
let wc = &mut workers[hw_id as usize];
match worker_prologue(
let _pro_pt = xenia_gpu::prof::is_on().then(|| {
xenia_gpu::prof::ScopeTimer::new(
&xenia_gpu::prof::PROLOGUE_NS,
&xenia_gpu::prof::PROLOGUE_CALLS,
)
});
let prologue_outcome = worker_prologue(
wc,
kernel,
mem,
@@ -3368,7 +3433,9 @@ fn run_execution(
&mut db_writer,
thunk_map,
&mut stats,
) {
);
drop(_pro_pt);
match prologue_outcome {
PrologueOutcome::Continue => continue,
PrologueOutcome::BreakOuter => break 'outer,
PrologueOutcome::StepBlock {
@@ -3385,7 +3452,13 @@ fn run_execution(
// the per-round (timebase / coord / round_schedule)
// and per-slot (prologue) tax over hundreds of
// instructions instead of ~6. See `run_superblock`.
match run_superblock(
let _sb_pt = xenia_gpu::prof::is_on().then(|| {
xenia_gpu::prof::ScopeTimer::new(
&xenia_gpu::prof::RUNSB_NS,
&xenia_gpu::prof::RUNSB_CALLS,
)
});
let sb_outcome = run_superblock(
wc,
kernel,
mem,
@@ -3396,7 +3469,9 @@ fn run_execution(
thread_ref,
block_ptr,
pc_before,
) {
);
drop(_sb_pt);
match sb_outcome {
SlotOutcome::Continue => continue,
SlotOutcome::BreakOuter => break 'outer,
}
@@ -4348,6 +4423,12 @@ fn dump_thread_diagnostic(
if xenia_gpu::prof::enabled() {
xenia_gpu::prof::report(0);
}
// JIT differential harness: checked/skipped/mismatch tally at clean exit.
xenia_cpu::recompiler::report_diff_summary();
// JIT native-vs-interpreted block coverage at clean exit.
xenia_cpu::recompiler::report_jit_summary();
// JIT opcode-frequency histogram (XENIA_JIT_HIST) — drives M1 coverage.
xenia_cpu::recompiler::report_histogram();
// STEP-10 diagnostic (observe-only, env-gated `XENIA_DUMP_SLOTS=1`).
// Prints each scheduler slot's full runqueue with the fields needed to

4
crates/xenia-cpu/Cargo.toml
View File

@@ -10,6 +10,10 @@ xenia-memory = { workspace = true }
tracing = { workspace = true }
bitflags = { workspace = true }
thiserror = { workspace = true }
cranelift-jit = "0.128.4"
cranelift-frontend = "0.128.4"
cranelift-module = "0.128.4"
cranelift-codegen = "0.128.4"
[dev-dependencies]
serde = { workspace = true }

7
crates/xenia-cpu/src/block_cache.rs
View File

@@ -191,7 +191,12 @@ impl BlockCache {
/// included as the last instruction),
/// - reaching [`MAX_BLOCK_INSTRS`],
/// - the next PC would cross a 4 KiB guest page boundary.
fn build_block(start_pc: u32, mem: &dyn MemoryAccess, page_version: u64) -> DecodedBlock {
///
/// Exposed to the JIT ([`crate::jit`]) so the superblock compiler can build
/// the transient successor blocks of a chain without a `BlockCache` (they are
/// consumed at compile time, not cached). Same walk the cache uses, so a
/// superblock's blocks are byte-identical to the ones the interpreter runs.
pub fn build_block(start_pc: u32, mem: &dyn MemoryAccess, page_version: u64) -> DecodedBlock {
let mut instrs: Vec<DecodedInstr> = Vec::with_capacity(8);
let page_base = start_pc & GUEST_PAGE_MASK;
let mut cur = start_pc;

5
crates/xenia-cpu/src/context.rs
View File

@@ -62,6 +62,11 @@ pub const VSCR_SAT_MASK: u32 = 0x0000_0001;
/// PowerPC processor context. Holds all register state for one guest thread.
/// Mirrors PPCContext from ppc_context.h, minus JIT-specific fields.
// `Clone` supports the recompiler's differential harness, which snapshots a
// context to run the JIT speculatively against the interpreter. Cloning shares
// the `reservation_table` Arc; the harness clears it on the speculative copy so
// a speculative lwarx/stwcx cannot perturb the authoritative reservation state.
#[derive(Clone)]
#[repr(C, align(64))]
pub struct PpcContext {
// General purpose registers (R0-R31)

6
crates/xenia-cpu/src/interpreter.rs
View File

@@ -214,7 +214,11 @@ pub fn step_block(
}
/// Execute a decoded instruction, updating context and memory.
fn execute(ctx: &mut PpcContext, mem: &dyn MemoryAccess, instr: &DecodedInstr) -> StepResult {
///
/// `pub(crate)` so the recompiler ([`crate::recompiler`]) can reuse the exact
/// per-opcode semantics as its interpreter-fallback path — the JIT lowers from
/// this same source of truth, and uncompiled opcodes route straight back here.
pub(crate) fn execute(ctx: &mut PpcContext, mem: &dyn MemoryAccess, instr: &DecodedInstr) -> StepResult {
match instr.opcode {
// ===== ALU: Immediate =====
PpcOpcode::addi => {

2537
crates/xenia-cpu/src/jit.rs Normal file
View File

File diff suppressed because it is too large Load Diff

2
crates/xenia-cpu/src/lib.rs
View File

@@ -5,9 +5,11 @@ pub mod dispatch_rec;
pub mod disasm;
pub mod fpscr;
pub mod interpreter;
pub mod jit;
pub mod opcode;
pub mod overflow;
pub mod phaser;
pub mod recompiler;
pub mod reservation;
pub mod scheduler;
pub mod trap;

445
crates/xenia-cpu/src/recompiler.rs Normal file
View File

@@ -0,0 +1,445 @@
//! PPC block recompiler (JIT) — **M0 foundation**.
//!
//! This is the seam a staged block-recompiler plugs into. M0 ships **no real
//! codegen**: [`run_block`] executes a block by falling back to the
//! interpreter's [`crate::interpreter::execute`] for every instruction, so it
//! is bit-identical to [`crate::interpreter::step_block`] by construction. Its
//! purpose is to prove the integration seam and the **differential harness**
//! before any lowering exists. Later stages replace the per-instruction
//! fallback with lowered ops (closure-threaded, then Cranelift machine code)
//! opcode-by-opcode, each gated through the harness below.
//!
//! ## Determinism is the whole game
//! `cycle_count`/`timebase` are bumped once per guest instruction and drive the
//! IPM clock the movie/scheduler depend on. [`run_block`] reproduces
//! `step_block`'s counting and stop semantics **exactly**. The interpreter is
//! the permanent reference oracle; the recompiler must never diverge from it.
//!
//! ## Differential harness (`XENIA_JIT_DIFF`) — in-process
//! The guest is only *coarsely* deterministic: `coord_idle_advance` ticks vsync
//! from wall-clock when the scheduler idles, so vblank ISRs land at
//! wall-clock-dependent instruction boundaries and two separate runs are **not**
//! bit-identical. Comparing across runs would measure that jitter, not JIT
//! divergence. So the harness is **in-process** ([`diff_step`]):
//!
//! * the **interpreter is authoritative** — it drives the real context and
//! memory, so a JIT bug can never corrupt the run;
//! * the **JIT runs speculatively** on a *clone* of the pre-block context
//! against an [`OverlayMemory`] (writes buffered, reads fall through to real
//! *pre-block* memory), then its registers are compared to the
//! interpreter's;
//! * blocks that touch **MMIO** (a device callback can't be run twice) or are
//! **`sync_sensitive`** (reservation/barrier ops share cross-thread state)
//! are skipped, not compared.
//!
//! Run with `XENIA_JIT_DIFF=1`; [`report_diff_summary`] prints checked/skipped/
//! mismatch counts at exit. Zero mismatches ⇒ the recompiler matches the
//! interpreter on every comparable block.
use std::cell::{Cell, RefCell};
use std::collections::HashMap;
use std::sync::atomic::{AtomicU64, AtomicU8, Ordering};
use crate::block_cache::DecodedBlock;
use crate::context::PpcContext;
use crate::interpreter::{execute, step_block, StepResult};
use crate::jit::{JitCache, MemEnv, RET_CONTINUE};
use crate::opcode::PpcOpcode;
use xenia_memory::MemoryAccess;
/// Cached env gate. 0 = uninitialised, 1 = on, 2 = off.
#[inline]
fn cached_flag(cell: &AtomicU8, var: &str) -> bool {
match cell.load(Ordering::Relaxed) {
1 => true,
2 => false,
_ => {
let on = std::env::var_os(var).is_some();
cell.store(if on { 1 } else { 2 }, Ordering::Relaxed);
on
}
}
}
/// `XENIA_JIT` — route block execution through the recompiler instead of the
/// interpreter's `step_block`. Opt-in; cheap cached check on the hot path.
#[inline]
pub fn jit_enabled() -> bool {
static F: AtomicU8 = AtomicU8::new(0);
cached_flag(&F, "XENIA_JIT")
}
/// `XENIA_JIT_DIFF` — enable the in-process differential harness ([`diff_step`]).
#[inline]
pub fn diff_enabled() -> bool {
static F: AtomicU8 = AtomicU8::new(0);
cached_flag(&F, "XENIA_JIT_DIFF")
}
/// `XENIA_JIT_CHAIN` — enable native superblock chaining (the JIT compiles a
/// chain of same-page blocks into one function with direct block-to-block
/// jumps). This is only the *env request*; the scheduler additionally requires
/// the diff harness OFF and no diagnostic probes / mem-watch armed before it
/// actually enables chaining (see the config install in `run_superblock`).
#[inline]
pub fn chain_requested() -> bool {
static F: AtomicU8 = AtomicU8::new(0);
cached_flag(&F, "XENIA_JIT_CHAIN")
}
// ---- opcode histogram (XENIA_JIT_HIST) — data-drives M1 coverage ---------
#[inline]
fn hist_enabled() -> bool {
static F: AtomicU8 = AtomicU8::new(0);
cached_flag(&F, "XENIA_JIT_HIST")
}
thread_local! {
static HIST: RefCell<HashMap<PpcOpcode, u64>> = RefCell::new(HashMap::new());
static HIST_TICK: Cell<u32> = const { Cell::new(0) };
}
/// Sample 1-in-16 instructions — relative opcode frequencies are unbiased at
/// this rate and the HashMap cost stops dominating the run.
#[inline]
fn hist_tally(op: PpcOpcode) {
let t = HIST_TICK.with(|c| {
let n = c.get().wrapping_add(1);
c.set(n);
n
});
if t & 0xF == 0 {
HIST.with(|h| *h.borrow_mut().entry(op).or_insert(0) += 1);
}
}
/// Print the opcode-frequency histogram (top 40 + cumulative %), so M1 coverage
/// targets the opcodes that actually dominate boot+movie execution.
pub fn report_histogram() {
if !hist_enabled() {
return;
}
let mut v: Vec<(PpcOpcode, u64)> =
HIST.with(|h| h.borrow().iter().map(|(&k, &c)| (k, c)).collect());
v.sort_by(|a, b| b.1.cmp(&a.1));
let total: u64 = v.iter().map(|(_, c)| c).sum();
eprintln!("=== JIT opcode histogram (total={total}, {} distinct) ===", v.len());
let mut cum = 0u64;
for (i, (op, c)) in v.iter().take(40).enumerate() {
cum += c;
eprintln!(
" {:>2}. {:<14?} {:>12} {:>5.1}% cum {:>5.1}%",
i + 1,
op,
c,
100.0 * *c as f64 / total as f64,
100.0 * cum as f64 / total as f64
);
}
}
// Coverage telemetry: blocks executed as native code vs interpreted (uncovered).
static JIT_COMPILED_RUN: AtomicU64 = AtomicU64::new(0);
static JIT_INTERP_RUN: AtomicU64 = AtomicU64::new(0);
/// Execute one decoded block, natively if it is fully covered, else via the
/// interpreter fallback.
///
/// If [`JitCache::get_or_compile`] returns a compiled entry point, the whole
/// block runs as native code (which advances `pc`/`cycle_count`/`timebase`
/// exactly like the interpreter — validated by the diff harness). Otherwise
/// this is byte-for-byte the same loop as [`crate::interpreter::step_block`]:
/// bump `cycle_count`/`timebase` per instruction, bail on the first
/// non-`Continue` result, and stop on a PC discontinuity (only the terminator
/// may branch).
pub fn run_block(
ctx: &mut PpcContext,
mem: &dyn MemoryAccess,
block: &DecodedBlock,
jit: &mut JitCache,
remaining_budget: u64,
) -> StepResult {
if let Some(f) = jit.get_or_compile(block, mem) {
// NB: under superblock chaining one call runs *several* guest blocks,
// so this counter (and the coverage %) undercounts native guest blocks
// vs the per-block `JIT_INTERP_RUN`. It stays a native-vs-interpreted
// *call* ratio; treat the chained coverage % as a floor.
JIT_COMPILED_RUN.fetch_add(1, Ordering::Relaxed);
let env = jit.mem_env(mem);
// `remaining_budget` bounds a superblock's internal chaining; a
// single-block compilation ignores it. A compiled (super)block always
// runs to completion and returns `Continue` — covered ops never branch
// out / fault, and the chain only ever stops at a clean block boundary.
let raw = f(ctx as *mut PpcContext, &env as *const MemEnv, remaining_budget);
debug_assert_eq!(raw, RET_CONTINUE, "covered block returned non-Continue");
return StepResult::Continue;
}
JIT_INTERP_RUN.fetch_add(1, Ordering::Relaxed);
let hist = hist_enabled();
let mut result = StepResult::Continue;
for instr in &block.instrs {
let expected_next = instr.addr.wrapping_add(4);
if hist {
hist_tally(instr.opcode);
}
// M0 fallback: identical to the interpreter. Future stages dispatch
// lowered ops here and only fall back for uncompiled opcodes.
result = execute(ctx, mem, instr);
ctx.cycle_count += 1;
ctx.timebase += 1;
if !matches!(result, StepResult::Continue) {
return result;
}
if ctx.pc != expected_next {
break;
}
}
result
}
// ---- in-process differential harness -------------------------------------
static CHECKED: AtomicU64 = AtomicU64::new(0);
static SKIPPED: AtomicU64 = AtomicU64::new(0);
static MISMATCH: AtomicU64 = AtomicU64::new(0);
const MAX_MISMATCH_PRINTS: u64 = 20;
/// Run one block under the differential harness. The **interpreter is
/// authoritative** (drives `ctx`+`mem`); the JIT runs speculatively on a clone
/// against an overlay and its registers are compared. Returns the interpreter's
/// `StepResult`.
pub fn diff_step(
ctx: &mut PpcContext,
mem: &dyn MemoryAccess,
block: &DecodedBlock,
jit: &mut JitCache,
) -> StepResult {
// Reservation/barrier blocks share cross-thread state; don't speculate.
if block.sync_sensitive {
SKIPPED.fetch_add(1, Ordering::Relaxed);
return step_block(ctx, mem, block);
}
// Speculative JIT run on a clone against buffered/overlay memory. Runs
// BEFORE the authoritative interpreter so its reads see pre-block memory.
let mut cand = ctx.clone();
cand.reservation_table = None; // never touch the shared reservation table
let overlay = OverlayMemory::new(mem);
// Chaining is disabled under the diff harness (a multi-block superblock
// can't be compared against one interpreter `step_block`), so this always
// runs a single block; the budget is irrelevant.
let _ = run_block(&mut cand, &overlay, block, jit, u64::MAX);
let touched_mmio = overlay.touched_mmio.get();
// Authoritative interpreter run: commits real ctx + memory.
let auth = step_block(ctx, mem, block);
if touched_mmio {
SKIPPED.fetch_add(1, Ordering::Relaxed);
} else {
CHECKED.fetch_add(1, Ordering::Relaxed);
compare(&cand, ctx, block.start_pc);
}
auth
}
/// Compare speculative-JIT (`cand`) vs authoritative-interpreter (`auth`)
/// register state. Reports the first [`MAX_MISMATCH_PRINTS`] divergent blocks.
/// Covers the integer + FP + control state (M0/M1 scope); vector regs join when
/// VMX opcodes are lowered.
fn compare(cand: &PpcContext, auth: &PpcContext, start_pc: u32) {
let mut diffs: Vec<String> = Vec::new();
for i in 0..32 {
if cand.gpr[i] != auth.gpr[i] {
diffs.push(format!("r{i} jit={:#018x} int={:#018x}", cand.gpr[i], auth.gpr[i]));
}
}
for i in 0..32 {
if cand.fpr[i].to_bits() != auth.fpr[i].to_bits() {
diffs.push(format!(
"f{i} jit={:#018x} int={:#018x}",
cand.fpr[i].to_bits(),
auth.fpr[i].to_bits()
));
}
}
if cand.lr != auth.lr {
diffs.push(format!("lr jit={:#x} int={:#x}", cand.lr, auth.lr));
}
if cand.ctr != auth.ctr {
diffs.push(format!("ctr jit={:#x} int={:#x}", cand.ctr, auth.ctr));
}
if cand.pc != auth.pc {
diffs.push(format!("pc jit={:#x} int={:#x}", cand.pc, auth.pc));
}
for i in 0..8 {
if cand.cr[i].as_u8() != auth.cr[i].as_u8() {
diffs.push(format!("cr{i} jit={:#x} int={:#x}", cand.cr[i].as_u8(), auth.cr[i].as_u8()));
}
}
if (cand.xer_ca, cand.xer_ov, cand.xer_so, cand.xer_tbc)
!= (auth.xer_ca, auth.xer_ov, auth.xer_so, auth.xer_tbc)
{
diffs.push("xer".into());
}
if cand.fpscr != auth.fpscr {
diffs.push(format!("fpscr jit={:#x} int={:#x}", cand.fpscr, auth.fpscr));
}
if cand.cycle_count != auth.cycle_count {
diffs.push(format!("cycles jit={} int={}", cand.cycle_count, auth.cycle_count));
}
if !diffs.is_empty() {
let n = MISMATCH.fetch_add(1, Ordering::Relaxed);
if n < MAX_MISMATCH_PRINTS {
eprintln!("JIT-DIFF MISMATCH block={start_pc:#010x}: {}", diffs.join(", "));
}
}
}
/// Print checked/skipped/mismatch tallies (call at clean exit).
pub fn report_diff_summary() {
if !diff_enabled() {
return;
}
let (c, s, m) = (
CHECKED.load(Ordering::Relaxed),
SKIPPED.load(Ordering::Relaxed),
MISMATCH.load(Ordering::Relaxed),
);
eprintln!(
"=== JIT-DIFF SUMMARY: checked={c} skipped(mmio/sync)={s} MISMATCHES={m} — {} ===",
if m == 0 { "CLEAN ✓" } else { "DIVERGENCE ✗" }
);
}
/// Print native-vs-interpreted block coverage (call at clean exit). Shown
/// whenever the JIT is engaged (`XENIA_JIT` or `XENIA_JIT_DIFF`) so we can see
/// how much of the real boot+movie workload the current covered set captures.
pub fn report_jit_summary() {
if !jit_enabled() && !diff_enabled() {
return;
}
let compiled = JIT_COMPILED_RUN.load(Ordering::Relaxed);
let interp = JIT_INTERP_RUN.load(Ordering::Relaxed);
let total = compiled + interp;
let pct = if total == 0 { 0.0 } else { 100.0 * compiled as f64 / total as f64 };
eprintln!(
"=== JIT COVERAGE: native blocks={compiled} interpreted={interp} ({pct:.2}% of {total} block-runs native) ==="
);
}
// ---- overlay memory (speculative-write buffer) ---------------------------
/// A [`MemoryAccess`] wrapper for speculative JIT execution: reads fall through
/// to the real (pre-block) memory, writes are buffered in a byte-granular
/// overlay (never committed), and any MMIO touch sets `touched_mmio` so the
/// caller can skip comparing that block. Big-endian byte order throughout,
/// matching `GuestMemory`.
struct OverlayMemory<'a> {
real: &'a dyn MemoryAccess,
overlay: RefCell<HashMap<u32, u8>>,
touched_mmio: Cell<bool>,
}
impl<'a> OverlayMemory<'a> {
fn new(real: &'a dyn MemoryAccess) -> Self {
Self { real, overlay: RefCell::new(HashMap::new()), touched_mmio: Cell::new(false) }
}
#[inline]
fn rb(&self, addr: u32) -> u8 {
match self.overlay.borrow().get(&addr) {
Some(&b) => b,
None => self.real.read_u8(addr),
}
}
#[inline]
fn wb(&self, addr: u32, b: u8) {
self.overlay.borrow_mut().insert(addr, b);
}
/// Returns true (and flags the block) if `addr` is MMIO — callers must not
/// touch it speculatively.
#[inline]
fn is_mmio_stop(&self, addr: u32) -> bool {
if self.real.is_mmio(addr) {
self.touched_mmio.set(true);
true
} else {
false
}
}
}
impl<'a> MemoryAccess for OverlayMemory<'a> {
fn read_u8(&self, a: u32) -> u8 {
if self.is_mmio_stop(a) {
return 0;
}
self.rb(a)
}
fn read_u16(&self, a: u32) -> u16 {
if self.is_mmio_stop(a) {
return 0;
}
((self.rb(a) as u16) << 8) | self.rb(a.wrapping_add(1)) as u16
}
fn read_u32(&self, a: u32) -> u32 {
if self.is_mmio_stop(a) {
return 0;
}
((self.rb(a) as u32) << 24)
| ((self.rb(a.wrapping_add(1)) as u32) << 16)
| ((self.rb(a.wrapping_add(2)) as u32) << 8)
| self.rb(a.wrapping_add(3)) as u32
}
fn read_u64(&self, a: u32) -> u64 {
if self.is_mmio_stop(a) {
return 0;
}
let hi = self.read_u32(a) as u64;
let lo = self.read_u32(a.wrapping_add(4)) as u64;
(hi << 32) | lo
}
fn write_u8(&self, a: u32, v: u8) {
if self.is_mmio_stop(a) {
return;
}
self.wb(a, v);
}
fn write_u16(&self, a: u32, v: u16) {
if self.is_mmio_stop(a) {
return;
}
self.wb(a, (v >> 8) as u8);
self.wb(a.wrapping_add(1), v as u8);
}
fn write_u32(&self, a: u32, v: u32) {
if self.is_mmio_stop(a) {
return;
}
self.wb(a, (v >> 24) as u8);
self.wb(a.wrapping_add(1), (v >> 16) as u8);
self.wb(a.wrapping_add(2), (v >> 8) as u8);
self.wb(a.wrapping_add(3), v as u8);
}
fn write_u64(&self, a: u32, v: u64) {
self.write_u32(a, (v >> 32) as u32);
self.write_u32(a.wrapping_add(4), v as u32);
}
// execute() never calls translate/translate_mut for real memory (only the
// test mock does), so returning None is safe here.
fn translate(&self, _a: u32) -> Option<*const u8> {
None
}
fn translate_mut(&self, _a: u32) -> Option<*mut u8> {
None
}
fn page_version(&self, a: u32) -> u64 {
self.real.page_version(a)
}
fn is_mmio(&self, a: u32) -> bool {
self.real.is_mmio(a)
}
}

87
crates/xenia-gpu/src/prof.rs
View File

@@ -45,6 +45,16 @@ pub static KERNEL_CALLS: AtomicU64 = AtomicU64::new(0);
pub static BUILD_NS: AtomicU64 = AtomicU64::new(0); // block decode / cache lookup
pub static BUILD_CALLS: AtomicU64 = AtomicU64::new(0);
// Top-level lockstep-loop attribution (sums to ~100% with idle/GPU-pacer):
pub static ROUND_NS: AtomicU64 = AtomicU64::new(0); // per-round tax (clock/timestamp/isr/schedule)
pub static ROUND_CALLS: AtomicU64 = AtomicU64::new(0);
pub static PROLOGUE_NS: AtomicU64 = AtomicU64::new(0); // worker_prologue (⊇ KERNEL)
pub static PROLOGUE_CALLS: AtomicU64 = AtomicU64::new(0);
pub static RUNSB_NS: AtomicU64 = AtomicU64::new(0); // run_superblock (⊇ STEP + chain BUILD + EPILOGUE)
pub static RUNSB_CALLS: AtomicU64 = AtomicU64::new(0);
pub static EPILOGUE_NS: AtomicU64 = AtomicU64::new(0); // worker_epilogue (subset of RUNSB)
pub static EPILOGUE_CALLS: AtomicU64 = AtomicU64::new(0);
/// Cached on/off state so the per-block hot path never touches the
/// environment. 0 = uninitialised, 1 = on, 2 = off.
static ENABLED: std::sync::atomic::AtomicU8 = std::sync::atomic::AtomicU8::new(0);
@@ -142,52 +152,43 @@ pub fn report(_ignored: u64) {
} else {
step_instr as f64 / (step_ns as f64 / 1e3) // instr / us = MIPS
};
eprintln!("=== XENIA_PROFILE (wall {:.1} ms) ===", ms(wall_ns));
eprintln!(
" interp step_block : {:>10.1} ms {:>5.1}% ({} calls, {} instr, {:.1} MIPS)",
ms(step_ns),
pct(step_ns),
g(&STEP_CALLS),
step_instr,
mips
);
eprintln!(
" texture decode+up : {:>10.1} ms {:>5.1}% ({} calls, {} MiB)",
ms(tex_ns),
pct(tex_ns),
g(&TEXDEC_CALLS),
g(&TEXDEC_BYTES) / (1024 * 1024)
);
eprintln!(
" host draw submit : {:>10.1} ms {:>5.1}% ({} calls)",
ms(draw_ns),
pct(draw_ns),
g(&DRAW_CALLS)
);
let kern_ns = g(&KERNEL_NS);
let build_ns = g(&BUILD_NS);
let round_ns = g(&ROUND_NS);
let prologue_ns = g(&PROLOGUE_NS);
let runsb_ns = g(&RUNSB_NS);
let epilogue_ns = g(&EPILOGUE_NS);
// Derived leftovers (honest lumping — see comments):
// runsb_other = chain-loop body (arithmetic/stop-checks) + in-chain block lookups
// prologue_other = block lookup (first block) + scheduler bookkeeping
let runsb_other = runsb_ns.saturating_sub(step_ns).saturating_sub(epilogue_ns);
let prologue_other = prologue_ns.saturating_sub(kern_ns);
let line = |label: &str, ns: u64, extra: &str| {
eprintln!(" {:<20}{:>10.1} ms {:>5.1}% {}", label, ms(ns), pct(ns), extra);
};
eprintln!("=== XENIA_PROFILE (wall {:.1} ms) ===", ms(wall_ns));
eprintln!(" -- TOP LEVEL (single-thread lockstep loop; sums ~100%) --");
line("per-round tax", round_ns, &format!("({} rounds)", g(&ROUND_CALLS)));
line("worker_prologue", prologue_ns, &format!("({} visits)", g(&PROLOGUE_CALLS)));
line("run_superblock", runsb_ns, &format!("({} visits)", g(&RUNSB_CALLS)));
let top = round_ns + prologue_ns + runsb_ns;
eprintln!(
" kernel HLE export : {:>10.1} ms {:>5.1}% ({} calls)",
ms(kern_ns),
pct(kern_ns),
g(&KERNEL_CALLS)
" ---- top accounted {:.1}% ; idle/GPU-pacer/misc {:.1}%",
pct(top),
pct(wall_ns.saturating_sub(top))
);
eprintln!(
" block decode/cache: {:>10.1} ms {:>5.1}% ({} calls)",
ms(build_ns),
pct(build_ns),
g(&BUILD_CALLS)
);
eprintln!(
" frontbuffer present: {:>9.1} ms {:>5.1}% ({} calls)",
ms(pres_ns),
pct(pres_ns),
g(&PRESENT_CALLS)
);
let accounted = step_ns + tex_ns + draw_ns + pres_ns + kern_ns + build_ns;
eprintln!(
" ---- accounted {:.1}% ; remainder (locks/kernel/scheduler/idle) {:.1}%",
pct(accounted),
pct(wall_ns.saturating_sub(accounted))
eprintln!(" -- SUB-ATTRIBUTION --");
line(
"interp step_block",
step_ns,
&format!("[in run_superblock] ({} calls, {} instr, {:.1} MIPS)", g(&STEP_CALLS), step_instr, mips),
);
line("worker_epilogue", epilogue_ns, "[in run_superblock]");
line("run_superblock other", runsb_other, "[chain-loop body + in-chain lookups]");
line("kernel HLE export", kern_ns, &format!("[in worker_prologue] ({} calls)", g(&KERNEL_CALLS)));
line("worker_prologue other", prologue_other, "[first-block lookup + sched bookkeeping]");
line("block decode/cache", build_ns, &format!("[split across prologue+runsb] ({} calls)", g(&BUILD_CALLS)));
line("texture decode+up", tex_ns, &format!("({} calls, {} MiB)", g(&TEXDEC_CALLS), g(&TEXDEC_BYTES) / (1024 * 1024)));
line("host draw submit", draw_ns, &format!("({} calls)", g(&DRAW_CALLS)));
line("frontbuffer present", pres_ns, &format!("({} calls)", g(&PRESENT_CALLS)));
}

8
crates/xenia-kernel/src/state.rs
View File

@@ -648,6 +648,14 @@ impl KernelState {
}
}
/// The import-thunk address band `(lo, hi)` (inclusive), or `None` if no
/// thunks are registered. Exposed so the JIT superblock compiler can refuse
/// to chain into the thunk band (those PCs need the full worker-prologue
/// dispatch, so the chain must stop and hand back to the round).
pub fn thunk_band(&self) -> Option<(u32, u32)> {
self.thunk_addr_band
}
/// Resolve a `(module, ordinal)` to its registered thunk address.
pub fn resolve_thunk(&self, module: ModuleId, ordinal: u16) -> Option<u32> {
self.thunks_by_ordinal.get(&(module, ordinal)).copied()

49
crates/xenia-memory/src/access.rs
View File

@@ -1,3 +1,34 @@
/// Raw pointers/constants a JIT needs to inline the load fast-path (skip the
/// trait-object call). Only the real [`crate::heap::GuestMemory`] exposes this;
/// wrappers that intercept accesses (e.g. the recompiler's speculative overlay)
/// return `None` from [`MemoryAccess::fast_mem`] so the JIT falls back to the
/// trait-object slow path and their interception is preserved.
///
/// The flat mapping means a guest address `a` translates to `membase + a`. A
/// load is safe to inline only when `a` is **not** MMIO and its page is
/// committed: `(a & mmio_mask) != mmio_value` AND page-table entry
/// `page_table[a >> 12]` has the COMMIT bit (bit 49). Unmapped pages are
/// `PROT_NONE`, so the committed check must precede the raw load.
#[derive(Clone, Copy)]
pub struct FastMem {
/// Base of the flat 4 GiB guest mapping. `membase + guest_addr` is the host
/// address. Never null for a real mapping (used by the JIT as the
/// "inline enabled" sentinel).
pub membase: *mut u8,
/// Base of the per-page allocation table (`AtomicU64` entries, reinterpreted
/// as `u64`). Entry `page_table[addr >> 12]` bit 49 = COMMIT.
pub page_table: *const u64,
/// MMIO fast-reject mask/value: `(addr & mask) == value` is the *necessary*
/// condition for `addr` to be MMIO, so `!=` proves non-MMIO.
pub mmio_mask: u32,
pub mmio_value: u32,
/// Pointer to the monotonic MMIO-access counter (`AtomicU64` as `*const
/// u64`). A superblock JIT samples this across a block boundary to detect an
/// MMIO touch and stop chaining there (preserving the interpreter's
/// fine-grained MMIO ordering).
pub mmio_count: *const u64,
}
/// Trait for all guest memory access. Every load/store goes through this,
/// enabling MMIO checking and debugger observation on every access.
/// This is the key abstraction that eliminates the need for MMIO exception handlers.
@@ -48,6 +79,24 @@ pub trait MemoryAccess {
}
}
/// True if `addr` falls in an MMIO region (a load/store there invokes a
/// device callback with side effects rather than touching backing RAM).
///
/// Default `false` (mock memories have no MMIO). `GuestMemory` overrides it.
/// Used by the recompiler's differential harness to *skip* comparing blocks
/// that touch MMIO — a device callback cannot be safely executed twice.
fn is_mmio(&self, _addr: u32) -> bool {
false
}
/// Raw pointers/constants for a JIT to inline the load fast-path, or `None`
/// to force the trait-object slow path. Default `None` (mock memories and
/// interception wrappers have no flat mapping to expose). `GuestMemory`
/// overrides it. See [`FastMem`].
fn fast_mem(&self) -> Option<FastMem> {
None
}
/// Get a direct host pointer for the given guest address.
/// Returns None if the address is invalid or in an MMIO region.
fn translate(&self, addr: u32) -> Option<*const u8>;

19
crates/xenia-memory/src/heap.rs
View File

@@ -504,6 +504,25 @@ impl GuestMemory {
}
impl MemoryAccess for GuestMemory {
#[inline]
fn is_mmio(&self, addr: u32) -> bool {
self.find_mmio(addr).is_some()
}
fn fast_mem(&self) -> Option<crate::access::FastMem> {
Some(crate::access::FastMem {
membase: self.membase,
// AtomicU64 is #[repr(transparent)] over u64, so the element pointer
// reinterprets directly. The JIT reads entries with a plain load —
// sound because alloc publishes each entry before the guest can
// access the page (single-thread lockstep; x86 loads are acquire).
page_table: self.page_table.as_ptr() as *const u64,
mmio_mask: self.mmio_aperture_mask,
mmio_value: self.mmio_aperture_value,
mmio_count: &self.mmio_access_count as *const AtomicU64 as *const u64,
})
}
// Tier-3 perf: `#[inline]` on the hot read/write paths lets LLVM
// fold the MMIO + mapping checks into the interpreter's load/store
// handlers, hoisting the "not-MMIO, mapped" branch out of the loop

2
crates/xenia-memory/src/lib.rs
View File

@@ -7,7 +7,7 @@ mod platform;
use thiserror::Error;
pub use access::MemoryAccess;
pub use access::{FastMem, MemoryAccess};
pub use heap::{set_writer_ctx, GuestMemory, HeapType};
pub use mmio::MmioRegion;
pub use page_table::PageEntry;