dc3de9dbc4feba48a0ab9c6137d936db805ffd13
71 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
MechaCat02
|
4b78c67605 |
[iterate-4A] jit: superblock chaining B2 — bcx two-way + loop back-edges
Extends B1's forward-linear superblock into a full same-page CFG: chains
through conditional bcx (BOTH directions) and loop back-edges (a successor
already in the chain jumps to its existing IR block), not just unconditional
bx + fall-through. This roughly doubles the block-merge ratio (run_block calls
145.7M interp -> 116.2M B1 (1.25x) -> 74.7M B2 (1.95x)) and halves the
run_superblock dispatch bucket (30.4% interp -> 26.2% B1 -> 16.2% B2).
Mechanism:
- enumerate_chain is now a two-pass CFG builder: BFS the reachable, chainable
(same-page / covered / non-thunk) blocks deduped by start PC (so a back-edge
becomes a loop, not a new node), capped at MAX_CHAIN_NODES=64; then resolve
each terminator to Succ edges (node indices or Exit). bx/fall -> One; bcx ->
Two{taken,fall} (each side Node or Exit); bclrx/bcctrx/uncovered -> Exit.
- emit_bcx extracted from emit_op's bcx arm: computes the taken predicate ONCE
(decrementing CTR at most once) and returns it; emit_node_body captures it so
the two-way boundary branches on the SAME value (recomputing would double the
CTR decrement). Single-block compile is unchanged (ignores the return).
- No Cranelift Variables/phi needed for the loop CFG: the budget/MMIO state
lives in memory (cycle_count, *mmio_count), so emit_stop reloads it and
compares against entry-sampled constants (cycle_entry/mmio_entry), which the
entry block dominates across back-edges. Budget = (cycle_count - cycle_entry)
>= remaining, correct across loop iterations; a native loop spins until the
budget is spent then hands back — same bound as the runner. MMIO check still
skipped for load/store-free blocks. seal_all_blocks handles the arbitrary CFG.
- Degenerate guard fixed: compile single-block only when the ENTRY has no
chainable successor (Succ::Exit) — a one-node self-loop is NOT degenerate.
Validation: 3 new unit tests (bcx always-taken chains to target; bcx not-taken
falls through; self-loop runs natively until budget then stops — all bit-equal
to the interpreter over the same PCs) -> 7/7 jit tests pass. Diff MISMATCHES=0
over 47.95M blocks (the emit_bcx refactor didn't perturb op bodies).
XENIA_JIT_CHAIN=1 movie plays: tid25 resumes (decode handoff / interleaving
preserved under the deeper chaining).
Perf finding (honest): despite ~2x merge and half the dispatch, wall is ~parity
with B1 (both ~-6% vs interp same-batch). step_block MIPS dropped (~82 vs B1
~102) because the deeper/bigger native superblocks cost more per boundary (B1's
compile-time-constant budget became a runtime cycle_count reload, needed for
loops) and Cranelift's default regalloc spills more in larger functions. The
dispatch savings are real but offset by native-code overhead that grows with
chain length. Converting the extra chaining into wall time now needs
host-register allocation (tighter code) — the clear next lever. Env gate,
diff-off-under-chain, and probe/mem-watch exclusivity unchanged from B1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
MechaCat02
|
2c17a511f7 |
[iterate-4A] jit: native superblock chaining B1 (bx + fall-through), −6.4% wall
Increment B1 of block-linking: compile a chain of same-page, fully-covered blocks into ONE Cranelift function whose internal block-to-block edges are direct machine jumps, eliminating the per-block return to run_block and the run_superblock dispatch tax for the chain. Env-gated XENIA_JIT_CHAIN (off by default), and additionally disabled under the diff harness or with diagnostic probes / mem-watch armed (chaining runs several blocks natively without the per-block-entry observation those need, so they are mutually exclusive). Mechanism (cranelift-jit can't patch finalized code, so no QEMU-style TB chaining): SUPERBLOCK COMPILATION. compile_chain enumerates the forward-linear chain from the entry — following static bx targets and fall-throughs, all same-page (so the entry page_version is a sound cache key), outside the thunk band, and covered — and emits one node per block plus a shared exit. At every internal boundary it re-checks the two runtime stop-conditions exactly as the Rust runner does: an MMIO touch (*mmio_count advanced vs the entry sample) and the instruction budget (cumulative >= remaining, passed as a runtime param); either stops the chain with pc/cycle/timebase already correct, and the runner re-dispatches from the clean boundary. B1 scope is forward-linear: a bx or fall-through continues; a conditional bcx, dynamic bclrx/bcctrx, off-page / thunk / uncovered / already-visited successor ends the chain (B2 adds bcx two-way + loop back-edges). Being *more* conservative (stopping earlier) is always safe. Correctness rests on composition: each node body is the same emit_node_body IR the single-block path emits (already diff-validated bit-exact), so only the chain glue is new — validated by two new unit tests (multi-block chain vs interpreter over the same PCs; budget cut at the exact boundary) plus e2e. Boundary MMIO check is skipped for load/store-free blocks (a pure-ALU block can't advance mmio_count), dropping a memory load+compare per ALU boundary. Plumbing: build_block exposed (xenia-cpu); KernelState::thunk_band getter; MemEnv mmio_count null-safety (points at a static zero when no flat mapping); CompiledFn gains a remaining-budget param (single-block ignores it); all compiled entries share the (ctx, mem_env, remaining) ABI. Validation: diff MISMATCHES=0 over 25.0M blocks (single-block op bodies unbroken by the emit_node_body refactor); XENIA_JIT_CHAIN=1 movie plays, tid25 resumes, source-read=12; fair perf (RUST_LOG=warn) interp 44.1s vs chain 41.3s = −6.4% wall — run_superblock "other" 30.4%->26.2%, run_block calls 145.7M->116.2M (1.25x block merge), step_block at parity (100.6->102.4 MIPS; the earlier "slower" reading was cranelift IR-logging inside the STEP timer). Uncommitted probe knobs unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
56dbf52a5f |
[iterate-4A] jit: wire compiled-block cache + cover bx/bcx/bclrx (diff-clean, 6.57% native)
Wire the Cranelift block-JIT into execution and add branch coverage. Both
increments validated bit-exact against the interpreter via the in-process
differential harness on a full boot+movie run (movie plays, clean exit):
wiring only (addi/addis): checked 146.8M blocks, 0.01% native, MISMATCHES=0
+ branches (bx/bcx/bclrx): checked 148.5M blocks, 6.57% native (9.76M), MISMATCHES=0
jit.rs
* JitCache: direct-mapped 64K-slot compiled-block cache keyed (start_pc,
page_version) identically to BlockCache, so self-modifying / DMA'd code
invalidates native code the same way. Caches the None ("uncovered") verdict
so an uncovered block is compile-attempted at most once per (pc,version).
Owns the Jit/JITModule (keeps every CompiledFn valid for its lifetime).
* covered(): addi/addis + bx/bcx/bclrx. bcctrx excluded (indirect target +
dispatch_rec diagnostic hook the native path would skip).
* pc-handling refactor: a branch terminator writes pc itself (writes_pc());
the block epilogue stores end_pc only for straight-line (max-len / page-
boundary) blocks. cycle/timebase still += N (covered ops never fault/yield;
a branch is always the last instruction).
* Branch lowering uses immediate targets — the interpreter's ctx.pc equals the
instruction address at emit time, so bx/bcx relative targets are constants.
emit_branch_taken mirrors the interpreter: optional CTR decrement, ctr_ok =
(ctr as u32 vs 0) inverted by BO3, cond_ok = CR-bit BI byte == BO1 (both BO
sub-cases const-fold), combined with select. bclrx reads lr&!3 before the LK
link overwrites lr.
recompiler.rs
* run_block / diff_step take &mut JitCache. run_block runs the native fn when
get_or_compile returns one (else interpreter fallback); diff_step runs it on
the speculative clone/OverlayMemory so every native block is diff-checked.
* report_jit_summary(): native-vs-interpreted block counts (XENIA_JIT / _DIFF).
main.rs
* WorkerCtx owns a JitCache; run_superblock routing passes it to
diff_step/run_block; report_jit_summary() at clean exit.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
MechaCat02
|
07cd272412 |
[iterate-4A] jit: opcode-frequency histogram (XENIA_JIT_HIST) for coverage targeting
Gated 1-in-16 sampled opcode histogram in the recompiler (tallied in run_block, printed sorted with cumulative % at clean exit). Drives which opcodes the JIT lowers first. Boot+movie result: workload is FP-heavy — addi 15%, lwz 12%, rlwinm 10%, lfs 9%, stfs 7.5%, lfsx 5%, fmaddsx 4.6%, bc 3.7%, fmulsx 3.6%, stw 3.4% ... top-20 = 90%, ~33% floating-point. This is why we go straight to a Cranelift machine-code JIT (which attacks the FP op bodies) rather than a closure-threaded stage (which only removes dispatch). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
ac289bca5e |
[iterate-4A] jit M0: recompiler seam + in-process differential harness
Foundation for a staged PPC block-recompiler (plan: closure-threaded -> Cranelift -> block-linking). No codegen yet — proves the integration seam and the correctness harness before any lowering exists. New crates/xenia-cpu/src/recompiler.rs: - run_block: executes a DecodedBlock; M0 falls back to the interpreter's execute() for every opcode, so it is bit-identical to step_block. Later stages dispatch lowered ops here and fall back only for uncompiled opcodes. - gates jit_enabled()/diff_enabled() (XENIA_JIT / XENIA_JIT_DIFF, cached). - In-process differential harness (diff_step): the interpreter is AUTHORITATIVE (drives real ctx+mem, so a JIT bug can never corrupt a run); the JIT runs SPECULATIVELY on a ctx clone against OverlayMemory (writes buffered in a byte HashMap, reads fall through to real pre-block memory), then registers are compared. Blocks touching MMIO or sync_sensitive (reservation/barrier) are skipped — a device callback can't be run twice and reservation state is shared cross-thread. report_diff_summary() prints checked/skipped/mismatch. Why in-process: the guest is only COARSELY deterministic — coord_idle_advance ticks vsync from wall-clock when idle, so two separate runs are not bit-exact and a cross-run signature compare would measure jitter, not JIT divergence. Supporting changes: PpcContext #[derive(Clone)] (harness clears the speculative clone's reservation_table Arc); is_mmio() on the MemoryAccess trait (default false) + GuestMemory impl via find_mmio; execute() made pub(crate); run_superblock routes the block body (DIFF->diff_step, JIT->run_block, else step_block). Validated: full boot+movie XENIA_JIT_DIFF=1 run = checked 218.4M blocks, skipped(mmio/sync) 511.6K (0.23%), MISMATCHES=0 CLEAN; movie plays (ADVreads=30, tid25 resumes); diff-mode ~2x slower (test-only path). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
e07f93ed0a |
[iterate-4A] perf: superblock budget 128->192 (movie-validated ~7% wall) + profiler attribution
Speed frontier, increment 1 (measure-first, then bank the cheap win). Profiler (crates/xenia-gpu/src/prof.rs): extend XENIA_PROFILE with hierarchical attribution of the single-thread lockstep loop — ROUND (per-round tax) / PROLOGUE (worker_prologue) / RUNSB (run_superblock) top level, plus the STEP/EPILOGUE/KERNEL/BUILD subsets. Gated zero-cost via is_on() (off-check emits 0 lines). Movie-time split: interp step_block 42%, run_superblock chain-loop body ~30%, worker_prologue+epilogue ~22%, kernel HLE 6%, per-round tax 0.6%, texture 0.4%. => overhead-bound; the ceiling is a JIT (attacks the ~72% interp + dispatch). The per-round tax is a non-lever (0.6%). Budget (crates/xenia-app/src/main.rs SUPERBLOCK_INSTR_BUDGET 128->192): the per-slot-visit tax (prologue+epilogue ~22%) scales with slot-visit count, and chains are NOT break-limited here — 128->192 cuts visits 27.4M->18.9M (-31%), 128->256 -44%. Banked 192 (not 256): the movie decode pipeline is more timing-sensitive than boot. At 256 the decode worker tid25 (0x82506588) intermittently fails to resume (1-of-2 runs) with a weakened feeder loop (source-read 27->5-11) — a scheduling race the coarser interleaving exposes. 192 is deterministic and safe: 3/3 runs byte-identical (source-read=21, tid25-resume=1, ADVreads=30) at both env-override and compiled-default, ~-7% movie wall, well below the 384 boot cliff. XENIA_SUPERBLOCK_BUDGET still overrides for A/B. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
2c883e9d5e |
[iterate-4A] diagnostics: XENIA_PROFILE wall-time profiler + probe/tooling snapshot
Handoff snapshot of the env-gated diagnostic scaffolding used across the intro-video RE. Kept out of the milestone commits (645feb8..5573ac1) to keep those clean; committed here so nothing is lost on handoff. New — XENIA_PROFILE wall-time profiler (crates/xenia-gpu/src/prof.rs): Coarse buckets attributing playback wall time to interpreter (step_block), kernel HLE (call_export), block decode/cache (lookup_or_build), texture decode, host draw, and present; prints periodic snapshots (every 500M guest instr, or every 500 presents) + a clean-exit report. Hot path is gated on a cached is_on() (one relaxed load) so it is zero-cost when XENIA_PROFILE is unset. Call sites: main.rs run_superblock / parallel worker (step_block, lookup_or_build, call_export), texture_cache ensure_cached, render.rs present + dispatch_xenos_draws. First profile (movie playback, headless single-thread lockstep): effective ~35 MIPS; interpreter body ~40% @ ~95-102 MIPS; texture decode 0.3% (cache works); present ~0%; the rest is per-block dispatch + scheduler plumbing (~13 instr/block over 229M blocks). Overhead-bound, not interpreter-body bound; the levers are coarser execution units (superblock chaining) and ultimately a JIT. Pre-existing read-only probe knobs (were uncommitted; env-gated, observe-only): XENIA_RET_CAPTURE_PC/_REG/_MEM, LOG_RESUMES, LOG_WAITS, LOG_SIGNAL, FORCE_TID, STARVE_LIMIT, INCUMBENT_PICK, INSTR_PER_MS, DUMP_FRAME, DUMP_WGSL, BIND_LOG, CONST_LOG, DISPATCH_REC, AUDIT_PC_TRACE. Tooling: sylph-run.sh (movie oracle loop, 180s default timeout), zq.py (DuckDB disasm/xref helper). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
43441523f9 |
[iterate-4A] intro-video: re-baseline boot golden 50M -> 200M after clock fix
The clock fix (INSTRUCTIONS_PER_MS 10_000 -> 1_000_000, commit
|
||
|
MechaCat02
|
3e17d37b4e |
[iterate-4A] intro-video RE: XENIA_AUDIT_PC_TRACE (per-tid PC-range r3/r4/r5/r30/r31 trace)
Observe-only diagnostic (lockstep digest unaffected) added during the milestone-2 video investigation. `XENIA_AUDIT_PC_TRACE=lo:hi:tid` logs AUDIT-TRACE r3/r4/r5/r30/r31 for blocks in [lo,hi) on a given tid — used to capture the demux registrar's stream-id (r4) at sub_82509E40. Handoff (full detail in memory cont.10ff-10ii): - Video decode is GUEST SOFTWARE (decoder vt 0x82009f70, 615 fns/30k instr, zero host/video imports) — no host codec to build; fix is upstream. - Video IS registered/routed/queued (refutes "not selected"); the engine never reaches ready-state, pump sub_825078D8 bails 5262x, begin-playback sub_825076F0 never fires -> 2000ms readiness timeout. - CANARY oracle measured: video producer sub_824FF678 drives demux SM sub_825211A0 72x+ (loops/pulls continuously); OURS drives it 1x then the pull loop sub_824FA8A0 exits on no-data (assembly sub_8250A2B0 returns 0x8050000B = incomplete unit). Root: ours delivers INCOMPLETE video units where canary delivers complete ones. - NEXT: compare ours-vs-canary fragment chain [slot+24] for the video stream (missing end-fragment / fragment-header mis-parse). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
7e9ee1ac33 |
intro-video RE: deep demux-core instrumentation + handoff hygiene
Investigation probes for the ADV.wmv intro-video deadlock (feeder's demux source-read 0x825211a0 returns no-data 0x80500000 where canary returns 0). - xenia-kernel/state.rs: extend AUDIT-DEREF (fire_audit_pc_probe_if_match) to deep-dump the demux-state object (to +0xfc), its buffer descriptor [sub+4], the byte-source sub-objects ([sub+0]/[+0x30]/[+0x94] and [src0+0x2c]), and to WALK the windowed-buffer cached-block linked list (AUDIT-BLK: per-block 64-bit start/size/dataptr) exactly as mapper 0x82522118 does. Read-only, env-gated (XENIA_AUDIT_DEREF); lockstep digest unaffected. - xenia-app/main.rs: XENIA_DUMP_SLOTS observe-only scheduler runqueue dump. - xenia-app/tests/sylpheed_oracles.rs: resolve ISO via SYLPHEED_ISO, then the repo-root sylpheed.iso symlink, then default; fix path typo. - .gitignore: exclude .claude/ (71k files / 66GB agent worktrees) and the local investigation artifacts (audit-runs/, exit-thread-state.json, zq_*.py). Findings (full chain in handoff notes): all ASF header metadata parses correctly (packet size 16415, count 4728, 2 streams); cursors correct (cur=5868, limit=77.6M); the windowed buffer works and slides (512B blocks); no-data originates deep in the byte-read/parse chain, not from an empty buffer. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
23189b95af |
[iterate-4A] Milestone-2: XMA audio decoder + RE tooling (dispatch recorder, analyzer vtable-fix, non-perturbing probes)
Milestone-2 (intro video dat/movie/ADV.wmv) audio path + major RE tooling. XMA AUDIO (built, working, deterministic, tested): - APU MMIO 0x7FEA0000 + 320x64B register-mapped context array; real XMACreateContext/Release (xma.rs); real FFmpeg xma2 decoder XMA_CONTEXT_DATA->S16BE PCM (xma_decode.rs, xma2_codec.rs, ffmpeg-sys-next). Decode runs synchronously on the CPU thread (deterministic, no host thread). - Audio-worker scheduler fix (main.rs LR_HALT restore + scheduler.rs): the XAudio render-callback worker was wrongly exited after ~2 deliveries; now survives -> guest drives XMA decode (70 kicks). - XAudioSubmitRenderDriverFrame made faithful. Golden sylpheed_n50m re-baselined; tests pass. RE TOOLING: - Runtime indirect-dispatch recorder (dispatch_rec.rs): records (call-site->target, r3, lr); env-gated XENIA_DISPATCH_REC, filters XENIA_DISPATCH_REC_TARGETS/_SITES; deterministic, observe-only. - Repaired static analyzer (vtables.rs): vtable extraction silently fragmented vtables with non-function head slots (missed the XMV engine vtable). Fixed via vptr-write-anchoring -> engine fully typed (vtables 722->1150 on rebuild). - Fixed probe HEISENBUG (main.rs run_superblock): --audit-pc-probe-hex/--mem-watch no longer disable superblock chaining; probes fire inside the chain loop -> scheduling identical armed-vs-unarmed, movie subsystem now observable. Fixed a --quiet bug swallowing armed trace reports. VIDEO still doesn't play (B, guest-side): the XMV engine never issues begin-playback (sub_825076F0, vtable 0x8200a1e8 slot21) -> never primes -> 2000ms timeout. Narrowed to the ARM2 engine-setup wrappers; no honest our-side gate-fix (masking forbidden). See HANDOFF-iterate-4A-milestone2.md for new-machine setup (incl. the FFmpeg apt deps + sylpheed.db regeneration) and continuation pointers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
acb29db444 |
[iterate-3AL] Superblock dispatch: chain basic blocks per slot-visit (~1.6x boot-to-splash)
Replace the one-basic-block-per-slot-per-round lockstep dispatch with a
SUPERBLOCK runner: each slot-visit chains straight-line blocks through
their terminating branches up to a deterministic instruction budget,
amortizing the per-round (timebase/coord/round_schedule) and per-slot
(worker_prologue) dispatch tax over ~128 instructions instead of ~6.
Yield-points (end the chain, return to the round) are pure functions of
guest state, preserving the lockstep cross-thread interleaving correctness:
- non-Continue step result (Yield/SystemCall/Trap/Unimpl/Halted);
db16cyc Yield is the spin-wait producer hand-off.
- sync-sensitive block: lwarx/ldarx/stwcx./stdcx. or sync/eieio/isync
(new PpcOpcode::is_sync_sensitive, flagged on DecodedBlock at build).
- MMIO touch: new GuestMemory::mmio_access_count() watermark, sampled
per block, keeps GPU/register ordering at one-block granularity.
- next PC leaves ordinary guest code (import thunk / halt sentinel /
unmapped) -> hand to the full worker_prologue next round.
- instruction budget reached.
Instruction-count/clock accounting stays exact: per-block cycle_count
deltas are summed and handed to worker_epilogue once (instruction_count +
decrement_quantum advance by the precise retired count). XENIA_SUPERBLOCK_BUDGET=1
reproduces the old one-block schedule byte-for-byte.
Budget tuned to 128 (env-overridable): boot progression stays healthy up
to 256, sharp cliff at ~384 (a boot producer/consumer handoff starves);
128 is 3x below the cliff. Also scale the inline-GPU per-round fairness
cap with the budget (flat 64 throttled GPU command processing 17x under
superblocks and collapsed the present loop).
PERF (check -n 100M --gpu-inline): 25.3 -> 42.7 MIPS (1.69x); 1B: 26.0 ->
41.4 MIPS (1.59x). Callgrind n=5M: host instructions 2.178B -> 1.507B
(-31%); worker_prologue -90%, coord_pre_round -91%, begin_slot_visit /
round_schedule_into / coord_post_round / update_timestamp_bundle each
~-90%; interpreter execute byte-identical (real work unchanged).
GATES: C1 boot progression 150M draws 7391/swaps 2164 (baseline 7415/2172),
1B draws 88547/swaps 29228 linear no stall, K8888 decode + RTs=2 intact.
C2 determinism: n50m stable digest byte-identical across fresh runs;
golden re-baselined intentionally (pacing-only deltas: imports 333453->243387,
draws 1274->1279). C3 milestone-1 render: texture_decodes/draws/swaps/
present cadence track baseline (3AJ fade-in pacing preserved). C4: 690
tests green (+2 sync_sensitive).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
dc1320cd4b |
[iterate-3AK] Perf quick-wins: ~21% faster boot-to-splash (22→27 MIPS)
Profile-driven low-risk optimizations attacking the ~48% per-block / per-round host-bookkeeping tax found by the callgrind profile. Measured on the bounded headless workload `check -n 100000000 --gpu-inline`: baseline ~4490 ms (22.3 MIPS) -> ~3700 ms (27.0 MIPS), +21%. Tier A (determinism-neutral; n50m golden byte-IDENTICAL, exit 0): 1. mem-watch write path: gate capture_mem_watch_old/check_mem_watch behind one has_mem_watch() predicted branch in write_u8/16/32/64 + write_bulk so the common (no-watch) store does no out-of-line call. check_mem_watch (4.8%) gone from the profile. 2. round-schedule alloc churn: add Scheduler::round_schedule_into filling a reusable [u8; HW_THREAD_COUNT] stack buffer; the lockstep round loop no longer __rust_alloc/__rust_dealloc a Vec<u8> per round. Identical ordering/RNG-advance. __rust_alloc/dealloc gone from the profile. 3. probe-firing: hoist a single KernelState::any_probe_active() guard to worker_prologue so the four fire_*_if_match calls don't happen at all when no probe is configured (was 4x call overhead/visit). All four gone from the profile. 4. thunk-map hash: range-reject pc against the registered import-thunk address band (KernelState::pc_in_thunk_band, two int compares) before the thunk_map.get(&pc) HashMap lookup. hash_one (4.3%) gone. Tier B (#5, time-granularity change — LANDED, no re-baseline needed): 5. update_timestamp_bundle: throttle to a 0.25 ms quantum (only re-write the KeTimeStampBundle when the deterministic clock advanced >= 2500 units). Inclusive cost 8.65% -> 1.08%. The quantum is far below the 1 ms granularity any guest deadline math needs (tick_count stays fresh; the hub gate is +66 ms; the fade-in is vsync-counter driven per 3AH, not this bundle). VERIFIED: n50m stable digest BYTE-IDENTICAL to the existing golden (so no re-baseline), 150M boot reaches the splash (draws=7415, swaps=2172, gpu.texture.decode{K8888}=448, RTs=2 — all match the post-3AJ baseline), 688 tests green, release n50m oracle ok. Remaining headroom: interpreter::execute (13%), decrement_quantum (8%), step_block (7%) are now the top self-costs — the structural superblock/ JIT lever is the next step for the larger gain. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
9d24dd0eaa |
[iterate-3AJ] Present-anchor vsync so the splash logo fade-in renders
The publisher/dev splash logo's intro fade-IN was skipped: the logo popped in at full brightness instead of ramping dim->bright like the canary oracle. Root (measured, iterate-3AF/3AI): ours' guest vsync counter is fed by a fixed-instruction-quantum proxy (one vsync per 150k retired instructions). During the ~1.1s splash asset-load the title's frame pump runs ~10M instructions inside a single guest frame, so the proxy fired ~66 vsyncs in that one frame. The pump's per-frame delta (counter_now - counter_last) was therefore ~66 on the first tick, which the anim tick (sub_823CDBF8) divides into the fade counter [item+72] @ 0x40c0add0 -> the counter JUMPED 0->0x42(66) in one step, landing past the fade-in region. Canary's wall-clock 60Hz vblank advances ~1 per heavy load frame, so its counter ramps smoothly 0->66 and the fade-in renders. Fix: anchor the lockstep vsync ticker to the guest's real present rate (VdSwap count), mirroring real hardware where the title double-buffers at vblank, so one heavy guest frame advances the vsync counter by ~1 instead of ~66. - interrupts.rs: tick_vsync_instr now takes the live present count. Two regimes: (1) bootstrap, before the guest's first present, keeps the original fixed instruction quantum unchanged -- the iterate-2W present-loop bootstrap needs vsyncs delivered BEFORE it can present (measured: callback registered ~6M instr, first delivered vsync and first present coincide; pure present-driven vsync would deadlock). (2) present-anchored, after the first present: one vblank per present, plus a small DRY_FALLBACK_CAP=4 instruction-quantum fallback per dry window so a non-presenting frame still ticks a few vsyncs (a small ramp like canary's 0/5/10/2/1...) without re-spiking to 66. - handle.rs: cheap GpuBackend::swaps_seen() accessor. - main.rs: pass the live present count into the lockstep ticker. Not masking: the fade dt/counter is never clamped or synthesized; the guest naturally computes a smooth dt once vblank tracks presents. Verified: - V1: fade counter 0x40c0add0 now ramps 0,6,8,10,12,13,+1... (was a 0->0x42 jump; direct baseline-vs-fix mem-watch). - V2 (--ui readback via per-frame logo vertex-alpha): logo alpha ramps 102,136,204,221,238,254 (dim->bright fade-IN) vs baseline all 255 (pop-in). Real artwork (has_real_vertices) still renders; milestone-1 intact. - V3: 150M boot progression intact -- texture_decodes=2, RTs=2, tex_cache=1 unchanged; draws/swaps higher (tighter present loop), 1B sanity linear, no stall/collapse. - V4: 50M --gpu-inline --stable-digest byte-identical 2x; golden re-baselined intentionally (pacing-only delta: draws 718->1274, swaps 147->259; structural fields unchanged). 688 tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
504592ac13 |
[iterate-3O] Real-render slice: replay guest geometry in --ui (Route A)
Replace the synthetic placeholder triangle in the --ui window with the
splash's REAL guest geometry, proving the faithful-render pipe end to end.
Architecture: Route A (UI-side replay). A per-draw capture channel carries
each PM4_DRAW_INDX*'s real state to the UI, which replays it through the
existing wgpu Xenos pipeline. The deterministic headless core is untouched:
capture is gated on an Option<Vec<DrawCapture>> that is None in headless
mode and only enabled on the --ui path, so the --gpu-inline n50m golden is
byte-identical (verified 2x).
The hard part was sourcing real vertices. The WGSL VS already does
format-aware vertex fetch from the b4 storage buffer at the address from the
fetch constant -- but b4 was never populated and the fetch address is an
absolute guest dword address. The slice:
* xenia-gpu/draw_capture.rs: parse the active VS, find its first vertex
fetch, read that fetch constant, copy a bounded window of guest memory
at the fetch base. Best-effort: has_real_vertices=false falls back to
procedural geometry (never fabricated pixels).
* gpu_system.rs: accumulate one DrawCapture per draw into frame_captures.
* exports.rs (vd_swap): drain + publish the frame's captures to the UI.
* ui_bridge/bridge.rs: new publish_geometry channel + UiHandles.geometry.
* WGSL (interp + translator): rebase the absolute fetch address by a new
DrawConstants.vertex_base_dwords so it indexes the uploaded window.
* render.rs: dispatch_xenos_captures uploads each draw's real vertex
window + matching shader, issues real DrawRequests (real prim type,
host vertex count, vs/ps keys).
* app.rs: prefer the real-capture replay; HUD adds real-geo=N counter.
Verified in --ui on Sylpheed: "first Xenos capture batch replayed (real
geometry) captures=24 real_vertex_draws=24" -- all draws resolved a real
guest vertex window; WGSL compiles; no validation errors over 1616 swaps.
Still synthetic-free but not yet pixel-perfect: textures/UVs, DMA index
buffers (auto-index only for now), and kCopy resolve routing are staged
for follow-ups. Faithful: real vertex data, prim types, shaders, constants.
cargo test --workspace green; n50m golden unchanged (2x byte-identical).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
6bb4355e3d |
[iterate-3M] Fix Xenos shader CF/fetch decode so the textured logo binds
The publisher splash (title idx0) rendered FLAT in ours while canary samples a texture: ours never decoded the logo's textured pixel shader (E59B2B3D, a `tfetch2D` sprite) even though our guest IM_LOADs the exact same microcode canary does (verified byte-identical against the Wine oracle). The shader was misparsed as flat. Three coupled bugs in the ucode decoder, all off vs canary `gpu/ucode.h`: 1. CF opcode table was off-by-one (`control_flow.rs`): mapped opcode 0→Exec and 1→Exit, but Xenos has 0=kNop, 1=kExec, 2=kExecEnd, 3..6/13..14 the cond-exec variants, 7/8 loop, 9/10 call/return, 11 condjmp, 12 alloc, 15 mark-vs-fetch-done. So a real `kExec` clause was read as a terminal `Exit`, truncating the CF block and dropping every instruction (incl. the `tfetch`) after it. Added Nop/MarkVsFetchDone variants; parse now ends on an END-bit exec clause. 2. exec/loop `address` is an absolute instruction-triple index from shader dword 0, but indexed our post-CF `instructions` slice directly (`ucode/mod.rs`). Rebase addresses by the CF triple count so `address*3` lands on the right instruction. 3. Fetch instruction bitfields were wrong (`ucode/fetch.rs`): `const_index` read from bit 5 (actually `src_reg`) instead of bit 20, and texture `dimension` from dword1 instead of dword2 bit14. The logo's `tfetch ..,tf0` was read as `tf1`, whose empty fetch-constant failed to decode → no texture. Also the `sequence` fetch/ALU bit is bit[0] of each pair, not bit[1] (`shader_metrics.rs`, `translator.rs`, `xenos_interp.wgsl`). Result (--gpu-inline, deterministic 2x): the active PS's `tfetch_slots` now resolves slot 0, the tf0 fetch-constant decodes (fmt K8888), and `gpu.texture.decode` fires (137x at -n 50M; texture_cache_entries 0→1, the only golden field that changed — all draw/swap counts unchanged). The same fixes correct the WGSL uber-shader's fetch/CF walk for the threaded/--ui path. Added a regression test that parses the real E59B2B3D microcode and asserts a tfetch slot is found. Golden re-baselined (texture_cache_entries 0→1). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
a91f4c550b |
[iterate-2W] Sustain the title present loop: viewport-size register + ISR CPU impersonation
The title's per-frame loop (sub_822F1AA8) is clock-B-paced and only re-fires when the swap count [controller+88] changes, which advances only on source=1 CP swap-complete interrupts. Each present batch the guest submits (via the sub_824CE348 -> sub_824BF4D0 builder) ends with a WAIT_REG_MEM on a per-CPU swap-acknowledge fence [GCTX+0] (GCTX = [device+10772]); the GPU parks there until the graphics ISR (sub_824BE9A0) clears that CPU's bit. Two coupled gaps kept ours emitting only ONE source=1 then dead-locking (draws plateaued at 28, run halted ~19.27M): 1. GPU MMIO register 0x1961 (AVIVO_D1MODE_VIEWPORT_SIZE) read as 0. The swap callback sub_824CE2B8 divides by its low 12 bits (display height) as a refresh-pacing term, so a 0 read tripped its `twi` divide-by-zero guard and aborted the ISR before it reached the fence-clear. Mirror canary GraphicsSystem::ReadRegister (graphics_system.cc:311): return 0x050002D0 (1280x720). 2. The ISR ran on an arbitrary borrowed thread, so [r13+268] (the PCR processor number) did not match the interrupt's target CPU. The ISR clears `1 << current_cpu` from the fence; running on the wrong CPU cleared the wrong bit and the fence (bit 2, from cpu_mask 0x4) never reached 0. Carry the target CPU through the interrupt queue (bit index of the PM4_INTERRUPT cpu_mask for CP, 2 for vsync per canary DispatchInterruptCallback(0, 2)) and impersonate it on the borrowed thread's PCR around the ISR, mirroring canary EmulateCPInterruptDPC -> XThread::SetActiveCpu. With both fixes the fence clears, the GPU drains each present batch, source=1 sustains per-present, clock B advances, and the loop runs continuously. Draws climb linearly with the budget (no re-stall): 50M 28->718, 200M ->3411, 1B ->18734; swaps 2->147/950/6060. No "Unanticipated CPU_INTERRUPT" trap. Inline-deterministic (--stable-digest byte-identical x2); n50m golden re-baselined. 675 tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
66bd805726 |
[iterate-2V] VdSwap: stop bumping primary CP_RB_WPTR out-of-band (canary-faithful)
Ours' `vd_swap` wrote its 64-dword XE_SWAP block at the guest's reserved `buffer_ptr` slot AND then bumped the primary ring `CP_RB_WPTR` out-of-band via `state.gpu.extend_write_ptr_by(64)`. That bump was a bug: `buffer_ptr` (~0x4add6efc) is NOT inside the primary ring (base ~0x4adcd000, 8192 dwords) — it lives ~10k dwords past it, in the renderer indirect-buffer region. The bogus WPTR bump pushed the GPU read-pointer PAST the guest's real write-pointer; the drain treated the overshoot as a circular wrap and re-executed the splash's draw indirect-buffers ~2×, inflating draws to 78 (the real splash geometry is ~28 draws; 12 INDIRECT_BUFFERs vs the real 6). Canary's `VdSwap_entry` (xenia-canary xboxkrnl_video.cc:518-548) writes the fetch-constant patch + PM4_XE_SWAP + NOP pad into the reserved slot and returns — it NEVER touches CP_RB_WPTR. The guest advances the primary ring write-pointer itself via its own doorbell once it has populated the slot; swap-complete CP interrupts come only from the game's in-stream PM4_INTERRUPT packets, never from VdSwap. This fix removes only the out-of-band `extend_write_ptr_by(64)` call, keeping the buffer_ptr block write intact and byte-faithful to canary. Effect at `--gpu-inline -n 50M`: draws 78→28, INDIRECT_BUFFER 12→6 (re-execution artifact gone), swaps 4→2. The run now halts at ~19.27M instructions (worker threads exit) instead of spinning to 50M, because removing the corruption unmasks the real per-present-interrupt deadlock — the title loop needs a per-present PM4_INTERRUPT that the stalled game never submits. That deadlock is a SEPARATE, known gate tracked/addressed elsewhere; it is intentionally NOT papered over here. Re-baselined golden crates/xenia-app/tests/golden/sylpheed_n50m.json to the new honest values (regenerated twice, byte-identical). sylpheed_n2m.json is unaffected (draws=0 at 2M). cargo test --workspace: 675 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
ad9c8e4cb8 |
[iterate-2U] VdGlobalDevice: allocate a real device cell so the swap counter (clock B) can advance
Sylpheed's title loop re-runs its per-frame manager update sub_821741C8
only when "clock B" ([controller+88], the swap count) changes. Clock B's
sole source is the CP swap-complete callback sub_824CE2B8, which bumps
[gfx+15160] via the TWO-LEVEL deref [[VdGlobalDevice]+0]+15160, where
VdGlobalDevice is the kernel variable export 0x01BE at guest .data
0x82000750.
Ours patched that import slot with literal 0 (the old "passed through to
Vd* shims, write 0" behaviour). Consequences, both confirmed at runtime:
* the guest's graphics init stores its D3D device object via
`stw r31, 0([0x82000750])` (sub_824C6DC0 @0x824C6F18) — with the slot
0, that store lands at address 0;
* the swap callback reads [[0x82000750]] = [0] = 0 and increments
[0+15160] (the null page) instead of the real device's swap counter.
So [gfx+15160] never moved, clock B stayed frozen at 0, sub_821741C8
fired exactly once, and the game submitted one render batch (the 78-draw
splash) then stalled.
Fix mirrors xenia-canary RegisterVideoExports (xboxkrnl_video.cc:557-564)
exactly: allocate a 4-byte cell, point the import slot at it, zero the
cell. The guest then stores its device into the cell, and the callback's
two-level deref resolves correctly. Verified: [0x82000750] now holds a
real cell whose [+0] is the device (gfx state), the swap callback bumps
[gfx+15160] 0->1, clock B advances, and the per-frame chain steps forward
(sub_821741C8 fires 1->2x, GamePart update sub_821C7CB8 0->1x).
Determinism: --gpu-inline digest re-baselined and byte-identical across
runs. The fix shifts the early execution trajectory (clock B unfreezing),
so the n50m golden moves imports 451500->178937 and instructions
50000001->50000014; draws/swaps/RTs/shaders unchanged (78/4/2/3). n2m
golden unchanged (early boot, pre-fix-effect). 675 workspace tests green;
sylpheed_n50m oracle green.
Note: this breaks the FIRST hard blocker (clock B could never advance at
all). Full per-frame sustain (draws past 78) needs a further step: each
GamePart update must submit a per-frame command buffer (with PM4_INTERRUPT)
during the asset-streaming phase to keep generating CP interrupts; ours
currently produces only the single seed interrupt from the initial batch,
so the chain advances once and re-stalls. Tracked for the next iterate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
873c197ff1 |
[iterate-2T] VdSwap: route present through ring PM4_XE_SWAP, drop out-of-band swap interrupt
Make ours' VdSwap present path faithful to xenia-canary `VdSwap_entry` (xboxkrnl_video.cc:518-548): write the reserved 64-dword ring slot with a PM4_TYPE0 fetch-constant patch + PM4_TYPE3(PM4_XE_SWAP) + NOP padding, then let the natural drain consume the swap packet in command-stream order. Remove the synthetic CP swap-complete interrupt that `notify_xe_swap` raised out-of-band. Root found this session (the actual present-path bug): ours' `notify_xe_swap` pushed an `InterruptSource::Swap` (→ INTERRUPT_SOURCE_CP) interrupt directly from the VdSwap HLE, decoupled from the GPU command stream. When that interrupt reached the graphics ISR `sub_824BE9A0` before D3D had armed its swap-callback slot (`[gfx+10772]+16` still the `0xBADF00D` placeholder), the ISR took its error path and hit the assert "ERR[D3D]: Unanticipated CPU_INTERRUPT. Sign of a corrupt command buffer?" (`bl sub_824C5DF0; twi` at 0x824BE9DC) — 2x per run on master. Canary's VdSwap raises NO interrupt; swap-complete CP interrupts come only from in-stream PM4_INTERRUPT packets, which are naturally ordered after the callback-arming Type-0 writes. Routing the swap through the ring packet matches that ordering and eliminates the trap (2 -> 0). Canary oracle confirmation (muted, audit_mem_watch + audit_jit_prolog_pc): canary's early/loading loop is present-driven — swap counter [gfx+15160] (0xBE56CA38) advances ~per-vblank from vblank 65 onward, reaching 0xD02 (3330) in ~60s via 6184 CP source=1 interrupts, with VdSwap called only ONCE. So the present interrupts are entirely in-stream, not from the VdSwap export. This is a correctness/faithfulness fix; it does NOT cascade. draws stay 78 at 200M and 1B because the upstream gate persists: the game submits one render batch then stalls (renderer sub_82506xxx 0x; 2nd title thread 0x821748F0 never spawns). The per-frame loop sub_822F1AA8 runs ~1207 iterations on vsync but clock B (swap count) only advances ~once, so the manager update sub_821741C8 fires once. That is the iterate-2Q/2F title-pipeline gate, not a present/ interrupt bug. swaps 3 -> 4 (the in-stream PM4_XE_SWAP now drains). Deterministic in inline mode (n50m --gpu-inline --stable-digest regenerated byte-identical twice; golden re-baselined: swaps 3 -> 4). cargo test --workspace 675 passing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
1ae472bd2b |
[iterate-2S] GPU: implement CP SCRATCH_REG memory writeback — arms Sylpheed's swap-callback slot
Sylpheed renders the splash (draws=78, iterate-2O) then plateaus: the
title's per-frame manager (sub_821741C8) only re-fires when "clock B"
([gfx+15160], swap count) changes, which only the CP swap-complete
callback sub_824CE2B8 increments. The graphics ISR sub_824BE9A0
indirect-calls that callback via [[gfx+10772]+16] on CP (source=1)
interrupts, but the slot stayed NULL so the callback never ran.
Root (runtime-verified, ours-side GPU): the guest arms the slot through
the Xenos CP scratch-register writeback path, which ours never
implemented. The arming IB (drained by ours at 0x4adf5180) contains a
Type-0 register write of the callback PC 0x824ce2b8 into SCRATCH_REG4
(0x057C). On hardware/canary, writing a SCRATCH_REG{n} mirrors the value
to SCRATCH_ADDR + n*4 in memory when the matching SCRATCH_UMSK bit is
set. Runtime values: SCRATCH_ADDR=0x0b1d5000 (the [gfx+10772]
descriptor), SCRATCH_UMSK=0x20033 (bit 4 set), so SCRATCH_REG4 ->
0x0b1d5010 = descriptor+16 = the callback slot (0x4b1d5010). Ours
decoded the Type-0 write into the register file but performed no
writeback (case a: drained-but-mishandled), so the slot stayed NULL.
Fix mirrors canary's CommandProcessor::HandleSpecialRegisterWrite
(command_processor.cc:545-552): a scratch_register_writeback() helper
called from handle_type0/handle_type1 after every register write; for
SCRATCH_REG0..7 with the UMSK bit set, it writes the value (big-endian,
as mem.write_u32 already stores) to SCRATCH_ADDR + n*4 (projected via
physical_to_backing). Deterministic given identical register state;
proven by unit test.
Cascade (verified by runtime probe): slot 0x4b1d5010 now armed with
0x824ce2b8; on the 2-3 CP interrupts that fire, the ISR reads the slot
and bcctrl's into sub_824CE2B8 (runs 2x; 0x cascade on master);
sub_824CE2B8 increments clock B ([gfx+15160]). The cascade does NOT yet
reach draws>78: there are only ~3 CP interrupts (from the initial 9825-
packet batch), and the title render loop stalls upstream (the iterate-2Q
title-respawn gate) before it submits more PM4_INTERRUPT work, so the
callback can't bootstrap a self-sustaining loop. This is the remaining
update-17/18 arming gap closed; the upstream stall is the next gate.
The default threaded GPU backend drains the ring on a separate host
thread, so with the callback now doing work the exact CP-interrupt
delivery instruction varies run to run (pre-existing GPU-thread race).
Pin the n50m oracle test to --gpu-inline (instruction-count
deterministic) and re-baseline its golden; bit-exact across repeated
runs. New unit test scratch_reg_write_mirrors_to_memory_when_umsk_enabled.
Tests: 675 pass (was 674). Golden re-baselined + determinism verified.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
034ec8b47f |
[iterate-2O] GPU: drain indirect buffers correctly — Sylpheed renders splash (draws 0→78)
Ours' GPU never drained the D3D driver's system command buffer past the first 11-dword indirect buffer, so DRAW_INDX / reg-0x57C-arm packets never executed and draws stayed 0 (the long-hunted render gate; see UPDATE-18). Runtime tracing (temporary, removed) showed the guest submits 6 INDIRECT_BUFFER packets at boot (CP_RB_WPTR 22→37) but ours executed exactly ONE IB and then spun 15.7M packets inside it. Three coupled command-processor bugs, all corrected to match canary: 1. `sync_with_mmio` applied the primary CP_RB_WPTR to whichever ring was active, including an executing indirect buffer — `37 % 11 = 3` clobbered the IB's write pointer so its read pointer looped 0→2→5→0 forever and never popped back to the primary ring. CP_RB_WPTR governs ONLY the primary ring; while an IB executes, the primary is the bottom of the IB stack. Canary executes each IB through a separate `RingBuffer reader_` (command_processor.cc), so the primary write pointer is structurally inapplicable to an IB. 2. Indirect buffers were treated as circular rings: read wrapped at `size_dwords` (`11 % 11 = 0`) and never reached the fixed write pointer, so even without the clobber the IB could not terminate. An IB is a fixed *linear* sub-stream; add `RingBufferView.indirect` and drain `[0, ib_size)` monotonically, then pop. 3. `is_ready` only checked the active ring, so an IB that now correctly exhausts would never get `execute_one` called again to pop back to the primary ring (whose WPTR may have advanced). Check the whole IB stack. Also: the ring was sized `1 << size_log2` bytes (1024 dwords) vs canary's `1 << (size_log2 + 3)` (8192 dwords) — an 8× undersize that desynced WPTR-wrap math from the guest. Fixed in `GpuSystem::initialize_ring_buffer` (and the dead bookkeeping copy in `vd_initialize_ring_buffer`). Cascade (deterministic; threaded-default backend, byte-identical across runs): reg 0x57C now written, IB jumps 1→12, packets 15.7M→9,825, and the splash renders — draws 0→78, shaders 0→3, render_targets 0→2, swaps 2→3 — stable at 50M / 200M / 1B. Boot then reaches a new downstream gate (draws plateau at 78, interrupts keep climbing → engine alive, not deadlocked). golden `sylpheed_n50m.json` re-baselined (draws 78). `cargo test --workspace` green (674; +2 ring_view regression tests). vd_swap's synthetic-swap short-circuit is now redundant but left untouched (cascade works without changing it); cleaning it up is a separate follow-up. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
93f60a3ba0 |
[iterate-2M] PCR+0x10C (PRCB.current_cpu): init per-HW-thread to unwedge spin-barrier
Ours never initialized the PRCB `current_cpu` byte at PCR+0x10C (prcb_data@0x100 + current_cpu@0xC). Canary sets it from `GetFakeCpuNumber(affinity)` (xthread.cc:847 `pcr->prcb_data.current_cpu = cpu_index`), which equals the HW thread id ours already writes at PCR+0x2C. Left unwritten it read 0 for every thread. Guest spin-barrier `sub_824D1328` (used by the audio/update pump threads at entries 0x824D2878 / 0x824D2940, ours tid 9 / tid 10) indexes a per-HW-thread occupancy byte array via `lbz r11, 268(r13)` then `stbx ..., [array+index]`. With index 0 for all threads, every thread marked slot 0; the multi-byte rendezvous signature it then spins on (`ld [obj+0x164]` compared against the packed per-slot expectation) could never assemble. Both pump threads busied at pc 0x824d140c/0x824d1410 forever (Ready, 5M+ barrier iterations) and never ran their `KeSetEvent` loops — so the events they signal (the 21k-per-thread heartbeat in canary) never fired, starving the downstream worker handshake. Fix: write `hw_id` to PCR+0x10C alongside PCR+0x2C in both the static thread image init (thread.rs) and the dynamic PcrWriter (state.rs, used by scheduler spawn + affinity migration) so the two stay in sync. Runtime-verified BOTH engines. Post-fix the pump threads escape the barrier (barrier iterations 5M+ -> 3) and advance into their loop bodies, now correctly Blocked(WaitAny) at pc 0x824d28d0 / 0x824d29c0 (was spinning at 0x824d140c). imports at n50M 339,766 -> 451,508; deterministic (two cold runs byte-identical). draws still 0 (a later, separate render gate). golden re-baselined. cargo test --workspace: 672 passed, 0 failed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
ed2e0e72fd |
[iterate-2J] KeTimeStampBundle deterministic tick: fix frozen+mislaid guest clock
The xboxkrnl data export KeTimeStampBundle (ordinal 0x00AD, import slot
0x820007d0 — confirmed via sylpheed.db imports table) was set up with TWO
defects in the import-patch pass:
1. FROZEN: the block was written once at boot and never updated, so every
field stayed a constant for the whole run (observed: the guest's clock
reader sub_824AA830 = [[0x820007d0]+0x10] returned a constant
0x01d6bc0c from 5M..150M instructions).
2. WRONG LAYOUT: it stuffed the FILETIME high-dword at +0x10. The canonical
X_TIME_STAMP_BUNDLE (xenia-canary kernel_state.h) is:
+0x00 interrupt_time u64 (100ns since boot)
+0x08 system_time u64 (FILETIME 100ns since 1601)
+0x10 tick_count u32 (milliseconds since boot)
+0x14 padding
so [block+0x10] is tick_count in ms, not a FILETIME dword.
Fix (deterministic, no wall-clock):
* Initialize the block with the correct field layout (tick_count = 0 at
boot, system_time = FILETIME base, interrupt_time = 0).
* Store the block VA on KernelState::timestamp_bundle_addr during the
import patch.
* Add KernelState::update_timestamp_bundle(mem, clock) and call it every
round in BOTH the lockstep (run_execution) and parallel
(run_execution_parallel) outer loops, right where the deterministic
Scheduler::global_clock is advanced. The clock is the retired-instruction
monotonic global_clock, so every guest-visible time value stays a pure
function of guest progress (lockstep byte-reproducible).
* Cadence: 1 global_clock unit = 100ns (coherent with parse_timeout, which
divides 100ns timeouts by 100 onto the same basis), so
INSTRUCTIONS_PER_MS = 10_000. tick_count now advances 0 -> ~4999ms over
a 50M-instruction window. Also make KeQuerySystemTime read the same
100ns clock instead of a frozen FILETIME constant.
Verification: tick_count at 0x40002010 now advances (deadline arm at
0x82450d0c stores clock+66 = 0x260,0x269,...,0x51d,... advancing, vs the
frozen 0x01d6bc4e before the fix). Determinism: two cold --stable-digest
runs are byte-identical; the n50m golden is UNCHANGED (the clock-affected
counter is not in the stable digest). 672/672 tests pass.
HONEST CAVEAT — the predicted render cascade did NOT materialize on this
branch. The diagnosed consuming gate at 0x82450b10 (the clock-vs-deadline
compare in the worker-hub channel loop sub_82450A68) is unreachable here:
the loop always branches away at 0x82450b0c ([this+220] >= channel-index),
so the hub already dispatches sub_82450B68 342x in BOTH the frozen and
fixed builds. Guest trajectory (imports 339766@50M / 1738001@200M /
9212446@1B), draws (0), swaps (2) and thread topology (tid14 Ready, not
blocked on 0x109c) are identical frozen-vs-fixed. This commit is therefore
a correct latent-clock-bug fix and determinism-safe prerequisite, NOT the
render unblock. The 0x109c/tid14 starvation premise was not reproduced at
f75bc96; the next gate must be re-localized.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
de21c7a544 |
[iterate-2G] db16cyc spin-hint cooperative yield: unblock title-screen 0x10a0 gate
The silph title state machine (tid13) blocked on event 0x10a0, never signaled. Root: the event's producer chain runs on the silph worker (entry 0x821C4AD0, our tid14), which was starved. tid14 shares a HW slot with a guest spinlock/ barrier participant (sub_824D1328, entry 0x824D2940) that busy-spins on the db16cyc hint `or r31,r31,r31` (encoding 0x7FFFFB78) at 0x824D140C. Under our round-robin lockstep the spinner consumed its whole block every round and starved the co-located tid14 (only 9 progress hits over 200M instr) — so the producer never reached the event-create/duplicate/signal dance the canary oracle performs (handle F80000E8 set by the submitter F8000044 via a duplicated handle). Fix (canary-faithful): recognize the db16cyc spin hint exactly as canary's InstrEmit_orx does (code 0x7FFFFB78 -> DelayExecution) and surface it as a new StepResult::Yield. The scheduler's yield_current() promotes every Ready peer on the slot past STARVE_LIMIT so begin_slot_visit picks one next round, then they reset and the spinner reclaims the slot — fair alternation, no priority inversion, pure function of slot state (deterministic). Result (lockstep, cache-persist, -n 200M): tid14 progresses past its old stall into a real wait; tid13 advances off 0x10a0 to a new event; hub/submitter re-enter their wait loops. imports 280k->592k, packets 124M->164M, swaps 1->2. draws still 0 (the splash's first draw is a further-upstream gate). Determinism preserved (two cold n50m runs byte-identical). n50m golden re-baselined (imports 90296->339766, swaps 1->2; draws unchanged 0). n2m golden unchanged (db16cyc not reached in first 2M). Tests 670/670. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
f3b7e8b760 |
[iterate-2F] Scheduler anti-starvation floor: fix job-4 handoff render gate
The lockstep scheduler's pick_runnable is strict priority (max_by_key (priority, -idx)). On a cooperative single-host HW slot, a CPU-bound spinner that never blocks (the silph poll loop pinned by affinity to hw=5) wins pick_runnable every round forever, permanently starving a co-located peer (the submitter, tid6) that the spinner is actually waiting on. On real hardware those threads run on separate SMT contexts concurrently, so the spinner never starves the submitter; ours collapses them onto one slot with no anti-starvation, turning priority (or equal-priority index order) into permanent starvation. The starved submitter never dequeued job-4 -> the worker-hub (tid5) blocked INFINITE on completion event 0x1080 -> silph (tid13) wedged on 0x1078 -> no vsync -> draws_seen=0, the publisher splash never renders. (decrement_quantum's within-slot rotation is dead: begin_slot_visit unconditionally re-pick_runnable()s each round, discarding the rotated running_idx. The fix is therefore evaluated at pick time, not via that discarded rotation.) Fix (Option A, bounded anti-starvation, deterministic): - Add per-thread steps_starved counter to GuestThread. - begin_slot_visit increments it for every Ready peer passed over this visit, resets it to 0 for the picked thread. - pick_runnable selects by effective_priority: once steps_starved reaches STARVE_LIMIT (4096) the thread is lifted to i32::MAX and wins exactly one pick, then resets. The genuinely higher-priority thread still wins ~4095/4096 visits -- the boost grants periodic forward progress only, it does NOT invert priority. Pure function of counter/priority/index -> deterministic (no wall-clock, no RNG). Cascade (lockstep exec, XENIA_CACHE_PERSIST=1, -n 200M): - submitter dequeue sub_82458508 now fires 4x (was 3x); the 4th job (buf 0x40baa2c0) is dequeued at cycle 6.15M. - hub tid5 leaves Blocked(0x1080) -> now Ready (no more INFINITE wait). - GPU packets 0 -> 116,101,363 (command stream now flowing). - tid13 (silph::UImpl) advances past the old 0x1078 wedge to a NEW downstream wait (handle 0x10a0); 3 new threads spawn (tid14/15/16). - draws_seen still 0 -> the splash's first draw is a NEW downstream gate, not this starvation. Determinism: two cold lockstep `check -n 5M` runs byte-identical (full and stable digests). New n50m stable digest deterministic across two cold runs. Golden re-baselined: instructions 50000007->50000003, imports 92317->90296 (trajectory shift from the changed pick order). Tests: 666/666 (+1 test_anti_starvation_bounded_progress). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
7e2603a9e5 |
[iterate-2E] Extend coherent monotonic clock to lockstep (timebase-desync livelock fix)
Lockstep livelocked the scheduler the same way --parallel did before
|
||
|
MechaCat02
|
5aaadfec36 |
[iterate-2E] Add XENIA_AUDIT_DEREF pointer-chase probe
On each AUDIT-PC-PROBE fire, treat gpr[reg] as a base object, dump its first 64 bytes, follow [base+off] to a sub-object, dump that, then follow [[base+off]+0] to its vtable and dump 48 slots. Env-gated (XENIA_AUDIT_DEREF=<reg>:<off>), read-only, lockstep digest unaffected. Captures the live work-item + stream object + vtable at sub_824510E0 before the pool recycles the slot — which overturned the prior session's "infinite spin" diagnosis: the streaming read PROGRESSES 68/68 128KB chunks of a 9MB file, then the hub (tid=5) blocks INFINITE on a self-created Event/Manual (0x1060) that is never signaled. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
0332d1990d |
[Track 2] Parallel-scoped global clock fixes timebase-desync livelock
In --parallel mode a long run livelocked: the scheduler spun "advanced to deadline 3000 waking hw=2 idx=0" ~14k times in microseconds. Root cause: each guest thread owns ctx.timebase (+1/instr in step_block), and all kernel deadline arithmetic read Scheduler::ctx(hw_id).timebase as "now". But the parallel worker extracts its PpcContext via mem::replace(ctx_mut_ref, PpcContext::new()) — leaving a ZEROED timebase in the slot while it steps unlocked — and advance_all_timebases_to only walks runqueue (never idle_ctx). So the coordinator's coord_pre_round drain and a woken thread's parse_timeout could read a zeroed/stale basis decoupled from the deadline the scheduler just advanced to. The thread re-armed the same constant deadline forever; the global clock never moved. Fix: add a single monotonic Scheduler::global_clock, advanced by the per-block retired-instruction count on each parallel writeback and floored up by advance_all_timebases_to. Kernel deadline reads route through KernelState::now_basis_at(hw_id), which returns global_clock ONLY when parallel_active; lockstep keeps reading the exact pre-existing ctx(hw_id).timebase expression, so the deterministic lockstep trace is byte-identical (sylpheed_n50m golden unchanged, zero re-baseline). Verified: - 50M --parallel run completes (was: hung). Deadlines now strictly increasing 5.4M -> 49.1M (18097 unique of 18116; max repeat 2) vs pre-fix constant 3000 x ~14000. - sylpheed_n50m golden byte-identical via plain `check` (no persist). - Full suite 665/665 green. Note: an intermittent parallel hang/crash (~1-2/20 at -n 5M) is pre-existing (master 1/20, this build 2/20 — within noise) and distinct from the timebase livelock: it is a parallel-race class (e.g. the unsafe block_ptr deref in run_execution_parallel). Tracked separately; lockstep remains the recommendation for long runs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
341196a111 |
[Issue-1 PPCBUG-020] Word-form ALU ops produce full 64-bit results
Xenon is a 64-bit PPC core (32-bit *pointer* ABI, but 64-bit registers and integer arithmetic). The interpreter was truncating every word-form integer ALU writeback to 32 bits and zero-extending, on a false "MSR.SF=0 / 32-bit ABI" premise. This silently corrupted any genuine 64-bit value flowing through word-form arithmetic. Confirmed load-bearing via runtime ours-vs-canary capture: Sylpheed's millisecond->LARGE_INTEGER timeout converter sub_824ACA88 does `clrldi; mulli r11,r11,-10000; std`. For a 16 ms wait the correct result is -160000 = 0xFFFFFFFF_FFFD8F00 (relative). canary stores exactly that; ours' truncating `mulli` stored 0x00000000_FFFD8F00 (positive) -> the i64 timeout read as a huge *absolute* deadline -> a ~26000x over-wait that froze the main frame loop. After the fix the timeout matches canary and the previously-frozen frame/worker loops run (parallel boot NtWaitForMultipleObjectsEx 94 -> 30428; KeWaitForSingleObject/critical-section loops resume). Fix mirrors canary's INT64 emitters (ppc_emit_alu.cc) op-by-op for the 17 data-losing word-form ops: addis, addic(.), subfic(.), mulli, add(c/e/ze/me)x, subf(c/e/ze/me)x, negx, mullwx. Only the result *writeback* widens to full 64 bit; the 32-bit carry (XER[CA]) and overflow (XER[OV]) computations and the CR0 i32 view are preserved byte-identical (the low 32 bits of the new result equal the old truncated result), so this is a strict no-op for clean 32-bit values and only restores the previously-zeroed upper bits for genuine 64-bit values. Genuinely-32-bit ops (rlwinm/slw/srw/cmpw, mulhw/divw whose upper bits are ISA-undefined) are left untouched. Updated 7 unit tests that asserted the truncation (they encoded the bug) to the canary-correct full-64-bit values. Re-baselined the sylpheed_n50m golden (imports 40454 -> 1790936: the unwedged frame/worker loops now cycle under the instruction-count timebase); sylpheed_n2m unchanged (pre-frame-loop). Lockstep determinism preserved (two 50M runs identical). Full suite 660/660. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
b20c99f141 |
[Subsystem-fixes] 6 verified ours-vs-canary divergence fixes
From the 2026-06-12 5-subsystem differential audit. All verified against canary as oracle; 660/660 workspace tests green (655 + 5 new). 1. nt_create_event polarity (exports.rs) — `manual_reset = gpr[5] != 0` was INVERTED. Canary xboxkrnl_threading.cc:668 `Initialize(!event_type,..)` + xevent.cc:41 (type 0 = NotificationEvent = manual, type 1 = Sync = auto). Now `== 0`. Was the dormant 2.AI fix on chore/portable-snapshot, never merged. The Ke-path was already correct; only the Nt-path was wrong. 2. 2.AF deadline drain (main.rs coord_pre_round) — expired KeWait/KeDelay deadlines never fired under load because advance_to_next_wake_if_due was only called in coord_idle_advance (no-Ready-threads path). Added a per-round drain loop; covers BOTH lockstep and parallel outer loops since both call coord_pre_round. Was the dormant 2.AF fix, never merged. 3. handle slab-recycle ABA guard (state.rs + scheduler.rs) — release_handle_slot (my round-34 regression) recycled a closed slot even with a thread still parked on it, risking a stale-waiter wake when the slot is re-minted. Added Scheduler::any_thread_waiting_on; decline to recycle a still-waited slot. 4. vpkpx pixel-pack (vmx.rs) — wrong field mapping (~100% mismatch). Now exact canary ppc_emit_altivec.cc:1795 shift/mask (red 6b out[15:10] from w[24:19], green out[9:5] from w[14:10], blue out[4:0] from w[7:3]; no fabricated alpha bit). +unit test. 5. VFS GDFX attribute plumbing (vfs/*, exports.rs query fns) — VfsEntry now carries the real on-disc attribute byte (GDFX dirent +12, canary disc_image_device.cc:136/154) instead of inferring directory-ness from path shape. Query exports report the real FILE_ATTRIBUTE_* bits. Candidate driver of the XamShowDirtyDiscErrorUI gate. +tests. 6. MmGetPhysicalAddress region-aware mirror (exports.rs) — flat 0x1FFFFFFF mask missed canary's +0x1000 host_address_offset for 0xE0000000+ mirror (memory.cc:2317). Read-only query; proven byte-identical 50M digest. +test. Investigated and intentionally NOT changed: - zero-on-recommit: no-op; ours has no region-reuse path (bump allocators, free is a stub). - 32-bit ALU writeback truncation (PPCBUG-020): documented-deliberate; premise (MSR.SF=0) is questionable but flipping it is out of scope here. - KeSetEvent/NtSetEvent return value: ours returns true previous state (hardware-faithful); canary returns constant 1 — NOT an ours bug. sylpheed_n50m golden will need re-baselining (legit behavior change). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
db90ad0f7d |
[AUDIT-059 R-D2] Phase D auto-signal POC confirms audit-049 wedge diagnosis
Hook NtCreateEvent for the silph::UImpl tid=13 chain (entry=0x821748F0, start_context=0x4024a840, frame-1 LR=0x821CB15C inside sub_821CB030+0x128) and auto-signal the resulting handle after XENIA_SILPH_UI_AUTOSIGNAL_DELAY instructions. Env-gated; default off. SR4 verdict B (partial unwedge): - handle 0x1078 signal_attempts 0->1 - tid=13 Blocked(WaitAny[0x1078]) -> Ready pc=0x824a9108 - ExCreateThread 10 -> 12 (new silph::UImpl tid=14, worker tid=15) - New downstream wedges 0x1084 + 0x1088 - cxx_throw runtime_error on tid=5 inside R26 dispatcher (BST not-registered instance lhs=0x715a7af0) - VdSwap stays 1; no draws (POC is diagnostic, not final fix) Confirms Phase C diagnosis end-to-end. The real signaler must (a) drive NtSetEvent on the silph KEVENT AND (b) register the dispatcher's BST instance upstream; this POC only does (a). Reading-error class #20: ctx.lr at kernel export entry is the thunk wrapper's return slot, NOT the guest caller's post-bl PC. Walk back-chain 1 step to get frames[1].lr. Reading-error class #21: --parallel and lockstep have SEPARATE outer loops in main.rs (run_execution_parallel line 2928 vs run_execution line 2706). Per-round hooks must be wired in BOTH paths. Files: - crates/xenia-cpu/src/scheduler.rs: GuestThread.start_entry/start_context fields + spawn() population + current_thread_entry_and_ctx() helper - crates/xenia-kernel/src/state.rs: AutoSignalPending struct, env-parsed silph_autosignal_delay, pending Vec, last_cycle_hint, set_now_cycle_hint, maybe_register_silph_autosignal (walks back-chain), fire_due_silph_autosignals - crates/xenia-kernel/src/exports.rs: hook in nt_create_event - crates/xenia-app/src/main.rs: fire-site + cycle hint in both outer loops - audit-runs/audit-059-handle-disambiguation/round-D2-autosignal-poc/FINDINGS.md Tests 655/655 green. Default behavior byte-identical when env unset. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
9340ff4592 |
[Audit] --audit-r3-dump-bytes: dump N bytes at r3 when probe fires
AUDIT-059 round 15 — diagnostic. When `--audit-r3-dump-bytes=N` is set, every `--audit-pc-probe-hex` fire emits a paired `AUDIT-R3-DUMP` line with N bytes of guest memory from r3 as u32 lanes (4-byte aligned, cap 256B). Sized for the 80-byte stack-local struct at sub_82452DC0's `r31+96` (probe sub_8245B000 entry where r3 IS the struct ptr). Settable via `XENIA_AUDIT_R3_DUMP_BYTES` env. Read-only; lockstep digest unaffected (empty-set fast path in fire_audit_pc_probe_if_match). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
bcd018659b |
[Audit] --audit-mem-dump-chain: deref a guest address N levels for diagnosis
Round-14 of AUDIT-2BF (singleton-dump). The bctrl at sub_822F1AA8+0x90
(PC 0x822F1B4C) loads [0x828E1F08] (a global singleton), dereferences
its vtable, and indirect-calls vtable[0]. Canary returns; ours hangs.
To name the resolved target we need to dump the (singleton, vtable,
vtable[0]) chain on probe firing.
Adds `--audit-mem-read-hex` / `XENIA_AUDIT_MEM_READ` taking a single
guest VA. When set and any `--audit-pc-probe-hex` PC fires, the kernel
emits a paired `AUDIT-MEM-READ` line with three guest reads:
AUDIT-MEM-READ addr=0x828E1F08 val=<*addr> vtable=<**addr> \
vtable[0]=<***addr+0> vtable[24]=<***addr+24> ...
`vtable[24]` is included as the slot-6 method (audit-059 round 9
documented the canary silph chain dispatching slot 6 of a vtable here).
Read-only; lockstep digest unaffected. ~30 LOC across state.rs and
main.rs. `cmd_check` opts out of the flag (same policy as the existing
audit_pc_probe_hex).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
09e59e09b7 |
Audit-2BF.delta: add --audit-pc-probe-hex for silph-init bctrl probe
Adds a per-PC probe analogous to --lr-trace / --branch-probe but tuned
for the silph init chain's virtual-dispatch site at sub_82172BA0+0x1E8
(PC 0x82172D88, the bctrl after a 3-deep `lwz` chain that loads vtable
slot 6). Each fire emits one AUDIT-PC-PROBE line with (pc, tid, hw,
cycle, lr, r3, r11) plus four guest-memory dereferences off r3 — the
vtable, slot-6 method pointer, auxiliary handle field, and embedded
sub-object vtable — so the line can be compared head-to-head with
canary's round-9 capture (r3=0xBCCC52C0, [r3+0]=0x820A3644,
slot6=sub_821B55D8, [r3+0xC]=0xF80000D8, [r3+0x30]=0x820A1870) to
identify whether ours dispatches to the wrong vtable on a correct
object (case A) or to a wrong object entirely (case B).
Why this addition rather than reuse of an existing probe: --lr-trace
emits JSONL designed for canary-side diffing and only captures
r3/r4/r5/r6/lr (no memory dereferences); --branch-probe captures CR
flags and lr but again no memory; --ctor-probe is single-shot per PC
and walks the stack back-chain. None of them load the four indirect
fields needed to identify a vtable-shape divergence.
Implementation:
- state.rs: new HashSet<u32> field `audit_pc_probe_pcs` and helper
`fire_audit_pc_probe_if_match(hw_id, mem)`. Empty-set fast-path
keeps the cost to one is_empty() check per worker_prologue call
when the flag is unused. Read-only — no guest state mutation,
lockstep digest unchanged.
- main.rs: new CLI flag --audit-pc-probe-hex with bare-hex comma
parsing (tolerates `0x` prefix), settable also via
XENIA_AUDIT_PC_PROBE env var. Threaded through cmd_exec_inner;
cmd_check passes None so check digests are unaffected.
Probe wired into worker_prologue alongside fire_ctor_probe / fire_-
branch_probe / fire_lr_trace. Like its siblings, it fires once per
basic-block entry — known limitation (audit-045 reading-error class
13); use a block-entry PC if probing a mid-block instruction.
Verification: kernel 127/127, app 5/5 non-ignored, no behaviour
change with empty flag.
Cross-references audit-059 round 9's canary capture and lays the
groundwork for the round-10 ours-side comparison.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
5a8fe21ad5 |
Iterate-2.BF.γ: refine is_in_callback gate to per-thread exclusion
Lockstep vsync delivery was capped at 54/run despite the ticker firing
333 periods and dispatcher being called 1.2M times. Root cause: the
blanket `is_in_callback()` gate skipped dispatch entirely whenever the
async audio path held `interrupts.saved`, which is essentially the
entire boot (audio worker rarely hits its LR_HALT_SENTINEL between
back-to-back callbacks). 5.85M dispatch_skip_in_callback events drowned
out the 55 with-pending windows.
Graphics dispatch (iterate-2.BE) runs the ISR synchronously and
restores the borrowed context before returning — it doesn't touch
`interrupts.saved`. The only real conflict is if graphics picks the
*same* thread audio borrowed (which would stomp audio's
SavedCallbackCtx). Replace the blanket gate with per-thread exclusion:
when audio is mid-flight, exclude only its `injected_ref` from
victim selection. Falls through to the existing no-victim drop if
that's the only candidate.
Lockstep (50M instr): gpu.interrupt.delivered{source=0} 54 → 295
(5.5×), all 333 ticker periods either delivered or unarmed (no more
queue_full_drops). Wallclock unchanged ~3 s.
Parallel (30M instr): 1193 → 3458 baseline lift (2.9×), no regression.
Tests: xenia-kernel 127/127, xenia-app 5/5 non-ignored. Lockstep
goldens will drift (interrupts.delivered is in the digest); deferred
to next iterate.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
51489e34db |
Iterate-2.BE Path β: tick vsync from coord_idle_advance
The iterate-2.BE host-driven synchronous ISR dispatcher relies on
something queueing v-syncs. In lockstep that's `tick_vsync_instr`,
called from `coord_pre_round` per round. If the scheduler stalls into
`coord_idle_advance` (no Ready threads), the instruction counter
freezes — the accumulator stops incrementing, the ticker stops
queueing, and the dispatcher is left starved for the duration of the
idle wait.
Tick `tick_vsync_wallclock` at the top of `coord_idle_advance` so
v-syncs keep firing on host time even when the guest scheduler is
parked. The dispatcher in the outer loop drains whatever we queue on
the next iteration. Same MMIO `D1MODE_VBLANK_VLINE_STATUS` bit-set as
the production path.
Note: empirically in Sylpheed at 50M/500M instruction horizons,
`coord_idle_advance` is never reached (tids 9/10/12 stay Ready through
the early-boot deadlock), so this commit doesn't move
`gpu.interrupt.delivered{source=0}` off 54 for this title at these
horizons. It is the correct fix for the documented starvation pattern
and will activate as soon as the kernel reaches a state where Ready
threads drop to zero with timers/waits pending.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
9a93152981 |
Iterate-2.BE: host-driven synchronous graphics ISR delivery
Replaces the victim-thread-mutate-then-wait scheme for vsync / CP
interrupts with synchronous in-line dispatch on the coordinator host
thread. Mirrors canary's EmulateCPInterruptDPC -> Processor::Execute
path (kernel_state.cc:1370, processor.cc:413): pick a guest thread,
borrow its PpcContext, jam ISR PC + args in, run the interpreter
inline until LR_HALT_SENTINEL, restore the borrowed context.
Why: audit-059 measured gpu.interrupt.delivered{source=0} = 54 over
3.9 s vs canary's 4712 over 30 s. Per-second shortfall ~11×. Old
asynchronous LR-sentinel injection (try_inject_graphics_interrupt)
needed a Ready or Blocked guest thread to land on; once the Sylpheed
main thread and worker threads all idled post-boot, no victim was
available and every queued vsync got dropped. Host-driven dispatch
decouples delivery from guest-thread readiness.
Smoke test (lockstep): unchanged 54 — under current Sylpheed boot
trajectory the ticker is gated by guest-instruction progress, not
victim availability; lockstep stalls into idle-advance after ~5M
instructions of real work and the synthetic tick_vsync_instr stops
firing. Under --parallel (wallclock ticker) gpu.interrupt.delivered
climbs to ~1131 over a 128 s run, confirming the synchronous
dispatcher itself works as intended. Architectural piece is now in
place; raising the lockstep delivery rate requires ticking the
synthetic vsync inside coord_idle_advance, which is a separate
change.
Changes:
- crates/xenia-kernel/src/interrupts.rs: doc-comment update only.
SavedCallbackCtx + CALLBACK_STACK_PAD retained — the audio
callback path (audit-048) still uses the asynchronous LR-sentinel
inject on a dedicated per-client worker.
- crates/xenia-app/src/main.rs:
* dispatch_graphics_interrupts(kernel, mem, &mut stats,
&mut decode_cache, thunk_map): new fn. Drains the full FIFO per
call. Victim selection same shape (Ready preferred, else
Blocked, skip Idle/Exited/ServicingIrq), but the call is
synchronous - we run step_cached + import-thunk dispatch inline
on the borrowed ctx until pc == LR_HALT_SENTINEL.
MAX_INSTRS_PER_ISR = 1M safety budget.
* coord_pre_round: graphics-IRQ injection call removed. Audio
path unchanged (still calls try_inject_audio_callback).
* run_execution + run_execution_parallel: each now owns a
persistent isr_decode_cache and calls
dispatch_graphics_interrupts after coord_pre_round.
* try_inject_graphics_interrupt: deleted (118 LOC).
No new public APIs, no new dependencies, no changes to xenia-cpu.
Tests: workspace 765 passed / 0 failed / 4 ignored (parallel_stress
+ sylpheed_n50m, all gated). Kernel 127/127, app 5/5, cpu 288/288.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
ac2f89a7bb |
Re-baseline sylpheed_n50m golden post-AUDIT-054
instructions: 50000002 → 50000001 (1-instr shift from FILE_DIRECTORY_FILE plumbing on NtCreateFile path; all other digest fields unchanged — imports/swaps/draws/render-targets/shaders/textures all match prior golden). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
25704c5811 |
Re-baseline sylpheed_n50m golden post-AUDIT-032
Companion to
|
||
|
MechaCat02
|
49f3eafa15 |
AUDIT-032: dedicated audio worker thread per client (Plan B)
Replaces APUBUG-PRODUCER-001's random-victim-hijack audio injection with a dedicated per-client guest worker thread, mirroring xenia-canary's apu/audio_system.cc:84-159 WorkerThreadMain pattern in xenia-rs's threading model. Audio callback ticker is now safe to enable by default. ## What changed - xenia-kernel/src/xaudio.rs: new XAudioState fields worker_handles + worker_refs (one slot per of XAUDIO_MAX_CLIENTS=8). Synthetic park-handle helper (0xF000_0000 | client_idx) — outside the normal alloc range so wake_eligible_waiters never finds it; the only legitimate state-flip is via try_inject_audio_callback. - xenia-kernel/src/exports.rs: xaudio_register_render_driver spawns a 64KB-stack guest thread (create_suspended=true) via state.scheduler.spawn after registration succeeds. Immediately flips the spawned thread's state from Blocked(Suspended) to Blocked(WaitAny[synthetic]) so it's parked but not woken. Stores the kernel handle so find_by_handle resolves a fresh ThreadRef after slot compaction. Failure paths log + leave xaudio.worker_refs[i] = None, in which case the ticker drops fires (no random-victim fallback). - xenia-app/src/main.rs: try_inject_audio_callback resolves the worker via worker_handles[index] instead of scanning runqueues for a Ready or Blocked victim. The PC+r3 injection and SavedCallbackCtx capture are unchanged; the existing LR_HALT restore path re-blocks the worker on its synthetic handle for the next tick. Flag handling reworked: --xaudio-tick / XENIA_XAUDIO_TICK now act as explicit override (truthy = force on, falsey = force off, absent = use the KernelState default). - xenia-kernel/src/state.rs: xaudio_tick_enabled default flipped from false to true. Pre-fix it was off because the random-victim hijack regressed swaps=2->1; with the dedicated worker that whole class of regression is gone. ## Cascade verification at -n 500M (audit-runs/audit-048-audio-host-pump/) Pre-fix baseline: audit-runs/audit-047-gamma-wedges/ours-end-state.log. | Dim | Predicted (AUDIT-032) | Observed | |-----|-------------------------------------|---------------------------------| | A | tid=9 leaves Blocked[0x828A3254] | Ready @ pc=0x824d1404 | | B | tid=10 leaves Blocked[0x828A3230] | Ready @ same pc/lr | | C | XAudioSubmitRenderDriverFrame > 0 | Mixer setup path executed | | D | KeReleaseSemaphore 0 -> non-zero | 0 -> 1; xaudio.callback.delivered=1 | Bonus: audit-042's tid=6 worker pair on 0x10A0+0x10A4 also went Blocked->Ready as a downstream effect. Boot trajectory shifted significantly: NtWaitForSingleObjectEx 1,489,791 -> 30; NtSetEvent 3,334 -> 68; new exports firing (StfsCreateDevice, ObCreateSymbolicLink, XamContentCreateEnumerator, XamEnumerate, XamTaskSchedule, ExCreateThread x10, KeSetAffinityThread x7, NtCreateSemaphore x4, NtWaitForMultipleObjectsEx x94, NtDuplicateObject x14, XeCryptSha, XeKeysConsolePrivateKeySign). The system left the audio-wait busy loop and entered the savegame/content/crypto init phase. swaps regressed 2 -> 1 (degenerate splash repeat lost; main thread now advances past splash entirely, blocked on a different handle). draws unchanged at 0 — expected per AUDIT-032 (audio gate != renderer gate). ## Tests + scope - cargo build --release succeeds, no new warnings. - cargo test -p xenia-kernel --lib: 127/127 pass (incl. xaudio). - cargo test -p xenia-app --lib: 5/5 non-ignored pass. - Lockstep goldens (sylpheed_n2m / sylpheed_n50m) WILL drift on this fix and need re-baselining as a follow-up commit. 75 net non-comment LOC across 4 files, well under AUDIT-032's 60-120 LOC budget. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
e428ce33aa |
M9.5 + M11.5 + VMX + SJIS/UTF-8: close the post-M5.5 deferred set
Closes the four remaining deferred follow-up items in one bundle. All four are smaller-scope and additive; lockstep determinism unaffected (analyzer-only changes). ## M9.5 — __CxxFrameHandler scope-table parsing - New `xenia_analysis::eh_scope` module. Magic-scans .rdata for the three documented MSVC FuncInfo signatures (0x19930520/21/22) on 4-byte alignment. Each match is parsed as the documented struct (BE u32 fields), with sanity caps on max_state / n_try_blocks / pointer validity. - Walks pUnwindMap (UnwindMapEntry, 8 bytes) and pTryBlockMap (TryBlockMapEntry, 20 bytes) into one row each. - New tables eh_funcinfo, eh_unwind_map, eh_try_blocks. - Sylpheed yield: 2,588 FuncInfo (all version 0x19930522) / 10,019 unwind entries / 315 try-blocks. ## M11.5 — Static-init driver chain detection - New `xenia_analysis::static_init` module. Walks every function looking for the canonical _initterm loop: lwz cursor; mtctr; bcctrl; addi cursor, cursor, 4 bounded by a compare against another constant register. Extracts (array_start, array_end) and reads the array. - Reuses `function_pointer_arrays` table — drivers' arrays land with kind='static_init' (replacing M11's prologue-heuristic output where the structurally-grounded pattern fires). - Sylpheed yield: 0 drivers detected — the binary's static-init structure does not match the canonical CRT loop. Infrastructure ready; future M11.6 can relax. ## VMX vector-store xrefs (M6 follow-up) - Adds AltiVec/VMX X-form load/store XOs to the M6 opcode-31 dispatch: lvx/lvxl/lvebx/lvehx/lvewx (reads) and stvx/stvxl/stvebx/stvehx/stvewx (writes), all addr_mode= 'x_form_indexed'. Static resolution still requires both rA and rB constant. - Sylpheed yield: 110 newly-detected stvx writes. ## Shift_JIS + UTF-8 localised-string detection (M7 follow-up) - Extends `xenia_analysis::strings::analyze` with scan_shift_jis (JIS X 0208 lead/trail byte ranges + half-width katakana pass-through) and scan_utf8 (2- and 3-byte sequences). At least one multi-byte unit required so pure-ASCII strings aren't double-counted. - SJIS bytes rendered as \xHH escapes for diagnostic readability; full SJIS→UTF-8 decoding deferred. - Sylpheed yield: 790 Shift_JIS strings (Japanese debug + UI text) + 39 UTF-8. ## Tests - +2 EH (parses_minimal_funcinfo_v0, rejects_bogus_max_state) - +2 static_init (detects_canonical_initterm_loop, rejects_function_without_pattern) - +2 strings (detects_shift_jis_string, detects_utf8_multibyte_string) Tests 649→655 (+6 unit tests). DB schema golden + write_analysis_results signature updated for new EH parameter. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
56ffa40a6a |
M5.5: this-flow indirect-dispatch resolution via vptr-write inference
Closes the dominant case M5 could not resolve — `lwz vt, off(this);
lwz fn, slot(vt); mtctr; bcctrl` (real C++ dispatch). Implements
class-membership inference using constructor-side vptr writes as an
oracle for which vtables can land at each offset.
## Algorithm
Phase 1 — vptr-write scan: walk every function with the existing
lis+addi register tracker. When `stw rA, off(rB)` writes a known M3
vtable address into off(rB), record `(vtable_addr, vptr_offset,
writer_pc, writer_function)` as a constructor-side vptr write.
Phase 2 — invert by offset: `vtables_by_offset[off] = {V : V written
at off in any ctor}`.
Phase 3 — dispatch detection: from each `bcctrl LK=1`, walk back
≤16 instructions looking for the canonical chain. Bail on register
clobber, branch, or label (basic-block) boundary.
Phase 4 — edge emission: for `(dispatch_pc, vptr_off, slot)`, emit one
`xrefs.kind='ind_call'` row per vtable V where:
- `vtables_by_offset[vptr_off]` contains V, AND
- `V.length > slot` (V actually has a method at that slot)
Multi-candidate sites (the common case at offset 0) are an
over-approximation; downstream queries filter to single-candidate sites
for high confidence:
`WHERE candidate_count=1` in `indirect_dispatch_sites`.
## Schema
NEW TABLES:
- `vptr_writes(writer_pc, vtable_address, vptr_offset, writer_function)`
- `indirect_dispatch_sites(dispatch_pc PK, vptr_offset, slot, candidate_count)`
- `indirect_dispatch_candidates(dispatch_pc, vtable_address, method_address)`
NEW INDICES on vtable_address / vptr_offset / method_address /
(vptr_offset, slot) for fast joins.
## Sylpheed yield
- 567 vptr writes / 214 vtables / 29 offsets (offset 0 = 88%).
- 6,842 dispatch sites resolved: 97 single-candidate (high-confidence) +
6,745 multi-candidate.
- 687,963 ind_call xref rows.
- 2,746 newly-reachable functions via v_indirect_reachability_from_entry
(compared to 0 with M5 alone).
- Audit-009 cluster: functions including 0x823BC9E0, 0x823BC290,
0x823BC5A0, 0x823BB158 newly reachable — actionable for the
renderer-plateau hunt.
Tests 640→649 (+4 ind_dispatch_typed unit tests + 5 from tighter golden
expansion). Schema golden + write_analysis_results signature updated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
77034b6cbf |
audit-038: persistent cache:/* VFS via host-FS backing
Replaces the "Synthesized empty file" stub for cache:/* paths with a
real host-FS HostPathDevice-style mount. Each KernelState gets a fresh
per-process tmpdir under /tmp/xenia-rs-cache-<pid>-<id>/ which is
cleared on init for lockstep determinism (mirrors canary's
xenia_main.cc:649 RegisterSymbolicLink("cache:", "\\CACHE") +
HostPathDevice in xenia-canary/src/xenia/vfs/devices/host_path_device.cc).
NtCreateFile now honours create_disposition for cache: paths:
FILE_OPEN -> NOT_FOUND if missing
FILE_CREATE -> NAME_COLLISION if present
FILE_OPEN_IF -> open or create
FILE_OVERWRITE_IF -> create or truncate
FILE_OVERWRITE -> NOT_FOUND if missing, else truncate
FILE_SUPERSEDE -> create or truncate
NtReadFile / NtWriteFile / NtSetInformationFile (XFileEndOfFileInformation)
/ NtQueryInformationFile / NtQueryFullAttributesFile route through
std::fs against the per-handle host_path; non-cache paths keep their
legacy semantics (read-only disc image, synth-empty stubs).
Verified by audit-037 cascade:
- sub_82459D18 (cache-miss restore): 0 fires (was firing constantly)
- sub_8245D230 (resize/zero-fill): 0 fires (was firing constantly)
- 105+ real cache-file writes per 500M run; 4+ MB of game data persisting
to disk per boot; cache:/recent, cache:/access, cache:/d4ea*.tmp, etc.
- Lockstep deterministic at instructions=100000004 / imports=987485
across 3+ reruns (digest shifted as expected; goldens re-baselined).
- swaps=2 plateau still in place; cluster L1 unactivated. Cascade
dimension D (cluster activation) — UNKNOWN, no L1 fires.
Tests 640 -> 645 (+5 cache-specific unit tests; full workspace green).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
5af792c9fc |
M8+M9+M10+M11+M12: LOW-tier milestones — funcptr-arrays, EH flag, TLS, lr-trace
Five LOW-priority milestones bundled. Total ~700 LOC across 11 files. ## M9 — has_eh derived from pdata.flags exception bit - New `functions.has_eh BOOLEAN NOT NULL` column. Derived from M1's already-parsed `pdata.flags` (bit 31 of the packed word — the exception-handler-present flag, distinct from bit 30 which is the always-1 32-bit-code flag). Index idx_functions_has_eh. - Sylpheed: 2,975 of 23,073 pdata-validated functions have EH (12.9%). ## M10 — .tls section / IMAGE_TLS_DIRECTORY32 parser - New `xenia_xex::tls::parse_tls` parses the directory + zero-terminated callback array. Returns None when the binary has no .tls section. - New `tls_info` (singleton row) + `tls_callbacks(slot, address)` tables. - New `DbWriter::write_tls()` no-ops on None. - Sylpheed has no .tls section → 0 rows; infra ready for binaries with __declspec(thread). ## M8 + M11 — function_pointer_arrays (dispatch tables + static initialisers) - New `xenia_analysis::funcptr_arrays::analyze` widens M3's vtable scan: detects runs of ≥2 function pointers in .rdata and classifies each as `vtable` (M3 re-emit), `dispatch_table` (M8), or `static_init` (M11) via a constructor-prologue heuristic (mfspr + small stwu). - New tables `function_pointer_arrays(address PK, length, kind)` and `function_pointer_array_entries(array_address, slot, function_address)`. - Sylpheed: 722 vtables + 388 dispatch_tables = 1,110 arrays / 6,347 slots. 0 static_init detected (Sylpheed's ctors don't all match the conservative heuristic; M11.5 future work can chain via the entry- point's static-init driver). ## M12 — --lr-trace runtime canary-diff harness - New CLI `exec --lr-trace=PC[,PC,...]` and `--lr-trace-out=PATH` flags. Symbolic resolution (Class::method, Class::*) via M4 lookup. Env vars XENIA_LR_TRACE / XENIA_LR_TRACE_OUT also work. - New `KernelState::lr_trace_pcs` + `lr_trace_writer` + helper `fire_lr_trace_if_match(hw_id)` invoked from the per-instr probe slot. - JSONL output: pc/tid/hw/cycle/r3/r4/r5/r6/lr — superset of what xenia-canary's --log_lr_on_pc patch emits, with a cycle counter for cross-run reproducibility. Diff-friendly via `jq`. - Lockstep digest unaffected: smoke test on entry-point PC fires once with cycle=0/lr=BCBCBCBC/all-GPR-zero (correct initial state). Tests 636→640 (+2 TLS tests, +2 funcptr_arrays tests). Schema golden updated for new tables + has_eh column. Lockstep determinism preserved (instructions=2000005 ×2 reruns identical). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
38d8871e8d |
M6: addr_mode column on xrefs + extended store/load classes
Adds finer-grained addressing-mode classification to every data xref row plus new dispatch for instruction families not previously emitted: - New `xrefs.addr_mode VARCHAR NULL` column. NULL for control-flow edges (call / ind_call / j / br); one of d_form / lis_addi / lis_ori / multiword / x_form_indexed / x_form_byterev / atomic / dcbz for data edges. Index idx_xrefs_addr_mode. - New `xenia_analysis::xref::AddrMode` enum + Xref::addr_mode field. - Opcode 46/47 (lmw/stmw) expand to one xref per slot — D-form multi-word load/store now resolves all (32-rS) consecutive addresses. - Opcode 31 X-form dispatch — stwx/stbx/sthx/stwux/stbux/sthux/stdx/stdux, lwzx/lbzx/lhzx/lhax/lwzux/lbzux/lhzux/lhaux/ldx/ldux, stwcx./stdcx. (atomic), stwbrx/sthbrx/lwbrx/lhbrx (byte-reverse), dcbz (cache-line clear). - X-form rows are emitted ONLY when both rA and rB resolve to known constants (rare but present); the dominant runtime-indexed pattern remains correctly skipped. Sylpheed yield (regen on master + merge): - 442 newly-detected x_form_indexed reads (lwzx/lhzx into static tables). - 40 newly-detected atomic writes (stwcx./stdcx. with resolvable address). - 28,834 lis_addi refs, 18,485 d_form reads, 3,288 d_form writes — every pre-existing data row now tagged. - 0 multiword / dcbz / byterev (these instructions exist but aren't on lis+addi-tracked code paths). Tests 633→636 (+3 xref unit tests covering AddrMode tag uniqueness, data-edge addr_mode round-trip, control-edge None invariant). Schema golden updated (xrefs gains addr_mode column). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
ab4fe211e5 |
M5+M7: indirect-dispatch reachability + .rdata string detection
Two MEDIUM milestones bundled (both opportunistic per plan; both small).
## M5 — indirect-dispatch reachability
- `xenia_analysis::indirect`: per-basic-block register tracker over each
detected function. Recognises the canonical static-vtable pattern
`lis+addi → lwz off(rA) → mtctr → bcctrl` where rA holds a known M3
vtable address. Emits one `Xref { kind: IndirectCall }` per resolvable
bcctrl site.
- PowerPC ABI awareness: `bl`-style calls clobber volatile r0..r12 + ctr
but preserve non-volatile r13..r31, so a vtable pointer parked in r30/r31
before a call survives.
- Label-based basic-block boundaries kill register state — bounds
false-positive risk for jump-IN paths.
- New `XrefKind::IndirectCall` variant (DB tag `'ind_call'`).
- New SQL view `v_indirect_reachability_from_entry` — strict superset of
`v_reachability_from_entry`, taking `ind_call` edges in the BFS.
Sylpheed yield: 0 edges detected. The binary's 1,001 static lis+addi
references into vtables are nearly all constructor-side vptr writes, not
dispatches; real method dispatch goes through `this->vptr` which requires
alias analysis we explicitly don't do. Documented in SCHEMA.md as the
expected limitation. Three unit tests cover the synthetic-correctness path.
## M7 — string / constant-pool detection
- `xenia_analysis::strings`: scans `.rdata` for runs of ≥ 6 printable
ASCII bytes (NUL-terminated) and ≥ 6 UTF-16LE code units (basic-plane
printable ASCII, NUL u16 terminator).
- New `strings(address PK, encoding, length, content)` table + encoding index.
- Implicit cross-ref via existing `xrefs.kind='ref'` rows whose target
matches a strings.address.
Sylpheed yield: 6,311 ASCII strings (including embedded HLSL shader source
and AS_CB_SURFACE_SWIZZLE_* assertion strings). 9,132 lis+addi sites
cross-reference detected strings — names source PCs near each string in
one query. Four unit tests cover encoding detection, NUL termination, and
short-run rejection.
Tests 626→633 (+3 indirect, +4 strings).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
4ff08f6116 |
M4: class-aware probe tokens via M3 vtable+method tables
CLI extension only — no schema change. Adds symbolic resolution for --pc-probe / --branch-probe / --ctor-probe tokens: - `0xADDR` / `2186674160` — numeric (current behavior, no DB load). - `Class::method` — joins classes × methods × demangled_names. - `Class::*` — joins classes × methods (all slots). - `function_name` — falls back to functions.name for free functions / saverestore stubs / labels. New `xenia_analysis::lookup::resolve_probe_token(db_path, token)` opens the DB read-only ONLY when a token is non-numeric, so legacy numeric flows pay no IO. New `--probe-db PATH` flag (or `XENIA_PROBE_DB` env / default `sylpheed.db` next to the .iso) selects the DB. Symbolic resolution happens BEFORE any guest exec, so it cannot affect the lockstep digest. Verified deterministic across two reruns at -n 2M (instructions=2000005 identical). End-to-end smoke test on Sylpheed: `--pc-probe='ANON_Class_6B674251::*'` resolves to all 45 method PCs of that anonymous class (matching the methods-table row count for that vtable). Tests 621→626 (+5 lookup unit tests covering numeric passthrough, symbolic-without-DB error, Class::method resolution, Class::* expansion, and functions.name fallback). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
1d6c51fbf8 |
M3: vtable scan + MSVC RTTI walk + 3 new tables
Adds detection of statically-allocated MSVC vtables in .rdata/.data: - New `xenia_analysis::vtables` walks read-only sections looking for runs of ≥3 contiguous big-endian u32 values where each value lands on a known function start (from M1's corrected functions table). 2-slot runs are rejected to keep false-positive rate down. - For each candidate the MSVC RTTI walk vtable[-1] → CompleteObjectLocator → TypeDescriptor → mangled name is attempted; on success the demangled class name is recorded along with a best-effort RTTIClassHierarchyDescriptor walk to fill base_classes_json. On failure (RTTI stripped — common for shipped game binaries) the class is named ANON_Class_<fnv1a-hash> keyed by sorted method-PC list, so identical vtables collapse to one entry. - DB: new tables `vtables`, `methods`, `classes` with indices on function_address and rtti_present. `write_analysis_results` takes a `&[Vtable]` slice; `write_disasm` (back-compat) passes empty. - cmd_dis wires the scan after xref analysis using `func_analysis.functions.keys()` as the function-start oracle. Validation on Sylpheed (RTTI stripped, as expected): 722 vtables / 499 unique classes / 5571 methods. Sanity invariant: every methods.function_address joins to functions.address (0 broken refs). Largest vtable: 131 slots. Tests 617→621 (+4 vtable unit tests covering 3-slot detect, 2-slot reject, synth name stability, and synth name divergence). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
70120465a3 |
M1: parse .pdata RUNTIME_FUNCTION; cross-validate function boundaries
Adds an authoritative function-boundary source from the linker: - New `xenia_xex::pdata` parses .pdata 8-byte entries (BeginAddress + packed prolog/length/flags). Bit layout per Microsoft PE32 PowerPC spec: prolog in bits 0..7, function_length in bits 8..29, flags in 30..31. - `func::analyze_with_pdata` unions pdata BeginAddresses into the candidate set, attaches `pdata_validated`/`pdata_length` to each `FuncInfo`, and trims any function whose `end` overlaps the next start (catches mis-merge where one row spanned two prologues — the audit-031 sub_824D23B0/sub_824D29F0 case). - DB: extends `functions` with `pdata_validated BOOLEAN`, `pdata_length BIGINT`; new table `pdata_entries`; index on pdata_validated. - New `crates/xenia-analysis/SCHEMA.md` documents M1 layer + forward work. Validation on Sylpheed: 25481 functions (was 12156) / 23073 pdata_validated / 0 orphans / 0 mis-merges. Audit-031 mis-merge resolved: sub_824D29F0 now has its own row with `pdata_length=280` (70 dwords); sub_824D23B0 now correctly ends at 0x824D2878 (`pdata_length=1224` matches prologue walk). Tests 605→610. New 5-test pdata unit suite covers bit layout + sentinel + out-of-range filtering + real-world layout round-trip. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |