iterate-4E/native-threads
256 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
MechaCat02
|
869790bab9 |
fix(tools): self-locate DB / cwd instead of the stale renamed path
zq.py and sylph-run.sh hard-coded '/home/fabi/RE - Project Sylpheed/...' (the dashed dir was renamed 'RE Project Sylpheed'), so both were broken. Resolve relative to the script directory now; zq.py honours $SYLPHEED_DB. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
19700736b8 |
[iterate-4E] Stage 1'a: enable host-atomic ReservationTable in native mode
native_active is now folded into parallel_active, so XENIA_NATIVE_THREADS=1 (a) enables kernel.reservations (lwarx/stwcx route through the inter-thread ReservationTable) and (b) sets kernel.parallel_active for the wall-clock vsync/coordination paths. Closes the PPCBUG-108 landmine flagged in the rework pressure-test: native mode runs guest code on multiple host threads concurrently, so the legacy per-PpcContext reservation fallback (which cannot observe cross-thread stores) is incorrect. Previously pure-native (no --parallel) left the table disabled — silently wrong in release, debug_assert in debug. Gate GREEN 3/3 (golden byte-identical, native renders, stress no-deadlock). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
48e166579b |
[iterate-4E] Stage 0: native-threads mode flag + functional gate
Scaffolding for the canary-model rework (host-thread-per-guest-thread, opt-in, non-deterministic). No behavior change with the flag off. - XENIA_NATIVE_THREADS=1 selects the new executor; folded into the parallel spawn gate (reuses Arc<Mutex<KernelState>> + worker dispatch). - native_threads_enabled() = single source of truth for guarded branches. - run_execution_native(): Stage-0 body delegates to the iterate-4D free-run executor verbatim (native == freerun for now); Stage 1' will replace the worker set with one host thread per guest thread. - native-gate.sh: functional oracle replacing byte-goldens for the MT path — (1) lockstep golden byte-identity (flag-off safety net), (2) native render milestone (draws>0 && swaps>0), (3) native deadlock stress. GREEN 3/3 on current code. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
fe797556c2 |
[iterate-4D] multi-core: barrier-less free-run coordinator (in-flight-slot skip)
Attacks the ACTUAL bottleneck the profile found (the ~40%-of-worker-time quiesce barrier — NOT lock contention, which was 4.8%). Removes the per-tick phaser rendezvous entirely: workers free-run continuously; the coordinator just takes the kernel lock each tick (like a 7th participant), runs the same housekeeping, releases, and unparks idle workers. The only reason the barrier existed was `dispatch_graphics_interrupts` borrowing a guest thread's ctx as the ISR victim — which races with a worker that has that ctx EXTRACTED for its unlocked region. Fixed with a per-slot in-flight flag: a worker sets its bit under the kernel lock right after mem::replace-ing its ctx out, clears it under the lock after writeback; the coordinator reads the flags (under the lock → stable snapshot) and passes an in_flight_mask to dispatch_graphics_interrupts, which SKIPS in-flight slots for victim selection. Lockstep + the barrier executor pass mask 0 (skip nothing → byte-identical). Measured (n=2B, JIT): typically ~17s = ~1.4× over lockstep-JIT (~24s), plays the full video (2.0B instrs, 12041 draws / 7440 swaps — the responsive coordinator delivers vsync faster so the guest advances more per instruction). Remaining run-to-run variance (occasional ~65s) is EXTERNAL — the box shows loadavg ~3 and a `powersave` governor with turbo off; it survived every code change (barrier, mmio, sync, tick, barrier-removal) precisely because it isn't the code. Validation: parallel_stress_short 20/20 ok under FREERUN=1; lockstep golden byte-identical (interp + JIT). Opt-in XENIA_PARALLEL_FREERUN=1. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
5a2d4947ea |
[iterate-4D] multi-core: free-run lock-contention profiler + tick=2000 default; REFUTES lock-splitting
Adds XENIA_PARALLEL_PROFILE=1 to the free-run executor: reports, summed across
workers, kernel-lock WAIT (contention), HELD, unlocked REGION, and idle-park
counts. This measured the actual bottleneck before committing to fine-grained
locking — and refuted it:
n=2B free-run profile (worker-thread-time basis):
kernel-lock WAIT = 4.8% <- contention is NOT the wall
kernel-lock HELD = 0.5%
unlocked REGION = 27.1% <- productive
idle_parks = 2.45M ; ~40% = workers blocked at the quiesce barrier
So fine-grained kernel locking would NOT help — the wall is COORDINATION
(the per-tick quiesce barrier + idle-park wake latency) and guest spin-waits.
Also raises the coordinator tick default 200µs -> 2000µs: the barrier is on the
critical path, so the small tick was the dominant cost. This roughly halves the
good-case time (n=2B ~17-18s vs lockstep-JIT ~24s = ~1.35×). BUT free-run is
still BIMODAL — a coordination pathology intermittently latches (~60s). (A/B
tests that looked like "XENIA_GPU_THREAD makes it fast" were run-ordering noise:
use_threaded=true either way — the flag is a no-op here; the split was the
pathology.) Next lever = attack the coordination bimodality (precise cross-slot
wake to kill the ≤50µs idle-park latency; or skip in-flight slots in the ISR
housekeeping to drop the barrier), NOT lock-splitting.
Opt-in (XENIA_PARALLEL_FREERUN=1); lockstep golden byte-identical (interp+JIT).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
MechaCat02
|
0b00b72292 |
[iterate-4D] multi-core Phase C: free-running executor (opt-in) + diagnosis
Removes the per-round phaser barrier that capped the coarse parallel path at ~parity. Workers now FREE-RUN their HW slots (pick slot's thread, run a parallel-safe region on the extracted ctx, writeback+epilogue under the lock, repeat — no waiting on peers); a coordinator thread runs the same housekeeping (coord_pre_round tickers/timers, dispatch_graphics_interrupts, inline-GPU drain, coord_idle_advance) only on a wall-clock cadence, quiescing the workers at a 7-party phaser (via a global `quiesce` AtomicBool, robust vs the earlier epoch-diff which desynced a late-starting worker) so ctx-borrowing housekeeping stays race-free. New: run_execution_parallel_freerun, parallel_region_budget() (default 2048, XENIA_PARALLEL_BUDGET; decoupled from lockstep's 128 so the golden is untouched), scheduler slot_runnable()/any_runnable(), a per-thread `retired=` field in the XENIA_DUMP_SLOTS diagnostic, and dropping the global mmio_access_count region break in the parallel driver (that shared counter, bumped by ANY worker, collapsed every region to one block). STATUS — correct but NOT yet a win. Measured (n=2B, --gpu-inline, JIT): - runs the full 2B and plays the video (4939 draws / 1361 swaps). - BIMODAL: good runs ~21s (edges out lockstep-JIT's ~24s) but a kernel-mutex-contention / guest-spin-wait pathology intermittently latches and makes a run 2-3x slower (~57-87s). Root cause: the single Arc<Mutex<KernelState>> serializes the 6 workers, so the ~4.3x thread-parallelism the workload exposes (measured: work spread across ~5-8 balanced guest threads, top only ~7%) collapses to ~parity. Region-tuning levers (barrier granularity, MMIO break, GPU cadence, sync break) were each measured and none crack it — the real win requires FINE-GRAINED kernel locking. Gated OPT-IN behind XENIA_PARALLEL_FREERUN=1; default --parallel stays the per-round barrier executor. Lockstep untouched (6-config golden byte-identical); parallel_stress_short 20/20 ok under FREERUN=1. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
cce887d69b |
[iterate-4D] multi-core: coarse-grained parallel-safe superblock driver (Phase A+B)
The --parallel worker did a full 7-party phaser barrier + kernel-lock dance around every single ~13-instruction interpreter block, making it 20x SLOWER than lockstep (2.7 vs 55 MIPS at n=400M) and JIT-less. Replace the single-block unlocked window with run_superblock_parallel_unlocked: a whole straight-line region on the extracted ctx + per-worker caches (block + JIT), stopping at the first import/halt/mmio/sync/budget boundary for the locked epilogue to handle. Touches zero KernelState in the lock-free window (thunk band cached once under the lock via new KernelState::thunk_addr_band). Same JIT seam as run_superblock, so XENIA_JIT unset = interp, set = JIT — one driver covers Phase A and B. Measured (--gpu-inline): - recovered the parallel path 16-30x (2.7 -> 43-95 MIPS) - video-phase n=2B parallel-JIT budget=8192: 21.0s vs lockstep-JIT 23.8s (+13%), but violently budget-fragile (4096 = 79.9s) -> the hard per-round barrier + load imbalance caps it well below the 4.3x runnable-width ceiling. Phase C (free-running workers) needed for the real multiplier. Determinism: lockstep path untouched; 6-config n=200M golden byte-identical (incl. config6 JIT+chain+budget=1 == interp+budget=1). xenia-jit 24 tests green; parallel_stress_short 20/20 ok. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
eb7c6f98cd |
[iterate-4C] JIT native FP-arith Phase 2: fused madd family (the 27.7% op)
Native fmadds/fmsubs/fnmadds/fnmsubs + double siblings via hardware FMA (vfmadd213sd/vfmsub213sd — single rounding, matching the interpreter's correctly-rounded f64::mul_add; negated forms flip the result sign with xorpd, and NaN — which the interpreter preserves unnegated — is caught by the finite guard → deopt, so unconditional negate is exact on the fast path). Gated on a runtime cpuid FMA3 check (emit::host_has_fma, cached OnceLock); on a non-FMA host these arms fall back to the interpreter's software FMA — byte-identical, so goldens hold on any host (NEVER emit mulsd+addsd, which double-rounds). fp_arith_matches extended to the madd family (guarded on host FMA3). All 6-config GOLDEN n200m BYTE-IDENTICAL; 24 xenia-jit tests green. MEASURED (2B-instr run through the video, --gpu-inline, best-of-3) — CLEAN 3-way: interp 72.6s (27.5 MIPS) region JIT (pre-FP) 27.3s (73.2 MIPS) = 2.66x over interp region JIT + native FP 23.3s (85.8 MIPS) = 3.12x over interp So native FP's OWN contribution is ~1.17x (+17%) on top of the region JIT — it removed 62% of all interpreter fallbacks (306.6M→117.3M, the FP-arith share) but that translates to only ~17% wall-time here because the region JIT already made non-FP code fast and the --gpu-inline drain + plumbing dominate the remainder. This matches the conservative ~1.3-1.5x plan estimate (region JIT is the bigger CPU lever on the video; FP is a solid, deterministic increment and a multiplier for a future multi-core mode). Video still ~7x from real-time → multi-core next. |
||
|
MechaCat02
|
0672f37c9f |
[iterate-4C] JIT native FP-arith Phase 1: fadds/fsubs/fmuls/fdivs (+doubles)
Emit native x64 for the hot FP-arith ops (currently 61.6% of interpreter fallbacks through the intro video; the JIT emitted ZERO native FP arithmetic before). Phase 1 = add/sub/mul/div, single (op59) + double (op63), Rc=0. Correctness rests on a lemma (validated vs fpscr.rs): if the final result is ±NORMAL or ±ZERO, the interpreter set NO FPSCR exception bit and FPRF=classify(result). So the fast path is: RN-nearest guard (FPSCR RN_MASK=0x3, low 2 bits — NOT 0xC0000000) → native movsd/addsd/subsd/mulsd/divsd in xmm (single narrows via cvtsd2ss;cvtss2sd = to_single at RN-nearest, bit-exact) → result guard (reject inf/NaN; for doubles also reject subnormal → UX+DENORMAL) → store + cheap inline FPRF (sign+iszero → one of ±NORMAL/±ZERO codes). Every guarded (rare) case DEOPTS to jit_interpret_one for that one instruction — counter-neutral (state.retire() defers the +1 on both paths; interp_one bumps no counter and touches only fpr/fpscr/pc, never ctx.gpr, so no RegCache flush). Rc=1 forms fall back (update_cr1 not replicated). FMA (madd) family deferred to Phase 2 (needs hardware FMA3). Infra: Offsets.fpscr; MemHelpers.interpret_one; emit_fp_arith + FpArith enum + emit_fprf_normal_or_zero (emit.rs); DynasmLabelApi import (dynamic labels/relocs in emit.rs). check_fp now SEEDS and ASSERTS fpscr (the real safety net) — added check_fp_seeded. New tests: fp_arith_matches (fuzz + edge floats), fp_arith_directed_rounding_matches (RN!=nearest → deopt), fp_arith_block_matches (multi-op native↔deopt↔fallback boundary). 24 xenia-jit tests green. GOLDEN n200m BYTE-IDENTICAL across all 6 configs (interp / JIT / +CHAIN(region) / +REGCACHE / +REGION_MAX=1 / +budget=1==interp+budget=1). Whole-video speedup measured after Phase 2 (the fmadds family is the single biggest op at 27.7%). |
||
|
MechaCat02
|
3f509ebc5c |
[iterate-4C] --ui: make threaded GPU the default (was opt-in XENIA_UI_GPU_THREAD)
Measured ~5× faster boot under --ui (~12 → ~63 MIPS) by moving the per-frame PM4 drain (draws, YUV texture decode, resolves) + UI publish off the emulation thread onto the GPU worker — canary's async-command-thread model. User-confirmed visuals identical to the inline path (splash + intro video render correctly). Flip the default: `--ui` now uses the threaded GPU backend. Opt back to inline with `--gpu-inline` or `XENIA_UI_GPU_INLINE=1`. The deterministic golden path already passes `--gpu-inline`, so goldens are unaffected (n2m re-checked MATCH). `XENIA_UI_GPU_THREAD` retained as a no-op alias. Headless default (M1.9 threaded) unchanged. Note: threading the GPU only helps GPU-bound phases (boot). The intro VIDEO is CPU-bound on multi-threaded 720p-YUV software decode (FP-arith-heavy, mostly JIT fallback), so it stays ~real-video-length; that needs the multi-core / native-FP levers, not GPU threading. |
||
|
MechaCat02
|
0e24e980fb |
[iterate-4C] M3: default connected controller for user 0 (reach the Press A title loop)
Sylpheed's boot reaches its front-end TITLE shell (steady frame loop — 2219 swaps over 5B instr; XamResetInactivity/XamEnableInactivityProcessing/ XamUserGetSigninState front-end mgmt; XamInputGetCapabilities polled 8880x) but then STALLS: headless has no HID device, so GetCapabilities returns DEVICE_NOT_CONNECTED and the game loops forever waiting for a controller. It never advances to read input (XamInputGetState / XamInputGetKeystrokeEx = 0). Canary reaches the "Press A" menu because its default input drivers (xinput / winkey) present an always-connected controller. Match that: present a default always-connected, IDLE (no-buttons) virtual controller for user 0 (a console always exposes controller slots; a real gamepad under --ui still supplies actual button state — see xam_input_get_state, which prefers ui.snapshot_gamepad()). Set XENIA_NO_PAD=1 to present no controller (the raw headless no-HID trajectory). MEASURED (headless, this is not inference): with the controller connected the game advances into the interactive input-poll loop — XamInputGetKeystrokeEx 0 -> 2164 (once per frame) XamInputGetState 0 -> 2165 XamInputSetState (rumble) 0 -> 1 i.e. the "Press A" title-screen loop (attract video repeats while it polls for A), exactly the M3 milestone behavior. Visual confirmation of the on-screen "Press A" prompt is the remaining step (--ui + frame readback). GOLDEN RE-BASELINE (deliberate): a connected controller changes the guest trajectory WITHIN the 200M window (the title polls input before 200M), so both sylpheed_n200m.json and sylpheed_n2m.json are re-baselined. Change is proven ISOLATED: XENIA_NO_PAD=1 reproduces the OLD goldens byte-identical at both n. New n200m: instructions 200000203, imports 575647, draws 3208, swaps 910 (was 200000239 / 575447 / 3165 / 895 — small front-end-input increments; RTs/ shaders/textures unchanged). All 6 JIT configs match the new golden byte- identical (interp / JIT / JIT+CHAIN / +REGCACHE / REGION_MAX=1 / budget=1== interp+budget=1); 131 xenia-kernel + 21 xenia-jit tests green. |
||
|
MechaCat02
|
f562e3ec2b |
[iterate-4C] JIT Phase B incr.2: region-granularity compilation (the real dispatch win)
Compile a straight-line/same-page REGION rooted at a block into one CompiledBlock (XENIA_JIT_CHAIN): the set of same-page blocks reachable by compile-time-known direct branches (BFS, capped by XENIA_JIT_REGION_MAX=32 + 512-instr budget). Internal transitions between member blocks become plain native jmps to dynasm labels — no per-block prologue/epilogue, no jit_chain_next lookup, no stack churn — while the lockstep yield guards (chain-enabled / budget / mmio) are evaluated inline at each block boundary and a sync-sensitive block ends the region. Any successor not inlined (indirect target, cross-page) falls to the increment-1 boundary chain: jit_chain_next tail-jumps to the next region on a hit, or signals Rust to build+compile it on a miss. Byte-identical schedule to run_superblock: the SAME blocks run in the SAME order with the SAME per-block budget/mmio/sync checks — only the dispatch MECHANISM differs. Inlining is purely an optimization; the boundary chain is the always-correct fallback, and inlined successors are same-page (⇒ mapped, non-thunk) with the LR_HALT sentinel filtered, so next_pc_breaks_chain's runtime arms are statically satisfied. THROUGHPUT (best-of-5, n=200M --gpu-inline): interp 3.76s · plain JIT 3.98s · increment-1 chain 3.97s · REGION 3.49s. Region is ~1.08-1.14x FASTER than the interpreter (first clear JIT win on the boot bench) and ~1.14x faster than increment-1's throughput-neutral block chaining — confirming the thesis that the real dispatch cost was the per-block prologue/epilogue increment 1 kept. Region-size sweep: gains saturate by max=32 (max=64 identical). GOLDEN n200m BYTE-IDENTICAL across 6 configs: interp / JIT / JIT+CHAIN(region) / JIT+CHAIN+REGCACHE / JIT+CHAIN+REGION_MAX=1 / JIT+CHAIN+budget=1 == interp+budget=1. 21 xenia-jit tests (added region_multiblock_matches_interp_ superblock + region_sync_block_ends_superblock). Bug found+fixed during bring-up: a region chained PAST a sync-sensitive block via the boundary chain instead of ending the superblock (compile_block gives sync blocks the plain epilogue; the region must too) — this diverged the whole 200M schedule (draws 3201 vs 3165). Fix: sync block tail = unconditional jmp l_region_ret; BFS never expands past a sync block. The sync regression test asserts the region stops after A+B (cycle==4) instead of looping to budget. New: xenia_cpu::block_cache::decode_block (pub standalone decoder for the region builder) + DecodedBlock: Clone. Diagnostic knobs XENIA_JIT_REGION_MAX (cap, default 32; =1 reproduces increment-1 chaining) and XENIA_JIT_NOREGION (route chain compiles through compile_block — the A/B knob that isolated the sync bug). Default XENIA_JIT/XENIA_JIT_CHAIN both still OFF; shipping path unchanged. |
||
|
MechaCat02
|
d132fb6d8d |
[iterate-4C] JIT Phase B incr.1: native tail-chaining (XENIA_JIT_CHAIN, default off)
Compiled blocks tail-jump straight to their successor's compiled code
(canary-style indirection dispatch) instead of returning to the Rust
superblock loop between blocks, checking the lockstep yield guards
INLINE. Gated behind XENIA_JIT_CHAIN (default off) — the shipping JIT
path (Phase A) is byte-for-byte unchanged.
Mechanism:
- JitEnv gains chain_{enabled,stop,deadline,mmio_ptr,mmio_before,cache}.
- Chaining epilogue (non-sync blocks only, when chain_active): flush cache
to ctx, then inline guards — budget (ctx.cycle_count >= start+budget,
== interp total_executed>=budget), mmio (per-block prologue snapshot vs
live counter), chain_enabled gate — then jit_chain_next(env, next_pc)
(pure JIT-cache freshness lookup, returns host entry or null). On a hit:
restore env->rdi, unwind THIS frame (pop to entry rsp), tail-jmp to the
successor (no stack growth; the return addr rides through to the final
block's ret). On null: set chain_stop=Miss, return to Rust.
- run_jit_chain drives it; run_superblock_jit_chained (used when chaining
compiled in + budget>1 + no probe/mem-watch armed) re-enters on a Miss to
build+compile the chainable next block. sync blocks keep the plain
epilogue (end the superblock like interp). GuestMemory::
mmio_access_count_ptr exposes the counter for the inline check.
BYTE-IDENTICAL golden n200m across: interp, JIT, JIT+REGCACHE, JIT+CHAIN,
JIT+CHAIN+REGCACHE, and JIT+CHAIN+budget=1 == interp+budget=1. 19 jit
tests green. (Fixed: prologue mmio snapshot must be chain_enabled-guarded
— a chaining-compiled block is also run via run_jit_block with a null
mmio_ptr on the Phase A / budget==1 / probes-armed paths.)
THROUGHPUT: NEUTRAL on the boot bench (JIT vs JIT+CHAIN statistically
equal, within noise). As the canary study predicted: block-granularity
chaining keeps the per-block prologue/epilogue (unwind+rewind per
boundary) and jit_chain_next ~= the run_fresh it replaces, so the
reclaimed Rust-loop overhead is Amdahl-swamped. The determinism-critical
machinery (inline yield guards, tail-jmp discipline, dispatch ABI) is now
PROVEN byte-identical — the foundation for region/function-granularity
compilation (increment 2), which is where the real dispatch win lives.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
MechaCat02
|
fec9e8de28 |
[iterate-4C] JIT Phase A: skip redundant block_cache lookup on chained hits
The JIT was ~1.077x SLOWER than the interpreter on the boot bench because it paid a DOUBLE lookup per block: block_cache.lookup_or_build (to get the DecodedBlock — the ~9.7% "block decode/cache" bucket) THEN JitCache's own lookup. On a JIT-cache HIT the DecodedBlock is not needed: the freshness key is (start_pc, mem.page_version(pc)), reconstructible from mem alone (build_block stops at the 4 KiB page boundary, so both caches key on the same single-page version). Phase A adds a JIT-specialized superblock runner that runs chained (2nd..Nth) blocks straight from the JIT cache, skipping block_cache on hits: - CompiledBlock now carries sync_sensitive (copied from DecodedBlock) — the chain STOP guard needs it and a JIT hit has no DecodedBlock. - JitCache::run_fresh(pc, ctx, mem): lookup-only fast path; computes pv itself via mem.page_version (SMC coherence); Some((result, sync)) on a fresh hit, None on a miss (never compiles — compilation stays on the DecodedBlock path). New unit test run_fresh_hit_miss. - run_superblock_jit (parallel to run_superblock, used when jit_cache is Some): first block + JIT misses use block_cache.lookup_or_build + run_or_compile (rebuild inline on miss so chain length — and the schedule — is unchanged); chained hits use run_fresh. Non-Continue break lazily rebuilds a block_ptr for worker_epilogue's SYSCALL/Trap diagnostics. Same raw-ctx-ptr discipline; shared next_pc_breaks_chain helper keeps both loops' chaining decisions in lockstep. Interp run_superblock untouched except the extracted helper call. Results (n=200M --gpu-inline): block_cache calls 11.4M -> 991k (-91% on the JIT run — run_fresh handles 91% of block acquisitions). Throughput: JIT 1.077x slower -> ~1.03x FASTER than interp (best-of-8 interleaved: JIT 3.73s vs interp 3.84s) — first time the JIT beats the interpreter. Golden n200m BYTE-IDENTICAL: interp==golden (path untouched), JIT==golden, JIT+REGCACHE==golden, and JIT budget=1 == interp budget=1. 19 jit tests green. Phase B (native inline chaining, targets the 23.6% loop body) deferred — see plan. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
cce35658a5 |
[iterate-4C] JIT: native update-form loads/stores (lwzu/stwu/ldu/stdu/...)
Port the D/DS-form update loads (lwzu, lhzu, lhau, lbzu, ldu) and stores (stwu, sthu, stbu, stdu) — ~10% of the boot fallback executions (stwu 6.1% + lwzu 2.4% + ldu/stdu 1.4% + ...). EA = gpr[rA] + EXTS(disp) with rA read directly (update forms are invalid for rA==0, and interp reads gpr[rA] with no 0-substitution); rD/mem then gpr[rA]=ea truncated to u32 and zero-extended (matches interp `... as u32; gpr[ra]=ea as u64`). Stores capture rS into rdx BEFORE the rA writeback so `stXu rS,d(rA)` with rS==rA stores the OLD rA value (interp order). New emit helpers emit_update_ea_writeback / emit_load_update / emit_load64_update / emit_store_update; all cache-aware (gld64/gst64 accessors). New differential test update_loadstore_matches (regcache forced on), incl. the store rS==rA case. 18 tests green. Fallback executions 25.55M -> 22.97M (-10.1%, n=200M). Golden n200m BYTE-IDENTICAL under interp, XENIA_JIT=1, and +XENIA_JIT_REGCACHE=1. Whole-run ratio 1.079x -> 1.077x: as prior evidence predicted, opcode coverage does not flip the JIT on this FP-video-decode-bound boot bench (62% of remaining fallbacks are fmadds/fmuls/fadds/fsubs, expensive in both paths). Correctness/coverage win; real speed lever is dispatch elimination (block-linking), not more opcodes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
7ec6941fe8 |
[iterate-4C] perf: cache running-thread ctx ptr across superblock chain
run_superblock re-resolved the running thread's PpcContext on every chained block via two bounds-checked slot lookups (ctx_mut_ref for the step + ctx(hw_id) for next_pc) — ~20M double-lookups per 100M-instr window. The running thread is FIXED for the whole chain (an import thunk or any sync-sensitive/MMIO op breaks the chain before any kernel mutation could restructure the runqueue; step_block is pure guest interpretation; the probe closure is read-only), so its context heap slot is stable. Resolve the raw *mut PpcContext once before the loop and reuse it — same raw-pointer discipline as block_ptr. ~3% faster clean wall (1.91s -> 1.85s, n=100M, best of 5). Golden sylpheed_n200m BYTE-IDENTICAL under interp, XENIA_JIT=1, and XENIA_JIT=1+XENIA_JIT_REGCACHE=1. Found via exclusive-attribution profiling: the run is overhead-bound with the non-step cost spread thin across the per-block chaining loop (no single dominant lever); step_block itself is ~46% and only the JIT attacks it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
1ffa3fb56d |
[iterate-4C] JIT Phase 4c: per-block register caching (XENIA_JIT_REGCACHE)
Pin the 3 hottest guest GPRs (static ra/rb/rd frequency) into callee-saved host regs r12/r13/r14 across a block. RegCache + gld64/gld32/gst64 accessors (dynasm Rq/Rd dynamic regs); ALL 75 GPR-access sites in the emitters routed through the accessors. Prologue loads cached regs / epilogue flushes (both l_cont+l_exit merge); fallback path flushes-before / reloads-after (interpreter touches ctx.gpr; mem/FP helpers don't, and cached regs are callee-saved, so no flush around those). RA=0 literal-0 rule unaffected (emit_ea/load_ra_or_zero never read gpr[0]). Gated by XENIA_JIT_REGCACHE (default off; forced on under cfg(test)); conditional prologue keeps the cache-off codegen unchanged. Golden n200m BYTE-IDENTICAL in all 3 configs (interp / JIT / JIT+RC); 17 differential tests pass with the cache FORCED ON. Measured (min of 6): JIT+RC 4.16s vs JIT 4.14s vs interp 3.80s -> NEUTRAL on this benchmark: per-block setup (5 push/pop + 3 load/flush) + fallback flush/reload (+6 mem ops each) cancels the intra-block savings (blocks ~13 instrs, fallbacks frequent). Correct + default-off; may help more CPU-bound workloads (gameplay, larger hot loops). Definitively: no JIT technique (coverage, deferral, regcache) beats interp on this ~60%-overhead boot/render bench. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
506b9554a5 |
[iterate-4C] JIT Phase 4b: indexed loads/stores + shifts + mflr/mtlr
Native lwzx/lhzx/lhax/lbzx/stwx/sthx/stbx (via refactored emit_load_tail/ emit_store_tail + emit_ea_x), slw/srw/sld/srd (variable shifts, non-rc, cmovae zeroes for count>=width), and mfspr/mtspr for the pure LR(8)/CTR(9) registers only (other SPRs have side effects -> fallback). Differential tests indexed_loadstore_matches / shifts_match (sh spanning width boundaries) / spr_lr_ctr_matches. Golden n200m BYTE-IDENTICAL with XENIA_JIT (17 tests green). Measured: fallback execs 13.5M->10.8M (64% below the original 29.7M). BUT throughput ratio only 1.10->1.09 -> DEFINITIVE: on this boot/render benchmark opcode coverage alone can't beat interp. Cutting fallbacks 64% moved the ratio 7 points because the run is ~60% non-CPU-step overhead (Amdahl) AND the remaining fallback is FP-arith (fmadds/fmuls/fadds ~57%), expensive in BOTH paths (small wrapper delta) + FPSCR determinism risk. Below-1.0 needs register caching (attacks native-op cost directly) or a more CPU-bound workload. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
4194ed77a3 |
[iterate-4C] JIT Phase 4a: native FP + 64-bit loads/stores
Native lfs/lfsx/lfd/lfdx/stfs/stfsx/stfd/stfdx and ld/ldx. FP goes through dedicated helpers (jit_read_f32_as_f64/read_f64/store_f32/store_f64) that call the EXACT interpreter mem methods + as-casts, so the f32<->f64 conversion is bit-identical by construction (no hand-written cvtss2sd / NaN-payload risk); FP stores replicate the reservation-invalidation. emit_ea_x (indexed EA), emit_fp_load/store, emit_int_load64 helpers; fpr offset added. Differential test fp_loadstore_matches (2000 seeds, FPR edge patterns 0/-0/inf/NaN/denormal/rounding, asserts fpr-by-bits + mem). Golden n200m BYTE-IDENTICAL with XENIA_JIT (14 tests green). Measured (XENIA_JIT_STATS): total fallback execs 29.7M->13.5M (-55%); throughput ratio vs interp 1.16->1.10. Remaining fallback now FP-arith- dominated: fmadds 20% + fmuls 16% + fadds 8% (~44%). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
afc3692223 |
[iterate-4C] JIT: gated fallback-opcode histogram (XENIA_JIT_STATS)
jit_interpret_one tallies fallback executions per opcode (behind XENIA_JIT_STATS, off by default); dump_fallback_stats() prints the top 30 after a check run. Measured on Sylpheed n100M: the fallback tax is FP- dominated — lfs/stfs/lfsx/stfsx ~50%, fmadds/fmuls/fadds/fmr ~21%, ld 4.5%, shifts ~4%, mtspr/mfspr only ~3%. Refutes the earlier "mflr/mtlr dominates" guess; FP is the crossover lever. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
e8d0dc4a2d |
[iterate-4C] JIT Phase 3b: defer pc/counter accounting per block
Strip the 3 per-instruction memory RMWs (pc+=4, cycle++, timebase++) from straight-line native ops. EmitState.pending accumulates retired native instrs at compile time; counters are materialized in bulk (add [cycle],N) only at observability points, and pc is written absolutely (from the known addr) only where read. Invariant: at every block-exit edge ctx.pc/ cycle_count/timebase are exactly the interpreter's values (nothing observes them mid-block; fallbacks flush+set-pc first; native branches set pc absolutely from addr and flush). Branches rewritten to use compile-time addr (no more [pc] reads). New multi_instr_block_matches test diffs a JIT block vs the real step_block (native+fallback+branch mix). Golden n200m BYTE-IDENTICAL with and without XENIA_JIT (13 tests green). Throughput 4.55->4.44s; ratio vs interp still ~1.16 -> gap is fallback tax (mflr/mtlr, indexed/update ld-st, shifts still fallback) + the run being only ~40% CPU-step, NOT the native-path RMWs. Deferral is also the flush-discipline substrate for register caching. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
781a82cc0a |
[iterate-4C] JIT Phase 3a: native rotate/mask family
Native rlwinm/rlwimi/rlwnm (32-bit rol+and, mask folded to a compile-time
constant) and rldicl/rldicr (64-bit rol + 64-bit mask), including the Rc
recording forms via emit_cr0_from_reg (test + signed setcc into cr[0],
reusing the compare CR path). rlw_mask/rld_mask replicated as
compile-time helpers. Differential test rotates_match: 2000 seeds x
{rc,non-rc} x {32,64-bit} x xer_so, asserting full GPR+CR+counters.
Golden n200m BYTE-IDENTICAL with and without XENIA_JIT (12 tests green).
Throughput 4.7->4.55s (interp ~3.9s): opcode coverage now at diminishing
returns; the remaining gap is memory-traffic-per-instruction (no register
caching) + fallback tax, not missing opcodes -> Phase 4 (register cache)
is the crossover lever.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
MechaCat02
|
701e4c399a |
[iterate-4C] JIT Phase 2b-ii: native branches (bx/bcx/bclrx)
try_emit_native now returns a 3-state Emit enum (Fallback/Native/Branch); compile_block appends the pc==expected_next discontinuity check after a native branch, mirroring step_block. bx (aa/rel, lk), bcx (CTR decrement + CTR/CR condition, cmov-selected target), bclrx (return via lr & !3) emitted natively; bcctrx stays on fallback (dispatch_rec side effect). count_only postlude (counters, no pc+=4) for branches. Differential tests: unconditional/conditional/return branches over exhaustive BO bits, CTR==1->0 boundary, lr alignment mask, both lk states, asserting pc/lr/ctr/cr/counters. Golden n200m BYTE-IDENTICAL with and without XENIA_JIT (11 tests green). Throughput 4.85->4.7s (interp ~3.9s); crossover still pending rotate/shift (Phase 3). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
5e521f7f53 |
[iterate-4C] JIT Phase 2b-i: native compares + CR field emission
Native cmpi/cmpli/cmp/cmpl via x64 cmp + setcc into cr[bf].{lt,gt,eq},
so from xer_so. CrField flag offsets resolved with offset_of! (the type
is not repr(C), so byte positions are not assumed). Both 32/64-bit (L)
widths handled. Differential test compares_match (2000 seeds x both
widths x both xer_so values) asserts full CR nibble + counters; check()
now also asserts CR for all opcodes. Golden n200m BYTE-IDENTICAL with
and without XENIA_JIT.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
MechaCat02
|
0f1130e2e6 |
[iterate-4C] JIT Phase 2a: native loads/stores + backed-memory diff tests
emit.rs: native x64 for the hot D/DS-form memory ops — lbz, lhz, lha, lwz, stb, sth, stw, std. EA = (rA==0?0:gpr[rA]) + EXTS(disp) computed inline; the access itself calls a focused extern "C" helper (jit_read_u*/ jit_store_u*) that reconstructs &dyn MemoryAccess (and &PpcContext for stores) from JitEnv and calls the SAME big-endian trait methods the interpreter uses -> MMIO routing / mem-watch / page-version bumps identical. Stores replicate the interpreter arms' reservation-invalidation prologue exactly (no-op without a reservation table). Load extension (zx8/zx16/ sx16/zx32) via movzx/movsx, matching each arm. tests.rs: VecMem (backed big-endian mock) + loads_match/stores_match diff tests (2000 seeds each) asserting GPR + full memory image + counters vs the interpreter, incl. RA=0 and byte-swap. Gate: golden n200m BYTE-IDENTICAL with XENIA_JIT=1; cargo test -p xenia-jit green (9 tests). Throughput -n 200M --gpu-inline: 3.6s interp, 4.85s JIT (down from 6.2s at Phase 1). Remaining fallback = branches (every block terminator) + compares -> the crossover. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
231c35a28f |
[iterate-4C] JIT Phase 1: native integer ALU emitters + differential tests
crates/xenia-jit/src/emit.rs: native x64 for the hot non-recording integer ALU ops — addi, addis, ori, oris, xori, xoris, or(x), and(x), xor(x), add(x), subf(x), neg(x). Each mirrors its interpreter arm exactly, then does pc+=4 and the cycle/timebase bumps. Per-instance guards fall back to the interpreter for forms the emitter can't yet reproduce faithfully: recording (`.`/Rc) forms, OE overflow forms, and the db16cyc spin hint (or r31,r31,r31 -> Yield). RA=0 literal-zero rule handled at emit time. Differential harness (tests.rs): each opcode run through interpret_one vs a JIT-compiled 1-instr block over 2000 random register seeds (+ edge values 0/0xFFFFFFFF/0x80000000/INT64 boundaries); asserts full GPR/PC/XER/cycle/ timebase/StepResult equality. Plus a guard test that recording/OE/db16cyc forms are NOT natively emitted. Gate: golden n200m BYTE-IDENTICAL with XENIA_JIT=1; `cargo test -p xenia-jit` green. Throughput -n 200M --gpu-inline: 4.3s interp, 6.2s JIT (down from 7.1s all-fallback skeleton). Still fallback-bound on loads/stores/branches -> Phase 2 is the crossover. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
1d56218d83 |
[iterate-4C] JIT Phase 0: dynasm skeleton (all-fallback, golden byte-identical)
New crate `xenia-jit` — the PPC->x64 block-JIT runtime substrate, behind
default-OFF XENIA_JIT. Phase 0 ports ZERO opcodes to native: every guest
instruction is emitted as `call jit_interpret_one` (the interpreter), so a
JIT-compiled block is byte-identical to step_block by construction. This
proves the ABI/counters/exit-semantics/mem-helpers/code-cache before any
opcode is hand-written.
- JitEnv{ctx, mem (fat raw ptr), last_result}; compiled block =
extern "C" fn(*mut JitEnv)->u32 (StepResult discriminant, 0=Continue).
Emitted code pins ctx in r15 + env in rbx (callee-saved across calls),
offsets via offset_of!.
- Determinism postlude: cycle_count/timebase +=1 after every retired
instruction; block stops at the same instruction as the interpreter
(non-Continue result, or taken-branch pc discontinuity).
- Per-slot JitCache mirrors BlockCache's (start_pc, page_version) gate;
each CompiledBlock OWNS a copy of its decoded instrs so baked instr
pointers can't dangle after a block-cache eviction.
- xenia-cpu: `pub fn interpret_one` (execute without the cycle bump — the
JIT owns counting).
- Seam: run_superblock main.rs:3154 dispatches to the JIT when enabled;
WorkerCtx gains an Option<JitCache> (Some only when XENIA_JIT set and
RET-CAPTURE debug env unset). observe_per_instruction gate unchanged, so
tooling runs never reach the JIT.
Gate: golden n200m BYTE-IDENTICAL both with and without XENIA_JIT=1.
Throughput -n 200M --gpu-inline: 4.5s interp vs 7.1s all-fallback skeleton
(the per-instruction call overhead Phase 1 removes for hot opcodes).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
MechaCat02
|
77c2d7bce9 |
[iterate-4B] spike: multi-core feasibility measurement (concurrency probe)
Throwaway diagnostic for the multi-core-vs-JIT decision. Two real numbers: 1. Existing --parallel is ~25x SLOWER, not faster: 100.9s wall vs 4.0s lockstep at -n 200M --gpu-inline (237s user across 6 threads burned on coarse-mutex + phaser contention). Naive coarse-locking inverts the win. 2. XENIA_CONCURRENCY_PROBE (env-gated, zero cost off): per-round histogram of runnable HW-slot width. Sylpheed at -n 300M: AVG WIDTH = 3.83 (width>=4 in 77% of rounds) => a *perfect* host-thread-per-guest-thread design has a ~3.8x Amdahl ceiling. The parallelism is real; the existing vehicle just can't capture it. Golden n200m byte-identical (probe inert unless env set). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
9851873e42 |
[iterate-4B] perf: quick hot-path wins (bulk quantum-decrement + ctx coalesce)
Two byte-identical interpreter-dispatch optimizations (headless golden
n200m unchanged). Together ~13% faster on the -n 200M --gpu-inline
benchmark (4.33s -> ~3.8s, ~46 -> ~53 MIPS).
- scheduler.rs: `decrement_quantum_by(n)` — the superblock epilogue looped
`for _ in 0..executed { decrement_quantum() }` (~one bounds-checked call
per retired guest instruction, the largest fixed per-superblock cost).
QUANTUM_DEFAULT (50k) >> a superblock's instr count, so the quantum
boundary is crossed at most once/call: common path is one subtraction,
the rare boundary step defers to decrement_quantum for exact rotation
(reload + same-priority peer hand-off) semantics. Byte-identical.
- main.rs run_superblock: resolve the running thread's PpcContext ONCE per
block (was three `ctx_mut_ref` double-indexed slot lookups: cycle-before,
the step, cycle-after).
Profiled remaining breakdown (-n 300M --gpu-inline): step_block body 50%
(67 MIPS, the JIT target), block decode/cache 9% (page-version-bound),
kernel HLE 8%, scheduler/lock remainder ~31%. Sub-10% cheap wins remain
(block-linking is page-version-gated so only partial); the 5-10x lift is
the JIT.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
MechaCat02
|
913b566a26 |
[iterate-4B] A.5: threaded GPU under --ui (opt-in XENIA_UI_GPU_THREAD)
Moves the per-frame GPU work (PM4 drain: draws, YUV texture decodes, resolves — ~12 ms/swap) AND the per-swap UI publish off the emulation thread onto the GPU worker thread when --ui is combined with XENIA_UI_GPU_THREAD=1. This is the fix for the dominant remaining --ui cost identified after A.1 (the inline drain at exports.rs:3145). Default OFF: plain --ui still forces the inline backend (the safe, milestone-2-verified path), so this cannot regress the shipped result. - xenia-gpu/handle.rs: new UiPublishHooks + WorkerSwapInfo (pure xenia-gpu types so the worker needs no kernel dep); GpuCommand::InstallUiHooks; GpuWorker gains ui_hooks + last_published_swaps; the worker runs GpuSystem::run_ui_publish (blobs/constants/texture/geometry + bulk frontbuffer detile + notify) level-triggered on stats.swaps_seen. - xenia-kernel/exports.rs: vd_swap returns immediately in threaded+ui mode (no blocking drain, no emulation-thread publish); the worker discovers the in-stream PM4_XE_SWAP and publishes. Inline/headless paths untouched. - xenia-app/main.rs: XENIA_UI_GPU_THREAD knob; run_with_ui takes Arc<GuestMemory> (removes the old --ui+gpu-thread try_unwrap panic); builds the publish hooks from the live UiBridge and installs them on the worker; enables frame capture on the worker's GpuSystem before spawn. Headless golden sylpheed_n200m byte-identical (inline path unchanged); threaded headless smoke clean. Threaded --ui visual correctness pending human verification (no self-screenshots). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
79d0026a31 |
[iterate-4B] --ui perf: bulk frontbuffer detile + present knob + shader/blob caches
Windowed (--ui) mode forces inline GPU, so VdSwap's per-swap UI publish runs on the emulation thread — profiled at ~87% of it (~4 MIPS effective vs ~35 headless). This lands the low-risk, headless-untouched wins (golden n200m byte-identical): - A.1 (biggest): VdSwap frontbuffer detile now uses one bounded GuestMemory ::read_bulk instead of ~3.7MB of per-byte read_u8 through the MMIO handler (~15 ms/swap). Bounds-checked to stay in the committed backing window. - A.4: XENIA_PRESENT_MODE (immediate|mailbox|fifo) + XENIA_FRAME_LATENCY knobs (render.rs); default (Mailbox-else-Fifo, latency 2) unchanged. - A.3a: cache parse_shader/pack_for_wgsl per blob key on RenderState instead of re-parsing every draw every frame (blobs are immutable) — the movie-relevant UI-thread win. - A.2: publish shader-blob map to the UI only when it changed (shader_blobs_version on GpuSystem; publish_xenos_assets blobs arg is now Option, None = keep previous). Constants still published every swap. Deferred (profiling-justified — target the measured bottlenecks, not these): - A.2 texture-gate, A.3b bind-group cache: zero benefit for the movie (its texture keys rotate every frame → always-miss) + staleness/leak risk. - A.3c submit-batching: the UI thread's bottleneck is the vsync-blocked present, not per-draw submits; GPUBUG-111 regression risk not justified. - A.5 (threaded GPU under --ui): the structural win; separate follow-up (needs the publish bridge moved to the worker + human visual verification). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
cc9ebbb5e7 |
[iterate-4A] docs: handoff for intro-video-done milestone + speed frontier
HANDOFF-intro-video-done.md: what to push/copy, the 6 branch commits, build/run, root-cause summary, and the profiled emulator-speed frontier with ranked levers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
2c883e9d5e |
[iterate-4A] diagnostics: XENIA_PROFILE wall-time profiler + probe/tooling snapshot
Handoff snapshot of the env-gated diagnostic scaffolding used across the intro-video RE. Kept out of the milestone commits (645feb8..5573ac1) to keep those clean; committed here so nothing is lost on handoff. New — XENIA_PROFILE wall-time profiler (crates/xenia-gpu/src/prof.rs): Coarse buckets attributing playback wall time to interpreter (step_block), kernel HLE (call_export), block decode/cache (lookup_or_build), texture decode, host draw, and present; prints periodic snapshots (every 500M guest instr, or every 500 presents) + a clean-exit report. Hot path is gated on a cached is_on() (one relaxed load) so it is zero-cost when XENIA_PROFILE is unset. Call sites: main.rs run_superblock / parallel worker (step_block, lookup_or_build, call_export), texture_cache ensure_cached, render.rs present + dispatch_xenos_draws. First profile (movie playback, headless single-thread lockstep): effective ~35 MIPS; interpreter body ~40% @ ~95-102 MIPS; texture decode 0.3% (cache works); present ~0%; the rest is per-block dispatch + scheduler plumbing (~13 instr/block over 229M blocks). Overhead-bound, not interpreter-body bound; the levers are coarser execution units (superblock chaining) and ultimately a JIT. Pre-existing read-only probe knobs (were uncommitted; env-gated, observe-only): XENIA_RET_CAPTURE_PC/_REG/_MEM, LOG_RESUMES, LOG_WAITS, LOG_SIGNAL, FORCE_TID, STARVE_LIMIT, INCUMBENT_PICK, INSTR_PER_MS, DUMP_FRAME, DUMP_WGSL, BIND_LOG, CONST_LOG, DISPATCH_REC, AUDIT_PC_TRACE. Tooling: sylph-run.sh (movie oracle loop, 180s default timeout), zq.py (DuckDB disasm/xref helper). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
5573ac1c43 |
[iterate-4A] intro-video: correct MULSC/ADDSC/SUBSC operand addressing
The scalar-constant ALU ops (MULSC/ADDSC/SUBSC, opcodes 42-47) address their operands through src3 in a special way, not via src_a/src_b like ordinary scalar ops: temp reg = (src3_swiz & 0x3C) | (scalar_opc & 1), component = src3_swiz & 3 const idx = src3_reg (+256 for the PS constant bank), component = .w The op is then temp <op> const. We were feeding these the plain (src_a.x, src_b.x). For the movie's YUV->RGB luma scale that made the single MULSC0 compute r0.x * r0.x = Yb^2 (a squared luma) instead of r0.x * alu[511].w = Yb * 1.1643 (the limited- range 255/219 luma scale). The squared luma crushed shadows, so dark regions rendered as near-pure chroma -- the intro's dark background read purple/magenta while the bright logo looked roughly right. With correct addressing the luma is linear: the background drops to near black (was a purple ~(37,15,39), now ~(12,0,14)) and the logo reaches full white -- the authentic SQUARE ENIX intro. (The earlier attempt at this addressing failed only because the PS constant bank was still mis-indexed before the +256 fix, so the coefficients read zero -> black.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>intro-video-done |
||
|
MechaCat02
|
deb9292395 |
[iterate-4A] intro-video: complete RectangleList quad (fix diagonal seam)
A Xenos RectangleList gives 3 corners of a rectangle; the GPU synthesizes the 4th (v3 = v0 + v2 - v1) and tessellates the quad as two triangles (v0,v1,v2) + (v0,v2,v3). Our expansion emitted only the front triangle, so the movie's fullscreen YUV rect filled half the screen and left a diagonal seam down the frame (absent in canary). expand_rectangles now emits the full 6-index quad. Since v3 has no backing vertex in the guest window, the translated VS synthesizes its attributes: the RectangleList arm maps 6 host verts through [0,1,2, 0,2,3], flags the 4th corner, and emit_vfetch computes that corner's attributes (position and every interpolator) as v0 + v2 - v1 by reading the three real source vertices. Non-rect draws are unaffected (the synth flag is false). Verified: the movie background now fills the whole frame uniformly, no diagonal, matching canary. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
3559c8f6ef |
[iterate-4A] intro-video ROOT #3: render the YUV movie in correct color
The intro video (ADV.wmv) now plays end-to-end in correct color. Three stacked host-render-path bugs, each masked by the prior: #3a Multi-texture render path. The host bound a single texture slot, so the YUV pixel shader's three plane fetches (Y 1280x720 + U/V 640x360, all k_8) collapsed onto one texture. Expanded the Xenos pipeline to 8 tex+1 sampler slots (xenos_pipeline.rs, xenos_interp.wgsl, translator.rs headers); each tfetch selects its texture by fetch-constant slot; the DrawCapture textures tuple now carries the slot; render.rs uploads+binds every plane per-draw. Also added the scalar-constant ALU ops MULSC/ADDSC/ SUBSC (42-47) the YUV->RGB shader uses. #3b tfetch destination swizzle. decode_fetch read the tfetch dest as a 4-bit write mask (w1 & 0xF), but Xenos tfetch dword1[0:11] is a 12-bit destination swizzle (3 bits/component: 0-3=xyzw, 4/5=const 0/1, 6/7=keep). The result: all three plane fetches did a full-vec4 overwrite of the dest register, so only the last plane survived. Decode the real 12-bit swizzle (dest_swizzle) and emit per-lane writes so Y/U/V coexist in r1.x/.y/.z. #3c Pixel-shader constant bank. Xenos splits the 512-entry float-constant file: the vertex shader addresses c0..255 -> physical 0..255, but the pixel shader's c0..255 map to physical 256..511. The game uploads the YUV->RGB coefficients to physical 510/511. Our translator indexed the low half for PS constants, reading all-zero -> R=B=Y^2, G=0 (magenta). emit_alu now adds a const_base of 256 for pixel-stage constant reads. Plus a bounded (FIFO, 64-entry) host texture cache: the movie streams ~3 new-VA planes per frame, and the previously-unbounded cache exhausted GPU memory into a device-lost crash mid-playback. Verified visually: the SQUARE ENIX logo and ADV.wmv footage render in correct color (was magenta); the translated movie shader now reads alu[510]/alu[511]; frame green channel is nonzero and R != B. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
43441523f9 |
[iterate-4A] intro-video: re-baseline boot golden 50M -> 200M after clock fix
The clock fix (INSTRUCTIONS_PER_MS 10_000 -> 1_000_000, commit
|
||
|
MechaCat02
|
645feb8f5b |
[iterate-4A] intro-video: fix decode-timeout clock, feeder starvation, k_8 texture decode
Three layered root causes kept ADV.wmv from playing. This lands the first three fixes: the intro video now decodes end-to-end and uploads its YUV planes. The on-screen composite still needs a multi-texture render path (root #3, tracked separately — the shader interpreter binds one texture slot but the YUV->RGB pass samples three). 1. Clock scale (xenia-kernel/state.rs): INSTRUCTIONS_PER_MS 10_000 -> 1_000_000. KeTimeStampBundle tick_count = global_clock / INSTRUCTIONS_PER_MS. At 10_000 (~10 MIPS; global_clock further inflated ~58x by the serialized scheduler summing busy-spin) the movie handler's 2000 ms software-decode deadline (sub_821B4968 @0x821b68c0) tripped on a legitimate ~20M-instruction 720p-YUV420 decode and aborted playback (canary never enters that wait loop). 1_000_000 sits in the validated [~600k, ~2.77M] window that fits both the movie (decode <=2000 ms) and boot (the worker-hub +66 ms gate still elapses before the movie, ~66M instr). XENIA_INSTR_PER_MS overrides. 2. Scheduler fairness (xenia-cpu/scheduler.rs): pick_runnable's equal-priority tiebreak now prefers the incumbent (running_idx) so decrement_quantum's quantum rotation sticks. Previously each round re-picked the lowest index, so co-located equal-priority threads never alternated and the movie's demux feeder (tid24, co-located on hw=1) starved until the STARVE_LIMIT=4096 backstop -- the decode ring never refilled. Priority preemption unaffected. XENIA_INCUMBENT_PICK=0 rolls back. 3. k_8 texture decode (xenia-gpu/texture_cache.rs + xenia-ui/texture_cache_host.rs): the video uploads its YUV420 planes as linear k_8 textures (Y 1280x720, U/V 640x360); with no k_8 decoder ensure_cached rejected them and the frames never reached the GPU. Adds decode_k8 (1 byte/texel expanded to Rgba8Unorm) + the host Rgba8Unorm mapping. Read-only diagnostic probe knobs used to find these remain uncommitted in the working tree. Boot goldens re-baselined in a follow-up commit (the clock change intentionally moves the digest; see tests/golden/README.md). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
3e17d37b4e |
[iterate-4A] intro-video RE: XENIA_AUDIT_PC_TRACE (per-tid PC-range r3/r4/r5/r30/r31 trace)
Observe-only diagnostic (lockstep digest unaffected) added during the milestone-2 video investigation. `XENIA_AUDIT_PC_TRACE=lo:hi:tid` logs AUDIT-TRACE r3/r4/r5/r30/r31 for blocks in [lo,hi) on a given tid — used to capture the demux registrar's stream-id (r4) at sub_82509E40. Handoff (full detail in memory cont.10ff-10ii): - Video decode is GUEST SOFTWARE (decoder vt 0x82009f70, 615 fns/30k instr, zero host/video imports) — no host codec to build; fix is upstream. - Video IS registered/routed/queued (refutes "not selected"); the engine never reaches ready-state, pump sub_825078D8 bails 5262x, begin-playback sub_825076F0 never fires -> 2000ms readiness timeout. - CANARY oracle measured: video producer sub_824FF678 drives demux SM sub_825211A0 72x+ (loops/pulls continuously); OURS drives it 1x then the pull loop sub_824FA8A0 exits on no-data (assembly sub_8250A2B0 returns 0x8050000B = incomplete unit). Root: ours delivers INCOMPLETE video units where canary delivers complete ones. - NEXT: compare ours-vs-canary fragment chain [slot+24] for the video stream (missing end-fragment / fragment-header mis-parse). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
7e9ee1ac33 |
intro-video RE: deep demux-core instrumentation + handoff hygiene
Investigation probes for the ADV.wmv intro-video deadlock (feeder's demux source-read 0x825211a0 returns no-data 0x80500000 where canary returns 0). - xenia-kernel/state.rs: extend AUDIT-DEREF (fire_audit_pc_probe_if_match) to deep-dump the demux-state object (to +0xfc), its buffer descriptor [sub+4], the byte-source sub-objects ([sub+0]/[+0x30]/[+0x94] and [src0+0x2c]), and to WALK the windowed-buffer cached-block linked list (AUDIT-BLK: per-block 64-bit start/size/dataptr) exactly as mapper 0x82522118 does. Read-only, env-gated (XENIA_AUDIT_DEREF); lockstep digest unaffected. - xenia-app/main.rs: XENIA_DUMP_SLOTS observe-only scheduler runqueue dump. - xenia-app/tests/sylpheed_oracles.rs: resolve ISO via SYLPHEED_ISO, then the repo-root sylpheed.iso symlink, then default; fix path typo. - .gitignore: exclude .claude/ (71k files / 66GB agent worktrees) and the local investigation artifacts (audit-runs/, exit-thread-state.json, zq_*.py). Findings (full chain in handoff notes): all ASF header metadata parses correctly (packet size 16415, count 4728, 2 streams); cursors correct (cur=5868, limit=77.6M); the windowed buffer works and slides (512B blocks); no-data originates deep in the byte-read/parse chain, not from an empty buffer. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
23189b95af |
[iterate-4A] Milestone-2: XMA audio decoder + RE tooling (dispatch recorder, analyzer vtable-fix, non-perturbing probes)
Milestone-2 (intro video dat/movie/ADV.wmv) audio path + major RE tooling. XMA AUDIO (built, working, deterministic, tested): - APU MMIO 0x7FEA0000 + 320x64B register-mapped context array; real XMACreateContext/Release (xma.rs); real FFmpeg xma2 decoder XMA_CONTEXT_DATA->S16BE PCM (xma_decode.rs, xma2_codec.rs, ffmpeg-sys-next). Decode runs synchronously on the CPU thread (deterministic, no host thread). - Audio-worker scheduler fix (main.rs LR_HALT restore + scheduler.rs): the XAudio render-callback worker was wrongly exited after ~2 deliveries; now survives -> guest drives XMA decode (70 kicks). - XAudioSubmitRenderDriverFrame made faithful. Golden sylpheed_n50m re-baselined; tests pass. RE TOOLING: - Runtime indirect-dispatch recorder (dispatch_rec.rs): records (call-site->target, r3, lr); env-gated XENIA_DISPATCH_REC, filters XENIA_DISPATCH_REC_TARGETS/_SITES; deterministic, observe-only. - Repaired static analyzer (vtables.rs): vtable extraction silently fragmented vtables with non-function head slots (missed the XMV engine vtable). Fixed via vptr-write-anchoring -> engine fully typed (vtables 722->1150 on rebuild). - Fixed probe HEISENBUG (main.rs run_superblock): --audit-pc-probe-hex/--mem-watch no longer disable superblock chaining; probes fire inside the chain loop -> scheduling identical armed-vs-unarmed, movie subsystem now observable. Fixed a --quiet bug swallowing armed trace reports. VIDEO still doesn't play (B, guest-side): the XMV engine never issues begin-playback (sub_825076F0, vtable 0x8200a1e8 slot21) -> never primes -> 2000ms timeout. Narrowed to the ARM2 engine-setup wrappers; no honest our-side gate-fix (masking forbidden). See HANDOFF-iterate-4A-milestone2.md for new-machine setup (incl. the FFmpeg apt deps + sylpheed.db regeneration) and continuation pointers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
acb29db444 |
[iterate-3AL] Superblock dispatch: chain basic blocks per slot-visit (~1.6x boot-to-splash)
Replace the one-basic-block-per-slot-per-round lockstep dispatch with a
SUPERBLOCK runner: each slot-visit chains straight-line blocks through
their terminating branches up to a deterministic instruction budget,
amortizing the per-round (timebase/coord/round_schedule) and per-slot
(worker_prologue) dispatch tax over ~128 instructions instead of ~6.
Yield-points (end the chain, return to the round) are pure functions of
guest state, preserving the lockstep cross-thread interleaving correctness:
- non-Continue step result (Yield/SystemCall/Trap/Unimpl/Halted);
db16cyc Yield is the spin-wait producer hand-off.
- sync-sensitive block: lwarx/ldarx/stwcx./stdcx. or sync/eieio/isync
(new PpcOpcode::is_sync_sensitive, flagged on DecodedBlock at build).
- MMIO touch: new GuestMemory::mmio_access_count() watermark, sampled
per block, keeps GPU/register ordering at one-block granularity.
- next PC leaves ordinary guest code (import thunk / halt sentinel /
unmapped) -> hand to the full worker_prologue next round.
- instruction budget reached.
Instruction-count/clock accounting stays exact: per-block cycle_count
deltas are summed and handed to worker_epilogue once (instruction_count +
decrement_quantum advance by the precise retired count). XENIA_SUPERBLOCK_BUDGET=1
reproduces the old one-block schedule byte-for-byte.
Budget tuned to 128 (env-overridable): boot progression stays healthy up
to 256, sharp cliff at ~384 (a boot producer/consumer handoff starves);
128 is 3x below the cliff. Also scale the inline-GPU per-round fairness
cap with the budget (flat 64 throttled GPU command processing 17x under
superblocks and collapsed the present loop).
PERF (check -n 100M --gpu-inline): 25.3 -> 42.7 MIPS (1.69x); 1B: 26.0 ->
41.4 MIPS (1.59x). Callgrind n=5M: host instructions 2.178B -> 1.507B
(-31%); worker_prologue -90%, coord_pre_round -91%, begin_slot_visit /
round_schedule_into / coord_post_round / update_timestamp_bundle each
~-90%; interpreter execute byte-identical (real work unchanged).
GATES: C1 boot progression 150M draws 7391/swaps 2164 (baseline 7415/2172),
1B draws 88547/swaps 29228 linear no stall, K8888 decode + RTs=2 intact.
C2 determinism: n50m stable digest byte-identical across fresh runs;
golden re-baselined intentionally (pacing-only deltas: imports 333453->243387,
draws 1274->1279). C3 milestone-1 render: texture_decodes/draws/swaps/
present cadence track baseline (3AJ fade-in pacing preserved). C4: 690
tests green (+2 sync_sensitive).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
dc1320cd4b |
[iterate-3AK] Perf quick-wins: ~21% faster boot-to-splash (22→27 MIPS)
Profile-driven low-risk optimizations attacking the ~48% per-block / per-round host-bookkeeping tax found by the callgrind profile. Measured on the bounded headless workload `check -n 100000000 --gpu-inline`: baseline ~4490 ms (22.3 MIPS) -> ~3700 ms (27.0 MIPS), +21%. Tier A (determinism-neutral; n50m golden byte-IDENTICAL, exit 0): 1. mem-watch write path: gate capture_mem_watch_old/check_mem_watch behind one has_mem_watch() predicted branch in write_u8/16/32/64 + write_bulk so the common (no-watch) store does no out-of-line call. check_mem_watch (4.8%) gone from the profile. 2. round-schedule alloc churn: add Scheduler::round_schedule_into filling a reusable [u8; HW_THREAD_COUNT] stack buffer; the lockstep round loop no longer __rust_alloc/__rust_dealloc a Vec<u8> per round. Identical ordering/RNG-advance. __rust_alloc/dealloc gone from the profile. 3. probe-firing: hoist a single KernelState::any_probe_active() guard to worker_prologue so the four fire_*_if_match calls don't happen at all when no probe is configured (was 4x call overhead/visit). All four gone from the profile. 4. thunk-map hash: range-reject pc against the registered import-thunk address band (KernelState::pc_in_thunk_band, two int compares) before the thunk_map.get(&pc) HashMap lookup. hash_one (4.3%) gone. Tier B (#5, time-granularity change — LANDED, no re-baseline needed): 5. update_timestamp_bundle: throttle to a 0.25 ms quantum (only re-write the KeTimeStampBundle when the deterministic clock advanced >= 2500 units). Inclusive cost 8.65% -> 1.08%. The quantum is far below the 1 ms granularity any guest deadline math needs (tick_count stays fresh; the hub gate is +66 ms; the fade-in is vsync-counter driven per 3AH, not this bundle). VERIFIED: n50m stable digest BYTE-IDENTICAL to the existing golden (so no re-baseline), 150M boot reaches the splash (draws=7415, swaps=2172, gpu.texture.decode{K8888}=448, RTs=2 — all match the post-3AJ baseline), 688 tests green, release n50m oracle ok. Remaining headroom: interpreter::execute (13%), decrement_quantum (8%), step_block (7%) are now the top self-costs — the structural superblock/ JIT lever is the next step for the larger gain. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
9d24dd0eaa |
[iterate-3AJ] Present-anchor vsync so the splash logo fade-in renders
The publisher/dev splash logo's intro fade-IN was skipped: the logo popped in at full brightness instead of ramping dim->bright like the canary oracle. Root (measured, iterate-3AF/3AI): ours' guest vsync counter is fed by a fixed-instruction-quantum proxy (one vsync per 150k retired instructions). During the ~1.1s splash asset-load the title's frame pump runs ~10M instructions inside a single guest frame, so the proxy fired ~66 vsyncs in that one frame. The pump's per-frame delta (counter_now - counter_last) was therefore ~66 on the first tick, which the anim tick (sub_823CDBF8) divides into the fade counter [item+72] @ 0x40c0add0 -> the counter JUMPED 0->0x42(66) in one step, landing past the fade-in region. Canary's wall-clock 60Hz vblank advances ~1 per heavy load frame, so its counter ramps smoothly 0->66 and the fade-in renders. Fix: anchor the lockstep vsync ticker to the guest's real present rate (VdSwap count), mirroring real hardware where the title double-buffers at vblank, so one heavy guest frame advances the vsync counter by ~1 instead of ~66. - interrupts.rs: tick_vsync_instr now takes the live present count. Two regimes: (1) bootstrap, before the guest's first present, keeps the original fixed instruction quantum unchanged -- the iterate-2W present-loop bootstrap needs vsyncs delivered BEFORE it can present (measured: callback registered ~6M instr, first delivered vsync and first present coincide; pure present-driven vsync would deadlock). (2) present-anchored, after the first present: one vblank per present, plus a small DRY_FALLBACK_CAP=4 instruction-quantum fallback per dry window so a non-presenting frame still ticks a few vsyncs (a small ramp like canary's 0/5/10/2/1...) without re-spiking to 66. - handle.rs: cheap GpuBackend::swaps_seen() accessor. - main.rs: pass the live present count into the lockstep ticker. Not masking: the fade dt/counter is never clamped or synthesized; the guest naturally computes a smooth dt once vblank tracks presents. Verified: - V1: fade counter 0x40c0add0 now ramps 0,6,8,10,12,13,+1... (was a 0->0x42 jump; direct baseline-vs-fix mem-watch). - V2 (--ui readback via per-frame logo vertex-alpha): logo alpha ramps 102,136,204,221,238,254 (dim->bright fade-IN) vs baseline all 255 (pop-in). Real artwork (has_real_vertices) still renders; milestone-1 intact. - V3: 150M boot progression intact -- texture_decodes=2, RTs=2, tex_cache=1 unchanged; draws/swaps higher (tighter present loop), 1B sanity linear, no stall/collapse. - V4: 50M --gpu-inline --stable-digest byte-identical 2x; golden re-baselined intentionally (pacing-only delta: draws 718->1274, swaps 147->259; structural fields unchanged). 688 tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
c62a355418 |
[iterate-3AE] Fix spurious WHITE TRIANGLE flashing before each splash logo
The publisher and developer splash logos rendered correctly, but a fullscreen OPAQUE WHITE diagonal half-triangle flashed at boot, before each logo, and persisted across the dev-logo transition — canary shows a black background there. Readback-isolated it (env-gated frontbuffer grid + per-draw inventory, both removed) to the background-fill draws. ROOT (measured, refutes the prior "saturate/interpreter/depth" guesses): the position-only VS `0xd4c14f46` (one vfetch → oPos; exports NO color) paired with PS `0xed732b5a` (`ocolor0 = interp0`). The iterate-3T translator seeded `ointerp[0] = (1,1,1,1)` "so a VS that only exports position still yields a visible non-zero color" — a debug FAKE: it injects white that no guest value backs. So that fill's interp0 stayed white → opaque-white fullscreen triangle. Vertex windows of a WHITE frame and a steady BLACK frame were byte-identical; served_translated=true for all of them and depth is disabled in the replay, so the white came purely from the injected seed, not saturate/interp/depth. FIX (UI-translator only, golden byte-identical): - translator.rs: default un-exported interpolators to (0,0,0,0) instead of seeding interp0 white. A position-only VS now contributes nothing visible under its real blend (RGB=0 → black; A=0 → premult transparent), matching canary; every VS that really exports interp0 (the logo `0x03b7b020`, the color fill `0x36660986`) overwrites the seed → logos unaffected. - app.rs: clear the splash frontbuffer to BLACK, not the iterate-3S navy placeholder `[0.04,0.04,0.06]` (never matched to the guest). The fill is a fullscreen Xbox-360 RectangleList drawn as a single triangle in the replay (4th implied corner not yet synthesized), so its uncovered half exposed the clear; black makes the transition uniformly black like the oracle. (Full RectangleList→rectangle expansion is a separate follow-up.) READBACK (env-gated, removed): white-heavy frames 200+ → 0; navy frames 240 → 0; transition frames uniformly black; the publisher logo (white text + red dots) and the developer logos (colored, on black) still render. Determinism: changes feed only the UI translator/clear; n50m --gpu-inline --stable-digest byte-identical 2× and matches the committed golden (--expect exit 0). cargo test --workspace 686 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
3f8d3b6f1c |
[iterate-3AD] Fix 2nd splash logo rendering black: re-upload evolving atlas
The publisher (SQUARE ENIX) and the 2nd developer/studio splash logo share
one K8888 atlas at physical base 0x4dbee000, sampled at different UVs. The
publisher's white text occupies the top V-bands; the developer logo's
(bluish/gold) artwork is CPU-written into the SAME surface AFTER the publisher
frame, so the atlas evolves across frames.
The UI host texture cache (`texture_cache_host::upload`) only re-uploads a
`TextureKey` when `version_when_uploaded` increases. But the per-draw bind in
`render.rs` hardcoded `version_when_uploaded = 1` for every draw, so once the
atlas was first uploaded (during the publisher frame, with only the top bands
filled) the cache pinned that partial upload. The 2nd logo, sampling a V-band
that was still zero at first-upload time, read transparent-black -> rendered
nothing (the "white-triangle / black stub" the user saw after SQUARE ENIX).
Verdict: (G) a legitimate 2nd LOGO item whose real artwork lives in the same
evolving atlas — NOT a spurious 3rd item, and NOT a geometry/shader/blend gap.
Measured via readback: the 2nd-logo geometry rasterizes correctly (3 on-screen
quads), interp1 (UV) and interp0 (color) reach the PS with real values, the
texture content at the sampled bands exists — only the bound wgpu texture was
the stale partial upload.
Fix (UI-only, deterministic core untouched):
- `gpu_system`: thread the real content `version` (from `span_max_version`)
into `last_draw_textures` (now `(key, version, bytes)`).
- `draw_capture::DrawCapture.textures`: same 3-tuple.
- `render.rs`: use the real `version` (not a hardcoded 1) so the host cache
re-uploads when the guest fills more of the atlas.
- `exports.rs` `vd_swap`: the legacy single-texture `publish_texture` bridge
drops the version (`(key, _v, bytes) -> (key, bytes)`).
Readback (env-gated probe, removed before commit): after the fix the 2nd logo
renders real varied artwork (blue + gold texels in a centered strip) instead
of black. Determinism: `check -n50m --gpu-inline --stable-digest` byte-
identical to the
|
||
|
MechaCat02
|
c0c6088e4d |
[iterate-3AA] Fix logo upside-down: no Y-flip on the clip-enabled NDC path
DEFECT 1 (logo upside down) ROOT + FIX. The publisher "SQUARE ENIX" logo rendered vertically mirrored vs the canary oracle (white upright on black). Measured (env-gated readback + texture-row + per-vertex dumps, all removed): - The K8888 logo texture decodes UPRIGHT (text in the top rows 1..161; the red dots sit at ~43% from the texture top). NOT a decoder row-order bug. - The logo geometry is a centered QuadList whose vertices are emitted in *clip space* (Y-UP, e.g. pos.y +0.085 top / -0.104 bottom), with the texture V mapped top->bottom (UV v 0.001 at the top vertex, 0.090 at the bottom). On both the Xbox 360 (D3D9) and wgpu, clip +Y maps to the framebuffer top — so a clip-space position is portable with NO Y-flip. - `compute_ndc_xy` unconditionally negated Y (the flip the *screen-space* pixel path legitimately needs). For the clip-enabled logo this swapped top<->bottom vertices while leaving the texture V unchanged, so the sampled sub-rect read bottom-up: red dots rendered at 58% from the top (a clean vertical mirror) instead of 43%. FIX: keep the Y-flip only on the clip_disable (screen-space pixel) branch where the framebuffer Y-down->wgpu Y-up flip is real; the clip-enabled branch now passes clip-Y-up through identity. Readback after the fix: red dots at 42% from the top (= texture's 43%) -> logo UPRIGHT, still centered. DEFECT 2 (background) was already correct + faithful; 3Z's contradiction is REFUTED by direct readback: the bg fill (vs 0x36660986 / ps 0xed732b5a, fullscreen RectangleList) reads its real vertex color (raw 0x818000c7 = -32896.5 as float) into r0, the PS exports it, and the GPUBUG-115 RB-UNORM saturate (canary spirv_shader_translator.cc:3607) clamps it to 0 -> BLACK, matching canary. The seed r0=(gvidx,...) does NOT show through (it's overwritten by the color vfetch). No code change needed. Readback of the full frame now matches canary: WHITE upright "SQUARE ENIX" + red dots on a BLACK field. UI-capture-path only (`compute_ndc_xy` runs solely when frame_captures is Some, i.e. --ui; None headless) -> deterministic core untouched, n50m --gpu-inline --stable-digest exit 0 (DRAW_INDX 275 / K8888 decode 137, identical across runs). cargo test --workspace green. Temp probes removed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
f6f3aac673 |
[iterate-3Z] Fix logo color (yellow->white): k_8_8_8_8 vfetch + vfetch field/stride/saturate
Defect 2 of the three render-fidelity defects vs the canary oracle (the
publisher "SQUARE ENIX" logo rendered YELLOW instead of WHITE). Root,
measured by readback (env-gated probes, removed): the logo PS multiplies
the sampled texture by the interpolated vertex COLOR; the K8888 texture
itself decodes correctly (67,667 white texels + 2,087 red — the red dots —
zero yellow), so the yellow came from the vertex-color attribute decode.
Four coupled, canary-faithful fixes (all UI-translator/capture only — the
deterministic headless core is untouched; n50m --gpu-inline --stable-digest
golden byte-identical, exit 0):
- GPUBUG-112 (translator vfetch): VertexFormat 6 = k_8_8_8_8 (4x u8
normalized, 1 dword), NOT k_16_16 (which is 25) per canary xenos.h:643.
The logo color stream is k_8_8_8_8; decoding it as k_16_16 read only 2 of
4 channels and forced BLUE = 0 -> white texture x (R,G,0) = yellow. Now
unpacks all four 8-bit channels (canary spirv_shader_translator_fetch.cc
k_8_8_8_8 packed_offsets 0/8/16/24); added k_16_16 (format 25) too.
- GPUBUG-113 (ucode/fetch): vfetch is_signed / is_normalized / is_mini_fetch
bit positions were wrong (read bits 24/25, which sit inside exp_adjust).
Per canary ucode.h:757-758,764: signed=fomat_comp_all (w1 bit12),
normalized=(num_format_all==0) (w1 bit13), mini_fetch (w1 bit30).
- GPUBUG-114 (translator vfetch): a vfetch_mini reuses the address AND
STRIDE of the preceding full vfetch of the same stream (canary
ucode.h:733); its own stride field is 0. Track the last full stride per
fetch-const and inherit it so a mini color/UV attribute indexes by the
real vertex stride, not its tight dword count.
- GPUBUG-115 (translator PS export): saturate the color export to [0,1]
before the UNORM render-target write, mirroring canary
spirv_shader_translator.cc:3607 ("Saturate, flushing NaN to 0"). Without
it an out-of-range guest color writes garbage to the sRGB target.
Verified by env-gated frontbuffer readback (copy_texture_to_buffer, removed
before commit): the logo now renders WHITE text + RED dots (bbox centered
~y322-389), zero yellow anywhere. Workspace tests green (added 4: k_8_8_8_8
4-channel unpack, mini-fetch stride inheritance, vfetch bit decode, PS
saturate). Determinism: golden byte-identical.
Remaining (defects 1 & 3, see memory iterate-3Z): logo orientation and the
ed732b5a fullscreen background fill (renders ~white, canary shows black) —
both localized but not yet cleanly resolved; plan in the memory file.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
2a992db47b |
[iterate-3Y] Replay per-draw blend + write-mask so the logo composites visible
The publisher logo rendered its real artwork in isolation (3X) but was
overpainted in the full composite: every replayed draw used ONE fixed
SrcAlpha/OneMinusSrcAlpha pipeline + an opaque-magenta texture stub, so the
textured RectangleList draws whose sampler slot is shadowed by a vertex-fetch
constant (no resolvable texture) wrote opaque magenta over the logo.
Per-draw render-state inventory at the splash (env-gated probe, removed):
- logo QuadList vs=0x03b7b020 ps=0x03b79001: bc0=0x07010701
(One,OneMinusSrcAlpha — premultiplied alpha), cmask=0xF, ntex=1 (real K8888)
- RectangleList vs=0xd4c14f46 ps=0x03b79001: SAME premult blend, ntex=0
(slot 0 holds a type=3 vertex constant → texture decode rejects) → magenta
- opaque fill vs=0x36660986 ps=0xed732b5a: bc0=0x00010001 (One,Zero) — green
Draw order: the logo is drawn LAST per group, so order was not the problem;
the fixed pipeline state was.
Change (UI-side capture/replay only):
- draw_capture: capture RB_BLENDCONTROL0 + RB_COLOR_MASK (+ colorcontrol /
depthcontrol for follow-ups) per draw.
- xenos_pipeline: new RenderState{blend_control,color_mask}; map Xenos blend
factors/ops -> wgpu mirroring canary kBlendFactorMap/kBlendFactorAlphaMap;
One,Zero,Add => blend:None (opaque); zero-channel mask => ColorWrites; cache
translator AND interpreter pipelines keyed on (vs,ps,RenderState) /
RenderState so each draw composites with its real state.
- render: pass each capture's RenderState through both replay paths.
- dummy texture magenta(255,0,255,255) -> transparent(0,0,0,0): an
unresolvable texture now contributes nothing under its real premult blend
instead of fabricating opaque magenta (removes a fake, adds none).
Readback (env-gated, removed): full 1280x720 composite now shows the logo's
real artwork (maxR=255, 50-102 distinct colors/cell) in a centered strip; no
magenta anywhere. Background is uniform green (the 0xed732b5a opaque fill) — a
separate vertex-color/shader fidelity issue, NOT compositing (next iterate).
Determinism: UI-only; draw_capture additions only run when frame_captures=Some.
check -n50m --gpu-inline --stable-digest --expect = "matches golden" (2x).
cargo test --workspace = 682 passed. Temp probes removed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
89b5c39d8a |
[iterate-3X] Real splash logo geometry renders: fix vertex-fetch const_index_sel + per-draw submit
Two readback-proven root-cause fixes make the publisher-logo QuadList draw land its REAL captured vertex buffer (the texture was already correct from 3V). REFUTES iterate-3W's "logo geometry is auto-generated from vertex_id": the logo IS sourced from a 4-vertex QuadList buffer at guest physical 0x0adf60f0 (measured), it was just resolved at the wrong fetch-constant register. GPUBUG-110 (vertex fetch const_index_sel dropped). The Xenos vertex-fetch instruction encodes const_index (w0[20:24]) AND const_index_sel (w0[25:26]); the full constant index is const_index*3 + const_index_sel (canary ucode.h:700), packed 3 two-dword constants per 6-dword register group. ucode/fetch.rs decoded only const_index and read sub-slot 0 (fc*6). The logo vfetch is const_index=31, sel=2 -> the real base lives at reg 0x48BE, but ours read 0x48BA which held an unused 0x00000001 (base=0,size=0) slot. So resolve_vertex_window returned None -> has_real_vertices=false -> the logo fell to the procedural fullscreen magenta fallback. Fix: decode const_index_sel, add VertexFetch::const_reg_offset() = const_index*6 + sel*2, and use it in both draw_capture.rs (capture) and translator.rs (the WGSL endian term + no-window fallback base; the old expression there read the src_reg bits, not the const index). Measured: logo now resolves a 24-dword (4 verts x stride 6) window, base 0x0adf60f0. GPUBUG-111 (single batch encoder = last-draw-wins vertex data). In wgpu every queue.write_buffer staged before a single queue.submit is applied before ANY command in that submit runs. dispatch_xenos_captures recorded the whole batch into one encoder + one submit, so every draw read only the LAST draw's vertex buffer / per-draw uniforms. The logo quad therefore sampled the trailing fullscreen background quad's vertices and rasterized nothing where the logo was. Fix: submit one encoder per draw (frontbuffer LoadOp::Load composites identically). Measured (env-gated readback, removed): with this fix the logo draw in isolation renders real varied texels (e.g. (225,17,22)/(255,255,0)) in a centered strip (~20k px), vs 100% navy before. Determinism: all changes are UI-side (xenia-ui replay) or the UI translator / capture path (frame_captures None in headless); the fetch.rs field addition is purely additive and does not change any existing decoded value. Verified the deterministic core unchanged: check -n50M --gpu-inline --stable-digest exit 0 and all 136 metric counters byte-identical across two runs. All temp probes removed. cargo test --workspace green; new regression test vertex_fetch_const_index_sel_and_reg_offset. Known remaining (next iterate): a fullscreen flat QuadList (ps 0x03b79081, vertex color green, no texture) and other textureless draws overpaint the logo in the full composite (their per-draw blend/alpha render state is not yet replayed, and draw order alternates bg/logo). The logo artwork renders correctly in isolation; the composite is not yet clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |