Commit Graph

8657 Commits

Author SHA1 Message Date
Sylpheed RE agent
f9170fd3fe [RE] xam_input: log the keystroke path that REACHES the driver
The existing [RE-INPUT] line only fires when IsUIActive() swallows a keystroke,
so its silence is ambiguous: it means either 'the game is polling and getting
nothing' or 'the game is not polling at all'. That is exactly the fork the
title-screen investigation is stuck on - with the sign-in dialog fixed the
swallow count is 0 and the title still does not respond to (A).

Adds two logs on the other side: a rate-limited count of calls that get past
IsUIActive to a driver, and one line per keystroke actually handed to the guest
(rare by construction - one per physical press - so every one is logged, with
user index, virtual key and flags).
2026-08-19 08:35:26 +00:00
Sylpheed RE agent
53b208bf4d [RE] log_ui_draws: log the UI quad's colour attribute
A UI quad carries a k_8_8_8_8 colour next to its position, and its alpha is the
fade the bundle's keyframes animate. Logging it costs one more read per vertex
and turns the capture from "where is this quad" into "where is it and how faded".

It was added to separate two elements that decode to the same 1133x280 and sit on
opposite sides of a third in the declaration table. It does NOT separate them —
both rest at 255 — but it does check the fade decode against the running game for
the first time: every static element draws at exactly the resting alpha the
bundle predicts, and the only two quads whose alpha changes between consecutive
frames are the rotating effect pair and the PRESS (A) glow, which are the two
things visibly animating.
2026-08-19 01:09:04 +00:00
Sylpheed RE agent
4cd019b769 [RE] Arm the UI draw capture on F10 unconditionally
The capture used to require --log_ui_draws at LAUNCH, which put an unexplained
variable into every navigation run: across 12 runs, launching with the flag
correlates with the title screen refusing (A) — 0 of 7 with it, 4 of 5 without.

No mechanism was found. The cvar is read in exactly one place, when F10 arms a
capture, and F10 was never pressed in those runs; the per-draw hook is a single
relaxed atomic load; the startup config dumps of the two arms are byte-identical.
An interleaved A/B also ruled out the obvious confound (boot duration): the
latest title of all, 268 s, ACCEPTED (A), while a 232 s title refused.

So rather than keep a variable nobody can explain in the path of every run, arm
on F10 the way the ship capture next to it already does. The cvar stays, marked
obsolete, so existing command lines still parse. Verified: F10 with no capture
flags at all writes xenia_re_ui_draws_01.log.
2026-08-19 00:36:34 +00:00
Sylpheed RE agent
043002a871 [RE] Say so when a guest keystroke is swallowed by IsUIActive
XamInputGetKeystrokeEx returns X_ERROR_SUCCESS with a zeroed keystroke whenever
any Xenia dialog is up, before consulting a driver. That is correct behaviour and
an invisible one: from outside, scripted input simply stops working, the pad
driver logs nothing because it is never asked, and the guest keeps polling. It is
easy to enter that state by accident — F10 is both the RE capture hotkey and the
toolkit's menu-bar key, and a dialog closed by its own [x] rather than by the
menu toggle leaves the counter incremented.

So log it, rate-limited to one line per 600 swallowed calls, with the dialog
counters. On the runs this was written to diagnose it stays silent, which is what
ruled the theory out — a diagnostic that is useful when it does not fire.
2026-08-18 22:18:38 +00:00
Sylpheed RE agent
7dbb24e64f [RE] --create_profile_if_none: bootstrap a profile for scripted runs
The title screen's (A) opens the Sign In dialog when no profile exists, and the
game goes no further. That dialog cannot be completed from a scripted container:
"Create Profile" asks for a gamertag in an ImGui text field, and synthetic X key
events do not reach it — tried with the window focused, via XTEST and via
--window, char by char, after clicking the field. Mouse clicks work; text entry
does not.

So: when no profile exists on disk and this cvar is set, create one at startup
and sign it in. It uses ProfileManager's fixed `default_xuid`
(B13EBABEBABEBABE), which is what makes it usable from a script — a later run
passes --logged_profile_slot_0_xuid=B13EBABEBABEBABE and is deterministic.
Ignored when a profile already exists, so it is safe to leave in a launcher.

Verified: "RE bootstrap profile 'SylphRE' -> created", then on the next boot
"Found 1 Profiles" / "Loaded SylphRE (GUID: B13EBABEBABEBABE) to slot 0".

This does NOT get the game past the title — see the Reborn repo's
docs/re/canary-scripted-input-traps.md for what does and does not, and for the
content-path crash that is the actual blocker.
2026-08-18 20:15:35 +00:00
Sylpheed RE agent
8cca105eb9 [RE] log_ui_draws: bound the capture by cvar, and log the quad's Z
Three changes, each paid for by a measurement that could not be made without it:

* `--ui_draw_capture_frames` / `--ui_draw_capture_max` replace the compiled-in
  3-frame, 20k-draw bounds. A screen that redraws every frame needs 3; finding
  the frames in which a screen is BUILT needs hundreds, and that answer (the
  title screen never rebuilds — it submits the same 11 draws every frame) is
  only reachable by turning the window up.

* the vertex dump prints all three floats. Attribute 0 of a UI quad is
  k_32_32_32_FLOAT, so the stream carries a Z — the game's own layer key, if it
  had one. It does not: every Z is 0.00000, which is what makes submission order
  the whole of the paint order.

* positions print as floats rather than raw words, now that the format is known.
2026-08-18 20:15:35 +00:00
Sylpheed RE agent
2263470723 [RE] log_ui_draws: a per-draw, submission-order capture for 2D screens
Neither existing RE hook can say in what order the game paints a UI screen:
`log_draws` de-dups by vertex-declaration fingerprint, so a screen's sprites —
which share a declaration — collapse into one record; and the F10 ship capture
returns early on any draw without an f32x3 position stream, which is every UI
quad.

So: `--log_ui_draws` arms (on F10, alongside the ship capture) a snapshot of the
next `--ui_draw_capture_frames` submitted frames, undeduplicated, writing every
draw with its primitive type, index count, VS/PS hashes, each bound texture's
base and dimensions, and the quad's vertex positions. The positions are the part
that matters: a screen's sprites all share one shader and sample big texture
pages, so the geometry is what names an element.

Frames are counted here rather than from `counter_`. `counter_` looks like a
frame number — the VdSwap packet increments it — but MarkVblank() increments it
too, from the vblank thread, so it advances between two draws of the SAME frame.
Bounded by it, the first capture ended after one draw having "covered 6 frames".

Measured on the title screen: 11 draws a frame, every frame.
2026-08-18 18:57:18 +00:00
MechaCat02
a05350ced3 [Build] Fix the SPIR-V shader script hiding the error it is reporting
Steps 2 and 3 (spirv-opt, spirv-dis) run subprocess without text=True, so
result.stderr is bytes -- and `sys.stderr.write(bytes)` raises TypeError. The
tool's real message is replaced by a Python traceback at exactly the moment you
need it.

Building in a clean container, the visible failure was:

  ERROR: spirv-opt failed for guest_output_bilinear.ps.xesl
  TypeError: write() argument must be str, not bytes

with the actual cause -- `Unknown flag '--canonicalize-ids'`, i.e. a SPIRV-Tools
too old -- never printed. Use .buffer.write, as compile_shader_dxbc.py already
does at the same site.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 23:06:02 +02:00
MechaCat02
6fe509c89e [Audit] Make the ported Phase B snapshot actually build
Taking phase_b_snapshot.{cc,h} as files from the snapshot branch left it
referencing three cvars that live in that branch's cpu_flags and were not
carried across, so the kernel library failed to compile:

  phase_b_snapshot_dir / phase_b_snapshot_and_exit / phase_b_dump_section_content

Ported their DEFINE_/DECLARE_ pair verbatim. Also switched one
`std::filesystem::path base(...)` to brace init -- with a single named argument
that parses as a function declaration under a newer clang than the branch was
written against, and this tree builds with -Werror.

Verified on the linked binary: all fifteen instrumentation cvars from the three
merged lineages are present, and so is every output path the superset draw
logger emits (xenia_re_draws.log, xenia_re_shaders.log, xenia_re_files.log,
xenia_ship_capture_NN.log, with idx_raw / vsconst / psconst records).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 20:41:47 +02:00
MechaCat02
7a682d7cf4 [Docs] Land the PPC / Xbox 360 export reference tables
These were sitting in stash@{0} ("On canary_experimental: doc", 2026-04-11) as
untracked files only -- no tracked change was ever stashed with them, so they
were one `git stash drop` away from being lost:

  docs/ppc_instructions.{json,md}   PPC instruction reference
  docs/xbox360_exports.{json,md}    XAM / xboxkrnl export tables
  docs/xex2_format.md               XEX2 container format
  tools/generate_export_docs.py     the generator behind them

xex2_format.md in particular is the spec a static XEX reader would need to
pull the title's XACH achievement table and .rdata tables off the disc.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 18:10:16 +02:00
MechaCat02
55c47b17ca [Audit] Take the snapshot branch's unique probes (files only, not a merge)
auto/canary-instrumentation-snapshot-2026-07-28 was authored from a base 85
commits behind this line and re-adds the cross-build toolchain and the audit
probes as unrelated new files, so merging it conflicts in 15 places for no
gain. Everything it carries is already here except these eight, which exist
nowhere else:

  - phase_b_snapshot.{cc,h}          one-shot JSON state snapshot, fired from
                                     the JIT before the entry thread's first
                                     guest instruction (--phase_b_snapshot_dir)
  - audit_68_host_mem_watch_{base.cc,fwd.h}
  - audit_69_event_signal_watch.{cc,h}
  - audit_70_semaphore_release_watch.{cc,h}

Its xboxkrnl_xconfig.h is unrelated to the instrumentation and unreferenced
here, so it is deliberately left behind. The branch itself is kept (its
commits are not ancestors of this one).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 18:10:03 +02:00
MechaCat02
8d1e731ead Merge remote-tracking branch 'origin/capture-drawlog' into sylpheed-re
# Conflicts:
#	src/xenia/base/threading_posix.cc
#	src/xenia/gpu/command_processor.cc
#	src/xenia/gpu/command_processor.h
2026-08-17 18:08:52 +02:00
MechaCat02
2ae44a0edc Merge remote-tracking branch 'origin/capture-ship-placement' into sylpheed-re
# Conflicts:
#	src/xenia/gpu/command_processor.cc
#	src/xenia/kernel/xboxkrnl/xboxkrnl_video.cc
2026-08-17 18:07:25 +02:00
4040c701c5 [Kernel] frame probe: sample guest memory once per guest frame, for RE
Some checks failed
Orchestrator / Commit Message Validation (push) Has been skipped
Orchestrator / Lint (push) Failing after 1m52s
Orchestrator / Windows (x86-64) (push) Has been skipped
Orchestrator / Linux (x86-64) (push) Has been skipped
Orchestrator / Create Release (push) Has been skipped
Reading guest RAM from outside the emulator is easy -- Canary backs it with a
/dev/shm file, so a host process can just pread it -- but it is UNSYNCHRONISED.
The reader has no idea where the guest is in its update, so successive reads are
separated by an unknown, jittering number of guest updates.

That is not a theoretical concern. Measuring the player craft's angular velocity
this way produced 3x swings between adjacent 0.25 s windows, which made it
impossible to tell "the rate ramps up after the stick goes over" from "the
sampler is aliasing" -- the one question standing between a confirmed law shape
and its absolute scale. Widening the window does not rescue it either: a hold
long enough to average bleeds the speed, and speed is the variable under test.

So sample from inside, in VdSwap: the guest's own thread, once per guest frame,
at a fixed point in it. Consecutive lines are then exactly one frame apart by
construction and the frame counter is exact.

Which bytes to sample is not known at launch -- object addresses are found by
scanning at runtime -- so the regions come from a small control file that is
re-read whenever its mtime changes, the same trick the file input pad uses:

    0x40D10590 128        a guest VA and a byte count, one region per line

and each frame appends

    F <frame> H <host_ns> G <guest_ticks> R0 <hex> R1 <hex> ...

Reads are bounds-checked through LookupHeap/QueryProtect and copied out before
printing, so an unmapped or stale address logs "-" instead of taking the
emulator down. Header-only and off unless --frame_probe_log is set, so it adds
no build target and no cost to anyone not using it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 06:03:09 +00:00
15fe11d5d9 [HID] file-pad: implement GetKeystroke -- menus do not read GetState
Some checks failed
Orchestrator / Commit Message Validation (push) Has been skipped
Orchestrator / Lint (push) Failing after 2m3s
Orchestrator / Windows (x86-64) (push) Has been skipped
Orchestrator / Linux (x86-64) (push) Has been skipped
Orchestrator / Create Release (push) Has been skipped
The pad looked correct and did nothing. Its own log showed A arriving, the
emulator sat on "PRESS (A) BUTTON", and the title never advanced.

Cause: 360 front-ends poll XamInputGetKeystrokeEx, not XamInputGetState. This
title imports both, and its menus use the keystroke path; the driver returned
X_ERROR_EMPTY there, so every scripted press went into the void while GetState
faithfully reported a button nobody asked about.

Implement it edge-triggered, one event per call: KEYUPs for everything released
first, then KEYDOWNs, matching the SDL driver's ordering (so a thumb transition
clears before it sets). Deliberately NO auto-repeat -- scripted input wants
exactly one event per press, and repeat is precisely what makes menu steps
overshoot. Bits without a virtual key (guide, unused) are swallowed rather than
re-offered forever.

Verified on the real game: title -> main menu -> EXTRAS driven entirely from the
pad file, with `[file-pad] keystroke vk=5800 down/up` in the log for each press.
2026-08-13 20:27:38 +00:00
e3e17e4951 [HID] file-pad: nanosecond change detection, and log every state change
Some checks failed
Orchestrator / Commit Message Validation (push) Has been skipped
Orchestrator / Lint (push) Failing after 2m24s
Orchestrator / Windows (x86-64) (push) Has been skipped
Orchestrator / Linux (x86-64) (push) Has been skipped
Orchestrator / Create Release (push) Has been skipped
Two things that only show up once you actually script this pad.

st_mtime is whole seconds. Combined with size it looked like enough and is not:
a script stepping a menu writes several same-length states per second
(`press=A` then `press=B`, both 8 bytes), and every one after the first was
silently dropped -- the emulator simply did not react, with nothing in any log
to say why. Compare st_mtim.tv_nsec as well, and track whether the file existed
at all so a delete is registered once rather than every frame.

Also log one line per state change (not per frame, so it stays quiet). Driving
the emulator headless means there is nothing to watch; this line is the only
proof that a scripted press was picked up, which turns "did my input land?" from
a guess into a grep.
2026-08-13 20:14:29 +00:00
d15c8cfab6 [HID] file-pad: a controller driven by a text file, for scripted RE
Some checks failed
Orchestrator / Commit Message Validation (push) Has been skipped
Orchestrator / Lint (push) Failing after 2m24s
Orchestrator / Windows (x86-64) (push) Has been skipped
Orchestrator / Linux (x86-64) (push) Has been skipped
Orchestrator / Create Release (push) Has been skipped
The scripted-input tool this project uses for reverse engineering created its
pad through /dev/uinput. Input devices are NOT namespaced by the kernel, so a
uinput device created inside a container registers with the HOST's input stack:
every trigger hold and button press is delivered to whatever on the host reads
gamepads, not just to the emulator. That was noticed the hard way, and it made
every runtime experiment -- booting, menu navigation, unit harvesting, flight
measurement -- unusable from inside the box.

This driver takes the kernel out of the loop. Pad state lives in an ordinary
text file only the container can see; GetState re-reads it when it changes.
Nothing is registered with the host and no X server is involved. A bonus for RE:
analogue values are exact rather than whatever a virtual stick quantises to.

    press=A,START     buttons by name, comma separated
    buttons=0x1010    or the raw XINPUT mask
    lt=0 rt=255       triggers, 0..255
    lx=0 ly=0         thumbs, -32768..32767

Absent keys are neutral, so `press=A` alone is a valid file, and a missing or
empty file means no input -- the safe default if it is deleted mid-run.

Selected with --hid=file, path from --pad_file (default /tmp/xenia_pad.txt).
Deliberately NOT part of "any": this pad has to be asked for. Header-only, so it
adds no build target and no cost to anyone not using it.
2026-08-13 20:12:02 +00:00
31366e5cac [GPU] ship-capture: log each draw's index buffer, and key the de-dup on it
The offline XBG7 decoder can only assume where a block's index data lives; the
capture now states it. Each captured draw gains an `ib base=… count=… fmt=…
endian=… len=… delta_vb=… min=… max=… idx: …` line — the guest index base, the
draw's index count, and the min/max index value read out of guest memory, which
together say both where the index buffer sits relative to the vertex buffer and
how much of the vertex pool the draw covers.

Also mix the index range into the capture's de-dup key. The engine issues several
draws over one vertex buffer, each indexing a sub-range (a 119-vertex hull LOD
draws 21 indices first, then 225); keying on (vbase, transform) alone kept only
the first batch, which reads like a mysteriously short draw and cost an earlier
session a spurious "the capture disagrees with the descriptor" mystery.

Read-only diagnostics behind the existing F10 hotkey; no emulation behaviour
changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NptfmpjdpNCKEez6d2xvA9
2026-08-13 05:58:02 +00:00
MechaCat02
a08526dff0 [GPU] ship-capture: numbered per-press snapshots + capture every instance
Some checks failed
Orchestrator / Commit Message Validation (push) Has been skipped
Orchestrator / Lint (push) Failing after 1m45s
Orchestrator / Windows (x86-64) (push) Has been skipped
Orchestrator / Linux (x86-64) (push) Has been skipped
Orchestrator / Create Release (push) Has been skipped
F10 now writes xenia_ship_capture_NN.log (NN = press number) so several angles
can be captured in one run without overwriting, and de-dups by (vertex-buffer
address, c0..c2 WVP hash) instead of address alone -- the same part buffer
drawn at different transforms (two engine nacelles, each ship of a fleet) now
yields one record per distinct placement. Seen-set cap raised to 8192.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 20:35:20 +02:00
MechaCat02
93fe5d69df [GPU] ship-capture: also dump VS float constants (the placement matrix)
The captured vertex BUFFER holds LOCAL positions (byte-identical to the .xpr) —
capital-ship parts are placed entirely in the vertex shader, not in the buffer.
So the per-part world/WVP matrix lives in the VS float constants. Extend the F10
capture to dump the first 48 vec4 from the SQ_VS_CONST base per draw; diffing two
parts isolates the world matrix (the camera VP block is shared across draws).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 20:35:20 +02:00
MechaCat02
067a373734 [GPU] F10 ship-placement capture: dump world-space draw vertices for RE
One-shot snapshot for reversing capital-ship part placement. F10 arms
RequestShipCaptureFrame(); the next batch of draws is dumped to
xenia_ship_capture.log with each draw's guest vertex-buffer address, vertex/
index counts, and up to 64 WORLD-space vertex positions. Capital-ship parts are
transformed into world space before the draw (unlike stage geometry, which
matches the .xpr byte-for-byte), so these are ground truth for the assembled
placement; a reborn-side correlator affine-fits each decoded .xpr part to a
captured draw to recover its exact transform. De-duped by buffer address, gated
independently of the log_draws cvar, budget-capped at 8000 draws / 4096 buffers.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 20:35:20 +02:00
MechaCat02
877163792b [GPU] ship-capture: numbered per-press snapshots + capture every instance
Some checks failed
Orchestrator / Commit Message Validation (push) Has been skipped
Orchestrator / Lint (push) Failing after 2m10s
Orchestrator / Windows (x86-64) (push) Has been skipped
Orchestrator / Linux (x86-64) (push) Has been skipped
Orchestrator / Create Release (push) Has been skipped
F10 now writes xenia_ship_capture_NN.log (NN = press number) so several angles
can be captured in one run without overwriting, and de-dups by (vertex-buffer
address, c0..c2 WVP hash) instead of address alone -- the same part buffer
drawn at different transforms (two engine nacelles, each ship of a fleet) now
yields one record per distinct placement. Seen-set cap raised to 8192.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 19:15:43 +02:00
MechaCat02
3ba56a5005 [GPU] ship-capture: also dump VS float constants (the placement matrix)
Some checks failed
Orchestrator / Commit Message Validation (push) Has been skipped
Orchestrator / Lint (push) Failing after 1m51s
Orchestrator / Windows (x86-64) (push) Has been skipped
Orchestrator / Linux (x86-64) (push) Has been skipped
Orchestrator / Create Release (push) Has been skipped
The captured vertex BUFFER holds LOCAL positions (byte-identical to the .xpr) —
capital-ship parts are placed entirely in the vertex shader, not in the buffer.
So the per-part world/WVP matrix lives in the VS float constants. Extend the F10
capture to dump the first 48 vec4 from the SQ_VS_CONST base per draw; diffing two
parts isolates the world matrix (the camera VP block is shared across draws).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 10:18:40 +02:00
MechaCat02
87d02d59f1 [GPU] F10 ship-placement capture: dump world-space draw vertices for RE
One-shot snapshot for reversing capital-ship part placement. F10 arms
RequestShipCaptureFrame(); the next batch of draws is dumped to
xenia_ship_capture.log with each draw's guest vertex-buffer address, vertex/
index counts, and up to 64 WORLD-space vertex positions. Capital-ship parts are
transformed into world space before the draw (unlike stage geometry, which
matches the .xpr byte-for-byte), so these are ground truth for the assembled
placement; a reborn-side correlator affine-fits each decoded .xpr part to a
captured draw to recover its exact transform. De-duped by buffer address, gated
independently of the log_draws cvar, budget-capped at 8000 draws / 4096 buffers.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 09:49:45 +02:00
MechaCat02
201553aea3 Instrument: file.read offset tracer + XMA-PARAM probe (voice RE)
Some checks failed
Orchestrator / Commit Message Validation (push) Has been skipped
Orchestrator / Lint (push) Failing after 1m52s
Orchestrator / Windows (x86-64) (push) Has been skipped
Orchestrator / Linux (x86-64) (push) Has been skipped
Orchestrator / Create Release (push) Has been skipped
Additive, cvar-gated instrumentation used to reverse-engineer Project
Sylpheed's cutscene-voice storage (movie -> continuous sound-stream cue
region). Default-off; no behaviour change.

- event_log.cc: MaybeEmitFileRead emits `file.read` events with the real
  NtReadFile ByteOffset (from r10), length, handle->path, and buffer VA, so a
  sound.pNN read offset can be mapped to a sound.pak TOC entry.
  Gate: --phase_a_fileio_only=true --phase_a_event_log_path=<file>.
- cpu_flags.cc: phase_a_fileio_only cvar (+ xma_param_probe).
- xma_context_new.cc: XMA-PARAM probe in XmaContextNew::Decode (the decoder the
  title actually uses) logging ctx/buffer/read-offset/channels/packets/byte_size
  /signature — to confirm which bytes get decoded for a given cutscene.

Used to produce /tmp/rt_fileio.jsonl (RT01A playthrough), which cracked the
movie->voice mapping now implemented in sylpheed-reborn. Next: run the same
file.read trace on an unbound hokyu (e.g. hokyu_LS_s03A) to resolve which
VOICE_D the game streams for the 13 manifest-unbound resupply cutscenes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 06:56:45 +02:00
MechaCat02
ea9f037b82 [GPU,Kernel] RE instrumentation: per-buffer draw log + file-I/O guest map
Some checks failed
Orchestrator / Commit Message Validation (push) Has been skipped
Orchestrator / Lint (push) Failing after 2m35s
Orchestrator / Windows (x86-64) (push) Has been skipped
Orchestrator / Linux (x86-64) (push) Has been skipped
Orchestrator / Create Release (push) Has been skipped
Draw logger (command_processor.cc): also key each record on the index-buffer
guest base so distinct static index buffers (ship body vs hangar vs weapons) no
longer collapse to one record; dump the first decoded index VALUES + raw index
bytes (8-in-16/32 endian-corrected) and the stream-base hex, to pin an index
buffer's exact .xpr file offset and validate a mesh decode against ground truth.

File-I/O log (xboxkrnl_io.cc): --log_file_io cvar writes each distinct NtReadFile
as 'READ <name> off=.. len=.. -> guest=..' to xenia_re_files.log, giving the
file->guest-memory mapping. Cross-referenced with the draw log's buffer guest
addresses this resolves which .xpr (and where inside it) a mesh loaded from.
De-duped by (offset, guest), capped at 65536 entries.

Also ignore build-cross/ (Wine cross-build output).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 20:19:12 +02:00
MechaCat02
29efc0381a [GPU] log_draws: also dump vertex positions for file correlation
Extend the RE draw-logger to dump the first few vertex POSITIONS (read
from guest memory) under each draw. The f32 position bytes are identical
between the guest buffer and the on-disc .xpr (only f16 pairs are
rearranged on load), so these values can be grep'd for in a resource file
to locate a mesh whose in-file offset is otherwise unknown.

Validated: world-space stage geometry byte-matches Stage_*.xpr at exact
offsets. (Load-time-transformed meshes like the player ship don't match,
which is itself a useful finding.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 14:28:47 +02:00
MechaCat02
c6caa9e9c5 [GPU] Add log_draws: dump per-draw vertex declaration for RE
Reverse-engineering aid for decoding game mesh formats against GPU ground
truth. When the `log_draws` cvar is set, each distinct draw's primitive
type, index buffer (guest base / count / format / endianness), and full
per-stream vertex declaration (fetch-constant base + stride, and every
element's format + offset) is written to xenia_re_draws.log.

- command_processor.{h,cc}: CommandProcessor::LogDrawForRE(), no-op unless
  the cvar is set. De-dups by the vertex-declaration fingerprint (shader +
  primitive + element formats/offsets), so animated UI that redraws the
  same format into fresh buffers every frame collapses to one record --
  keeping it near-free (an earlier address-keyed de-dup flooded the log and
  stalled the GPU thread). Capped at 4096 distinct formats.
- pm4_command_processor_implement.h: call it from ExecutePacketType3Draw
  after IssueDraw (so the vertex shader has been analyzed). Backend-agnostic
  base path -- works for the Vulkan build.

Used to confirm Project Sylpheed's XBG7 mesh layout (triangle list, pos
f32x3 / normal f16x4 / uv f16x2, variable stride).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 14:28:47 +02:00
MechaCat02
3c8faf03f7 [Fix] threading_posix: reap thread handle once (mission-end freeze)
post_execution() runs on every successful Wait() and a Thread handle stays
signaled forever after exit, so multiple guest waits on the same exited handle
each call pthread_join() on an already-reaped pthread_t. On glibc that hangs
(recycled tid) -> the black-screen freeze at mission teardown. Guard reap with
an atomic exchange so it happens exactly once. Isolated from the audio/crash
WIP on phase-a-args-fileread; only this hunk, on stock 16e1eb8e2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 22:05:49 +02:00
goldislead
16e1eb8e28 [GPU] Clarify 64bpp clear register order
This resolves the TODO in GetColorClearShaderConstants about which 32 bit portion is in which register.

For 64bpp formats, RB_COLOR_CLEAR_LO holds the lower 32 bits of the packed clear value, and RB_COLOR_CLEAR holds the upper 32 bits.
2026-07-17 08:18:56 +02:00
MechaCat02
7b6902e08f [WIP] Audio/threading fixes + crash investigation; NEW ORACLE: crash is ours not the game
Some checks failed
Orchestrator / Commit Message Validation (push) Has been skipped
Orchestrator / Lint (push) Failing after 1m34s
Orchestrator / Windows (x86-64) (push) Has been skipped
Orchestrator / Linux (x86-64) (push) Has been skipped
Orchestrator / Create Release (push) Has been skipped
Snapshot for handoff. Contains the mission-audio + threading fixes and the
crash-investigation instrumentation (all diagnostic cvars default-OFF).

Fixes (behavioral):
- threading_posix.cc: reap-once guard on PosixCondition<Thread>::post_execution
  (double pthread_join at mission teardown -> fault loop -> audio death + freeze).
- xma_decoder.cc: work_event_->Set() in Pause() so the idle XMA worker observes
  paused_ and signals pause_fence_ (Pause() deadlock -> permanent audio death).
- audio_system / xma_context_master / xboxkrnl_audio / apu_flags / alsa: mission
  audio keepalive + guest_audio_flags + watchdogs.

Instrumentation (additive, default-off): xboxkrnl_debug cache-throw diag +
guest-catch dispatcher, xex_module PE/PDATA/EH scans, kernel_state mem_watch
(NOTE: mem_watch DEFAULTS TRUE -- an always-on host poll thread; prime crash suspect).

NEW ORACLE (see HANDOFF-crash-oracle-2026-07-16.md): stock 6e5b8324f built with
our toolchain + zero custom code = NO crash, NO sound-stop, plays the Ready Room.
=> the Ready-Room out_of_range crash is introduced by THESE changes, not the game
and not the (LTO-broken) build chain. Bisection plan + suspect ranking in the note.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 22:43:50 +02:00
NicknineTheEagle
c6298dd0e3 [XAM] Implemented game exit behavior in XamLoaderTerminateTitle 2026-07-16 11:48:30 +02:00
Gliniak
44e1a64f35 [XAM] Extend UI delay on close. Added delay to headless after notification sent.
Created thread to do async notification broadcast on close for headless.

This fixes game freeze in 534407FC
2026-07-16 11:17:16 +02:00
Gliniak
c2650c6e58 [Testing] Define full test paths, unify test results of vectors to unsigned type 2026-07-16 10:47:15 +02:00
Gliniak
7ff152a5a7 [X64] MAX_V128/MIN_V128 Proper NaN handling.
According to console. FFC00000->FFC00000 instead of FFFFFC00 as it was previously.

Test case: 415607E8 - "Evil Queen's Castle" level
2026-07-15 23:31:38 +02:00
The-Little-Wolf
51322122a5 [Xam/Info] - Implement XamFeatureEnabled
- Implement XamFeatureEnabled
2026-07-15 19:55:04 +02:00
Gliniak
deaf21e9e1 [CI] Fixed CI failure caused by incorrect variable check 2026-07-15 19:53:27 +02:00
The-Little-Wolf
2799234fbb [Xam/XMP] - Stub XMPGetNumSongsInTitlePlaylist
- Stub XMPGetNumSongsInTitlePlaylist
2026-07-15 09:18:46 +02:00
SaveEditors
e20f26963f [GPU] Initialize GPU registers to hardware reset defaults
RegisterFile zeroes the whole register file at construction, but a real
console comes up with non-zero values in several of the context
registers. If a game reads one of those before writing it, it gets 0
under emulation and the real default on hardware.

I read the reset values off a retail console (read-only) and checked
them against the AMD R6xx register reference and the r600g defaults in
Mesa, which set the same registers to the same values at context init.
Where the hardware read matches the driver default it's a genuine
power-on default, not leftover runtime state.

Left the tessellation levels out on purpose: they reset to 1.0f too, but
the backends do register + 1.0f, so seeding them here would change the
effective reset factor. That needs its own change.
2026-07-14 21:07:49 +02:00
Margen67
bad346717a [xb] Skip aarch64 submodule for x64 target 2026-07-14 03:24:00 -07:00
MechaCat02
0aa3eadca7 [Kernel] Add null-safe XThread::TryGetCurrentThread() accessor
The additive event_log extraction tracer needs a current-thread lookup that
returns nullptr instead of asserting when called from boot code before any
XThread exists (GetCurrentThread asserts on non-guest threads). New read-only
static accessor over the thread-local; no behaviour change. Fixes the native
Linux build of the phase-a args/file.read tracer.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 20:42:54 +02:00
MechaCat02
a40bc50159 [Phase A] Port args / file.read extraction tracing onto instrument-current
Bring the cvar-gated JSONL extraction tracer (event_log) forward from the
phase-a-tracing line so args/args_resolved + file.read events are
available alongside the GPU draw logging. Purely additive:

- src/xenia/kernel/event_log.{cc,h}: the tracer (auto-globbed into the
  kernel target). Tier 1 fills kernel.call args (raw r3..r10) +
  args_resolved.path under --phase_a_trace_args; Tier 2 resolves
  NtReadFile handle->path via the object table and emits file.read.
- cpu_flags: phase_a_event_log_path / _mem_writes / _trace_args /
  _hash_probe / kernel_emit_contention (all default-off).
- shim_utils.h: phase_a_bridge decl + EmitImportAndCallWithCtx/EmitReturn
  hooks in the active X::Trampoline, skipped when Enabled() is false.

Deliberately EXCLUDES the Tier-3 hash-probe HIR/emitter trap (the IPFB
name-hash was recovered statically, so the probe is moot). The
phase_a_hash_probe cvar is defined but inert.

Default-off => instrument-current behaviour byte-identical when unused.
Not yet compile-verified against instrument-current's toolchain.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 19:13:36 +02:00
MechaCat02
f970a5173f [GPU] log_draws: also dump vertex positions for file correlation
Some checks failed
Orchestrator / Commit Message Validation (push) Has been skipped
Orchestrator / Lint (push) Failing after 1m57s
Orchestrator / Windows (x86-64) (push) Has been skipped
Orchestrator / Linux (x86-64) (push) Has been skipped
Orchestrator / Create Release (push) Has been skipped
Extend the RE draw-logger to dump the first few vertex POSITIONS (read
from guest memory) under each draw. The f32 position bytes are identical
between the guest buffer and the on-disc .xpr (only f16 pairs are
rearranged on load), so these values can be grep'd for in a resource file
to locate a mesh whose in-file offset is otherwise unknown.

Validated: world-space stage geometry byte-matches Stage_*.xpr at exact
offsets. (Load-time-transformed meshes like the player ship don't match,
which is itself a useful finding.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 19:09:08 +02:00
MechaCat02
25eb17b91d [GPU] Add log_draws: dump per-draw vertex declaration for RE
Reverse-engineering aid for decoding game mesh formats against GPU ground
truth. When the `log_draws` cvar is set, each distinct draw's primitive
type, index buffer (guest base / count / format / endianness), and full
per-stream vertex declaration (fetch-constant base + stride, and every
element's format + offset) is written to xenia_re_draws.log.

- command_processor.{h,cc}: CommandProcessor::LogDrawForRE(), no-op unless
  the cvar is set. De-dups by the vertex-declaration fingerprint (shader +
  primitive + element formats/offsets), so animated UI that redraws the
  same format into fresh buffers every frame collapses to one record --
  keeping it near-free (an earlier address-keyed de-dup flooded the log and
  stalled the GPU thread). Capped at 4096 distinct formats.
- pm4_command_processor_implement.h: call it from ExecutePacketType3Draw
  after IssueDraw (so the vertex shader has been analyzed). Backend-agnostic
  base path -- works for the Vulkan build.

Used to confirm Project Sylpheed's XBG7 mesh layout (triangle list, pos
f32x3 / normal f16x4 / uv f16x2, variable stride).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 18:42:33 +02:00
MechaCat02
f10484834c [APU] Fix mission-audio silence: ALSA underrun keepalive + guest_audio_flags
Two Linux mission-audio fixes for Project Sylpheed (audio played in
intro/menu but died when a mission finished loading, never returning):

- alsa_audio_driver: when the guest stalls (mission "Preparing for Sortie"
  load) and the ring buffer empties, feed silence to keep the PCM alive
  instead of sleeping. Previously the small buffer drained, XRUN'd, and
  playback never recovered (matches the known "audio muted permanently").
- xconfig: make the guest speaker config a cvar (guest_audio_flags,
  default Digital Stereo) instead of hardcoded Dolby Digital surround.

(The practical fix on this box also needed log_mask=13 to stop kernel log
spam starving the audio pipeline during missions — config, not code.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 16:49:25 +02:00
MechaCat02
919e526fd8 [Fix] Screenshot: null-check presenter before use (fixes F12 crash)
TakeScreenshot() called GetGraphicsSystemPresenter()->CaptureGuestOutput()
and only tested the pointer for null AFTER dereferencing it, so taking a
screenshot with no live presenter dereferenced null and crashed. Check for
null first, and re-enable notifications on the capture-failure path too.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 15:44:11 +02:00
MechaCat02
2233fae235 [Build] Guard ThinLTO behind XENIA_ENABLE_LTO for memory-constrained hosts
Release enables -flto=thin unconditionally; the ThinLTO link spikes memory
past what a 15GB box can handle. Gate it behind an option (default ON, so
normal/CI builds are unchanged); pass -DXENIA_ENABLE_LTO=OFF to build
Release without the LTO memory spike.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 15:43:23 +02:00
MechaCat02
f81e90f03a audit: add poll-based arbitrary guest-VA memory-watch cvars
Adds audit_mem_watch_addr (comma-separated hex guest VAs) and
audit_mem_watch_size (1/2/4/8 bytes) to watch arbitrary guest memory
locations at runtime. Once per vblank, GraphicsSystem::MarkVblank()
reads each VA big-endian via Memory::TranslateVirtual and emits an
XELOGKERNEL "AUDIT-MEM-WATCH" line on every value change vs the prior
frame (same greppable log stream as AUDIT-HLC / AUDIT-MEM-READ).

Mechanism is poll / value-change: it captures WHEN a value changes
(vblank index) but NOT the writer guest-PC; pair with audit_jit_prolog_pc
on a suspected writer to recover the PC. Heap VAs vary per run/engine,
so they are supplied at runtime via cvar/CLI. Default empty => disabled
=> zero overhead and no emulation-behavior change.

Validated against the controller vsync "clock A" field (+0x58): the
watch fires once per change, advancing monotonically in lockstep with
vblanks. Game still boots and runs muted with the cvar unset.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 15:43:23 +02:00
MechaCat02
07f09f8fe0 [Audit] round 24: signal-path probes (XEvent::Set + IO + NtDuplicateObject)
Adds AUDIT-HLC probes for every path that signals an XEvent or duplicates
its handle, so the round-23 puzzle (waits completing without visible
NtSetEvent/KeSetEvent) can be cross-referenced by underlying X_KEVENT VA
and by primary handle.

  * NtCreateEvent_inner: always log kevent_va for handle/VA cross-ref
  * XEvent::Set: universal hook prints primary handle + kevent_va + LR;
    catches all paths into the event regardless of shim used
  * NtSetEvent: extended with PPC back-chain walk for caller's guest_lr
    (same idiom as the wait probe), so the signaler function is
    immediately identifiable
  * NtReadFile / NtReadFileScatter / NtWriteFile: log when signal_event
    path fires ev->Set() inline at IO completion (bypasses both
    NtSetEvent and KeSetEvent shims)
  * CompleteOverlappedEx (kernel_state.cc): log when overlapped event
    is signaled at completion
  * NtDuplicateObject: log src/dst handle pair (round-24 confirmed silph
    event is dup'd before signaling, explaining the handle-mismatch
    puzzle between NtSetEvent's `handle=` argument and XEvent::Set's
    primary handle())

Result: silph wait at sub_821CB030+0x1B0 on handle F80000A0 is signaled
via NtSetEvent on handle F80000A8 (the duplicate created by sub_8245D9D8
at lr=0x82450DF4 inside the worker chain sub_82450A28 ← 0xA68 ← 0xB68).
Verdict A confirmed: signaler exists and is reachable, but round-17.β
missed it because the search keyed on the wrong handle. No bypass path;
the worker chain is identical between canary and ours-impl, but ours
signals different handles (work-item queue divergence at sub_82452DC0).

Audit-only diagnostic. No semantic changes. Net delta +86 LOC across 7
files. All probes gated on the existing audit_handle_lifecycle cvar
(default off). Tested via Sylpheed boot (35 s, 4 MB log). Audit-handle-
lifecycle-probes branch, local-only.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-07-12 15:43:23 +02:00
MechaCat02
4670afbeb5 [Audit] --audit-r3-dump-bytes: dump N bytes at r3 when probe fires
AUDIT-059 round 15 — diagnostic. New cvar `audit_jit_prolog_r3_bytes`
(default 64 = existing behaviour, capped at 256, rounded up to 16B
multiple) controls how many bytes are dumped at host(r3) when the
audit_jit_prolog_pc probe fires. Set to 80 to capture audit-051's
stack-local struct at sub_82452DC0's r31+96.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-07-12 15:43:23 +02:00