This repository has been archived on 2026-09-16. You can view files and clone it. You cannot open issues or pull requests or push a commit.
MechaCat02 ac20afb1a9 analysis: complete, correct and faster XEX static analysis
Reworks the `dis --db` static-analysis pipeline. Everything here is derived
from the XEX alone; no run traces.

Completeness
- `.pdata` is now authoritative for function boundaries rather than a hint:
  all 23,073 linker-declared RUNTIME_FUNCTIONs are emitted and `end_address`
  is the declared end, not a prologue-walk guess. Adds tail-call targets
  (with a preceding-terminator fallback for the ~450 KB of .text `.pdata`
  leaves unclaimed): 12,156 -> 25,676 functions.
- New `jumptables.rs`: 394 switch tables / 6,356 case edges recovered from
  941 `bctr` sites. Table words are flagged `instructions.is_data` and kept
  out of the xref pass, and render as `.long` with case labels in `-o`.
- New `rtti.rs`: RTTI is not stripped in this title. 611 type descriptors,
  653 locators, 1,491 base-class records, giving a real inheritance graph
  with mdisp/pdisp/vdisp. Verified against the STL:
  std::out_of_range : std::logic_error : std::exception.
- New `xdbf.rs` + `xenia-xex/resources.rs`: the XDBF/SPA package the XEX
  names via its resource table -- 24 achievements (1000G, matching the
  project's independently derived figure), 888 localized strings across 6
  locales, 25 PNGs, title name/id/version. Located through the entry table,
  not a magic scan; a scan finds a phantom 7th string table where the entry
  table declares 6, which shifts every scan-derived language index.
- Strings scan extended to `.data`; metadata grows from 5 keys to 41.

Correctness
- `instructions.function` was a rolling window -- set at each function start
  and never cleared -- so every word in a `.pdata` gap was attributed to the
  preceding function. 55,227 rows were wrong and the bogus 100% attribution
  hid the gap. Now cleared at `end_address`; 0 wrong, honest 97.0%.
- Shift-JIS scanning accepted half-width katakana, which turned IEEE-754
  float tables into "Japanese" (`3f 66 66 66` = 0.9f reads as "fff"), and
  emitted escaped bytes rather than text. 837 mostly-noise rows -> 115 real
  strings, decoded via encoding_rs, with resync so a run starting one byte
  early reports the true address instead of mangling the first character.
- `indirect_dispatch_candidates` was a cross product, not a resolution: at
  vptr_offset 0 the (offset, slot) match hits nearly every class, so 6,556
  of 6,983 sites produced 1.80M of 1.81M rows (one site claiming 764
  callees) and the derived ind_call edges were 84% of the xrefs table. Adds
  `--max-indirect-candidates` (default 16); over-ceiling sites keep their
  row with a truthful count and a new `truncated` flag but emit no candidate
  rows and no xrefs. 1.81M -> 4,199 candidates, xrefs 2.16M -> 357k.
- xenia-xex: `TLS_INFO` and `DEFAULT_STACK_SIZE` header keys were swapped.
  `get_stack_size()` would have returned the TLS descriptor's file offset;
  it has no callers today, so nothing regressed.

Performance
- DuckDB autocommits per statement, so row-at-a-time INSERT paid a
  transaction + WAL flush per row: 221k rows took 20 minutes and the 1.8M
  candidate rows ~59 more, for an 81-minute build that never finished. One
  big transaction fixes the time but not the cause (uncommitted state grew
  to 16.6 GB RSS). Every table above a few thousand rows now uses the
  Appender; each converted sink documents why its key cannot collide or
  dedupes first. 2m20s wall, 486 MB peak, DB 635 MB -> 318 MB.

Also: 6 new SQL views, `zq.py` gains switch/switches/classes/class/str/
xdbf/ach, and an `analysis_report` example that runs the passes without
building a database. 80 tests pass (29 new).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 19:57:14 +02:00

xenia-rs

Rust reimplementation of the Xbox 360 emulator xenia, focused on reverse-engineering and preservation rather than full-speed play. The initial target is Project Sylpheed — Arc of Deception; getting the title disassembled, traced, and far enough into its init path to understand its engine.

Heavy cross-reference to xenia-canary for CPU context setup, kernel export behavior, and XEX loading semantics.

Status

  • XEX loader — XEX2 header parsing, LZX decompression, AES decryption, PE section parsing.
  • VFS / XISO — XGD2 dual-layer disc images (with the 0x0FD90000 partition offset).
  • PPC interpreter — 200+ opcodes, PowerPC 32/64-bit GPR/FPR, VMX128 decoding.
  • Static analyzer — function discovery (.pdata ground truth + prolog/epilog heuristics), cross-references, labels, save/restore helper detection, switch/jump-table recovery, MSVC RTTI class + vtable + inheritance recovery, string scan, EH scope tables, assembly text + DuckDB database output.
  • Kernel HLE — minimal subset driving Project Sylpheed: ~170 xboxkrnl + xam exports (critical sections, events, TLS, virtual memory, Vd stubs, XAM input/user/content).
  • Debugger — in-memory step/break, SQLite execution + import-call + branch tracing.

Not yet: GPU (xenos/xe-shader), APU audio, HID, kernel scheduler, full threading, exception delivery.

Workspace

crates/
  xenia-types       # shared primitive types, bitflags
  xenia-memory      # guest memory, paged allocator, page table
  xenia-cpu         # PPC decoder, interpreter, context
  xenia-xex         # XEX2 loader, PE parser, LZX, AES
  xenia-vfs         # XISO / disc-image reader
  xenia-kernel      # HLE kernel state, exports, XAM
  xenia-gpu         # (stub) Xenos command processor
  xenia-apu         # (stub) XAudio
  xenia-hid         # (stub) XInput
  xenia-debugger    # in-memory trace, breakpoints, step modes
  xenia-analysis    # function/xref analysis, assembly formatter, SQLite DbWriter
  xenia-app         # `xenia-rs` CLI binary

CLI

Build:

cargo build --release

The binary xenia-rs accepts XEX2 files or ISO / XISO disc images as input (the loader auto-detects discs and extracts default.xex).

info / browse / disasm

Quick header / disc / first-N-instructions inspection. See --help.

extract — unpack PE + metadata

xenia-rs extract <xex-or-iso> [-o <out-dir>] [--db <sqlite-path>]

Writes <name>.pe (decompressed/decrypted PE image) and <name>.xex.json (header metadata). With --db, also emits a SQLite database containing the base tables: metadata, sections, imports.

dis — full disassembly

xenia-rs dis <xex-or-iso> [-o <asm-file>] [--db <sqlite-path>] [--quiet]

Runs the full static-analysis pipeline and produces:

  • assembly text to stdout or -o <file> (unless --quiet)
  • optional DB with the base tables + analysis tables:
group tables
disasm instructions (is_data flags jump-table words), functions, labels, xrefs, pdata_entries
switches jump_tables, jump_table_entries, data_in_code
C++ / RTTI rtti_type_descriptors, rtti_locators, rtti_base_classes, vtables, methods, classes, demangled_names
other strings, eh_funcinfo/eh_unwind_map/eh_try_blocks, function_pointer_arrays, indirect_dispatch_*, vptr_writes, tls_info/tls_callbacks

--analyze sql additionally creates query views: v_switch_cases, v_class_hierarchy, v_class_methods, v_function_strings, v_call_graph, v_reachability_from_entry, v_indirect_reachability_from_entry, v_branch_xrefs, v_function_first_instruction, v_imports_called.

Function boundaries come from the linker's .pdata RUNTIME_FUNCTION table where it covers the function (exact), and from prologue/epilogue walking elsewhere (functions.pdata_validated says which). Rows in instructions with is_data = true are jump-table words, not code — filter them out of any instruction-level statistic.

To iterate on an analysis pass without paying for a full DB build:

cargo run --release -p xenia-analysis --example analysis_report -- <xex>

which runs the passes and prints their headline counts in seconds.

exec — interpret with tracing

xenia-rs exec <xex-or-iso> [-n <max-instrs>] [--db <sqlite-path>]
             [--trace-instructions] [--trace-imports] [--trace-branches]

Loads the title, initializes CPU state per xenia-canary, intercepts import thunks with HLE kernel calls, and interprets from the entry point. Without -n, runs until halt/fault. With --db, produces a DB that is a superset of dis --db plus opt-in trace tables:

flag table rows
--trace-instructions exec_trace one row per interpreted instruction (PC, r3/r4, LR, SP)
--trace-imports import_calls one row per kernel/XAM call (module, ordinal, args)
--trace-branches branch_trace taken branches classified as call/return/jump/branch

Cumulative DB layering

Each command's DB is a superset of the previous. A single xenia-rs exec <iso> --db full.db --trace-instructions --trace-imports --trace-branches produces the full picture in one pass — base tables, complete static disassembly, and runtime traces correlatable by address/cycle.

Performance knobs

  • XENIA_DB_BATCH_SIZE — rows per streaming commit / trace-buffer flush (default 100_000). Lower values reduce memory use; higher values reduce fsync overhead on slow disks.

The DB writer uses journal_mode=OFF, synchronous=OFF, locking_mode=EXCLUSIVE and commits in batches; no ANALYZE is run at finalize. Indices are created after bulk insertion with progress messages.

Example queries

-- Top 20 kernel functions called during early init
SELECT name, COUNT(*) FROM import_calls GROUP BY name ORDER BY 2 DESC LIMIT 20;

-- All basic-block leaders (targets of taken branches) not already labelled
SELECT DISTINCT bt.target
FROM branch_trace bt LEFT JOIN labels l ON l.address = bt.target
WHERE l.address IS NULL;

-- Correlate a traced call site with its static disassembly
SELECT et.cycle, i.disasm, i.ext_disasm
FROM exec_trace et JOIN instructions i ON i.address = et.address
WHERE et.address = 0x824AB748 ORDER BY et.cycle;

License

BSD-3-Clause, matching upstream xenia.

Description
No description provided
Readme 11 MiB
Languages
Rust 98%
WGSL 1.7%
Python 0.3%