This repository has been archived on 2026-09-16. You can view files and clone it. You cannot open issues or pull requests or push a commit.
MechaCat02 b4f19f1f48 harvest: import-thunk naming + schema-wide foreign keys (was uncommitted)
This was found UNCOMMITTED in the working tree on 2026-09-14, four days after
it was written (file mtimes 2026-09-10 21:01-21:23). It exists in exactly one
place and that place was not git:

  imports.rs / import_address   PR #32 (lifted tool): absent
                                iterate-4A (committed): absent
                                working tree: present

CONSOLIDATION.md Phase 5 ends with "delete the local clone" and Phase 7 drops
the emulator, so this had a deletion scheduled against it. Phase 1's own rule
is the reason to commit it: "a retired branch that was never pushed is not
retired -- it is deleted, quietly, by someone who thought it was safe." The
same applies, more sharply, to work that was never committed at all: Phase 0's
gate walks refs/heads, so a dirty working tree is invisible to it. Canary's
dirt was secured because someone knew it was there; this was not.

Two separable pieces of work:

1. Import-thunk recognition (`imports.rs`, 338 lines new).
   An XEX import is not a PLT jump: the linker emits a four-word thunk whose
   first two words are import RECORDS, rewritten by the loader at module load.
   On disc they are still records, so a PowerPC-only decoder renders them as
   two meaningless `.long`s before an indirect branch. This maps every word of
   every thunk -- and every direct branch targeting one -- back to its `imports`
   row, so disassembly can say `xboxkrnl.exe::RtlEnterCriticalSection` instead
   of `.long 0x01010194`. Shape-validated, not trusted: an entry is indexed
   only when the four words it points at actually have the thunk shape.
   Adds `instructions.import_address` (FK onto `imports.address`) +
   `import_role` ('record' | 'thunk' | 'call', NULL iff import_address IS NULL)
   + an index. `zq.py` gains `imp` and `impcalls`, and `dis` names imports.

2. Sixteen schema-wide foreign keys, declared wherever a column is *derived
   from* another table. CREATE TABLE and insertion order become load-bearing
   (a parent must exist before any child that references it);
   `write_analysis_results` documents its order.

   `functions` is deliberately NOT an FK parent, and the golden test now
   asserts that: DuckDB implements UPDATE as delete+insert, so an inbound FK
   would make `functions.name` un-updatable and break apply_re_symbols.sql,
   which re-applies RE symbol names after every regeneration.

Committed exactly as found -- not cleaned up, not rebased. `cargo check` was
still running when this was committed; preserving the tree does not depend on
it compiling, and editing it first would have destroyed the thing being
preserved.

Not ported into the lifted `sylpheed-xexdb` crate (PR #32). That is a separate
decision for the human: this repo is being retired, and the feature would have
to be carried across deliberately.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 20:12:50 +02:00

xenia-rs

Rust reimplementation of the Xbox 360 emulator xenia, focused on reverse-engineering and preservation rather than full-speed play. The initial target is Project Sylpheed — Arc of Deception; getting the title disassembled, traced, and far enough into its init path to understand its engine.

Heavy cross-reference to xenia-canary for CPU context setup, kernel export behavior, and XEX loading semantics.

Status

  • XEX loader — XEX2 header parsing, LZX decompression, AES decryption, PE section parsing.
  • VFS / XISO — XGD2 dual-layer disc images (with the 0x0FD90000 partition offset).
  • PPC interpreter — 200+ opcodes, PowerPC 32/64-bit GPR/FPR, VMX128 decoding.
  • Static analyzer — function discovery (.pdata ground truth + prolog/epilog heuristics), cross-references, labels, save/restore helper detection, switch/jump-table recovery, MSVC RTTI class + vtable + inheritance recovery, string scan, EH scope tables, assembly text + DuckDB database output.
  • Kernel HLE — minimal subset driving Project Sylpheed: ~170 xboxkrnl + xam exports (critical sections, events, TLS, virtual memory, Vd stubs, XAM input/user/content).
  • Debugger — in-memory step/break, SQLite execution + import-call + branch tracing.

Not yet: GPU (xenos/xe-shader), APU audio, HID, kernel scheduler, full threading, exception delivery.

Workspace

crates/
  xenia-types       # shared primitive types, bitflags
  xenia-memory      # guest memory, paged allocator, page table
  xenia-cpu         # PPC decoder, interpreter, context
  xenia-xex         # XEX2 loader, PE parser, LZX, AES
  xenia-vfs         # XISO / disc-image reader
  xenia-kernel      # HLE kernel state, exports, XAM
  xenia-gpu         # (stub) Xenos command processor
  xenia-apu         # (stub) XAudio
  xenia-hid         # (stub) XInput
  xenia-debugger    # in-memory trace, breakpoints, step modes
  xenia-analysis    # function/xref analysis, assembly formatter, SQLite DbWriter
  xenia-app         # `xenia-rs` CLI binary

CLI

Build:

cargo build --release

The binary xenia-rs accepts XEX2 files or ISO / XISO disc images as input (the loader auto-detects discs and extracts default.xex).

info / browse / disasm

Quick header / disc / first-N-instructions inspection. See --help.

extract — unpack PE + metadata

xenia-rs extract <xex-or-iso> [-o <out-dir>] [--db <sqlite-path>]

Writes <name>.pe (decompressed/decrypted PE image) and <name>.xex.json (header metadata). With --db, also emits a SQLite database containing the base tables: metadata, sections, imports.

dis — full disassembly

xenia-rs dis <xex-or-iso> [-o <asm-file>] [--db <sqlite-path>] [--quiet]

Runs the full static-analysis pipeline and produces:

  • assembly text to stdout or -o <file> (unless --quiet)
  • optional DB with the base tables + analysis tables:
group tables
disasm instructions (is_data flags jump-table words), functions, labels, xrefs, pdata_entries
switches jump_tables, jump_table_entries, data_in_code
C++ / RTTI rtti_type_descriptors, rtti_locators, rtti_base_classes, vtables, methods, classes, demangled_names
other strings, eh_funcinfo/eh_unwind_map/eh_try_blocks, function_pointer_arrays, indirect_dispatch_*, vptr_writes, tls_info/tls_callbacks

--analyze sql additionally creates query views: v_switch_cases, v_class_hierarchy, v_class_methods, v_function_strings, v_call_graph, v_reachability_from_entry, v_indirect_reachability_from_entry, v_branch_xrefs, v_function_first_instruction, v_imports_called.

Function boundaries come from the linker's .pdata RUNTIME_FUNCTION table where it covers the function (exact), and from prologue/epilogue walking elsewhere (functions.pdata_validated says which). Rows in instructions with is_data = true are jump-table words, not code — filter them out of any instruction-level statistic.

To iterate on an analysis pass without paying for a full DB build:

cargo run --release -p xenia-analysis --example analysis_report -- <xex>

which runs the passes and prints their headline counts in seconds.

exec — interpret with tracing

xenia-rs exec <xex-or-iso> [-n <max-instrs>] [--db <sqlite-path>]
             [--trace-instructions] [--trace-imports] [--trace-branches]

Loads the title, initializes CPU state per xenia-canary, intercepts import thunks with HLE kernel calls, and interprets from the entry point. Without -n, runs until halt/fault. With --db, produces a DB that is a superset of dis --db plus opt-in trace tables:

flag table rows
--trace-instructions exec_trace one row per interpreted instruction (PC, r3/r4, LR, SP)
--trace-imports import_calls one row per kernel/XAM call (module, ordinal, args)
--trace-branches branch_trace taken branches classified as call/return/jump/branch

Cumulative DB layering

Each command's DB is a superset of the previous. A single xenia-rs exec <iso> --db full.db --trace-instructions --trace-imports --trace-branches produces the full picture in one pass — base tables, complete static disassembly, and runtime traces correlatable by address/cycle.

Performance knobs

  • XENIA_DB_BATCH_SIZE — rows per streaming commit / trace-buffer flush (default 100_000). Lower values reduce memory use; higher values reduce fsync overhead on slow disks.

The DB writer uses journal_mode=OFF, synchronous=OFF, locking_mode=EXCLUSIVE and commits in batches; no ANALYZE is run at finalize. Indices are created after bulk insertion with progress messages.

Example queries

-- Top 20 kernel functions called during early init
SELECT name, COUNT(*) FROM import_calls GROUP BY name ORDER BY 2 DESC LIMIT 20;

-- All basic-block leaders (targets of taken branches) not already labelled
SELECT DISTINCT bt.target
FROM branch_trace bt LEFT JOIN labels l ON l.address = bt.target
WHERE l.address IS NULL;

-- Correlate a traced call site with its static disassembly
SELECT et.cycle, i.disasm, i.ext_disasm
FROM exec_trace et JOIN instructions i ON i.address = et.address
WHERE et.address = 0x824AB748 ORDER BY et.cycle;

License

BSD-3-Clause, matching upstream xenia.

Description
No description provided
Readme 11 MiB
Languages
Rust 98%
WGSL 1.7%
Python 0.3%