Closes the four remaining deferred follow-up items in one bundle.
All four are smaller-scope and additive; lockstep determinism
unaffected (analyzer-only changes).
## M9.5 — __CxxFrameHandler scope-table parsing
- New `xenia_analysis::eh_scope` module. Magic-scans .rdata for the
three documented MSVC FuncInfo signatures (0x19930520/21/22) on
4-byte alignment. Each match is parsed as the documented struct
(BE u32 fields), with sanity caps on max_state / n_try_blocks /
pointer validity.
- Walks pUnwindMap (UnwindMapEntry, 8 bytes) and pTryBlockMap
(TryBlockMapEntry, 20 bytes) into one row each.
- New tables eh_funcinfo, eh_unwind_map, eh_try_blocks.
- Sylpheed yield: 2,588 FuncInfo (all version 0x19930522) /
10,019 unwind entries / 315 try-blocks.
## M11.5 — Static-init driver chain detection
- New `xenia_analysis::static_init` module. Walks every function
looking for the canonical _initterm loop: lwz cursor; mtctr;
bcctrl; addi cursor, cursor, 4 bounded by a compare against another
constant register. Extracts (array_start, array_end) and reads
the array.
- Reuses `function_pointer_arrays` table — drivers' arrays land with
kind='static_init' (replacing M11's prologue-heuristic output where
the structurally-grounded pattern fires).
- Sylpheed yield: 0 drivers detected — the binary's static-init
structure does not match the canonical CRT loop. Infrastructure
ready; future M11.6 can relax.
## VMX vector-store xrefs (M6 follow-up)
- Adds AltiVec/VMX X-form load/store XOs to the M6 opcode-31
dispatch: lvx/lvxl/lvebx/lvehx/lvewx (reads) and
stvx/stvxl/stvebx/stvehx/stvewx (writes), all addr_mode=
'x_form_indexed'. Static resolution still requires both rA and rB
constant.
- Sylpheed yield: 110 newly-detected stvx writes.
## Shift_JIS + UTF-8 localised-string detection (M7 follow-up)
- Extends `xenia_analysis::strings::analyze` with scan_shift_jis (JIS
X 0208 lead/trail byte ranges + half-width katakana pass-through)
and scan_utf8 (2- and 3-byte sequences). At least one multi-byte
unit required so pure-ASCII strings aren't double-counted.
- SJIS bytes rendered as \xHH escapes for diagnostic readability;
full SJIS→UTF-8 decoding deferred.
- Sylpheed yield: 790 Shift_JIS strings (Japanese debug + UI text)
+ 39 UTF-8.
## Tests
- +2 EH (parses_minimal_funcinfo_v0, rejects_bogus_max_state)
- +2 static_init (detects_canonical_initterm_loop, rejects_function_without_pattern)
- +2 strings (detects_shift_jis_string, detects_utf8_multibyte_string)
Tests 649→655 (+6 unit tests). DB schema golden + write_analysis_results
signature updated for new EH parameter.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the dominant case M5 could not resolve — `lwz vt, off(this);
lwz fn, slot(vt); mtctr; bcctrl` (real C++ dispatch). Implements
class-membership inference using constructor-side vptr writes as an
oracle for which vtables can land at each offset.
## Algorithm
Phase 1 — vptr-write scan: walk every function with the existing
lis+addi register tracker. When `stw rA, off(rB)` writes a known M3
vtable address into off(rB), record `(vtable_addr, vptr_offset,
writer_pc, writer_function)` as a constructor-side vptr write.
Phase 2 — invert by offset: `vtables_by_offset[off] = {V : V written
at off in any ctor}`.
Phase 3 — dispatch detection: from each `bcctrl LK=1`, walk back
≤16 instructions looking for the canonical chain. Bail on register
clobber, branch, or label (basic-block) boundary.
Phase 4 — edge emission: for `(dispatch_pc, vptr_off, slot)`, emit one
`xrefs.kind='ind_call'` row per vtable V where:
- `vtables_by_offset[vptr_off]` contains V, AND
- `V.length > slot` (V actually has a method at that slot)
Multi-candidate sites (the common case at offset 0) are an
over-approximation; downstream queries filter to single-candidate sites
for high confidence:
`WHERE candidate_count=1` in `indirect_dispatch_sites`.
## Schema
NEW TABLES:
- `vptr_writes(writer_pc, vtable_address, vptr_offset, writer_function)`
- `indirect_dispatch_sites(dispatch_pc PK, vptr_offset, slot, candidate_count)`
- `indirect_dispatch_candidates(dispatch_pc, vtable_address, method_address)`
NEW INDICES on vtable_address / vptr_offset / method_address /
(vptr_offset, slot) for fast joins.
## Sylpheed yield
- 567 vptr writes / 214 vtables / 29 offsets (offset 0 = 88%).
- 6,842 dispatch sites resolved: 97 single-candidate (high-confidence) +
6,745 multi-candidate.
- 687,963 ind_call xref rows.
- 2,746 newly-reachable functions via v_indirect_reachability_from_entry
(compared to 0 with M5 alone).
- Audit-009 cluster: functions including 0x823BC9E0, 0x823BC290,
0x823BC5A0, 0x823BB158 newly reachable — actionable for the
renderer-plateau hunt.
Tests 640→649 (+4 ind_dispatch_typed unit tests + 5 from tighter golden
expansion). Schema golden + write_analysis_results signature updated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five LOW-priority milestones bundled. Total ~700 LOC across 11 files.
## M9 — has_eh derived from pdata.flags exception bit
- New `functions.has_eh BOOLEAN NOT NULL` column. Derived from M1's
already-parsed `pdata.flags` (bit 31 of the packed word — the
exception-handler-present flag, distinct from bit 30 which is the
always-1 32-bit-code flag). Index idx_functions_has_eh.
- Sylpheed: 2,975 of 23,073 pdata-validated functions have EH (12.9%).
## M10 — .tls section / IMAGE_TLS_DIRECTORY32 parser
- New `xenia_xex::tls::parse_tls` parses the directory + zero-terminated
callback array. Returns None when the binary has no .tls section.
- New `tls_info` (singleton row) + `tls_callbacks(slot, address)` tables.
- New `DbWriter::write_tls()` no-ops on None.
- Sylpheed has no .tls section → 0 rows; infra ready for binaries with
__declspec(thread).
## M8 + M11 — function_pointer_arrays (dispatch tables + static initialisers)
- New `xenia_analysis::funcptr_arrays::analyze` widens M3's vtable scan:
detects runs of ≥2 function pointers in .rdata and classifies each as
`vtable` (M3 re-emit), `dispatch_table` (M8), or `static_init` (M11)
via a constructor-prologue heuristic (mfspr + small stwu).
- New tables `function_pointer_arrays(address PK, length, kind)` and
`function_pointer_array_entries(array_address, slot, function_address)`.
- Sylpheed: 722 vtables + 388 dispatch_tables = 1,110 arrays / 6,347 slots.
0 static_init detected (Sylpheed's ctors don't all match the
conservative heuristic; M11.5 future work can chain via the entry-
point's static-init driver).
## M12 — --lr-trace runtime canary-diff harness
- New CLI `exec --lr-trace=PC[,PC,...]` and `--lr-trace-out=PATH` flags.
Symbolic resolution (Class::method, Class::*) via M4 lookup. Env vars
XENIA_LR_TRACE / XENIA_LR_TRACE_OUT also work.
- New `KernelState::lr_trace_pcs` + `lr_trace_writer` + helper
`fire_lr_trace_if_match(hw_id)` invoked from the per-instr probe slot.
- JSONL output: pc/tid/hw/cycle/r3/r4/r5/r6/lr — superset of what
xenia-canary's --log_lr_on_pc patch emits, with a cycle counter for
cross-run reproducibility. Diff-friendly via `jq`.
- Lockstep digest unaffected: smoke test on entry-point PC fires once
with cycle=0/lr=BCBCBCBC/all-GPR-zero (correct initial state).
Tests 636→640 (+2 TLS tests, +2 funcptr_arrays tests). Schema golden
updated for new tables + has_eh column. Lockstep determinism preserved
(instructions=2000005 ×2 reruns identical).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds finer-grained addressing-mode classification to every data xref row
plus new dispatch for instruction families not previously emitted:
- New `xrefs.addr_mode VARCHAR NULL` column. NULL for control-flow edges
(call / ind_call / j / br); one of d_form / lis_addi / lis_ori /
multiword / x_form_indexed / x_form_byterev / atomic / dcbz for data
edges. Index idx_xrefs_addr_mode.
- New `xenia_analysis::xref::AddrMode` enum + Xref::addr_mode field.
- Opcode 46/47 (lmw/stmw) expand to one xref per slot — D-form multi-word
load/store now resolves all (32-rS) consecutive addresses.
- Opcode 31 X-form dispatch — stwx/stbx/sthx/stwux/stbux/sthux/stdx/stdux,
lwzx/lbzx/lhzx/lhax/lwzux/lbzux/lhzux/lhaux/ldx/ldux,
stwcx./stdcx. (atomic),
stwbrx/sthbrx/lwbrx/lhbrx (byte-reverse),
dcbz (cache-line clear).
- X-form rows are emitted ONLY when both rA and rB resolve to known
constants (rare but present); the dominant runtime-indexed pattern
remains correctly skipped.
Sylpheed yield (regen on master + merge):
- 442 newly-detected x_form_indexed reads (lwzx/lhzx into static tables).
- 40 newly-detected atomic writes (stwcx./stdcx. with resolvable address).
- 28,834 lis_addi refs, 18,485 d_form reads, 3,288 d_form writes — every
pre-existing data row now tagged.
- 0 multiword / dcbz / byterev (these instructions exist but aren't on
lis+addi-tracked code paths).
Tests 633→636 (+3 xref unit tests covering AddrMode tag uniqueness,
data-edge addr_mode round-trip, control-edge None invariant). Schema
golden updated (xrefs gains addr_mode column).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two MEDIUM milestones bundled (both opportunistic per plan; both small).
## M5 — indirect-dispatch reachability
- `xenia_analysis::indirect`: per-basic-block register tracker over each
detected function. Recognises the canonical static-vtable pattern
`lis+addi → lwz off(rA) → mtctr → bcctrl` where rA holds a known M3
vtable address. Emits one `Xref { kind: IndirectCall }` per resolvable
bcctrl site.
- PowerPC ABI awareness: `bl`-style calls clobber volatile r0..r12 + ctr
but preserve non-volatile r13..r31, so a vtable pointer parked in r30/r31
before a call survives.
- Label-based basic-block boundaries kill register state — bounds
false-positive risk for jump-IN paths.
- New `XrefKind::IndirectCall` variant (DB tag `'ind_call'`).
- New SQL view `v_indirect_reachability_from_entry` — strict superset of
`v_reachability_from_entry`, taking `ind_call` edges in the BFS.
Sylpheed yield: 0 edges detected. The binary's 1,001 static lis+addi
references into vtables are nearly all constructor-side vptr writes, not
dispatches; real method dispatch goes through `this->vptr` which requires
alias analysis we explicitly don't do. Documented in SCHEMA.md as the
expected limitation. Three unit tests cover the synthetic-correctness path.
## M7 — string / constant-pool detection
- `xenia_analysis::strings`: scans `.rdata` for runs of ≥ 6 printable
ASCII bytes (NUL-terminated) and ≥ 6 UTF-16LE code units (basic-plane
printable ASCII, NUL u16 terminator).
- New `strings(address PK, encoding, length, content)` table + encoding index.
- Implicit cross-ref via existing `xrefs.kind='ref'` rows whose target
matches a strings.address.
Sylpheed yield: 6,311 ASCII strings (including embedded HLSL shader source
and AS_CB_SURFACE_SWIZZLE_* assertion strings). 9,132 lis+addi sites
cross-reference detected strings — names source PCs near each string in
one query. Four unit tests cover encoding detection, NUL termination, and
short-run rejection.
Tests 626→633 (+3 indirect, +4 strings).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CLI extension only — no schema change. Adds symbolic resolution for
--pc-probe / --branch-probe / --ctor-probe tokens:
- `0xADDR` / `2186674160` — numeric (current behavior, no DB load).
- `Class::method` — joins classes × methods × demangled_names.
- `Class::*` — joins classes × methods (all slots).
- `function_name` — falls back to functions.name for free functions /
saverestore stubs / labels.
New `xenia_analysis::lookup::resolve_probe_token(db_path, token)` opens the
DB read-only ONLY when a token is non-numeric, so legacy numeric flows pay
no IO. New `--probe-db PATH` flag (or `XENIA_PROBE_DB` env / default
`sylpheed.db` next to the .iso) selects the DB.
Symbolic resolution happens BEFORE any guest exec, so it cannot affect the
lockstep digest. Verified deterministic across two reruns at -n 2M
(instructions=2000005 identical).
End-to-end smoke test on Sylpheed: `--pc-probe='ANON_Class_6B674251::*'`
resolves to all 45 method PCs of that anonymous class (matching the
methods-table row count for that vtable).
Tests 621→626 (+5 lookup unit tests covering numeric passthrough,
symbolic-without-DB error, Class::method resolution, Class::* expansion,
and functions.name fallback).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds detection of statically-allocated MSVC vtables in .rdata/.data:
- New `xenia_analysis::vtables` walks read-only sections looking for runs of
≥3 contiguous big-endian u32 values where each value lands on a known
function start (from M1's corrected functions table). 2-slot runs are
rejected to keep false-positive rate down.
- For each candidate the MSVC RTTI walk vtable[-1] → CompleteObjectLocator
→ TypeDescriptor → mangled name is attempted; on success the demangled
class name is recorded along with a best-effort RTTIClassHierarchyDescriptor
walk to fill base_classes_json. On failure (RTTI stripped — common for
shipped game binaries) the class is named ANON_Class_<fnv1a-hash> keyed
by sorted method-PC list, so identical vtables collapse to one entry.
- DB: new tables `vtables`, `methods`, `classes` with indices on
function_address and rtti_present. `write_analysis_results` takes a
`&[Vtable]` slice; `write_disasm` (back-compat) passes empty.
- cmd_dis wires the scan after xref analysis using
`func_analysis.functions.keys()` as the function-start oracle.
Validation on Sylpheed (RTTI stripped, as expected): 722 vtables / 499
unique classes / 5571 methods. Sanity invariant: every methods.function_address
joins to functions.address (0 broken refs). Largest vtable: 131 slots.
Tests 617→621 (+4 vtable unit tests covering 3-slot detect, 2-slot reject,
synth name stability, and synth name divergence).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds an MSVC name-demangling layer in front of M3's vtable / RTTI work:
- New `xenia_analysis::demangle` wraps the `msvc-demangler` crate (a Rust
port of LLVM's `MicrosoftDemangle.cpp`). `demangle()` short-circuits on
non-mangled inputs (`?` prefix check); `demangle_or_raw()` always returns
a record (raw passthrough on parse failure).
- Heuristic split of the formatted demangled string into structured fields
`(namespace_path, class_name, method_name, params_signature)`. Top-level
paren / template-bracket aware, so `a::b<c::d>::e` and signatures with
templated arg types parse correctly.
- DB: new `demangled_names(address, mangled, raw_demangled, namespace_path,
class_name, method_name, params_signature)` with indices on address /
class_name / method_name. Populated from any label whose name starts with
`?` plus any import name that happens to be mangled.
For Sylpheed (a fully stripped binary) this table is empty out-of-the-box;
the layer's value lands in M3, which will append rows for every RTTI
TypeDescriptor name found in `.rdata`.
Tests 610→617 (+7 demangler unit tests covering early-out, raw fallback,
member function form, RTTI form, qname split, paren-template safety, and
top-level `::` splitting).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds an authoritative function-boundary source from the linker:
- New `xenia_xex::pdata` parses .pdata 8-byte entries (BeginAddress + packed
prolog/length/flags). Bit layout per Microsoft PE32 PowerPC spec: prolog in
bits 0..7, function_length in bits 8..29, flags in 30..31.
- `func::analyze_with_pdata` unions pdata BeginAddresses into the candidate
set, attaches `pdata_validated`/`pdata_length` to each `FuncInfo`, and trims
any function whose `end` overlaps the next start (catches mis-merge where
one row spanned two prologues — the audit-031 sub_824D23B0/sub_824D29F0
case).
- DB: extends `functions` with `pdata_validated BOOLEAN`, `pdata_length BIGINT`;
new table `pdata_entries`; index on pdata_validated.
- New `crates/xenia-analysis/SCHEMA.md` documents M1 layer + forward work.
Validation on Sylpheed: 25481 functions (was 12156) / 23073 pdata_validated /
0 orphans / 0 mis-merges. Audit-031 mis-merge resolved: sub_824D29F0 now has
its own row with `pdata_length=280` (70 dwords); sub_824D23B0 now correctly
ends at 0x824D2878 (`pdata_length=1224` matches prologue walk).
Tests 605→610. New 5-test pdata unit suite covers bit layout + sentinel +
out-of-range filtering + real-world layout round-trip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
