xenia-rs/crates/xenia-analysis/SCHEMA.md

xenia-analysis schema reference

Authoritative documentation for the DuckDB tables and SQL views produced by xenia-rs dis --db sylpheed.db. Track schema changes here alongside any update to the db_schema_golden test fixture.

The base + disasm tables (metadata, sections, imports, functions, labels, instructions, xrefs, opt-in exec_trace / import_calls / branch_trace) are documented inline in src/db.rs doc comment. This file collects layered analysis additions and forward-work notes.

---

Layer M1 — .pdata boundary correction (landed)

Schema additions

• functions.pdata_validated BOOLEAN NOT NULL — true when the row's address matches a RUNTIME_FUNCTION.BeginAddress from .pdata. Linker ground truth.
• functions.pdata_length BIGINT NULL — function_length (bytes) from the matching pdata entry; NULL when the row is prologue-only.
• New table pdata_entries(begin_address BIGINT PRIMARY KEY, end_address BIGINT, function_length BIGINT, prolog_length BIGINT, flags BIGINT) — every parsed .pdata RUNTIME_FUNCTION entry (raw, before any merge with prologue analysis).
• Index idx_functions_pdata_validated on functions(pdata_validated).

What this layer does

• Parses .pdata 8-byte RUNTIME_FUNCTION entries (PowerPC PE32 layout): word 0 BeginAddress (absolute VA), word 1 packed {prolog_length:8, function_length:22, flags:2}, both big-endian.
• Unions pdata BeginAddress values into the function-candidate set fed to the prologue walker, so functions our prologue heuristic missed still get rows.
• When pdata supplies a longer function_length than the prologue walk found, extends end_address to the pdata-implied end (catches mis-split where the walker stopped at an early blr).
• After the walker, performs a forward pass that trims function.end to the next start when they overlap (catches mis-merge where one row spanned two prologues — the audit-031 sub_824D23B0 / sub_824D29F0 case).

What this layer does NOT do

• Does not adjust prolog-derived frame_size / saved_gprs from .pdata's prolog_length field — those remain prologue-only inferences.
• Does not classify functions further than the existing is_leaf / is_saverestore columns. Class membership is M3.
• Does not detect functions whose entries are missing from BOTH .pdata and the bl-target scan (extremely rare; would require executable-byte linear sweep).

Reference docs

• Microsoft PE32+ exception data spec for PowerPC RUNTIME_FUNCTION.
• xenia-canary src/xenia/cpu/xex_module.cc:1570-1587 — canary's reference parser (extracts BeginAddress only; we additionally decode word 1).

Validation queries

-- All pdata entries found
SELECT COUNT(*) FROM pdata_entries;            -- ~23073 for Sylpheed
-- Functions cross-validated against pdata
SELECT COUNT(*) FROM functions WHERE pdata_validated;
-- Functions detected ONLY by prologue (orphans of pdata)
SELECT COUNT(*) FROM functions WHERE NOT pdata_validated;
-- Pdata orphans NOT yet in functions (should be 0 after this layer)
SELECT COUNT(*) FROM pdata_entries p
LEFT JOIN functions f ON f.address = p.begin_address
WHERE f.address IS NULL;
-- Audit-031 mis-merge resolved: 0x824D29F0 should have its own row
SELECT name FROM functions WHERE address = 2186674160;  -- 0x824D29F0

---

Layer M2 — MSVC C++ name demangler (landed)

Schema additions

• New table demangled_names(address BIGINT NULL, mangled VARCHAR NOT NULL, raw_demangled VARCHAR NOT NULL, namespace_path VARCHAR NULL, class_name VARCHAR NULL, method_name VARCHAR NULL, params_signature VARCHAR NULL).
• Indices on address, class_name, method_name.

What this layer does

• Wraps msvc_demangler::demangle (a Rust port of LLVM's MicrosoftDemangle.cpp) and splits the formatted output into structured fields via a heuristic top-level parser (handles templates and nested parens correctly).
• Populates demangled_names from any label whose name starts with ? plus any import name that happens to be mangled (defensive — typical kernel imports use C names).

What this layer does NOT do

• Does not parse the AST returned by msvc_demangler::parse — uses the formatted string and a heuristic split. Adequate for typical class member functions and RTTI strings; exotic template / lambda forms still get raw_demangled populated but may have NULL structured fields.
• Does not yet ingest RTTI strings discovered in .rdata — that's M3's job; M3 will append rows to this table at the addresses where it finds RTTI TypeDescriptors.

Reference docs

• msvc-demangler crate (https://docs.rs/msvc-demangler/0.11).
• LLVM MicrosoftDemangle.cpp (the parser this crate ports).

Layer M3 — Vtable + RTTI detection (landed)

Schema additions

• vtables(address PK, length, col_address NULL, class_name, rtti_present, base_classes_json NULL) — every detected static vtable.
• methods(vtable_address, slot, function_address, mangled_name NULL, demangled_name NULL, PRIMARY KEY (vtable_address, slot)) — one row per method slot.
• classes(name PK, vtable_address, rtti_present, base_classes_json NULL) — deduped by class name (first-detected vtable wins).
• Indices: methods.function_address, classes.rtti_present.

What this layer does

• Walks .rdata and .data looking for runs of ≥3 consecutive 4-byte BE values where each value is a known function start (from M1's corrected functions table). Single-2-method vtables are intentionally rejected to control false-positive rate.
• Attempts the MSVC RTTI walk vtable[-1] → CompleteObjectLocator → TypeDescriptor for each candidate. When successful, the demangled class ClassName string fills class_name and a best-effort RTTIClassHierarchyDescriptor walk fills base_classes_json (JSON array of base class names).
• Falls back to ANON_Class_<8-hex> keyed by FNV-1a hash of the sorted method-PC tuple when RTTI is absent (typical for shipped game binaries). Identical vtables across the binary (multiple instances) collapse to the same anonymous name.

What this layer does NOT do

• Vtables built at runtime in heap-allocated memory (e.g. by ctors copying static templates) are out of scope — only static .rdata/.data content.
• Multiple-inheritance "extra" vftables (one per base subobject) are detected as independent vtables with no link between them.
• Inheritance-tree walking beyond RTTIClassHierarchyDescriptor's direct base list is not attempted.

Reference docs

• openrce.org "Reversing Microsoft Visual C++" — RTTI layout articles (CompleteObjectLocator at vtable[-1]; TypeDescriptor at COL+0xC; mangled name at TD+0x8).

Layer M4 — Class-aware probe targeting (landed)

CLI extension only — no schema changes. The probe-token grammar adds three symbolic forms on top of the existing 0xADDR literal:

• Class::method — joins classes × methods × demangled_names to find every PC whose vtable belongs to that class and whose demangled method_name matches.
• Class::* — joins classes × methods to find every method PC of that class.
• function_name — falls back to functions.name lookup for free functions / saverestore stubs / labels.

Numeric tokens never touch the DB (preserves zero-IO fast path; lockstep digest unaffected). Symbolic tokens require the DuckDB at --probe-db PATH or XENIA_PROBE_DB; default is sylpheed.db next to the .iso when present.

Resolution happens BEFORE guest exec begins, so it cannot affect the lockstep digest.

See crates/xenia-analysis/src/lookup.rs.

---

Layer M5 — Indirect-dispatch reachability (landed)

Schema additions

• New value 'ind_call' in the xrefs.kind set.
• New SQL view v_indirect_reachability_from_entry — strict superset of v_reachability_from_entry, taking ind_call edges in the BFS.

What this layer does

• Walks each FuncAnalysis.functions entry with a per-basic-block register tracker. Recognises the canonical static-vtable pattern: lis+addi → lwz off(rA) → mtctr → bcctrl, where rA ends up holding a known vtable's start address from M3.
• Honours the PowerPC ABI: bl-style calls (op 18 / 16 with LK=1) clobber volatile r0..r12 + ctr but preserve non-volatile r13..r31, so a vtable pointer parked in r30/r31 before a call survives.
• Treats every M3 loc_* label as a basic-block boundary (kills register state) so jump-IN paths cannot induce false positives.

What this layer does NOT do (and observed impact)

• Vtable pointer loaded from a this-pointer field (lwz r_vt, off(rA) where rA = this) — by far the dominant pattern in real C++ — is unresolvable without alias / points-to analysis.
• On Sylpheed: the layer detects 0 edges. The binary's 1,001 lis+addi references into vtables are mostly constructor-side vptr writes (stw rVtable, vptr_offset(this)), not direct dispatches. The renderer hunt's audit-009 cluster therefore needs a future M5.5 with this-flow tracking before this layer surfaces it.

Reference docs

• IBM PowerPC ABI: register-save convention (volatile r0..r12 + ctr, non-volatile r13..r31).

Layer M7 — String / constant-pool detection (landed)

Schema additions

• New table strings(address PK, encoding, length, content).
• Index idx_strings_encoding.

What this layer does

• Scans .rdata for runs of length ≥ 6 of printable ASCII bytes followed by a NUL terminator.
• Scans .rdata for UTF-16LE runs of length ≥ 6 code units (printable-ASCII basic plane only) followed by a u16 NUL terminator.
• Cross-reference is implicit: existing xrefs.kind='ref' rows whose target falls in strings.address's exact match set name the referencing PCs. SQL: SELECT s.content, x.source FROM xrefs x JOIN strings s ON s.address = x.target WHERE x.kind='ref'.

What this layer does NOT do

• No UTF-8 multibyte / non-ASCII basic plane in either encoding.
• No .data scan (read-only-section bias).
• No multi-byte CJK encodings — Japanese text in localised builds may be represented in shift_jis / utf-8 with non-printable bytes that this scanner skips.

Sylpheed yield

• 6,311 ASCII strings (including full embedded HLSL shader source).
• 0 UTF-16LE strings (binary uses ASCII / native CJK encoding).
• 9,132 lis+addi sites cross-reference into the detected strings — names the source PCs that reference each string.

Layer M6 — Extended store-class xrefs + addr_mode column (landed)

Schema additions

• xrefs.addr_mode VARCHAR NULL — sub-classifies how the source instruction computes its target. NULL for control-flow edges (call / ind_call / j / br); one of the following tags for data edges:
  • d_form — standard signed-16 displacement (lwz/stw/lfs/stfs/etc.)
  • lis_addi — address materialised via lis + addi register tracking
  • lis_ori — address materialised via lis + ori
  • multiword — lmw / stmw (one xref per slot; up to 32-rS slots)
  • x_form_indexed — stwx / stbx / sthx / stwux / stbux / sthux / stdx / stdux / lwzx / lbzx / lhzx / lhax / lwzux / lbzux / lhzux / lhaux / ldx / ldux — emitted only when both rA and rB are tracked constants
  • x_form_byterev — stwbrx / sthbrx / lwbrx / lhbrx
  • atomic — stwcx. / stdcx. reservation-conditional stores
  • dcbz — cache-line clear (32-byte zero at rA+rB)
• Index idx_xrefs_addr_mode.

What this layer does

• Tags every existing data xref with its addressing mode (d_form for the bulk; lis_addi / lis_ori for the lift-and-add cases that produce DataRef rows).
• Adds new dispatch for opcode 47 (stmw) and 46 (lmw), expanding to per-slot DataWrite / DataRead rows.
• Adds new dispatch for opcode 31 X-form: stores, atomic, byte-reverse, dcbz. X-form rows are emitted ONLY when both rA and rB resolve to known constants (otherwise the address is runtime-dependent and we skip).

What this layer does NOT do

• VMX / VMX128 vector stores (opcode 31 with vector XO codes) are not emitted — they always have register-indexed addresses that the lis+addi tracker can't usually resolve, and detecting them adds noise without improving target resolution.
• The dominant runtime-of-stwx pattern (rA = base, rB = runtime index) is not resolved — by design; mem-watch covers the runtime side per VERIFY-B.

Sylpheed yield

• 28,834 lis_addi refs, 18,485 d_form reads, 3,288 d_form writes — the existing baseline now properly tagged.
• 442 newly-detected x_form_indexed reads — primarily lwzx/lhzx reads from in-table dispatch (each pair (rA,rB) resolved statically).
• 40 newly-detected atomic writes — every stwcx. site with a resolvable address; useful for reservation-table audits.
• 9 lis_ori refs.
• 0 multiword / dcbz / byterev — these instructions exist in the binary but are not in lis+addi-tracked code paths.

Forward work (M8–M12, not yet landed)

• M8 — dispatch-table heuristics beyond vtables (e.g. function-pointer arrays in .data).
• M9 — __CxxFrameHandler exception scope-table parsing.
• M10 — .tls section / TLS slot tracking.
• M11 — __xc_a / __xc_z static-initializer driver detection.
• M12 — comparative-PC-trace mode for canary diff (runtime side, not analyzer).