M8+M9+M10+M11+M12: LOW-tier milestones — funcptr-arrays, EH flag, TLS, lr-trace
Five LOW-priority milestones bundled. Total ~700 LOC across 11 files. ## M9 — has_eh derived from pdata.flags exception bit - New `functions.has_eh BOOLEAN NOT NULL` column. Derived from M1's already-parsed `pdata.flags` (bit 31 of the packed word — the exception-handler-present flag, distinct from bit 30 which is the always-1 32-bit-code flag). Index idx_functions_has_eh. - Sylpheed: 2,975 of 23,073 pdata-validated functions have EH (12.9%). ## M10 — .tls section / IMAGE_TLS_DIRECTORY32 parser - New `xenia_xex::tls::parse_tls` parses the directory + zero-terminated callback array. Returns None when the binary has no .tls section. - New `tls_info` (singleton row) + `tls_callbacks(slot, address)` tables. - New `DbWriter::write_tls()` no-ops on None. - Sylpheed has no .tls section → 0 rows; infra ready for binaries with __declspec(thread). ## M8 + M11 — function_pointer_arrays (dispatch tables + static initialisers) - New `xenia_analysis::funcptr_arrays::analyze` widens M3's vtable scan: detects runs of ≥2 function pointers in .rdata and classifies each as `vtable` (M3 re-emit), `dispatch_table` (M8), or `static_init` (M11) via a constructor-prologue heuristic (mfspr + small stwu). - New tables `function_pointer_arrays(address PK, length, kind)` and `function_pointer_array_entries(array_address, slot, function_address)`. - Sylpheed: 722 vtables + 388 dispatch_tables = 1,110 arrays / 6,347 slots. 0 static_init detected (Sylpheed's ctors don't all match the conservative heuristic; M11.5 future work can chain via the entry- point's static-init driver). ## M12 — --lr-trace runtime canary-diff harness - New CLI `exec --lr-trace=PC[,PC,...]` and `--lr-trace-out=PATH` flags. Symbolic resolution (Class::method, Class::*) via M4 lookup. Env vars XENIA_LR_TRACE / XENIA_LR_TRACE_OUT also work. - New `KernelState::lr_trace_pcs` + `lr_trace_writer` + helper `fire_lr_trace_if_match(hw_id)` invoked from the per-instr probe slot. - JSONL output: pc/tid/hw/cycle/r3/r4/r5/r6/lr — superset of what xenia-canary's --log_lr_on_pc patch emits, with a cycle counter for cross-run reproducibility. Diff-friendly via `jq`. - Lockstep digest unaffected: smoke test on entry-point PC fires once with cycle=0/lr=BCBCBCBC/all-GPR-zero (correct initial state). Tests 636→640 (+2 TLS tests, +2 funcptr_arrays tests). Schema golden updated for new tables + has_eh column. Lockstep determinism preserved (instructions=2000005 ×2 reruns identical). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
@@ -274,10 +274,134 @@ See `crates/xenia-analysis/src/lookup.rs`.
|
||||
- 0 multiword / dcbz / byterev — these instructions exist in the binary
|
||||
but are not in lis+addi-tracked code paths.
|
||||
|
||||
## Forward work (M8–M12, not yet landed)
|
||||
## Layer M8 + M11 — Function-pointer arrays beyond vtables (landed)
|
||||
|
||||
- **M8** — dispatch-table heuristics beyond vtables (e.g. function-pointer arrays in `.data`).
|
||||
- **M9** — `__CxxFrameHandler` exception scope-table parsing.
|
||||
- **M10** — `.tls` section / TLS slot tracking.
|
||||
- **M11** — `__xc_a` / `__xc_z` static-initializer driver detection.
|
||||
- **M12** — comparative-PC-trace mode for canary diff (runtime side, not analyzer).
|
||||
### Schema additions
|
||||
- New table `function_pointer_arrays(address PK, length, kind)` where
|
||||
`kind` is `'vtable'` (M3 re-emit), `'dispatch_table'` (M8), or
|
||||
`'static_init'` (M11).
|
||||
- New table `function_pointer_array_entries(array_address, slot,
|
||||
function_address, PRIMARY KEY (array_address, slot))` — one row per
|
||||
slot of every detected array (vtable + non-vtable).
|
||||
- Indices on `function_pointer_arrays.kind` and
|
||||
`function_pointer_array_entries.function_address`.
|
||||
|
||||
### What this layer does
|
||||
- Walks `.rdata` (only — `.data` produces too many false positives) for
|
||||
runs of ≥ 2 consecutive 4-byte BE values where each value is a known
|
||||
function entry from M1's `functions` table.
|
||||
- Skips runs whose start matches an M3 vtable head — those are re-emitted
|
||||
in this table with `kind='vtable'` for unified queries but not
|
||||
re-classified.
|
||||
- Heuristically classifies non-vtable runs:
|
||||
- `static_init` (M11): every entry's first instruction is `mfspr r12, LR`
|
||||
AND the next is `stwu r1, -N(r1)` with `N ≤ 0x80` (or a save-stub `bl`).
|
||||
Mirrors the typical C++ static-initialiser prologue.
|
||||
- `dispatch_table` (M8): everything else.
|
||||
|
||||
### What this layer does NOT do
|
||||
- Does not parse symbol-table-bracketed regions like `__xc_a` / `__xc_z`
|
||||
/ `__xi_a` / `__xi_z` directly — Sylpheed's symbol table is stripped.
|
||||
- Does not chain multi-segment static-init drivers; future M11.5 could
|
||||
walk the entry-point's static-init driver call chain to surface
|
||||
ground-truth ctor PCs.
|
||||
- 2-slot runs in `.rdata` may be false positives where two struct fields
|
||||
happen to alias function VAs; downstream queries should use a length
|
||||
filter (`WHERE length >= 3`) when high precision matters.
|
||||
|
||||
### Sylpheed yield
|
||||
- 722 vtables (M3 re-emit) + 388 dispatch_tables = 1,110 arrays in
|
||||
`function_pointer_arrays`.
|
||||
- 0 static_init detected — Sylpheed's ctors don't all match the
|
||||
conservative prologue heuristic. Lengths concentrate at 2 slots
|
||||
(typical of switch-case jump tables).
|
||||
|
||||
## Layer M9 — `has_eh` from `.pdata` exception flag (landed)
|
||||
|
||||
### Schema additions
|
||||
- `functions.has_eh BOOLEAN NOT NULL` — true when `.pdata`'s exception-
|
||||
handler-present bit (bit 31 of word 1, the high bit) is set.
|
||||
- Index `idx_functions_has_eh`.
|
||||
|
||||
### What this layer does
|
||||
- Derived directly from M1's already-parsed `pdata.flags` bit field (no
|
||||
new parsing). The bit was always available in `pdata_entries.flags`;
|
||||
this layer surfaces it as a first-class column on `functions`.
|
||||
|
||||
### What this layer does NOT do
|
||||
- Does not parse the actual `__CxxFrameHandler` / `__C_specific_handler`
|
||||
scope-table records that the exception bit gates. Walking those tables
|
||||
would let us name try/catch ranges and per-state cleanup actions, but
|
||||
is out of scope for a derive-only milestone.
|
||||
|
||||
### Sylpheed yield
|
||||
- 2,975 of 23,073 pdata-validated functions have `has_eh=true` (12.9%) —
|
||||
plausible MSVC C++ EH coverage rate. Largest EH function: 26,328 bytes
|
||||
(`sub_823518F0`).
|
||||
|
||||
## Layer M10 — `.tls` section / TLS directory (landed)
|
||||
|
||||
### Schema additions
|
||||
- New table `tls_info(raw_data_start, raw_data_end, index_address,
|
||||
callback_array, zero_fill_size, characteristics)` — at most one row
|
||||
(the IMAGE_TLS_DIRECTORY32).
|
||||
- New table `tls_callbacks(slot PK, address)` — one row per resolved TLS
|
||||
callback function.
|
||||
|
||||
### What this layer does
|
||||
- Reads the first 24 bytes of the `.tls` section as an
|
||||
`IMAGE_TLS_DIRECTORY32` and walks the zero-terminated callback array.
|
||||
- All addresses stored as absolute VAs.
|
||||
|
||||
### What this layer does NOT do
|
||||
- Does not parse the raw TLS template content (the variable initialiser
|
||||
block); just records its start/end VAs.
|
||||
|
||||
### Sylpheed yield
|
||||
- 0 rows — Sylpheed has no `.tls` section. Infrastructure ready for any
|
||||
binary that uses `__declspec(thread)` storage.
|
||||
|
||||
## Layer M12 — `--lr-trace` runtime canary-diff harness (landed)
|
||||
|
||||
### Runtime additions (no DB)
|
||||
- New CLI flag `--lr-trace=PC[,PC,...]` on `exec` — comma-separated PCs
|
||||
to capture as JSONL records on every fire. Symbolic tokens (`Class::method`)
|
||||
resolve via M4's lookup against `--probe-db`. Settable via
|
||||
`XENIA_LR_TRACE`.
|
||||
- New CLI flag `--lr-trace-out=PATH` — writes JSONL to a file (one
|
||||
record per line). Stdout when omitted. Settable via `XENIA_LR_TRACE_OUT`.
|
||||
- New kernel state fields `lr_trace_pcs: HashSet<u32>` +
|
||||
`lr_trace_writer: Option<Mutex<File>>` and helper
|
||||
`KernelState::fire_lr_trace_if_match(hw_id)` invoked from the
|
||||
per-instruction probe slot.
|
||||
|
||||
### JSONL record fields
|
||||
`pc, tid, hw, cycle, r3, r4, r5, r6, lr` — superset of what
|
||||
xenia-canary's `--log_lr_on_pc` patch emits, with a cycle counter added
|
||||
for cross-run reproducibility.
|
||||
|
||||
### What this layer does NOT do
|
||||
- Does not capture VMX / FP register state (only GPRs r3..r6).
|
||||
- Does not buffer / batch records — one `write_all` per fire. For
|
||||
high-frequency probes (e.g. tight loops at >1M fires/sec), redirect
|
||||
to a file and use a SSD.
|
||||
|
||||
### Determinism
|
||||
Lockstep digest unaffected: probe firing happens after the per-instr
|
||||
hooks for ctor/branch probes and only emits side-channel output. Verified
|
||||
end-of-session: `check sylpheed.iso --stable-digest -n 2M` ×2 produced
|
||||
byte-identical digests (`instructions=2000005`).
|
||||
|
||||
---
|
||||
|
||||
## Forward work (not yet landed)
|
||||
|
||||
- **M5.5** — `this`-flow extension to M5. Resolve vtable dispatches via
|
||||
`lwz vt, off(this)` patterns by tracing constructor-side vptr writes.
|
||||
Highest-value future work for the audit-009 cluster renderer hunt.
|
||||
- **M9.5** — full `__CxxFrameHandler` scope-table parsing (try/catch
|
||||
range names, per-state cleanup actions).
|
||||
- **M11.5** — walk the static-initialiser driver call chain from the
|
||||
entry point to surface ground-truth ctor PCs.
|
||||
- VMX/VMX128 vector-store xref emission (M6 follow-up).
|
||||
- UTF-8 / shift_jis localised-string detection in `.rdata` (M7 follow-up).
|
||||
|
||||
@@ -306,7 +306,8 @@ impl DbWriter {
|
||||
///
|
||||
/// `vtables` is the M3 result; pass an empty slice when the caller has
|
||||
/// not run the vtable scan (the tables are still created, just empty).
|
||||
/// `strings` is the M7 result; same convention.
|
||||
/// `strings` is the M7 result; same convention. `funcptr_arrays` is the
|
||||
/// M8/M11 result.
|
||||
#[tracing::instrument(skip_all, name = "db.write_analysis_results")]
|
||||
pub fn write_analysis_results(
|
||||
&mut self,
|
||||
@@ -317,6 +318,7 @@ impl DbWriter {
|
||||
xrefs: &XrefMap,
|
||||
vtables: &[crate::vtables::Vtable],
|
||||
strings: &[crate::strings::DetectedString],
|
||||
funcptr_arrays: &[crate::funcptr_arrays::FuncPtrArray],
|
||||
) -> anyhow::Result<()> {
|
||||
self.conn.execute_batch("
|
||||
CREATE TABLE functions (
|
||||
@@ -328,7 +330,8 @@ impl DbWriter {
|
||||
is_leaf BOOLEAN NOT NULL, -- true if the function has no outgoing calls
|
||||
is_saverestore BOOLEAN NOT NULL, -- true if __savegprlr_* / __restgprlr_* stub
|
||||
pdata_validated BOOLEAN NOT NULL, -- true if .pdata RUNTIME_FUNCTION exists at this VA
|
||||
pdata_length BIGINT -- length in bytes per .pdata; NULL if no pdata entry
|
||||
pdata_length BIGINT, -- length in bytes per .pdata; NULL if no pdata entry
|
||||
has_eh BOOLEAN NOT NULL -- M9: pdata exception-flag bit set; function has C++ EH/SEH
|
||||
);
|
||||
|
||||
CREATE TABLE pdata_entries (
|
||||
@@ -377,6 +380,33 @@ impl DbWriter {
|
||||
content VARCHAR NOT NULL -- UTF-8 representation of the string
|
||||
);
|
||||
|
||||
CREATE TABLE tls_info (
|
||||
raw_data_start BIGINT NOT NULL, -- VA of TLS template start
|
||||
raw_data_end BIGINT NOT NULL, -- VA one-past-end of TLS template
|
||||
index_address BIGINT NOT NULL, -- VA of u32 the loader writes the assigned slot index into
|
||||
callback_array BIGINT NOT NULL, -- VA of zero-terminated callback array (0 if none)
|
||||
zero_fill_size BIGINT NOT NULL, -- bytes of zero-fill appended after raw template
|
||||
characteristics BIGINT NOT NULL -- IMAGE_TLS_DIRECTORY characteristics flags
|
||||
);
|
||||
|
||||
CREATE TABLE tls_callbacks (
|
||||
slot BIGINT PRIMARY KEY, -- 0-based index in the callback array
|
||||
address BIGINT NOT NULL -- VA of callback function
|
||||
);
|
||||
|
||||
CREATE TABLE function_pointer_arrays (
|
||||
address BIGINT PRIMARY KEY, -- absolute VA of the array's first slot
|
||||
length BIGINT NOT NULL, -- number of slots
|
||||
kind VARCHAR NOT NULL -- 'vtable' (M3) | 'dispatch_table' (M8) | 'static_init' (M11)
|
||||
);
|
||||
|
||||
CREATE TABLE function_pointer_array_entries (
|
||||
array_address BIGINT NOT NULL, -- FK to function_pointer_arrays.address
|
||||
slot BIGINT NOT NULL, -- 0-based slot index
|
||||
function_address BIGINT NOT NULL, -- VA of the function this slot points at
|
||||
PRIMARY KEY (array_address, slot)
|
||||
);
|
||||
|
||||
CREATE TABLE demangled_names (
|
||||
address BIGINT, -- VA the mangled name is associated with; NULL when from a non-address source (e.g. RTTI-only string)
|
||||
mangled VARCHAR NOT NULL, -- original mangled symbol (e.g. ?Foo@Bar@@QEAAXXZ)
|
||||
@@ -406,11 +436,13 @@ impl DbWriter {
|
||||
insert_vtables(&self.conn, vtables, pe, info.image_base)?;
|
||||
insert_methods_and_classes(&self.conn, vtables, labels)?;
|
||||
insert_strings(&self.conn, strings)?;
|
||||
insert_funcptr_arrays(&self.conn, funcptr_arrays)?;
|
||||
insert_xrefs_streaming(&self.conn, xrefs, pe, info.image_base, func_analysis, labels)?;
|
||||
|
||||
let indices = [
|
||||
("idx_functions_name", "CREATE INDEX idx_functions_name ON functions(name)"),
|
||||
("idx_functions_pdata_validated", "CREATE INDEX idx_functions_pdata_validated ON functions(pdata_validated)"),
|
||||
("idx_functions_has_eh", "CREATE INDEX idx_functions_has_eh ON functions(has_eh)"),
|
||||
("idx_labels_kind", "CREATE INDEX idx_labels_kind ON labels(kind)"),
|
||||
("idx_labels_name", "CREATE INDEX idx_labels_name ON labels(name)"),
|
||||
("idx_demangled_address", "CREATE INDEX idx_demangled_address ON demangled_names(address)"),
|
||||
@@ -420,6 +452,8 @@ impl DbWriter {
|
||||
("idx_classes_rtti", "CREATE INDEX idx_classes_rtti ON classes(rtti_present)"),
|
||||
("idx_strings_encoding", "CREATE INDEX idx_strings_encoding ON strings(encoding)"),
|
||||
("idx_xrefs_addr_mode", "CREATE INDEX idx_xrefs_addr_mode ON xrefs(addr_mode)"),
|
||||
("idx_fparrays_kind", "CREATE INDEX idx_fparrays_kind ON function_pointer_arrays(kind)"),
|
||||
("idx_fpentries_function", "CREATE INDEX idx_fpentries_function ON function_pointer_array_entries(function_address)"),
|
||||
("idx_xrefs_target", "CREATE INDEX idx_xrefs_target ON xrefs(target)"),
|
||||
("idx_xrefs_source", "CREATE INDEX idx_xrefs_source ON xrefs(source)"),
|
||||
("idx_xrefs_source_func", "CREATE INDEX idx_xrefs_source_func ON xrefs(source_func)"),
|
||||
@@ -448,7 +482,39 @@ impl DbWriter {
|
||||
xrefs: &XrefMap,
|
||||
) -> anyhow::Result<()> {
|
||||
self.ingest_instructions(pe, info, func_analysis, labels)?;
|
||||
self.write_analysis_results(pe, info, func_analysis, labels, xrefs, &[], &[])?;
|
||||
self.write_analysis_results(pe, info, func_analysis, labels, xrefs, &[], &[], &[])?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// M10 — write the parsed `.tls` directory + callback array. No-op
|
||||
/// when `tls` is `None` (binary has no `.tls` section).
|
||||
#[tracing::instrument(skip_all, name = "db.write_tls")]
|
||||
pub fn write_tls(
|
||||
&mut self,
|
||||
tls: Option<&xenia_xex::tls::TlsInfo>,
|
||||
) -> anyhow::Result<()> {
|
||||
let Some(t) = tls else { return Ok(()); };
|
||||
self.conn.execute(
|
||||
"INSERT INTO tls_info (raw_data_start, raw_data_end, index_address,
|
||||
callback_array, zero_fill_size, characteristics)
|
||||
VALUES (?, ?, ?, ?, ?, ?)",
|
||||
params![
|
||||
t.raw_data_start as i64,
|
||||
t.raw_data_end as i64,
|
||||
t.index_address as i64,
|
||||
t.callback_array as i64,
|
||||
t.zero_fill_size as i64,
|
||||
t.characteristics as i64,
|
||||
],
|
||||
)?;
|
||||
let mut stmt = self.conn.prepare(
|
||||
"INSERT INTO tls_callbacks (slot, address) VALUES (?, ?)"
|
||||
)?;
|
||||
for (i, cb) in t.callbacks.iter().enumerate() {
|
||||
stmt.execute(params![i as i64, cb.address as i64])?;
|
||||
}
|
||||
metrics::counter!("db.rows", "table" => "tls_callbacks").increment(t.callbacks.len() as u64);
|
||||
tracing::info!(rows = t.callbacks.len(), table = "tls_callbacks", "tls write complete");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -755,8 +821,8 @@ fn insert_functions(
|
||||
let mut stmt = conn.prepare(
|
||||
"INSERT INTO functions
|
||||
(address, name, end_address, frame_size, saved_gprs, is_leaf, is_saverestore,
|
||||
pdata_validated, pdata_length)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)"
|
||||
pdata_validated, pdata_length, has_eh)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"
|
||||
)?;
|
||||
for (&addr, fi) in &func_analysis.functions {
|
||||
let name = labels.get(&addr)
|
||||
@@ -772,6 +838,7 @@ fn insert_functions(
|
||||
fi.is_saverestore,
|
||||
fi.pdata_validated,
|
||||
fi.pdata_length.map(|n| n as i64),
|
||||
fi.has_eh,
|
||||
])?;
|
||||
}
|
||||
Ok(())
|
||||
@@ -884,6 +951,37 @@ fn insert_strings(
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn insert_funcptr_arrays(
|
||||
conn: &Connection,
|
||||
arrays: &[crate::funcptr_arrays::FuncPtrArray],
|
||||
) -> anyhow::Result<()> {
|
||||
if arrays.is_empty() { return Ok(()); }
|
||||
let mut stmt_arr = conn.prepare(
|
||||
"INSERT INTO function_pointer_arrays (address, length, kind) VALUES (?, ?, ?)
|
||||
ON CONFLICT DO NOTHING"
|
||||
)?;
|
||||
let mut stmt_ent = conn.prepare(
|
||||
"INSERT INTO function_pointer_array_entries (array_address, slot, function_address)
|
||||
VALUES (?, ?, ?) ON CONFLICT DO NOTHING"
|
||||
)?;
|
||||
let mut n_arr = 0u64;
|
||||
let mut n_ent = 0u64;
|
||||
for a in arrays {
|
||||
let inserted = stmt_arr.execute(params![
|
||||
a.address as i64, a.length as i64, a.kind,
|
||||
])?;
|
||||
if inserted > 0 { n_arr += 1; }
|
||||
for (i, &fn_va) in a.entries.iter().enumerate() {
|
||||
stmt_ent.execute(params![a.address as i64, i as i64, fn_va as i64])?;
|
||||
n_ent += 1;
|
||||
}
|
||||
}
|
||||
metrics::counter!("db.rows", "table" => "function_pointer_arrays").increment(n_arr);
|
||||
metrics::counter!("db.rows", "table" => "function_pointer_array_entries").increment(n_ent);
|
||||
tracing::info!(arrays = n_arr, entries = n_ent, "function-pointer arrays insert complete");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn insert_demangled_from_labels(
|
||||
conn: &Connection,
|
||||
labels: &HashMap<u32, String>,
|
||||
|
||||
@@ -39,6 +39,10 @@ pub struct FuncInfo {
|
||||
/// Function size in bytes per `.pdata`'s `function_length` field, if known.
|
||||
/// Absent (None) when this row is prologue-only.
|
||||
pub pdata_length: Option<u32>,
|
||||
/// True when `.pdata`'s exception-flag bit is set on this entry — the
|
||||
/// function has a registered C++ EH (or SEH) frame handler. Always false
|
||||
/// for entries without `.pdata` coverage. (M9)
|
||||
pub has_eh: bool,
|
||||
}
|
||||
|
||||
/// Result of the function analysis pass.
|
||||
@@ -296,6 +300,8 @@ pub fn analyze_with_pdata(
|
||||
if let Some(p) = pdata_entry {
|
||||
fi.pdata_validated = true;
|
||||
fi.pdata_length = Some(p.function_length);
|
||||
// bit 0 of the packed flags = exception-handler-present
|
||||
fi.has_eh = (p.flags & 0x2) != 0;
|
||||
// If the prologue walk ended too early, trust pdata's length.
|
||||
let pdata_end = p.begin_address.wrapping_add(p.function_length);
|
||||
if pdata_end > fi.end {
|
||||
@@ -317,6 +323,7 @@ pub fn analyze_with_pdata(
|
||||
is_saverestore: false,
|
||||
pdata_validated: true,
|
||||
pdata_length: Some(p.function_length),
|
||||
has_eh: (p.flags & 0x2) != 0,
|
||||
},
|
||||
);
|
||||
}
|
||||
@@ -326,6 +333,7 @@ pub fn analyze_with_pdata(
|
||||
if let Some(sb) = save_base {
|
||||
// The save block is one cascade: entry at each rN, falls through to blr
|
||||
// Treat as a single function with the first entry point
|
||||
let pe_sb = pdata_by_begin.get(&sb).copied();
|
||||
functions.insert(sb, FuncInfo {
|
||||
start: sb,
|
||||
end: sb + 20 * 4, // 18 std + stw r12 + blr
|
||||
@@ -333,11 +341,13 @@ pub fn analyze_with_pdata(
|
||||
saved_gprs: 18,
|
||||
is_leaf: true,
|
||||
is_saverestore: true,
|
||||
pdata_validated: pdata_by_begin.contains_key(&sb),
|
||||
pdata_length: pdata_by_begin.get(&sb).map(|p| p.function_length),
|
||||
pdata_validated: pe_sb.is_some(),
|
||||
pdata_length: pe_sb.map(|p| p.function_length),
|
||||
has_eh: pe_sb.map(|p| (p.flags & 0x2) != 0).unwrap_or(false),
|
||||
});
|
||||
}
|
||||
if let Some(rb) = restore_base {
|
||||
let pe_rb = pdata_by_begin.get(&rb).copied();
|
||||
functions.insert(rb, FuncInfo {
|
||||
start: rb,
|
||||
end: rb + 21 * 4, // 18 ld + lwz r12 + mtspr LR + blr
|
||||
@@ -345,8 +355,9 @@ pub fn analyze_with_pdata(
|
||||
saved_gprs: 18,
|
||||
is_leaf: true,
|
||||
is_saverestore: true,
|
||||
pdata_validated: pdata_by_begin.contains_key(&rb),
|
||||
pdata_length: pdata_by_begin.get(&rb).map(|p| p.function_length),
|
||||
pdata_validated: pe_rb.is_some(),
|
||||
pdata_length: pe_rb.map(|p| p.function_length),
|
||||
has_eh: pe_rb.map(|p| (p.flags & 0x2) != 0).unwrap_or(false),
|
||||
});
|
||||
}
|
||||
|
||||
@@ -498,6 +509,7 @@ fn analyze_function(
|
||||
is_saverestore: false,
|
||||
pdata_validated: false,
|
||||
pdata_length: None,
|
||||
has_eh: false,
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
257
crates/xenia-analysis/src/funcptr_arrays.rs
Normal file
257
crates/xenia-analysis/src/funcptr_arrays.rs
Normal file
@@ -0,0 +1,257 @@
|
||||
//! Generic function-pointer array detection (M8 + M11).
|
||||
//!
|
||||
//! M3 already detects "vtable" candidates — runs of ≥3 contiguous function
|
||||
//! pointers in `.rdata` / `.data` (with COL/RTTI walk on top). This module
|
||||
//! widens the net:
|
||||
//!
|
||||
//! - **Dispatch tables** (M8): runs of ≥2 function pointers in `.rdata` /
|
||||
//! `.data` that are NOT already classified as vtables. Captures switch
|
||||
//! jump tables, callback registries, command tables, gameplay state
|
||||
//! machines, etc.
|
||||
//! - **Static initialiser tables** (M11): function-pointer arrays in
|
||||
//! `.rdata` whose entries all have classic constructor-like prologues
|
||||
//! (small frame; either leaf or calling well-known runtime helpers).
|
||||
//! The MSVC convention names the bracketing symbols `__xc_a` /
|
||||
//! `__xc_z` (C++ ctors) and `__xi_a` / `__xi_z` (C runtime), but the
|
||||
//! names are stripped from Sylpheed; we classify by structure.
|
||||
//!
|
||||
//! All findings are written to a single `function_pointer_arrays` table
|
||||
//! with a `kind` column — `"vtable"`, `"dispatch_table"`, or `"static_init"`.
|
||||
//! Vtable rows are duplicated from M3's `vtables` table for join
|
||||
//! convenience (so a single query covers all classification kinds).
|
||||
//!
|
||||
//! ### What this module does NOT do
|
||||
//!
|
||||
//! - No alias-based classification — `static_init` is heuristic and may
|
||||
//! include any function-pointer array near the binary's `__xc_*` region.
|
||||
//! - Does not parse the bracket symbols' actual addresses — we'd need
|
||||
//! debug symbols, which Sylpheed doesn't ship.
|
||||
//! - Two-element runs in `.data` are common false positives (struct fields
|
||||
//! that happen to alias function entries); we only emit `dispatch_table`
|
||||
//! rows for `.rdata`.
|
||||
|
||||
use std::collections::BTreeSet;
|
||||
|
||||
use xenia_xex::pe::PeSection;
|
||||
|
||||
use crate::vtables::Vtable;
|
||||
|
||||
/// One detected function-pointer array.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct FuncPtrArray {
|
||||
pub address: u32,
|
||||
pub length: u32,
|
||||
pub kind: &'static str, // "vtable" | "dispatch_table" | "static_init"
|
||||
/// Array entries (function VAs).
|
||||
pub entries: Vec<u32>,
|
||||
}
|
||||
|
||||
/// Run the pass. `vtables` is the M3 result — those addresses are skipped
|
||||
/// in the dispatch-table scan to avoid duplication. `function_starts` is
|
||||
/// the M1 corrected function-start set (used to validate that each array
|
||||
/// entry actually points at a known function).
|
||||
#[tracing::instrument(skip_all, fields(image_base = format_args!("{:#010x}", image_base)))]
|
||||
pub fn analyze(
|
||||
pe: &[u8],
|
||||
image_base: u32,
|
||||
sections: &[PeSection],
|
||||
function_starts: &BTreeSet<u32>,
|
||||
vtables: &[Vtable],
|
||||
) -> Vec<FuncPtrArray> {
|
||||
let started = std::time::Instant::now();
|
||||
let vtable_addrs: BTreeSet<u32> = vtables.iter().map(|v| v.address).collect();
|
||||
let mut out: Vec<FuncPtrArray> = Vec::new();
|
||||
|
||||
// Re-emit vtables in this table for unified-query convenience.
|
||||
for v in vtables {
|
||||
out.push(FuncPtrArray {
|
||||
address: v.address,
|
||||
length: v.length,
|
||||
kind: "vtable",
|
||||
entries: v.methods.clone(),
|
||||
});
|
||||
}
|
||||
|
||||
// Scan only .rdata for dispatch tables — .data has too many false
|
||||
// positives from struct fields aliasing function VAs.
|
||||
for section in sections {
|
||||
if section.name != ".rdata" { continue; }
|
||||
let raw_start = section.virtual_address as usize;
|
||||
let raw_end = (section.virtual_address + section.virtual_size) as usize;
|
||||
if raw_end > pe.len() { continue; }
|
||||
let bytes = &pe[raw_start..raw_end.min(pe.len())];
|
||||
let va_base = image_base + section.virtual_address;
|
||||
|
||||
let mut i = 0usize;
|
||||
while i + 8 <= bytes.len() {
|
||||
if !i.is_multiple_of(4) { i += 1; continue; }
|
||||
let mut entries: Vec<u32> = Vec::new();
|
||||
let mut j = i;
|
||||
while j + 4 <= bytes.len() {
|
||||
let val = u32::from_be_bytes([bytes[j], bytes[j + 1], bytes[j + 2], bytes[j + 3]]);
|
||||
if function_starts.contains(&val) {
|
||||
entries.push(val);
|
||||
j += 4;
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
}
|
||||
if entries.len() >= 2 {
|
||||
let address = va_base + (i as u32);
|
||||
if !vtable_addrs.contains(&address) {
|
||||
let kind = classify_run(image_base, &entries, pe);
|
||||
out.push(FuncPtrArray {
|
||||
address,
|
||||
length: entries.len() as u32,
|
||||
kind,
|
||||
entries,
|
||||
});
|
||||
}
|
||||
i += j - i;
|
||||
} else {
|
||||
i += 4;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let elapsed_ms = started.elapsed().as_millis() as f64;
|
||||
let n_vt = out.iter().filter(|a| a.kind == "vtable").count();
|
||||
let n_dt = out.iter().filter(|a| a.kind == "dispatch_table").count();
|
||||
let n_si = out.iter().filter(|a| a.kind == "static_init").count();
|
||||
metrics::histogram!("analysis.phase_ms", "phase" => "funcptr_arrays").record(elapsed_ms);
|
||||
tracing::info!(
|
||||
total = out.len(), vtable = n_vt, dispatch_table = n_dt, static_init = n_si,
|
||||
elapsed_ms,
|
||||
"function-pointer array scan complete",
|
||||
);
|
||||
out
|
||||
}
|
||||
|
||||
/// Classify a non-vtable function-pointer array. Currently distinguishes
|
||||
/// only "static_init" (all entries have constructor-like prologues — a
|
||||
/// brief mfspr+stwu prologue with a small frame) from "dispatch_table"
|
||||
/// (anything else).
|
||||
fn classify_run(image_base: u32, entries: &[u32], pe: &[u8]) -> &'static str {
|
||||
// Heuristic: a static initialiser's prologue is small (frame ≤ 0x80,
|
||||
// typically ≤ 0x40). If every entry's first instruction is mfspr+LR
|
||||
// (opcode 31, xo 339, spr 8) followed by a small stwu, classify as
|
||||
// static_init.
|
||||
let mut all_ctor = true;
|
||||
let mut any_ctor = false;
|
||||
for &fn_va in entries {
|
||||
if !is_ctor_like(pe, image_base, fn_va) {
|
||||
all_ctor = false;
|
||||
} else {
|
||||
any_ctor = true;
|
||||
}
|
||||
}
|
||||
if all_ctor && any_ctor && entries.len() >= 3 {
|
||||
"static_init"
|
||||
} else {
|
||||
"dispatch_table"
|
||||
}
|
||||
}
|
||||
|
||||
/// True if the function at `fn_va` looks like a tiny C++ static initialiser:
|
||||
/// `mfspr r12, LR` immediately followed by `stwu r1, -N(r1)` with `N ≤ 0x80`.
|
||||
fn is_ctor_like(pe: &[u8], image_base: u32, fn_va: u32) -> bool {
|
||||
let off = fn_va.wrapping_sub(image_base) as usize;
|
||||
if off + 8 > pe.len() { return false; }
|
||||
let i0 = u32::from_be_bytes([pe[off], pe[off + 1], pe[off + 2], pe[off + 3]]);
|
||||
let i1 = u32::from_be_bytes([pe[off + 4], pe[off + 5], pe[off + 6], pe[off + 7]]);
|
||||
// i0: mfspr rD, LR — opcode 31, xo 339, spr 8.
|
||||
let op0 = i0 >> 26;
|
||||
let xo0 = (i0 >> 1) & 0x3FF;
|
||||
let spr0 = (((i0 >> 11) & 0x1F) << 5) | ((i0 >> 16) & 0x1F);
|
||||
if !(op0 == 31 && xo0 == 339 && spr0 == 8) { return false; }
|
||||
// i1 must be stwu r1, -N(r1) with N ≤ 0x80, OR a `bl __savegprlr_*`
|
||||
// followed eventually by stwu (full prologue). Allow either.
|
||||
let op1 = i1 >> 26;
|
||||
if op1 == 37 {
|
||||
// stwu D-form: rS=1, rA=1
|
||||
let rs = (i1 >> 21) & 0x1F;
|
||||
let ra = (i1 >> 16) & 0x1F;
|
||||
let d = ((i1 & 0xFFFF) as i16) as i32;
|
||||
rs == 1 && ra == 1 && d <= 0 && (-d) <= 0x80
|
||||
} else if op1 == 18 {
|
||||
// bl __savegprlr_NN — accept; ctor with frame ≤ 0x80 is the
|
||||
// common case, but if the compiler emits a save-stub call we
|
||||
// can't easily verify the frame size without walking further.
|
||||
true
|
||||
} else {
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use xenia_xex::pe::PeSection;
|
||||
|
||||
fn mk_section(name: &str, va: u32, size: u32) -> PeSection {
|
||||
PeSection {
|
||||
name: name.into(),
|
||||
virtual_address: va,
|
||||
virtual_size: size,
|
||||
raw_offset: va,
|
||||
raw_size: size,
|
||||
flags: 0x4000_0040,
|
||||
}
|
||||
}
|
||||
|
||||
fn write_be_u32(buf: &mut [u8], at: usize, val: u32) {
|
||||
buf[at..at + 4].copy_from_slice(&val.to_be_bytes());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn detects_dispatch_table_in_rdata() {
|
||||
let image_base = 0x82000000u32;
|
||||
let rdata_va = 0x1000u32;
|
||||
let mut pe = vec![0u8; 0x4000];
|
||||
|
||||
// Two consecutive function pointers, no vtable shadowing them.
|
||||
let pcs = [image_base + 0x2000, image_base + 0x2010];
|
||||
for (i, p) in pcs.iter().enumerate() {
|
||||
write_be_u32(&mut pe, rdata_va as usize + i * 4, *p);
|
||||
}
|
||||
|
||||
let sections = vec![mk_section(".rdata", rdata_va, 0x100)];
|
||||
let mut starts = BTreeSet::new();
|
||||
for &p in &pcs { starts.insert(p); }
|
||||
|
||||
let arrs = analyze(&pe, image_base, §ions, &starts, &[]);
|
||||
assert_eq!(arrs.len(), 1);
|
||||
assert_eq!(arrs[0].kind, "dispatch_table");
|
||||
assert_eq!(arrs[0].length, 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn vtable_overrides_dispatch_classification() {
|
||||
let image_base = 0x82000000u32;
|
||||
let rdata_va = 0x1000u32;
|
||||
let mut pe = vec![0u8; 0x4000];
|
||||
|
||||
let pcs = [image_base + 0x2000, image_base + 0x2010, image_base + 0x2020];
|
||||
for (i, p) in pcs.iter().enumerate() {
|
||||
write_be_u32(&mut pe, rdata_va as usize + i * 4, *p);
|
||||
}
|
||||
let sections = vec![mk_section(".rdata", rdata_va, 0x100)];
|
||||
let mut starts = BTreeSet::new();
|
||||
for &p in &pcs { starts.insert(p); }
|
||||
|
||||
let vt = Vtable {
|
||||
address: image_base + rdata_va,
|
||||
length: 3,
|
||||
col_address: None,
|
||||
class_name: "ANON_test".into(),
|
||||
rtti_present: false,
|
||||
base_classes_json: None,
|
||||
methods: pcs.to_vec(),
|
||||
};
|
||||
let arrs = analyze(&pe, image_base, §ions, &starts, &[vt]);
|
||||
// Vtable + (no dispatch-table dup): the M3 vtable is re-emitted, but
|
||||
// the scan also skips the same address from re-classification.
|
||||
assert_eq!(arrs.len(), 1);
|
||||
assert_eq!(arrs[0].kind, "vtable");
|
||||
}
|
||||
}
|
||||
@@ -374,6 +374,7 @@ mod tests {
|
||||
is_saverestore: false,
|
||||
pdata_validated: false,
|
||||
pdata_length: None,
|
||||
has_eh: false,
|
||||
});
|
||||
let func_analysis = FuncAnalysis {
|
||||
functions,
|
||||
@@ -414,6 +415,7 @@ mod tests {
|
||||
is_saverestore: false,
|
||||
pdata_validated: false,
|
||||
pdata_length: None,
|
||||
has_eh: false,
|
||||
});
|
||||
let func_analysis = FuncAnalysis {
|
||||
functions,
|
||||
@@ -448,6 +450,7 @@ mod tests {
|
||||
is_saverestore: false,
|
||||
pdata_validated: false,
|
||||
pdata_length: None,
|
||||
has_eh: false,
|
||||
});
|
||||
let func_analysis = FuncAnalysis {
|
||||
functions,
|
||||
|
||||
@@ -11,6 +11,7 @@ pub mod vtables;
|
||||
pub mod lookup;
|
||||
pub mod indirect;
|
||||
pub mod strings;
|
||||
pub mod funcptr_arrays;
|
||||
|
||||
mod ordinals;
|
||||
pub use ordinals::resolve_ordinal;
|
||||
|
||||
@@ -67,6 +67,7 @@ fn synthetic_func_analysis(image_base: u32) -> FuncAnalysis {
|
||||
is_saverestore: false,
|
||||
pdata_validated: false,
|
||||
pdata_length: None,
|
||||
has_eh: false,
|
||||
},
|
||||
);
|
||||
FuncAnalysis {
|
||||
@@ -106,7 +107,7 @@ fn db_schema_matches_expected_columns() {
|
||||
w.write_base(&info).expect("write_base");
|
||||
w.ingest_instructions(&pe, &info, &func_analysis, &labels)
|
||||
.expect("ingest_instructions");
|
||||
w.write_analysis_results(&pe, &info, &func_analysis, &labels, &xrefs, &[], &[])
|
||||
w.write_analysis_results(&pe, &info, &func_analysis, &labels, &xrefs, &[], &[], &[])
|
||||
.expect("write_analysis_results");
|
||||
w.create_sql_views().expect("create_sql_views");
|
||||
}
|
||||
@@ -159,6 +160,7 @@ fn db_schema_matches_expected_columns() {
|
||||
("is_saverestore", "BOOLEAN"),
|
||||
("pdata_validated", "BOOLEAN"),
|
||||
("pdata_length", "BIGINT"),
|
||||
("has_eh", "BOOLEAN"),
|
||||
]),
|
||||
("pdata_entries", &[
|
||||
("begin_address", "BIGINT"),
|
||||
@@ -208,6 +210,28 @@ fn db_schema_matches_expected_columns() {
|
||||
("length", "BIGINT"),
|
||||
("content", "VARCHAR"),
|
||||
]),
|
||||
("tls_info", &[
|
||||
("raw_data_start", "BIGINT"),
|
||||
("raw_data_end", "BIGINT"),
|
||||
("index_address", "BIGINT"),
|
||||
("callback_array", "BIGINT"),
|
||||
("zero_fill_size", "BIGINT"),
|
||||
("characteristics", "BIGINT"),
|
||||
]),
|
||||
("tls_callbacks", &[
|
||||
("slot", "BIGINT"),
|
||||
("address", "BIGINT"),
|
||||
]),
|
||||
("function_pointer_arrays", &[
|
||||
("address", "BIGINT"),
|
||||
("length", "BIGINT"),
|
||||
("kind", "VARCHAR"),
|
||||
]),
|
||||
("function_pointer_array_entries", &[
|
||||
("array_address", "BIGINT"),
|
||||
("slot", "BIGINT"),
|
||||
("function_address", "BIGINT"),
|
||||
]),
|
||||
("xrefs", &[
|
||||
("source", "BIGINT"),
|
||||
("target", "BIGINT"),
|
||||
|
||||
Reference in New Issue
Block a user