Carries `xenia-rs` `harvest/import-thunk-naming` (b4f19f1) across into the lifted crate. That work was found UNCOMMITTED in the retired repo's working tree on 2026-09-14 and exists nowhere else: absent from `iterate-4A`, absent from this crate as lifted. CONSOLIDATION.md Phase 5 ends with "delete the local clone" and Phase 7 drops the emulator, so it had a deletion scheduled against it. Not a cherry-pick. The lift's base is exactly the harvest's parent (8401d4d), so each file was merged three-way -- lifted vs base vs harvest -- which is what made the port reviewable: 10 conflicts, 9 of them pure rustfmt reflow from the lift's formatting pass, and 1 semantic. The semantic one is insertion ORDER. `xdbf_achievements.image_id` is now a foreign key onto `xdbf_images(id)`, so the image rows must be inserted before the achievement rows. The merge moved that block; the conflict was the stale copy left at the old position. What arrives: * **Import-thunk recognition** (`imports.rs`, 338 lines). An XEX import is not a PLT jump: the linker emits a four-word thunk whose first two words are import RECORDS that the loader rewrites at module load. On disc they are still records, so a PowerPC-only decoder prints two meaningless `.long`s in front of an indirect branch. This maps every word of every thunk, and every direct branch into one, back to its `imports` row. Shape-validated rather than trusted: an entry is indexed only when the four words it points at actually have the thunk shape. Adds `instructions.import_address` (FK onto `imports.address`) and `import_role` (`'record'` | `'thunk'` | `'call'`, NULL iff import_address is NULL), plus an index. `tools/zq.py` gains `imp` and `impcalls`, and `dis` now names imports instead of printing `.long 0x01010194`. * **Sixteen schema-wide foreign keys**, declared wherever a column is derived from another table. CREATE TABLE and insertion order become load-bearing. `functions` is deliberately NOT an FK parent and the golden test now asserts zero inbound FKs onto it: DuckDB implements UPDATE as delete+insert, so one inbound FK would make `functions.name` un-updatable and break `apply_re_symbols.sql`, which re-applies RE symbol names after every regeneration. The database path needed no change in the binary: `DbWriter` builds its own index inside `ingest_instructions` from `info.import_libraries`. Only the two output paths (`enrich_section` for JSONL, `write_asm`) take it as an argument. Verified: `cargo test -p sylpheed-xexdb` = 10 passed / 0 failed, including `db_schema_golden` (41s, builds a real DuckDB) which locks the 16-FK set and the no-FK-onto-functions rule. `cargo clippy -p sylpheed-xexdb --all-targets -- -D warnings` clean; `cargo fmt --all --check` clean. Stacked on #32 -- it ports into a crate that only exists there. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
205 lines
7.5 KiB
Rust
205 lines
7.5 KiB
Rust
//! Analysis-side enrichment over [`sylpheed_ppc::disasm::iter_disasm`].
|
|
//!
|
|
//! Turns a stream of decoder-only [`sylpheed_ppc::disasm::DisasmItem`]s into a
|
|
//! stream of [`RichDisasmItem`]s carrying section name + enclosing function +
|
|
//! label name. The three sinks in [`crate::sinks`] (text, JSON, DuckDB) all
|
|
//! consume `RichDisasmItem`.
|
|
|
|
use std::collections::{BTreeSet, HashMap};
|
|
|
|
use sylpheed_ppc::disasm::DisasmItem;
|
|
|
|
use crate::func::FuncAnalysis;
|
|
use crate::imports::{ImportRole, ImportSites};
|
|
|
|
/// `DisasmItem` plus the analysis context (section/function/label).
|
|
#[derive(Debug, Clone)]
|
|
pub struct RichDisasmItem<'a> {
|
|
pub item: DisasmItem,
|
|
pub section: &'a str,
|
|
pub function: Option<u32>,
|
|
pub label: Option<&'a str>,
|
|
/// True when this word is data embedded in a code section (a recovered
|
|
/// jump table or its index map, or an XEX import record), so its decoded
|
|
/// text is meaningless.
|
|
pub is_data: bool,
|
|
/// The XEX import this word belongs to, as `(thunk head VA, role)`. The
|
|
/// head VA is `imports.address` for that import, so it is written out as
|
|
/// a foreign key. `None` for the overwhelming majority of instructions.
|
|
pub import: Option<(u32, ImportRole)>,
|
|
}
|
|
|
|
/// Walk one code section, yielding rich items annotated with section name,
|
|
/// enclosing function, and label-at-address.
|
|
///
|
|
/// `function` is the function that actually *contains* the address: it is set
|
|
/// on crossing a function start and cleared again at that function's
|
|
/// `end_address`. It is deliberately `None` in the gaps between functions.
|
|
///
|
|
/// It used to be a pure rolling window — set at each start and never cleared —
|
|
/// which silently attributed every gap word to whichever function happened to
|
|
/// precede it. On the reference title that mislabelled 55,227 instructions,
|
|
/// so `WHERE function = X` returned code that is not part of X, and the
|
|
/// resulting 100% attribution rate hid the fact that `.pdata` leaves ~450 KB
|
|
/// of `.text` unclaimed.
|
|
///
|
|
/// `data_words` is the set of 4-byte-aligned addresses inside code sections
|
|
/// that are known to hold data (see [`crate::jumptables::data_word_addresses`]).
|
|
/// Rows at those addresses are still emitted — their `raw` value is the table
|
|
/// entry a consumer wants — but flagged so nothing mistakes the decoded text
|
|
/// for a real instruction.
|
|
///
|
|
/// `import_sites` resolves XEX import thunks. It contributes two things: the
|
|
/// two import-record words at each thunk head are also flagged `is_data`
|
|
/// (they are loader-rewritten data, not code, and decode to nonsense), and
|
|
/// every word of a thunk plus every direct branch into one carries the
|
|
/// import it belongs to.
|
|
pub fn enrich_section<'a>(
|
|
image: &'a [u8],
|
|
image_base: u32,
|
|
section_name: &'a str,
|
|
va_start: u32,
|
|
va_end: u32,
|
|
func_analysis: &'a FuncAnalysis,
|
|
labels: &'a HashMap<u32, String>,
|
|
data_words: &'a BTreeSet<u32>,
|
|
import_sites: &'a ImportSites,
|
|
) -> impl Iterator<Item = RichDisasmItem<'a>> + 'a {
|
|
// (start, end) of the function currently being walked.
|
|
let mut current: Option<(u32, u32)> = None;
|
|
sylpheed_ppc::disasm::iter_disasm(image, image_base, va_start, va_end).map(move |item| {
|
|
// Leaving the current function must be handled before entering the
|
|
// next: a function often starts exactly at its predecessor's end.
|
|
if let Some((_, end)) = current
|
|
&& item.addr >= end
|
|
{
|
|
current = None;
|
|
}
|
|
if let Some(fi) = func_analysis.functions.get(&item.addr) {
|
|
current = Some((item.addr, fi.end));
|
|
}
|
|
let current_func = current.map(|(start, _)| start);
|
|
let label = labels.get(&item.addr).map(|s| s.as_str());
|
|
let import = import_sites.lookup(item.addr, item.text.branch_target);
|
|
let is_data =
|
|
data_words.contains(&item.addr) || matches!(import, Some((_, ImportRole::Record)));
|
|
RichDisasmItem {
|
|
item,
|
|
section: section_name,
|
|
function: current_func,
|
|
label,
|
|
is_data,
|
|
import,
|
|
}
|
|
})
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
use crate::func::{FuncAnalysis, FuncInfo};
|
|
use std::collections::BTreeMap;
|
|
|
|
fn fi(start: u32, end: u32) -> FuncInfo {
|
|
FuncInfo {
|
|
start,
|
|
end,
|
|
frame_size: 0,
|
|
saved_gprs: 0,
|
|
is_leaf: true,
|
|
is_saverestore: false,
|
|
pdata_validated: true,
|
|
pdata_length: Some(end - start),
|
|
pdata_prolog_length: None,
|
|
has_eh: false,
|
|
}
|
|
}
|
|
|
|
/// A word in the gap between two functions belongs to neither. Before the
|
|
/// containment check this walker carried the *preceding* function forward
|
|
/// across the gap, so `WHERE function = X` returned code outside X.
|
|
#[test]
|
|
fn gap_between_functions_is_unattributed() {
|
|
let image_base = 0x82000000u32;
|
|
// 6 words: [f0 f0] [gap gap] [f1 f1]
|
|
let image = vec![0x60u8; 0x40]; // `ori` — decodes cleanly, value irrelevant
|
|
let mut functions = BTreeMap::new();
|
|
functions.insert(image_base, fi(image_base, image_base + 8));
|
|
functions.insert(image_base + 16, fi(image_base + 16, image_base + 24));
|
|
let fa = FuncAnalysis {
|
|
functions,
|
|
save_gpr_base: None,
|
|
restore_gpr_base: None,
|
|
pdata_entries: Vec::new(),
|
|
};
|
|
let labels = HashMap::new();
|
|
let data_words = BTreeSet::new();
|
|
let got: Vec<(u32, Option<u32>)> = enrich_section(
|
|
&image,
|
|
image_base,
|
|
".text",
|
|
image_base,
|
|
image_base + 24,
|
|
&fa,
|
|
&labels,
|
|
&data_words,
|
|
&ImportSites::default(),
|
|
)
|
|
.map(|r| (r.item.addr, r.function))
|
|
.collect();
|
|
|
|
assert_eq!(
|
|
got,
|
|
vec![
|
|
(image_base, Some(image_base)), // inside f0
|
|
(image_base + 4, Some(image_base)), // inside f0
|
|
(image_base + 8, None), // gap — was wrongly f0
|
|
(image_base + 12, None), // gap — was wrongly f0
|
|
(image_base + 16, Some(image_base + 16)), // f1 starts
|
|
(image_base + 20, Some(image_base + 16)),
|
|
]
|
|
);
|
|
}
|
|
|
|
/// A function starting exactly at its predecessor's `end_address` must be
|
|
/// entered, not dropped: the leave check runs before the enter check.
|
|
#[test]
|
|
fn adjacent_functions_hand_over_cleanly() {
|
|
let image_base = 0x82000000u32;
|
|
let image = vec![0x60u8; 0x40];
|
|
let mut functions = BTreeMap::new();
|
|
functions.insert(image_base, fi(image_base, image_base + 8));
|
|
functions.insert(image_base + 8, fi(image_base + 8, image_base + 16));
|
|
let fa = FuncAnalysis {
|
|
functions,
|
|
save_gpr_base: None,
|
|
restore_gpr_base: None,
|
|
pdata_entries: Vec::new(),
|
|
};
|
|
let labels = HashMap::new();
|
|
let data_words = BTreeSet::new();
|
|
let got: Vec<Option<u32>> = enrich_section(
|
|
&image,
|
|
image_base,
|
|
".text",
|
|
image_base,
|
|
image_base + 16,
|
|
&fa,
|
|
&labels,
|
|
&data_words,
|
|
&ImportSites::default(),
|
|
)
|
|
.map(|r| r.function)
|
|
.collect();
|
|
assert_eq!(
|
|
got,
|
|
vec![
|
|
Some(image_base),
|
|
Some(image_base),
|
|
Some(image_base + 8),
|
|
Some(image_base + 8),
|
|
]
|
|
);
|
|
}
|
|
}
|