fabi/Sylpheed
1
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
13c895abff7e209eea23cd68b592452146483683
Sylpheed/crates/sylpheed-xexdb/tests/db_schema_golden.rs
Fabian Hamm 13c895abff feat(xexdb): port import-thunk naming + schema-wide foreign keys 
Carries `xenia-rs` `harvest/import-thunk-naming` (b4f19f1) across into the
lifted crate. That work was found UNCOMMITTED in the retired repo's working
tree on 2026-09-14 and exists nowhere else: absent from `iterate-4A`, absent
from this crate as lifted. CONSOLIDATION.md Phase 5 ends with "delete the local
clone" and Phase 7 drops the emulator, so it had a deletion scheduled against
it.

Not a cherry-pick. The lift's base is exactly the harvest's parent (8401d4d),
so each file was merged three-way -- lifted vs base vs harvest -- which is what
made the port reviewable: 10 conflicts, 9 of them pure rustfmt reflow from the
lift's formatting pass, and 1 semantic.

The semantic one is insertion ORDER. `xdbf_achievements.image_id` is now a
foreign key onto `xdbf_images(id)`, so the image rows must be inserted before
the achievement rows. The merge moved that block; the conflict was the stale
copy left at the old position.

What arrives:

* **Import-thunk recognition** (`imports.rs`, 338 lines). An XEX import is not
  a PLT jump: the linker emits a four-word thunk whose first two words are
  import RECORDS that the loader rewrites at module load. On disc they are
  still records, so a PowerPC-only decoder prints two meaningless `.long`s in
  front of an indirect branch. This maps every word of every thunk, and every
  direct branch into one, back to its `imports` row. Shape-validated rather
  than trusted: an entry is indexed only when the four words it points at
  actually have the thunk shape.

  Adds `instructions.import_address` (FK onto `imports.address`) and
  `import_role` (`'record'` | `'thunk'` | `'call'`, NULL iff import_address is
  NULL), plus an index. `tools/zq.py` gains `imp` and `impcalls`, and `dis`
  now names imports instead of printing `.long 0x01010194`.

* **Sixteen schema-wide foreign keys**, declared wherever a column is derived
  from another table. CREATE TABLE and insertion order become load-bearing.

  `functions` is deliberately NOT an FK parent and the golden test now asserts
  zero inbound FKs onto it: DuckDB implements UPDATE as delete+insert, so one
  inbound FK would make `functions.name` un-updatable and break
  `apply_re_symbols.sql`, which re-applies RE symbol names after every
  regeneration.

The database path needed no change in the binary: `DbWriter` builds its own
index inside `ingest_instructions` from `info.import_libraries`. Only the two
output paths (`enrich_section` for JSONL, `write_asm`) take it as an argument.

Verified: `cargo test -p sylpheed-xexdb` = 10 passed / 0 failed, including
`db_schema_golden` (41s, builds a real DuckDB) which locks the 16-FK set and
the no-FK-onto-functions rule. `cargo clippy -p sylpheed-xexdb --all-targets
-- -D warnings` clean; `cargo fmt --all --check` clean.

Stacked on #32 -- it ports into a crate that only exists there.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 20:43:16 +02:00

629 lines
20 KiB
Rust
Raw Blame History

//! DB schema golden — locks the column layout (names + types) of every
//! table written by `DbWriter`. A schema change here without a fixture
//! update fails the test, forcing a conscious decision before downstream
//! query consumers break.
//!
//! The fixture is constructed in-process (no XEX/ISO needed): a small
//! synthetic PE-shaped byte slice with one `.text` section of 4
//! instructions, plus an empty import-library list and one detected
//! function.
use std::collections::{BTreeMap, BTreeSet, HashMap};
use std::io::Write;
use duckdb::Connection;
use sylpheed_xex::pe::PeSection;
use sylpheed_xexdb::DbWriter;
use sylpheed_xexdb::formatter::DisasmInfo;
use sylpheed_xexdb::func::{FuncAnalysis, FuncInfo};
use sylpheed_xexdb::rtti::RttiResult;
use sylpheed_xexdb::xref::XrefMap;
/// Build a 16-byte `.text` section: 4 instructions (mflr / nop / blr / nop).
fn synthetic_pe() -> (
Vec<u8>,
Vec<PeSection>,
Vec<sylpheed_xex::header::ImportLibrary>,
) {
// VA layout: image_base + 0x1000 = .text start (so RVA = 0x1000).
// The DB writer expects pe[rva] to hold the byte at that RVA, so the
// buffer must be at least 0x1000 + section_size bytes long.
const RVA: usize = 0x1000;
const TEXT: [u32; 4] = [
// mfspr r12, LR (a.k.a. mflr r12) — opcode 31, xo 339, spr 8 (LR).
// Encoded with spr halves swapped per the ISA: spr_field = (8<<5).
(31u32 << 26) | (12 << 21) | ((8 << 5) << 11) | (339 << 1),
0x60000000, // nop (ori r0, r0, 0)
(19u32 << 26) | (20 << 21) | (16 << 1), // blr (bclr 20, 0)
0x60000000, // nop
];
let mut pe = vec![0u8; RVA + 16];
for (i, &word) in TEXT.iter().enumerate() {
pe[RVA + i * 4..RVA + i * 4 + 4].copy_from_slice(&word.to_be_bytes());
}
let sections = vec![PeSection {
name: ".text".to_string(),
virtual_address: 0x1000,
virtual_size: 16,
raw_offset: 0x1000,
raw_size: 16,
flags: 0x60000020, // CODE | EXECUTE | READ
}];
let import_libraries = vec![]; // No imports in the fixture.
(pe, sections, import_libraries)
}
fn synthetic_func_analysis(image_base: u32) -> FuncAnalysis {
// Single function covering all four .text instructions.
let entry = image_base + 0x1000;
let mut functions = BTreeMap::new();
functions.insert(
entry,
FuncInfo {
start: entry,
end: entry + 16,
frame_size: 0,
saved_gprs: 0,
is_leaf: true,
is_saverestore: false,
pdata_validated: false,
pdata_length: None,
pdata_prolog_length: None,
has_eh: false,
},
);
FuncAnalysis {
functions,
save_gpr_base: None,
restore_gpr_base: None,
pdata_entries: Vec::new(),
}
}
#[test]
fn db_schema_matches_expected_columns() {
let (pe, sections, libs) = synthetic_pe();
let image_base = 0x82000000u32;
let entry = image_base + 0x1000;
let info = DisasmInfo {
image_base,
entry_point: entry,
original_pe_name: Some("synthetic.exe"),
title_id: Some(0xDEADBEEF),
media_id: Some(0xCAFEF00D),
sections: &sections,
import_libraries: &libs,
xex_header: None,
};
let func_analysis = synthetic_func_analysis(image_base);
let mut labels: HashMap<u32, String> = HashMap::new();
labels.insert(entry, "entry_point".to_string());
let xrefs: XrefMap = XrefMap::new();
let tmp = std::env::temp_dir().join("sylpheed_xexdb_schema_golden.duckdb");
let _ = std::fs::remove_file(&tmp);
{
let mut w = DbWriter::open_fresh(&tmp).expect("open fresh DB");
w.write_base(&info).expect("write_base");
w.ingest_instructions(&pe, &info, &func_analysis, &labels, &BTreeSet::new())
.expect("ingest_instructions");
w.write_analysis_results(
&pe,
&info,
&func_analysis,
&labels,
&xrefs,
&[],
&[],
&[],
None,
&[],
&[],
&RttiResult::default(),
None,
)
.expect("write_analysis_results");
w.create_sql_views().expect("create_sql_views");
}
let conn = Connection::open(&tmp).expect("reopen DB");
// Lock the column layout per table. Pairs are (name, type).
let expected: &[(&str, &[(&str, &str)])] = &[
("metadata", &[("key", "VARCHAR"), ("value", "VARCHAR")]),
(
"sections",
&[
("name", "VARCHAR"),
("virtual_address", "BIGINT"),
("virtual_size", "BIGINT"),
("raw_offset", "BIGINT"),
("raw_size", "BIGINT"),
("flags", "BIGINT"),
("is_code", "BOOLEAN"),
],
),
(
"imports",
&[
("library", "VARCHAR"),
("ordinal", "BIGINT"),
("name", "VARCHAR"),
("record_type", "BIGINT"),
("address", "BIGINT"),
],
),
(
"instructions",
&[
("address", "BIGINT"),
("raw", "BIGINT"),
("mnemonic", "VARCHAR"),
("operands", "VARCHAR"),
("disasm", "VARCHAR"),
("ext_mnemonic", "VARCHAR"),
("ext_operands", "VARCHAR"),
("ext_disasm", "VARCHAR"),
("target_hex", "BIGINT"),
("section", "VARCHAR"),
("function", "BIGINT"),
("label", "VARCHAR"),
("is_data", "BOOLEAN"),
("import_address", "BIGINT"),
("import_role", "VARCHAR"),
],
),
(
"functions",
&[
("address", "BIGINT"),
("name", "VARCHAR"),
("end_address", "BIGINT"),
("frame_size", "BIGINT"),
("saved_gprs", "BIGINT"),
("is_leaf", "BOOLEAN"),
("is_saverestore", "BOOLEAN"),
("pdata_validated", "BOOLEAN"),
("pdata_length", "BIGINT"),
("prolog_length", "BIGINT"),
("has_eh", "BOOLEAN"),
],
),
(
"jump_tables",
&[
("bctr_pc", "BIGINT"),
("function", "BIGINT"),
("table_address", "BIGINT"),
("entry_count", "BIGINT"),
("table_slots", "BIGINT"),
("index_map_address", "BIGINT"),
("index_map_count", "BIGINT"),
("case_bound", "BIGINT"),
("kind", "VARCHAR"),
],
),
(
"jump_table_entries",
&[
("bctr_pc", "BIGINT"),
("case_index", "BIGINT"),
("target_address", "BIGINT"),
],
),
(
"data_in_code",
&[
("address", "BIGINT"),
("length", "BIGINT"),
("kind", "VARCHAR"),
],
),
(
"rtti_type_descriptors",
&[
("address", "BIGINT"),
("mangled_name", "VARCHAR"),
("demangled_name", "VARCHAR"),
],
),
(
"rtti_locators",
&[
("address", "BIGINT"),
("subobject_offset", "BIGINT"),
("cd_offset", "BIGINT"),
("type_descriptor", "BIGINT"),
("class_hierarchy", "BIGINT"),
("vtable_address", "BIGINT"),
],
),
(
"rtti_base_classes",
&[
("class_hierarchy", "BIGINT"),
("base_index", "BIGINT"),
("type_descriptor", "BIGINT"),
("name", "VARCHAR"),
("num_contained_bases", "BIGINT"),
("mdisp", "BIGINT"),
("pdisp", "BIGINT"),
("vdisp", "BIGINT"),
("attributes", "BIGINT"),
],
),
(
"pdata_entries",
&[
("begin_address", "BIGINT"),
("end_address", "BIGINT"),
("function_length", "BIGINT"),
("prolog_length", "BIGINT"),
("flags", "BIGINT"),
],
),
(
"labels",
&[
("address", "BIGINT"),
("name", "VARCHAR"),
("kind", "VARCHAR"),
],
),
(
"xdbf_entries",
&[
("namespace", "BIGINT"),
("namespace_name", "VARCHAR"),
("id", "BIGINT"),
("body_offset", "BIGINT"),
("size", "BIGINT"),
("magic", "VARCHAR"),
],
),
(
"xdbf_achievements",
&[
("id", "BIGINT"),
("name", "VARCHAR"),
("unlocked_desc", "VARCHAR"),
("locked_desc", "VARCHAR"),
("label_id", "BIGINT"),
("description_id", "BIGINT"),
("unachieved_id", "BIGINT"),
("image_id", "BIGINT"),
("gamerscore", "BIGINT"),
("flags", "BIGINT"),
],
),
(
"xdbf_strings",
&[
("language", "BIGINT"),
("language_name", "VARCHAR"),
("string_id", "BIGINT"),
("value", "VARCHAR"),
],
),
(
"xdbf_images",
&[
("id", "BIGINT"),
("is_title_icon", "BOOLEAN"),
("body_offset", "BIGINT"),
("size", "BIGINT"),
("format", "VARCHAR"),
],
),
(
"demangled_names",
&[
("address", "BIGINT"),
("mangled", "VARCHAR"),
("raw_demangled", "VARCHAR"),
("namespace_path", "VARCHAR"),
("class_name", "VARCHAR"),
("method_name", "VARCHAR"),
("params_signature", "VARCHAR"),
],
),
(
"vtables",
&[
("address", "BIGINT"),
("length", "BIGINT"),
("col_address", "BIGINT"),
("class_name", "VARCHAR"),
("rtti_present", "BOOLEAN"),
("base_classes_json", "VARCHAR"),
],
),
(
"methods",
&[
("vtable_address", "BIGINT"),
("slot", "BIGINT"),
("function_address", "BIGINT"),
("mangled_name", "VARCHAR"),
("demangled_name", "VARCHAR"),
],
),
(
"classes",
&[
("name", "VARCHAR"),
("vtable_address", "BIGINT"),
("rtti_present", "BOOLEAN"),
("base_classes_json", "VARCHAR"),
],
),
(
"strings",
&[
("address", "BIGINT"),
("encoding", "VARCHAR"),
("length", "BIGINT"),
("content", "VARCHAR"),
("section", "VARCHAR"),
],
),
(
"tls_info",
&[
("raw_data_start", "BIGINT"),
("raw_data_end", "BIGINT"),
("index_address", "BIGINT"),
("callback_array", "BIGINT"),
("zero_fill_size", "BIGINT"),
("characteristics", "BIGINT"),
],
),
(
"tls_callbacks",
&[("slot", "BIGINT"), ("address", "BIGINT")],
),
(
"function_pointer_arrays",
&[
("address", "BIGINT"),
("length", "BIGINT"),
("kind", "VARCHAR"),
],
),
(
"function_pointer_array_entries",
&[
("array_address", "BIGINT"),
("slot", "BIGINT"),
("function_address", "BIGINT"),
],
),
(
"indirect_dispatch_sites",
&[
("dispatch_pc", "BIGINT"),
("vptr_offset", "BIGINT"),
("slot", "BIGINT"),
("candidate_count", "BIGINT"),
("truncated", "BOOLEAN"),
],
),
(
"indirect_dispatch_candidates",
&[
("dispatch_pc", "BIGINT"),
("vtable_address", "BIGINT"),
("method_address", "BIGINT"),
],
),
(
"vptr_writes",
&[
("writer_pc", "BIGINT"),
("vtable_address", "BIGINT"),
("vptr_offset", "BIGINT"),
("writer_function", "BIGINT"),
],
),
(
"eh_funcinfo",
&[
("address", "BIGINT"),
("magic", "BIGINT"),
("max_state", "BIGINT"),
("p_unwind_map", "BIGINT"),
("n_try_blocks", "BIGINT"),
("p_try_block_map", "BIGINT"),
("n_ip_map_entries", "BIGINT"),
("p_ip_to_state_map", "BIGINT"),
("p_es_type_list", "BIGINT"),
("eh_flags", "BIGINT"),
],
),
(
"eh_unwind_map",
&[
("funcinfo_address", "BIGINT"),
("state_index", "BIGINT"),
("to_state", "BIGINT"),
("action_pc", "BIGINT"),
],
),
(
"eh_try_blocks",
&[
("funcinfo_address", "BIGINT"),
("try_index", "BIGINT"),
("try_low", "BIGINT"),
("try_high", "BIGINT"),
("catch_high", "BIGINT"),
("n_catches", "BIGINT"),
("p_handler_array", "BIGINT"),
],
),
(
"xrefs",
&[
("source", "BIGINT"),
("target", "BIGINT"),
("kind", "VARCHAR"),
("addr_mode", "VARCHAR"),
("instruction", "VARCHAR"),
("source_func", "BIGINT"),
("source_label", "VARCHAR"),
("target_label", "VARCHAR"),
],
),
];
let mut errs: Vec<String> = Vec::new();
for (table, cols) in expected {
let mut stmt = conn
.prepare(&format!("PRAGMA table_info('{}')", table))
.unwrap_or_else(|e| panic!("prepare PRAGMA for {table}: {e}"));
let rows: Vec<(String, String)> = stmt
.query_map([], |row| {
let name: String = row.get(1)?;
let ty: String = row.get(2)?;
Ok((name, ty))
})
.expect("query")
.map(|r| r.unwrap())
.collect();
if rows.len() != cols.len() {
writeln!(
std::io::stderr(),
"{table}: column count mismatch (got {}, expected {})",
rows.len(),
cols.len()
)
.ok();
errs.push(format!("{table}: count {} vs {}", rows.len(), cols.len()));
}
for (i, (got, expected_col)) in rows.iter().zip(cols.iter()).enumerate() {
if got.0 != expected_col.0 || got.1 != expected_col.1 {
errs.push(format!(
"{table} col {i}: got ({}, {}) expected ({}, {})",
got.0, got.1, expected_col.0, expected_col.1
));
}
}
}
assert!(
errs.is_empty(),
"schema drift detected:\n {}",
errs.join("\n ")
);
// Verify row counts in the populated tables.
let n_instr: i64 = conn
.query_row("SELECT COUNT(*) FROM instructions", [], |r| r.get(0))
.unwrap();
assert_eq!(
n_instr, 4,
"expected 4 instruction rows from the synthetic PE"
);
// The synthetic mflr should produce target_hex = NULL, blr likewise (indirect).
let n_with_target: i64 = conn
.query_row("SELECT COUNT(target_hex) FROM instructions", [], |r| {
r.get(0)
})
.unwrap();
assert_eq!(
n_with_target, 0,
"indirect-only fixture should have no direct branch targets"
);
// SQL views must be queryable. The `_` in SQL LIKE is a single-char
// wildcard, so we list the names explicitly rather than `LIKE 'v_%'`
// (which also matches DuckDB's built-in `views` system view).
let expected_views = [
"v_branch_xrefs",
"v_call_graph",
"v_function_first_instruction",
"v_imports_called",
"v_indirect_reachability_from_entry",
"v_reachability_from_entry",
];
for v in expected_views {
let exists: i64 = conn
.query_row(
"SELECT COUNT(*) FROM duckdb_views() WHERE view_name = ?",
[v],
|r| r.get(0),
)
.unwrap();
assert_eq!(exists, 1, "missing SQL view: {v}");
}
// ── Foreign keys ────────────────────────────────────────────────────
//
// Lock the declared FK set. These are correctness assertions baked into
// the schema: a build that would produce a dangling reference fails at
// insert time rather than shipping a broken database.
let expected_fks = [
("classes", "vtables"),
("data_in_code", "instructions"),
("eh_try_blocks", "eh_funcinfo"),
("eh_unwind_map", "eh_funcinfo"),
("function_pointer_array_entries", "function_pointer_arrays"),
("indirect_dispatch_candidates", "indirect_dispatch_sites"),
("indirect_dispatch_candidates", "vtables"),
("instructions", "imports"),
("jump_table_entries", "jump_tables"),
("methods", "vtables"),
("rtti_base_classes", "rtti_type_descriptors"),
("rtti_locators", "rtti_type_descriptors"),
("vptr_writes", "vtables"),
("vtables", "rtti_locators"),
("xdbf_achievements", "xdbf_images"),
("xrefs", "instructions"),
];
for (child, parent) in expected_fks {
let n: i64 = conn
.query_row(
"SELECT COUNT(*) FROM duckdb_constraints() \
WHERE constraint_type = 'FOREIGN KEY' AND table_name = ? \
AND referenced_table = ?",
[child, parent],
|r| r.get(0),
)
.unwrap();
assert_eq!(n, 1, "missing foreign key: {child} -> {parent}");
}
// `functions` must never become a foreign-key parent. DuckDB implements
// UPDATE as delete-then-insert, so a single inbound FK makes the table
// un-updatable — even for an update that does not touch `address` — and
// that breaks `apply_re_symbols.sql`, which re-stamps reverse-engineered
// names onto `functions.name` after every regeneration. Adding one back
// looks harmless in review and only fails at symbol-stamping time, hence
// this guard.
let inbound_to_functions: i64 = conn
.query_row(
"SELECT COUNT(*) FROM duckdb_constraints() \
WHERE constraint_type = 'FOREIGN KEY' \
AND referenced_table = 'functions'",
[],
|r| r.get(0),
)
.unwrap();
assert_eq!(
inbound_to_functions, 0,
"no table may declare a foreign key onto `functions`: it would make \
functions.name un-updatable and break apply_re_symbols.sql",
);
let _ = std::fs::remove_file(&tmp);
}
Reference in New Issue View Git Blame Copy Permalink