7 Commits

Author SHA1 Message Date
sim
e909c7c133 chore: retire the last dead paths and names from the consolidation
Nothing here changes what a tool computes; it changes where tools look.

- tools/re-capture: 33 censuses globbed /work/sylph_extract, a path that has
  existed nowhere since /work became a clone, so they matched nothing and
  printed empty results. They now resolve the disc through a new disc.py
  from $SYLPHEED_DISC and exit loudly without it (the #44 fix, generalised).
  Nine scripts that imported siblings from the retired Reborn checkout or an
  old session scratchpad now import from their own directory. unitgroup.py
  only needs the variable when --pak is not given.
- sylpheed-xex: the loader only ever uses the XEX2 retail key. The dead
  devkit key and a doc comment claiming a devkit fallback that does not
  exist are gone; Project Sylpheed is a retail XEX2, so no XEX1 key either.
- sylpheed-viewer: real_font_rasterizes looked for /tmp/sylph_extract and so
  always skipped. It reads $SYLPHEED_DISC now, and passes against the disc.
- Comments and docs that named xenia-rs, the Reborn repository or /work/*.pe
  as places to look now name sylpheed.db, Canary's ppc_context.h and the
  flat .pe; docs/re/README.md no longer says the native Canary build does not
  run.

Historical records keep their original paths: findings that were measured
against /work/xenia-rs/sylpheed.db still say so.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 22:30:28 +02:00
sim
2f838040ca docs: stop telling people to use the retired xenia-rs
Instructions and generated text that still sent readers to `xenia-rs`, which is
archived and deleted locally. Provenance and dated research records are left as
they are — "lifted from xenia-rs", or a finding stating what database it was
measured against, is still true, and rewriting it would falsify the record.

Changed, because each one tells someone what to do today or writes stale text:

  docs/re/README.md          the RE toolchain guide queried `xenia-rs/sylpheed.db`
                             with `xenia-rs/zq.py`, and its "Dynamic" bullet said
                             to prefer xenia-rs's probe suite OVER Canary. Now:
                             `sylpheed.db` at the repo root (how to build it and
                             re-stamp names), `tools/zq.py`, dynamic = Canary, and
                             `.rdata` reads from the `.pe` (offset = VA - 0x82000000)
                             in place of `--dump-addr`.
  challenge-mission-gate.md, structures/achievements.md
                             reproduction commands `python3 xenia-rs/zq.py dis …`
  crates/sylpheed-xexdb/SCHEMA.md
                             titled after the retired `xenia-analysis` crate,
                             citing `xenia-rs dis`, pointing at a `xenia-analysis`
                             source path
  sylpheed-xexdb/src/formatter.rs
                             WROTE "generated by xenia-rs" into every disassembly
                             it produced (no test pins the banner)
  sylph-xexdb.rs, db.rs      "SQLite" / "DuckDB writer for xenia-rs"
  sylpheed-formats/src/hash.rs
                             pointed at `xenia-rs/RE_SYMBOLS.md`, now at
                             `docs/re/RE_SYMBOLS.md`

Verified: every rewritten reproduction command was extracted from the doc and
run as written against the regenerated database — all 5 exit 0 and return the
disassembly they describe. The README's function count (25 676) is the database's
own. `cargo fmt --all -- --check` clean.

Not changed, deliberately: the README's Oracle bullet says Canary's native Linux
ELF "crashes / does not run". That is about Canary, not xenia-rs, and it conflicts
with a July note that the native build works — unverified either way here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 20:37:08 +02:00
13c895abff feat(xexdb): port import-thunk naming + schema-wide foreign keys
Carries `xenia-rs` `harvest/import-thunk-naming` (b4f19f1) across into the
lifted crate. That work was found UNCOMMITTED in the retired repo's working
tree on 2026-09-14 and exists nowhere else: absent from `iterate-4A`, absent
from this crate as lifted. CONSOLIDATION.md Phase 5 ends with "delete the local
clone" and Phase 7 drops the emulator, so it had a deletion scheduled against
it.

Not a cherry-pick. The lift's base is exactly the harvest's parent (8401d4d),
so each file was merged three-way -- lifted vs base vs harvest -- which is what
made the port reviewable: 10 conflicts, 9 of them pure rustfmt reflow from the
lift's formatting pass, and 1 semantic.

The semantic one is insertion ORDER. `xdbf_achievements.image_id` is now a
foreign key onto `xdbf_images(id)`, so the image rows must be inserted before
the achievement rows. The merge moved that block; the conflict was the stale
copy left at the old position.

What arrives:

* **Import-thunk recognition** (`imports.rs`, 338 lines). An XEX import is not
  a PLT jump: the linker emits a four-word thunk whose first two words are
  import RECORDS that the loader rewrites at module load. On disc they are
  still records, so a PowerPC-only decoder prints two meaningless `.long`s in
  front of an indirect branch. This maps every word of every thunk, and every
  direct branch into one, back to its `imports` row. Shape-validated rather
  than trusted: an entry is indexed only when the four words it points at
  actually have the thunk shape.

  Adds `instructions.import_address` (FK onto `imports.address`) and
  `import_role` (`'record'` | `'thunk'` | `'call'`, NULL iff import_address is
  NULL), plus an index. `tools/zq.py` gains `imp` and `impcalls`, and `dis`
  now names imports instead of printing `.long 0x01010194`.

* **Sixteen schema-wide foreign keys**, declared wherever a column is derived
  from another table. CREATE TABLE and insertion order become load-bearing.

  `functions` is deliberately NOT an FK parent and the golden test now asserts
  zero inbound FKs onto it: DuckDB implements UPDATE as delete+insert, so one
  inbound FK would make `functions.name` un-updatable and break
  `apply_re_symbols.sql`, which re-applies RE symbol names after every
  regeneration.

The database path needed no change in the binary: `DbWriter` builds its own
index inside `ingest_instructions` from `info.import_libraries`. Only the two
output paths (`enrich_section` for JSONL, `write_asm`) take it as an argument.

Verified: `cargo test -p sylpheed-xexdb` = 10 passed / 0 failed, including
`db_schema_golden` (41s, builds a real DuckDB) which locks the 16-FK set and
the no-FK-onto-functions rule. `cargo clippy -p sylpheed-xexdb --all-targets
-- -D warnings` clean; `cargo fmt --all --check` clean.

Stacked on #32 -- it ports into a crate that only exists there.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 20:43:16 +02:00
MechaCat02
1de202d88d test(ppc): bring the disassembler's golden fixtures across
Three tests failed on the extracted tree for a reason that was not a port
defect: disasm_goldens.rs resolves its fixtures through a relative path to
xenia-cpu/tests/golden/, and I had copied the test without the data.

The fixtures (base + extended mnemonics, VMX128 registers) and the generator
test now live in sylpheed-ppc, and the shim test in sylpheed-xexdb points at
them. serde/serde_json become dev-dependencies of sylpheed-ppc, which the
fixtures need and the library does not.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 20:43:48 +02:00
MechaCat02
c9dd2cb705 fix(xexdb): clear the lint gate on the imported crates
rustfmt, then clippy -D warnings across the three new crates. Mechanical,
except three decisions that are stated rather than silently allowed:

  * lzx.rs gets file-scoped needless_range_loop/explicit_counter_loop allows.
    Index arithmetic IS the algorithm -- LZX is defined over symbol indices,
    Huffman slots and window positions, and a decompressor that is merely
    idiomatic is worth nothing if it is not bit-exact.
  * sylpheed-xexdb gets crate-scoped allows for needless_range_loop (nine
    sites index reg[r] where r is the PowerPC register number -- the index is
    the meaning), too_many_arguments and type_complexity. This code arrived
    whole from a retired repository; a refactor here would be an unreviewed
    edit dressed as a lint fix.
  * Everything else clippy asked for is FIXED, including all 14 doc-indent
    sites, the let-else, and a Prepared type alias in the binary.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 20:25:44 +02:00
MechaCat02
62b0f79590 wip: tool README, DuckDB wording, zq.py escape hatch 2026-09-13 19:42:26 +02:00
MechaCat02
86da451bfe wip: extract the xexdb tool closure 2026-09-13 19:31:49 +02:00