Instructions and generated text that still sent readers to `xenia-rs`, which is
archived and deleted locally. Provenance and dated research records are left as
they are — "lifted from xenia-rs", or a finding stating what database it was
measured against, is still true, and rewriting it would falsify the record.
Changed, because each one tells someone what to do today or writes stale text:
docs/re/README.md the RE toolchain guide queried `xenia-rs/sylpheed.db`
with `xenia-rs/zq.py`, and its "Dynamic" bullet said
to prefer xenia-rs's probe suite OVER Canary. Now:
`sylpheed.db` at the repo root (how to build it and
re-stamp names), `tools/zq.py`, dynamic = Canary, and
`.rdata` reads from the `.pe` (offset = VA - 0x82000000)
in place of `--dump-addr`.
challenge-mission-gate.md, structures/achievements.md
reproduction commands `python3 xenia-rs/zq.py dis …`
crates/sylpheed-xexdb/SCHEMA.md
titled after the retired `xenia-analysis` crate,
citing `xenia-rs dis`, pointing at a `xenia-analysis`
source path
sylpheed-xexdb/src/formatter.rs
WROTE "generated by xenia-rs" into every disassembly
it produced (no test pins the banner)
sylph-xexdb.rs, db.rs "SQLite" / "DuckDB writer for xenia-rs"
sylpheed-formats/src/hash.rs
pointed at `xenia-rs/RE_SYMBOLS.md`, now at
`docs/re/RE_SYMBOLS.md`
Verified: every rewritten reproduction command was extracted from the doc and
run as written against the regenerated database — all 5 exit 0 and return the
disassembly they describe. The README's function count (25 676) is the database's
own. `cargo fmt --all -- --check` clean.
Not changed, deliberately: the README's Oracle bullet says Canary's native Linux
ELF "crashes / does not run". That is about Canary, not xenia-rs, and it conflicts
with a July note that the native build works — unverified either way here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Run 204 gave this repository its first clippy measurement — 48 errors, 43
of them in `sylpheed-formats`. This clears that 43 to zero under the exact
invocation CI runs, `cargo clippy -p sylpheed-formats -- -D warnings`.
Why this crate first, and why it is safe to touch:
Every one of the 43 sites was checked against the line ranges that
`auto/frame-blend-draw-path` (495 commits) and `auto/port-p6-audio` (366)
actually modify. None of them overlap. Eleven of the fifteen affected
files are byte-identical on both branches, including `mesh.rs` and
`texture.rs`, which carry 28 of the hits between them. The three sites in
`audio.rs`, `ui_layout.rs` and `slb.rs` that live in files those branches
do change fall outside every modified hunk. The collision argument that
defers #12 does not transfer here; it was tested rather than assumed.
It also unblocks a measurement. `-D warnings` turns a lint in this crate
into a hard compile error, so its dependents never build — `sylpheed-cli`
and `sylpheed-viewer` have never been linted at all, and viewer is the
largest crate in the workspace. Both depend only on `sylpheed-formats`
(`sylpheed-export` pins it from a git tag instead), so this commit is what
makes their real counts knowable.
38 applied by `cargo clippy --fix` — chunks_exact_to_as_chunks,
manual_div_ceil / is_multiple_of / range_contains, unnecessary_map_or,
needless_borrow, let_and_return, dead_code, unused_mut/variables.
Purely local expression rewrites: 38 insertions, 39 deletions.
2 by hand: a doc continuation that markdown was parsing as a list, and
`d / frame` behind a `frame > 0` guard becoming `checked_div`.
3 `#[allow(clippy::too_many_arguments)]` with a stated reason.
On those three allows: 8 parameters against a threshold of 7, in the mesh
anchor path. The real fix is a shared params struct across
`anchor_pool_mesh`, `validate_block` and `validate_block_report` — the
latter two take the same eight arguments and one delegates to the other —
which is a change to the decoder's signatures and belongs to whoever owns
that path, not to a CI-lint pass.
This is not the shape PROTOCOL.md forbids. `continue-on-error` suppresses
everything, present and future, at the job level, and cannot tell "not
yet" from "no longer". A site-local `#[allow]` with a reason is a decision
recorded where it applies: one lint, one function, and any new violation
anywhere else still fails the build.
`sylpheed-export`'s remaining 5 are deliberately untouched — three of them
sit inside hunks both long-lived branches modify, and that crate blocks
nothing. Left for #13.
Refs #13
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01McNbzUeq1KRBWs4G6X2YVj
Found the routines in the disassembly DB rather than guessing from data:
sub_82447DF0 IDXD tag hash (lbz+extsb, modulus 0x00FFFFDF, magic 0x2101)
sub_82447E70 IXUD tag hash (lhz, 64-bit, modulus 0xFFFFFF67 then 0x00FFFFDF)
Both transcribed instruction-for-instruction into Python and Rust.
IXUD SOLVED. It defeated every single-modulus search because it chains TWO
exact moduli -- the loop reduces mod 2^32-153 in 64-bit arithmetic and only the
result is folded mod 2^24-33. A polynomial mod M1 folded through M2 is not a
polynomial mod anything, which is exactly why the gcd test returned 1. Verified
independently: 86/86 record keys and 108,261/108,261 field tags in
GP_MAIN_GAME_E.pak, and NoRecord -> 0x1c6d9c96.
CORRECTION 1: tag_hash must SIGN-EXTEND each byte (extsb). My reconstruction
used unsigned bytes and matched all 1.27M disc names -- every one is ASCII --
while disagreeing on ~90% of random inputs with a byte >= 0x80 (verified:
18096/20000). The disc could never have caught this; only the disassembly did.
CORRECTION 2: name_hash's reduction is EXACT, not lossy. The module doc claimed
the missing conditional subtract made it something other than %. rlwinm r6,r6,
9,23,31 is just hi>>23, and with RECIP = floor(2^55/M)+1 that is Granlund-
Montgomery magic division -- 0 wrong at every quotient boundary across the full
32-bit domain. Retracted.
cargo test -p sylpheed-formats --lib hash: 10/10.
Closes the 4-byte record key. tag_hash is name_hash's shape -- byte-sum
checksum in the top byte over a 24-bit modular polynomial -- with two different
constants: modulus 0x00FFFFDF (2^24-33, prime) instead of 0x00FFF9D7, and no
lowercasing, so tags are case-sensitive. name_hash explains 0 of 8643.
Recovered from the tables rather than the executable: every inline field name
is a known (name -> tag) pair, and comparing names differing in one character
gives the per-position weights 1, 0x100, 0x10000, 0x21, 0x2100, ... -- a byte
leaving bit 24 re-enters as 33, i.e. reduction mod 2^24-33. Holds where it is
easy to get wrong (distance 8 and 9 carry correctly).
A record's key is the tag of its own name: FormationSet rosters 362/362,
UnitGroup rosters 281/281, S02 squadron names 111/111 -- so records can be
addressed by name without reading the roster first.
Implemented in Python (unitgroup.tag_hash) and Rust
(sylpheed_formats::hash::tag_hash) with 3 new unit tests carrying disc-derived
vectors; cargo test -p sylpheed-formats --lib hash is 8/8 green.
Not settled: the guest routine is unlocated, so this uses exact modular
arithmetic where the game may use a Barrett step without final fixup.
Add hash::recover_toc_name + TOC_NAME_SCHEMES (confirmed path schemes:
unit\<ID>.tbl, weapon\<ID>.tbl, message\<ID>.tbl, effect\<ID>.tbl,
<name>.tbl) that reproduce an entry's original backslash path from its
internal identity string via the recovered name-hash.
`pak list` now extracts identifier candidates from each IDXD entry
(ID/Name/Model + pool tokens) and prints the resolved path when a scheme
matches — e.g. unit\UN_f001_TCAF_DeltaSaber_T.tbl,
weapon\Weapon_…_Missile.tbl, message\CharacterCARL.tbl — plus a
name-resolved count. 308/1004 resolved on GP_MAIN_GAME_E; the remainder
use deeper cross-referenced paths (deferred).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The pak TOC key was previously an unknown hash (crc32/fnv/djb2/… ruled
out), forcing content-only lookup. Recovered it by static RE of the
retail title:
sub_824609C8 (pak lookup-by-name) dups the requested path, lowercases
it (sub_825F4F90), hashes with sub_82455C78, then binary-searches the
sorted TOC. sub_82455C78 is a per-byte Barrett-reduced polynomial hash
(modulus 0x00FFF9D7, reciprocal 0x80031493): the low 24 bits are the
modular hash, the top byte an 8-bit additive checksum of the bytes.
New `hash` module reproduces it exactly (faithful op sequence, no
textbook %). Verified against the real disc: name_hash("files.tbl") ==
0x83421153 and name_hash("eng\\weapon.tbl") == 0x900C8DCD, both present
in retail TOCs. Add PakArchive::find_by_name / read_by_name and a disc
integration test resolving eng\weapon.tbl by path (→ valid IDXD).
Note: retail paths use backslash separators (eng\weapon.tbl).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
