Nothing here changes what a tool computes; it changes where tools look.
- tools/re-capture: 33 censuses globbed /work/sylph_extract, a path that has
existed nowhere since /work became a clone, so they matched nothing and
printed empty results. They now resolve the disc through a new disc.py
from $SYLPHEED_DISC and exit loudly without it (the #44 fix, generalised).
Nine scripts that imported siblings from the retired Reborn checkout or an
old session scratchpad now import from their own directory. unitgroup.py
only needs the variable when --pak is not given.
- sylpheed-xex: the loader only ever uses the XEX2 retail key. The dead
devkit key and a doc comment claiming a devkit fallback that does not
exist are gone; Project Sylpheed is a retail XEX2, so no XEX1 key either.
- sylpheed-viewer: real_font_rasterizes looked for /tmp/sylph_extract and so
always skipped. It reads $SYLPHEED_DISC now, and passes against the disc.
- Comments and docs that named xenia-rs, the Reborn repository or /work/*.pe
as places to look now name sylpheed.db, Canary's ppc_context.h and the
flat .pe; docs/re/README.md no longer says the native Canary build does not
run.
Historical records keep their original paths: findings that were measured
against /work/xenia-rs/sylpheed.db still say so.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Found the routines in the disassembly DB rather than guessing from data:
sub_82447DF0 IDXD tag hash (lbz+extsb, modulus 0x00FFFFDF, magic 0x2101)
sub_82447E70 IXUD tag hash (lhz, 64-bit, modulus 0xFFFFFF67 then 0x00FFFFDF)
Both transcribed instruction-for-instruction into Python and Rust.
IXUD SOLVED. It defeated every single-modulus search because it chains TWO
exact moduli -- the loop reduces mod 2^32-153 in 64-bit arithmetic and only the
result is folded mod 2^24-33. A polynomial mod M1 folded through M2 is not a
polynomial mod anything, which is exactly why the gcd test returned 1. Verified
independently: 86/86 record keys and 108,261/108,261 field tags in
GP_MAIN_GAME_E.pak, and NoRecord -> 0x1c6d9c96.
CORRECTION 1: tag_hash must SIGN-EXTEND each byte (extsb). My reconstruction
used unsigned bytes and matched all 1.27M disc names -- every one is ASCII --
while disagreeing on ~90% of random inputs with a byte >= 0x80 (verified:
18096/20000). The disc could never have caught this; only the disassembly did.
CORRECTION 2: name_hash's reduction is EXACT, not lossy. The module doc claimed
the missing conditional subtract made it something other than %. rlwinm r6,r6,
9,23,31 is just hi>>23, and with RECIP = floor(2^55/M)+1 that is Granlund-
Montgomery magic division -- 0 wrong at every quotient boundary across the full
32-bit domain. Retracted.
cargo test -p sylpheed-formats --lib hash: 10/10.
Closes the 4-byte record key. tag_hash is name_hash's shape -- byte-sum
checksum in the top byte over a 24-bit modular polynomial -- with two different
constants: modulus 0x00FFFFDF (2^24-33, prime) instead of 0x00FFF9D7, and no
lowercasing, so tags are case-sensitive. name_hash explains 0 of 8643.
Recovered from the tables rather than the executable: every inline field name
is a known (name -> tag) pair, and comparing names differing in one character
gives the per-position weights 1, 0x100, 0x10000, 0x21, 0x2100, ... -- a byte
leaving bit 24 re-enters as 33, i.e. reduction mod 2^24-33. Holds where it is
easy to get wrong (distance 8 and 9 carry correctly).
A record's key is the tag of its own name: FormationSet rosters 362/362,
UnitGroup rosters 281/281, S02 squadron names 111/111 -- so records can be
addressed by name without reading the roster first.
Implemented in Python (unitgroup.tag_hash) and Rust
(sylpheed_formats::hash::tag_hash) with 3 new unit tests carrying disc-derived
vectors; cargo test -p sylpheed-formats --lib hash is 8/8 green.
Not settled: the guest routine is unlocated, so this uses exact modular
arithmetic where the game may use a Barrett step without final fixup.
stage\UnitGroup_S<NN>.tbl is now fully readable. A squadron record is Count
member tuples -- (unit model, message set, n, identity/nameplate) -- followed by
five named fields: Count, SideID, AIID, FormationID, DisableInterval. The
property entries carry their own field names inline, so the tag hash never has
to be inverted.
Two independent self-checks validate it corpus-wide, 1160/1160 each across all
28 stage tables on the disc:
- the length identity len(fields) == Count * 4 + 5, which is what pins the
member-tuple width at 4 and the named-field count at 5;
- agreement with the file's own Enumerate_Squadrons roster, which maps record
key to squadron id independently of the per-record string offset.
Adds tools/re-capture/unitgroup.py (pure static, runs no emulator) with a
--all --check self-check mode, and commits the Stage 02 dump as evidence.
Corrections to the container layout written yesterday, all three wrong:
- the 20-byte "(tag, 0, 0, count, size) section header" does not exist. It
was the file's last 16-byte record followed by a plain npool word. The
corrected layout is uniform across all 28 files; the old one failed on 9.
- squadron ids do not use a separate string base. Every offset in the file is
relative to the one string pool. The earlier "109 of 111" score was an
artefact of the uniform 7-byte id stride and had silently shifted every
name by three entries, which is why 17 TC*-named squadrons came out as
SideID=ADAN. The roster record refuted it outright.
- the roster is not always the last record; 9 stages put it elsewhere, so it
is found by its missing Count.
Refuted and kept: the 4-byte record key is not the squadron id's name hash
(0 of 112).
Not settled: what the key encodes, the member tuple's third field n, and where
the arrival interval values live. DisableInterval is only a per-squadron flag
(Yes for 31 of 1160); the durations, triggers and arrival positions are not in
this file. Formation_*.tbl and EnumSquadron_Test.tbl are next.
