From 6b248d71e9cccd4f94c0bb8801af3afaacf2667b Mon Sep 17 00:00:00 2001 From: MechaCat02 Date: Thu, 9 Jul 2026 21:45:26 +0200 Subject: [PATCH] docs: RE symbol map + DB re-stamp for cracked IPFB name-hash MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Record the guest functions identified while cracking the IPFB TOC name-hash: Pak_FindEntryByName (0x824609C8), Pak_HashPathName (0x82460928), Sylph_NameHash (0x82455C78, the Barrett poly-hash), Str_ToLowerAscii (0x825F4F90), Pak_IsIPFBHeader (0x824607B0), Archive_StreamReadCrc32 (0x82458508, content CRC — not the name-hash), Res3D_LoadMeshChunk (0x82640290), and g_Crc32Table (0x828992F0). RE_SYMBOLS.md is the source of truth (names/addresses/signatures + confidence, the name-hash algorithm, and the confirmed TOC key path schemes unit\.tbl / weapon\.tbl). apply_re_symbols.sql re-stamps functions.name onto sylpheed.db (wiped on regen); already applied. Co-Authored-By: Claude Opus 4.8 --- RE_SYMBOLS.md | 74 ++++++++++++++++++++++++++++++++++++++++++++ apply_re_symbols.sql | 13 ++++++++ 2 files changed, 87 insertions(+) create mode 100644 RE_SYMBOLS.md create mode 100644 apply_re_symbols.sql diff --git a/RE_SYMBOLS.md b/RE_SYMBOLS.md new file mode 100644 index 0000000..d784b67 --- /dev/null +++ b/RE_SYMBOLS.md @@ -0,0 +1,74 @@ +# Project Sylpheed — reverse-engineered symbol map + +Durable record of guest functions/data in the retail title (`default.xex`, +base `0x82000000`) that have been positively identified by RE. Addresses are +guest virtual addresses. This is the source of truth for names; the DuckDB +`sylpheed.db` (this dir) `functions.name` column can be re-stamped from here +with `apply_re_symbols.sql` (names are wiped on a full DB regen). + +Confidence: **H** = behaviour proven (reproduced/verified against data), **M** = +strongly inferred from disassembly, **L** = tentative. + +## Archive (IPFB `.pak`) subsystem + +| Address | Name | Conf | Description | +|---|---|---|---| +| `0x824609C8` | `Pak_FindEntryByName` | H | Look up a TOC entry by path. Args `(r3 = pak object, r4 = char* path)`. Calls `Pak_HashPathName(path)`, then binary-searches the sorted 12-byte TOC (base `pak+4`, count `pak+8`, stride 12; compares the 32-bit `name_hash`). Returns the matching record pointer or 0. | +| `0x82460928` | `Pak_HashPathName` | H | Duplicate `r3 = char* path`, lowercase it in place (`Str_ToLowerAscii`), hash with `Sylph_NameHash`, free the copy, return the 32-bit hash. This is the path→key bridge. | +| `0x82455C78` | `Sylph_NameHash` | H | The IPFB TOC / cache-path name-hash. See [Name-hash](#name-hash-sylph_namehash). Input must already be lowercased. `r3 = char*` → `r3 = u32 hash`. | +| `0x825F4F90` | `Str_ToLowerAscii` | H | In-place ASCII lowercase (`A`–`Z` → `+0x20`); other bytes pass through. `r3 = char*`. (When `r3 == 0` it takes an unrelated init branch.) | +| `0x824607B0` | `Pak_IsIPFBHeader` | H | Validate an IPFB header. Loads the header's first word and checks it equals `"IPFB"` (`0x49504642`) via a delta trick: `first_word - 0x49504620 == 0x22` (`'B' - ' '`). `r5 = header ptr`. Returns bool-ish in `r3`. | +| `0x82460AD0`… | *(pak header parse, unregistered container)* | M | The pak-open/validate path around `0x82460DC0`–`0x82460EC4` builds an expected-header template (`"IPFB"`, `0x10000000` flags) on the stack and calls `Pak_IsIPFBHeader`. Function-boundary detection missed the enclosing frame (entry ≈ `0x82460E34`). | +| `0x82458508` | `Archive_StreamReadCrc32` | M | Streaming block reader that validates content with a reflected CRC-32 (init `0xFFFFFFFF`, final `~`, table at `g_Crc32Table`). Compares the computed CRC to a stored expected value (`state+144`). This is a **content-integrity** CRC, *not* the name-hash. | +| `0x828992F0` | `g_Crc32Table` (data) | H | 256-entry reflected CRC-32 lookup table. Referenced only by `Archive_StreamReadCrc32` and its sibling loop at `0x82457AF8`. | + +## 3D resource subsystem + +| Address | Name | Conf | Description | +|---|---|---|---| +| `0x82640290` | `Res3D_LoadMeshChunk` | L | 3D-resource loader: walks 12-byte records, checks chunk tags (`0x1A22AA26`, `0x2DA2AA24`), and copies `float` triples (x/y/z vertices) via `lfs`/`stfs`. Part of the DefTables/`machines\\…` mesh path. | + +## Name-hash (`Sylph_NameHash`) + +Per-byte Barrett-reduced polynomial hash over the **lowercased** name: + +``` +A = 0 ; B = 0 +for each (sign-extended) byte c of the lowercased name: + A = (A << 8) + c # 32-bit + A = A - (((A * 0x8003_1493) >> 32) rol 9 & 0x1FF) * 0x00FF_F9D7 # A mod 0x00FF_F9D7 (no final fixup) + B = B + c +hash = ((B & 0xFF) << 24) | (A & 0x00FF_FFFF) +``` + +- Modulus `0x00FF_F9D7`, reciprocal `0x8003_1493` (the `mulhwu`/`mullw` Barrett step). +- Low 24 bits = modular polynomial hash; top byte = 8-bit additive checksum of the bytes. +- No trailing conditional subtract — the value is defined by the exact op sequence. +- Faithful Rust reproduction: `sylpheed-formats/src/hash.rs` (`name_hash`). + +**Verified** against retail TOCs: `name_hash("files.tbl") == 0x8342_1153`, +`name_hash("eng\\weapon.tbl") == 0x900C_8DCD`, `name_hash("eng\\strings.tbl") == 0x10C8_0B87`, +`name_hash("jpn\\weapon.tbl") == 0x9E85_FEFF`. + +## TOC key path conventions (preimages) + +The 32-bit TOC key is `name_hash` of a **backslash** path with the entry's +internal identity string. Confirmed schemes: + +| Scheme | Example | Where | +|---|---|---| +| `unit\.tbl` | `unit\UN_f001_TCAF_DeltaSaber_T_EX5.tbl` → `0x7C96296C` | craft/ship entries in `GP_MAIN_GAME_*.pak` | +| `weapon\.tbl` | `weapon\Weapon_DSaber_P_wep_26_Missile.tbl` | weapon entries | +| `\.tbl` | `eng\weapon.tbl`, `jpn\strings.tbl` | localized loadout/text tables | +| `.tbl` | `files.tbl` | root manifest | + +`` = the entry's internal `ID` field (IDXD string pool). Craft/weapon +entries also self-describe by content, so stats are readable without the key. + +## Open threads + +- **Defaulted IDXD fields** (`*Ratio`, the `*Count` family) carry no value on + disc. The large 16-byte-record binary node table inside a craft IDXD is + **spatial/mesh data**, not defaults (ruled out: it is not keyed by + `name_hash(field_key)`). Defaults must live in per-`schema_hash` + schema-definition tables (e.g. craft schema `0x43FAA517`) — location TBD. diff --git a/apply_re_symbols.sql b/apply_re_symbols.sql new file mode 100644 index 0000000..bba0268 --- /dev/null +++ b/apply_re_symbols.sql @@ -0,0 +1,13 @@ +-- Re-stamp reverse-engineered function names onto sylpheed.db. +-- Source of truth: docs/RE_SYMBOLS.md. Re-run after any full DB regen +-- (regen wipes functions.name back to sub_XXXX). +-- python3 -c "import duckdb; duckdb.connect('sylpheed.db').execute(open('apply_re_symbols.sql').read())" +-- Addresses are decimal (DuckDB rejects 0x literals in some contexts); hex in comments. + +UPDATE functions SET name = 'Pak_FindEntryByName' WHERE address = 2185628104; -- 0x824609C8 +UPDATE functions SET name = 'Pak_HashPathName' WHERE address = 2185627944; -- 0x82460928 +UPDATE functions SET name = 'Sylph_NameHash' WHERE address = 2185583736; -- 0x82455C78 +UPDATE functions SET name = 'Str_ToLowerAscii' WHERE address = 2187284368; -- 0x825F4F90 +UPDATE functions SET name = 'Pak_IsIPFBHeader' WHERE address = 2185627568; -- 0x824607B0 +UPDATE functions SET name = 'Archive_StreamReadCrc32' WHERE address = 2185594120; -- 0x82458508 +UPDATE functions SET name = 'Res3D_LoadMeshChunk' WHERE address = 2187592336; -- 0x82640290