Compare commits
3 Commits
5950dc86d5
...
dc3de9dbc4
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
dc3de9dbc4 | ||
|
|
f51c7c8ca9 | ||
|
|
6b248d71e9 |
121
RE_SYMBOLS.md
Normal file
121
RE_SYMBOLS.md
Normal file
@@ -0,0 +1,121 @@
|
||||
# Project Sylpheed — reverse-engineered symbol map
|
||||
|
||||
Durable record of guest functions/data in the retail title (`default.xex`,
|
||||
base `0x82000000`) that have been positively identified by RE. Addresses are
|
||||
guest virtual addresses. This is the source of truth for names; the DuckDB
|
||||
`sylpheed.db` (this dir) `functions.name` column can be re-stamped from here
|
||||
with `apply_re_symbols.sql` (names are wiped on a full DB regen).
|
||||
|
||||
Confidence: **H** = behaviour proven (reproduced/verified against data), **M** =
|
||||
strongly inferred from disassembly, **L** = tentative.
|
||||
|
||||
## Archive (IPFB `.pak`) subsystem
|
||||
|
||||
| Address | Name | Conf | Description |
|
||||
|---|---|---|---|
|
||||
| `0x824609C8` | `Pak_FindEntryByName` | H | Look up a TOC entry by path. Args `(r3 = pak object, r4 = char* path)`. Calls `Pak_HashPathName(path)`, then binary-searches the sorted 12-byte TOC (base `pak+4`, count `pak+8`, stride 12; compares the 32-bit `name_hash`). Returns the matching record pointer or 0. |
|
||||
| `0x82460928` | `Pak_HashPathName` | H | Duplicate `r3 = char* path`, lowercase it in place (`Str_ToLowerAscii`), hash with `Sylph_NameHash`, free the copy, return the 32-bit hash. This is the path→key bridge. |
|
||||
| `0x82455C78` | `Sylph_NameHash` | H | The IPFB TOC / cache-path name-hash. See [Name-hash](#name-hash-sylph_namehash). Input must already be lowercased. `r3 = char*` → `r3 = u32 hash`. |
|
||||
| `0x825F4F90` | `Str_ToLowerAscii` | H | In-place ASCII lowercase (`A`–`Z` → `+0x20`); other bytes pass through. `r3 = char*`. (When `r3 == 0` it takes an unrelated init branch.) |
|
||||
| `0x824607B0` | `Pak_IsIPFBHeader` | H | Validate an IPFB header. Loads the header's first word and checks it equals `"IPFB"` (`0x49504642`) via a delta trick: `first_word - 0x49504620 == 0x22` (`'B' - ' '`). `r5 = header ptr`. Returns bool-ish in `r3`. |
|
||||
| `0x82460AD0`… | *(pak header parse, unregistered container)* | M | The pak-open/validate path around `0x82460DC0`–`0x82460EC4` builds an expected-header template (`"IPFB"`, `0x10000000` flags) on the stack and calls `Pak_IsIPFBHeader`. Function-boundary detection missed the enclosing frame (entry ≈ `0x82460E34`). |
|
||||
| `0x82458508` | `Archive_StreamReadCrc32` | M | Streaming block reader that validates content with a reflected CRC-32 (init `0xFFFFFFFF`, final `~`, table at `g_Crc32Table`). Compares the computed CRC to a stored expected value (`state+144`). This is a **content-integrity** CRC, *not* the name-hash. |
|
||||
| `0x828992F0` | `g_Crc32Table` (data) | H | 256-entry reflected CRC-32 lookup table. Referenced only by `Archive_StreamReadCrc32` and its sibling loop at `0x82457AF8`. |
|
||||
|
||||
## IDXD reflective serialization framework
|
||||
|
||||
Generic tagged-object (de)serializer shared by ~15 object families (craft,
|
||||
weapon, message/character, effects, …). The `.pak` IDXD payloads are read
|
||||
through this.
|
||||
|
||||
| Address | Name | Conf | Description |
|
||||
|---|---|---|---|
|
||||
| `0x824486C0` | `IdxdLoad_Dispatch` | M | Entry: reads the magic and dispatches across variants `IDXD` (`0x49445844`), `IDX2`, `IDX3`, `IDXC`, `IXUD` → the matching parser. **15 callers** (one per object family). The target object is passed in `r3` — already constructed (defaults set) by the caller. |
|
||||
| `0x82448D00` | `IdxdLoad_Variant2` | M | Sibling variant loader (also materializes the `IDXD` magic). |
|
||||
| `0x82449640` | `Idxd_Parse` | M | The IDXD body parser: skips a BOM, then loops the string pool — per field, reads the name token, `Reflect_FindFieldIndex`, then dispatches on the **value's first char** (jump table at `0x824497F8`, index `firstchar - 0x22`) to type-specific writers. Only fields **present** in the pool are written. |
|
||||
| `0x8244A2F0` | `Reflect_FindFieldIndex` | M | Look up a field-name string in a global name-vector at `0x820B4F18` (std::vector-like: count `+20`, data `+24`/SSO), via a `strncmp`-style compare (`0x825EDCE0`). Returns index or `-1`. |
|
||||
| `0x8244A4B0` | `Reflect_SetField` | L | Apply a value to `object` using its per-instance property registry at `object+64` and a config string at `0x820B4F28`. |
|
||||
| `0x820B4F18` | `g_FieldNameRegistry` (data) | L | Global reflection name-vector (runtime-built) used by `Reflect_FindFieldIndex`. |
|
||||
|
||||
**Defaulted fields:** a field is written to the IDXD pool only when it differs
|
||||
from its default, so `Idxd_Parse` never sets defaulted fields — their values are
|
||||
established by each **type's constructor** (in the 15 `IdxdLoad_Dispatch`
|
||||
callers) before load, via this same registry. Recovering them statically means
|
||||
reversing those constructors and joining `field-name → struct-offset →
|
||||
default-store`; a runtime dump of a loaded object's registry gives
|
||||
`name → value` directly. (Open — see below.)
|
||||
|
||||
## 3D resource subsystem
|
||||
|
||||
| Address | Name | Conf | Description |
|
||||
|---|---|---|---|
|
||||
| `0x82640290` | `Res3D_LoadMeshChunk` | L | 3D-resource loader: walks 12-byte records, checks chunk tags (`0x1A22AA26`, `0x2DA2AA24`), and copies `float` triples (x/y/z vertices) via `lfs`/`stfs`. Part of the DefTables/`machines\<model>\…` mesh path. |
|
||||
|
||||
## Name-hash (`Sylph_NameHash`)
|
||||
|
||||
Per-byte Barrett-reduced polynomial hash over the **lowercased** name:
|
||||
|
||||
```
|
||||
A = 0 ; B = 0
|
||||
for each (sign-extended) byte c of the lowercased name:
|
||||
A = (A << 8) + c # 32-bit
|
||||
A = A - (((A * 0x8003_1493) >> 32) rol 9 & 0x1FF) * 0x00FF_F9D7 # A mod 0x00FF_F9D7 (no final fixup)
|
||||
B = B + c
|
||||
hash = ((B & 0xFF) << 24) | (A & 0x00FF_FFFF)
|
||||
```
|
||||
|
||||
- Modulus `0x00FF_F9D7`, reciprocal `0x8003_1493` (the `mulhwu`/`mullw` Barrett step).
|
||||
- Low 24 bits = modular polynomial hash; top byte = 8-bit additive checksum of the bytes.
|
||||
- No trailing conditional subtract — the value is defined by the exact op sequence.
|
||||
- Faithful Rust reproduction: `sylpheed-formats/src/hash.rs` (`name_hash`).
|
||||
|
||||
**Verified** against retail TOCs: `name_hash("files.tbl") == 0x8342_1153`,
|
||||
`name_hash("eng\\weapon.tbl") == 0x900C_8DCD`, `name_hash("eng\\strings.tbl") == 0x10C8_0B87`,
|
||||
`name_hash("jpn\\weapon.tbl") == 0x9E85_FEFF`.
|
||||
|
||||
## TOC key path conventions (preimages)
|
||||
|
||||
The 32-bit TOC key is `name_hash` of a **backslash** path with the entry's
|
||||
internal identity string. Confirmed schemes:
|
||||
|
||||
| Scheme | Example | Where |
|
||||
|---|---|---|
|
||||
| `unit\<ID>.tbl` | `unit\UN_f001_TCAF_DeltaSaber_T_EX5.tbl` → `0x7C96296C` | craft/ship entries in `GP_MAIN_GAME_*.pak` |
|
||||
| `weapon\<ID>.tbl` | `weapon\Weapon_DSaber_P_wep_26_Missile.tbl` | weapon entries |
|
||||
| `message\<ID>.tbl` | `message\CharacterCARL.tbl` | character/dialog entries |
|
||||
| `effect\<ID>.tbl` | — | effect definitions |
|
||||
| `<lang>\<name>.tbl` | `eng\weapon.tbl`, `jpn\strings.tbl` | localized loadout/text tables |
|
||||
| `<name>.tbl` | `files.tbl` | root manifest / DefTables resource entries |
|
||||
|
||||
`<ID>` = the entry's internal `ID` field (IDXD string pool). Craft/weapon
|
||||
entries also self-describe by content, so stats are readable without the key.
|
||||
|
||||
Coverage with these schemes (IDXD entries, whole-pool ID extraction):
|
||||
GP_MAIN_GAME_E 308/1004 (≈31%), DefTables 804/1425 (≈56%). Shipped in
|
||||
`sylpheed-formats` (`hash::recover_toc_name`, `pak list` prints resolved
|
||||
paths). The unresolved remainder use deeper cross-referenced directory
|
||||
paths (e.g. per-mission dialog `MSG_*` entries) — a later hunt, not an
|
||||
extraction limit.
|
||||
|
||||
## Open threads
|
||||
|
||||
- **Defaulted IDXD fields.** A field is omitted from the pool exactly when it
|
||||
equals the schema default, so present values are all *overrides* and the
|
||||
defaults cannot be inferred from them. Gap is large: across the 534 craft
|
||||
(`schema 0x43FAA517`) entries, explicit-presence is `Acceleration` 42%,
|
||||
`MaximumVelocity` 51%, `Turn_AngularVelocity` 31%, `FCSRange` 34%,
|
||||
`ShieldRatio` 49% (`Size_X` 100%, `HP` 85%). So 40–70% of craft rely on
|
||||
unknown defaults for core flight/defense stats.
|
||||
- Ruled out: the big 16-byte-record binary node table inside a craft IDXD is
|
||||
**spatial/mesh data**, not defaults (not keyed by `name_hash(field_key)`).
|
||||
- `schema_hash` is **not** a code immediate and **not** `name_hash(typename)`.
|
||||
- Defaults are set by each type's **constructor** (one per `IdxdLoad_Dispatch`
|
||||
caller) via the reflection registry, not by `Idxd_Parse`.
|
||||
- **Two finish routes.** (A, static) reverse the craft constructor: read its
|
||||
field registrations (`name → offset`) and default-init stores
|
||||
(`offset → value`), join them — heavy, and offset↔name correlation is
|
||||
error-prone. (B, runtime) dump a loaded craft object's registry
|
||||
(`obj+64`) / struct from xenia-rs or Canary — a fully-loaded craft already
|
||||
holds every default; one dump yields `name → value`. **B is the efficient
|
||||
finish** given the framework is generic; A's framework map above is the
|
||||
prerequisite either way.
|
||||
20
apply_re_symbols.sql
Normal file
20
apply_re_symbols.sql
Normal file
@@ -0,0 +1,20 @@
|
||||
-- Re-stamp reverse-engineered function names onto sylpheed.db.
|
||||
-- Source of truth: docs/RE_SYMBOLS.md. Re-run after any full DB regen
|
||||
-- (regen wipes functions.name back to sub_XXXX).
|
||||
-- python3 -c "import duckdb; duckdb.connect('sylpheed.db').execute(open('apply_re_symbols.sql').read())"
|
||||
-- Addresses are decimal (DuckDB rejects 0x literals in some contexts); hex in comments.
|
||||
|
||||
UPDATE functions SET name = 'Pak_FindEntryByName' WHERE address = 2185628104; -- 0x824609C8
|
||||
UPDATE functions SET name = 'Pak_HashPathName' WHERE address = 2185627944; -- 0x82460928
|
||||
UPDATE functions SET name = 'Sylph_NameHash' WHERE address = 2185583736; -- 0x82455C78
|
||||
UPDATE functions SET name = 'Str_ToLowerAscii' WHERE address = 2187284368; -- 0x825F4F90
|
||||
UPDATE functions SET name = 'Pak_IsIPFBHeader' WHERE address = 2185627568; -- 0x824607B0
|
||||
UPDATE functions SET name = 'Archive_StreamReadCrc32' WHERE address = 2185594120; -- 0x82458508
|
||||
UPDATE functions SET name = 'Res3D_LoadMeshChunk' WHERE address = 2187592336; -- 0x82640290
|
||||
|
||||
-- IDXD reflective serialization framework (defaulted-field hunt, 2026-07-09)
|
||||
UPDATE functions SET name = 'IdxdLoad_Dispatch' WHERE address = 2185529024; -- 0x824486C0 (magic dispatch IDXD/IDX2/IDX3/IDXC/IXUD; 15 callers)
|
||||
UPDATE functions SET name = 'IdxdLoad_Variant2' WHERE address = 2185530624; -- 0x82448D00 (sibling variant loader)
|
||||
UPDATE functions SET name = 'Idxd_Parse' WHERE address = 2185532992; -- 0x82449640 (tokenize pool; per-field lookup+set)
|
||||
UPDATE functions SET name = 'Reflect_FindFieldIndex' WHERE address = 2185536240; -- 0x8244A2F0 (field-name -> index in name-vector @0x820B4F18)
|
||||
UPDATE functions SET name = 'Reflect_SetField' WHERE address = 2185536688; -- 0x8244A4B0 (apply value via object registry @obj+64)
|
||||
Reference in New Issue
Block a user