fabi/Sylpheed
1
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity

re: the '504 unnamed keys' is 504 entries, not 504 names -- it is 42 keys

Browse Source 
Independently reproduced across all 33 paks: 7750 IDXD objects, 1485577 unnamed
field entries, 7094 distinct never-named keys splitting cleanly into 7052 in an
ordinal band (<=0x2198, 94.6% equal to their own field index) and 42 hash-shaped
(>=0x2677C), with ZERO keys in the gap between. The 42 carry exactly 504
entries -- six language copies of one object times two records.

So the preimage target was 42, not 504, and my earlier wording invited the
misreading. Cross-referenced to the idxd-unnamed-keys write-up, which shows the
42 belong to <lang>\script\ID.tbl and cannot be recovered from a 24-bit hash.
This commit is contained in:
Sylpheed RE agent
2026-08-26 03:40:55 +00:00
parent e0648a815c
commit a6fb570677
2 changed files with 16 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/re/INDEX.md
View File

@@ -12,7 +12,7 @@ Promote to a prose `structures/…md` file when a format needs behavioural notes
|--------|-------|-----------------------|-------|
| IPFB `.pak` archive | ✅ | `sylpheed-formats/src/pak.rs` + `tests/pak_idxd_disc.rs` | header + 12-byte TOC, Z1/zlib payloads |
| name-hash (TOC keys) | ✅ | `sylpheed-formats/src/hash.rs` | Barrett-reduction hash; recovers original paths |
| IDXD object/table | ✅ | `sylpheed-formats/src/idxd.rs` + `tests/idxd_records_disc.rs` ([container](structures/idxd-container.md)) | **The binary record/index region in front of the string pool is DECODED** (2026-08-25), closing the parser's long-standing "not yet decoded" note. Uniform 16-byte records `{name_hash, name_off, field_begin, field_end}` sorted by hash and binary-searched, then a field count, 12-byte fields `{key, name_off, value_off}` sorted by key, a pool size, and the string pool; the trailing `pool_size == file_len - pool_base` identity makes the layout self-checking. Verified over the **whole disc** with **zero** failures: 7 750/7 750 objects, 190 782/190 782 records reproducing their stored `tag_hash`, 1 271 462/1 271 462 named fields reproducing their key — and `IXUD` is the same container with `ixud_hash`, UTF-16BE and all offsets in **chars** (1 104/1 104 objects, 628 165/628 165 fields). **Field names are stored on disc** — a field's middle word points at its own name — so nothing needs preimage search except the **504** fields disc-wide that are hash-keyed with no name; the other 1 485 073 nameless fields are *positional*, keyed by a literal integer (line slots, movie ids). ⚠️ **Two long-held beliefs WITHDRAWN**: the word at `0x08` is **not a schema hash**, it is record 0's `name_hash` (7 750/7 750) — the header has no type field at all, so an object's kind is known only from the caller that loads it; and the field's middle word is **not** an always-`0xFFFFFFFF` flags word. The first was caught by a test asserting that every movie id names a real record: `1005 -> STAGE10_PHASE01` failed because `tag_hash("STAGE10_PHASE01")` **is** `0x067025B9`, that table's supposed schema id. 🟡 the legacy value-before-key string-pool reader is now known to be an *approximation* of the real table, and every number derived from it is re-checkable but not yet re-checked |
| IDXD object/table | ✅ | `sylpheed-formats/src/idxd.rs` + `tests/idxd_records_disc.rs` ([container](structures/idxd-container.md)) | **The binary record/index region in front of the string pool is DECODED** (2026-08-25), closing the parser's long-standing "not yet decoded" note. Uniform 16-byte records `{name_hash, name_off, field_begin, field_end}` sorted by hash and binary-searched, then a field count, 12-byte fields `{key, name_off, value_off}` sorted by key, a pool size, and the string pool; the trailing `pool_size == file_len - pool_base` identity makes the layout self-checking. Verified over the **whole disc** with **zero** failures: 7 750/7 750 objects, 190 782/190 782 records reproducing their stored `tag_hash`, 1 271 462/1 271 462 named fields reproducing their key — and `IXUD` is the same container with `ixud_hash`, UTF-16BE and all offsets in **chars** (1 104/1 104 objects, 628 165/628 165 fields). **Field names are stored on disc** — a field's middle word points at its own name — so nothing needs preimage search except the **504** field entries disc-wide that are hash-keyed with no name — which are only **42 distinct keys**, each in 12 places (see [idxd-unnamed-keys.md](structures/idxd-unnamed-keys.md)), and are provably unrecoverable from the hash alone; the other 1 485 073 nameless fields are *positional*, keyed by a literal integer (line slots, movie ids). ⚠️ **Two long-held beliefs WITHDRAWN**: the word at `0x08` is **not a schema hash**, it is record 0's `name_hash` (7 750/7 750) — the header has no type field at all, so an object's kind is known only from the caller that loads it; and the field's middle word is **not** an always-`0xFFFFFFFF` flags word. The first was caught by a test asserting that every movie id names a real record: `1005 -> STAGE10_PHASE01` failed because `tag_hash("STAGE10_PHASE01")` **is** `0x067025B9`, that table's supposed schema id. 🟡 the legacy value-before-key string-pool reader is now known to be an *approximation* of the real table, and every number derived from it is re-checkable but not yet re-checked |
| XPR2 texture + cubemap | 🟡/✅ | `sylpheed-formats/src/texture.rs` + [colour check](xpr2-colour-check.md) | de-tile + A8R8G8B8 and DXT1. **Channel order ✅ confirmed against the running game**: the Delta Saber's decoded atlas is orange-dominant (median saturated hue 23.3°, *zero* cool pixels) and the game renders the same hull at 9.3° — a red↔blue swap would sit at ≈200°. Exact fidelity (gamma/sRGB curve, premultiplied alpha, per-channel scale) is 🟡 untested, since a hue comparison cannot see it; cubemap face ordering ❔ |
| T8aD 2D texture | ✅ | `sylpheed-formats/src/t8ad.rs` | **100 % of the disc decodes** (19 216/19 216, measured). The "~15 % deferred variants" were a wrong model, not a variant: a surface is a list of **arbitrary sub-rectangles**, each with a 16-byte header of `dst X, dst Y, width, height`, not a 256×256 grid — `0x1c` is the **rectangle count**. Uncovered area stays transparent. **Colours ✅ CONFIRMED** ([k8888](structures/texture-color-k8888.md)) |
| RATC bundle | ✅ | `sylpheed-formats/src/ratc.rs` | child listing confirmed. **"One level deep" is not a limitation — there is nothing deeper**: 2 859 bundles hold 18 002 children at depth 1 and **0 at depth 2**, with no parse failures. Nested RATC blobs are **leaf records that reference siblings by name** (`opt `, the sprite name): 3 311 leaves, all embedding sibling names, **10 144 of 10 148 references resolve**. The 4 that do not are one dangling asset — `pmbase.rat` → `pmbase.t32` in `GP_STAGE_CLEAR.pak`'s four language builds, and `pmbase.t32` is **on the disc nowhere** |

18
docs/re/structures/idxd-container.md
View File

@@ -60,8 +60,18 @@ Disc-wide census of all **2,757,039** fields:
| unnamed, literal key (`< 0x10000`) | 1,485,073 |
| unnamed, hash-shaped key | 504 |
Those **504** are the entire remaining preimage problem on the disc. ❔ Their
names are unrecovered.
Those **504** are the entire remaining preimage problem on the disc — but the
count is easy to misread, so state it precisely: 504 is a count of **field
entries**, not of distinct names. They are **42 distinct keys**, each appearing
in 12 places (six byte-identical language copies of one object × two records).
The preimage target was never 504 names; it is 42.
❔ Their names are unrecovered, and
[idxd-unnamed-keys.md](idxd-unnamed-keys.md) shows why that is not a matter of
more compute: the 42 belong to `<lang>\script\ID.tbl`, and at 7 characters one
of them already has 1 176 preimages under a 24-bit modulus. 30 of the 42 have
their trailing digits pinned algebraically; the prefixes cannot be recovered
from the hash alone.
## Two corrections
@@ -144,7 +154,9 @@ of the four legacy magics), so that path is read-only knowledge, untestable here
## What this does *not* settle
* ❔ The names of the 504 unnamed hash-keyed fields.
* ❔ The names of the 42 unnamed hash-keyed fields (504 entries — see
[idxd-unnamed-keys.md](idxd-unnamed-keys.md); shown to be unrecoverable from
the hash alone).
* ❔ How much of the existing corpus the old string-pool reader got wrong. The
legacy `get_f32`/`get_raw` path infers a field's value from *pool adjacency*
(`<value>\0<key>\0`). That adjacency is a consequence of the field table, not a