re: the 42 hash-keyed nameless fields are an ISL script link map — names not cracked

All 42 live in six byte-identical copies of <lang>\script\ID.tbl inside
GP_READY_ROOM.pak, the artefact of the pass the executable announces as
"PrepareScript - isl script prescanning start". Two records, FILE and
OFFSET, carry the same 42 keys: symbol hash -> (.isb file, offset).

Confirmed independently, not inferred: 41 of the 42 keys occur verbatim as
little-endian words inside the .isb bytecode, and where they occur is a
coherent call graph -- main.isb calls the 30 stage/challenge/tutorial
symbols, those call the five helpers in function.isb, the tutorials call the
six in function_tutorial.isb, and main.isb's own symbol is called by nothing.
Ordinals do not appear as call targets in code; hashes do.

30 of the 42 names have their last characters PINNED by the hash's own
algebra. Bumping the character k places from the end moves the low 24 bits
by 256^k mod M and the top byte by 1, and the measured deltas are exactly
+0x01000001 across stage01..09, stage10..16 and challenge01..06, and
+0x01010000 across tutorial0101..0601. So those names end in the same digits
as their .isb filename. The prefixes are not recovered.

NULL RESULT on the names, and quantified rather than asserted. Seven attacks
(1.2M disc + executable strings, printf substitution, a hand guess list,
two-token composition over a mined vocabulary at S = 3.5e8, exhaustive
meet-in-the-middle preimage search) produced zero hits above the noise floor.
The search is not broken -- fed tag_hash("Stage01") it returns "Stage01" plus
three collisions, as 63^6 / 2^32 predicts. It returns nothing for the real
targets at <=6 characters, so the prefixes are longer, and at 7 characters
one target already has 1 176 measured preimages. A 24-bit modulus cannot
name an 8+ character identifier uniquely; only a corpus holding the actual
string can, and the disc does not hold it.

Refutations kept rather than deleted: the filename-stem hypothesis (direct),
the shared-prefix-plus-stem hypothesis (algebraic -- A's byte sum is pinned
twice and the two values disagree, 9 pairs, 9 contradictions), names inside
the .isb payloads (there are no ASCII names there at all), the hash constants
sitting in the executable (absent in both endiannesses), and IXUD holding
more of the same (8 unnamed tags, all ordinals 0..7).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE

docs/re/structures/idxd-unnamed-keys.md

@@ -0,0 +1,253 @@
+# IDXD field keys that carry no name
+
+- **Confidence:** ✅ the census and the identification of the hash-keyed set ·
+  ✅ the last characters of 30 of the 42 names · ❌ **no name was recovered**
+- **Tool:** `tools/re-capture/idxd_unnamed_keys.py` (`census` / `idtbl` / `crack` / `selftest`)
+- **Data:** [`docs/re/data/idxd-unnamed-field-keys.txt`](../data/idxd-unnamed-field-keys.txt)
+- **Seen in:** every `dat/*.pak` + `hidden/*.pak` on the disc; the interesting 42
+  are in `dat/GP_READY_ROOM.pak`, file `<lang>\script\ID.tbl`
+- **Depends on:** the IDXD record/field table and `tag_hash`, which live on branch
+  `auto/re-idxd-container` — **not on `main`**, so this note transcribes what it
+  needs and the tool carries its own copy of `tag_hash` rather than importing one.
+
+An IDXD field entry is `(key, name_off, value_off)`. Usually `name_off` points at
+the field's own name in the string pool and `key == tag_hash(name)`. When
+`name_off` is `0xFFFFFFFF` the pool holds no name and the reader has to know what
+`key` means from somewhere else. This note answers: how many such keys are there,
+what are they, and can the names be inverted out of the hash?
+
+**Short answer: 7 094 distinct keys, of which 7 052 are not hashes at all and 42
+are. None of the 42 names was recovered — but they are now identified, their call
+graph is reconstructed, and 30 of them have their last two-to-four characters
+pinned algebraically.**
+
+## ✅ The census
+
+Measured by walking all 33 `dat/*.pak` plus `hidden/DefTables.pak` and
+`hidden/MiscBin.pak` (`.pNN` segments joined, `Z1` payloads inflated):
+
+| | |
+|---|---|
+| IDXD objects / records / field entries | 7 750 / 190 782 / **2 757 039** |
+| named field entries | 1 271 462 |
+| unnamed field entries (`name_off == 0xFFFFFFFF`) | 1 485 577 |
+| distinct keys named *somewhere* on disc | 12 174 |
+| **distinct keys never named anywhere** | **7 094** |
+|  … below `0x00010000` | 7 052 |
+|  … in `[0x10000, 0x01000000)` | **0** |
+|  … at or above `0x01000000` | **42** |
+
+Parse failures: 0. `tag_hash` reproduces the key of **1 271 462 / 1 271 462**
+named field entries disc-wide (`idxd_unnamed_keys.py selftest`), which is the
+validation gate for everything below — far past the 200-pair bar this
+investigation was asked to clear.
+
+### ✅ "504 hash-keyed fields disc-wide" is 504 **entries**, not 504 keys
+
+`INDEX.md` (on `auto/re-idxd-container`) records *504 fields disc-wide that are
+hash-keyed with no name*. That number is exactly reproduced here — and it counts
+**field entries**, not distinct keys:
+
+```
+42 distinct keys  x  6 language copies of the same object  x  2 records = 504
+```
+
+The three unnamed populations add up:
+`504` (hash-keyed) `+ 1 380 324` (ordinal) `+ 104 749` (key `0x00000000`, i.e.
+ordinal 0, held apart only because `tag_hash("") == 0` makes it look "named"
+somewhere) `= 1 485 577`. So the brute-force target was never 504 names; it was
+**42**.
+
+### ✅ The gap at `[0x10000, 0x01000000)` is what separates hashes from ordinals
+
+`tag_hash` puts the byte-sum checksum of the name in the **top byte**. A real
+hash therefore falls below `0x01000000` only when the name's byte sum is `0 mod
+256` — about 1 name in 256 — *and* its low 24 bits are also tiny. Of the 12 174
+keys that do carry a name, 75 have a zero top byte, and the smallest of those is
+`0x0002677C`; the never-named ordinal band tops out at `0x2198`. There is a clean
+three-order-of-magnitude gap and **not one key in it**. The split is measured,
+not assumed.
+
+## ✅ The 7 052 ordinals are not hashes and never were
+
+They are author-assigned element ids written straight into the tag slot. For
+1 404 924 of the 1 485 577 unnamed entries the key **equals the field's index
+within its record**; a further 42 750 are index + 1. The rest are hand-numbered
+with deliberate gaps, e.g. the hangar `WEAPONS` record runs
+`…0x30 0x31 … 0x35`, jumps to `0x50 … 0x55`, then `0x61` — an id space, not a
+counter. `tables.pak`'s `FILES` record numbers 5 135 resources from `0x3E9`
+(1001) upwards.
+
+**Do not feed these to a reverse lookup.** Confirmed by the strongest possible
+test: `tag_hash("BGM_001") = 0xC662435B`, while the key of the field whose value
+is `BGM_001.slb` is `0x000003E9`. Zero of 5 135 `FILES` keys match a hash of
+their own value under any stem/case/extension variant.
+
+Full per-key listing: Part 3 of the data file. Per-schema value profiles: Part 2.
+
+## ✅ What the 42 hash-shaped keys are
+
+All 42 live in **six byte-identical copies** of one object — `eng jpn fra deu ita
+esp \script\ID.tbl` inside `dat/GP_READY_ROOM.pak` (resolved through the IPFB TOC
+by `name_hash("eng\\script\\ID.tbl") = 0xC40BC120`, and pointed at by the
+`SCRIPT_PATH` field of `tables.pak`'s `BASE_INFO`, whose value is
+`dat\GP_READY_ROOM.pak+eng\script\`).
+
+The object has exactly two records, `FILE` and `OFFSET`, carrying **the same 42
+keys in the same order** — it is a column store:
+
+```
+key(tag_hash of an ISL script symbol)  ->  FILE   = the .isb it is defined in
+                                           OFFSET = where in that file
+```
+
+It is the artefact of the pass the executable announces as
+`silph::GamePart_ReadyRoom::Impl::PrepareScript - isl script prescanning start.`
+— a link map from symbol hash to definition site. The 34 `.isb` files it names
+are all present in the same pak (`eng\script\stage01.isb` etc.).
+
+### ✅ Independent confirmation: the keys appear in the bytecode
+
+41 of the 42 keys occur verbatim as **little-endian 32-bit words** inside the
+`.isb` bytecode, and where they occur reconstructs a coherent call graph
+(Part 4 of the data file):
+
+| defined in | referenced from |
+|---|---|
+| `stage01..16.isb`, `challenge01..06.isb`, `tutorial0N01.isb` (30 symbols) | `main.isb` only |
+| `function.isb` (5 symbols) | the stage and challenge scripts |
+| `function_tutorial.isb` (6 symbols) | the six tutorial scripts |
+| `main.isb` (1 symbol) | **nothing** — the root entry point |
+
+That is exactly the shape a launcher + shared-helper-library arrangement should
+have, and it is the reason these keys can be called hashes with confidence rather
+than "large ordinals": ordinals are not stored as call targets in code.
+
+🟡 `OFFSET` is **base 36**. Every value uses only `[0-9a-z]`, and read as base 36
+they are small ascending numbers per file: `function.isb` → 1, 9, 19, 35, 58;
+`function_tutorial.isb` → 1, 12, 23, 34, 45, 50; `tutorial0101.isb` → 1, 61, 191.
+The first symbol of every file is 1. Whether the unit is a statement index, a
+line, or a word offset is ❔ — not settled here.
+
+## ✅ 30 of the 42 names end in the digits of their own filename
+
+This is *forced* by the hash, not guessed. `tag_hash`'s low 24 bits are a
+base-256 polynomial mod `0x00FFFFDF`, so bumping the character `k` places from
+the end changes the low bits by `256^k mod M` (1, `0x100`, `0x10000`, `0x21`, …)
+and the top byte by 1. Measured:
+
+| family | observed key deltas | forced conclusion |
+|---|---|---|
+| `stage01..09` | `+0x01000001` × 8 | last character increments by 1 |
+| `stage10..16` | `+0x01000001` × 6 | ditto |
+| `stage09 → stage10` | `0x8FA349EF → 0x87A34AE6` | low bits `+0xF7 = 0x100 − 9`, top `−8`: the last two characters go `"09" → "10"` |
+| `challenge01..06` | `+0x01000001` × 5 | last character increments by 1 |
+| `tutorial0101 → 0201 … 0601` | `+0x01010000` × 5 | the character **two** from the end increments |
+| `tutorial0101`'s two siblings | `+0x01000001`, `+0x02000002` | last character `+1`, `+2` |
+
+So the sixteen stage symbols end `"01" … "16"`, the six challenge symbols end
+`"01" … "06"`, and the eight tutorial symbols end `"0101" "0102" "0103" "0201"
+"0301" "0401" "0501" "0601"` — matching their `.isb` filenames exactly. The
+remaining 12 (`function.isb` × 5, `function_tutorial.isb` × 6, `main.isb` × 1) are
+unconstrained.
+
+That is the whole of the recovered name information. **The prefixes are not
+recovered.**
+
+## ❌ Cracking: a null result, and the arithmetic that says why
+
+`tag_hash` maps onto `256 × 0x00FFFFDF ≈ 2^32` values, so a search over `S`
+candidates yields on average `S / 2^32` **false** preimages *per target*. That is
+the number every attempt below is judged against.
+
+| attack | candidates (space `S`) | E[false] per target | hits |
+|---|---|---|---|
+| every distinct disc string (IDXD names, record names, field values) | 105 393 | 0.000025 | **0** |
+| every string in the executable (`.pe`) | 75 912 | 0.000018 | **0** |
+| every ASCII run ≥3 in every pak payload except `sound.pak` (incl. `.ssb` symbol tables, XML, XPR) | 1 023 671 | 0.00024 | **0** |
+| all three combined, hashed against all 42 | 1 204 976 | 0.00028 | **0** |
+| `printf`-style substitution: 49 139 format strings × 80 numeric substitutions | 161 200 | 0.00004 | **0** |
+| hand-built guess list (`main`/`Stage`/`Mission`/`ReadyRoom`/… × prefixes × decorations × numbers) | 506 520 | 0.00012 | **0** |
+| two-token composition over a vocabulary mined from disc + exe identifiers (`t1 + sep + t2`, sep ∈ `"" _ -`) — this is what `idxd_unnamed_keys.py crack --pe …` runs | 3.48 × 10⁸ | 0.081 | **0** |
+| the same, with each family's forced digit suffix appended | 1.18 × 10¹⁰ across the sweep | 2.8 total | 1 (`"Cj-format 01"` — noise, and the same string for all six challenge keys, i.e. **one** false prefix) |
+| meet-in-the-middle exhaustive preimage, `[A-Za-z0-9_]`, ≤ 6 chars before the known suffix | 6.3 × 10¹⁰ | ~15 | noise only |
+
+The last row is the important one. Exhaustive search **succeeds** — it is not
+that the search is broken. Fed `tag_hash("Stage01")` it returns
+`['F4D_kl01', 'Stage01', 'be_zT01', 'cd_Yu01']`: the right answer plus three
+collisions, which is what `63^5 / 2^32 ≈ 0.2` … `63^6 / 2^32 ≈ 15` predicts. Fed
+the real target `0x87A349E7` with suffix `"01"` it returns **nothing**, so the
+prefix is **longer than 6 characters** over that alphabet.
+
+And that is where it stops being useful. Measured, not extrapolated: at 7
+characters target `0xA40C6DF7` has **1 176** preimages (`'1T1E1B6'`,
+`'24A7O0G'`, `'4quqkkC'`, …). At 8 the expectation is ≈ 58 000. A hash with a
+24-bit modulus simply does not have enough entropy to name an 8+ character
+identifier uniquely, so **no amount of compute recovers these names from the hash
+alone** — only a corpus containing the actual string can, and the disc does not
+contain it.
+
+### 🔴 Refuted along the way (kept, not deleted)
+
+* **"The symbol is the filename stem, in some case/decoration."** Refuted
+  directly: `tag_hash` of `stage01`, `Stage01`, `STAGE01`, `stage_01`,
+  `mission01`, and 30-odd relatives is not `0x87A349E7`.
+* **"The symbol is `<common prefix> + <filename stem> + <number>`."** Refuted
+  *algebraically*, without searching. If `name = A + "stage01"` and
+  `name = A + "challenge01"` share one `A`, then `A`'s byte sum is pinned twice
+  and the two values must agree. They do not, for lowercase, Capitalised or
+  UPPER stems, with or without a `_`/`-` separator — 9 pairs tested, 9
+  contradictions.
+* **"The name is somewhere in the `.isb` files."** Refuted: the `.isb` payloads
+  contain no ASCII symbol names at all. They reference symbols purely by hash —
+  which is *why* the prescan link map exists.
+* **"The hash constants might be in the executable next to a name table."**
+  Refuted: none of the 42 values occurs in the `.pe` image in either endianness.
+* **"IXUD might hold more hash-keyed nameless fields."** Refuted by measurement:
+  534 IXUD objects, 624 488 field entries, **8** distinct unnamed tags — all of
+  them ordinals `0…7` (the weapon `CATEGORY_DESC` list). Zero hash-shaped.
+* **⚠️ "There are ~504 keys to crack."** Withdrawn — 504 is the *entry* count.
+  See above.
+
+## Coverage / limits
+
+- The census covers every IDXD object the disc has (7 750). `sound.pak` holds no
+  IDXD objects and `dat/movie/*.wmv` and `hidden/resource3d/*.xpr` are not
+  containers; strings from `resource3d` were nevertheless folded into the corpus
+  attack.
+- The 42 targets are fully characterised as *references*; their *spelling* is
+  unrecovered and, per the arithmetic above, unrecoverable from the disc.
+
+## Evidence log (append-only; newest last)
+
+- `2026-08-26` — Disc-wide walk of 7 750 IDXD objects / 2 757 039 field entries
+  with `tag_hash` self-check at 1 271 462/1 271 462. Census → confidence
+  `CONFIRMED` for the counts and for the ordinal/hash split.
+- `2026-08-26` — 42 hash-shaped keys localised to six copies of
+  `<lang>\script\ID.tbl`; cross-checked against the `.isb` bytecode, where 41 of
+  42 appear as little-endian call targets forming a consistent call graph →
+  `CONFIRMED` that they are ISL script-symbol hashes.
+- `2026-08-26` — Key-delta arithmetic pins the trailing digits of 30 of the 42
+  names → `CONFIRMED` (forced by the hash's polynomial structure, three
+  independent families agreeing).
+- `2026-08-26` — Seven corpus/composition/brute-force attacks, none producing a
+  hit above the noise floor; exhaustive search validated on a known name and
+  shown to return nothing at ≤ 6 characters → the names are **not recovered**,
+  and the negative result is quantified rather than asserted.
+
+## Open questions / what would raise confidence
+
+- **The 12 unconstrained names** (`function.isb` ×5, `function_tutorial.isb` ×6,
+  `main.isb` ×1) have no digit structure to exploit at all.
+- **What would actually work:** a build artefact containing ISL source or a symbol
+  list (none on the retail disc); or observing the game *construct* one of these
+  strings at runtime — `PrepareScript` builds the table from `.isb` contents, but
+  whatever `main.isb`'s root symbol is called, the name existed only in the
+  compiler's input. A `--mem-watch` on the ReadyRoom lookup path would show
+  whether any caller ever passes a literal name rather than a precomputed hash;
+  that is the one remaining honest route and it was **not** attempted here (this
+  was a static-only investigation).
+- **`OFFSET`'s unit** — base-36 is settled, what it counts is not.
+- The `.isb` instruction encoding is only sketched here (a length byte at
+  `byte[2]`, call payloads leading with the target hash). It is not the `.ssb`
+  ISL bytecode documented elsewhere and deserves its own note.