re: which archives the disc can name -- 100% for menu paks, 0.0% for the 2D paks and READY_ROOM
Chasing more prefixes for the 419 HUD config paths would have been the same mistake twice, so this censuses the whole disc: harvest every plausible asset-name string from every archive (6027), hash each under the 16 known path prefixes, and ask per archive what fraction of its TOC that explains. idxd-container.md and idxd-tag-hash.md own the hash; neither says which archives are reachable by it. The result is bimodal. GP_TITLE 16/16, GP_PAUSE_MENU 11/11, GP_STAGE_CLEAR 44/44, GP_CHALLENGE 151/151, GP_MOVIE_THEATER 56/56, MiscBin 40/40, GP_GAMEOVER, GP_BUNK, GP_SYSTEM, GP_TUTORIAL and fonts are at 100%; tables.pak 78/79, GP_DIALOG 139/140, the six language paks 115/117; GP_MAIN_GAME_* 751/1119. Then the cliff: the six GP_MAIN_GAME_*2D.pak at 0 of 711 each, and GP_READY_ROOM at 6 of 1106 -- the largest UI pak on the disc, not previously noted anywhere. Eleven paks at 100% in the same run is the control that makes 0.0% a finding rather than a failed guess. So the 419 HUD paths are not missing assets: nothing in the 2D paks is reachable by name from the disc's own strings at all. Those TOC keys hash names that are not written anywhere readable. Also refuted first: the 419 values under 13 name transformations, every one scoring 0 against the E2D 711 and against all 16630 entries. Not settled: what those names are. The lever left is the hash's shape -- the top byte is the character-sum checksum -- but that needs a name corpus the disc does not contain. New structure doc, artefact and regenerator; the other eight regenerate byte-identical.
This commit is contained in:
Sylpheed RE agent
@@ -827,6 +827,42 @@ unknown, what evidence exists, and what the first step would be. Move an item in
|
||||
field; the stat keys in records not opened here (`MegaTons`, `BulletLimit`,
|
||||
`CrewCount`, `HP_CLASS`); why the roster lists 59 of 131.
|
||||
|
||||
* ✅ **(2026-08-27) WHICH ARCHIVES THE DISC CAN NAME — 100 % for eleven menu
|
||||
paks, **0.0 %** for all six 2D paks AND `GP_READY_ROOM`.**
|
||||
[structures/archive-naming](structures/archive-naming.md), artefact
|
||||
`data/archive-naming.txt`, regenerator `tools/re-capture/archive_naming.py`.
|
||||
Chasing more prefixes for the 419 HUD paths would have been the same mistake
|
||||
twice, so this censuses the whole disc: harvest **every** plausible asset-name
|
||||
string from **every** archive (**6 027**), hash each under the 16 known
|
||||
prefixes, and ask per archive what fraction of its TOC that explains.
|
||||
`idxd-container.md`/`idxd-tag-hash.md` own the hash; neither says which
|
||||
archives are REACHABLE by it.
|
||||
🔑 **The result is bimodal.** `GP_TITLE` 16/16, `GP_PAUSE_MENU` 11/11,
|
||||
`GP_STAGE_CLEAR` 44/44, `GP_CHALLENGE` 151/151, `GP_MOVIE_THEATER` 56/56,
|
||||
`MiscBin` 40/40, `GP_GAMEOVER`, `GP_BUNK`, `GP_SYSTEM`, `GP_TUTORIAL`, `fonts`
|
||||
— **100 %**; `tables.pak` 78/79, `GP_DIALOG` 139/140, the six language paks
|
||||
115/117, `GP_SAVE_LOAD`/`GP_LEADERBOARD`/`GP_OPTIONS`/`GP_MISSION_LOG` 91-94 %;
|
||||
`GP_MAIN_GAME_*` 751/1119 = 67 %. Then the cliff: **the six
|
||||
`GP_MAIN_GAME_*2D.pak` at 0 of 711 EACH**, and **`GP_READY_ROOM` at 6 of
|
||||
1 106 = 0.5 %** — the largest UI pak on the disc, and not previously noted
|
||||
anywhere in `docs/re/`.
|
||||
🧪 Eleven paks at 100 % in the same run is the control that makes 0.0 % a
|
||||
finding rather than a failed guess: the method works; these archives are
|
||||
outside it.
|
||||
🔑 **So the 419 HUD paths are not "missing assets"** — nothing in the 2D paks
|
||||
is reachable by name from the disc's own strings at all. Those TOC keys hash
|
||||
names that are **not written anywhere readable**: composed by the tool chain or
|
||||
by the executable, or listed outside these archives.
|
||||
❌ Also refuted first: the 419 config values under **13 name transformations**
|
||||
(as-is, forward slashes, basename, basename-no-ext, no-ext, UPPER, `.rat`
|
||||
substituted, `2d\`/`eng\`/`hud\`/`GP_MAIN_GAME_2D\` prefixed, first-directory
|
||||
stripped) — **every one scored 0**, against the E2D 711 and against all 16 630
|
||||
entries on the disc.
|
||||
🟡 Not settled: what those names are. The one lever left is the hash's shape —
|
||||
the top byte is the character-sum checksum, so candidates are cheap to reject —
|
||||
but that needs a name corpus the disc does not contain. Also unexplained:
|
||||
`DefTables` 8.9 % and `GP_HANGAR_ARSENAL` 22.6 % sitting in the middle.
|
||||
|
||||
* ✅ **(2026-08-27) THE WHOLE IN-GAME HUD CONFIG IS OPEN — 16 records, 419 asset
|
||||
paths, a per-stage `ResourceTable` — and NONE of it is name-hash addressed.**
|
||||
[structures/hud-config](structures/hud-config.md), artefact
|
||||
|
||||
@@ -99,6 +99,7 @@ files, which is how the same ground got covered twice.
|
||||
| [`structures/sound-cue-table.md`](structures/sound-cue-table.md) | The cue index in `tables.pak` — message id -> cue -> sound id -> `.slb` bank | ✅ CONFIRMED, 1 326/1 338 script message ids bind to a bank; SOUNDS and FILES agree on the same 12 absentees, 0 orphan files |
|
||||
| [`structures/cutscene-message-table.md`](structures/cutscene-message-table.md) | Cutscene dialogue — speaker, portrait, on-screen seconds, audio cue per page | ✅ CONFIRMED, field count = 9·PageCount+2 for all 7 PageCounts, 1 252/1 252 caption keys match, 138 ids close both ways |
|
||||
| [`structures/isl-mission-timer.md`](structures/isl-mission-timer.md) | The mission timer — two sequential countdowns, not a limit and a warning | ✅ CONFIRMED — read from `sub_822639B8`; the threshold reading is |
|
||||
| [`structures/archive-naming.md`](structures/archive-naming.md) | Which archives the disc can name — 100% for menu paks, **0.0%** for the six 2D paks and `GP_READY_ROOM` | ✅ CONFIRMED — 6027 names × 16 prefixes, bimodal |
|
||||
| [`structures/hud-config.md`](structures/hud-config.md) | The in-game HUD configuration — 16 records, 419 asset paths, and a per-stage `ResourceTable` | ✅ CONFIRMED — 0 of 419 config paths resolve as pak entries |
|
||||
| [`structures/mission-script-manifest.md`](structures/mission-script-manifest.md) | `Stage\script.tbl`'s 11 non-`MISSION` fields, and the mission dialogue table they name | ✅ CONFIRMED — 1 of 7 values resolves (`message\MissionDialogMessage.tbl`, 200 records), controls in the same sweep |
|
||||
| [`structures/stage-numbering-and-player-craft.md`](structures/stage-numbering-and-player-craft.md) | The disc's stage numbering (S01–S16, S18–S29) and the player's craft per stage | ✅✅ CONFIRMED — two independent files close on 6 + 15 + 1 + 6 = 28 |
|
||||
|
||||
49
docs/re/data/archive-naming.txt
Normal file
49
docs/re/data/archive-naming.txt
Normal file
@@ -0,0 +1,49 @@
|
||||
# How much of each pak's TOC can be named from the disc's own strings
|
||||
# Regenerate: python3 tools/re-capture/archive_naming.py
|
||||
# See docs/re/structures/archive-naming.md
|
||||
|
||||
candidate asset-name strings harvested: 6027
|
||||
path prefixes tried: 16
|
||||
|
||||
archive entries named pct
|
||||
sound.pak 9519 1 0.0%
|
||||
GP_HANGAR_ARSENAL.pak 1538 347 22.6%
|
||||
DefTables.pak 1465 130 8.9%
|
||||
GP_MAIN_GAME_D.pak 1119 751 67.1%
|
||||
GP_MAIN_GAME_E.pak 1119 751 67.1%
|
||||
GP_MAIN_GAME_F.pak 1119 751 67.1%
|
||||
GP_MAIN_GAME_I.pak 1119 751 67.1%
|
||||
GP_MAIN_GAME_J.pak 1119 751 67.1%
|
||||
GP_MAIN_GAME_S.pak 1119 751 67.1%
|
||||
GP_READY_ROOM.pak 1106 6 0.5%
|
||||
GP_MAIN_GAME_D2D.pak 711 0 0.0%
|
||||
GP_MAIN_GAME_E2D.pak 711 0 0.0%
|
||||
GP_MAIN_GAME_F2D.pak 711 0 0.0%
|
||||
GP_MAIN_GAME_I2D.pak 711 0 0.0%
|
||||
GP_MAIN_GAME_J2D.pak 711 0 0.0%
|
||||
GP_MAIN_GAME_S2D.pak 711 0 0.0%
|
||||
GP_DEBRIEFING_PILOTLOG.pak 234 78 33.3%
|
||||
GP_CHALLENGE.pak 151 151 100.0%
|
||||
GP_DIALOG.pak 140 139 99.3%
|
||||
deu.pak 117 115 98.3%
|
||||
eng.pak 117 115 98.3%
|
||||
esp.pak 117 115 98.3%
|
||||
fra.pak 117 115 98.3%
|
||||
ita.pak 117 115 98.3%
|
||||
jpn.pak 117 115 98.3%
|
||||
GP_SAVE_LOAD.pak 108 102 94.4%
|
||||
GP_MISSION_SELECT.pak 88 22 25.0%
|
||||
GP_LEADERBOARD.pak 87 81 93.1%
|
||||
tables.pak 79 78 98.7%
|
||||
GP_MOVIE_THEATER.pak 56 56 100.0%
|
||||
GP_STAGE_CLEAR.pak 44 44 100.0%
|
||||
MiscBin.pak 40 40 100.0%
|
||||
GP_OPTIONS.pak 26 24 92.3%
|
||||
GP_MISSION_LOG.pak 24 22 91.7%
|
||||
GP_TITLE.pak 16 16 100.0%
|
||||
GP_GAMEOVER.pak 12 12 100.0%
|
||||
GP_PAUSE_MENU.pak 11 11 100.0%
|
||||
GP_BUNK.pak 10 10 100.0%
|
||||
fonts.pak 3 3 100.0%
|
||||
GP_SYSTEM.pak 2 2 100.0%
|
||||
GP_TUTORIAL.pak 2 2 100.0%
|
||||
66
docs/re/structures/archive-naming.md
Normal file
66
docs/re/structures/archive-naming.md
Normal file
@@ -0,0 +1,66 @@
|
||||
# ✅ Which archives the disc can name — and the two it cannot
|
||||
|
||||
The previous pass found that **0 of 419** in-game HUD config paths hash to a pak
|
||||
entry ([hud-config](hud-config.md)). Chasing more prefixes would have been the
|
||||
same mistake twice, so this censuses the whole disc instead: harvest **every**
|
||||
plausible asset-name string from **every** archive, hash each under the 16 known
|
||||
path prefixes, and ask per archive **what fraction of its TOC that explains**.
|
||||
|
||||
`idxd-container.md` and `idxd-tag-hash.md` own the hash (`name_hash` = a
|
||||
checksum byte over a 24-bit modular value, case-insensitive, distinct from
|
||||
`tag_hash`); neither says which archives are *reachable* by it. Artefact
|
||||
`../data/archive-naming.txt`, regenerator `tools/re-capture/archive_naming.py`
|
||||
(**6 027** candidate names × 16 prefixes).
|
||||
|
||||
## The result is bimodal
|
||||
|
||||
| archive | entries | named | |
|
||||
|---|---:|---:|---|
|
||||
| `GP_TITLE`, `GP_PAUSE_MENU`, `GP_STAGE_CLEAR`, `GP_CHALLENGE`, `GP_MOVIE_THEATER`, `GP_GAMEOVER`, `GP_BUNK`, `GP_SYSTEM`, `GP_TUTORIAL`, `MiscBin`, `fonts` | 2–151 | **all** | **100 %** |
|
||||
| `tables.pak` | 79 | 78 | 98.7 % |
|
||||
| `GP_DIALOG` | 140 | 139 | 99.3 % |
|
||||
| `deu/eng/esp/fra/ita/jpn.pak` | 117 | 115 | 98.3 % |
|
||||
| `GP_SAVE_LOAD` / `GP_LEADERBOARD` / `GP_OPTIONS` / `GP_MISSION_LOG` | 24–108 | | 91–94 % |
|
||||
| `GP_MAIN_GAME_{D,E,F,I,J,S}` | 1 119 | 751 | 67.1 % |
|
||||
| `GP_DEBRIEFING_PILOTLOG` | 234 | 78 | 33.3 % |
|
||||
| `GP_HANGAR_ARSENAL` | 1 538 | 347 | 22.6 % |
|
||||
| `DefTables` | 1 465 | 130 | 8.9 % |
|
||||
| `GP_MISSION_SELECT` | 88 | 22 | 25.0 % |
|
||||
| **`GP_READY_ROOM`** | **1 106** | **6** | **0.5 %** |
|
||||
| **`GP_MAIN_GAME_{D,E,F,I,J,S}2D`** | **711** each | **0** | **0.0 %** |
|
||||
| `sound.pak` | 9 519 | 1 | 0.0 % (a separate format — [sound-pak-contents](sound-pak-contents.md)) |
|
||||
|
||||
🔑 **The six 2D paks are at exactly 0.0 %, all six, 711 entries each** — while
|
||||
eleven menu paks are at 100 % by the same method in the same run. That is the
|
||||
control that makes it a finding rather than a failed guess: the method works, and
|
||||
these archives are outside it.
|
||||
|
||||
🔑 **`GP_READY_ROOM` is a second such archive** — 1 106 entries, **6** named.
|
||||
Not previously noted anywhere in `docs/re/`; it is the largest UI pak on the
|
||||
disc and its names are as unreachable as the 2D paks'.
|
||||
|
||||
## What that means for the 419
|
||||
|
||||
The in-game HUD's asset paths are not "missing". **Nothing in the 2D paks is
|
||||
reachable by name from the disc's own strings**, whatever the extension. So the
|
||||
2D and READY_ROOM TOC keys are hashes of names that are **not written anywhere
|
||||
we can read** — built by the tool chain, or from a string the executable
|
||||
composes, or from a name list held outside these archives.
|
||||
|
||||
## 🧪 Also refuted here — 13 name transformations
|
||||
|
||||
Before the census, the 419 config values were re-hashed under 13 rewrites:
|
||||
as-is, forward slashes, basename, basename without extension, without extension,
|
||||
uppercased, `.rat` substituted, `2d\`/`eng\`/`hud\`/`GP_MAIN_GAME_2D\` prefixed,
|
||||
and first-directory stripped. **Every one scored 0, both against the E2D pak's
|
||||
711 entries and against all 16 630 entries on the disc.**
|
||||
|
||||
## 🟡 Not settled
|
||||
|
||||
* What names the 2D and `GP_READY_ROOM` TOCs actually hash. The one lever left is
|
||||
the hash's own shape — the top byte is the character-sum checksum, so any
|
||||
candidate is cheap to reject — but that needs a name corpus this disc does not
|
||||
contain.
|
||||
* Why `DefTables` (8.9 %) and `GP_HANGAR_ARSENAL` (22.6 %) sit in the middle;
|
||||
probably just that most of their entries are tables referenced by records this
|
||||
harvest does not treat as filenames. Not chased.
|
||||
47
tools/re-capture/archive_naming.py
Normal file
47
tools/re-capture/archive_naming.py
Normal file
@@ -0,0 +1,47 @@
|
||||
#!/usr/bin/env python3
|
||||
"""How much of each pak's TOC can be named from the asset strings on the disc.
|
||||
|
||||
Harvests every plausible asset-name string from every archive, hashes each under
|
||||
the known path prefixes, and reports per archive how many of its entries are
|
||||
explained. Regenerates docs/re/data/archive-naming.txt.
|
||||
"""
|
||||
import sys, os, glob, re, collections
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.insert(0, HERE)
|
||||
import unitgroup as U
|
||||
from unit_substructures import pak_entries
|
||||
|
||||
EXT = r'(prt|t32|rat|tbl|xpr|ttc|ttf|TTC|TTF|bin|col|rgn|prm)'
|
||||
PRE = ['', '2d\\', 'ui\\', 'hud\\', 'Stage\\', 'stage\\', 'message\\', 'language\\',
|
||||
'prt\\', 'view\\'] + [l + '\\' for l in ('eng', 'jpn', 'fra', 'deu', 'ita', 'esp')]
|
||||
|
||||
def main():
|
||||
paks = sorted(glob.glob('/work/sylph_extract/**/*.pak', recursive=True))
|
||||
names = set()
|
||||
toc = {}
|
||||
for pk in paks:
|
||||
ents = pak_entries(pk)
|
||||
toc[os.path.basename(pk)] = [h for h, b in ents]
|
||||
for h, b in ents:
|
||||
for m in re.finditer((r'[A-Za-z0-9_\\.\-]{4,60}\.' + EXT).encode(), b):
|
||||
names.add(m.group(0).decode('latin-1'))
|
||||
|
||||
H = set()
|
||||
for n in names:
|
||||
for p in PRE:
|
||||
H.add(U.name_hash(p + n))
|
||||
|
||||
print("# How much of each pak's TOC can be named from the disc's own strings")
|
||||
print("# Regenerate: python3 tools/re-capture/archive_naming.py")
|
||||
print("# See docs/re/structures/archive-naming.md")
|
||||
print("\n candidate asset-name strings harvested: %d" % len(names))
|
||||
print(" path prefixes tried: %d" % len(PRE))
|
||||
print("\n archive entries named pct")
|
||||
rows = sorted(((k, v) for k, v in toc.items()), key=lambda r: -len(r[1]))
|
||||
for nm, ent in rows:
|
||||
nd = sum(1 for h in ent if h in H)
|
||||
print(" %-30s %6d %6d %5.1f%%" % (nm, len(ent), nd, 100.0 * nd / len(ent) if ent else 0))
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user