4e9d7deeb8
Chasing more prefixes for the 419 HUD config paths would have been the same mistake twice, so this censuses the whole disc: harvest every plausible asset-name string from every archive (6027), hash each under the 16 known path prefixes, and ask per archive what fraction of its TOC that explains. idxd-container.md and idxd-tag-hash.md own the hash; neither says which archives are reachable by it. The result is bimodal. GP_TITLE 16/16, GP_PAUSE_MENU 11/11, GP_STAGE_CLEAR 44/44, GP_CHALLENGE 151/151, GP_MOVIE_THEATER 56/56, MiscBin 40/40, GP_GAMEOVER, GP_BUNK, GP_SYSTEM, GP_TUTORIAL and fonts are at 100%; tables.pak 78/79, GP_DIALOG 139/140, the six language paks 115/117; GP_MAIN_GAME_* 751/1119. Then the cliff: the six GP_MAIN_GAME_*2D.pak at 0 of 711 each, and GP_READY_ROOM at 6 of 1106 -- the largest UI pak on the disc, not previously noted anywhere. Eleven paks at 100% in the same run is the control that makes 0.0% a finding rather than a failed guess. So the 419 HUD paths are not missing assets: nothing in the 2D paks is reachable by name from the disc's own strings at all. Those TOC keys hash names that are not written anywhere readable. Also refuted first: the 419 values under 13 name transformations, every one scoring 0 against the E2D 711 and against all 16630 entries. Not settled: what those names are. The lever left is the hash's shape -- the top byte is the character-sum checksum -- but that needs a name corpus the disc does not contain. New structure doc, artefact and regenerator; the other eight regenerate byte-identical.
48 lines
1.9 KiB
Python
48 lines
1.9 KiB
Python
#!/usr/bin/env python3
|
|
"""How much of each pak's TOC can be named from the asset strings on the disc.
|
|
|
|
Harvests every plausible asset-name string from every archive, hashes each under
|
|
the known path prefixes, and reports per archive how many of its entries are
|
|
explained. Regenerates docs/re/data/archive-naming.txt.
|
|
"""
|
|
import sys, os, glob, re, collections
|
|
|
|
HERE = os.path.dirname(os.path.abspath(__file__))
|
|
sys.path.insert(0, HERE)
|
|
import unitgroup as U
|
|
from unit_substructures import pak_entries
|
|
|
|
EXT = r'(prt|t32|rat|tbl|xpr|ttc|ttf|TTC|TTF|bin|col|rgn|prm)'
|
|
PRE = ['', '2d\\', 'ui\\', 'hud\\', 'Stage\\', 'stage\\', 'message\\', 'language\\',
|
|
'prt\\', 'view\\'] + [l + '\\' for l in ('eng', 'jpn', 'fra', 'deu', 'ita', 'esp')]
|
|
|
|
def main():
|
|
paks = sorted(glob.glob('/work/sylph_extract/**/*.pak', recursive=True))
|
|
names = set()
|
|
toc = {}
|
|
for pk in paks:
|
|
ents = pak_entries(pk)
|
|
toc[os.path.basename(pk)] = [h for h, b in ents]
|
|
for h, b in ents:
|
|
for m in re.finditer((r'[A-Za-z0-9_\\.\-]{4,60}\.' + EXT).encode(), b):
|
|
names.add(m.group(0).decode('latin-1'))
|
|
|
|
H = set()
|
|
for n in names:
|
|
for p in PRE:
|
|
H.add(U.name_hash(p + n))
|
|
|
|
print("# How much of each pak's TOC can be named from the disc's own strings")
|
|
print("# Regenerate: python3 tools/re-capture/archive_naming.py")
|
|
print("# See docs/re/structures/archive-naming.md")
|
|
print("\n candidate asset-name strings harvested: %d" % len(names))
|
|
print(" path prefixes tried: %d" % len(PRE))
|
|
print("\n archive entries named pct")
|
|
rows = sorted(((k, v) for k, v in toc.items()), key=lambda r: -len(r[1]))
|
|
for nm, ent in rows:
|
|
nd = sum(1 for h in ent if h in H)
|
|
print(" %-30s %6d %6d %5.1f%%" % (nm, len(ent), nd, 100.0 * nd / len(ent) if ent else 0))
|
|
|
|
if __name__ == "__main__":
|
|
main()
|