re: which archives the disc can name -- 100% for menu paks, 0.0% for the 2D paks and READY_ROOM
Chasing more prefixes for the 419 HUD config paths would have been the same mistake twice, so this censuses the whole disc: harvest every plausible asset-name string from every archive (6027), hash each under the 16 known path prefixes, and ask per archive what fraction of its TOC that explains. idxd-container.md and idxd-tag-hash.md own the hash; neither says which archives are reachable by it. The result is bimodal. GP_TITLE 16/16, GP_PAUSE_MENU 11/11, GP_STAGE_CLEAR 44/44, GP_CHALLENGE 151/151, GP_MOVIE_THEATER 56/56, MiscBin 40/40, GP_GAMEOVER, GP_BUNK, GP_SYSTEM, GP_TUTORIAL and fonts are at 100%; tables.pak 78/79, GP_DIALOG 139/140, the six language paks 115/117; GP_MAIN_GAME_* 751/1119. Then the cliff: the six GP_MAIN_GAME_*2D.pak at 0 of 711 each, and GP_READY_ROOM at 6 of 1106 -- the largest UI pak on the disc, not previously noted anywhere. Eleven paks at 100% in the same run is the control that makes 0.0% a finding rather than a failed guess. So the 419 HUD paths are not missing assets: nothing in the 2D paks is reachable by name from the disc's own strings at all. Those TOC keys hash names that are not written anywhere readable. Also refuted first: the 419 values under 13 name transformations, every one scoring 0 against the E2D 711 and against all 16630 entries. Not settled: what those names are. The lever left is the hash's shape -- the top byte is the character-sum checksum -- but that needs a name corpus the disc does not contain. New structure doc, artefact and regenerator; the other eight regenerate byte-identical.
This commit is contained in:
47
tools/re-capture/archive_naming.py
Normal file
47
tools/re-capture/archive_naming.py
Normal file
@@ -0,0 +1,47 @@
|
||||
#!/usr/bin/env python3
|
||||
"""How much of each pak's TOC can be named from the asset strings on the disc.
|
||||
|
||||
Harvests every plausible asset-name string from every archive, hashes each under
|
||||
the known path prefixes, and reports per archive how many of its entries are
|
||||
explained. Regenerates docs/re/data/archive-naming.txt.
|
||||
"""
|
||||
import sys, os, glob, re, collections
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.insert(0, HERE)
|
||||
import unitgroup as U
|
||||
from unit_substructures import pak_entries
|
||||
|
||||
EXT = r'(prt|t32|rat|tbl|xpr|ttc|ttf|TTC|TTF|bin|col|rgn|prm)'
|
||||
PRE = ['', '2d\\', 'ui\\', 'hud\\', 'Stage\\', 'stage\\', 'message\\', 'language\\',
|
||||
'prt\\', 'view\\'] + [l + '\\' for l in ('eng', 'jpn', 'fra', 'deu', 'ita', 'esp')]
|
||||
|
||||
def main():
|
||||
paks = sorted(glob.glob('/work/sylph_extract/**/*.pak', recursive=True))
|
||||
names = set()
|
||||
toc = {}
|
||||
for pk in paks:
|
||||
ents = pak_entries(pk)
|
||||
toc[os.path.basename(pk)] = [h for h, b in ents]
|
||||
for h, b in ents:
|
||||
for m in re.finditer((r'[A-Za-z0-9_\\.\-]{4,60}\.' + EXT).encode(), b):
|
||||
names.add(m.group(0).decode('latin-1'))
|
||||
|
||||
H = set()
|
||||
for n in names:
|
||||
for p in PRE:
|
||||
H.add(U.name_hash(p + n))
|
||||
|
||||
print("# How much of each pak's TOC can be named from the disc's own strings")
|
||||
print("# Regenerate: python3 tools/re-capture/archive_naming.py")
|
||||
print("# See docs/re/structures/archive-naming.md")
|
||||
print("\n candidate asset-name strings harvested: %d" % len(names))
|
||||
print(" path prefixes tried: %d" % len(PRE))
|
||||
print("\n archive entries named pct")
|
||||
rows = sorted(((k, v) for k, v in toc.items()), key=lambda r: -len(r[1]))
|
||||
for nm, ent in rows:
|
||||
nd = sum(1 for h in ent if h in H)
|
||||
print(" %-30s %6d %6d %5.1f%%" % (nm, len(ent), nd, 100.0 * nd / len(ent) if ent else 0))
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user