re: the Enumerate declaration tables close DefTables (130 -> 1413 of 1425 IDXD)
The string harvest could never name DefTables because those names are not
spelled out as "something.tbl" anywhere on the disc -- they are declared.
An IDXD object whose single record is named Enumerate is a DECLARATION TABLE:
its field NAMES are the names of other objects, each resolving as
name_hash("<field name>.tbl"). EnumLODSet_test.tbl declares 676,
EnumGameModel_test.tbl 360; the disc holds 144 such objects (138 in DefTables,
one in each GP_MAIN_GAME_*) declaring 1298 distinct names.
Route 2 names +1283 entries route 1 could not. 130 + 1283 + 12 = 1425, no
overlap, 99.2 % coverage. Zero partials: of 5 suffixes x 6 prefixes, ('', .tbl)
scored 1036/1036 and every other combination scored 0. Residual in full: 8
declaration tables nothing declares, 2 LOD sets (Model rou_e004 / rou_e013), 2
motion sets; 15 declared names have no pak entry at all.
REFUTED alongside it: the 40 XPR2 manifests are not the naming source -- their
82 Name= values and 82 DataFile/Source paths resolve 0 entries under any of the
5 suffixes. They share the MODEL namespace only: 40 of the 82 appear as the
Model field value inside the tables.
Artefact +17/-18, every line paired, byte-identical across two runs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
This commit is contained in:
@@ -19,6 +19,8 @@ PRE = ['', '2d\\', 'ui\\', 'hud\\', 'Stage\\', 'stage\\', 'message\\', 'language
|
||||
def main():
|
||||
paks = sorted(glob.glob('/work/sylph_extract/**/*.pak', recursive=True))
|
||||
names = set()
|
||||
decl = set()
|
||||
nenum = collections.Counter()
|
||||
toc = {}
|
||||
for pk in paks:
|
||||
ents = pak_entries(pk)
|
||||
@@ -26,6 +28,18 @@ def main():
|
||||
for h, b in ents:
|
||||
for m in re.finditer((r'[A-Za-z0-9_\\.\-]{4,60}\.' + EXT).encode(), b):
|
||||
names.add(m.group(0).decode('latin-1'))
|
||||
# ROUTE 2 seed: an object whose single record is named `Enumerate`
|
||||
# is a DECLARATION TABLE — its field names are other objects' names.
|
||||
if b[:4] == b'IDXD' and b'Enumerate' in b:
|
||||
try:
|
||||
recs = U.parse(b)
|
||||
except Exception:
|
||||
continue
|
||||
if len(recs) == 1 and recs[0]['squadron'] == 'Enumerate':
|
||||
nenum[os.path.basename(pk)] += 1
|
||||
for _t, fn, _v in recs[0]['fields']:
|
||||
if fn:
|
||||
decl.add(fn)
|
||||
|
||||
H = set()
|
||||
for n in names:
|
||||
@@ -62,6 +76,29 @@ def main():
|
||||
for k, c in sorted(fam.items(), key=lambda kv: (-kv[1], kv[0])):
|
||||
print(" x%-4d %s" % (c, k))
|
||||
|
||||
# ROUTE 2 -- the declaration tables. An `Enumerate` object lists, as FIELD
|
||||
# NAMES, the names of other objects; each resolves as `<name>.tbl`. This is
|
||||
# the naming source the string harvest cannot see, because these names are
|
||||
# never spelled out as `something.tbl` anywhere on the disc.
|
||||
DECL = {}
|
||||
for n in sorted(decl):
|
||||
DECL.setdefault(U.name_hash(n + '.tbl'), n + '.tbl')
|
||||
print("\n## ROUTE 2 the `Enumerate` declaration tables")
|
||||
print(" declaration objects found: %d distinct names they declare: %d"
|
||||
% (sum(nenum.values()), len(decl)))
|
||||
for nm, c in sorted(nenum.items(), key=lambda kv: (-kv[1], kv[0])):
|
||||
print(" x%-5d %s" % (c, nm))
|
||||
gained = 0
|
||||
for nm, ent in rows:
|
||||
g = sum(1 for h in ent if h in DECL and h not in NAMES)
|
||||
if g:
|
||||
print(" %-30s +%d newly named" % (nm, g))
|
||||
gained += g
|
||||
print(" entries named by route 2 that route 1 could not: %d" % gained)
|
||||
miss = [v for k, v in sorted(DECL.items()) if not any(k in ent for ent in toc.values())]
|
||||
print(" declared names with NO pak entry: %d %s" % (len(miss), sorted(miss)[:6]))
|
||||
NAMES.update({k: v for k, v in DECL.items() if k not in NAMES})
|
||||
|
||||
# WHY an archive is low-coverage: split its UNNAMED entries by content.
|
||||
print("\n## What the UNNAMED entries are (magic of every entry not resolved above)")
|
||||
print(" %-28s %6s %6s %8s %9s %6s" % (
|
||||
|
||||
Reference in New Issue
Block a user