5ad6e9397c
Censusing ptc_pack's own naming vocabulary turned up a third variant of the prefix trap, and this one had been corrupting a number the corpus carried. 268 of ptc_pack's names do not start with eff_ at all. They start with EF_IDX_, as in EF_IDX_eff_d001_f. A regex anchored at eff_ chops that prefix off and merges distinct names, which is exactly where the earlier figure of 532 came from. Enumerating maximal [A-Za-z0-9_] runs gives 727. The two earlier traps were a STORED name being longer (rot_n001_break) and a BOUND name being a prefix (eff_f0002 inside eff_f0002_barnhaze); this is the third - a prefix the pattern cannot see at all, because its anchor sits in the middle of the real name. Looking each bound name up bare AND under EF_IDX_ resolves 25 of the 34 that were unlocated. The map is now 128 of 137, and the residual is 9, small enough to print: eff_e0044, eff_f0002, eff_f0002_barn, eff_h308, eff_j002_e01, eff_j002_e02, eff_m010_wep_85, eff_m011_wep_85, eff_n0071. All 17 eff_l### are among the recovered. This withdraws my own previous correction. I had recorded Base.xpr (53) as holding more bound effects than ptc_pack (46), and struck out "ptc_pack is the effect library". With the prefixed keys counted ptc_pack holds 71 - it IS the larger library, and the 46 was an undercount from the same truncating pattern. Two shared libraries remains right; which one is bigger does not. The suffix vocabulary: 106 distinct tokens over the 727 names - IDX 223 (the prefix above), _f 137, _e 119, _root 87, _col 54, _mdl 45, _break 43, _ring 38, _ALL 17, _haze 14, _thunder 10. That census counts ALL tokens rather than trailing ones, which is precisely how the EF_IDX_ PREFIX surfaced inside what I had first labelled a suffix list - the mislabel found the bug. Testing the structural candidates the way _hangar was tested, does the suffixed name have a bare parent: _ALL 17 names 17 of 17 _root 87 64 of 87 _break 30 15 of 30 _e 74 0 of 74 _f 61 0 of 61 _root is strictly terminal - 87 of 87, and it never appears mid-name. The compound shapes put it outermost: _e_root 19, _f_root 18, _break_root 13, bare _root 30. So the order is <stem>_[<faction>|<break>]_root and _root reads as a hierarchy marker rather than a variant - though 64 of 87 having a bare parent means it is not simply the parent of an existing node, and _break at 15 of 30 is likewise not a plain destroyed-twin-of-everything. _e/_f never have a bare parent, 0 of 135. That is independent asset-side confirmation of the faction law: an effect is authored per faction and there is no faction-neutral original for either side to derive from. effect-homes.txt changes 5/30 and every line pairs: five values changed (103->128, 34->9, ptc_pack 46->71 and its sort position, the residual header, 3-digit 80->105 of 110) plus 25 pure deletions, exactly the 25 recovered names. All are 3-digit, so the 4-digit line is unchanged at 23 of 27. The other sixteen artefacts are byte-identical.
80 lines
3.0 KiB
Python
80 lines
3.0 KiB
Python
#!/usr/bin/env python3
|
|
"""Which .xpr package holds each effect the datasheets bind.
|
|
|
|
Effect names are matched EXACTLY, not as substrings: Base.xpr contains
|
|
eff_f0002_barnhaze, so a substring test for "eff_f0002" reports a false hit for
|
|
a resource that is not there. Names are pulled as maximal [A-Za-z0-9_] runs
|
|
containing "eff_", which yields whole resource names -- 268 of ptc_pack's 727
|
|
carry an EF_IDX_ prefix, so a pattern anchored at "eff_" would truncate them and
|
|
silently merge distinct names. A bound name is looked up bare AND under that
|
|
prefix.
|
|
|
|
Regenerates docs/re/data/effect-homes.txt.
|
|
"""
|
|
import glob, os, re, subprocess, sys, collections
|
|
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
|
from unit_substructures import pak_entries
|
|
import unitgroup as U
|
|
|
|
XPR = '/work/sylph_extract/hidden/resource3d'
|
|
FX = re.compile(r'(FxModel|EffectName|Effect_|ShellModel|CoverModel)')
|
|
|
|
def width(n):
|
|
m = re.match(r'eff_[a-z](\d+)', n)
|
|
return len(m.group(1)) if m else 0
|
|
|
|
def main():
|
|
if not os.path.isdir(XPR):
|
|
print('disc not mounted; nothing to do'); return
|
|
home = collections.defaultdict(set)
|
|
for path in sorted(glob.glob(os.path.join(XPR, '*.xpr'))):
|
|
out = subprocess.run(['grep', '-o', '-a', '[A-Za-z0-9_]*eff_[A-Za-z0-9_]*', path],
|
|
capture_output=True, text=True, errors='replace').stdout
|
|
for name in set(out.split()):
|
|
home[name].add(os.path.basename(path))
|
|
|
|
bound = set()
|
|
for pk in sorted(glob.glob('/work/sylph_extract/**/*.pak', recursive=True)):
|
|
for _h, b in pak_entries(pk):
|
|
if b[:4] != b'IDXD':
|
|
continue
|
|
try:
|
|
recs = U.parse(b)
|
|
except Exception:
|
|
continue
|
|
for r in recs:
|
|
for _t, f, v in r['fields']:
|
|
if f and FX.search(f) and isinstance(v, str) and v.startswith('eff_'):
|
|
bound.add(v)
|
|
|
|
# a name may be stored bare or under the EF_IDX_ index prefix
|
|
def where(n):
|
|
return home.get(n, set()) | home.get('EF_IDX_' + n, set())
|
|
|
|
res = sorted(n for n in bound if where(n))
|
|
un = sorted(n for n in bound if not where(n))
|
|
print('effect names bound by a datasheet field : %d' % len(bound))
|
|
print('resolved to an .xpr package (exact name): %d' % len(res))
|
|
print('unlocated : %d' % len(un))
|
|
print()
|
|
per = collections.Counter()
|
|
for n in res:
|
|
for f in where(n):
|
|
per[f] += 1
|
|
print('resolved, by package:')
|
|
for f, k in sorted(per.items(), key=lambda kv: (-kv[1], kv[0])):
|
|
print(' %-26s %d' % (f, k))
|
|
print()
|
|
print('by digit-width:')
|
|
for k in (3, 4, 0):
|
|
s = [n for n in bound if width(n) == k]
|
|
if s:
|
|
print(' %d-digit: %d resolved of %d' % (k, len([n for n in s if where(n)]), len(s)))
|
|
print()
|
|
print('the %d unlocated, in full:' % len(un))
|
|
for n in un:
|
|
print(' %s' % n)
|
|
|
|
if __name__ == '__main__':
|
|
main()
|