re: the effect->package map, 103 of 137 - and eff_f0002 was a substring artefact

Enumerating eff_* names per .xpr across all 166 packages and matching the bound
names EXACTLY gives a real home for 103 of the 137, up from the 71 ptc_pack
alone accounted for. Only 36 packages carry an effect name at all, and two
dominate:

  Base.xpr                  53
  ptc_pack.xpr              46
  Stage_S28.xpr              2
  five rou_f001_wep_NN.xpr   1 each

So there are TWO shared effect libraries, not one - and ptc_pack.xpr is the only
*_pack bundle on the disc, so no third shared library is hiding. By digit-width:
3-digit 80 resolved of 110, 4-digit 23 of 27. The previous iteration's split
survives and sharpens - the four-digit series really does live outside ptc_pack
(that zero stands), and now we can say where: Base.xpr.

Correction to the previous commit. It reported eff_f0002 and eff_f0002_barn as
present in Base.xpr. Both were SUBSTRING artefacts: what the file actually holds
is eff_f0002_barnhaze, one longer resource name that grep -l eff_f0002 and
grep -l eff_f0002_barn each match inside. Neither bound name is there. This is
the corpus's own paid-for prefix lesson arriving from the other direction - last
time it was rot_n001 vs rot_n001_break with the stored name longer; here the
BOUND name was the prefix. The new map is exact-keyed and does not have this
failure mode, so the earlier positive is withdrawn.

34 names remain unlocated, dominated by a family the last pass did not single
out: eff_l### with 17 of the 34, then h 4, s 4, j 2, m 2, t 1, and four
four-digit names - eff_e0044, eff_f0002, eff_f0002_barn, eff_n0071.

Scope note worth keeping: the j 22 / t 14 clustering reported last time was the
residual against ptc_pack ALONE; against all packages those families are largely
accounted for and l is what is left. Both numbers are right for their own
population, which is exactly why a residual has to say what it was measured
against.

New artefact with its regenerator: tools/re-capture/effect_homes.py ->
docs/re/data/effect-homes.txt, which lists all 34 by name. All sixteen existing
artefacts byte-identical.

docs/re/BACKLOG.md

@@ -6707,3 +6707,31 @@ cluster on `j` (22) and `t` (14); `ptc_pack` holds just one `j` name, so
 🟡 **Not closed**: 66 of 137 bound effects still have no located home,
 `eff_n0071` included. ❔ Next: the `eff_j###` family and the four-digit series —
 not another disc-wide grep.
+
+---
+
+## ✅❌ 2026-08-28 — the effect→package map (103 of 137), and a substring correction
+
+Item (a). Enumerating `eff_*` **per `.xpr`** across all 166 packages and matching
+bound names **exactly** locates **103 of the 137**, up from 71. Only 36 packages
+carry an effect name; two dominate: **`Base.xpr` 53** and **`ptc_pack.xpr` 46**
+(then `Stage_S28.xpr` 2 and five `rou_f001_wep_NN.xpr` at 1 each).
+
+🔑 **Two shared effect libraries, not one** — and `ptc_pack.xpr` is the only
+`*_pack` bundle on the disc. By digit-width: **3-digit 80 of 110, 4-digit 23 of
+27**, so the previous iteration's split survives and sharpens — the four-digit
+series lives outside `ptc_pack`, in `Base.xpr`.
+
+❌ **Correction to the previous entry**: `eff_f0002` and `eff_f0002_barn` do
+**not** ship. `Base.xpr` contains **`eff_f0002_barnhaze`**, and both substring
+greps matched inside it. The corpus's paid-for prefix lesson arriving from the
+other direction — this time the *bound* name was a prefix of the *stored* one.
+
+🟡 **34 remain**, dominated by **`eff_l###` (17)**, then `h` 4, `s` 4, `j` 2,
+`m` 2, `t` 1, and four four-digit names including `eff_n0071`. ⚠️ Scope note: the
+`j` 22 / `t` 14 clustering reported last time was the residual against
+`ptc_pack` alone; against all packages `l` is what is left. **Say which
+population a residual is measured against.**
+
+New artefact: `tools/re-capture/effect_homes.py` → `docs/re/data/effect-homes.txt`
+(lists all 34 by name).

docs/re/data/effect-homes.txt

@@ -0,0 +1,53 @@
+effect names bound by a datasheet field : 137
+resolved to an .xpr package (exact name): 103
+unlocated                               : 34
+
+resolved, by package:
+   Base.xpr                   53
+   ptc_pack.xpr               46
+   Stage_S28.xpr              2
+   rou_f001_wep_28.xpr        1
+   rou_f001_wep_27.xpr        1
+   rou_f001_wep_12.xpr        1
+   rou_f001_wep_82.xpr        1
+   rou_f001_wep_59.xpr        1
+
+by digit-width:
+   3-digit: 80 resolved of 110
+   4-digit: 23 resolved of 27
+
+the 34 unlocated, in full:
+   eff_e0044
+   eff_f0002
+   eff_f0002_barn
+   eff_h000
+   eff_h006
+   eff_h308
+   eff_h310
+   eff_j002_e01
+   eff_j002_e02
+   eff_l002_f
+   eff_l010_e
+   eff_l011_e
+   eff_l101_e
+   eff_l101_f
+   eff_l102_e
+   eff_l102_f
+   eff_l104_e
+   eff_l104_f
+   eff_l105_e
+   eff_l105_f
+   eff_l106_e
+   eff_l106_f
+   eff_l107_e
+   eff_l108_e
+   eff_l201_e
+   eff_l201_f
+   eff_m010_wep_85
+   eff_m011_wep_85
+   eff_n0071
+   eff_s100
+   eff_s901_e01_B
+   eff_s901_e02_D
+   eff_s901_e03_D
+   eff_t130

docs/re/structures/unit-substructure-records.md

@@ -308,3 +308,56 @@ So `eff_j###` is a third grouping that is almost entirely elsewhere.
 no located home, `eff_n0071` among them. But the route is now a real one with a
 number on it, and the next step is the `eff_j###` family and the four-digit
 series rather than another disc-wide grep.
+
+### ✅ The effect→package map — 103 of 137 located, and a correction
+
+Enumerating `eff_*` names **per `.xpr`** across all 166 packages and matching the
+bound names **exactly** gives a real home for **103 of the 137**, up from the 71
+`ptc_pack` alone accounted for. Only **36** packages carry an effect name at
+all, and two dominate:
+
+| package | bound effects it holds |
+|---|---|
+| `Base.xpr` | **53** |
+| `ptc_pack.xpr` | **46** |
+| `Stage_S28.xpr` | 2 |
+| five `rou_f001_wep_NN.xpr` | 1 each |
+
+🔑 **There are two shared effect libraries, not one.** `ptc_pack.xpr` is the
+particle library the previous iteration found; **`Base.xpr` is the larger home**
+for bound effects. `ptc_pack.xpr` is also the *only* `*_pack` library on the
+disc — no other shared bundle exists.
+
+By digit-width: **3-digit 80 resolved of 110, 4-digit 23 of 27.** So the
+previous iteration's reading survives and sharpens — the four-digit series
+really does live outside `ptc_pack` (that zero stands), and now we can say where:
+`Base.xpr`.
+
+### ❌ Correction: `eff_f0002` does **not** ship — that was a substring hit
+
+The previous iteration reported `eff_f0002` and `eff_f0002_barn` as present in
+`Base.xpr`. **Both were substring artefacts.** What the file actually contains
+is **`eff_f0002_barnhaze`** — one longer resource name that `grep -l eff_f0002`
+and `grep -l eff_f0002_barn` both match inside. Neither bound name is there.
+
+This is the corpus's own paid-for lesson arriving from the other direction: a
+prefix hit is not an exact hit (cf. `rot_n001` vs `rot_n001_break`), and here it
+was the *bound* name that was a prefix of the *stored* one. The map above is
+exact-keyed, so it does not have this failure; the earlier positive is withdrawn.
+
+### 🟡 The 34 that remain
+
+The unlocated residual is dominated by a family the previous iteration did not
+single out — **`eff_l###` with 17 of the 34** — then `eff_h###` 4, `eff_s###` 4,
+`eff_j###` 2, `eff_m###` 2, `eff_t###` 1, and four four-digit names
+(`eff_e0044`, `eff_f0002`, `eff_f0002_barn`, `eff_n0071`).
+
+⚠️ Note the scope shift: the `j` 22 / `t` 14 clustering reported last time was
+the residual **against `ptc_pack` alone**; against *all* packages those families
+are largely accounted for and `l` is what is left. Both numbers are right for
+their own population — which is the reason to say which population a residual
+is measured against.
+
+Regenerate with `tools/re-capture/effect_homes.py` →
+[`docs/re/data/effect-homes.txt`](../data/effect-homes.txt), which lists all 34
+by name.

tools/re-capture/effect_homes.py

@@ -0,0 +1,72 @@
+#!/usr/bin/env python3
+"""Which .xpr package holds each effect the datasheets bind.
+
+Effect names are matched EXACTLY, not as substrings: Base.xpr contains
+eff_f0002_barnhaze, so a substring test for "eff_f0002" reports a false hit for
+a resource that is not there. Names are pulled as maximal [A-Za-z0-9_] runs
+containing "eff_", which yields bare resource names.
+
+Regenerates docs/re/data/effect-homes.txt.
+"""
+import glob, os, re, subprocess, sys, collections
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from unit_substructures import pak_entries
+import unitgroup as U
+
+XPR = '/work/sylph_extract/hidden/resource3d'
+FX = re.compile(r'(FxModel|EffectName|Effect_|ShellModel|CoverModel)')
+
+def width(n):
+    m = re.match(r'eff_[a-z](\d+)', n)
+    return len(m.group(1)) if m else 0
+
+def main():
+    if not os.path.isdir(XPR):
+        print('disc not mounted; nothing to do'); return
+    home = collections.defaultdict(set)
+    for path in sorted(glob.glob(os.path.join(XPR, '*.xpr'))):
+        out = subprocess.run(['grep', '-o', '-a', '[A-Za-z0-9_]*eff_[A-Za-z0-9_]*', path],
+                             capture_output=True, text=True, errors='replace').stdout
+        for name in set(out.split()):
+            home[name].add(os.path.basename(path))
+
+    bound = set()
+    for pk in sorted(glob.glob('/work/sylph_extract/**/*.pak', recursive=True)):
+        for _h, b in pak_entries(pk):
+            if b[:4] != b'IDXD':
+                continue
+            try:
+                recs = U.parse(b)
+            except Exception:
+                continue
+            for r in recs:
+                for _t, f, v in r['fields']:
+                    if f and FX.search(f) and isinstance(v, str) and v.startswith('eff_'):
+                        bound.add(v)
+
+    res = sorted(n for n in bound if n in home)
+    un = sorted(n for n in bound if n not in home)
+    print('effect names bound by a datasheet field : %d' % len(bound))
+    print('resolved to an .xpr package (exact name): %d' % len(res))
+    print('unlocated                               : %d' % len(un))
+    print()
+    per = collections.Counter()
+    for n in res:
+        for f in home[n]:
+            per[f] += 1
+    print('resolved, by package:')
+    for f, k in per.most_common():
+        print('   %-26s %d' % (f, k))
+    print()
+    print('by digit-width:')
+    for k in (3, 4, 0):
+        s = [n for n in bound if width(n) == k]
+        if s:
+            print('   %d-digit: %d resolved of %d' % (k, len([n for n in s if n in home]), len(s)))
+    print()
+    print('the %d unlocated, in full:' % len(un))
+    for n in un:
+        print('   %s' % n)
+
+if __name__ == '__main__':
+    main()