From 23a6cfd979ffe849a73ac03fc04be30b099e6986 Mon Sep 17 00:00:00 2001 From: Sylpheed RE agent Date: Fri, 28 Aug 2026 10:01:53 +0000 Subject: [PATCH] re: the EF_IDX_ prefix - ptc_pack has 727 names, and the map reaches 128 of 137 Censusing ptc_pack's own naming vocabulary turned up a third variant of the prefix trap, and this one had been corrupting a number the corpus carried. 268 of ptc_pack's names do not start with eff_ at all. They start with EF_IDX_, as in EF_IDX_eff_d001_f. A regex anchored at eff_ chops that prefix off and merges distinct names, which is exactly where the earlier figure of 532 came from. Enumerating maximal [A-Za-z0-9_] runs gives 727. The two earlier traps were a STORED name being longer (rot_n001_break) and a BOUND name being a prefix (eff_f0002 inside eff_f0002_barnhaze); this is the third - a prefix the pattern cannot see at all, because its anchor sits in the middle of the real name. Looking each bound name up bare AND under EF_IDX_ resolves 25 of the 34 that were unlocated. The map is now 128 of 137, and the residual is 9, small enough to print: eff_e0044, eff_f0002, eff_f0002_barn, eff_h308, eff_j002_e01, eff_j002_e02, eff_m010_wep_85, eff_m011_wep_85, eff_n0071. All 17 eff_l### are among the recovered. This withdraws my own previous correction. I had recorded Base.xpr (53) as holding more bound effects than ptc_pack (46), and struck out "ptc_pack is the effect library". With the prefixed keys counted ptc_pack holds 71 - it IS the larger library, and the 46 was an undercount from the same truncating pattern. Two shared libraries remains right; which one is bigger does not. The suffix vocabulary: 106 distinct tokens over the 727 names - IDX 223 (the prefix above), _f 137, _e 119, _root 87, _col 54, _mdl 45, _break 43, _ring 38, _ALL 17, _haze 14, _thunder 10. That census counts ALL tokens rather than trailing ones, which is precisely how the EF_IDX_ PREFIX surfaced inside what I had first labelled a suffix list - the mislabel found the bug. Testing the structural candidates the way _hangar was tested, does the suffixed name have a bare parent: _ALL 17 names 17 of 17 _root 87 64 of 87 _break 30 15 of 30 _e 74 0 of 74 _f 61 0 of 61 _root is strictly terminal - 87 of 87, and it never appears mid-name. The compound shapes put it outermost: _e_root 19, _f_root 18, _break_root 13, bare _root 30. So the order is _[|]_root and _root reads as a hierarchy marker rather than a variant - though 64 of 87 having a bare parent means it is not simply the parent of an existing node, and _break at 15 of 30 is likewise not a plain destroyed-twin-of-everything. _e/_f never have a bare parent, 0 of 135. That is independent asset-side confirmation of the faction law: an effect is authored per faction and there is no faction-neutral original for either side to derive from. effect-homes.txt changes 5/30 and every line pairs: five values changed (103->128, 34->9, ptc_pack 46->71 and its sort position, the residual header, 3-digit 80->105 of 110) plus 25 pure deletions, exactly the 25 recovered names. All are 3-digit, so the 4-digit line is unchanged at 23 of 27. The other sixteen artefacts are byte-identical. --- docs/re/BACKLOG.md | 32 +++++++++ docs/re/data/effect-homes.txt | 35 ++------- .../structures/unit-substructure-records.md | 72 +++++++++++++++++++ tools/re-capture/effect_homes.py | 17 +++-- 4 files changed, 121 insertions(+), 35 deletions(-) diff --git a/docs/re/BACKLOG.md b/docs/re/BACKLOG.md index 0d04a00..2dbd507 100644 --- a/docs/re/BACKLOG.md +++ b/docs/re/BACKLOG.md @@ -6794,3 +6794,35 @@ for those kinds; there was nothing to test. regex requiring `UN_###__`; the looser form finds **71**. The extra is **`UN_e910_core_ADAN_GeneratorCore`**, which inserts a sub-kind before the faction tag. Grammar is `UN_###_[_]_`. + +--- + +## ❌✅ 2026-08-28 — `EF_IDX_` prefix: `ptc_pack` has 727 names, and the map reaches 128 of 137 + +Item (a). Censusing `ptc_pack`'s naming vocabulary found a third prefix trap that +had been corrupting a carried number. + +❌ **`ptc_pack` holds 727 distinct names, not 532.** **268 do not start with +`eff_`** — they start with **`EF_IDX_`** (e.g. `EF_IDX_eff_d001_f`). A regex +anchored at `eff_` truncates that and merges distinct names. ⚠️ Third variant of +the trap: the stored name longer (`rot_n001_break`), the bound name a prefix +(`eff_f0002`), and now **a prefix the pattern cannot see because its anchor sits +mid-name**. + +✅ **Looking each bound name up bare AND under `EF_IDX_` resolves 25 of the 34.** +The map is now **128 of 137**, residual **9**: `eff_e0044`, `eff_f0002`, +`eff_f0002_barn`, `eff_h308`, `eff_j002_e01`, `eff_j002_e02`, `eff_m010_wep_85`, +`eff_m011_wep_85`, `eff_n0071`. + +❌ **Withdraws my own previous correction**: I had recorded `Base.xpr` (53) as +holding more bound effects than `ptc_pack` (46). With prefixed keys `ptc_pack` +holds **71** — it *is* the larger library; 46 was an undercount from the same +pattern. + +✅ **Suffix vocabulary** (106 tokens over 727 names): `IDX` 223, `_f` 137, `_e` +119, `_root` 87, `_col` 54, `_mdl` 45, `_break` 43, `_ring` 38, `_ALL` 17. +**`_root` is strictly terminal, 87 of 87, never mid-name**; compounds place it +outermost (`_e_root` 19, `_f_root` 18, `_break_root` 13, bare 30). Bare-parent +test: **`_ALL` 17/17**, `_root` 64/87, `_break` 15/30, and 🔑 **`_e`/`_f` 0 of +135** — asset-side confirmation that effects are authored per faction with no +faction-neutral original. diff --git a/docs/re/data/effect-homes.txt b/docs/re/data/effect-homes.txt index 35b8fd8..93e2dbc 100644 --- a/docs/re/data/effect-homes.txt +++ b/docs/re/data/effect-homes.txt @@ -1,10 +1,10 @@ effect names bound by a datasheet field : 137 -resolved to an .xpr package (exact name): 103 -unlocated : 34 +resolved to an .xpr package (exact name): 128 +unlocated : 9 resolved, by package: + ptc_pack.xpr 71 Base.xpr 53 - ptc_pack.xpr 46 Stage_S28.xpr 2 rou_f001_wep_12.xpr 1 rou_f001_wep_27.xpr 1 @@ -13,41 +13,16 @@ resolved, by package: rou_f001_wep_82.xpr 1 by digit-width: - 3-digit: 80 resolved of 110 + 3-digit: 105 resolved of 110 4-digit: 23 resolved of 27 -the 34 unlocated, in full: +the 9 unlocated, in full: eff_e0044 eff_f0002 eff_f0002_barn - eff_h000 - eff_h006 eff_h308 - eff_h310 eff_j002_e01 eff_j002_e02 - eff_l002_f - eff_l010_e - eff_l011_e - eff_l101_e - eff_l101_f - eff_l102_e - eff_l102_f - eff_l104_e - eff_l104_f - eff_l105_e - eff_l105_f - eff_l106_e - eff_l106_f - eff_l107_e - eff_l108_e - eff_l201_e - eff_l201_f eff_m010_wep_85 eff_m011_wep_85 eff_n0071 - eff_s100 - eff_s901_e01_B - eff_s901_e02_D - eff_s901_e03_D - eff_t130 diff --git a/docs/re/structures/unit-substructure-records.md b/docs/re/structures/unit-substructure-records.md index 8ee1a15..10b6c94 100644 --- a/docs/re/structures/unit-substructure-records.md +++ b/docs/re/structures/unit-substructure-records.md @@ -445,3 +445,75 @@ counted 42 + 26 + 2 = 70 unit objects using a regex that required extra is **`UN_e910_core_ADAN_GeneratorCore`**, which inserts a sub-kind token *before* the faction tag. So the ID grammar is `UN_###_[_]_`, and both counts were right for their own pattern. + +### ❌ `ptc_pack` holds 727 names, not 532 — an `EF_IDX_` prefix the pattern truncated + +Censusing the package's own naming vocabulary turned up a third variant of the +prefix trap, and this one had been corrupting a number the corpus carried. + +**268 of `ptc_pack`'s names do not start with `eff_`** — they start with +**`EF_IDX_`**, e.g. `EF_IDX_eff_d001_f`. A regex anchored at `eff_` chops that +prefix off and merges distinct names, which is where the earlier figure of 532 +came from. Enumerating **maximal `[A-Za-z0-9_]` runs** gives **727**. + +⚠️ The two earlier traps were a *stored* name being longer (`rot_n001_break`) and +a *bound* name being a prefix (`eff_f0002` ⊂ `eff_f0002_barnhaze`). This is the +third: **a prefix the pattern cannot see at all**, because the anchor sits in the +middle of the real name. + +### ✅ The map improves to 128 of 137 — and a claim of mine is withdrawn + +Looking each bound name up **bare and under `EF_IDX_`** resolves **25 of the 34** +that were unlocated. The map is now: + +| package | bound effects held | +|---|---| +| **`ptc_pack.xpr`** | **71** | +| `Base.xpr` | 53 | +| `Stage_S28.xpr` | 2 | +| five `rou_f001_wep_NN.xpr` | 1 each | + +❌ **This withdraws the previous section's correction.** I had recorded that +`Base.xpr` holds *more* bound effects than `ptc_pack` (53 vs 46) and struck out +"`ptc_pack` is the effect library". With the prefixed keys counted, `ptc_pack` +holds **71** — it *is* the larger library, and the 46 was an undercount from the +same truncating pattern. Two shared libraries remains right; which is bigger does +not. + +🟡 **Nine names remain unlocated**, small enough to print in full: `eff_e0044`, +`eff_f0002`, `eff_f0002_barn`, `eff_h308`, `eff_j002_e01`, `eff_j002_e02`, +`eff_m010_wep_85`, `eff_m011_wep_85`, `eff_n0071`. + +### ✅ The suffix vocabulary, and `_root` as the outermost token + +Splitting all 727 names on `_` gives 106 distinct tokens. The frequent ones are +`IDX` 223 (the prefix above), `_f` 137, `_e` 119, `_root` 87, `_col` 54, +`_mdl` 45, `_break` 43, `_ring` 38, `_ALL` 17, `_haze` 14, `_thunder` 10 — a mix +of one structural marker, the faction pair, and descriptive part names. + +⚠️ Method note: that census counts **all** tokens, not trailing ones — which is +exactly how the `EF_IDX_` *prefix* surfaced in what I first labelled a suffix +list. The mislabel found the bug. + +Testing the two structural candidates the way `_hangar` was tested — does the +suffixed name have a bare parent? + +| token | names | bare parent present | +|---|---|---| +| `_ALL` | 17 | **17 of 17** | +| `_root` | 87 | 64 of 87 | +| `_break` | 30 | 15 of 30 | +| `_e` | 74 | **0 of 74** | +| `_f` | 61 | **0 of 61** | + +🔑 **`_root` is strictly terminal — 87 of 87, and `_root` never appears +mid-name.** The compound shapes place it outermost: `_e_root` 19, `_f_root` 18, +`_break_root` 13, bare `_root` 30. So the order is +`_[|]_root`, and `_root` reads as a hierarchy marker rather +than a variant — though 64 of 87 having a bare parent means it is not simply "the +parent of an existing node", and `_break` at 15 of 30 is likewise not a plain +"destroyed twin of every effect". + +🔑 **`_e`/`_f` never have a bare parent — 0 of 135.** That is independent, +asset-side confirmation of the faction law: an effect is authored per faction, +and there is no faction-neutral original for either side to derive from. diff --git a/tools/re-capture/effect_homes.py b/tools/re-capture/effect_homes.py index 5c1aa52..f86d4eb 100644 --- a/tools/re-capture/effect_homes.py +++ b/tools/re-capture/effect_homes.py @@ -4,7 +4,10 @@ Effect names are matched EXACTLY, not as substrings: Base.xpr contains eff_f0002_barnhaze, so a substring test for "eff_f0002" reports a false hit for a resource that is not there. Names are pulled as maximal [A-Za-z0-9_] runs -containing "eff_", which yields bare resource names. +containing "eff_", which yields whole resource names -- 268 of ptc_pack's 727 +carry an EF_IDX_ prefix, so a pattern anchored at "eff_" would truncate them and +silently merge distinct names. A bound name is looked up bare AND under that +prefix. Regenerates docs/re/data/effect-homes.txt. """ @@ -44,15 +47,19 @@ def main(): if f and FX.search(f) and isinstance(v, str) and v.startswith('eff_'): bound.add(v) - res = sorted(n for n in bound if n in home) - un = sorted(n for n in bound if n not in home) + # a name may be stored bare or under the EF_IDX_ index prefix + def where(n): + return home.get(n, set()) | home.get('EF_IDX_' + n, set()) + + res = sorted(n for n in bound if where(n)) + un = sorted(n for n in bound if not where(n)) print('effect names bound by a datasheet field : %d' % len(bound)) print('resolved to an .xpr package (exact name): %d' % len(res)) print('unlocated : %d' % len(un)) print() per = collections.Counter() for n in res: - for f in home[n]: + for f in where(n): per[f] += 1 print('resolved, by package:') for f, k in sorted(per.items(), key=lambda kv: (-kv[1], kv[0])): @@ -62,7 +69,7 @@ def main(): for k in (3, 4, 0): s = [n for n in bound if width(n) == k] if s: - print(' %d-digit: %d resolved of %d' % (k, len([n for n in s if n in home]), len(s))) + print(' %d-digit: %d resolved of %d' % (k, len([n for n in s if where(n)]), len(s))) print() print('the %d unlocated, in full:' % len(un)) for n in un: