re: the Enumerate declaration tables close DefTables (130 -> 1413 of 1425 IDXD)
The string harvest could never name DefTables because those names are not
spelled out as "something.tbl" anywhere on the disc -- they are declared.
An IDXD object whose single record is named Enumerate is a DECLARATION TABLE:
its field NAMES are the names of other objects, each resolving as
name_hash("<field name>.tbl"). EnumLODSet_test.tbl declares 676,
EnumGameModel_test.tbl 360; the disc holds 144 such objects (138 in DefTables,
one in each GP_MAIN_GAME_*) declaring 1298 distinct names.
Route 2 names +1283 entries route 1 could not. 130 + 1283 + 12 = 1425, no
overlap, 99.2 % coverage. Zero partials: of 5 suffixes x 6 prefixes, ('', .tbl)
scored 1036/1036 and every other combination scored 0. Residual in full: 8
declaration tables nothing declares, 2 LOD sets (Model rou_e004 / rou_e013), 2
motion sets; 15 declared names have no pak entry at all.
REFUTED alongside it: the 40 XPR2 manifests are not the naming source -- their
82 Name= values and 82 DataFile/Source paths resolve 0 entries under any of the
5 suffixes. They share the MODEL namespace only: 40 of the 82 appear as the
Model field value inside the tables.
Artefact +17/-18, every line paired, byte-identical across two runs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
This commit is contained in:
Claude (auto)
@@ -4793,3 +4793,61 @@ manifests name meshes like `machines\rou_e104\objects\…xbg` — whether those
|
||||
names hash to the LOD/motion table keys is untested). The 337 unnamed IDXD in
|
||||
each `GP_MAIN_GAME_*` are also uncharacterised — the same census would say what
|
||||
they are, and was not run this iteration.
|
||||
|
||||
---
|
||||
|
||||
## ✅✅ 2026-08-27 — the `Enumerate` declaration tables close `DefTables` (99.2 %)
|
||||
|
||||
Item taken: "do the XPR2 manifest model names hash to the `DefTables` LOD/motion
|
||||
tables?" — **refuted**, and the real naming source found next to it.
|
||||
|
||||
**Docs checked first:** `archive-naming.md` (mine); `isl-condition-builtins.md`
|
||||
owns the 40 XPR2 manifests; `stage-definition-table.md` / `stage-mission-tables.md`
|
||||
own the names `EnumLODSet_test.tbl` / `EnumGameModel_test.tbl` and record that
|
||||
they live in `DefTables.pak` — but **neither says what is inside them**.
|
||||
|
||||
🔴 **REFUTED — the XPR2 manifests are not a naming source.** Their 82 `Name=`
|
||||
values and 82 `DataFile`/`Source` paths, hashed bare and under
|
||||
`.tbl`/`.xbg`/`.xpr`/`.prt`/`game:\`, resolve **0** `DefTables` entries. They do
|
||||
share the *model* namespace: 40 of the 82 appear as the `Model` field VALUE
|
||||
inside the LOD/motion tables (1 272 distinct `Model` values). ⚠️ The first
|
||||
attempt regexed `Name="` and found 0 values — the manifests write `Name = "…"`
|
||||
with spaces; a regex miss looks exactly like a null result.
|
||||
|
||||
✅✅ **An IDXD object whose single record is named `Enumerate` is a DECLARATION
|
||||
TABLE: its FIELD NAMES are the names of other objects**, each resolving as
|
||||
`name_hash("<field name>.tbl")`. `EnumLODSet_test.tbl` declares 676 names,
|
||||
`EnumGameModel_test.tbl` 360; the disc holds **144** such objects (138 in
|
||||
`DefTables`, one in each `GP_MAIN_GAME_*`) declaring **1 298** distinct names.
|
||||
|
||||
| | |
|
||||
|---|---:|
|
||||
| `DefTables` named by the string harvest (route 1) | 130 |
|
||||
| **named by the declaration tables (route 2)** | **+1 283** |
|
||||
| still unnamed | **12** |
|
||||
| **coverage** | **1 413 / 1 425 IDXD = 99.2 %** |
|
||||
|
||||
`130 + 1283 + 12 = 1425`, no overlap between routes. ✅ **Zero partials and the
|
||||
suffix is exact**: of 5 suffixes × 6 prefixes, `('', '.tbl')` scored 1 036/1 036
|
||||
on the `_test` declarations and every other combination scored 0.
|
||||
|
||||
**Residual, in full:** 8 declaration tables nothing declares (the other 130 are
|
||||
named by route 1, from the stage tables that reference them), 2
|
||||
`Generic+Level_0` LOD sets whose `Model` reads `rou_e004`/`rou_e013`, and 2
|
||||
motion sets. **15 declared names have no pak entry** — content that did not ship.
|
||||
|
||||
`archive_naming.py` extended with route 2; artefact **+17/−18**, every line
|
||||
paired (the DefTables census row 130→1413 and its shape list collapsing 17→3).
|
||||
Byte-identical across two runs.
|
||||
|
||||
🟡 Not settled: **why those 12 are unreachable** — no attempt was made to name
|
||||
the 8 orphan declaration tables or the 4 stragglers. And the six
|
||||
`GP_MAIN_GAME_*` `Enumerate` objects named **nothing** there; what they declare
|
||||
and where it lives was not chased.
|
||||
|
||||
❔ Handed forward: with `DefTables` closed, the largest unnamed block is the
|
||||
identical **337 IDXD in each `GP_MAIN_GAME_*`**, 53 shapes led by `Shell+Weapon`
|
||||
(90), `Faces+Generic` (64), the unit shape
|
||||
`Effect+Explosion+Generic+Maneuver+Mass+SE+Shield+StructureCount` (44) and a
|
||||
chatter table (8) — weapon/unit/chatter families the corpus already documents
|
||||
under other names. Characterising them is the next item.
|
||||
|
||||
@@ -102,7 +102,7 @@ files, which is how the same ground got covered twice.
|
||||
| [`structures/mission-scoring.md`](structures/mission-scoring.md) | Mission scoring and the S/A/B/C/D rank thresholds — 24 tables x 3 difficulties | ✅ CONFIRMED, one 22-field schema over 72 records; difficulty moves 10 of 22 fields and never the RankScore thresholds; ✅ all 24 named stage\StageParameter_S<NN>.tbl (+ _Tutorial shared by six, + _Test) |
|
||||
| [`structures/stage-settings-table.md`](structures/stage-settings-table.md) | The per-stage settings object — Camera rigs, Player limits, Difficulty_* damage record | ✅ CONFIRMED, 24 settings objects vs 29 resource objects both using Phase_1/2/3; Camera is 13/14 fields constant; GravityFactor non-zero in 4 stages; IsBoss16Enable identifies one object; the Phase block is 94 fields of which 68 never vary between phases; all 24 objects named StageParameter_S<NN> |
|
||||
| [`structures/isl-mission-timer.md`](structures/isl-mission-timer.md) | The mission timer — two sequential countdowns, not a limit and a warning | ✅ CONFIRMED — read from `sub_822639B8`; the threshold reading is |
|
||||
| [`structures/archive-naming.md`](structures/archive-naming.md) | Which archives the disc can name — 100% for menu paks, **0.0%** for the six 2D paks and `GP_READY_ROOM` | ✅ CONFIRMED — 6027 names × 16 prefixes, bimodal; unnamed entries split by content — three UI paks have ZERO unnamed IDXD (artwork only), DefTables is the only real data gap (1295 LOD/motion tables) |
|
||||
| [`structures/archive-naming.md`](structures/archive-naming.md) | Which archives the disc can name — 100% for menu paks, **0.0%** for the six 2D paks and `GP_READY_ROOM` | ✅ CONFIRMED — 6027 names × 16 prefixes, bimodal; unnamed entries split by content — three UI paks have ZERO unnamed IDXD (artwork only), DefTables is the only real data gap (1295 LOD/motion tables); **route 2 = the `Enumerate` declaration tables** closes DefTables to 1413/1425 IDXD (99.2 %), 12 left; XPR2 manifests refuted as a naming source |
|
||||
| [`structures/hud-config.md`](structures/hud-config.md) | The in-game HUD configuration — 16 records, 419 asset paths, and a per-stage `ResourceTable` | ✅ CONFIRMED — 0 of 419 config paths resolve as pak entries |
|
||||
| [`structures/mission-script-manifest.md`](structures/mission-script-manifest.md) | `Stage\script.tbl`'s 11 non-`MISSION` fields, and the mission dialogue table they name | ✅ CONFIRMED — 1 of 7 values resolves (`message\MissionDialogMessage.tbl`, 200 records), controls in the same sweep |
|
||||
| [`structures/stage-numbering-and-player-craft.md`](structures/stage-numbering-and-player-craft.md) | The disc's stage numbering (S01–S16, S18–S29) and the player's craft per stage | ✅✅ CONFIRMED — two independent files close on 6 + 15 + 1 + 6 = 28 |
|
||||
|
||||
@@ -1752,10 +1752,23 @@
|
||||
GP_TUTORIAL.pak 2 named, 1 families
|
||||
x2 puscr.prt
|
||||
|
||||
## ROUTE 2 the `Enumerate` declaration tables
|
||||
declaration objects found: 144 distinct names they declare: 1298
|
||||
x138 DefTables.pak
|
||||
x1 GP_MAIN_GAME_D.pak
|
||||
x1 GP_MAIN_GAME_E.pak
|
||||
x1 GP_MAIN_GAME_F.pak
|
||||
x1 GP_MAIN_GAME_I.pak
|
||||
x1 GP_MAIN_GAME_J.pak
|
||||
x1 GP_MAIN_GAME_S.pak
|
||||
DefTables.pak +1283 newly named
|
||||
entries named by route 2 that route 1 could not: 1283
|
||||
declared names with NO pak entry: 15 ['GameModel_eff_e0033.tbl', 'GameModel_eff_e0058.tbl', 'GameModel_eff_e0059.tbl', 'GameModel_eff_e0060.tbl', 'GameModel_rou_e106_wep_02.tbl', 'LOD_Effect_eff_e0033.tbl']
|
||||
|
||||
## What the UNNAMED entries are (magic of every entry not resolved above)
|
||||
archive IDXD named unnamedI unnamedUI LSTA
|
||||
GP_HANGAR_ARSENAL.pak 180 180 0 1149 42
|
||||
DefTables.pak 1425 130 1295 0 0
|
||||
DefTables.pak 1425 1413 12 0 0
|
||||
GP_MAIN_GAME_D.pak 1004 667 337 0 0
|
||||
GP_MAIN_GAME_E.pak 1004 667 337 0 0
|
||||
GP_MAIN_GAME_F.pak 1004 667 337 0 0
|
||||
@@ -1779,24 +1792,10 @@
|
||||
GP_MISSION_LOG.pak 0 0 0 0 2
|
||||
|
||||
record-name sets of the UNNAMED IDXD entries, per archive:
|
||||
DefTables.pak 1295 unnamed IDXD in 17 shapes
|
||||
x294 Generic Level_0 Level_1
|
||||
x247 Generic Level_0
|
||||
x212 Default EnumMotions Generic ReferenceFrames
|
||||
x209 Generic Level_0 Level_1 Level_2 Level_3
|
||||
x182 Default EnumMotions Generic Motion_break Motion_dead ReferenceFrames
|
||||
x61 Default EnumMotions Generic Motion_break ReferenceFrames
|
||||
x57 Generic Level_0 Level_1 Level_2
|
||||
x8 Default EnumMotions Generic Motion_break Motion_down ReferenceFrames
|
||||
x8 Default EnumMotions Generic Motion_break Motion_selected ReferenceFrames
|
||||
DefTables.pak 12 unnamed IDXD in 3 shapes
|
||||
x8 Enumerate
|
||||
x3 Default EnumMotions Generic Motion_FlipTurn Motion_ShieldAttack Motion_SideRoll_L Motion_SideRol
|
||||
x1 Damage Damage2 Default EnumMotions Generic ReferenceFrames
|
||||
x1 Default Default2 Default3 EnumMotions Generic ReferenceFrames
|
||||
x1 Default EnumMotions Generic Motion_AfterBurner Motion_break ReferenceFrames
|
||||
x1 Default EnumMotions Generic Motion_dead ReferenceFrames
|
||||
x1 Default EnumMotions Generic Motion_down ReferenceFrames
|
||||
x1 EnumMotions Generic Motion_attackA_charge Motion_attackA_charge_b1 Motion_attackA_charge_b2 Moti
|
||||
x2 Default EnumMotions Generic Motion_break Motion_dead ReferenceFrames
|
||||
x2 Generic Level_0
|
||||
GP_MAIN_GAME_D.pak 337 unnamed IDXD in 53 shapes
|
||||
x90 Shell Weapon
|
||||
x64 Faces Generic
|
||||
|
||||
@@ -67,6 +67,66 @@ redo the work.
|
||||
is now iterated **sorted** — a plain `set` gave a different winner for collided
|
||||
hashes on each run and the artefact failed its own byte-identical check.
|
||||
|
||||
## ✅✅ ROUTE 2 — the `Enumerate` declaration tables close `DefTables` (2026-08-27)
|
||||
|
||||
The string harvest could never name `DefTables`, because **those names are not
|
||||
spelled out as `something.tbl` anywhere on the disc**. They are declared instead.
|
||||
|
||||
🔑 **An IDXD object whose single record is named `Enumerate` is a DECLARATION
|
||||
TABLE: its FIELD NAMES are the names of other objects, and each one resolves as
|
||||
`name_hash("<field name>.tbl")`.** `EnumLODSet_test.tbl` declares 676 names,
|
||||
`EnumGameModel_test.tbl` 360 — and the disc holds **144** such objects (138 in
|
||||
`DefTables`, one in each of the six `GP_MAIN_GAME_*`), declaring **1 298**
|
||||
distinct names between them.
|
||||
|
||||
| | |
|
||||
|---|---:|
|
||||
| entries `DefTables` named by the string harvest (route 1) | 130 |
|
||||
| **entries named by the declaration tables (route 2)** | **+1 283** |
|
||||
| still unnamed | **12** |
|
||||
| **`DefTables` IDXD coverage** | **1 413 / 1 425 = 99.2 %** |
|
||||
|
||||
`130 + 1283 + 12 = 1425` — the arithmetic closes, with **no overlap** between
|
||||
the two routes.
|
||||
|
||||
✅ **Zero partials, and the suffix is exact.** Every one of the 1 036
|
||||
`_test`-declared names resolves, and it resolves **only** as `<name>.tbl` with
|
||||
**no** path prefix — of the 5 suffixes × 6 prefixes tried, `('', '.tbl')` scored
|
||||
1 036 and every other combination scored 0.
|
||||
|
||||
**What the residual is**, in full: 8 declaration tables that nothing declares (the
|
||||
other 130 are named by route 1, from the stage tables that reference them), plus
|
||||
**4** ordinary tables — two `Generic+Level_0` LOD sets whose `Model` fields read
|
||||
`rou_e004` and `rou_e013`, and two motion sets. **15 declared names have no pak
|
||||
entry** (`GameModel_eff_e0033`, `…e0058`–`e0060`, `rou_e106_wep_02`, and their
|
||||
`LOD_Effect_*` twins) — declarations for content that did not ship.
|
||||
|
||||
### 🔴 The XPR2 manifests are NOT the naming source — refuted
|
||||
|
||||
The obvious guess was the 40 XPR2 manifests, which name meshes
|
||||
(`<XBGMesh Name = "rou_e104_brg_01_d" DataFile = "machines\rou_e104\objects\…xbg"/>`).
|
||||
Their **82** `Name=` values and 82 `DataFile`/`Source` paths, hashed bare and
|
||||
under `.tbl` / `.xbg` / `.xpr` / `.prt` / `game:\`, resolve **0** `DefTables`
|
||||
entries.
|
||||
|
||||
They are not unrelated, though: **40 of the 82 manifest names appear as the
|
||||
`Model` field VALUE inside the LOD/motion tables** (1 272 distinct `Model`
|
||||
values in `DefTables`). Manifest and table share the *model* namespace; the pak
|
||||
key is a different namespace, and only the declaration tables bridge it.
|
||||
|
||||
⚠️ First attempt used `Name="` and found **0 values** — the manifests write
|
||||
`Name = "…"` with spaces. A regex miss looks exactly like a null result.
|
||||
|
||||
### ⚠️ Bonus, not this item: the `GP_MAIN_GAME_*` residual is already-owned data
|
||||
|
||||
With `DefTables` closed, the largest remaining unnamed block is the identical
|
||||
**337 IDXD in each of the six `GP_MAIN_GAME_*`**, in 53 record-name shapes whose
|
||||
top entries are `Shell+Weapon` (90), `Faces+Generic` (64),
|
||||
`Effect+Explosion+Generic+Maneuver+Mass+SE+Shield+StructureCount` (44) and a
|
||||
chatter table (`ATTACK_IS_AVOIDED …`, 8) — i.e. **weapon, unit and chatter
|
||||
tables**, all families the corpus already documents. Characterising them
|
||||
properly is still open.
|
||||
|
||||
## ✅ WHY each archive is low-coverage — two different reasons (2026-08-27)
|
||||
|
||||
Splitting every **unnamed** entry by its magic turns the percentage table into an
|
||||
|
||||
@@ -19,6 +19,8 @@ PRE = ['', '2d\\', 'ui\\', 'hud\\', 'Stage\\', 'stage\\', 'message\\', 'language
|
||||
def main():
|
||||
paks = sorted(glob.glob('/work/sylph_extract/**/*.pak', recursive=True))
|
||||
names = set()
|
||||
decl = set()
|
||||
nenum = collections.Counter()
|
||||
toc = {}
|
||||
for pk in paks:
|
||||
ents = pak_entries(pk)
|
||||
@@ -26,6 +28,18 @@ def main():
|
||||
for h, b in ents:
|
||||
for m in re.finditer((r'[A-Za-z0-9_\\.\-]{4,60}\.' + EXT).encode(), b):
|
||||
names.add(m.group(0).decode('latin-1'))
|
||||
# ROUTE 2 seed: an object whose single record is named `Enumerate`
|
||||
# is a DECLARATION TABLE — its field names are other objects' names.
|
||||
if b[:4] == b'IDXD' and b'Enumerate' in b:
|
||||
try:
|
||||
recs = U.parse(b)
|
||||
except Exception:
|
||||
continue
|
||||
if len(recs) == 1 and recs[0]['squadron'] == 'Enumerate':
|
||||
nenum[os.path.basename(pk)] += 1
|
||||
for _t, fn, _v in recs[0]['fields']:
|
||||
if fn:
|
||||
decl.add(fn)
|
||||
|
||||
H = set()
|
||||
for n in names:
|
||||
@@ -62,6 +76,29 @@ def main():
|
||||
for k, c in sorted(fam.items(), key=lambda kv: (-kv[1], kv[0])):
|
||||
print(" x%-4d %s" % (c, k))
|
||||
|
||||
# ROUTE 2 -- the declaration tables. An `Enumerate` object lists, as FIELD
|
||||
# NAMES, the names of other objects; each resolves as `<name>.tbl`. This is
|
||||
# the naming source the string harvest cannot see, because these names are
|
||||
# never spelled out as `something.tbl` anywhere on the disc.
|
||||
DECL = {}
|
||||
for n in sorted(decl):
|
||||
DECL.setdefault(U.name_hash(n + '.tbl'), n + '.tbl')
|
||||
print("\n## ROUTE 2 the `Enumerate` declaration tables")
|
||||
print(" declaration objects found: %d distinct names they declare: %d"
|
||||
% (sum(nenum.values()), len(decl)))
|
||||
for nm, c in sorted(nenum.items(), key=lambda kv: (-kv[1], kv[0])):
|
||||
print(" x%-5d %s" % (c, nm))
|
||||
gained = 0
|
||||
for nm, ent in rows:
|
||||
g = sum(1 for h in ent if h in DECL and h not in NAMES)
|
||||
if g:
|
||||
print(" %-30s +%d newly named" % (nm, g))
|
||||
gained += g
|
||||
print(" entries named by route 2 that route 1 could not: %d" % gained)
|
||||
miss = [v for k, v in sorted(DECL.items()) if not any(k in ent for ent in toc.values())]
|
||||
print(" declared names with NO pak entry: %d %s" % (len(miss), sorted(miss)[:6]))
|
||||
NAMES.update({k: v for k, v in DECL.items() if k not in NAMES})
|
||||
|
||||
# WHY an archive is low-coverage: split its UNNAMED entries by content.
|
||||
print("\n## What the UNNAMED entries are (magic of every entry not resolved above)")
|
||||
print(" %-28s %6s %6s %8s %9s %6s" % (
|
||||
|
||||
Reference in New Issue
Block a user