b273ee9f48
The refutation used the wrong hash function. A record key is tag_hash of the
record's own name, not the pak-TOC name_hash which lowercases and uses a
different modulus:
tag_hash("Formation_4_Bird") = 0x22a5eeed <- exactly the record key
name_hash("Formation_4_Bird") = 0x6286edad <- the wrong hash, hence
the recorded "0 of 16"
Formation_4_Bird IS a record name in FormationSet_S02.tbl and every record in
that table keys by tag_hash of its name. Across all 28 UnitGroup_S<NN>.tbl the
same holds for field keys: 6960 of 6960 are tag_hash(field name), 0 are
name_hash. So FormationID resolves by a direct hash lookup and the roster
indirection is NOT required for it.
This is exactly the trap idxd-container.md warns about, and it cost this file
a spurious refutation. The original text is kept below the correction.
The squadron half of the claim still stands and is marked so: a squadron
record's key is not tag_hash of the squadron id either -- tag_hash("TCN001")
is 0x760cdeaa against keys running 0x659aff47 -- because the record's NAME is
not the squadron id. That roster indirection is real.
Also closes the file's standing "the key derivation stays open, a second hash
function is unidentified": that function is tag_hash, recovered since.
Found while checking a subagent's claim rather than taking it -- it reported
the hash correction, and verifying it turned up that the FormationID half was
a stronger result than reported.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
213 lines
10 KiB
Markdown
213 lines
10 KiB
Markdown
# `stage\UnitGroup_S<NN>.tbl` — the per-stage squadron roster
|
||
|
||
Status: ✅ container format and field semantics, validated across all 28 stage
|
||
tables present on the disc; 🟡 one member field; ❔ the record key and the
|
||
arrival-interval *values*.
|
||
|
||
Reached from the per-stage definition record's `EnumerateSquadron` field — see
|
||
[stage-definition-table.md](stage-definition-table.md). Tool:
|
||
`tools/re-capture/unitgroup.py` (pure static; runs no emulator).
|
||
|
||
```
|
||
python3 tools/re-capture/unitgroup.py S02 # full roster
|
||
python3 tools/re-capture/unitgroup.py --all --check # self-check every stage
|
||
```
|
||
|
||
A committed dump of Stage 02 is at [`../data/unitgroup-s02.txt`](../data/unitgroup-s02.txt).
|
||
|
||
## ✅ Container
|
||
|
||
```
|
||
0x00 "IDXD"
|
||
0x04 u32 nrec
|
||
0x08 nrec x 16 record: (key, squadron_off, field_lo, field_hi)
|
||
u32 npool
|
||
npool x 12 property entry: (tag, name_off | 0xffffffff, value_off)
|
||
u32 strsize -- and STR + strsize == filesize, exactly
|
||
strsize string pool; [0] = "Enumerate_Squadrons", [0x14] = ""
|
||
```
|
||
|
||
**Every offset in the file is relative to `STR`**, the string-pool base. A
|
||
record's fields are pool entries `[field_lo, field_hi)`. An entry with
|
||
`name_off == 0xffffffff` is *positional*; otherwise the entry carries its own
|
||
field name inline, so the table is self-describing and the tag hash never has to
|
||
be inverted.
|
||
|
||
Exactly one record per file is not a squadron: it is the **`Enumerate_Squadrons`
|
||
roster**, whose entries map `key -> squadron id` for every other record. It is
|
||
usually the last record but not always — in S05, S06, S07, S08, S09, S11, S13,
|
||
S28 and S29 it sits elsewhere, so find it by its missing `Count`, not by
|
||
position.
|
||
|
||
## ✅ A squadron record
|
||
|
||
Five named fields, always present and always these five:
|
||
|
||
| field | values |
|
||
|---|---|
|
||
| `Count` | 1 (×1082), 2 (×50), 3 (×8), 4 (×20) |
|
||
| `SideID` | `ADAN` (745), `TCAF` (294), `Neutral` (121) |
|
||
| `AIID` | 31 distinct, e.g. `AI_ADAN_CraftSquadron_Veteran`, `AI_TCAF_BirdFlight`, `AI_ADAN_Fleet`, `AI_Structure`, `AI_TraceRoute` |
|
||
| `FormationID` | e.g. `Formation_4_Bird`, `Formation_ADAN_Turret07_30`, `Formation_1_only` |
|
||
| `DisableInterval` | `No` (1129), **`Yes` (31)** |
|
||
|
||
…preceded by `Count` **member tuples** of four positional entries:
|
||
|
||
```
|
||
(unit model, message set, n, identity)
|
||
```
|
||
|
||
* unit model — 122 distinct, and they are our XBG7 mesh names:
|
||
`UN_f001_TCAF_DeltaSaber_T_Player`, `UN_e010_ADAN_Attacker_S`,
|
||
`UN_e007_ADAN_Turret`, `UN_bf001_TCAF_SchlosBase`.
|
||
* message set — the squadron's radio chatter, e.g. `MessageSet_Ellen`,
|
||
`MessageSet_ADAN_plA`.
|
||
* `n` — ✅ **the number of units this tuple instantiates**, filling slots of the
|
||
squadron's formation. Integer 1…30, dominated by 1 (735) and 9 (197). See
|
||
*"What `n` counts"* below — the old note that it is "not the `_NN` suffix of
|
||
`FormationID`" was right but drew the wrong conclusion.
|
||
* identity — 63 distinct: named pilots (`ELLEN`, `SANDRA`, `RAYMOND`, `YOJI`),
|
||
ship nameplates (`NP_Charon`, `NP_Amalthea`, `NP_Olympus`), carrier tags
|
||
(`Carrier_ADAN01`), cargo tags (`CARGO_1`), or empty.
|
||
|
||
Worked example — Stage 02, squadron `TCN004`, the player's own flight:
|
||
|
||
```
|
||
TCN004 TCAF Count=2 DisableInterval=No Formation_2_Rhino1 AI_TCAF_RhinoFlight
|
||
unit=UN_f001_TCAF_DeltaSaber_T_Player msg=MessageSet_Katana n=1 id=Character_Player_Test
|
||
unit=UN_f001_TCAF_DeltaSaber_T msg=MessageSet_Ellen n=1 id=ELLEN
|
||
```
|
||
|
||
## ✅ Two independent self-checks, both corpus-wide
|
||
|
||
1. **`len(fields) == Count * 4 + 5`** — holds for **1160 of 1160** squadrons
|
||
across all 28 stage tables. This is what pins the member-tuple width at 4 and
|
||
the named-field count at 5; it was not assumed.
|
||
2. **Roster agreement** — the `Enumerate_Squadrons` record's `key -> id` mapping
|
||
agrees with the name each record resolves independently through its own
|
||
`squadron_off`, for **1160 of 1160**.
|
||
|
||
S17 has no `stage\UnitGroup_S17.tbl` in `GP_MAIN_GAME_E.pak`, matching the
|
||
missing S17 stage record.
|
||
|
||
## ❌ Refuted along the way
|
||
|
||
* **The record key is not the squadron id's name hash.**
|
||
`name_hash("TCN001") = 0xd639f1a4`, but the keys run
|
||
`0x659aff47, 0x659b0046, 0x669b0047, …` — 0 of 112 match. What the key encodes
|
||
is still ❔. Its byte structure (`b0` ramping, `b2` taking small signed values)
|
||
looks like a packed tuple, untested.
|
||
* **Squadron ids do not use a separate string base.** An earlier reading here
|
||
put them at `STR + 0x15` and scored "109 of 111", which looked like a near-fit
|
||
but was an artefact of the uniform 7-byte id stride — it silently shifted
|
||
every name by three entries (record 0 read as `ADN104` instead of `ADN101`,
|
||
and 17 `TC*`-named squadrons came out with `SideID = ADAN`). The roster record
|
||
refuted it outright, and with the correct base — plain `STR` — agreement is
|
||
111/111. The lesson is that a 98 %-looking score on a self-consistent stride
|
||
is not evidence; the roster was an independent oracle and should have been
|
||
consulted first.
|
||
* **The 20-byte "section header"** described in an earlier draft of
|
||
[stage-definition-table.md](stage-definition-table.md) does not exist. What
|
||
looked like `(tag, 0, 0, count, size)` was the file's last 16-byte record
|
||
followed by the `npool` word. The corrected layout above is uniform across all
|
||
28 files, which the earlier reading was not — it failed on 9 of them.
|
||
|
||
## What is still open
|
||
|
||
* The meaning of the 4-byte record key.
|
||
* The member tuple's third field `n`.
|
||
* **The interval values.** `DisableInterval` is a per-squadron *flag* and 31
|
||
squadrons set it, but the interval durations, spawn triggers and arrival
|
||
positions are not in this file. The next places to look are
|
||
`Formation_*.tbl` (the formation geometry and possibly its timing) and
|
||
`EnumSquadron_Test.tbl`, both named by the stage record.
|
||
|
||
|
||
## ✅ What `n` counts — bounded by the formation, 1159 of 1160
|
||
|
||
`n` is the **number of units the member tuple instantiates**. The evidence is a
|
||
hard, falsifiable inequality against a table `UnitGroup` never mentions.
|
||
|
||
`FormationSet_S<NN>.tbl` is the same IDXD container, and a formation record is a
|
||
**slot list**: one named field `FrameCount` plus 8 positional fields per slot —
|
||
`FrameCount = 4 → 33` fields, `14 → 113`, `30 → 241`, `32 → 257`, i.e.
|
||
`1 + 8·FrameCount`, exactly.
|
||
|
||
Resolving each squadron's `FormationID` to its formation (below) and comparing:
|
||
|
||
```
|
||
sum(n) <= FrameCount: HOLDS 1159 VIOLATED 1 unresolved 0
|
||
of the holders, sum(n) == FrameCount exactly: 539
|
||
```
|
||
|
||
Across all 28 stages, **every** `FormationID` resolves and **every** squadron
|
||
but one fits its formation, with 46 % filling it exactly. A field unrelated to
|
||
formation size would not do that.
|
||
|
||
**The one violation is a debug leftover**, recorded rather than swept up: S20
|
||
(a tutorial stage), `Formation_1_only` (`FrameCount = 1`) with `n = 2` —
|
||
`AIID = AI_Test`, `msg = MessageSet_test`, no identity, unit
|
||
`UN_e015_ADAN_Puppy_2`. It is still a literal violation of the invariant.
|
||
|
||
### ✅ The formation name's trailing number IS `FrameCount`
|
||
|
||
`Formation_ADAN_Turret07_30` → `FrameCount = 30`; `Formation_TCAF_ArrowHead02_32`
|
||
→ 32; `Formation_4_Bird` → 4 (leading, this family has no suffix). So the old
|
||
observation that `n = 9` does not match the `_30` in
|
||
`Formation_ADAN_Turret07_30` is correct and **not** evidence against `n` being a
|
||
count: 9 units occupy 9 of that formation's 30 slots.
|
||
|
||
### ❌ WITHDRAWN — "`FormationID` does not hash into the table"
|
||
|
||
**It does.** The refutation below used the wrong hash function. A record key is
|
||
[`tag_hash`](idxd-tag-hash.md) of the record's own name — *not* the pak-TOC
|
||
`name_hash`, which lowercases and uses a different modulus. Checked directly:
|
||
|
||
```
|
||
tag_hash("Formation_4_Bird") = 0x22a5eeed <- exactly the record key
|
||
name_hash("Formation_4_Bird") = 0x6286edad <- the wrong hash, hence "0 of 16"
|
||
```
|
||
|
||
`Formation_4_Bird` **is** a record name in `FormationSet_S02.tbl`, and every
|
||
record in that table keys by `tag_hash` of its name. Across all 28
|
||
`UnitGroup_S<NN>.tbl` the same holds for field keys: **6960 / 6960** are
|
||
`tag_hash(field name)` and **0** are `name_hash`. So `FormationID` resolves by a
|
||
direct hash lookup and the roster indirection is **not required** for it.
|
||
|
||
This is the exact trap [`idxd-container.md`](idxd-container.md) warns about, and
|
||
it cost this file a spurious ❌.
|
||
|
||
🟡 The *squadron* half of the original claim still stands: a squadron record's
|
||
key is not `tag_hash` of the squadron id either (`tag_hash("TCN001")` =
|
||
`0x760cdeaa`, against keys running `0x659aff47…`), because the record's **name**
|
||
is not the squadron id. The roster maps id → record, and that indirection is real.
|
||
|
||
### The original text, kept
|
||
|
||
`FormationID` does **not** hash into the table: `name_hash("Formation_4_Bird")`
|
||
= `0x6286edad`, and the record key is `0x22a5eeed` — 0 of 16 resolve that way.
|
||
`FormationSet_S02.tbl` has **17 records for 16 formations**, and the extra one is
|
||
a **name roster**, exactly like `Enumerate_Squadrons` here: it carries no
|
||
`FrameCount` and its fields are `(tag, name, "")` triples whose **tags are the
|
||
record keys**. Find it by its missing `FrameCount`, map name → key, done.
|
||
|
||
That is the same convention twice in two different tables, which is worth
|
||
remembering for the next one: **a table's record keys are resolved by a roster
|
||
record inside the table, not by hashing the name.**
|
||
|
||
### 🟡 What this does not settle
|
||
|
||
* It shows `sum(n)` *fits* the formation, not that each unit is separately
|
||
instantiated. "Slots consumed" and "units spawned" are indistinguishable here.
|
||
* It does **not** close the 387-vs-~300 gap in
|
||
[roster-to-craft-link](../roster-to-craft-link.md). Σ`n` over all of Stage 02
|
||
is 387 against 296–300 live craft. That count was measured mid-mission after
|
||
kills, and squadrons deploy across phases, so the two are not comparable as
|
||
they stand — but the earlier note rejected `n = 387` as *the* craft count on
|
||
the assumption that everything deploys at once, and that assumption is still
|
||
untested.
|
||
* ✅ The **key derivation is no longer ❔**: the "second hash function" is
|
||
[`tag_hash`](idxd-tag-hash.md), recovered since this was written. Record keys
|
||
and field keys are both `tag_hash` of their own name — 6960 / 6960 field keys
|
||
across all 28 `UnitGroup` tables, 0 matching `name_hash`.
|