Syplheed-Reborn/docs/re/structures/unit-group-table.md

# `stage\UnitGroup_S<NN>.tbl` — the per-stage squadron roster

Status: ✅ container format and field semantics, validated across all 28 stage
tables present on the disc; 🟡 one member field; ❔ the record key and the
arrival-interval *values*.

Reached from the per-stage definition record's `EnumerateSquadron` field — see
[stage-definition-table.md](stage-definition-table.md). Tool:
`tools/re-capture/unitgroup.py` (pure static; runs no emulator).

```
python3 tools/re-capture/unitgroup.py S02          # full roster
python3 tools/re-capture/unitgroup.py --all --check # self-check every stage
```

A committed dump of Stage 02 is at [`../data/unitgroup-s02.txt`](../data/unitgroup-s02.txt).

## ✅ Container

```
0x00              "IDXD"
0x04    u32       nrec
0x08    nrec x 16 record: (key, squadron_off, field_lo, field_hi)
        u32       npool
        npool x 12 property entry: (tag, name_off | 0xffffffff, value_off)
        u32       strsize          -- and STR + strsize == filesize, exactly
        strsize   string pool; [0] = "Enumerate_Squadrons", [0x14] = ""
```

**Every offset in the file is relative to `STR`**, the string-pool base. A
record's fields are pool entries `[field_lo, field_hi)`. An entry with
`name_off == 0xffffffff` is *positional*; otherwise the entry carries its own
field name inline, so the table is self-describing and the tag hash never has to
be inverted.

Exactly one record per file is not a squadron: it is the **`Enumerate_Squadrons`
roster**, whose entries map `key -> squadron id` for every other record. It is
usually the last record but not always — in S05, S06, S07, S08, S09, S11, S13,
S28 and S29 it sits elsewhere, so find it by its missing `Count`, not by
position.

## ✅ A squadron record

Five named fields, always present and always these five:

| field | values |
|---|---|
| `Count` | 1 (×1082), 2 (×50), 3 (×8), 4 (×20) |
| `SideID` | `ADAN` (745), `TCAF` (294), `Neutral` (121) |
| `AIID` | 31 distinct, e.g. `AI_ADAN_CraftSquadron_Veteran`, `AI_TCAF_BirdFlight`, `AI_ADAN_Fleet`, `AI_Structure`, `AI_TraceRoute` |
| `FormationID` | e.g. `Formation_4_Bird`, `Formation_ADAN_Turret07_30`, `Formation_1_only` |
| `DisableInterval` | `No` (1129), **`Yes` (31)** |

…preceded by `Count` **member tuples** of four positional entries:

```
(unit model, message set, n, identity)
```

* unit model — 122 distinct, and they are our XBG7 mesh names:
  `UN_f001_TCAF_DeltaSaber_T_Player`, `UN_e010_ADAN_Attacker_S`,
  `UN_e007_ADAN_Turret`, `UN_bf001_TCAF_SchlosBase`.
* message set — the squadron's radio chatter, e.g. `MessageSet_Ellen`,
  `MessageSet_ADAN_plA`.
* `n` — ✅ **the number of units this tuple instantiates**, filling slots of the
  squadron's formation. Integer 1…30, dominated by 1 (735) and 9 (197). See
  *"What `n` counts"* below — the old note that it is "not the `_NN` suffix of
  `FormationID`" was right but drew the wrong conclusion.
* identity — 63 distinct: named pilots (`ELLEN`, `SANDRA`, `RAYMOND`, `YOJI`),
  ship nameplates (`NP_Charon`, `NP_Amalthea`, `NP_Olympus`), carrier tags
  (`Carrier_ADAN01`), cargo tags (`CARGO_1`), or empty.

Worked example — Stage 02, squadron `TCN004`, the player's own flight:

```
TCN004  TCAF  Count=2  DisableInterval=No  Formation_2_Rhino1  AI_TCAF_RhinoFlight
    unit=UN_f001_TCAF_DeltaSaber_T_Player  msg=MessageSet_Katana  n=1  id=Character_Player_Test
    unit=UN_f001_TCAF_DeltaSaber_T         msg=MessageSet_Ellen   n=1  id=ELLEN
```

## ✅ Two independent self-checks, both corpus-wide

1. **`len(fields) == Count * 4 + 5`** — holds for **1160 of 1160** squadrons
   across all 28 stage tables. This is what pins the member-tuple width at 4 and
   the named-field count at 5; it was not assumed.
2. **Roster agreement** — the `Enumerate_Squadrons` record's `key -> id` mapping
   agrees with the name each record resolves independently through its own
   `squadron_off`, for **1160 of 1160**.

S17 has no `stage\UnitGroup_S17.tbl` in `GP_MAIN_GAME_E.pak`, matching the
missing S17 stage record.

## ❌ Refuted along the way

* **The record key is not the squadron id's name hash.**
  `name_hash("TCN001") = 0xd639f1a4`, but the keys run
  `0x659aff47, 0x659b0046, 0x669b0047, …` — 0 of 112 match. What the key encodes
  is still ❔. Its byte structure (`b0` ramping, `b2` taking small signed values)
  looks like a packed tuple, untested.
* **Squadron ids do not use a separate string base.** An earlier reading here
  put them at `STR + 0x15` and scored "109 of 111", which looked like a near-fit
  but was an artefact of the uniform 7-byte id stride — it silently shifted
  every name by three entries (record 0 read as `ADN104` instead of `ADN101`,
  and 17 `TC*`-named squadrons came out with `SideID = ADAN`). The roster record
  refuted it outright, and with the correct base — plain `STR` — agreement is
  111/111. The lesson is that a 98 %-looking score on a self-consistent stride
  is not evidence; the roster was an independent oracle and should have been
  consulted first.
* **The 20-byte "section header"** described in an earlier draft of
  [stage-definition-table.md](stage-definition-table.md) does not exist. What
  looked like `(tag, 0, 0, count, size)` was the file's last 16-byte record
  followed by the `npool` word. The corrected layout above is uniform across all
  28 files, which the earlier reading was not — it failed on 9 of them.

## What is still open

* The meaning of the 4-byte record key.
* The member tuple's third field `n`.
* **The interval values.** `DisableInterval` is a per-squadron *flag* and 31
  squadrons set it, but the interval durations, spawn triggers and arrival
  positions are not in this file. The next places to look are
  `Formation_*.tbl` (the formation geometry and possibly its timing) and
  `EnumSquadron_Test.tbl`, both named by the stage record.


## ✅ What `n` counts — bounded by the formation, 1159 of 1160

`n` is the **number of units the member tuple instantiates**. The evidence is a
hard, falsifiable inequality against a table `UnitGroup` never mentions.

`FormationSet_S<NN>.tbl` is the same IDXD container, and a formation record is a
**slot list**: one named field `FrameCount` plus 8 positional fields per slot —
`FrameCount = 4 → 33` fields, `14 → 113`, `30 → 241`, `32 → 257`, i.e.
`1 + 8·FrameCount`, exactly.

Resolving each squadron's `FormationID` to its formation (below) and comparing:

```
sum(n) <= FrameCount:   HOLDS 1159    VIOLATED 1    unresolved 0
   of the holders, sum(n) == FrameCount exactly:  539
```

Across all 28 stages, **every** `FormationID` resolves and **every** squadron
but one fits its formation, with 46 % filling it exactly. A field unrelated to
formation size would not do that.

**The one violation is a debug leftover**, recorded rather than swept up: S20
(a tutorial stage), `Formation_1_only` (`FrameCount = 1`) with `n = 2` —
`AIID = AI_Test`, `msg = MessageSet_test`, no identity, unit
`UN_e015_ADAN_Puppy_2`. It is still a literal violation of the invariant.

### ✅ The formation name's trailing number IS `FrameCount`

`Formation_ADAN_Turret07_30` → `FrameCount = 30`; `Formation_TCAF_ArrowHead02_32`
→ 32; `Formation_4_Bird` → 4 (leading, this family has no suffix). So the old
observation that `n = 9` does not match the `_30` in
`Formation_ADAN_Turret07_30` is correct and **not** evidence against `n` being a
count: 9 units occupy 9 of that formation's 30 slots.

### ❌ WITHDRAWN — "`FormationID` does not hash into the table"

**It does.** The refutation below used the wrong hash function. A record key is
[`tag_hash`](idxd-tag-hash.md) of the record's own name — *not* the pak-TOC
`name_hash`, which lowercases and uses a different modulus. Checked directly:

```
tag_hash("Formation_4_Bird") = 0x22a5eeed   <- exactly the record key
name_hash("Formation_4_Bird") = 0x6286edad  <- the wrong hash, hence "0 of 16"
```

`Formation_4_Bird` **is** a record name in `FormationSet_S02.tbl`, and every
record in that table keys by `tag_hash` of its name. Across all 28
`UnitGroup_S<NN>.tbl` the same holds for field keys: **6960 / 6960** are
`tag_hash(field name)` and **0** are `name_hash`. So `FormationID` resolves by a
direct hash lookup and the roster indirection is **not required** for it.

This is the exact trap [`idxd-container.md`](idxd-container.md) warns about, and
it cost this file a spurious ❌.

🟡 The *squadron* half of the original claim still stands: a squadron record's
key is not `tag_hash` of the squadron id either (`tag_hash("TCN001")` =
`0x760cdeaa`, against keys running `0x659aff47…`), because the record's **name**
is not the squadron id. The roster maps id → record, and that indirection is real.

### The original text, kept

`FormationID` does **not** hash into the table: `name_hash("Formation_4_Bird")`
= `0x6286edad`, and the record key is `0x22a5eeed` — 0 of 16 resolve that way.
`FormationSet_S02.tbl` has **17 records for 16 formations**, and the extra one is
a **name roster**, exactly like `Enumerate_Squadrons` here: it carries no
`FrameCount` and its fields are `(tag, name, "")` triples whose **tags are the
record keys**. Find it by its missing `FrameCount`, map name → key, done.

That is the same convention twice in two different tables, which is worth
remembering for the next one: **a table's record keys are resolved by a roster
record inside the table, not by hashing the name.**

### 🟡 What this does not settle

* It shows `sum(n)` *fits* the formation, not that each unit is separately
  instantiated. "Slots consumed" and "units spawned" are indistinguishable here.
* It does **not** close the 387-vs-~300 gap in
  [roster-to-craft-link](../roster-to-craft-link.md). Σ`n` over all of Stage 02
  is 387 against 296–300 live craft. That count was measured mid-mission after
  kills, and squadrons deploy across phases, so the two are not comparable as
  they stand — but the earlier note rejected `n = 387` as *the* craft count on
  the assumption that everything deploys at once, and that assumption is still
  untested.
* ✅ The **key derivation is no longer ❔**: the "second hash function" is
  [`tag_hash`](idxd-tag-hash.md), recovered since this was written. Record keys
  and field keys are both `tag_hash` of their own name — 6960 / 6960 field keys
  across all 28 `UnitGroup` tables, 0 matching `name_hash`.

## ✅❌ Declared × deployed — the weapon law does **not** generalise, and the whole exception is S14

[weapon-struct-runtime](weapon-struct-runtime.md) established that nothing is
*mounted* without being *declared* — 0 of 131, an empty cell. Running the same
partition over units refutes the general form of that law.

Three vocabularies had to be separated first, because they are not the same set:

| set | source | size |
|---|---|---:|
| **declared** | field names of the **31** `EnumUnit` records | 118 |
| **deployed** | slot 0 of each `UnitGroup` member tuple | 122 |
| datasheet | `Generic` records carrying `HP` | **114** |

The datasheet count re-derives [unit-datasheet-static](unit-datasheet-static.md)'s
own **114** from a different direction, as a control. `declared` and `deployed`
share one vocabulary (`UN_*`), overlapping on 109; the datasheet's `ID`s are a
third naming, disjoint from both.

| | deployed | not deployed |
|---|---:|---:|
| **declared** | 109 | 9 |
| **not declared** | **13** | 1 |

❌ **The cell that was empty for weapons holds 13 for units.** Units *can* be
deployed without any table declaring them.

✅ **But the 13 are one coherent family, not strays** — every one is
`UN_S14_p1_Asteroid_cmesh_*` or `UN_S14_p2_Asteroid_cmesh_*`. Asking the other
population settles it:

| stage | asteroid meshes deployed | of those, declared |
|---|---:|---:|
| S01 | 8 | **8** |
| S04 | 10 | **10** |
| S05 | 5 | **5** |
| S08 | 6 | **6** |
| S13 | 4 | **4** |
| **S14 p1** | 9 | **0** |
| **S14 p2** | 4 | **0** |

**33 of 33 declared everywhere else; 0 of 13 in S14.** And
**non-asteroid deployed-but-undeclared = 0** — the entire violation is this one
stage's asteroid field. So the law survives in a narrower form: *every unit that
is deployed is declared, except S14's asteroid collision meshes.*

🟡 **Why S14 differs is not settled here.** It is suggestive that S14 is the
stage whose asteroid table S28 borrows (see
[asteroid-fields](asteroid-fields.md)), but nothing measured connects the borrow
to the missing declarations — **a coincidence worth recording, not a cause.**

The remaining cells are small and benign: 9 declared-but-never-deployed units,
and one datasheet id, `Ship_`, that is neither.

### ✅ Sharper, per stage: `EnumUnit_S<NN>` **is** the deployed set — 20 of 22 exactly

The partition above was global, which understated the result. Joining each
stage's own `EnumerateUnit` table to its own `EnumerateSquadron` table gives a
much tighter statement:

| | stages |
|---|---|
| `declared == deployed`, **exactly** | **20** |
| declares 4 it never deploys | S16 |
| deploys 13 it never declares | S14 |

Per stage the counts are equal and the undeclared count is 0 — S01 13/13,
S02 14/14, S03 15/15, S04 23/23, S05 13/13, S06 19/19, S07 17/17, S08 23/23,
S09 21/21, S10 3/3, S11 10/10, S12 15/15, S13 18/18, S15 18/18, S24 11/11,
S25 15/15, S26 15/15, S27 16/16, S28 9/9, S29 18/18. **Every per-stage
deployed-but-undeclared count is 0 except S14's, and the 13 there are the whole
disc-wide total** — so the global "13" is one stage's, not a scattering.

🔑 **S16 explains what the table actually is.** It declares **6** and deploys
**2**. Its six are `UN_e009_ADAN_Phantom`, `UN_f002_TCAF_DeltaSaber_W`,
`…_W_Player`, `Test_ADAN_PrometheusDriver_InsideP2_01`, **`UN_e901_ADAN_Boss`**
and **`UN_e910_core_ADAN_GeneratorCore`** — and the last two are exactly the
units [stage16-boss](stage16-boss.md) shows are brought in by their own loader
chain (`sub_82315AE8`), not by the squadron roster. So `EnumUnit_S<NN>` is the
stage's **load manifest**, not a mirror of the roster: a unit that code spawns is
declared but never deployed. That also names 2 of the 9 disc-wide
declared-but-never-deployed units; 6 more are `Ship_Test*` placeholders and the
ninth is that Prometheus-driver test entry, so **all 9 are accounted for**.

🟡 **S14 remains the one unexplained cell**, and it is not a missing table:
`EnumUnit_S14.tbl` exists and declares six names —
`UN_e004_ADAN_ElanPlus_N`, `UN_e006_ADAN_Vindicator_Margras`,
`UN_f002_TCAF_DeltaSaber_W`, `…_W_Player`, `UN_f201_TCAF_Tanker`,
**`UN_mn040_Asteroid_Big`**. Note the last: S14 *does* declare an asteroid, just
not the 13 `Asteroid_cmesh` entities it deploys. So "asteroids are exempt from
the manifest" is **refuted for S14 itself**; whatever the cause, it is specific
to the collision-mesh entities.