Sylpheed/docs/re/structures/unit-group-table.md

stage\UnitGroup_S<NN>.tbl — the per-stage squadron roster

Status: ✅ container format and field semantics, validated across all 28 stage tables present on the disc; 🟡 one member field; ❔ the record key and the arrival-interval values.

Reached from the per-stage definition record's EnumerateSquadron field — see stage-definition-table.md. Tool: tools/re-capture/unitgroup.py (pure static; runs no emulator).

python3 tools/re-capture/unitgroup.py S02          # full roster
python3 tools/re-capture/unitgroup.py --all --check # self-check every stage

A committed dump of Stage 02 is at ../data/unitgroup-s02.txt.

✅ Container

0x00              "IDXD"
0x04    u32       nrec
0x08    nrec x 16 record: (key, squadron_off, field_lo, field_hi)
        u32       npool
        npool x 12 property entry: (tag, name_off | 0xffffffff, value_off)
        u32       strsize          -- and STR + strsize == filesize, exactly
        strsize   string pool; [0] = "Enumerate_Squadrons", [0x14] = ""

Every offset in the file is relative to STR, the string-pool base. A record's fields are pool entries [field_lo, field_hi). An entry with name_off == 0xffffffff is positional; otherwise the entry carries its own field name inline, so the table is self-describing and the tag hash never has to be inverted.

Exactly one record per file is not a squadron: it is the Enumerate_Squadrons roster, whose entries map key -> squadron id for every other record. It is usually the last record but not always — in S05, S06, S07, S08, S09, S11, S13, S28 and S29 it sits elsewhere, so find it by its missing Count, not by position.

✅ A squadron record

Five named fields, always present and always these five:

field	values
Count	1 (×1082), 2 (×50), 3 (×8), 4 (×20)
SideID	ADAN (745), TCAF (294), Neutral (121)
AIID	31 distinct, e.g. AI_ADAN_CraftSquadron_Veteran, AI_TCAF_BirdFlight, AI_ADAN_Fleet, AI_Structure, AI_TraceRoute
FormationID	e.g. Formation_4_Bird, Formation_ADAN_Turret07_30, Formation_1_only
DisableInterval	No (1129), Yes (31)

…preceded by Count member tuples of four positional entries:

(unit model, message set, n, identity)

• unit model — 122 distinct, and they are our XBG7 mesh names: UN_f001_TCAF_DeltaSaber_T_Player, UN_e010_ADAN_Attacker_S, UN_e007_ADAN_Turret, UN_bf001_TCAF_SchlosBase.
• message set — the squadron's radio chatter, e.g. MessageSet_Ellen, MessageSet_ADAN_plA.
• n — ✅ the number of units this tuple instantiates, filling slots of the squadron's formation. Integer 1…30, dominated by 1 (735) and 9 (197). See "What n counts" below — the old note that it is "not the _NN suffix of FormationID" was right but drew the wrong conclusion.
• identity — 63 distinct: named pilots (ELLEN, SANDRA, RAYMOND, YOJI), ship nameplates (NP_Charon, NP_Amalthea, NP_Olympus), carrier tags (Carrier_ADAN01), cargo tags (CARGO_1), or empty.

Worked example — Stage 02, squadron TCN004, the player's own flight:

TCN004  TCAF  Count=2  DisableInterval=No  Formation_2_Rhino1  AI_TCAF_RhinoFlight
    unit=UN_f001_TCAF_DeltaSaber_T_Player  msg=MessageSet_Katana  n=1  id=Character_Player_Test
    unit=UN_f001_TCAF_DeltaSaber_T         msg=MessageSet_Ellen   n=1  id=ELLEN

✅ Two independent self-checks, both corpus-wide

1. len(fields) == Count * 4 + 5 — holds for 1160 of 1160 squadrons across all 28 stage tables. This is what pins the member-tuple width at 4 and the named-field count at 5; it was not assumed.
2. Roster agreement — the Enumerate_Squadrons record's key -> id mapping agrees with the name each record resolves independently through its own squadron_off, for 1160 of 1160.

S17 has no stage\UnitGroup_S17.tbl in GP_MAIN_GAME_E.pak, matching the missing S17 stage record.

❌ Refuted along the way

• The record key is not the squadron id's name hash. name_hash("TCN001") = 0xd639f1a4, but the keys run 0x659aff47, 0x659b0046, 0x669b0047, … — 0 of 112 match. What the key encodes is still ❔. Its byte structure (b0 ramping, b2 taking small signed values) looks like a packed tuple, untested.
• Squadron ids do not use a separate string base. An earlier reading here put them at STR + 0x15 and scored "109 of 111", which looked like a near-fit but was an artefact of the uniform 7-byte id stride — it silently shifted every name by three entries (record 0 read as ADN104 instead of ADN101, and 17 TC*-named squadrons came out with SideID = ADAN). The roster record refuted it outright, and with the correct base — plain STR — agreement is 111/111. The lesson is that a 98 %-looking score on a self-consistent stride is not evidence; the roster was an independent oracle and should have been consulted first.
• The 20-byte "section header" described in an earlier draft of stage-definition-table.md does not exist. What looked like (tag, 0, 0, count, size) was the file's last 16-byte record followed by the npool word. The corrected layout above is uniform across all 28 files, which the earlier reading was not — it failed on 9 of them.

What is still open

• The meaning of the 4-byte record key.
• The member tuple's third field n.
• The interval values. DisableInterval is a per-squadron flag and 31 squadrons set it, but the interval durations, spawn triggers and arrival positions are not in this file. The next places to look are Formation_*.tbl (the formation geometry and possibly its timing) and EnumSquadron_Test.tbl, both named by the stage record.

✅ What n counts — bounded by the formation, 1159 of 1160

n is the number of units the member tuple instantiates. The evidence is a hard, falsifiable inequality against a table UnitGroup never mentions.

FormationSet_S<NN>.tbl is the same IDXD container, and a formation record is a slot list: one named field FrameCount plus 8 positional fields per slot — FrameCount = 4 → 33 fields, 14 → 113, 30 → 241, 32 → 257, i.e. 1 + 8·FrameCount, exactly.

Resolving each squadron's FormationID to its formation (below) and comparing:

sum(n) <= FrameCount:   HOLDS 1159    VIOLATED 1    unresolved 0
   of the holders, sum(n) == FrameCount exactly:  539

Across all 28 stages, every FormationID resolves and every squadron but one fits its formation, with 46 % filling it exactly. A field unrelated to formation size would not do that.

The one violation is a debug leftover, recorded rather than swept up: S20 (a tutorial stage), Formation_1_only (FrameCount = 1) with n = 2 — AIID = AI_Test, msg = MessageSet_test, no identity, unit UN_e015_ADAN_Puppy_2. It is still a literal violation of the invariant.

✅ The formation name's trailing number IS FrameCount

Formation_ADAN_Turret07_30 → FrameCount = 30; Formation_TCAF_ArrowHead02_32 → 32; Formation_4_Bird → 4 (leading, this family has no suffix). So the old observation that n = 9 does not match the _30 in Formation_ADAN_Turret07_30 is correct and not evidence against n being a count: 9 units occupy 9 of that formation's 30 slots.

❌ WITHDRAWN — "FormationID does not hash into the table"

It does. The refutation below used the wrong hash function. A record key is tag_hash of the record's own name — not the pak-TOC name_hash, which lowercases and uses a different modulus. Checked directly:

tag_hash("Formation_4_Bird") = 0x22a5eeed   <- exactly the record key
name_hash("Formation_4_Bird") = 0x6286edad  <- the wrong hash, hence "0 of 16"

Formation_4_Bird is a record name in FormationSet_S02.tbl, and every record in that table keys by tag_hash of its name. Across all 28 UnitGroup_S<NN>.tbl the same holds for field keys: 6960 / 6960 are tag_hash(field name) and 0 are name_hash. So FormationID resolves by a direct hash lookup and the roster indirection is not required for it.

This is the exact trap idxd-container.md warns about, and it cost this file a spurious ❌.

🟡 The squadron half of the original claim still stands: a squadron record's key is not tag_hash of the squadron id either (tag_hash("TCN001") = 0x760cdeaa, against keys running 0x659aff47…), because the record's name is not the squadron id. The roster maps id → record, and that indirection is real.

The original text, kept

FormationID does not hash into the table: name_hash("Formation_4_Bird") = 0x6286edad, and the record key is 0x22a5eeed — 0 of 16 resolve that way. FormationSet_S02.tbl has 17 records for 16 formations, and the extra one is a name roster, exactly like Enumerate_Squadrons here: it carries no FrameCount and its fields are (tag, name, "") triples whose tags are the record keys. Find it by its missing FrameCount, map name → key, done.

That is the same convention twice in two different tables, which is worth remembering for the next one: a table's record keys are resolved by a roster record inside the table, not by hashing the name.

🟡 What this does not settle

• It shows sum(n) fits the formation, not that each unit is separately instantiated. "Slots consumed" and "units spawned" are indistinguishable here.
• It does not close the 387-vs-~300 gap in roster-to-craft-link. Σn over all of Stage 02 is 387 against 296–300 live craft. That count was measured mid-mission after kills, and squadrons deploy across phases, so the two are not comparable as they stand — but the earlier note rejected n = 387 as the craft count on the assumption that everything deploys at once, and that assumption is still untested.
• ✅ The key derivation is no longer ❔: the "second hash function" is tag_hash, recovered since this was written. Record keys and field keys are both tag_hash of their own name — 6960 / 6960 field keys across all 28 UnitGroup tables, 0 matching name_hash.

✅❌ Declared × deployed — the weapon law does not generalise, and the whole exception is S14

weapon-struct-runtime established that nothing is mounted without being declared — 0 of 131, an empty cell. Running the same partition over units refutes the general form of that law.

Three vocabularies had to be separated first, because they are not the same set:

set	source	size
declared	field names of the 31 EnumUnit records	118
deployed	slot 0 of each UnitGroup member tuple	122
datasheet	Generic records carrying HP	114

The datasheet count re-derives unit-datasheet-static's own 114 from a different direction, as a control. declared and deployed share one vocabulary (UN_*), overlapping on 109; the datasheet's IDs are a third naming, disjoint from both.

	deployed	not deployed
declared	109	9
not declared	13	1

❌ The cell that was empty for weapons holds 13 for units. Units can be deployed without any table declaring them.

✅ But the 13 are one coherent family, not strays — every one is UN_S14_p1_Asteroid_cmesh_* or UN_S14_p2_Asteroid_cmesh_*. Asking the other population settles it:

stage	asteroid meshes deployed	of those, declared
S01	8	8
S04	10	10
S05	5	5
S08	6	6
S13	4	4
S14 p1	9	0
S14 p2	4	0

33 of 33 declared everywhere else; 0 of 13 in S14. And non-asteroid deployed-but-undeclared = 0 — the entire violation is this one stage's asteroid field. So the law survives in a narrower form: every unit that is deployed is declared, except S14's asteroid collision meshes.

🟡 Why S14 differs is not settled here. It is suggestive that S14 is the stage whose asteroid table S28 borrows (see asteroid-fields), but nothing measured connects the borrow to the missing declarations — a coincidence worth recording, not a cause.

The remaining cells are small and benign: 9 declared-but-never-deployed units, and one datasheet id, Ship_, that is neither.

✅ Sharper, per stage: EnumUnit_S<NN> is the deployed set — 20 of 22 exactly

The partition above was global, which understated the result. Joining each stage's own EnumerateUnit table to its own EnumerateSquadron table gives a much tighter statement:

	stages
declared == deployed, exactly	20
declares 4 it never deploys	S16
deploys 13 it never declares	S14

Per stage the counts are equal and the undeclared count is 0 — S01 13/13, S02 14/14, S03 15/15, S04 23/23, S05 13/13, S06 19/19, S07 17/17, S08 23/23, S09 21/21, S10 3/3, S11 10/10, S12 15/15, S13 18/18, S15 18/18, S24 11/11, S25 15/15, S26 15/15, S27 16/16, S28 9/9, S29 18/18. Every per-stage deployed-but-undeclared count is 0 except S14's, and the 13 there are the whole disc-wide total — so the global "13" is one stage's, not a scattering.

🔑 S16 explains what the table actually is. It declares 6 and deploys 2. Its six are UN_e009_ADAN_Phantom, UN_f002_TCAF_DeltaSaber_W, …_W_Player, Test_ADAN_PrometheusDriver_InsideP2_01, UN_e901_ADAN_Boss and UN_e910_core_ADAN_GeneratorCore — and the last two are exactly the units stage16-boss shows are brought in by their own loader chain (sub_82315AE8), not by the squadron roster. So EnumUnit_S<NN> is the stage's load manifest, not a mirror of the roster: a unit that code spawns is declared but never deployed. That also names 2 of the 9 disc-wide declared-but-never-deployed units; 6 more are Ship_Test* placeholders and the ninth is that Prometheus-driver test entry, so all 9 are accounted for.

🟡 S14 remains the one unexplained cell, and it is not a missing table: EnumUnit_S14.tbl exists and declares six names — UN_e004_ADAN_ElanPlus_N, UN_e006_ADAN_Vindicator_Margras, UN_f002_TCAF_DeltaSaber_W, …_W_Player, UN_f201_TCAF_Tanker, UN_mn040_Asteroid_Big. Note the last: S14 does declare an asteroid, just not the 13 Asteroid_cmesh entities it deploys. So "asteroids are exempt from the manifest" is refuted for S14 itself; whatever the cause, it is specific to the collision-mesh entities.