Sylpheed/docs/re/structures/unit-group-table.md

stage\UnitGroup_S<NN>.tbl — the per-stage squadron roster

Status: ✅ container format and field semantics, validated across all 28 stage tables present on the disc; 🟡 one member field; ❔ the record key and the arrival-interval values.

Reached from the per-stage definition record's EnumerateSquadron field — see stage-definition-table.md. Tool: tools/re-capture/unitgroup.py (pure static; runs no emulator).

python3 tools/re-capture/unitgroup.py S02          # full roster
python3 tools/re-capture/unitgroup.py --all --check # self-check every stage

A committed dump of Stage 02 is at ../data/unitgroup-s02.txt.

✅ Container

0x00              "IDXD"
0x04    u32       nrec
0x08    nrec x 16 record: (key, squadron_off, field_lo, field_hi)
        u32       npool
        npool x 12 property entry: (tag, name_off | 0xffffffff, value_off)
        u32       strsize          -- and STR + strsize == filesize, exactly
        strsize   string pool; [0] = "Enumerate_Squadrons", [0x14] = ""

Every offset in the file is relative to STR, the string-pool base. A record's fields are pool entries [field_lo, field_hi). An entry with name_off == 0xffffffff is positional; otherwise the entry carries its own field name inline, so the table is self-describing and the tag hash never has to be inverted.

Exactly one record per file is not a squadron: it is the Enumerate_Squadrons roster, whose entries map key -> squadron id for every other record. It is usually the last record but not always — in S05, S06, S07, S08, S09, S11, S13, S28 and S29 it sits elsewhere, so find it by its missing Count, not by position.

✅ A squadron record

Five named fields, always present and always these five:

field	values
Count	1 (×1082), 2 (×50), 3 (×8), 4 (×20)
SideID	ADAN (745), TCAF (294), Neutral (121)
AIID	31 distinct, e.g. AI_ADAN_CraftSquadron_Veteran, AI_TCAF_BirdFlight, AI_ADAN_Fleet, AI_Structure, AI_TraceRoute
FormationID	e.g. Formation_4_Bird, Formation_ADAN_Turret07_30, Formation_1_only
DisableInterval	No (1129), Yes (31)

…preceded by Count member tuples of four positional entries:

(unit model, message set, n, identity)

• unit model — 122 distinct, and they are our XBG7 mesh names: UN_f001_TCAF_DeltaSaber_T_Player, UN_e010_ADAN_Attacker_S, UN_e007_ADAN_Turret, UN_bf001_TCAF_SchlosBase.
• message set — the squadron's radio chatter, e.g. MessageSet_Ellen, MessageSet_ADAN_plA.
• n — ✅ the number of units this tuple instantiates, filling slots of the squadron's formation. Integer 1…30, dominated by 1 (735) and 9 (197). See "What n counts" below — the old note that it is "not the _NN suffix of FormationID" was right but drew the wrong conclusion.
• identity — 63 distinct: named pilots (ELLEN, SANDRA, RAYMOND, YOJI), ship nameplates (NP_Charon, NP_Amalthea, NP_Olympus), carrier tags (Carrier_ADAN01), cargo tags (CARGO_1), or empty.

Worked example — Stage 02, squadron TCN004, the player's own flight:

TCN004  TCAF  Count=2  DisableInterval=No  Formation_2_Rhino1  AI_TCAF_RhinoFlight
    unit=UN_f001_TCAF_DeltaSaber_T_Player  msg=MessageSet_Katana  n=1  id=Character_Player_Test
    unit=UN_f001_TCAF_DeltaSaber_T         msg=MessageSet_Ellen   n=1  id=ELLEN

✅ Two independent self-checks, both corpus-wide

1. len(fields) == Count * 4 + 5 — holds for 1160 of 1160 squadrons across all 28 stage tables. This is what pins the member-tuple width at 4 and the named-field count at 5; it was not assumed.
2. Roster agreement — the Enumerate_Squadrons record's key -> id mapping agrees with the name each record resolves independently through its own squadron_off, for 1160 of 1160.

S17 has no stage\UnitGroup_S17.tbl in GP_MAIN_GAME_E.pak, matching the missing S17 stage record.

❌ Refuted along the way

• The record key is not the squadron id's name hash. name_hash("TCN001") = 0xd639f1a4, but the keys run 0x659aff47, 0x659b0046, 0x669b0047, … — 0 of 112 match. What the key encodes is still ❔. Its byte structure (b0 ramping, b2 taking small signed values) looks like a packed tuple, untested.
• Squadron ids do not use a separate string base. An earlier reading here put them at STR + 0x15 and scored "109 of 111", which looked like a near-fit but was an artefact of the uniform 7-byte id stride — it silently shifted every name by three entries (record 0 read as ADN104 instead of ADN101, and 17 TC*-named squadrons came out with SideID = ADAN). The roster record refuted it outright, and with the correct base — plain STR — agreement is 111/111. The lesson is that a 98 %-looking score on a self-consistent stride is not evidence; the roster was an independent oracle and should have been consulted first.
• The 20-byte "section header" described in an earlier draft of stage-definition-table.md does not exist. What looked like (tag, 0, 0, count, size) was the file's last 16-byte record followed by the npool word. The corrected layout above is uniform across all 28 files, which the earlier reading was not — it failed on 9 of them.

What is still open

• The meaning of the 4-byte record key.
• The member tuple's third field n.
• The interval values. DisableInterval is a per-squadron flag and 31 squadrons set it, but the interval durations, spawn triggers and arrival positions are not in this file. The next places to look are Formation_*.tbl (the formation geometry and possibly its timing) and EnumSquadron_Test.tbl, both named by the stage record.

✅ What n counts — bounded by the formation, 1159 of 1160

n is the number of units the member tuple instantiates. The evidence is a hard, falsifiable inequality against a table UnitGroup never mentions.

FormationSet_S<NN>.tbl is the same IDXD container, and a formation record is a slot list: one named field FrameCount plus 8 positional fields per slot — FrameCount = 4 → 33 fields, 14 → 113, 30 → 241, 32 → 257, i.e. 1 + 8·FrameCount, exactly.

Resolving each squadron's FormationID to its formation (below) and comparing:

sum(n) <= FrameCount:   HOLDS 1159    VIOLATED 1    unresolved 0
   of the holders, sum(n) == FrameCount exactly:  539

Across all 28 stages, every FormationID resolves and every squadron but one fits its formation, with 46 % filling it exactly. A field unrelated to formation size would not do that.

The one violation is a debug leftover, recorded rather than swept up: S20 (a tutorial stage), Formation_1_only (FrameCount = 1) with n = 2 — AIID = AI_Test, msg = MessageSet_test, no identity, unit UN_e015_ADAN_Puppy_2. It is still a literal violation of the invariant.

✅ The formation name's trailing number IS FrameCount

Formation_ADAN_Turret07_30 → FrameCount = 30; Formation_TCAF_ArrowHead02_32 → 32; Formation_4_Bird → 4 (leading, this family has no suffix). So the old observation that n = 9 does not match the _30 in Formation_ADAN_Turret07_30 is correct and not evidence against n being a count: 9 units occupy 9 of that formation's 30 slots.

❌ WITHDRAWN — "FormationID does not hash into the table"

It does. The refutation below used the wrong hash function. A record key is tag_hash of the record's own name — not the pak-TOC name_hash, which lowercases and uses a different modulus. Checked directly:

tag_hash("Formation_4_Bird") = 0x22a5eeed   <- exactly the record key
name_hash("Formation_4_Bird") = 0x6286edad  <- the wrong hash, hence "0 of 16"

Formation_4_Bird is a record name in FormationSet_S02.tbl, and every record in that table keys by tag_hash of its name. Across all 28 UnitGroup_S<NN>.tbl the same holds for field keys: 6960 / 6960 are tag_hash(field name) and 0 are name_hash. So FormationID resolves by a direct hash lookup and the roster indirection is not required for it.

This is the exact trap idxd-container.md warns about, and it cost this file a spurious ❌.

🟡 The squadron half of the original claim still stands: a squadron record's key is not tag_hash of the squadron id either (tag_hash("TCN001") = 0x760cdeaa, against keys running 0x659aff47…), because the record's name is not the squadron id. The roster maps id → record, and that indirection is real.

The original text, kept

FormationID does not hash into the table: name_hash("Formation_4_Bird") = 0x6286edad, and the record key is 0x22a5eeed — 0 of 16 resolve that way. FormationSet_S02.tbl has 17 records for 16 formations, and the extra one is a name roster, exactly like Enumerate_Squadrons here: it carries no FrameCount and its fields are (tag, name, "") triples whose tags are the record keys. Find it by its missing FrameCount, map name → key, done.

That is the same convention twice in two different tables, which is worth remembering for the next one: a table's record keys are resolved by a roster record inside the table, not by hashing the name.

🟡 What this does not settle

• It shows sum(n) fits the formation, not that each unit is separately instantiated. "Slots consumed" and "units spawned" are indistinguishable here.
• It does not close the 387-vs-~300 gap in roster-to-craft-link. Σn over all of Stage 02 is 387 against 296–300 live craft. That count was measured mid-mission after kills, and squadrons deploy across phases, so the two are not comparable as they stand — but the earlier note rejected n = 387 as the craft count on the assumption that everything deploys at once, and that assumption is still untested.
• ✅ The key derivation is no longer ❔: the "second hash function" is tag_hash, recovered since this was written. Record keys and field keys are both tag_hash of their own name — 6960 / 6960 field keys across all 28 UnitGroup tables, 0 matching name_hash.

✅❌ Declared × deployed — the weapon law does not generalise, and the whole exception is S14

weapon-struct-runtime established that nothing is mounted without being declared — 0 of 131, an empty cell. Running the same partition over units refutes the general form of that law.

Three vocabularies had to be separated first, because they are not the same set:

set	source	size
declared	field names of the 31 EnumUnit records	118
deployed	slot 0 of each UnitGroup member tuple	122
datasheet	Generic records carrying HP	114

The datasheet count re-derives unit-datasheet-static's own 114 from a different direction, as a control. declared and deployed share one vocabulary (UN_*), overlapping on 109; the datasheet's IDs are a third naming, disjoint from both.

	deployed	not deployed
declared	109	9
not declared	13	1

❌ The cell that was empty for weapons holds 13 for units. Units can be deployed without any table declaring them.

✅ But the 13 are one coherent family, not strays — every one is UN_S14_p1_Asteroid_cmesh_* or UN_S14_p2_Asteroid_cmesh_*. Asking the other population settles it:

stage	asteroid meshes deployed	of those, declared
S01	8	8
S04	10	10
S05	5	5
S08	6	6
S13	4	4
S14 p1	9	0
S14 p2	4	0

33 of 33 declared everywhere else; 0 of 13 in S14. And non-asteroid deployed-but-undeclared = 0 — the entire violation is this one stage's asteroid field. So the law survives in a narrower form: every unit that is deployed is declared, except S14's asteroid collision meshes.

🟡 Why S14 differs is not settled here. It is suggestive that S14 is the stage whose asteroid table S28 borrows (see asteroid-fields), but nothing measured connects the borrow to the missing declarations — a coincidence worth recording, not a cause.

The remaining cells are small and benign: 9 declared-but-never-deployed units, and one datasheet id, Ship_, that is neither.

✅ Sharper, per stage: EnumUnit_S<NN> is the deployed set — 20 of 22 exactly

The partition above was global, which understated the result. Joining each stage's own EnumerateUnit table to its own EnumerateSquadron table gives a much tighter statement:

	stages
declared == deployed, exactly	20
declares 4 it never deploys	S16
deploys 13 it never declares	S14

Per stage the counts are equal and the undeclared count is 0 — S01 13/13, S02 14/14, S03 15/15, S04 23/23, S05 13/13, S06 19/19, S07 17/17, S08 23/23, S09 21/21, S10 3/3, S11 10/10, S12 15/15, S13 18/18, S15 18/18, S24 11/11, S25 15/15, S26 15/15, S27 16/16, S28 9/9, S29 18/18. Every per-stage deployed-but-undeclared count is 0 except S14's, and the 13 there are the whole disc-wide total — so the global "13" is one stage's, not a scattering.

🔑 S16 explains what the table actually is. It declares 6 and deploys 2. Its six are UN_e009_ADAN_Phantom, UN_f002_TCAF_DeltaSaber_W, …_W_Player, Test_ADAN_PrometheusDriver_InsideP2_01, UN_e901_ADAN_Boss and UN_e910_core_ADAN_GeneratorCore — and the last two are exactly the units stage16-boss shows are brought in by their own loader chain (sub_82315AE8), not by the squadron roster. So EnumUnit_S<NN> is the stage's load manifest, not a mirror of the roster: a unit that code spawns is declared but never deployed. That also names 2 of the 9 disc-wide declared-but-never-deployed units; 6 more are Ship_Test* placeholders and the ninth is that Prometheus-driver test entry, so all 9 are accounted for.

🟡 S14 remains the one unexplained cell, and it is not a missing table: EnumUnit_S14.tbl exists and declares six names — UN_e004_ADAN_ElanPlus_N, UN_e006_ADAN_Vindicator_Margras, UN_f002_TCAF_DeltaSaber_W, …_W_Player, UN_f201_TCAF_Tanker, UN_mn040_Asteroid_Big. Note the last: S14 does declare an asteroid, just not the 13 Asteroid_cmesh entities it deploys. So "asteroids are exempt from the manifest" is refuted for S14 itself; whatever the cause, it is specific to the collision-mesh entities.

✅ S14 resolved: the 13 are dangling deployments, not a manifest omission

The previous pass left S14 as the one unexplained cell and offered no cause. Two measurements settle what kind of thing it is.

1. The deployment side is identical. S14's asteroid-cmesh squadron records are indistinguishable from a stage that declares its own — same squadron ids (GNN901, GNN902, …) and the same five named fields with the same values:

Count 1   SideID Neutral   AIID AI_Structure   FormationID Formation_1_only   DisableInterval No

So nothing about how S14 deploys them differs. The wrong record would not have been a control here; these are the same kind of record, compared field for field.

2. The definition side is where they differ — completely. Asking whether each cmesh entity has a unit definition at all:

stage	deployed	declared in EnumUnit	has a Generic record with HP	has any Generic record
S01	8	8	8	8
S04	10	10	10	10
S05	5	5	5	5
S08	6	6	6	6
S13	4	4	4	4
S14 p1	9	0	0	0
S14 p2	4	0	0	0

🔑 33 / 33 / 33 for the five, and 0 / 0 / 0 for S14. The 13 are not merely absent from the load manifest — they have no unit definition anywhere on the disc, not even an un-HP'd Generic row. UnitGroup_S14.tbl names entities that do not exist.

That reframes the exception and rescues the law. Nothing is "deployed without being loaded" in any working sense: S14's roster holds 13 dangling references, and the manifest is consistent with them being unloadable. It also explains why the omission is exactly the cmesh class and nothing else in S14 — the other six units S14 deploys are declared and defined normally.

🟡 What this predicts, and what would settle it, needs the emulator. Either those 13 collision meshes simply fail to appear in Stage 14 — a shipped bug — or some route outside the unit chain supplies them. Flying S14 and checking whether the large asteroids collide would distinguish the two. NEEDS-HUMAN; no static route here can tell them apart.

⚠️ Note the two asteroid populations are different sets and must not be conflated: asteroid-fields' AsteroidGroup_00N tables scatter mob_n0NN models, while these are UN_<stage>_Asteroid_cmesh_* squadron entities. S28.Phase_1 borrows S14_p2_asteroid.tbl (the scatter table) but deploys no cmesh entity of its own, so the borrow does not carry the dangling references with it.