fabi/Syplheed-Reborn
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

re: decode the UnitGroup squadron roster, and correct yesterday's container reading

Browse Source 
stage\UnitGroup_S<NN>.tbl is now fully readable. A squadron record is Count
member tuples -- (unit model, message set, n, identity/nameplate) -- followed by
five named fields: Count, SideID, AIID, FormationID, DisableInterval. The
property entries carry their own field names inline, so the tag hash never has
to be inverted.

Two independent self-checks validate it corpus-wide, 1160/1160 each across all
28 stage tables on the disc:

  - the length identity len(fields) == Count * 4 + 5, which is what pins the
    member-tuple width at 4 and the named-field count at 5;
  - agreement with the file's own Enumerate_Squadrons roster, which maps record
    key to squadron id independently of the per-record string offset.

Adds tools/re-capture/unitgroup.py (pure static, runs no emulator) with a
--all --check self-check mode, and commits the Stage 02 dump as evidence.

Corrections to the container layout written yesterday, all three wrong:

  - the 20-byte "(tag, 0, 0, count, size) section header" does not exist. It
    was the file's last 16-byte record followed by a plain npool word. The
    corrected layout is uniform across all 28 files; the old one failed on 9.
  - squadron ids do not use a separate string base. Every offset in the file is
    relative to the one string pool. The earlier "109 of 111" score was an
    artefact of the uniform 7-byte id stride and had silently shifted every
    name by three entries, which is why 17 TC*-named squadrons came out as
    SideID=ADAN. The roster record refuted it outright.
  - the roster is not always the last record; 9 stages put it elsewhere, so it
    is found by its missing Count.

Refuted and kept: the 4-byte record key is not the squadron id's name hash
(0 of 112).

Not settled: what the key encodes, the member tuple's third field n, and where
the arrival interval values live. DisableInterval is only a per-squadron flag
(Yes for 31 of 1160); the durations, triggers and arrival positions are not in
this file. Formation_*.tbl and EnumSquadron_Test.tbl are next.
This commit is contained in:
Sylpheed RE agent
2026-08-24 11:07:33 +00:00
parent 7482142abf
commit b39aabf5c3
5 changed files with 528 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
docs/re/BACKLOG.md
View File

@@ -283,10 +283,27 @@ search cannot find a *schedule*.
🔴 Refuted along the way: the 16-byte record key is **not** the squadron
ID's name hash (0 of 112).
**First step (next):** read the `UnitGroup` per-record payload — three u32s per
entry — and find where the interval *values* live (the file's second section,
`Formation_*.tbl`, or `EnumSquadron_Test.tbl`). Also: S17–S23 have no stage
record in `GP_MAIN_GAME_E.pak`; find where they are.
* ✅ **`UnitGroup` is fully decoded** (2026-08-24). Container and field semantics
in [`structures/unit-group-table.md`](structures/unit-group-table.md), tool
`tools/re-capture/unitgroup.py`, Stage 02 dump committed at
[`data/unitgroup-s02.txt`](data/unitgroup-s02.txt). A squadron record is
`Count` member tuples — (unit model, message set, `n`, identity/nameplate) —
followed by five named fields `Count / SideID / AIID / FormationID /
DisableInterval`. Validated corpus-wide by two independent self-checks, each
**1160/1160** across all 28 stage tables: the `Count*4+5` length identity, and
agreement with the file's own `Enumerate_Squadrons` roster.
🔴 Two refutations recorded: the record key is **not** the squadron-ID name
hash, and an earlier "109/111" reading of the squadron-id string base was an
artefact of the uniform 7-byte id stride (it shifted every name by three).
**First step (next):** find where the arrival *interval values* live.
`DisableInterval` is only a per-squadron flag (`Yes` for 31 of 1160); the
durations, spawn triggers and arrival positions are not in `UnitGroup`. The
stage record names two untouched candidates: `Formation_*.tbl` (formation
geometry, possibly its timing) and `EnumSquadron_Test.tbl`. Also still open: the
4-byte record key, the member field `n`, and the missing S17–S23 stage records
(S17 has no `UnitGroup_S17.tbl` either, so the whole stage is absent from
`GP_MAIN_GAME_E.pak`).
## The dynamic-RE state is not in git, and it was gone

228
docs/re/data/unitgroup-s02.txt Normal file
View File

@@ -0,0 +1,228 @@
S02: 111 squadrons, 116 members, Count*4+5 111/111, roster agrees 111/111
ADN101 ADAN Count=1 DisableInterval=No Formation_ADAN_Turret07_30 AI_ADAN_CraftSquadron_Veteran
unit=UN_e007_ADAN_Turret msg=MessageSet_ADAN_plA n=9 pilot=
ADN110 ADAN Count=1 DisableInterval=No Formation_ADAN_Turret09_30 AI_ADAN_CraftSquadron_Rookie
unit=UN_e007_ADAN_Turret msg=MessageSet_ADAN_plA n=9 pilot=
ADN111 ADAN Count=1 DisableInterval=No Formation_ADAN_Turret09_30 AI_ADAN_CraftSquadron_Veteran
unit=UN_e007_ADAN_Turret msg=MessageSet_ADAN_plB n=9 pilot=
ADN201 ADAN Count=1 DisableInterval=No Formation_ADAN_Destroyer02_03 AI_ADAN_Fleet
unit=UN_e106_ADAN_Destroyer msg=MessageSet_ADAN_ShipA n=1 pilot=
ADN103 ADAN Count=1 DisableInterval=No Formation_ADAN_Turret08_30 AI_ADAN_CraftSquadron_Rookie
unit=UN_e007_ADAN_Turret msg=MessageSet_ADAN_plB n=9 pilot=
ADN112 ADAN Count=1 DisableInterval=No Formation_ADAN_Turret09_30 AI_ADAN_CraftSquadron_Rookie
unit=UN_e007_ADAN_Turret msg=MessageSet_ADAN_plC n=9 pilot=
ADN202 ADAN Count=1 DisableInterval=No Formation_ADAN_Destroyer02_03 AI_ADAN_Fleet
unit=UN_e106_ADAN_Destroyer msg=MessageSet_ADAN_ShipA n=1 pilot=
ADN310 ADAN Count=1 DisableInterval=No Formation_ADAN_Turret03_30 AI_ADAN_CraftSquadron_Rookie
unit=UN_e007_ADAN_Turret msg=MessageSet_ADAN_plBC n=15 pilot=
ADN104 ADAN Count=1 DisableInterval=No Formation_ADAN_Turret08_30 AI_ADAN_CraftSquadron_Rookie
unit=UN_e007_ADAN_Turret msg=MessageSet_ADAN_plB n=9 pilot=
ADN131 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e106_ADAN_Destroyer msg=MessageSet_ADAN_ShipA n=1 pilot=
ADN140 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e105_ADAN_Cruiser msg=MessageSet_ADAN_ShipB n=1 pilot=
ADN203 ADAN Count=1 DisableInterval=No Formation_ADAN_Turret08_30 AI_ADAN_CraftSquadron_Rookie
unit=UN_e007_ADAN_Turret msg=MessageSet_ADAN_plC n=18 pilot=
ADN221 ADAN Count=1 DisableInterval=No Formation_ADAN_Turret07_30 AI_ADAN_CraftSquadron_Veteran
unit=UN_e007_ADAN_Turret msg=MessageSet_ADAN_plC n=9 pilot=
ADN230 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e106_ADAN_Destroyer msg=MessageSet_ADAN_ShipA n=1 pilot=
ADN311 ADAN Count=1 DisableInterval=No Formation_ADAN_Turret09_30 AI_ADAN_CraftSquadron_Rookie
unit=UN_e007_ADAN_Turret msg=MessageSet_ADAN_plB n=9 pilot=
ADN105 ADAN Count=1 DisableInterval=No Formation_ADAN_Turret09_30 AI_ADAN_CraftSquadron_Rookie
unit=UN_e007_ADAN_Turret msg=MessageSet_ADAN_plC n=9 pilot=
ADN114 ADAN Count=1 DisableInterval=No Formation_ADAN_Turret09_30 AI_ADAN_CraftSquadron_Rookie
unit=UN_e007_ADAN_Turret msg=MessageSet_ADAN_plB n=9 pilot=
ADN132 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e106_ADAN_Destroyer msg=MessageSet_ADAN_ShipA n=1 pilot=
ADN204 ADAN Count=1 DisableInterval=No Formation_ADAN_AttackerS03_12 AI_ADAN_BomberSquadron_Rookie
unit=UN_e010_ADAN_Attacker_S msg=MessageSet_ADAN_plA n=4 pilot=
ADN222 ADAN Count=1 DisableInterval=No Formation_ADAN_Turret09_30 AI_ADAN_CraftSquadron_Rookie
unit=UN_e007_ADAN_Turret msg=MessageSet_ADAN_plA n=9 pilot=
ADN231 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e105_ADAN_Cruiser msg=MessageSet_ADAN_ShipB n=1 pilot=
ADN312 ADAN Count=1 DisableInterval=No Formation_ADAN_Turret03_30 AI_ADAN_CraftSquadron_Veteran
unit=UN_e007_ADAN_Turret msg=MessageSet_ADAN_plA n=15 pilot=
ADN321 ADAN Count=1 DisableInterval=No Formation_ADAN_Turret09_30 AI_ADAN_CraftSquadron_Rookie
unit=UN_e007_ADAN_Turret msg=MessageSet_ADAN_plA n=9 pilot=
ADN330 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e106_ADAN_Destroyer msg=MessageSet_ADAN_ShipA n=1 pilot=
ADN106 ADAN Count=1 DisableInterval=No Formation_ADAN_Turret09_30 AI_ADAN_CraftSquadron_Rookie
unit=UN_e007_ADAN_Turret msg=MessageSet_ADAN_plA n=9 pilot=
ADN115 ADAN Count=1 DisableInterval=No Formation_ADAN_Turret09_30 AI_ADAN_CraftSquadron_Rookie
unit=UN_e007_ADAN_Turret msg=MessageSet_ADAN_plC n=9 pilot=
ADN133 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e106_ADAN_Destroyer msg=MessageSet_ADAN_ShipA n=1 pilot=
ADN205 ADAN Count=1 DisableInterval=No Formation_ADAN_Turret09_30 AI_ADAN_CraftSquadron_Rookie
unit=UN_e007_ADAN_Turret msg=MessageSet_ADAN_plA n=9 pilot=
ADN223 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Frigates
unit=UN_e108_ADAN_ASFrigate msg=MessageSet_ADAN_ShipA n=1 pilot=
ADN232 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e105_ADAN_Cruiser msg=MessageSet_ADAN_ShipB n=1 pilot=
ADN313 ADAN Count=1 DisableInterval=No Formation_1_only AI_TraceRoute
unit=UN_e201_ADAN_ISCMissile msg= n=1 pilot=
ADN322 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Frigates
unit=UN_e108_ADAN_ASFrigate msg=MessageSet_ADAN_ShipA n=1 pilot=
ADN331 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e105_ADAN_Cruiser msg=MessageSet_ADAN_ShipB n=1 pilot=
ADN134 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e106_ADAN_Destroyer msg=MessageSet_ADAN_ShipA n=1 pilot=
ADN206 ADAN Count=1 DisableInterval=No Formation_ADAN_AttackerS03_12 AI_ADAN_BomberSquadron_Rookie
unit=UN_e010_ADAN_Attacker_S msg=MessageSet_ADAN_plB n=4 pilot=
ADN224 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Frigates
unit=UN_e108_ADAN_ASFrigate msg=MessageSet_ADAN_ShipA n=1 pilot=
ADN314 ADAN Count=1 DisableInterval=No Formation_ADAN_Turret03_30 AI_ADAN_CraftSquadron_Veteran
unit=UN_e007_ADAN_Turret msg=MessageSet_ADAN_plAC n=15 pilot=
ADN323 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Frigates
unit=UN_e108_ADAN_ASFrigate msg=MessageSet_ADAN_ShipA n=1 pilot=
ADN332 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e105_ADAN_Cruiser msg=MessageSet_ADAN_ShipB n=1 pilot=
ADN108 ADAN Count=1 DisableInterval=No Formation_ADAN_Turret09_30 AI_ADAN_CraftSquadron_Rookie
unit=UN_e007_ADAN_Turret msg=MessageSet_ADAN_plC n=9 pilot=
ADT102 ADAN Count=1 DisableInterval=No Formation_ADAN_AttackerS03_12 AI_ADAN_BomberSquadron_Rookie
unit=UN_e010_ADAN_Attacker_S msg=MessageSet_ADAN_plC n=4 pilot=
ADN135 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e106_ADAN_Destroyer msg=MessageSet_ADAN_ShipA n=1 pilot=
ADN207 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e106_ADAN_Destroyer msg=MessageSet_ADAN_ShipA n=1 pilot=
ADN225 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Frigates
unit=UN_e108_ADAN_ASFrigate msg=MessageSet_ADAN_ShipA n=1 pilot=
ADN324 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Frigates
unit=UN_e108_ADAN_ASFrigate msg=MessageSet_ADAN_ShipA n=1 pilot=
ADN109 ADAN Count=1 DisableInterval=No Formation_ADAN_Turret09_30 AI_ADAN_CraftSquadron_Rookie
unit=UN_e007_ADAN_Turret msg=MessageSet_ADAN_plB n=9 pilot=
ADN136 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e106_ADAN_Destroyer msg=MessageSet_ADAN_ShipA n=1 pilot=
ADN208 ADAN Count=1 DisableInterval=No Formation_ADAN_AttackerS03_12 AI_ADAN_BomberSquadron_Rookie
unit=UN_e010_ADAN_Attacker_S msg=MessageSet_ADAN_plB n=6 pilot=
ADN226 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Frigates
unit=UN_e108_ADAN_ASFrigate msg=MessageSet_ADAN_ShipA n=1 pilot=
ADT301 ADAN Count=1 DisableInterval=No Formation_1_only AI_TraceRoute
unit=UN_e201_ADAN_ISCMissile msg= n=1 pilot=
ADN325 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Frigates
unit=UN_e108_ADAN_ASFrigate msg=MessageSet_ADAN_ShipA n=1 pilot=
ADT113 ADAN Count=1 DisableInterval=No Formation_ADAN_AttackerS03_12 AI_ADAN_BomberSquadron_Rookie
unit=UN_e010_ADAN_Attacker_S msg=MessageSet_ADAN_plA n=4 pilot=
ADN137 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e106_ADAN_Destroyer msg=MessageSet_ADAN_ShipA n=1 pilot=
ADN209 ADAN Count=1 DisableInterval=No Formation_ADAN_AttackerS03_12 AI_ADAN_BomberSquadron_Rookie
unit=UN_e010_ADAN_Attacker_S msg=MessageSet_ADAN_plB n=6 pilot=
ADN227 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Frigates
unit=UN_e108_ADAN_ASFrigate msg=MessageSet_ADAN_ShipA n=1 pilot=
ADT302 ADAN Count=1 DisableInterval=No Formation_1_only AI_TraceRoute
unit=UN_e201_ADAN_ISCMissile msg= n=1 pilot=
ADN326 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e106_ADAN_Destroyer msg=MessageSet_ADAN_ShipA n=1 pilot=
ADN138 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e105_ADAN_Cruiser msg=MessageSet_ADAN_ShipB n=1 pilot=
ADS151 ADAN Count=1 DisableInterval=No Formation_ADAN_AttackerS03_12 AI_ADAN_BomberSquadron_Rookie
unit=UN_e010_ADAN_Attacker_S msg=MessageSet_ADAN_plB n=4 pilot=
ADN228 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e106_ADAN_Destroyer msg=MessageSet_ADAN_ShipA n=1 pilot=
ADT303 ADAN Count=1 DisableInterval=No Formation_1_only AI_TraceRoute
unit=UN_e201_ADAN_ISCMissile msg= n=1 pilot=
ADN327 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e106_ADAN_Destroyer msg=MessageSet_ADAN_ShipA n=1 pilot=
ADN139 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e105_ADAN_Cruiser msg=MessageSet_ADAN_ShipB n=1 pilot=
ADN229 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e106_ADAN_Destroyer msg=MessageSet_ADAN_ShipA n=1 pilot=
ADS251 ADAN Count=1 DisableInterval=No Formation_ADAN_AttackerS03_12 AI_ADAN_BomberSquadron_Rookie
unit=UN_e010_ADAN_Attacker_S msg=MessageSet_ADAN_plB n=9 pilot=
ADT304 ADAN Count=1 DisableInterval=No Formation_1_only AI_TraceRoute
unit=UN_e201_ADAN_ISCMissile msg= n=1 pilot=
ADN328 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e106_ADAN_Destroyer msg=MessageSet_ADAN_ShipA n=1 pilot=
ADT107 ADAN Count=1 DisableInterval=No Formation_ADAN_AttackerS03_12 AI_ADAN_BomberSquadron_Rookie
unit=UN_e010_ADAN_Attacker_S msg=MessageSet_ADAN_plB n=4 pilot=
ADT305 ADAN Count=1 DisableInterval=No Formation_1_only AI_TraceRoute
unit=UN_e201_ADAN_ISCMissile msg= n=1 pilot=
ADN329 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e106_ADAN_Destroyer msg=MessageSet_ADAN_ShipA n=1 pilot=
ADS351 ADAN Count=1 DisableInterval=No Formation_1_only AI_ADAN_Fleet
unit=UN_e106_ADAN_Destroyer msg=MessageSet_ADAN_ShipA n=1 pilot=
ADT306 ADAN Count=1 DisableInterval=No Formation_1_only AI_TraceRoute
unit=UN_e201_ADAN_ISCMissile msg= n=1 pilot=
ADT307 ADAN Count=1 DisableInterval=No Formation_1_only AI_TraceRoute
unit=UN_e201_ADAN_ISCMissile msg= n=1 pilot=
ADT308 ADAN Count=1 DisableInterval=No Formation_1_only AI_TraceRoute
unit=UN_e201_ADAN_ISCMissile msg= n=1 pilot=
TCN000 Neutral Count=1 DisableInterval=No Formation_1_only AI_Structure
unit=UN_bf001_TCAF_SchlosBase msg= n=1 pilot=
TCN001 TCAF Count=2 DisableInterval=No Formation_2_Rhino1 AI_TCAF_RhinoFlight
unit=UN_f001_TCAF_DeltaSaber_T_Player msg=MessageSet_Katana n=1 pilot=Character_Player_Test
unit=UN_f001_TCAF_DeltaSaber_T msg=MessageSet_Ellen n=1 pilot=ELLEN
TCN002 TCAF Count=2 DisableInterval=No Formation_2_Rhino3 AI_TCAF_RhinoLeader
unit=UN_f001_TCAF_DeltaSaber_T msg=MessageSet_Raymond n=1 pilot=RAYMOND
unit=UN_f001_TCAF_DeltaSaber_T msg=MessageSet_Gene n=1 pilot=GENE
TCN003 TCAF Count=4 DisableInterval=No Formation_4_Bird AI_TCAF_BirdFlight
unit=UN_f001_TCAF_DeltaSaber_T msg=MessageSet_Sandra n=1 pilot=SANDRA
unit=UN_f001_TCAF_DeltaSaber_T msg=MessageSet_Billy n=1 pilot=BILLY
unit=UN_f001_TCAF_DeltaSaber_T msg=MessageSet_Antonius n=1 pilot=ANTONIUS
unit=UN_f001_TCAF_DeltaSaber_T msg=MessageSet_Carl n=1 pilot=CARL
TCN004 TCAF Count=1 DisableInterval=No Formation_TCAF_Acropolis04_01 AI_TCAF_Acropolis
unit=UN_f101_TCAF_Acropolis msg=MessageSet_Acropolis n=1 pilot=
TCN131 TCAF Count=1 DisableInterval=No Formation_1_only AI_TraceRoute
unit=UN_f105_TCAF_Cruiser msg= n=1 pilot=
TCN221 TCAF Count=1 DisableInterval=No Formation_TCAF_ArrowHead02_32 AI_TCAF_CraftSquadron
unit=UN_f003_TCAF_ArrowHead msg=MessageSet_TCAF_plA n=8 pilot=
TCN105 TCAF Count=1 DisableInterval=No Formation_TCAF_Destroyer01_02 AI_TCAF_Fleet_Escort
unit=UN_f106_TCAF_Destroyer msg=MessageSet_TCAF_AcropolisGuard n=2 pilot=
TCN132 TCAF Count=1 DisableInterval=No Formation_1_only AI_TCAF_Fleet
unit=UN_f106_TCAF_Destroyer msg=MessageSet_TCAF_ShipA n=1 pilot=
TCN222 TCAF Count=1 DisableInterval=No Formation_TCAF_ArrowHead03_64 AI_TCAF_CraftSquadron
unit=UN_f003_TCAF_ArrowHead msg=MessageSet_TCAF_plB n=8 pilot=
TCN321 TCAF Count=1 DisableInterval=No Formation_TCAF_ArrowHead03_64 AI_TCAF_CraftSquadron
unit=UN_f003_TCAF_ArrowHead msg=MessageSet_TCAF_plA n=8 pilot=
TCN330 TCAF Count=1 DisableInterval=No Formation_1_only AI_TCAF_Fleet
unit=UN_f105_TCAF_Cruiser msg=MessageSet_TCAF_ShipC n=1 pilot=
TCN133 TCAF Count=1 DisableInterval=No Formation_1_only AI_TCAF_Fleet
unit=UN_f105_TCAF_Cruiser msg=MessageSet_TCAF_ShipC n=1 pilot=
TCN223 TCAF Count=1 DisableInterval=No Formation_1_only AI_TCAF_Fleet
unit=UN_f106_TCAF_Destroyer msg=MessageSet_TCAF_ShipA n=1 pilot=
TCN322 TCAF Count=1 DisableInterval=No Formation_TCAF_ArrowHead03_64 AI_TCAF_CraftSquadron
unit=UN_f003_TCAF_ArrowHead msg=MessageSet_TCAF_plB n=8 pilot=
TCN134 TCAF Count=1 DisableInterval=No Formation_1_only AI_TCAF_Fleet
unit=UN_f106_TCAF_Destroyer msg=MessageSet_TCAF_ShipA n=1 pilot=
TCN224 TCAF Count=1 DisableInterval=No Formation_1_only AI_TCAF_Fleet
unit=UN_f106_TCAF_Destroyer msg=MessageSet_TCAF_ShipA n=1 pilot=
TCN323 TCAF Count=1 DisableInterval=No Formation_1_only AI_TCAF_Fleet
unit=UN_f106_TCAF_Destroyer msg=MessageSet_TCAF_ShipA n=1 pilot=
TCN135 TCAF Count=1 DisableInterval=No Formation_1_only AI_TCAF_Fleet
unit=UN_f105_TCAF_Cruiser msg=MessageSet_TCAF_ShipC n=1 pilot=
TCN207 TCAF Count=1 DisableInterval=No Formation_TCAF_Destroyer04_02 AI_TCAF_Fleet
unit=UN_f106_TCAF_Destroyer msg=MessageSet_TCAF_FleetAA n=2 pilot=
TCN225 TCAF Count=1 DisableInterval=No Formation_1_only AI_TCAF_Fleet
unit=UN_f106_TCAF_Destroyer msg=MessageSet_TCAF_ShipA n=1 pilot=
TCN324 TCAF Count=1 DisableInterval=No Formation_1_only AI_TCAF_Fleet
unit=UN_f106_TCAF_Destroyer msg=MessageSet_TCAF_ShipA n=1 pilot=
TCN208 TCAF Count=1 DisableInterval=No Formation_TCAF_Destroyer05_01 AI_TCAF_Fleet
unit=UN_f106_TCAF_Destroyer msg=MessageSet_TCAF_ShipC n=1 pilot=
TCN226 TCAF Count=1 DisableInterval=No Formation_1_only AI_TCAF_Fleet
unit=UN_f105_TCAF_Cruiser msg=MessageSet_TCAF_ShipC n=1 pilot=
TCN325 TCAF Count=1 DisableInterval=No Formation_1_only AI_TCAF_Fleet
unit=UN_f106_TCAF_Destroyer msg=MessageSet_TCAF_ShipA n=1 pilot=
TCN334 TCAF Count=1 DisableInterval=No Formation_1_only AI_TraceRoute
unit=UN_f105_TCAF_Cruiser msg= n=1 pilot=
TCN227 TCAF Count=1 DisableInterval=No Formation_1_only AI_TCAF_Fleet
unit=UN_f105_TCAF_Cruiser msg=MessageSet_TCAF_ShipC n=1 pilot=
TCN326 TCAF Count=1 DisableInterval=No Formation_1_only AI_TCAF_Fleet
unit=UN_f106_TCAF_Destroyer msg=MessageSet_TCAF_ShipA n=1 pilot=
TCN335 TCAF Count=1 DisableInterval=No Formation_TCAF_ArrowHead03_64 AI_TCAF_CraftSquadron
unit=UN_f003_TCAF_ArrowHead msg=MessageSet_TCAF_plA n=8 pilot=
TCS151 TCAF Count=1 DisableInterval=No Formation_1_only AI_TCAF_Fleet
unit=UN_f106_TCAF_Destroyer msg=MessageSet_TCAF_ShipA n=1 pilot=NP_Charon
TCN327 TCAF Count=1 DisableInterval=No Formation_1_only AI_TCAF_Fleet
unit=UN_f106_TCAF_Destroyer msg=MessageSet_TCAF_ShipA n=1 pilot=
TCS251 TCAF Count=1 DisableInterval=No Formation_1_only AI_TCAF_Fleet
unit=UN_f105_TCAF_Cruiser msg=MessageSet_TCAF_ShipB n=1 pilot=NP_Amalthea
TCN328 TCAF Count=1 DisableInterval=No Formation_1_only AI_TCAF_Fleet
unit=UN_f105_TCAF_Cruiser msg=MessageSet_TCAF_ShipC n=1 pilot=
TCT206 TCAF Count=1 DisableInterval=No Formation_1_only AI_TCAF_Fleet
unit=UN_f105_TCAF_Cruiser msg=MessageSet_TCAF_Caliban n=1 pilot=NP_Caliban
TCS252 TCAF Count=1 DisableInterval=No Formation_TCAF_ArrowHead03_64 AI_TCAF_CraftSquadron
unit=UN_f003_TCAF_ArrowHead msg=MessageSet_TCAF_plB n=4 pilot=
TCN329 TCAF Count=1 DisableInterval=No Formation_1_only AI_TCAF_Fleet
unit=UN_f105_TCAF_Cruiser msg=MessageSet_TCAF_ShipC n=1 pilot=
TCN099 TCAF Count=1 DisableInterval=No Formation_1_only AI_TraceRoute
unit=UN_f201_TCAF_Tanker msg=MessageSet_TCAF_Supply n=1 pilot=

40
docs/re/structures/stage-definition-table.md
View File

@@ -97,26 +97,19 @@ scheduler uses: squadrons are gated on an *interval*, and individual records can
opt out of it. That is consistent with the observed behaviour that a mission's
live enemy count rises and falls rather than being fixed.
## 🟡 Container layout
## ✅ Container layout — see [unit-group-table.md](unit-group-table.md)
The file is a multi-section IDXD container, and `pak dump`'s current labels for
it are wrong — it prints the first section's key as "schema".
**Superseded.** An earlier draft of this section described a multi-section
container with a 20-byte `(tag, 0, 0, count, size)` header at `0x708`, 112
records, and a separate squadron-id string base. That reading was wrong on all
three points: the "header" was the file's last 16-byte record followed by a
plain `npool` word, and it failed to parse 9 of the 28 stage tables.
```
0x0000 "IDXD"
0x0004 u32 be = 0x70 = 112 record count
0x0008 112 x 16-byte entries: u32 key, u32 a, u32 b, u32 c
0x0708 next section: u32 tag (0x c5c9f110), 0,0, u32 count=0x6f, u32 size=0x46a,
then the SAME 112 keys again with a narrower payload
...
0x3c1d 112 squadron-ID strings: TCN001.. ADN101.. ADT102.. TCS151.. ADS151..
(prefixes ADN, ADS, ADT, TCN, TCS, TCT)
0x3f26 field-name / value string pool (the vocabulary above)
```
The 112×16 entry array was confirmed by boundary, not assumed: keys are strictly
increasing for exactly 112 entries and the run breaks at `0x708`, where the next
section header sits.
The corrected, uniform layout — header, 16-byte records, a 12-byte property
pool, a string-pool size trailer that checks out against the file length exactly,
and every offset relative to one string base — is documented in
[unit-group-table.md](unit-group-table.md), together with the two corpus-wide
self-checks that validate it (1160/1160 on both).
### ❌ Refuted: the record key is not the squadron-ID name hash
@@ -128,10 +121,9 @@ than a hash, but that is untested.
## What is still open
* The per-record payload: which of the three words per entry selects the
formation, the AI, the side, the count.
* The meaning of the record key.
* **The interval schedule itself.** `DisableInterval` names the mechanism but
the interval *values* have not been located — they may be in the second
section, in `Formation_*.tbl`, or in `EnumSquadron_Test.tbl`.
* S17–S23 stage records.
* **The interval schedule itself.** `DisableInterval` names the mechanism and 31
squadrons set it, but the interval *values* have not been located — next
candidates are `Formation_*.tbl` and `EnumSquadron_Test.tbl`.
* S17–S23 stage records. S17 also has no `UnitGroup_S17.tbl`, so its whole
stage is absent from `GP_MAIN_GAME_E.pak` rather than merely unnamed.

122
docs/re/structures/unit-group-table.md Normal file
View File

@@ -0,0 +1,122 @@
# `stage\UnitGroup_S<NN>.tbl` — the per-stage squadron roster
Status: ✅ container format and field semantics, validated across all 28 stage
tables present on the disc; 🟡 one member field; ❔ the record key and the
arrival-interval *values*.
Reached from the per-stage definition record's `EnumerateSquadron` field — see
[stage-definition-table.md](stage-definition-table.md). Tool:
`tools/re-capture/unitgroup.py` (pure static; runs no emulator).
```
python3 tools/re-capture/unitgroup.py S02 # full roster
python3 tools/re-capture/unitgroup.py --all --check # self-check every stage
```
A committed dump of Stage 02 is at [`../data/unitgroup-s02.txt`](../data/unitgroup-s02.txt).
## ✅ Container
```
0x00 "IDXD"
0x04 u32 nrec
0x08 nrec x 16 record: (key, squadron_off, field_lo, field_hi)
u32 npool
npool x 12 property entry: (tag, name_off | 0xffffffff, value_off)
u32 strsize -- and STR + strsize == filesize, exactly
strsize string pool; [0] = "Enumerate_Squadrons", [0x14] = ""
```
**Every offset in the file is relative to `STR`**, the string-pool base. A
record's fields are pool entries `[field_lo, field_hi)`. An entry with
`name_off == 0xffffffff` is *positional*; otherwise the entry carries its own
field name inline, so the table is self-describing and the tag hash never has to
be inverted.
Exactly one record per file is not a squadron: it is the **`Enumerate_Squadrons`
roster**, whose entries map `key -> squadron id` for every other record. It is
usually the last record but not always — in S05, S06, S07, S08, S09, S11, S13,
S28 and S29 it sits elsewhere, so find it by its missing `Count`, not by
position.
## ✅ A squadron record
Five named fields, always present and always these five:
| field | values |
|---|---|
| `Count` | 1 (×1082), 2 (×50), 3 (×8), 4 (×20) |
| `SideID` | `ADAN` (745), `TCAF` (294), `Neutral` (121) |
| `AIID` | 31 distinct, e.g. `AI_ADAN_CraftSquadron_Veteran`, `AI_TCAF_BirdFlight`, `AI_ADAN_Fleet`, `AI_Structure`, `AI_TraceRoute` |
| `FormationID` | e.g. `Formation_4_Bird`, `Formation_ADAN_Turret07_30`, `Formation_1_only` |
| `DisableInterval` | `No` (1129), **`Yes` (31)** |
…preceded by `Count` **member tuples** of four positional entries:
```
(unit model, message set, n, identity)
```
* unit model — 122 distinct, and they are our XBG7 mesh names:
`UN_f001_TCAF_DeltaSaber_T_Player`, `UN_e010_ADAN_Attacker_S`,
`UN_e007_ADAN_Turret`, `UN_bf001_TCAF_SchlosBase`.
* message set — the squadron's radio chatter, e.g. `MessageSet_Ellen`,
`MessageSet_ADAN_plA`.
* `n` — 🟡 **unidentified**; integer 1…30, dominated by 1 (735) and 9 (197).
It is *not* the `_NN` suffix of `FormationID` (`Formation_ADAN_Turret07_30`
pairs with `n = 9`). Plausibly a spawn or wing size, untested.
* identity — 63 distinct: named pilots (`ELLEN`, `SANDRA`, `RAYMOND`, `YOJI`),
ship nameplates (`NP_Charon`, `NP_Amalthea`, `NP_Olympus`), carrier tags
(`Carrier_ADAN01`), cargo tags (`CARGO_1`), or empty.
Worked example — Stage 02, squadron `TCN004`, the player's own flight:
```
TCN004 TCAF Count=2 DisableInterval=No Formation_2_Rhino1 AI_TCAF_RhinoFlight
unit=UN_f001_TCAF_DeltaSaber_T_Player msg=MessageSet_Katana n=1 id=Character_Player_Test
unit=UN_f001_TCAF_DeltaSaber_T msg=MessageSet_Ellen n=1 id=ELLEN
```
## ✅ Two independent self-checks, both corpus-wide
1. **`len(fields) == Count * 4 + 5`** — holds for **1160 of 1160** squadrons
across all 28 stage tables. This is what pins the member-tuple width at 4 and
the named-field count at 5; it was not assumed.
2. **Roster agreement** — the `Enumerate_Squadrons` record's `key -> id` mapping
agrees with the name each record resolves independently through its own
`squadron_off`, for **1160 of 1160**.
S17 has no `stage\UnitGroup_S17.tbl` in `GP_MAIN_GAME_E.pak`, matching the
missing S17 stage record.
## ❌ Refuted along the way
* **The record key is not the squadron id's name hash.**
`name_hash("TCN001") = 0xd639f1a4`, but the keys run
`0x659aff47, 0x659b0046, 0x669b0047, …` — 0 of 112 match. What the key encodes
is still ❔. Its byte structure (`b0` ramping, `b2` taking small signed values)
looks like a packed tuple, untested.
* **Squadron ids do not use a separate string base.** An earlier reading here
put them at `STR + 0x15` and scored "109 of 111", which looked like a near-fit
but was an artefact of the uniform 7-byte id stride — it silently shifted
every name by three entries (record 0 read as `ADN104` instead of `ADN101`,
and 17 `TC*`-named squadrons came out with `SideID = ADAN`). The roster record
refuted it outright, and with the correct base — plain `STR` — agreement is
111/111. The lesson is that a 98 %-looking score on a self-consistent stride
is not evidence; the roster was an independent oracle and should have been
consulted first.
* **The 20-byte "section header"** described in an earlier draft of
[stage-definition-table.md](stage-definition-table.md) does not exist. What
looked like `(tag, 0, 0, count, size)` was the file's last 16-byte record
followed by the `npool` word. The corrected layout above is uniform across all
28 files, which the earlier reading was not — it failed on 9 of them.
## What is still open
* The meaning of the 4-byte record key.
* The member tuple's third field `n`.
* **The interval values.** `DisableInterval` is a per-squadron *flag* and 31
squadrons set it, but the interval durations, spawn triggers and arrival
positions are not in this file. The next places to look are
`Formation_*.tbl` (the formation geometry and possibly its timing) and
`EnumSquadron_Test.tbl`, both named by the stage record.

141
tools/re-capture/unitgroup.py Normal file
View File

@@ -0,0 +1,141 @@
#!/usr/bin/env python3
"""Decode a stage's squadron roster: hidden/dat `stage\\UnitGroup_S<NN>.tbl`.
Pure static work — reads the extracted disc, runs no emulator.
python3 tools/re-capture/unitgroup.py S02
python3 tools/re-capture/unitgroup.py --all --check
Format (see docs/re/structures/unit-group-table.md):
"IDXD" magic
u32 nrec record count; exactly one record is the
Enumerate_Squadrons roster, not a squadron
nrec x 16 (key, squadron_string_off, field_lo, field_hi)
u32 npool property-entry count
npool x 12 (tag, name_off | 0xffffffff, value_off)
u32 strsize string-pool byte size; STR+strsize == len(file)
strsize bytes string pool; [0] is "Enumerate_Squadrons",
[0x14] is the empty string. Every offset in
the file -- squadron ids included -- is relative
to STR.
A squadron's field list is `Count` member tuples of four positional entries
(unit model, message set, a number, pilot character) followed by five named
entries: Count, SideID, AIID, FormationID, DisableInterval. So
len(fields) == Count * 4 + 5
which holds for 1160 of 1160 squadrons across the 28 stage tables present in
GP_MAIN_GAME_E.pak.
"""
import argparse, glob, os, struct, sys, zlib
MODULUS, RECIP = 0x00FFF9D7, 0x80031493
def _rotl(v, n): return ((v << n) | (v >> (32 - n))) & 0xFFFFFFFF
def name_hash(s):
bs = bytearray(s.encode('latin-1', 'replace'))
for i, x in enumerate(bs):
if 65 <= x <= 90: bs[i] = x + 0x20
a = b = 0
for byte in bs:
c = ((byte - 256) if byte > 127 else byte) & 0xFFFFFFFF
a = ((_rotl(a, 8) & 0xFFFFFF00) + c) & 0xFFFFFFFF
b = (b + c) & 0xFFFFFFFF
q = _rotl(((a * RECIP) >> 32) & 0xFFFFFFFF, 9) & 0x1FF
a = (a - (q * MODULUS)) & 0xFFFFFFFF
return (((b << 24) & 0xFF000000) | (a & 0x00FFFFFF)) & 0xFFFFFFFF
def read_entry(pak, h):
idx = open(pak, 'rb').read()
base = pak[:-4]
data = b''.join(open(p, 'rb').read() for p in sorted(glob.glob(base + '.p[0-9][0-9]')))
for i in range(struct.unpack_from('>I', idx, 4)[0]):
k, off, csize = struct.unpack_from('>III', idx, 16 + i * 12)
if k == h:
s = data[off:off + csize]
return zlib.decompress(s[10:]) if s[:2] == b'Z1' else s
raise KeyError('%#010x not in %s' % (h, os.path.basename(pak)))
def parse(b):
u = lambda o: struct.unpack_from('>I', b, o)[0]
if b[:4] != b'IDXD': raise ValueError('bad magic')
nrec = u(0x04)
npool_off = 0x08 + nrec * 16
npool = u(npool_off)
pool = npool_off + 4
strsize_off = pool + npool * 12
STR = strsize_off + 4
if STR + u(strsize_off) != len(b):
raise ValueError('string-pool size trailer mismatch')
def s(o):
return b[o:b.index(b'\x00', o)].decode('latin-1')
out = []
for i in range(nrec):
key, sq, lo, hi = (u(0x08 + i * 16 + j * 4) for j in range(4))
fields = []
for j in range(hi - lo):
o = pool + (lo + j) * 12
tag, name, val = u(o), u(o + 4), u(o + 8)
fields.append((tag, s(STR + name) if name != 0xFFFFFFFF else None, s(STR + val)))
out.append({'key': key, 'squadron': s(STR + sq), 'fields': fields})
return out
def named(rec): return {n: v for _, n, v in rec['fields'] if n is not None}
def members(rec):
pos = [v for _, n, v in rec['fields'] if n is None]
return [tuple(pos[i:i + 4]) for i in range(0, len(pos), 4)]
def is_roster(rec): return 'Count' not in named(rec)
def roster(recs):
"""One record enumerates key -> squadron id; use it to cross-check the rest.
It is usually the last record but not always (S05, S28, ... put it earlier),
so find it by its lack of a Count field rather than by position.
"""
for r in recs:
if is_roster(r):
return {tag: n for tag, n, _ in r['fields']}
return {}
def load_stage(pak, stage):
return parse(read_entry(pak, name_hash('stage\\UnitGroup_%s.tbl' % stage)))
def main():
ap = argparse.ArgumentParser()
ap.add_argument('stage', nargs='?', default='S02')
ap.add_argument('--pak', default='/work/sylph_extract/dat/GP_MAIN_GAME_E.pak')
ap.add_argument('--all', action='store_true', help='every stage S01..S29')
ap.add_argument('--check', action='store_true', help='only report the Count*4+5 identity')
a = ap.parse_args()
stages = ['S%02d' % n for n in range(1, 30)] if a.all else [a.stage]
held = total = 0
for st in stages:
try:
recs = load_stage(a.pak, st)
except KeyError as e:
print('%s: absent (%s)' % (st, e)); continue
squads = [r for r in recs if not is_roster(r)]
ok = sum(1 for r in squads if len(r['fields']) == int(named(r)['Count']) * 4 + 5)
rst = roster(recs)
agree = sum(1 for r in squads if rst.get(r['key']) == r['squadron'])
held += ok; total += len(squads)
print('%s: %3d squadrons, %4d members, Count*4+5 %d/%d, roster agrees %d/%d'
% (st, len(squads), sum(len(members(r)) for r in squads), ok, len(squads),
agree, len(squads)))
if a.check: continue
for r in squads:
d = named(r)
print(' %-10s %-8s Count=%-3s DisableInterval=%-4s %-30s %s'
% (r['squadron'], d['SideID'], d['Count'], d['DisableInterval'],
d['FormationID'], d['AIID']))
for m in members(r):
print(' unit=%-34s msg=%-24s n=%-4s pilot=%s' % m)
if len(stages) > 1:
print('\nTOTAL %d squadrons, identity held %d/%d' % (total, held, total))
return 0 if held == total else 1
if __name__ == '__main__':
sys.exit(main())