diff --git a/docs/re/BACKLOG.md b/docs/re/BACKLOG.md index b7b5d74e..70b4b0e5 100644 --- a/docs/re/BACKLOG.md +++ b/docs/re/BACKLOG.md @@ -4029,3 +4029,67 @@ cinematic — before crashing at the same `0x82307128`. The crash is intermitten in **where** it fires, not whether, so there is no menu route around it. See [`title-crash-stl-tree.md`](title-crash-stl-tree.md) for the end-to-end measurement and for what has been ruled out (`--mem_watch=false`, twice). + +--- + +## ✅ 2026-08-27 — `ORDOR_SQUADRON_EXTENDED` opened: the reactive-chatter rule table + +The biggest unopened record name in the 3 496-name census (**864 records**) is a +record in the **chatter rule table**: 144 objects per language pack × 6 packs. +Write-up: [`structures/preset-message-rules.md`](structures/preset-message-rules.md); +artefact `data/preset-messages.txt`; regenerator +`tools/re-capture/preset_messages.py`. + +**Docs checked before claiming anything** — `squadron-orders.md`, +`sound-cue-table.md`, `cutscene-message-table.md`, `ixud-localised-text.md`, +`isl-message-dialogue-link.md`, `movie-subtitles.md`, `mission-phase-deployment.md`, +`mission-phase-membership.md`. Two of them own parts of this: + +* `ixud-localised-text.md` owns "`PresetMessage_*` is reactive combat chatter + keyed by speaker class", and `mission-phase-deployment.md` owns the route in + (`UnitMessageSet_S.tbl` → `PresetMessage_Phase1/2/3`). **What neither says** + is what is inside the table. +* `cutscene-message-table.md` owns the `Generic`+`Message_NNN` layout for the 32 + cutscene objects per pack. **What it does not say** is that the same layout + carries **137 more** objects per pack for in-mission chatter. Control: of 273 + `PresetMessage_*` names that hit an IDXD entry, 136 are rule tables and 137 are + message tables — the prefix names two different shapes. + +✅ **One schema, no variants.** All 9 216 event records (144 × 64) carry the same +seven named fields — `EffectiveTime, Interval, Priority, Probability, +MessageCount, IntervalFluctuation, Pattern` — then positional `(message id, +Yes/No)` pairs. **`MessageCount · 2 == positional count` in 9 216/9 216, zero +mismatches**, which is what pins the pairing. + +✅ **Named twice, independently, with the same answer.** Harvested +`PresetMessage_*` strings hashed under `message\` → 136/144; the +`PresetMessage_Phase1/2/3` declarations of the 22 `UnitMessageSet_S.tbl` +tables → 136/144; **the two routes name the same set** (set identity, not a +matching count). + +✅ **Joins to the settled cue table**: 2 388 of 2 405 distinct message ids +resolve (99.3 %) through `sound-cue-table.md`'s index. + +⚠️ **A trap that doc had already paid for, and I walked into it anyway.** +Rewriting `MSG_`→`VOICE_` unconditionally scores 2 353/2 405, and the 52 misses +looked like a finding ("lines with no recording"). 44 of them were a naming +artefact: `sound-cue-table.md` states that a name already carrying `VOICE_` keeps +the one it has. Applying the documented exception gives 2 388/2 405. Reading the +owning doc caught this *after* the first write-up was drafted — the earlier grep +found the ownership, but the rule inside it still had to be read. + +✅ **Correction to `squadron-orders.md`** — it rendered the `0x820AEEB0` enum as +twelve `ORDER_*` names. The executable's own strings misspell all four SQUADRON +entries as **`ORDOR_*`**, and the disc data carries the identical misspelling, +which is how the two sides join. That doc's 🔴 "this is a UI/comms axis, not the +order classes" now has its explanation: the twelve are bark *events*. + +❔ **Not settled.** What the seven numeric fields mean (schema only, not +measured against the running game); the `Yes`/`No` element of each pair; which of +the 49 `Sperkers` speakers is a wingman (the "13 tables voice every player-facing +event" partition is a count, not a verified roster). 🟡 **8 of the 144 rule +tables are named by neither route** — nothing on the disc declares them. +`UnitMessageSet_S.tbl` exists for 22 of the 28 stages and the six missing are +exactly **S18–S23, the tutorials** (same set as the missing `AIParams_SNN.tbl`), +so tutorial chatter is the plausible reading — but there is no tutorial +`UnitMessageSet` to confirm it, so it stays a reading. diff --git a/docs/re/INDEX.md b/docs/re/INDEX.md index 94666008..c706b1b2 100644 --- a/docs/re/INDEX.md +++ b/docs/re/INDEX.md @@ -98,6 +98,7 @@ files, which is how the same ground got covered twice. | [`structures/sound-pak-contents.md`](structures/sound-pak-contents.md) | Census of `sound.pak`, and the limit of the leading-region rule | ✅ CONFIRMED, 5 135/5 135 names hash into the TOC, **9 519/9 519** entries accounted for, and a full 4 114-bank manifest (408.3 min of audio) computed from PsuedoBytesPerSec without decoding; ⚠️ leading-region rule holds for 1 571/4 382 eng and 0/5 100 jpn | | [`structures/sound-cue-table.md`](structures/sound-cue-table.md) | The cue index in `tables.pak` — message id -> cue -> sound id -> `.slb` bank | ✅ CONFIRMED, 1 326/1 338 script message ids bind to a bank; SOUNDS and FILES agree on the same 12 absentees, 0 orphan files | | [`structures/cutscene-message-table.md`](structures/cutscene-message-table.md) | Cutscene dialogue — speaker, portrait, on-screen seconds, audio cue per page | ✅ CONFIRMED, field count = 9·PageCount+2 for all 7 PageCounts, 1 252/1 252 caption keys match, 138 ids close both ways | +| [`structures/preset-message-rules.md`](structures/preset-message-rules.md) | The reactive-chatter rule table — 64 events, the (message, Yes/No) pair payload, the `Sperkers` roster | ✅ CONFIRMED, MessageCount·2 == positional count in 9 216/9 216; two independent naming routes agree on the same 136/144; 2 388/2 405 message ids bind to a cue | | [`structures/isl-mission-timer.md`](structures/isl-mission-timer.md) | The mission timer — two sequential countdowns, not a limit and a warning | ✅ CONFIRMED — read from `sub_822639B8`; the threshold reading is | | [`structures/archive-naming.md`](structures/archive-naming.md) | Which archives the disc can name — 100% for menu paks, **0.0%** for the six 2D paks and `GP_READY_ROOM` | ✅ CONFIRMED — 6027 names × 16 prefixes, bimodal | | [`structures/hud-config.md`](structures/hud-config.md) | The in-game HUD configuration — 16 records, 419 asset paths, and a per-stage `ResourceTable` | ✅ CONFIRMED — 0 of 419 config paths resolve as pak entries | diff --git a/docs/re/data/preset-messages.txt b/docs/re/data/preset-messages.txt new file mode 100644 index 00000000..d4db6eb6 --- /dev/null +++ b/docs/re/data/preset-messages.txt @@ -0,0 +1,309 @@ +# Reactive combat-chatter rule tables (regenerated by preset_messages.py) + +rule tables in GP_MAIN_GAME_E.pak : 144 of 1004 IDXD entries +per language pack : {'GP_MAIN_GAME_D.pak': 144, 'GP_MAIN_GAME_E.pak': 144, 'GP_MAIN_GAME_F.pak': 144, 'GP_MAIN_GAME_I.pak': 144, 'GP_MAIN_GAME_J.pak': 144, 'GP_MAIN_GAME_S.pak': 144} +total across the disc : 864 +distinct record-name tuples : 1 +records per table : 65 (64 events + Sperkers) + +distinct named-field schemas : 1 + x9216 EffectiveTime, Interval, Priority, Probability, MessageCount, IntervalFluctuation, Pattern +MessageCount*2 == positional count: 9216/9216 (mismatches 0) +second-of-pair values : {'No': 8940, 'Yes': 388} + +the 64 event records, and how many of the 144 tables give each one lines: + ATTACK_IS_AVOIDED 114 + ATTACK_SUCCEEDED 114 + AVOIDED_BY_PLAYER 50 + AVOIDED_PLAYER_ATTACK 15 + AVOID_ATTACK 118 + COUNTER 114 + COUNTER_TO_PLAYER 45 + DAMAGED 124 + DAMAGED_BY_PLAYER_50_30 0 + DAMAGED_BY_PLAYER_GREATER_50 128 + DAMAGED_BY_PLAYER_LESS_30 127 + DAMAGED_TO_PLAYER 50 + EXTENDED 95 + EXTENDED_BY_PLAYER 86 + FOUND 111 + FOUND_PLAYER 45 + HP_LESS_10 121 + HP_LESS_30 0 + HP_LESS_50 121 + KILLED 82 + KILLED_BY_PLAYER 96 + KILL_PLAYER 2 + KILL_UNIT 114 + LOCKON_1 13 + LOCKON_10 13 + LOCKON_11 13 + LOCKON_12 13 + LOCKON_2 13 + LOCKON_3 13 + LOCKON_4 13 + LOCKON_5 13 + LOCKON_6 13 + LOCKON_7 13 + LOCKON_8 13 + LOCKON_9 13 + LOCKON_DOWN 13 + LOCKON_UP 13 + LOST_BRIDGE_BY_PLAYER 18 + LOST_SHIELD_BY_PLAYER 18 + LOST_THRUSTER_BY_PLAYER 18 + LOST_TURRET_BY_PLAYER 18 + ORDER_SQUAD_ATTACK 7 + ORDER_SQUAD_COVER 7 + ORDER_SQUAD_EXTENDED 7 + ORDER_SQUAD_FORMATION 7 + ORDER_WINGMAN_ATTACK 13 + ORDER_WINGMAN_COVER 13 + ORDER_WINGMAN_EXTENDED 13 + ORDER_WINGMAN_FORMATION 13 + ORDOR_SQUADRON_ATTACK 7 + ORDOR_SQUADRON_COVER 7 + ORDOR_SQUADRON_EXTENDED 7 + ORDOR_SQUADRON_FORMATION 7 + PLAYER_AMMO_LESS_00 13 + PLAYER_AMMO_LESS_10 13 + PLAYER_AMMO_LESS_40 13 + PLAYER_HP_LESS_10 13 + PLAYER_HP_LESS_30 13 + PLAYER_HP_LESS_50 13 + RETURNED 0 + SHIELD_DOWNED 61 + SHIELD_DOWNED_BY_PLAYER 46 + TALK 0 + TM_TO_PLAYER 0 + +Sperkers roster: 49 distinct speakers; sizes {0: 3, 1: 84, 2: 37, 3: 4, 4: 6, 5: 1, 6: 1, 7: 2, 8: 1, 9: 2, 10: 2, 11: 1} + CharacterELLEN 19 + CharacterCARL 15 + CharacterGENE 15 + CharacterSANDRA 13 + CharacterKATANA 13 + CharacterBILLY 13 + CharacterANTONIUS 13 + CharacterRAYMOND 12 + CharacterNATALIE 10 + CharacterYOJI 10 + CharacterTCAFOPA 9 + CharacterTCAFOPB 8 + CharacterCRICHTON 8 + CharacterTCAFA 8 + CharacterTCAFB 8 + CharacterADANPLA 6 + CharacterADANPLB 6 + CharacterADANPLC 6 + CharacterTCAFC 6 + CharacterADANOPB 5 + CharacterADANOPA 5 + CharacterMARGRAS 5 + CharacterADANPLD 4 + CharacterADANPLE 4 + CharacterADANPLF 4 + CharacterADANA 4 + CharacterADANB 4 + CharacterADANC 4 + CharacterSHIPSQUIRE1 4 + CharacterWAYNE 3 + CharacterHOWARD 3 + CharacterSHIPLADY1 3 + CharacterIAN 3 + CharacterRUSSELL 3 + CharacterVIOLETA 3 + CharacterTCAFPLA 2 + CharacterTCAFPLB 2 + CharacterTASKENT 2 + CharacterHIGGINS 1 + CharacterADANCOMMANDER 1 + CharacterTCAFSUPPLY 1 + CharacterVIEIRA 1 + CharacterTCAFMIRANDA 1 + CharacterADANETERNAL 1 + CharacterADANMASSIVE 1 + CharacterTCAFOLYMPUS 1 + CharacterMARGRAS2 1 + CharacterADANCAMAEL 1 + CharacterADANMETATRON 1 + +distinct message ids : 2405 +resolved in tables.pak SOUNDS : 2388 (99.3%) +unresolved : 17 + by family: {'ADAN': 4, 'ADPL': 3, 'RHIN': 1, 'VOICE': 9} + MSG_ADAN_569 + MSG_ADAN_668 + MSG_ADAN_788 + MSG_ADAN_789 + MSG_ADPL_045 + MSG_ADPL_046 + MSG_ADPL_047 + MSG_RHIN_341 + MSG_VOICE_C_286 + MSG_VOICE_D_400 + MSG_VOICE_D_419 + MSG_VOICE_D_443 + MSG_VOICE_D_444 + MSG_VOICE_D_445 + MSG_VOICE_D_446 + MSG_VOICE_D_447 + MSG_VOICE_D_449 + +naming route 1 (any harvested PresetMessage_ string): 136/144 +naming route 2 (UnitMessageSet_S* declarations) : 136/144 +the two routes agree on the same set : True +UnitMessageSet_S* tables present: 22 of 28 (S01 S02 S03 S04 S05 S06 S07 S08 S09 S10 S11 S12 S13 S14 S15 S16 S24 S25 S26 S27 S28 S29) +missing: S18 S19 S20 S21 S22 S23 +distinct declared PresetMessage tables: 137 + +CONTROL — PresetMessage_* names that hit an IDXD entry but are NOT rule tables: 137 + their record-name families: {('Generic', 'Message_'): 134, ('Generic',): 3} + +rule tables, by name: + 0xde14f001 message\PresetMessage_ADAN_Camael.tbl + 0x31092781 message\PresetMessage_ADAN_Commander.tbl + 0x660be19b message\PresetMessage_ADAN_Eternal.tbl + 0x4de6f4ce message\PresetMessage_ADAN_FleetAA.tbl + 0x4eed1dce message\PresetMessage_ADAN_FleetAB.tbl + 0x4e101fed message\PresetMessage_ADAN_FleetBA.tbl + 0x4f1648ed message\PresetMessage_ADAN_FleetBB.tbl + 0x4f3944e3 message\PresetMessage_ADAN_FleetCA.tbl + 0x503f6de3 message\PresetMessage_ADAN_FleetCB.tbl + 0x7d898a7a message\PresetMessage_ADAN_Glance_S07-2.tbl + 0x733ffa0f message\PresetMessage_ADAN_Massive.tbl + 0xe5b45769 message\PresetMessage_ADAN_Metatron.tbl + 0x50a3be19 message\PresetMessage_ADAN_ShipA_01.tbl + 0xe41a87df message\PresetMessage_ADAN_ShipA_02_S12-1.tbl + 0x5197392f message\PresetMessage_ADAN_ShipB_01.tbl + 0xe5767fd1 message\PresetMessage_ADAN_ShipB_01_S03-2.tbl + 0x528ab445 message\PresetMessage_ADAN_ShipC_01.tbl + 0xe991cee2 message\PresetMessage_ADAN_ShipC_02_S15-1.tbl + 0xabb657f0 message\PresetMessage_ADAN_plAB_S01-1.tbl + 0xac95b821 message\PresetMessage_ADAN_plAC_S01-1.tbl + 0x494882e2 message\PresetMessage_ADAN_plA_S01-1.tbl + 0xadf51dd8 message\PresetMessage_ADAN_plBC_S01-1.tbl + 0x4a27e313 message\PresetMessage_ADAN_plB_S01-1.tbl + 0x4b074344 message\PresetMessage_ADAN_plC_S01-1.tbl + 0x1ced0fc8 message\PresetMessage_ADAN_plDEF_S15-1.tbl + 0xb5cae6f3 message\PresetMessage_ADAN_plDE_S03-3.tbl + 0xb6aa4724 message\PresetMessage_ADAN_plDF_S03-3.tbl + 0x503eda97 message\PresetMessage_ADAN_plD_S03-3.tbl + 0xb709b304 message\PresetMessage_ADAN_plEF_S03-3.tbl + 0x511e3ac8 message\PresetMessage_ADAN_plE_S03-3.tbl + 0x52fd94d0 message\PresetMessage_ADAN_plF_S03-3.tbl + 0xa6e5b0b5 message\PresetMessage_Acropolis_01_S02-1.tbl + 0xac82d3e4 message\PresetMessage_Acropolis_02_S07-1.tbl + 0xa871f99c message\PresetMessage_Acropolis_03_S11-1.tbl + 0x4bf26dc4 message\PresetMessage_Antonius_01_S01-2.tbl + 0x4cf19786 message\PresetMessage_Antonius_02_S02-1.tbl + 0x50034271 message\PresetMessage_Antonius_03_S03-3.tbl + 0x536e1ae6 message\PresetMessage_Antonius_04_S07-1.tbl + 0x4f5d409e message\PresetMessage_Antonius_05_S11-1.tbl + 0xfad127f0 message\PresetMessage_Antonius_06_S28.tbl + 0xf66565a9 message\PresetMessage_Billy_01_S01-2.tbl + 0xf7648f6b message\PresetMessage_Billy_02_S02-1.tbl + 0xfb76342d message\PresetMessage_Billy_03_S03-3.tbl + 0xfee10ca2 message\PresetMessage_Billy_04_S07-1.tbl + 0xfad0325a message\PresetMessage_Billy_05_S11-1.tbl + 0xa51488ed message\PresetMessage_Billy_06_S28.tbl + 0x7ce33a27 message\PresetMessage_Carl_01_S01-2.tbl + 0x7de263e9 message\PresetMessage_Carl_02_S02-1.tbl + 0x81f408ab message\PresetMessage_Carl_03_S03-3.tbl + 0x845ee749 message\PresetMessage_Carl_04_S07-1.tbl + 0x804e0d01 message\PresetMessage_Carl_05_S11-1.tbl + 0x2be8d24f message\PresetMessage_Carl_06_S28.tbl + 0xe9b7a641 message\PresetMessage_Ellen_01_S01-1.tbl + 0xebbcf903 message\PresetMessage_Ellen_02_S02-1.tbl + 0xefce9dc5 message\PresetMessage_Ellen_03_S03-3.tbl + 0xf2397c63 message\PresetMessage_Ellen_04_S07-1.tbl + 0xf77113b6 message\PresetMessage_Ellen_05_S09-3.tbl + 0x99745ef7 message\PresetMessage_Ellen_06_S28.tbl + 0x79eb166f message\PresetMessage_Gene_01_S02-1.tbl + 0x7dfcbb31 message\PresetMessage_Gene_02_S03-3.tbl + 0x806799cf message\PresetMessage_Gene_03_S07-1.tbl + 0x859f3122 message\PresetMessage_Gene_04_S09-3.tbl + 0x82d412fc message\PresetMessage_Gene_05_S15-2.tbl + 0x6300daef message\PresetMessage_Howard_01_S05-2.tbl + 0x640004b1 message\PresetMessage_Howard_02_S06-1.tbl + 0x6361021c message\PresetMessage_Howard_03_S13-1.tbl + 0x49ced33f message\PresetMessage_Katana_01_S01-1.tbl + 0x4bb45c70 message\PresetMessage_Katana_02_S01-2.tbl + 0x4cb38632 message\PresetMessage_Katana_03_S02-1.tbl + 0x4e990f63 message\PresetMessage_Katana_04_S02-2.tbl + 0x51a48b25 message\PresetMessage_Katana_05_S03-3.tbl + 0x540f69c3 message\PresetMessage_Katana_06_S07-1.tbl + 0x5840d816 message\PresetMessage_Katana_07_S09-2.tbl + 0x5a266147 message\PresetMessage_Katana_08_S09-3.tbl + 0x51975723 message\PresetMessage_Katana_09_S10-1.tbl + 0x4a425a04 message\PresetMessage_Katana_10_S11-1.tbl + 0x4c47acc6 message\PresetMessage_Katana_11_S12-1.tbl + 0x4e4cff88 message\PresetMessage_Katana_12_S13-1.tbl + 0x517844db message\PresetMessage_Katana_13_S15-1.tbl + 0xf93b44ee message\PresetMessage_Katana_15_S28.tbl + 0x632e5d87 message\PresetMessage_Lady1.tbl + 0x64348687 message\PresetMessage_Lady2.tbl + 0x653aaf87 message\PresetMessage_Lady3.tbl + 0xca38072d message\PresetMessage_Margras_01_S03-3.tbl + 0xcb3730ef message\PresetMessage_Margras_02_S04-2.tbl + 0xc86c12c9 message\PresetMessage_Margras_03_S10-1.tbl + 0xcfc8c66f message\PresetMessage_Margras_05_S14-2.tbl + 0xd0c7f031 message\PresetMessage_Margras_06_S15-1.tbl + 0xd532e9aa message\PresetMessage_Raymond_02_S02-1.tbl + 0xd9448e6c message\PresetMessage_Raymond_03_S03-3.tbl + 0x81791b91 message\PresetMessage_Raymond_04_S28.tbl + 0x131f8802 message\PresetMessage_Raymond_S01-1.tbl + 0x53b2e415 message\PresetMessage_Sandra_01_S01-2.tbl + 0x54b20dd7 message\PresetMessage_Sandra_02_S02-1.tbl + 0x58c3b299 message\PresetMessage_Sandra_03_S03-3.tbl + 0x5b2e9137 message\PresetMessage_Sandra_04_S07-1.tbl + 0x571db6ef message\PresetMessage_Sandra_05_S11-1.tbl + 0x02351472 message\PresetMessage_Sandra_06_S28.tbl + 0x52223719 message\PresetMessage_Squire1.tbl + 0x53286019 message\PresetMessage_Squire2.tbl + 0x542e8919 message\PresetMessage_Squire3.tbl + 0x5534b219 message\PresetMessage_Squire4.tbl + 0x9bfa033e message\PresetMessage_TCAF_17thFleetAA.tbl + 0x9c003267 message\PresetMessage_TCAF_17thFleetAB.tbl + 0x9c232e5d message\PresetMessage_TCAF_17thFleetBA.tbl + 0xf021197c message\PresetMessage_TCAF_17thFleetShipC.tbl + 0x64ecd655 message\PresetMessage_TCAF_AcropolisGuard.tbl + 0x4f33ea2e message\PresetMessage_TCAF_Caliban.tbl + 0x57a53d3b message\PresetMessage_TCAF_FleetAA.tbl + 0x58ab663b message\PresetMessage_TCAF_FleetAB.tbl + 0x58ce6231 message\PresetMessage_TCAF_FleetBA.tbl + 0x59f78727 message\PresetMessage_TCAF_FleetCA.tbl + 0x5afdb027 message\PresetMessage_TCAF_FleetCB.tbl + 0xa6b0cfde message\PresetMessage_TCAF_Janus.tbl + 0x616c4182 message\PresetMessage_TCAF_Miranda.tbl + 0x9ec12cb7 message\PresetMessage_TCAF_Olympus.tbl + 0x6a18ba6a message\PresetMessage_TCAF_Pandora.tbl + 0x9aafd76a message\PresetMessage_TCAF_ShipA.tbl + 0x9bb6006a message\PresetMessage_TCAF_ShipB.tbl + 0x9cbc296a message\PresetMessage_TCAF_ShipC.tbl + 0x326328f2 message\PresetMessage_TCAF_Supply.tbl + 0x22beec7a message\PresetMessage_TCAF_Vinson.tbl + 0xb704e242 message\PresetMessage_TCAF_plAB_S02-2.tbl + 0x554cc1d2 message\PresetMessage_TCAF_plA_S02-2.tbl + 0x562c2203 message\PresetMessage_TCAF_plB_S02-2.tbl + 0xd74c69b3 message\PresetMessage_Taskent_01_S03-3.tbl + 0xd84b9375 message\PresetMessage_Taskent_02_S04-2.tbl + 0xd4edb4dc message\PresetMessage_Violeta_01_S08-1.tbl + 0xd7f9309e message\PresetMessage_Violeta_02_S09-2.tbl + 0xd0964834 message\PresetMessage_Violeta_03_S11-1.tbl + 0x027f9fba message\PresetMessage_Wayne_01_S05-2.tbl + 0x037ec97c message\PresetMessage_Wayne_02_S06-1.tbl + 0x02dfc6e7 message\PresetMessage_Wayne_03_S13-1.tbl + 0x9a44099c message\PresetMessage_Yoji_01_S06-2.tbl + 0x9b43335e message\PresetMessage_Yoji_02_S07-1.tbl + 0xa07acab1 message\PresetMessage_Yoji_03_S09-3.tbl + 0x5483c09d + 0x59d48b31 + 0x9d29575d + 0x9d4c5353 + 0x9e527c53 + 0xcde33d3e + 0xee14c77c + 0xef1af07c + +unnamed rule tables: 8 diff --git a/docs/re/structures/preset-message-rules.md b/docs/re/structures/preset-message-rules.md new file mode 100644 index 00000000..086132eb --- /dev/null +++ b/docs/re/structures/preset-message-rules.md @@ -0,0 +1,167 @@ +# The reactive-chatter rule table + +**✅ Settled 2026-08-27** (static; regenerator +`tools/re-capture/preset_messages.py`, artefact +[`data/preset-messages.txt`](../data/preset-messages.txt)). + +The corpus already knew *that* `PresetMessage_*` is reactive combat chatter — +[`ixud-localised-text.md`](ixud-localised-text.md) names the IXUD family +("the bulk: reactive combat chatter keyed by speaker class") and +[`mission-phase-deployment.md`](../mission-phase-deployment.md) found the link +that reaches it (`message\UnitMessageSet_S.tbl` gives every crew a +`PresetMessage_Phase1/2/3`). What neither says is **what is inside the table** — +which events exist, what governs when a line fires, and how a line reaches a +recording. That is this document. + +## Two files per set + +A chatter set is a **pair** of IDXD objects in each `dat/GP_MAIN_GAME_.pak`: + +| name | role | +|---|---| +| `message\PresetMessage_.tbl` | the **rule** table — this document | +| `message\PresetMessage__msg.tbl` | the **message** table | + +The `_msg` half is *not* new: it is exactly the record layout +[`cutscene-message-table.md`](cutscene-message-table.md) settled for +`MSG_DEMO_*` — a `Generic` count record plus `Message_NNN` records with `ID` / +`PageCount` and 9 positional fields per page. What that document does not say is +that the same layout carries **137 more objects per language pack** for +in-mission chatter, against the 32 cutscene ones it counted. The control run +confirms the split cleanly: of the 273 `PresetMessage_*` names that hit an IDXD +entry, **136 are rule tables and 137 are `Generic`+`Message_NNN` message +tables** (3 of those are `Generic`-only, i.e. empty). + +## The rule table + +**144 per language pack, 864 across the six `GP_MAIN_GAME_.pak`** — one +identical count in each, and `ORDOR_SQUADRON_EXTENDED` (the misspelling is the +game's, see below) is the largest single record name on the disc for exactly +that reason. + +Every one of the 144 has the **same 65 records**: 64 event records plus a +`Sperkers` (sic) roster. One record-name tuple, no variants. + +### Event records + +All 9 216 event records (144 × 64) share **one** named-field schema, with no +exceptions: + +``` +EffectiveTime Interval Priority Probability MessageCount +IntervalFluctuation Pattern +``` + +After the seven named fields come the record's positional payload — +[a record's unnamed positional fields are its payload] — in **pairs**: + + (message id, Yes/No) e.g. ('MSG_BIRD_020', 'No') + +`MessageCount` is the number of pairs, and that identity is **exact**: +`MessageCount · 2 == positional-field count` in **9 216 of 9 216** records, zero +mismatches. That is what pins the pairing. The second element of each pair is +`No` 8 940 times and `Yes` 388 times; its meaning is not settled. + +### The 64 events + +The full census with per-event table counts is in the artefact. The shape of +that census is itself informative: + +* **Player-facing events are voiced only by the wingman tables.** Every + `LOCKON_1..12`, `LOCKON_UP`, `LOCKON_DOWN`, `PLAYER_HP_LESS_10/30/50` and + `PLAYER_AMMO_LESS_00/10/40` record carries lines in **exactly 13** of the 144 + tables — the same 13 as `ORDER_WINGMAN_{FORMATION,ATTACK,COVER,EXTENDED}`. +* The squad/squadron order events carry lines in **7** tables each. +* Generic combat events are near-universal: `DAMAGED_BY_PLAYER_GREATER_50` 128, + `DAMAGED` 124, `HP_LESS_10` and `HP_LESS_50` 121 each. +* **Five events are declared everywhere and populated nowhere** — + `DAMAGED_BY_PLAYER_50_30`, `HP_LESS_30`, `RETURNED`, `TALK`, `TM_TO_PLAYER` + have `MessageCount 0` in all 144 tables. They are slots the shipped data never + uses. + +Note the near-miss pairs that are *not* redundant: `HP_LESS_*` (the speaker's +own hull) versus `PLAYER_HP_LESS_*`, and `AVOIDED_BY_PLAYER` versus +`AVOIDED_PLAYER_ATTACK`. The `_PLAYER` half of each pair is the smaller +population. + +### `Sperkers` + +The last record is a roster of speaker names as **field names** with empty +values — `CharacterELLEN`, `CharacterADANCAMAEL`, … **49 distinct speakers** +across the 144 tables. Most tables name one speaker (84 of 144); the rest name +2–11, and three name none. + +## The join to audio is the settled cue table + +The pair's first element is a message id, and it resolves through the cue index +[`sound-cue-table.md`](sound-cue-table.md) already documents — `MSG_` becomes +`VOICE_`, then `SOUNDS` in `dat/tables.pak` gives the numeric id and `FILES` the +`.slb` bank. + +**2 388 of 2 405 distinct message ids resolve (99.3 %).** The 17 that do not +are characterised, not swept up: 8 are ordinary four-letter-family ids +(`MSG_ADAN_569/668/788/789`, `MSG_ADPL_045/046/047`, `MSG_RHIN_341`) and 9 are +`MSG_VOICE_*`, seven of them the consecutive run `MSG_VOICE_D_443`…`_449`. A +consecutive run with no recording reads as content cut late. All 17 are listed +in the artefact. + +⚠️ **The wrinkle in that doc is real and I walked into it.** Rewriting `MSG_` to +`VOICE_` unconditionally gives `VOICE_VOICE_D_443`, which resolved only 2 353 of +2 405 and would have been written up as "52 lines with no recording", with 44 of +them a naming artefact. `sound-cue-table.md` states the exception explicitly — +a name that already carries `VOICE_` keeps the one it has — and applying it +moves the count to 2 388. Recorded here because the same trap is one line of +code away in any port. + +## Naming, by two independent routes that agree + +The rule tables are pak entries keyed by `name_hash`, so they had to be named +before any of this could be read. + +1. **Harvested strings** — every `PresetMessage_*` string appearing anywhere in + any IDXD object on the disc, hashed under the `message\` prefix: **136/144**. +2. **Declarations** — the `PresetMessage_Phase1/2/3` values of the 22 + `message\UnitMessageSet_S.tbl` tables: **136/144**. + +The two routes name **the same set** (verified as a set identity, not a matching +count). Names are of the form +`PresetMessage_[_]_S-.tbl`, e.g. +`PresetMessage_Ellen_03_S03-3.tbl`, `PresetMessage_ADAN_Camael.tbl` — so a set is +scoped to a character *and* a stage-phase, which is what makes 144 of them. + +### Residual, characterised + +* **8 rule tables stay unnamed** and — this is the point — they are unnamed by + *both* routes: nothing on the disc declares them. Their hashes are in the + artefact. +* `UnitMessageSet_S.tbl` exists for **22 of the 28 shipped stages**. The six + missing are exactly **S18–S23, the tutorials**, which is the same set that has + no `AIParams_SNN.tbl` ([`challenge-mission-gate.md`](challenge-mission-gate.md)). + So the 8 undeclared tables are plausibly tutorial chatter, but that is a + reading — no tutorial `UnitMessageSet` exists to confirm it. +* The 22 stage tables declare **137** distinct `PresetMessage` sets against 136 + rule tables resolved, so one declared set has no rule table under the + `message\` prefix. + +## ✅ Correction to `squadron-orders.md` + +That document renders the voice-line enum at `0x820AEEB0` as +`ORDER_{WINGMAN,SQUAD,SQUADRON}_{FORMATION,ATTACK,COVER,EXTENDED}`. The +executable's own strings are **not** uniform: the WINGMAN and SQUAD names are +`ORDER_`, and all four SQUADRON names are misspelled **`ORDOR_`**. The disc data +carries the identical misspelling, which is how the two sides were joined. + +That document's 🔴 warning — that this enum is a UI/comms axis and is **not** the +order-class enum — is **confirmed and now explained**: all twelve names are +record names in this chatter rule table, i.e. they are the twelve *events* +"a squadron order was issued", each with its own lines, priority and interval. +They are triggers for barks, not order types. + +## 🟡 Not settled + +* What `EffectiveTime`, `Interval`, `IntervalFluctuation`, `Priority`, + `Probability` and `Pattern` mean numerically — they are read here as a schema, + not measured against the running game. +* The `Yes`/`No` second element of each pair. +* Which of the 49 speakers is a wingman — the "13 tables" partition above is a + count, not a verified roster. diff --git a/docs/re/structures/squadron-orders.md b/docs/re/structures/squadron-orders.md index 8d0d59bd..119402ed 100644 --- a/docs/re/structures/squadron-orders.md +++ b/docs/re/structures/squadron-orders.md @@ -57,11 +57,21 @@ as 🟡 rather than levelled up to match the other three. ## 🔴 A different enum that is NOT this -`sub_82320B48` selects a voice line from -`ORDER_{WINGMAN,SQUAD,SQUADRON}_{FORMATION,ATTACK,COVER,EXTENDED}` -(`0x820AEEB0`…). The names are tempting — ATTACK and COVER look like they should -map onto the order classes — but there is **no evidence** they index the order -types, and it is a UI/comms axis. Not connected; do not conflate them. +`sub_82320B48` selects a voice line from twelve strings at `0x820AEEB0`…. Note +the spelling, which is the game's own and matters because it is the join key: +the WINGMAN and SQUAD names are `ORDER_`, and **all four SQUADRON names are +misspelled `ORDOR_`** — `ORDOR_SQUADRON_{FORMATION,ATTACK,COVER,EXTENDED}`. (An +earlier revision of this document rendered all twelve as `ORDER_`.) + +The names are tempting — ATTACK and COVER look like they should map onto the +order classes — but there is **no evidence** they index the order types, and it +is a UI/comms axis. Not connected; do not conflate them. + +**✅ Settled 2026-08-27, and the warning above holds.** All twelve are record +names in the reactive-chatter rule table, carrying the same misspelling on the +data side — see [`preset-message-rules.md`](preset-message-rules.md). They are +the twelve *events* "a squadron order was issued", each with its own lines, +priority and interval. Triggers for barks, not order types. ## 🟡 Not settled diff --git a/tools/re-capture/preset_messages.py b/tools/re-capture/preset_messages.py new file mode 100644 index 00000000..a85ee697 --- /dev/null +++ b/tools/re-capture/preset_messages.py @@ -0,0 +1,227 @@ +#!/usr/bin/env python3 +"""Regenerate docs/re/data/preset-messages.txt. + +The reactive combat-chatter system: a per-character *rule* table +(`message\\PresetMessage_[_][_S-].tbl`) holding 64 event +records plus a `Sperkers` speaker roster, paired with a `_msg.tbl` message table +in the layout cutscene-message-table.md already documents. + +Run: python3 preset_messages.py > ../../docs/re/data/preset-messages.txt +""" +import sys, os, glob, collections +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from unit_substructures import pak_entries +import unitgroup as U + +DAT = '/work/sylph_extract/dat' +PAK = os.path.join(DAT, 'GP_MAIN_GAME_E.pak') +MARKER = 'ORDOR_SQUADRON_EXTENDED' +STAGES = ['S%02d' % i for i in list(range(1, 17)) + list(range(18, 30))] + + +def idxd_entries(path): + out = {} + for h, s in pak_entries(path): + if s[:4] == b'IDXD': + out[h] = s + return out + + +def cue_names(): + for h, s in pak_entries(os.path.join(DAT, 'tables.pak')): + if s[:4] != b'IDXD': + continue + try: + recs = U.parse(s) + except Exception: + continue + for r in recs: + if r['squadron'] == 'SOUNDS': + return {n for _, n, _ in r['fields'] if n is not None} + return set() + + +def to_cue(m): + """sound-cue-table.md's rule: MSG_ becomes VOICE_, but a name that already + carries VOICE_ keeps the one it has rather than doubling it.""" + if not m.startswith('MSG_'): + return m + rest = m[4:] + return rest if rest.startswith('VOICE_') else 'VOICE_' + rest + + +def main(): + entries = idxd_entries(PAK) + rule = {} + for h, s in entries.items(): + try: + recs = U.parse(s) + except Exception: + continue + if any(r['squadron'] == MARKER for r in recs): + rule[h] = recs + + # how many language packs carry the same set + packs = sorted(glob.glob(os.path.join(DAT, 'GP_MAIN_GAME_?.pak'))) + percopy = {} + for p in packs: + n = 0 + for h, s in pak_entries(p): + if s[:4] == b'IDXD' and MARKER.encode() in s: + n += 1 + percopy[os.path.basename(p)] = n + + print('# Reactive combat-chatter rule tables (regenerated by preset_messages.py)') + print() + print('rule tables in GP_MAIN_GAME_E.pak : %d of %d IDXD entries' % (len(rule), len(entries))) + print('per language pack : %s' % percopy) + print('total across the disc : %d' % sum(percopy.values())) + + # one record-name shape? + shapes = collections.Counter(tuple(r['squadron'] for r in recs) for recs in rule.values()) + print('distinct record-name tuples : %d' % len(shapes)) + shape = shapes.most_common(1)[0][0] + print('records per table : %d (64 events + Sperkers)' % len(shape)) + + # named-field schema, and the MessageCount identity + schemas = collections.Counter() + ok = bad = 0 + msgs = set() + flags = collections.Counter() + perevent = collections.Counter() + for recs in rule.values(): + for r in recs: + if r['squadron'] == 'Sperkers': + continue + names = tuple(n for _, n, _ in r['fields'] if n is not None) + schemas[names] += 1 + pos = [v for _, n, v in r['fields'] if n is None] + try: + mc = int(U.named(r)['MessageCount']) + except Exception: + mc = -1 + if mc * 2 == len(pos): + ok += 1 + else: + bad += 1 + if pos: + perevent[r['squadron']] += 1 + for i, v in enumerate(pos): + if i % 2: + flags[v] += 1 + else: + msgs.add(v) + print() + print('distinct named-field schemas : %d' % len(schemas)) + for names, c in schemas.most_common(): + print(' x%-6d %s' % (c, ', '.join(names))) + print('MessageCount*2 == positional count: %d/%d (mismatches %d)' % (ok, ok + bad, bad)) + print('second-of-pair values : %s' % dict(flags)) + + print() + print('the 64 event records, and how many of the %d tables give each one lines:' % len(rule)) + for name in sorted(n for n in shape if n != 'Sperkers'): + print(' %-32s %3d' % (name, perevent[name])) + + # speakers + spk = collections.Counter() + per = collections.Counter() + for recs in rule.values(): + for r in recs: + if r['squadron'] == 'Sperkers': + names = [n for _, n, _ in r['fields'] if n is not None] + per[len(names)] += 1 + for n in names: + spk[n] += 1 + print() + print('Sperkers roster: %d distinct speakers; sizes %s' % ( + len(spk), dict(sorted(per.items())))) + for n, c in spk.most_common(): + print(' %-28s %3d' % (n, c)) + + # the join into the sound-cue table + cues = cue_names() + hit = sorted(m for m in msgs if to_cue(m) in cues) + miss = sorted(m for m in msgs if to_cue(m) not in cues) + print() + print('distinct message ids : %d' % len(msgs)) + print('resolved in tables.pak SOUNDS : %d (%.1f%%)' % ( + len(hit), 100.0 * len(hit) / max(1, len(msgs)))) + print('unresolved : %d' % len(miss)) + fam = collections.Counter(m.split('_')[1] for m in miss if m.count('_') >= 2) + print(' by family: %s' % dict(fam)) + for m in miss: + print(' %s' % m) + + # naming, two independent routes + harvest = set() + for p in sorted(glob.glob(os.path.join(DAT, '**', '*.pak'), recursive=True)): + for h, s in pak_entries(p): + if s[:4] != b'IDXD': + continue + try: + recs = U.parse(s) + except Exception: + continue + for r in recs: + harvest.add(r['squadron']) + for _, n, v in r['fields']: + if n: + harvest.add(n) + harvest.add(v) + byharvest = {} + for c in harvest: + if c.startswith('PresetMessage_'): + byharvest.setdefault(U.name_hash('message\\' + c), 'message\\' + c) + + declared = collections.Counter() + ums_found = [] + for st in STAGES: + h = U.name_hash('message\\UnitMessageSet_%s.tbl' % st) + if h not in entries: + continue + ums_found.append(st) + for r in U.parse(entries[h]): + d = U.named(r) + for k in ('PresetMessage_Phase1', 'PresetMessage_Phase2', 'PresetMessage_Phase3'): + if d.get(k): + declared[d[k]] += 1 + bydecl = {} + for v in declared: + bydecl.setdefault(U.name_hash('message\\' + v), 'message\\' + v) + + A = {h: n for h, n in byharvest.items() if h in rule} + B = {h: n for h, n in bydecl.items() if h in rule} + print() + print('naming route 1 (any harvested PresetMessage_ string): %d/%d' % (len(A), len(rule))) + print('naming route 2 (UnitMessageSet_S* declarations) : %d/%d' % (len(B), len(rule))) + print('the two routes agree on the same set : %s' % (set(A) == set(B))) + print('UnitMessageSet_S* tables present: %d of %d (%s)' % ( + len(ums_found), len(STAGES), ' '.join(ums_found))) + print('missing: %s' % ' '.join(s for s in STAGES if s not in ums_found)) + print('distinct declared PresetMessage tables: %d' % len(declared)) + + # control: the same prefix also names a DIFFERENT shape + other = {h: n for h, n in byharvest.items() if h in entries and h not in rule} + print() + print('CONTROL — PresetMessage_* names that hit an IDXD entry but are NOT rule tables: %d' % len(other)) + oshape = collections.Counter() + for h in other: + try: + recs = U.parse(entries[h]) + except Exception: + continue + oshape[tuple(sorted({r['squadron'].rstrip('0123456789') for r in recs}))] += 1 + print(' their record-name families: %s' % dict(oshape)) + + print() + print('rule tables, by name:') + for h in sorted(rule, key=lambda k: A.get(k, '~%#010x' % k)): + print(' %#010x %s' % (h, A.get(h, ''))) + un = [h for h in rule if h not in A] + print() + print('unnamed rule tables: %d' % len(un)) + + +if __name__ == '__main__': + main()