diff --git a/docs/re/BACKLOG.md b/docs/re/BACKLOG.md index 70b4b0e5..c87f60bb 100644 --- a/docs/re/BACKLOG.md +++ b/docs/re/BACKLOG.md @@ -4093,3 +4093,52 @@ tables are named by neither route** β€” nothing on the disc declares them. exactly **S18–S23, the tutorials** (same set as the missing `AIParams_SNN.tbl`), so tutorial chatter is the plausible reading β€” but there is no tutorial `UnitMessageSet` to confirm it, so it stays a reading. + +--- + +## βœ…πŸ”΄ 2026-08-27 β€” the 8 undeclared chatter rule tables: all named, and NOT tutorial + +Continues the entry above. **144/144 rule tables are now named**, and the +reading I left in that entry is **refuted**. + +**Two more naming routes, both cheap:** + +* **Route 3 β€” strip `_msg`.** A message table's name gives its rule table's + name. **137/144**, a strict superset of route 1, and it hits **zero** entries + that are not rule tables. It adds `PresetMessage_Katana_14_S16-2.tbl`. +* **Route 5 β€” sweep the grammar.** `PresetMessage_[_NN][_S-

].tbl` + over the 86 `` tokens the other routes expose. Adds + `PresetMessage_Margras_04_S14-1.tbl`. **1/144.** + +βœ… **Route 4 β€” the roster predicts the name, 6 of 6.** Among the already-named +two-speaker tables the rule is transparent: `_Fleet` speaks +(`OP`, ``), and `TCAF_..Ship` speaks (`ADANPL`, +`TCAF`). The six remaining unnamed tables are six **gaps in two series**. +Predicted from their rosters, then hashed: `TCAF_FleetBB`, `TCAF_17thFleetBB`, +`TCAF_17thFleetCA`, `TCAF_17thFleetCB`, `TCAF_17thFleetShipA`, +`TCAF_17thFleetShipB` β€” **6/6**. **Control:** four same-shaped names the series +does not contain (`TCAF_FleetCC`, `TCAF_17thFleetCC`, `TCAF_ShipA_01`, +`TCAF_17thFleetShipC_01`) score **0/4**, so it is not the case that any plausible +name lands on an unnamed slot. + +πŸ”΄ **Refuted: "the undeclared tables are tutorial chatter."** It was the obvious +reading β€” `UnitMessageSet` exists for 22 of 28 stages and the six missing are +exactly S18–S23. But the eight are two **story**-stage tables (`Katana_14_S16-2`, +`Margras_04_S14-1`) and six TCAF fleet/ship tables carrying **no stage tag at +all**. Not one is a tutorial. Recorded rather than deleted: the inference from +"S18–S23 have no `UnitMessageSet`" to "the undeclared tables are theirs" skipped +the step of *looking at what is in them* β€” all six residuals speak `MSG_TCAF_*` +with TCAF fleet rosters. + +βœ… Also settled in the same sweep: **every one of the 144 rule tables has its +`_msg` companion present**, in all six language packs (the earlier count of 137 +message tables was a count of *harvested names*, not of entries β€” six of the +eight have message tables whose names appear as strings nowhere). + +What the tutorial evidence does support, narrowly: no rule table is tagged +`_S18`…`_S23` and the tutorials ship no `UnitMessageSet`, so on this evidence the +six tutorial missions have **no reactive chatter of their own**. That is a +smaller claim than the one it replaces. + +❔ Still open from the entry above: what the seven numeric fields mean, the +`Yes`/`No` pair element, and which speakers are wingmen. diff --git a/docs/re/INDEX.md b/docs/re/INDEX.md index c706b1b2..1587ca05 100644 --- a/docs/re/INDEX.md +++ b/docs/re/INDEX.md @@ -98,7 +98,7 @@ files, which is how the same ground got covered twice. | [`structures/sound-pak-contents.md`](structures/sound-pak-contents.md) | Census of `sound.pak`, and the limit of the leading-region rule | βœ… CONFIRMED, 5 135/5 135 names hash into the TOC, **9 519/9 519** entries accounted for, and a full 4 114-bank manifest (408.3 min of audio) computed from PsuedoBytesPerSec without decoding; ⚠️ leading-region rule holds for 1 571/4 382 eng and 0/5 100 jpn | | [`structures/sound-cue-table.md`](structures/sound-cue-table.md) | The cue index in `tables.pak` β€” message id -> cue -> sound id -> `.slb` bank | βœ… CONFIRMED, 1 326/1 338 script message ids bind to a bank; SOUNDS and FILES agree on the same 12 absentees, 0 orphan files | | [`structures/cutscene-message-table.md`](structures/cutscene-message-table.md) | Cutscene dialogue β€” speaker, portrait, on-screen seconds, audio cue per page | βœ… CONFIRMED, field count = 9Β·PageCount+2 for all 7 PageCounts, 1 252/1 252 caption keys match, 138 ids close both ways | -| [`structures/preset-message-rules.md`](structures/preset-message-rules.md) | The reactive-chatter rule table β€” 64 events, the (message, Yes/No) pair payload, the `Sperkers` roster | βœ… CONFIRMED, MessageCountΒ·2 == positional count in 9 216/9 216; two independent naming routes agree on the same 136/144; 2 388/2 405 message ids bind to a cue | +| [`structures/preset-message-rules.md`](structures/preset-message-rules.md) | The reactive-chatter rule table β€” 64 events, the (message, Yes/No) pair payload, the `Sperkers` roster | βœ… CONFIRMED, MessageCountΒ·2 == positional count in 9 216/9 216; five naming routes union to 144/144 (six names predicted from the speaker roster before hashing, control 0/4); 2 388/2 405 message ids bind to a cue | | [`structures/isl-mission-timer.md`](structures/isl-mission-timer.md) | The mission timer β€” two sequential countdowns, not a limit and a warning | βœ… CONFIRMED β€” read from `sub_822639B8`; the threshold reading is | | [`structures/archive-naming.md`](structures/archive-naming.md) | Which archives the disc can name β€” 100% for menu paks, **0.0%** for the six 2D paks and `GP_READY_ROOM` | βœ… CONFIRMED β€” 6027 names Γ— 16 prefixes, bimodal | | [`structures/hud-config.md`](structures/hud-config.md) | The in-game HUD configuration β€” 16 records, 419 asset paths, and a per-stage `ResourceTable` | βœ… CONFIRMED β€” 0 of 419 config paths resolve as pak entries | diff --git a/docs/re/data/preset-messages.txt b/docs/re/data/preset-messages.txt index d4db6eb6..1ab50062 100644 --- a/docs/re/data/preset-messages.txt +++ b/docs/re/data/preset-messages.txt @@ -152,7 +152,15 @@ unresolved : 17 naming route 1 (any harvested PresetMessage_ string): 136/144 naming route 2 (UnitMessageSet_S* declarations) : 136/144 -the two routes agree on the same set : True +naming route 3 (strip _msg from a message table) : 137/144 +naming route 4 (predicted from the Sperkers roster) : 6/144 +routes 1 and 2 agree on the same set : True +route 3 is a superset of route 1 : True +route 3 names nothing that is not a rule table : True +CONTROL β€” same-shaped names outside the series, hits : 0/4 +naming route 5 (sweep of the grammar, 86 tokens): 1/144 +UNION of all five routes : 144/144 +rule tables whose _msg companion is absent : 0 UnitMessageSet_S* tables present: 22 of 28 (S01 S02 S03 S04 S05 S06 S07 S08 S09 S10 S11 S12 S13 S14 S15 S16 S24 S25 S26 S27 S28 S29) missing: S18 S19 S20 S21 S22 S23 distinct declared PresetMessage tables: 137 @@ -240,6 +248,7 @@ rule tables, by name: 0x4c47acc6 message\PresetMessage_Katana_11_S12-1.tbl 0x4e4cff88 message\PresetMessage_Katana_12_S13-1.tbl 0x517844db message\PresetMessage_Katana_13_S15-1.tbl + 0x5483c09d message\PresetMessage_Katana_14_S16-2.tbl 0xf93b44ee message\PresetMessage_Katana_15_S28.tbl 0x632e5d87 message\PresetMessage_Lady1.tbl 0x64348687 message\PresetMessage_Lady2.tbl @@ -247,6 +256,7 @@ rule tables, by name: 0xca38072d message\PresetMessage_Margras_01_S03-3.tbl 0xcb3730ef message\PresetMessage_Margras_02_S04-2.tbl 0xc86c12c9 message\PresetMessage_Margras_03_S10-1.tbl + 0xcde33d3e message\PresetMessage_Margras_04_S14-1.tbl 0xcfc8c66f message\PresetMessage_Margras_05_S14-2.tbl 0xd0c7f031 message\PresetMessage_Margras_06_S15-1.tbl 0xd532e9aa message\PresetMessage_Raymond_02_S02-1.tbl @@ -266,12 +276,18 @@ rule tables, by name: 0x9bfa033e message\PresetMessage_TCAF_17thFleetAA.tbl 0x9c003267 message\PresetMessage_TCAF_17thFleetAB.tbl 0x9c232e5d message\PresetMessage_TCAF_17thFleetBA.tbl + 0x9d29575d message\PresetMessage_TCAF_17thFleetBB.tbl + 0x9d4c5353 message\PresetMessage_TCAF_17thFleetCA.tbl + 0x9e527c53 message\PresetMessage_TCAF_17thFleetCB.tbl + 0xee14c77c message\PresetMessage_TCAF_17thFleetShipA.tbl + 0xef1af07c message\PresetMessage_TCAF_17thFleetShipB.tbl 0xf021197c message\PresetMessage_TCAF_17thFleetShipC.tbl 0x64ecd655 message\PresetMessage_TCAF_AcropolisGuard.tbl 0x4f33ea2e message\PresetMessage_TCAF_Caliban.tbl 0x57a53d3b message\PresetMessage_TCAF_FleetAA.tbl 0x58ab663b message\PresetMessage_TCAF_FleetAB.tbl 0x58ce6231 message\PresetMessage_TCAF_FleetBA.tbl + 0x59d48b31 message\PresetMessage_TCAF_FleetBB.tbl 0x59f78727 message\PresetMessage_TCAF_FleetCA.tbl 0x5afdb027 message\PresetMessage_TCAF_FleetCB.tbl 0xa6b0cfde message\PresetMessage_TCAF_Janus.tbl @@ -297,13 +313,5 @@ rule tables, by name: 0x9a44099c message\PresetMessage_Yoji_01_S06-2.tbl 0x9b43335e message\PresetMessage_Yoji_02_S07-1.tbl 0xa07acab1 message\PresetMessage_Yoji_03_S09-3.tbl - 0x5483c09d - 0x59d48b31 - 0x9d29575d - 0x9d4c5353 - 0x9e527c53 - 0xcde33d3e - 0xee14c77c - 0xef1af07c -unnamed rule tables: 8 +unnamed rule tables: 0 diff --git a/docs/re/structures/preset-message-rules.md b/docs/re/structures/preset-message-rules.md index 086132eb..32353347 100644 --- a/docs/re/structures/preset-message-rules.md +++ b/docs/re/structures/preset-message-rules.md @@ -113,35 +113,68 @@ a name that already carries `VOICE_` keeps the one it has β€” and applying it moves the count to 2 388. Recorded here because the same trap is one line of code away in any port. -## Naming, by two independent routes that agree +## Naming β€” five routes, and all 144 land The rule tables are pak entries keyed by `name_hash`, so they had to be named -before any of this could be read. +before any of this could be read. Five routes, each an independent transform: -1. **Harvested strings** β€” every `PresetMessage_*` string appearing anywhere in - any IDXD object on the disc, hashed under the `message\` prefix: **136/144**. -2. **Declarations** β€” the `PresetMessage_Phase1/2/3` values of the 22 - `message\UnitMessageSet_S.tbl` tables: **136/144**. +| route | source | named | +|---|---|---| +| 1 | any `PresetMessage_*` string appearing in any IDXD object, under `message\` | 136/144 | +| 2 | the `PresetMessage_Phase1/2/3` values of the 22 `UnitMessageSet_S.tbl` | 136/144 | +| 3 | strip `_msg` from a **message**-table name to get its rule table's name | 137/144 | +| 4 | predicted from the `Sperkers` roster (below) | 6/144 | +| 5 | sweep of the naming grammar over the 86 `` tokens routes 1–4 expose | 1/144 | +| | **union** | **144/144** | -The two routes name **the same set** (verified as a set identity, not a matching -count). Names are of the form -`PresetMessage_[_]_S-.tbl`, e.g. -`PresetMessage_Ellen_03_S03-3.tbl`, `PresetMessage_ADAN_Camael.tbl` β€” so a set is +Routes 1 and 2 name **the same set** (checked as a set identity, not a matching +count). Route 3 is a strict superset of route 1 and hits **no** entry that is not +a rule table. Every one of the 144 has its `_msg` companion present. + +Names are of the form `PresetMessage_[_][_S-].tbl`, e.g. +`PresetMessage_Ellen_03_S03-3.tbl`, `PresetMessage_ADAN_Camael.tbl` β€” a set is scoped to a character *and* a stage-phase, which is what makes 144 of them. -### Residual, characterised +### βœ… The roster predicts the name -* **8 rule tables stay unnamed** and β€” this is the point β€” they are unnamed by - *both* routes: nothing on the disc declares them. Their hashes are in the - artefact. -* `UnitMessageSet_S.tbl` exists for **22 of the 28 shipped stages**. The six - missing are exactly **S18–S23, the tutorials**, which is the same set that has - no `AIParams_SNN.tbl` ([`challenge-mission-gate.md`](challenge-mission-gate.md)). - So the 8 undeclared tables are plausibly tutorial chatter, but that is a - reading β€” no tutorial `UnitMessageSet` exists to confirm it. -* The 22 stage tables declare **137** distinct `PresetMessage` sets against 136 - rule tables resolved, so one declared set has no rule table under the - `message\` prefix. +Route 4 is the interesting one, because the prediction came first. Among the +tables already named, the two-speaker ones expose a transparent rule: + + _Fleet speaks (OP, ) + TCAF_..Ship speaks (ADANPL, TCAF) + +`TCAF_Fleet{AA,AB,BA,CA,CB}` and `TCAF_17thFleet{AA,AB,BA}` and +`TCAF_17thFleetShipC` are named; **six gaps in those series** were predicted from +the six unnamed tables' rosters and then hashed: + +| roster | predicted name | result | +|---|---|---| +| `TCAFOPB`, `TCAFB` | `TCAF_FleetBB` | βœ… `0x59d48b31` | +| `TCAFOPB`, `TCAFB` | `TCAF_17thFleetBB` | βœ… `0x9d29575d` | +| `TCAFOPA`, `TCAFC` | `TCAF_17thFleetCA` | βœ… `0x9d4c5353` | +| `TCAFOPB`, `TCAFC` | `TCAF_17thFleetCB` | βœ… `0x9e527c53` | +| `ADANPLA`, `TCAFA` | `TCAF_17thFleetShipA` | βœ… `0xee14c77c` | +| `ADANPLB`, `TCAFB` | `TCAF_17thFleetShipB` | βœ… `0xef1af07c` | + +**6 of 6.** The control matters as much as the hits: four same-shaped names the +series does *not* contain β€” `TCAF_FleetCC`, `TCAF_17thFleetCC`, `TCAF_ShipA_01`, +`TCAF_17thFleetShipC_01` β€” score **0 of 4**, so this is not "any plausible name +hits an unnamed slot". + +### πŸ”΄ Refuted: the undeclared tables are not tutorial chatter + +The eight tables that neither route 1 nor route 2 names are undeclared because +**nothing on the disc declares them** β€” no `UnitMessageSet` entry, and their own +names appear as strings nowhere. `UnitMessageSet_S.tbl` exists for 22 of the +28 stages and the six missing are exactly **S18–S23, the tutorials**, so +"the undeclared eight are tutorial chatter" was the obvious reading. It is +**wrong**: the eight are `PresetMessage_Katana_14_S16-2` and +`PresetMessage_Margras_04_S14-1` β€” both **story** stages β€” plus the six TCAF +fleet/ship tables above, which carry no stage tag at all. Not one is a tutorial. + +What that leaves is a different and smaller claim: the tutorials ship **no** +`UnitMessageSet`, and no rule table is tagged `_S18`…`_S23`, so on this evidence +the six tutorial missions have no reactive chatter of their own. ## βœ… Correction to `squadron-orders.md` diff --git a/tools/re-capture/preset_messages.py b/tools/re-capture/preset_messages.py index a85ee697..c09d75eb 100644 --- a/tools/re-capture/preset_messages.py +++ b/tools/re-capture/preset_messages.py @@ -8,7 +8,7 @@ in the layout cutscene-message-table.md already documents. Run: python3 preset_messages.py > ../../docs/re/data/preset-messages.txt """ -import sys, os, glob, collections +import sys, os, re, glob, collections sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) from unit_substructures import pak_entries import unitgroup as U @@ -170,9 +170,15 @@ def main(): harvest.add(n) harvest.add(v) byharvest = {} + byderived = {} for c in harvest: - if c.startswith('PresetMessage_'): - byharvest.setdefault(U.name_hash('message\\' + c), 'message\\' + c) + if not c.startswith('PresetMessage_'): + continue + byharvest.setdefault(U.name_hash('message\\' + c), 'message\\' + c) + # route 3: a message table's name gives its rule table's name + if c.endswith('_msg.tbl'): + b = 'message\\' + c[:-len('_msg.tbl')] + '.tbl' + byderived.setdefault(U.name_hash(b), b) declared = collections.Counter() ums_found = [] @@ -190,12 +196,60 @@ def main(): for v in declared: bydecl.setdefault(U.name_hash('message\\' + v), 'message\\' + v) + # route 4: names predicted from the Sperkers roster before they were hashed. + # TCAF_Fleet speaks (TCAFOP, TCAF); TCAF_..Ship speaks + # (ADANPL, TCAF). Six gaps in two series, all six confirmed. + PREDICTED = ['TCAF_FleetBB', 'TCAF_17thFleetBB', 'TCAF_17thFleetCA', + 'TCAF_17thFleetCB', 'TCAF_17thFleetShipA', 'TCAF_17thFleetShipB'] + # same-shaped names that the series does NOT contain, as the control + CONTROL = ['TCAF_FleetCC', 'TCAF_17thFleetCC', 'TCAF_ShipA_01', + 'TCAF_17thFleetShipC_01'] + bypred = {} + for c in PREDICTED: + n = 'message\\PresetMessage_%s.tbl' % c + bypred.setdefault(U.name_hash(n), n) + A = {h: n for h, n in byharvest.items() if h in rule} B = {h: n for h, n in bydecl.items() if h in rule} + C = {h: n for h, n in byderived.items() if h in rule} + D = {h: n for h, n in bypred.items() if h in rule} print() print('naming route 1 (any harvested PresetMessage_ string): %d/%d' % (len(A), len(rule))) print('naming route 2 (UnitMessageSet_S* declarations) : %d/%d' % (len(B), len(rule))) - print('the two routes agree on the same set : %s' % (set(A) == set(B))) + print('naming route 3 (strip _msg from a message table) : %d/%d' % (len(C), len(rule))) + print('naming route 4 (predicted from the Sperkers roster) : %d/%d' % (len(D), len(rule))) + print('routes 1 and 2 agree on the same set : %s' % (set(A) == set(B))) + print('route 3 is a superset of route 1 : %s' % set(A).issubset(C)) + print('route 3 names nothing that is not a rule table : %s' % all( + h in rule for h in byderived if h in entries)) + print('CONTROL β€” same-shaped names outside the series, hits : %d/%d' % ( + sum(1 for c in CONTROL if U.name_hash('message\\PresetMessage_%s.tbl' % c) in rule), + len(CONTROL))) + A = dict(A); A.update(C); A.update(D) + # route 5: sweep the naming grammar the other 143 names describe, + # PresetMessage_[_NN][_S-

].tbl, over every already seen. + whos = set() + for n in A.values(): + base = n.rsplit('\\', 1)[-1] + m = re.match(r'PresetMessage_(.+?)(?:_\d\d)?(?:_S\d\d-\d)?\.tbl$', base) + if m: + whos.add(m.group(1)) + stages = ['_S%02d-%d' % (st, ph) for st in range(1, 30) for ph in range(1, 5)] + E = {} + for w in sorted(whos): + for nn in [''] + ['_%02d' % i for i in range(1, 21)]: + for st in [''] + stages: + n = 'message\\PresetMessage_%s%s%s.tbl' % (w, nn, st) + h = U.name_hash(n) + if h in rule and h not in A: + E[h] = n + print('naming route 5 (sweep of the grammar, %d tokens): %d/%d' % ( + len(whos), len(E), len(rule))) + A.update(E) + print('UNION of all five routes : %d/%d' % (len(A), len(rule))) + missing_msg = [n for h, n in A.items() + if U.name_hash(n[:-4] + '_msg.tbl') not in entries] + print('rule tables whose _msg companion is absent : %d' % len(missing_msg)) print('UnitMessageSet_S* tables present: %d of %d (%s)' % ( len(ums_found), len(STAGES), ' '.join(ums_found))) print('missing: %s' % ' '.join(s for s in STAGES if s not in ums_found))