diff --git a/docs/re/BACKLOG.md b/docs/re/BACKLOG.md index dbc5afe6..45aaec33 100644 --- a/docs/re/BACKLOG.md +++ b/docs/re/BACKLOG.md @@ -4199,3 +4199,49 @@ event or per speaker, and what `IntervalFluctuation` randomises against all live in the *consumer* of the 40-byte object, which this iteration did not read. `Pattern`'s `Window` is named, not measured. Also still open from the earlier entries: which of the 49 speakers are wingmen. + +--- + +## βœ…πŸ”΄ 2026-08-27 β€” how the three phase tables merge, and one refuted handle + +Continues the entry above. **Docs checked:** `preset-message-rules.md` (mine), +`mission-phase-deployment.md` and `mission-phase-membership.md` (they own +`UnitMessageSet_S.tbl` and its `CrewCount` + `PresetMessage_Phase1/2/3` β€” +what they do **not** say is what happens when the three tables are loaded), +`idxd-tag-hash.md` (owns the hash family the map key comes from). + +βœ… **The three phase tables become ONE map.** `sub_82215A58` references exactly +`CrewCount` and `PresetMessage_Phase1/2/3` and reaches the loader from a **single** +call site (`0x82215D98`). The map is keyed by a hash of the *event record's name* +(`sub_82455C78`). All 64 event names recur in every phase table, so **every merge +collides**. + +βœ… **`sub_82213840` is the reconciliation comparator** β€” called from nowhere but +the loader. It compares the two message-id vectors elementwise plus `+16` +`Probability`, `+17` `Priority`, `+18` `Pattern`, `+20` `Interval`, `+24` +`IntervalFluctuation`, `+28` `EffectiveTime`. It does **not** compare `+32` (the +`Yes` mask) or `+36`. On a collision the loader frees the newcomer and keeps the +incumbent either way; a mismatch also clears the byte it returns, which the +caller propagates. + +βœ… **Measured on disc** (regenerator extended, numbers in `data/preset-messages.txt`): +298 `UnitMessageSet` records name β‰₯2 phase tables; **26 432** duplicate-key +insertions; **26 208 (99.15 %)** identical; **224 different**, of which **189 +differ only in the message list**. β‡’ **The phase tables are not additive** β€” +phase 1 wins and **224 authored variants never play**. A port that merges +additively, or lets a later phase override, will not match the game. + +βœ… **A negative worth keeping:** **zero** pairs agree on the six compared fields +while disagreeing on the `Yes` mask, so the comparator's exclusion of `+32` +changes nothing on the shipped disc. It looks like a latent bug until counted. + +πŸ”΄ **Refuted handle for the firing policy.** I tried to find the consumer by +intersecting functions that touch `+16`/`+17`/`+18`/`+20`/`+28`/`+32`/`+36`. +It returns dozens of unrelated functions across the binary β€” **offset shape is +not an identifier**, which the corpus already warns about and I re-learned by +spending a query on it. The remaining handle is the map's consumer reached via +`sub_82215A58` ← `sub_82214050`. + +❔ **Still open:** the firing policy proper (who rolls `Probability`, compares +`Priority`, ticks `Interval`, what `IntervalFluctuation` randomises); which of +the 49 speakers are wingmen. diff --git a/docs/re/INDEX.md b/docs/re/INDEX.md index 1763e855..fd2e142b 100644 --- a/docs/re/INDEX.md +++ b/docs/re/INDEX.md @@ -98,7 +98,7 @@ files, which is how the same ground got covered twice. | [`structures/sound-pak-contents.md`](structures/sound-pak-contents.md) | Census of `sound.pak`, and the limit of the leading-region rule | βœ… CONFIRMED, 5 135/5 135 names hash into the TOC, **9 519/9 519** entries accounted for, and a full 4 114-bank manifest (408.3 min of audio) computed from PsuedoBytesPerSec without decoding; ⚠️ leading-region rule holds for 1 571/4 382 eng and 0/5 100 jpn | | [`structures/sound-cue-table.md`](structures/sound-cue-table.md) | The cue index in `tables.pak` β€” message id -> cue -> sound id -> `.slb` bank | βœ… CONFIRMED, 1 326/1 338 script message ids bind to a bank; SOUNDS and FILES agree on the same 12 absentees, 0 orphan files | | [`structures/cutscene-message-table.md`](structures/cutscene-message-table.md) | Cutscene dialogue β€” speaker, portrait, on-screen seconds, audio cue per page | βœ… CONFIRMED, field count = 9Β·PageCount+2 for all 7 PageCounts, 1 252/1 252 caption keys match, 138 ids close both ways | -| [`structures/preset-message-rules.md`](structures/preset-message-rules.md) | The reactive-chatter rule table β€” 64 events, the (message, Yes/No) pair payload, the `Sperkers` roster | βœ… CONFIRMED, the seven fields read off `sub_82213980` (three are seconds Γ—60, the Yes/No element is a u32 mask, hence the 32 clamp); MessageCountΒ·2 == positional count in 9 216/9 216; five naming routes union to 144/144 (six names predicted from the speaker roster before hashing, control 0/4); 2 388/2 405 message ids bind to a cue | +| [`structures/preset-message-rules.md`](structures/preset-message-rules.md) | The reactive-chatter rule table β€” 64 events, the (message, Yes/No) pair payload, the `Sperkers` roster | βœ… CONFIRMED, the seven fields read off `sub_82213980`, phase tables merge non-additively (224 variants discarded) (three are seconds Γ—60, the Yes/No element is a u32 mask, hence the 32 clamp); MessageCountΒ·2 == positional count in 9 216/9 216; five naming routes union to 144/144 (six names predicted from the speaker roster before hashing, control 0/4); 2 388/2 405 message ids bind to a cue | | [`structures/isl-mission-timer.md`](structures/isl-mission-timer.md) | The mission timer β€” two sequential countdowns, not a limit and a warning | βœ… CONFIRMED β€” read from `sub_822639B8`; the threshold reading is | | [`structures/archive-naming.md`](structures/archive-naming.md) | Which archives the disc can name β€” 100% for menu paks, **0.0%** for the six 2D paks and `GP_READY_ROOM` | βœ… CONFIRMED β€” 6027 names Γ— 16 prefixes, bimodal | | [`structures/hud-config.md`](structures/hud-config.md) | The in-game HUD configuration β€” 16 records, 419 asset paths, and a per-stage `ResourceTable` | βœ… CONFIRMED β€” 0 of 419 config paths resolve as pak entries | diff --git a/docs/re/data/preset-messages.txt b/docs/re/data/preset-messages.txt index a43af8f3..b1e181b6 100644 --- a/docs/re/data/preset-messages.txt +++ b/docs/re/data/preset-messages.txt @@ -162,6 +162,19 @@ unresolved : 17 MSG_VOICE_D_447 MSG_VOICE_D_449 +phase merge (the loader folds Phase1/2/3 into one map): + UnitMessageSet records naming >= 2 phase tables : 298 + duplicate-key insertions to reconcile : 26432 + identical on the compared fields : 26208 + DIFFERENT -> loader clears its ok flag : 224 + compared fields equal but Yes mask differs : 0 + x189 MESSAGE_LIST + x17 Probability, Priority, Pattern, Interval, EffectiveTime, MESSAGE_LIST + x14 Probability, Priority, Pattern, Interval, MESSAGE_LIST + x2 Probability, Priority, Interval, IntervalFluctuation, MESSAGE_LIST + x1 Probability, Priority, Pattern, Interval, IntervalFluctuation, EffectiveTime, MESSAGE_LIST + x1 Probability, Priority, Interval, IntervalFluctuation + naming route 1 (any harvested PresetMessage_ string): 136/144 naming route 2 (UnitMessageSet_S* declarations) : 136/144 naming route 3 (strip _msg from a message table) : 137/144 diff --git a/docs/re/structures/preset-message-rules.md b/docs/re/structures/preset-message-rules.md index fde27601..ce0a2387 100644 --- a/docs/re/structures/preset-message-rules.md +++ b/docs/re/structures/preset-message-rules.md @@ -265,13 +265,71 @@ can never reach, because it skips the record first. Thirteen dead records out of ### 🟑 What this does *not* settle -The loader tells us the units and the storage, not the policy. `Priority`'s -comparison rule, whether `Interval` is a cooldown per event or per speaker, and -what `IntervalFluctuation` randomises against are all in the *consumer* of the -40-byte object, which this iteration did not read. `Pattern`'s two live values +The loader and the merge tell us units, storage and duplicate handling β€” not +the *firing* policy. `Priority`'s comparison rule, whether `Interval` is a +cooldown per event or per speaker, and what `IntervalFluctuation` randomises +against all live in the consumer of the map, reachable via +`sub_82215A58` ← `sub_82214050`. πŸ”΄ One handle was tried and refuted: searching +for functions that touch `+16`/`+17`/`+18`/`+20`/`+28`/`+32`/`+36` returns +dozens of unrelated matches, because those offsets recur across unrelated object +layouts. Offset shape is not an identifier. `Pattern`'s two live values are named, but that `Window` means an on-screen text window is a reading of the name, not something measured. +## βœ… How the three phase tables are merged β€” and what that costs + +**Settled 2026-08-27.** `sub_82215A58` is the `UnitMessageSet` reader: it +references exactly `CrewCount`, `PresetMessage_Phase1`, `PresetMessage_Phase2` +and `PresetMessage_Phase3`, and it reaches the loader from **one** call site +(`0x82215D98`). So a crew's three phase tables are loaded into **one** map, not +three. The map is keyed by a hash of the *event record's name* +(`sub_82455C78`, the rolling-accumulator / character-sum-top-byte shape +[`idxd-tag-hash.md`](idxd-tag-hash.md) documents; which member of that family +this entry point is was not pinned here, and the finding does not depend on it). + +Because all 64 event names recur in every phase table, **every merge collides**. +`sub_82213980`'s insert therefore has a reconciliation arm, and +`sub_82213840` β€” a function called from nowhere else β€” is its comparator. + +### `sub_82213840` compares six of the eight fields + +It walks the two message-id vectors elementwise, then compares `+16` +(`Probability`), `+17` (`Priority`), `+18` (`Pattern`), `+20` (`Interval`), +`+24` (`IntervalFluctuation`) and `+28` (`EffectiveTime`), and returns a bool. +It does **not** look at `+32` (the `Yes` mask) or `+36`. + +On a colliding key the loader compares, and then β€” whatever the answer β€” +**frees the newcomer and keeps the incumbent**. A mismatch additionally clears +the byte the loader returns (`lbz r3, 80(r31)` at the epilogue); the caller +turns that into a flag at `0x82215D9C`…`0x82215DDC` and propagates it. + +### What the disc actually does with that + +Measured over the 22 `UnitMessageSet_S.tbl` (numbers in the artefact): + +| | | +|---|---| +| `UnitMessageSet` records naming β‰₯ 2 phase tables | **298** | +| duplicate-key insertions to reconcile | **26 432** | +| identical on the compared fields | **26 208** (99.15 %) | +| **different** β€” loader clears its flag | **224** | +| compared fields equal but `Yes` mask differs | **0** | + +Of the 224, **189 differ only in the message list** β€” per-phase dialogue +variants for the same event and the same timings. The other 35 also move +`Probability`/`Priority`/`Pattern`/`Interval`. + +βœ… **The phase tables are not additive.** For a given crew and event, phase 1's +rule wins and the later phases' are discarded β€” so **224 authored variants never +play**. A port that merges them additively, or that lets a later phase override, +will not behave like the game. + +βœ… **The comparator's blind spot is never exercised.** There is no record pair +that agrees on the six compared fields and disagrees on the `Yes` mask, so the +fact that `+32` is excluded from the comparison changes nothing on the shipped +disc. Worth recording precisely because it is the kind of gap that looks like a +latent bug until it is counted. + ## 🟑 Not settled * The *policy* the seven fields drive β€” see "what this does not settle" above. diff --git a/tools/re-capture/preset_messages.py b/tools/re-capture/preset_messages.py index 021f5869..6c9c8adb 100644 --- a/tools/re-capture/preset_messages.py +++ b/tools/re-capture/preset_messages.py @@ -190,6 +190,63 @@ def main(): for m in miss: print(' %s' % m) + # phase-merge reconciliation: sub_82215A58 loads a unit's Phase1/2/3 tables + # into ONE map keyed by the event record name, and sub_82213840 compares a + # duplicate against the incumbent on (message list, +16, +17, +18, +20, +24, + # +28) -- not on the Yes mask at +32. + CMP = ('Probability', 'Priority', 'Pattern', 'Interval', + 'IntervalFluctuation', 'EffectiveTime') + def _core(rec): + d = U.named(rec) + pos = [v for _, nm, v in rec['fields'] if nm is None] + return ((tuple(d.get(f) for f in CMP), tuple(pos[0::2])), tuple(pos[1::2])) + sets = dup = same = diff = mask_only = 0 + why = collections.Counter() + for st in STAGES: + h = U.name_hash('message\\UnitMessageSet_%s.tbl' % st) + if h not in entries: + continue + for r in U.parse(entries[h]): + d = U.named(r) + phases = [d.get('PresetMessage_Phase%d' % i) for i in (1, 2, 3)] + phases = [n for n in phases if n] + if len(phases) < 2: + continue + sets += 1 + seen = {} + for n in phases: + hh = U.name_hash('message\\' + n) + if hh not in entries: + continue + for rec in U.parse(entries[hh]): + if rec['squadron'] == 'Sperkers': + continue + core, mask = _core(rec) + k = rec['squadron'] + if k not in seen: + seen[k] = (core, mask) + continue + dup += 1 + if seen[k][0] == core: + same += 1 + if seen[k][1] != mask: + mask_only += 1 + else: + diff += 1 + w = [f for f, a, b in zip(CMP, seen[k][0][0], core[0]) if a != b] + if seen[k][0][1] != core[1]: + w.append('MESSAGE_LIST') + why[tuple(w)] += 1 + print() + print('phase merge (the loader folds Phase1/2/3 into one map):') + print(' UnitMessageSet records naming >= 2 phase tables : %d' % sets) + print(' duplicate-key insertions to reconcile : %d' % dup) + print(' identical on the compared fields : %d' % same) + print(' DIFFERENT -> loader clears its ok flag : %d' % diff) + print(' compared fields equal but Yes mask differs : %d' % mask_only) + for w, c in why.most_common(): + print(' x%-5d %s' % (c, ', '.join(w))) + # naming, two independent routes harvest = set() for p in sorted(glob.glob(os.path.join(DAT, '**', '*.pak'), recursive=True)):