From 0f7ce6d6cba8238ca26dee5ce03faf09efe99275 Mon Sep 17 00:00:00 2001 From: "Claude (auto)" Date: Thu, 27 Aug 2026 16:35:56 +0000 Subject: [PATCH] re: the seven chatter fields, read off sub_82213980 The one function referencing all seven field-name strings is the rule table's loader. Interval / IntervalFluctuation / EffectiveTime are SECONDS, emitted as *60 frame counts; Probability is a percentage and zero skips the record; Pattern is a 4-arm enum of which only Sound and Window ship; the Yes/No pair element is a u32 bitmask, which is why MessageCount is clamped to 32 (max on disc is 26). 40-byte object layout recorded. 13 dead records characterised. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE --- docs/re/BACKLOG.md | 57 ++++++++++++++ docs/re/INDEX.md | 2 +- docs/re/data/preset-messages.txt | 12 +++ docs/re/structures/preset-message-rules.md | 87 +++++++++++++++++++++- tools/re-capture/preset_messages.py | 37 +++++++++ 5 files changed, 190 insertions(+), 5 deletions(-) diff --git a/docs/re/BACKLOG.md b/docs/re/BACKLOG.md index c87f60bb..dbc5afe6 100644 --- a/docs/re/BACKLOG.md +++ b/docs/re/BACKLOG.md @@ -4142,3 +4142,60 @@ smaller claim than the one it replaces. ❔ Still open from the entry above: what the seven numeric fields mean, the `Yes`/`No` pair element, and which speakers are wingmen. + +--- + +## ✅ 2026-08-27 — the seven chatter fields, read off the engine's loader + +Continues the two entries above. `sub_82213980` is the **only** function in the +executable that references any of the seven field-name strings — they sit +contiguously at `0x820A5794`…`0x820A5800`, and all seven xrefs land in it. It is +the rule table's loader: it walks records with `sub_82448AA0` /`sub_824482D0` / +`sub_82448BC8` and builds a **40-byte** object per event. + +**Docs checked:** `preset-message-rules.md` (mine), `movie-subtitles.md` (its +"Pattern" is a *filename*-pattern table heading — unrelated), +`mission-phase-deployment.md` (owns the radio **delivery category** +`None`/`Emergency`/`Killed`/`Noise` in `ScriptMessage_S02_msg.tbl` — a different +table on a different axis; `Killed` sits at `0x820A5784`, immediately before this +string block, which is precisely the adjacency that invites conflating them). + +✅ **The three time fields are SECONDS at 60 Hz.** The loader emits +`mulli rN, rN, 60` for `Interval`, `IntervalFluctuation` and `EffectiveTime`, and +for nothing else. Disc values are all small ints (`{0,3,5,10}`, `{0,3,4,5,30,120}`, +`{0,2}`), so seconds is the only reading that survives — `120` s = two minutes. + +✅ **`Pattern` is a delivery channel with two dead arms.** `strcmp` against +`"Log"` → 2, `"Window"` → 3, `"Demo"` → 4, default **1**. The disc says only +`Sound` (8 395, → default 1) and `Window` (821, → 3): **`Log` and `Demo` exist in +code and in no data**. + +✅ **The `Yes`/`No` element is a 32-bit mask — and that explains the clamp.** Two +passes over the positional payload: `sub_82448BC8(rec, 2·i)` takes the even slots +(message ids), `sub_82448BC8(rec, 2·i+1)` the odd, `strcmp`s each against `"Yes"` +(`0x820A57A4`) and sets `1 << i` in a `u32` at `[obj+32]`. The loader clamps +`MessageCount` to **32** — the same constant as the mask width. **The data +agrees:** max `MessageCount` on disc is **26**, nothing exceeds the clamp, and no +record's `Yes` count exceeds its own `MessageCount`. This re-derives the pair +layout from the *executable* side, independently of the +`MessageCount · 2 == positional count` identity it was first inferred from. + +✅ **`Probability` is a gate, not just a weight.** Read as a sign-extended byte; +**zero skips the record before the object is allocated**. Values are percentages +(`1,2,3,5,10,20,25,30,40,50,60,100`). + +Object layout: `+16` Probability u8, `+17` Priority u8, `+18` Pattern u8, +`+20` Interval·60, `+24` IntervalFluctuation·60, `+28` EffectiveTime·60, +`+32` the Yes mask, `+36` zero. + +**Residual, characterised.** Cross-tab of `MessageCount == 0` against +`Probability == 0` over all 9 216 records: 6 786 zero on both, 2 417 set on both, +**11** with a `Probability` but no lines, **2** with lines and `Probability 0` — +those two ship dialogue the loader can never reach. Thirteen dead records. + +❔ **Not settled: the policy.** The loader gives units and storage, not +behaviour. `Priority`'s comparison rule, whether `Interval` is a cooldown per +event or per speaker, and what `IntervalFluctuation` randomises against all live +in the *consumer* of the 40-byte object, which this iteration did not read. +`Pattern`'s `Window` is named, not measured. Also still open from the earlier +entries: which of the 49 speakers are wingmen. diff --git a/docs/re/INDEX.md b/docs/re/INDEX.md index 1587ca05..1763e855 100644 --- a/docs/re/INDEX.md +++ b/docs/re/INDEX.md @@ -98,7 +98,7 @@ files, which is how the same ground got covered twice. | [`structures/sound-pak-contents.md`](structures/sound-pak-contents.md) | Census of `sound.pak`, and the limit of the leading-region rule | ✅ CONFIRMED, 5 135/5 135 names hash into the TOC, **9 519/9 519** entries accounted for, and a full 4 114-bank manifest (408.3 min of audio) computed from PsuedoBytesPerSec without decoding; ⚠️ leading-region rule holds for 1 571/4 382 eng and 0/5 100 jpn | | [`structures/sound-cue-table.md`](structures/sound-cue-table.md) | The cue index in `tables.pak` — message id -> cue -> sound id -> `.slb` bank | ✅ CONFIRMED, 1 326/1 338 script message ids bind to a bank; SOUNDS and FILES agree on the same 12 absentees, 0 orphan files | | [`structures/cutscene-message-table.md`](structures/cutscene-message-table.md) | Cutscene dialogue — speaker, portrait, on-screen seconds, audio cue per page | ✅ CONFIRMED, field count = 9·PageCount+2 for all 7 PageCounts, 1 252/1 252 caption keys match, 138 ids close both ways | -| [`structures/preset-message-rules.md`](structures/preset-message-rules.md) | The reactive-chatter rule table — 64 events, the (message, Yes/No) pair payload, the `Sperkers` roster | ✅ CONFIRMED, MessageCount·2 == positional count in 9 216/9 216; five naming routes union to 144/144 (six names predicted from the speaker roster before hashing, control 0/4); 2 388/2 405 message ids bind to a cue | +| [`structures/preset-message-rules.md`](structures/preset-message-rules.md) | The reactive-chatter rule table — 64 events, the (message, Yes/No) pair payload, the `Sperkers` roster | ✅ CONFIRMED, the seven fields read off `sub_82213980` (three are seconds ×60, the Yes/No element is a u32 mask, hence the 32 clamp); MessageCount·2 == positional count in 9 216/9 216; five naming routes union to 144/144 (six names predicted from the speaker roster before hashing, control 0/4); 2 388/2 405 message ids bind to a cue | | [`structures/isl-mission-timer.md`](structures/isl-mission-timer.md) | The mission timer — two sequential countdowns, not a limit and a warning | ✅ CONFIRMED — read from `sub_822639B8`; the threshold reading is | | [`structures/archive-naming.md`](structures/archive-naming.md) | Which archives the disc can name — 100% for menu paks, **0.0%** for the six 2D paks and `GP_READY_ROOM` | ✅ CONFIRMED — 6027 names × 16 prefixes, bimodal | | [`structures/hud-config.md`](structures/hud-config.md) | The in-game HUD configuration — 16 records, 419 asset paths, and a per-stage `ResourceTable` | ✅ CONFIRMED — 0 of 419 config paths resolve as pak entries | diff --git a/docs/re/data/preset-messages.txt b/docs/re/data/preset-messages.txt index 1ab50062..a43af8f3 100644 --- a/docs/re/data/preset-messages.txt +++ b/docs/re/data/preset-messages.txt @@ -11,6 +11,18 @@ distinct named-field schemas : 1 MessageCount*2 == positional count: 9216/9216 (mismatches 0) second-of-pair values : {'No': 8940, 'Yes': 388} +value census of the seven named fields (all 9216 event records): + EffectiveTime 4 distinct 0=8566 3=453 5=187 10=10 + Interval 6 distinct 0=6820 3=821 4=1 5=894 30=182 120=498 + IntervalFluctuation 2 distinct 0=8403 2=813 + Priority 8 distinct 1=7738 2=112 3=423 4=339 5=244 6=237 7=118 9=5 + Probability 13 distinct 0=6788 1=967 2=335 3=253 5=98 10=90 20=15 25=22 30=31 40=21 50=196 60=8 100=392 + Pattern 2 distinct Sound=8395 Window=821 + MessageCount 25 distinct 0=6797 1=419 2=556 3=465 4=335 5=236 6=105 7=65 8=57 9=34 10=43 11=23 12=11 13=6 14=8 15=5 16=14 17=8 18=6 19=7 20=7 21=2 23=2 25=4 26=1 + (MessageCount==0, Probability==0) cross-tab: {'both zero': 6786, 'both set': 2417, 'no lines but Probability>0': 11, 'lines but Probability==0 (dead)': 2} + records with MessageCount > 32 (the reader clamps here): 0 + Yes-bits per record: {0: 9022, 1: 127, 2: 28, 3: 14, 4: 10, 5: 1, 6: 1, 7: 7, 9: 2, 10: 2, 11: 1, 14: 1} + the 64 event records, and how many of the 144 tables give each one lines: ATTACK_IS_AVOIDED 114 ATTACK_SUCCEEDED 114 diff --git a/docs/re/structures/preset-message-rules.md b/docs/re/structures/preset-message-rules.md index 32353347..fde27601 100644 --- a/docs/re/structures/preset-message-rules.md +++ b/docs/re/structures/preset-message-rules.md @@ -190,11 +190,90 @@ record names in this chatter rule table, i.e. they are the twelve *events* "a squadron order was issued", each with its own lines, priority and interval. They are triggers for barks, not order types. +## ✅ The seven fields, read off the engine's own loader + +**Settled 2026-08-27** from `sub_82213980`, the one function in the executable +that references all seven field-name strings (they sit contiguously at +`0x820A5794`…`0x820A5800`; the query is the string-xref join, and the seven +xrefs land in that single function). It walks the rule table's records through +`sub_82448AA0` / `sub_824482D0` / `sub_82448BC8` and builds a **40-byte** runtime +object per event. The disc-side value census in the artefact agrees with it at +every point. + +| field | disc values (all 9 216 records) | what the loader does | +|---|---|---| +| `EffectiveTime` | `0` 8566, `3` 453, `5` 187, `10` 10 | `× 60` → `[obj+28]` | +| `Interval` | `0` 6820, `3` 821, `4` 1, `5` 894, `30` 182, `120` 498 | `× 60` → `[obj+20]` | +| `IntervalFluctuation` | `0` 8403, `2` 813 | `× 60` → `[obj+24]` | +| `Priority` | `1`…`7`, `9` (no `8`); `1` is 7 738 | byte → `[obj+17]` | +| `Probability` | `0` 6788, then `1`,`2`,`3`,`5`,`10`,`20`,`25`,`30`,`40`,`50`,`60`,`100` | byte → `[obj+16]`; **`0` makes the loader skip the record entirely** | +| `Pattern` | `Sound` 8395, `Window` 821 | string → small enum → `[obj+18]` | +| `MessageCount` | `0`…`26` | pair count; **clamped to 32** | + +✅ **The three time fields are SECONDS, converted to 60 Hz frames.** The loader +emits `mulli rN, rN, 60` for `Interval`, `IntervalFluctuation` and +`EffectiveTime` and for nothing else. Every disc value is a small integer +(`120` s = two minutes is the largest), which is what makes seconds the only +reading that survives. + +✅ **`Pattern` is a delivery channel, and the enum has two dead arms.** The +loader `strcmp`s the value against `"Log"` → 2, `"Window"` → 3, `"Demo"` → 4, +defaulting to **1**. The disc only ever says `Sound` (→ default 1) or `Window` +(→ 3); **`Log` and `Demo` are declared in code and never used in data**. Do not +conflate this with the radio *delivery category* (`None`/`Emergency`/`Killed`/ +`Noise`) that `mission-phase-deployment.md` found in `ScriptMessage_S02_msg.tbl` +— different table, different axis. (`Killed` does sit at `0x820A5784`, +immediately before this string block, which is exactly the kind of adjacency +that invites the mistake.) + +✅ **The `Yes`/`No` element is a 32-bit mask, and that is why `MessageCount` is +capped at 32.** The loader runs two passes over the positional payload: +`sub_82448BC8(rec, 2·i)` collects the **even** slots (the message ids) and +`sub_82448BC8(rec, 2·i+1)` the **odd** ones, `strcmp`s each against `"Yes"` +(`0x820A57A4`), and sets `1 << i` in a `u32` stored at `[obj+32]`. The clamp and +the mask width are the same constant. **Confirmation from the data:** the largest +`MessageCount` on the disc is **26**, no record exceeds the clamp, and no +record's `Yes` count ever exceeds its `MessageCount`. This also independently +re-derives the pair layout from the executable side, having first been inferred +from `MessageCount · 2 == positional count`. + +### The 40-byte runtime object + +``` ++16 u8 Probability (per cent) ++17 u8 Priority (1..9) ++18 u8 Pattern (1 = Sound, 3 = Window; 2 = Log and 4 = Demo unused) ++20 u32 Interval * 60 (frames) ++24 u32 IntervalFluctuation * 60 ++28 u32 EffectiveTime * 60 ++32 u32 the Yes bitmask ++36 u32 zero at construction +``` + +`[obj+16]`'s byte is also the loader's gate: it is read with `sub_824482D0` and +sign-extended, and a **zero `Probability` skips the record before the object is +ever allocated**. + +### Residual, characterised + +Cross-tabbing `MessageCount == 0` against `Probability == 0` over all 9 216 +records: **6 786** are zero on both (the event is simply not used by that +speaker), **2 417** are set on both, **11** carry a `Probability` but no lines, +and **2** carry lines with `Probability 0` — those two ship dialogue the loader +can never reach, because it skips the record first. Thirteen dead records out of +9 216. + +### 🟡 What this does *not* settle + +The loader tells us the units and the storage, not the policy. `Priority`'s +comparison rule, whether `Interval` is a cooldown per event or per speaker, and +what `IntervalFluctuation` randomises against are all in the *consumer* of the +40-byte object, which this iteration did not read. `Pattern`'s two live values +are named, but that `Window` means an on-screen text window is a reading of the +name, not something measured. + ## 🟡 Not settled -* What `EffectiveTime`, `Interval`, `IntervalFluctuation`, `Priority`, - `Probability` and `Pattern` mean numerically — they are read here as a schema, - not measured against the running game. -* The `Yes`/`No` second element of each pair. +* The *policy* the seven fields drive — see "what this does not settle" above. * Which of the 49 speakers is a wingman — the "13 tables" partition above is a count, not a verified roster. diff --git a/tools/re-capture/preset_messages.py b/tools/re-capture/preset_messages.py index c09d75eb..021f5869 100644 --- a/tools/re-capture/preset_messages.py +++ b/tools/re-capture/preset_messages.py @@ -118,6 +118,43 @@ def main(): print('MessageCount*2 == positional count: %d/%d (mismatches %d)' % (ok, ok + bad, bad)) print('second-of-pair values : %s' % dict(flags)) + # value census of the seven named fields, against sub_82213980's reader + FIELDS = ['EffectiveTime', 'Interval', 'IntervalFluctuation', 'Priority', + 'Probability', 'Pattern', 'MessageCount'] + vals = {f: collections.Counter() for f in FIELDS} + cross = collections.Counter() + yes_bits = collections.Counter() + over32 = 0 + for recs in rule.values(): + for r in recs: + if r['squadron'] == 'Sperkers': + continue + d = U.named(r) + for f in FIELDS: + vals[f][d.get(f)] += 1 + n, pr = int(d['MessageCount']), int(d['Probability']) + cross[(n == 0, pr == 0)] += 1 + if n > 32: + over32 += 1 + pos = [v for _, nm, v in r['fields'] if nm is None] + yes_bits[sum(1 for i, v in enumerate(pos) if i % 2 and v == 'Yes')] += 1 + print() + print('value census of the seven named fields (all %d event records):' % sum(vals['Priority'].values())) + for f in FIELDS: + v = vals[f] + try: + order = sorted(v, key=lambda x: int(x)) + except (TypeError, ValueError): + order = sorted(v, key=str) + print(' %-20s %2d distinct %s' % ( + f, len(v), ' '.join('%s=%d' % (k, v[k]) for k in order))) + print(' (MessageCount==0, Probability==0) cross-tab: %s' % { + 'both zero': cross[(True, True)], 'both set': cross[(False, False)], + 'no lines but Probability>0': cross[(True, False)], + 'lines but Probability==0 (dead)': cross[(False, True)]}) + print(' records with MessageCount > 32 (the reader clamps here): %d' % over32) + print(' Yes-bits per record: %s' % dict(sorted(yes_bits.items()))) + print() print('the 64 event records, and how many of the %d tables give each one lines:' % len(rule)) for name in sorted(n for n in shape if n != 'Sperkers'):