From acbcd8d75cb1694f0857d95e3ecce4dfd46ba022 Mon Sep 17 00:00:00 2001 From: Sylpheed RE agent Date: Thu, 27 Aug 2026 13:37:31 +0000 Subject: [PATCH] re: Stage\script.tbl's 11 non-MISSION fields, and the mission dialogue table structures/mission-script-ssb.md owns this manifest and names its 40 fields but never read the 11 that are not MISSION = StageNN.ssb. They are DIALOG_MESSAGE, DIALOG_LOCAL_STRING, FONT (+size), TEXT_POS, TEXT_LINES and the five pgmsg_*.prt. Two sibling records were also unread: GP_TEST (PATH = dat\GP_TEST\, a debug archive not on the disc) and TEXTS (a second text style). Probed 7 values x 33 prefixes x 41 archives. One resolves: message\MissionDialogMessage.tbl, in all six GP_MAIN_GAME_* paks -- 200 records, 25280 bytes, every name S_P_ with five kinds 40 each (HINT_PAUSE, HINT, OBJECTIVE, GRAPH, LOSE), fields positional and tagged 0..3, each value a message key. An index from (stage, phase, kind) to the localised strings, on the same S_P keying the ISL corpus already uses. Control: the 40 stage-phases span stages 1-16 and 24-29 -- a subset of the 28 shipped, and the six with no hints are exactly 18-23, the tutorials. A fifth independent route to the story/tutorial split, and it gives the phase count per stage. The other six do not resolve, with the control in the same sweep: MissionDialog_local_string.tbl and all five pgmsg_*.prt are not a pak entry under any of the 33 prefixes, while message\MissionDialogMessage.tbl and Stage\script.tbl both resolve in 6 archives. name_hash is CASE-INSENSITIVE (message\ == Message\); tag_hash is not. New structure doc, artefact and regenerator; the other five artefacts regenerate byte-identical. --- docs/re/BACKLOG.md | 35 ++++++++ docs/re/INDEX.md | 1 + docs/re/data/script-manifest.txt | 49 +++++++++++ docs/re/structures/mission-script-manifest.md | 77 +++++++++++++++++ tools/re-capture/script_manifest.py | 86 +++++++++++++++++++ 5 files changed, 248 insertions(+) create mode 100644 docs/re/data/script-manifest.txt create mode 100644 docs/re/structures/mission-script-manifest.md create mode 100644 tools/re-capture/script_manifest.py diff --git a/docs/re/BACKLOG.md b/docs/re/BACKLOG.md index 0f37cbab..6a71e208 100644 --- a/docs/re/BACKLOG.md +++ b/docs/re/BACKLOG.md @@ -827,6 +827,41 @@ unknown, what evidence exists, and what the first step would be. Move an item in field; the stat keys in records not opened here (`MegaTons`, `BulletLimit`, `CrewCount`, `HP_CLASS`); why the roster lists 59 of 131. +* โœ… **(2026-08-27) `Stage\script.tbl`'s 11 NON-`MISSION` FIELDS, READ AT LAST โ€” + and one of them lands on the mission DIALOGUE table.** + [structures/mission-script-manifest](structures/mission-script-manifest.md), + artefact `data/script-manifest.txt`, regenerator + `tools/re-capture/script_manifest.py`. + `structures/mission-script-ssb.md` owns this manifest and names its 40 fields + but never read the 11 that are not `MISSION = StageNN.ssb`. They are + `DIALOG_MESSAGE = MissionDialogMessage.tbl`, `DIALOG_LOCAL_STRING = + MissionDialog_local_string.tbl`, `FONT = dat\fonts.pak+HGRGE00.TTF`, + `FONT_SIZE 15`, `TEXT_POS 128,128,16`, `TEXT_LINES 30`, and the five + `pgmsg_{start,end,update,failed,restart}.prt`. Two sibling records were also + unread: **`GP_TEST` (`PATH = dat\GP_TEST\`, a debug archive NOT on the disc)** + and `TEXTS` (a second text style, 33/3/`296,565,39`). + ๐Ÿ”‘ Probed 7 values x 33 prefixes x 41 archives. **One resolves: + `message\MissionDialogMessage.tbl`**, in all six `GP_MAIN_GAME_*` paks โ€” + **200 records, 25 280 bytes**, every name `S_P_` with five kinds + 40 each (`HINT_PAUSE`, `HINT`, `OBJECTIVE`, `GRAPH`, `LOSE`), fields + **positional, tagged 0..3**, each value a message key + (`S10_P1_HINT_PAUSE` -> `S10_P1_Hint_Pause_00.._03`). An index from + (stage, phase, kind) to the localised strings, on the same `S_P` keying + the ISL corpus already uses. + ๐Ÿงช **Control**: the 40 stage-phases span stages **1-16 and 24-29** โ€” a subset + of the 28 shipped, and the six with NO hints are exactly **18-23, the + tutorials**. A fifth independent route to the story/tutorial split, and it + hands over the **phase count per stage** (1 to 3; S02/S03/S06/S09 have 3, + S10/S13 and all of S24-S29 have 1). + ๐ŸŸก **The other six do not resolve**, with the control in the same sweep: + `MissionDialog_local_string.tbl` and all five `pgmsg_*.prt` are not a pak entry + under any of the 33 prefixes, while `message\MissionDialogMessage.tbl` and + `Stage\script.tbl` both resolve in 6 archives. Same wall + `mission-phase-advance.md` hit for the manifest's KEYS; this closes the VALUES. + โš ๏ธ **`name_hash` is CASE-INSENSITIVE** (`message\` == `Message\`); `tag_hash` + is not. Already noted in `sound-pak-contents.md`; repeated because it doubles + the apparent hit count of a prefix sweep. + * โœ… **(2026-08-27) THE STAGE NUMBERING IS CONSISTENT EVERYWHERE โ€” and the manifest still declares the cut S17.** Also a **correction to my own claim** in the entry below. diff --git a/docs/re/INDEX.md b/docs/re/INDEX.md index 1bf10949..5c0fe369 100644 --- a/docs/re/INDEX.md +++ b/docs/re/INDEX.md @@ -99,6 +99,7 @@ files, which is how the same ground got covered twice. | [`structures/sound-cue-table.md`](structures/sound-cue-table.md) | The cue index in `tables.pak` โ€” message id -> cue -> sound id -> `.slb` bank | โœ… CONFIRMED, 1 326/1 338 script message ids bind to a bank; SOUNDS and FILES agree on the same 12 absentees, 0 orphan files | | [`structures/cutscene-message-table.md`](structures/cutscene-message-table.md) | Cutscene dialogue โ€” speaker, portrait, on-screen seconds, audio cue per page | โœ… CONFIRMED, field count = 9ยทPageCount+2 for all 7 PageCounts, 1 252/1 252 caption keys match, 138 ids close both ways | | [`structures/isl-mission-timer.md`](structures/isl-mission-timer.md) | The mission timer โ€” two sequential countdowns, not a limit and a warning | โœ… CONFIRMED โ€” read from `sub_822639B8`; the threshold reading is | +| [`structures/mission-script-manifest.md`](structures/mission-script-manifest.md) | `Stage\script.tbl`'s 11 non-`MISSION` fields, and the mission dialogue table they name | โœ… CONFIRMED โ€” 1 of 7 values resolves (`message\MissionDialogMessage.tbl`, 200 records), controls in the same sweep | | [`structures/stage-numbering-and-player-craft.md`](structures/stage-numbering-and-player-craft.md) | The disc's stage numbering (S01โ€“S16, S18โ€“S29) and the player's craft per stage | โœ…โœ… CONFIRMED โ€” two independent files close on 6 + 15 + 1 + 6 = 28 | | [`structures/hangar-loadout-system.md`](structures/hangar-loadout-system.md) | The Hangar loadout system โ€” loadout โ†’ per-slot allow-list โ†’ arsenal item | โœ… CONFIRMED โ€” 60/60 slot refs, 70/88 item refs (18 = the `No_Equipment` sentinel) | | [`structures/arsenal-item-weapon-chain.md`](structures/arsenal-item-weapon-chain.md) | The 59-of-131 arsenal question โ€” an item names a hardpoint slot, not a weapon | โœ…โœ… CONFIRMED โ€” 0/59 `PlayerWeapon` are a `Weapon.ID`, 59/59 are a `Turret_NNN` slot | diff --git a/docs/re/data/script-manifest.txt b/docs/re/data/script-manifest.txt new file mode 100644 index 00000000..88bf3eae --- /dev/null +++ b/docs/re/data/script-manifest.txt @@ -0,0 +1,49 @@ +# Stage\script.tbl -- the mission-script manifest +# Regenerate: python3 tools/re-capture/script_manifest.py +# See docs/re/structures/mission-script-manifest.md + +## the three records + SCRIPTS 40 fields + GP_TEST 1 fields + PATH 'dat\\GP_TEST\\' + TEXTS 3 fields + FONT_SIZE '33' + TEXT_LINES '3' + TEXT_POS '296,565,39' + +## SCRIPTS -- the 11 non-MISSION fields, read for the first time + DIALOG_LOCAL_STRING 'MissionDialog_local_string.tbl' + DIALOG_MESSAGE 'MissionDialogMessage.tbl' + FONT 'dat\\fonts.pak+HGRGE00.TTF' + FONT_SIZE '15' + MISSION_END_PRT 'pgmsg_end.prt' + MISSION_FAILED_PRT 'pgmsg_failed.prt' + MISSION_RESTART_PRT 'pgmsg_restart.prt' + MISSION_START_PRT 'pgmsg_start.prt' + MISSION_UPDATE_PRT 'pgmsg_update.prt' + TEXT_LINES '30' + TEXT_POS '128,128,16' + +## do those values resolve as pak entries? (33 prefixes x 41 archives) + MissionDialogMessage.tbl FOUND as message\MissionDialogMessage.tbl in 6 archives + MissionDialog_local_string.tbl not found + pgmsg_start.prt not found + pgmsg_end.prt not found + pgmsg_update.prt not found + pgmsg_failed.prt not found + pgmsg_restart.prt not found + CONTROL Stage\script.tbl resolves in 6 archives + NOTE name_hash is CASE-INSENSITIVE (message\ == Message\); tag_hash is not. + +## message\MissionDialogMessage.tbl -- 200 records, 25280 bytes + name families S_P_: {'HINT_PAUSE': 40, 'HINT': 40, 'LOSE': 40, 'GRAPH': 40, 'OBJECTIVE': 40} + field counts: {4: 120, 3: 40, 1: 40} + stages present (22): [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 24, 25, 26, 27, 28, 29] + phases per stage: {1: 2, 2: 3, 3: 3, 4: 2, 5: 2, 6: 3, 7: 2, 8: 2, 9: 3, 10: 1, 11: 2, 12: 2, 13: 1, 14: 2, 15: 2, 16: 2, 24: 1, 25: 1, 26: 1, 27: 1, 28: 1, 29: 1} + CONTROL subset of the 28 shipped stages: True ; shipped with NO hints: [18, 19, 20, 21, 22, 23] + + sample S10_P1_HINT_PAUSE: + tag 0 'S10_P1_Hint_Pause_00' + tag 1 'S10_P1_Hint_Pause_01' + tag 2 'S10_P1_Hint_Pause_02' + tag 3 'S10_P1_Hint_Pause_03' diff --git a/docs/re/structures/mission-script-manifest.md b/docs/re/structures/mission-script-manifest.md new file mode 100644 index 00000000..8ba48236 --- /dev/null +++ b/docs/re/structures/mission-script-manifest.md @@ -0,0 +1,77 @@ +# โœ… `Stage\script.tbl` โ€” the 11 non-`MISSION` fields, and where they point + +[mission-script-ssb](mission-script-ssb.md) owns this manifest and names its +40 fields; **it never read the 11 that are not `MISSION = StageNN.ssb`.** +This does, and follows each value into the archives. Artefact +`../data/script-manifest.txt`, regenerator `tools/re-capture/script_manifest.py`. + +``` +DIALOG_MESSAGE MissionDialogMessage.tbl +DIALOG_LOCAL_STRING MissionDialog_local_string.tbl +FONT dat\fonts.pak+HGRGE00.TTF FONT_SIZE 15 +TEXT_POS 128,128,16 TEXT_LINES 30 +MISSION_START_PRT pgmsg_start.prt MISSION_END_PRT pgmsg_end.prt +MISSION_UPDATE_PRT pgmsg_update.prt MISSION_FAILED_PRT pgmsg_failed.prt +MISSION_RESTART_PRT pgmsg_restart.prt +``` + +Two sibling records in the same file, also unread until now: +`GP_TEST` (`PATH = dat\GP_TEST\` โ€” a debug archive that is **not on the disc**) +and `TEXTS` (`FONT_SIZE 33`, `TEXT_LINES 3`, `TEXT_POS 296,565,39` โ€” a second, +larger text style beside `SCRIPTS`'s own 15/30/`128,128,16`). + +## ๐Ÿ”‘ `message\MissionDialogMessage.tbl` โ€” found, and it is keyed by stage AND phase + +Probed 7 values ร— 33 prefixes ร— 41 archives. **One resolves**: +`message\MissionDialogMessage.tbl`, in all six `GP_MAIN_GAME_*` paks. +**200 records, 25 280 bytes**, every name of the form +`S_P_` with five kinds, 40 each: + +| kind | fields | reading | +|---|---:|---| +| `HINT_PAUSE` | 4 | the pause menu's hint lines | +| `HINT` | 4 | ๐ŸŸก in-mission hint | +| `OBJECTIVE` | 4 | ๐ŸŸก the objective text | +| `GRAPH` | 3 | ๐ŸŸก | +| `LOSE` | 1 | ๐ŸŸก the failure line | + +The fields are **positional, tagged 0โ€ฆ3**, and each value is a message key: +`S10_P1_HINT_PAUSE` โ†’ `S10_P1_Hint_Pause_00 โ€ฆ _03`. So this is an index from +(stage, phase, kind) to the localised strings โ€” the same `S_P` keying the +ISL corpus uses for [mission phases](isl-phase-guards.md). + +๐Ÿงช **Control โ€” the 22 stages are exactly the STORY stages.** The 40 stage-phases +span stages **1โ€“16 and 24โ€“29**; that is a subset of the 28 shipped, and the six +with no hints at all are **18โ€“23**, the tutorials +([stage-numbering-and-player-craft](stage-numbering-and-player-craft.md)). A +fifth independent route to the same story/tutorial split, and it also gives the +**phase count per stage** โ€” 1 to 3, e.g. S02 and S03 have 3, S10 and S13 have 1. + +## ๐ŸŸก The other six values do not resolve โ€” with a working control + +`MissionDialog_local_string.tbl` and all five `pgmsg_*.prt` are **not a pak +entry** under any of the 33 prefixes tried (`message\`, `Stage\`, `2d\`, `ui\`, +`prt\`, six language dirs, and the language dirs under `Stage\` and `message\`). +The control is in the same loop and in the same sweep: +`message\MissionDialogMessage.tbl` **does** resolve, and so does +`Stage\script.tbl` itself, both in 6 archives. + +This is the same wall [mission-phase-advance](../mission-phase-advance.md) hit +for the manifest's KEYS โ€” but that sweep probed `MISSION1..33` and +`MISSION_*_PRT` as names. This one probes the **values**, which had never been +read, and closes them the same way. + +## โš ๏ธ `name_hash` is CASE-INSENSITIVE + +`name_hash("message\\a.tbl") == name_hash("Message\\A.TBL")`. `tag_hash` is +**not** (`tag_hash("Abc") != tag_hash("abc")`). Already noted for archive lookup +in [sound-pak-contents](sound-pak-contents.md); worth repeating because it +doubles the apparent hit count of any prefix sweep. + +## ๐ŸŸก Not settled + +* Where the five `pgmsg_*.prt` screen parts and + `MissionDialog_local_string.tbl` actually live. +* What `GRAPH` selects, and why `LOSE` carries exactly one entry. +* `dat\GP_TEST\` is named by the manifest and is not on the disc โ€” a debug path + left in, not a missing asset. Not chased. diff --git a/tools/re-capture/script_manifest.py b/tools/re-capture/script_manifest.py new file mode 100644 index 00000000..45a271f5 --- /dev/null +++ b/tools/re-capture/script_manifest.py @@ -0,0 +1,86 @@ +#!/usr/bin/env python3 +"""`Stage\\script.tbl` -- the mission-script manifest, and what its values point at. + +Regenerates docs/re/data/script-manifest.txt. +""" +import sys, os, glob, re, collections + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, HERE) +import unitgroup as U +from unit_substructures import pak_entries + +def main(): + mg = glob.glob('/work/sylph_extract/**/GP_MAIN_GAME_E.pak', recursive=True)[0] + E = dict(pak_entries(mg)) + recs = U.parse(E[U.name_hash('Stage\\script.tbl')]) + + print("# Stage\\script.tbl -- the mission-script manifest") + print("# Regenerate: python3 tools/re-capture/script_manifest.py") + print("# See docs/re/structures/mission-script-manifest.md") + + print("\n## the three records") + for r in recs: + n = U.named(r) + print(" %-10s %2d fields" % (r['squadron'], len(r['fields']))) + if r['squadron'] == 'SCRIPTS': continue + for k, v in sorted(n.items()): + print(" %-22s %r" % (k, v)) + + sc = [r for r in recs if r['squadron'] == 'SCRIPTS'][0] + print("\n## SCRIPTS -- the 11 non-MISSION fields, read for the first time") + for t, n, v in sorted(sc['fields'], key=lambda x: (x[1] or '')): + if n and n.startswith('MISSION') and v.endswith('.ssb'): continue + print(" %-22s %r" % (n, v)) + + idx = {} + for pk in sorted(glob.glob('/work/sylph_extract/**/*.pak', recursive=True)): + idx[os.path.basename(pk)] = {h for h, b in pak_entries(pk)} + targets = ['MissionDialogMessage.tbl', 'MissionDialog_local_string.tbl', + 'pgmsg_start.prt', 'pgmsg_end.prt', 'pgmsg_update.prt', + 'pgmsg_failed.prt', 'pgmsg_restart.prt'] + pre = ['', 'Stage\\', 'stage\\', 'message\\', 'msg\\', 'dialog\\', 'script\\', + 'SCRIPTS\\', 'GP_SCRIPT\\', '2d\\', 'ui\\', 'prt\\', 'view\\', 'dat\\', 'hidden\\'] + for l in ['eng', 'jpn', 'fra', 'deu', 'ita', 'esp']: + pre += [l + '\\', 'Stage\\' + l + '\\', 'message\\' + l + '\\'] + print("\n## do those values resolve as pak entries? (%d prefixes x %d archives)" + % (len(pre), len(idx))) + for t in targets: + hits = sorted({p + t for p in pre if any(U.name_hash(p + t) in s for s in idx.values())}) + if hits: + for hname in hits[:1]: + where = sorted(pk for pk, s in idx.items() if U.name_hash(hname) in s) + print(" %-32s FOUND as %-40s in %d archives" % (t, hname, len(where))) + else: + print(" %-32s not found" % t) + c = U.name_hash('Stage\\script.tbl') + print(" CONTROL Stage\\script.tbl resolves in %d archives" + % sum(1 for s in idx.values() if c in s)) + print(" NOTE name_hash is CASE-INSENSITIVE (message\\ == Message\\); tag_hash is not.") + + b = E[U.name_hash('message\\MissionDialogMessage.tbl')] + dr = U.parse(b) + pat = re.compile(r'^S(\d+)_P(\d+)_(.+)$') + fam = collections.Counter(); st = collections.defaultdict(set); nf = collections.Counter() + for r in dr: + m = pat.match(r['squadron']) + if m: + fam[m.group(3)] += 1 + st[int(m.group(1))].add(int(m.group(2))) + nf[len(r['fields'])] += 1 + print("\n## message\\MissionDialogMessage.tbl -- %d records, %d bytes" % (len(dr), len(b))) + print(" name families S_P_: %s" % dict(fam)) + print(" field counts: %s" % dict(nf)) + print(" stages present (%d): %s" % (len(st), sorted(st))) + print(" phases per stage: %s" % {k: len(v) for k, v in sorted(st.items())}) + H = {h for h, _ in pak_entries(mg)} + ship = {i for i in range(0, 40) if U.name_hash('stage\\UnitGroup_S%02d.tbl' % i) in H} + print(" CONTROL subset of the %d shipped stages: %s ; shipped with NO hints: %s" + % (len(ship), set(st) <= ship, sorted(ship - set(st)))) + r0 = [r for r in dr if r['squadron'].endswith('_HINT_PAUSE')][0] + print("\n sample %s:" % r0['squadron']) + for t, n, v in r0['fields']: + print(" tag %-3s %-8s %r" % (t, n or '', v)) + +if __name__ == "__main__": + main()