From fdcb5e94e03615674b912c3fee70af4f18da6f97 Mon Sep 17 00:00:00 2001 From: Sylpheed RE agent Date: Wed, 26 Aug 2026 09:50:29 +0000 Subject: [PATCH] re: census the T8aD paint-order key -- it is pak-local, not a global vocabulary The page rested on twelve values from two screens. This walks all 4525 sprites on the disc. * The field is a u16 at +0x0A. The upper half of the 32-bit word the page reads is zero in 4525/4525. Nothing above changes -- 0x00008100 sorts the same as 0x8100 -- but a future value with the high half set would mean something had been misread rather than that the layer got deeper. * It is an enumeration: 45 values for 4525 sprites, one of which (0x8100) covers 1188 of them. * The reading worth trying -- a global layer vocabulary shared across the UI -- is refuted. Only 4 of 45 keys cross a pak family and 33 of 45 live only in GP_MAIN_GAME_2D; every other pak owns a narrow high-byte band (0x90-0x94 for the in-game overlays, 0xa4 mission log, 0xb1-0xb2 save/load). A screen that owns one or two keys is not ordering itself with them. That supports "group id in the high bits, order in the low bits", which is what the page already suspected, but it does NOT test it: paint order has been measured on two screens and both are inside GP_MAIN_GAME_2D, so there is no ground truth to check the split against. Left amber. The first number I got was 37/45 shared, which would have supported precisely the wrong conclusion. It came from counting paks instead of pak families: the six GP_MAIN_GAME_*2D paks are the same screens in six languages and their key sets are byte-for-byte identical. Recorded on the page, because the shape recurs -- a corpus with near-duplicate members manufactures agreement. --- docs/re/BACKLOG.md | 11 +++++ docs/re/data/paint-key-census.txt | 17 +++++++ docs/re/structures/ui-paint-order-key.md | 60 ++++++++++++++++++++++++ tools/re-capture/paint_key_census.py | 49 +++++++++++++++++++ 4 files changed, 137 insertions(+) create mode 100644 docs/re/data/paint-key-census.txt create mode 100755 tools/re-capture/paint_key_census.py diff --git a/docs/re/BACKLOG.md b/docs/re/BACKLOG.md index 393c478..ca9ed97 100644 --- a/docs/re/BACKLOG.md +++ b/docs/re/BACKLOG.md @@ -143,6 +143,17 @@ where translucent sprites overlap. * ❔ What the field's bits mean — `0x8000`/`0x80a0`/`0xa110` look like flag words with a layer in some bits, not a plain depth. Sorting the whole word works on both measured screens; which bits carry the layer is unknown. + **Censused 2026-08-26** over all **4 525** disc sprites + (`tools/re-capture/paint_key_census.py`): ✅ the field is a **`u16` at `+0x0A`** + (upper half zero **4 525/4 525**), ✅ it is an **enumeration — 45 values**, and + 🔴 **it is pak-local, so it is not a global layer vocabulary**: only **4/45** + keys cross a pak family and **33/45** live only in `GP_MAIN_GAME_2D`, each + other pak owning a narrow high-byte band (`0x90`–`0x94` overlays, `0xa4` + mission log, `0xb1`–`0xb2` save/load). Supports "group id in the high bits, + order in the low bits" but does **not** test it — both measured screens are + inside one pak. ⚠️ My first count said 37/45 shared; that was the six + *language* copies of `GP_MAIN_GAME_2D` (identical key sets) counted as six + paks, and it pointed at the opposite conclusion. * ❔ A third measured permutation, to promote "holds on two" to a rule. The cheapest is a screen whose object is resident at the same time as the title's. * ❔ 341 builds now composite in an order no capture has checked. diff --git a/docs/re/data/paint-key-census.txt b/docs/re/data/paint-key-census.txt new file mode 100644 index 0000000..f64a8a5 --- /dev/null +++ b/docs/re/data/paint-key-census.txt @@ -0,0 +1,17 @@ +pak families: {'GP_CHALLENGE.pak': 2, 'GP_DEBRIEFING_PILOTLOG.pak': 2, 'GP_HANGAR_ARSENAL.pak': 3, 'GP_LEADERBOARD.pak': 3, 'GP_MAIN_GAME_2D.pak': 34, 'GP_MISSION_LOG.pak': 1, 'GP_MISSION_SELECT.pak': 1, 'GP_OPTIONS.pak': 1, 'GP_SAVE_LOAD.pak': 3, 'GP_STAGE_CLEAR.pak': 1} + +the six language 2D paks have identical key sets: True (6 paks) +keys in more than one FAMILY: 4/45 -> ['0000', '9092', '9110', '9200'] +keys confined to GP_MAIN_GAME_2D alone: 33/45 + +per family, its keys: + GP_CHALLENGE.pak: ['9110', '9200'] + GP_DEBRIEFING_PILOTLOG.pak: ['9091', '9092'] + GP_HANGAR_ARSENAL.pak: ['9200', '9300', '9400'] + GP_LEADERBOARD.pak: ['9108', '9110', '9200'] + GP_MAIN_GAME_2D.pak: ['0000', '7d00', '7e00', '7e10', '7e20', '7e40', '7e50', '7e51', '7e52', '7e80', '7e90', '7f00', '7f10', '7f20', '7f30', '7f40', '7f60', '7f61', '8000', '8010', '8020', '8030', '8040', '8080', '8100', '8108', '8110', '8200', '8210', '8300', '8800', '8880', '8900', '9820'] + GP_MISSION_LOG.pak: ['a410'] + GP_MISSION_SELECT.pak: ['9110'] + GP_OPTIONS.pak: ['0000'] + GP_SAVE_LOAD.pak: ['b100', 'b110', 'b210'] + GP_STAGE_CLEAR.pak: ['9092'] diff --git a/docs/re/structures/ui-paint-order-key.md b/docs/re/structures/ui-paint-order-key.md index 1800841..c5b8276 100644 --- a/docs/re/structures/ui-paint-order-key.md +++ b/docs/re/structures/ui-paint-order-key.md @@ -323,3 +323,63 @@ The new group is suggestive: 10 and 11 are `kind = 0x2002` and 8 and 12 are So `kind` is not it either, and the count of refuted candidates is now seven: declaration order, RATC child order, first keyframe time, resting time, resting X/Y, the `T8aD` header words, and `kind`. + +## The corpus-wide census (2026-08-26) + +Everything above rests on twelve values from two screens. This walks **all 4 525 +`T8aD` sprites on the disc**, so the open question can be asked of the whole +corpus. `tools/re-capture/paint_key_census.py`, output in +[`../data/paint-key-census.txt`](../data/paint-key-census.txt). + +### ✅ The field is a `u16` at `+0x0A`, not a word at `+0x08` + +The upper half of the 32-bit word this page reads is **zero in 4 525 / 4 525** +sprites. Nothing above changes — `0x00008100` sorts identically to `0x8100` — +but the port should read two bytes, and a future value with the high half set +would mean something has been misread rather than that the layer got deeper. + +### ✅ It is an enumeration: 45 values for 4 525 sprites + +Not a per-sprite depth. The commonest key alone (`0x8100`) covers 1 188 sprites, +and the whole range is `0x0000`, `0x7d00`–`0x9820`, then `0xa410`, `0xb100`–`0xb210`. + +### 🔴 The keys are **pak-local** — so this is not a global layer vocabulary + +That was the reading worth trying, and it fails. Collapsing the six language +variants of `GP_MAIN_GAME_*2D.pak` into one family: + +| | | +|---|---| +| keys appearing in more than one pak family | **4 / 45** (`0x0000`, `0x9092`, `0x9110`, `0x9200`) | +| keys confined to `GP_MAIN_GAME_2D` alone | **33 / 45** | + +and each auxiliary pak occupies its own narrow high-byte band: + +| pak | its keys | +|---|---| +| `GP_MAIN_GAME_2D` | `0x7d00`–`0x9820` (34 keys) | +| `GP_DEBRIEFING_PILOTLOG` | `0x9091`, `0x9092` | +| `GP_LEADERBOARD` | `0x9108`, `0x9110`, `0x9200` | +| `GP_CHALLENGE` | `0x9110`, `0x9200` | +| `GP_HANGAR_ARSENAL` | `0x9200`, `0x9300`, `0x9400` | +| `GP_STAGE_CLEAR` | `0x9092` | +| `GP_MISSION_SELECT` | `0x9110` | +| `GP_MISSION_LOG` | `0xa410` | +| `GP_SAVE_LOAD` | `0xb100`, `0xb110`, `0xb210` | + +A screen that owns one or two keys is not using them to order itself internally — +it has nothing to order against. So the high bits look like a **group or owner +id** and the low bits like the ordering within it, which is the shape this page +already suspected. 🟡 That remains a *description of 45 numbers*: paint order has +been measured on two screens only, both inside `GP_MAIN_GAME_2D`, so there is no +ground truth anywhere else and the split between "group" and "order" bits is +still untested. + +### ⚠️ The first count I got was 37 / 45, and it was about localisation + +Counting paks rather than pak *families* makes every `GP_MAIN_GAME_2D` key look +like it appears in six paks. The six are the same screens in six languages — and +their key sets are **byte-for-byte identical** (verified: one distinct set across +all six). The inflated number would have supported exactly the wrong conclusion, +that the keys form a vocabulary shared across the whole UI. Recorded because the +shape recurs: a corpus with near-duplicate members will manufacture agreement. diff --git a/tools/re-capture/paint_key_census.py b/tools/re-capture/paint_key_census.py new file mode 100755 index 0000000..de20cd5 --- /dev/null +++ b/tools/re-capture/paint_key_census.py @@ -0,0 +1,49 @@ +#!/usr/bin/env python3 +"""Census the T8aD paint-order key across every sprite on the disc. + +`structures/ui-paint-order-key.md` established that this field sorts a screen's +paint order, from twelve values on two measured screens. This walks all 4 525 +sprites so the open question -- what the bits mean -- is asked of the whole +corpus rather than of one screen. + + ./paint_key_census.py + +Note the field is a **u16 at +0x0A**: the doc reads a 32-bit word at +0x08 and +its upper half is zero in every sprite. + +The language variants of `GP_MAIN_GAME_*2D.pak` are collapsed into one family. +They hold the same screens in six languages, and counting them separately makes +every one of their keys look shared across six paks -- which is how this script +first reported "37 of 45 keys appear in more than one pak", a statement about +localisation and not about the format. +""" + +import sys, glob, struct, re, collections +S="/tmp/claude-1000/-home-fabi-RE-Project-Sylpheed/b113cc12-4769-4ed8-ad66-c2b48f800773/scratchpad" +sys.path.insert(0,S+"/wt-root/Syplheed-Reborn/tools/re-capture") +exec(open(S+"/regn_decode.py").read().split("# ── the object")[0]) +Hh=lambda b,o: struct.unpack_from(">H",b,o)[0] +U=lambda b,o: struct.unpack_from(">I",b,o)[0] +def family(n): return re.sub(r'_[DEFIJS]2D\.pak$','_2D.pak',n) +byfam=collections.defaultdict(collections.Counter); keys=collections.Counter() +langsets=collections.defaultdict(set) +ROOT = sys.argv[1] if len(sys.argv) > 1 else "/work/sylph_extract" +for f in sorted(glob.glob(ROOT + "/**/*.pak", recursive=True)): + try: E=pak_entries(f) + except Exception: continue + base=f.split('/')[-1] + for h,b in (E.items() if isinstance(E,dict) else E): + if b[:4]!=b'T8aD': continue + k=Hh(b,10); keys[k]+=1; byfam[family(base)][k]+=1 + if base.endswith('2D.pak'): langsets[base].add(k) +fams=sorted(byfam) +print("pak families:",{f:len(c) for f,c in byfam.items()}) +print("\nthe six language 2D paks have identical key sets:", + len({frozenset(v) for v in langsets.values()})==1, f"({len(langsets)} paks)") +multi=[k for k in keys if sum(1 for f in fams if k in byfam[f])>1] +print(f"keys in more than one FAMILY: {len(multi)}/{len(keys)} -> {[('%04x'%k) for k in sorted(multi)]}") +only2d=[k for k in keys if list(f for f in fams if k in byfam[f])==['GP_MAIN_GAME_2D.pak']] +print(f"keys confined to GP_MAIN_GAME_2D alone: {len(only2d)}/{len(keys)}") +print("\nper family, its keys:") +for f in fams: + print(f" {f:>28}: {sorted('%04x'%k for k in byfam[f])}")