diff --git a/docs/re/BACKLOG.md b/docs/re/BACKLOG.md index 393c4789..ca9ed97d 100644 --- a/docs/re/BACKLOG.md +++ b/docs/re/BACKLOG.md @@ -143,6 +143,17 @@ where translucent sprites overlap. * ❔ What the field's bits mean — `0x8000`/`0x80a0`/`0xa110` look like flag words with a layer in some bits, not a plain depth. Sorting the whole word works on both measured screens; which bits carry the layer is unknown. + **Censused 2026-08-26** over all **4 525** disc sprites + (`tools/re-capture/paint_key_census.py`): ✅ the field is a **`u16` at `+0x0A`** + (upper half zero **4 525/4 525**), ✅ it is an **enumeration — 45 values**, and + 🔴 **it is pak-local, so it is not a global layer vocabulary**: only **4/45** + keys cross a pak family and **33/45** live only in `GP_MAIN_GAME_2D`, each + other pak owning a narrow high-byte band (`0x90`–`0x94` overlays, `0xa4` + mission log, `0xb1`–`0xb2` save/load). Supports "group id in the high bits, + order in the low bits" but does **not** test it — both measured screens are + inside one pak. ⚠️ My first count said 37/45 shared; that was the six + *language* copies of `GP_MAIN_GAME_2D` (identical key sets) counted as six + paks, and it pointed at the opposite conclusion. * ❔ A third measured permutation, to promote "holds on two" to a rule. The cheapest is a screen whose object is resident at the same time as the title's. * ❔ 341 builds now composite in an order no capture has checked. diff --git a/docs/re/data/paint-key-census.txt b/docs/re/data/paint-key-census.txt new file mode 100644 index 00000000..f64a8a54 --- /dev/null +++ b/docs/re/data/paint-key-census.txt @@ -0,0 +1,17 @@ +pak families: {'GP_CHALLENGE.pak': 2, 'GP_DEBRIEFING_PILOTLOG.pak': 2, 'GP_HANGAR_ARSENAL.pak': 3, 'GP_LEADERBOARD.pak': 3, 'GP_MAIN_GAME_2D.pak': 34, 'GP_MISSION_LOG.pak': 1, 'GP_MISSION_SELECT.pak': 1, 'GP_OPTIONS.pak': 1, 'GP_SAVE_LOAD.pak': 3, 'GP_STAGE_CLEAR.pak': 1} + +the six language 2D paks have identical key sets: True (6 paks) +keys in more than one FAMILY: 4/45 -> ['0000', '9092', '9110', '9200'] +keys confined to GP_MAIN_GAME_2D alone: 33/45 + +per family, its keys: + GP_CHALLENGE.pak: ['9110', '9200'] + GP_DEBRIEFING_PILOTLOG.pak: ['9091', '9092'] + GP_HANGAR_ARSENAL.pak: ['9200', '9300', '9400'] + GP_LEADERBOARD.pak: ['9108', '9110', '9200'] + GP_MAIN_GAME_2D.pak: ['0000', '7d00', '7e00', '7e10', '7e20', '7e40', '7e50', '7e51', '7e52', '7e80', '7e90', '7f00', '7f10', '7f20', '7f30', '7f40', '7f60', '7f61', '8000', '8010', '8020', '8030', '8040', '8080', '8100', '8108', '8110', '8200', '8210', '8300', '8800', '8880', '8900', '9820'] + GP_MISSION_LOG.pak: ['a410'] + GP_MISSION_SELECT.pak: ['9110'] + GP_OPTIONS.pak: ['0000'] + GP_SAVE_LOAD.pak: ['b100', 'b110', 'b210'] + GP_STAGE_CLEAR.pak: ['9092'] diff --git a/docs/re/structures/ui-paint-order-key.md b/docs/re/structures/ui-paint-order-key.md index 18008415..c5b8276d 100644 --- a/docs/re/structures/ui-paint-order-key.md +++ b/docs/re/structures/ui-paint-order-key.md @@ -323,3 +323,63 @@ The new group is suggestive: 10 and 11 are `kind = 0x2002` and 8 and 12 are So `kind` is not it either, and the count of refuted candidates is now seven: declaration order, RATC child order, first keyframe time, resting time, resting X/Y, the `T8aD` header words, and `kind`. + +## The corpus-wide census (2026-08-26) + +Everything above rests on twelve values from two screens. This walks **all 4 525 +`T8aD` sprites on the disc**, so the open question can be asked of the whole +corpus. `tools/re-capture/paint_key_census.py`, output in +[`../data/paint-key-census.txt`](../data/paint-key-census.txt). + +### ✅ The field is a `u16` at `+0x0A`, not a word at `+0x08` + +The upper half of the 32-bit word this page reads is **zero in 4 525 / 4 525** +sprites. Nothing above changes — `0x00008100` sorts identically to `0x8100` — +but the port should read two bytes, and a future value with the high half set +would mean something has been misread rather than that the layer got deeper. + +### ✅ It is an enumeration: 45 values for 4 525 sprites + +Not a per-sprite depth. The commonest key alone (`0x8100`) covers 1 188 sprites, +and the whole range is `0x0000`, `0x7d00`–`0x9820`, then `0xa410`, `0xb100`–`0xb210`. + +### 🔴 The keys are **pak-local** — so this is not a global layer vocabulary + +That was the reading worth trying, and it fails. Collapsing the six language +variants of `GP_MAIN_GAME_*2D.pak` into one family: + +| | | +|---|---| +| keys appearing in more than one pak family | **4 / 45** (`0x0000`, `0x9092`, `0x9110`, `0x9200`) | +| keys confined to `GP_MAIN_GAME_2D` alone | **33 / 45** | + +and each auxiliary pak occupies its own narrow high-byte band: + +| pak | its keys | +|---|---| +| `GP_MAIN_GAME_2D` | `0x7d00`–`0x9820` (34 keys) | +| `GP_DEBRIEFING_PILOTLOG` | `0x9091`, `0x9092` | +| `GP_LEADERBOARD` | `0x9108`, `0x9110`, `0x9200` | +| `GP_CHALLENGE` | `0x9110`, `0x9200` | +| `GP_HANGAR_ARSENAL` | `0x9200`, `0x9300`, `0x9400` | +| `GP_STAGE_CLEAR` | `0x9092` | +| `GP_MISSION_SELECT` | `0x9110` | +| `GP_MISSION_LOG` | `0xa410` | +| `GP_SAVE_LOAD` | `0xb100`, `0xb110`, `0xb210` | + +A screen that owns one or two keys is not using them to order itself internally — +it has nothing to order against. So the high bits look like a **group or owner +id** and the low bits like the ordering within it, which is the shape this page +already suspected. 🟡 That remains a *description of 45 numbers*: paint order has +been measured on two screens only, both inside `GP_MAIN_GAME_2D`, so there is no +ground truth anywhere else and the split between "group" and "order" bits is +still untested. + +### ⚠️ The first count I got was 37 / 45, and it was about localisation + +Counting paks rather than pak *families* makes every `GP_MAIN_GAME_2D` key look +like it appears in six paks. The six are the same screens in six languages — and +their key sets are **byte-for-byte identical** (verified: one distinct set across +all six). The inflated number would have supported exactly the wrong conclusion, +that the keys form a vocabulary shared across the whole UI. Recorded because the +shape recurs: a corpus with near-duplicate members will manufacture agreement. diff --git a/tools/re-capture/paint_key_census.py b/tools/re-capture/paint_key_census.py new file mode 100755 index 00000000..de20cd53 --- /dev/null +++ b/tools/re-capture/paint_key_census.py @@ -0,0 +1,49 @@ +#!/usr/bin/env python3 +"""Census the T8aD paint-order key across every sprite on the disc. + +`structures/ui-paint-order-key.md` established that this field sorts a screen's +paint order, from twelve values on two measured screens. This walks all 4 525 +sprites so the open question -- what the bits mean -- is asked of the whole +corpus rather than of one screen. + + ./paint_key_census.py + +Note the field is a **u16 at +0x0A**: the doc reads a 32-bit word at +0x08 and +its upper half is zero in every sprite. + +The language variants of `GP_MAIN_GAME_*2D.pak` are collapsed into one family. +They hold the same screens in six languages, and counting them separately makes +every one of their keys look shared across six paks -- which is how this script +first reported "37 of 45 keys appear in more than one pak", a statement about +localisation and not about the format. +""" + +import sys, glob, struct, re, collections +S="/tmp/claude-1000/-home-fabi-RE-Project-Sylpheed/b113cc12-4769-4ed8-ad66-c2b48f800773/scratchpad" +sys.path.insert(0,S+"/wt-root/Syplheed-Reborn/tools/re-capture") +exec(open(S+"/regn_decode.py").read().split("# ── the object")[0]) +Hh=lambda b,o: struct.unpack_from(">H",b,o)[0] +U=lambda b,o: struct.unpack_from(">I",b,o)[0] +def family(n): return re.sub(r'_[DEFIJS]2D\.pak$','_2D.pak',n) +byfam=collections.defaultdict(collections.Counter); keys=collections.Counter() +langsets=collections.defaultdict(set) +ROOT = sys.argv[1] if len(sys.argv) > 1 else "/work/sylph_extract" +for f in sorted(glob.glob(ROOT + "/**/*.pak", recursive=True)): + try: E=pak_entries(f) + except Exception: continue + base=f.split('/')[-1] + for h,b in (E.items() if isinstance(E,dict) else E): + if b[:4]!=b'T8aD': continue + k=Hh(b,10); keys[k]+=1; byfam[family(base)][k]+=1 + if base.endswith('2D.pak'): langsets[base].add(k) +fams=sorted(byfam) +print("pak families:",{f:len(c) for f,c in byfam.items()}) +print("\nthe six language 2D paks have identical key sets:", + len({frozenset(v) for v in langsets.values()})==1, f"({len(langsets)} paks)") +multi=[k for k in keys if sum(1 for f in fams if k in byfam[f])>1] +print(f"keys in more than one FAMILY: {len(multi)}/{len(keys)} -> {[('%04x'%k) for k in sorted(multi)]}") +only2d=[k for k in keys if list(f for f in fams if k in byfam[f])==['GP_MAIN_GAME_2D.pak']] +print(f"keys confined to GP_MAIN_GAME_2D alone: {len(only2d)}/{len(keys)}") +print("\nper family, its keys:") +for f in fams: + print(f" {f:>28}: {sorted('%04x'%k for k in byfam[f])}")