fabi/Syplheed-Reborn
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

re: retract the paint-key census and redo it over all 21 184 sprites

Browse Source 
The census filtered pak entries whose own first four bytes are T8aD.  A sprite
is usually a child of a RATC bundle, and a bundle entry's magic is RATC, so a
top-level magic filter cannot see one:

    top-level T8aD entries (counted)    4 525 sprites,  45 keys
    T8aD inside RATC bundles (missed)  16 659 sprites, 204 keys
    both                               21 184 sprites, 216 keys

171 of the 216 keys exist only inside bundles.  The sharpest statement of the
error: that census never saw GP_TITLE.pak at all -- the pak holding both of the
screens this page's entire evidence comes from.

Retracted: "45 values", "the keys are pak-local", "each auxiliary pak occupies
its own narrow high-byte band".  On the full population 68/216 keys (31%, not
9%) cross a pak family and the per-pak ranges overlap heavily -- GP_BUNK
0x8000-0xa110, GP_TITLE 0x8000-0xc150, GP_LEADERBOARD 0x8000-0xf100.  The tidy
banding was an artifact of seeing one or two keys per pak.  So the key looks
like a shared vocabulary, which is the opposite of what I published.

Survives, now on the full population: the field is a u16 at +0x0A (upper half
zero 21 184/21 184), and it is an enumeration (216 values for 21 184 sprites).

Three wrong numbers on this page now, all the same shape -- a statistic computed
over a population I had not checked was the population in question.  Stated once
at the end of the section rather than three times: check the sampling frame
before the statistic.
This commit is contained in:
Sylpheed RE agent
2026-08-26 10:06:23 +00:00
parent dbeab6aa9c
commit 0a64728fae
4 changed files with 145 additions and 107 deletions
Download Patch File Download Diff File Expand all files Collapse all files

108
docs/re/structures/ui-paint-order-key.md
View File

@@ -326,72 +326,72 @@ X/Y, the `T8aD` header words, and `kind`.
## The corpus-wide census (2026-08-26)
Everything above rests on twelve values from two screens. This walks **4 525
`T8aD` sprites on the disc**, so the open question can be asked of a corpus.
`tools/re-capture/paint_key_census.py`, output in
[`../data/paint-key-census.txt`](../data/paint-key-census.txt).
[`../data/paint-key-census.txt`](../data/paint-key-census.txt). **All 21 184
`T8aD` sprites on the disc** — pak entries *and* `RATC` children.
> ⚠️ **Read the next paragraph before using these numbers.** The census counts
> pak entries whose *own* first four bytes are `T8aD`. But the sprites this page
> measures paint order on are **children of a `RATC` bundle** — `ui_layout.rs`
> reaches them through `ratc::parse`, and a bundle entry's magic is `RATC`, so a
> child `T8aD` is invisible to a top-level magic filter. The two populations may
> overlap partly, wholly, or not at all, and **which one the 45 keys describe is
> being measured, not assumed.** Until that lands, treat every count below as a
> statement about top-level `T8aD` entries only.
>
> This is the same failure shape as the 37/45 note further down: a number
> computed over a population I had not checked was the population in question.
### ⚠️ The first version of this section counted 4 525 sprites, and was retracted
It filtered pak entries whose own first four bytes are `T8aD`. But a sprite is
usually a **child of a `RATC` bundle**, and a bundle entry's magic is `RATC`, so
a top-level magic filter cannot see one:
| | sprites | distinct keys |
|---|---|---|
| top-level `T8aD` entries (what was counted) | 4 525 | 45 |
| `T8aD` inside `RATC` bundles (what was missed) | **16 659** | **204** |
| both | **21 184** | **216** |
**171 of the 216 keys exist only inside bundles.** The sharpest way to put it:
that census never saw **`GP_TITLE.pak` at all** — the pak holding both screens
this page's evidence comes from. Every number it produced described a fifth of
the corpus, chosen by an accident of container nesting.
Retracted from it: "45 values", "the keys are pak-local", and "each auxiliary pak
occupies its own narrow high-byte band". What survives is below.
### ✅ The field is a `u16` at `+0x0A`, not a word at `+0x08`
The upper half of the 32-bit word this page reads is **zero in 4 525 / 4 525**
sprites. Nothing above changes — `0x00008100` sorts identically to `0x8100` —
but the port should read two bytes, and a future value with the high half set
would mean something has been misread rather than that the layer got deeper.
The upper half of the 32-bit word this page reads is **zero in 21 184 / 21 184**
sprites. Nothing above changes — `0x00008100` sorts identically to `0x8100` — but
the port should read two bytes, and a value with the high half set would mean
something has been misread rather than that the layer got deeper.
### ✅ It is an enumeration: 45 values for 4 525 sprites
### ✅ It is an enumeration: 216 values for 21 184 sprites
Not a per-sprite depth. The commonest key alone (`0x8100`) covers 1 188 sprites,
and the whole range is `0x0000`, `0x7d00`–`0x9820`, then `0xa410`, `0xb100`–`0xb210`.
Not a per-sprite depth. The commonest key (`0x8000`) covers 1 768 sprites,
`0x8100` another 1 524.
### 🔴 The keys are **pak-local** — so this is not a global layer vocabulary
### 🔴 Refuted: the keys are **not** partitioned by pak
That was the reading worth trying, and it fails. Collapsing the six language
variants of `GP_MAIN_GAME_*2D.pak` into one family:
This was the previous section's headline and the fuller population kills it.
| | |
|---|---|
| keys appearing in more than one pak family | **4 / 45** (`0x0000`, `0x9092`, `0x9110`, `0x9200`) |
| keys confined to `GP_MAIN_GAME_2D` alone | **33 / 45** |
| | 4 525-sprite version | **all 21 184** |
|---|---|---|
| keys crossing a pak family | 4 / 45 = 9 % | **68 / 216 = 31 %** |
| keys confined to `GP_MAIN_GAME_2D` | 33 / 45 | **52 / 216** |
and each auxiliary pak occupies its own narrow high-byte band:
And the per-pak *ranges overlap heavily* rather than banding: `GP_BUNK`
`0x8000`–`0xa110`, `GP_CHALLENGE` `0x8000`–`0x9200`, `GP_HANGAR_ARSENAL`
`0x8000`–`0x9412`, `GP_LEADERBOARD` `0x8000`–`0xf100`, `GP_TITLE`
`0x8000`–`0xc150`. The tidy "each pak owns a high-byte band" picture was an
artifact of seeing one or two keys per pak. Only three paks are actually narrow
(`GP_GAMEOVER` `0xd840`–`0xd862`, `GP_MISSION_LOG` `0xa400`–`0xa420`,
`GP_SYSTEM` `0xa010`–`0xa112`).
| pak | its keys |
|---|---|
| `GP_MAIN_GAME_2D` | `0x7d00`–`0x9820` (34 keys) |
| `GP_DEBRIEFING_PILOTLOG` | `0x9091`, `0x9092` |
| `GP_LEADERBOARD` | `0x9108`, `0x9110`, `0x9200` |
| `GP_CHALLENGE` | `0x9110`, `0x9200` |
| `GP_HANGAR_ARSENAL` | `0x9200`, `0x9300`, `0x9400` |
| `GP_STAGE_CLEAR` | `0x9092` |
| `GP_MISSION_SELECT` | `0x9110` |
| `GP_MISSION_LOG` | `0xa410` |
| `GP_SAVE_LOAD` | `0xb100`, `0xb110`, `0xb210` |
So the key is closer to a **shared vocabulary** than to a per-screen depth — the
opposite of what the under-sampled census said. ❔ Which bits carry the layer
remains unknown, and is now a question about 216 values rather than 45.
A screen that owns one or two keys is not using them to order itself internally —
it has nothing to order against. So the high bits look like a **group or owner
id** and the low bits like the ordering within it, which is the shape this page
already suspected. 🟡 That remains a *description of 45 numbers*: paint order has
been measured on two screens only, both inside `GP_MAIN_GAME_2D`, so there is no
ground truth anywhere else and the split between "group" and "order" bits is
still untested.
### ✅ The language paks are one screen set six times
### ⚠️ The first count I got was 37 / 45, and it was about localisation
The six `GP_MAIN_GAME_*2D.pak` have **byte-for-byte identical key sets**. This is
why counting paks rather than pak *families* first reported "37 of 45 keys appear
in more than one pak" — a statement about localisation, not about the format.
Counting paks rather than pak *families* makes every `GP_MAIN_GAME_2D` key look
like it appears in six paks. The six are the same screens in six languages — and
their key sets are **byte-for-byte identical** (verified: one distinct set across
all six). The inflated number would have supported exactly the wrong conclusion,
that the keys form a vocabulary shared across the whole UI. Recorded because the
shape recurs: a corpus with near-duplicate members will manufacture agreement.
### The recurring failure, stated once
Three published numbers on this page were wrong the same way: each was computed
over a population I had not checked was the population in question — the six
language copies, then the top-level-only sprites, then the size of that gap. The
analysis was never the problem. **Check the sampling frame before the statistic.**