diff --git a/docs/port/HANDOFF.md b/docs/port/HANDOFF.md index bb0b073..4c2ae3a 100644 --- a/docs/port/HANDOFF.md +++ b/docs/port/HANDOFF.md @@ -120,7 +120,7 @@ authored version can be deleted. ⚠️ **Correlation only — untested.** Nothing yet shows it *means* additive; the test is to blend bit-`0x02` sprites additively and re-correlate the title against the capture. Note `ptlogo_back2eff` is `0x8830` despite its name, so - this is a field and not a naming pattern. ❔ Not diagnosed — but two candidate meanings are now dead: **not** "the name contains `eff`" — and not even the one-way "bit ⇒ `eff` name", which held 10/10 on the title but fails on **2 657 of 4 995** bit-set sprites disc-wide (`P(eff|set) = 0.468`). What survives is a 3.3× association, and the counterexamples are rings, glows and lights — effect-like art without the naming convention and **not** "the element is transient" (`pteff03`/`pteff03a` carry the bit and persist to `t=250`). ⚠️ Practical note for an asset pipeline: `T8aD` headers appear in **RATC child order** (18/18 verified on build 4 against decoded dimensions), which is the only sound way to attribute a header to a name when two sprites share a size — and two here do. + this is a field and not a naming pattern. ❔ Not diagnosed — but two candidate meanings are now dead: **not** "the name contains `eff`" — and not even the one-way "bit ⇒ `eff` name", which held 10/10 on the title but fails on **2 657 of 4 995** bit-set sprites disc-wide (`P(eff|set) = 0.468`). What survives is a 3.3× association, and the counterexamples are rings, glows and lights — effect-like art without the naming convention and **not** "the element is transient" (`pteff03`/`pteff03a` carry the bit and persist to `t=250`). 🛑 **Parked** — four candidate meanings are now dead (additive blend; `eff` name, both directions; transient element; **premultiplied alpha**, refuted because flagged sprites violate `RGB ≤ A` *more* than unflagged, 55.5 % vs 33.7 %) and none produced a positive account. It blocks nothing — your screens composite at 0.947 correlation against a capture without it. ⚠️ Practical note for an asset pipeline: `T8aD` headers appear in **RATC child order** (18/18 verified on build 4 against decoded dimensions), which is the only sound way to attribute a header to a name when two sprites share a size — and two here do. ⚠️ **The rotated draw is NOT the swoosh.** It was identified as the swoosh by elimination; that is **refuted** — its quads span y −209…925 in screen space, the swoosh is a band at y 126…360. ✅ **It is the two `ptloop` sweeps** diff --git a/docs/re/METHOD.md b/docs/re/METHOD.md index 394a0e6..87baa98 100644 --- a/docs/re/METHOD.md +++ b/docs/re/METHOD.md @@ -700,3 +700,15 @@ agent's loop prompt, i.e. nowhere durable. See [`README.md`](README.md) for the `px_bunk_line` — rings, glows, lights, thin lines. That the exceptions are all effect-like artwork *without* the naming convention says more about the field than the rule it broke did. +* **Report a classifier's lift over its base rate, not its accuracy.** The + `0x02` bit's association with bright-RGB/low-alpha art gives a best single + threshold of 76.5 % accuracy — which sounds like a finding until you notice + that always guessing the majority class scores 64.1 %. The lift is 12 points, + the distributions overlap, and it is a tendency rather than a rule. An accuracy + quoted without its base rate is not interpretable. +* **Park a field after N failed hypotheses, and say what was eliminated.** Four + candidate meanings for one flag bit were each refuted at a cost of roughly an + iteration apiece, and the bit blocks nothing measurable. Stopping is the right + call, but only if the negative space is written down — additive blend, name, + lifetime, premultiplied alpha — so the next attempt starts where this one ended + rather than at the beginning. diff --git a/docs/re/REFUTED.md b/docs/re/REFUTED.md index 2d09056..4984c9f 100644 --- a/docs/re/REFUTED.md +++ b/docs/re/REFUTED.md @@ -511,3 +511,8 @@ neighbourhood, not just the line. fails on **2 657 of 4 995** bit-set ones. `P(eff|set) = 0.468` against `P(eff|clear) = 0.144` — an association, not an implication. [`ui-paint-order-key.md`](structures/ui-paint-order-key.md) +* "`T8aD +0x04` bit `0x02` selects premultiplied alpha" → **refuted.** + Premultiplied requires `RGB ≤ A` everywhere; over 170 decoded textures the + flagged group violates it on a median **52.5 %** of pixels against **30.2 %** + unflagged — both far from premultiplied, and the flagged group *further*. + [`ui-paint-order-key.md`](structures/ui-paint-order-key.md) diff --git a/docs/re/data/eff-bit-alpha-test.txt b/docs/re/data/eff-bit-alpha-test.txt new file mode 100644 index 0000000..1574d08 --- /dev/null +++ b/docs/re/data/eff-bit-alpha-test.txt @@ -0,0 +1,7 @@ +matched 170 decoded textures to a T8aD flag word + bit SET n= 61 mean %(RGB>A) 55.52 median 52.52 + bit clear n=109 mean %(RGB>A) 33.66 median 30.17 + + premultiplied would require ~0% for the flagged group -> REFUTED + best single-threshold accuracy: 76.5% at %(RGB>A) > 83.5 + (base rate, always-guess-majority: 64.1%) diff --git a/docs/re/structures/ui-paint-order-key.md b/docs/re/structures/ui-paint-order-key.md index 15b1eb5..6b36bd7 100644 --- a/docs/re/structures/ui-paint-order-key.md +++ b/docs/re/structures/ui-paint-order-key.md @@ -582,3 +582,45 @@ authoring intent rather than by name, which is a description, not a decode. `T8aD`, validated 17/18 on build 4 against the child order. The one mismatch is the known `pteff04.t32` → registered as `8AX` case, so this is the **element** (`opt `) name, not the sprite's registered name. + +### 🔴 Premultiplied alpha — a fourth candidate, refuted + +A per-sprite flag controlling **premultiplied vs straight alpha** would matter +enormously to a port (it changes the blend equation) and has a sharp static +signature: a premultiplied texture has `RGB ≤ A` everywhere. +[`tools/re-capture/eff_bit_alpha_test.py`](../../../tools/re-capture/eff_bit_alpha_test.py), +over the 170 decoded `GP_TITLE` textures that pair to a flag word +([data](../data/eff-bit-alpha-test.txt)): + +| group | n | mean %(RGB>A) | median | +|---|---|---|---| +| bit SET | 61 | **55.5** | 52.5 | +| bit clear | 109 | **33.7** | 30.2 | + +Premultiplied requires ~0 % for the flagged group. Both groups are far from it, +and the flagged group violates **more** — the opposite of the hypothesis. +**Refuted.** + +🟡 What is left is a weak association: flagged sprites carry more bright-RGB / +low-alpha pixels, which is what glow art looks like. But the distributions +overlap badly — the best single threshold classifies **76.5 %** against a +**64.1 %** base rate, a 12-point lift. That is a tendency, not a rule, and +nothing to build on. + +### ⚠️ One re-test worth remembering + +"`0x02` selects an additive blend" was refuted by *"blending those sprites +additively worsens every measure against the capture"* — but that ran against a +title render which has since been fixed twice (the `rest_plateau` bug, and the +`8AX` background our composer drops). The refutation may still stand; it was +simply measured through a renderer with known other errors. If `ui_layout::blit` +ever gains additive blending, it is worth one re-run. + +### 🛑 Parking this field + +Four candidate meanings are now dead — additive blend, `eff` name (both +directions), transient element, premultiplied alpha — and none of the four +searches produced a positive account. The bit blocks nothing: the port's screens +composite at 0.947 correlation against a capture without it. **Parked**, with the +search space narrowed and the attribution method (child order, not size) left +sound for whoever picks it up. diff --git a/tools/re-capture/eff_bit_alpha_test.py b/tools/re-capture/eff_bit_alpha_test.py new file mode 100755 index 0000000..349b7af --- /dev/null +++ b/tools/re-capture/eff_bit_alpha_test.py @@ -0,0 +1,58 @@ +"""Does bit 0x02 separate sprites by their RGB-vs-alpha content? + +Premultiplied alpha predicts RGB <= A everywhere for the flagged group. Tested +below and refuted. What remains is a description: how often RGB exceeds A, which +is the signature of glow art (bright colour carried at low alpha). + +Bit comes from the T8aD header; the name from the string immediately preceding +it (validated 17/18 on build 4 against the RATC child order). +""" +import struct, zlib, glob, re, os +import numpy as np +from PIL import Image +NAME = re.compile(rb'[A-Za-z0-9_.]{2,31}\x00') +base = "/work/sylph_extract/dat/GP_TITLE" +stub = open(base + ".pak", "rb").read() +n = struct.unpack_from(">I", stub, 4)[0] +blob = b"".join(open(s, "rb").read() for s in sorted(glob.glob(base + ".p[0-9][0-9]"))) +flags = {} # (entry_hash, name, w, h) -> flags +for i in range(n): + h_, off, sz = struct.unpack_from(">III", stub, 0x10 + 12 * i) + st = blob[off:off + sz] + if len(st) < 10: continue + try: d = zlib.decompress(st[10:]) if st[:2] == b"Z1" else st + except Exception: continue + for m in re.finditer(b"T8aD", d): + o = m.start() + try: + fl = struct.unpack_from(">I", d, o + 4)[0] + w = struct.unpack_from(">I", d, o + 0x14)[0] + hh = struct.unpack_from(">I", d, o + 0x18)[0] + except Exception: continue + if not (0 < w <= 4096 and 0 < hh <= 4096): continue + ms = list(NAME.finditer(d[max(0, o - 64):o])) + if not ms: continue + flags[(f"{h_:08x}", ms[-1].group()[:-1].decode("latin1"), w, hh)] = fl +rows = [] +for f in sorted(glob.glob("/tmp/tex/*.png")): + b = os.path.basename(f) + m = re.match(r"([0-9a-f]{8})_(.+)_(\d+)x(\d+)\.png$", b) + if not m: continue + key = (m.group(1), m.group(2), int(m.group(3)), int(m.group(4))) + fl = flags.get(key) + if fl is None: continue + a = np.asarray(Image.open(f).convert("RGBA")).astype(int) + rgb = a[:, :, :3].max(axis=2); al = a[:, :, 3] + rows.append((bool(fl & 2), 100 * float((rgb > al).mean()), m.group(2))) +s = [r[1] for r in rows if r[0]]; c = [r[1] for r in rows if not r[0]] +print(f"matched {len(rows)} decoded textures to a T8aD flag word") +print(f" bit SET n={len(s):3d} mean %(RGB>A) {np.mean(s):6.2f} median {np.median(s):6.2f}") +print(f" bit clear n={len(c):3d} mean %(RGB>A) {np.mean(c):6.2f} median {np.median(c):6.2f}") +print(f"\n premultiplied would require ~0% for the flagged group -> REFUTED") +# separability: what threshold best splits them, and how well? +best = (0, None) +for t in np.arange(0, 100, 0.5): + acc = (sum(x > t for x in s) + sum(x <= t for x in c)) / len(rows) + if acc > best[0]: best = (acc, t) +print(f" best single-threshold accuracy: {100*best[0]:.1f}% at %(RGB>A) > {best[1]}") +print(f" (base rate, always-guess-majority: {100*max(len(s),len(c))/len(rows):.1f}%)")