diff --git a/docs/port/HANDOFF.md b/docs/port/HANDOFF.md index 0f8a35e..f3da9ec 100644 --- a/docs/port/HANDOFF.md +++ b/docs/port/HANDOFF.md @@ -282,8 +282,21 @@ authored version can be deleted. pose k **to** pose k+1; neither is held unless they are equal — which is a plateau, and the plateau path has already returned by then. **So any element with no two adjacent identical poses has a guessed rest pose**, in our renderer - and in anything built from it. You can test for that in one pass over the - keyframes; consider flagging it rather than inheriting our guess. + and in anything built from it. Measured disc-wide: **3 807 of 15 493 elements + (24.57 %)**, of which **195 get a degenerate `scale = 0 %` pose**, and the two + candidate rules agree only **50.2 %** of the time. + ✅ **This does not block you.** On the five screens the port needs, 14 elements + are affected and the candidate rules **agree on 13**. The single disagreement + is `palogo_anima_eff.t32` on the developer splash, and the current answer is + the defensible one there — see below. Still worth flagging plateau-less + elements in an export rather than silently inheriting our guess; it is one pass + over the keyframes. + 🔴 **"rest = last keyframe" is refuted** as the fix. The splash has three + sibling glows with identical structure and times, differing in one byte + (`a=212` vs `a=255` at `t=45`); that rule would make `anima_eff` alone + invisible while its two siblings stay lit. The capture agrees weakly — box-mean + ratios capture/render are gamearts 0.717, seta 0.723, **anima 0.772**, and a + glow we drew that the game does not would put anima *below* its siblings. [`structures/ui-resting-pose.md`](../re/structures/ui-resting-pose.md) 🟡 **The shift is still not adopted**, now for a different reason: it flips this element to the visibly wrong answer, so it and a decision about diff --git a/docs/re/METHOD.md b/docs/re/METHOD.md index 3224ee9..71f3ce2 100644 --- a/docs/re/METHOD.md +++ b/docs/re/METHOD.md @@ -396,3 +396,15 @@ agent's loop prompt, i.e. nowhere durable. See [`README.md`](README.md) for the the plateau the first path already caught. So the fallback is guaranteed to be guessing every time it is reached. When a heuristic's justification is the same condition that routes around it, it has no justification. +* **Siblings adjudicate a rule that a single element cannot.** Deciding what + `rest()` should return for an element with no held pose looked like it needed a + capture we do not have. It did not: the developer splash carries three glows + with identical structure and identical times, differing in one alpha byte, and + any rule that makes one of them behave completely differently from the other + two is producing an artefact. Parallel constructions in the same bundle are a + free control — look for them before concluding a question needs new measurement. +* **Backgrounding with `&` in a compound command loses the working directory.** + `cd X && cmd &` followed by more work in the same call left the shell reset, and + a heredoc two lines later wrote nowhere — while the `echo` after it still + printed "written". A success message after a failed redirect is not a success. + Use `(setsid nohup … &)` and keep the file-writing in its own call. diff --git a/docs/re/REFUTED.md b/docs/re/REFUTED.md index 07b979a..77aac26 100644 --- a/docs/re/REFUTED.md +++ b/docs/re/REFUTED.md @@ -402,3 +402,9 @@ neighbourhood, not just the line. poses; an endpoint is only held when the two poses are equal, which is a plateau, which the earlier path already returned for. Every element that reaches the fallback has a guessed rest pose. +* "`rest()` for a plateau-less element should be the **last keyframe**" → **mine, + and refuted.** The developer splash's three sibling glows are structurally + identical and differ by one byte (`a=212` vs `a=255` at `t=45`); that rule + makes `palogo_anima_eff` alone invisible while `gamearts_eff` and `seta_eff` + stay lit. Capture box-mean ratios (0.717 / 0.723 / **0.772**) go the other way + too. [`structures/ui-resting-pose.md`](structures/ui-resting-pose.md) diff --git a/docs/re/data/plateau-census.txt b/docs/re/data/plateau-census.txt new file mode 100644 index 0000000..23a9a5d --- /dev/null +++ b/docs/re/data/plateau-census.txt @@ -0,0 +1,25 @@ +CONTROL GP_TITLE build 7: fallback elements [8, 13, 14] (want [8, 13, 14]) + index 8 is 'ptlogo_eff3.t32' (want ptlogo_eff3.t32) + +elements with a keyframe group, disc-wide: 15493 + no plateau -> rest pose is GUESSED: 3807 (24.57 %) + +most common guessed elements: + 1002 pb_name_eff.t32 + 90 gbp301i.t32 + 84 pb_warp_eff.t32 + 78 pbb_destroyer.t32 + 70 gbp303i.t32 + 60 pb_w_line.t32 + 54 pbmwindow_eff.t32 + 54 pbfriendly.t32 + 32 pzeff00.prm + 32 pzeff02.t32 + 32 px_mission_area.rat + 30 pbr_fighter.t32 + +--- the 3807 plateau-less elements, by what each candidate rule returns + current rule (longest dwell) returns an INVISIBLE pose : 1711 ( 44.9 %) + current rule returns a ZERO-SCALE (degenerate) pose : 195 ( 5.1 %) + 'rest = last keyframe' returns an INVISIBLE pose : 1618 ( 42.5 %) + the two rules agree : 1911 ( 50.2 %) diff --git a/docs/re/structures/ui-resting-pose.md b/docs/re/structures/ui-resting-pose.md index b119aab..42f8c08 100644 --- a/docs/re/structures/ui-resting-pose.md +++ b/docs/re/structures/ui-resting-pose.md @@ -320,3 +320,67 @@ Any element whose keyframes contain **no two adjacent identical poses** has a built from it. That is a property a consumer can test for itself in one pass over the keyframes, and it is worth flagging in an export rather than silently inheriting our guess. + +## The size of the defect, and why it does not block the menu port + +[`tools/re-capture/plateau_census.py`](../../../tools/re-capture/plateau_census.py) +walks every `GP_*.pak` placement region directly (the CLI route decodes every +texture and is far too slow for a disc-wide pass). Its control reproduces +`GP_TITLE` build 7's three fallback elements and names `ptlogo_eff3.t32` among +them before it counts anything. Output: [`data/plateau-census.txt`](../data/plateau-census.txt). + +| | | +|---|---| +| elements with a keyframe group, disc-wide | **15 493** | +| no plateau → **rest pose is guessed** | **3 807 (24.57 %)** | +| of those, current rule returns an **invisible** pose | 1 711 (44.9 %) | +| of those, current rule returns a **zero-scale** pose | **195 (5.1 %)** | +| the two candidate rules **agree** | 1 911 (50.2 %) | + +A returned pose with `scale = 0 %` is not a pose at all, and 195 elements get +one. Disc-wide the choice of rule is not cosmetic: the candidates agree only half +the time. + +### ✅ But on the five screens the port needs, the exposure is one element + +| screen | plateau-less | rules differ | +|---|---|---| +| main menu (entry 5) | 5 / 16 | 0 | +| `EXTRAS` (entry 6) | 5 / 18 | 0 | +| title (entry 4) | 2 / 24 | 0 | +| developer splash (entry 11) | 2 / 7 | **1** | + +Thirteen of the fourteen affected elements get the same answer either way. The +one disagreement is `palogo_anima_eff.t32`. + +### 🔴 And "rest = last keyframe" is refuted on it + +That was the alternative I named last iteration. The developer splash carries +**three sibling glows**, identical in structure and in every time: + +``` +palogo_gamearts_eff 15:a=0 30:a=255 45:a=255 -:a=0 → plateau → visible +palogo_seta_eff 15:a=0 30:a=255 45:a=255 -:a=0 → plateau → visible +palogo_anima_eff 15:a=0 30:a=255 45:a=212 -:a=0 → no plateau +``` + +They differ in **one byte** — `212` where the others have `255`. Under "last +keyframe", `anima_eff` alone goes invisible while its two siblings stay lit. A +rule that makes one of three parallel elements behave differently because of a +single alpha count is producing an artefact, not a decode. + +The capture agrees weakly. Comparing box means in +[`live-splash-developer.png`](../captures/title-builds/live-splash-developer.png) +against our render (the screenshot is 1279×675, top-aligned, so only ratios are +comparable): gamearts **0.717**, seta **0.723**, anima **0.772**. If our render +were adding a glow the game does not draw, anima's ratio would sit *below* its +siblings'. It sits above. + +### 🟡 Where this leaves it + +The **defect** is established and measured: `rest()` guesses for 24.57 % of +elements disc-wide and returns a degenerate zero-scale pose for 195 of them. +The **fix** is not decided — "last keyframe" is refuted, and the current rule +survives on the only captured element that discriminates. ⚠️ For the menu port +specifically this is **not a blocker**: one element on one screen, and our +current answer for it is the defensible one. diff --git a/tools/re-capture/plateau_census.py b/tools/re-capture/plateau_census.py new file mode 100644 index 0000000..804f6ee --- /dev/null +++ b/tools/re-capture/plateau_census.py @@ -0,0 +1,138 @@ +"""Disc-wide: which elements reach rest()'s dwell fallback, i.e. have no plateau? + +Reimplements the placement-region walk from ui_layout.rs::parse_placements so the +census runs in seconds instead of via `screen info --geometry` (which decodes +every texture). CONTROL: it must reproduce the three known fallback elements of +GP_TITLE build 7 (indices 8, 13, 14) and find ptlogo_eff3 among them. +""" +import struct, zlib, glob, os, collections + +DECL_AT, DECL_ENTRY, KEYFRAME = 0x20, 60, 40 + +def entries(base): + stub = open(base + ".pak", "rb").read() + if stub[:4] != b"IPFB": return + n = struct.unpack_from(">I", stub, 4)[0] + segs = sorted(glob.glob(base + ".p[0-9][0-9]")) + if not segs: return + blob = b"".join(open(s, "rb").read() for s in segs) + for i in range(n): + h, off, sz = struct.unpack_from(">III", stub, 0x10 + 12 * i) + st = blob[off:off + sz] + if len(st) < 10: continue + try: yield i, h, (zlib.decompress(st[10:]) if st[:2] == b"Z1" else st) + except Exception: continue + +def elements(d): + """(index, name, [poses]) per element with a keyframe group.""" + if d[:4] != b"RATC": return [] + count = struct.unpack_from(">I", d, 0x14)[0] + if not (0 < count < 4096): return [] + names = [] + for i in range(count): + o = DECL_AT + i * DECL_ENTRY + if o + DECL_ENTRY > len(d): return [] + names.append(d[o:o+28].split(b"\0")[0].decode("ascii", "replace")) + out, pos = [], DECL_AT + count * DECL_ENTRY + for _ in range(count): + if pos + 8 > len(d): break + idx, frames = struct.unpack_from(">II", d, pos) + if idx >= count or frames == 0 or frames > 4096: break + first = pos + 12 + end = first + frames * KEYFRAME - 4 + poses = [] + for k in range(frames): + blk = first + k * KEYFRAME + if blk + 36 > len(d) or blk + 36 > end: break + poses.append(struct.unpack_from(">IiiIIi i", d, blk)[:1] + + struct.unpack_from(">II", d, blk + 16) + + struct.unpack_from(">ii", d, blk + 28)) + if poses: out.append((idx, names[idx], poses)) + pos = end + return out + +def has_plateau(poses): + return any(poses[i] == poses[i+1] for i in range(len(poses) - 1)) + +# ---- CONTROL ------------------------------------------------------------- +ctl = {i: (n, p) for i, n, p in elements( + dict((i, d) for i, h, d in entries("/work/sylph_extract/dat/GP_TITLE"))[7])} +fb = sorted(i for i, (n, p) in ctl.items() if not has_plateau(p)) +print(f"CONTROL GP_TITLE build 7: fallback elements {fb} (want [8, 13, 14])") +print(f" index 8 is {ctl[8][0]!r} (want ptlogo_eff3.t32)") +assert fb == [8, 13, 14] and ctl[8][0].startswith("ptlogo_eff3"), "control failed" + +# ---- census -------------------------------------------------------------- +tot = fb_n = 0 +by_name = collections.Counter() +worst = [] +for pak in sorted(glob.glob("/work/sylph_extract/dat/GP_*.pak")): + base = pak[:-4] + for i, h, d in entries(base): + for idx, name, poses in elements(d): + tot += 1 + if not has_plateau(poses): + fb_n += 1 + by_name[name] += 1 + worst.append((os.path.basename(base), i, idx, name, len(poses))) +print(f"\nelements with a keyframe group, disc-wide: {tot}") +print(f" no plateau -> rest pose is GUESSED: {fb_n} ({100*fb_n/max(tot,1):.2f} %)") +print(f"\nmost common guessed elements:") +for n, c in by_name.most_common(12): print(f" {c:5d} {n}") + +# ---- candidate rules for a plateau-less element -------------------------- +# pose tuple = (fade, scale_x, scale_y, x, y); times read separately. +def dwell_pick(times, poses): + best = (0, -1) + for k in range(len(times) - 1): + if times[k] is None or times[k + 1] is None: continue + d = times[k + 1] - times[k] + if d >= best[1]: best = (k, d) + return best[0] + +def elements_t(d): + if d[:4] != b"RATC": return [] + count = struct.unpack_from(">I", d, 0x14)[0] + if not (0 < count < 4096): return [] + names = [] + for i in range(count): + o = DECL_AT + i * DECL_ENTRY + if o + DECL_ENTRY > len(d): return [] + names.append(d[o:o+28].split(b"\0")[0].decode("ascii", "replace")) + out, pos = [], DECL_AT + count * DECL_ENTRY + for _ in range(count): + if pos + 8 > len(d): break + idx, frames = struct.unpack_from(">II", d, pos) + if idx >= count or frames == 0 or frames > 4096: break + first = pos + 12; end = first + frames * KEYFRAME - 4 + poses, times = [], [] + for k in range(frames): + blk = first + k * KEYFRAME + if blk + 36 > len(d) or blk + 36 > end: break + poses.append(struct.unpack_from(">I", d, blk) + + struct.unpack_from(">II", d, blk + 16) + + struct.unpack_from(">ii", d, blk + 28)) + times.append(struct.unpack_from(">i", d, blk + 36)[0] if blk + 40 <= end else None) + if poses: out.append((idx, names[idx], times, poses)) + pos = end + return out + +alpha = lambda p: (p[0] >> 24) & 0xff +stats = collections.Counter() +for pak in sorted(glob.glob("/work/sylph_extract/dat/GP_*.pak")): + for i, h, d in entries(pak[:-4]): + for idx, name, times, poses in elements_t(d): + if has_plateau(poses): continue + cur = poses[dwell_pick(times, poses)] + last = poses[-1] + stats["n"] += 1 + stats["cur_invisible"] += alpha(cur) == 0 + stats["last_invisible"] += alpha(last) == 0 + stats["cur_zero_scale"] += (cur[1] == 0 or cur[2] == 0) + stats["agree"] += cur == last +n = stats["n"] +print(f"\n--- the {n} plateau-less elements, by what each candidate rule returns") +print(f" current rule (longest dwell) returns an INVISIBLE pose : {stats['cur_invisible']:5d} ({100*stats['cur_invisible']/n:5.1f} %)") +print(f" current rule returns a ZERO-SCALE (degenerate) pose : {stats['cur_zero_scale']:5d} ({100*stats['cur_zero_scale']/n:5.1f} %)") +print(f" 'rest = last keyframe' returns an INVISIBLE pose : {stats['last_invisible']:5d} ({100*stats['last_invisible']/n:5.1f} %)") +print(f" the two rules agree : {stats['agree']:5d} ({100*stats['agree']/n:5.1f} %)")