re(ui): put a number on the render-vs-capture tone difference

Closes an observation I left dangling last iteration ("the capture is ~4x
darker than the render") and puts a figure on the  INDEX's texture row
already carried: exact gamma/sRGB fidelity untested because a hue
comparison cannot see it.

Geometry first: cross-correlating the main-menu capture against our
render over +/-6 px puts the best alignment at exactly dy=0 dx=0,
correlation 0.9466. Only the tone differs.

Two methods failed before one worked, and both failures are recorded.
Three dark patches gave "4x darker" -- the whole-frame best linear scale
is 0.914, so three patches from one region are not a transfer curve. A
pixel-wise fit over 854,685 pixels then produced a NON-MONOTONIC transfer
(render 96-127 mapping brighter than render 128-159) with mean abs error
10-14 for every candidate model. That is edge misalignment, not a tone
curve: at correlation 0.947 a bright pixel routinely lands on a dark one.

Flat patches fix it -- 16x16 blocks with std < 8 in BOTH images, a
threshold chosen from the counts (0/83/404/1055/1788 at std<3/5/8/12/20):

  main menu  404 patches  gamma 1.491  err 0.28   (best linear 0.276, 0.34)
  EXTRAS     382 patches  gamma 1.493  err 0.22   (best linear 0.273, 0.28)
  title      506 patches  gamma 1.338  err 1.08   (best linear 0.842, 9.02)

Reach, stated because it is narrow: those patches span only render values
~0-60, where gamma and a plain scale are nearly indistinguishable -- the
two menus decide nothing (0.28 vs 0.34, 0.22 vs 0.28) and only the title
separates them. Nothing constrains midtones or highlights.

The held-out control FAILED TO DISCRIMINATE and is reported as such: the
splash's 2918 flat patches are pure black (render 0-4), so every model
scores ~0.00. That is a test with no power, not corroboration.

Confound left open: this compares our composite to what canary DISPLAYS,
and canary applies kernel_display_gamma_type = 2 (BT.709). The exponent
may be its output stage. The discriminating run -- set it to 0, recapture,
refit -- needs one emulator session reaching the main menu and was not
done.

Classified measured, not decoded; HANDOFF says plainly that a port
applying it is authoring.
This commit is contained in:
Sylpheed RE agent
2026-08-29 01:05:25 +00:00
parent 1e82096519
commit 969947e0e6
4 changed files with 113 additions and 1 deletions

View File

@@ -19,7 +19,7 @@ Promote to a prose `structures/…md` file when a format needs behavioural notes
| name-hash (TOC keys) | ✅ | `sylpheed-formats/src/hash.rs` | Barrett-reduction hash; recovers original paths |
| IDXD object/table | ✅ | `sylpheed-formats/src/idxd.rs` + `tests/idxd_records_disc.rs` ([container](structures/idxd-container.md)) | **The binary record/index region in front of the string pool is DECODED** (2026-08-25), closing the parser's long-standing "not yet decoded" note. Uniform 16-byte records `{name_hash, name_off, field_begin, field_end}` sorted by hash and binary-searched, then a field count, 12-byte fields `{key, name_off, value_off}` sorted by key, a pool size, and the string pool; the trailing `pool_size == file_len - pool_base` identity makes the layout self-checking. Verified over the **whole disc** with **zero** failures: 7 750/7 750 objects, 190 782/190 782 records reproducing their stored `tag_hash`, 1 271 462/1 271 462 named fields reproducing their key — and `IXUD` is the same container with `ixud_hash`, UTF-16BE and all offsets in **chars** (1 104/1 104 objects, 628 165/628 165 fields). **Field names are stored on disc** — a field's middle word points at its own name — so nothing needs preimage search except the **504** field entries disc-wide that are hash-keyed with no name — which are only **42 distinct keys**, each in 12 places (the page for these was never written — the finding is in this row), and are provably unrecoverable from the hash alone; the other 1 485 073 nameless fields are *positional*, keyed by a literal integer (line slots, movie ids). ⚠️ **Two long-held beliefs WITHDRAWN**: the word at `0x08` is **not a schema hash**, it is record 0's `name_hash` (7 750/7 750) — the header has no type field at all, so an object's kind is known only from the caller that loads it; and the field's middle word is **not** an always-`0xFFFFFFFF` flags word. The first was caught by a test asserting that every movie id names a real record: `1005 -> STAGE10_PHASE01` failed because `tag_hash("STAGE10_PHASE01")` **is** `0x067025B9`, that table's supposed schema id. 🟡 the legacy value-before-key string-pool reader is now known to be an *approximation* of the real table, and every number derived from it is re-checkable but not yet re-checked |
| IDXD nameless field keys | ✅/❌ | [idxd-unnamed-keys](structures/idxd-unnamed-keys.md) + [`tools/re-capture/idxd_unnamed_keys.py`](../../tools/re-capture/idxd_unnamed_keys.py) | Census of every field entry whose `name_off` is `0xFFFFFFFF`, disc-wide: **7 750 objects, 2 757 039 field entries, 0 parse failures**, `tag_hash` reproducing **1 271 462/1 271 462** named keys. **7 094 distinct keys are never named — and 7 052 of them are not hashes at all**, but author-assigned element ids (equal to the field's own index in 1 404 924 of 1 485 577 cases; `tag_hash("BGM_001")` is `0xC662435B` while the key valued `BGM_001.slb` is `0x000003E9`). ⚠️ **The "504 hash-keyed nameless fields" figure is 504 ENTRIES, not 504 names** — 42 distinct keys × 6 language copies × 2 records. All 42 are **ISL script-symbol hashes** in `<lang>\script\ID.tbl` (GP_READY_ROOM.pak), the link map built by `PrepareScript`'s "isl script prescanning"; 41 of 42 appear as little-endian call targets inside the `.isb` bytecode, forming a coherent launcher/helper call graph. The hash's own algebra pins the **trailing digits of 30 of the 42 names** (deltas of exactly `+0x01000001` across `stage01..09`, `stage10..16`, `challenge01..06`; `+0x01010000` across `tutorial0101..0601`). ❌ **No name was cracked, and the negative is quantified**: seven attacks up to a 3.5×10⁸ composition space found nothing above the noise floor; exhaustive preimage search recovers `"Stage01"` from its own hash but returns nothing for the real targets at ≤6 characters, and at 7 characters one target already has **1 176** preimages — a 24-bit modulus cannot name an 8+ character identifier uniquely |
| XPR2 texture + cubemap | 🟡/✅ | `sylpheed-formats/src/texture.rs` + [colour check](xpr2-colour-check.md) | de-tile + A8R8G8B8 and DXT1. **Channel order ✅ confirmed against the running game**: the Delta Saber's decoded atlas is orange-dominant (median saturated hue 23.3°, *zero* cool pixels) and the game renders the same hull at 9.3° — a red↔blue swap would sit at ≈200°. Exact fidelity (gamma/sRGB curve, premultiplied alpha, per-channel scale) is 🟡 untested, since a hue comparison cannot see it; cubemap face ordering ❔ |
| XPR2 texture + cubemap | 🟡/✅ | `sylpheed-formats/src/texture.rs` + [colour check](xpr2-colour-check.md) | de-tile + A8R8G8B8 and DXT1. **Channel order ✅ confirmed against the running game**: the Delta Saber's decoded atlas is orange-dominant (median saturated hue 23.3°, *zero* cool pixels) and the game renders the same hull at 9.3° — a red↔blue swap would sit at ≈200°. Exact fidelity (gamma/sRGB curve, premultiplied alpha, per-channel scale) is 🟡 untested, since a hue comparison cannot see it; cubemap face ordering ❔. **2026-08-29, for the UI path only:** our composite is brighter than the emulator's frame by a gamma of **≈1.341.49** across three screens ([tone curve](structures/ui-render-tone-curve.md)) — 🟡 measured, not decoded, constrained only over render values ~060, and possibly canary's own `kernel_display_gamma_type = 2` (BT.709) output stage rather than the game's |
| T8aD 2D texture | ✅ | `sylpheed-formats/src/t8ad.rs` | **100 % of the disc decodes** (19 216/19 216, measured). The "~15 % deferred variants" were a wrong model, not a variant: a surface is a list of **arbitrary sub-rectangles**, each with a 16-byte header of `dst X, dst Y, width, height`, not a 256×256 grid — `0x1c` is the **rectangle count**. Uncovered area stays transparent. **Colours ✅ CONFIRMED** ([k8888](structures/texture-color-k8888.md)) |
| RATC bundle | ✅ | `sylpheed-formats/src/ratc.rs` | child listing confirmed. **"One level deep" is not a limitation — there is nothing deeper**: 2 859 bundles hold 18 002 children at depth 1 and **0 at depth 2**, with no parse failures. Nested RATC blobs are **leaf records that reference siblings by name** (`opt `, the sprite name): 3 311 leaves, all embedding sibling names, **10 144 of 10 148 references resolve**. The 4 that do not are one dangling asset — `pmbase.rat``pmbase.t32` in `GP_STAGE_CLEAR.pak`'s four language builds, and `pmbase.t32` is **on the disc nowhere** |
| LSTA sprite list | ✅ | `sylpheed-formats/src/lsta.rs` | A display list of inline elements: **T8aD sprites and `PRMD` primitives**. The `count` at `0x04` is **exact and counts both**`count == T8aD + PRMD` for **64/64** lists on the disc, which retires the old "a few entries disagree" note (it compared sprites against a total including primitives). **All 1 281 sprite frames decode** after the T8aD rectangle-list fix |