fabi/Sylpheed
1
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity

formats: a keyframe's time comes before its pose, and none of them was missing

Browse Source 
The placement region is `frames` records of `{u32 time; 36-byte pose}` after an
8-byte header, so the time word PRECEDES the pose it belongs to. Our parser's
40-byte window opened at the pose, four bytes into the record, and then read the
word at its `+36` as that pose's time -- which is the NEXT pose's. Every pose
field was right; only the time association slipped by one.

Two things the corpus has carried for weeks are that off-by-one and nothing
else: "a group's data stops 4 bytes short of its final block's time slot", and
"the last keyframe carries no time". The group is not short (8 + frames*40 is
exact) and no time is missing -- the first pose's time is the lead-in word at
`header + 8` that `parse_placements` skipped without asking what it was.

Disc-wide, 33 archives, 13 991 groups, each test with a control:

  A  lead-in prepended to the shifted times is non-decreasing  13991/13991
  B  a non-zero lead-in is strictly below the next time         5058/5058
     control (another group's lead-in, same bundle)            70.9%
  C  multi-segment alpha ramp at a constant rate, corrected    857/1540
     the same, under the old reading                             0/1042

C is the one that cannot be argued with: interpolation between keyframes is
linear, and under the old reading not one multi-keyframe ramp on the disc comes
out at a constant rate.

Adoption is free on every static composite, which is what the corpus previously
declined it over. `SYLPHEED_KF_TIME_SHIFT=1` moved GP_TITLE build 7 by 13.1% of
its pixels because it left pose 0 untimed; with the lead-in restored, all 12
GP_TITLE builds render byte-identically, and across 217 builds in six archives
only two elements pick a different rest pose -- both times between two poses
that are equally invisible.

`SYLPHEED_KF_TIME_SHIFT` is gone; `SYLPHEED_KF_TIME_LEGACY=1` restores the old
reading for A/B work.

ui_header_time_disc needed one line: 546 bundles whose every group is a single
static pose now report max_time = 0 where they previously reported no time at
all. Excluding them, the result it guards strengthened -- the bound holds over
2 859 bundles instead of 2 313, still with zero violations.

Not established: the executable's own parser. Reach is written down.

docs/re/ui-keyframe-record-layout.md

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nsxw1A9JseUw99Yw1ZRQzY
This commit is contained in:
sylph-decoder
2026-08-29 14:01:39 +00:00
parent 7e57601c15
commit a9755170ee
6 changed files with 712 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files

53
crates/sylpheed-formats/src/ui_layout.rs
View File

@@ -111,12 +111,22 @@ pub struct Keyframe {
/// starts are negative. /// starts are negative.
pub x: i32, pub x: i32,
pub y: i32, pub y: i32,
/// Keyframe time, or `None` for the group's **last** frame. /// The time at which this pose is reached.
/// ///
/// A group's data stops 4 bytes short of its final block's time slot — that /// ✅ Always `Some` since 2026-08-29. A placement group is an 8-byte header
/// word is already the next group's element index. Reading it anyway is /// followed by `frames` records of `{ u32 time; 36-byte pose }`, so the time
/// where a stray `time = 1869640736` comes from, and it silently corrupts /// word **precedes** the pose it belongs to. Our block window starts at the
/// the max-dwell pick in [`Element::rest`]. /// pose, so pose `k`'s time is the previous stride's `+36` word, and pose
/// 0's is the group's lead-in word at `header + 8`.
///
/// ⚠️ The old reading took `+36` as *this* pose's time. That left the final
/// pose — the end of every fade-out — untimed, and it is where the "a
/// group's data stops 4 bytes short of its final block's time slot" note and
/// the stray `time = 1869640736` both came from. There is no short group and
/// no missing word; the association was off by one.
/// See `docs/re/ui-keyframe-record-layout.md`.
///
/// `Option` is retained for the `SYLPHEED_KF_TIME_LEGACY=1` escape hatch.
pub time: Option<u32>, pub time: Option<u32>,
} }
@@ -493,8 +503,10 @@ fn mark_focused_states(elements: &mut [Element]) {
/// Read the placement region that follows the declaration table, filling in each /// Read the placement region that follows the declaration table, filling in each
/// element's keyframe group. /// element's keyframe group.
fn parse_placements(bundle: &[u8], elements: &mut [Element]) -> Vec<usize> { fn parse_placements(bundle: &[u8], elements: &mut [Element]) -> Vec<usize> {
// Experiment gate, default off; see the `time` field below. // Escape hatch for the pre-2026-08-29 reading, which mis-associated every
let shift_times = std::env::var("SYLPHEED_KF_TIME_SHIFT").as_deref() == Ok("1"); // keyframe time by one slot and could not time a group's final pose at all.
// See `docs/re/ui-keyframe-record-layout.md`.
let legacy_times = std::env::var("SYLPHEED_KF_TIME_LEGACY").as_deref() == Ok("1");
let count = elements.len(); let count = elements.len();
let mut order = Vec::with_capacity(count); let mut order = Vec::with_capacity(count);
let mut pos = DECL_TABLE_AT + count * DECL_ENTRY; let mut pos = DECL_TABLE_AT + count * DECL_ENTRY;
@@ -507,7 +519,13 @@ fn parse_placements(bundle: &[u8], elements: &mut [Element]) -> Vec<usize> {
if idx >= count || frames == 0 || frames > 4096 { if idx >= count || frames == 0 || frames > 4096 {
break; break;
} }
// Group header is (index, count) then one lead-in word; blocks follow. // The region is `frames` records of 40 bytes, each `{ u32 time; 36-byte
// pose }`, after an 8-byte header — so the word at `pos + 8` is the
// FIRST pose's time, and each 40-byte stride's `+36` word is the time of
// the pose that follows it. Our block window is offset 4 bytes into the
// record (it starts at the pose), which is why the pose field offsets
// below are right while the times were off by one.
let first_time = be32(bundle, pos + 8);
let first = pos + 12; let first = pos + 12;
// The region is packed so that the next group's header sits 4 bytes // The region is packed so that the next group's header sits 4 bytes
// inside the last block — i.e. the group owns `frames * 40 - 4` bytes of // inside the last block — i.e. the group owns `frames * 40 - 4` bytes of
@@ -529,16 +547,17 @@ fn parse_placements(bundle: &[u8], elements: &mut [Element]) -> Vec<usize> {
tint: be32(bundle, blk + 24), tint: be32(bundle, blk + 24),
x: be32(bundle, blk + 28) as i32, x: be32(bundle, blk + 28) as i32,
y: be32(bundle, blk + 32) as i32, y: be32(bundle, blk + 32) as i32,
// Only a block wholly inside the group carries a time. // Pose `k`'s time is the word that PRECEDES it: the group's
// // lead-in word for `k == 0`, and the previous stride's `+36`
// ⚠️ Which block a time word BELONGS TO is under test — see // otherwise. Every pose is timed; nothing is missing and nothing
// `docs/re/ui-keyframe-time-unit.md`. Set `SYLPHEED_KF_TIME_SHIFT=1` // is special-cased. Checked disc-wide — see
// to read `W[k-1]` as block `k`'s time ("the word is the time the // `docs/re/ui-keyframe-record-layout.md`.
// NEXT pose is reached") instead of `W[k]`. Default is unchanged. time: if legacy_times {
time: if shift_times {
(k >= 1).then(|| be32(bundle, blk - KEYFRAME + 36))
} else {
(blk + 40 <= group_end).then(|| be32(bundle, blk + 36)) (blk + 40 <= group_end).then(|| be32(bundle, blk + 36))
} else if k == 0 {
Some(first_time)
} else {
Some(be32(bundle, blk - KEYFRAME + 36))
}, },
}); });
} }

24
crates/sylpheed-formats/tests/ui_header_time_disc.rs
View File

@@ -120,8 +120,17 @@ fn header_0x08_against_the_keyframe_times() {
worst.push(format!("{pak}: max keyframe {max_time} > header {dur}")); worst.push(format!("{pak}: max keyframe {max_time} > header {dur}"));
} }
} }
let r = ((max_time as f64 / dur as f64) * 10.0).round() as u32; // ⚠️ Bundles whose every group is a single static pose contribute
*ratio.entry(r.min(30)).or_default() += 1; // `max_time == 0` and say nothing about whether `+0x08` is a length.
// Before 2026-08-29 they were invisible here, because the old keyframe
// time reading left a one-frame group's only pose untimed; the corrected
// record layout (`docs/re/ui-keyframe-record-layout.md`) gives it the
// group's lead-in time, which is 0. They are excluded rather than
// allowed to swamp the histogram's zero bucket — 546 of them do.
if max_time > 0 {
let r = ((max_time as f64 / dur as f64) * 10.0).round() as u32;
*ratio.entry(r.min(30)).or_default() += 1;
}
}); });
eprintln!("bundles with keyframe times and a non-zero +0x08: {animated}"); eprintln!("bundles with keyframe times and a non-zero +0x08: {animated}");
@@ -142,11 +151,18 @@ fn header_0x08_against_the_keyframe_times() {
assert!(animated > 0, "no animated bundles — the sweep is broken"); assert!(animated > 0, "no animated bundles — the sweep is broken");
// MEASURED 2026-08-24. +0x08 bounds the keyframe times in EVERY one of the // MEASURED 2026-08-24, re-measured 2026-08-29 under the corrected keyframe
// 2313 bundles that have both, and 444 of them reach it exactly. The // record layout. +0x08 bounds the keyframe times in EVERY one of the 2 859
// bundles that have both (2 313 before the correction, which could not time
// a group's final pose at all), and 444 of them reach it exactly. The
// spread-out ratio histogram is what rules out the boring explanation: a // spread-out ratio histogram is what rules out the boring explanation: a
// large unrelated constant would bound everything too, but then the ratios // large unrelated constant would bound everything too, but then the ratios
// would pile up near zero instead of peaking at 1.0. // would pile up near zero instead of peaking at 1.0.
//
// ✅ The correction STRENGTHENS this result rather than weakening it: 546
// more bundles now carry a readable last-pose time, and `over` is still 0 —
// i.e. the newly-visible times, which are the LATEST in every group, still
// do not run past the header's.
assert_eq!(over, 0, "a keyframe time runs past the header's +0x08"); assert_eq!(over, 0, "a keyframe time runs past the header's +0x08");
assert!(exact > 400, "the bound is never attained — it may be unrelated"); assert!(exact > 400, "the bound is never attained — it may be unrelated");
let near_one = ratio.get(&10).copied().unwrap_or(0); let near_one = ratio.get(&10).copied().unwrap_or(0);

204
crates/sylpheed-formats/tests/ui_keyframe_record_disc.rs Normal file
View File

@@ -0,0 +1,204 @@
//! A placement group's time word **precedes** the pose it belongs to.
//!
//! A group is an 8-byte header `{u32 element_index, u32 frame_count}` followed
//! by `frame_count` records of 40 bytes, each `{u32 time; 36-byte pose}`. The
//! parser's block window starts at the *pose*, four bytes into the record, so
//! the word at a block's `+36` is the time of the pose that FOLLOWS it, and the
//! first pose's time is the group's lead-in word at `header + 8`.
//!
//! The old reading took `+36` as the block's own time. That is off by one, and
//! it is where two long-standing oddities came from: the group looked four bytes
//! short, and the final pose — the end of every fade-out — carried no time.
//!
//! Full argument and disc-wide census: `docs/re/ui-keyframe-record-layout.md`.
//!
//! Two checks here, both disc-wide:
//!
//! 1. **Every pose is timed, and the times are non-decreasing.** Under the old
//! reading the last pose has no time at all, so this cannot even be asked.
//! 2. **A monotone alpha ramp of three or more segments runs at a constant
//! rate.** Interpolation between keyframes is linear
//! (`docs/re/ui-keyframe-time-unit.md`), so a correct time assignment makes
//! multi-keyframe ramps come out at a constant d(alpha)/d(time). The old
//! reading achieves this on **zero** ramps on the whole disc.
use std::path::{Path, PathBuf};
use sylpheed_formats::{pak::PakArchive, ratc, ui_layout};
fn disc_root() -> Option<PathBuf> {
if let Ok(p) = std::env::var("SYLPHEED_DISC") {
let p = PathBuf::from(p);
if p.join("dat").is_dir() {
return Some(p);
}
}
let default = Path::new(
"/home/fabi/RE - Project Sylpheed/Project Sylpheed - Arc of Deception (USA, Europe) (En,Ja)",
);
if default.join("dat").is_dir() {
return Some(default.to_path_buf());
}
None
}
fn for_each_build(root: &Path, mut f: impl FnMut(&str, &[u8])) {
let mut paks: Vec<PathBuf> = std::fs::read_dir(root.join("dat"))
.expect("dat/")
.flatten()
.map(|e| e.path())
.filter(|p| p.extension().and_then(|s| s.to_str()) == Some("pak"))
.collect();
paks.sort();
for p in &paks {
let name = p.file_name().unwrap().to_string_lossy().to_string();
let Ok(arc) = PakArchive::open(p) else { continue };
for e in arc.entries() {
let Ok(bytes) = arc.read(e) else { continue };
if ratc::is_ratc(&bytes) {
f(&name, &bytes);
}
}
}
}
/// Maximal runs of strictly monotone alpha with at least `min_seg` segments.
fn monotone_ramps(alphas: &[i32], min_seg: usize) -> Vec<(usize, usize)> {
let mut out = Vec::new();
let (mut i, n) = (0usize, alphas.len());
while i + 1 < n {
if alphas[i] == alphas[i + 1] {
i += 1;
continue;
}
let up = alphas[i + 1] > alphas[i];
let mut j = i + 1;
while j + 1 < n && ((alphas[j + 1] > alphas[j]) == up) && alphas[j + 1] != alphas[j] {
j += 1;
}
if j - i >= min_seg {
out.push((i, j));
}
i = j;
}
out
}
fn constant_rate(times: &[u32], alphas: &[i32], a: usize, b: usize) -> Option<bool> {
let mut rates = Vec::new();
for k in a..b {
let dt = times[k + 1].checked_sub(times[k])?;
if dt == 0 {
return None;
}
rates.push((alphas[k + 1] - alphas[k]).unsigned_abs() as f64 / dt as f64);
}
let mean = rates.iter().sum::<f64>() / rates.len() as f64;
if mean == 0.0 {
return None;
}
let worst = rates.iter().map(|r| (r - mean).abs()).fold(0.0, f64::max);
Some(worst / mean <= 0.06)
}
#[test]
fn every_pose_is_timed_and_the_times_are_ordered() {
let Some(root) = disc_root() else {
eprintln!("SKIP: extracted disc not found (set SYLPHEED_DISC to enable)");
return;
};
if std::env::var("SYLPHEED_KF_TIME_LEGACY").as_deref() == Ok("1") {
eprintln!("SKIP: the legacy time reading is selected");
return;
}
let (mut groups, mut untimed, mut out_of_order) = (0usize, 0usize, 0usize);
let mut examples: Vec<String> = Vec::new();
for_each_build(&root, |pak, bytes| {
let Some(build) = ui_layout::parse_build(bytes) else {
return;
};
if build.from_fallback {
return;
}
for el in &build.elements {
if el.keyframes.is_empty() {
continue;
}
groups += 1;
if el.keyframes.iter().any(|k| k.time.is_none()) {
untimed += 1;
if examples.len() < 8 {
examples.push(format!("{pak}: {} has an untimed pose", el.name));
}
continue;
}
let t: Vec<u32> = el.keyframes.iter().map(|k| k.time.unwrap()).collect();
if t.windows(2).any(|w| w[1] < w[0]) {
out_of_order += 1;
if examples.len() < 8 {
examples.push(format!("{pak}: {} times {t:?} descend", el.name));
}
}
}
});
eprintln!("placement groups: {groups}; untimed: {untimed}; out of order: {out_of_order}");
for e in &examples {
eprintln!(" {e}");
}
assert!(groups > 10_000, "expected the whole disc, saw {groups} groups");
assert_eq!(untimed, 0, "every pose must carry a time");
assert_eq!(out_of_order, 0, "keyframe times must not descend");
}
#[test]
fn multi_segment_alpha_ramps_run_at_a_constant_rate() {
let Some(root) = disc_root() else {
eprintln!("SKIP: extracted disc not found (set SYLPHEED_DISC to enable)");
return;
};
if std::env::var("SYLPHEED_KF_TIME_LEGACY").as_deref() == Ok("1") {
eprintln!("SKIP: the legacy time reading is selected");
return;
}
let (mut ramps, mut constant) = (0usize, 0usize);
for_each_build(&root, |_pak, bytes| {
let Some(build) = ui_layout::parse_build(bytes) else {
return;
};
if build.from_fallback {
return;
}
for el in &build.elements {
if el.keyframes.iter().any(|k| k.time.is_none()) {
continue;
}
let t: Vec<u32> = el.keyframes.iter().map(|k| k.time.unwrap()).collect();
let a: Vec<i32> = el
.keyframes
.iter()
.map(|k| ((k.fade >> 24) & 0xff) as i32)
.collect();
for (lo, hi) in monotone_ramps(&a, 3) {
if let Some(ok) = constant_rate(&t, &a, lo, hi) {
ramps += 1;
constant += usize::from(ok);
}
}
}
});
let share = 100.0 * constant as f64 / ramps as f64;
eprintln!("multi-segment alpha ramps: {constant}/{ramps} at a constant rate ({share:.1}%)");
assert!(ramps > 500, "expected the whole disc, saw {ramps} ramps");
// The old reading scores 0 of 1042. Half is a floor, not a target: the rest
// are genuinely shaped ramps, authored with keyframes that are not evenly
// spaced. Anything near zero means the time assignment has slipped again.
assert!(
share > 45.0,
"only {share:.1}% of ramps run at a constant rate — the time \
association has probably slipped (the old off-by-one scored 0%)"
);
}

14
docs/re/data/kf-record-census.txt Normal file
View File

@@ -0,0 +1,14 @@
paks scanned : 33
placement groups : 13991
A. lead-in prepended to the shifted times is non-decreasing
13991/13991 = 100.000%
B. non-zero lead-in is strictly less than the next time
5058/5058 = 100.000%
control (another group's lead-in, same bundle): 35837/50580 = 70.852%
gap to the next time, most common: [(10, 2076), (1, 2022), (30, 116), (40, 80), (12, 78), (90, 78), (149, 78), (20, 78)]
C. constant d(alpha)/d(time) across a multi-segment ramp
corrected (time precedes pose): 857/1540 = 55.649%
old (+36 is own time) : 0/1042 = 0.000%

224
docs/re/ui-keyframe-record-layout.md Normal file
View File

@@ -0,0 +1,224 @@
# A keyframe's time word comes **before** its pose — the placement record, decoded
**Status:** ✅ `CONFIRMED`, **decoded**. The field, plus a disc-wide check
(13 991 placement groups over 33 archives, three tests, each with a control) and
a regression test that runs against the disc
(`crates/sylpheed-formats/tests/ui_keyframe_record_disc.rs`).
This closes the one thing [MISSION](../port/MISSION.md) **Q1** still had open —
*"the interpolation law is settled; the group TIMELINE for multi-keyframe
elements is not"* — and it dissolves, rather than decides, the argument in
[`ui-keyframe-time-unit.md`](ui-keyframe-time-unit.md) about whether to adopt
`SYLPHEED_KF_TIME_SHIFT`. Both sides of that argument were reasoning about a
missing word that is not missing.
## The record
A build bundle's placement region is a run of groups, one per element. A group
is an 8-byte header followed by `frame_count` **records of 40 bytes**:
```text
u32 element_index
u32 frame_count
┐
u32 time │ record 0 ← the time comes FIRST
36 pose ┘
u32 time ┐ record 1
36 pose ┘
…
u32 time ┐ record n−1
36 pose ┘
```
Total group size: `8 + frame_count * 40`.
The 36-byte pose is what the parser already reads correctly — fade ARGB, the
three signed rotation words, scale X/Y, tint, X, Y — at offsets 0…35 of the
pose, i.e. 4…39 of the record.
## What was wrong, and why it looked right for so long
Our parser opened its 40-byte window **at the pose**, four bytes into record 0,
and then read the word at window `+36` as that pose's time. That word is
record `k+1`'s `time` — the time of the *next* pose. Every pose field lands
correctly (the window is aligned to a pose, and poses are what it reads); only
the time association slips by one.
Two long-standing oddities in the corpus are that off-by-one, and nothing else:
| the oddity as recorded | what it actually was |
|---|---|
| *"a group's data stops 4 bytes short of its final block's time slot — that word is already the next group's element index"* | the group is **not** short. `8 + frames*40` is exact. The parser was reading 4 bytes past the last pose because its window began 4 bytes early |
| *"the last keyframe carries no time"* — `Keyframe::time` was `Option<u32>`, `None` on every group's final pose | the final pose's time is the *previous* stride's `+36` word. **Every** pose is timed |
| the stray `time = 1869640736` (= `"ohnm"`, ASCII from the next record) that "silently corrupts the max-dwell pick in `Element::rest`" | the same over-read |
The first pose's time is the group's **lead-in word** at `header + 8` — the word
[`parse_placements`](../../crates/sylpheed-formats/src/ui_layout.rs) skipped as
*"one lead-in word"* without asking what it was.
## The disc-wide check
`tools/re-capture/kf_record_census.py`, output committed at
[`data/kf-record-census.txt`](data/kf-record-census.txt). Run it with
```bash
python3 tools/re-capture/kf_record_census.py "$SYLPHEED_DISC"/dat/*.pak
```
### A. The lead-in word takes its place in the sequence
Prepending the lead-in to the shifted time series must give a non-decreasing
sequence. **13 991 of 13 991 groups — 100.000 %.** (15 493 including
single-pose groups, which are trivially ordered; the regression test counts
those and also finds 0 out of order.)
### B. The 5 058 non-zero lead-ins are times, not padding
If the lead-in were padding, a flag, or a count, 5 058 of them would not all
happen to fall strictly below the group's next time.
| | result |
|---|---|
| non-zero lead-ins | 5 058 |
| strictly less than the next time | **5 058 — 100.000 %**, none equal |
| **control**: another group's lead-in from the same bundle | 35 837 / 50 580 = **70.9 %** |
The gap to the next time piles up at **10** (2 076 groups) and **1** (2 022) —
ramp lengths, not arbitrary numbers. And the values themselves read as times:
`GP_DIALOG` entry 9's `pzeff02.t32` runs `167 → 197 → 217 → 232`; entry 25's
`pznoise.rat` runs `40 → 80 → 230 → 260`.
### C. A multi-keyframe ramp only runs at a constant rate under this reading
Interpolation between two keyframes is linear — measured against the running
game, in [`ui-keyframe-time-unit.md`](ui-keyframe-time-unit.md). So where an
author chains three or more keyframes through a monotone alpha ramp, a correct
time assignment should often make `d(alpha)/d(time)` come out constant, and a
wrong one should scramble it.
| reading | multi-segment alpha ramps at a constant rate (±6 %) |
|---|---|
| **corrected** — time precedes pose | **857 / 1 540 = 55.6 %** |
| old — `+36` is the block's own time | **0 / 1 042 = 0.0 %** |
**Zero.** Not one ramp on the whole disc. The 44 % that are not constant under
the corrected reading are genuinely shaped ramps — authors do place keyframes
unevenly — so 56 % is a floor, not a fit.
A worked example, `pgloading_loop4.rat` on `GP_TITLE` build 11:
| | times | alphas | rate per unit |
|---|---|---|---|
| corrected | 0, 4, 6, 7, 8, 32, 38 | 0, 128, 192, 224, 255, 255, 0 | **32, 32, 32, 31** — then hold, then out |
| old | 4, 6, 7, 8, 32, 38, *(none)* | 0, 128, 192, 224, 255, 255, 0 | 64, 64, 32, 1.3 — then hold, then an **untimed** fade-out |
## What it costs to adopt: nothing, on every static composite
This is the change the corpus previously declined to make, because
`SYLPHEED_KF_TIME_SHIFT=1` moved `GP_TITLE` build 7 by 13.1 % of its pixels and
made the EN/JP twin brightness disagree (70.94 vs 76.32 against build 4's
71.41). **That was the missing first time word, not the shift.**
With the lead-in restored as pose 0's time:
| check | result |
|---|---|
| `GP_TITLE`, all 12 builds rendered under both readings | **12 / 12 byte-identical PNGs**, build 7 included |
| 217 builds over 6 UI archives, `rest()` pose per element | **2 builds differ**: `GP_TITLE` 7 and `GP_DIALOG` 31 |
| what those 2 differences are | `ptlogo_eff3.t32`: `(98,42)` vs `(108,72)` — **both α = 0**, so neither paints. `pzstg14_2.t32`: one pixel of Y |
| renders of those 2 builds | **identical** |
So the build-7 luminance objection is withdrawn: it was `rest()`'s dwell
fallback picking the 200 %-scale bloom because pose 0 had no time to be compared
against. Given a time, the dwell rule picks an invisible pose — the same
*visible* result the old reading produced, by a rule that is now sound.
⚠️ **`Element::rest()` is unchanged and is still a heuristic.** The times feed
it; they do not fix it. `structures/ui-resting-pose.md` stands as written.
## Against the oracle
The committed `log_ui_draws` capture of the developer splash
([`captures/ui-timing/splash-build-quads.csv`](captures/ui-timing/splash-build-quads.csv))
is the check that this is the game's reading and not merely a tidier one.
`palogo_gamearts_eff.t32` — lead-in 0, `W = [15, 30, 45, –]`, alphas
`[0, 255, 255, 0]`:
| phase | corrected | old | captured |
|---|---|---|---|
| fade in | t 0→15 (7.5 f) | t 15→30 (7.5 f) | frames 94–101, **7 f** |
| hold | t 15→30 (7.5 f) | t 30→45 (7.5 f) | frames 101–107, **7 f** |
| fade out | t 30→45 (7.5 f) | **untimed** | frames 108–115, **8 f** |
The glow's *durations* do not discriminate — that was already recorded — but its
**end does**: the old reading cannot say when the fade-out finishes, and the
capture plainly shows it finishing.
`palogo_gamearts.t32` — lead-in 0, `W = [15, 30, 190, 194, 206, 210, –]`, alphas
`[0, 0, 255, 255, 232, 32, 0]`:
| | corrected | old | captured |
|---|---|---|---|
| fade in | t 15→30, **7.5 f**, in the same window as its own glow | t 30→190, **80 f** | already at 255 when the quad first appears (frame 116) |
| hold at 255 | t 30→190, **80 f** | t 190→194, **2 f** | frames ≤116–198, **≥ 83 f** |
| fade out | t 190→210, **10 f** | t 194→? , untimed end | frames 199–211, **13 f** |
A logo whose bloom layer fades in over 7.5 frames while the logo itself takes 80
is not a thing anyone authored. This replicates the 26× result already in
[`ui-keyframe-time-unit.md`](ui-keyframe-time-unit.md) and adds the reason.
⚠️ **Reach.** The capture's absolute frame numbers sit about 18 frames later than
the glow-derived calibration `t = 2f − 171` predicts for the *logo* — the
fade-out starts at frame 199 where the calibration says 180.5. Durations match;
the two elements' groups do not appear to start on the same frame. That offset is
**not explained here** and is not needed for this result, which is about which
word is which. It is the same lateness `ui-keyframe-time-unit.md` records as
"17 frames late" and leaves open.
## What changed in the code
[`crates/sylpheed-formats/src/ui_layout.rs`](../../crates/sylpheed-formats/src/ui_layout.rs):
* `parse_placements` reads `header + 8` as pose 0's time and the previous
stride's `+36` as pose `k`'s. Every pose gets a time.
* `SYLPHEED_KF_TIME_SHIFT` is gone. `SYLPHEED_KF_TIME_LEGACY=1` restores the old
reading for A/B work.
* `Keyframe::time` stays `Option<u32>` only so the legacy gate still type-checks.
Under the default it is always `Some`.
New test, disc-gated: `tests/ui_keyframe_record_disc.rs` — every pose timed and
ordered (15 493 groups), and ≥ 45 % of multi-segment alpha ramps at a constant
rate (the old reading scores 0 %).
⚠️ `tests/ui_header_time_disc.rs` needed one line: bundles whose every group is a
single static pose now report `max_time = 0` where before they reported no time
at all, and 546 of them were swamping the ratio histogram's zero bucket. The
result it guards **strengthened** — the bound `max_time ≤ header +0x08` now holds
over **2 859** bundles instead of 2 313, still with **0** violations, and the
newly readable times are the latest in every group.
## What is NOT established
❔ **The executable's own parser was not found.** Reach: queried the disassembly
database for functions carrying a `mulli` by 40 (the record stride) and by 60
(the declaration-entry stride) — 26 functions have the first, none have both, and
PowerPC compilers synthesise both constants as shift-adds, so the query is weak
rather than negative. Nothing here rests on a database row; every number above
comes from the disc bytes or from a committed capture. Finding the interpolator
would upgrade this from *decoded from the container's own arithmetic and a
disc-wide census* to *decoded from the code*, and would also settle the 18-frame
group-start offset above.
## For the port
The pose values you already have do not move. What moves is **when** each pose is
reached:
* pose `k`'s time is the word **before** it, not after it;
* pose 0 has a time — usually 0, but 5 058 groups on the disc start late;
* the **last** pose has a time, so an exit ramp now has an end. Anything you
authored to cover "the final keyframe has no time" can come out.
Static composites are unaffected: `screen render` produces byte-identical output
on all 12 `GP_TITLE` builds.

214
tools/re-capture/kf_record_census.py Executable file
View File

@@ -0,0 +1,214 @@
#!/usr/bin/env python3
"""Disc-wide test of the UI placement-group record layout.
A placement group in a screen bundle is an 8-byte header `{u32 element_index,
u32 frame_count}` followed by `frame_count` records of 40 bytes, each
`{u32 time; 36-byte pose}`. The time word therefore **precedes** the pose it
belongs to.
Our parser's block window starts at the pose, so under the old reading a
block's `+36` word was taken as *its own* time — off by one, which left the
group's final pose (the end of every fade-out) untimed and made the group look
4 bytes short.
This script tests the corrected reading against the whole disc, with controls:
A. monotonicity — prepending the lead-in word to the shifted time series must
give a non-decreasing sequence, for every group.
B. the non-zero lead-ins — a lead-in that is a *time* must be strictly less
than the next time. Control: swap in another group's lead-in from the
same bundle.
C. ramp linearity — within a monotone alpha ramp of >=3 segments, is
d(alpha)/d(time) constant? Keyframe interpolation is linear
(`docs/re/ui-keyframe-time-unit.md`), so a correct time assignment should
make multi-keyframe ramps come out at a constant rate far more often than
an incorrect one.
Usage: python3 tools/re-capture/kf_record_census.py "$SYLPHEED_DISC"/dat/*.pak
"""
import collections
import os
import random
import struct
import sys
import zlib
def be32(b, o):
return struct.unpack_from(">I", b, o)[0]
def load_pak(pak_path):
"""Entries of an IPFB archive, transparently un-Z1'd. See sylpheed-formats::pak."""
idx = open(pak_path, "rb").read()
if idx[:4] != b"IPFB":
return []
n = be32(idx, 4)
data = b""
base = os.path.splitext(pak_path)[0]
for i in range(100):
p = f"{base}.p{i:02d}"
if not os.path.exists(p):
break
data += open(p, "rb").read()
out = []
for k in range(n):
_h, off, sz = struct.unpack_from(">III", idx, 0x10 + 12 * k)
blob = data[off : off + sz]
if blob[:2] == b"Z1":
try:
blob = zlib.decompress(blob[0x0A:])
except Exception:
blob = b""
out.append(blob)
return out
def elem_name(b, o):
s = b[o : o + 28]
z = s.find(b"\0")
return (s[:z] if z >= 0 else s).decode("latin1")
def groups(bundle):
"""(element_index, name, frames, lead_in, W, alphas) per placement group.
`W[k]` is the word at the k-th 40-byte stride's `+36`; `W[-1]` is None
because it lies outside the group. Under the corrected reading the pose
times are `[lead_in] + W[:-1]`.
"""
n = be32(bundle, 0x14)
names = [elem_name(bundle, 0x20 + i * 60) for i in range(n)]
pos = 0x20 + n * 60
out = []
for _ in range(n):
if pos + 8 > len(bundle):
break
idx, frames = be32(bundle, pos), be32(bundle, pos + 4)
if idx >= n or frames == 0 or frames > 4096:
break
lead_in = be32(bundle, pos + 8)
first = pos + 12
end = first + frames * 40 - 4
if end > len(bundle):
break
W, alphas = [], []
for k in range(frames):
blk = first + k * 40
W.append(be32(bundle, blk + 36) if blk + 40 <= end else None)
alphas.append(be32(bundle, blk) >> 24)
out.append((idx, names[idx], frames, lead_in, W, alphas))
pos = end
return out
def is_build(raw):
if raw[:4] != b"RATC" or len(raw) < 0x20:
return False
n = be32(raw, 0x14)
return 0 < n <= 4096 and 0x20 + n * 60 <= len(raw)
def monotone_ramps(alphas, min_segments=3):
out, i, n = [], 0, len(alphas)
while i < n - 1:
if alphas[i] == alphas[i + 1]:
i += 1
continue
d = 1 if alphas[i + 1] > alphas[i] else -1
j = i + 1
while j < n - 1 and (alphas[j + 1] - alphas[j]) * d > 0:
j += 1
if j - i >= min_segments:
out.append((i, j))
i = j
return out
def constant_rate(times, alphas, a, b, tol=0.06):
rates = []
for k in range(a, b):
dt = times[k + 1] - times[k]
if dt <= 0:
return None
rates.append(abs(alphas[k + 1] - alphas[k]) / dt)
mean = sum(rates) / len(rates)
if mean == 0:
return None
return max(abs(r - mean) for r in rates) / mean <= tol
def main(paks):
rnd = random.Random(20260829)
per_bundle = collections.defaultdict(list)
total = mono = 0
ramp_new = [0, 0]
ramp_old = [0, 0]
for p in paks:
for ei, raw in enumerate(load_pak(p)):
if not is_build(raw):
continue
try:
gs = groups(raw)
except Exception:
continue
for _idx, nm, _frames, lead_in, W, alphas in gs:
rest = W[:-1]
if not rest or any(w is None for w in rest):
continue
total += 1
t_new = [lead_in] + rest
if all(t_new[i] <= t_new[i + 1] for i in range(len(t_new) - 1)):
mono += 1
per_bundle[(os.path.basename(p), ei)].append((lead_in, rest, nm))
for a, b in monotone_ramps(alphas):
r = constant_rate(t_new, alphas, a, b)
if r is not None:
ramp_new[1] += 1
ramp_new[0] += r
# old reading: `+36` is the block's own time, last pose untimed
a_old = alphas[:-1]
for a, b in monotone_ramps(a_old):
r = constant_rate(rest, a_old, a, b)
if r is not None:
ramp_old[1] += 1
ramp_old[0] += r
nz = nz_ok = ctrl = ctrl_ok = 0
gaps = collections.Counter()
for rows in per_bundle.values():
pool = [l for l, _, _ in rows]
for lead_in, rest, _nm in rows:
if lead_in == 0:
continue
nz += 1
nz_ok += lead_in < rest[0]
gaps[rest[0] - lead_in] += 1
for _ in range(10):
ctrl += 1
ctrl_ok += rnd.choice(pool) < rest[0]
pct = lambda a, b: f"{100 * a / b:.3f}%" if b else "n/a"
print(f"paks scanned : {len(paks)}")
print(f"placement groups : {total}")
print()
print("A. lead-in prepended to the shifted times is non-decreasing")
print(f" {mono}/{total} = {pct(mono, total)}")
print()
print("B. non-zero lead-in is strictly less than the next time")
print(f" {nz_ok}/{nz} = {pct(nz_ok, nz)}")
print(f" control (another group's lead-in, same bundle): "
f"{ctrl_ok}/{ctrl} = {pct(ctrl_ok, ctrl)}")
print(f" gap to the next time, most common: {gaps.most_common(8)}")
print()
print("C. constant d(alpha)/d(time) across a multi-segment ramp")
print(f" corrected (time precedes pose): {ramp_new[0]}/{ramp_new[1]} = "
f"{pct(ramp_new[0], ramp_new[1])}")
print(f" old (+36 is own time) : {ramp_old[0]}/{ramp_old[1]} = "
f"{pct(ramp_old[0], ramp_old[1])}")
if __name__ == "__main__":
if len(sys.argv) < 2:
sys.exit(__doc__)
main(sys.argv[1:])