fabi/Syplheed-Reborn
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

formats: sweep the bundle header - no screen flag, but three of its words decode

Browse Source 
The backlog asked what makes a bundle a screen rather than a fragment, and the
obvious suspect was the 32-byte header. Swept over all 2859 composable bundles
with a real declaration table.

The answer to the question is NO, and it is asserted rather than argued: no bit
of the flags word at +0x10 labels a screen. The best any bit manages is bit 13 -
403 bundles, 179 of them carrying a full-screen element, a 44% hit rate against a
12.8% base - and the commonest bit is set on 91% of everything. Enrichment, not a
marker.

The sweep found more than it was asked for, though. The header is not dead space:
+0x18 is 1280 on 2829 bundles and +0x1c is 720 on 2823 - the design resolution at
bundle level, the same pair the parser already reads out of a .rat record, and
asserted here. And +0x04 takes only three values, 0x3C0000 on 2843 and 0x1E0000
on 12, which are exactly 60.0 and 30.0 in 16.16 fixed point, with +0x08 taking
30/1200/120/60 - a frame rate and a duration in frames would fit a format whose
records are keyframe lists. That reading is marked amber: it comes from the
values alone and is not verified against an animation.

Also recorded, since the file will not say: element counts are min 1, median 2,
p75 5, p95 23, max 56, and only 365 bundles carry a full-screen element. The
population is mostly fragments and the separation is shape.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
This commit is contained in:
Sylpheed RE agent
2026-08-24 04:26:06 +00:00
parent 92dd1da65a
commit 27fcb69fdf
2 changed files with 236 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

190
crates/sylpheed-formats/tests/ui_screen_vs_fragment_disc.rs Normal file
View File

@@ -0,0 +1,190 @@
//! Is there a FIELD that says "this bundle is a screen", or only a shape?
//!
//! `is_composable` admits 1 786 more bundles than there are screens, most of
//! them 2–5-element button+glow fragments, and the backlog asks what separates
//! the two. The obvious place to look is the 32-byte bundle header: six words
//! besides the magic and the entry count, none of them read by anything.
//!
//! This sweeps every composable bundle on the disc and asks two questions the
//! same way the `opt `/focus sweeps did — by counting, not by looking at one
//! example:
//!
//! 1. do those six header words ever vary at all?
//! 2. what does the population actually look like — element counts, and how
//! many carry a full-screen (1280×720) element?
//!
//! A negative on (1) is a real answer: it would mean the file does not label a
//! screen, and a port has to decide by shape or by who references the bundle.
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use sylpheed_formats::{pak::PakArchive, ratc, ui_layout};
fn disc_root() -> Option<PathBuf> {
if let Ok(p) = std::env::var("SYLPHEED_DISC") {
let p = PathBuf::from(p);
if p.join("dat").is_dir() {
return Some(p);
}
}
let default = Path::new(
"/home/fabi/RE - Project Sylpheed/Project Sylpheed - Arc of Deception (USA, Europe) (En,Ja)",
);
if default.join("dat").is_dir() {
return Some(default.to_path_buf());
}
None
}
fn for_each_build(root: &Path, mut f: impl FnMut(&str, &[u8])) {
let mut paks: Vec<PathBuf> = std::fs::read_dir(root.join("dat"))
.expect("dat/")
.flatten()
.map(|e| e.path())
.filter(|p| p.extension().and_then(|s| s.to_str()) == Some("pak"))
.collect();
paks.sort();
for p in &paks {
let name = p.file_name().unwrap().to_string_lossy().to_string();
let Ok(arc) = PakArchive::open(p) else { continue };
for e in arc.entries() {
let Ok(bytes) = arc.read(e) else { continue };
if ratc::is_ratc(&bytes) {
f(&name, &bytes);
}
}
}
}
fn be32(b: &[u8], at: usize) -> u32 {
u32::from_be_bytes([b[at], b[at + 1], b[at + 2], b[at + 3]])
}
#[test]
fn the_bundle_header_does_not_label_a_screen() {
let Some(root) = disc_root() else {
eprintln!("SKIP: extracted disc not found (set SYLPHEED_DISC to enable)");
return;
};
// header word offset -> value -> how many bundles
let mut header: HashMap<usize, HashMap<u32, usize>> = HashMap::new();
let mut bundles = 0usize;
let mut counts: Vec<usize> = Vec::new();
let mut with_fullscreen = 0usize;
// Cross-tabulate the one header word that looks like flags against the two
// shape signals a "screen" would have: a full-screen element, and size.
let mut flag_bits: HashMap<u32, (usize, usize, usize)> = HashMap::new(); // bit -> (set, set&fullscreen, set&big)
for_each_build(&root, |_pak, bytes| {
if bytes.len() < 0x20 {
return;
}
let Some(build) = ui_layout::parse_build(bytes) else {
return;
};
if build.from_fallback {
return;
}
bundles += 1;
counts.push(build.elements.len());
if build
.elements
.iter()
.any(|e| (e.pivot_x as u64 * 2, e.pivot_y as u64 * 2) == (1280, 720))
{
with_fullscreen += 1;
}
let fullscreen = build
.elements
.iter()
.any(|e| (e.pivot_x as u64 * 2, e.pivot_y as u64 * 2) == (1280, 720));
let big = build.elements.len() >= 10;
let flags = be32(bytes, 0x10);
for bit in 0..32u32 {
if flags & (1 << bit) != 0 {
let e = flag_bits.entry(bit).or_default();
e.0 += 1;
if fullscreen {
e.1 += 1;
}
if big {
e.2 += 1;
}
}
}
for off in [0x04, 0x08, 0x0c, 0x10, 0x18, 0x1c] {
*header
.entry(off)
.or_default()
.entry(be32(bytes, off))
.or_default() += 1;
}
});
counts.sort_unstable();
let pct = |p: f64| counts[((counts.len() as f64 - 1.0) * p) as usize];
eprintln!("composable bundles with a real declaration table: {bundles}");
eprintln!(
"element counts: min {} p25 {} median {} p75 {} p95 {} max {}",
counts[0],
pct(0.25),
pct(0.50),
pct(0.75),
pct(0.95),
counts[counts.len() - 1]
);
eprintln!("bundles carrying a full-screen (1280x720) element: {with_fullscreen}");
let mut offs: Vec<_> = header.keys().copied().collect();
offs.sort();
for off in offs {
let vals = &header[&off];
let mut v: Vec<_> = vals.iter().collect();
v.sort_by_key(|(_, n)| std::cmp::Reverse(**n));
eprintln!(
" header +{off:#04x}: {} distinct value(s), commonest {:?}",
vals.len(),
&v[..v.len().min(4)]
);
}
let mut bits: Vec<_> = flag_bits.iter().collect();
bits.sort();
eprintln!(" +0x10 bits: bit -> (bundles with it set, of those full-screen, of those >=10 elements)");
for (bit, (n, fs, big)) in bits {
eprintln!(" bit {bit:2}: {n:5} full-screen {fs:5} big {big:5}");
}
eprintln!(
" for reference: {bundles} bundles, {with_fullscreen} full-screen, {} with >=10 elements",
counts.iter().filter(|&&c| c >= 10).count()
);
assert!(bundles > 0, "no composable bundles — the sweep is broken");
// MEASURED 2026-08-24. The question was "does a field say this bundle is a
// screen"; the answer is no, and the numbers that make it no are asserted.
//
// The best any bit of the flags word manages is bit 13, set on 403 bundles
// of which 179 carry a full-screen element — a 44 % hit rate against a
// 12.8 % base. Enrichment, not a marker. Bit 15 is set on 91 % of ALL
// bundles, which is the opposite failure.
let base = with_fullscreen as f64 / bundles as f64;
for (bit, (n, fs, _big)) in &flag_bits {
let rate = *fs as f64 / *n as f64;
assert!(
rate < 0.95 || *n < 50,
"bit {bit} looks like a screen marker after all: {fs}/{n} full-screen \
against a {base:.3} base — re-open the question"
);
}
// The header is NOT dead space, which is the other half of the result.
assert!(
header[&0x18].get(&1280).copied().unwrap_or(0) > bundles * 9 / 10,
"+0x18 is not the design width after all"
);
assert!(
header[&0x1c].get(&720).copied().unwrap_or(0) > bundles * 9 / 10,
"+0x1c is not the design height after all"
);
}

46
docs/re/structures/ui-rat-layout.md
View File

@@ -496,3 +496,49 @@ are byte coincidences in binary data — the scan is unaligned). What those say
untested. The claims above are about the links a screen's element table can
reach, which is what a compositor follows; they are not a statement about every
`opt ` in the file.
## The 32-byte bundle header — swept (2026-08-24)
The backlog asked what makes a bundle a **screen** rather than a fragment, and
the obvious suspect was the bundle header: six words besides the `RATC` magic and
the entry count at `0x14`, none of them read by anything. Swept over all **2 859**
composable bundles with a real declaration table
(`tests/ui_screen_vs_fragment_disc.rs`):
| offset | distinct values | reading |
|---|---|---|
| `+0x04` | **3** — `0x3C0000` ×2 843, `0x1E0000` ×12, `0x3C0001` ×4 | 🟡 **frame rate in 16.16**: `0x3C0000` is exactly `60.0`, `0x1E0000` exactly `30.0` |
| `+0x08` | 22 — 30, 1200, 120, 60, … | 🟡 a **duration in frames** (0.5 s, 20 s, 2 s, 1 s at 60) |
| `+0x0c` | 170 | ❔ |
| `+0x10` | 83 — `0x9400`, `0x9200`, `0x8212`, … | ❔ flags; bit 15 set on **91 %** of all bundles |
| `+0x18` | **2** — `1280` ×2 829 | ✅ **design width** |
| `+0x1c` | **3** — `720` ×2 823 | ✅ **design height** |
So the header is not dead space. `+0x18`/`+0x1c` are the design resolution at
bundle level — the same pair the parser already reads out of a `.rat` record —
and the two words before the count look like a frame rate and a duration, which
would fit a format whose records are keyframe lists. 🟡 The rate/duration reading
is from the **values alone** and is not verified against an animation; the
resolution one is asserted.
### 🔴 But no bit of it says "screen"
Cross-tabulating every bit of `+0x10` against the two shapes a screen would have:
```
bundles 2859 with a full-screen (1280x720) element 365 with >=10 elements 464
bit 15: 2605 set, 347 full-screen <- set on 91% of everything
bit 12: 2013 set, 299 full-screen
bit 13: 403 set, 179 full-screen <- the best enrichment: 44% vs a 12.8% base
```
The best any bit manages is **44 %** against a **12.8 %** base, and the most
common bit is set on nine bundles in ten. That is enrichment, not a label, and
the test asserts it so a future pass does not re-litigate it from one example.
### What a "screen" looks like, since the file will not say
Element counts over those 2 859 bundles: **min 1, p25 1, median 2, p75 5, p95 23,
max 56**, and only **365** carry a full-screen element. The population really is
mostly fragments, and the separation is **shape** — or which bundle references
which, which the PAK cannot answer directly because its entries are name-hashed.