re(ui): the 60 nameless RATC children are frames, not children -- .tan decoded
Closes the reach caveat the `opt ` name fix left behind: 60 of 18 002 RATC children carry no `opt ` block, and it was not established whether they lack one or sit past our 128-byte window. Neither. They are not children. `examples/ratc_optless_children.rs` re-runs `ratc::parse`'s own guards over the disc and reports which one fired: all 60 are "tag beyond the window", none is rejected by length, gap or charset, none is child #0, and all 60 live in six bundles of one archive. Within a bundle the distances back to the nearest tag are an exact arithmetic progression, step 60 600 -- ten different records finding the SAME tag, because there is only one. Reading a bundle directly: children 1..10 are equal-size T8aD blocks under a single `opt ` name, `pb_f15_eg_anm.tan`. `.tan` is a FRAME SEQUENCE. One block declares the resource; its payload is a run of T8aD frames. Disc-wide, over all 18 718 `opt ` names in all 33 paks: a RATC bundle names exactly six kinds of resource -- `.t32` 14 756, `.rat` 3 311, `.prm` 367, `.tbm` 224, `.sbo` 54, `.tan` 6. Six `.tan`, ten frames each = 60, the entire population with nothing left over. The negative is closed, not narrowed. Consequence recorded but deliberately not fixed: `ratc::parse` over-reports there, listing a `.tan`'s frames as anonymous children. Nothing in the menu milestone reads a `.tan` -- it occurs only in GP_READY_ROOM, which S1 ruled out -- so no screen the port draws changes. Also a METHOD entry for this container OOM-killing `slb_leading_segment_disc` under default test parallelism (SIGKILL, no assertion; 8/8 pass with --test-threads=1).
This commit is contained in:
75
tools/re-capture/ratc_opt_name_census.py
Normal file
75
tools/re-capture/ratc_opt_name_census.py
Normal file
@@ -0,0 +1,75 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Every `opt ` name in every RATC bundle on the disc, by extension -- and the
|
||||
`.tan` frame sequences among them.
|
||||
|
||||
Written to close the reach caveat in docs/re/structures/ratc-child-names.md:
|
||||
60 of 18 002 RATC children carry no `opt ` block of their own. They are not
|
||||
children. They are the ten frames of the disc's only `.tan` resource, and one
|
||||
`opt ` block names the whole run.
|
||||
|
||||
python3 tools/re-capture/ratc_opt_name_census.py
|
||||
|
||||
Reads $SYLPHEED_DISC/dat/*.pak directly (IPFB TOC + Z1/zlib entries), so it does
|
||||
not depend on the Rust parser it is checking. Takes a few minutes.
|
||||
"""
|
||||
|
||||
import struct, zlib, glob, os, collections, bisect
|
||||
MAG = (b"T8aD", b"RATC", b"ttcf", b"\x89PNG")
|
||||
ext = collections.Counter()
|
||||
tan_sites = []
|
||||
opt_total = 0
|
||||
DISC = os.environ.get("SYLPHEED_DISC", "/work/sylph_extract")
|
||||
for pakpath in sorted(glob.glob(f"{DISC}/dat/*.pak")):
|
||||
base = pakpath[:-4]
|
||||
pak = open(pakpath, "rb").read()
|
||||
if pak[:4] != b"IPFB": continue
|
||||
n = struct.unpack_from(">I", pak, 4)[0]
|
||||
toc = [struct.unpack_from(">III", pak, 0x10 + 12*i) for i in range(n)]
|
||||
segs = sorted(glob.glob(base + ".p[0-9][0-9]"))
|
||||
if not segs: continue
|
||||
data = b"".join(open(s, "rb").read() for s in segs)
|
||||
for ei, (h, off, cs) in enumerate(toc):
|
||||
raw = data[off:off+cs]
|
||||
try:
|
||||
b = zlib.decompress(raw[10:]) if raw[:2] == b"Z1" else raw
|
||||
except Exception:
|
||||
continue
|
||||
if b[:4] != b"RATC": continue
|
||||
names = []
|
||||
p = b.find(b"opt ")
|
||||
while p >= 0:
|
||||
ln = struct.unpack_from(">I", b, p+4)[0] if p+8 <= len(b) else 0
|
||||
if 0 < ln <= 64 and p+8+ln <= len(b):
|
||||
nm = b[p+8:p+8+ln].decode('latin1', 'replace')
|
||||
if nm and all(32 < ord(c) < 127 for c in nm):
|
||||
names.append((p, nm)); opt_total += 1
|
||||
ext[os.path.splitext(nm)[1].lower()] += 1
|
||||
p = b.find(b"opt ", p+4)
|
||||
offs, i = [], 4
|
||||
while i + 4 <= len(b):
|
||||
if b[i:i+4] in MAG:
|
||||
offs.append(i); i += 4
|
||||
else: i += 1
|
||||
if not names or not offs: continue
|
||||
npos = [p for p, _ in names]
|
||||
# each child -> index of the nearest preceding opt
|
||||
owner = collections.defaultdict(list)
|
||||
for k, o in enumerate(offs):
|
||||
j = bisect.bisect_left(npos, o) - 1
|
||||
if j >= 0: owner[j].append(k)
|
||||
for j, (p, nm) in enumerate(names):
|
||||
if not nm.lower().endswith(".tan"): continue
|
||||
ks = owner.get(j, [])
|
||||
if not ks: continue
|
||||
sizes = sorted({(offs[k+1] if k+1 < len(offs) else len(b)) - offs[k] for k in ks})
|
||||
tan_sites.append((os.path.basename(pakpath), ei, nm, len(ks), sizes))
|
||||
print(f"`opt ` blocks disc-wide: {opt_total}")
|
||||
print("\nby extension:")
|
||||
for e, c in ext.most_common(25):
|
||||
print(f" {e or '(none)':10} x{c}")
|
||||
print(f"\n.tan resources with >=1 child: {len(tan_sites)}")
|
||||
seen = collections.Counter()
|
||||
for t in tan_sites:
|
||||
seen[(t[0], t[2], t[3], tuple(t[4]))] += 1
|
||||
for (pk, nm, fr, sz), c in sorted(seen.items()):
|
||||
print(f" {pk:24} {nm:30} frames={fr:3} sizes={list(sz)} x{c} bundles")
|
||||
Reference in New Issue
Block a user