diff --git a/docs/port/HANDOFF.md b/docs/port/HANDOFF.md index 879d9596..deaeb407 100644 --- a/docs/port/HANDOFF.md +++ b/docs/port/HANDOFF.md @@ -6478,3 +6478,48 @@ drop it. capture-only labels, **fails its first independent test**: I validated it on the two captures that produced the labels. Two of four checks fire on a third. Treat it as unproven, not as a check. + +## 🟑 Shaping the residue above (issue #9) β€” the sweepβ†’plate lead stays withdrawn; nothing new to author + +[`docs/re/f6-residue-shaping.md`](../re/f6-residue-shaping.md). No new number +for you here β€” this is a static review of the three failures above, and the +withdrawal stands exactly as it was. + +**What's new:** the three failures aren't three independent misses. Two of +them (the `ptcopyright`/parent ramp ratio, the sweep-leads-plate lead) both +miss **in the same direction** (0.75Γ—, 0.71Γ— of predicted), and the corpus +already has a decoded reason they might: `ui-layout`'s own +[`f6-unit10`](../re/f6-unit10-parent-alpha-gates-the-sweep.md) says the sweep +is gated by its own parent's alpha, "not by anything to do with the plate" β€” +i.e. the sweep-family and plate-family clocks are **declared as separately +triggered**. A ratio between two independently-triggered elements' timings +is not guaranteed constant just because each element's own animation is. The +third failure (the pulse's own amplitude curve fit) is a different, narrower +thing β€” a single element's self-consistency check, not a cross-element one β€” +and is not explained by the same argument. + +⚠️ **This is a hypothesis the existing write-up points at, not a new +measurement.** The gross mislabeling that caused the *original* 1.7Γ— conflict +was fixed two days before the out-of-sample prereg was written +([`f6-plate-identity-and-clock-conflict-resolved.md`](../re/f6-plate-identity-and-clock-conflict-resolved.md)), +so that specific bug is ruled out as the cause of *this* residue β€” but +whether the cross-group phase genuinely varies boot to boot, versus an +artifact in how the checker derives ratios from raw frame numbers, is still +open. **Nothing here reinstates the withdrawn 0.138–0.141 lead.** + +**`check_labels.py` repaired, not patched.** It now reports two groups: +*identity* checks (which element you're looking at β€” clock-free, 3-for-3 +including `f6c`) gate the exit code; *timing* checks (a cross-element ratio +and a self-consistency curve fit β€” 0-for-2 on `f6c`) print in full but no +longer read as a label failure. Tolerances are untouched β€” widening them to +pass `f6c` would be tuning the check on the case that failed it. Verified +with synthetic data shaped like the real `f6c` residue (3/3 identity pass, +0/2 timing fail, exit 0) and with the selftest's injected mislabel confirmed +to still fail an *identity* check (exit 1) β€” no capture available in this +container to run it against real logs. + +**Follow-on filed, not started:** three more independent no-input boots with +`check_labels.py`'s per-run diagnostics saved, to see whether the timing +residue clusters (points at a real boot-order dependency) or scatters (points +at instrument sensitivity). `state/proposed` β€” new exploratory dynamic RE, +not a continuation of this shaping pass. diff --git a/docs/re/f6-residue-shaping.md b/docs/re/f6-residue-shaping.md new file mode 100644 index 00000000..8397f61c --- /dev/null +++ b/docs/re/f6-residue-shaping.md @@ -0,0 +1,120 @@ +# Shaping the F6 out-of-sample residue (issue #9) + +`f6-out-of-sample-RESULT.md` already did the honest thing: three predictions +held, three failed, and the failing three were withdrawn rather than patched. +This page answers what issue #9 asked for β€” what remains open in my own words, +what instrument would close it, and whether `check_labels.py` is worth +repairing or withdrawing. + +## The residue is not three independent failures β€” it's one, seen three ways + +| # | quantity | what it actually measures | +|---|---|---| +| 4 | pulse amplitude vs `ptbtn00f.rat`'s declared curve | a **single element's own** curve, content-lag-aligned to itself | +| 5 | `ptcopyright` ramp / parent (sweep-gate) ramp | a **ratio across two separately-triggered elements** | +| 6 | sweep leads `ptbtn00f`, in sweep-loops | a **cross-element phase offset** | + +Items 5 and 6 both fail **in the same direction** (0.75Γ— and 0.71Γ— of +predicted) β€” that is not two coincidental misses, it is one relationship +measured two ways. Item 4 is a different animal: it never compares two +elements, so it needs its own explanation, below. + +### Why 5 and 6 plausibly share a cause, and it isn't mislabeling + +[`f6-plate-identity-and-clock-conflict-resolved.md`](f6-plate-identity-and-clock-conflict-resolved.md) +(2026-09-02) fixed the gross mislabeling β€” `ptbtn00f` is the pulsing plate, +`x=-0.54` is `ptcopyright` β€” **two days before** the prereg +(`8927d2f`, 2026-09-04) was written. `check_labels.py` already encodes the +corrected identities. So "the label is wrong" in the sense that cost us the +original 1.7Γ— conflict is ruled out as the explanation for *this* residue β€” +that bug was fixed and stayed fixed in-sample. + +What is *not* ruled out, and is already sitting in the corpus: +[`f6-unit10-parent-alpha-gates-the-sweep.md`](f6-unit10-parent-alpha-gates-the-sweep.md) +states plainly that the sweep is switched on by its own parent's declared +alpha ramp, "not by anything to do with the plate." The sweep-family clock and +the plate-family clock are **declared as separately gated**. A ratio between +two independently-triggered elements' timings is only a constant if both +triggers fire in a fixed relative order and interval to each other β€” which is +an empirical claim about the boot sequence, not something either element's own +`.rat` declaration can guarantee. Items 5 and 6 are exactly that kind of +cross-group ratio; items 1–3 are not (each stays inside one sweep-family +element or one loop-ratio internal to it), which is the cleanest read I have +for why 1–3 are 3-for-3 and 5–6 are 0-for-2. + +I am **not** asserting this is confirmed β€” it is the hypothesis the existing +corpus already points at, not a new measurement. The alternative (an +instrument artifact in how `check_labels.py` derives `parent_ramp` and +`ptcopyright_ramp` from raw frame numbers in a differently-shaped capture) is +not excluded either. Both are live. + +### Item 4 needs its own account + +The amplitude-curve fit doesn't compare two elements; it compares +`ptbtn00f`'s own drawn alpha, within its own cycle, against its own disc-read +curve, after a content-based lag search. Its `upf` (title-units-per-frame) +comes from **this same run's** `pulse_period` β€” so if `f6c`'s automatic +slot-finder measured a noisier `pulse_period` (fewer clean cycles, a +frame-drop landing differently), the curve fit inherits that error even if the +underlying decoded curve is right. `check_labels.py` already prints the best +lag and the mean error; it does not yet print the per-run `pulse_period` +sample count or variance, which is the number that would tell them apart. + +## What instrument would close it + +Not more reasoning from what's already written down β€” that's what this page +is, and it is the ceiling of what a static review can do. **More independent +no-input boots**, ⟨capture⟩, with `check_labels.py`'s per-run diagnostics +(lag, sample count, per-cycle period variance) saved alongside the pass/fail +line rather than discarded. Two outcomes distinguish the two live hypotheses: + +* items 5–6 **cluster** into two or more distinct values across runs (e.g. a + bimodal split rather than a spread) β†’ points at a real, possibly discrete, + boot-order dependency in when each group's trigger fires; +* items 5–6 **scatter** without structure, while item 4's error tracks + `pulse_period`'s own sample-to-sample noise β†’ points at instrument + sensitivity in the frame-based measurement, not a game fact. + +Three more runs (total n = 6) is the smallest batch that could show clustering +vs scatter; two would not be enough to tell either from noise, which is the +same lesson `f6-out-of-sample-RESULT.md` already drew from n = 2. Filed as a +follow-on item rather than run here, since it is new exploratory dynamic RE +and the mission asks for state/approved before starting it, not a +continuation of #9's own scope. + +## `check_labels.py` β€” repair, don't patch the tolerance + +**Not worth silently loosening.** Widening tolerance 4/5 until `f6c` passes +would be tuning a check on the case that failed it β€” the exact error class +this whole session has been naming. The tool staying red is more honest than +a quietly-widened green. + +**Worth repairing its claim, though**, because right now a failure reads as +"N LABEL(S) DRIFTED" for checks 4–5 exactly as it would for checks 1–3, and +those are different claims: 1–3 test *which element you're looking at* +(clock-free, robust, still 3-for-3 including out of sample); 4–5 test +*whether a cross-element or self-consistency timing number holds*, which the +analysis above says may not even be a constant. Splitting them lets a future +reader β€” human or agent β€” tell "the identification is wrong" from "the +identification is right and the timing varies" without re-deriving this page. + +Applied below: `check_labels.py` now prints identity checks and timing +checks as two labelled groups, and only identity checks affect the exit code. +Timing-check failures still print in full, still say FAIL, and still return a +nonzero *count* in the summary line β€” they are demoted from gating, not +hidden. + +## Answering issue #9's pass criterion + +Concrete follow-on: + +* **New item** (state/proposed, this iteration): "F6 β€” cluster or scatter? + Three more independent no-input boots, diagnostics saved, to tell a real + cross-group phase dependency from an instrument artifact in the + frame-based ratios." Instrument: ⟨capture⟩ Γ— 3, `check_labels.py` + (repaired) with diagnostics logged per run. +* `check_labels.py` repaired in place (this commit) to stop conflating the + two claims β€” no emulator needed for that part, verified by + `--selftest` still detecting the seeded mislabel. +* The sweepβ†’plate separation constant stays withdrawn, as it already was. + Nothing here reinstates it. diff --git a/tools/re-capture/check_labels.py b/tools/re-capture/check_labels.py index 22844771..801ce16b 100755 --- a/tools/re-capture/check_labels.py +++ b/tools/re-capture/check_labels.py @@ -119,8 +119,20 @@ def features(path, swap_plate=False): f['pulse_amp_err'] = 99.0 return f +# ⚠️ Two different claims live here, and docs/re/f6-residue-shaping.md is why +# they are no longer reported as one. IDENTITY checks test *which element you +# are looking at* -- clock-free ratios internal to the sweep family, robust +# out of sample (3-for-3 on `f6c`, the first capture not used to derive them). +# TIMING checks test whether a cross-element phase or a self-consistency curve +# fit holds to a specific number -- and out of sample, both failed (0-for-2 on +# `f6c`), in a way the identity checks did not. Folding a TIMING failure into +# "N LABEL(S) DRIFTED" reads as "the identification is wrong", which out-of- +# sample evidence does not support; what may not hold is that the *timing +# relationship* is a constant at all. See f6-residue-shaping.md before +# tightening these tolerances -- they were already tuned on n=2 once. + # label -> (derived ratio, declared value, tolerance, what the label asserts) -def checks(f): +def identity_checks(f): return [ ("pteff03a is the 720-unit leaf (not a second copy of the 600)", f['pteff03a_period']/f['pteff03_period'], 720/600, 0.05), @@ -128,37 +140,51 @@ def checks(f): f['pteff03a_len']/f['pteff03_len'], 800/600, 0.06), ("the pulsing slot is ptbtn00f (120-unit loop vs the sweep's 600 leaf units)", f['pulse_period']/f['pteff03_period'], 0.1, 0.05), + ] + +def timing_checks(f): + return [ ("the -0.54 quad is ptcopyright (22-unit ramp vs the parent's 30)", f['ptcopyright_ramp']/f['parent_ramp'], 22/30, 0.08), ("the pulse AMPLITUDE matches ptbtn00f.rat's declared 8-key curve (peak 80)", f['pulse_amp_err'], 0.0, None), ] -def run(swap=False): +def _run_group(f, group, indent=" "): bad = 0 + for label, got, want, tol in group(f): + if tol is None: # absolute: mean |alpha| error, <=3 levels + err, ok = got, got <= 3.0 + print(f"{indent}[{'PASS' if ok else 'FAIL'}] mean |alpha| error {got:5.2f} levels (tol 3.00, best lag {f.get('pulse_lag',0):.2f} fr) {label}") + bad += not ok + continue + err = abs(got-want)/want + ok = err <= tol + bad += not ok + print(f"{indent}[{'PASS' if ok else 'FAIL'}] {got:.4f} vs {want:.4f} ({err*100:4.1f}%, tol {tol*100:.0f}%) {label}") + return bad + +def run(swap=False): + id_bad, timing_bad = 0, 0 for name, path in CAPS.items(): f = features(path, swap_plate=swap) print(f" {name}:") - for label, got, want, tol in checks(f): - if tol is None: # absolute: mean |alpha| error, <=3 levels - err, ok = got, got <= 3.0 - print(f" [{'PASS' if ok else 'FAIL'}] mean |alpha| error {got:5.2f} levels (tol 3.00, best lag {f.get('pulse_lag',0):.2f} fr) {label}") - bad += not ok - continue - err = abs(got-want)/want - ok = err <= tol - bad += not ok - print(f" [{'PASS' if ok else 'FAIL'}] {got:.4f} vs {want:.4f} ({err*100:4.1f}%, tol {tol*100:.0f}%) {label}") - return bad + print(f" identity (gates the exit code):") + id_bad += _run_group(f, identity_checks, indent=" ") + print(f" timing (reported, not gating -- see f6-residue-shaping.md):") + timing_bad += _run_group(f, timing_checks, indent=" ") + return id_bad, timing_bad if __name__ == '__main__': if '--selftest' in sys.argv: print("SELFTEST β€” the plate label deliberately pointed at ptcopyright.") print("A check that cannot fail here would not have caught the real error.\n") - bad = run(swap=True) + id_bad, timing_bad = run(swap=True) + bad = id_bad + timing_bad print(f"\n{'OK: mislabel detected' if bad else 'BROKEN: mislabel NOT detected'} ({bad} failures)") sys.exit(0 if bad else 1) print("Element labels in the F5/F6 findings, re-derived from captures + disc:\n") - bad = run() - print(f"\n{'all labels agree' if not bad else str(bad)+' LABEL(S) DRIFTED'}") - sys.exit(1 if bad else 0) + id_bad, timing_bad = run() + print(f"\n{'all identity checks agree' if not id_bad else str(id_bad)+' IDENTITY CHECK(S) DRIFTED'}" + f"; {timing_bad} timing check(s) failed (informational)") + sys.exit(1 if id_bad else 0)