b9afe324da
One fresh no-input boot against predictions registered before it. Sweep- internal quantities pass on a run that had no hand in deriving them: leaf period ratio 1.1753, size ratio 1.3009, pulse/sweep 0.0963. Those are now 3-for-3 across captures. The three plate-relative quantities fail, together and in the same direction: ptcopyright ramp ratio 0.550 vs 0.733 (0.75x) and sweep->plate separation 0.0996 vs 0.138-0.141 (0.71x). So the separation is not a constant -- three runs give 0.138, 0.141, 0.0996 -- and my published "2.2% agreement" was n=2. Withdrawn. The F6 conclusion survives qualitatively; the number does not. Also records that check_labels.py fails its first independent test. I validated it on the two captures that produced the labels, which is an instrument validated on its own training data. Two of four checks fire on f6c. I am not guessing which of labels/tolerances/run-variation is responsible. Caught mid-analysis that I had first run the checks against a log still being written; re-ran on the complete log and the numbers are identical, so the failures are not a truncation artefact. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jc4pciRArGHfxGGhEbwp5t
2.7 KiB
🔴 Out-of-sample: three of six predictions failed
The predictions are in
f6-out-of-sample-prereg.md, committed before the
capture. One fresh no-input boot, f6c.
| # | quantity | predicted | f6c |
|
|---|---|---|---|---|
| 1 | pteff03a / pteff03 period |
1.200 ±5 % | 1.1753 | ✅ |
| 2 | strip length ratio | 1.333 ±6 % | 1.3009 | ✅ |
| 3 | pulse period / sweep loop | 0.100 ±5 % | 0.0963 | ✅ |
| 4 | pulse amplitude vs declared curve | ≤3 levels | 8.73 | 🔴 |
| 5 | ptcopyright ramp / parent ramp |
0.733 ±8 % | 0.5500 | 🔴 |
| 6 | sweep leads ptbtn00f |
0.138–0.141 loops | 0.0996 | 🔴 |
What holds
Everything sweep-internal, across three captures. The two leaves' period ratio, their size ratio, and the pulse period as a fraction of the sweep loop all reproduce on a run that had no hand in deriving them. Those stay ✅.
What does not
The three plate-relative quantities. And they fail together and in the same
direction: 5 lands at 0.75× its prediction, 6 at 0.71×. That is not noise — the
plate and ptcopyright both arrive earlier relative to the sweep in this run than
in the two that produced the numbers.
🔴 So the sweep→plate separation is not a constant. Three runs give 0.138, 0.141, 0.0996. My published "0.138 and 0.141, agreeing to 2.2 %" was n = 2, and two runs agreeing is not reproducibility. The figure is withdrawn.
⚠️ The F6 conclusion survives; its number does not. With no input the glow still clearly precedes the plate — 0.0996 of a loop is a wide margin — and with Ⓐ the two still coincide. What is dead is any specific value for the lead.
🔴 And the check I built last iteration fails its first independent test
check_labels.py was written to catch label drift, and I validated it on the
two captures that produced the labels. On the first capture it did not see, two
of its four checks fire.
That is an instrument validated on its own training data — the same class as every other error this session, one level up. A detector tuned on n = 2 and failing on n = 3 is not yet a detector, and I presented it as one.
I cannot yet say which of three causes is responsible, and I am not going to
guess: the labels could be wrong in f6c, the tolerances could be fitted, or the
game's plate timing could genuinely vary run to run. The diagnostic that would
separate them is more captures — n = 3 is where this became visible and n = 3
is not enough to resolve it.
Reach
Three captures, one screen, one emulator build. The sweep-internal results are now 3-for-3; the plate-relative ones are 0-for-1 out of sample after 2-for-2 in-sample.