diff --git a/docs/re/f6-out-of-sample-RESULT.md b/docs/re/f6-out-of-sample-RESULT.md new file mode 100644 index 00000000..457ccbec --- /dev/null +++ b/docs/re/f6-out-of-sample-RESULT.md @@ -0,0 +1,57 @@ +# πŸ”΄ Out-of-sample: **three of six predictions failed** + +The predictions are in +[`f6-out-of-sample-prereg.md`](f6-out-of-sample-prereg.md), committed before the +capture. One fresh no-input boot, `f6c`. + +| # | quantity | predicted | `f6c` | | +|---|---|---|---|---| +| 1 | `pteff03a` / `pteff03` period | 1.200 Β±5 % | **1.1753** | βœ… | +| 2 | strip length ratio | 1.333 Β±6 % | **1.3009** | βœ… | +| 3 | pulse period / sweep loop | 0.100 Β±5 % | **0.0963** | βœ… | +| 4 | pulse amplitude vs declared curve | ≀3 levels | **8.73** | πŸ”΄ | +| 5 | `ptcopyright` ramp / parent ramp | 0.733 Β±8 % | **0.5500** | πŸ”΄ | +| 6 | sweep leads `ptbtn00f` | 0.138–0.141 loops | **0.0996** | πŸ”΄ | + +## What holds + +**Everything sweep-internal, across three captures.** The two leaves' period +ratio, their size ratio, and the pulse period as a fraction of the sweep loop all +reproduce on a run that had no hand in deriving them. Those stay βœ…. + +## What does not + +**The three plate-relative quantities.** And they fail *together and in the same +direction*: 5 lands at 0.75Γ— its prediction, 6 at 0.71Γ—. That is not noise β€” the +plate and `ptcopyright` both arrive earlier relative to the sweep in this run than +in the two that produced the numbers. + +πŸ”΄ **So the sweepβ†’plate separation is not a constant.** Three runs give +**0.138, 0.141, 0.0996**. My published "0.138 and 0.141, agreeing to 2.2 %" was +**n = 2**, and two runs agreeing is not reproducibility. The figure is withdrawn. + +⚠️ **The F6 conclusion survives; its number does not.** With no input the glow +still clearly precedes the plate β€” 0.0996 of a loop is a wide margin β€” and with β’Ά +the two still coincide. What is dead is any specific value for the lead. + +## πŸ”΄ And the check I built last iteration fails its first independent test + +`check_labels.py` was written to catch label drift, and I validated it on **the +two captures that produced the labels**. On the first capture it did not see, two +of its four checks fire. + +That is an instrument validated on its own training data β€” the same class as +every other error this session, one level up. **A detector tuned on n = 2 and +failing on n = 3 is not yet a detector**, and I presented it as one. + +I cannot yet say which of three causes is responsible, and I am not going to +guess: the labels could be wrong in `f6c`, the tolerances could be fitted, or the +game's plate timing could genuinely vary run to run. The diagnostic that would +separate them is more captures β€” `n = 3` is where this became visible and `n = 3` +is not enough to resolve it. + +## Reach + +Three captures, one screen, one emulator build. The sweep-internal results are +now 3-for-3; the plate-relative ones are 0-for-1 out of sample after 2-for-2 +in-sample.