re: the fast probe stalls -- its own dense negatives are withdrawn

Cross-checked the instrument built last iteration against an independent
grabber while both watched the same screen, and it fails.

A single long-lived ffmpeg x11grab stream degrades and then freezes:

  862 frames in 540.1 s = 1.60 fps        (it starts at 3.98)
  t=450/480/510/540 s: surface mean 5.21, identical every time

At that same moment `import` read surface mean 125.65, and a freshly
started ffmpeg stream read 122.43 -- agreeing with import to 3%. So the
acquisition was broken, not the analysis: the stream replayed a stale
frame while the screen was 24x brighter.

That withdraws last iteration's headline. "2391 frames over 600 s from
t=0, max glyph 0" cannot distinguish "the title never appeared" from "the
stream froze early and repeated one frame 2391 times". Its 3.98 fps was
measured over the first 20 s, before the degradation. Sample count is not
coverage unless the samples are known independent.

Fixed: the stream is now torn down and restarted every 30 s. Startup is
~0.3 s, cheap against the title's window, and it guarantees live frames.

Separately, the cache hypothesis was tested and is SUPPORTED. cache,
cache0, cache1, cache_host moved aside (to /tmp/xenia-cache-aside, not
deleted) and the surface renders again: import reads mean 54.8 and 68.6
with 100% non-black warm content, against 0.07 and 0.08% non-black in the
black run; 773 of 862 probe frames had >2% non-black. One run each side
and many kill -9s before the black one, so it is supported, not proven --
the old caches are kept for reproduction.

Still no title, but that number now comes from a stalling probe and
establishes nothing either way.

METHOD: validating a probe on static images tests its analysis, not its
acquisition -- cross-check against an independent grabber during a run.
This commit is contained in:
Sylpheed RE agent
2026-08-29 02:01:47 +00:00
parent 8f07819b02
commit 801a9116c4
4 changed files with 95 additions and 6 deletions

View File

@@ -532,3 +532,15 @@ agent's loop prompt, i.e. nowhere durable. See [`README.md`](README.md) for the
non-zero frame-to-frame rmse), so the failure mode changed over the session.
Reaching back to re-explain earlier results with the newest cause would have
been the fourth over-reach in a row.
* **Validating a probe on static test images tests its ANALYSIS, not its
ACQUISITION.** The fast probe's counter was controlled against committed
captures and matched exactly (753 / 327), which proved the numpy expression
right and the frame source untested. The source was the broken half: a
long-lived x11grab stream degrades from 3.98 to 1.60 fps and then freezes,
repeating one stale frame. The check that catches it is to read the same screen
with an **independent grabber at the same moment**`import` said 125.65 where
the stream said 5.21. Do that during a run, not only at design time.
* **A dense negative from one instrument is worth less than one cross-check.**
"2 391 frames, zero hits" reads as overwhelming and collapses to nothing once
the frames may all be the same frame. Sample count is not evidence of coverage
unless the samples are known to be independent.

View File

@@ -451,3 +451,9 @@ neighbourhood, not just the line.
attached at t=0: **2 391 frames over 600 s at 3.98 fps from launch**, max glyph
0. The plate did not appear at any point in the first ten minutes.
[`capture-harness-status.md`](capture-harness-status.md)
* "2 391 frames over 600 s from t=0, max glyph 0, therefore the title never
appears in the first ten minutes" → **withdrawn: the instrument stalls.** A
single long-lived x11grab stream degrades 3.98 → 1.60 fps and then freezes,
repeating one stale frame; cross-checked, it read surface mean 5.21 where
`import` read 125.65 at the same moment. A dense negative from a frozen stream
is not a negative. [`capture-harness-status.md`](capture-harness-status.md)

View File

@@ -170,3 +170,56 @@ nothing reaches the screen.
**The test is one line and one run:** move `cache*` aside and boot again. Not
done — it needs a fresh 10-minute run, and this iteration had spent its emulator
budget proving the negative above.
---
## 🔴 The fast probe STALLS — and that invalidates its own dense negatives
**2026-08-29.** The instrument built last iteration is unfit for long runs, found
by cross-checking it against an independent grabber *while both watched the same
screen*.
A single long-lived `ffmpeg` x11grab stream degrades and then freezes:
```
862 frames in 540.1 s = 1.60 fps (it starts at 3.98 fps)
t=450s surface mean 5.21 nonblack 10.1%
t=480s surface mean 5.21 nonblack 10.1%
t=510s surface mean 5.21 nonblack 10.1%
t=540s surface mean 5.21 nonblack 10.1%
```
Four consecutive 30-second marks reporting an identical value. At that same
moment an `import` grab of the same display read **surface mean 125.65**, and a
*freshly started* ffmpeg stream read **122.43** — agreeing with `import` to 3 %.
So the acquisition, not the analysis, was broken: the stream was replaying a
stale frame while the screen was 24× brighter.
🔴 **Therefore the previous section's headline is withdrawn.** "2 391 frames over
600 s from t=0, max glyph 0" cannot distinguish *the title never appeared* from
*the stream froze early and repeated one frame 2 391 times*. Its 3.98 fps was
measured over the first 20 s, before the degradation.
**Fixed** in `tools/re-capture/fast_title_probe.py`: the stream is torn down and
restarted every 30 s. Startup costs ~0.3 s, cheap against the title's window, and
it buys a guarantee the frames are live.
## ✅ Clearing the shader cache restored rendering
The cache hypothesis from the previous section was tested: `cache`, `cache0`,
`cache1` and `cache_host` moved aside (to `/tmp/xenia-cache-aside`, not deleted),
then a fresh boot.
The surface **renders again**, confirmed with the independent grabber:
`import` reads surface mean **54.8** and **68.6** on successive samples with 100 %
non-black warm content, against **0.07** and 0.08 % non-black in the black run.
The probe's own summary agrees where it can be trusted — **773 of 862 frames had
> 2 % non-black**.
🟡 **Supported, not proven.** One run each side, and the emulator was `kill -9`'d
many times before the black run, so a corrupt pipeline cache is plausible but a
single A/B does not exclude run-to-run variation. The old caches are kept in
`/tmp/xenia-cache-aside` if anyone wants to reproduce the failure.
**Still no title.** Even rendering, `max glyph 0` — but that number now comes
from a stalling probe, so it establishes nothing either way.

View File

@@ -1,4 +1,16 @@
"""Fast title probe: one long-lived x11grab stream, ~3.5 fps, glyph count per frame.
"""Fast title probe: a PERIODICALLY RESTARTED x11grab stream + glyph count.
🔴 DO NOT run one x11grab stream for the whole session. Measured 2026-08-29: a
single long-lived stream degrades from 3.98 fps to 1.60 fps and then FREEZES,
reporting a stale frame indefinitely. Cross-checked at one moment against an
independent `import` grab: the stream said surface mean 5.21 while `import` said
125.65, and the stream repeated that same value for four consecutive 30 s marks.
Any negative result from a stalled stream is worthless -- an earlier "2391
frames, zero hits" claim came from exactly that failure.
So the stream is torn down and restarted every RESTART_S seconds. Startup costs
~0.3 s, which is cheap against the title's few-second window and buys a
guarantee that the frames are live.
Replaces the two-`screenshot` polling loop, which costs ~4-11 s per grab while
xenia runs and so samples every ~41 s -- slower than the title screen lasts.
@@ -10,13 +22,19 @@ import numpy as np
W,H = 1280,720
LIMIT = float(sys.argv[1]) if len(sys.argv)>1 else 120
NEED = int(sys.argv[2]) if len(sys.argv)>2 else 400
p = subprocess.Popen(["ffmpeg","-loglevel","error","-f","x11grab","-draw_mouse","0",
"-video_size",f"{W}x{H}","-i",":98","-r","4","-f","rawvideo","-pix_fmt","rgb24","-"],
stdout=subprocess.PIPE, bufsize=W*H*3*2)
n=W*H*3; t0=time.time(); frames=0; best=0; hits=0
RESTART_S = 30 # tear down and restart the stream this often
def _open():
return subprocess.Popen(["ffmpeg","-loglevel","error","-f","x11grab","-draw_mouse","0",
"-video_size",f"{W}x{H}","-i",":98","-r","4","-f","rawvideo","-pix_fmt","rgb24","-"],
stdout=subprocess.PIPE, bufsize=W*H*3*2)
p=_open()
n=W*H*3; t0=time.time(); frames=0; best=0; hits=0; seg=time.time()
while time.time()-t0 < LIMIT:
if time.time()-seg > RESTART_S: # a stalled stream reports stale frames forever
p.kill(); p=_open(); seg=time.time()
buf=p.stdout.read(n)
if len(buf)<n: break
if len(buf)<n:
p.kill(); p=_open(); seg=time.time(); continue
a=np.frombuffer(buf,np.uint8).reshape(H,W,3).astype(int)
r,g,b=a[:,:,0],a[:,:,1],a[:,:,2]
c=int(((g>130)&(g-r>45)&(g-b>45)).sum())