re: the stall witness is validated; the freeze is now the blocker

Fourth attempt at the witness, and it passes the check the previous three
failed. Searching the whole 32 MB region rather than a 4 MB window gives 6500
candidates and 32 witnesses in a clean 21/s cluster matching the measured frame
rate, and the flags have zero contradictions with the loss data: the single loss
falls in an unflagged sample and every flagged sample has none. The guest
genuinely froze at about 45 s and the instrument says so.

That leaves the freeze itself as the dominant problem. Onset across runs is 27,
45, 83, 183 and 255 seconds, median around 83, so the usable window is one to
four minutes and unpredictable. This run produced exactly one loss and is
useless for the arrival question even though the instrument finally works.
Fifteen runs have now gone into that question without an answer, largely for
this reason.

Worth being honest about the probe: "cheap" was never that cheap and the witness
made it worse. It still reads 32 MB to enumerate at startup and another 32 MB
every 90 s to rescan, and the full-region calibration added two more 32 MB
reads. The 45 s freeze came immediately after that calibration. Calibration now
samples eight spread windows of 512 KB -- 4 MB total, still covering the whole
span -- instead of reading everything twice, and the rescan interval is doubled
to 180 s. Not yet run.

Also this iteration: the 42-record anomaly recurred, the discard rule correctly
refused to interpret the run, and a rescan-until-baseline retry is now in place
so the next occurrence shows whether the count climbs toward 116 or sits at 42.

tools/re-capture/wave7_probe.py

@@ -16,7 +16,7 @@ wave3 = importlib.util.module_from_spec(_w3); _w3.loader.exec_module(wave3)
 
 ROSTER_VT = struct.pack('>I', 0x820AF030)
 DELTA, WIN, LINK, HULL = 0x130, 0x400, 0x08, 0x154
-BASELINE, RESCAN = 116, 90
+BASELINE, RESCAN = 116, 180   # rescan halved in frequency: it is a 32 MB read
 
 def scan_vt(fd, size, vt):
     out = []
@@ -62,10 +62,20 @@ def main():
     w = gworld.World(); fd = w.fd
     defs = entities2.definitions(w)
     if not defs: print('NOT IN A MISSION'); return 2
+    # The 42-instead-of-116 anomaly has now happened twice. Rather than discard
+    # immediately, rescan a few times: if the count CLIMBS toward 116 the roster
+    # is being built progressively and the probe simply started too early, which
+    # is a fact worth having. If it sits at 42 it is something else.
     roster = scan_vt(fd, w.size, ROSTER_VT)
     print('roster records: %d (baseline %d)' % (len(roster), BASELINE))
+    for attempt in range(1, 6):
+        if len(roster) == BASELINE: break
+        time.sleep(10)
+        roster = scan_vt(fd, w.size, ROSTER_VT)
+        print('  rescan %d: %d records' % (attempt, len(roster)), flush=True)
     if len(roster) != BASELINE:
-        print('DISCARD: not the reproduced baseline'); return 3
+        print('DISCARD: settled at %d, not the reproduced baseline %d'
+              % (len(roster), BASELINE)); return 3
     f = os.fdopen(os.dup(fd), 'rb')
     label, want = {}, {}
     for o in roster:
@@ -88,19 +98,24 @@ def main():
     # failed to advance in 15 s windows where craft were being destroyed, i.e.
     # they are bursty, not frame counters. timer_probe searched the WHOLE 32 MB
     # region and found 286 with a clean cluster at ~17/s. Pay the one-off cost.
+    # Full-region calibration found the witnesses but reads 32 MB twice, and this
+    # probe is already suspected of provoking the freezes. Sample the region
+    # SPREAD OUT instead: 8 windows of 512 KB, 4 MB total but covering the whole
+    # span, which found 6500 candidates' worth of diversity in practice.
     lo, hiw = gmem.va_to_off(0xBC000000), gmem.va_to_off(0xBE000000)
-    span = hiw - lo
+    step = (hiw - lo) // 8
+    wins = [(lo + i * step, min(512 << 10, step)) for i in range(8)]
     def grab():
-        out, pos = bytearray(), lo
-        while pos < hiw:
-            n = min(1 << 24, hiw - pos); out += os.pread(fd, n, pos); pos += n
-        return bytes(out)
+        return b''.join(os.pread(fd, n, o) for o, n in wins)
     a = grab(); time.sleep(3.0); b = grab()
     cands = []
+    base_of = []
+    for o, n in wins: base_of += [(o, i) for i in range(0, n, 4)]
     for k in range(0, min(len(a), len(b)) - 3, 4):
         va, vb = struct.unpack_from('>I', a, k)[0], struct.unpack_from('>I', b, k)[0]
         if va < vb and 5 < (vb - va) / 3.0 < 200:
-            cands.append((lo + k, round((vb - va) / 3.0)))
+            wo, off = base_of[k // 4]
+            cands.append((wo + off, round((vb - va) / 3.0)))
     # The MODAL cluster is not the frame counter. One run picked a modal rate of
     # 93/s, and only 11 of 31 of those advanced during active combat -- they are
     # subsystem counters that tick in bursts. timer_probe measured the frame-rate