re: multi-witness vote works, its threshold did not, and freezes persist

The graded output is the improvement. Instead of a single bit it reports how
many witnesses agree, and the sequence tells a coherent story: 11, 9, 7, 4, 1,
then 0 of 31, with the drop to zero at t=183 s coinciding exactly with the last
loss and 106 s of nothing after it. That is a real freeze, identified.

The threshold was wrong though. Flagging a stall at "fewer than half" marked the
entire run stalled, including samples in which craft were destroyed, so 11 of 31
advancing is a healthy guest rather than a stalled one. The cause is the cluster
choice: the modal rate was 93/s, far above the ~16.5/s frame rate timer_probe
measured, and those are subsystem counters that tick in bursts and sit idle in
most 15 s windows even while the game runs. Picking the modal cluster was
convenient rather than principled.

Fixed to prefer the cluster whose rate falls in the frame-rate band of 8-40/s,
falling back to modal only if none exists, and to flag a stall only when zero
witnesses advance, which is the signal the data actually supports. Not yet run.

The uncomfortable part: this run used the cheap probe and still froze, at about
183 s. The previous iteration's "0 stalled samples" came from the unreliable
single-word witness and cannot stand as validation. What the evidence supports
now is that the no-probe control ran 300 s clean, the heavy probe froze at 27 to
255 s, and the cheap probe froze at 183 s -- one run on each arm. Cheap sampling
plausibly helps but does not remove the freeze, and it is equally possible the
freeze is stochastic and the control was lucky. Recorded as unresolved rather
than resolved in the probe's favour.

Practical consequence: the usable window is roughly three minutes per run,
sometimes less, whether or not the probe is cheap. Experiments needing longer
have to survive a freeze or be redesigned around one.

tools/re-capture/wave7_probe.py

@@ -91,10 +91,18 @@ def main():
         va, vb = struct.unpack_from('>I', a, k)[0], struct.unpack_from('>I', b, k)[0]
         if va < vb and 5 < (vb - va) / 3.0 < 200:
             cands.append((lo + k, round((vb - va) / 3.0)))
-    modal = collections.Counter(r for _, r in cands).most_common(1)
-    ticks = [o for o, r in cands if modal and r == modal[0][0]][:32]
-    print('tick witnesses: %d candidates, modal rate %s/s, using %d'
-          % (len(cands), modal[0][0] if modal else '-', len(ticks))
+    # The MODAL cluster is not the frame counter. One run picked a modal rate of
+    # 93/s, and only 11 of 31 of those advanced during active combat -- they are
+    # subsystem counters that tick in bursts. timer_probe measured the frame-rate
+    # cluster at ~17/s, matching Canary's 14-19 fps on this box, so prefer a
+    # cluster in that band and fall back to modal only if none exists.
+    rates = collections.Counter(r for _, r in cands)
+    band = [r for r in rates if 8 <= r <= 40]
+    pick = max(band, key=lambda r: rates[r]) if band else (
+        rates.most_common(1)[0][0] if rates else None)
+    ticks = [o for o, r in cands if r == pick][:32]
+    print('tick witnesses: %d candidates, using %d at %s/s (frame-rate band)'
+          % (len(cands), len(ticks), pick)
           if ticks else 'tick witnesses: NONE -- RUN UNVALIDATED')
     last_ticks = [struct.unpack('>I', os.pread(fd, 4, o))[0] for o in ticks]
 
@@ -115,7 +123,10 @@ def main():
         if ticks:
             now = [struct.unpack('>I', os.pread(fd, 4, o))[0] for o in ticks]
             moved = sum(1 for x, y in zip(last_ticks, now) if y > x)
-            if moved * 2 < len(ticks):
+            # "Fewer than half" was too strict: 11 of 31 advanced while craft
+            # were being destroyed. The unambiguous signal in that run was
+            # 0 of 31, which coincided exactly with losses stopping.
+            if moved == 0:
                 st = '  *** GUEST STALLED (%d/%d witnesses moved) ***' % (moved, len(ticks))
                 stalls += 1
             last_ticks = now