re: the stall witness works, and stalls may be caused by the probe

First run with the tick witness. It flagged a stall from t=27 s and every sample
after, and the pilot's own telemetry -- which the probe never reads -- agrees:
35 distinct speed values across the whole log and exactly 1 in the last 400
lines, against 236 in the first 400 of a healthy run. All variation is in the
first ~50 s. The witness is validated.

It earned its keep on that same run. Without it the output reads as "no arrivals
across 313 seconds with 300 craft resident" -- clean, quotable and completely
worthless, because the game was frozen for 90 % of it. Rule adopted: a run whose
witness reports a stall is discarded, and every write-up states the witness
result. Flat samples are not evidence unless the witness says the guest was
advancing.

Stalls are frequent and early. The last three long runs stalled at roughly 255 s,
83 s (after the player died) and 27 s. That makes long observation windows
unreliable, and long windows are exactly what the arrival question needs.

Leading suspect is the probe itself, and it is recorded because it is
uncomfortable rather than despite it. AGENT.md warns that a full memory scan
competes with the emulator for every core under lavapipe, and these probes have
grown heavier each iteration: wave6 now reads the entire 32 MB entity heap plus
about 300 extra preads every 12 seconds while the game renders. If that is the
cause, the instrument has been degrading the thing it measures and the earlier
"no arrival" results were collected under conditions the game was struggling
with.

Next is a control that needs no new decoding: run the hunting pilot for 300 s
with no probe at all and judge from the pilot log alone. If it does not stall,
sampling has to get much cheaper -- narrow the scan to the roster region, sample
less often, or reread only the craft bases already located instead of rescanning
the heap.

The multi-squadron kill-threshold test did not run: the guest froze before
anything was destroyed, so there were no losses to threshold.
This commit is contained in:
Sylpheed RE agent
2026-08-24 15:57:09 +00:00
parent 44e9f8dc94
commit e1e035fb57
2 changed files with 87 additions and 2 deletions

View File

@@ -391,8 +391,20 @@ search cannot find a *schedule*.
objectives, specific squadrons untested). 🔴 **Guest stalled at ~t=255 s**
last 400 pilot lines have 1 distinct speed vs 236 in the first 400 — so trailing
flat samples in ANY earlier run may be a frozen guest, not a quiet mission.
Probe now carries a frame-rate tick witness printing `*** GUEST STALLED ***`
(implemented, **not yet run**).
Probe now carries a frame-rate tick witness printing `*** GUEST STALLED ***`.
* ✅🔴 **The stall witness WORKS and is now mandatory**
([`guest-stalls.md`](guest-stalls.md), 2026-08-24). Validated against the
pilot's independent telemetry (35 distinct speeds all run, 1 in the last 400
lines; witness flagged t=27 s). It immediately **discarded** a run that would
otherwise have read as "no arrivals across 313 s" — worthless, the game was
frozen for 90 % of it. **Rule: a stalled run is discarded, and every write-up
must state the witness result.** 🔴 Stalls are frequent and early — last three
long runs stalled at ~255 s, 83 s and **~27 s**. 🟡 **Leading suspect: the probe
itself** — it reads the whole 32 MB heap plus ~300 `pread`s every 12 s while
lavapipe renders, and AGENT.md warns a full scan competes for every core.
**Next: run the pilot 300 s with NO probe**, judge from its log alone; if it
does not stall, the instrument is degrading what it measures and sampling must
get much cheaper. ❔ Multi-squadron threshold test did not run (zero losses).
⚠️ The ~210 s title movie at boot is the binding constraint on observable game
time per turn.
Earlier framing:

73
docs/re/guest-stalls.md Normal file
View File

@@ -0,0 +1,73 @@
# The guest stalls, often — and the probe now detects it
Status: ✅ the stall witness works and is validated against independent
telemetry; ✅ it correctly discarded a run that would otherwise have been written
up as evidence; 🔴 stalls are frequent and early enough to threaten the whole
wave line of work; 🟡 the probe's own memory scanning is the leading suspect and
is untested.
## ✅ The witness works
`wave6_probe.py` locates a word that advances at frame rate and prints
`*** GUEST STALLED ***` on any sample where it fails to advance. Its first run:
```
tick witness at 0x11c225810, ~26.7/s
t= 14s craft=300 deployed=41 ARRIVALS=0 losses=0
t= 27s craft=300 deployed=41 ARRIVALS=0 losses=0 *** GUEST STALLED ***
... stalled for every remaining sample to t=313s
```
**Validated independently.** The pilot's own telemetry, which the probe never
sees, agrees: **35 distinct speed values across the entire log, and exactly 1 in
the last 400 lines**, against 236 in the first 400 lines of a healthy run. All
the variation is in the first ~50 s. The witness flagged the stall at t = 27 s and
the telemetry puts it in the same place.
## ✅ It earned its keep immediately
Without it, this run reads as *"no arrivals across 313 seconds with 300 craft
resident"* — a clean, quotable, and completely worthless result, since the game
was frozen for 90 % of it. That is exactly the failure the previous iteration
warned about, caught automatically on the first run after the fix.
**Operational rule:** a run whose witness reports a stall is discarded, and every
future write-up must state the witness result. Flat samples are not evidence
unless the witness says the guest was advancing.
## 🔴 Stalls are frequent and early
| run | stall onset | usable window |
|---|---|---|
| keep-out 1400 (previous) | ~255 s | ~255 s |
| keep-out 600 | player died 83 s, frozen after | 83 s |
| this run | **~27 s** | ~27 s |
Two of the last three long runs froze, one of them almost immediately. This is
consistent with the pre-existing freeze recorded elsewhere in the corpus, but its
frequency makes long observation windows unreliable — and long windows are
exactly what the arrival question needs.
## 🟡 Leading suspect: the probe itself
`docker/agent/AGENT.md` warns that a full memory scan "competes with the emulator
for every core under lavapipe", and the probes have grown heavier every
iteration. `wave6_probe` now reads the whole 32 MB entity heap **plus** a 1 KB
`pread` per craft — roughly 300 extra syscalls — every 12 seconds, while the
emulator is trying to render.
That is a plausible cause of the freezes and it has never been tested. It would
also be an uncomfortable one: it would mean the instrument has been degrading the
thing it measures, and that the "no arrival" results were collected under
conditions the game was struggling with.
**Test:** run the hunting pilot for 300 s with **no probe at all**, and read the
outcome only from the pilot's own log — distinct speed values over time. If it
does not stall, the probe is the cause and the sampling has to get much cheaper
(narrow the scan to the roster records' own region, sample less often, or read
only the craft bases already located rather than rescanning).
## Not settled this iteration
The multi-squadron kill-threshold test did not run: the guest froze before the
pilot destroyed anything, so there were zero losses and nothing to threshold.