From 9daf9dc5920dfd3ba114670eaffed4df31432d43 Mon Sep 17 00:00:00 2001 From: Sylpheed RE agent Date: Mon, 24 Aug 2026 20:28:15 +0000 Subject: [PATCH] =?UTF-8?q?re:=20sample=20from=20the=20first=20moment=20?= =?UTF-8?q?=E2=80=94=20still=20flat,=20and=20the=20wait=20variant=20caught?= =?UTF-8?q?=20the=20ready=20room?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit early_probe defers everything expensive: no witness calibration, no per-record labelling, and the heap scan uses bytes.find. Setup now completes in 0.5 to 0.8 seconds instead of about 25, so the first sample lands essentially at flight detection. It is still flat. Deployed reads 41 at flight+0.8 s and every sample after, with the only change in 252 s being 41 to 40 when one squadron was wiped out. No climb at any point. Under the frames reading that is expected, since the whole phase-1 schedule would be finished within eight seconds of mission start. The attempt to get ahead of flight detection did not work, and the reason is worth recording. A second run started the probe before the launch and waited for the roster to appear. It appeared with 116 records, but the numbers were deployed 39 and craft 276, flat for the whole window. That is the ready room: the roster is built before take-off, so waiting for it catches the pre-flight scene, and the probe's window expired around the time flight actually began. Waiting for the roster is not the same as catching mission start, and the test as designed does not do what it claims. The two runs together do suggest something, held at 🟡 because they are different runs. The ready room shows 39 deployed and 276 craft while flight shows 41 and 300, so two records and twenty-four craft appear between them. That points at deployment being a single step at take-off rather than a schedule unfolding during the mission. Next is one run of about 400 s with the probe waiting for the roster, long enough to span ready room, take-off and flight in a single continuous series, which would show the 39 to 41 step directly or refute it. --- docs/re/BACKLOG.md | 10 +++ docs/re/mission-arrival-watch.md | 52 ++++++++++++++ tools/re-capture/early2_session.sh | 12 ++++ tools/re-capture/early_probe.py | 107 +++++++++++++++++++++++++++++ tools/re-capture/early_session.sh | 10 +++ 5 files changed, 191 insertions(+) create mode 100755 tools/re-capture/early2_session.sh create mode 100755 tools/re-capture/early_probe.py create mode 100755 tools/re-capture/early_session.sh diff --git a/docs/re/BACKLOG.md b/docs/re/BACKLOG.md index 96b16ef0..f350b837 100644 --- a/docs/re/BACKLOG.md +++ b/docs/re/BACKLOG.md @@ -484,6 +484,16 @@ search cannot find a *schedule*. first sample (~25 s in) — explaining every null result, and consistent with `deployed=41` already at t=0. **Test: sample at flight+2 s vs flight+30 s** — a reordering of the probe, not new decoding. +* ✅🔴 **(2026-08-24) Early sampling done; still flat.** `early_probe.py` cuts + setup from ~25 s to **0.5–0.8 s** (no calibration, no labelling, `bytes.find` + scan). At flight+0.8 s **deployed=41 already**, flat for 252 s (only 41→40, one + squadron wiped). 🔴 **The `--wait` variant caught the READY ROOM, not flight** — + the roster is built before take-off, so waiting for it is not catching mission + start; that window (deployed=39, craft=276, flat 200 s) tests nothing. + 🟡 **But ready room 39/276 vs flight 41/300 suggests deployment is a single step + at TAKE-OFF**, not a schedule unfolding in-mission (different runs, so + suggestive only). **Next: one ~400 s run spanning ready room → take-off → + flight** to catch the 39→41 step in one continuous series. 🔴 `SYLPH_HZ=3` refuted as a lever: it gave the LOWEST frame rate (8/s) with the most kills, so pilot polling is not the throttle. * ~~🚧 BLOCKER: t=210/240 unreachable in one turn~~ — **superseded, see above**; diff --git a/docs/re/mission-arrival-watch.md b/docs/re/mission-arrival-watch.md index 699111ad..7442cebb 100644 --- a/docs/re/mission-arrival-watch.md +++ b/docs/re/mission-arrival-watch.md @@ -560,3 +560,55 @@ under any seconds reading it is flat there and climbs later. That requires the probe's ~25 s of enumeration and calibration to happen *after* a first cheap sample, which is a reordering rather than new decoding. + +--- + +# Sampling from the first moment (2026-08-24) + +## ✅ Setup cost cut from ~25 s to ~0.5 s + +`early_probe.py` defers everything expensive: no witness calibration (two 32 MB +reads), no per-record labelling, and the heap scan uses `bytes.find`. Setup +completes in **0.5–0.8 s**, so the first sample lands essentially at flight +detection instead of 25 s after it. + +## 🔴 Still flat: deployed is 41 at flight+0.8 s and never climbs + +``` ++ 0.8s deployed= 41 craft=298 ++ 2.1s deployed= 41 craft=298 + … every sample to +252s … ++252.0s deployed= 40 craft=282 +``` + +The only change in 252 s is 41 → 40, one squadron wiped out. **No climb at any +point**, so no arrival is observable from flight detection onward — which under +the frames reading is expected, since the whole phase-1 schedule would be over +within 8 s of mission start. + +## 🔴 A flaw in the `--wait` variant: it caught the ready room + +To get ahead of flight detection, a second run started the probe *before* the +launch and waited for the roster to appear. It did appear — 116 records — but the +numbers were **`deployed = 39`, `craft = 276`**, flat for the whole 200 s window. + +That is not the mission. **The roster is built before take-off**, so waiting for +it catches the READY ROOM, and the probe's window expired around the time flight +actually began. Waiting for the roster is not the same as catching mission start, +and the test as designed does not do what it claims. + +## 🟡 But the two runs together suggest deployment happens at take-off + +| state | deployed | craft | +|---|---|---| +| ready room | 39 | 276 | +| in flight (from t+0.8 s onward) | 41 | 300 | + +Two more records and 24 more craft appear between the ready room and flight. +These are **different runs**, so this is suggestive rather than measured — but it +points at deployment being a single step at take-off rather than a schedule +unfolding during the mission. + +**Next:** one run with the probe waiting for the roster and sampling for ~400 s, +long enough to span ready room → take-off → flight in a single continuous +series. That would show the 39 → 41 step directly, or refute it. diff --git a/tools/re-capture/early2_session.sh b/tools/re-capture/early2_session.sh new file mode 100755 index 00000000..6517b0b8 --- /dev/null +++ b/tools/re-capture/early2_session.sh @@ -0,0 +1,12 @@ +#!/usr/bin/env bash +# Probe FIRST, launch second: the probe waits for a roster to appear so it sees +# the mission from its first moment, not from when screen_id notices flight. +set -u +export HOME=/sylph-home/re SDL_AUDIODRIVER=dummy DISPLAY=:98 +export PYTHONPATH=/sylph-home/.local/lib/python3.12/site-packages +SD="$(cd "$(dirname "$0")" && pwd)" +nohup python3 "$SD/early_probe.py" --wait "${1:-45}" "${2:-200}" /tmp/early2-probe.log 2>&1 & +PROBE=$! +"$SD/launch_mission.sh" fly || { echo "BOOT FAILED"; kill $PROBE 2>/dev/null; exit 1; } +wait $PROBE +echo "EARLY2 DONE" diff --git a/tools/re-capture/early_probe.py b/tools/re-capture/early_probe.py new file mode 100755 index 00000000..eddabc9b --- /dev/null +++ b/tools/re-capture/early_probe.py @@ -0,0 +1,107 @@ +#!/usr/bin/env python3 +"""Does `deployed` climb in the first seconds of flight? + +If the route table's times are FRAMES at 30 Hz, Stage 02's phase-1 entries at +t = 90/120/170/210/240 are 3-8 seconds, so every arrival happens before the +normal probe's first sample (~25 s in, after enumeration and calibration). This +samples from the very first moment instead, with the expensive setup deferred: + + * no witness calibration up front -- it costs two 32 MB reads; + * no per-record labelling -- names are not needed to count deployed records; + * `deployed` = distinct roster records referenced by a live craft, which needs + one heap scan (bytes.find, ~seconds) plus ~300 small preads. + +Frames reading -> deployed climbs during the first ~10 s, then is flat. +Seconds reading -> deployed is flat early and climbs at 164 s+ of wall-clock. +""" +import os, sys, time, struct, collections +sys.path.insert(0, __file__.rsplit('/', 1)[0]) +import gmem, gworld, entities2 + +ROSTER_VT = struct.pack('>I', 0x820AF030) +DELTA, WIN, LINK, HULL = 0x130, 0x400, 0x08, 0x154 + +def scan_vt(fd, size, vt): + out = [] + for a, b in gmem.extents(fd, size): + pos = a + while pos < b: + m = min(1 << 24, b - pos) + blob = os.pread(fd, m, pos) + i = blob.find(vt) + while i != -1: + if (pos + i) % 4 == 0: out.append(pos + i) + i = blob.find(vt, i + 1) + pos += m + return sorted(out) + +def deployed(fd, defs, want): + lo, hi = gmem.va_to_off(entities2.ENT_VA_LO), gmem.va_to_off(entities2.ENT_VA_HI) + per = collections.Counter(); tot = 0; pos = lo + while pos < hi: + m = min(1 << 24, hi - pos) + blob = os.pread(fd, m, pos) + for needle in defs: + i = blob.find(needle) + while i != -1: + if (pos + i) % 4 == 0: + base = pos + i - DELTA + tot += 1 + try: + alive = struct.unpack('>f', os.pread(fd, 4, base + HULL))[0] > 0 + except Exception: + alive = False + if alive: + head = os.pread(fd, WIN, base) + for j in range(0, len(head) - 3, 4): + (p,) = struct.unpack_from('>I', head, j) + if p in want: per[want[p]] += 1; break + i = blob.find(needle, i + 1) + pos += m + return len(per), tot, per + +def main(): + args = [a for a in sys.argv[1:] if not a.startswith('--')] + fast_for = int(args[0]) if len(args) > 0 else 60 + total = int(args[1]) if len(args) > 1 else 240 + wait = '--wait' in sys.argv + w = gworld.World(); fd = w.fd + if wait: + # Flight DETECTION lags the mission start, so even a sample at + # flight+0.8 s may be several seconds of game time late -- and under the + # frames reading the entire phase-1 schedule is 8 s long. Start before + # the mission exists and poll for the roster to appear instead. + print('waiting for a mission to load...', flush=True) + while True: + r = scan_vt(fd, w.size, ROSTER_VT) + if len(r) >= 30: + print('roster appeared: %d records' % len(r), flush=True); break + time.sleep(0.5) + t0 = time.time() + defs = entities2.definitions(w) + roster = scan_vt(fd, w.size, ROSTER_VT) + want = {} + for o in roster: + va = gmem.primary_va(o) + if va is not None: want[va + LINK] = o + print('setup done at +%.1fs: %d roster records, %d definitions' + % (time.time() - t0, len(roster), len(defs)), flush=True) + if not roster or not defs: + print('NOT IN A MISSION'); return 2 + series = [] + while time.time() - t0 < total: + el = time.time() - t0 + d, tot, per = deployed(fd, defs, want) + series.append((round(el, 1), d, tot)) + print(' +%6.1fs deployed=%3d craft=%3d strengths %s' + % (el, d, tot, sorted(collections.Counter(per.values()).items())), flush=True) + time.sleep(1 if time.time() - t0 < fast_for else 15) + d0 = series[0][1]; dmax = max(s[1] for s in series) + print('\nfirst deployed=%d max deployed=%d climb=%d' % (d0, dmax, dmax - d0)) + print('verdict: %s' % ('DEPLOYED CLIMBED -- consistent with arrivals happening' + if dmax > d0 else + 'flat -- no arrival observed in this window')) + return 0 + +if __name__ == '__main__': + sys.exit(main()) diff --git a/tools/re-capture/early_session.sh b/tools/re-capture/early_session.sh new file mode 100755 index 00000000..80bfdfd5 --- /dev/null +++ b/tools/re-capture/early_session.sh @@ -0,0 +1,10 @@ +#!/usr/bin/env bash +set -u +export HOME=/sylph-home/re SDL_AUDIODRIVER=dummy DISPLAY=:98 +export PYTHONPATH=/sylph-home/.local/lib/python3.12/site-packages +SD="$(cd "$(dirname "$0")" && pwd)" +"$SD/launch_mission.sh" fly || { echo "BOOT FAILED"; exit 1; } +# No pilot: it costs ~15 s to bind and the question is what happens in the first +# seconds. An unattended craft is fine here -- arrivals are not player-driven +# under the reading being tested. +python3 "$SD/early_probe.py" "${1:-60}" "${2:-240}"; echo "EARLY DONE rc=$?"