diff --git a/docs/re/BACKLOG.md b/docs/re/BACKLOG.md index 6da08d37..45a90bf9 100644 --- a/docs/re/BACKLOG.md +++ b/docs/re/BACKLOG.md @@ -320,6 +320,15 @@ search cannot find a *schedule*. record per member (n=1, NOT promoted; the refutation needs another stage's save, which we do not have — only slot 01 / Stage 02 exists). ❔ Still unmeasured: whether the timetable's `t` is frames or seconds. +* 🔴 **Diffing inside the 116 records did not find the arrival flag** + (2026-08-24, same doc). 10 of 116 records are dynamic, 106 never change a byte + in 170 s — more support for the pre-allocated roster, though turret-heavy S02 + means "inert" ≠ "not arrived". No field transitions in groups of 3 at the + predicted times. **Blocked on two of my own defects**: the record→unit-ID + label resolved to `?` for all 116 (reuse `unit_discover.py`, do not re-derive), + and `RECLEN=0x200` was assumed, not measured — the busiest fields sit at the + very end of the window, which is what spilling into the next object looks like. + Fix both before diffing these records again. * 🔴 **Probed 2026-08-24 and refuted the cheap hypothesis** — the phase state is NOT adjacent to the loaded table strings; see [`mission-phase-runtime.md`](mission-phase-runtime.md). The run did confirm diff --git a/docs/re/mission-wave-arrivals.md b/docs/re/mission-wave-arrivals.md index c3e1954d..a5689f51 100644 --- a/docs/re/mission-wave-arrivals.md +++ b/docs/re/mission-wave-arrivals.md @@ -82,3 +82,71 @@ Cold, the boot spent **204 s** in the title movie before the first Ⓐ — the boot exceeded a 580 s turn budget and the first attempt was killed with its output still in the pipe. Log runs to a file (`> /tmp/x.log 2>&1`) rather than piping to `tail`, or a timeout loses the evidence as well as the run. + +--- + +# Follow-up: diffing inside the 116 records (2026-08-24) + +Status: 🟡 supporting evidence for the pre-allocated-roster reading; 🔴 the +arrival flag was **not** found; 🔴 two defects in my own probe recorded. + +The count being flat meant an arrival must flip a field *inside* a record, so +this diffed all 116 records (0x200 bytes each) every 5 s for 170 s of Stage 02 +flight, under the pilot. `tools/re-capture/wave2_probe.py`. + +## 🟡 Only 10 of 116 records are dynamic + +Across the whole run, **10 distinct records** ever changed a byte. The other +**106 never changed at all** — not one word, over 170 s of live mission. + +That is a second, independent line of support for the reading in the section +above: if all 116 were live entities they would be moving, and position words +would churn in most of them every tick. Instead the overwhelming majority are +inert, which is what a pre-allocated roster of not-yet-activated members looks +like. It is still not proof — Stage 02's roster is heavy with +`UN_e007_ADAN_Turret` entries, and a turret does not move even when it is very +much alive, so "inert" and "not yet arrived" are not distinguishable from this +run alone. + +## 🔴 No transition matching the arrival timetable + +The prediction under test was groups of 3, 3, 3, 2, 1 records changing state at +t = 90, 120, 170, 210, 240 s. Nothing of the sort appeared. Field-change events +are spread evenly across ticks (1–7 words per 5 s tick, 1052 in total), with no +cluster at any predicted time and no field that transitions once for exactly +three records. + +So one of these is true, and this run cannot say which: + +* the timetable's `t` is **not** seconds (at 30 Hz the whole phase-1 schedule + finishes inside the first 8 s, before the probe's first sample); +* arrival is not marked in the `0x820af030` record at all; +* phase 1 was not where the mission actually was during the run. + +## 🔴 Two defects in this probe + +* **`label()` failed for all 116 records** — every one resolved to `?`. The + `object+0x04 → name_record+0x10 → char*` chain from + [structures/unit-struct-runtime.md](structures/unit-struct-runtime.md) did not + resolve here. Without labels the records cannot be tied back to squadrons, + which is exactly what would have made the result decisive: "these three + records that changed at t=90 are `ADN110`, `ADN111`, `ADN112`" is evidence, + "records 18, 32, 99" is not. `unit_discover.py` already solves this + association and should be reused rather than re-derived. +* **`RECLEN = 0x200` is probably too wide.** The busiest fields by far are + `+0x1e0 … +0x1fc` — the last eight words of the window — which is what + spilling into the *next* object looks like. The record's true size was assumed, + not measured. + +* **A method error worth keeping:** the first attempt deferred all analysis to + the end of the run, and the turn's timeout killed it with 240 s of collected + data still in memory and nothing written. The probe now streams every + transition to `/tmp/wave2-trans.tsv` and prints a partial ranking every 60 s. + Combined with the 219 s cold-boot title movie, a run that only reports at the + end has perhaps 300 s of usable budget and one bad estimate loses everything. + +## Next + +Fix the record→squadron association first (reuse `unit_discover.py`), and +measure the record stride instead of assuming `0x200`. Until a record can be +named, no diff of these records can confirm or refute an arrival. diff --git a/tools/re-capture/wave2_probe.py b/tools/re-capture/wave2_probe.py new file mode 100755 index 00000000..f792d4db --- /dev/null +++ b/tools/re-capture/wave2_probe.py @@ -0,0 +1,111 @@ +#!/usr/bin/env python3 +"""Diff the 116 spawned-entity records over a mission to find the arrival flag. + +wave2 follows mission-wave-arrivals.md: the record COUNT is flat at 116 (one per +UnitGroup roster member), so an arrival must flip a field *inside* a record +rather than create one. Route_S02 predicts phase-1 arrivals at t = 90, 120, 170, +210, 240 in groups of 3, 3, 3, 2, 1 -- in units that are still unmeasured. A +per-record word that transitions at ~90 s supports seconds; everything landing +inside the first ~8 s supports frames at 30 Hz. +""" +import os, sys, time, struct, collections, importlib.util + +SD = __file__.rsplit('/', 1)[0] +spec = importlib.util.spec_from_file_location('gmem', SD + '/gmem.py') +gmem = importlib.util.module_from_spec(spec); spec.loader.exec_module(gmem) + +VT = struct.pack('>I', 0x820AF030) +RECLEN = 0x200 + +def find_records(f, fd, size): + offs = [] + for start, end in gmem.extents(fd, size): + pos = start + while pos < end: + f.seek(pos); buf = f.read(min(1 << 24, end - pos)) + if not buf: break + i = buf.find(VT) + while i != -1: + if (pos + i) % 4 == 0: offs.append(pos + i) + i = buf.find(VT, i + 1) + pos += len(buf) + return offs + +def read_at(f, off, n): + f.seek(off); return f.read(n) + +def label(f, size, off): + """unit id string: object+0x04 -> name record, +0x10 -> char*""" + try: + nr = struct.unpack('>I', read_at(f, off + 4, 4))[0] + o = gmem.va_to_off(nr) + if o is None or o + 0x14 > size: return '?' + sp = struct.unpack('>I', read_at(f, o + 0x10, 4))[0] + so = gmem.va_to_off(sp) + if so is None: return '?' + b = read_at(f, so, 48) + return b.split(b'\x00')[0].decode('latin-1') or '?' + except Exception: + return '?' + +def main(): + secs = int(sys.argv[1]) if len(sys.argv) > 1 else 260 + every = int(sys.argv[2]) if len(sys.argv) > 2 else 5 + path = gmem.mem_path() + fd = os.open(path, os.O_RDONLY); size = os.fstat(fd).st_size + f = os.fdopen(os.dup(fd), 'rb') + offs = find_records(f, fd, size) + if not offs: + print('NO ENTITY RECORDS -- not in a mission'); return 2 + ids = [label(f, size, o) for o in offs] + print('%d records; distinct unit ids: %d' % (len(offs), len(set(ids)))) + print(' ', collections.Counter(ids).most_common(8)) + prev = [read_at(f, o, RECLEN) for o in offs] + t0 = time.time() + # Stream every transition to disk. The first version of this probe deferred + # ALL analysis to the end and a timeout destroyed the whole run's evidence. + jl = open('/tmp/wave2-trans.tsv', 'w') + jl.write('t\trec\tfield\told\tnew\n') + # transitions[(field_off)] -> list of (t, rec_index, old, new) + trans = collections.defaultdict(list) + while time.time() - t0 < secs: + time.sleep(every) + el = round(time.time() - t0) + nch = 0 + for k, o in enumerate(offs): + cur = read_at(f, o, RECLEN) + if cur == prev[k]: continue + for w in range(0, RECLEN, 4): + a = prev[k][w:w+4]; b = cur[w:w+4] + if a != b: + ov, nv = struct.unpack('>I', a)[0], struct.unpack('>I', b)[0] + trans[w].append((el, k, ov, nv)) + jl.write('%d\t%d\t0x%03x\t%08x\t%08x\n' % (el, k, w, ov, nv)) + nch += 1 + prev[k] = cur + print(' t=%4ds changed words this tick: %d' % (el, nch), flush=True) + jl.flush() + if el % 60 < every: # partial ranking, timeout-proof + r = sorted(trans.items(), key=lambda kv: -len({x[1] for x in kv[1]})) + print(' partial: ' + '; '.join( + '+0x%03x:%drec' % (w, len({x[1] for x in ev})) for w, ev in r[:6]), + flush=True) + jl.close() + print('\n--- fields by how many records ever changed them ---') + rank = sorted(trans.items(), key=lambda kv: -len({x[1] for x in kv[1]})) + for w, ev in rank[:18]: + recs = {x[1] for x in ev} + times = collections.Counter(x[0] for x in ev) + print(' +0x%03x %3d records, %3d events; busiest ticks %s' + % (w, len(recs), len(ev), times.most_common(5))) + print('\n--- fields that changed for only a FEW records (arrival-like) ---') + for w, ev in rank: + recs = {x[1] for x in ev} + if not (1 <= len(recs) <= 12): continue + print(' +0x%03x records=%s' % (w, sorted(recs)[:12])) + for t, k, a, b in ev[:10]: + print(' t=%4ds rec %3d (%s) %08x -> %08x' % (t, k, ids[k], a, b)) + return 0 + +if __name__ == '__main__': + sys.exit(main()) diff --git a/tools/re-capture/wave2_session.sh b/tools/re-capture/wave2_session.sh new file mode 100755 index 00000000..c61f0d98 --- /dev/null +++ b/tools/re-capture/wave2_session.sh @@ -0,0 +1,15 @@ +#!/usr/bin/env bash +set -u +export HOME=/sylph-home/re SDL_AUDIODRIVER=dummy DISPLAY=:98 +export PYTHONPATH=/sylph-home/.local/lib/python3.12/site-packages +SD="$(cd "$(dirname "$0")" && pwd)" +SECS="${1:-300}"; EVERY="${2:-10}" +CFG=/tmp/nav-wave.json +"$SD/launch_mission.sh" fly || { echo "BOOT FAILED"; exit 1; } +if python3 "$SD/entities2.py" self 0x130 "$CFG" >/dev/null 2>&1; then + nohup python3 "$SD/pilot.py" "$CFG" "$SECS" /tmp/wave2-pilot.log 2>&1 & + PILOT=$!; echo "--- pilot flying" +else PILOT=""; echo "--- BIND FAILED, unattended craft"; fi +python3 "$SD/wave2_probe.py" "$SECS" "$EVERY"; rc=$? +[ -n "$PILOT" ] && kill "$PILOT" 2>/dev/null +echo "WAVE SESSION DONE rc=$rc"