diff --git a/docs/re/BACKLOG.md b/docs/re/BACKLOG.md index cc42d00..39b1598 100644 --- a/docs/re/BACKLOG.md +++ b/docs/re/BACKLOG.md @@ -326,10 +326,14 @@ search cannot find a *schedule*. (monotone 298→280). The hunting pilot **does** kill (hull crossing at t=57s). 🔴 "No births in 164 s" does NOT separate the wave models — everything is pre-allocated, so an arrival must be a STATE CHANGE, not an allocation. - 🟡 Turret `sum(n)`=216 vs 214 sites found (vs `Count`=21) suggests the mystery - member field `n` is the craft count per member — NOT promoted: the probe counts - definition-pointer *sites*, and the player (1 member → 2 sites) proves some - entities hold several. **Next: collapse sites into entities, then re-test.** + 🔴 **`n` = craft-per-member is WITHDRAWN** (re-tested 2026-08-24): sites really + are entities 1:1 (all same-unit gaps ≥0x1000, hull plausible 298/298), so the + confound was not the explanation — and with it gone, `DeltaSaber_T`, `Player` + and `Acropolis` all come out at exactly **2×** `sum(n)`. An undershoot is + explainable by phases; an overshoot is not. `n` back to ❔. Formation slot count + also rejected (630 turret slots vs 214 live). ✅ Side result: `FormationSet` + `FrameCount` = slot count. ❔ **Open: the 116 roster records and the 298 live + entities are different structures; the expansion rule between them is unknown.** * 🟡 **COMPETING MODEL (user, 2026-08-24): waves are event-gated, not clock-driven** — released by kills/objectives rather than at a fixed time, with the route's start time being a post-trigger delay. Fits the evidence better: diff --git a/docs/re/mission-liveness-probe.md b/docs/re/mission-liveness-probe.md index 252181c..1130951 100644 --- a/docs/re/mission-liveness-probe.md +++ b/docs/re/mission-liveness-probe.md @@ -84,3 +84,73 @@ implied entity base and take distinct bases. Then the table above becomes a real test rather than a suggestive one. The same de-duplication is a precondition for using this probe to watch an arrival, since an arrival is now known to be a state change rather than an allocation. + +--- + +# Sites really are entities — and `n` does not survive the fix (2026-08-24) + +## ✅ The multiplicity confound is refuted + +The worry was that some entity types hold several pointers to their definition, +so "sites" over-counted entities. Measured, it does not happen: + +* **Gaps between consecutive same-unit sites are all ≥ `0x1000`**, and 274 of + them are *exactly* `0x1000`. Entities are page-spaced; there are no pairs + sitting a few bytes apart that a clustering pass could merge. +* Clustering at any threshold below `0x1000` therefore gives **ratio 1.00 for + every unit type** — 298 sites, 298 entities. +* **Hull is plausible (`0 < h ≤ 200000`) on 298 of 298** clustered bases at + delta `0x130`. These are real objects, not stray pointer words. + +So the previous run's counts were right after all, and the reason the player +shows two is that there genuinely are two such objects — not two pointers to one. + +*(The delta spectrum came back empty this run: `entities2.moving()` found no +movers in its sample window, so `find_delta` had nothing to vote on. It does not +affect the conclusion — every gap is ≥ `0x1000`, so any threshold below that +gives the same clustering — but the threshold was a fallback, not a measurement, +and that is worth knowing if this probe is reused.)* + +## 🔴 With the confound gone, the `n` = craft-per-member reading fails + +| unit | members | `sum(n)` | formation slots | **live entities** | +|---|---|---|---|---| +| `UN_e007_ADAN_Turret` | 21 | 216 | 630 | **214** | +| `UN_e010_ADAN_Attacker_S` | 9 | 45 | 126 | **32** | +| `UN_f001_TCAF_DeltaSaber_T` | 7 | 7 | 23 | **14** | +| `UN_e106_ADAN_Destroyer` | 19 | 19 | 23 | **14** | +| `UN_f106_TCAF_Destroyer` | 14 | 16 | 16 | **10** | +| `UN_e105_ADAN_Cruiser` | 7 | 7 | 7 | **6** | +| `UN_f105_TCAF_Cruiser` | 11 | 11 | 11 | **4** | +| `UN_f001_..._Player` | 1 | 1 | 3 | **2** | +| `UN_f101_TCAF_Acropolis` | 1 | 1 | 6 | **2** | + +`sum(n)` fits the turret row well (216 against 214, with kills already recorded +before the sample). It does **not** generalise: `DeltaSaber_T`, `Player` and +`Acropolis` all come out at exactly **twice** their `sum(n)`, and an undershoot +can be blamed on phases 2–3 not having started but an *overshoot* cannot. + +Two different multipliers are at work and only one of them is explained. **`n` +goes back to ❔** — the 🟡 in the previous section is withdrawn. What the turret +row shows is only that a roster member expands into many craft; it does not show +that `n` is the expansion factor. + +## 🔴 Formation slot count is not the expansion factor either + +Tested because the formation names carry a size suffix. `FormationSet_S02.tbl` +gives 630 turret slots against 214 live — an overshoot of 3×. Rejected. + +## ✅ Side result: `FormationSet` `FrameCount` is the slot count + +The frame count of a formation record is the number of slots, and the name +suffix usually agrees: `Formation_ADAN_Turret07_30` → 30, +`Formation_TCAF_ArrowHead03_64` → 64, `Formation_4_Bird` → 4. One exception +found: `Formation_ADAN_AttackerS03_12` has 14 slots, not 12, so the suffix is a +label rather than a guarantee. + +## Also worth recording + +The live entity population (**298**) is not the 116 `0x820af030` roster records. +Both structures exist simultaneously: 116 records, one per `UnitGroup` member, +and 298 heap objects, one per actual craft. Whatever maps one onto the other is +the expansion rule, and it is still unknown. diff --git a/tools/re-capture/dedupe_probe.py b/tools/re-capture/dedupe_probe.py new file mode 100755 index 0000000..6ab6e2d --- /dev/null +++ b/tools/re-capture/dedupe_probe.py @@ -0,0 +1,98 @@ +#!/usr/bin/env python3 +"""Collapse definition-pointer SITES into distinct entities, then re-test `n`. + +liveness_probe.py counts every aligned word equal to a unit-definition VA. Some +entity types hold more than one such pointer -- the player is one roster member +and yields two sites -- so its counts are sites, not entities, and the +`sum(n)` == entities test could not be believed (mission-liveness-probe.md). + +This measures the multiplicity instead of assuming it: + 1. the delta spectrum -- how many distinct (entity -> def pointer) offsets exist; + 2. the gap distribution between consecutive sites of the SAME unit type, which + shows whether sites cluster in pairs; + 3. a clustering threshold derived from that gap distribution, not guessed; + 4. hull sanity: a real entity's f32 at +0x154 should be a plausible HP. +""" +import os, sys, collections, struct +sys.path.insert(0, __file__.rsplit('/', 1)[0]) +import gmem, gworld, entities2 + +HULL_OFF = 0x154 + +def sites(fd, defs): + lo, hi = gmem.va_to_off(entities2.ENT_VA_LO), gmem.va_to_off(entities2.ENT_VA_HI) + out, pos = [], lo + while pos < hi: + n = min(1 << 24, hi - pos) + blob = os.pread(fd, n, pos) + for k in range(0, len(blob) - 3, 4): + nm = defs.get(blob[k:k+4]) + if nm: out.append((pos + k, nm)) + pos += n + return out + +def f32(fd, off): + try: return struct.unpack('>f', os.pread(fd, 4, off))[0] + except Exception: return None + +def main(): + w = gworld.World(); fd = w.fd + defs = entities2.definitions(w) + print('definitions: %d' % len(defs)) + if not defs: print('NOT IN A MISSION'); return 2 + S = sites(fd, defs) + print('sites: %d' % len(S)) + print('by unit:', collections.Counter(n for _, n in S).most_common(8)) + + # (1) delta spectrum, measured on movers + movers = entities2.moving(fd, w.size) + votes = entities2.find_delta(fd, defs, movers) + print('\n--- delta spectrum (entity -> def pointer offsets) ---') + for d, c in votes.most_common(8): + print(' %+#08x seen %d' % (d, c)) + + # (2) gaps between consecutive sites of the same unit + bygroup = collections.defaultdict(list) + for off, nm in S: bygroup[nm].append(off) + gaps = collections.Counter() + for nm, offs in bygroup.items(): + offs.sort() + for i in range(1, len(offs)): gaps[offs[i] - offs[i-1]] += 1 + print('\n--- gaps between same-unit sites (smallest 10) ---') + for g, c in sorted(gaps.items())[:10]: print(' %#08x (%6d) x%d' % (g, g, c)) + + # (3) cluster with a threshold taken from the spectrum, not guessed + span = max(votes.most_common(4), key=lambda kv: kv[0])[0] - min( + d for d, _ in votes.most_common(4)) if len(votes) > 1 else 0 + thresh = max(0x40, abs(span) + 4) + print('\nclustering threshold from delta spread: %#x' % thresh) + ents = collections.Counter() + for nm, offs in bygroup.items(): + offs.sort(); last = None + for o in offs: + if last is None or o - last > thresh: ents[nm] += 1 + last = o + print('\n--- sites vs clustered entities ---') + sc = collections.Counter(n for _, n in S) + print('%-34s %7s %9s %6s' % ('unit', 'sites', 'entities', 'ratio')) + for nm, c in sc.most_common(): + e = ents[nm] + print('%-34s %7d %9d %6.2f' % (nm, c, e, c / e if e else 0)) + print('\ntotal sites=%d entities=%d' % (len(S), sum(ents.values()))) + + # (4) hull sanity on clustered bases for the top delta + d0 = votes.most_common(1)[0][0] if votes else 0x130 + ok = bad = 0 + for nm, offs in bygroup.items(): + offs.sort(); last = None + for o in offs: + if last is None or o - last > thresh: + h = f32(fd, o - d0 + HULL_OFF) + if h is not None and 0 < h <= 200000: ok += 1 + else: bad += 1 + last = o + print('hull plausible on %d clustered bases, implausible on %d (delta %#x)' % (ok, bad, d0)) + return 0 + +if __name__ == '__main__': + sys.exit(main()) diff --git a/tools/re-capture/dedupe_session.sh b/tools/re-capture/dedupe_session.sh new file mode 100755 index 0000000..21d54be --- /dev/null +++ b/tools/re-capture/dedupe_session.sh @@ -0,0 +1,16 @@ +#!/usr/bin/env bash +set -u +export HOME=/sylph-home/re SDL_AUDIODRIVER=dummy DISPLAY=:98 +export PYTHONPATH=/sylph-home/.local/lib/python3.12/site-packages +SD="$(cd "$(dirname "$0")" && pwd)" +SECS="${1:-180}"; EVERY="${2:-10}"; HUNT="${3:-1}" +CFG=/tmp/nav-live.json +"$SD/launch_mission.sh" fly || { echo "BOOT FAILED"; exit 1; } +if python3 "$SD/entities2.py" self 0x130 "$CFG" >/dev/null 2>&1; then + SYLPH_HUNT="$HUNT" SYLPH_KILL_TURRETS=1 nohup python3 "$SD/pilot.py" "$CFG" "$SECS" \ + /tmp/live-pilot.log 2>&1 & + PILOT=$!; echo "--- pilot (SYLPH_HUNT=$HUNT)" +else PILOT=""; echo "--- BIND FAILED, no pilot"; fi +python3 "$SD/dedupe_probe.py" ; rc=$? +[ -n "$PILOT" ] && kill "$PILOT" 2>/dev/null +echo "LIVENESS DONE rc=$rc"