Syplheed-Reborn/docs/re/autopilot-memory-driven.md

Memory-driven autopilot — build log and current state

Status: 🟡 PARTIAL — infrastructure works, the craft is not yet flown. Started 2026-07-29. This is the honest state, not a plan: what is proven, what is not, and the one thing that most cheaply unblocks the rest.

Goal

Fly and fight a mission by reading the game's own world state out of guest RAM and driving the pad from it — the RE payoff being an oracle for the reimplementation's flight model and AI, and a way to reach missions the save cannot otherwise reach (unit coverage for unit-struct-runtime is capped at 21/110 because definitions load per stage).

What works (verified)

Piece	Tool	Evidence
Live guest-RAM reads at loop rate	gworld.py	/dev/shm file opened once, pread per tick; a whole-RAM scan is ~6 s, a targeted read is microseconds
Whole-RAM float scanning	numpy over SEEK_DATA extents	1 270 orthonormal 3×3 blocks located in 6.2 s
Entity enumeration by unit type	gworld.py entities	116 live instances in Stage 02, typed by unit ID, incl. exactly one …_Player
Pad control at loop rate	flight_probe.Pad	writes command lines straight into the vgamepad FIFO; the vgamepad CLI spawns a process per command and its tap/hold sleep inside the server, so neither is usable in a control loop
Unattended mission entry	launch_mission.sh	title → LOAD GAME → slot 01 → READY ROOM → TAKE OFF → in flight, repeatable
Hangar loadout	launch_mission.sh --hangar	the "Recommended" control is AUTO SELECT — "Mount most suitable weapons", already the default cursor position. At 5 % progress it is a no-op: only two weapons are developed, and they are already mounted
Finding moving objects	findplayer.py	position triples recovered from motion alone — straight-line, constant-speed filter over K whole-RAM samples

What is NOT solved

Identifying our own craft, and typing the other entities. Both remain open, and the autopilot cannot work without them.

1. The 0x820af030 class is not the live entity. It has one object per spawned thing and carries the unit-ID string, so it looked like the entity list — but over a 29 s in-flight capture all 384 words of it are constant (whatchanges.py). It is a static spawn record. The earlier claim in unit-struct-runtime.md that this class is "the spawned entity instance — live state" is wrong in the second half: it is per-spawn, but it is not live state. The parts of that document that depend on the definition class 0x820af844 are unaffected.
2. No transform in or one hop from that object: no orthonormal 3×3, and no unit quaternion, within 0x2000 of it or behind any of its 121 pointers.
3. Input correlation finds a self-object, but not obviously the craft. Holding hard-left then hard-right yaw and looking for a position whose turn axis reverses (findself.py, selfstate.py) gives clean hits (cos ≈ −0.99), but they cluster at 0x40009xxx in what looks like an 8-corner box with ±45 000 coordinates — a camera/skybox volume that follows the player, not the craft. Its speed (≈359/s) is suspiciously close to the HUD's 350, which supports "follows the player" but is not proof of identity.
4. Entity typing is unavailable: no definition pointer within ±0x800 of the self-position, so the trick of learning one object's layout and applying it to all the others has nothing to anchor on. Without typing, the 33 418 moving triples in a firefight cannot be separated into enemies, friendlies and bullets, so there is nothing to aim at.

Dead ends, recorded so they are not re-run

• Speed-scan for the player object (findspeed.py, the classic two-state value scan: coast → boost → coast). Sound method, but RT is not the throttle — 5 864 floats matched the cruise speed and none rose. The control actually bound to acceleration was never established, and the run that would have established it ended in GAME OVER.
• Comparing orientation matrices 2 s apart. At a real turn rate that is far outside the small-angle regime, so the skew part of A·Bᵀ is not the rotation vector and the "angular velocity" comes out as ~30 000. Sample incrementally (6 Hz) and re-check orthonormality on every read — blocks found by a scan get overwritten between the scan and the read.

The next step that unblocks the most

Find the game's own target pointer instead of typing entities ourselves. The HUD has a lock-on system (a TARGET marker and a target-cycle button), so a global almost certainly holds a pointer to the currently-targeted entity. Reading that gives an enemy's live object address directly — which yields both target selection and the entity layout (position offset within it), i.e. it collapses problems 3 and 4 into one. It is also cheap to find: cycle the target with the pad and watch which pointer-shaped global changes in step.

Operational notes

• An unattended craft dies — the ship flies straight into a firefight, and two long scans were invalidated by a GAME OVER mid-run. Any scan longer than ~30 s needs either a survivable holding pattern or a fresh mission.
• Xvfb and the emulator die on their own every few minutes here, cleanly (exit 0), cause unidentified. Everything that must not be interrupted is run as one background task that starts the display, the emulator, the navigation and the measurement together, so nothing has to survive between tool calls.
• pgrep cannot be used for liveness in this container: PID 1 is sleep infinity and never reaps, so dead processes linger as <defunct> and still match by name. Use ps -o stat= and skip Z, or xdpyinfo for X.
• numpy is not installed system-wide; pip install --break-system-packages numpy puts it in /sylph-home/.local, which is only on sys.path when HOME=/sylph-home. Scripts run with HOME=/sylph-home/re need PYTHONPATH=/sylph-home/.local/lib/python3.12/site-packages.

Files

gworld.py (live reader + entity list) · flight_probe.py (scripted inputs + sampling, and the Pad FIFO client) · flight_analyze.py · whatchanges.py (encoding-agnostic "which words are live") · findplayer.py · findself.py · findrot_global.py · findspeed.py · liveents.py · selfstate.py (the whole chain → JSON) · autopilot2.py (PD controller; untested — it has never had a valid config to run against) · launch_mission.sh.