Syplheed-Reborn/docs/re/autopilot-memory-driven.md

Memory-driven autopilot — build log and current state

Status: 🟢 IT FLIES AND SHOOTS — it does not yet survive. Updated 2026-07-29 (second pass). The autopilot reads the live world, picks hostile targets, pursues them and opens fire. What it cannot do is stay alive: there is no evasion or shield management, so it dies before a mission ends. This is the honest state, not a plan.

What the loop does now, observed

[ 82.5] tgt=e007_ADAN_Turret  d=3384  yaw= -7.3 pit=+14.6 stick=(-0.13,-0.34) fire=0
[ 85.7] tgt=e007_ADAN_Turret  d=2797  yaw= +1.9 pit=+27.2 stick=(+0.20,-0.59) fire=0
[117.3] tgt=e007_ADAN_Turret  d=4211  yaw= +5.7 pit= +9.0 stick=(+0.25,-0.40) fire=1

Distance closes monotonically, yaw error is driven from −8° to ~0, and once both errors are inside the firing cone it holds RB and the ammo counter falls. A rescan reports the scene as e.g. 136 entities {'TCAF': 16, 'ADAN': 120}.

The chain that made it work — each link checked, not assumed:

1. Entity typing. A live entity's definition pointer sits at position + 0x130. One heap scan then yields every craft with its unit type, which is what separates 20-odd real combatants from ~30 000 moving particles. Verified by the result being coherent: wingmen, enemy turrets and attackers, friendly capital ships, and exactly one …_Player.
2. Orientation. A 3×3 rotation at position − 0x70, stored with a 16-byte row stride (a 4×4 transform whose translation row is the position). An earlier search for nine contiguous floats structurally could not find this, which is why the first pass concluded "no transform". The binding is confirmed independently: its row 2 matches the craft's measured direction of travel with cos = +1.000.
3. The fire button is RB — established by consequence, not by guessing: of RB/LB/A/B/X/Y/RT/LT, pressing RB is the only one that makes the nose-ammo counter in RAM fall (5958 → 5940). RT is not the throttle, and no button tested is.
4. Control. PD on the aiming error with the derivative taken from the craft's own body angular velocity (from two consecutive rotation matrices), and target selection weighted by off-boresight angle (score = d·(1 + 3·(θ/π)²)) rather than pure nearest — closing on a target 90° off the nose only raises the bearing rate, which is what held the first run outside its firing cone at a steady ~27° pitch error.

Goal

Fly and fight a mission by reading the game's own world state out of guest RAM and driving the pad from it — the RE payoff being an oracle for the reimplementation's flight model and AI, and a way to reach missions the save cannot otherwise reach (unit coverage for unit-struct-runtime is capped at 21/110 because definitions load per stage).

What works (verified)

Piece	Tool	Evidence
Live guest-RAM reads at loop rate	gworld.py	/dev/shm file opened once, pread per tick; a whole-RAM scan is ~6 s, a targeted read is microseconds
Whole-RAM float scanning	numpy over SEEK_DATA extents	1 270 orthonormal 3×3 blocks located in 6.2 s
Entity enumeration by unit type	gworld.py entities	116 live instances in Stage 02, typed by unit ID, incl. exactly one …_Player
Pad control at loop rate	flight_probe.Pad	writes command lines straight into the vgamepad FIFO; the vgamepad CLI spawns a process per command and its tap/hold sleep inside the server, so neither is usable in a control loop
Unattended mission entry	launch_mission.sh	title → LOAD GAME → slot 01 → READY ROOM → TAKE OFF → in flight, repeatable
Hangar loadout	launch_mission.sh --hangar	the "Recommended" control is AUTO SELECT — "Mount most suitable weapons", already the default cursor position. At 5 % progress it is a no-op: only two weapons are developed, and they are already mounted
Finding moving objects	findplayer.py	position triples recovered from motion alone — straight-line, constant-speed filter over K whole-RAM samples

What is NOT solved

Survival, and therefore mission completion. The loop has no evasion, no shield/armour awareness and no throttle control, so it flies a straight pursuit into defended space and is eventually shot down — every long run so far has ended in GAME OVER. Completing a mission needs, at least: reading own shield/armour, breaking off when hit, and prioritising the mission's actual objective targets over the nearest turret.

Superseded (kept because the reasoning still matters)

The notes below were written before the chain above worked. They remain true as statements about 0x820af030, which is not the live entity —

1. The 0x820af030 class is not the live entity. It has one object per spawned thing and carries the unit-ID string, so it looked like the entity list — but over a 29 s in-flight capture all 384 words of it are constant (whatchanges.py). It is a static spawn record. The earlier claim in unit-struct-runtime.md that this class is "the spawned entity instance — live state" is wrong in the second half: it is per-spawn, but it is not live state. The parts of that document that depend on the definition class 0x820af844 are unaffected.
2. No transform in or one hop from that object: no orthonormal 3×3, and no unit quaternion, within 0x2000 of it or behind any of its 121 pointers.
3. Input correlation finds a self-object, but not obviously the craft. Holding hard-left then hard-right yaw and looking for a position whose turn axis reverses (findself.py, selfstate.py) gives clean hits (cos ≈ −0.99), but they cluster at 0x40009xxx in what looks like an 8-corner box with ±45 000 coordinates — a camera/skybox volume that follows the player, not the craft. Its speed (≈359/s) is suspiciously close to the HUD's 350, which supports "follows the player" but is not proof of identity.
4. Entity typing is unavailable: no definition pointer within ±0x800 of the self-position, so the trick of learning one object's layout and applying it to all the others has nothing to anchor on. Without typing, the 33 418 moving triples in a firefight cannot be separated into enemies, friendlies and bullets, so there is nothing to aim at.

Dead ends, recorded so they are not re-run

• Speed-scan for the player object (findspeed.py, the classic two-state value scan: coast → boost → coast). Sound method, but RT is not the throttle — 5 864 floats matched the cruise speed and none rose. The control actually bound to acceleration was never established, and the run that would have established it ended in GAME OVER.
• Comparing orientation matrices 2 s apart. At a real turn rate that is far outside the small-angle regime, so the skew part of A·Bᵀ is not the rotation vector and the "angular velocity" comes out as ~30 000. Sample incrementally (6 Hz) and re-check orthonormality on every read — blocks found by a scan get overwritten between the scan and the read.

The next step that unblocks the most

Find the game's own target pointer instead of typing entities ourselves. The HUD has a lock-on system (a TARGET marker and a target-cycle button), so a global almost certainly holds a pointer to the currently-targeted entity. Reading that gives an enemy's live object address directly — which yields both target selection and the entity layout (position offset within it), i.e. it collapses problems 3 and 4 into one. It is also cheap to find: cycle the target with the pad and watch which pointer-shaped global changes in step.

Operational notes

• An unattended craft dies — the ship flies straight into a firefight, and two long scans were invalidated by a GAME OVER mid-run. Any scan longer than ~30 s needs either a survivable holding pattern or a fresh mission.
• Xvfb and the emulator die on their own every few minutes here, cleanly (exit 0), cause unidentified. Everything that must not be interrupted is run as one background task that starts the display, the emulator, the navigation and the measurement together, so nothing has to survive between tool calls.
• pgrep cannot be used for liveness in this container: PID 1 is sleep infinity and never reaps, so dead processes linger as <defunct> and still match by name. Use ps -o stat= and skip Z, or xdpyinfo for X.
• numpy is not installed system-wide; pip install --break-system-packages numpy puts it in /sylph-home/.local, which is only on sys.path when HOME=/sylph-home. Scripts run with HOME=/sylph-home/re need PYTHONPATH=/sylph-home/.local/lib/python3.12/site-packages.

Files

gworld.py (live reader + entity list) · flight_probe.py (scripted inputs + sampling, and the Pad FIFO client) · flight_analyze.py · whatchanges.py (encoding-agnostic "which words are live") · findplayer.py · findself.py · findrot_global.py · findspeed.py · liveents.py · selfstate.py (the whole chain → JSON) · autopilot2.py (PD controller; untested — it has never had a valid config to run against) · launch_mission.sh.