Syplheed-Reborn/docs/re/autopilot-memory-driven.md

# Memory-driven autopilot — build log and current state

**Status: 🟢 IT FLIES, KILLS AND SURVIVES — but it loses the mission anyway.**
Updated 2026-07-30. `pilot.py` flew Stage 02 for **300 s with the hull untouched
at 1500/1500** and scored the first confirmed autopilot kill (`YOU KILLED
WARPLANES 0001` on the HUD, screenshots `shots/pilot1-*.png`); the scene's
hostile count fell from 134 to 111 over the run. The day before, every run was
dead inside 35 s. What is still missing is the *end* of a mission: the objective
counter (`REMAINING OB`) rises as new waves spawn, and nothing yet tracks which
targets actually close it out — and the second run proved the point the hard
way: `GAME OVER` with the hull at 1500/1500, because Stage 02 is an **escort**
and the ACROPOLIS was sunk while the pilot chased fighters two kilometres away.

## 2026-07-30 — the numbers survival needs

Three things the loop was missing were measured this session, each by
consequence rather than by reading a field and hoping.

### Hull is `position + 0x154` ✅ CONFIRMED

The unit definition already had `HP` solved at `+0x054`
([unit-struct-runtime](structures/unit-struct-runtime.md)); the Delta Saber's is
**1500**. A craft that has taken no damage must therefore *contain that number*,
which turns "find the HP field" into a two-float lookup rather than a value scan
(`own_state.py`). It appears once in the entity object, at `pos+0x154`, and the
trace across a death settles it (`ctrl_probe.py` capture, `binq.py trace`):

```
  t      phase        hull
  0.00   base         1500.00      <- == definition HP
 23.25   rest_A       1380.00      <- first hit, -120
 26.68 … 27.18  B     1320 … 930   <- seven hits in 0.5 s
 31.93   X            150.00
 35.21   Y            -30.00       <- goes negative
 35.26   Y           -180.00       -> GAME OVER on screen
```

Damage arrives in 30/60/90-point steps and the field goes *negative* at death,
so it is the raw hull counter, not a clamped display value. **1500 hull lost in
12 s** of sitting in a turret's line of fire is the whole reason every earlier
run died.

### Shield is `position + 0x430` 🟡 PROBABLE — not yet confirmed live

Same anchor trick: the definition's shield `MaxValue` is **400** and
`ChargeSpeed` **25**, and the entity object holds `400.0` at `+0x430`, `+0x434`
and `+0x438`, with `25.0` at `+0x448`. Which of the three is the *current* value
is unproven — the capture that spanned the death used a ±0x400 window and
cropped them out. `ctrl_probe.py` now samples ±0x800.

### `RT` accelerates, `LT` brakes, and the throttle is a *setting* ✅ CONFIRMED

`ctrl_probe.py` holds each input in turn and measures the craft's own speed as
displacement per second from the position triple, so no speed field is needed.
Distance flown / phase duration, one 3 s hold each, sticks neutral:

| phase | speed (units/s) | | phase | speed (units/s) |
|---|---|---|---|---|
| base (no input) | 488 | | A | 287 |
| **RT** | **1510** | | B | 139 |
| rest after RT | 1056 | | X | 125 |
| **LT** | **174** | | Y | (dying) |
| rest after LT | 428 | | LB / LS / RS / RY / RX / dpad | no effect |

RT triples the speed, LT cuts it to a third, and **the braked state persists**:
after the LT phase the craft sat at 125–140 units/s with the sticks and triggers
neutral for the remaining 40 s, and nothing but RT brought it back. So these are
a throttle setting, not a momentary boost — which also means a control loop must
send only the *changes*.

This **corrects** the earlier note in this file ("`RT` is *not* the throttle,
and no button tested is"). That conclusion came from `findspeed.py`, which
assumed the control and went looking for a *field* that rose; measuring the
speed directly reverses it.

### Two method corrections

* **Pick the attitude block by the flight path, not by address order.** The
  player object contains **20** orthonormal 3×3 blocks (identity frames, bone
  or camera frames), and `pos-0x70` and `pos-0x30` hold the *same* matrix.
  Taking `found[0]` wrote a config with `rot_delta = -0x764` and a nonsense
  forward axis; `entities2.py self` now scores every block against the measured
  direction of travel and picks the best (`cos = +1.000`, row 2, sign +1).
* **The entity-heap scan has to be numpy.** A per-word Python loop over the
  16 MB entity region costs seconds per scan, which is the whole budget of a
  10 Hz control loop; `np.isin` over a `>u4` view is milliseconds.

### Stage 02 as an autopilot testbed (from the in-flight HUD)

`OBJECTIVE: shoot down all invading enemy fighters while watching out for
attacks on the ACROPOLIS` · `DEFEAT: your fighter is shot down, or the ACROPOLIS
is sunk` · `HINT: you can resupply at the ACROPOLIS`. The HUD shows
**`REMAINING OB 004`** — only four objective targets — so this mission is
winnable by an autopilot that survives. It also shows separate **SHIELD** and
**ARMOR** bars (matching a 400-point shield over 1500 hull), `A/B 7,635`
afterburner, and `NOSE BM 06000` / `MAIN MPM 00300` ammo.

## What the loop did before that, observed

```
[ 82.5] tgt=e007_ADAN_Turret  d=3384  yaw= -7.3 pit=+14.6 stick=(-0.13,-0.34) fire=0
[ 85.7] tgt=e007_ADAN_Turret  d=2797  yaw= +1.9 pit=+27.2 stick=(+0.20,-0.59) fire=0
[117.3] tgt=e007_ADAN_Turret  d=4211  yaw= +5.7 pit= +9.0 stick=(+0.25,-0.40) fire=1
```

Distance closes monotonically, yaw error is driven from −8° to ~0, and once both
errors are inside the firing cone it holds RB and the ammo counter falls. A
rescan reports the scene as e.g. `136 entities {'TCAF': 16, 'ADAN': 120}`.

**The chain that made it work** — each link checked, not assumed:

1. **Entity typing.** A live entity's definition pointer sits at
   **position + 0x130**. One heap scan then yields every craft *with its unit
   type*, which is what separates 20-odd real combatants from ~30 000 moving
   particles. Verified by the result being coherent: wingmen, enemy turrets and
   attackers, friendly capital ships, and exactly one `…_Player`.
2. **Orientation.** A 3×3 rotation at **position − 0x70**, stored with a
   **16-byte row stride** (a 4×4 transform whose translation row *is* the
   position). An earlier search for nine *contiguous* floats structurally could
   not find this, which is why the first pass concluded "no transform". The
   binding is confirmed independently: its row 2 matches the craft's measured
   direction of travel with **cos = +1.000**.
3. **The fire button is RB** — established by consequence, not by guessing:
   of RB/LB/A/B/X/Y/RT/LT, pressing RB is the only one that makes the nose-ammo
   counter in RAM fall (5958 → 5940). (This entry also claimed `RT` is *not* the
   throttle — **wrong**, see the 2026-07-30 measurement above.)
4. **Control.** PD on the aiming error with the derivative taken from the
   craft's own body angular velocity (from two consecutive rotation matrices),
   and target selection weighted by off-boresight angle
   (`score = d·(1 + 3·(θ/π)²)`) rather than pure nearest — closing on a target
   90° off the nose only raises the bearing rate, which is what held the first
   run outside its firing cone at a steady ~27° pitch error.

## Goal

Fly and fight a mission by reading the game's own world state out of guest RAM
and driving the pad from it — the RE payoff being an oracle for the
reimplementation's flight model and AI, and a way to reach missions the save
cannot otherwise reach (unit coverage for
[unit-struct-runtime](structures/unit-struct-runtime.md) is capped at 21/110
because definitions load **per stage**).

## What works (verified)

| Piece | Tool | Evidence |
|---|---|---|
| Live guest-RAM reads at loop rate | `gworld.py` | `/dev/shm` file opened once, `pread` per tick; a whole-RAM scan is ~6 s, a targeted read is microseconds |
| Whole-RAM float scanning | numpy over `SEEK_DATA` extents | 1 270 orthonormal 3×3 blocks located in 6.2 s |
| Entity enumeration by unit type | `gworld.py entities` | 116 live instances in Stage 02, typed by unit ID, incl. exactly one `…_Player` |
| Pad control at loop rate | `flight_probe.Pad` | writes command lines straight into the vgamepad FIFO; the `vgamepad` CLI spawns a process per command and its `tap`/`hold` sleep *inside* the server, so neither is usable in a control loop |
| Unattended mission entry | `launch_mission.sh` | title → LOAD GAME → slot 01 → READY ROOM → TAKE OFF → in flight, repeatable |
| Hangar loadout | `launch_mission.sh --hangar` | the "Recommended" control is **AUTO SELECT — "Mount most suitable weapons"**, already the default cursor position. At 5 % progress it is a no-op: only two weapons are developed, and they are already mounted |
| Finding moving objects | `findplayer.py` | position triples recovered from motion alone — straight-line, constant-speed filter over K whole-RAM samples |

## What is NOT solved

**Survival, and therefore mission completion.** The loop has no evasion, no
shield/armour awareness and no throttle control, so it flies a straight pursuit
into defended space and is eventually shot down — every long run so far has
ended in GAME OVER. Completing a mission needs, at least: reading own
shield/armour, breaking off when hit, and prioritising the mission's actual
objective targets over the nearest turret.

### Superseded (kept because the reasoning still matters)

The notes below were written before the chain above worked. They remain true as
statements about `0x820af030`, which is *not* the live entity —

1. **The `0x820af030` class is not the live entity.** It has one object per
   spawned thing and carries the unit-ID string, so it looked like the entity
   list — but over a 29 s in-flight capture **all 384 words of it are
   constant** (`whatchanges.py`). It is a static spawn record. The earlier
   claim in `unit-struct-runtime.md` that this class is "the spawned entity
   instance — live state" is **wrong in the second half**: it is per-spawn, but
   it is not live state. The parts of that document that depend on the
   *definition* class `0x820af844` are unaffected.
2. **No transform in or one hop from that object**: no orthonormal 3×3, and no
   unit quaternion, within `0x2000` of it or behind any of its 121 pointers.
3. **Input correlation finds *a* self-object, but not obviously the craft.**
   Holding hard-left then hard-right yaw and looking for a position whose turn
   axis reverses (`findself.py`, `selfstate.py`) gives clean hits
   (`cos ≈ −0.99`), but they cluster at `0x40009xxx` in what looks like an
   8-corner box with ±45 000 coordinates — a camera/skybox volume that follows
   the player, not the craft. Its speed (≈359/s) is suspiciously close to the
   HUD's 350, which supports "follows the player" but is not proof of identity.
4. **Entity typing is unavailable**: no definition pointer within ±0x800 of the
   self-position, so the trick of learning one object's layout and applying it
   to all the others has nothing to anchor on. Without typing, the 33 418
   moving triples in a firefight cannot be separated into enemies, friendlies
   and bullets, so there is nothing to aim at.

## Dead ends, recorded so they are not re-run

* **Speed-scan for the player object** (`findspeed.py`, the classic two-state
  value scan: coast → boost → coast). Sound method, but **`RT` is not the
  throttle** — 5 864 floats matched the cruise speed and none rose. The control
  actually bound to acceleration was never established, and the run that would
  have established it ended in GAME OVER.
* **Comparing orientation matrices 2 s apart.** At a real turn rate that is far
  outside the small-angle regime, so the skew part of `A·Bᵀ` is not the rotation
  vector and the "angular velocity" comes out as ~30 000. Sample incrementally
  (6 Hz) and re-check orthonormality on every read — blocks found by a scan get
  overwritten between the scan and the read.

## The second run lost the mission **without being hit** (2026-07-30)

A second 240 s flight, with the two fixes above, ended on the `GAME OVER`
screen — while the hull read **1500/1500 on the last live tick**. Nothing shot
us down. The other defeat condition fired: *the ACROPOLIS is sunk*. The HUD had
been showing a red `WARNING` banner for a while, and the pilot spent the whole
run pursuing an `e010_ADAN_Attacker_S` two kilometres away.

So surviving is necessary and not sufficient, and "nearest hostile fighter" is
the wrong objective function for this stage. **The mission is an escort.** What
follows:

* **Prioritise hostiles by their distance to the protected asset, not to us.**
  The attackers worth killing are the ones closing on the ACROPOLIS.
* **The protected asset's health is readable with the same anchor as ours** —
  hull at `position + 0x154`, its maximum being its own definition's `HP`. That
  gives a live "are we winning" signal for the escort, and it should drive the
  target choice directly.
* A frozen tail in the log (identical position, speed and target for the last
  five seconds) is what mission-end looks like from the outside, **not** an
  emulator wedge. Worth knowing before diagnosing the wrong thing.
* Practical: do **not** pipe a long run's log through `tail` — that discards
  everything but the end, and the interesting part of this run is gone.

## After survival, the blocker is lethality (2026-07-30)

The 300 s run took **no damage at all** and killed **one** warplane, spending
~800 rounds of nose ammo (`06000` → `05193`) to do it, while `REMAINING OB` rose
from `004` to `011` as fresh waves spawned. So attrition at this rate never
finishes the mission, and the ranking of open problems has changed:

1. **Hit rate.** It opens fire at 2–5 km with a 9° cone and a crude lead
   (`p + v·d/speed`, no projectile speed). The `Shell` records in
   [weapon-struct-runtime](structures/weapon-struct-runtime.md) carry the real
   projectile speed and `MaximumRange` per weapon — the lead and the firing
   range should come from *those*, not from constants.
2. **Which targets count.** `REMAINING OB` is the mission's own objective
   counter and it is on screen, so it is in RAM; finding it turns "shoot
   whatever is nearest" into "shoot what closes the mission". Objective-marked
   entities also draw an `OB` badge in the HUD, so the flag is likely a word in
   the entity object.
3. **Confirming the shield word** — needs a run that actually takes damage; the
   pilot is now good enough at avoiding that to make it awkward, so drive
   straight at a turret on purpose with `--dry` steering disabled.
4. **Does the ACROPOLIS repair?** RETIRE mode has never triggered (the hull
   never fell), so the resupply hint is still untested.

## The next step that unblocks the most (superseded — kept for the reasoning)

**Update 2026-07-30: this is no longer the blocker.** Entity typing via the
definition pointer already solved target selection, so the game's own target
pointer is now a convenience rather than a prerequisite. It would still be the
cheapest route to problem 2 above (objective targets), because whatever the HUD
locks on to is what the game itself considers a target.

**Find the game's own target pointer instead of typing entities ourselves.**
The HUD has a lock-on system (a `TARGET` marker and a target-cycle button), so
a global almost certainly holds a pointer to the currently-targeted entity.
Reading that gives an enemy's live object address directly — which yields both
target selection *and* the entity layout (position offset within it), i.e. it
collapses problems 3 and 4 into one. It is also cheap to find: cycle the target
with the pad and watch which pointer-shaped global changes in step.

## Operational notes

* An unattended craft **dies** — the ship flies straight into a firefight, and
  two long scans were invalidated by a GAME OVER mid-run. Any scan longer than
  ~30 s needs either a survivable holding pattern or a fresh mission.
* `Xvfb` and the emulator die on their own every few minutes here, cleanly
  (exit 0), cause unidentified. Everything that must not be interrupted is run
  as **one background task** that starts the display, the emulator, the
  navigation and the measurement together, so nothing has to survive between
  tool calls.
* `pgrep` cannot be used for liveness in this container: PID 1 is
  `sleep infinity` and never reaps, so dead processes linger as `<defunct>` and
  still match by name. Use `ps -o stat=` and skip `Z`, or `xdpyinfo` for X.
* numpy is not installed system-wide; `pip install --break-system-packages
  numpy` puts it in `/sylph-home/.local`, which is only on `sys.path` when
  `HOME=/sylph-home`. Scripts run with `HOME=/sylph-home/re` need
  `PYTHONPATH=/sylph-home/.local/lib/python3.12/site-packages`.

## Files

`pilot.py` (**the survival loop**) · `ctrl_probe.py` (input → speed calibration,
plus a per-tick window of the player object) · `binq.py` (query that capture) ·
`own_state.py` (definition-anchored hull/shield lookup) · `fly_session.sh`
(boot → mission → bind → fly, one task) · `wait_flight.sh` (wait for the real
HUD instead of a fixed sleep) · `navigator.py` (drift-aware steering + CPA
avoidance, reused by the pilot) ·
`gworld.py` (live reader + entity list) · `flight_probe.py` (scripted inputs +
sampling, and the `Pad` FIFO client) · `flight_analyze.py` · `whatchanges.py`
(encoding-agnostic "which words are live") · `findplayer.py` · `findself.py` ·
`findrot_global.py` · `findspeed.py` · `liveents.py` · `selfstate.py` (the
whole chain → JSON) · `autopilot2.py` (PD controller; **untested — it has never
had a valid config to run against**) · `launch_mission.sh`.