Files
Sylpheed/docs/re/mission-outcome-stage02.md
Claude (auto-RE) 5587a68e16 re: making turrets targets does not save the escort — the effect is inside the noise
The Stage-02 outcome write-up ended by naming "turrets near the asset must
become targets" as the fix worth ~50% of the escort damage. That was an
inference from a co-presence attribution, not a measurement. Run it and it
does not hold.

Also corrects the run labelling: pilot.py gained the SYLPH_KILL_TURRETS gate
two minutes before mission02 started, so mission02 was already a treatment run,
not a second baseline. Only mission01 (0 of 3968 pilot frames targeting a
turret) is the baseline.

At a common t=428s: baseline 46.9% escort hull, treatment 44.5% and 53.0%. The
two runs of the same arm differ by more than either differs from the baseline,
and the escort still reaches zero at t~590-670s in all three. So the
transferable finding is the power limit: one 430s flight cannot resolve an
effect below ~9 percentage points, and every single-run pilot conclusion,
including this one's, is inside it.

What does reproduce: the assault is scripted (onset 166/167/166s), and the
e007/e010 damage split is 50/50 in all three arms including the one that never
fires at a turret -- so that attribution measures the wave script, not us.

Also records that the viewer's include_external hypothesis in BACKLOG is dead
(it defaults true and is threaded through unchanged).
2026-08-11 05:29:52 +00:00

9.5 KiB
Raw Blame History

Why Stage 02 is never won — the escort sinks at ~11 minutes (2026-08-10)

Status: measured, one 500 s run. The standing open item since 2026-07-29 was "no mission completed". This is the first session whose deliverable was the ending rather than a measurement, and it settles why: the mission is lost before it can be won, and the pilot's survival policy is what guarantees it.

Run: tools/re-capture/mission_run.sh 500 mission01 — boot → Stage 02 in flight → pilot.py (escort-weighted targeting, target commitment, guided missiles) for 500 s, with every entity's hull sampled at 2 Hz and a screenshot every 30 s. Artifacts at /sylph-home/re/mission01/ (9 MB mission.jsonl, not committed).

The escort's decay is linear, and it ends the mission

t (s) ACROPOLIS hull %
0160 25000 100 %
180 24510 98.0
280 20279 81.1
380 13799 55.2
480 8541 34.2
485 (end) 8182 32.7

Untouched until t ≈ 170 s, then ≈53 HP/s with no let-up — so the asset reaches zero at t ≈ 640 s, and the whole-run average rate puts it at 722 s. Either way the escort is dead at 1012 minutes, and "the ACROPOLIS is sunk" is a defeat condition (mission-escort-state).

This also retires a suspicion: the 240 s time-box of earlier runs was not hiding a win, and the ~500 s ceiling of a single blocking tool call is not the binding constraint. A longer session would simply watch the loss arrive.

What is actually killing it — and the conflict that follows

Attributing damage by co-presence (which hostiles are within 3000 units of the asset in the sample where its hull drops, damage split evenly among the classes present — suggestive, not per-shot proof), only two classes are ever near it:

class samples present attributed damage
UN_e007_ADAN_Turret 200 8483
UN_e010_ADAN_Attacker_S 194 8334

Roughly half the damage comes from turrets — and pilot.py treats turrets as keep-out zones at 2500 units, never as targets. That rule is not arbitrary: a turret is what shot down every pilot before 2026-07-30, and it is why the craft now survives. But it means the policy that keeps the pilot alive also guarantees the escort dies. Survival and the objective are in direct conflict, and the pilot currently resolves it entirely in favour of survival.

The HUD at t≈485 s says the same thing from the game's side:

  • YOU KILLED WARSHIPS 0000 — not one warship in 500 s, across every run ever;
  • YOU KILLED WARPLANES 0009 — fighters only;
  • REMAINING OB 004 → 008 — objectives are being added by waves faster than any are cleared, so the pilot is not touching the objective set at all;
  • SHIELD and ARMOR bars full, hull 1500/1500, 120 missiles spent.

Stage 02 at t≈485 s

The conclusion that matters

The pilot optimises the wrong thing. It maximises survival and fighter kills; the mission scores objectives and the escort, and the fighter population (134 → 92) is close to irrelevant to both. An untouched 1500/1500 hull at the moment the escort passes 33 % is not a good run — it is unspent risk budget.

Concretely, for the next attempt, in priority order: (Step 1 below was run on 2026-08-11 and did not hold — see the follow-up A/B at the end of this file before acting on it.)

  1. Turrets near the asset must become targets, not keep-out zones — accepting hull damage is the only way to cut ~50 % of the incoming escort damage. The keep-out rule should be scoped to turrets that are not threatening the asset, rather than applied globally.
  2. Engage warships. WARSHIPS 0000 forever means the objective class has never been attacked; REMAINING OB rising is the scoreboard saying so.
  3. Re-check whether the escort damage rate actually falls once turrets die — that is the experiment that tells us whether (1) is sufficient or whether the bombers need dedicated intercept too.

Method note, learned the hard way

A harness-tracked background task does not protect the display: the same run launched in the background lost Xvfb 11 s in, at the turn boundary (skip_intro exit 3, "DISPLAY LOST"). The comment in launch_mission.sh saying the script may be run as a tracked background task is wrong; the one-blocking-foreground-call rule still stands, which caps a single attempt at the tool's 600 s timeout. And do not pipe a long run through tail — the first attempt printed nothing because timeout killed the pipeline before it flushed; the on-disk artifacts are what survived.


Follow-up A/B (2026-08-11): making turrets targets does not save the escort

Status: measured (3 runs), and it falsifies the causal claim above. The section above ends by naming step 1 — "turrets near the asset must become targets" — as the fix worth ~50 % of the escort damage. That was an inference from an attribution, never a measurement. It has now been run, and it does not hold.

First, a correction to the run labelling

pilot.py gained the SYLPH_KILL_TURRETS gate at 21:20:28 on 2026-08-10; /sylph-home/re/mission02 started at 21:22:25, i.e. after it, with the gate on. So mission02 was never a second baseline — it is a treatment run that the previous session produced but never reported. The arms are:

run turret rule evidence (pilot.log)
mission01 off (baseline) tgt=e007 on 0 of 3968 lines
mission02 on tgt=e007 on 1588 of 3844 lines
mission03 on (this session) tgt=e007 on 1218 of 3514 lines

mission01 predates the edit, so the only difference between the arms is the gate. Everything else — build, save slot, nav route, loadout — is identical.

The result, all three runs truncated to a common t = 428 s

run rule ACROPOLIS @428 s % decay rate turrets seen turrets killed attack onset
mission01 off 11735 46.9 % 50.3 HP/s 92 30 166 s
mission02 on 11127 44.5 % 52.7 HP/s 98 34 167 s
mission03 on 13257 53.0 % 44.4 HP/s 88 41 166 s

The two runs of the same arm differ by 8.5 percentage points (2130 HP) — more than either differs from the baseline. The baseline sits between the two treatment runs on escort hull and on decay rate. So the effect of the turret rule is not resolvable at n=1 per arm, and the honest statement is: it was not demonstrated. Extrapolating each run's own rate, the escort still reaches zero at t ≈ 590670 s in every arm — the mission is lost in all three.

The rule does do the mechanical thing it was written to do (turret kills 30 → 34, 41; 12181588 frames spent with a turret as the committed target). It just does not convert into escort hull.

Stage 02 at t≈430 s with the turret rule on — WARSHIPS 0000, REMAINING OB 008

What is reproducible across all three runs

These are the numbers to build on, because they repeat to within a percent:

  • The assault on the ACROPOLIS is scripted, not emergent. Onset at 166 / 167 / 166 s — ±1 s across three runs with completely different pilot behaviour in between. Nothing the pilot does moves it.
  • The damage split is a property of the scenario, not of our targeting. e007 turrets vs e010 bombers is 50.6/49.4, 50.0/50.0, 51.0/49.0 — and the baseline, in which we never fired at a turret at all, splits the same way. A co-presence attribution that is invariant to whether we attack one of the two classes is measuring the wave script, not our contribution.
  • Turrets die anyway: 30 of 92 are already dead by t=428 s in the baseline, from friendly fire. The turret rule adds ~411 kills on top of that, which is why its escort-hull effect is small enough to be buried in run-to-run noise.
  • WARSHIPS 0000 and REMAINING OB rising (004 → 008) in every run, including this one.

Per-run series: captures/escort-decay-ab.csv (10 s grid, hull + turret alive/killed counts for all three runs).

The methodological finding, which is the transferable one

A single 430 s flight cannot resolve an escort-hull effect smaller than ~9 percentage points. Every pilot conclusion drawn from one run — including step 1 above, and including anything drawn from mission03 alone — is inside the noise. Each run costs ~10 minutes of wall clock and one blocking foreground call, so a properly powered A/B is ~6 runs ≈ 1 hour. Any future pilot tuning must budget that, or not be believed.

What this means for the reason we wanted a win

Story progress was wanted only because unit definitions instantiate per stage, so unit-field coverage is stuck at 21/110 (unit struct). This result says the pilot route to that coverage is expensive and unproven: the escort dies on a script at ~1011 minutes, our best lever moved it by less than the noise, and the objective class (WARSHIPS) has never been touched in any run ever. Before spending another hour on pilot tuning, the cheaper lever to price is the save file — if the profile's save data can be read and understood, stage unlock state is a much shorter path to the same coverage, and the save format is itself something the reimplementation needs. Marked NEEDS-HUMAN as a direction choice; no save-file modification has been attempted.