Sylpheed/docs/re/script-runtime-probe.md

Reading the live script state — the real phase counter, and per-squadron liveness

Status: ✅ ScriptMission and ScriptPhase located in a running mission with no debugger, validated arithmetically; ✅ the true phase ordinal read live; 🟡 the per-unit state encoding needs care.

This unblocks mission-phase-membership, which was stuck because "38 enemies died" could not say whether the right ones did. Chasing craft→squadron was the wrong angle: the script VM already keeps that table, indexed by the .ssb symbol-table-2 index.

Tool: tools/re-capture/squadron_state.py.

✅ Locating the objects, without gdb

1. Find the .ssb header in guest memory — 20 bytes of version + code offset
   • the two symbol-table offsets, distinctive enough to hit once. For a Stage 02 run it sat at 0xAB840010.
2. code base = file base + header code offset (0x24) → 0xAB840034.
3. [ScriptMission+24] is that code base, so scan for a word equal to it.
4. Validate arithmetically, not by eye: [ScriptMission+44] must equal file base + symtab1 offset + 4. Measured 0xAB874C94; predicted 0xAB840010 + 0x34C80 + 4 = 0xAB874C94. Exact.

That check is what makes this trustworthy — the candidate is confirmed against a number taken from the file on disc, not against "it looks like a pointer". A second candidate that also pointed at the code base failed it and was discarded.

ScriptMission 0xBC7A2A20
  +4  ScriptPhase*  = 0xBE14DD80      +20 state     = 1  ("phase running")
  +24 code base     = 0xAB840034      +28 pc        = 0xAB84007C
  +40 PHASE ORDINAL = 1               +44 symtab1   = 0xAB874C94  ✓
ScriptPhase 0xBE14DD80
  +196 finished = 0    +244 symtab1 = 0xAB874C94    +324 unit array = 0xBC43B560

ScriptPhase+324 → +4 → an array of per-unit records. It holds 122 records — exactly the size of Stage 02's symbol table 2, which is an independent confirmation that the index space is the one the bytecode uses.

✅ The real phase counter reads 1 — the mirror was the wrong field

[ScriptMission+40] reads 1 in a phase-1 mission. The runtime mirror at [*(0x828F35F8)+236], which three earlier runs polled, reads 0 — because ChangePhase is only posted once the ordinal exceeds 1.

So the mirror is not a phase readout at all in phase 1, and +40 is. It is reachable from /dev/shm with no debugger, which is what made three runs of polling the wrong address avoidable in hindsight.

✅ PINNED: state 1 = not yet deployed, state 2 = active — and arrivals are real

Watching [ScriptMission+40] and the three objective squadrons together across a live run (data/phase-watch-s02.txt):

[   1.8s] phase=1 finished=0 active= 24  ADN110:1  ADN111:1  ADN112:1
[  58.2s] phase=1 finished=0 active= 27  ADN110:1  ADN111:1  ADN112:1
[  90.7s] phase=1 finished=0 active= 30  ADN110:1  ADN111:1  ADN112:1
[ 120.8s] phase=1 finished=0 active= 29  ADN110:1  ADN111:1  ADN112:1
[ 143.4s] phase=1 finished=0 active= 32  ADN110:2  ADN111:2  ADN112:2   <-- arrive
[ 223.3s] phase=1 finished=0 active= 35  ADN110:2  ADN111:2  ADN112:2

All three flip 1 → 2 at ~143 s, and the count of records in state 2 climbs 24 → 35 over the same window. So for these squadrons state 1 is "not yet deployed", not "gone" — the built-in table's shorthand "1/3/4 = gone/dead/invalid" is incomplete, and reading state != 2 as "destroyed" would have been wrong in exactly the way flagged last iteration. Good that it was flagged rather than assumed.

✅ This also answers a much older question: arrivals DO happen

mission-arrival-watch and the wave work recorded "0 confirmed arrivals" after many runs, measured by watching the craft population. The script's own unit table shows arrivals plainly: eleven more records enter state 2 within four minutes, three of them the phase-1 objective squadrons at a distinct moment.

The old negative was not wrong about what it measured — it was measuring the wrong structure. Craft counts conflate deployment with attrition; the per-unit state field does not.

⚠️ Both runs of this experiment froze — at ~70 s and ~253 s — so the window above is all that was observed, and no phase advance was reached. The freeze witness caught both immediately, which is the only reason the truncation is visible rather than silently producing a flat line.

🟡 The per-unit state encoding is not what the summary implies

For the three phase-1 objective squadrons, early in a fresh mission:

ADN110  idx=1  obj=True  state=1
ADN111  idx=2  obj=True  state=1
ADN112  idx=5  obj=True  state=1
records with state==2 (active): 27-29 of 122

The built-in table describes +16 as "2 = active; 1/3/4 = gone/dead/invalid". But these three have a live object pointer and state 1, in a mission that has barely started and where nothing has been shot. So either state 1 does not mean "gone", or it means "not yet deployed" — not settled, and worth pinning before any conclusion is drawn from it. Reading state != 2 as "destroyed" would be exactly the kind of plausible-but-wrong inference this corpus keeps catching.

What this makes possible

The decisive phase experiment is no longer blocked on attribution — and it has now partly run: the arrival of the three objective squadrons is directly observed. What is still missing is a run that survives long enough (no freeze) for them to be destroyed, which is when [ScriptMission+40] should step to 2. Two attempts froze first.

tools/re-capture/phase_watch.py is the harness: it samples the real counter and the watched squadrons together, witnesses the freeze every 60 s, and prints only on change.

✅ 2026-08-25 — the bounded scan fixes the freeze, and the state machine is confirmed live

The sweeps were the cost. Bounding the pointer scan to 0xBC000000–0xBD000000 (with a full-sweep fallback) drops find_mission from a full ~371 MB walk to 0.7 s. The run then went 694 s with the probe attached and no freeze, against 3 of 3 frozen inside ~4 minutes with the unbounded version. n=1, but it is the first probe-attached run to survive past four minutes.

Full trace in data/phase-watch-s02-full.txt:

[   0.7s] phase=1 finished=0 active=24  ADN110:1 ADN111:1 ADN112:1
[ 113.8s] phase=1 finished=0 active=33  ADN110:2 ADN111:2 ADN112:2   <- arrive
[ 191.0s] phase=1 finished=0 active=36
[ 433.0s] phase=1 finished=0 active=30  ADN110:2 ADN111:4 ADN112:2   <- ADN111 destroyed
[ 631.7s] phase=1 finished=0 active=27
[ 694.9s] phase=1 finished=1 active=27                               <- phase ends

✅ State 4 = destroyed — a squadron death caught in the act

ADN111 goes 2 → 4 at 433 s while the active count falls 36 → 27 over the same window. Together with the earlier 1 → 2 arrival this pins three points of the encoding: 1 = not yet deployed, 2 = active, 4 = destroyed.

✅ The mission-over branch, observed exactly as disassembled

The phase ended at 694.9 s, but the ordinal did not advance — and the reason is the branch mission-phase-advance read out of sub_82260710:

if ([phase+300] == 2)  post 994 ; state = 0 ; MISSION OVER
else                   state = 5 ; [mission+40] += 1   NEXT PHASE

Measured at the end of the run:

field	value	meaning
[phase+300]	2	last-phase flag set (built-in 39)
[mission+20]	0	the mission-over state
[phase+196]	1	phase finished
[mission+40]	1	ordinal unchanged — correct for this branch

Every field matches the disassembled branch, on the live oracle. The static reading of the phase state machine is confirmed — for the mission-over half.

🔴 This was a LOSS, not a phase clear

screen_id shows the GAME OVER frame, the escort was down to 35.7 %, and the pilot logged DEAD at 676 s. So a lose path ran built-in 39 (MARK_LAST_PHASE) and then END_PHASE, which is why the mission ended instead of advancing.

Two of the three objective squadrons were still alive (ADN110 and ADN112 at state 2), so this says nothing about whether destroying all three clears phase 1 — that prediction is still untested. What it does establish is that the else branch is the only way to reach phase 2, and it requires [phase+300] != 2 at the moment the phase ends.

Still not observed: a phase ADVANCE. Five attempts. The obstacle is no longer the freeze or the instrument — it is keeping the escort alive long enough to win.

🔴 2026-08-25 — the "win the mission" route is not converging

Two more attempts, and the honest summary is that flying to a phase clear is the expensive way to test the prediction.

The pilot's gun-fire rate is 1.6 % — 81 fire frames in 4986 samples — but that is not the blocker it looks like. The nose gun is Power 15 unguided; the main mount is Power 200 guided, and the pilot fired ~70 missiles in ~500 s. The damage is coming from missiles, and fire= in the log only tracks the gun. Worth writing down because the log invites the wrong conclusion.

SYLPH_KILL_TURRETS=1 made things worse, not better. The idea was to align DEFEND with the objective by letting it kill turrets attacking the escort. Measured: 3387 of 11112 samples (30 %) chased a target more than 20 000 units away — turrets are static and spread across the map, so the pilot commits to distant ones and stops defending anything. The escort still fell to 48.5 %, and no additional objective squadron died. Refuted as an improvement.

Both runs ended the same way as before: ADN111 destroyed (again — it is evidently the one closest to the action), ADN110 and ADN112 untouched at state 2, no phase advance. Six attempts now.

🟡 The bounded scan delays freezes but does not remove them

bounded-scan run	outcome
1	clean to 694 s, ended by the mission's own lose branch
2	froze at ~682 s

Against 3-of-3 frozen inside ~4 minutes with the unbounded sweep, that is still a large improvement — but "the sweeps were the cause" is too strong. They were a cost; something else also freezes runs at ~11 minutes.

The cheaper experiment to run instead

Stop trying to win. The prediction is that finished goes to 1 with [phase+300] != 2 when ADN110/111/112 all reach state 4. Guest memory is writable (tools/re-capture/gpoke.py), so set the two surviving squadrons' +16 to 4 directly and watch whether the phase ends and the ordinal steps to 2. That tests the condition in seconds rather than fighting a mission the autopilot is not good enough to win, and a wrong answer is as informative as a right one — if nothing happens, the condition is not what the bytecode reading says.

🔴 2026-08-25 — poking all three squadrons to "destroyed" does NOT end the phase

The direct test, run instead of a seventh attempt at winning. All three objective squadrons were live (state 2) when the poke went in.

ADN110 rec=0xBCA48BC0  +4=0x0000001A  +16=2
ADN111 rec=0xBCA48C60  +4=0x0000001B  +16=2
ADN112 rec=0xBCA48D00  +4=0x0000001C  +16=2
STICK TEST on ADN110 +16: was=2 wrote=4 after2s=4 -> STICKS
poked all 3
  [+ 5s .. +60s]  phase=1 finished=0  states={ADN110:4, ADN111:4, ADN112:4}

The write sticks — and nothing happens. Sixty seconds with all three reading state 4 (the value a naturally-destroyed squadron takes, measured earlier on ADN111), and [ScriptPhase+196] stayed 0 and the ordinal stayed 1.

So "phase 1 clears when ADN110/111/112 are destroyed" is not confirmed, and the simplest form of it is refuted. The bytecode reading — three unit_state polls then set_flag(8) — is solid; what does not follow is that flipping this field is equivalent to the kill.

🟡 Why it probably did nothing: the poll was not running

That the poke persisted for 60 s is itself the clue. Built-in 69 is documented as normalising +16 when it polls, so if the condition coroutine were running its unit_state polls, it should have overwritten the value within a frame. It did not — which points at the condition being evaluated only when a trigger fires, not on every frame. Poking state without firing the trigger changes a value nobody reads.

🔴 The per-unit record layout is not what the built-in summary says

Dumping ADN110's record contradicts "+4 live object (NULL = absent)":

+0  = 2          +12 = 0x42480000 (50.0f)     +20 = 9
+4  = 26         +16 = 4  (state)             +128 = 0x3F733333 (0.95f)

+4 is 26/27/28 for the three squadrons — small consecutive integers, not pointers (an undeployed squadron, ADN201, has +4 = 0 and +16 = 0). And +20 = 9 is exactly these squadrons' member count n, which the roster gives independently — so the record is per-squadron and carries its strength.

Earlier readings printed obj=yes because the probe tested that word for non-zero, not for pointer-ness. That is a reporting bug in my own tool, and it made a small index look like a live object.

Not settled: what +4 indexes (a route or symtab-1 index is the obvious guess, given the values), and how to make the condition actually re-evaluate. Firing the trigger — built-in 100 pushes onto [phase+272] — is the next thing to look at.

🔴 The corrected poke (rec+4 = 0) also does nothing — and I have no positive control

Built-in 69 reads rec+4, not rec+16 (see isl-builtins), so the corrected simulation of "this squadron is gone" is rec+4 = 0, which takes the handler's documented early exit. Ran it with all three squadrons active:

STICK TEST on ADN110 +4: was=26 wrote=0 after2s=0 -> STICKS
poked all 3
  [+5s .. +60s]  phase=1 finished=0  states={ADN110:2, ADN111:2, ADN112:2}

No phase end, and +16 never changed either — it stayed 2 for a full minute while the handle it is supposedly derived from read zero.

🟡 The leading explanation: the condition coroutine is not running

Both pokes persist untouched, and nothing recomputes +16. If the phase-1 condition were polling unit_state on these squadrons each frame, one of these writes should have provoked something. The consistent reading is that the polls at 0xF524 live in a coroutine that only runs when a trigger starts it (built-in 1 start_coroutine, fed from the queue at [phase+272]), so changing unit state without firing the trigger changes data nobody is reading.

🔴 The gap in this method: no positive control

Two pokes have now produced no observable effect, and I cannot distinguish "the game ignored the write" from "the write never reached the game". The stick test only proves the value persisted in the shared-memory file — it does not prove Canary's guest sees it.

That is a real hole, and it should have been closed before drawing conclusions from a null result. The control to run first: poke something with a visible effect — the player's hull, or a HUD counter — and confirm the change appears on screen. Until that passes, every "poking X did nothing" here is unfalsifiable rather than informative, and the two null results above must be read as inconclusive, not as evidence about the condition.

What still stands from these runs is only what was observed, not poked: the arrival transitions, the destruction of ADN111, and the mission-over branch.

🟡 The positive control ran — and is still inconclusive

tools/re-capture/poke_control.sh (self-retrying; succeeded on attempt 1, no freeze) poked the player's hull at position + 0x154 to 1, on the theory that the game visibly reacts to a dead player.

hull before: 0x447A0000        (= 1000.0f)
poke        0x447A0000 -> 0x00000001   OK
hull after : 0x447A0000        (12 s later -- the game put it back)
screen before: flight     screen after: flight

What it establishes: the hull word is continuously rewritten by the game, so a poke there cannot persist — unlike the unit-record fields, which held our value for 60 s untouched. That asymmetry is itself informative: it separates fields the game actively maintains from fields nobody is writing.

Why it is still not a control. I looked for a visible reaction and found a red WARNING banner in the after-frame — but the before-frame already shows MISSILE ALERT, i.e. the ship was under attack in both. The banner is not attributable to the poke, and I am not going to count it. A value being overwritten proves the game writes that address; it does not prove the game read ours.

New fact, and a correction: hull at pos + 0x154 reads 0x447A0000 — a float, 1000.0 — not the 1500 the pilot logs. The pilot's hull= is a different field or scale, and the two should not be conflated.

The refinement that would settle it

Poke in a tight loop for several seconds so the value is low whenever the game samples it, rather than once between two of its own writes. If hull is authoritative, the ship dies and the screen goes to GAME OVER — unambiguous. If it survives a sustained low hull, the field is a readout and the authoritative copy is elsewhere. Either answer is worth having.

✅ The self-retrying harness works and is the reusable part of this iteration: boot → verify animating → locate → act, with a freeze at any step costing one retry instead of a whole iteration.

✅ THE CONTROL PASSES — pokes reach the guest, and hull is authoritative

Hammering the write instead of doing it once settles it:

hull before: 0x44BB8000        (= 1500.0f)
hammered hull=1 for 15 s -- 944,387 writes
hull after : 0x00000001        (the game STOPPED rewriting it)
screen: flight -> other

The after-frame (captures/poke-control-kill.png) shows the flight HUD gone, the ship trailing fire, and a radio line: "I've lost contact with Rhino 3!" — Rhino 3 being the player's own callsign. The game read the poked value and killed the player.

So, established:

• Writes to /dev/shm/xenia_memory_* do reach the running guest. The plumbing works.
• Hull at player position + 0x154 is authoritative, not a readout.
• A single write loses a race — the game rewrites hull continuously, so one poke lands between two of its own writes. Hammering wins; 15 s was ample.

✅ This upgrades two earlier "inconclusive" results to genuine negatives

The poke experiments on the unit records were downgraded to inconclusive because I could not tell "the game ignored it" from "the write never arrived". The write arrives. And those pokes persisted untouched for 60 s — nothing overwrote them — so the game genuinely saw state = 4 and handle = 0 on all three objective squadrons and did nothing.

That is now real evidence for the standing explanation: the phase-1 condition coroutine is not polling during ordinary flight, and the polls at 0xF524 run only when a trigger starts them.

🔴 Withdrawn: "the pilot's hull= is a different field"

Last iteration I read 0x447A0000 (1000.0f) at pos + 0x154 and concluded it could not be the 1500 the pilot logs. Wrong. This run reads 0x44BB8000 = 1500.0f at the same offset. It is the same field; the value simply differs between runs (craft or loadout). The "different field or scale" note is retracted.

✅ The JIT keeps the guest context in %rsi — from Canary's own source

The watchpoint's ceiling was that the writer is JIT code with no host symbols. The way past it is in the emulator's source, not the disassembly:

src/xenia/cpu/backend/x64/x64_emitter.cc:881
    Xbyak::Reg64 X64Emitter::GetContextReg() const { return rsi; }
    Xbyak::Reg64 X64Emitter::GetMembaseReg()  const { return rdi; }

So at any JIT instruction, %rsi is the PPCContext* — which is also why the faulting instruction read 0x110(%rsi): it was loading a guest register. The whole guest register file is available at the moment of the write, and a guest code address (0x82xxxxxx) picked out of it resolves against sylpheed.db to name the calling function.

trigger_watch.sh now dumps x/128wx $rsi at the hit rather than a useless host backtrace. ⚠️ Reading the LR by offset would need PPCContext's layout; scanning the dump for 0x82…-range words avoids parsing a 900-line struct and is what the script does.

🔴 The re-run could not locate the mission — unexplained

The run reached flight (readyroom at 18s, IN FLIGHT at 40s), the pilot bound, the guest was animating, and yet find_mission returned NOTFOUND. Narrowing it:

• the .ssb header is not in guest memory — 0 hits for its 20-byte signature, where previous runs hit it immediately;
• ADN110 is not in guest memory either — so the script's symbol table is not resident;
• but Stage02.ssb (the manifest string) is present, at 0xBDA6C50B.

So guest memory is readable and the manifest is loaded, while the script itself is not — in a mission that is demonstrably flying. That contradicts four earlier runs where the header was found within seconds of flight.

I do not have an explanation, and I am not going to invent one. Candidates worth separating next time: the script is loaded later than I assumed and the earlier runs sampled later; the probe raced a load; or this run entered flight by a different path. The cheap discriminator is to poll for the header from the moment flight starts and record when it appears, rather than sampling once.