Last iteration I said launch_mission died because skip_intro only tests for
the title on a static frame, gated at rmse <= 1500, and that run logged 1503
and 1549 just above the cut. I also said the fix was NOT to nudge 1500 but
to measure both signals through a boot first. Measured, and the diagnosis
does not survive.
boot_trace.sh logs the two signals skip_intro decides on -- frame-to-frame
RMSE and the is_title.py green-glyph count -- through a clean boot with no
presses at all. One run, 29 samples over 484 s:
8 samples had rmse <= 1500, so the gate OPENED eight times
0 samples had glyph > 0, so the title was never seen
At t=145 s the RMSE was 1205, comfortably inside the threshold, and the
glyph test was called and answered zero. A frame can be perfectly static
without being the title -- the intro movie has long quiet stretches, three
reading RMSE exactly 0. So 1503/1549 were almost certainly movie frames too,
and raising the constant would have admitted two more of them.
What is left is narrower and honest: the interactive title never appeared,
rather than appearing and being missed by a threshold.
The limitation is recorded rather than buried: the tracer intended 1 s
sampling and achieved 16.9 s, because each iteration forks two screenshots,
ImageMagick compare and a fresh Python. So this does NOT prove the title
never appeared -- only that it was absent from 29 samples. A window shorter
than ~17 s falls between them. The recorded next step is to make the tracer
sample at the rate it claims before concluding anything stronger.
Artifact: docs/re/captures/boot-signal-trace.tsv.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Third independent line for yesterday's built-in 100 rename, from the callee
this time. sub_8226E3B8 was labelled "push", which is what made built-in 100
look like push_trigger. It reads the element count, returns immediately when
the container is EMPTY, and otherwise walks the node list splicing nodes out
until it is empty. A push links one node; this unlinks all of them. It is
clear(). The append is sub_8226E160, reached from built-ins 19 and 25.
So the rename now rests on the handler, the usage (all 12 Stage 02 sites sit
in the phase terminator next to timer_stop / clear_flag(-1) /
MARK_LAST_PHASE), and the callee.
The dynamic half did NOT run, and the write-up says so. phase_watch.py now
samples [phase+272+20] (triggers queued) and [phase+216+8] (coroutines
alive) so a phase terminator's effect on the VM is visible in one line —
written here, never yet exercised against a live guest.
Boot-nav could not reach the title in 381 s. Diagnosed rather than retried:
skip_intro.sh only runs the title test on a static frame, gated at
rmse <= 1500, and this run measured 1503 at 104 s and 1549 at 139 s — just
above the cut — so is_title.py was never called and the one allowed press
was never spent. Recorded in BACKLOG with the explicit instruction NOT to
raise the constant: the first step is to log rmse and the glyph count
through a whole boot and look at the two distributions, because tuning a
threshold to make one run pass is fitting to a single sample.
Also reaped a stale lock: a gdb orphaned 2h14m earlier was holding
/tmp/xenia-canary.lock with an already-defunct emulator child.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
All re-read twice — the handler, and the thing it calls — because each had
been named from its shape rather than its effect.
* id 11 `yield` -> `end_coroutine`. 0x82272624 is li r11,1 ; li r3,3 ;
stw r11,164(r31), and the dispatcher's r3==3 arm erases the thread from
the active list and returns it to the free list. It destroys the thread.
2945 sites game-wide, 372 in Stage 02 — the most-used built-in there was.
* id 5 `await_label` -> `kill_coroutine(label)`. sub_82273B08 kills the
thread parked at the target pc, or itself if the target is its own pc.
It waits for nothing.
* id 100 `push_trigger` -> `reset_phase_threads`. It clears the trigger
container and then frees every thread whose pc differs from the caller's
— the opposite of pushing a trigger. Corroborated by usage: its 12 Stage
02 sites all sit in the phase terminator, next to timer_stop,
clear_flag(-1) and MARK_LAST_PHASE.
One name recovered from the game's own text: opcode 992 prints
"RequestScriptMessage %s" at 0x820A5700, so id 64 is request_script_message
(2683 sites).
Return codes documented properly: 1 = restart the coroutine from its entry
(previously not recorded at all), 3 = terminate. And the blocking set was
wrong in two places — it is 102, 120, 137, 142, 143. Id 97 does NOT block;
its handler ends `b 0x822724F8`, so it always returns 0.
Unit-operand resolution settled from DATA over all 28 stages rather than by
reading 147 handlers: a slot qualifies only if every value is a valid
symtab-2 index, it takes >=15 distinct values, AND its maximum reaches most
of the table — that last clause is what discriminates, since every small
integer is trivially "in range". 31 built-ins at slot 4, 8 at slot 12, one
at slot 20. It also refutes set_flag's slot 0, whose maximum overruns the
table, and the resolver now declines rather than inventing a name.
New and unexplained: symtab-2 holds two types, 2 and 8, and built-ins 95 and
128 take type 8 at slot 12 in 100% of their sites.
A downstream inference is withdrawn with it: the note reading the live
trigger counter attributed it to "the script arming watches as it goes" via
built-in 100. The measurement stands; the attribution does not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Settled from the disassembly, no run needed. Built-in 69's tail maps the
lifecycle lookup into [phase+164]: handle == 0 takes the early exit at
0x8226AF44 and returns 0, while the destroyed states return 2, 3 or 4 (and two
of those also normalise the record's +16). Those are different values.
So poking rec+4 = 0 made the predicate report ABSENT -- the same answer an
undeployed unit gives -- and never the answer the script branches on. The
condition was polling at 5 Hz throughout and correctly saw 'not here'.
Both null results are now fully explained, and neither was evidence about the
condition: the first poke wrote a field nobody reads, the second wrote the wrong
value into the right field. Simulating a kill needs the handle to stay valid
while the lifecycle lookup returns 3/4/5, i.e. the write belongs in whatever
sub_82301240 reads, not in the script's own record.
Also names built-ins 46/47/48 as squadron_trace / squadron_attack /
squadron_escort in isl.py.
sub_822FE040 is a fully unrolled registration: 1023 slots filled with a default,
then 57 explicit writes, of which 48 are real handlers and nine are a shared
accept-and-discard stub. Return convention is nonzero = consumed, 0 = retry,
which is how the interpreter waits for a named unit to exist.
Opcode 995 is the ONLY handler touching the phase mirror [*(0x828F35F8)+236] --
the sole read and sole write in the table -- independently confirming why polling
that mirror saw nothing during phase 1. And no handler spawns or despawns a unit:
256 is the strongest deploy candidate but is unconfirmed because the message ids
are write-only in this image.
WITHDRAWN, verified wrong: I had recorded the writes to '+20' in sub_8226E7D8 /
sub_8226E930 as block initialisations by container constructors. At
0x8226E86C-0x8226E8E0 they do li r3,28 / bl 0x8230C160 then
lis r10,0xAB03 / ori r7,r10,0xE4BA / stw r7,4(r3): they build an INTERPRETER
COMMAND RECORD for opcode 996 and push it, i.e. AddSelector and RemoveSelector,
with a 32-entry cap. The stw to 20(r3) is the command record's +20, a different
object. Wrong twice: not constructors, and not that container.
Also flags that sub_8230C398 -- gated on *(0x82899CE0) == 16 at both call sites
-- looks like Stage 16's script compiled in C++, which 'debug defaults' does not
survive given the .ssb loader explicitly refuses mission 16.
The watchpoint plus Canary's source settle it. At the write, the guest context
(rsi, per x64_emitter.cc:881) holds 0x8226EAE0, inside sub_8226EAB8. That
function is a generic list-node insert: it reads the count at 8(r30), guards
against 0x3FFFFFFF overflow, does addi r11,r11,1 / stw r11,8(r30), then links the
node. It has 16 callers, so it is a shared container helper.
That explains why two static searches missed it. The trigger container at
phase+272 EMBEDS an inner list object at +12 -- which is why the push does
'addi r31, r30, 12' -- and the inner object keeps its count at its own +8. So
272 + 12 + 8 = 292 = the watched word, and the instruction is stw r11, 8(r30)
with r30 = phase+284. Searching for 'stw rN, 20(rM)' could never have found it.
Also resolves the earlier open item on +12: it is the embedded list object, not
a list head pointer, which is why it read 0x000A0009 instead of an address.
Method note kept: the static hunt assumed the field's offset in the OUTER object
would appear in the writing instruction. A watchpoint is indifferent to the
addressing form, which is why it was the right tool after two failed offset
searches.
ssb_watch.py polls the .ssb header, a symbol string and find_mission from the
moment flight is detected. On a normal run all three are present at the FIRST
sample (t=0.0, mission 0xBC79C960), so 'the script loads later than assumed' and
'the probe raced a load' are both out -- there is no window in which a healthy
mission flies without its script in memory.
A detail worth more than it looks: at that sample screen_id reported 'other',
not 'flight'. The script is loaded and the ScriptMission locatable BEFORE the
flight HUD appears, so residency is not gated on the HUD, and a run showing the
HUD without the script is in a state a healthy run never passes through.
That leaves the third candidate: the anomalous run's mission never loaded and
its IN FLIGHT was a misdetection -- consistent with it being frozen on a black
screen when checked afterwards. Not proven, but now the only surviving
explanation rather than one of three.
From Canary's own source (x64_emitter.cc:881) GetContextReg() returns rsi, so at
any JIT instruction %rsi is the PPCContext* -- which is also why the faulting
instruction read 0x110(%rsi), a guest register load. That is the way past the
watchpoint's ceiling: the guest register file is available at the write, and a
0x82xxxxxx word picked out of it resolves against sylpheed.db to name the caller.
trigger_watch.sh now dumps x/128wx instead of a useless host backtrace.
The re-run then failed for an unrelated and unexplained reason: it reached
flight, the pilot bound, the guest was animating, and find_mission returned
NOTFOUND. Narrowed: the .ssb header is absent from guest memory (0 hits where
earlier runs hit immediately), ADN110 is absent too, but the manifest string
'Stage02.ssb' IS present at 0xBDA6C50B. So memory is readable and the manifest
is loaded while the script is not, in a mission that is flying.
No explanation offered. The cheap discriminator for next time is to poll for the
header from the moment flight starts and record when it appears, instead of
sampling once.
All 28 StageNN.ssb decoded: 2,085,628 bytes, 25,705 call sites, 108 of 147
built-ins used. Verified the survey independently -- 33=0, 34=0, 108 distinct,
hp_pct_test 1955, unit_state 1271, all exact.
global_counter0/1 have ZERO call sites in all 28 stages. The handlers exist and
are wired; no mission calls them. So 'does a wave start after N kills?' is
answered for the whole game, not just Stage 02: no mission counts kills.
Two idioms change how the counts read. hp_pct_test(unit, 0.0) IS a destruction
test -- 1786 of 1955 calls (91%) pass 0.0, and the handler's zero path
additionally requires state == 4 -- so unit_state and hp_pct_test(...,0) are
interchangeable and stages just pick one. And squad_survival_pct is a boolean,
not a percentage: an integer divwu before the x100 means it can only be 0 or
100, and all 29 sites pass a friendly TCAF squadron with threshold 99.9, making
it 'has this escort lost anybody'. The game has no destroy-N%-of-a-squadron
objective.
Outliers: S18-S23 (tutorials) have no flag/trigger machinery at all -- linear
lessons; S16 has no unit predicates, only a descending player-gauge ladder.
Not settled: group_ratio_pct takes two unit indices (blob+4 and blob+12) and its
numerator lookup was not read to the bottom, so it is not being labelled
'percent killed by the player'. isl.py's UNIT_ARG omitted 71/72.
All 25 opcodes now have meanings. Ops 2/4/6/8 are integer compound assignment
(+= -= *= /=) and 3/5/7/9 the float versions; 10 and 11 are integer and float
compare writing three condition bits; 13-18 are je/jne/jl/jle/jg/jge; 21-24 are
push.i/push.f/pop.i/pop.f over deques at phase+44 and phase+64.
The shared-handler question is answered: the dispatcher leaves the opcode in r4
and the shared thunks never overwrite it, so those helpers take an extra opcode
argument and index a secondary table (0x82271448, 0x8227152C).
CORRECTION to my own tool and note: the branch/jump base is [phase+232], which
the phase initialiser sets to 0x24 + the phase's entry from the mission-level
stream -- 0xE4 / 0x14AA8 / 0x24B4C for Stage 02's three phases, not the file's
0x24. Measured on phase 1: base 0xE4 puts 525 of 525 branch targets on an
instruction boundary; base 0x24 manages 188. isl.py had been using 0x24 for
every phase, so its jump targets were wrong throughout. Fixed via
isl.phase_bases().
That also settles two things mission-script-ssb.md left open: offsets ARE
code-base-relative, and 0x1883's operand IS a code pointer -- the earlier worry
that some 'land on IEEE floats' was an artefact of adding the wrong base.
Hammering settles what a single write could not: hull 0x44BB8000 (1500.0f),
944,387 writes of 1 over 15s, and afterwards the value HELD at 1 -- the game
stopped rewriting it. The screen left 'flight', the HUD is gone, the ship is
burning, and Natalie radios 'I've lost contact with Rhino 3!', the player's own
callsign. The game read the poked value and killed the player.
Established: writes to /dev/shm reach the running guest; hull at pos+0x154 is
authoritative, not a readout; and a single write loses a race against the game's
own continuous writes.
This upgrades two earlier results from inconclusive to genuine negatives. The
unit-record pokes were downgraded because I could not tell 'ignored' from 'never
arrived'. The write arrives -- and those pokes persisted untouched for 60s, so
the game genuinely saw state=4 and handle=0 on all three objective squadrons and
did nothing. That is real evidence the phase-1 condition coroutine is not polling
and its checks run only when a trigger starts them.
Withdrawn: last iteration's claim that the pilot's hull= is a different field or
scale. I read 1000.0f at pos+0x154 and inferred a mismatch with the logged 1500;
this run reads 1500.0f at the same offset. Same field, different value per run.
poke_control.sh (self-retrying, succeeded on attempt 1 with no freeze) set the
player's hull at pos+0x154 to 1. Twelve seconds later the game had put
0x447A0000 back.
That establishes an asymmetry worth having: the hull word is continuously
rewritten by the game, while the unit-record fields held our value untouched for
60s. It separates fields the game maintains from fields nobody writes.
But it is not yet a control. The after-frame shows a red WARNING banner -- and
the before-frame already shows MISSILE ALERT, so the ship was under attack in
both and the banner is not attributable to the poke. A value being overwritten
proves the game writes that address, not that it read ours.
Correction: hull at pos+0x154 is 0x447A0000, a FLOAT 1000.0, not the 1500 the
pilot logs -- those are different fields or scales and should not be conflated.
The settling refinement: poke in a tight loop for several seconds so the value is
low whenever the game samples it. If hull is authoritative the ship dies and the
screen goes to GAME OVER; if it survives, the field is a readout.
The reusable part is the harness: boot -> verify animating -> locate -> act, with
a freeze at any step costing one retry rather than the iteration.
Ran the direct test instead of a seventh attempt at winning. All three objective
squadrons were live (state 2); the write to +16 sticks, and 60s later with all
three reading 4 -- the value a naturally-destroyed squadron takes, measured on
ADN111 -- [ScriptPhase+196] is still 0 and the ordinal still 1.
So 'phase 1 clears when ADN110/111/112 are destroyed' is not confirmed and its
simplest form is refuted. The bytecode reading (three unit_state polls then
set_flag(8)) stands; what does not follow is that flipping the field equals the
kill.
The persistence is the clue: built-in 69 normalises +16 when it polls, so a
running condition coroutine should have overwritten the poke within a frame. It
did not, which points at the condition being evaluated only when a trigger fires.
Also corrects the per-unit record layout: +4 is 26/27/28 for the three
squadrons -- small consecutive integers, NOT the 'live object pointer' the
built-in summary describes (an undeployed squadron has +4=0). +20 = 9 is exactly
their member count n from the roster, so the record is per-squadron and carries
its strength. My own probe printed 'obj=yes' by testing that word for non-zero
rather than pointer-ness, which made an index look like an object.
Bounding the pointer scan to 0xBC000000-0xBD000000 (with a full-sweep fallback)
drops find_mission from a ~371MB walk to 0.7s. The run then went 694s with the
probe attached and NO freeze, against 3-of-3 frozen inside ~4 minutes with the
unbounded version. n=1, but the first probe-attached run to survive.
State encoding pinned to three points: 1 = not yet deployed, 2 = active,
4 = destroyed. ADN111 caught going 2 -> 4 at 433s while the active count fell
36 -> 27.
The phase ended at 694.9s WITHOUT the ordinal advancing, and every field matches
the branch read statically from sub_82260710: [phase+300]=2 (last-phase flag),
[mission+20]=0 (mission-over state), [phase+196]=1 (finished), [mission+40]=1
(unchanged). The static state machine is confirmed on the live oracle for the
mission-over half.
But this was a LOSS, not a clear: GAME OVER on screen, escort at 35.7%, pilot
DEAD at 676s, and two of the three objective squadrons still at state 2. So the
'destroy all three clears phase 1' prediction remains untested. What is
established is that the else-branch is the only route to phase 2 and needs
[phase+300] != 2 when the phase ends.
Five attempts, still no phase advance observed -- the obstacle is now keeping the
escort alive, not the freeze or the instrument.
A pilot-only run froze at t~150.7s (frozen.py: max_pixel_delta=0), found by
accident when the sweep-free experiment aborted at startup and the run flew with
no script probe attached. The tally is now 3-of-3 frozen with the probe versus
1-of-3 without, not 3-versus-0. Still a lean, but not the clean separation the
previous entry claimed, and marked down accordingly -- the fourth time a freeze
conclusion here has had to be softened by one more run.
The sweep-free test itself could not run: ScriptMission is re-allocated per run,
so the address from an earlier run (0xBC7A2A20) read back all zeros. The cheap
self-consistency check ([m+44] must equal the phase's [+244]) rejected it
instead of reporting garbage, which is the part that worked.
Names the cheaper replacement: a BOUNDED pointer scan. Every ScriptMission seen
so far sits in 0xBC79xxxx-0xBC7Axxxx, so ~32MB instead of ~371MB would cut the
sweep cost roughly tenfold. Not yet implemented.
Watching [ScriptMission+40] and the three phase-1 objective squadrons together:
all of ADN110/111/112 flip state 1 -> 2 at ~143s, while records in state 2 climb
24 -> 35 over four minutes.
So state 1 means 'not yet deployed' for these, not 'gone'. The built-in table's
'1/3/4 = gone/dead/invalid' shorthand is incomplete, and reading state != 2 as
destroyed would have been wrong exactly as flagged last iteration.
This also answers a much older question: mission-arrival-watch.md and the wave
work recorded '0 confirmed arrivals' across many runs by watching the CRAFT
population. The script's own unit table shows arrivals plainly -- eleven records
enter state 2 within four minutes. The old negative measured the wrong
structure; craft counts conflate deployment with attrition, the per-unit state
field does not.
Both attempts froze (at ~70s and ~253s), so no phase advance was reached. The
freeze witness caught both immediately, which is why the truncation is visible
instead of a silently flat line.
New harness tools/re-capture/phase_watch.py.
Unblocks the phase experiment, which was stuck because '38 enemies died' could
not say whether the right ones did. Chasing craft->squadron was the wrong angle:
the script VM keeps that table itself, indexed by the .ssb symbol-table-2 index.
Route: find the .ssb header in guest memory (0xAB840010 for a Stage 02 run),
code base = filebase + 0x24, scan for a word equal to it, then VALIDATE
arithmetically -- [ScriptMission+44] must equal filebase + symtab1 offset + 4.
Measured 0xAB874C94, predicted 0xAB874C94, exact. A second candidate that also
pointed at the code base failed that check and was discarded; without it either
would have looked plausible.
ScriptPhase+324 -> +4 is an array of 122 per-unit records -- exactly the size of
Stage 02's symbol table 2, an independent confirmation of the index space.
[ScriptMission+40] reads 1 in a phase-1 mission. The mirror at
[*(0x828F35F8)+236] that three earlier runs polled reads 0, because ChangePhase
only posts once the ordinal exceeds 1 -- so +40 is the real counter and is
reachable from /dev/shm with no debugger.
Flagged rather than asserted: the three objective squadrons read state=1 with a
LIVE object pointer in a mission where nothing has been shot, which does not fit
the built-in table's '1 = gone'. Reading state != 2 as destroyed would be a
plausible-but-wrong inference; the encoding needs pinning first.
New tool tools/re-capture/squadron_state.py, verified end to end against the
manual reading.
SYLPH_PREFER=Turret, escort at 100%, 1070s, phase field 0 throughout, 4029
Turret-targeting pilot samples. It looked like a clean negative -- 'hunting the
objective squadrons does not advance the phase' -- and it is worthless:
frozen.py reports max_pixel_delta=0, and the pilot's first sample at t=0.0
already has spd=0 with the same yaw/pitch/target/d=7186 it still had at 1070s.
The mission froze on entry to flight and nothing was ever shot, while screen_id
said 'flight' the whole time.
I was one step from writing this up as a fact about the game; running the freeze
test rather than trusting a plausible log is what caught it.
phase_probe.py now calls frozen.py every 60s and prints a GUEST FROZEN banner
inline, so a dead-world reading is labelled in the data instead of discovered
later. Note frozen.frozen() returns a TUPLE (is_frozen, max_delta) -- testing it
directly is always truthy and would have made the witness fire on every check.
Verified against the frozen guest.
Also recorded: the pilot log is itself a freeze witness -- identical
yaw/pitch/target across thousands of samples is a dead world, not patience.
Resolving symbol-table-2 indices turns the bytecode into mission logic. At
0xF524 Stage02.ssb polls unit_state on ADN110, ADN111 and ADN112, updates each
one's objective marker, then latches set_flag(8) -- exactly the
trigger/predicate/set_flag/END_PHASE shape predicted from the disassembly, now
observed in the mission's own code with names the roster tables already gave.
The 12 END_PHASE sites are outro sequences (wait_cmds_drained / fade_sound(3) /
builtin85(3) / wait_s(3) / END_PHASE / yield) -- the terminator, not the
decision.
Fixes a decode bug that hid every argument: the tracker only followed
local[i] = special[0], but the common form is an immediate written straight into
local[i] (k=01,03), so every unit predicate printed with NO arguments. The
disassembly looked complete while being empty exactly where it mattered.
Also records the live probe result: the phase mirror at [*(0x828F35F8)+236]
stayed 0 for ~530s of actively-hunting flight, no advance observed -- which is
what the static analysis predicts for phase 1, since ChangePhase only posts once
the ordinal exceeds 1.
Table at 0x8227226C is 147 big-endian absolute VAs (verified structurally: it
ends exactly where the first handler begins, all targets inside sub_82272220).
Arguments are not in the instruction -- every handler does c_str() on
[phase+20], a packed blob, which is what the local[] staging fills. Return 2 =
yield; five built-ins block by skipping the pc advance.
Recovered the ScriptPhase state layout: 32-entry float and flag register files,
int/double result registers, the timer block, and the runtime unit array at
+324 indexed by symbol-table-2 index -- a direct hook from bytecode call sites
to the two .ssb symbol tables.
Spot-checked two claims against the disassembly rather than trusting them: id 4
loads a DOUBLE into the thread countdown and returns 2 (wait_s), and id 24 reads
current/initial squadron member counts (squad_survival_pct). Both exact.
Counting Stage02.ssb: unit_state 255, hp_pct_test 167, dist_lt 92, unit_alive
71, unit_relation 52 -- and squad_survival_pct, group_ratio_pct and the two
global counters are NOT called at all. So Stage 02's phases are gated on named
units (destroyed / HP / proximity), never on an aggregate count, even though the
kill-counter primitives exist in the VM. That answers the standing 'next wave
after N kills or after an event?' question for this stage: specific units, not a
number.
isl.py now names the built-ins, so the run-up to the first END_PHASE reads
wait_cmds_drained / fade_sound(3) / builtin85(3) / wait_s(3) / END_PHASE.
Not settled: 3 handlers unresolved (55, 75, 105); the 1024-slot interpreter
command table is only partly recovered.
Resolver table 0x82271D74 gives four kinds: 0 global[i], 1 immediate,
2 special[i] ([phase+164]/[phase+168]), 3 local[i] ([phase+20+i]). Byte[0] is
the rvalue kind, byte[1] the lvalue kind, so the recurring instruction pair is
argument staging -- values land in local[] at offsets 0,4,8,0xC and the next
call consumes them. A built-in's arguments are not in its own instruction.
Fixed a decode that would have been believed: immediates in set.f are DOUBLES
carried as two words (op 1 stores with stfd). Reading the high word as a float
gives 2.125 where the script means 3.0.
isl.py now tracks staging and prints call arguments, so the run-up to the first
END PHASE in Stage02 reads as builtin=64(0x42,2,1,9,1,-1) / 120 / 59(3) / 85(3)
/ 4(3) / 6. Three built-ins taking 3 just before the phase ends look like a
wait-seconds family -- flagged as unconfirmed until the built-in table is read.
Read the encoding off the interpreter rather than guessing: instruction is a
big-endian u32 whose LOW byte is the opcode (25 of them, table 0x822635FC),
byte[2] is the instruction length -- every handler advances the pc by it -- and
bytes[0..1] are operand kinds. Op 12 is a jump whose operand is relative to the
code base [phase+232], which settles that offsets are code-base-relative for
this opcode. Op 19 is the built-in call: id in word@+4, and word@+8 is a
monotonically increasing STATEMENT id (0x245, 0x248, 0x24A, ...).
Confirmed by disassembling Stage02.ssb: the stream decodes cleanly from the code
base and routines terminate on ret exactly where expected.
Scanning the code region on the call encoding: 2846 call sites, 73 of the 147
built-ins used. The phase-control ones are located -- built-in 6 (end phase) at
12 sites, 62 at 3, 39 (mark last phase) at 8 -- so a phase has several exit
paths, as a mission with win and lose branches should.
New tool tools/re-capture/isl.py with --calls and --to (resync-into-target,
needed because instructions are variable-length so you cannot walk backwards).
Not settled: the 147 built-ins are uncharacterised, so this is structure without
meaning -- we can see THAT a phase ends, not WHAT was tested.
Found the routines in the disassembly DB rather than guessing from data:
sub_82447DF0 IDXD tag hash (lbz+extsb, modulus 0x00FFFFDF, magic 0x2101)
sub_82447E70 IXUD tag hash (lhz, 64-bit, modulus 0xFFFFFF67 then 0x00FFFFDF)
Both transcribed instruction-for-instruction into Python and Rust.
IXUD SOLVED. It defeated every single-modulus search because it chains TWO
exact moduli -- the loop reduces mod 2^32-153 in 64-bit arithmetic and only the
result is folded mod 2^24-33. A polynomial mod M1 folded through M2 is not a
polynomial mod anything, which is exactly why the gcd test returned 1. Verified
independently: 86/86 record keys and 108,261/108,261 field tags in
GP_MAIN_GAME_E.pak, and NoRecord -> 0x1c6d9c96.
CORRECTION 1: tag_hash must SIGN-EXTEND each byte (extsb). My reconstruction
used unsigned bytes and matched all 1.27M disc names -- every one is ASCII --
while disagreeing on ~90% of random inputs with a byte >= 0x80 (verified:
18096/20000). The disc could never have caught this; only the disassembly did.
CORRECTION 2: name_hash's reduction is EXACT, not lossy. The module doc claimed
the missing conditional subtract made it something other than %. rlwinm r6,r6,
9,23,31 is just hi>>23, and with RECIP = floor(2^55/M)+1 that is Granlund-
Montgomery magic division -- 0 wrong at every quotient boundary across the full
32-bit domain. Retracted.
cargo test -p sylpheed-formats --lib hash: 10/10.
Closes the 4-byte record key. tag_hash is name_hash's shape -- byte-sum
checksum in the top byte over a 24-bit modular polynomial -- with two different
constants: modulus 0x00FFFFDF (2^24-33, prime) instead of 0x00FFF9D7, and no
lowercasing, so tags are case-sensitive. name_hash explains 0 of 8643.
Recovered from the tables rather than the executable: every inline field name
is a known (name -> tag) pair, and comparing names differing in one character
gives the per-position weights 1, 0x100, 0x10000, 0x21, 0x2100, ... -- a byte
leaving bit 24 re-enters as 33, i.e. reduction mod 2^24-33. Holds where it is
easy to get wrong (distance 8 and 9 carry correctly).
A record's key is the tag of its own name: FormationSet rosters 362/362,
UnitGroup rosters 281/281, S02 squadron names 111/111 -- so records can be
addressed by name without reading the roster first.
Implemented in Python (unitgroup.tag_hash) and Rust
(sylpheed_formats::hash::tag_hash) with 3 new unit tests carrying disc-derived
vectors; cargo test -p sylpheed-formats --lib hash is 8/8 green.
Not settled: the guest routine is unlocated, so this uses exact modular
arithmetic where the game may use a Barrett step without final fixup.
FormationSet_S<NN>.tbl records are slot lists -- 1 + 8*FrameCount fields,
exactly. Resolving every squadron's FormationID and comparing gives
sum(n) <= FrameCount holding 1159/1160 across all 28 stages, 0 unresolved, with
539 filling the formation exactly. The single violation is a debug leftover
(S20, AI_Test / MessageSet_test, Formation_1_only with n=2) and is recorded.
The old 'n is not the _NN suffix of FormationID' observation was right but drew
the wrong conclusion: the suffix IS FrameCount, so n=9 against _30 just means 9
units in 9 of 30 slots.
Also: FormationID does not hash into its table (0/16). FormationSet carries a
name roster record -- no FrameCount, fields are (tag, name, '') with the tags
being the record keys -- the same convention as Enumerate_Squadrons. Second
occurrence of 'keys are resolved by an in-table roster, not by hashing'.
Does not close the 387-vs-300 gap, and the key derivation stays open.
Caught the freeze by waiting for the event (frozen.py + in_flight) instead of
sleeping a guessed interval; freeze_waitobj.sh splits into boot/watch so the
wait is not capped by one Bash call. Verified hard: a frame minutes later is
byte-identical to the capture.
Healthy vs frozen, same run: 20 -> 24 wait frames, XEvent 19 -> 23,
XSemaphore 8 -> 7. The signature is per-thread -- 17 of 24 threads sit on the
exact object they were on, four previously-running threads park, and T74/T75
move off a semaphore onto an event. So the freeze is not a whole-emulator stall.
Also corrects the previous entry's test: screen_id reads 'flight' during a
freeze by design, which is why frozen.py exists. Re-testing the saved frames
says that run was genuinely healthy, but it was right by luck.
heavy_read.py added to test whether the instrument provokes the freeze: I/O is
free (371 MB in 0.1s, page cache), the cost is Python-level CPU. One data point
-- 670s clean, then frozen 54s after the inducer started -- recorded as n=1, not
as causation.
23 wait frames, 30 objects, nothing unresolved -- the second deref turns every
former miss into a resolved object, as predicted. XEvent 20 / XSemaphore 9 /
XTimer 1; every WaitMultiple thread waits on a pair, and 78/79/80 and 64/65 are
worker groups sharing a handle.
%ebp does not survive as the count -- WaitMultiple reuses it at 8fc158 -- so the
array is bounded by reading until an entry stops resolving instead.
The frozen capture is still not taken: screen_id reads 'flight' at the second
capture and out to ~470s, so the mission never black-screened. The diff in the
data file is two healthy captures and is recorded as such.
The 8 threads whose [rbx] did not resolve to a vtable were never in
XObject::Wait. The backtrace grep matched WaitMultiple as a substring, and
there %rbx is the XObject** array (mov %rsi,%rbx) with the count in %ebp, so
[rbx] is objects[0] -- an object pointer, needing a second deref -- not a
vtable. The unwind restored rbx correctly for all 18.
freeze_waitobj.sh now takes the function and frame index from the backtrace and
applies the matching read, and captures twice in one run (healthy and after the
~270s black-screen) so the comparison is within-run. waitobj_report.py tabulates
both and diffs them, discarding any value info symbol cannot resolve.
Re-extracting the same gdb capture per thread rather than by grep qualifies the
previous entry. Of eighteen threads whose frame 3 is XObject::Wait, eight have
[rbx] equal to the XEvent vtable plus sixteen, two equal to the XSemaphore
vtable plus sixteen, and eight hold a pointer into the mmap region that is not a
vtable at all. So the waits are on two distinct kernel types, XEvent and
XSemaphore, and the earlier claim that the object is an XEvent was right for the
majority but not the whole picture.
The eight non-vtable readings are the method checking itself rather than a
failure. A polymorphic object's first word is always a vtable pointer, so those
values are simply not this: rbx could not be restored for those frames and the
unwind returned whatever the register held. A reading counts only if [rbx] lands
in the binary's vtable range and resolves to a "vtable for" symbol; anything
else is discarded rather than interpreted. Ten of eighteen resolve and the rest
are honestly unknown.
That also settles the previous entry's worry that the 0x7ffc and 0x7ffd
addresses looked like stack. They are the shared mmap region, which holds thread
stacks and large allocations alike, so the vtable check rather than the address
range is what separates an object from a stack slot.
The follow-up run that would have added /proc/<pid>/maps classification and a
wider object dump never booted -- EMULATOR GONE at 0s, skip_intro exit 4 --
most likely a stale emulator or lockfile from the preceding gdb session, whose
process tree is parented differently and escaped the usual cleanup. So the map
classification, the multi-word object dump and the frozen-state capture are all
still unrun.
Executed the wait-object read on a live run under gdb. For every thread whose
frame 3 is XObject::Wait, rbx gives a pointer whose first quadword is
0x5555562db8f0, which is the PIE base plus 0xd878f0, and the symbol table has
vtable for xe::kernel::XEvent at 0xd878e0. The stored pointer is the vtable
symbol plus sixteen, since offset-to-top and RTTI come first, so it matches
exactly. The same vtable appears on every sampled waiting thread while the this
pointers differ, meaning many threads waiting on different XEvent instances. The
whole chain needs no DWARF and no rebuild, as the static groundwork predicted.
Two caveats, both recorded rather than smoothed over.
This is a healthy-play snapshot, not the freeze. The capture landed at 195 s of
flight with screen_id reporting flight and a non-black mean, because boot under
gdb costs about 300 s and the entire experiment has to fit inside one call --
a timeout kills the process group and takes the emulator with it, which lost an
earlier attempt outright. So this describes what threads wait on during normal
play, which is the control the frozen capture never had, but it is not the
frozen case.
And the this addresses look like host stack rather than heap. Either xenia
places these objects somewhere unusual, or rbx at frame 3 is not Wait's this
after the unwind and the vtable match is coincidence. An exact plus-sixteen
match on a known symbol is hard to get by accident, but the address range is not
what was expected. The settling check is to dump a few words at rbx and see
whether they look like an XEvent -- vtable, KernelState pointer, handle and type
fields -- or like saved registers.
A two-segment OB watch made the mechanism visible. Segment 1 ran clean, with the
witness reporting zero stalled samples of fifty and OB steady at 4 for 250
seconds. Segment 2, attaching to the same still-alive emulator, got "NOT IN A
MISSION (no unit definitions)".
The emulator was alive; the mission was gone. The pilot's hull reads 1500 in
every line, so the player never died. Its log stops at t=267 s with consecutive
byte-identical lines, the freeze signature. And the screen is now entirely
black, mean (0,0,0). This is the pre-existing mission-end black-screen freeze
already recorded in the corpus, caught in the act. Segment 1's clean witness is
not a contradiction, since it sampled to about 250 s and the freeze came at 267.
That corrects an earlier claim. mission-arrival-watch.md records the blocker as
removed because the emulator survives between calls, which is true, but it was
stated as though observation could be extended indefinitely within a turn. The
emulator surviving is not the mission surviving. The binding limit is the game's
own black-screen at roughly four and a half minutes and no amount of chaining
crosses it. The earlier chained run that reached 435 s of cumulative
verified-live observation did so across two segments of one mission, which fits:
its second segment froze at t=135 s.
The consequence is worth stating plainly. Any experiment needing more than about
four and a half minutes of one continuous mission is not doable on this build,
chained or not, and that includes clearing sixteen marked fighters at two kills
per five minutes. The freeze is therefore now the highest-value target in its own
right, since fixing it would unblock the arrival question, the phase-advance
question and the OB series together. It is also a long-standing known defect
rather than anything this work introduced.
ob_watch.py verifies the address against a confidence-gated HUD reading before
reporting anything, and two consecutive fresh launches settle the question:
0xbdb59668 held 3165285888 against a HUD of 4 on one, and exactly 4 on the next.
The address is not stable across launches, the old note that it recurs in about
five runs of seven was right, and the gate did its job by refusing to report a
series from an address that did not describe that run. The watcher now hunts the
address on the current run when confirmation fails, using the same intersection
method, so it no longer depends on a lucky launch.
The HUD reader is also gated now. ob_read returns a best and second score per
digit and those were printed but never checked, which is how one misread
poisoned an intersection and produced a wrong refutation of big-endian u32. A
reading is accepted only if every digit scores at least 0.80 with a margin of at
least 0.05, the rule ob_read's own docstring states.
The measurement itself is a negative. With a clean witness, zero stalled samples
of fifty, OB held at 4 for 250 seconds while the pilot targeted e010 for 1964
ticks and fired on 1635 of them. Constant fire at the marked attackers and not
one decrement, so it destroyed none, which matches the roughly two marked kills
per five minutes measured earlier. The fire rate itself rose from 4.6 % of ticks
in an earlier diagnosis to 83 % here without producing more kills.
Recorded as unreproduced rather than explained away: the run that found the
address saw the counter rise 4, 8, 12 over five minutes, and that reading was
confirmed against the HUD. This run was flat over a comparable window. Both
observations are sound and they disagree, so the rise is not a stable property
of the mission's first five minutes and presumably depends on progress this run
never reached.
Ported onto the shared harness, the HUD changed from 4 to 8 and the intersection
collapsed in a single step: one u32be survivor at 0xbdb59668, with the u16be and
u8 hits at 0xbdb5966a and 0xbdb5966b being the low half and low byte of that same
word.
Verified live rather than asserted. Reading screenshot and memory together three
times, the one legible HUD frame showed 012 against mem@0xbdb59668 = 12; the
other two frames were unreadable rather than mismatched. This independently
rediscovers the address the earliest sessions found by digit-transition hunting,
by a completely different method.
It also corrects an earlier conclusion. A previous run intersected HUD readings
of 4 and then 11, got zero survivors in every encoding, and that was written up
as eliminating big-endian u32 for the whole region. This run shows u32be holds
the counter, so the refutation was wrong. The likely cause is the input:
ob_digits.png has templates for 0 1 2 4 8 only, so values containing other
digits are misread rather than rejected, and "11" was probably one of those. A
single bad reading poisons an intersection permanently, because it removes the
true address and nothing later can restore it. The lesson is that an
intersection method needs individually verifiable inputs -- the reader's
confidence scores were printed but never gated on.
One observation reopens the arrival question in a useful way: the counter
increases, 4 then 8 then 12 across about five minutes, measured in memory so not
a digit misread. A count of remaining marked targets that rises means targets
are being added during the mission. That does not contradict the deployment
finding, since the roster is fixed at load, but it does mean the game marks new
objective targets as the mission proceeds. Watching this one address across a
whole mission is now the obvious next experiment and costs almost nothing.
Four probes were written from a blank file and each re-learned the same lessons
by losing a run: that a flat run cannot be told from a frozen guest without a
stall witness, that results held to the end of a run are destroyed by a turn
timeout, that a roster count which is not the stage's member count means a
different stage loaded and must be discarded, and that a run's witness state has
to be read before its numbers. Writing each lesson down did not stop the next
probe repeating it, because each probe started from nothing.
probeharness.py makes them structural. Probe(baseline=N) discovers the roster,
rescans up to five times and refuses to start if the count never reaches the
baseline. The witness is calibrated on construction, sampled by tick() and
reported by status() and summary(), so a probe cannot forget it, and when no
witness is found it reports UNVALIDATED rather than zero stalls. emit() flushes
on every line. craft(), strengths(), alive() and heap() supply the
roster-to-craft link, per-record liveness and the raw heap, so a new probe
writes only its own logic.
Verified rather than asserted: deploy_probe.py reimplements the per-record
deployment watch on top of it in about forty lines against wave7_probe's
hundred and fifty, and its first live run was clean -- 116 roster records, 32
witnesses at 10/s, zero stalled samples, seven losses tracked, and the TSV
written incrementally. Nothing about the result is new, which is the point: the
harness reproduces a known-good measurement.
The existing probes are deliberately not ported. They work, and rewriting them
would risk changing results other documents cite. New probes should use the
harness; old ones should be ported when they next need a change.
ob_by_hud.py now scans seven readings of the same bytes and keeps a separate
candidate set for each, as byte offsets: u32 big and little endian, u16 both
endiannesses at both alignments, and u8. The big-endian u32 reading had been
refuted, so widening rather than assuming is the point.
u32le is much the tightest at 154 candidates against u32be's 4452. That is a
hint about the encoding rather than a result, since a rarer bit pattern narrows
faster regardless of meaning.
The run is inconclusive. The HUD read 4 at every sample, so there was no second
value to collapse the sets against, and from t=136 the candidate counts are
byte-identical across five samples in all seven encodings, which is what a
frozen guest looks like -- nothing in 32 MB changed at all. The probe had no
stall witness, so the run cannot prove it either way. One is added now.
Worth stating plainly: this is the fourth probe written without a witness and
the third whose flat output could not be distinguished from a freeze. Each time
the fix gets applied to that one script. The durable fix is the shared probe
harness already noted in this file, and the lesson recurring four times is
itself the argument for building it.
What the hunt needs is unchanged: two HUD readings at different values in
non-stalled samples. The counter moves on kills, which lands back on the combat
limit, though the earlier 4 to 11 observation shows it does move.
The correlation route is gated on marked-fighter kills, which the pilot manages
at about two per five minutes. ob_read.py already reads the counter off the
screen, so ob_by_hud.py matches the displayed value against memory directly and
needs no kills at all: screenshot, read the digits, keep heap words equal to that
value, intersect across readings.
Four readings at value 4 narrowed 6156 candidates to 4312, the expected slow
drift. Then the HUD read 11 and the intersection collapsed to zero. A word
holding this counter must equal 4 at the first four samples and 11 at the last,
and none does, so within the entity heap read as big-endian u32 the counter does
not exist. It may be u16, u8, little-endian, or outside that region. Both
previous hunts assumed big-endian u32 there, so this eliminates the assumption
rather than merely failing to find anything.
The displayed value also went up, from 4 to 11 over about 340 seconds. A pure
countdown of remaining marked targets should not rise, and the deployment work
says phase 1 gains no new participants. Three readings are possible and none is
tested: the cell being read is not REMAINING OB, the digits are misread, or the
counter genuinely counts something that can increase. The two clean readings
scored 0.95 to 0.98 against their templates, but 4 and 11 use only digits that
are in the strip, which is exactly the selection effect that would hide a wrong
reading -- the template set covers 0 1 2 4 8 only, and most samples came back
unreadable.
Next is widening the scan to u16 and u8 and to little-endian, and beyond the
entity heap, which is a change to one function and costs no combat. Extending
ob_digits.png with the missing digits would also raise the sample yield, since
only two of eleven readings in a 480 s run were usable.
The incremental-save fix is verified. A fresh mission caught one e010 event at
t=241 s and wrote 1187 candidates to disk immediately; the turn timeout then
fired exactly as before, but this time the data survived. The session also
clears the candidate file at launch, since candidate offsets are only meaningful
within one emulator instance and resuming across launches would intersect
unrelated addresses.
The correlation itself was wrong though. It matched on the delta alone, so any
two float bit patterns whose integer representations differ by the loss count
qualified, and in a heap full of positions and velocities that is thousands of
words. The 1187 survivors were things like 1044450858, about 0.1f, and
3212461993, a negative float. Candidates must now also look like a counter --
a small non-negative integer in both samples -- which removes the noise by
construction instead of hoping the intersection washes it out.
The follow-up attach logged zero events across 520 s, which reads like the
combat-effectiveness limit again. It was not: 25 of its 26 samples were flagged
GUEST STALLED, so the guest was frozen for essentially the whole window. The
witness added last iteration did its job, and the lesson is about reading it --
the run summary quoted "0 events" first and the stall count only surfaced on a
deliberate check. A run's witness result should be the first thing looked at,
before any interpretation of what the run showed.
Still unfinished, with no address identified. What is needed is unchanged, two
or three e010 kill events in non-stalled samples, and the two obstacles are now
clearly separate: the freeze rate, and a pilot that manages about two
marked-fighter kills per five minutes.
The objective text settles what the counter is, so the hunt can be a correlation
rather than a value scan: keep every word in the 32 MB heap that fell by the same
amount, in the same interval, as a named e010 loss. One event cut roughly eight
million words to 1056, so two or three more should leave a handful.
The run did not get them. The turn's timeout fired at t=219 s and the probe saved
its candidate set only at the end, so the 1056 were discarded and the follow-up
attach started from nothing. That is the same mistake already recorded in
guest-stalls.md, where an earlier probe deferred all analysis to the end and a
timeout killed it with 240 s of data in memory and nothing written. The lesson
was written down and then repeated in a new script four iterations later.
The attach had a second gap: 535 s with zero losses of any kind, which is
indistinguishable from a freeze, and ob_probe2 carried no stall witness so the
run cannot say which it was.
Both are fixed. Candidates are written after every event and SYLPH_OB_RESUME=1
reloads them so a chained attach keeps intersecting on the same mission, and the
witness from wave7_probe is carried here.
The underlying pattern is worth naming: each new probe starts from scratch and
re-earns the same lessons about saving incrementally and validating liveness. A
shared probe harness would stick where written-down lessons have not.
The hunt itself is unfinished. No address is identified, and finishing needs a
run that catches two or three marked-fighter kills, which is the same
combat-effectiveness limit already recorded -- about two per five minutes against
a dozen turrets.
pilot.py gained SYLPH_PREFER, a unit-name substring whose matches get their
target score multiplied by 0.05 while everything else is multiplied by 4.0. With
SYLPH_PREFER=e010 a clean 320 s run, zero stalls by the witness, killed eight
turrets and two Attacker_S.
The preference is real -- e010 kills went from roughly one across all previous
runs to two in a single run -- but it is weak. Turrets still outnumber attackers
four to one in the kill log, because target commitment and simple proximity keep
pulling the nose back to them, and phase 1 fields 108 turret craft against 16
attackers. Deployed stayed at 41 throughout, so no phase advance.
That quantifies the blocker. Clearing the marked attackers means destroying 16
craft, and at two per 320 s that is about 2560 seconds, roughly 43 minutes of
continuous verified-live flight across many chained attaches, against a freeze
rate of about two runs in five.
This is no longer a reverse-engineering problem. Everything needed to observe
the phase advance is built and validated -- the roster-to-craft link, the
liveness read, the stall witness, chained attaches and the discard rule. What is
missing is a pilot good enough to complete the mission objective, which is
game-playing work with an uncertain payoff.
The choice is recorded rather than made, because it is about how much effort one
confirmation is worth rather than a technical unknown: invest in the pilot,
accept the static answer where only the trigger is inferred rather than
observed, or attempt one very long chained run betting against the freeze rate.
early_probe defers everything expensive: no witness calibration, no per-record
labelling, and the heap scan uses bytes.find. Setup now completes in 0.5 to 0.8
seconds instead of about 25, so the first sample lands essentially at flight
detection.
It is still flat. Deployed reads 41 at flight+0.8 s and every sample after, with
the only change in 252 s being 41 to 40 when one squadron was wiped out. No
climb at any point. Under the frames reading that is expected, since the whole
phase-1 schedule would be finished within eight seconds of mission start.
The attempt to get ahead of flight detection did not work, and the reason is
worth recording. A second run started the probe before the launch and waited for
the roster to appear. It appeared with 116 records, but the numbers were
deployed 39 and craft 276, flat for the whole window. That is the ready room:
the roster is built before take-off, so waiting for it catches the pre-flight
scene, and the probe's window expired around the time flight actually began.
Waiting for the roster is not the same as catching mission start, and the test
as designed does not do what it claims.
The two runs together do suggest something, held at 🟡 because they are
different runs. The ready room shows 39 deployed and 276 craft while flight
shows 41 and 300, so two records and twenty-four craft appear between them. That
points at deployment being a single step at take-off rather than a schedule
unfolding during the mission.
Next is one run of about 400 s with the probe waiting for the roster, long
enough to span ready room, take-off and flight in a single continuous series,
which would show the 39 to 41 step directly or refute it.
The previous iteration declared route entries t=210 and t=240 unreachable
because a shell call is capped at 595 s. That was wrong, and it rested on an
assumption I never checked: launch_mission.sh leaves the emulator running and it
survives between Bash calls within a turn. Verified directly -- 611 s elapsed
and still running after the launching call had returned.
attach_session.sh attaches a pilot and probe to an already-running mission, and
attaches chain, so the window is bounded by the turn rather than by one call.
On one continuous mission: 300 s of clean observation followed by an attach that
ran 135 s before freezing, giving 435 s of cumulative verified-live wall-clock.
The craft population fell from 300 to 258, so 42 were destroyed, and deployed
fell 41 to 38. Zero confirmed arrivals throughout -- the longest verified-live
observation so far. That covers the first four phase-1 route entries, t=90, 120,
170 and 210; t=240 needs 436 s and was missed by a second.
The coverage claim is qualified rather than asserted. The wall-to-game
conversion uses 55 %, from a 16.5/s frame rate against an assumed 30 Hz tick,
but the witness has measured 8, 11, 11, 21 and 24 per second across runs. At the
low end the factor is 0.27 and the same 435 s covers only about 117
game-seconds, reaching t=90 alone. So what is established is that nothing
arrived in 435 s of verified-live phase-1 flight with 42 kills; how many
scheduled arrival times that spans depends on a tick rate that is not yet
pinned, and pinning it is what would make the result sharp.
Also refuted: SYLPH_HZ=3, a lower pilot poll rate, produced the lowest
calibrated frame rate of any run at 8/s while landing the most kills at 26. The
pilot's polling is not the throttle and lowering it does not buy game time.
Correcting the previous iteration's wording. It claimed on n=1 that disabling
the periodic rescan "removes the freeze". With more runs that is too strong: the
configuration is now clean at 210, 240 and 300 seconds and frozen at 60 on a
fourth run.
The tally across configurations: the heavy probe froze at 27, 45, 83, 183 and
255 seconds; the cheap probe with a 90 s rescan froze at 183; the cheap probe
with no rescan is clean in three runs of four past 200 s. That is a real and
large improvement in survival, but it is a change in probability rather than a
fix, and the documentation now says so.
The witness handled the frozen run correctly -- losses stop at t=45 and stalls
are flagged from t=60 onward, with no contradiction between them.
Separately, the candidate search was the last Python loop over eight million
words and most of the remaining startup cost. Replaced with two numpy vector
operations; 7250 candidates found and startup is no longer the bottleneck. Boot
now dominates: about 190 s of title movie plus 35 s to flight against a 595 s
turn cap leaves roughly 350 s of observation, so boot is the only remaining
lever if longer windows are needed.
Practical rule recorded: do not treat a single frozen run as evidence. Run,
check the witness, discard the frozen ones and keep the clean ones -- about
three in four are usable now, which is workable where it previously was not.
Second consecutive run with no stall flag on any sample, which confirms that
disabling the periodic rescan is what fixed the freezes. Nine losses, four
flicker increases all correctly rejected, and zero confirmed arrivals. The
trustworthy negative now extends to 240 s of verified-live flight, roughly 132
game-seconds, so nothing arrives past the route table's t=90 or t=120 entries
either.
The flicker rate is worth noting: about one spurious increase per minute. That
is the rate at which the old rule would have been manufacturing arrivals.
The arithmetic of the run exposed a waste. Boot finished at 249 s, the probe ran
240 s, and the turn's 595 s cap fired, leaving about 100 s unaccounted for
between them -- the witness calibration and the initial craft enumeration.
enumerate_craft was iterating every 4-byte word of 32 MB in Python, eight
million steps, to find fourteen fixed needles. Replaced with bytes.find() per
definition VA, which is the same search at C speed and is what the vtable scan
already did. Not yet run.
If that recovers most of the hundred seconds the observation window grows from
about 240 s to about 340 s, roughly 187 game-seconds, which would finally reach
the t=170 route entry -- the first of the schedule's later arrivals that no run
has yet been able to observe.
Two changes this iteration, one failed and reverted, one that worked.
Trimming the witness calibration to eight spread 512 KB windows instead of the
full region found 17 candidates, none of them frame counters, and the witness
then reported 0 of 17 stalled on every sample of a run that recorded 13 losses.
That is a total contradiction, caught by the same internal check that exposed
the previous three witness failures. The frame-rate cluster is sparse and spread
sampling misses it. Reverted: two 32 MB reads once at startup is simply the
price of a witness that works.
The recurring cost was the periodic rescan, a 32 MB read every 90 to 180
seconds, and it exists only to catch craft appearing from nowhere -- which the
roster work already established does not happen, since every participant is
allocated at mission load and an arrival is a state change rather than an
allocation. Disabled.
The result is the first fully clean probed run: 3875 candidates, 32 witnesses at
11/s, no stall flag on any sample from t=0 to t=210, and eight losses spread
across it. Previous probed runs froze at 27, 45, 83, 183 and 255 seconds. This
one ended on the turn timeout. One run, so not proven, but together with the
clean no-probe control it points at recurring heavy reads rather than at memory
reading as such.
That also produces the first arrival result that means what it says. Every
earlier one carried a caveat -- a stalled guest, an unvalidated witness, a probe
degrading what it measured. This one has a validated witness reporting no
stalls, a demonstrably live guest, and a clean end: zero confirmed arrivals over
210 s of verified-live Stage 02 flight, roughly 115 game-seconds, while the
player destroyed eight craft.
It does not settle the question. The route table's t = 170, 210 and 240 entries
remain out of reach in a single turn. But it does establish that nothing arrives
in the first ~115 game-seconds of phase 1 under those conditions, which none of
the previous fifteen runs could honestly claim.
Fourth attempt at the witness, and it passes the check the previous three
failed. Searching the whole 32 MB region rather than a 4 MB window gives 6500
candidates and 32 witnesses in a clean 21/s cluster matching the measured frame
rate, and the flags have zero contradictions with the loss data: the single loss
falls in an unflagged sample and every flagged sample has none. The guest
genuinely froze at about 45 s and the instrument says so.
That leaves the freeze itself as the dominant problem. Onset across runs is 27,
45, 83, 183 and 255 seconds, median around 83, so the usable window is one to
four minutes and unpredictable. This run produced exactly one loss and is
useless for the arrival question even though the instrument finally works.
Fifteen runs have now gone into that question without an answer, largely for
this reason.
Worth being honest about the probe: "cheap" was never that cheap and the witness
made it worse. It still reads 32 MB to enumerate at startup and another 32 MB
every 90 s to rescan, and the full-region calibration added two more 32 MB
reads. The 45 s freeze came immediately after that calibration. Calibration now
samples eight spread windows of 512 KB -- 4 MB total, still covering the whole
span -- instead of reading everything twice, and the rescan interval is doubled
to 180 s. Not yet run.
Also this iteration: the 42-record anomaly recurred, the discard rule correctly
refused to interpret the run, and a rescan-until-baseline retry is now in place
so the next occurrence shows whether the count climbs toward 116 or sits at 42.
Two harness problems, one fixed and verified, one diagnosed.
Bind: three consecutive attempts failed and the session aborted rather than
flying an unattended craft, which is last iteration's guard working as intended.
The cause is that entities2 self finds the player by motion between two samples,
so a craft sitting still at mission start is invisible. The session now holds the
throttle for three seconds before each attempt. Verified: the next run bound
immediately and produced 11 losses with zero confirmed arrivals, making fourteen
runs without one. One increase, 10 to 11, was surfaced and correctly not counted.
Witness: the frame-rate-band selection is still unreliable. It found only five
candidates in band and reported 0 of 5 moved in samples where craft were being
destroyed, and a counter genuinely running at 24/s cannot fail to advance across
a 15 s sample -- it would gain about 360. They are bursty counters that moved
during the 3 s calibration and then stopped.
The cause is mine. When the witness was bolted into wave7_probe the candidate
search was narrowed to a 4 MB window, while timer_probe searched the whole 32 MB
region and found 286 candidates with a clean cluster near 17/s. The narrowing
was never justified, it was just cheaper to write. Now searches the full region
once at startup, which is a one-off cost rather than per sample, and warns
explicitly when fewer than eight witnesses are available so a weak vote is
visible instead of silently trusted. Not yet run.
Worth recording: this is the third attempt at this witness, and all three
failures share one pattern -- a shortcut in selecting the witness rather than in
the voting logic. Each was caught only because the flagged samples contradicted
the loss events in the same output. That internal contradiction is the real
check and should survive into any future version.
The graded output is the improvement. Instead of a single bit it reports how
many witnesses agree, and the sequence tells a coherent story: 11, 9, 7, 4, 1,
then 0 of 31, with the drop to zero at t=183 s coinciding exactly with the last
loss and 106 s of nothing after it. That is a real freeze, identified.
The threshold was wrong though. Flagging a stall at "fewer than half" marked the
entire run stalled, including samples in which craft were destroyed, so 11 of 31
advancing is a healthy guest rather than a stalled one. The cause is the cluster
choice: the modal rate was 93/s, far above the ~16.5/s frame rate timer_probe
measured, and those are subsystem counters that tick in bursts and sit idle in
most 15 s windows even while the game runs. Picking the modal cluster was
convenient rather than principled.
Fixed to prefer the cluster whose rate falls in the frame-rate band of 8-40/s,
falling back to modal only if none exists, and to flag a stall only when zero
witnesses advance, which is the signal the data actually supports. Not yet run.
The uncomfortable part: this run used the cheap probe and still froze, at about
183 s. The previous iteration's "0 stalled samples" came from the unreliable
single-word witness and cannot stand as validation. What the evidence supports
now is that the no-probe control ran 300 s clean, the heavy probe froze at 27 to
255 s, and the cheap probe froze at 183 s -- one run on each arm. Cheap sampling
plausibly helps but does not remove the freeze, and it is equally possible the
freeze is stochastic and the control was lucky. Recorded as unresolved rather
than resolved in the probe's favour.
Practical consequence: the usable window is roughly three minutes per run,
sometimes less, whether or not the probe is cheap. Experiments needing longer
have to survive a freeze or be redesigned around one.
Run 12, cheap probe with a bound pilot: 16 losses over 290 s and zero confirmed
arrivals, making twelve runs without one. The persistence rule earned its place
immediately -- an increase of 13 to 15 was surfaced and correctly not counted,
since it does not start from zero. Under the previous rule it would have been
invisible, and a similar flicker straddling zero was nearly written up last
iteration as the first arrival.
The stall witness, on the other hand, is unreliable. Thirteen samples were
flagged GUEST STALLED while recording losses in those same samples, and a frozen
guest cannot destroy craft, so they are false positives and the run was healthy.
The cause is the selection rule: it took the first word in a 4 MB window whose
rate fell in a plausible band, and plenty of counters advance intermittently
without saying anything about whether frames are being rendered. timer_probe had
already solved this properly -- 286 candidates, a rate histogram with a dominant
cluster near 17/s -- and that lesson was not carried over when the witness was
bolted onto the probe.
Now fixed to a majority vote: collect every candidate, keep the modal-rate
cluster, sample up to 32 of them, and report a stall only when fewer than half
advance. It also prints RUN UNVALIDATED when no witness is found, because an
earlier run printed "stalled samples=0" alongside "tick witness: NONE", and a
witness that does not exist cannot report zero stalls. Not yet run.
Consequence worth flagging: the "0 stalled samples" that validated the cheap
probe last iteration came from this same unreliable witness and should be
re-confirmed under the majority rule. The pilot-log speed analysis that
established the stalls in the first place is unaffected.
Also fixed: the entity bind now retries three times and aborts if it never
takes, instead of silently flying an unattended craft -- one run was wasted that
way this iteration, producing no kills and no information.
First run of wave7_probe: zero stalled samples across the whole run, against
three consecutive heavy-probe runs that stalled at roughly 27, 83 and 255 s. The
fix works. The guest is also visibly healthier -- 19 losses against 8 in a
heavy-probe run of comparable length -- so starving the emulator had been
suppressing the very activity the probe existed to watch.
It also reported the first arrival of the whole line of work, and that arrival
does not survive inspection. A record went 1 -> 0 at t=229, 0 -> 2 at t=259 and
2 -> 0 at t=274. Two craft appearing and vanishing within fifteen seconds is not
a wave.
The same log contains the giveaway: at t=60 a record read 13 and at t=75 the
same record read 14, an increase, with nothing printed, because the probe only
surfaced decreases. The hull-based liveness read flickers, and a flicker that
straddles zero was indistinguishable from an arrival under the old rule. The
count therefore stands at zero confirmed arrivals in eleven runs.
Two changes, neither yet exercised: every increase is printed rather than only
those from zero, and an increase from zero counts only if it persists across two
consecutive samples, with a candidate that returns to zero discarded as flicker.
Recorded because it was close: under the old rule this run would have been
written up as "first arrival observed", which would have been the
strongest-looking result so far and wrong.
