From Canary's own source (x64_emitter.cc:881) GetContextReg() returns rsi, so at
any JIT instruction %rsi is the PPCContext* -- which is also why the faulting
instruction read 0x110(%rsi), a guest register load. That is the way past the
watchpoint's ceiling: the guest register file is available at the write, and a
0x82xxxxxx word picked out of it resolves against sylpheed.db to name the caller.
trigger_watch.sh now dumps x/128wx instead of a useless host backtrace.
The re-run then failed for an unrelated and unexplained reason: it reached
flight, the pilot bound, the guest was animating, and find_mission returned
NOTFOUND. Narrowed: the .ssb header is absent from guest memory (0 hits where
earlier runs hit immediately), ADN110 is absent too, but the manifest string
'Stage02.ssb' IS present at 0xBDA6C50B. So memory is readable and the manifest
is loaded while the script is not, in a mission that is flying.
No explanation offered. The cheap discriminator for next time is to poll for the
header from the moment flight starts and record when it appears, instead of
sampling once.
All 28 StageNN.ssb decoded: 2,085,628 bytes, 25,705 call sites, 108 of 147
built-ins used. Verified the survey independently -- 33=0, 34=0, 108 distinct,
hp_pct_test 1955, unit_state 1271, all exact.
global_counter0/1 have ZERO call sites in all 28 stages. The handlers exist and
are wired; no mission calls them. So 'does a wave start after N kills?' is
answered for the whole game, not just Stage 02: no mission counts kills.
Two idioms change how the counts read. hp_pct_test(unit, 0.0) IS a destruction
test -- 1786 of 1955 calls (91%) pass 0.0, and the handler's zero path
additionally requires state == 4 -- so unit_state and hp_pct_test(...,0) are
interchangeable and stages just pick one. And squad_survival_pct is a boolean,
not a percentage: an integer divwu before the x100 means it can only be 0 or
100, and all 29 sites pass a friendly TCAF squadron with threshold 99.9, making
it 'has this escort lost anybody'. The game has no destroy-N%-of-a-squadron
objective.
Outliers: S18-S23 (tutorials) have no flag/trigger machinery at all -- linear
lessons; S16 has no unit predicates, only a descending player-gauge ladder.
Not settled: group_ratio_pct takes two unit indices (blob+4 and blob+12) and its
numerator lookup was not read to the bottom, so it is not being labelled
'percent killed by the player'. isl.py's UNIT_ARG omitted 71/72.
trigger_watch.sh + host_addr.py translate the guest VA to a host address
(0xBE14DEA4 -> host 0x1BE14DEA4) and watch it. It fires: Thread 50 'Main
XThread', old 0, new 16777216 -- which is big-endian 1 read little-endian, so
the count going 0 -> 1, independently confirming the field. The write happens on
the guest's own main thread, not an emulator worker.
But the writer cannot be named from the host stack: the faulting PC is
0xa0c65f23 with no symbol, executing 'mov 0x110(%rsi),%rbx', i.e. Xenia's
JIT-compiled guest code, and the frames above it are not host-unwindable. So the
watchpoint answers when and which thread, not which guest function.
Recorded as a ceiling of the method rather than retried blindly. The way past it
is that the JIT holds the guest context in a register (%rsi here), so the guest
PC is recoverable from the context block -- which needs Xenia's context layout
from the xenia-rs sources on this box, a separate tractable piece of work.
All 25 opcodes now have meanings. Ops 2/4/6/8 are integer compound assignment
(+= -= *= /=) and 3/5/7/9 the float versions; 10 and 11 are integer and float
compare writing three condition bits; 13-18 are je/jne/jl/jle/jg/jge; 21-24 are
push.i/push.f/pop.i/pop.f over deques at phase+44 and phase+64.
The shared-handler question is answered: the dispatcher leaves the opcode in r4
and the shared thunks never overwrite it, so those helpers take an extra opcode
argument and index a secondary table (0x82271448, 0x8227152C).
CORRECTION to my own tool and note: the branch/jump base is [phase+232], which
the phase initialiser sets to 0x24 + the phase's entry from the mission-level
stream -- 0xE4 / 0x14AA8 / 0x24B4C for Stage 02's three phases, not the file's
0x24. Measured on phase 1: base 0xE4 puts 525 of 525 branch targets on an
instruction boundary; base 0x24 manages 188. isl.py had been using 0x24 for
every phase, so its jump targets were wrong throughout. Fixed via
isl.phase_bases().
That also settles two things mission-script-ssb.md left open: offsets ARE
code-base-relative, and 0x1883's operand IS a code pointer -- the earlier worry
that some 'land on IEEE floats' was an artefact of adding the wrong base.
Three approaches, none successful. sub_8226E160, earlier flagged as 'enqueue a
pending trigger', has exactly one caller and is a specific operation rather than
the general append. Writes to the count at +20 inside the container code number
only four, and all four are part of a block initialisation (stw to 0/8/12/16/20/
24 in consecutive instructions) in sub_8226E7D8 and sub_8226E930 -- constructors,
called from 0x8226E560 and from ScriptMission's own constructor at 0x822608A0.
So the increment that takes the count 0 -> 1 -> 2, which is measured live, does
not appear as a plain stw to 20(rM) anywhere in the container's code. It is
inlined, uses another addressing form, or lives somewhere I have not looked.
Recorded as not found rather than guessed: inferring from the shape of nearby
functions is exactly what produced the 'push' mislabel last iteration.
Names the approach that would settle it: a gdb watchpoint on
ScriptPhase+272+20 during a live mission. The address is known at runtime, the
count demonstrably changes within ~2 minutes, and a watchpoint reports the
writing instruction directly instead of inferring it from static shape.
Walked the container's list from +16 for 200s. The structure holds: +20 tracks
the node count, nodes chain through their first word, and entries appear as the
mission runs (0 -> 1 -> 2, then stable).
The record layout does not. I expected node+8 to hold small symbol indices,
which the pop's out-parameters made natural. Every field is a guest heap pointer
(0xBC..), so trigger records reference objects rather than table indices, and
those objects are unidentified.
Also records a false resolution I introduced: a line printed 'f4=0(ADN101)'
because the raw value is 0 and my formatter mapped index 0 to symbol-table-2's
first entry. ADN101 is not in that record -- the pretty-printer invented a name
for a null. A resolver must refuse values that were never indices.
And corrects the previous section: sub_8226E3B8 is a CLEAR, not a push. Its tail
decrements a counter, calls an erase helper, and loops while [+20] != 0. So
built-in 100 clears the queue then rebuilds the thread list, matching the
built-in table's own wording; 'push' was my label, not the disassembly's. Two
callers: vt2 (script) and 0x8226D420 (an engine site). What appends a node is
still unidentified.
Read from a running Stage 02 mission: the container at ScriptPhase+272 has +20
moving 0 -> 1 -> 2 over the first two minutes while the phase ordinal stays 1.
So +20 is a real count of currently registered triggers -- the script arming
watches as it goes, and Stage 02 has 12 push_trigger sites -- and it is readable
from /dev/shm with no debugger. First direct view of what the script is waiting
for.
Corrects one field from the static reading: +12 is not (only) a list head. It
reads 0x000A0009, which is not a pointer, even though the push's
'addi r31, r30, 12' made list-head the obvious interpretation. Recorded as
unresolved rather than kept quietly.
Chasing what makes the phase-1 condition re-evaluate, since the polls do not run
continuously.
Two method corrections: searching the VM range for '272(rN)' mostly returns
VTABLE slot offsets -- 0x82273174 lwz r11,272(r11) is followed by mtctr/bctrl,
a virtual call through slot 68, not an access to the phase field. And
[phase+272] is not a pointer to a queue but an EMBEDDED container: vt2
(sub_82265DD0) is 'addi r3,r3,272 ; b 0x8226E3B8', passing phase+272 as this.
Layout from the push/pop pair (sub_8226E3B8 from built-in 100, sub_8226E220
called every frame from sub_8226D740): +12 list head, +16 current node, +20
element count (zero = empty, tested first by the pop), +24 scratch. The pop
returns the record through out-parameters read from node+8: three u32s, a
double at +16, another u32 at +24 -- matching the six pointers sub_8226D740
passes in.
The actionable part is [phase+272+20], a live pending-trigger count readable
from /dev/shm. Watching it alongside [ScriptMission+40] should show when the
engine hands the script an event, which is when condition coroutines start --
the thing every phase experiment so far has been blind to.
Layout is from disassembly only; not yet verified live.
Hammering settles what a single write could not: hull 0x44BB8000 (1500.0f),
944,387 writes of 1 over 15s, and afterwards the value HELD at 1 -- the game
stopped rewriting it. The screen left 'flight', the HUD is gone, the ship is
burning, and Natalie radios 'I've lost contact with Rhino 3!', the player's own
callsign. The game read the poked value and killed the player.
Established: writes to /dev/shm reach the running guest; hull at pos+0x154 is
authoritative, not a readout; and a single write loses a race against the game's
own continuous writes.
This upgrades two earlier results from inconclusive to genuine negatives. The
unit-record pokes were downgraded because I could not tell 'ignored' from 'never
arrived'. The write arrives -- and those pokes persisted untouched for 60s, so
the game genuinely saw state=4 and handle=0 on all three objective squadrons and
did nothing. That is real evidence the phase-1 condition coroutine is not polling
and its checks run only when a trigger starts them.
Withdrawn: last iteration's claim that the pilot's hull= is a different field or
scale. I read 1000.0f at pos+0x154 and inferred a mismatch with the logged 1500;
this run reads 1500.0f at the same offset. Same field, different value per run.
poke_control.sh (self-retrying, succeeded on attempt 1 with no freeze) set the
player's hull at pos+0x154 to 1. Twelve seconds later the game had put
0x447A0000 back.
That establishes an asymmetry worth having: the hull word is continuously
rewritten by the game, while the unit-record fields held our value untouched for
60s. It separates fields the game maintains from fields nobody writes.
But it is not yet a control. The after-frame shows a red WARNING banner -- and
the before-frame already shows MISSILE ALERT, so the ship was under attack in
both and the banner is not attributable to the poke. A value being overwritten
proves the game writes that address, not that it read ours.
Correction: hull at pos+0x154 is 0x447A0000, a FLOAT 1000.0, not the 1500 the
pilot logs -- those are different fields or scales and should not be conflated.
The settling refinement: poke in a tight loop for several seconds so the value is
low whenever the game samples it. If hull is authoritative the ship dies and the
screen goes to GAME OVER; if it survives, the field is a readout.
The reusable part is the harness: boot -> verify animating -> locate -> act, with
a freeze at any step costing one retry rather than the iteration.
This iteration set out to run the positive control the poke results need -- poke
the player's hull, which the pilot logs every sample, and confirm the guest sees
it. It did not run: the bind took two attempts (first fwd_cos -0.94, second 1.0)
and by the time the player was being located the guest had frozen, with
entities2 reporting '0 moving triples' and frozen.py confirming
max_pixel_delta=0.
Session tally: froze at ~70/126/150/253/610-682s and this run; ran clean at
694s (ended by the game), 936s and 1064s. Roughly two in three freeze, each
costing a ~5min boot plus the window. The freeze has now truncated more
experiments than every other cause combined.
Restates that it is probably not ours: the mission-end freeze is recorded as
pre-existing in both our build and the official AppImage, and this session
produced a pilot-only freeze at 150s with no probe attached. The
probe-correlation lead is real but never became a clean split.
Consequence: experiments needing more than ~2 minutes of live mission should
checkpoint and resume, or detect the freeze and re-run themselves. Every tool
here witnesses the freeze; none survives it.
rec+4 = 0 (the early-exit path built-in 69 actually tests) sticks for 60s with
all three squadrons poked, and neither the phase nor +16 changes. +16 stayed 2
while the handle it derives from read zero.
Leading explanation: the phase-1 condition coroutine is not polling. Both pokes
persist untouched and nothing recomputes anything, which fits the polls at
0xF524 living in a coroutine that only runs when a trigger starts it
(built-in 1, fed from [phase+272]).
But the honest problem is bigger: two pokes have now produced no observable
effect and I cannot tell 'the game ignored the write' from 'the write never
reached the game'. The stick test only proves the value persisted in the
shared-memory FILE, not that Canary's guest sees it. That control should have
come first, so both null results are downgraded to INCONCLUSIVE rather than
evidence about the condition.
Next: poke something with a visible effect (player hull, a HUD counter) and
confirm it on screen before trusting any further null result. What still stands
from these runs is only what was observed rather than poked -- the arrivals,
ADN111's destruction, and the mission-over branch.
Disassembling built-in 69's handler at 0x8226ADF0 instead of trusting the
summary: it loads rec = base[idx], tests rec+4 against zero (early exit =
absent), passes rec+4 to sub_82301240 for the lifecycle lookup, and compares the
byte at rec+104. rec+16 is never touched on that path.
That explains the null poke result exactly -- I wrote into a field the condition
does not consult. +16 remains a faithful OBSERVABLE (the arrival and death
transitions were real) but it is a readout, not the input. It also explains why
rec+4 holds small consecutive integers: it is a handle that sub_82301240
resolves, not a pointer.
The corrected simulation of 'this squadron is gone' is rec+4 = 0, taking the
documented early exit.
Flags a general caution: this built-in table came from a subagent's reading of
handler behaviour, and this is the second per-offset field description that did
not survive the disassembly. The identifications of which built-in does what
have held up; the offset meanings are leads to verify.
Ran the direct test instead of a seventh attempt at winning. All three objective
squadrons were live (state 2); the write to +16 sticks, and 60s later with all
three reading 4 -- the value a naturally-destroyed squadron takes, measured on
ADN111 -- [ScriptPhase+196] is still 0 and the ordinal still 1.
So 'phase 1 clears when ADN110/111/112 are destroyed' is not confirmed and its
simplest form is refuted. The bytecode reading (three unit_state polls then
set_flag(8)) stands; what does not follow is that flipping the field equals the
kill.
The persistence is the clue: built-in 69 normalises +16 when it polls, so a
running condition coroutine should have overwritten the poke within a frame. It
did not, which points at the condition being evaluated only when a trigger fires.
Also corrects the per-unit record layout: +4 is 26/27/28 for the three
squadrons -- small consecutive integers, NOT the 'live object pointer' the
built-in summary describes (an undeployed squadron has +4=0). +20 = 9 is exactly
their member count n from the roster, so the record is per-squadron and carries
its strength. My own probe printed 'obj=yes' by testing that word for non-zero
rather than pointer-ness, which made an index look like an object.
Two more attempts. Recording three things rather than another flat negative.
The pilot's gun-fire rate is 1.6% (81 of 4986 frames), but that is not the
blocker: the nose gun is Power 15 unguided while the main mount is Power 200
guided, and ~70 missiles went out in ~500s. The log's fire= field tracks only
the gun and invites the wrong conclusion.
SYLPH_KILL_TURRETS=1 was tried to align DEFEND with the objective and is
REFUTED as an improvement: 3387 of 11112 samples (30%) chased targets over
20,000 units away, because turrets are static and spread out, so the pilot
commits to distant ones and defends nothing. Escort still fell to 48.5% and no
additional objective squadron died.
The bounded scan delays freezes but does not remove them -- one run clean to
694s, one frozen at ~682s, against 3-of-3 inside 4 minutes unbounded. So 'the
sweeps were the cause' is too strong; they were a cost.
Six attempts, no phase advance. Names the cheaper experiment: guest memory is
writable, so set the two surviving squadrons' +16 to 4 with gpoke and watch
whether finished goes to 1 and the ordinal steps to 2. That tests the condition
directly, and a null result is equally informative.
Bounding the pointer scan to 0xBC000000-0xBD000000 (with a full-sweep fallback)
drops find_mission from a ~371MB walk to 0.7s. The run then went 694s with the
probe attached and NO freeze, against 3-of-3 frozen inside ~4 minutes with the
unbounded version. n=1, but the first probe-attached run to survive.
State encoding pinned to three points: 1 = not yet deployed, 2 = active,
4 = destroyed. ADN111 caught going 2 -> 4 at 433s while the active count fell
36 -> 27.
The phase ended at 694.9s WITHOUT the ordinal advancing, and every field matches
the branch read statically from sub_82260710: [phase+300]=2 (last-phase flag),
[mission+20]=0 (mission-over state), [phase+196]=1 (finished), [mission+40]=1
(unchanged). The static state machine is confirmed on the live oracle for the
mission-over half.
But this was a LOSS, not a clear: GAME OVER on screen, escort at 35.7%, pilot
DEAD at 676s, and two of the three objective squadrons still at state 2. So the
'destroy all three clears phase 1' prediction remains untested. What is
established is that the else-branch is the only route to phase 2 and needs
[phase+300] != 2 when the phase ends.
Five attempts, still no phase advance observed -- the obstacle is now keeping the
escort alive, not the freeze or the instrument.
A pilot-only run froze at t~150.7s (frozen.py: max_pixel_delta=0), found by
accident when the sweep-free experiment aborted at startup and the run flew with
no script probe attached. The tally is now 3-of-3 frozen with the probe versus
1-of-3 without, not 3-versus-0. Still a lean, but not the clean separation the
previous entry claimed, and marked down accordingly -- the fourth time a freeze
conclusion here has had to be softened by one more run.
The sweep-free test itself could not run: ScriptMission is re-allocated per run,
so the address from an earlier run (0xBC7A2A20) read back all zeros. The cheap
self-consistency check ([m+44] must equal the phase's [+244]) rejected it
instead of reporting garbage, which is the part that worked.
Names the cheaper replacement: a BOUNDED pointer scan. Every ScriptMission seen
so far sits in 0xBC79xxxx-0xBC7Axxxx, so ~32MB instead of ~371MB would cut the
sweep cost roughly tenfold. Not yet implemented.
phase_watch.py runs give a sharper tally than the earlier inducer test, because
the only difference between the groups is one tool: pilot alone went clean to
936s and 1064s (2 runs), while pilot + phase_watch froze at ~70s, ~253s and
~126s (3 runs). Same boot path, same pilot, same mission.
What phase_watch adds is find_mission(), which does TWO full sweeps of guest
memory at startup plus ~130 small reads every 5s. The sweeps are the same shape
as the heavy_read.py scans the earlier n=2 experiment implicated.
Still correlation, not cause -- 3 vs 2, and the earlier inducer test produced a
clean counter-example. But it is the best-supported version of 'the instrument
provokes the freeze' so far, and it is my instrument, which makes it actionable.
The cheap test is named: find_mission only needs to run once, so re-run the
watch with the sweeps replaced by an address from a prior run. Freezes stopping
implicates the sweeps; continuing implicates the per-sample reads.
Cost so far: three attempts at observing a phase advance, all truncated inside
four minutes.
ADN110/111/112 all carry first-keyframe time 170 on their _p1F routes, and in a
live run all three flipped from not-deployed to active at ~143s on the probe
clock. The probe zero sits roughly 10-20s into the mission, putting the arrival
at ~155-165s of mission time against a predicted 170.
That pins the unit as seconds: 170 frames at 30fps is 5.7s, so they would have
been active at the first sample and demonstrably were not. Three squadrons
sharing one timetable entry changing state in the same 5s window is not
coincidence. Flagged as a match rather than an exact measurement -- the probe
clock is not aligned to phase start.
Supersedes mission-arrival-watch.md's headline negative: 'no arrival has ever
been observed across six runs' was an instrument limit, not a fact about the
game. Its craft-counting analysis stays accurate -- craft counts conflate
deployment with attrition and cannot see an arrival at all.
Watching [ScriptMission+40] and the three phase-1 objective squadrons together:
all of ADN110/111/112 flip state 1 -> 2 at ~143s, while records in state 2 climb
24 -> 35 over four minutes.
So state 1 means 'not yet deployed' for these, not 'gone'. The built-in table's
'1/3/4 = gone/dead/invalid' shorthand is incomplete, and reading state != 2 as
destroyed would have been wrong exactly as flagged last iteration.
This also answers a much older question: mission-arrival-watch.md and the wave
work recorded '0 confirmed arrivals' across many runs by watching the CRAFT
population. The script's own unit table shows arrivals plainly -- eleven records
enter state 2 within four minutes. The old negative measured the wrong
structure; craft counts conflate deployment with attrition, the per-unit state
field does not.
Both attempts froze (at ~70s and ~253s), so no phase advance was reached. The
freeze witness caught both immediately, which is why the truncation is visible
instead of a silently flat line.
New harness tools/re-capture/phase_watch.py.
Unblocks the phase experiment, which was stuck because '38 enemies died' could
not say whether the right ones did. Chasing craft->squadron was the wrong angle:
the script VM keeps that table itself, indexed by the .ssb symbol-table-2 index.
Route: find the .ssb header in guest memory (0xAB840010 for a Stage 02 run),
code base = filebase + 0x24, scan for a word equal to it, then VALIDATE
arithmetically -- [ScriptMission+44] must equal filebase + symtab1 offset + 4.
Measured 0xAB874C94, predicted 0xAB874C94, exact. A second candidate that also
pointed at the code base failed that check and was discarded; without it either
would have looked plausible.
ScriptPhase+324 -> +4 is an array of 122 per-unit records -- exactly the size of
Stage 02's symbol table 2, an independent confirmation of the index space.
[ScriptMission+40] reads 1 in a phase-1 mission. The mirror at
[*(0x828F35F8)+236] that three earlier runs polled reads 0, because ChangePhase
only posts once the ordinal exceeds 1 -- so +40 is the real counter and is
reachable from /dev/shm with no debugger.
Flagged rather than asserted: the three objective squadrons read state=1 with a
LIVE object pointer in a mission where nothing has been shot, which does not fit
the built-in table's '1 = gone'. Reading state != 2 as destroyed would be a
plausible-but-wrong inference; the encoding needs pinning first.
New tool tools/re-capture/squadron_state.py, verified end to end against the
manual reading.
Checked liveness before investing this time (animating, max_delta=254) and the
run was real: 823 distinct speeds, hull 1500->1485, escort 100%->30.7%, ADAN
population 127->89 (38 destroyed) over ~1064s, no GUEST FROZEN banner. The phase
mirror stayed 0 throughout.
But this still does not test the prediction. The claim is that phase 1 clears
when ADN110/111/112 -- 27 specific Turrets -- die; what was measured is that 38
enemies of some kind died. SYLPH_PREFER biases target choice but the pilot went
to DEFEND and spent the run on e010_ADAN_Attacker_S protecting the escort.
The missing instrument is per-squadron liveness: nothing maps a live craft back
to its roster squadron, the same gap roster-to-craft-link.md records as unsolved.
So 'no advance after 38 kills' is compatible with both 'the condition is wrong'
and 'the right 27 were never killed'. The decisive experiment is blocked on
attribution, not on flying time.
Last iteration ended with 'what reads the flag file is unknown' after an offset
search failed. The framing was wrong: I was hunting an engine-side reader, but
the consumer is the script itself, through built-in 9 (read_freg), which loads
[phase+88][i] into the double result register.
Stage02.ssb calls read_freg 12 times -- the same count as set_flag (12) and
clear_flag (12) -- so the latch is symmetric and entirely inside the VM. That
closes the middle of the set_flag -> ... -> END_PHASE chain: one coroutine
latches, another reads it back and branches.
Also records an unconfirmed observation: op10 + op13 pairs with consecutive
small immediates and their own code offsets look like a case/branch dispatch.
Flagged as a pattern, not named -- neither handler has been read.
SYLPH_PREFER=Turret, escort at 100%, 1070s, phase field 0 throughout, 4029
Turret-targeting pilot samples. It looked like a clean negative -- 'hunting the
objective squadrons does not advance the phase' -- and it is worthless:
frozen.py reports max_pixel_delta=0, and the pilot's first sample at t=0.0
already has spd=0 with the same yaw/pitch/target/d=7186 it still had at 1070s.
The mission froze on entry to flight and nothing was ever shot, while screen_id
said 'flight' the whole time.
I was one step from writing this up as a fact about the game; running the freeze
test rather than trusting a plausible log is what caught it.
phase_probe.py now calls frozen.py every 60s and prints a GUEST FROZEN banner
inline, so a dead-world reading is labelled in the data instead of discovered
later. Note frozen.frozen() returns a TUPLE (is_frozen, max_delta) -- testing it
directly is always truthy and would have made the witness fire on every check.
Verified against the frozen guest.
Also recorded: the pilot log is itself a freeze witness -- identical
yaw/pitch/target across thousands of samples is a dead world, not patience.
Resolving the three squadrons the phase-1 script condition polls against the
roster: all three are phase-1 (route-name map), all fly UN_e007_ADAN_Turret, all
n=9, all on Formation_ADAN_Turret09_30 -- so the condition watches 27 Turrets.
Three layers agree independently: route names say phase 1, the roster says what
they fly and how many, and the script polls exactly those three before latching
set_flag(8).
Also confirms on the data that 'Turret' is a craft type, not ship-mounted AA:
UN_e007_ADAN_Turret is flown by AI_ADAN_CraftSquadron_Rookie/_Veteran in a
30-slot craft formation, never AI_Structure.
Polled the runtime phase mirror every 3s for 936s of hunting flight. It never
left 0, which the static work predicts rather than contradicts: ChangePhase is
only posted once the ordinal exceeds 1, so a phase-1 run never writes it. That
does make the mirror useless as a phase-1 readout -- it reads 0, not 1, and
[ScriptMission+40] needs a debugger.
The run ended in GAME OVER with the player's hull at full 1500, so a script
lose-branch fired rather than the player dying. The pilot log narrows it: the
tracked asset fell from 100.8% to 23.4% with ASSET-HIT events, while the pilot
avoided f101_TCAF_Acropolis 2588 times. 'Escort lost' is a measured candidate,
not a guess -- but the asset ended at 23.4%, not 0, so the threshold (or whether
another unit's loss ended it) is not established.
Two runs now (530s, 936s) and no phase advance observed. Since phase 1 polls
unit_state on ADN110/111/112 and the pilot hunts whatever is nearest rather than
the objective, biasing it at those squadrons is the next step.
Refuted: sub_8226D740 does NOT read the flag file. Its three addi rN,r31,120 are
fields of a local trigger-record buffer passed to sub_8226E220 as out-params;
the phase lives in r26/r29. Offsets 88 and 120 are too common image-wide (2091
and 5228 instructions) to isolate the register files by offset search, so what
reads the flags is still unknown and the set_flag -> END_PHASE chain has a gap.
Resolving symbol-table-2 indices turns the bytecode into mission logic. At
0xF524 Stage02.ssb polls unit_state on ADN110, ADN111 and ADN112, updates each
one's objective marker, then latches set_flag(8) -- exactly the
trigger/predicate/set_flag/END_PHASE shape predicted from the disassembly, now
observed in the mission's own code with names the roster tables already gave.
The 12 END_PHASE sites are outro sequences (wait_cmds_drained / fade_sound(3) /
builtin85(3) / wait_s(3) / END_PHASE / yield) -- the terminator, not the
decision.
Fixes a decode bug that hid every argument: the tracker only followed
local[i] = special[0], but the common form is an immediate written straight into
local[i] (k=01,03), so every unit predicate printed with NO arguments. The
disassembly looked complete while being empty exactly where it mattered.
Also records the live probe result: the phase mirror at [*(0x828F35F8)+236]
stayed 0 for ~530s of actively-hunting flight, no advance observed -- which is
what the static analysis predicts for phase 1, since ChangePhase only posts once
the ordinal exceeds 1.
Table at 0x8227226C is 147 big-endian absolute VAs (verified structurally: it
ends exactly where the first handler begins, all targets inside sub_82272220).
Arguments are not in the instruction -- every handler does c_str() on
[phase+20], a packed blob, which is what the local[] staging fills. Return 2 =
yield; five built-ins block by skipping the pc advance.
Recovered the ScriptPhase state layout: 32-entry float and flag register files,
int/double result registers, the timer block, and the runtime unit array at
+324 indexed by symbol-table-2 index -- a direct hook from bytecode call sites
to the two .ssb symbol tables.
Spot-checked two claims against the disassembly rather than trusting them: id 4
loads a DOUBLE into the thread countdown and returns 2 (wait_s), and id 24 reads
current/initial squadron member counts (squad_survival_pct). Both exact.
Counting Stage02.ssb: unit_state 255, hp_pct_test 167, dist_lt 92, unit_alive
71, unit_relation 52 -- and squad_survival_pct, group_ratio_pct and the two
global counters are NOT called at all. So Stage 02's phases are gated on named
units (destroyed / HP / proximity), never on an aggregate count, even though the
kill-counter primitives exist in the VM. That answers the standing 'next wave
after N kills or after an event?' question for this stage: specific units, not a
number.
isl.py now names the built-ins, so the run-up to the first END_PHASE reads
wait_cmds_drained / fade_sound(3) / builtin85(3) / wait_s(3) / END_PHASE.
Not settled: 3 handlers unresolved (55, 75, 105); the 1024-slot interpreter
command table is only partly recovered.
Resolver table 0x82271D74 gives four kinds: 0 global[i], 1 immediate,
2 special[i] ([phase+164]/[phase+168]), 3 local[i] ([phase+20+i]). Byte[0] is
the rvalue kind, byte[1] the lvalue kind, so the recurring instruction pair is
argument staging -- values land in local[] at offsets 0,4,8,0xC and the next
call consumes them. A built-in's arguments are not in its own instruction.
Fixed a decode that would have been believed: immediates in set.f are DOUBLES
carried as two words (op 1 stores with stfd). Reading the high word as a float
gives 2.125 where the script means 3.0.
isl.py now tracks staging and prints call arguments, so the run-up to the first
END PHASE in Stage02 reads as builtin=64(0x42,2,1,9,1,-1) / 120 / 59(3) / 85(3)
/ 4(3) / 6. Three built-ins taking 3 just before the phase ends look like a
wait-seconds family -- flagged as unconfirmed until the built-in table is read.
Read the encoding off the interpreter rather than guessing: instruction is a
big-endian u32 whose LOW byte is the opcode (25 of them, table 0x822635FC),
byte[2] is the instruction length -- every handler advances the pc by it -- and
bytes[0..1] are operand kinds. Op 12 is a jump whose operand is relative to the
code base [phase+232], which settles that offsets are code-base-relative for
this opcode. Op 19 is the built-in call: id in word@+4, and word@+8 is a
monotonically increasing STATEMENT id (0x245, 0x248, 0x24A, ...).
Confirmed by disassembling Stage02.ssb: the stream decodes cleanly from the code
base and routines terminate on ret exactly where expected.
Scanning the code region on the call encoding: 2846 call sites, 73 of the 147
built-ins used. The phase-control ones are located -- built-in 6 (end phase) at
12 sites, 62 at 3, 39 (mark last phase) at 8 -- so a phase has several exit
paths, as a mission with win and lose branches should.
New tool tools/re-capture/isl.py with --calls and --to (resync-into-target,
needed because instructions are variable-length so you cannot walk backwards).
Not settled: the 147 built-ins are uncharacterised, so this is structure without
meaning -- we can see THAT a phase ends, not WHAT was tested.
mission-phase-advance.md recorded the bytecode as not on the disc. It is; the
earlier grep failed because every pak entry is Z1+zlib, so no name is plaintext.
The loader resolves table KEYS, not a filename: GamePart name -> GP_SCRIPT ->
script.tbl (name_hash 0x75FE4656), whose SCRIPTS record is a 40-field manifest
mapping MISSION1..MISSION29 to StageNN.ssb. Verified: Stage02.ssb is 226,596
bytes, md5 aff69b5a..., byte-identical across all six language paks.
28 scripts exist; S17 is the missing one -- which agrees with the table sweep
(S17 in none of the five families) AND with the loader guard
sub_8225EC78 'if (n == 16 || n > 32) return', proving that mission number is
0-based. Three independent routes to the same conclusion.
Header decoded from ScriptMission::Load rather than guessed: version, code
offset 0x24, two symbol tables. Stage02 symtab1 = 326 symbols (197 message ids,
119 route names, 10 subobjectives), symtab2 = 122 (111 unit ids, 11 characters).
Two loops close: the 111 unit ids are exactly the 111 UnitGroup_S02 squadrons,
and the route names appear as SCRIPT SYMBOLS -- which is why nothing in the
executable parses Route_*_p<N>* names. The _pN convention is the script's.
Refuted: the .embsec_ sections are code (32,368 instructions, 108 functions),
not script; MiscBin.pak and DefTables.pak have zero hits.
Open: the 25 ISL opcodes and 147 built-ins are not decoded, and the
mission-level stream at +0x24 is only partly read.
Third run with the inducer on from flight start went 900s without freezing, so
the tally is 2 for / 1 against. Stated confound: the inducer was far weaker this
run (~47-50s per CPU pass vs ~4.2s in run 2, because two subagents were
saturating the box -- 22 passes in 15 min instead of ~150). That is consistent
with either reading, so it is not scored as a confirmation. What it does
establish is that the inducer is not sufficient, and that future runs must
report its measured rate rather than just that it was on.
Second healthy stability sample: 9 stable / 14 vary vs 12 / 13 in the first,
intersecting to only 8 threads. So the stable set is run-dependent and must be
measured within the run it is used in -- the same lesson as gdb thread numbering,
one level up.
The frozen half of the distribution experiment is still uncollected.
Closes the open question from mission-phase-membership.md. The static sweep
found no trigger because there is none in the data: each phase ends when its own
per-mission script says so.
[ScriptMission+40] is the 1/2/3 phase ordinal -- initialised at 0x822606B0 and
incremented at exactly one site, 0x822609F8-0x82260A00 (checked: only one stw to
40(rN) in the whole state machine). Its guard is [ScriptPhase+196] != 0, and
that flag has only two writers, vtable slots 0 and 1, reached only from built-ins
6 and 62 of the phase-script VM's 147-entry command table. Built-in 39 sets
[phase+300]=2, which ends the mission instead of advancing.
Refutes all four candidate triggers: no kill counter, timer, trigger volume or
message event appears on the path. Any such condition lives inside the script,
which is why three phases of one stage can have three different clear conditions.
CScriptInterpreter::ChangePhase (sub_822FF330, opcode 995) is the consumer, not
the cause; it writes a second phase mirror at [*(0x828F35F8)+236] -- a probe
target.
Not settled: the script bytecode is not on the disc under any obvious name.
Tooling correction: the .pe is not stale, it is a flat VA image (offset = VA -
0x82000000), verified 7/7 against the DB. Also: instructions.raw is an INTEGER,
and decoding it as a hex string silently compares nothing -- that nearly got
this correction recorded backwards.
Found the routines in the disassembly DB rather than guessing from data:
sub_82447DF0 IDXD tag hash (lbz+extsb, modulus 0x00FFFFDF, magic 0x2101)
sub_82447E70 IXUD tag hash (lhz, 64-bit, modulus 0xFFFFFF67 then 0x00FFFFDF)
Both transcribed instruction-for-instruction into Python and Rust.
IXUD SOLVED. It defeated every single-modulus search because it chains TWO
exact moduli -- the loop reduces mod 2^32-153 in 64-bit arithmetic and only the
result is folded mod 2^24-33. A polynomial mod M1 folded through M2 is not a
polynomial mod anything, which is exactly why the gcd test returned 1. Verified
independently: 86/86 record keys and 108,261/108,261 field tags in
GP_MAIN_GAME_E.pak, and NoRecord -> 0x1c6d9c96.
CORRECTION 1: tag_hash must SIGN-EXTEND each byte (extsb). My reconstruction
used unsigned bytes and matched all 1.27M disc names -- every one is ASCII --
while disagreeing on ~90% of random inputs with a byte >= 0x80 (verified:
18096/20000). The disc could never have caught this; only the disassembly did.
CORRECTION 2: name_hash's reduction is EXACT, not lossy. The module doc claimed
the missing conditional subtract made it something other than %. rlwinm r6,r6,
9,23,31 is just hi>>23, and with RECIP = floor(2^55/M)+1 that is Granlund-
Montgomery magic division -- 0 wrong at every quotient boundary across the full
32-bit domain. Retracted.
cargo test -p sylpheed-formats --lib hash: 10/10.
Swept all 33 dat paks plus hidden/DefTables.pak: 7,750 IDXD objects, 190,782
records, 1,271,462 (name -> tag) pairs, 0 failures, 0 parse failures.
The key rule is stronger than first written: a record's key is tag_hash of its
own inline name, 190,782/190,782 -- no roster needed (roster-based recovery only
reaches ~30% of records). Verified independently here on UnitGroup_S02, 112/112.
The modulus is uniquely pinned, not just consistent: gcd of
poly256(name) - (tag & 0xFFFFFF) over all pairs is exactly 0x00FFFFDF, prime.
Case-sensitivity is load-bearing, not incidental: the disc has 17 name pairs
differing only in case (UNIT/Unit, TYPE/Type, STAGE/Stage, ...); name_hash
collides on all 17, tag_hash separates all 17.
Records the limit too: IXUD, the wide-string sibling, uses a DIFFERENT low-24
hash (0/19,808) though it keeps the same top-byte checksum (19,808/19,808). Its
offsets are counted in 16-bit chars (STR + 2*strsize == filesize) -- newly
decoded, all 534 parse. Its low bits are provably not a modular polynomial (gcd
= 1; exhaustive Barrett search over M in [2^20,2^25) finds nothing; fixed-position
deltas are non-constant), so it is lossy and needs the XEX.
Six captures across one healthy run: 12 thread states stable, 13 vary. Every
thread previously reported as a freeze signature is in the VARIES set --
including T68/T69, which I had kept as 'what reproduces across both freezes'.
They park and unpark during ordinary play, landing on the same objects they hold
when frozen, so seeing them parked while frozen is not evidence.
Net: no thread-level freeze signature has survived. Both frozen diffs are
consistent with healthy variation, and one-sample-per-state was never capable of
separating them.
Still standing: the 12 stable threads hold the same object in all six captures
and none of them moved in either frozen capture -- so 'not a whole-emulator
stall' survives, now resting on the stable set being undisturbed rather than on
a count of unchanged threads.
Also noted: gdb thread numbers are not comparable across runs (this run has
T132-T142, earlier runs had T104-T106), so future cross-run work must key on the
object address or guest thread id.
Route_S<NN>.tbl names every route Route_<squadron>_p<N><kind> -- 120/120 for
S02 -- so the squadron-to-phase map was on the disc all along, in a table read
until now only for fly-in geometry. 108/111 S02 squadrons map; the 3 misses are
typos in the route table and all are phase 2.
That resolves the 387-vs-300 gap after four failed attempts, by refuting its
premise: there were never 300 craft. Phase 1's member-level n multiset
{1x25, 2x1, 4x4, 9x12} matches the measured craft-per-record fan-in
{2x24, 4x1, 8x4, 18x12} bucket for bucket at exactly 2x, so 300 = 2*(151-1),
the -1 being the documented 41-vs-42 off-by-one.
The 2x is the probe double-counting, not real units: a 2x reading needs 16
Delta Sabers for 8 named pilots, and breaks the formation capacity bound in 20
of 37 phase-1 squadrons (16 of them on Formation_1_only, one slot) where the 1x
reading has 0 violations. All three measured craft totals being even is a
further tell.
Refuted: UnitGroup has no phase/spawn/delay field (1019 = sum(Count)*4 + 5*111,
every slot accounted for); DisableInterval is No for all 111 S02 squadrons;
stage\EnumSquadron_Test.tbl does not exist.
Confirms tag_hash independently: UnitGroup record key == tag_hash(squadron id),
1160/1160.
Still open: what advances a phase -- no static table encodes a trigger.
Closes the 4-byte record key. tag_hash is name_hash's shape -- byte-sum
checksum in the top byte over a 24-bit modular polynomial -- with two different
constants: modulus 0x00FFFFDF (2^24-33, prime) instead of 0x00FFF9D7, and no
lowercasing, so tags are case-sensitive. name_hash explains 0 of 8643.
Recovered from the tables rather than the executable: every inline field name
is a known (name -> tag) pair, and comparing names differing in one character
gives the per-position weights 1, 0x100, 0x10000, 0x21, 0x2100, ... -- a byte
leaving bit 24 re-enters as 33, i.e. reduction mod 2^24-33. Holds where it is
easy to get wrong (distance 8 and 9 carry correctly).
A record's key is the tag of its own name: FormationSet rosters 362/362,
UnitGroup rosters 281/281, S02 squadron names 111/111 -- so records can be
addressed by name without reading the roster first.
Implemented in Python (unitgroup.tag_hash) and Rust
(sylpheed_formats::hash::tag_hash) with 3 new unit tests carrying disc-derived
vectors; cargo test -p sylpheed-formats --lib hash is 8/8 green.
Not settled: the guest routine is unlocated, so this uses exact modular
arithmetic where the game may use a Barrett step without final fixup.
Run 2 put the inducer on from flight start and froze ~96s in, against 670s
clean with it off -- n=2, contrast sharp, confounder (elapsed mission time)
still untouched.
Withdraws last iteration's 'T74/T75 move off a semaphore onto an event' as the
signature to chase: it does not reproduce. In run 2 they are on XEvent while
HEALTHY and stay there. The healthy state varies between instants, so a
one-sample-per-state diff cannot separate a freeze transition from ordinary
variation -- I read a difference of samples as a difference of states.
Reproduces across both: T68 and T69 go from not-waiting to waiting, T69 on a
semaphore and T68 on an event both times. And 21 of 24 threads unchanged in run
2 (17 of 24 in run 1), so 'not a whole-emulator stall' now has two independent
captures behind it.
Next: repeat the capture several times within one healthy run to establish which
thread states are stable before reading any frozen diff.
FormationSet_S<NN>.tbl records are slot lists -- 1 + 8*FrameCount fields,
exactly. Resolving every squadron's FormationID and comparing gives
sum(n) <= FrameCount holding 1159/1160 across all 28 stages, 0 unresolved, with
539 filling the formation exactly. The single violation is a debug leftover
(S20, AI_Test / MessageSet_test, Formation_1_only with n=2) and is recorded.
The old 'n is not the _NN suffix of FormationID' observation was right but drew
the wrong conclusion: the suffix IS FrameCount, so n=9 against _30 just means 9
units in 9 of 30 slots.
Also: FormationID does not hash into its table (0/16). FormationSet carries a
name roster record -- no FrameCount, fields are (tag, name, '') with the tags
being the record keys -- the same convention as Enumerate_Squadrons. Second
occurrence of 'keys are resolved by an in-table roster, not by hashing'.
Does not close the 387-vs-300 gap, and the key derivation stays open.
The 'S17-S23 have no stage record' gap was an artefact of enumerating by the
literal 'Stage_S<NN>'. Tutorial records omit it -- they name no per-stage .xpr
and pull AIParams/weapons/strings/subobjectives/nameplate/collision from a
shared _Tutorial set -- so all six were skipped. stagetbl.py Stage_S18 returns a
full six-record definition and always would have.
Counting distinct *_S<NN> names across all 1119 decompressed entries: UnitGroup
and Route cover 28 stages (S01-S16, S18-S23, S24-S29); Stage literal and
AIParams cover 22; SUBObjectiveSettings 16 (story only). S17 appears in none --
it is not a stage that lost its data, it does not exist.
Tutorial records carry the same Phase_1/2/3 structure as story stages, so the
tutorial is not a special mission type at the data layer.
Refuted en route: GP_TUTORIAL.pak does not hold the config -- 2 RATC entries,
zero IDXD, like GP_CHALLENGE.pak.
