The control needed no new decoding: hunting pilot, keep-out 1400, 300 s, and no
memory probe at all, judged only from the pilot's own log. It ran healthy the
whole way -- 211, 257, 250, 241 and 200 distinct speed values in successive 60 s
windows, with no decline in the last one -- against probed runs that stalled at
roughly 27, 83 and 255 s and ended with a single distinct speed across 400 lines.
So the instrument was degrading the thing it measured. AGENT.md warned that a
full scan competes with the emulator for every core under lavapipe, and the
probes grew heavier each iteration until wave6 was reading the entire 32 MB
entity heap plus about 300 extra preads every 12 seconds while the game
rendered.
The control is n=1. The contrast is stark and the mechanism was predicted in
advance, so this is recorded as confirmed enough to act on rather than proven; a
second clean control would settle it.
What it costs is worth stating plainly. Every "no arrival" result so far was
collected while the probe was running, under conditions the game was struggling
with and in some runs after it had frozen. That does not make them wrong -- the
elimination test at t=163 s sits in a window the witness later showed was live --
but none of them is as strong as written, and the arrival question deserves
re-running with cheap sampling before silence is read as evidence.
wave7_probe.py is the fix: one full enumeration, then polling only the hull word
at each known craft base, about 1.2 KB per sample instead of 32 MB, with a full
rescan every 90 s to catch anything genuinely new. Three or four heavy scans per
run instead of twenty-five. Implemented but not yet run, so the claim that it
stops the stalling is untested and the next run must report the witness first.
SYLPH_KEEPOUT makes the hunt keep-out a knob instead of a hard-coded 600. At
1400 the pilot both kills and survives: hull 1500 and escorted asset 100 % for
the whole run, ENGAGE throughout, eight loss events against seven in the run
where it died at 83 s.
That let the elimination test actually run. An e007 Turret squadron was ground
18 -> 10 -> 8 -> 4 -> 2 -> 0, reaching zero at t=163 s with deployed falling 41
to 40 -- an enemy squadron destroyed outright for the first time in nine runs.
No arrival followed. ARRIVALS=0 at every sample including all those after t=163.
The hypothesis that a wave is released when a squadron is wiped out rather than
merely damaged does not survive its first test. Two further losses occurred at
t=176 and t=202, so the mission was demonstrably still live and still processing
kills; it simply produced no arrival. This refutes elimination-of-one-squadron
as the trigger. It does not refute event-gating generally -- a threshold across
several squadrons, an objective completion, or a specific squadron could all
still be the gate.
The valid window is smaller than the log suggests, about 90 s after the
elimination rather than 143 s, because the guest stalled around t=255 s. The
pilot's telemetry gives it away: the last 400 log lines contain one distinct
speed value against 236 in the first 400, with no timestamp gaps -- the process
kept logging while the game stopped advancing. Nothing in the probe output
distinguished a stall from a quiet mission, and the same ambiguity affects the
trailing flat samples of earlier runs.
wave6_probe now locates a counter advancing at frame rate, samples it each tick
and prints GUEST STALLED when it fails to advance, so future runs validate
themselves. Implemented but not yet exercised in a run.
Correction first. The sed used to derive each session script from the last
stripped the probe's arguments, so wave5, census and wave6 sessions invoked
their probes with no arguments and every derived probe has been running on its
own defaults. The previous iteration's claim that the run was "cut at 240 s by
the turn timeout, not the planned 330 s" is therefore wrong: the probe used its
default of 240. The pilot got the requested duration while the probe watched for
a different one, and the numbers were plausible enough that it went unnoticed.
No earlier conclusion is invalidated -- the windows were real, just shorter than
intended and misattributed. All three sessions now pass SECS and EVERY.
First n -> 0 ever observed: the player's own record went 2 -> 0 at t=83 s and
deployed fell 41 to 40. The signal does register elimination, not just damage.
No arrival followed, which is weak evidence against the squadron-elimination
trigger since the record eliminated was the player rather than an enemy
squadron. Two other turret records dropped from 18 in the same sample; noted
without interpretation.
The important finding is what came after. For the remaining 220 seconds the
mission was frozen -- exactly 288 craft, zero losses, zero arrivals, across 18
consecutive samples. So the usable observation window is not the probe duration
but however long the player survives. A 340 s probe that loses its pilot at 83 s
yields 83 s of evidence and 257 s of nothing, and several earlier "nothing over
240 s" results may have been much shorter in practice than they look.
That also explains why pilot.py was written to survive rather than to shoot. The
SYLPH_HUNT mode added two iterations ago drops TURRET_KEEPOUT from 2500 to 600,
buying kills at the cost of exactly the survival the run depends on.
The elimination test itself did not run: the squadron reached 14, not 0, before
the pilot died. What is needed is a pilot that kills and survives -- hunt turrets
but keep the evade and retire behaviour, or a keep-out between the two extremes.
That is tuning, not discovery.
The prime suspect from the previous iteration was that the phase scheduler
simply is not running, which would have made every arrival result meaningless.
It is refuted.
timer_probe.py takes three equally-spaced snapshots of the 32 MB game heap with
no pilot -- exactly the condition where nothing had been observed to change --
and keeps words whose two successive deltas are both positive and agree within
12 %, so linear rather than merely noisy. 286 words qualify, with a large
cluster advancing in lockstep at 16.5 per second.
That rate is not a coincidence: the existing performance notes put Canary
playback on this box at roughly 14-19 fps, and the dominant counter sits inside
that band, so these read as per-frame counters.
Which gives a mundane explanation for six arrival-free runs. If the scheduler is
frame-driven and the title targets 30 Hz, game time advances at about 55 % of
wall-clock here, so the 168, 190 and 240 second runs covered roughly 92, 105 and
132 seconds of game time. Route_S02 schedules phase-1 arrivals at 90, 120, 170,
210 and 240, so the longest run passed the first two and came nowhere near the
last three. No model has to be wrong for the observations to be empty.
Kept at 🟡, not promoted: two links are assumed rather than measured -- that
these counters are frame counters, and that the game's tick is 30 Hz. The values
also do not fit a naive frame count, since the cluster read 14193 about 255 s
after emulator start, which matches neither 16.5 nor 30 per second, so their
origin is genuinely unknown and no counter is claimed to be the mission clock.
Next is one long run, ~350 s of probe, watching for a 0 -> n transition near
t = 163 s and 218 s wall, where the t = 90 and t = 120 route entries land at 55 %
speed. If 350 s still yields nothing, the frame-rate explanation is itself
refuted and the event-gated model returns as the front-runner. The ~210 s title
movie at boot is the binding constraint on how much game time one turn can
observe.
wave6_probe refuses to interpret a run whose roster count is not the reproduced
baseline of 116, per the discard rule. This run passed and its first sample is
identical to the earlier link run: 116 records, 300 craft, 41 deployed, strength
histogram 2x24, 4x1, 8x4, 18x12. The deployment is deterministic at mission
start.
The pilot failed to bind, which accidentally supplied the control condition the
kill-versus-no-kill experiment needed. With nobody flying, not one craft was
destroyed in four minutes -- exactly 300 across all 22 samples -- against 16-20
losses in each piloted run. So losses are attributable to the player being in
the fight, and NPC crossfire destroys nothing by itself. That was an open
question two iterations ago.
Still no arrival. Zero 0 -> n transitions in either condition, across roughly
fifteen minutes of cumulative Stage 02 flight and windows up to 240 s. The 75
records holding no craft at mission start still hold none at the end. Against a
route table scheduling phase-1 arrivals at t = 90/120/170/210/240, that is now a
strong negative rather than a null result.
Three readings survive: the timetable's t is not seconds (at 30 Hz the whole
phase-1 schedule finishes inside 8 s, before any first sample); arrivals are
event-gated and no run supplied the trigger; or the mission is not advancing its
phase clock at all.
The third has never been checked and is the cheapest to eliminate, so it is now
the prime suspect. Nothing in six runs has confirmed that mission time advances:
the craft count freezes without a pilot, REMAINING OB has never read as a
counter, and no clock has been located. Every "no arrival" observation is
consistent with a scheduler that simply is not running, in which case the
arrival results so far are measuring a stopped clock.
Two further runs sampled the counts from the instant flight is detected. Both
give 116 roster records in the very first sample, flat thereafter, with 14 unit
definitions throughout. That is not a load race: a race would climb toward 116
from below, and it never does.
The leading explanation was arithmetic rather than behavioural -- census_probe
counts raw aligned vtable hits while wave5_probe counted distinct primary_va
values, which are different quantities if an offset can alias to a VA another
offset also claims. Measuring both in the same run refutes it: 116 raw hits and
116 distinct VAs in every sample.
So the outlier stands unexplained. Four runs now read 116/300, 116/300, 116/296
and one 42/170, with save drift, freezing, load race and VA aliasing each
individually refuted. Recorded rather than dropped, because a one-in-four
anomaly in the measurement apparatus is what later turns into a wrong
conclusion. Practical rule adopted: discard a run that disagrees with 116 rather
than interpreting it, and reproduce any finding in at least two runs.
Useful new fact: craft declines steadily and monotonically in both runs, 300 to
288 and 296 to 280, roughly 16-20 losses over 168 s under the hunting pilot.
Whether the player or NPC crossfire is responsible is not determined here, but
destruction is visible in this signal, which is half of what the kill-versus-
no-kill experiment needs.
Still not observed in any run: an arrival.
The link from the previous iteration gives a per-record live strength, and the
measurement is internally sound: 11 records at 2, 2 at 8 and 11 at 12 sums to
exactly the 170 craft counted, with 24 of 42 records deployed.
It does not reproduce. The previous run saw 116 roster records and 300 craft
with strengths 2/4/8/18; this one sees 42 and 170 with 2/8/12 -- same disc, same
save slot, same launch script, same stage.
Two explanations were checked and both fail. The save has not drifted:
game01/savedata is unmodified since 2026-08-23 and only the profile .gpd files
were written today. The guest was not frozen: the pilot's telemetry over the
same 190 s shows speed varying across dozens of values and a live engagement
with an e007 Turret at 259-680 m.
So the discrepancy is real and unexplained, and until it is understood
per-record counts cannot be used as a time series. Recorded as the blocker
rather than worked around. The same within-run-only discipline already noted for
the global craft count now applies one level down.
Also recorded: zero arrivals and zero losses across 190 s. That is weak evidence
against clock-driven arrivals at t = 90/120/170 seconds, since a 0 -> n
transition should have appeared and none did. It is no test of the event-gated
model at all, because nothing was killed -- fc=0, no record lost strength, and
the player's hull never moved off 1500. The pilot closes to 259 m and misses, so
the gap is accuracy rather than engagement.
The previous iteration was right that neither structure points at the other's
base, and wrong to stop there. Craft point eight bytes into a roster record.
Resolving the 5101 "pointers into the roster range" by distance to the nearest
roster base gives a distribution that is neither flat nor diffuse: +0x08 appears
exactly 300 times, matching the 300 live craft, while every other delta appears
at most twice and 4780 of 5104 land more than 0x4000 from any base.
A count matching a count is the coincidence that has already misled this corpus
three times, so the link was tested on something a coincidence cannot survive.
Each side's unit type is resolved by a different chain -- the craft's from its
definition pointer at +0x130, the record's from its +0x04 name chain -- and they
must agree. They do, on all 300, with zero disagreements, and the fan-out
histogram is [(1, 300)]: every craft has exactly one such pointer, none has zero
or two.
The fan-in is the useful half. Only 41 of the 116 roster records are referenced
at all; the other 75 have no craft. Craft per referenced record is discrete:
24 records with 2, one with 4, four with 8, and twelve with 18 -- summing to
exactly 300. The twelve holding 18 are all e007 Turret.
This is the observable four iterations have been hunting. A member with zero
craft is not deployed; a member's craft count is its live strength; so an
arrival is craft appearing for a record that had none and a kill is that count
falling. Both are attributable to a named squadron, because the record is a
UnitGroup member and UnitGroup names the squadron that Route_S02.tbl ties to a
phase and an arrival time. The chain from a memory observation to a static route
entry is complete.
Not claimed: that the 75 undeployed members are waiting on phases 2 and 3. That
is a plausible reading, but the run only shows they have no craft.
Fourth failed attempt to explain how 116 roster members become ~300 live craft,
recorded so it is not retried. Count (116), the member field n (387) and
formation slots (630) were rejected earlier; this rejects a head pointer.
link_probe.py scans the first 0x400 bytes of every object for an address of the
other kind. Roster record to craft base: 0 of 116. Craft to roster base: 0 of
300. Both directions empty.
The scan also measured the address spaces, which is the useful part. The two
structures live in different regions about 19 MB apart -- roster records at
0xbc372c00..0xbc9bc720, craft at 0xbdb2fd80..0xbdcd1d80 -- and the 14 unit
definitions match Stage 02's 14 distinct unit types exactly. The craft count is
not fixed: three runs at comparable mission times gave 296, 298 and 300, so the
population must be compared within a run and never across runs.
One thread is left open but explicitly not counted as evidence. Craft objects
hold 5101 words pointing somewhere into the roster VA range, which sounds like a
lot until you notice that range spans 6.5 MB and holds many allocations besides
the 116 records. The test that would settle it is the distance from each such
pointer to the nearest roster-record base: a spike at a single delta means a
link at base+X, a flat distribution kills the thread. Not run yet.
The association may not be a pointer at all -- an index, a hash, or a third
object such as a squadron instance would all look like this. Since an arrival is
already known to be a state change rather than an allocation, finding that
mediating structure matters more than finding a flag: it is what would let an
observed change be attributed to a named squadron and hence to a route.
The suspected confound turned out not to exist. Gaps between consecutive
same-unit definition-pointer sites are all >= 0x1000, with 274 of them exactly
0x1000, so entities are page-spaced and there are no near-adjacent pairs to
merge. Clustering at any threshold below 0x1000 gives ratio 1.00 for every unit
type, and hull is plausible on 298 of 298 clustered bases at delta 0x130. The
player shows two objects because there are two, not because one holds two
pointers.
That removes the excuse the previous iteration had used to keep the reading
alive, and the reading does not survive: sum(n) fits the turret row well (216
against 214, with kills already recorded), but DeltaSaber_T, Player and
Acropolis all come out at exactly twice their sum(n). An undershoot can be
blamed on phases 2-3 not having started; an overshoot cannot. n goes back to ❔
and the previous 🟡 is withdrawn. All the turret row establishes is that a
roster member expands into many craft, not that n is the factor.
Formation slot count was tested as the alternative and rejected outright: 630
turret slots against 214 live.
Side result worth keeping: a FormationSet record's FrameCount is its slot count,
and the name suffix usually agrees -- Turret07_30 -> 30, ArrowHead03_64 -> 64,
4_Bird -> 4 -- with one exception, AttackerS03_12 having 14 slots, so the suffix
is a label and not a guarantee.
Also recorded: the 298 live entities are not the 116 roster records. Both
structures exist at once, and the rule mapping one onto the other is the real
open question.
Probe caveat noted in the doc: entities2.moving() found no movers this run, so
the delta spectrum was empty and the clustering threshold was a fallback rather
than a measurement. It does not change the conclusion, since every gap exceeds
any threshold below 0x1000.
entities2.moving() finds entities by displacement between two samples, so
anything stationary is invisible -- the entire explanation for the +/-10 swing
that made the previous run's count useless. liveness_probe.py enumerates by
definition pointer over the entity heap instead, moving or not, and reads hull
as f32 at position+0x154. The series is monotone rather than oscillating:
298 -> 280 over 164 s, with the decline matching the 18 disappearance events
exactly.
The hunting pilot does kill: one hull crossing caught directly, an
e010_ADAN_Attacker_S at t=57 s. The previous run's worry that SYLPH_HUNT shoots
but never destroys anything is settled.
Recorded as a non-result so the next run does not misread it: zero births in
164 s does NOT favour either wave model. The roster finding already established
that every participant is allocated at mission load, so neither a clock nor an
event model would produce an allocation. An arrival must be a state change on an
existing entity.
The mystery member field n now has a candidate meaning: the number of craft a
roster member spawns. Static sum(n) for turrets is 216 against 214 sites found,
with the count already falling before the first sample, where Count alone
predicts 21 -- off by an order of magnitude.
Not promoted, and the reason is a confound in my own measurement rather than the
data: the probe counts definition-pointer SITES, not entities. The player is one
member and yields two sites, and DeltaSaber_T yields exactly double its sum(n),
so some entity types hold several pointers to their definition. Until sites are
collapsed into distinct entities the turret match could be a coincidence between
a x1 multiplicity and a x1 ratio. The capital-ship rows undershoot for a
separate and expected reason: phases 2 and 3 have not started.
The user asked for an actively hunting pilot, since a player who kills nothing
cannot trigger an event-gated wave and both previous runs used the survival
pilot.
pilot.py gains SYLPH_HUNT=1. The substantive change is which contacts ENGAGE may
shoot: it previously skipped every "hard" target -- "turrets and hulls are not
the objective" -- and stood off 2500 units from turrets, on the assumption that
an e007 Turret is an AA mount on a capital ship. It is a craft, one of the main
enemy types of the first six missions, and at 100 HP the cheapest kill on the
field. Under SYLPH_HUNT it is a target and the keep-out drops to 600. The run
confirms the pilot engages: steady ENGAGE, fire=1, committed to an
e010_ADAN_Attacker_S at ~2.2 km, hull and escorted asset untouched over 160 s.
Withdrawn: "only 10 of 116 records ever changed a byte in 170 s". This run
measured 41-56 records changing in every 10 s tick. The old figure does not
reproduce.
I cannot say why, because I changed two variables at once -- the record bound
(fixed 0x200 to bounded-by-next-record) and the pilot (survival to hunting).
Either explains it. That is a design error, and the honest outcome is a
retraction without a replacement explanation rather than a story that fits.
The conclusion it had supported is unaffected: the roster identity now rests on
the exact 10-of-10 unit-composition match measured independently.
Still open, and explicitly not concluded: the pilot's own entity scan shows ADAN
drifting 147 -> 129 -> 142, and the late rise has the shape of an arrival, but
the sample-to-sample swing is +/-10, the same size as the effect. AGENT.md warns
that polling faster than the guest updates manufactures a curve out of noise, so
no wave conclusion is drawn. The run probably did not kill anything either
(fc=0, asset at 100%), so it does not test the event-gated model. A stable
per-record liveness field and a working kill counter are both needed first;
REMAINING OB at 0xbdb59668 still does not read as a counter.
Both defects from the previous iteration are fixed by measuring instead of
assuming, and the fix immediately promotes a 🟡 result to ✅.
Labelling: the previous probe assumed object+0x04 -> name_record+0x10 -> char*
and resolved 0 of 116. wave3_probe.py searches for the chain per record instead,
the way unit_discover.py does, and resolves 116 of 116 -- every one through the
pointer at +0x04 with the string at delta 0x00, not 0x10. The 0x10 belongs to
the definition object (vtable 0x820af844); the spawned-entity record
(0x820af030) uses 0x00. Carrying one over to the other cost the last run.
Stride: measured, not assumed. Gaps between consecutive records are min 32,
median 800, with common values 800, 640, 608, 576, 416 and 32. There is no fixed
record size, so the old RECLEN=0x200 window truncated large records and overran
small ones -- which is why its busiest fields were the last words of the window.
Future diffs must bound each record by the next record's address.
With labels available, the "116 records == 116 roster members" claim was tested
properly and is promoted from 🟡 to ✅. The multiset of unit types matches the
static roster exactly: Turret 21/21, e106 Destroyer 19/19, f106 Destroyer 14/14,
f105 Cruiser 11/11, ASFrigate 9/9, ISCMissile 9/9, Attacker_S 9/9, e105 Cruiser
7/7, DeltaSaber_T 7/7, ArrowHead 6/6 -- 10 of 10 exact. A coincidental total is
possible; a coincidental distribution over ten unit types is not. The game
allocates one record per roster member at mission load.
Not settled: REMAINING OB at 0xbdb59668 held 95748078 unchanged all run. That
address is known to be run-dependent, and this was one of the misses, so the run
cannot say whether the pilot killed anything. Re-hunting it is a precondition
for the kill-versus-no-kill test, not an optional extra.
Since the record count is flat, an arrival would have to flip a field inside a
record. This diffed all 116 records every 5 s for 170 s of Stage 02 flight.
The prediction under test -- groups of 3, 3, 3, 2, 1 records changing state at
t = 90, 120, 170, 210, 240 s -- did not appear. Changes are spread evenly across
ticks with no cluster at any predicted time and no field that transitions once
for exactly three records. Three explanations survive and this run cannot
separate them: the timetable's t is not seconds (at 30 Hz the whole phase-1
schedule finishes inside 8 s, before the first sample); arrival is not marked in
these records; or the mission was not in phase 1.
Supporting, not conclusive: only 10 of 116 records ever changed a byte, and 106
never changed at all. Live entities would be moving, so that fits the
pre-allocated roster reading -- but Stage 02's roster is turret-heavy and a
turret does not move while alive, so "inert" and "not yet arrived" are not
distinguishable here.
Two defects in my own probe, recorded rather than quietly fixed:
- label() resolved to '?' for all 116 records, so nothing could be tied back
to a squadron. That association is what would have made the result
decisive -- "the three records that changed at t=90 are ADN110, ADN111,
ADN112" is evidence; "records 18, 32, 99" is not. unit_discover.py already
solves this and should be reused.
- RECLEN=0x200 was assumed, not measured. The busiest fields are the last
eight words of the window, which is what spilling into the next object
looks like.
Method error kept: the first attempt deferred all analysis to the end and the
turn timeout killed it with 240 s of data in memory and nothing written. The
probe now streams transitions to disk and prints a partial ranking every 60 s.
With a 219 s cold-boot title movie, an end-only report has ~300 s of budget and
one bad estimate loses the whole run.
The routes' first-keyframe time is the arrival schedule. It is not always zero,
and grouping Stage 02's 120 route records by phase and first-frame time gives a
timetable: phase 1 releases 25 routes at t=0 then 3, 3, 3, 2, 1 at t = 90, 120,
170, 210, 240. Phase 2 has every one of its 37 routes at t=0, which is what
pins the meaning: t is measured from the start of its phase, not of the mission.
Entering a phase releases that phase's t=0 group and the rest follow on the
offsets.
That completes the data side of the question this line of work started from --
the schedule is data, split across UnitGroup (who) and Route (when, and the path
flown in), with no fixed enemy count anywhere.
Refuted: counting spawned-entity records does not reveal arrivals. One Stage 02
flight, 210 s sampled every 15 s, counting aligned 0x820af030 in an 8.3 MB span:
flat at 116 throughout, no step at 90, 120, 170, 210 or anywhere.
The reason looks more useful than the refutation. UnitGroup_S02's Count fields
sum to exactly 116 members, and there are exactly 116 records from the first
sample on, so the game most likely allocates one record per roster member at
mission load and a route arrival activates an existing record rather than
creating one. Kept at 🟡, not promoted: n=1, and the obvious refutation -- check
another stage's record count against its member sum (S01=42, S16=2, S29=95) --
needs a save for another stage, and only slot 01 / Stage 02 exists. Noted as the
blocker rather than worked around.
Not settled: whether the timetable's t is frames or seconds (at 30 Hz t=240 is
8 s; as seconds it is 4 min), and where an arrival is observable in memory. The
live flag is presumably a field inside those 116 records, which is a well-scoped
next probe now that the record set is bounded and located.
Operational note recorded: cold boot spent 204 s in the title movie, so a 300 s
probe overran the turn and the first attempt died with its output still in the
pipe. Log to a file rather than piping to tail.
Static reading had gone as far as it could: the stage record splits a mission
into Phase_1..3 and every arrival route is phase-tagged, but nothing in the data
says what ends a phase. So this took it to the oracle -- one Stage 02 flight,
160 s under the survival pilot.
Confirmed, and this is the useful half: every string the static decode predicts
is present in live guest memory -- Phase_1, Phase_2, Route_ADN101_p1F,
SUBOBJ_010, AI_ADAN_CraftSquadron_Veteran, UnitGroup_S02.tbl. The game loads
exactly the tables the stage record names, under exactly the names we resolved,
and they can be located in RAM by content. That is the first dynamic
confirmation of the whole static table layer.
Refuted: the phase state is not adjacent to those strings. The probe reported
862 changed words around the anchors, which looks like a signal until you read
the values -- each word takes its predecessor's previous value and every value
points into the same region. It is one block shifted down four bytes, a single
memmove in a pointer list, occurring once between t=66s and t=89s. Diffing
around a string anchor was the cheap thing to try and it did not work.
Also recorded: a defect in my own probe. It scraped hit addresses with
0x([0-9a-f]{8}), but gmem.py find prints both the backing-file offset and the
guest VA, so half the anchors were file offsets read as addresses. Fixed to
match the va column only. It did not change the conclusion -- the anchor that
produced the shift was a real VA -- but a negative result from one of those
junk anchors would have been worthless.
Not settled: what advances a phase. Next handles are watching Route_ADN101_p1F
fire against entity positions, or working back from the SUBOBJ_*_Mes_L1 HUD
strings; the phase state is more likely near the known mutable REMAINING OB
counter than near the tables.
Following the real stage record (not the _Test template dumped earlier) reaches
the whole mission-parameter layer, all of it in the same self-describing IDXD
container as the squadron roster.
The big one: a stage is divided into Phase_N blocks -- three for Stage 02, each
with its own map path, map mesh, asteroid definition and background -- and
Route_S<NN>.tbl holds the arrival paths, with records named
Route_<squadron>_p<phase><kind>
tying a UnitGroup squadron id to a phase and to a time-stamped keyframed path of
(time, quat x4, pos x3). Route_ADN101_p1F is 3 frames at t = 0, 20, 30. The
identity len(fields) == FrameCount * 8 + 1 holds for 1449 of 1449 route records
across the 28 stages that have one, and 16/16 for FormationSet_S02.
Also decoded: SUBObjectiveSettings (per-objective bonus points by difficulty,
unlock item id, HUD strings) and AIParams (34 profiles, firing/guard/muster/
counter ranges plus 14 manoeuvre weights for Squad-type AI). The AIParams
numbers are exact original values from static RE and are portable as they are.
Adds tools/re-capture/stagetbl.py, which resolves a stage record by content and
can --follow every table it names, and commits two dumps as evidence.
Refuted and kept: the eight-value keyframe is the common case, not universal.
Formation_Fleet_01 has FrameCount=1 with 136 positional fields and
Formation_Fleet_02 has FrameCount=8 with 32, so a parser must not assume the
stride.
Corrects stage-definition-table.md, which was written from the _Test template
and is missing EnumerateSubobjective, EnumerateAIParams, BackGroundID and the
WingmanIconID fields the real record carries.
Not settled: what advances a phase -- the stage declares Phase_1..3 and routes
are phase-tagged, but nothing static says what ends one. That is a question for
the oracle, not for more static reading. Also open: the route-name kind letters
F/S/A/M/B, what activates a sub-objective, and StageMessageSet_S<NN>.tbl, which
does not resolve in GP_MAIN_GAME_E.pak.
stage\UnitGroup_S<NN>.tbl is now fully readable. A squadron record is Count
member tuples -- (unit model, message set, n, identity/nameplate) -- followed by
five named fields: Count, SideID, AIID, FormationID, DisableInterval. The
property entries carry their own field names inline, so the tag hash never has
to be inverted.
Two independent self-checks validate it corpus-wide, 1160/1160 each across all
28 stage tables on the disc:
- the length identity len(fields) == Count * 4 + 5, which is what pins the
member-tuple width at 4 and the named-field count at 5;
- agreement with the file's own Enumerate_Squadrons roster, which maps record
key to squadron id independently of the per-record string offset.
Adds tools/re-capture/unitgroup.py (pure static, runs no emulator) with a
--all --check self-check mode, and commits the Stage 02 dump as evidence.
Corrections to the container layout written yesterday, all three wrong:
- the 20-byte "(tag, 0, 0, count, size) section header" does not exist. It
was the file's last 16-byte record followed by a plain npool word. The
corrected layout is uniform across all 28 files; the old one failed on 9.
- squadron ids do not use a separate string base. Every offset in the file is
relative to the one string pool. The earlier "109 of 111" score was an
artefact of the uniform 7-byte id stride and had silently shifted every
name by three entries, which is why 17 TC*-named squadrons came out as
SideID=ADAN. The roster record refuted it outright.
- the roster is not always the last record; 9 stages put it elsewhere, so it
is found by its missing Count.
Refuted and kept: the 4-byte record key is not the squadron id's name hash
(0 of 112).
Not settled: what the key encodes, the member tuple's third field n, and where
the arrival interval values live. DisableInterval is only a per-squadron flag
(Yes for 31 of 1160); the durations, triggers and arrival positions are not in
this file. Formation_*.tbl and EnumSquadron_Test.tbl are next.
With the per-entity searches refuted at word and bit level, the question became
which object owns the counter. Sampling +-0x200 around it across a 4->8
transition: the control interval moved 0 of 256 words, and the step moved nine -
the counter plus four words holding ASCII '4' -> '8' NUL-padded, and four
pointers into 0xbcad2xxx that swap with them. Read live at HUD 008, all four
character slots hold '8'.
So the neighbourhood is the HUD's rendered text for this counter, which reframes
the address: it is the HUD widget's value rather than "the mission's own
objective counter" as this file called it.
Recorded against that, because it is already measured: there is no separate
mission-side copy moving on the same step. ob_hunt scans all of guest memory and
requires a match across two transitions, and it left exactly one address.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Run 8 died 54 s into its boot and this time bash named it: "176880 Killed" on the
run-canary line, which is SIGKILL. So the third failure mode is not an internal
fault - something outside the process is killing it.
And it is still not the OOM killer. Checked immediately after: oom_kill remained
0 and the allocation-stall counter did not move from 4421, so during run 8 the
cgroup never reached its limit (5.35 GB of 7 GiB), and the host had 13.8 GB
available. Two kills, no OOM evidence either time.
Rather than keep guessing after the fact, freeze_watch.sh now samples host
MemAvailable, cgroup memory.current and oom_kill on every poll and dumps the last
five samples when the process disappears - so the next occurrence carries its own
contemporaneous reading.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
An agent's commands run through a wrapper shell whose command line contains the
pattern being searched for, so `pgrep -f 'pilot\.py'` matches the shell running
that very pgrep, and `pkill -f 'fly_session|pilot\.py'` kills it mid-script. In
one session this killed two running scripts and twice made an "is it already
running?" guard answer yes because it had found itself. The [p]ilot bracket trick
does not help, because the real invocation appears on the wrapper's command line
too.
Recorded with what works instead: pgrep -x on the exact name, or ps -eo args
filtered against the snapshot-bash path that every wrapper carries, then kill by
explicit pid.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
v2 counts every KeWaitForSingleObject call per thread per second, tracks how many
distinct objects it saw, and logs the last result. Its HEALTHY baseline is a
result on its own: over a full 22-minute run the main thread cleared 500 calls/s
in 224 separate windows, peaking at 1235 calls/s over up to THIRTEEN distinct
objects, and the last result was X_STATUS_SUCCESS in all 314 windows. Not one
timeout.
So the game normally does hundreds of successful waits a second across many
objects - exactly the blind spot v1 could not see, and the reason a timeout-streak
counter reported the same single poller whether the game was frozen or healthy.
Stated as a consequence rather than a triumph: 500/s is NOT self-selecting, since
the main thread clears it constantly, so the freeze signal has to be a different
shape - a thread far above 1235/s, a new thread, or a window whose result is not
SUCCESS. That still needs a frozen sample; run 5 ended in GAME OVER at ~22 min
without freezing.
freeze_watch.sh now summarises the rate probe per thread when it captures.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Catching the in-mission freeze by hand costs a tool call every 25 s, and it
arrives anywhere between ten seconds and never - roughly half the runs that reach
flight freeze and the other half do not. freeze_watch.sh polls frozen.py, checks
the flight HUD is still on screen (the GAME OVER screen ANIMATES, so a run that
ended there is not a freeze and must not be scored as one), and on detection
snapshots what the stuck-wait probe has said.
Taking the probe snapshot at the moment of detection matters: the comparison
against the healthy-run baseline - one thread polling one Event at BE56BB5C - is
then made from the same instant rather than reconstructed afterwards from a log
that kept growing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Both exist for one diagnostic. KeWaitForSingleObject is declared kHighFrequency,
so it is silent even at LOG_LEVEL=3 unless
--log_high_frequency_kernel_calls=true is passed - and that flag writes ~157 MB
in ten minutes and slows the emulator so far that the title had still not
appeared when skip_intro's fixed 600 s budget ran out. The run was then scored
"BOOT FAILED" when it was only slow.
EXTRA_FLAGS passes one-off diagnostic cvars without making them defaults;
SKIP_INTRO_TIMEOUT lets a deliberately-slowed run have the time it needs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The gdb route works, and the recipe is written down: run-canary execs $XENIA_BIN,
so a wrapper that execs "gdb --args <real binary>" keeps the lockfile, the flags
and the process name (gdb forks and execs the real binary, so ps -C xenia_canary
still finds the inferior) while satisfying ptrace_scope=1 by being the parent.
The Release binary is not stripped - 26595 symtab entries - so frames have names.
The handle SIGSEGV/SIGBUS/SIG32-35 lines are mandatory: xenia uses SIGSEGV for
guest memory watches and the RT signals for thread suspend.
A run froze after ~4 minutes of flight, screen still "flight" rather than GAME
OVER, and all 79 threads had backtraces. EVERY ONE is in a wait - guest threads
in KeWaitForSingleObject / NtWaitForSingleObjectEx / SelfSuspend, the GPU command
processor parked idle, the main thread in poll().
It is nevertheless burning 1253 ticks per 10 s: 403 in the TimerQueue thread
(nanosleep inside TimerThreadMain) and 290 + 274 in two guest threads that the
backtrace shows blocked in KeWaitForSingleObject. A thread genuinely blocked
cannot burn 28% of a core, so those two are CYCLING - a timed wait that expires
and is re-entered - with the timer thread servicing them hot. Three samples
minutes apart show identical frames.
That refines the earlier "the guest is spinning, not deadlocked": the CPU burn is
real but it is in the WAIT PATH inside the kernel layer, not in guest code. The
shape is an event that never gets signalled.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The freeze's named next step is more expensive than it looked, and the reasons
are recorded before anyone starts it: xenia's stack walker is a TWENTY-LINE STUB
on POSIX (Create logs "unimplemented" and returns nullptr), so
ThreadDebugInfo::guest_pc is never filled here; PPCContext carries no live PC
either. A reverse host->guest map is buildable - the code cache already learns
the mapping in OnCodePlaced - but that plus a way to sample another thread's RIP
is a real emulator feature, not a patch.
The cheap route that does exist: gdb is installed and ptrace_scope is 1, so
attaching to a running emulator is refused but launching it UNDER gdb is not -
run-canary execs $XENIA_BIN, so a wrapper that execs "gdb --args <real binary>"
keeps the lockfile and flags and makes gdb the parent. That would separate
"spinning in guest JIT code" from "spinning in a xenia loop", which is the fork
this is stuck on. Not attempted yet.
ob_bitflag now locates the counter itself from the three addresses measured so
far, instead of refusing when the default one is wrong. That costs no
transitions, and with roughly half of all runs ending early, transitions are the
scarce resource.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
ob_bitflag.py is the follow-on the word-level refutation named: for every 4-byte
offset in the window and every one of its 32 bits, count how many entities have
it set, keep the pairs whose count is exactly the counter, and require them to
match again after a transition. Both polarities, since an objective could be
marked by a bit that is CLEAR on it.
Three runs, no verification, and the reasons are recorded: run 1 gave 187 + 33
candidates at counter 4 and then reported "the counter never moved" for 700 s -
about a mission that had ENDED in GAME OVER partway through; run 2 hit the same
dead mission; run 3 had the counter at a different address (the guard refused,
correctly) and then froze after one filter.
The hole is closed. frozen() asks whether the guest is ANIMATING, and the GAME
OVER screen animates happily - mean colour (114,22,63) - so every liveness check
passed while the mission was over. frozen.in_flight() classifies the screen with
screen_id, and ob_hunt/ob_flag/ob_bitflag now abort with NO LONGER IN FLIGHT.
That is the second confident negative in this investigation that was really about
a dead world, so the rule is written down: before believing "X never happened",
show that the thing that would produce X was still running.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
With the pilot finally shooting, the experiment completed both halves. Sample A
at counter 12 over 120 entities gave 2 offsets where exactly 12 entities share a
value; the counter then went 12 -> 11 and NEITHER survived. So within +-0x400 of
an entity's position triple there is no 4-byte word whose shared-value population
tracks REMAINING OB.
The limits are recorded as part of the result, because they bound it: the test
asks which entities share an EXACT 32-bit value, so a single bit ORed into a word
that also carries health or a timer would never show up - a bit-level version of
the same differential is the follow-on. Anything outside the window, or on
entities that entities2 cannot see (it types by position CHANGING, so stationary
objectives are invisible), is untested too, and the populations differed a lot
between samples - 120 against 194.
Separately: REMAINING OB went 12 -> 11, the first decrement of this whole
investigation, while pilot.py logged 411 fire=1 samples and the HUD reached YOU
KILLED WARPLANES 0003. Stated carefully - it does NOT show the counter counts
kills, since an earlier run had the hostile population fall by a third with no
movement; it shows some kills close something the counter tracks.
Two robustness fixes: ob_flag retries an empty entity sample (one void run was
caused by exactly that), and Pad releases everything on interpreter exit - a
file-backed pad PERSISTS after its writer dies, so a tool killed mid-press would
leave a button held and the game would walk through menus on its own.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Measured, not argued: command a 45-degree-off-the-nose error and watch whether it
shrinks, both axes, both sides, two pulse widths, with the opposite sign as a
control. Yaw's sign is correct (45 -> 11.3/32.7 at 0.6 s, 41.6/33.8 at 1.2 s).
Pitch's is inverted - the pilot's own sign GREW the error every time
(48.3/55.6/70.7/91.7) and the opposite shrank it every time (30.7/41.5/15.7/9.6).
A method artefact is recorded because it gave the opposite answer first: a 3 s
full-deflection pulse overshoots a 45-degree error so far that BOTH signs look
wrong (45 -> 164 and 45 -> 178). A long pulse cannot answer a sign question.
Verified against the game rather than by inspection. Before: fire=1 in 0 of 13521
samples, |aim yaw| pinned at 90.0, target 36-43 km away. After: 43 of 1732, aim
down to 2.3 degrees, range median 6.3 km, and the HUD's own ammunition counters
moving - NOSE BM 06000 -> 05723, MAIN MPM 00300 -> 00298.
Also fixed a leftover of the same FIFO era: pilot.py called pad.f.write("tap A
90") for the target-select double tap, which raised AttributeError once Pad
stopped having an `f`. Pad gained tap()/dpad(); ctrl_probe.py and target_probe.py
still use pad.f and now say so in place.
Still open: YOU KILLED is 0000 after 250 s of firing and REMAINING OB is still
012. The craft shoots, closes and selects; whether it destroys anything is next.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
flight_probe.Pad opened /tmp/sylph-vgamepad.fifo, "the vgamepad server's FIFO".
That server was removed when the uinput pad was replaced by Canary's --hid=file
driver, because a uinput device is not namespaced and scripted presses leaked to
the host's desktop. The FIFO is now an ordinary 91-byte file nothing reads, and
/tmp/xenia_pad.txt - the file the emulator polls - was 0 bytes while the
"autopilot" was flying. So every axis, trigger and button from pilot.py,
autopilot3.py, aim_probe.py and flight_probe.py went nowhere, silently.
The corpus already carried this trap for the SHELL scripts
(canary-scripted-input-traps.md, "every call here failed silently"). This class
was missed, and every flight tool imports it.
Verified against the oracle rather than by inspection, on runs confirmed
animating at both ends of every phase: before, full stick produced 0.00 degrees
of heading change over 4 s while the ship travelled 350-735 units, and the
attitude matrix at pos-0x70 was byte-identical; after, LX=-1 turns 12.72 degrees
and LX=+1 swings the flight direction from [1,0,0] to [0.13,-0.14,-0.98], with
the matrix moving 0.44/0.31 under stick and 0.0000 at neutral.
That also REFUTES the "stale attitude matrix" suspicion from the earlier pass -
pos-0x70 is live and tracks the ship; it only looked dead because nothing was
turning the ship.
Still open and said plainly: with the pad fixed a 150 s pilot run still fires 0
times, with |aim yaw| still exactly 90.0 and the target 36-43 km away. Steering
works now, so what is left is the stick SIGN against the pilot's error convention
or a target selection that commits to something too far to close. Both are
finally testable.
findrot_global/findself/findspeed/selfstate still write to the dead FIFO and now
say so in place; they are not repaired because none has been re-run since.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
fire=1 appears in 0 of 13521 samples. The gate that closes is measured rather
than guessed: over the 3004 samples that had a target, |aim yaw| is EXACTLY 90.0
degrees every time, which is the `if ez < 0` branch in sticks() - the target is
astern - with pitch near 180 and a range that grows 22km -> 49km and plateaus.
The craft flies away from what it is chasing for fifteen minutes and the turn
never completes.
What is NOT established is why, and the attempt is withdrawn rather than kept:
aim_probe.py reported the forward vector pinned at [-1,0,0] with 0.00 deg/s under
neutral, full-left and full-right stick, which looks like a stale attitude
matrix - but the guest had FROZEN partway through the probe, confirmed after the
fact by frozen.py and by the player position being identical across 3 s. A dead
world holds every matrix still. The probe is committed because it is the right
experiment; its numbers are not evidence.
One confusion resolved: today's entities2.py "0 moving triples" bind failures are
the freeze, not a tool defect - moving() types entities by position CHANGING, so
a frozen world yields nothing by construction.
live_delta.py gains a per-1MB-region summary; a flat list is useless at 236000
hits.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
live_delta.py exists for the freeze: a frozen run is guest code spinning, and
spinning code waits on guest memory, so the words still changing while frozen
should be the loop's own bookkeeping and whatever it polls. Measured control on a
RUNNING game, so the frozen number has something to be compared against:
1863296 words change in 4 s, 2.08% of the 357.7 MB of data extents. The sparse
4.6 GB costs nothing - the walk is over SEEK_DATA extents.
The freeze did not reproduce this run, so the frozen half of that measurement is
still owed.
ob_flag.py: an unreadable HUD frame is now retried rather than counted as a
mismatch. ob_read returns None on a frame it cannot read - an explosion across
the plate, a flash - and the sanity check treated that as "the address is wrong"
and aborted a perfectly good run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The control this file never had: a run with the Kernel channel on, analysed WHILE
STILL FLYING, has 2738 refused resumes on one pair - more than the frozen run's
1171. The target's own lines show why. The game runs a self-suspending worker
(NtSuspendThread on itself, a manager thread resumes it, thousands of times), and
a self-suspended thread is not host-suspended, so the host Resume legitimately
returns false EVERY cycle: 3115 refusals against 3115 resumes.
The error is named rather than buried: the warning's commit says "~7 times in a
normal boot" and this file generalised that from boot to gameplay, where the
number is thousands. The 150x anomaly was an artefact of the baseline. The
zero-CPU threads go with it - the healthy run has four of those too.
What the instrumented reproduction DOES establish is sharper than the lead was.
The last kernel event in 690000 lines is "Thread F8000204 self-suspending", with
self-suspends 3116 against resumes 3115 - but the resumer never issues another
NtResumeThread at all, so nothing was dropped in flight; every thread stopped
together. And the guest is SPINNING, not deadlocked: over 10 s while frozen the
main thread is in state R gaining 409 ticks and guest threads gain ~680 in total
while making not one kernel call. So it is guest code waiting on something in
guest memory, and the next question is which guest PC.
Two corrections fall out: "the log stopped growing" is not a freeze detector (it
goes quiet for 25 s in normal flight), and 0xbdb59668 held the counter again -
4 of 6 runs now.
freeze_report.py makes the analysis repeatable, and refuses to answer "did this
thread ever run" when the Kernel channel was off rather than reporting a false NO.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
A Stage 02 run froze at TIME 01:02 with a radio line caught mid-word. The
emulator was alive at ~200% CPU with its main thread in state R, two screenshots
six seconds apart were byte-identical, and the log ended in a spin: 1171 of the
run's 1200 "host resume was refused" lines are the single pair F80002AC ->
F8000240, starting at the line immediately after F80002AC is created, and the log
never grows again. The commit that added that warning records what normal looks
like - about 7 in a whole boot - so this is a 150x anomaly on one pair rather
than noise. F8000240 itself appears exactly once outside the spin, at creation,
and calls nothing.
Eleven of the frozen process's 79 host threads have zero CPU, four of them
consecutive late-created guest threads - the same signature as the lost resume
that c1b57f93b fixed for the title screen. That fix IS in this build, so either
there is a second window in that race or this only looks alike.
The inference is named as one: nothing here maps a guest handle to a host tid, so
"the zero-CPU threads are the ones being resumed" is a reading of two consistent
observations. And the refusals could equally be the game's reaction to a worker
stuck for another reason - log_mask=13 has the Kernel channel disabled, so not
one of F8000240's waits is visible. The next experiment is written down
concretely: reproduce with LOG_MASK=12 LOG_LEVEL=3 and map the handle to a tid.
Also caps ob_hunt's survivor listing at 40 - an aborted run printed all 21482 and
buried the line that mattered.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Three things from one Stage 02 run.
The address recurs a third time: HUD=4 RAM=4 at 0xbdb59668, so 3 of the 5 runs
measured put the counter exactly there.
The counter is NOT a live class head-count. With the counter at 4 the typed
entity list was 8 attackers, 7 friendly Delta Sabers, 7 turrets and the player -
no class has 4 members and no pair of them sums to 4. That sharpens the corpus's
existing "012 against 118 live ADAN" note from "not the hostile count" to "not
the count of any class this enumeration can see".
The flag experiment itself proves nothing, and why is the useful part. It found
20 offsets where exactly 4 of 23 entities agree, then reported "the counter never
moved" for 600 s. The guest had stopped advancing ten seconds into flight:
pilot.py logged 724 s of identical speed/yaw/pitch, and two screenshots six
seconds apart were byte-identical, max delta 0 over 863325 pixels - while
screen_id said "flight", the emulator burned 212% CPU and every liveness check
passed. So that was a fact about a dead world. Withdrawn along with it: the claim
in ob_session.sh that the counter climbs on its own in the first minutes, which
one advancing run supports and this one cannot.
frozen.py makes it a single call, checked in both directions (0 on the frozen
pair, 254 on two frames of a live run), and ob_hunt/ob_flag now say GUEST FROZEN
rather than waiting out their timeouts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
One filter to select (004 -> 008, 35897 -> 7), a second on a transition it was
NOT selected by (008 -> 012, 7 -> 1), leaving exactly one address; then three
live paired readings, RAM/screenshot/RAM, all agreeing with the HUD.
The address is the same one run 1 reported. That does NOT reverse yesterday's
refutation and the entry says so explicitly: runs 2 and 3 read a hard 0 there on
an allocated page while the HUD counted, so "it is there in every run" stays
refuted. What is withdrawn is the stronger claim that the number was meaningless
- it recurs exactly, in 2 of the 4 runs measured, and run 3's amber candidate
0xbdb49668 sits one 64 KB page below it at the identical page offset 0x9668. The
practical rule is therefore: try 0xbdb59668, check it against the HUD, re-scan
when it reads 0.
Why run 3 failed and run 4 did not is also recorded, because it is a method
lesson rather than luck: the evidence was always in the first four minutes of the
stage, and the earlier runs simply could not look often enough - every HUD
reading cost a human round trip, so the 008 step went by between two of them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Both hand attempts at this address managed exactly one filter and then lost the
counter, because every reading of the HUD costs a human round trip. ob_read.py
does it by template correlation over three fixed digit cells, and ob_hunt.py
uses that to run the whole method unattended: confirm the HUD on both sides of
the 0.9 s scan, then filter on each following transition, labelling the first as
the one it selected on and the rest as verification.
Two things measured rather than assumed:
* Normalisation is the point, not a nicety. The plate is translucent, so an
explosion turns it orange - on the 008 reference frame a cyan-stroke mask finds
6 of ~70 stroke columns and would silently read nothing. Per-cell mean/std
normalisation reads it correctly at 0.843.
* The accept rule is two-sided because the margin is narrow: over 12 cells from
four frames of known value the correct digit scores 0.843..1.000 and the best
WRONG digit reaches 0.789 (0 and 8 are similar outlines). So a floor of 0.80
AND a 0.05 margin over the runner-up, against a smallest observed correct
margin of 0.093. A bare threshold fitted between those two numbers would be
fitted to twelve samples.
Templates exist for 0 1 2 4 8 - the digits actually seen. Anything else reads as
"?" and callers must treat a "?" as unknown, never as a value. Rejects both
negatives tested: GAME OVER scores 0.05-0.18, the main menu 0.07-0.14.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
It tested one named pixel, (450,640), described as "inside the SHIELD bar". It is
not inside anything. `screenshot` crops to the GAME SURFACE, 1279x675, while
those constants were chosen for a 1280x720 window: measured on a frame that was
unmistakably in flight (TIME 01:18.05, REMAINING OB 004), the SHIELD bar's green
rows are 590-605 and the ARMOR bar's are 650-670, and (450,640) sits in the gap
between them reading (64,66,116) - blue.
That is why the corpus already carried "wait_flight.sh reported NEVER REACHED
FLIGHT while the game was plainly in flight" as an unexplained note, and it is
exactly the failure screen_id.py's own header warns about. Reproduced live here:
the script sat for five minutes reporting nothing while tapping A into the
cockpit every six seconds.
Now classified by whole-image statistics - screen_id.py's "flight" class, green
fraction 1.3-1.5% against <0.1% on every menu - which no crop or scale can move.
Two consecutive frames are required so an explosion in a cutscene cannot pass,
and the A tapping (which dismisses the objective card) STOPS once flight is seen,
because in the cockpit A is a weapon press and not a "continue".
Tested against the running game before being believed: 0 with "IN FLIGHT at 19s"
on the same session the old test had been failing on, and then in the unattended
launch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Two separate reasons the scripted route stopped, both measured rather than guessed:
* LOAD -> READY ROOM is not 28 s. Both runs on 2026-08-23 overran it, so the next
press was eaten by the transition and the run ended up in OPTIONS once and
BRIEFINGS once. wait_screen.sh now waits for the screen, with an optional
--tap that clears a dialog the caller cannot know about (a freshly restored
profile inserts "Auto-Save is active. OK?" here).
* The READY ROOM is DRAWN long before it is USABLE: it comes up with a
"Preparing to Sortie" spinner and TAKE OFF greyed out. The two states differ by
1.7 units of blue whole-image, so screen_id.py cannot separate them and should
not try. take_off_armed.py tests the label instead: 0.0000 bright pixels while
preparing, 0.1633 once armed, on three captures from two runs. It carries its
own position check - the always-enabled BRIEFINGS label below reads 0.1027 in
all three, to four decimals, so if that reference is dark the boxes are off the
labels and the answer is "unknown", not a confident wrong one.
screen_id.py gains a "readyroom" class from the same measurements; nothing else
reclassifies.
Verified end to end and unattended: boot -> title -> LOAD GAME -> slot 01 ->
READY ROOM -> TAKE OFF -> "IN FLIGHT at 34s", pilot bound and engaging.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
ob_sample.py pairs a live read of a guest VA with a screenshot and a crop of the
HUD counter, and reads the word again AFTER the shot so a sample whose two reads
disagree can be thrown away instead of believed -- that race is what left the
first three-snapshot filter with zero survivors.
ob_session.sh is fly_session.sh plus that sampler, because the evidence this
needs is a transition, and an unattended craft is dead in about a minute. Its
header records that launch_mission.sh does not yet finish unattended on restored
state, so nobody discovers that halfway through a boot.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The mission's objective counter is a big-endian u32 at guest VA 0xbdb59668 in a
Stage 02 run on the upstream baseline. It was selected on an 18->24 transition
and then tracked 24->23->22 against the HUD on its own - four readings, two
changes it was not filtered on.
That last point is the whole discipline here, because the first attempt failed it.
An earlier differential over 19->18 also produced exactly one candidate,
0xbc22e83c, which matched the transition it was selected on and was still wrong:
read live it held 26 while the HUD showed 017. One matching transition is not
evidence.
A second trap is recorded too: a three-snapshot filter requiring 19 -> 19 -> 18
left ZERO survivors, because the value moves between the memory copy and the
screenshot that reads it. Filtering on the next DISTINCT value instead found the
counter on the first try.
ob_scan.py carries the method: scan one snapshot, then filter the candidate set
against live /dev/shm/xenia_memory_* at each new value, so only the first pass
needs a 4.8 GB copy.
Stated plainly as unsettled: the ADDRESS is from one run and cross-run stability
is untested, so the durable result is the method rather than the number. And what
the counter counts - whether every OB-badged entity is one of them, and whether
that badge is a flag in the entity object - is the follow-on the pilot actually
needs to CHOOSE targets rather than just know how many remain.
Finding REMAINING OB in RAM means getting to the escort stage, and
launch_mission.sh exists for exactly that. It carried three of the dead patterns
this session has been clearing: six vgamepad calls (the command no longer exists,
so the entire scripted route pressed nothing), the XUID E0300000EFBEA3D4 which
has no profile behind it and therefore opens the sign-in dialog that swallows
every keystroke, and the emulator's stdout going to /dev/null. wait_flight.sh had
a fourth, in the tap that skips the launch cinematic.
Fixed, the route works far past where it used to stop: boot -> title -> LOAD GAME
-> slot 01 -> READY ROOM -> TAKE OFF -> the Stage 02 briefing map, with ACROPOLIS
labelled on it.
And it stops there: 384 dumps at PC 0x82307128 and no flight HUD within 300s. The
briefing is still animating (two shots a minute apart differ by RMSE 2986), so
the emulator is not frozen - flight simply never starts.
One discriminator worth recording: the TUTORIAL reaches flight and has run
completely crash-free, while this path crashes during the briefing, which is
where the stage content loads. That is consistent with the cache-flush shape the
crash has always had, and points at WHAT is being loaded rather than at how far
the navigation gets.
REMAINING OB is therefore still not found - the mission was never entered.
ship_capture_window.sh polls for the flight screen and presses F10 the moment it
appears rather than after a fixed sleep. One run gave three results.
First, the mission ran with ZERO crashes through t+152s - the first clean mission
run, where the three before it ended at 13243 / 11898 / 11497 - and it renders
and plays: player ship, starfield, full HUD, no dialog.
Second, the capture armed and wrote its file, so the mechanism works in-mission.
Third, and against expectation, the file holds NO ship geometry. 329KB, 181
deduped draws: 180 of them share a single vertex shader, all stride=28 vcount=3
prim=8 at full-screen coordinates, plus one full-screen quad, and not one draw
has positions outside the 1280x720 rectangle. The budget is not the limit -
kShipCaptureBudget is 8000 and only 181 distinct (vbase, WVP) pairs were seen -
and the scene was definitely drawing. The known-good capture from an earlier
session is 2.9 MB.
Fourth, a correction to the previous commit. It said the cache is "REFUTED as the
cure". Too strong: this run used the IDENTICAL complete cache as tut4 and
produced 0 crashes against tut4's 11497. What four runs support is that a
complete cache is not SUFFICIENT to prevent the storm and that run-to-run
variance dominates a 3-run comparison - not that the cache does nothing.
Next step is static: compare this capture's shape against the known-good one to
find why the 3D draws never reach CaptureShipDrawForRE.
resume_reliability.sh counts, per boot, whether the title's loader thread became
the CALLER of a kernel call. Counting boots where the press reached the title:
before the fix 1 of 5 ran the loader; after it, 5 of 5. Three of the post-fix
boots were driven far enough to classify the screen and all three reached the
MAIN MENU, each with the same shape - title at 216-252s, 40 kernel calls by the
loader, 6 ResolvePath reads. The failing boots before had zero of both, every
time.
Caveat kept in the doc: pre-fix runs were interleaved with compiles and a race is
load-sensitive, so the counts corroborate the mechanism rather than carry it.
Also fixes a self-inflicted confound. Two runs came back 'title_at_s=none' with
the emulator log ending in 'Killed', which I first read as the title detector
being too strict. It was not: an earlier copy of the script was still running
because 'pkill -f resume_reliability.sh' had matched the shell running it rather
than the script, and each of that copy's iterations kills the emulator BY NAME -
outside run-canary's lockfile. The harness now takes its own lockfile and refuses
a second copy, and the note records that pkill -f matches your own shell.
resume_reliability.sh boots N times and records, per boot, whether the title's
loader thread was created and whether it ever became the CALLER of a kernel call.
That is the thing the threading fix targets, and it is binary; "did the menu
appear" is a worse test because it needs the game to get further and the attract
loop confuses it.
Two defects fixed on the way, both found by the harness disagreeing with itself:
* is_title.py's glyph count alone still fired at 113s and 173s, during the
intro movie, and the single press was wasted there. Measured, not guessed:
on those runs NO loader thread was created at all, and the title handler
always creates one - even on the boots where it then fails to run. The title
check now also requires the frame to be STATIC, reusing the RMSE between the
two grabs 0.6s apart that the movie branch already computes, and raises the
glyph threshold to 800 (a real title measures ~1450-1520, the SQUARE ENIX
logo 0).
* `grep -ac ... || echo 0` corrupted every CSV field it guarded: grep exits 1
on zero matches, so the fallback APPENDED a second value and the field came
out "0 0".
Retrospective before/after from the logs already on disc, using ResolvePath after
the resume (logged at every verbosity, so it is comparable across all runs):
before the fix 1 of 5 boots that created a loader thread went on to load assets;
after it, 2 of 2. n is small and a proper count is running.
The stalled loader thread is a lost wakeup in Xenia's POSIX threading, fixed on
the canary branch as a60fe7d11 and written up here. A thread created suspended
publishes state_ and suspend_count_ in two separate lock scopes, and Resume()
waits only for state_ before testing suspend_count_ == 0 - so a resumer in the
gap drops the resume and the thread waits forever. The Linux XThread::Resume
discards that false, which is why the guest saw success.
On the first clean boot after the fix the loader thread is the CALLER on 20
kernel-call lines and issues 4 ResolvePath reads. Every failed boot before it had
exactly zero of both.
Stated plainly as not shown: that boots now reach the menu RELIABLY. One post-fix
boot, and it is confounded by the harness.
Which is the second half. skip_intro.sh's title test has now been wrong twice in
opposite directions: originally one absolute pixel (625,618) - a 1280x720
coordinate against the 1279x675 game surface, so it read the copyright line and
timed out with the title on screen - and then my replacement, screen_id.py, which
is too loose and called the SQUARE ENIX publisher logo "title" 151s into a boot,
spending the script's single press there. is_title.py now counts the green (A)
glyph over the whole frame: geometry-independent and specific, measured at 0
pixels on the logo and 1520 on a real title.
Two findings, the second only visible because of the first.
1. The harness has always had kernel logging switched off. log_mask DISABLES
categories (Kernel=1, Apu=2, Cpu=4, Gpu=8), so the long-standing --log_mask=13
meant Kernel+Cpu+Gpu off; kernel calls also log at Debug while log_level
defaults to Info. Seeing one needs BOTH LOG_MASK=12 and LOG_LEVEL=3, and no
boot log this project has taken ever contained a kernel call. boot_menu.sh now
takes LOG_MASK / LOG_LEVEL / EXTRA_FLAGS. A whole boot at Debug with Kernel on
is 23 MB, so the default was costing far more than it saved.
2. With that on, a captured failure shows the (A) handler doing everything right:
XamUserGetXUID(0, 7, ...)
NtCreateEvent(...)
ExCreateThread(..., entry=821748F0, ..., 00000001)
ExCreateThread Active: Thread Initially Suspended,
XThreadF80000CC (1F) Stack: 70880000-70900000
NtResumeThread(F80000CC, ...)
and the thread then never executing. Measured two independent ways: it makes
ZERO kernel calls - it appears 13 times in the log and every one is as an
ARGUMENT, never as the calling thread, while five other threads make 31905
calls after the resume - and its host thread has 00:00:00 CPU time while the
process sits at 546% and has burned 37 minutes of CPU in 6:46 wall.
A spinning thread burns CPU. This one has not run at all.
So the chain is: press delivered -> handler runs -> thread created suspended ->
resumed -> never scheduled. Input, the cache-flush crash and the game's own logic
are all excluded. A lost resume is a race, which is the first explanation that
fits the ~1-in-3 success rate.
Not settled: where the resume is lost, and no successful boot has been captured
with kernel logging to compare against.
Chasing why (A) does not advance the title turned up four harness bugs, one real
root cause, and a refutation of that root cause as the explanation.
FIXED, each verified:
* boot_menu.sh sent the emulator stdout to /dev/null, which is why none of the
rest was visible. It now keeps a log and prints its path.
* skip_intro.sh probed pixel (625,618) for the green (A) glyph - a 1280x720
coordinate. screenshot returns the 1279x675 game surface, where that point
is on the copyright line and reads (8,17,31). Symptom: a 600s TIMEOUT with
the title on screen. Now classified with screen_id.py; next boot printed
"TITLE at 239s -> A".
* skip_intro.sh no longer taps through the movies. Making that press real (the
vgamepad fix) was a REGRESSION: the boot then reached the title in 90s and
that title accepted nothing. menu_draw_capture.sh records the same finding
independently.
* boot_menu.sh signs in whichever profile exists instead of a hard-coded XUID.
ROOT CAUSE FOUND: naming a XUID with no profile behind it opens a sign-in
dialog, xam_dialogs_shown_ goes to 1, and IsUIActive() then discards every
XamInputGetKeystrokeEx. Measured 8,388,601 swallowed calls in one boot with the
pad log showing presses arriving normally throughout. Only B13EBABEBABEBABE
exists on disc; the script asked for E0300000EFBEA3D4. Signing in the real
profile takes the swallow count 8,388,601 -> 0.
AND REFUTED: that is not why the title is stuck. With a valid profile and zero
swallowed keystrokes, a single (A) on the title that ends the boot still does
nothing. The dialog bug was real and worth fixing; it is not the explanation.
Next probe is specific: --log_mask=13 records no kernel calls, so it is not even
known whether the game polls at the title. A log on the SUCCESSFUL
XamInputGetKeystrokeEx return, mirroring the one already on the swallow path,
answers it in one boot.
Trying to read a third paint order off the running game turned up two bugs in
the capture harness, both of which fail in ways that look like the game
misbehaving rather than the script being wrong.
1. `--audio` is not a cvar in this tree, and eight boot scripts passed it.
Xenia calls ShowSimpleMessageBox from ParseLaunchArguments, BEFORE logging is
initialised, so the symptom is a 10x10 window, no log, no guest memory and a
dialog that blocks on XIfEvent forever - i.e. a hang deep in the emulator.
run-canary`s own header documents this exact trap; the scripts predate it.
Removed from all eight.
2. `vgamepad` no longer exists - the uinput pad was replaced by the --hid=file
driver and pad.py - but skip_intro.sh still called it. The script runs
without `set -e`, so the call failed silently and the title branch pressed
nothing while still exiting 0. A caller was told "TITLE -> A" with the game
sitting on the title screen. It now presses through pad.py and exits 6 if
that fails.
The first bug is fixed and verified: the boot now reaches the title screen with
PRESS (A) BUTTON. The second is fixed but does NOT unblock the title - see the
next commit.
