Closes the 4-byte record key. tag_hash is name_hash's shape -- byte-sum
checksum in the top byte over a 24-bit modular polynomial -- with two different
constants: modulus 0x00FFFFDF (2^24-33, prime) instead of 0x00FFF9D7, and no
lowercasing, so tags are case-sensitive. name_hash explains 0 of 8643.
Recovered from the tables rather than the executable: every inline field name
is a known (name -> tag) pair, and comparing names differing in one character
gives the per-position weights 1, 0x100, 0x10000, 0x21, 0x2100, ... -- a byte
leaving bit 24 re-enters as 33, i.e. reduction mod 2^24-33. Holds where it is
easy to get wrong (distance 8 and 9 carry correctly).
A record's key is the tag of its own name: FormationSet rosters 362/362,
UnitGroup rosters 281/281, S02 squadron names 111/111 -- so records can be
addressed by name without reading the roster first.
Implemented in Python (unitgroup.tag_hash) and Rust
(sylpheed_formats::hash::tag_hash) with 3 new unit tests carrying disc-derived
vectors; cargo test -p sylpheed-formats --lib hash is 8/8 green.
Not settled: the guest routine is unlocated, so this uses exact modular
arithmetic where the game may use a Barrett step without final fixup.
Run 2 put the inducer on from flight start and froze ~96s in, against 670s
clean with it off -- n=2, contrast sharp, confounder (elapsed mission time)
still untouched.
Withdraws last iteration's 'T74/T75 move off a semaphore onto an event' as the
signature to chase: it does not reproduce. In run 2 they are on XEvent while
HEALTHY and stay there. The healthy state varies between instants, so a
one-sample-per-state diff cannot separate a freeze transition from ordinary
variation -- I read a difference of samples as a difference of states.
Reproduces across both: T68 and T69 go from not-waiting to waiting, T69 on a
semaphore and T68 on an event both times. And 21 of 24 threads unchanged in run
2 (17 of 24 in run 1), so 'not a whole-emulator stall' now has two independent
captures behind it.
Next: repeat the capture several times within one healthy run to establish which
thread states are stable before reading any frozen diff.
FormationSet_S<NN>.tbl records are slot lists -- 1 + 8*FrameCount fields,
exactly. Resolving every squadron's FormationID and comparing gives
sum(n) <= FrameCount holding 1159/1160 across all 28 stages, 0 unresolved, with
539 filling the formation exactly. The single violation is a debug leftover
(S20, AI_Test / MessageSet_test, Formation_1_only with n=2) and is recorded.
The old 'n is not the _NN suffix of FormationID' observation was right but drew
the wrong conclusion: the suffix IS FrameCount, so n=9 against _30 just means 9
units in 9 of 30 slots.
Also: FormationID does not hash into its table (0/16). FormationSet carries a
name roster record -- no FrameCount, fields are (tag, name, '') with the tags
being the record keys -- the same convention as Enumerate_Squadrons. Second
occurrence of 'keys are resolved by an in-table roster, not by hashing'.
Does not close the 387-vs-300 gap, and the key derivation stays open.
The 'S17-S23 have no stage record' gap was an artefact of enumerating by the
literal 'Stage_S<NN>'. Tutorial records omit it -- they name no per-stage .xpr
and pull AIParams/weapons/strings/subobjectives/nameplate/collision from a
shared _Tutorial set -- so all six were skipped. stagetbl.py Stage_S18 returns a
full six-record definition and always would have.
Counting distinct *_S<NN> names across all 1119 decompressed entries: UnitGroup
and Route cover 28 stages (S01-S16, S18-S23, S24-S29); Stage literal and
AIParams cover 22; SUBObjectiveSettings 16 (story only). S17 appears in none --
it is not a stage that lost its data, it does not exist.
Tutorial records carry the same Phase_1/2/3 structure as story stages, so the
tutorial is not a special mission type at the data layer.
Refuted en route: GP_TUTORIAL.pak does not hold the config -- 2 RATC entries,
zero IDXD, like GP_CHALLENGE.pak.
Caught the freeze by waiting for the event (frozen.py + in_flight) instead of
sleeping a guessed interval; freeze_waitobj.sh splits into boot/watch so the
wait is not capped by one Bash call. Verified hard: a frame minutes later is
byte-identical to the capture.
Healthy vs frozen, same run: 20 -> 24 wait frames, XEvent 19 -> 23,
XSemaphore 8 -> 7. The signature is per-thread -- 17 of 24 threads sit on the
exact object they were on, four previously-running threads park, and T74/T75
move off a semaphore onto an event. So the freeze is not a whole-emulator stall.
Also corrects the previous entry's test: screen_id reads 'flight' during a
freeze by design, which is why frozen.py exists. Re-testing the saved frames
says that run was genuinely healthy, but it was right by luck.
heavy_read.py added to test whether the instrument provokes the freeze: I/O is
free (371 MB in 0.1s, page cache), the cost is Python-level CPU. One data point
-- 670s clean, then frozen 54s after the inducer started -- recorded as n=1, not
as causation.
23 wait frames, 30 objects, nothing unresolved -- the second deref turns every
former miss into a resolved object, as predicted. XEvent 20 / XSemaphore 9 /
XTimer 1; every WaitMultiple thread waits on a pair, and 78/79/80 and 64/65 are
worker groups sharing a handle.
%ebp does not survive as the count -- WaitMultiple reuses it at 8fc158 -- so the
array is bounded by reading until an entry stops resolving instead.
The frozen capture is still not taken: screen_id reads 'flight' at the second
capture and out to ~470s, so the mission never black-screened. The diff in the
data file is two healthy captures and is recorded as such.
The 8 threads whose [rbx] did not resolve to a vtable were never in
XObject::Wait. The backtrace grep matched WaitMultiple as a substring, and
there %rbx is the XObject** array (mov %rsi,%rbx) with the count in %ebp, so
[rbx] is objects[0] -- an object pointer, needing a second deref -- not a
vtable. The unwind restored rbx correctly for all 18.
freeze_waitobj.sh now takes the function and frame index from the backtrace and
applies the matching read, and captures twice in one run (healthy and after the
~270s black-screen) so the comparison is within-run. waitobj_report.py tabulates
both and diffs them, discarding any value info symbol cannot resolve.
Re-extracting the same gdb capture per thread rather than by grep qualifies the
previous entry. Of eighteen threads whose frame 3 is XObject::Wait, eight have
[rbx] equal to the XEvent vtable plus sixteen, two equal to the XSemaphore
vtable plus sixteen, and eight hold a pointer into the mmap region that is not a
vtable at all. So the waits are on two distinct kernel types, XEvent and
XSemaphore, and the earlier claim that the object is an XEvent was right for the
majority but not the whole picture.
The eight non-vtable readings are the method checking itself rather than a
failure. A polymorphic object's first word is always a vtable pointer, so those
values are simply not this: rbx could not be restored for those frames and the
unwind returned whatever the register held. A reading counts only if [rbx] lands
in the binary's vtable range and resolves to a "vtable for" symbol; anything
else is discarded rather than interpreted. Ten of eighteen resolve and the rest
are honestly unknown.
That also settles the previous entry's worry that the 0x7ffc and 0x7ffd
addresses looked like stack. They are the shared mmap region, which holds thread
stacks and large allocations alike, so the vtable check rather than the address
range is what separates an object from a stack slot.
The follow-up run that would have added /proc/<pid>/maps classification and a
wider object dump never booted -- EMULATOR GONE at 0s, skip_intro exit 4 --
most likely a stale emulator or lockfile from the preceding gdb session, whose
process tree is parented differently and escaped the usual cleanup. So the map
classification, the multi-word object dump and the frozen-state capture are all
still unrun.
Executed the wait-object read on a live run under gdb. For every thread whose
frame 3 is XObject::Wait, rbx gives a pointer whose first quadword is
0x5555562db8f0, which is the PIE base plus 0xd878f0, and the symbol table has
vtable for xe::kernel::XEvent at 0xd878e0. The stored pointer is the vtable
symbol plus sixteen, since offset-to-top and RTTI come first, so it matches
exactly. The same vtable appears on every sampled waiting thread while the this
pointers differ, meaning many threads waiting on different XEvent instances. The
whole chain needs no DWARF and no rebuild, as the static groundwork predicted.
Two caveats, both recorded rather than smoothed over.
This is a healthy-play snapshot, not the freeze. The capture landed at 195 s of
flight with screen_id reporting flight and a non-black mean, because boot under
gdb costs about 300 s and the entire experiment has to fit inside one call --
a timeout kills the process group and takes the emulator with it, which lost an
earlier attempt outright. So this describes what threads wait on during normal
play, which is the control the frozen capture never had, but it is not the
frozen case.
And the this addresses look like host stack rather than heap. Either xenia
places these objects somewhere unusual, or rbx at frame 3 is not Wait's this
after the unwind and the vtable match is coincidence. An exact plus-sixteen
match on a known symbol is hard to get by accident, but the address range is not
what was expected. The settling check is to dump a few words at rbx and see
whether they look like an XEvent -- vtable, KernelState pointer, handle and type
fields -- or like saved registers.
Pure static analysis, no run spent. XObject::Wait's prologue does mov %rdi,%rbx
at 8fbc9c, so the this pointer lives in a callee-saved register rather than a
stack slot. And the binary carries full unwind information: .eh_frame with
127231 FDEs, which survives in Release builds because C++ exceptions need it,
including an FDE covering 8fbc90 to 8fbde2 that tracks rbx explicitly.
Together those mean that from a thread frozen deep in pthread_cond_wait, moving
to the XObject::Wait frame and reading rbx yields the XObject being waited on --
gdb reconstructs callee-saved registers during the unwind from .eh_frame alone,
with no DWARF involved. Reading the first quadword at that pointer gives the
vtable, and vtable symbols are in the symtab, so the object's concrete type is
identifiable too.
This revises the previous entry, which listed route 1 as per-frame archaeology
that must be redone whenever the binary changes, and route 2, a RelWithDebInfo
rebuild, as what would make the question easy. Route 1 is neither expensive nor
fragile: two gdb commands per thread, no rebuild, and the oracle stays
byte-identical to the binary every other measurement in this corpus was taken
against.
Not yet executed on a frozen run, which is the next step and is now a small one.
Two corrections to the plan the freeze file leaves for the next pass, both cheap
to establish and both worth having before a run is spent discovering them.
The export in the frozen frame is confirmed. The shim frame carries its ordinal
as a template parameter, 176 decimal, which is 0xB0, and xboxkrnl_table.inc line
190 maps that to KeWaitForSingleObject. The identification was previously read
off frame names; it is now pinned by ordinal.
The file's stated next step does not work on this binary. It says the wait
target is one info-args-equivalent away now that a debugger can be attached, but
readelf shows zero debug sections and only a symtab of 27807 entries. The
Release build is not stripped, which is why frames carry function names, but it
has no debug info at all, so gdb can name a frame and nothing inside it: no info
args, no p *this, no member access. The earlier note conflated "not stripped"
with "has debug info".
Two routes remain and their costs are now recorded. A prologue-guided stack read
can recover XObject::Wait's spilled this pointer from a known stack slot, needing
no rebuild and keeping the oracle byte-identical to the one every other
measurement in this corpus used, at the price of per-frame archaeology that must
be redone if the binary changes. Alternatively a RelWithDebInfo build via
build-canary makes this and every future freeze question straightforward, at the
price of a full compile and a binary that differs from the measured one.
Neither is attempted here. Recorded so the next pass picks with the costs
visible.
One new data point on the in-mission freeze. The refuted resume-spin lead rested
on refused resumes being normal -- thousands during gameplay, more in a healthy
run than a frozen one. Today's freeze log contains zero of them across 1147
lines, and the mission still froze at about 267 s with a black screen, against
2447 in an older log. So the warning is not even necessary for a freeze, let
alone sufficient, which closes the lead from the other side.
Also refuted today, before this file was found: the burst of BaseHeap::Release
failures at the end of the short log looks like a freeze signature and is not.
In the longer log the same failures span lines 1044 to 5210 and the log
continues for 2700 lines afterwards; they begin at mission load in both runs and
are routine.
The uncomfortable part is that both were already settled in
mission-freeze-resume-spin.md, as was the 0xbdb59668 address that a previous
iteration rediscovered independently. That is twice in one session that existing
work was redone.
The cause is mechanical rather than forgetfulness. docs/re/INDEX.md listed 20 of
43 notes and none of the recent ones, so searching the index for prior work on
the freeze returned nothing -- the corpus was searched, but the search was
blind. INDEX.md now carries a generated table of every note under docs/re/, 59
entries with title and status, and states outright that it should be searched
before starting an investigation. Regenerating it is a few lines of Python and
should be redone whenever notes are added.
A two-segment OB watch made the mechanism visible. Segment 1 ran clean, with the
witness reporting zero stalled samples of fifty and OB steady at 4 for 250
seconds. Segment 2, attaching to the same still-alive emulator, got "NOT IN A
MISSION (no unit definitions)".
The emulator was alive; the mission was gone. The pilot's hull reads 1500 in
every line, so the player never died. Its log stops at t=267 s with consecutive
byte-identical lines, the freeze signature. And the screen is now entirely
black, mean (0,0,0). This is the pre-existing mission-end black-screen freeze
already recorded in the corpus, caught in the act. Segment 1's clean witness is
not a contradiction, since it sampled to about 250 s and the freeze came at 267.
That corrects an earlier claim. mission-arrival-watch.md records the blocker as
removed because the emulator survives between calls, which is true, but it was
stated as though observation could be extended indefinitely within a turn. The
emulator surviving is not the mission surviving. The binding limit is the game's
own black-screen at roughly four and a half minutes and no amount of chaining
crosses it. The earlier chained run that reached 435 s of cumulative
verified-live observation did so across two segments of one mission, which fits:
its second segment froze at t=135 s.
The consequence is worth stating plainly. Any experiment needing more than about
four and a half minutes of one continuous mission is not doable on this build,
chained or not, and that includes clearing sixteen marked fighters at two kills
per five minutes. The freeze is therefore now the highest-value target in its own
right, since fixing it would unblock the arrival question, the phase-advance
question and the OB series together. It is also a long-standing known defect
rather than anything this work introduced.
ob_watch.py verifies the address against a confidence-gated HUD reading before
reporting anything, and two consecutive fresh launches settle the question:
0xbdb59668 held 3165285888 against a HUD of 4 on one, and exactly 4 on the next.
The address is not stable across launches, the old note that it recurs in about
five runs of seven was right, and the gate did its job by refusing to report a
series from an address that did not describe that run. The watcher now hunts the
address on the current run when confirmation fails, using the same intersection
method, so it no longer depends on a lucky launch.
The HUD reader is also gated now. ob_read returns a best and second score per
digit and those were printed but never checked, which is how one misread
poisoned an intersection and produced a wrong refutation of big-endian u32. A
reading is accepted only if every digit scores at least 0.80 with a margin of at
least 0.05, the rule ob_read's own docstring states.
The measurement itself is a negative. With a clean witness, zero stalled samples
of fifty, OB held at 4 for 250 seconds while the pilot targeted e010 for 1964
ticks and fired on 1635 of them. Constant fire at the marked attackers and not
one decrement, so it destroyed none, which matches the roughly two marked kills
per five minutes measured earlier. The fire rate itself rose from 4.6 % of ticks
in an earlier diagnosis to 83 % here without producing more kills.
Recorded as unreproduced rather than explained away: the run that found the
address saw the counter rise 4, 8, 12 over five minutes, and that reading was
confirmed against the HUD. This run was flat over a comparable window. Both
observations are sound and they disagree, so the rise is not a stable property
of the mission's first five minutes and presumably depends on progress this run
never reached.
Ported onto the shared harness, the HUD changed from 4 to 8 and the intersection
collapsed in a single step: one u32be survivor at 0xbdb59668, with the u16be and
u8 hits at 0xbdb5966a and 0xbdb5966b being the low half and low byte of that same
word.
Verified live rather than asserted. Reading screenshot and memory together three
times, the one legible HUD frame showed 012 against mem@0xbdb59668 = 12; the
other two frames were unreadable rather than mismatched. This independently
rediscovers the address the earliest sessions found by digit-transition hunting,
by a completely different method.
It also corrects an earlier conclusion. A previous run intersected HUD readings
of 4 and then 11, got zero survivors in every encoding, and that was written up
as eliminating big-endian u32 for the whole region. This run shows u32be holds
the counter, so the refutation was wrong. The likely cause is the input:
ob_digits.png has templates for 0 1 2 4 8 only, so values containing other
digits are misread rather than rejected, and "11" was probably one of those. A
single bad reading poisons an intersection permanently, because it removes the
true address and nothing later can restore it. The lesson is that an
intersection method needs individually verifiable inputs -- the reader's
confidence scores were printed but never gated on.
One observation reopens the arrival question in a useful way: the counter
increases, 4 then 8 then 12 across about five minutes, measured in memory so not
a digit misread. A count of remaining marked targets that rises means targets
are being added during the mission. That does not contradict the deployment
finding, since the roster is fixed at load, but it does mean the game marks new
objective targets as the mission proceeds. Watching this one address across a
whole mission is now the obvious next experiment and costs almost nothing.
Four probes were written from a blank file and each re-learned the same lessons
by losing a run: that a flat run cannot be told from a frozen guest without a
stall witness, that results held to the end of a run are destroyed by a turn
timeout, that a roster count which is not the stage's member count means a
different stage loaded and must be discarded, and that a run's witness state has
to be read before its numbers. Writing each lesson down did not stop the next
probe repeating it, because each probe started from nothing.
probeharness.py makes them structural. Probe(baseline=N) discovers the roster,
rescans up to five times and refuses to start if the count never reaches the
baseline. The witness is calibrated on construction, sampled by tick() and
reported by status() and summary(), so a probe cannot forget it, and when no
witness is found it reports UNVALIDATED rather than zero stalls. emit() flushes
on every line. craft(), strengths(), alive() and heap() supply the
roster-to-craft link, per-record liveness and the raw heap, so a new probe
writes only its own logic.
Verified rather than asserted: deploy_probe.py reimplements the per-record
deployment watch on top of it in about forty lines against wave7_probe's
hundred and fifty, and its first live run was clean -- 116 roster records, 32
witnesses at 10/s, zero stalled samples, seven losses tracked, and the TSV
written incrementally. Nothing about the result is new, which is the point: the
harness reproduces a known-good measurement.
The existing probes are deliberately not ported. They work, and rewriting them
would risk changing results other documents cite. New probes should use the
harness; old ones should be ported when they next need a change.
ob_by_hud.py now scans seven readings of the same bytes and keeps a separate
candidate set for each, as byte offsets: u32 big and little endian, u16 both
endiannesses at both alignments, and u8. The big-endian u32 reading had been
refuted, so widening rather than assuming is the point.
u32le is much the tightest at 154 candidates against u32be's 4452. That is a
hint about the encoding rather than a result, since a rarer bit pattern narrows
faster regardless of meaning.
The run is inconclusive. The HUD read 4 at every sample, so there was no second
value to collapse the sets against, and from t=136 the candidate counts are
byte-identical across five samples in all seven encodings, which is what a
frozen guest looks like -- nothing in 32 MB changed at all. The probe had no
stall witness, so the run cannot prove it either way. One is added now.
Worth stating plainly: this is the fourth probe written without a witness and
the third whose flat output could not be distinguished from a freeze. Each time
the fix gets applied to that one script. The durable fix is the shared probe
harness already noted in this file, and the lesson recurring four times is
itself the argument for building it.
What the hunt needs is unchanged: two HUD readings at different values in
non-stalled samples. The counter moves on kills, which lands back on the combat
limit, though the earlier 4 to 11 observation shows it does move.
The correlation route is gated on marked-fighter kills, which the pilot manages
at about two per five minutes. ob_read.py already reads the counter off the
screen, so ob_by_hud.py matches the displayed value against memory directly and
needs no kills at all: screenshot, read the digits, keep heap words equal to that
value, intersect across readings.
Four readings at value 4 narrowed 6156 candidates to 4312, the expected slow
drift. Then the HUD read 11 and the intersection collapsed to zero. A word
holding this counter must equal 4 at the first four samples and 11 at the last,
and none does, so within the entity heap read as big-endian u32 the counter does
not exist. It may be u16, u8, little-endian, or outside that region. Both
previous hunts assumed big-endian u32 there, so this eliminates the assumption
rather than merely failing to find anything.
The displayed value also went up, from 4 to 11 over about 340 seconds. A pure
countdown of remaining marked targets should not rise, and the deployment work
says phase 1 gains no new participants. Three readings are possible and none is
tested: the cell being read is not REMAINING OB, the digits are misread, or the
counter genuinely counts something that can increase. The two clean readings
scored 0.95 to 0.98 against their templates, but 4 and 11 use only digits that
are in the strip, which is exactly the selection effect that would hide a wrong
reading -- the template set covers 0 1 2 4 8 only, and most samples came back
unreadable.
Next is widening the scan to u16 and u8 and to little-endian, and beyond the
entity heap, which is a change to one function and costs no combat. Extending
ob_digits.png with the missing digits would also raise the sample yield, since
only two of eleven readings in a 480 s run were usable.
With the value filter in place, an attach watching turret losses -- frequent,
unlike marked-fighter kills -- narrowed from 374 candidates to 2 to 0 across
three events. That is exactly how a correlation search should behave, and it
ends in a refutation rather than fizzling out: no plain u32 in the entity heap
decrements consistently with turret kills.
Witness checked first, per the rule adopted last iteration: 12 of 25 samples
stalled, but all four kill events fall in the early non-stalled stretch between
t=45 and t=177, so the events themselves are sound.
The negative fits the objective text rather than contradicting it. Stage 02
phase 1 asks to shoot down all invading enemy fighters, and turrets are not
fighters, so a counter of remaining marked targets should not move when a turret
dies. It also rules out the cheaper reading that REMAINING OB is a general kill
or enemy-remaining tally: it ignored ten turret deaths.
The method is proven and the search region is right, since 0xbdb59668 where the
counter was originally seen lies inside the scanned range. What is missing is
unchanged -- two or three e010 kill events in non-stalled samples -- against the
same two obstacles, the freeze rate and a pilot managing about two
marked-fighter kills per five minutes. One clean 220 s run this iteration
produced zero.
The incremental-save fix is verified. A fresh mission caught one e010 event at
t=241 s and wrote 1187 candidates to disk immediately; the turn timeout then
fired exactly as before, but this time the data survived. The session also
clears the candidate file at launch, since candidate offsets are only meaningful
within one emulator instance and resuming across launches would intersect
unrelated addresses.
The correlation itself was wrong though. It matched on the delta alone, so any
two float bit patterns whose integer representations differ by the loss count
qualified, and in a heap full of positions and velocities that is thousands of
words. The 1187 survivors were things like 1044450858, about 0.1f, and
3212461993, a negative float. Candidates must now also look like a counter --
a small non-negative integer in both samples -- which removes the noise by
construction instead of hoping the intersection washes it out.
The follow-up attach logged zero events across 520 s, which reads like the
combat-effectiveness limit again. It was not: 25 of its 26 samples were flagged
GUEST STALLED, so the guest was frozen for essentially the whole window. The
witness added last iteration did its job, and the lesson is about reading it --
the run summary quoted "0 events" first and the stall count only surfaced on a
deliberate check. A run's witness result should be the first thing looked at,
before any interpretation of what the run showed.
Still unfinished, with no address identified. What is needed is unchanged, two
or three e010 kill events in non-stalled samples, and the two obstacles are now
clearly separate: the freeze rate, and a pilot that manages about two
marked-fighter kills per five minutes.
The previous commit landed the doc and the probes but its BACKLOG edit asserted
on text that had since moved, so the index did not mention the hunt at all.
That is the third time an assert-then-replace patch has silently failed this
way. The pattern is the problem: the assertion aborts the edit but the commit
that follows still succeeds, so a missing index entry looks like a clean run.
Locating the anchor first and verifying the insert afterwards, as done here, is
the habit to keep.
The objective text settles what the counter is, so the hunt can be a correlation
rather than a value scan: keep every word in the 32 MB heap that fell by the same
amount, in the same interval, as a named e010 loss. One event cut roughly eight
million words to 1056, so two or three more should leave a handful.
The run did not get them. The turn's timeout fired at t=219 s and the probe saved
its candidate set only at the end, so the 1056 were discarded and the follow-up
attach started from nothing. That is the same mistake already recorded in
guest-stalls.md, where an earlier probe deferred all analysis to the end and a
timeout killed it with 240 s of data in memory and nothing written. The lesson
was written down and then repeated in a new script four iterations later.
The attach had a second gap: 535 s with zero losses of any kind, which is
indistinguishable from a freeze, and ob_probe2 carried no stall witness so the
run cannot say which it was.
Both are fixed. Candidates are written after every event and SYLPH_OB_RESUME=1
reloads them so a chained attach keeps intersecting on the same mission, and the
witness from wave7_probe is carried here.
The underlying pattern is worth naming: each new probe starts from scratch and
re-earns the same lessons about saving incrementally and validating liveness. A
shared probe harness would stick where written-down lessons have not.
The hunt itself is unfinished. No address is identified, and finishing needs a
run that catches two or three marked-fighter kills, which is the same
combat-effectiveness limit already recorded -- about two per five minutes against
a dozen turrets.
TextIndex in sylpheed-formats::localization already indexes per-phase objective
text under S<NN>_P<n>_Objective_<i>, with hints and lose conditions alongside.
Querying it settles what the wave work spent many iterations circling, and a
disc test now pins it.
Stage 02 phase 1 is "Shoot down all invading enemy fighters while watching out
for attacks on the ACROPOLIS", losing if the ACROPOLIS is sunk. Phase 2 is
"Protect the Admiral's ship the CALIBAN until it has entered the safe zone",
losing if the CALIBAN is sunk. Phase 3 is "Destroy the interstellar cruise
missiles headed for Schlos Base", matching phase 3's roster of exactly nine
ISCMissile and SUBOBJ_013.
That explains the phase advances and shows they are not all the same kind. Phase
1 is a kill-all-marked-fighters gate -- fighters, not the turrets every run
killed almost exclusively. Phase 2 is positional and ends when the CALIBAN
reaches a safe zone, so "what advances a phase" never had a single answer.
It also closes a loop to the first session. The hints say red mission markers
[OB] indicate your targets, so REMAINING OB -- the HUD counter hunted in the
earliest iterations and located at 0xbdb59668 -- is the count of remaining
marked objective targets, which is phase-1 progress itself. The old reframing
that the counter is not a roster was right, and this names what it is. It is
also the correct signal to watch for an advance: not deployed, which only
changes once the next phase deploys, but REMAINING OB reaching zero, which is
the cause.
Method lesson recorded because the reflex will recur: localization.rs has had
objectives, hints and lose_conditions for some time, and several iterations were
spent reconstructing the same information by hand from SUBObjective strings and
the guide script. The reconstruction was not wasted -- it produced the IXUD
UTF-16BE and language\ prefix findings independently -- but the corpus should
have been searched before the disc was, and grep -rl TextIndex crates/ would
have saved the detour.
Minor discrepancy noted, not investigated: ixud.rs documents the string pool as
UTF-16BE and localization.rs as UTF-16LE.
pilot.py gained SYLPH_PREFER, a unit-name substring whose matches get their
target score multiplied by 0.05 while everything else is multiplied by 4.0. With
SYLPH_PREFER=e010 a clean 320 s run, zero stalls by the witness, killed eight
turrets and two Attacker_S.
The preference is real -- e010 kills went from roughly one across all previous
runs to two in a single run -- but it is weak. Turrets still outnumber attackers
four to one in the kill log, because target commitment and simple proximity keep
pulling the nose back to them, and phase 1 fields 108 turret craft against 16
attackers. Deployed stayed at 41 throughout, so no phase advance.
That quantifies the blocker. Clearing the marked attackers means destroying 16
craft, and at two per 320 s that is about 2560 seconds, roughly 43 minutes of
continuous verified-live flight across many chained attaches, against a freeze
rate of about two runs in five.
This is no longer a reverse-engineering problem. Everything needed to observe
the phase advance is built and validated -- the roster-to-craft link, the
liveness read, the stall witness, chained attaches and the discard rule. What is
missing is a pilot good enough to complete the mission objective, which is
game-playing work with an uncertain payoff.
The choice is recorded rather than made, because it is about how much effort one
confirmation is worth rather than a technical unknown: invest in the pilot,
accept the static answer where only the trigger is inferred rather than
observed, or attempt one very long chained run betting against the freeze rate.
The localised string tables are decoded. language\*_local_string.tbl uses a
third archive prefix, language\, and is not IDXD: the magic is IXUD and the
strings are UTF-16 big-endian, which is why earlier ASCII-oriented dumps of
these files produced garbage.
That makes Stage 02's nine sub-objectives readable in words -- rescue the ally
destroyer, destroy all enemy fighters, sink all enemy warships, rescue the
Amalthea, keep the Caliban undamaged, sink the destroyer targeting the
Acropolis, destroy all enemy cruise missiles. Ninety-one objectives are defined
across the game.
The guide script then confirms the phase structure twice over. It reads as three
acts -- marked attackers and protecting the Acropolis, then a warship whose
engine, weapons and shield projector must be destroyed, then cruise missiles
taken down one by one -- and each act matches the per-phase roster composition
found independently. Phase 3 fields exactly nine UN_e201_ADAN_ISCMissile against
SUBOBJ_013's "You destroyed all enemy cruise missiles", an exact match between
two files decoded on different days by different routes.
And it gives a mundane explanation for every null result in the arrival work.
The phase-1 objective is stated outright in the script: the attackers with the
orange markers are the targets, meaning UN_e010_ADAN_Attacker_S, of which phase
1 fields four squadrons. Every loss line in every run log reads
UN_e007_ADAN_Turret, with e010 appearing twice in total, because SYLPH_HUNT was
built to make turrets targets and SYLPH_KEEPOUT was tuned for them. The 42 kills
were overwhelmingly the wrong 42.
Kept at 🟡 because the objective text is certain but the inference that this
particular objective gates the phase advance comes from the script's structure
rather than from measurement. The test is a pilot that prioritises e010 over
e007, which is a target-preference knob rather than new decoding, with the
existing probe watching deployed jump from 41 toward the phase-2 roster.
StageMessageSet_S02.tbl is resolved. stage-mission-tables.md had recorded it as
absent from the pak; it is present, and the archive prefix is message\ rather
than stage\ like its siblings. Sweeping prefix candidates over the full 16630
entry TOC found it immediately. It holds a voice mapping, a unit message set
pointer, and a list of 31 message tables including ScriptMessage_S02_msg.tbl and
the per-character PresetMessage sets.
message\UnitMessageSet_S02.tbl then gives an independent confirmation of the
phase structure. Its 26 records have exactly four named fields: CrewCount and
PresetMessage_Phase1, Phase2 and Phase3. Every message set carries a different
preset table per phase, so phases now appear in three unrelated table families --
the stage record, the route names, and the message sets. They are not a quirk of
one file.
The promising lead did not survive. ScriptMessage_S02_msg.tbl has 149 records
whose third positional field takes values like Killed and None, which looked
like a trigger. Enumerated across S01 and S02 it is a small closed vocabulary of
None, Emergency, Killed and Noise -- delivery categories for radio chatter,
meaning how a line is played rather than mission control flow -- and the fourth
field is 1 in every record, so it is not an argument either. Rejected.
That leaves no table anywhere carrying a phase-advance condition. Everything
found is a consequence of the phase and never its cause, which points at the
advance logic living in code. The only place left is the executable, and
default.xex is encrypted on disc, so the decrypted image exists only in guest
RAM and reaching it needs a run plus function-level work. Recorded as the honest
end of the static search rather than continuing to guess at table names.
Cross-referencing Route_S02 with UnitGroup_S02 gives per-phase membership: phase
1 has 37 squadrons and 42 members, phase 2 has 34 and 36, phase 3 has 44 and 49,
with squadrons appearing in more than one phase so the totals overlap. The
measured deployed count, stable across many runs and from the first observable
instant, is 41 of 116 records.
41 against 42. The deployed set is the phase-1 roster, to within one record.
That makes every previously puzzling observation coherent. All 116 records exist
at mission load, confirmed on two stages, but only the current phase's squadrons
hold craft. Deployed is 41 before the ready room ends and never moves because
phase 1 deploys as a unit at load rather than on a schedule. No arrival was ever
observed in up to 435 s of verified-live flight with 42 kills because phase 1
never completed in any run, and phases 2 and 3 hold the other 85 members. And
the route times are within-phase fly-in timings rather than release times, which
fits the frames reading where Stage 01 phase 2's 1500, 1800 and 2100 are 50, 60
and 70 seconds at 30 Hz.
So enemies come into play per phase, deployed together at phase start, with each
squadron's route giving its entrance path and the moment it flies in.
The off-by-one is not explained and is recorded rather than rounded away. The
player's record may be counted differently, one squadron may hold craft in a way
the liveness read misses, or one phase-1 member may genuinely not deploy. A
one-record gap is exactly the size of the errors that have misled this corpus
before.
What advances a phase is now the single remaining question. It is directly
observable, since deployed would jump toward the phase-2 roster. It did not
happen in 435 s of verified-live flight with 42 craft destroyed, so it is
neither a short timer nor a small kill count. SUBObjectiveSettings names nine
objectives but carries only rewards and HUD strings, no trigger. Not yet
searched: StageMessageSet_S02.tbl, which has never resolved in the pak, and the
executable, whose decrypted image exists only in guest RAM.
The run meant to catch a deployment step at take-off removed the hypothesis
instead. One continuous series of 64 samples over about 380 s, started when the
roster first appeared and running through the ready room, the transition to
flight at +47 s, and several minutes of flight, shows deployed at 41 and craft
at 292 with not one change in either number.
So the previous iteration's suggestion that deployment is a single step at
take-off is withdrawn. It rested on ready room 39/276 against flight 41/300,
which came from different runs and was flagged 🟡 for that reason. Measured
within one run the values are identical before and after take-off, so that gap
was cross-run variance.
What this does establish is stronger. The roster is already deployed when the
roster first exists: 41 of 116 records hold craft before the ready room ends and
nothing changes afterwards. Together with the roster identity confirmed on two
stages and the earlier piloted runs, Stage 02 phase 1 has no observable
in-mission arrival at all -- not in the ready room, not at take-off, and not in
up to 435 s of verified-live flight during which the player destroyed 42 craft.
The route times still mean something, since 0/90/120/170/210/240 and Stage 01's
1500/1800/2100 are structured rather than noise. The frames reading remains the
best fit, and under it they are most likely fly-in animation timings applied at
load rather than release times.
Two things stay open and are recorded as such. Phases 2 and 3 have never been
tested: every run has stayed in phase 1 and phase advance was never located, so
a phase transition is now the most likely place an arrival could exist. And this
particular run had no pilot, so it adds nothing on event-gating; the piloted
435 s run already covers that ground.
early_probe defers everything expensive: no witness calibration, no per-record
labelling, and the heap scan uses bytes.find. Setup now completes in 0.5 to 0.8
seconds instead of about 25, so the first sample lands essentially at flight
detection.
It is still flat. Deployed reads 41 at flight+0.8 s and every sample after, with
the only change in 252 s being 41 to 40 when one squadron was wiped out. No
climb at any point. Under the frames reading that is expected, since the whole
phase-1 schedule would be finished within eight seconds of mission start.
The attempt to get ahead of flight detection did not work, and the reason is
worth recording. A second run started the probe before the launch and waited for
the roster to appear. It appeared with 116 records, but the numbers were
deployed 39 and craft 276, flat for the whole window. That is the ready room:
the roster is built before take-off, so waiting for it catches the pre-flight
scene, and the probe's window expired around the time flight actually began.
Waiting for the roster is not the same as catching mission start, and the test
as designed does not do what it claims.
The two runs together do suggest something, held at 🟡 because they are
different runs. The ready room shows 39 deployed and 276 craft while flight
shows 41 and 300, so two records and twenty-four craft appear between them. That
points at deployment being a single step at take-off rather than a schedule
unfolding during the mission.
Next is one run of about 400 s with the probe waiting for the roster, long
enough to span ready room, take-off and flight in a single continuous series,
which would show the 39 to 41 step directly or refute it.
The rescan-until-baseline retry settled the anomaly: five rescans over 50 s
stuck at 42, so not a load race. Inspecting the live mission rather than
guessing, the records include UN_S01_Asteroid_cmesh_* and UnitGroup_S01.tbl is
resident while UnitGroup_S02.tbl is not. The launch sometimes loads Stage 01
instead of Stage 02, and the discard rule caught every such run, which is why
none of them contaminated a result.
That accident supplies a test previously recorded as blocked. The roster
identity needed checking against a different stage, and no second save exists.
Stage 01 provides one: static 42 members and 13 distinct units against live 42
roster records and 13 definitions, with composition matching 20/20, 6/6, 5/5 and
2/2. One record per roster member now holds on two independent stages with
completely different rosters.
Stage 01's timetable also bears on the unit question, which has been open since
the schedule was found. Its phase 2 has entries at t=1500, 1800 and 2100. As
seconds that is 25 to 35 minutes into a single phase, with three arrivals spread
across the last ten minutes of it, which is not a plausible mission. At 30 Hz
they are 50, 60 and 70 seconds, which is exactly the shape of a paced phase.
If the unit is frames, Stage 02's phase-1 entries at 90, 120, 170, 210 and 240
are 3 to 8 seconds, every one of them before the probe's first sample, which
lands about 25 s after flight is detected. That would explain every null result
in this line of work at a stroke, and it fits what the probe has always seen:
deployed already at 41 by t=0 and never changing.
Kept at 🟡 -- it is an inference from the implausibility of one reading rather
than a measurement -- but it is now the leading explanation, ahead of
event-gating. The test is to sample at flight+2 s and compare with flight+30 s,
which needs the probe's enumeration and calibration reordered after a first
cheap sample rather than any new decoding.
The previous iteration declared route entries t=210 and t=240 unreachable
because a shell call is capped at 595 s. That was wrong, and it rested on an
assumption I never checked: launch_mission.sh leaves the emulator running and it
survives between Bash calls within a turn. Verified directly -- 611 s elapsed
and still running after the launching call had returned.
attach_session.sh attaches a pilot and probe to an already-running mission, and
attaches chain, so the window is bounded by the turn rather than by one call.
On one continuous mission: 300 s of clean observation followed by an attach that
ran 135 s before freezing, giving 435 s of cumulative verified-live wall-clock.
The craft population fell from 300 to 258, so 42 were destroyed, and deployed
fell 41 to 38. Zero confirmed arrivals throughout -- the longest verified-live
observation so far. That covers the first four phase-1 route entries, t=90, 120,
170 and 210; t=240 needs 436 s and was missed by a second.
The coverage claim is qualified rather than asserted. The wall-to-game
conversion uses 55 %, from a 16.5/s frame rate against an assumed 30 Hz tick,
but the witness has measured 8, 11, 11, 21 and 24 per second across runs. At the
low end the factor is 0.27 and the same 435 s covers only about 117
game-seconds, reaching t=90 alone. So what is established is that nothing
arrived in 435 s of verified-live phase-1 flight with 42 kills; how many
scheduled arrival times that spans depends on a tick rate that is not yet
pinned, and pinning it is what would make the result sharp.
Also refuted: SYLPH_HZ=3, a lower pilot poll rate, produced the lowest
calibrated frame rate of any run at 8/s while landing the most kills at 26. The
pilot's polling is not the throttle and lowering it does not buy game time.
Computed the observation ceiling, which should have been done several iterations
ago instead of being approached one run at a time. A turn's shell call is capped
at 595 s; boot costs about 220 s, of which roughly 190 s is a title movie the
launch script waits out because tapping breaks the title; probe startup is about
25 s. That leaves roughly 350 s of observation, and at the measured 55 % of
real-time that is about 193 game-seconds.
Against the phase-1 route schedule: t=90 needs 164 s of wall-clock and t=120
needs 218 s, both already observed with no arrival. t=170 needs 309 s and is
reachable but only on a run that does not freeze. t=210 needs 382 s and t=240
needs 436 s, and neither fits in a turn at all.
So the tail of Stage 02's phase-1 schedule cannot be tested under this harness
however many runs are attempted. Recorded as a blocker rather than improvised
around. Two things would unblock it and neither is mine to decide: a safe way to
skip the title movie, which is more than half the budget and would roughly
double the window to about 297 game-seconds covering every entry, or a longer
shell timeout if the harness permits one.
This run is discarded: frozen at t=90 with 15 stalled samples and 3 losses,
correctly flagged and not interpreted. The freeze tally is now three clean of
five, lower than the three-of-four quoted last iteration.
What stands is unchanged: zero confirmed arrivals over 300 s of verified-live
flight, about 165 game-seconds, covering the t=90 and t=120 entries. That is the
strongest statement this harness can currently support.
Correcting the previous iteration's wording. It claimed on n=1 that disabling
the periodic rescan "removes the freeze". With more runs that is too strong: the
configuration is now clean at 210, 240 and 300 seconds and frozen at 60 on a
fourth run.
The tally across configurations: the heavy probe froze at 27, 45, 83, 183 and
255 seconds; the cheap probe with a 90 s rescan froze at 183; the cheap probe
with no rescan is clean in three runs of four past 200 s. That is a real and
large improvement in survival, but it is a change in probability rather than a
fix, and the documentation now says so.
The witness handled the frozen run correctly -- losses stop at t=45 and stalls
are flagged from t=60 onward, with no contradiction between them.
Separately, the candidate search was the last Python loop over eight million
words and most of the remaining startup cost. Replaced with two numpy vector
operations; 7250 candidates found and startup is no longer the bottleneck. Boot
now dominates: about 190 s of title movie plus 35 s to flight against a 595 s
turn cap leaves roughly 350 s of observation, so boot is the only remaining
lever if longer windows are needed.
Practical rule recorded: do not treat a single frozen run as evidence. Run,
check the witness, discard the frozen ones and keep the clean ones -- about
three in four are usable now, which is workable where it previously was not.
Second consecutive run with no stall flag on any sample, which confirms that
disabling the periodic rescan is what fixed the freezes. Nine losses, four
flicker increases all correctly rejected, and zero confirmed arrivals. The
trustworthy negative now extends to 240 s of verified-live flight, roughly 132
game-seconds, so nothing arrives past the route table's t=90 or t=120 entries
either.
The flicker rate is worth noting: about one spurious increase per minute. That
is the rate at which the old rule would have been manufacturing arrivals.
The arithmetic of the run exposed a waste. Boot finished at 249 s, the probe ran
240 s, and the turn's 595 s cap fired, leaving about 100 s unaccounted for
between them -- the witness calibration and the initial craft enumeration.
enumerate_craft was iterating every 4-byte word of 32 MB in Python, eight
million steps, to find fourteen fixed needles. Replaced with bytes.find() per
definition VA, which is the same search at C speed and is what the vtable scan
already did. Not yet run.
If that recovers most of the hundred seconds the observation window grows from
about 240 s to about 340 s, roughly 187 game-seconds, which would finally reach
the t=170 route entry -- the first of the schedule's later arrivals that no run
has yet been able to observe.
Two changes this iteration, one failed and reverted, one that worked.
Trimming the witness calibration to eight spread 512 KB windows instead of the
full region found 17 candidates, none of them frame counters, and the witness
then reported 0 of 17 stalled on every sample of a run that recorded 13 losses.
That is a total contradiction, caught by the same internal check that exposed
the previous three witness failures. The frame-rate cluster is sparse and spread
sampling misses it. Reverted: two 32 MB reads once at startup is simply the
price of a witness that works.
The recurring cost was the periodic rescan, a 32 MB read every 90 to 180
seconds, and it exists only to catch craft appearing from nowhere -- which the
roster work already established does not happen, since every participant is
allocated at mission load and an arrival is a state change rather than an
allocation. Disabled.
The result is the first fully clean probed run: 3875 candidates, 32 witnesses at
11/s, no stall flag on any sample from t=0 to t=210, and eight losses spread
across it. Previous probed runs froze at 27, 45, 83, 183 and 255 seconds. This
one ended on the turn timeout. One run, so not proven, but together with the
clean no-probe control it points at recurring heavy reads rather than at memory
reading as such.
That also produces the first arrival result that means what it says. Every
earlier one carried a caveat -- a stalled guest, an unvalidated witness, a probe
degrading what it measured. This one has a validated witness reporting no
stalls, a demonstrably live guest, and a clean end: zero confirmed arrivals over
210 s of verified-live Stage 02 flight, roughly 115 game-seconds, while the
player destroyed eight craft.
It does not settle the question. The route table's t = 170, 210 and 240 entries
remain out of reach in a single turn. But it does establish that nothing arrives
in the first ~115 game-seconds of phase 1 under those conditions, which none of
the previous fifteen runs could honestly claim.
Fourth attempt at the witness, and it passes the check the previous three
failed. Searching the whole 32 MB region rather than a 4 MB window gives 6500
candidates and 32 witnesses in a clean 21/s cluster matching the measured frame
rate, and the flags have zero contradictions with the loss data: the single loss
falls in an unflagged sample and every flagged sample has none. The guest
genuinely froze at about 45 s and the instrument says so.
That leaves the freeze itself as the dominant problem. Onset across runs is 27,
45, 83, 183 and 255 seconds, median around 83, so the usable window is one to
four minutes and unpredictable. This run produced exactly one loss and is
useless for the arrival question even though the instrument finally works.
Fifteen runs have now gone into that question without an answer, largely for
this reason.
Worth being honest about the probe: "cheap" was never that cheap and the witness
made it worse. It still reads 32 MB to enumerate at startup and another 32 MB
every 90 s to rescan, and the full-region calibration added two more 32 MB
reads. The 45 s freeze came immediately after that calibration. Calibration now
samples eight spread windows of 512 KB -- 4 MB total, still covering the whole
span -- instead of reading everything twice, and the rescan interval is doubled
to 180 s. Not yet run.
Also this iteration: the 42-record anomaly recurred, the discard rule correctly
refused to interpret the run, and a rescan-until-baseline retry is now in place
so the next occurrence shows whether the count climbs toward 116 or sits at 42.
Two harness problems, one fixed and verified, one diagnosed.
Bind: three consecutive attempts failed and the session aborted rather than
flying an unattended craft, which is last iteration's guard working as intended.
The cause is that entities2 self finds the player by motion between two samples,
so a craft sitting still at mission start is invisible. The session now holds the
throttle for three seconds before each attempt. Verified: the next run bound
immediately and produced 11 losses with zero confirmed arrivals, making fourteen
runs without one. One increase, 10 to 11, was surfaced and correctly not counted.
Witness: the frame-rate-band selection is still unreliable. It found only five
candidates in band and reported 0 of 5 moved in samples where craft were being
destroyed, and a counter genuinely running at 24/s cannot fail to advance across
a 15 s sample -- it would gain about 360. They are bursty counters that moved
during the 3 s calibration and then stopped.
The cause is mine. When the witness was bolted into wave7_probe the candidate
search was narrowed to a 4 MB window, while timer_probe searched the whole 32 MB
region and found 286 candidates with a clean cluster near 17/s. The narrowing
was never justified, it was just cheaper to write. Now searches the full region
once at startup, which is a one-off cost rather than per sample, and warns
explicitly when fewer than eight witnesses are available so a weak vote is
visible instead of silently trusted. Not yet run.
Worth recording: this is the third attempt at this witness, and all three
failures share one pattern -- a shortcut in selecting the witness rather than in
the voting logic. Each was caught only because the flagged samples contradicted
the loss events in the same output. That internal contradiction is the real
check and should survive into any future version.
The graded output is the improvement. Instead of a single bit it reports how
many witnesses agree, and the sequence tells a coherent story: 11, 9, 7, 4, 1,
then 0 of 31, with the drop to zero at t=183 s coinciding exactly with the last
loss and 106 s of nothing after it. That is a real freeze, identified.
The threshold was wrong though. Flagging a stall at "fewer than half" marked the
entire run stalled, including samples in which craft were destroyed, so 11 of 31
advancing is a healthy guest rather than a stalled one. The cause is the cluster
choice: the modal rate was 93/s, far above the ~16.5/s frame rate timer_probe
measured, and those are subsystem counters that tick in bursts and sit idle in
most 15 s windows even while the game runs. Picking the modal cluster was
convenient rather than principled.
Fixed to prefer the cluster whose rate falls in the frame-rate band of 8-40/s,
falling back to modal only if none exists, and to flag a stall only when zero
witnesses advance, which is the signal the data actually supports. Not yet run.
The uncomfortable part: this run used the cheap probe and still froze, at about
183 s. The previous iteration's "0 stalled samples" came from the unreliable
single-word witness and cannot stand as validation. What the evidence supports
now is that the no-probe control ran 300 s clean, the heavy probe froze at 27 to
255 s, and the cheap probe froze at 183 s -- one run on each arm. Cheap sampling
plausibly helps but does not remove the freeze, and it is equally possible the
freeze is stochastic and the control was lucky. Recorded as unresolved rather
than resolved in the probe's favour.
Practical consequence: the usable window is roughly three minutes per run,
sometimes less, whether or not the probe is cheap. Experiments needing longer
have to survive a freeze or be redesigned around one.
Run 12, cheap probe with a bound pilot: 16 losses over 290 s and zero confirmed
arrivals, making twelve runs without one. The persistence rule earned its place
immediately -- an increase of 13 to 15 was surfaced and correctly not counted,
since it does not start from zero. Under the previous rule it would have been
invisible, and a similar flicker straddling zero was nearly written up last
iteration as the first arrival.
The stall witness, on the other hand, is unreliable. Thirteen samples were
flagged GUEST STALLED while recording losses in those same samples, and a frozen
guest cannot destroy craft, so they are false positives and the run was healthy.
The cause is the selection rule: it took the first word in a 4 MB window whose
rate fell in a plausible band, and plenty of counters advance intermittently
without saying anything about whether frames are being rendered. timer_probe had
already solved this properly -- 286 candidates, a rate histogram with a dominant
cluster near 17/s -- and that lesson was not carried over when the witness was
bolted onto the probe.
Now fixed to a majority vote: collect every candidate, keep the modal-rate
cluster, sample up to 32 of them, and report a stall only when fewer than half
advance. It also prints RUN UNVALIDATED when no witness is found, because an
earlier run printed "stalled samples=0" alongside "tick witness: NONE", and a
witness that does not exist cannot report zero stalls. Not yet run.
Consequence worth flagging: the "0 stalled samples" that validated the cheap
probe last iteration came from this same unreliable witness and should be
re-confirmed under the majority rule. The pilot-log speed analysis that
established the stalls in the first place is unaffected.
Also fixed: the entity bind now retries three times and aborts if it never
takes, instead of silently flying an unattended craft -- one run was wasted that
way this iteration, producing no kills and no information.
First run of wave7_probe: zero stalled samples across the whole run, against
three consecutive heavy-probe runs that stalled at roughly 27, 83 and 255 s. The
fix works. The guest is also visibly healthier -- 19 losses against 8 in a
heavy-probe run of comparable length -- so starving the emulator had been
suppressing the very activity the probe existed to watch.
It also reported the first arrival of the whole line of work, and that arrival
does not survive inspection. A record went 1 -> 0 at t=229, 0 -> 2 at t=259 and
2 -> 0 at t=274. Two craft appearing and vanishing within fifteen seconds is not
a wave.
The same log contains the giveaway: at t=60 a record read 13 and at t=75 the
same record read 14, an increase, with nothing printed, because the probe only
surfaced decreases. The hull-based liveness read flickers, and a flicker that
straddles zero was indistinguishable from an arrival under the old rule. The
count therefore stands at zero confirmed arrivals in eleven runs.
Two changes, neither yet exercised: every increase is printed rather than only
those from zero, and an increase from zero counts only if it persists across two
consecutive samples, with a candidate that returns to zero discarded as flicker.
Recorded because it was close: under the old rule this run would have been
written up as "first arrival observed", which would have been the
strongest-looking result so far and wrong.
The control needed no new decoding: hunting pilot, keep-out 1400, 300 s, and no
memory probe at all, judged only from the pilot's own log. It ran healthy the
whole way -- 211, 257, 250, 241 and 200 distinct speed values in successive 60 s
windows, with no decline in the last one -- against probed runs that stalled at
roughly 27, 83 and 255 s and ended with a single distinct speed across 400 lines.
So the instrument was degrading the thing it measured. AGENT.md warned that a
full scan competes with the emulator for every core under lavapipe, and the
probes grew heavier each iteration until wave6 was reading the entire 32 MB
entity heap plus about 300 extra preads every 12 seconds while the game
rendered.
The control is n=1. The contrast is stark and the mechanism was predicted in
advance, so this is recorded as confirmed enough to act on rather than proven; a
second clean control would settle it.
What it costs is worth stating plainly. Every "no arrival" result so far was
collected while the probe was running, under conditions the game was struggling
with and in some runs after it had frozen. That does not make them wrong -- the
elimination test at t=163 s sits in a window the witness later showed was live --
but none of them is as strong as written, and the arrival question deserves
re-running with cheap sampling before silence is read as evidence.
wave7_probe.py is the fix: one full enumeration, then polling only the hull word
at each known craft base, about 1.2 KB per sample instead of 32 MB, with a full
rescan every 90 s to catch anything genuinely new. Three or four heavy scans per
run instead of twenty-five. Implemented but not yet run, so the claim that it
stops the stalling is untested and the next run must report the witness first.
First run with the tick witness. It flagged a stall from t=27 s and every sample
after, and the pilot's own telemetry -- which the probe never reads -- agrees:
35 distinct speed values across the whole log and exactly 1 in the last 400
lines, against 236 in the first 400 of a healthy run. All variation is in the
first ~50 s. The witness is validated.
It earned its keep on that same run. Without it the output reads as "no arrivals
across 313 seconds with 300 craft resident" -- clean, quotable and completely
worthless, because the game was frozen for 90 % of it. Rule adopted: a run whose
witness reports a stall is discarded, and every write-up states the witness
result. Flat samples are not evidence unless the witness says the guest was
advancing.
Stalls are frequent and early. The last three long runs stalled at roughly 255 s,
83 s (after the player died) and 27 s. That makes long observation windows
unreliable, and long windows are exactly what the arrival question needs.
Leading suspect is the probe itself, and it is recorded because it is
uncomfortable rather than despite it. AGENT.md warns that a full memory scan
competes with the emulator for every core under lavapipe, and these probes have
grown heavier each iteration: wave6 now reads the entire 32 MB entity heap plus
about 300 extra preads every 12 seconds while the game renders. If that is the
cause, the instrument has been degrading the thing it measures and the earlier
"no arrival" results were collected under conditions the game was struggling
with.
Next is a control that needs no new decoding: run the hunting pilot for 300 s
with no probe at all and judge from the pilot log alone. If it does not stall,
sampling has to get much cheaper -- narrow the scan to the roster region, sample
less often, or reread only the craft bases already located instead of rescanning
the heap.
The multi-squadron kill-threshold test did not run: the guest froze before
anything was destroyed, so there were no losses to threshold.
SYLPH_KEEPOUT makes the hunt keep-out a knob instead of a hard-coded 600. At
1400 the pilot both kills and survives: hull 1500 and escorted asset 100 % for
the whole run, ENGAGE throughout, eight loss events against seven in the run
where it died at 83 s.
That let the elimination test actually run. An e007 Turret squadron was ground
18 -> 10 -> 8 -> 4 -> 2 -> 0, reaching zero at t=163 s with deployed falling 41
to 40 -- an enemy squadron destroyed outright for the first time in nine runs.
No arrival followed. ARRIVALS=0 at every sample including all those after t=163.
The hypothesis that a wave is released when a squadron is wiped out rather than
merely damaged does not survive its first test. Two further losses occurred at
t=176 and t=202, so the mission was demonstrably still live and still processing
kills; it simply produced no arrival. This refutes elimination-of-one-squadron
as the trigger. It does not refute event-gating generally -- a threshold across
several squadrons, an objective completion, or a specific squadron could all
still be the gate.
The valid window is smaller than the log suggests, about 90 s after the
elimination rather than 143 s, because the guest stalled around t=255 s. The
pilot's telemetry gives it away: the last 400 log lines contain one distinct
speed value against 236 in the first 400, with no timestamp gaps -- the process
kept logging while the game stopped advancing. Nothing in the probe output
distinguished a stall from a quiet mission, and the same ambiguity affects the
trailing flat samples of earlier runs.
wave6_probe now locates a counter advancing at frame rate, samples it each tick
and prints GUEST STALLED when it fails to advance, so future runs validate
themselves. Implemented but not yet exercised in a run.
Correction first. The sed used to derive each session script from the last
stripped the probe's arguments, so wave5, census and wave6 sessions invoked
their probes with no arguments and every derived probe has been running on its
own defaults. The previous iteration's claim that the run was "cut at 240 s by
the turn timeout, not the planned 330 s" is therefore wrong: the probe used its
default of 240. The pilot got the requested duration while the probe watched for
a different one, and the numbers were plausible enough that it went unnoticed.
No earlier conclusion is invalidated -- the windows were real, just shorter than
intended and misattributed. All three sessions now pass SECS and EVERY.
First n -> 0 ever observed: the player's own record went 2 -> 0 at t=83 s and
deployed fell 41 to 40. The signal does register elimination, not just damage.
No arrival followed, which is weak evidence against the squadron-elimination
trigger since the record eliminated was the player rather than an enemy
squadron. Two other turret records dropped from 18 in the same sample; noted
without interpretation.
The important finding is what came after. For the remaining 220 seconds the
mission was frozen -- exactly 288 craft, zero losses, zero arrivals, across 18
consecutive samples. So the usable observation window is not the probe duration
but however long the player survives. A 340 s probe that loses its pilot at 83 s
yields 83 s of evidence and 257 s of nothing, and several earlier "nothing over
240 s" results may have been much shorter in practice than they look.
That also explains why pilot.py was written to survive rather than to shoot. The
SYLPH_HUNT mode added two iterations ago drops TURRET_KEEPOUT from 2500 to 600,
buying kills at the cost of exactly the survival the run depends on.
The elimination test itself did not run: the squadron reached 14, not 0, before
the pilot died. What is needed is a pilot that kills and survives -- hunt turrets
but keep the evade and retire behaviour, or a keep-out between the two extremes.
That is tuning, not discovery.
The per-record instrument works and resolves individual squadrons. Over 240 s
with the hunting pilot, seven loss events all landed on the same record,
tracking one e007 Turret squadron from 18 craft down to 2 while deployed held at
41 and the global count fell 300 to 284. Losses come in steps of 2 after an
opening drop of 4, which is unexplained and recorded rather than smoothed over.
This weakens the frame-rate explanation from the previous iteration. At ~16.5
fps against a 30 Hz tick, game time runs at about 55 % of wall-clock, so route
entries t = 90 and t = 120 land near 163 s and 218 s wall. The run reached 234 s
wall, roughly 129 game-seconds, passing both, and no arrival occurred at either.
"The runs were too short" no longer covers t = 90 and t = 120, though it still
covers 170, 210 and 240 -- the run was cut at 240 s by the turn timeout rather
than the planned 330 s, so t = 170 was never reached.
It also sharpens the event-gated model into something testable. The squadron
ended at 2, not 0, and no squadron has been eliminated in any run so far. If the
trigger is a squadron being wiped out rather than merely damaged, every
observation to date is explained: seven kills produced no arrival because they
never finished anything off.
Next is the cheapest decisive experiment yet available: run 60-90 s longer so
that squadron reaches zero and watch for a 0 -> n in the following samples. The
~210 s title movie at boot remains the binding constraint, leaving about 350 s of
observation per turn.
