Decodes the table found at the end of every phase region. Layout:
int N
N x [ int offset ; float t ; int kind ] -- 8-byte typed records,
tag 0x19 int, 0x1A float
1 + 3N matches the record count in every phase measured (Stage 02: 76/40/55
records for N = 25/13/18).
Checks, all independent of each other:
schedule entries disc-wide 675
0x1A float records disc-wide 675 (counted by a different route)
offsets landing on the instruction stream 675/675 = 100.0%
control, random 4-aligned offsets 33.3%
The floats are seconds -- 0, 0.5, 1, 4, 5, 30, 50, 60, 90, 120, 150, 170, 180, 210,
240, 270, 300, 330, 360, 420, 570, 1020, 1080, 1140, 1170 -- and the targets are small
one-shot coroutines that set arguments, call one built-in and end_coroutine. kind is
0 (556) or 5 (119) and is not identified.
Runtime cross-check, recorded as consistency rather than confirmation: the closed
REMAINING OB work measured Stage 02's squadron arrivals at t = 0, 120 and 210 s over
n=5 emulator runs, and all three appear in phase 1's static schedule, with 120 and 210
each appearing TWICE. These are round numbers and phase 1 has ~22 distinct times over
0-1170, so presence alone is not unlikely; the doubling is the sharper detail and was
not predicted in advance.
New artefacts data/isl-stage02-schedule.txt and data/isl-schedule-all.txt with a
committed generator (isl_report.py schedule). calls, phase-ends, conditions and
phase-guards all regenerate byte-identical.
Not settled and said so: kind is unread; the consumer is unread, so the decode rests
on the structural checks above; whether the clock is per-phase or per-mission is an
inference from the layout; and this is NOT what starts the unreachable code -- 0 of
the 675 targets are unreached run-starts, so that ~15% gap stands.
Disassembling the three Stage-02 entry_a targets shows opcodes 0x19 and 0x1A, and the
ISL dispatcher's table has 25 entries (cmplwi 0x18). They are not instructions. Each
phase region ENDS with a trailing data table of 8-byte typed records -- tag 0x19 = int,
tag 0x1A = IEEE float (0.0, 0.5, 1.0, 4.0) -- and entry_a is where it starts.
Confirmed across the disc: in 44 of 44 phases the first offset whose opcode exceeds
0x18 is exactly that phase's entry_a, with zero exceptions, and only two tags ever
appear (1394 x 0x19, 675 x 0x1A). So the record is
0x1883, base, size, 0, code_end, force_end_handler
one boundary and one entry, not two entries as the previous commit said.
That also retires this thread's own "82 of 88 land on a valid instruction = 93.2% vs a
38.6% control" as TOO WEAK a test: a data record has length 8 and passes "nonzero,
even". The entry_b result stands on different evidence -- those targets were matched
against isl.call_sites(), an independent enumeration.
isl.linear_offsets was decoding all 2069 data records as instructions, 1.23% of the
stream. Now each phase's walk stops at its boundary:
decoded instructions 168251 -> 166182 (= 168251 - 2069, as predicted)
opcode > 0x18 2069 -> 0
call sites covered 25705/25705 -> 25705/25705
exits unreachable 0 -> 0
conditions unknown 400 -> 400
Recorded because the first attempt at the fix was worse than the bug: it destroyed 36%
of the stream (168251 -> 107596, exits 0 -> 74) because linear_offsets is ONE global
walk from the first phase base, so stopping at phase 1's table lost every later phase.
It has to skip the region and resume at the next base. A count moving hard in the
wrong direction is the same signal as one that will not move.
Still open: the table's contents are undecoded -- its int values land on the
instruction stream 46/51 against a 29.5% chance rate, but 0 of them are unreached
run-starts, so this is not what starts the unreachable code either.
data/isl-phase-guards-all.txt goes from 5 of 177 unreachable exits to 0.
The cheap first step failed, usefully. An unreached routine's entry offset does NOT
appear as a word anywhere in the file, in any encoding: phase-relative 6.6% against an
11.5% control on reached offsets, absolute 1.6% vs 3.3%, and the /4 forms 0-1.6% vs
6.6-8.2%. Every variant is at or below its control, which rules out the whole family
of "some instruction operand points at them". It also rules out dead code: Stage 02's
3069 unreached instructions contain 485 calls, including start_coroutine x75,
squadron_attack x59, set_group_speed x42 and objective_marker x13.
The answer is the mission-level stream that isl-bytecode.md already partly read. Each
0x1883 record is
0x1883, base_delta, size, 0, entry_a, entry_b ; entries PHASE-RELATIVE
Measured over all 28 stages, 82 of those 88 values land on a valid instruction --
93.2%, against a 38.6% chance rate for a random 4-aligned offset. In Stage 02
entry_b is the phase's force-end handler: 0x1482C, 0x249F0 and 0x34A10, two of which
were exactly the unreachable exits, and the third being already reachable is the
consistency check.
Seeding them: exits unreachable 5 -> 0. Those exits now report 0 necessary
conditions, which is what an engine-entered abort handler should look like.
Recorded because it is the same mistake twice: the first seeding attempt moved NOTHING
(reach 85.0% -> 85.0%, exits 5 -> 5). dominating_conditions() builds its own entry set
and did not use the one I had patched -- fix-the-instance-not-the-class again, caught
only because an unchanged count is by now a standing signal.
Not settled and stated: reach went only 85.0% -> 85.2%, so what starts the other ~15%
of code is still unknown, and the negative above says it is not an operand in the file;
entry_a is unidentified; 6 of the 88 values do not land on an instruction.
Dominance said a phase cannot end unless X. A port also needs "once X holds, it
must end", and that is a must-reach set: nodes from which END_PHASE is unavoidable,
as a least fixpoint where n qualifies when it has successors and ALL of them qualify.
The conservatism is deliberate and is the honest answer: a loop never enters the set,
because a poll loop reaches its exit only if the polled predicate eventually becomes
true, which is a liveness property rather than a graph one.
A dominating condition is a TRIGGER when the successor it takes on being satisfied
lies in that set. Over all 28 stages: 732 dominating conditions, 234 triggers
(31.97%). isl_report.py phase-guards now tags every line precond / TRIGGER.
The split lands where it should. Stage 02's phase-1 objective exit is six
preconditions -- player alive, TCN004 destroyed, t <= 210, ADT102/ADT107/ADT113
destroyed -- and exactly ONE trigger: hp_pct_test(ADN101, 0) != 1. Destroying ADN101
is what fires the phase. That is a sentence a port can implement.
Per-exit distribution over 172 reachable exits: 89 have exactly one trigger, 42 have
none, 41 have several. The 42 with none are not a failure -- they are the exits no
branch fires; Stage 02's 0x006260 ends on read_freg(0) < 1200, a timeout, and time
passing is not a property of the graph, so declining to call it a trigger is correct.
Recorded as a heuristic rather than a rule: "the first trigger is the point of no
return" holds for 33 of the 41 multi-trigger exits, with 8 counterexamples where a
precondition appears after a trigger. The likely cause is that the listing is
ordered by file offset, which is not execution order -- coroutines and jumps let a
lower offset run later. Not asserted.
calls, phase-ends and conditions all regenerate byte-identical; the two phase-guards
artefacts change only by gaining the tags.
isl_report.py now accepts a directory, so the dominance analysis runs over the whole
disc: data/isl-phase-guards-all.txt, 177 phase exits, of which only 5 (2.8%) are
reachable from no static entry. CFG reach ranges 69.5% (S26) to 95.8% (S25), median
about 4 dominating conditions per exit.
The lopsided number in the per-stage table was the six TUTORIAL stages, S18-S23, each
with exactly ONE exit and exactly ONE dominating condition. That could have been a
degenerate result, so I looked: it is the same condition in all six,
END_PHASE <- builtin104() != 1
and isl-builtins.md reached built-in 104 from call-site USAGE alone -- "S18-S23 only,
followed by wait_s 39/39, preceded by end_coroutine 37/39, a textbook poll loop".
Usage said 104 is the tutorial's polled test; dominance says it is the tutorial's
clear condition. Two unrelated methods, six for six.
Stage 16 -- the corpus outlier whose script may be compiled C++ -- resolves as well:
read_freg(0) < 600, player_gauge0_test, player_gauge1_test, and two builtin141 calls
differing in a single argument (0 vs -4000), which is the shape of a position or zone
test. builtin141 is unread, so it is not named.
Stage 02's separate artefact regenerates byte-identical.
Also added: an RLIMIT_AS cap in isl_report's entry point. The dominator pass
OOM-killed a run earlier on this 15 GB box; a bad input should now fail the process
rather than the machine.
Still not settled and stated in the doc: dominance gives necessary, not sufficient,
conditions; the 5 unreachable exits need the trigger queue at phase+272; builtin104,
builtin141 and builtin7 all appear in clear conditions and are unread.
Closes the backlog's "which condition guards each END_PHASE". With the CFG from the
previous commit this is a graph query, not new machinery.
The obvious query is WRONG for this language, and I implemented it first: "one
successor reaches END_PHASE and the other does not" finds 1/62/1 guards across Stage
02's three phases, and the 1s are both the same read_freg(0) < 1200 timeout -- every
objective test missed. The cause is the dominant idiom: a POLL LOOP's loop-back
branch also reaches the exit, one iteration later, so neither successor discriminates.
The asymmetric 1/62/1 is what exposed it; a uniform number would have read as
plausible.
Dominance has no such blind spot: a condition dominates an exit when every path from
an entry passes through it, so it is NECESSARY for the phase to end that way, and a
poll loop's test dominates its own exit by construction. Iterative dominators
converge in 3 passes over 15670/18739 instructions (83.6%).
Result for Stage 02 -- every exit in all three phases is dominated by
unit_hp_pct(TCN001, Character_Player_Test) != 0, the player's ship being alive, which
falls out rather than being assumed. Beyond that, phase 1's objective exit requires
hp_pct_test on ADT102, ADT107 and ADT113; phase 3's requires ADT301 and ADT302;
read_freg(0) gates at 210 / 300 and times out at 1200; random(3) and random(5)
dominate only the exits that pick one of several closing lines.
Two of the 15 exits are reachable from NO static entry, both FORCE_END_PHASE. That
agrees with the independently measured 389 unreachable routines: they are started from
the trigger queue at phase+272, by data rather than code.
Practical note recorded: the first dominator run was OOM-killed -- 6743 nodes each
holding a Python set of up to 6743 elements. Integer bitmasks run in seconds.
Not settled, and said so: dominance gives necessary, not sufficient, conditions; only
Stage 02's artefact is committed; one listed condition is still an unresolved
<unknown>; read_freg's units are inferred from the gate values, not read.
calls, phase-ends and conditions all regenerate byte-identical.
The linear walk's 10% unknown was a floor imposed by the method: a block entered only
by a branch has a well-defined state, just not one a straight-line pass can see.
tools/re-capture/isl_cfg.py replaces it with a worklist fixpoint that joins each
block's state over its ACTUAL predecessors -- a value survives only if every
predecessor agrees.
Over all 28 stages:
instructions reached by the CFG 85.0%
condition sites, unknown LHS 756 (10.00%) -> 402 (5.32%)
of those, never reached at all 389
joined away (predecessors disagree) 13
both resolve but DISAGREE 161 <- linear walk was wrong here
Those 161 are on top of the 889 the previous jmp fix caught.
Two zero-results on the way, both my own bug, both caught because the number looked
wrong rather than because a test failed:
* The first CFG run reached only 36% of instructions and made things WORSE (35%
unknown). Cause: the phase bases reach almost nothing. Most routines are
COROUTINES the engine starts from its trigger queue, with no static predecessor,
so every start_coroutine target has to be seeded as an entry.
* That seeding then found ZERO entries in a file with 216 start_coroutine calls,
because the target is staged in TWO steps -- special[0] = imm, then
local[0] = special[0] -- and I matched only the direct-immediate form.
Reachability went 36% -> 64% -> 85% as each was fixed.
The 389 still unreached are an honest limit rather than a gap: nothing in the bytecode
starts them; they are entered from the trigger queue at phase+272, by data rather than
code, so no purely static analysis reaches them.
isl_report.py conditions now uses isl_cfg; calls and phase-ends regenerate
byte-identical. Stage 02 unknowns drop from 71 to 25.
Reading builtin80's body (0x82268460) to name it: it is NOT a predicate. It
allocates a 20-byte object, stamps vtable 0x820A8CB0, magic 0xAB0311BA and the
unit's live object into it, pushes it onto a queue via the same helper push.i uses,
and returns 1 -- or 0 when the unit is absent. A command.
That made the conditions listing impossible: it showed a six-way switch
`if builtin80(TCT206) == 0 … == 5` on a function returning 1 or 0. Disassembling the
site shows two unconditional `jmp`s between the call and the compare, so 0x1B6C0 is
reached ONLY by a branch and its special[0] has nothing to do with builtin80.
op12 is unconditional -- the next instruction is never reached by fall-through -- and
the tracker walked through it exactly as it had walked through end_coroutine. Last
iteration I fixed the instance and not the class, leaving 22x more bad sites in place
than the fix removed.
A/B over all 28 stages, 7563 sites, resetting at jmp as well:
sites whose operands change 889 (11.75%)
LHS unresolved, before -> after 34 (0.45%) -> 756 (10.00%)
So the previous commit's headline "0.0% unresolved" was a MISSING CHECK, not a strong
result: the linear walk always had some value to report, and reporting it was the bug.
10% is the honest figure and the other 90% is trustworthy for a reason.
Also corrected: isl-unit-args.md illustrated its diff with 0x1B6C0, which is one of
the bogus sites. The UNIT_ARG result itself stands -- it came from reading
implementations, not from this listing -- but the example was picked from bad output.
Not done, and said so: recovering the 756 needs a dataflow join over each block's
actual predecessors, a CFG fixpoint rather than a linear pass. The branch targets are
all known so the CFG is available; the analysis is not written.
calls and phase-ends regenerate byte-identical; conditions changes on 187 lines.
isl.py's UNIT_ARG decides whether a built-in's slot-4 operand prints as a unit name
or a raw number. It was inferred statistically from operand ranges and, by its own
comment, listed a slot "only when the ratio stayed below 1.0" -- conservative.
The vtable base makes it a lookup instead: every unit-taking built-in's implementation
opens with lwz 324(phase) / lwz 4(argbase) / rlwinm 2,0,29 / lwzx / lwz 4(rec). Read
directly for all 147:
implementation indexes [phase+324] by an argument 55
of the statistical set's 31, confirmed 31 (zero false positives)
UNIT_ARG claims a unit, implementation does not 0
implementation says unit, UNIT_ARG missed it 24
The 24 include builtin80, group_ratio_pct, is_engaged, set_unit_flags,
squadron_trace, wait_units_ready and deploy_and_wait. Hand-verified by reading
builtin7, 16, 80, 105, 117 and 136.
Recorded because it nearly passed: my FIRST control -- whether the additions' operands
resolve to a symbol-table-2 index -- is worthless. The additions score 100.0%, but so
do the 31 baseline (100.0%) AND the 92 built-ins in neither set (99.3%). Symtab 2 is
dense enough that almost any small integer lands in it. A control the negative class
also passes is not evidence.
The control that discriminates is the tag word: a symbol operand is a two-word pair
whose first word is the constant 1, so slot0 == 1 exactly when slot 4 is a unit --
100.0% (13677 calls) / 100.0% (140) / 2.5% (2903). A 40x separation.
Artefacts: isl-stage02.txt and -phase-ends.txt regenerate byte-identical; -conditions
changes on 28 sites, every diff line pairing, each a raw number becoming a unit name.
Left unnamed on purpose: all 24. builtin80 returns a small enum (tested 0..4 in a
switch) but its body past the liveness check is unread; builtin103 is a predicate over
[phase+10152]/[phase+10156]; builtin105 tests a unit record's +16 against 4.
The listing showed end_coroutine as the left-hand side of 34 comparisons disc-wide.
That is impossible -- it returns no value a script can test -- so it was the bug
reporting itself.
The recorded fix ("set special[0] only for built-ins that write [phase+164]") is
REFUTED. end_coroutine's handler 0x82272624 is `addi r11,r0,1 ; addi r3,r0,3 ;
stw r11,164(r31)` -- it DOES write [phase+164], so that filter would have kept it.
Reading the handler before writing the filter is what caught this.
The real cause: end_coroutine returns 3, which DESTROYS the thread. Execution does
not continue past it, so the instructions following it in the flat stream belong to
a different routine and every tracked value is stale. The linear walk that makes
the decode possible is exactly what walks across that boundary.
A/B over all 28 stages, 7563 sites, resetting the tracker at end_coroutine:
sites whose operands change 34 (0.45%)
LHS = end_coroutine, before -> after 34 -> 0
left as an explicit unknown 34 (0.45%)
The two counts being equal is the result: the leak was confined to exactly the sites
that displayed the impossible value, so the other 7529 conditions were never
affected. Those 34 now print "<unknown: reached after a coroutine boundary>".
Not done, and said so: their RHS is still exact and the LHS is recoverable by seeding
the tracker at coroutine entries, whose targets are staged slot 0 of start_coroutine.
data/isl-stage02-conditions.txt regenerated; calls and phase-ends both byte-identical.
The deque ops are an EXPRESSION STACK: push the left operand, evaluate the right
(a built-in call, whose result lands in special[0]), pop the comparand back into
special[1], compare. Tracking that through the linear decode is enough to recover
what each site tests.
Evidence the model is right, not just plausible:
push vs pop across all 28 stages 1877 vs 1877
files that underflow or end unbalanced 0 of 28
Stage 02 pop.i sites followed by cmp.i 319 / 319
ops immediately before a pop.i call x313, cmp.a x6
isl.conditions() recovers 7563 condition sites disc-wide with 0.0% left as an
unresolved special[N]; 83.2% have a built-in call as the LHS and 99.7% compare
against a plain number. Most-tested: hp_pct_test 1955, unit_state 1257,
unit_relation 796, dist_lt 450, unit_alive 413.
They read as conditions now:
if unit_alive(TCN105) != 1
if hp_pct_test(ADT308, 0) != 1
if dist_lt(ADT308, TCN000, 15000) != 1 (world unit = 1 m, so 15 km)
if unit_state(ADT308) == 1
data/isl-stage02-conditions.txt was a stale artefact with NO generator -- the thing
isl_report.py's docstring complained about. It has one now (isl_report.py
conditions). The calls and phase-ends artefacts both regenerate byte-identical, so
the change is additive.
Recorded rather than glossed: 15 of Stage 02's 965 sites (1.6%) attribute the LHS to
end_coroutine, which returns no value -- the tracker sets special[0] on EVERY call,
so those show a stale value and are wrong, not imprecise. The fix is to set it only
for built-ins that write [phase+164], which the vtable work makes checkable.
Answers what the previous commit left open: naming the branches did not give a
clear condition, because that needs the operand chain feeding each compare.
First, a correction to my own work. isl-bytecode.md -- which OWNS the opcode table
-- already named ops 21-24 push.i/push.f/pop.i/pop.f. isl-branches.md, which I
wrote last iteration, said op21 and op23 were unread. The stale file was mine.
Verified from the thunks rather than accepted: 21 pushes [phase+168] onto the deque
at phase+44, 22 pushes [phase+184] onto phase+64, and the 23/24 handlers touch only
r3+168 and r3+184. So pop.i lands in special[1].
New: the 147-entry built-in table is a thin DISPATCH LAYER, not implementations.
Each stub resolves the local[] argument base and tail-calls a fixed ScriptPhase
vtable slot. 112 of 147 dispatch that way; 17 write [phase+164] inline; 0 write
+184. Every named predicate is in the vtable group -- unit_state 184, unit_alive
188, hp_pct_test 64, dist_lt 56, is_engaged 252, timer_elapsed 372 -- which is the
control that the split separates engine queries from script bookkeeping.
The vtable is 0x820A84BC, derived from a known implementation rather than a stride:
MARK_LAST_PHASE is documented as [phase+300]=2; the function 0x8226B498 is exactly
that stub; it appears as a data word at exactly one address, 0x820A8570; built-in
39 uses slot 180. The check NOT used in the derivation: built-in 40 mark_not_last
uses slot 176, and slot 176 holds the [phase+300]=1 stub. Predicted and confirmed.
The db's own vptr_writes independently lists 0x820A84BC, written at 0x82261B80.
unit_state = slot 184 = 0x8226ADF0, which indexes [phase+324] by local[4] and writes
its answer to [phase+164] = special[0] at both exits. The phase-3 poll loop now
reads end to end: unit_state(ADT308) -> special[0]; pop.i -> special[1]; cmp.i; beq.
isl.py names ops 21-24; the calls artefact regenerates with NO diff.
Left open and said so: the other 111 vtable slots, which comparand each site pushes,
the 35 non-vtable built-ins, and the vtable's length.
Closes the backlog item that was the last thing between the flat decode and a
per-phase clear condition, and closes isl-builtins.md's standing "op10 + op13 look
like a switch -- NOT confirmed".
op10 resolves two operands, issues a SIGNED cmp, and writes three condition bits to
a bitset at phase+24: bit 0 = EQ, bit 1 = GT, bit 2 = LT. op11 is the same machine
for floats via fcmpu. op13-op18 branch on those bits to [phase+232] + word@+4 --
the same phase-relative target form as the unconditional op12:
13 bit0 set beq 16 bits 2 then 0 ble
14 bit0 clear bne 18 bits 1 then 0 bge
15 bit2 set blt 17 bit1 set bgt
13/14/15/17 are byte-identical apart from the bit index and the polarity. All six
relations are present and each appears exactly once; that completeness is the check
that the reading is right, rather than the usage pattern -- which the item
explicitly warned against.
Operand order recorded because it is easy to reverse: LHS = (kind byte[1], word@+4),
RHS = (kind byte[0], word@+8).
Method note in the doc: the jump table at 0x822635FC holds THUNKS, and the handler
is the bl target inside each. My first pass guessed handler addresses at a fixed
stride, landed mid-function, and produced a 20-line "difference" that was pure
misalignment.
isl.py names the ops; data/isl-stage02.txt is regenerated and every diff line pairs
exactly, only the op-name column changing (op10->cmp.i x5, op13->beq x4,
op14->bne x1). data/isl-stage02-phase-ends.txt now shows the phase-3 poll loop
reading as one: unit_state(ADT308) -> op23 -> cmp.i -> beq back to 0xFEB4.
Left unnamed on purpose: op23 (0x82271C30) and op21 (0x82175C20).
Two files (isl_report.py's docstring and structures/isl-builtins.md) recorded the
same blocker on a faithful per-phase condition listing: that it needs the coroutine
entry points from start_coroutine's operand. Measured against isl.call_sites(),
which enumerates by scanning the encoding rather than by decoding and so is an
independent denominator:
linear + jumps, stopping at ret (what the tool did) 133 / 2846 = 4.7%
linear + jumps, continuing past ret 2275 / 2846 = 79.9%
... + following start_coroutine (the recorded fix) 2355 / 2846 = 82.7%
plain linear decode, no control flow at all 2846 / 2846 = 100.0%
Following the coroutine entries buys 2.8 points. Disc-wide, a plain linear decode
from the first phase base reaches 25705/25705 call sites over all 28 stages, and
28/28 decode clean to code_end with no desync.
The real bug was isl.dis ending on `if op == 20: break`. Op 20 is `ret`, but this
is a coroutine VM -- the thread suspends and resumes at the FOLLOWING instruction,
so code continues past it. dis() now takes stop_at_ret (default True, preserving
the old output: data/isl-stage02.txt regenerates byte-identical) and
isl.linear_offsets() is the correct walk.
By-product, kept with its control: start_coroutine's target is staged slot 0 --
73/83 phase-1 sites land on a valid instruction, against a 38.7% chance rate for an
arbitrary 4-aligned offset.
New artefact data/isl-stage02-phase-ends.txt with a committed generator
(isl_report.py phase-ends). It shows END_PHASE's call site is the WRONG place to
read a clear condition: all 12 Stage-02 sites sit in one stereotyped outro. Not
settled, and stated as such: op10/op13/op14/op21/op23 are unread handlers, so the
condition in the poll loop upstream cannot be named yet.
pilot.py gains SYLPH_WEAKEST=1, which scales a target's score by its remaining
hull (pos+0x154) so the pilot finishes what is already hurt instead of
re-engaging whatever is nearest. Motivated by b69cc23: over ~8 minutes the pilot
damaged 14 of 16 e010 attackers (hulls 360..500) and killed none, because 500 HP
spread across a squadron kills nobody.
STATUS: the flag is implemented and its targeting works -- 3105 of 3105 target
samples selected e010 -- but it is UNVERIFIED in combat, because the run it was
written for was lost.
That loss is the second half of this commit. The run printed "READY ROOM / >>>
HUD / Stage 02 OK" and I began the experiment; there was no FLIGHT: line, because
the flight check failed three times and fell through silently while the next line
read like success. The game was frozen on a near-black screen (screen_id `other`,
mean 10.8/2.8/2.1, frozen.py max_pixel_delta=0) and the pilot's every sample from
t=0.0 to t=406.1 is byte-identical with speed 0.
assert_stage.py could not have caught it: it reads the DEFINITION table, which is
populated when the STAGE loads, independently of whether the mission is running.
Recorded in nav-guards.md with the rule -- enforce the flight gate with a non-zero
exit, and run the three-crop TIME liveness check before any experiment.
Third Stage 02 run, pilot with SYLPH_PREFER=e010 (312 fire=1 samples), per-class
live counts logged every 11s beside the HUD (new tools/re-capture/class_count.py).
CONTROL CONFIRMED: the live turret population fell 108 -> 101 -- seven e007 deaths
-- and REMAINING OB never decremented, only rose. Previously this was inferred
from a run whose kill log happened to be turrets; it is now measured with the
classes counted directly.
ARRIVAL TIMING n=3: 004 -> 008 in (108.7s, 125.5s] and 008 -> 012 in (204.2s,
221.7s], both brackets containing the predicted 120 and 210.
The live e010 count sat at exactly 16 in 20 of 26 samples -- precisely phase 1's
e010 roster (ADT102/ADT107/ADT113/ADS151, each n=4) -- an independent runtime
corroboration of the static roster.
DECREMENT STILL UNPROVEN: the e010 floor never fell, so no marked attacker died
and the counter had no chance to move. Three runs have failed to kill one. The
blocker is combat effectiveness, not instrumentation.
Artifact recorded: six of 26 class samples read 17-28. Spikes are always upward
and transient -- the tool dedups on a position triple read just after the pattern
scan, so an entity written between the two reads is counted twice.
Every earlier refutation in this file carried the caveat that entities2.typed
types entities by their position CHANGING, so a stationary objective is invisible
to it. This session's definition-pointer enumeration does not have that limit, so
the sweep was re-run against it (ob_flag_all.py, guarded route, stage asserted,
HUD cropped beside each sample):
A: HUD 004, 147 entities -> 152 candidates
B: HUD 012, 133 entities -> 15 candidates
intersection: 1
The lone survivor pos+0x0250 = 239d6732 is the same offset AND identical value
this file already characterised as a per-group word. Membership test: all 12
holders are UN_e010_ADAN_Attacker_S, 12 of 16 live attackers. It is a squad
parameter, and it survived only because that population equalled the counter at
both samples.
Also reconfirms "not a class head-count" on 147 entities including capital ships.
Trap recorded: the first sweep reported 298 entities and a class with head-count
exactly 4 -- a perfect-looking hit that was pure artifact. Deduping by ADDRESS
leaves the measured exact 2x duplication (pairs 0x1000 apart, byte-identical
positions) intact and doubles every population. Dedup on the position VALUE.
The app owns TWO windows of class xenia_canary -- measured in the tree right now
as 10x10+10+10 and 1280x745+1+20. Largest-by-area picks the game window while
both are present, but during a load or mode switch the game window is briefly
absent from the tree, the 10x10 helper wins by default, and the crop produces a
10x710 SLIVER. That is exactly the grab screen_id classified as `menu` on
2026-08-26, which let a nav guard pass on garbage and loaded the wrong stage.
Ignore candidates narrower than 640 so nothing is selected in that case and the
existing fall-through hands back the raw root grab -- itself a valid full frame.
Verified on the selection logic directly: with both windows listed the pick is
unchanged (1280x745+1+45); with only the helper listed the old logic returned
10x10+10+10 and the new one selects nothing. Pairs with 30e53f5, which rejects
such a frame at the consumer.
assert_stage.py checks the DEFINITION table against an expected stage marker and
earned its keep immediately: its first live run reported MISMATCH -- the capture
had a live flight HUD and would have been filed as Stage 02, but was the S01
tutorial. That is exactly the failure that silently invalidated an earlier
cross-run comparison.
require_menu (launch_mission.sh) refuses to press until screen_id reads `menu`.
It is NOT sufficient, and this refutes my previous explanation: the run DID
confirm the menu and still loaded the tutorial. The real cause was that the
guard's own capture was a 10x710 sliver which classified as `menu` -- fixed
separately in 30e53f5.
Left open: whether the menu guard suffices now that slivers are rejected (not
re-run), and why the capture was a sliver at all when the other shots in the same
run were 1279x675.
Every statistic in screen_id is an AREA FRACTION, so a capture that is not a game
frame still produces clean numbers. Measured 2026-08-26: a guard shot came back
10x710 -- a sliver -- and classified as `menu` with green=0.0000, white=0.0157.
The guard passed, the fixed key sequence went out anyway, and the run loaded a
TUTORIAL instead of the save's Stage 02.
This is the second time this failure has been paid for. bin/screenshot's own
header records the first (2026-08-18): a second window of class "xenia_canary"
meant grabs came back as slivers and "a whole session's screen ids were noise".
That fix hardened the CAPTURE side only, so the same failure still reached the
oracles by any other path. Reject it at the point the answer is consumed too:
features() now returns None below 640x360 and classify() reports `none`.
Verified: the 10x710 sliver -> `none`; readyroom, flight and the briefing capture
all still classify as before.
The briefing map is cyan and satisfies every clause of the menu rule (b-r > 30,
r < 45, little white), with no earlier rule claiming it -- so it was labelled
`menu`. That made wait_screen.sh report NEVER REACHED READY ROOM on a run that
had successfully done LOAD GAME -> slot 01 -> YES and was three screens further
on: a working route scored as a failed one, pointing the next debugging step at
an input path that was fine.
Cyan has b and g nearly equal (b-g ~ 5) where the menu's blue leads its green
(b-g ~ 32), so `r < 20 and g > 30 and b - g < 20` separates them; the r floor
keeps the title screen out. The file's own docstring already carried the
briefing's mean as an aside -- it just never had a class.
Verified against all eight signatures the file documents (2 menu variants, title,
ready room, flight, 3 briefing measurements): no regressions, and the captured
briefing image now reads `briefing`.
Menu navigation here has always been screenshot-driven, which is unusable under
--gpu=null -- the only configuration where the game does not hit the
software-rasterizer freeze. Without a memory signal, the one backend that runs
is the one that cannot be steered.
Snapshotting 0x82800000+3 MB at each menu of a rendered run and keeping the
4-byte words that differ between screens and hold small integers leaves exactly
four of 786 432. One has the property that matters -- it changes on a screen
transition and holds steady when only the highlight moves:
0x828A690C screen id 1 title, 3 main menu, 4 extras
0x828F38AC menu cursor (second copy at 0x828F38BC)
0x828F37B4 per-menu value
Verified on a fresh --gpu=null run with no display at all, driving the same keys
blind: title 1/2/12, main menu 3/4/45, after 4x down 3/12/45, extras 4/14/49 --
4/4 exact against the rendered run, across two runs and two GPU backends. That
is the check that matters, since this corpus has already had to mark one runtime
address run-dependent.
tools/re-capture/menu_state.py reads them; `menu_state.py watch` prints on
change.
Open: the rest of the sequence into a mission. Blind driving reached extras
(screen 4) and a further A did not move it, so MISSION SELECT needs a cursor
move first. Screen ids beyond 4 are unmapped, and the rendered run freezes on
entering that screen -- so map ids up to the freeze, then step blind past it.
Last iteration concluded the freeze is "entering MISSION SELECT" and made
avoiding that screen the next experiment. Ran it; the conclusion was too
narrow.
First, a liveness metric that actually separates the states: two frames five
seconds apart, percentage of pixels changed. The menu animates, so healthy is
99.80-99.97% and frozen is 0.00%. No navigation script needed, and no
classifier. Committed as tools/re-capture/route_liveness_probe.sh; this is what
should have been used from the first run.
Then the menu's FIRST item, NEW GAME, which never touches MISSION SELECT:
main menu 99.80% alive
after A on NEW GAME 7.33%
after the next A 0.00% -- frozen, and screen_id calls it "flight"
with the same 134217728-byte AllocRange failure in the log.
So the correct statement is broader: the game freezes on the first content load
after the main menu, whichever item is taken. MISSION SELECT was just the route
every earlier run used. "Avoid MISSION SELECT" is withdrawn -- there is nothing
to avoid, and that also puts the memory account back at the centre, since ~379
MB live plus a 128 MB content load fails on any route.
Worth repeating because it caught me twice: screen_id.py called a frozen frame
"flight" on a run that never left the menus. Liveness first, classification
second.
challenge-mission-gate.md §5.6 attributes the 128 MB heap failure to a careless
cleared-stage mask poke, concludes that poking only real story ids does not blow
the heap, and ends by asking for the control: repeat without the poke.
Ran it. nav_to_flight.sh gains SYLPH_NO_POKE=1, which skips the write; only
Stage 1 is selectable without it, so the control changes stage too, which makes
the agreement stronger rather than weaker.
poked 0x0001FFFE Stage 02 frozen, 128 MB request refused
control untouched Stage 01 frozen, 128 MB request refused
Both logs carry not merely the same error but the same numbers:
requested 134217728 bytes, parent free 28969/131072 pages
28969 in both, across two stages and two boots. So the poke does not cause it
and neither does the stage; the guest reproducibly arrives at a 128 MB request
with ~113 MB free. An identical free-page count across independent runs also
says the allocation pattern is deterministic -- not a race, not host pressure.
The control was verified three ways, because the first attempt was confounded:
two emulators were alive at once (the previous one survived a pkill). The mask
was read back as 0x0 from the live mapping, the log was confirmed to be this
run's, and the liveness test was repeated after killing the stale process so
exactly one emulator was running -- three frames at rmse 0.00, and an 8 MB slab
of guest RAM with 0 bytes changed over 3 s.
The census filtered pak entries whose own first four bytes are T8aD. A sprite
is usually a child of a RATC bundle, and a bundle entry's magic is RATC, so a
top-level magic filter cannot see one:
top-level T8aD entries (counted) 4 525 sprites, 45 keys
T8aD inside RATC bundles (missed) 16 659 sprites, 204 keys
both 21 184 sprites, 216 keys
171 of the 216 keys exist only inside bundles. The sharpest statement of the
error: that census never saw GP_TITLE.pak at all -- the pak holding both of the
screens this page's entire evidence comes from.
Retracted: "45 values", "the keys are pak-local", "each auxiliary pak occupies
its own narrow high-byte band". On the full population 68/216 keys (31%, not
9%) cross a pak family and the per-pak ranges overlap heavily -- GP_BUNK
0x8000-0xa110, GP_TITLE 0x8000-0xc150, GP_LEADERBOARD 0x8000-0xf100. The tidy
banding was an artifact of seeing one or two keys per pak. So the key looks
like a shared vocabulary, which is the opposite of what I published.
Survives, now on the full population: the field is a u16 at +0x0A (upper half
zero 21 184/21 184), and it is an enumeration (216 values for 21 184 sprites).
Three wrong numbers on this page now, all the same shape -- a statistic computed
over a population I had not checked was the population in question. Stated once
at the end of the section rather than three times: check the sampling frame
before the statistic.
The page rested on twelve values from two screens. This walks all 4525 sprites
on the disc.
* The field is a u16 at +0x0A. The upper half of the 32-bit word the page
reads is zero in 4525/4525. Nothing above changes -- 0x00008100 sorts the
same as 0x8100 -- but a future value with the high half set would mean
something had been misread rather than that the layer got deeper.
* It is an enumeration: 45 values for 4525 sprites, one of which (0x8100)
covers 1188 of them.
* The reading worth trying -- a global layer vocabulary shared across the UI
-- is refuted. Only 4 of 45 keys cross a pak family and 33 of 45 live only
in GP_MAIN_GAME_2D; every other pak owns a narrow high-byte band (0x90-0x94
for the in-game overlays, 0xa4 mission log, 0xb1-0xb2 save/load). A screen
that owns one or two keys is not ordering itself with them.
That supports "group id in the high bits, order in the low bits", which is what
the page already suspected, but it does NOT test it: paint order has been
measured on two screens and both are inside GP_MAIN_GAME_2D, so there is no
ground truth to check the split against. Left amber.
The first number I got was 37/45 shared, which would have supported precisely
the wrong conclusion. It came from counting paks instead of pak families: the
six GP_MAIN_GAME_*2D paks are the same screens in six languages and their key
sets are byte-for-byte identical. Recorded on the page, because the shape
recurs -- a corpus with near-duplicate members manufactures agreement.
Two follow-ups on yesterday's^Wthis morning's CollisionSet write-up.
1. The _cmesh <-> render-model link, which I recorded as UNTESTED because
matching stems against .xbg object names covered 4 of 158. The disc keeps
only one build manifest, so that corpus was never going to answer it. The
right corpus is the GameResourceID field of the DefTables / GP_MAIN_GAME
records -- 480 distinct values. Against those, with a control that shuffles
the characters of each stem:
ship/mob stems prefixed by a real resource id 108/112 = 96.4%
same stems, characters shuffled (control) 0/112 = 0.0%
asteroid stems prefixed (expected none) 0/46
So a CollisionSet entry is <GameResourceID>[_<part>]_cmesh. The 0/46 on
asteroids matters as much as the 108/112: a test that fired on everything
would be the bound-check hazard again.
2. The world unit. Sweeping every pak for a name carrying a kilometre figure
returns mapmesh_box_500km.col/.rgn and nothing else -- 162 references, all to
that one pair. The reading rests on a single filename with no corroborating
instance anywhere in the data, so no static test can settle it; marking it
blocked on the oracle rather than leaving it as an open static question.
My objection's premise did survive: rou_e010 is a real GameResourceID and
e010_ADAN_Attacker_S is in the stage tables, so the 133-unit mesh does belong
to a craft the game calls an attacker. Whether the trailing _S means "small"
is a further guess (there are _EX4 / _HF / _HF_Wayne variants), so it stays
suggestive rather than evidence.
All 18 blobs are byte-identical: the per-stage naming is nominal, and every
stage points at one shared 1675148-byte library stored eighteen times. That
identical size was the reason to open the item, and it turned out to be the
answer to it.
Record layout: {u32 size, u32 name_len, char name[name_len], u32 nv, u32 nt,
f32[3] x nv, u32[3] x nt}, next record at off + 8 + size. The indices are u32
here where MCOL uses u16 -- two different serialisers in one archive.
What makes this a decode rather than a plausible reading: the walk consumes the
file to the byte over 158 variable-length records, with the size word predicted
from the two counts 158/158. A wrong field would desynchronise within a few
records and could not land exactly on the end. All indices in range 158/158;
98.24% of edges shared by exactly two triangles; 147/158 fully manifold.
158 meshes, 90 836 triangles: per-part ship proxies (_bdy/_brg/_eng/_wep/_sld,
the XBG7 sub-part vocabulary) plus 46 stage asteroid meshes whose prefixes are
exactly the stages that have an _AsteroidVolume_wp MCOL.
Two things this file makes me walk back:
* The "1 unit = 1 metre" reading from mapmesh_box_500km is downgraded to
amber. The 500000 arithmetic stands, but it implies that a craft the game's
own tables call "small" is 133 m and that rob_f002 is 447 km -- 89% of the
arena width. The format check survives; the interpretation has no
independent support.
* The _cmesh <-> render-model name link is recorded as UNTESTED, not
confirmed: only one .xbg build manifest survives on the disc, so matching
stems against object names covers 4 of 158, which is no coverage at all.
The names live outside MiscBin: they are the MapPath / MapMesh /
CollisionMeshes field values of the per-stage StageResource object (IDXD schema
3c9ae32e, in every GP_MAIN_GAME_<lang>.pak), and each hashes with the ordinary
pak name_hash straight to a TOC entry. 40/40 resolve, no collisions -- the 11
REGN as <stem>.rgn, the 11 MCOL as <stem>.col, and the 18 remaining blobs as
CollisionSet_S01..S16 / _Tutorial / _test.bin. The .pe string table at 651540
was the way in: MapMesh and MapPath sit adjacent there.
This upgrades the pairing claim. The first section of mcol-collision.md could
only say REGN and MCOL had matching *distributions* of bbox and cell size, and
flagged that as not an object-to-object link. A phase record names one .rgn and
one .col, and all 11/11 pairs share a stem and agree exactly on both.
The names also check the format work from outside it: mapmesh_box_500km.col is
the object decoded here as 8 vertices and 12 triangles spanning exactly
+-250000, and its name says that cube is 500 km across -- so one world unit is
one metre, and a wrong stride could not have produced a box that measures what
its own filename claims. 70 of the 87 phases use it: most stages' only
collision is the arena wall, and _AsteroidVolume_ names the rest.
Still open: the 18 CollisionSet_*.bin are named but not decoded (all exactly
1675148 bytes), and CMapColliderBridge in the RTTI names the runtime consumer
without following it into the code.
The 0x50 header word, which the first section of this page had dismissed as "a
large value", is two u16 counts: vertices and triangles. They give the two
remaining blocks their stride, and every derived length is exact in 11/11 --
len(0x54) == align16(12*nv), len(0x58) == align16(6*nt), and nt equals the
bounding-sphere count decoded last iteration.
Checks that cannot pass by accident:
* sphere i is the TIGHT bounding sphere of triangle i, 4768/4768, with
max|v-c|/r median 0.99990 (a fixed 1.0001 epsilon), against a 1.32%
random-triangle control;
* the mesh is watertight -- every edge shared by exactly two triangles,
7152/7152, zero degenerate triangles, zero unreferenced vertices;
* the two smallest objects are 8 vertices and 12 triangles whose positions
are the eight +-250000 corners of the map bbox: a bare bounding cube.
The cell lists are a correct broad phase: with an exact triangle/box SAT test
only 3 overlapping triangles in 18 577 entries are absent, so a query walking
one cell's list cannot miss a hit. The 730 conservative extras bracket the
builder's own test between exact-SAT and AABB, which retires the 18 unexplained
"sphere misses" from the previous commit as that same margin.
mcol_probe.py gains `mesh` and `obj`; `verify` now runs all three checks and its
output is recorded in docs/re/data/mcol-verify.txt.
The unexplained ~0.75 ratio left at the end of the last iteration was my own
stride. I had read the block as 12-byte points because REGN's vertex section
is 12 bytes, and never checked it: len(0x5C) is not a multiple of 12 in 5 of
the 11 objects, so that stride was never arithmetically possible.
At stride 16 the relation is exact in 11/11 -- max u16 == len(0x5C)/16 - 1 --
and the record reads as {centre f32[3], radius f32}. Powered test, since a
u16 is reached through a specific grid cell: the sphere it names reaches that
cell in 18 559/18 577 = 99.90%, against a 12.02% random-sphere control. Both
fields carry signal (centre alone 26.75%, radius shuffled 70.19%).
The converse -- is the list *exactly* the intersecting set? -- is 0.38%, which
is the expected direction: a bounding sphere is conservative, so membership
implies overlap but not the reverse. The tighter geometry is in 0x54/0x58,
still undecoded. 18 entries (0.10%) go the wrong way and are recorded as open.
tools/re-capture/regn_decode.py is copied unchanged from auto/regn-reader so
the probe's POF0 reader is the known-good one rather than a second copy.
dat/tables.pak holds a 5798-entry SOUNDS record (cue name -> sound id) and a
5135-entry FILES record (.slb bank paths). Cue names are the join key, so a
script message id now resolves all the way to the bank that voices it:
MSG_VOICE_D_257 -> VOICE_D_257 -> 6945 -> jpn\etc\VOICE_D_257.slb.
The prefix rule is MSG_ -> VOICE_, not strip-MSG_. My first rule was the
latter; it left 88 names unresolved and I was about to write those families up
as text-only announcements, until VOICE_TCAF_592.slb turned up in FILES and
refuted it. Corrected rule resolves 1326 of 1338, and SOUNDS and FILES agree on
exactly the same 12 absentees.
Separately, MSG_DEMO_* is driven by its own IDXD tables in the language packs,
which carry speaker, portrait, on-screen seconds and audio cue per page. Field
count is 9*PageCount+2 for all 7 distinct PageCounts; 1252/1252 caption-key
slots match <ID>_<page>_<line>; the 78 multi-page records equal the 78 counted
independently from the caption side; 138 ids close exactly against the caption
table both ways.
Does not settle the known VOICE_D_452 wrong-recording case -- every cue id is
distinct, so bank sharing is not happening at this layer.
Measured the <id>_<page>_<line> key structure: a page is one subtitle box of
3 or 4 wrapped lines, and successive pages are successive utterances by
possibly different speakers. 452 of 4091 ids span more than one page, up to 8.
That refutes the isl_dialogue.py committed two commits ago, which read only
page 000 -- 356 of the 1338 script message names are multi-page, so a quarter
of its output was truncated to the opening utterance. Tool now walks pages
until one is empty; Stage 02 sample regenerated (43 of 213 calls multi-page).
The 2683/2683 resolution figure is unaffected: it counted ids that have text,
and every id does have a page 000. What was wrong was the rendered text.
Built-in 64's slot-0 operand is a symbol-table-1 type-6 message id, and every
one of them now has caption text: 2683 of 2683 call sites across the 28 stage
scripts, 1338 distinct names, no residue of any kind.
This only became reachable once build_caption_text was switched to the IXUD
field table (537 -> 8800 lines); before that most of these names had nothing
to resolve to.
Adds isl_dialogue.py plus a committed Stage 02 sample. Does not settle which
recording plays for a given line, multi-page captions, or the other five
languages.
sub_822FE040 fills 1023 eight-byte slots at table+32 with a default and then
overwrites individual ones; slot = (N - 32) / 8 from each std r9, N(r31).
Symbolically executing it yields 57 populated slots, matching the count the
corpus recorded, now with the full opcode -> handler map committed as
docs/re/data/isl-command-table.txt and regenerable from
tools/re-capture/isl_cmdtab.py.
Nine opcodes point at 0x82391BA8, which is `li r3,1 ; blr` -- accept and
discard. 768, 769, 774, 775, 776, 791, 792, 793 and 805 are dead in this
build, which is why the built-ins posting them do nothing.
Opcodes 800-802's entries are thunks 8 bytes apart into 60-instruction
handlers that differ in exactly two words: a descriptor offset and a unit
message id.
800 builtin 26 0xED0802DE
801 builtin 28 0xED0803DE
802 builtin 29, 101 0xED0804DE
That fixes the id format as 0xED08 nn DE, and the ids known from other work
fit it: opcode 514 -> 00DE, 803 -> 07DE, 999 -> 0FDE.
Stopped one link short of the semantics, and saying so: the pump's arm for
0xED0802DE does not apply an effect. It walks the unit's child list at
[unit+320]/[unit+324] and REBROADCASTS to each child as 0xED0902DE. So 0xED08
is the to-unit family and 0xED09 the to-child one, and the terminal effect is
further on. 26/28/29 remain unnamed.
The command table is the reusable part -- it answers "what does this opcode
reach" for every future built-in question, not just this family.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
1146 sites in 22 stages, the second-largest unnamed built-in. Its method
sub_822646B8 (vtable slot 300) and built-in 2's sub_822642E0 (slot 12) are
190 and 199 instructions and differ in exactly one block. Diffed instruction
by instruction, 108 adds:
lwz r11, 16(r29) the blob's slot-16 int
cmpwi/blt/cmpwi/bgt range-check n to [0, 31]
slw r21, r25, r11 r21 = 1 << n <- a 32-bit selector
... default 1 when out of range
Everywhere built-in 2 passes its r21, 108 passes r20 and reserves r21 for the
mask, so the bit is an EXTRA argument to the same call rather than a
replacement. Both post the identical command word AB0100BA, opcode 256.
The operand is always a valid bit index: over all 1146 sites slot 16 is in
[0, 31], 1146 of 1146, none outside, so the out-of-range default never fires
in shipped content. Fifteen distinct values clustered at 16 (531x), 31
(165x), 20 (161x) and 2 (90x); 21 of 22 stages use more than one.
What the bit SELECTS is not established and the name does not claim it. There
is plainly a 32-bit space -- built-in 92 reportedly allocates a free bit by
OR-ing over live units, which would make 108 its "place in a named slot"
counterpart -- but I have not verified that, so the name says only what is
proven: the same deploy as built-in 2, with an extra selector.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The highest-traffic unnamed built-in: 1197 sites across all 28 stages.
sub_822659F0 read directly:
* indexes [phase+324]'s record array by the slot-4 symbol;
* returns 0 immediately when the live object [record+4] is NULL, so it
registers an object that already exists rather than spawning one;
* sets [record+16] = 2, the documented "active" state every unit predicate
tests;
* stores sub_82301118's packed result into [record+20] (low 16) and
[record+24] (high 16) -- member counts;
* posts opcode 513 (0xAB0201BA) either way; the slot-8 mode (1 in 999 sites,
0 in 198) only decides whether cmd+20 is also set to 1.
The ordering test: if this activates a unit for the script, no predicate
should test a unit before it. Over all 28 stages, for every (stage, unit)
pair having both, activate_unit comes first 517 times and a predicate first
0 times. Recorded as file order rather than proven execution order --
coroutines can interleave -- but 517 with no exceptions is not a coincidence.
344 units are tested without ever being activated (live from mission start)
and 203 activated without being tested.
Also recorded: a tidy closure that FAILS. squad_survival_pct reports current
over initial and activate_unit snapshots counts, so the snapshot looks like
the baseline. It is not -- built-in 24 reads [record+16] for the state then
calls sub_823011B0 and sub_82301118 on the LIVE object, never touching
+20/+24. What reads those two fields is unidentified.
Named coverage is now 57 of 108 distinct ids and ~80% of call-site traffic.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The member object sub_82348830 returns is the per-member unit DEFINITION, and
that identification is not a guess: the same spawn loop builds two aggregates
and each lands on a semantically apt field with the apt reducer.
group +192 min, seeded FLT_MAX member +164 = CruisingVelocity
group +472 sum member +84 = HP
A wrong struct would have to make both offsets land on apt fields AND pair
each with the apt reducer. Minimum of a speed, sum of hit points: a
formation's cruise limit and its total health.
The quantitative test over all 1360 sites, joining each to its craft's
definition:
value <= the craft's MaximumVelocity 1355 / 1360 = 99.6% (5 fail)
value <= the craft's CruisingVelocity 1042 / 1360 = 76.6% (318 fail)
The test discriminates -- the cruise bound breaks 318 times, the hull maximum
5 -- so the script sets a COMMANDED SPEED, free to exceed the cruise default
and bounded by what the hull can do.
The turret anomaly that stopped me naming this two iterations ago was my own
artefact. UN_e007_ADAN_Turret's definition carries MaximumVelocity 500 and
CruisingVelocity 280: the data models turrets as if mobile, so a script value
of 400 is legal and simply never manifests. I had assumed turrets have no
velocity fields and treated 13% of the traffic as a refutation.
Recorded as unsettled: the five overshoots are UN_e106_ADAN_Destroyer 200 vs
a 150 maximum (x2) and UN_e011_ADAN_Attacker_B_HF/_Wayne 500 vs 450 (x3).
Designer overrides or an engine clamp; not established.
Named set_group_speed. Default = the slowest member's CruisingVelocity;
mode 1 restores it, mode 3 sets it, mode 2 hands it a global constant.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
77, 78, 81, 82 and 135 were unnamed. The engine has five contiguous strings
-- MISSION_START_PRT at 0x820A83F0, then _END_, _UPDATE_, _FAILED_,
_RESTART_ -- and five sequential ScriptPhase fields at +388/+392/+396/+400/
+404, stored in ascending order by one constructor region. Five names, five
fields, five unnamed built-ins.
Which is which is decided by call-site structure, measured over all 28
stages, and it is exact:
39 MARK_LAST_PHASE 89 sites -> 82 in 89/89
82 banner_mission_failed <- 39 in 89/89, then wait_s 89/89
40 mark_not_last 50 sites -> 78 (27) + 81 (17) + END_PHASE (6) = 50
78 banner_mission_complete <- 40 in 27/27
81 banner_objective_update <- 40 in 17/17
77 banner_mission_start 22 sites in 22 stages, one per stage,
after play_bgm
135 banner_mission_restart 16 sites, after play_bgm, phase >= 2
39 -> 82 is a perfect pairing and 40's sites partition exactly three ways.
Stated as inferred rather than read: the string-to-field pairing itself comes
from both sequences ascending in the same order; my operand tracker did not
catch the string loads in that constructor. The ROLES above do not depend on
it.
76 is left unnamed on purpose. It has 38 sites = 22 + 16, exactly 77's count
plus 135's, and precedes them; its body sets [phase+332] = 1 and nothing in
the image reads that field. Suggestive arithmetic is not a name.
Flagged as a consequence: MARK_LAST_PHASE is followed by the FAILED banner in
89 of 89 sites and mark_not_last by END or UPDATE. So [phase+300] = 2 reads
less like "this is the last phase" than "end the mission now,
unsuccessfully" -- the existing names for that pair may be mis-framing it.
Artifact regenerated: docs/re/data/isl-stage02.txt.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Verified rather than adopted: a subagent proposed that every even operand
slot is a type tag. Measured, the strong form is false and a precise form is
true.
TRUE: a SYMBOL operand is two words, a tag holding the constant 1 followed
by the index. Slot 0 is the integer 1 in 19899/19899 calls whose slot 4 is a
unit; slot 8 is tag-shaped in 100% of calls for every built-in taking a
second unit; slot 16 is 1 in 152/152 for built-in 128, the only one taking a
third. The 24 built-ins whose slot 0 is NOT the constant are exactly those
taking no symbol there. This explains the unit slots 4/12/20 rather than
replacing them.
FALSE as stated: slot 8 is a bare double for built-ins 4, 20, 24, 26, 28,
29, 90, 106 and 127, and built-in 75 carries five bare indices at 0/4/8/12/16
with no tags at all. Each built-in has a fixed signature and is 100%
self-consistent; none of the 34 with >=20 sites mixes the two.
Symbol table 1 has three types -- 1 routes (1362), 6 messages (2247), 7
effects (81) -- and its operand slots are type-pure, measured the same way.
Resolving them makes listings say what the script means:
`request_script_message(MSG_VOICE_D_257, ...)`, a fourth independent
confirmation of that name. Slots 24@4, 46@12 and 114@4 resolve 100% but MIX
types 6 and 1, so they are left unresolved rather than guessed.
Two more names withdrawn, neither replaced:
* 88 `camera_at` -- ZERO call sites in all 28 stages; never testable.
* 90 `camera_at_route` -- 8 sites, all Stage 02 phase 3, first operand is
symtab-1 type 7 `eff_n0071`, an EFFECT name, in 8/8, with a per-missile
Route_ADT301..308_p3M at slot 20. Not aimed at a camera.
Left unnamed on purpose: replacing a guessed name with another guess is how
the three names corrected earlier today went wrong.
Also flagged: 115 `named_event`'s only symbol operand is an eff_* name in
84/84 sites, so that name is suspect too. Not renamed pending a handler read.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Last iteration I said launch_mission died because skip_intro only tests for
the title on a static frame, gated at rmse <= 1500, and that run logged 1503
and 1549 just above the cut. I also said the fix was NOT to nudge 1500 but
to measure both signals through a boot first. Measured, and the diagnosis
does not survive.
boot_trace.sh logs the two signals skip_intro decides on -- frame-to-frame
RMSE and the is_title.py green-glyph count -- through a clean boot with no
presses at all. One run, 29 samples over 484 s:
8 samples had rmse <= 1500, so the gate OPENED eight times
0 samples had glyph > 0, so the title was never seen
At t=145 s the RMSE was 1205, comfortably inside the threshold, and the
glyph test was called and answered zero. A frame can be perfectly static
without being the title -- the intro movie has long quiet stretches, three
reading RMSE exactly 0. So 1503/1549 were almost certainly movie frames too,
and raising the constant would have admitted two more of them.
What is left is narrower and honest: the interactive title never appeared,
rather than appearing and being missed by a threshold.
The limitation is recorded rather than buried: the tracer intended 1 s
sampling and achieved 16.9 s, because each iteration forks two screenshots,
ImageMagick compare and a fresh Python. So this does NOT prove the title
never appeared -- only that it was absent from 29 samples. A window shorter
than ~17 s falls between them. The recorded next step is to make the tracer
sample at the rate it claims before concluding anything stronger.
Artifact: docs/re/captures/boot-signal-trace.tsv.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Third independent line for yesterday's built-in 100 rename, from the callee
this time. sub_8226E3B8 was labelled "push", which is what made built-in 100
look like push_trigger. It reads the element count, returns immediately when
the container is EMPTY, and otherwise walks the node list splicing nodes out
until it is empty. A push links one node; this unlinks all of them. It is
clear(). The append is sub_8226E160, reached from built-ins 19 and 25.
So the rename now rests on the handler, the usage (all 12 Stage 02 sites sit
in the phase terminator next to timer_stop / clear_flag(-1) /
MARK_LAST_PHASE), and the callee.
The dynamic half did NOT run, and the write-up says so. phase_watch.py now
samples [phase+272+20] (triggers queued) and [phase+216+8] (coroutines
alive) so a phase terminator's effect on the VM is visible in one line —
written here, never yet exercised against a live guest.
Boot-nav could not reach the title in 381 s. Diagnosed rather than retried:
skip_intro.sh only runs the title test on a static frame, gated at
rmse <= 1500, and this run measured 1503 at 104 s and 1549 at 139 s — just
above the cut — so is_title.py was never called and the one allowed press
was never spent. Recorded in BACKLOG with the explicit instruction NOT to
raise the constant: the first step is to log rmse and the glyph count
through a whole boot and look at the two distributions, because tuning a
threshold to make one run pass is fitting to a single sample.
Also reaped a stale lock: a gdb orphaned 2h14m earlier was holding
/tmp/xenia-canary.lock with an already-defunct emulator child.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
All re-read twice — the handler, and the thing it calls — because each had
been named from its shape rather than its effect.
* id 11 `yield` -> `end_coroutine`. 0x82272624 is li r11,1 ; li r3,3 ;
stw r11,164(r31), and the dispatcher's r3==3 arm erases the thread from
the active list and returns it to the free list. It destroys the thread.
2945 sites game-wide, 372 in Stage 02 — the most-used built-in there was.
* id 5 `await_label` -> `kill_coroutine(label)`. sub_82273B08 kills the
thread parked at the target pc, or itself if the target is its own pc.
It waits for nothing.
* id 100 `push_trigger` -> `reset_phase_threads`. It clears the trigger
container and then frees every thread whose pc differs from the caller's
— the opposite of pushing a trigger. Corroborated by usage: its 12 Stage
02 sites all sit in the phase terminator, next to timer_stop,
clear_flag(-1) and MARK_LAST_PHASE.
One name recovered from the game's own text: opcode 992 prints
"RequestScriptMessage %s" at 0x820A5700, so id 64 is request_script_message
(2683 sites).
Return codes documented properly: 1 = restart the coroutine from its entry
(previously not recorded at all), 3 = terminate. And the blocking set was
wrong in two places — it is 102, 120, 137, 142, 143. Id 97 does NOT block;
its handler ends `b 0x822724F8`, so it always returns 0.
Unit-operand resolution settled from DATA over all 28 stages rather than by
reading 147 handlers: a slot qualifies only if every value is a valid
symtab-2 index, it takes >=15 distinct values, AND its maximum reaches most
of the table — that last clause is what discriminates, since every small
integer is trivially "in range". 31 built-ins at slot 4, 8 at slot 12, one
at slot 20. It also refutes set_flag's slot 0, whose maximum overruns the
table, and the resolver now declines rather than inventing a name.
New and unexplained: symtab-2 holds two types, 2 and 8, and built-ins 95 and
128 take type 8 at slot 12 in 100% of their sites.
A downstream inference is withdrawn with it: the note reading the live
trigger counter attributed it to "the script arming watches as it goes" via
built-in 100. The measurement stands; the attribution does not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Settled from the disassembly, no run needed. Built-in 69's tail maps the
lifecycle lookup into [phase+164]: handle == 0 takes the early exit at
0x8226AF44 and returns 0, while the destroyed states return 2, 3 or 4 (and two
of those also normalise the record's +16). Those are different values.
So poking rec+4 = 0 made the predicate report ABSENT -- the same answer an
undeployed unit gives -- and never the answer the script branches on. The
condition was polling at 5 Hz throughout and correctly saw 'not here'.
Both null results are now fully explained, and neither was evidence about the
condition: the first poke wrote a field nobody reads, the second wrote the wrong
value into the right field. Simulating a kill needs the handle to stay valid
while the lifecycle lookup returns 3/4/5, i.e. the write belongs in whatever
sub_82301240 reads, not in the script's own record.
Also names built-ins 46/47/48 as squadron_trace / squadron_attack /
squadron_escort in isl.py.
sub_822FE040 is a fully unrolled registration: 1023 slots filled with a default,
then 57 explicit writes, of which 48 are real handlers and nine are a shared
accept-and-discard stub. Return convention is nonzero = consumed, 0 = retry,
which is how the interpreter waits for a named unit to exist.
Opcode 995 is the ONLY handler touching the phase mirror [*(0x828F35F8)+236] --
the sole read and sole write in the table -- independently confirming why polling
that mirror saw nothing during phase 1. And no handler spawns or despawns a unit:
256 is the strongest deploy candidate but is unconfirmed because the message ids
are write-only in this image.
WITHDRAWN, verified wrong: I had recorded the writes to '+20' in sub_8226E7D8 /
sub_8226E930 as block initialisations by container constructors. At
0x8226E86C-0x8226E8E0 they do li r3,28 / bl 0x8230C160 then
lis r10,0xAB03 / ori r7,r10,0xE4BA / stw r7,4(r3): they build an INTERPRETER
COMMAND RECORD for opcode 996 and push it, i.e. AddSelector and RemoveSelector,
with a 32-entry cap. The stw to 20(r3) is the command record's +20, a different
object. Wrong twice: not constructors, and not that container.
Also flags that sub_8230C398 -- gated on *(0x82899CE0) == 16 at both call sites
-- looks like Stage 16's script compiled in C++, which 'debug defaults' does not
survive given the .ssb loader explicitly refuses mission 16.
The watchpoint plus Canary's source settle it. At the write, the guest context
(rsi, per x64_emitter.cc:881) holds 0x8226EAE0, inside sub_8226EAB8. That
function is a generic list-node insert: it reads the count at 8(r30), guards
against 0x3FFFFFFF overflow, does addi r11,r11,1 / stw r11,8(r30), then links the
node. It has 16 callers, so it is a shared container helper.
That explains why two static searches missed it. The trigger container at
phase+272 EMBEDS an inner list object at +12 -- which is why the push does
'addi r31, r30, 12' -- and the inner object keeps its count at its own +8. So
272 + 12 + 8 = 292 = the watched word, and the instruction is stw r11, 8(r30)
with r30 = phase+284. Searching for 'stw rN, 20(rM)' could never have found it.
Also resolves the earlier open item on +12: it is the embedded list object, not
a list head pointer, which is why it read 0x000A0009 instead of an address.
Method note kept: the static hunt assumed the field's offset in the OUTER object
would appear in the writing instruction. A watchpoint is indifferent to the
addressing form, which is why it was the right tool after two failed offset
searches.
ssb_watch.py polls the .ssb header, a symbol string and find_mission from the
moment flight is detected. On a normal run all three are present at the FIRST
sample (t=0.0, mission 0xBC79C960), so 'the script loads later than assumed' and
'the probe raced a load' are both out -- there is no window in which a healthy
mission flies without its script in memory.
A detail worth more than it looks: at that sample screen_id reported 'other',
not 'flight'. The script is loaded and the ScriptMission locatable BEFORE the
flight HUD appears, so residency is not gated on the HUD, and a run showing the
HUD without the script is in a state a healthy run never passes through.
That leaves the third candidate: the anomalous run's mission never loaded and
its IN FLIGHT was a misdetection -- consistent with it being frozen on a black
screen when checked afterwards. Not proven, but now the only surviving
explanation rather than one of three.
From Canary's own source (x64_emitter.cc:881) GetContextReg() returns rsi, so at
any JIT instruction %rsi is the PPCContext* -- which is also why the faulting
instruction read 0x110(%rsi), a guest register load. That is the way past the
watchpoint's ceiling: the guest register file is available at the write, and a
0x82xxxxxx word picked out of it resolves against sylpheed.db to name the caller.
trigger_watch.sh now dumps x/128wx instead of a useless host backtrace.
The re-run then failed for an unrelated and unexplained reason: it reached
flight, the pilot bound, the guest was animating, and find_mission returned
NOTFOUND. Narrowed: the .ssb header is absent from guest memory (0 hits where
earlier runs hit immediately), ADN110 is absent too, but the manifest string
'Stage02.ssb' IS present at 0xBDA6C50B. So memory is readable and the manifest
is loaded while the script is not, in a mission that is flying.
No explanation offered. The cheap discriminator for next time is to poll for the
header from the moment flight starts and record when it appears, instead of
sampling once.
All 28 StageNN.ssb decoded: 2,085,628 bytes, 25,705 call sites, 108 of 147
built-ins used. Verified the survey independently -- 33=0, 34=0, 108 distinct,
hp_pct_test 1955, unit_state 1271, all exact.
global_counter0/1 have ZERO call sites in all 28 stages. The handlers exist and
are wired; no mission calls them. So 'does a wave start after N kills?' is
answered for the whole game, not just Stage 02: no mission counts kills.
Two idioms change how the counts read. hp_pct_test(unit, 0.0) IS a destruction
test -- 1786 of 1955 calls (91%) pass 0.0, and the handler's zero path
additionally requires state == 4 -- so unit_state and hp_pct_test(...,0) are
interchangeable and stages just pick one. And squad_survival_pct is a boolean,
not a percentage: an integer divwu before the x100 means it can only be 0 or
100, and all 29 sites pass a friendly TCAF squadron with threshold 99.9, making
it 'has this escort lost anybody'. The game has no destroy-N%-of-a-squadron
objective.
Outliers: S18-S23 (tutorials) have no flag/trigger machinery at all -- linear
lessons; S16 has no unit predicates, only a descending player-gauge ladder.
Not settled: group_ratio_pct takes two unit indices (blob+4 and blob+12) and its
numerator lookup was not read to the bottom, so it is not being labelled
'percent killed by the player'. isl.py's UNIT_ARG omitted 71/72.
All 25 opcodes now have meanings. Ops 2/4/6/8 are integer compound assignment
(+= -= *= /=) and 3/5/7/9 the float versions; 10 and 11 are integer and float
compare writing three condition bits; 13-18 are je/jne/jl/jle/jg/jge; 21-24 are
push.i/push.f/pop.i/pop.f over deques at phase+44 and phase+64.
The shared-handler question is answered: the dispatcher leaves the opcode in r4
and the shared thunks never overwrite it, so those helpers take an extra opcode
argument and index a secondary table (0x82271448, 0x8227152C).
CORRECTION to my own tool and note: the branch/jump base is [phase+232], which
the phase initialiser sets to 0x24 + the phase's entry from the mission-level
stream -- 0xE4 / 0x14AA8 / 0x24B4C for Stage 02's three phases, not the file's
0x24. Measured on phase 1: base 0xE4 puts 525 of 525 branch targets on an
instruction boundary; base 0x24 manages 188. isl.py had been using 0x24 for
every phase, so its jump targets were wrong throughout. Fixed via
isl.phase_bases().
That also settles two things mission-script-ssb.md left open: offsets ARE
code-base-relative, and 0x1883's operand IS a code pointer -- the earlier worry
that some 'land on IEEE floats' was an artefact of adding the wrong base.
