fix/canary-native-binary-path
9 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Sylpheed RE agent
|
c696657613 |
re: the trailing data table is a per-phase TIMELINE of scheduled routines
Decodes the table found at the end of every phase region. Layout:
int N
N x [ int offset ; float t ; int kind ] -- 8-byte typed records,
tag 0x19 int, 0x1A float
1 + 3N matches the record count in every phase measured (Stage 02: 76/40/55
records for N = 25/13/18).
Checks, all independent of each other:
schedule entries disc-wide 675
0x1A float records disc-wide 675 (counted by a different route)
offsets landing on the instruction stream 675/675 = 100.0%
control, random 4-aligned offsets 33.3%
The floats are seconds -- 0, 0.5, 1, 4, 5, 30, 50, 60, 90, 120, 150, 170, 180, 210,
240, 270, 300, 330, 360, 420, 570, 1020, 1080, 1140, 1170 -- and the targets are small
one-shot coroutines that set arguments, call one built-in and end_coroutine. kind is
0 (556) or 5 (119) and is not identified.
Runtime cross-check, recorded as consistency rather than confirmation: the closed
REMAINING OB work measured Stage 02's squadron arrivals at t = 0, 120 and 210 s over
n=5 emulator runs, and all three appear in phase 1's static schedule, with 120 and 210
each appearing TWICE. These are round numbers and phase 1 has ~22 distinct times over
0-1170, so presence alone is not unlikely; the doubling is the sharper detail and was
not predicted in advance.
New artefacts data/isl-stage02-schedule.txt and data/isl-schedule-all.txt with a
committed generator (isl_report.py schedule). calls, phase-ends, conditions and
phase-guards all regenerate byte-identical.
Not settled and said so: kind is unread; the consumer is unread, so the decode rests
on the structural checks above; whether the clock is per-phase or per-mission is an
inference from the layout; and this is NOT what starts the unreachable code -- 0 of
the 675 targets are unreached run-starts, so that ~15% gap stands.
|
||
|
Sylpheed RE agent
|
6cbfe0524f |
re: the sufficient side -- each phase exit now names the condition that FIRES it
Dominance said a phase cannot end unless X. A port also needs "once X holds, it must end", and that is a must-reach set: nodes from which END_PHASE is unavoidable, as a least fixpoint where n qualifies when it has successors and ALL of them qualify. The conservatism is deliberate and is the honest answer: a loop never enters the set, because a poll loop reaches its exit only if the polled predicate eventually becomes true, which is a liveness property rather than a graph one. A dominating condition is a TRIGGER when the successor it takes on being satisfied lies in that set. Over all 28 stages: 732 dominating conditions, 234 triggers (31.97%). isl_report.py phase-guards now tags every line precond / TRIGGER. The split lands where it should. Stage 02's phase-1 objective exit is six preconditions -- player alive, TCN004 destroyed, t <= 210, ADT102/ADT107/ADT113 destroyed -- and exactly ONE trigger: hp_pct_test(ADN101, 0) != 1. Destroying ADN101 is what fires the phase. That is a sentence a port can implement. Per-exit distribution over 172 reachable exits: 89 have exactly one trigger, 42 have none, 41 have several. The 42 with none are not a failure -- they are the exits no branch fires; Stage 02's 0x006260 ends on read_freg(0) < 1200, a timeout, and time passing is not a property of the graph, so declining to call it a trigger is correct. Recorded as a heuristic rather than a rule: "the first trigger is the point of no return" holds for 33 of the 41 multi-trigger exits, with 8 counterexamples where a precondition appears after a trigger. The likely cause is that the listing is ordered by file offset, which is not execution order -- coroutines and jumps let a lower offset run later. Not asserted. calls, phase-ends and conditions all regenerate byte-identical; the two phase-guards artefacts change only by gaining the tags. |
||
|
Sylpheed RE agent
|
2876449965 |
re: phase-guards for all 28 stages, and a 6/6 cross-check from an unrelated method
isl_report.py now accepts a directory, so the dominance analysis runs over the whole
disc: data/isl-phase-guards-all.txt, 177 phase exits, of which only 5 (2.8%) are
reachable from no static entry. CFG reach ranges 69.5% (S26) to 95.8% (S25), median
about 4 dominating conditions per exit.
The lopsided number in the per-stage table was the six TUTORIAL stages, S18-S23, each
with exactly ONE exit and exactly ONE dominating condition. That could have been a
degenerate result, so I looked: it is the same condition in all six,
END_PHASE <- builtin104() != 1
and isl-builtins.md reached built-in 104 from call-site USAGE alone -- "S18-S23 only,
followed by wait_s 39/39, preceded by end_coroutine 37/39, a textbook poll loop".
Usage said 104 is the tutorial's polled test; dominance says it is the tutorial's
clear condition. Two unrelated methods, six for six.
Stage 16 -- the corpus outlier whose script may be compiled C++ -- resolves as well:
read_freg(0) < 600, player_gauge0_test, player_gauge1_test, and two builtin141 calls
differing in a single argument (0 vs -4000), which is the shape of a position or zone
test. builtin141 is unread, so it is not named.
Stage 02's separate artefact regenerates byte-identical.
Also added: an RLIMIT_AS cap in isl_report's entry point. The dominator pass
OOM-killed a run earlier on this 15 GB box; a bad input should now fail the process
rather than the machine.
Still not settled and stated in the doc: dominance gives necessary, not sufficient,
conditions; the 5 unreachable exits need the trigger queue at phase+272; builtin104,
builtin141 and builtin7 all appear in clear conditions and are unread.
|
||
|
Sylpheed RE agent
|
983c5d112c |
re: the per-phase clear conditions, by dominance over the ISL CFG
Closes the backlog's "which condition guards each END_PHASE". With the CFG from the previous commit this is a graph query, not new machinery. The obvious query is WRONG for this language, and I implemented it first: "one successor reaches END_PHASE and the other does not" finds 1/62/1 guards across Stage 02's three phases, and the 1s are both the same read_freg(0) < 1200 timeout -- every objective test missed. The cause is the dominant idiom: a POLL LOOP's loop-back branch also reaches the exit, one iteration later, so neither successor discriminates. The asymmetric 1/62/1 is what exposed it; a uniform number would have read as plausible. Dominance has no such blind spot: a condition dominates an exit when every path from an entry passes through it, so it is NECESSARY for the phase to end that way, and a poll loop's test dominates its own exit by construction. Iterative dominators converge in 3 passes over 15670/18739 instructions (83.6%). Result for Stage 02 -- every exit in all three phases is dominated by unit_hp_pct(TCN001, Character_Player_Test) != 0, the player's ship being alive, which falls out rather than being assumed. Beyond that, phase 1's objective exit requires hp_pct_test on ADT102, ADT107 and ADT113; phase 3's requires ADT301 and ADT302; read_freg(0) gates at 210 / 300 and times out at 1200; random(3) and random(5) dominate only the exits that pick one of several closing lines. Two of the 15 exits are reachable from NO static entry, both FORCE_END_PHASE. That agrees with the independently measured 389 unreachable routines: they are started from the trigger queue at phase+272, by data rather than code. Practical note recorded: the first dominator run was OOM-killed -- 6743 nodes each holding a Python set of up to 6743 elements. Integer bitmasks run in seconds. Not settled, and said so: dominance gives necessary, not sufficient, conditions; only Stage 02's artefact is committed; one listed condition is still an unresolved <unknown>; read_freg's units are inferred from the gate values, not read. calls, phase-ends and conditions all regenerate byte-identical. |
||
|
Sylpheed RE agent
|
29eda81ede |
re: recover ISL conditions by CFG dataflow instead of a linear walk
The linear walk's 10% unknown was a floor imposed by the method: a block entered only
by a branch has a well-defined state, just not one a straight-line pass can see.
tools/re-capture/isl_cfg.py replaces it with a worklist fixpoint that joins each
block's state over its ACTUAL predecessors -- a value survives only if every
predecessor agrees.
Over all 28 stages:
instructions reached by the CFG 85.0%
condition sites, unknown LHS 756 (10.00%) -> 402 (5.32%)
of those, never reached at all 389
joined away (predecessors disagree) 13
both resolve but DISAGREE 161 <- linear walk was wrong here
Those 161 are on top of the 889 the previous jmp fix caught.
Two zero-results on the way, both my own bug, both caught because the number looked
wrong rather than because a test failed:
* The first CFG run reached only 36% of instructions and made things WORSE (35%
unknown). Cause: the phase bases reach almost nothing. Most routines are
COROUTINES the engine starts from its trigger queue, with no static predecessor,
so every start_coroutine target has to be seeded as an entry.
* That seeding then found ZERO entries in a file with 216 start_coroutine calls,
because the target is staged in TWO steps -- special[0] = imm, then
local[0] = special[0] -- and I matched only the direct-immediate form.
Reachability went 36% -> 64% -> 85% as each was fixed.
The 389 still unreached are an honest limit rather than a gap: nothing in the bytecode
starts them; they are entered from the trigger queue at phase+272, by data rather than
code, so no purely static analysis reaches them.
isl_report.py conditions now uses isl_cfg; calls and phase-ends regenerate
byte-identical. Stage 02 unknowns drop from 71 to 25.
|
||
|
Sylpheed RE agent
|
f32def9a05 |
re: resolve every ISL condition's comparand -- the clear conditions are readable
The deque ops are an EXPRESSION STACK: push the left operand, evaluate the right (a built-in call, whose result lands in special[0]), pop the comparand back into special[1], compare. Tracking that through the linear decode is enough to recover what each site tests. Evidence the model is right, not just plausible: push vs pop across all 28 stages 1877 vs 1877 files that underflow or end unbalanced 0 of 28 Stage 02 pop.i sites followed by cmp.i 319 / 319 ops immediately before a pop.i call x313, cmp.a x6 isl.conditions() recovers 7563 condition sites disc-wide with 0.0% left as an unresolved special[N]; 83.2% have a built-in call as the LHS and 99.7% compare against a plain number. Most-tested: hp_pct_test 1955, unit_state 1257, unit_relation 796, dist_lt 450, unit_alive 413. They read as conditions now: if unit_alive(TCN105) != 1 if hp_pct_test(ADT308, 0) != 1 if dist_lt(ADT308, TCN000, 15000) != 1 (world unit = 1 m, so 15 km) if unit_state(ADT308) == 1 data/isl-stage02-conditions.txt was a stale artefact with NO generator -- the thing isl_report.py's docstring complained about. It has one now (isl_report.py conditions). The calls and phase-ends artefacts both regenerate byte-identical, so the change is additive. Recorded rather than glossed: 15 of Stage 02's 965 sites (1.6%) attribute the LHS to end_coroutine, which returns no value -- the tracker sets special[0] on EVERY call, so those show a stale value and are wrong, not imprecise. The fix is to set it only for built-ins that write [phase+164], which the vtable work makes checkable. |
||
|
Sylpheed RE agent
|
228680edf0 |
re: the ISL stream is flat -- refute the "needs coroutine entry points" blocker
Two files (isl_report.py's docstring and structures/isl-builtins.md) recorded the same blocker on a faithful per-phase condition listing: that it needs the coroutine entry points from start_coroutine's operand. Measured against isl.call_sites(), which enumerates by scanning the encoding rather than by decoding and so is an independent denominator: linear + jumps, stopping at ret (what the tool did) 133 / 2846 = 4.7% linear + jumps, continuing past ret 2275 / 2846 = 79.9% ... + following start_coroutine (the recorded fix) 2355 / 2846 = 82.7% plain linear decode, no control flow at all 2846 / 2846 = 100.0% Following the coroutine entries buys 2.8 points. Disc-wide, a plain linear decode from the first phase base reaches 25705/25705 call sites over all 28 stages, and 28/28 decode clean to code_end with no desync. The real bug was isl.dis ending on `if op == 20: break`. Op 20 is `ret`, but this is a coroutine VM -- the thread suspends and resumes at the FOLLOWING instruction, so code continues past it. dis() now takes stop_at_ret (default True, preserving the old output: data/isl-stage02.txt regenerates byte-identical) and isl.linear_offsets() is the correct walk. By-product, kept with its control: start_coroutine's target is staged slot 0 -- 73/83 phase-1 sites land on a valid instruction, against a 38.7% chance rate for an arbitrary 4-aligned offset. New artefact data/isl-stage02-phase-ends.txt with a committed generator (isl_report.py phase-ends). It shows END_PHASE's call site is the WRONG place to read a clear condition: all 12 Stage-02 sites sit in one stereotyped outro. Not settled, and stated as such: op10/op13/op14/op21/op23 are unread handlers, so the condition in the poll loop upstream cannot be named yet. |
||
|
Sylpheed RE agent
|
009bca880e |
re: an ISL symbol operand is a (tag, index) pair — and two more names withdrawn
Verified rather than adopted: a subagent proposed that every even operand slot is a type tag. Measured, the strong form is false and a precise form is true. TRUE: a SYMBOL operand is two words, a tag holding the constant 1 followed by the index. Slot 0 is the integer 1 in 19899/19899 calls whose slot 4 is a unit; slot 8 is tag-shaped in 100% of calls for every built-in taking a second unit; slot 16 is 1 in 152/152 for built-in 128, the only one taking a third. The 24 built-ins whose slot 0 is NOT the constant are exactly those taking no symbol there. This explains the unit slots 4/12/20 rather than replacing them. FALSE as stated: slot 8 is a bare double for built-ins 4, 20, 24, 26, 28, 29, 90, 106 and 127, and built-in 75 carries five bare indices at 0/4/8/12/16 with no tags at all. Each built-in has a fixed signature and is 100% self-consistent; none of the 34 with >=20 sites mixes the two. Symbol table 1 has three types -- 1 routes (1362), 6 messages (2247), 7 effects (81) -- and its operand slots are type-pure, measured the same way. Resolving them makes listings say what the script means: `request_script_message(MSG_VOICE_D_257, ...)`, a fourth independent confirmation of that name. Slots 24@4, 46@12 and 114@4 resolve 100% but MIX types 6 and 1, so they are left unresolved rather than guessed. Two more names withdrawn, neither replaced: * 88 `camera_at` -- ZERO call sites in all 28 stages; never testable. * 90 `camera_at_route` -- 8 sites, all Stage 02 phase 3, first operand is symtab-1 type 7 `eff_n0071`, an EFFECT name, in 8/8, with a per-missile Route_ADT301..308_p3M at slot 20. Not aimed at a camera. Left unnamed on purpose: replacing a guessed name with another guess is how the three names corrected earlier today went wrong. Also flagged: 115 `named_event`'s only symbol operand is an eff_* name in 84/84 sites, so that name is suspect too. Not renamed pending a handler read. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE |
||
|
Sylpheed RE agent
|
1ba5d0a4a3 |
re: three ISL built-in names were wrong, including the most-used one
All re-read twice — the handler, and the thing it calls — because each had been named from its shape rather than its effect. * id 11 `yield` -> `end_coroutine`. 0x82272624 is li r11,1 ; li r3,3 ; stw r11,164(r31), and the dispatcher's r3==3 arm erases the thread from the active list and returns it to the free list. It destroys the thread. 2945 sites game-wide, 372 in Stage 02 — the most-used built-in there was. * id 5 `await_label` -> `kill_coroutine(label)`. sub_82273B08 kills the thread parked at the target pc, or itself if the target is its own pc. It waits for nothing. * id 100 `push_trigger` -> `reset_phase_threads`. It clears the trigger container and then frees every thread whose pc differs from the caller's — the opposite of pushing a trigger. Corroborated by usage: its 12 Stage 02 sites all sit in the phase terminator, next to timer_stop, clear_flag(-1) and MARK_LAST_PHASE. One name recovered from the game's own text: opcode 992 prints "RequestScriptMessage %s" at 0x820A5700, so id 64 is request_script_message (2683 sites). Return codes documented properly: 1 = restart the coroutine from its entry (previously not recorded at all), 3 = terminate. And the blocking set was wrong in two places — it is 102, 120, 137, 142, 143. Id 97 does NOT block; its handler ends `b 0x822724F8`, so it always returns 0. Unit-operand resolution settled from DATA over all 28 stages rather than by reading 147 handlers: a slot qualifies only if every value is a valid symtab-2 index, it takes >=15 distinct values, AND its maximum reaches most of the table — that last clause is what discriminates, since every small integer is trivially "in range". 31 built-ins at slot 4, 8 at slot 12, one at slot 20. It also refutes set_flag's slot 0, whose maximum overruns the table, and the resolver now declines rather than inventing a name. New and unexplained: symtab-2 holds two types, 2 and 8, and built-ins 95 and 128 take type 8 at slot 12 in 100% of their sites. A downstream inference is withdrawn with it: the note reading the live trigger counter attributed it to "the script arming watches as it goes" via built-in 100. The measurement stands; the attribution does not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE |