re: resolve every ISL condition's comparand -- the clear conditions are readable

The deque ops are an EXPRESSION STACK: push the left operand, evaluate the right
(a built-in call, whose result lands in special[0]), pop the comparand back into
special[1], compare.  Tracking that through the linear decode is enough to recover
what each site tests.

Evidence the model is right, not just plausible:
  push vs pop across all 28 stages          1877 vs 1877
  files that underflow or end unbalanced    0 of 28
  Stage 02 pop.i sites followed by cmp.i    319 / 319
  ops immediately before a pop.i            call x313, cmp.a x6

isl.conditions() recovers 7563 condition sites disc-wide with 0.0% left as an
unresolved special[N]; 83.2% have a built-in call as the LHS and 99.7% compare
against a plain number.  Most-tested: hp_pct_test 1955, unit_state 1257,
unit_relation 796, dist_lt 450, unit_alive 413.

They read as conditions now:
  if unit_alive(TCN105) != 1
  if hp_pct_test(ADT308, 0) != 1
  if dist_lt(ADT308, TCN000, 15000) != 1      (world unit = 1 m, so 15 km)
  if unit_state(ADT308) == 1

data/isl-stage02-conditions.txt was a stale artefact with NO generator -- the thing
isl_report.py's docstring complained about.  It has one now (isl_report.py
conditions).  The calls and phase-ends artefacts both regenerate byte-identical, so
the change is additive.

Recorded rather than glossed: 15 of Stage 02's 965 sites (1.6%) attribute the LHS to
end_coroutine, which returns no value -- the tracker sets special[0] on EVERY call,
so those show a stale value and are wrong, not imprecise.  The fix is to set it only
for built-ins that write [phase+164], which the vtable work makes checkable.
This commit is contained in:
Sylpheed RE agent
2026-08-27 05:28:50 +00:00
parent 0287a78633
commit f32def9a05
5 changed files with 1214 additions and 343 deletions

View File

@@ -1412,9 +1412,26 @@ premise was wrong.**
db's own `vptr_writes`. `unit_state` = slot 184 = `0x8226ADF0`, which indexes
`[phase+324]` by `local[4]` and writes its answer to **`[phase+164]` =
`special[0]`**. So: **result → `special[0]`, comparand popped → `special[1]`,
then `cmp.i` + branch.** ▶️ **Still open:** the other 111 vtable slots are a
then `cmp.i` + branch.** ~~▶️ **Still open:** the other 111 vtable slots are a
lookup but unread; which comparand each site pushes (needs `push.i` tracked
through the decode); the 35 non-vtable built-ins; the vtable's length. The condition lives in the `op10`/`op13` poll loop upstream of
through the decode); the 35 non-vtable built-ins; the vtable's length.~~
**(2026-08-27) COMPARANDS RESOLVED — [structures/isl-conditions](structures/isl-conditions.md).**
The deques are an **expression stack**: push the left operand, evaluate the
right (result → `special[0]`), pop → `special[1]`, compare. Evidence: push/pop
balance **1877/1877 across 28 stages with ZERO underflows**, and **319/319**
Stage-02 `pop.i` sites are immediately followed by `cmp.i`. `isl.conditions()`
now recovers **7563 condition sites disc-wide with 0.0 % left unresolved**
(83.2 % have a built-in call as LHS, 99.7 % compare against a plain number).
`data/isl-stage02-conditions.txt` finally has a generator
(`isl_report.py conditions`); the other two artefacts regenerate identical.
Top predicates: `hp_pct_test` 1955, `unit_state` 1257, `unit_relation` 796,
`dist_lt` 450. 🟡 **1.6 % are WRONG:** 15 sites attribute the LHS to
`end_coroutine`, which returns nothing — the tracker sets `special[0]` on every
call, so those show a stale value. Fix = only set it for built-ins that write
`[phase+164]`. ▶️ **Still open:** the 35 unnamed built-ins (`builtin16` 132
sites, `builtin105` 117, `builtin103` 115 — each now a vtable-slot lookup);
which condition guards each `END_PHASE` (needs the control flow between them);
the vtable's length. The condition lives in the `op10`/`op13` poll loop upstream of
the outro — e.g. phase 3 polls `unit_state(ADT308)` and branches back to
`0xFEB4` until it passes. Artefact: `data/isl-stage02-phase-ends.txt`.
* 🐛 **(2026-08-25) The nav fix is NOT fully reliable.** `dialog_up.py` works on

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,94 @@
# ✅ Every condition site, with its comparand — the clear conditions are readable
This is what the whole ISL chain was for. `data/isl-stage02-conditions.txt` has
existed for a long time as a **stale artefact with no generator**
`isl_report.py`'s own docstring says so. It has one now, and the conditions are
resolved rather than printed as `special[N]`.
## ✅ The deque ops are an expression stack
`isl-bytecode.md` names ops 2124 `push.i`/`push.f`/`pop.i`/`pop.f` over deques
at `phase+44` / `phase+64`. What they are *used for* is the missing piece:
```
set.i special[0] = 1
set.i special[1] = special[0]
push.i ; save the comparand
set.i local[0] = 1
set.i local[4] = 0x49
call unit_state(ADT308) ; result -> special[0] (clobbers it)
pop.i ; restore comparand -> special[1]
cmp.i special[0], special[1]
beq -> 0xFEB4
```
A textbook stack-machine lowering: **push the left operand, evaluate the right,
pop, compare.** Tracking the stack through the decode is therefore enough to
recover what every site actually tests.
### The evidence it is a stack, not something else
| check | result |
|---|---|
| `push` vs `pop` across all 28 stages | **1877 vs 1877** |
| files where the deque underflows or ends unbalanced | **0 of 28** |
| Stage 02 `pop.i` sites immediately followed by `cmp.i` | **319 / 319** |
| ops immediately preceding a `pop.i` | `call` ×313, `cmp.a` ×6 |
Perfect balance with zero underflow across 28 independent files is not something
a wrong model produces, and `pop.i``cmp.i` at 319/319 makes `pop.i` a reliable
marker for a condition site.
## ✅ Result — disc-wide
`isl.conditions()` walks the linear stream tracking `special[]`, `local[]` and
the stack, then reports each compare with its branch:
| | all 28 stages |
|---|---|
| condition sites | **7 563** |
| **either side left unresolved as `special[N]`** | **0 — 0.0 %** |
| LHS is a resolved built-in call | 6 292 — 83.2 % |
| RHS is a plain number | 7 544 — 99.7 % |
**Most-tested predicates disc-wide:** `hp_pct_test` 1955, `unit_state` 1257,
`unit_relation` 796, `dist_lt` 450, `request_script_message` 425,
`unit_alive` 413, `read_freg` 187.
And they read as conditions:
```
0x032F4 ph1 if unit_alive(TCN105) != 1 -> 0x3388
0x04D70 ph1 if unit_hp_pct(TCN001, Character_Player_Test) != 0 -> 0x51FC
0x2C4E0 ph3 if hp_pct_test(ADT308, 0) != 1 -> 0x2CF8C
0x30790 ph3 if dist_lt(ADT308, TCN000, 15000) != 1 -> 0x307B0
0x33FA0 ph3 if unit_state(ADT308) != 2 -> 0x34030
0x349E0 ph3 if unit_state(ADT308) == 1 -> 0x34A00
```
`dist_lt(ADT308, TCN000, 15000)` — with the world unit established as 1 metre,
that is a **15 km** proximity test.
## 🟡 A caveat the numbers themselves expose
15 of Stage 02's 965 sites (1.6 %) attribute the left-hand side to
**`end_coroutine`**, which does not return a value. The tracker sets
`special[0]` on *every* call, so where the right-hand side of a comparison was
not actually a value-producing built-in, the LHS shown is a **stale**
`special[0]` from an earlier call. Those sites are wrong, not merely imprecise.
The fix is to set `special[0]` only for built-ins that write `[phase+164]`
which [`isl-builtin-dispatch`](isl-builtin-dispatch.md) makes checkable, since
112 of 147 are vtable slots whose implementations can be tested for that store.
Not done; recorded so the 1.6 % is not read as accurate.
## 🟡 Not settled
* The **35 unnamed built-ins** still print as `builtinN``builtin103` (115
sites), `builtin105` (117) and `builtin16` (132) are the highest-traffic
unknowns, and each is now a vtable-slot lookup away.
* **Which condition guards each `END_PHASE`.** Every site is readable, but
linking a condition to the phase exit it eventually reaches needs the control
flow between them, which this listing does not follow.
* Only Stage 02's artefact is committed; the other 27 generate from the same
command but are not in the tree.

View File

@@ -354,6 +354,82 @@ def dis(b, off, count=40, code_base=0x24, args=True, sym2=None, sym1=None,
return out
REL = {13: '==', 14: '!=', 15: '<', 16: '<=', 17: '>', 18: '>='}
def conditions(b, sym1=None, sym2=None):
"""Every condition site, with its comparand resolved.
The deque ops are an EXPRESSION STACK, which is what makes this possible:
`push.i` saves `special[1]`, the right-hand side is evaluated (usually a
built-in call, whose result lands in `special[0]`), `pop.i` restores the
saved comparand into `special[1]`, and `cmp.i` compares the two.
MEASURED, and this is why the model is trusted: across all 28 stages
push and pop balance at 1877 each with ZERO underflows, and in Stage 02
all 319 `pop.i` sites are immediately followed by `cmp.i`.
"""
bases = phase_bases(b)
sp, loc, stack = {}, {}, []
out, pend = [], None
for off in linear_offsets(b):
w = struct.unpack_from('>I', b, off)[0]
op, ln = w & 0xFF, (w >> 8) & 0xFF
src_kind, dst_kind = (w >> 24) & 0xFF, (w >> 16) & 0xFF
words = [struct.unpack_from('>I', b, off + i)[0]
for i in range(4, max(ln, 4), 4) if off + i + 4 <= len(b)]
def val(kind, operand, lo=0):
if kind == 1:
if op == 1:
return '%.6g' % struct.unpack('>d',
struct.pack('>II', operand, lo))[0]
return str(operand)
if kind == 2:
return sp.get(operand, 'special[%d]' % operand)
if kind == 3:
return loc.get(operand, 'local[%d]' % operand)
return 'global[%d]' % operand
if op in (0, 1) and len(words) >= 2:
v = val(src_kind, words[1], words[2] if len(words) > 2 else 0)
(sp if dst_kind == 2 else loc)[words[0]] = v
elif op == 19 and words:
nm = BUILTIN.get(words[0], 'builtin%d' % words[0])
args = []
tags = {sl - 4 for sl, ids in UNIT_SLOTS.items()
if words[0] in ids and sl in loc}
for sl in sorted(loc):
v = loc[sl]
if sl in tags and v == '1':
continue
if v.isdigit():
i = int(v)
if sym2 and words[0] in UNIT_SLOTS.get(sl, ()) and i in sym2:
v = sym2[i][1]
elif sym1 and words[0] in SYM1_SLOTS.get(sl, ()) and i in sym1:
v = sym1[i][1]
args.append(v)
sp[0] = '%s(%s)' % (nm, ', '.join(args))
loc = {}
elif op == 21:
stack.append(sp.get(1))
elif op == 22:
stack.append(sp.get(1))
elif op in (23, 24):
sp[1] = stack.pop() if stack else None
elif op in (10, 11):
pend = (off, val(dst_kind, words[0]), val(src_kind, words[1]))
elif op in REL and pend:
ph = sum(1 for x in bases if x <= off)
tgt = bases[ph - 1] + words[0] if words else None
out.append({'off': pend[0], 'phase': ph, 'lhs': pend[1],
'rel': REL[op], 'rhs': pend[2], 'branch': off,
'target': tgt})
pend = None
return out
def call_sites(b):
"""Every `call` in the code region. Scans on the encoding, not by decoding,
so a bad length somewhere cannot hide the rest of the file."""

View File

@@ -9,6 +9,7 @@ arguments) and once when three built-in names were corrected. Keeping the
generator in the tree is the point of this file.
isl_report.py <StageNN.ssb> phase-ends -> every END_PHASE with its context
isl_report.py <StageNN.ssb> conditions -> every condition site, comparand resolved
The "needs the coroutine entry points" blocker recorded here is REFUTED: the
instruction stream is FLAT and `isl.linear_offsets` reaches 25705/25705 call
@@ -93,11 +94,42 @@ def emit_phase_ends(b, path):
print(' ' + line)
def emit_conditions(b, path):
"""Every condition site with its comparand — what `data/isl-stage02-conditions.txt`
never had a generator for."""
import collections
cs = isl.conditions(b, isl.symbols(b, 1), isl.symbols(b, 2))
bases = isl.phase_bases(b)
print('# %s — condition sites, comparands resolved' % path)
print()
print('Generated by `tools/re-capture/isl_report.py conditions`.')
print()
print('The deque ops are an expression stack: `push.i` saves the comparand,')
print('the right-hand side is evaluated (its result lands in `special[0]`),')
print('`pop.i` restores the comparand into `special[1]`, then `cmp.i` compares.')
print()
print('%d condition sites; phase bases %s'
% (len(cs), ' '.join('0x%x' % x for x in bases)))
h = collections.Counter(c['lhs'].split('(')[0] for c in cs if '(' in c['lhs'])
print()
print('## most-tested predicates')
for k, n in h.most_common(20):
print(' %-28s %4d' % (k, n))
for ph in range(1, len(bases) + 1):
rows = [c for c in cs if c['phase'] == ph]
print()
print('## phase %d%d sites' % (ph, len(rows)))
for c in rows:
print(' 0x%06X if %s %s %s -> 0x%X'
% (c['off'], c['lhs'], c['rel'], c['rhs'], c['target']))
def main():
path = sys.argv[1]
b = isl.load(path)
name = path.replace('\\', '/').split('/')[-1]
{'calls': emit_calls, 'phase-ends': emit_phase_ends}[sys.argv[2]](b, name)
{'calls': emit_calls, 'phase-ends': emit_phase_ends,
'conditions': emit_conditions}[sys.argv[2]](b, name)
if __name__ == '__main__':