02d3c9c82b
Disassembling the three Stage-02 entry_a targets shows opcodes 0x19 and 0x1A, and the
ISL dispatcher's table has 25 entries (cmplwi 0x18). They are not instructions. Each
phase region ENDS with a trailing data table of 8-byte typed records -- tag 0x19 = int,
tag 0x1A = IEEE float (0.0, 0.5, 1.0, 4.0) -- and entry_a is where it starts.
Confirmed across the disc: in 44 of 44 phases the first offset whose opcode exceeds
0x18 is exactly that phase's entry_a, with zero exceptions, and only two tags ever
appear (1394 x 0x19, 675 x 0x1A). So the record is
0x1883, base, size, 0, code_end, force_end_handler
one boundary and one entry, not two entries as the previous commit said.
That also retires this thread's own "82 of 88 land on a valid instruction = 93.2% vs a
38.6% control" as TOO WEAK a test: a data record has length 8 and passes "nonzero,
even". The entry_b result stands on different evidence -- those targets were matched
against isl.call_sites(), an independent enumeration.
isl.linear_offsets was decoding all 2069 data records as instructions, 1.23% of the
stream. Now each phase's walk stops at its boundary:
decoded instructions 168251 -> 166182 (= 168251 - 2069, as predicted)
opcode > 0x18 2069 -> 0
call sites covered 25705/25705 -> 25705/25705
exits unreachable 0 -> 0
conditions unknown 400 -> 400
Recorded because the first attempt at the fix was worse than the bug: it destroyed 36%
of the stream (168251 -> 107596, exits 0 -> 74) because linear_offsets is ONE global
walk from the first phase base, so stopping at phase 1's table lost every later phase.
It has to skip the region and resume at the next base. A count moving hard in the
wrong direction is the same signal as one that will not move.
Still open: the table's contents are undecoded -- its int values land on the
instruction stream 46/51 against a 29.5% chance rate, but 0 of them are unreached
run-starts, so this is not what starts the unreachable code either.
575 lines
27 KiB
Python
Executable File
575 lines
27 KiB
Python
Executable File
#!/usr/bin/env python3
|
|
"""Disassemble the ISL script bytecode inside a `Stage\\StageNN.ssb`.
|
|
|
|
The VM is `ScriptPhase::Update` (`sub_82263408`). Everything below is read off
|
|
the dispatcher and its 25 handlers, not guessed:
|
|
|
|
0x822635D4 lwz r11,0(r31) ; instruction = one big-endian u32
|
|
0x822635D8 clrlwi r4,r11,24 ; OPCODE = the LOW byte (= byte[3])
|
|
0x822635DC cmplwi 0x18 ; 25 opcodes
|
|
0x822635FC jump table (25 absolute VAs)
|
|
|
|
Each handler advances the pc by `lbz r11,2(r31); add r31,r11,r31`, so
|
|
**byte[2] is the instruction length in bytes**, and bytes [0]/[1] are operand
|
|
kind selectors passed to the operand resolvers as `r4`.
|
|
|
|
op 0 `lbz 0` + word@+8 -> resolve ; `lbz 1` + word@+4 -> lvalue ; stw
|
|
(integer assignment; resolvers 0x82271D40 / 0x82272030)
|
|
op 1 same shape with fmr/stfd (float assignment; 0x82271F10/0x82272120)
|
|
op 12 JUMP: r31 = [phase+232] + word@+4
|
|
-> jump operands are relative to `[phase+232]`, which is **PER PHASE**,
|
|
not the file's 0x24. The phase initialiser sub_82270DF8 writes it
|
|
as 0x24 + the phase's entry from the mission-level stream, whose
|
|
three `0x1883` records carry 0xC0 / 0x14A84 / 0x24B28 for Stage 02
|
|
-> bases 0xE4 / 0x14AA8 / 0x24B4C.
|
|
MEASURED: with 0xE4, 525 of 525 phase-1 branch targets land on an
|
|
instruction boundary; with 0x24, only 188. Using 0x24 for every
|
|
phase -- which this tool did -- gives wrong targets in phases 2
|
|
and 3, and mostly-wrong ones in phase 1.
|
|
op 19 CALL BUILT-IN: `sub_82272220` reads the id from **word@+4**
|
|
(`lwz r11,4(r28); cmplwi 0x92` -> 147 built-ins, table 0x8227226C)
|
|
and word@+8 into [phase+200].
|
|
op 20 sets r29=1 and takes the suspend path -> yield/return.
|
|
|
|
Handler return codes drive the outer loop: 0 = continue, 1 = suspend,
|
|
2/3 = other exits (`0x82263828`).
|
|
|
|
Instruction layout, confirmed by the decode reading cleanly from the code base
|
|
and by every routine ending on a `ret`:
|
|
|
|
byte[3] opcode | byte[2] length | byte[1],byte[0] operand kinds
|
|
following words: operands (12 bytes is the common `call` form)
|
|
|
|
**Operand kinds** (resolver table `0x82271D74`, 4 entries):
|
|
|
|
0 global[i] lis 0x828E / bl 82454A40 / lwzx -- indexed global array
|
|
1 immediate mr r3,r31 -- the operand word itself
|
|
2 special[i] [phase+164] if i==0 else [phase+168]
|
|
3 local[i] addi r3,r3,20 / lwzx -- [phase+20 + i]
|
|
|
|
so the recurring pair
|
|
|
|
set.i k=01,02 <A> <V> special[A] = V (immediate -> special)
|
|
set.i k=02,03 <B> <0> local[B] = special[0]
|
|
|
|
is **argument staging**: values land in `local[]` slots 0,4,8,0xC… and the next
|
|
`call` consumes them. That is why a built-in's arguments are not in its own
|
|
instruction.
|
|
|
|
A `call` carries the built-in id in word@+4 and a monotonically increasing
|
|
STATEMENT ID in word@+8 (0x245, 0x248, 0x24A, ... across a routine) -- the value
|
|
`sub_82272220` stores to `[phase+200]`, i.e. a source-position counter.
|
|
|
|
Usage: isl.py <file.ssb> <offset> [count] offsets are FILE offsets
|
|
isl.py <file.ssb> --entry <off> follow from a code-base offset
|
|
isl.py <file.ssb> --calls every built-in call site + histogram
|
|
isl.py <file.ssb> --to <target> [n] resync and disassemble INTO target
|
|
"""
|
|
import struct
|
|
import sys
|
|
|
|
CODE_BASE_FIELD = 0x08 # .ssb header: code offset (0x24 in every file)
|
|
|
|
# opcode -> (mnemonic, handler VA) from the jump table
|
|
KIND = {0: 'global', 1: 'imm', 2: 'special', 3: 'local'}
|
|
|
|
# op10 `cmp.i` resolves its two operands as
|
|
# LHS = resolve(kind byte[1], word@+4) `lbz r4,1(pc)` + `lwz r5,4(pc)`
|
|
# RHS = resolve(kind byte[0], word@+8) `lbz r4,0(pc)` + `lwz r5,8(pc)`
|
|
# and issues a SIGNED `cmp cr6, 0, LHS, RHS`. op11 `cmp.f` is the same shape
|
|
# through the float resolvers with `fcmpu`. So a listing line
|
|
# cmp.i k=01,02 00000000 00000002
|
|
# reads LHS = special[0], RHS = imm 2 -- "compare special[0] with 2".
|
|
|
|
# Built-in names, from the 147-entry table at 0x8227226C. Only the ones whose
|
|
# handler was actually read are named; the rest print as a bare id rather than a
|
|
# guess. See docs/re/structures/isl-builtins.md.
|
|
BUILTIN = {
|
|
1: 'start_coroutine', 2: 'deploy_squadron', 3: 'move_order', 4: 'wait_s',
|
|
5: 'kill_coroutine', 6: 'END_PHASE', 8: 'set_flag', 9: 'read_freg',
|
|
10: 'random', 11: 'end_coroutine', 12: 'activate_unit',
|
|
13: 'play_se', 14: 'play_bgm',
|
|
15: 'set_group_speed',
|
|
17: 'wait_frames', 18: 'dist_lt', 20: 'hp_pct_test', 24: 'squad_survival_pct',
|
|
26: 'damage_unit', 30: 'objective_marker', 31: 'objective_marker_at_route',
|
|
33: 'global_counter0', 34: 'global_counter1', 36: 'screen_fade',
|
|
39: 'MARK_LAST_PHASE', 40: 'mark_not_last', 43: 'play_voice',
|
|
45: 'play_voice_vol', 46: 'squadron_trace', 47: 'squadron_attack',
|
|
48: 'squadron_escort', 52: 'play_stream', 53: 'sound_busy', 54: 'stop_sound',
|
|
56: 'unit_relation', 59: 'fade_sound', 62: 'FORCE_END_PHASE', 64: 'request_script_message',
|
|
69: 'unit_state', 70: 'unit_alive', 72: 'group_ratio_pct', 73: 'timer_start',
|
|
74: 'timer_limit',
|
|
# ❌ 88 'camera_at' and 90 'camera_at_route' WITHDRAWN. 88 has ZERO call
|
|
# sites in all 28 stages, so its name was never testable. 90 has exactly 8,
|
|
# all in Stage 02 phase 3 (the cruise-missile act), and its first operand
|
|
# resolves to symbol-table-1 type 7 -- `eff_n0071`, an EFFECT name -- in
|
|
# 8/8, with a per-missile `Route_ADT30N_p3M` at slot 20. Whatever it does,
|
|
# it is not aimed at a camera. Left unnamed rather than renamed on a guess.
|
|
77: 'banner_mission_start', 78: 'banner_mission_complete',
|
|
81: 'banner_objective_update', 82: 'banner_mission_failed',
|
|
135: 'banner_mission_restart',
|
|
93: 'clear_flag', 94: 'is_engaged', 95: 'unit_hp_pct', 100: 'reset_phase_threads',
|
|
102: 'prompt_yes_no', 108: 'deploy_squadron_ex',
|
|
109: 'set_unit_flags', 115: 'named_event',
|
|
120: 'wait_cmds_drained', 123: 'timer_resume', 124: 'timer_stop',
|
|
125: 'timer_reset', 126: 'timer_elapsed', 127: 'timer_set',
|
|
132: 'player_gauge0_test', 133: 'player_gauge1_test', 134: 'player_byte',
|
|
137: 'wait_units_ready', 139: 'fade_to_black_end', 142: 'deploy_and_wait',
|
|
143: 'deploy_and_wait2', 145: 'random_rand',
|
|
}
|
|
|
|
OPS = {
|
|
0: 'set.i', 1: 'set.f',
|
|
2: 'cmp.a', 4: 'cmp.a', 6: 'cmp.a', 8: 'cmp.a',
|
|
3: 'cmp.b', 5: 'cmp.b', 7: 'cmp.b', 9: 'cmp.b',
|
|
# 10-18: a condition-code architecture, read off the handlers. op10/op11
|
|
# COMPARE and write three bits into a bitset at `phase+24`
|
|
# bit 0 = EQ bit 1 = GT bit 2 = LT
|
|
# and 13-18 branch on those bits to `[phase+232] + word@+4` -- the same
|
|
# phase-relative target form as the unconditional op12. All six relations
|
|
# are present, which is itself the check that the reading is right.
|
|
10: 'cmp.i', 11: 'cmp.f', 12: 'jmp',
|
|
13: 'beq', 14: 'bne', 15: 'blt', 16: 'ble', 17: 'bgt', 18: 'bge',
|
|
19: 'call', 20: 'ret',
|
|
# 21-24 name the deque ops -- see structures/isl-bytecode.md, verified from
|
|
# the thunks: 21 pushes [phase+168] onto the deque at phase+44, 22 pushes
|
|
# [phase+184] onto phase+64, and 23/24 pop back into +168 / +184.
|
|
21: 'push.i', 22: 'push.f', 23: 'pop.i', 24: 'pop.f',
|
|
}
|
|
|
|
|
|
def load(path):
|
|
return open(path, 'rb').read()
|
|
|
|
|
|
def symbols(b, which):
|
|
"""Parse a .ssb symbol table -> {index: (type, name)}.
|
|
|
|
Built-in argument blobs carry INDICES into these: fields that index
|
|
`[phase+244]` are symtab-1 (routes, messages, subobjectives) and fields that
|
|
index `[phase+324]` are symtab-2 (the unit ids). Resolving them is what turns
|
|
`unit_state(0x2b)` into `unit_state(ADN201)`.
|
|
"""
|
|
off = struct.unpack_from('>I', b, 0x0C if which == 1 else 0x10)[0]
|
|
cnt = struct.unpack_from('>I', b, off)[0]
|
|
base = off + 4
|
|
out = {}
|
|
for i in range(cnt):
|
|
o = struct.unpack_from('>I', b, base + 4 * i)[0]
|
|
if o == 0:
|
|
continue
|
|
rp = base + o
|
|
typ = struct.unpack_from('>I', b, rp)[0]
|
|
e = b.index(b'\0', rp + 4)
|
|
out[i] = (typ, b[rp + 4:e].decode('latin-1'))
|
|
return out
|
|
|
|
|
|
# Built-ins whose operand blob carries a symbol-table-2 (unit) index, by slot.
|
|
#
|
|
# Derived from the DATA, not from reading 147 handlers: across all 28 stages a
|
|
# slot qualifies only if every observed value is a valid symtab-2 index, it takes
|
|
# >=15 distinct values, and its maximum reaches most of the table (symtab-2 tops
|
|
# out at 122 entries, so a non-index slot overruns). That last clause is what
|
|
# makes the test discriminating -- plain range-checking cannot separate an index
|
|
# from a bool, because every small integer is "in range".
|
|
#
|
|
# It also refutes one tempting entry: `set_flag`'s slot 0 passes the range and
|
|
# spread tests but its maximum EXCEEDS the table (flag indices run 0..31 against
|
|
# tables as small as 40), so it is excluded. Slots are only listed here when the
|
|
# ratio stayed below 1.0.
|
|
# DERIVED FROM THE IMPLEMENTATIONS, not from operand ranges. Each of these
|
|
# built-ins resolves to a ScriptPhase vtable slot whose body does
|
|
# lwz rX, 324(rPhase) ; the unit array
|
|
# lwz rY, 4(rArgBase) ; local[4]
|
|
# rlwinm rY, rY, 2, 0, 29 ; x4
|
|
# lwzx ... ; -> the record
|
|
# The previous set was inferred statistically from operand ranges and listed a
|
|
# slot only "when the ratio stayed below 1.0", so it was CONSERVATIVE: all 31 of
|
|
# its entries are confirmed here (zero false positives) but it MISSED 24 more.
|
|
# Control, over all 28 stages: slot 0 is the tag constant 1 in 100.0% of the
|
|
# original 31's calls, 100.0% of the 24 additions', and only 2.5% of the 92
|
|
# built-ins in neither set.
|
|
UNIT_ARG = {2, 3, 7, 12, 15, 16, 18, 19, 20, 21, 22, 23, 24, 25, 26, 28, 29, 30,
|
|
32, 42, 44, 46, 47, 48, 49, 50, 51, 55, 56, 57, 58, 60, 61, 63, 69,
|
|
70, 72, 79, 80, 83, 91, 92, 94, 95, 101, 105, 108, 109, 117, 128,
|
|
136, 137, 141, 142, 143}
|
|
UNIT_ARG2 = {2, 18, 47, 48, 56, 79, 95, 128} # a SECOND unit index at blob[12]
|
|
UNIT_ARG3 = {128} # and a third at blob[20]
|
|
UNIT_SLOTS = {4: UNIT_ARG, 12: UNIT_ARG2, 20: UNIT_ARG3}
|
|
|
|
# WHY the unit indices sit at 4/12/20 and never at 0/8/16: a **symbol operand is
|
|
# a two-word pair** -- a tag word holding the constant 1, then the index. The
|
|
# tag is not data, so printing it puts a meaningless leading `0x1` in front of
|
|
# every unit predicate.
|
|
#
|
|
# Measured over all 28 stages:
|
|
# * slot 0 is the integer 1 in 19899 / 19899 calls whose slot 4 is a unit;
|
|
# * slot 8 is tag-shaped in 100% of calls for every built-in taking a second
|
|
# unit, and slot 16 is the constant 1 in 152/152 for built-in 128, the only
|
|
# one taking a third;
|
|
# * 24 built-ins have a slot 0 that is NOT the constant -- and every one of
|
|
# them takes no symbol at slot 4 (`start_coroutine` a code offset, `wait_s`
|
|
# a double, `set_flag` an index).
|
|
#
|
|
# The tag does NOT generalise to "every even slot is a tag": slot 8 is a bare
|
|
# double for built-ins 4, 20, 24, 26, 28, 29, 90, 106 and 127, and built-in 75
|
|
# carries five bare symbol indices at 0/4/8/12/16 with no tags at all. Each
|
|
# built-in has a fixed signature and is 100% consistent with itself; none mixes.
|
|
TAG_SLOTS = {slot - 4 for slot in UNIT_SLOTS}
|
|
|
|
# Symbol table 1 holds three types, and its slots were measured the same way as
|
|
# the unit slots (every observed value resolves, >=5 distinct values, and the
|
|
# resolved type is pure):
|
|
# type 1 (1362 entries) `Route_*` names
|
|
# type 6 (2247) message / objective names
|
|
# type 7 (81) `eff_*` effect names
|
|
SYM1_SLOTS = {
|
|
0: {64, 75, 115}, # 64 & 75 type 6; 115 type 7
|
|
4: {75, 136},
|
|
8: {75},
|
|
12: {2, 3, 7, 16, 19, 25, 75, 108, 143},
|
|
16: {75},
|
|
20: {90},
|
|
24: {48},
|
|
28: {128},
|
|
}
|
|
# Deliberately NOT listed: built-ins 24@4, 46@12 and 114@4 resolve 100% but mix
|
|
# type 6 and type 1, so the slot's meaning is not one thing. Recorded rather
|
|
# than guessed at.
|
|
|
|
# Symbol table 2 holds TWO entity types: type 2 (1160 entries disc-wide) and
|
|
# type 8 (249). They are not interchangeable -- built-ins 95 and 128 take a
|
|
# type-2 unit at slot 4 and, at slot 12, an operand that is type 8 in 100% of
|
|
# its 90 and 152 call sites respectively. What distinguishes the two classes is
|
|
# not yet established.
|
|
|
|
|
|
def linear_offsets(b, start=None):
|
|
"""Every instruction offset, decoding linearly from the first phase base.
|
|
|
|
MEASURED over all 28 stages: this reaches 25705/25705 of the call sites
|
|
`call_sites()` finds by scanning the encoding, and every file decodes clean
|
|
to `code_end` with no desync. The instruction stream is therefore FLAT --
|
|
reaching a call site needs no control-flow reconstruction at all.
|
|
"""
|
|
end = struct.unpack_from('>I', b, 0x0C)[0] # symtab1 = end of code
|
|
# Walk EACH PHASE separately: a phase's code runs from its base up to that
|
|
# phase's trailing data table, and the next phase's code begins at its own
|
|
# base. A single global walk stops dead at phase 1's table and loses every
|
|
# later phase -- that mistake cost 36% of the instruction stream.
|
|
bases = phase_bases(b)
|
|
if start is None or start in bases:
|
|
out = []
|
|
limits = list(bases[1:]) + [end]
|
|
for base, hi in zip(bases, limits):
|
|
out.extend(_walk_one(b, base, hi))
|
|
return out
|
|
return _walk_one(b, start, end)
|
|
|
|
|
|
def _walk_one(b, start, end):
|
|
out = []
|
|
off = start
|
|
while off + 4 <= end:
|
|
w = struct.unpack_from('>I', b, off)[0]
|
|
# The dispatcher's table has 25 entries (`cmplwi 0x18`), so an opcode
|
|
# above 0x18 is NOT an instruction. Each phase region ends with a
|
|
# trailing DATA table of 8-byte typed records -- tag 0x19 = int,
|
|
# tag 0x1A = IEEE float -- and its start is the FIRST entry of that
|
|
# phase's mission-level `0x1883` record. Measured: in 44 of 44 phases
|
|
# across all 28 stages the first opcode > 0x18 is exactly that value.
|
|
# Decoding those 2069 records as instructions was 1.23% of the stream.
|
|
if (w & 0xFF) > 0x18:
|
|
break
|
|
out.append(off)
|
|
ln = (w >> 8) & 0xFF
|
|
if ln == 0 or ln % 2:
|
|
break
|
|
off += ln
|
|
return out
|
|
|
|
|
|
def dis(b, off, count=40, code_base=0x24, args=True, sym2=None, sym1=None,
|
|
stop_at_ret=True):
|
|
"""`stop_at_ret` preserves the ORIGINAL behaviour and is wrong for reading.
|
|
|
|
op 20 is `ret`, but in a coroutine VM that is a YIELD: the thread suspends
|
|
and later resumes at the following instruction, so code continues after it.
|
|
Stopping there reaches only 133 of Stage 02's 2846 call sites (4.7%);
|
|
continuing reaches all 2846. Pass `stop_at_ret=False` to read a listing.
|
|
"""
|
|
out = []
|
|
staged = {} # local[] slot -> last value staged into it
|
|
pending = None # value most recently put in special[0]
|
|
for _ in range(count):
|
|
if off + 4 > len(b):
|
|
break
|
|
w = struct.unpack_from('>I', b, off)[0]
|
|
op = w & 0xFF
|
|
ln = (w >> 8) & 0xFF
|
|
k1 = (w >> 24) & 0xFF
|
|
k0 = (w >> 16) & 0xFF
|
|
name = OPS.get(op, 'op%d?' % op)
|
|
words = []
|
|
n = max(ln, 4)
|
|
for i in range(4, n, 4):
|
|
if off + i + 4 <= len(b):
|
|
words.append(struct.unpack_from('>I', b, off + i)[0])
|
|
extra = ''
|
|
if op in (0, 1) and len(words) >= 2:
|
|
# op 0/1: lvalue = (kind byte[1], word@+4); rvalue = (kind byte[0], word@+8)
|
|
rv = words[1]
|
|
extra = ' %s[%d] = %s%s' % (
|
|
KIND.get(k0, '?%d' % k0), words[0],
|
|
KIND.get(k1, '?%d' % k1),
|
|
('' if k1 == 1 else '[%s]' % rv) if True else '')
|
|
if k1 == 1:
|
|
if op == 1:
|
|
lo = words[2] if len(words) > 2 else 0
|
|
extra += ' %.6g' % struct.unpack(
|
|
'>d', struct.pack('>II', words[1], lo))[0]
|
|
else:
|
|
extra += ' 0x%X' % words[1]
|
|
# track the staging pattern so a call can show its arguments
|
|
if op in (0, 1) and len(words) >= 2:
|
|
if k0 == 2 and k1 == 1:
|
|
if op == 1:
|
|
# op 1 stores with stfd, so an immediate float operand is a
|
|
# DOUBLE carried as two words -- reading only the high word
|
|
# as a float gives 2.125 where the script means 3.0.
|
|
lo = words[2] if len(words) > 2 else 0
|
|
pending = '%.6g' % struct.unpack(
|
|
'>d', struct.pack('>II', words[1], lo))[0]
|
|
else:
|
|
pending = words[1]
|
|
elif k0 == 3 and k1 == 2 and pending is not None:
|
|
staged[words[0]] = pending
|
|
elif k0 == 3 and k1 == 1:
|
|
# local[i] = immediate, DIRECTLY -- the common form. Missing this
|
|
# made every unit predicate print with no arguments at all.
|
|
if op == 1:
|
|
lo = words[2] if len(words) > 2 else 0
|
|
staged[words[0]] = '%.6g' % struct.unpack(
|
|
'>d', struct.pack('>II', words[1], lo))[0]
|
|
else:
|
|
staged[words[0]] = words[1]
|
|
if op == 19 and words:
|
|
extra = ' %s' % BUILTIN.get(words[0], 'builtin%d' % words[0])
|
|
if args and staged:
|
|
parts = []
|
|
# The tag word in front of a symbol operand is not an argument.
|
|
tags = {slot - 4 for slot, ids in UNIT_SLOTS.items()
|
|
if words[0] in ids and slot in staged}
|
|
for slot, v in sorted(staged.items()):
|
|
if slot in tags and v == 1:
|
|
continue
|
|
txt = ('0x%X' % v) if isinstance(v, int) else v
|
|
# Resolve only a slot that is declared an index AND whose
|
|
# value really is one -- a resolver that invents a name for
|
|
# a non-index is worse than one that prints the raw number.
|
|
if (sym2 and words[0] in UNIT_SLOTS.get(slot, ())
|
|
and isinstance(v, int) and v in sym2):
|
|
txt = sym2[v][1]
|
|
elif (sym1 and words[0] in SYM1_SLOTS.get(slot, ())
|
|
and isinstance(v, int) and v in sym1):
|
|
txt = sym1[v][1]
|
|
parts.append(txt)
|
|
extra += '(' + ', '.join(parts) + ')'
|
|
staged = {}
|
|
elif op == 12 and words:
|
|
extra = ' -> code+0x%X (file 0x%X)' % (words[0], code_base + words[0])
|
|
out.append('%06X: %08X %-6s len=%-3d k=%02x,%02x %s%s' % (
|
|
off, w, name, ln, k1, k0,
|
|
' '.join('%08X' % x for x in words), extra))
|
|
if ln == 0:
|
|
out.append(' (length 0 -- stopping)')
|
|
break
|
|
off += ln
|
|
if op == 20 and stop_at_ret:
|
|
break
|
|
return out
|
|
|
|
|
|
REL = {13: '==', 14: '!=', 15: '<', 16: '<=', 17: '>', 18: '>='}
|
|
|
|
|
|
def conditions(b, sym1=None, sym2=None):
|
|
"""Every condition site, with its comparand resolved.
|
|
|
|
The deque ops are an EXPRESSION STACK, which is what makes this possible:
|
|
`push.i` saves `special[1]`, the right-hand side is evaluated (usually a
|
|
built-in call, whose result lands in `special[0]`), `pop.i` restores the
|
|
saved comparand into `special[1]`, and `cmp.i` compares the two.
|
|
|
|
MEASURED, and this is why the model is trusted: across all 28 stages
|
|
push and pop balance at 1877 each with ZERO underflows, and in Stage 02
|
|
all 319 `pop.i` sites are immediately followed by `cmp.i`.
|
|
|
|
The tracker resets at every CONTROL-FLOW BOUNDARY: `end_coroutine` (returns 3,
|
|
destroying the thread) and `jmp` (unconditional, so the next instruction is
|
|
not reached by fall-through). A linear walk cannot know a block's state when
|
|
that block is only ever entered by a branch, so those sites report an explicit
|
|
unknown instead of a stale value.
|
|
|
|
MEASURED over all 28 stages: resetting at `jmp` changes 889 of 7563 sites
|
|
(11.75%) and leaves 756 (10.00%) honestly unresolved. Recovering those needs
|
|
a real dataflow join over each block's actual predecessors, not a linear walk.
|
|
"""
|
|
bases = phase_bases(b)
|
|
sp, loc, stack = {}, {}, []
|
|
out, pend = [], None
|
|
for off in linear_offsets(b):
|
|
w = struct.unpack_from('>I', b, off)[0]
|
|
op, ln = w & 0xFF, (w >> 8) & 0xFF
|
|
src_kind, dst_kind = (w >> 24) & 0xFF, (w >> 16) & 0xFF
|
|
words = [struct.unpack_from('>I', b, off + i)[0]
|
|
for i in range(4, max(ln, 4), 4) if off + i + 4 <= len(b)]
|
|
|
|
def val(kind, operand, lo=0):
|
|
if kind == 1:
|
|
if op == 1:
|
|
return '%.6g' % struct.unpack('>d',
|
|
struct.pack('>II', operand, lo))[0]
|
|
return str(operand)
|
|
if kind == 2:
|
|
return sp.get(operand, 'special[%d]' % operand)
|
|
if kind == 3:
|
|
return loc.get(operand, 'local[%d]' % operand)
|
|
return 'global[%d]' % operand
|
|
|
|
if op in (0, 1) and len(words) >= 2:
|
|
v = val(src_kind, words[1], words[2] if len(words) > 2 else 0)
|
|
(sp if dst_kind == 2 else loc)[words[0]] = v
|
|
elif op == 19 and words:
|
|
nm = BUILTIN.get(words[0], 'builtin%d' % words[0])
|
|
args = []
|
|
tags = {sl - 4 for sl, ids in UNIT_SLOTS.items()
|
|
if words[0] in ids and sl in loc}
|
|
for sl in sorted(loc):
|
|
v = loc[sl]
|
|
if sl in tags and v == '1':
|
|
continue
|
|
if v.isdigit():
|
|
i = int(v)
|
|
if sym2 and words[0] in UNIT_SLOTS.get(sl, ()) and i in sym2:
|
|
v = sym2[i][1]
|
|
elif sym1 and words[0] in SYM1_SLOTS.get(sl, ()) and i in sym1:
|
|
v = sym1[i][1]
|
|
args.append(v)
|
|
sp[0] = '%s(%s)' % (nm, ', '.join(args))
|
|
loc = {}
|
|
if words[0] == 11:
|
|
# `end_coroutine` returns 3, which DESTROYS the thread -- execution
|
|
# does not continue past it, so the instructions that follow in the
|
|
# flat stream belong to a different routine and every tracked value
|
|
# is stale. Without this reset, 34 sites disc-wide reported
|
|
# `end_coroutine` itself as the left-hand side of a comparison,
|
|
# which is impossible: it returns no value a script can test.
|
|
sp, loc, stack, pend = {}, {}, [], None
|
|
elif op == 12:
|
|
# An UNCONDITIONAL jump: execution never reaches the following
|
|
# instruction by fall-through, so whatever this walk is carrying is
|
|
# not that block's state. Same bug class as `end_coroutine` below,
|
|
# and it is much bigger: 889 of 7563 sites (11.75%) had operands
|
|
# derived from state that leaked across a `jmp`.
|
|
sp, loc, stack, pend = {}, {}, [], None
|
|
elif op == 21:
|
|
stack.append(sp.get(1))
|
|
elif op == 22:
|
|
stack.append(sp.get(1))
|
|
elif op in (23, 24):
|
|
sp[1] = stack.pop() if stack else None
|
|
elif op in (10, 11):
|
|
pend = (off, val(dst_kind, words[0]), val(src_kind, words[1]))
|
|
elif op in REL and pend:
|
|
ph = sum(1 for x in bases if x <= off)
|
|
tgt = bases[ph - 1] + words[0] if words else None
|
|
lhs = pend[1]
|
|
if lhs.startswith('special['):
|
|
lhs = '<unknown: block entered by a branch, not by fall-through>'
|
|
out.append({'off': pend[0], 'phase': ph, 'lhs': lhs,
|
|
'rel': REL[op], 'rhs': pend[2], 'branch': off,
|
|
'target': tgt})
|
|
pend = None
|
|
return out
|
|
|
|
|
|
def call_sites(b):
|
|
"""Every `call` in the code region. Scans on the encoding, not by decoding,
|
|
so a bad length somewhere cannot hide the rest of the file."""
|
|
code_end = struct.unpack_from('>I', b, 0x0C)[0] # symtab1 = end of code
|
|
out = []
|
|
off = struct.unpack_from('>I', b, CODE_BASE_FIELD)[0]
|
|
while off + 12 <= code_end:
|
|
w = struct.unpack_from('>I', b, off)[0]
|
|
if (w & 0xFF) == 0x13 and ((w >> 8) & 0xFF) == 12 and (w >> 16) == 0:
|
|
bid = struct.unpack_from('>I', b, off + 4)[0]
|
|
if bid <= 0x92:
|
|
out.append((off, bid, struct.unpack_from('>I', b, off + 8)[0]))
|
|
off += 4
|
|
return out
|
|
|
|
|
|
def resync(b, target, back=400):
|
|
"""Find a start from which linear decode lands exactly on `target`.
|
|
|
|
Instructions are variable-length, so you cannot simply walk backwards; but a
|
|
wrong start almost always desynchronises into an invalid length, so trying
|
|
every 4-byte start in a window and keeping the one that hits the target
|
|
exactly is reliable in practice.
|
|
"""
|
|
for start in range(max(0, target - back), target, 4):
|
|
off = start
|
|
for _ in range(300):
|
|
if off >= target or off + 4 > len(b):
|
|
break
|
|
ln = (struct.unpack_from('>I', b, off)[0] >> 8) & 0xFF
|
|
if ln == 0 or ln % 2:
|
|
off = -1
|
|
break
|
|
off += ln
|
|
if off == target:
|
|
return start
|
|
return None
|
|
|
|
|
|
def phase_bases(b):
|
|
"""The per-phase code bases, from the mission-level stream's 0x1883 records."""
|
|
out = []
|
|
off = struct.unpack_from('>I', b, CODE_BASE_FIELD)[0]
|
|
for o in range(0x24, 0x100, 4):
|
|
if struct.unpack_from('>I', b, o)[0] == 0x1883:
|
|
out.append(off + struct.unpack_from('>I', b, o + 4)[0])
|
|
return out
|
|
|
|
|
|
if __name__ == '__main__':
|
|
b = load(sys.argv[1])
|
|
if sys.argv[2:3] == ['--calls']:
|
|
import collections
|
|
cs = call_sites(b)
|
|
h = collections.Counter(bid for _, bid, _ in cs)
|
|
print('%d call sites, %d distinct built-ins' % (len(cs), len(h)))
|
|
for bid, n in h.most_common():
|
|
print(' builtin %-4d %5d site(s)' % (bid, n))
|
|
sys.exit(0)
|
|
if sys.argv[2:3] == ['--to']:
|
|
t = int(sys.argv[3], 0)
|
|
st = resync(b, t)
|
|
if st is None:
|
|
print('could not resync into 0x%X' % t); sys.exit(1)
|
|
print('resync from 0x%X' % st)
|
|
print('\n'.join(dis(b, st, int(sys.argv[4], 0) if len(sys.argv) > 4 else 40,
|
|
sym2=symbols(b, 2))))
|
|
sys.exit(0)
|
|
code_base = struct.unpack_from('>I', b, CODE_BASE_FIELD)[0]
|
|
a = sys.argv[2]
|
|
if a == '--entry':
|
|
off = code_base + int(sys.argv[3], 0)
|
|
else:
|
|
off = int(a, 0)
|
|
cnt = int(sys.argv[4], 0) if len(sys.argv) > 4 else 40
|
|
print('code base 0x%X, disassembling from 0x%X' % (code_base, off))
|
|
print('\n'.join(dis(b, off, cnt, code_base)))
|