The page rested on twelve values from two screens. This walks all 4525 sprites
on the disc.
* The field is a u16 at +0x0A. The upper half of the 32-bit word the page
reads is zero in 4525/4525. Nothing above changes -- 0x00008100 sorts the
same as 0x8100 -- but a future value with the high half set would mean
something had been misread rather than that the layer got deeper.
* It is an enumeration: 45 values for 4525 sprites, one of which (0x8100)
covers 1188 of them.
* The reading worth trying -- a global layer vocabulary shared across the UI
-- is refuted. Only 4 of 45 keys cross a pak family and 33 of 45 live only
in GP_MAIN_GAME_2D; every other pak owns a narrow high-byte band (0x90-0x94
for the in-game overlays, 0xa4 mission log, 0xb1-0xb2 save/load). A screen
that owns one or two keys is not ordering itself with them.
That supports "group id in the high bits, order in the low bits", which is what
the page already suspected, but it does NOT test it: paint order has been
measured on two screens and both are inside GP_MAIN_GAME_2D, so there is no
ground truth to check the split against. Left amber.
The first number I got was 37/45 shared, which would have supported precisely
the wrong conclusion. It came from counting paks instead of pak families: the
six GP_MAIN_GAME_*2D paks are the same screens in six languages and their key
sets are byte-for-byte identical. Recorded on the page, because the shape
recurs -- a corpus with near-duplicate members manufactures agreement.
Two follow-ups on yesterday's^Wthis morning's CollisionSet write-up.
1. The _cmesh <-> render-model link, which I recorded as UNTESTED because
matching stems against .xbg object names covered 4 of 158. The disc keeps
only one build manifest, so that corpus was never going to answer it. The
right corpus is the GameResourceID field of the DefTables / GP_MAIN_GAME
records -- 480 distinct values. Against those, with a control that shuffles
the characters of each stem:
ship/mob stems prefixed by a real resource id 108/112 = 96.4%
same stems, characters shuffled (control) 0/112 = 0.0%
asteroid stems prefixed (expected none) 0/46
So a CollisionSet entry is <GameResourceID>[_<part>]_cmesh. The 0/46 on
asteroids matters as much as the 108/112: a test that fired on everything
would be the bound-check hazard again.
2. The world unit. Sweeping every pak for a name carrying a kilometre figure
returns mapmesh_box_500km.col/.rgn and nothing else -- 162 references, all to
that one pair. The reading rests on a single filename with no corroborating
instance anywhere in the data, so no static test can settle it; marking it
blocked on the oracle rather than leaving it as an open static question.
My objection's premise did survive: rou_e010 is a real GameResourceID and
e010_ADAN_Attacker_S is in the stage tables, so the 133-unit mesh does belong
to a craft the game calls an attacker. Whether the trailing _S means "small"
is a further guess (there are _EX4 / _HF / _HF_Wayne variants), so it stays
suggestive rather than evidence.
All 18 blobs are byte-identical: the per-stage naming is nominal, and every
stage points at one shared 1675148-byte library stored eighteen times. That
identical size was the reason to open the item, and it turned out to be the
answer to it.
Record layout: {u32 size, u32 name_len, char name[name_len], u32 nv, u32 nt,
f32[3] x nv, u32[3] x nt}, next record at off + 8 + size. The indices are u32
here where MCOL uses u16 -- two different serialisers in one archive.
What makes this a decode rather than a plausible reading: the walk consumes the
file to the byte over 158 variable-length records, with the size word predicted
from the two counts 158/158. A wrong field would desynchronise within a few
records and could not land exactly on the end. All indices in range 158/158;
98.24% of edges shared by exactly two triangles; 147/158 fully manifold.
158 meshes, 90 836 triangles: per-part ship proxies (_bdy/_brg/_eng/_wep/_sld,
the XBG7 sub-part vocabulary) plus 46 stage asteroid meshes whose prefixes are
exactly the stages that have an _AsteroidVolume_wp MCOL.
Two things this file makes me walk back:
* The "1 unit = 1 metre" reading from mapmesh_box_500km is downgraded to
amber. The 500000 arithmetic stands, but it implies that a craft the game's
own tables call "small" is 133 m and that rob_f002 is 447 km -- 89% of the
arena width. The format check survives; the interpretation has no
independent support.
* The _cmesh <-> render-model name link is recorded as UNTESTED, not
confirmed: only one .xbg build manifest survives on the disc, so matching
stems against object names covers 4 of 158, which is no coverage at all.
The names live outside MiscBin: they are the MapPath / MapMesh /
CollisionMeshes field values of the per-stage StageResource object (IDXD schema
3c9ae32e, in every GP_MAIN_GAME_<lang>.pak), and each hashes with the ordinary
pak name_hash straight to a TOC entry. 40/40 resolve, no collisions -- the 11
REGN as <stem>.rgn, the 11 MCOL as <stem>.col, and the 18 remaining blobs as
CollisionSet_S01..S16 / _Tutorial / _test.bin. The .pe string table at 651540
was the way in: MapMesh and MapPath sit adjacent there.
This upgrades the pairing claim. The first section of mcol-collision.md could
only say REGN and MCOL had matching *distributions* of bbox and cell size, and
flagged that as not an object-to-object link. A phase record names one .rgn and
one .col, and all 11/11 pairs share a stem and agree exactly on both.
The names also check the format work from outside it: mapmesh_box_500km.col is
the object decoded here as 8 vertices and 12 triangles spanning exactly
+-250000, and its name says that cube is 500 km across -- so one world unit is
one metre, and a wrong stride could not have produced a box that measures what
its own filename claims. 70 of the 87 phases use it: most stages' only
collision is the arena wall, and _AsteroidVolume_ names the rest.
Still open: the 18 CollisionSet_*.bin are named but not decoded (all exactly
1675148 bytes), and CMapColliderBridge in the RTTI names the runtime consumer
without following it into the code.
The 0x50 header word, which the first section of this page had dismissed as "a
large value", is two u16 counts: vertices and triangles. They give the two
remaining blocks their stride, and every derived length is exact in 11/11 --
len(0x54) == align16(12*nv), len(0x58) == align16(6*nt), and nt equals the
bounding-sphere count decoded last iteration.
Checks that cannot pass by accident:
* sphere i is the TIGHT bounding sphere of triangle i, 4768/4768, with
max|v-c|/r median 0.99990 (a fixed 1.0001 epsilon), against a 1.32%
random-triangle control;
* the mesh is watertight -- every edge shared by exactly two triangles,
7152/7152, zero degenerate triangles, zero unreferenced vertices;
* the two smallest objects are 8 vertices and 12 triangles whose positions
are the eight +-250000 corners of the map bbox: a bare bounding cube.
The cell lists are a correct broad phase: with an exact triangle/box SAT test
only 3 overlapping triangles in 18 577 entries are absent, so a query walking
one cell's list cannot miss a hit. The 730 conservative extras bracket the
builder's own test between exact-SAT and AABB, which retires the 18 unexplained
"sphere misses" from the previous commit as that same margin.
mcol_probe.py gains `mesh` and `obj`; `verify` now runs all three checks and its
output is recorded in docs/re/data/mcol-verify.txt.
The unexplained ~0.75 ratio left at the end of the last iteration was my own
stride. I had read the block as 12-byte points because REGN's vertex section
is 12 bytes, and never checked it: len(0x5C) is not a multiple of 12 in 5 of
the 11 objects, so that stride was never arithmetically possible.
At stride 16 the relation is exact in 11/11 -- max u16 == len(0x5C)/16 - 1 --
and the record reads as {centre f32[3], radius f32}. Powered test, since a
u16 is reached through a specific grid cell: the sphere it names reaches that
cell in 18 559/18 577 = 99.90%, against a 12.02% random-sphere control. Both
fields carry signal (centre alone 26.75%, radius shuffled 70.19%).
The converse -- is the list *exactly* the intersecting set? -- is 0.38%, which
is the expected direction: a bounding sphere is conservative, so membership
implies overlap but not the reverse. The tighter geometry is in 0x54/0x58,
still undecoded. 18 entries (0.10%) go the wrong way and are recorded as open.
tools/re-capture/regn_decode.py is copied unchanged from auto/regn-reader so
the probe's POF0 reader is the known-good one rather than a second copy.
dat/tables.pak holds a 5798-entry SOUNDS record (cue name -> sound id) and a
5135-entry FILES record (.slb bank paths). Cue names are the join key, so a
script message id now resolves all the way to the bank that voices it:
MSG_VOICE_D_257 -> VOICE_D_257 -> 6945 -> jpn\etc\VOICE_D_257.slb.
The prefix rule is MSG_ -> VOICE_, not strip-MSG_. My first rule was the
latter; it left 88 names unresolved and I was about to write those families up
as text-only announcements, until VOICE_TCAF_592.slb turned up in FILES and
refuted it. Corrected rule resolves 1326 of 1338, and SOUNDS and FILES agree on
exactly the same 12 absentees.
Separately, MSG_DEMO_* is driven by its own IDXD tables in the language packs,
which carry speaker, portrait, on-screen seconds and audio cue per page. Field
count is 9*PageCount+2 for all 7 distinct PageCounts; 1252/1252 caption-key
slots match <ID>_<page>_<line>; the 78 multi-page records equal the 78 counted
independently from the caption side; 138 ids close exactly against the caption
table both ways.
Does not settle the known VOICE_D_452 wrong-recording case -- every cue id is
distinct, so bank sharing is not happening at this layer.
Measured the <id>_<page>_<line> key structure: a page is one subtitle box of
3 or 4 wrapped lines, and successive pages are successive utterances by
possibly different speakers. 452 of 4091 ids span more than one page, up to 8.
That refutes the isl_dialogue.py committed two commits ago, which read only
page 000 -- 356 of the 1338 script message names are multi-page, so a quarter
of its output was truncated to the opening utterance. Tool now walks pages
until one is empty; Stage 02 sample regenerated (43 of 213 calls multi-page).
The 2683/2683 resolution figure is unaffected: it counted ids that have text,
and every id does have a page 000. What was wrong was the rendered text.
Built-in 64's slot-0 operand is a symbol-table-1 type-6 message id, and every
one of them now has caption text: 2683 of 2683 call sites across the 28 stage
scripts, 1338 distinct names, no residue of any kind.
This only became reachable once build_caption_text was switched to the IXUD
field table (537 -> 8800 lines); before that most of these names had nothing
to resolve to.
Adds isl_dialogue.py plus a committed Stage 02 sample. Does not settle which
recording plays for a given line, multi-page captions, or the other five
languages.
sub_822FE040 fills 1023 eight-byte slots at table+32 with a default and then
overwrites individual ones; slot = (N - 32) / 8 from each std r9, N(r31).
Symbolically executing it yields 57 populated slots, matching the count the
corpus recorded, now with the full opcode -> handler map committed as
docs/re/data/isl-command-table.txt and regenerable from
tools/re-capture/isl_cmdtab.py.
Nine opcodes point at 0x82391BA8, which is `li r3,1 ; blr` -- accept and
discard. 768, 769, 774, 775, 776, 791, 792, 793 and 805 are dead in this
build, which is why the built-ins posting them do nothing.
Opcodes 800-802's entries are thunks 8 bytes apart into 60-instruction
handlers that differ in exactly two words: a descriptor offset and a unit
message id.
800 builtin 26 0xED0802DE
801 builtin 28 0xED0803DE
802 builtin 29, 101 0xED0804DE
That fixes the id format as 0xED08 nn DE, and the ids known from other work
fit it: opcode 514 -> 00DE, 803 -> 07DE, 999 -> 0FDE.
Stopped one link short of the semantics, and saying so: the pump's arm for
0xED0802DE does not apply an effect. It walks the unit's child list at
[unit+320]/[unit+324] and REBROADCASTS to each child as 0xED0902DE. So 0xED08
is the to-unit family and 0xED09 the to-child one, and the terminal effect is
further on. 26/28/29 remain unnamed.
The command table is the reusable part -- it answers "what does this opcode
reach" for every future built-in question, not just this family.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
1146 sites in 22 stages, the second-largest unnamed built-in. Its method
sub_822646B8 (vtable slot 300) and built-in 2's sub_822642E0 (slot 12) are
190 and 199 instructions and differ in exactly one block. Diffed instruction
by instruction, 108 adds:
lwz r11, 16(r29) the blob's slot-16 int
cmpwi/blt/cmpwi/bgt range-check n to [0, 31]
slw r21, r25, r11 r21 = 1 << n <- a 32-bit selector
... default 1 when out of range
Everywhere built-in 2 passes its r21, 108 passes r20 and reserves r21 for the
mask, so the bit is an EXTRA argument to the same call rather than a
replacement. Both post the identical command word AB0100BA, opcode 256.
The operand is always a valid bit index: over all 1146 sites slot 16 is in
[0, 31], 1146 of 1146, none outside, so the out-of-range default never fires
in shipped content. Fifteen distinct values clustered at 16 (531x), 31
(165x), 20 (161x) and 2 (90x); 21 of 22 stages use more than one.
What the bit SELECTS is not established and the name does not claim it. There
is plainly a 32-bit space -- built-in 92 reportedly allocates a free bit by
OR-ing over live units, which would make 108 its "place in a named slot"
counterpart -- but I have not verified that, so the name says only what is
proven: the same deploy as built-in 2, with an extra selector.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The highest-traffic unnamed built-in: 1197 sites across all 28 stages.
sub_822659F0 read directly:
* indexes [phase+324]'s record array by the slot-4 symbol;
* returns 0 immediately when the live object [record+4] is NULL, so it
registers an object that already exists rather than spawning one;
* sets [record+16] = 2, the documented "active" state every unit predicate
tests;
* stores sub_82301118's packed result into [record+20] (low 16) and
[record+24] (high 16) -- member counts;
* posts opcode 513 (0xAB0201BA) either way; the slot-8 mode (1 in 999 sites,
0 in 198) only decides whether cmd+20 is also set to 1.
The ordering test: if this activates a unit for the script, no predicate
should test a unit before it. Over all 28 stages, for every (stage, unit)
pair having both, activate_unit comes first 517 times and a predicate first
0 times. Recorded as file order rather than proven execution order --
coroutines can interleave -- but 517 with no exceptions is not a coincidence.
344 units are tested without ever being activated (live from mission start)
and 203 activated without being tested.
Also recorded: a tidy closure that FAILS. squad_survival_pct reports current
over initial and activate_unit snapshots counts, so the snapshot looks like
the baseline. It is not -- built-in 24 reads [record+16] for the state then
calls sub_823011B0 and sub_82301118 on the LIVE object, never touching
+20/+24. What reads those two fields is unidentified.
Named coverage is now 57 of 108 distinct ids and ~80% of call-site traffic.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The member object sub_82348830 returns is the per-member unit DEFINITION, and
that identification is not a guess: the same spawn loop builds two aggregates
and each lands on a semantically apt field with the apt reducer.
group +192 min, seeded FLT_MAX member +164 = CruisingVelocity
group +472 sum member +84 = HP
A wrong struct would have to make both offsets land on apt fields AND pair
each with the apt reducer. Minimum of a speed, sum of hit points: a
formation's cruise limit and its total health.
The quantitative test over all 1360 sites, joining each to its craft's
definition:
value <= the craft's MaximumVelocity 1355 / 1360 = 99.6% (5 fail)
value <= the craft's CruisingVelocity 1042 / 1360 = 76.6% (318 fail)
The test discriminates -- the cruise bound breaks 318 times, the hull maximum
5 -- so the script sets a COMMANDED SPEED, free to exceed the cruise default
and bounded by what the hull can do.
The turret anomaly that stopped me naming this two iterations ago was my own
artefact. UN_e007_ADAN_Turret's definition carries MaximumVelocity 500 and
CruisingVelocity 280: the data models turrets as if mobile, so a script value
of 400 is legal and simply never manifests. I had assumed turrets have no
velocity fields and treated 13% of the traffic as a refutation.
Recorded as unsettled: the five overshoots are UN_e106_ADAN_Destroyer 200 vs
a 150 maximum (x2) and UN_e011_ADAN_Attacker_B_HF/_Wayne 500 vs 450 (x3).
Designer overrides or an engine clamp; not established.
Named set_group_speed. Default = the slowest member's CruisingVelocity;
mode 1 restores it, mode 3 sets it, mode 2 hands it a global constant.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
77, 78, 81, 82 and 135 were unnamed. The engine has five contiguous strings
-- MISSION_START_PRT at 0x820A83F0, then _END_, _UPDATE_, _FAILED_,
_RESTART_ -- and five sequential ScriptPhase fields at +388/+392/+396/+400/
+404, stored in ascending order by one constructor region. Five names, five
fields, five unnamed built-ins.
Which is which is decided by call-site structure, measured over all 28
stages, and it is exact:
39 MARK_LAST_PHASE 89 sites -> 82 in 89/89
82 banner_mission_failed <- 39 in 89/89, then wait_s 89/89
40 mark_not_last 50 sites -> 78 (27) + 81 (17) + END_PHASE (6) = 50
78 banner_mission_complete <- 40 in 27/27
81 banner_objective_update <- 40 in 17/17
77 banner_mission_start 22 sites in 22 stages, one per stage,
after play_bgm
135 banner_mission_restart 16 sites, after play_bgm, phase >= 2
39 -> 82 is a perfect pairing and 40's sites partition exactly three ways.
Stated as inferred rather than read: the string-to-field pairing itself comes
from both sequences ascending in the same order; my operand tracker did not
catch the string loads in that constructor. The ROLES above do not depend on
it.
76 is left unnamed on purpose. It has 38 sites = 22 + 16, exactly 77's count
plus 135's, and precedes them; its body sets [phase+332] = 1 and nothing in
the image reads that field. Suggestive arithmetic is not a name.
Flagged as a consequence: MARK_LAST_PHASE is followed by the FAILED banner in
89 of 89 sites and mark_not_last by END or UPDATE. So [phase+300] = 2 reads
less like "this is the last phase" than "end the mission now,
unsuccessfully" -- the existing names for that pair may be mis-framing it.
Artifact regenerated: docs/re/data/isl-stage02.txt.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Verified rather than adopted: a subagent proposed that every even operand
slot is a type tag. Measured, the strong form is false and a precise form is
true.
TRUE: a SYMBOL operand is two words, a tag holding the constant 1 followed
by the index. Slot 0 is the integer 1 in 19899/19899 calls whose slot 4 is a
unit; slot 8 is tag-shaped in 100% of calls for every built-in taking a
second unit; slot 16 is 1 in 152/152 for built-in 128, the only one taking a
third. The 24 built-ins whose slot 0 is NOT the constant are exactly those
taking no symbol there. This explains the unit slots 4/12/20 rather than
replacing them.
FALSE as stated: slot 8 is a bare double for built-ins 4, 20, 24, 26, 28,
29, 90, 106 and 127, and built-in 75 carries five bare indices at 0/4/8/12/16
with no tags at all. Each built-in has a fixed signature and is 100%
self-consistent; none of the 34 with >=20 sites mixes the two.
Symbol table 1 has three types -- 1 routes (1362), 6 messages (2247), 7
effects (81) -- and its operand slots are type-pure, measured the same way.
Resolving them makes listings say what the script means:
`request_script_message(MSG_VOICE_D_257, ...)`, a fourth independent
confirmation of that name. Slots 24@4, 46@12 and 114@4 resolve 100% but MIX
types 6 and 1, so they are left unresolved rather than guessed.
Two more names withdrawn, neither replaced:
* 88 `camera_at` -- ZERO call sites in all 28 stages; never testable.
* 90 `camera_at_route` -- 8 sites, all Stage 02 phase 3, first operand is
symtab-1 type 7 `eff_n0071`, an EFFECT name, in 8/8, with a per-missile
Route_ADT301..308_p3M at slot 20. Not aimed at a camera.
Left unnamed on purpose: replacing a guessed name with another guess is how
the three names corrected earlier today went wrong.
Also flagged: 115 `named_event`'s only symbol operand is an eff_* name in
84/84 sites, so that name is suspect too. Not renamed pending a handler read.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Last iteration I said launch_mission died because skip_intro only tests for
the title on a static frame, gated at rmse <= 1500, and that run logged 1503
and 1549 just above the cut. I also said the fix was NOT to nudge 1500 but
to measure both signals through a boot first. Measured, and the diagnosis
does not survive.
boot_trace.sh logs the two signals skip_intro decides on -- frame-to-frame
RMSE and the is_title.py green-glyph count -- through a clean boot with no
presses at all. One run, 29 samples over 484 s:
8 samples had rmse <= 1500, so the gate OPENED eight times
0 samples had glyph > 0, so the title was never seen
At t=145 s the RMSE was 1205, comfortably inside the threshold, and the
glyph test was called and answered zero. A frame can be perfectly static
without being the title -- the intro movie has long quiet stretches, three
reading RMSE exactly 0. So 1503/1549 were almost certainly movie frames too,
and raising the constant would have admitted two more of them.
What is left is narrower and honest: the interactive title never appeared,
rather than appearing and being missed by a threshold.
The limitation is recorded rather than buried: the tracer intended 1 s
sampling and achieved 16.9 s, because each iteration forks two screenshots,
ImageMagick compare and a fresh Python. So this does NOT prove the title
never appeared -- only that it was absent from 29 samples. A window shorter
than ~17 s falls between them. The recorded next step is to make the tracer
sample at the rate it claims before concluding anything stronger.
Artifact: docs/re/captures/boot-signal-trace.tsv.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Third independent line for yesterday's built-in 100 rename, from the callee
this time. sub_8226E3B8 was labelled "push", which is what made built-in 100
look like push_trigger. It reads the element count, returns immediately when
the container is EMPTY, and otherwise walks the node list splicing nodes out
until it is empty. A push links one node; this unlinks all of them. It is
clear(). The append is sub_8226E160, reached from built-ins 19 and 25.
So the rename now rests on the handler, the usage (all 12 Stage 02 sites sit
in the phase terminator next to timer_stop / clear_flag(-1) /
MARK_LAST_PHASE), and the callee.
The dynamic half did NOT run, and the write-up says so. phase_watch.py now
samples [phase+272+20] (triggers queued) and [phase+216+8] (coroutines
alive) so a phase terminator's effect on the VM is visible in one line —
written here, never yet exercised against a live guest.
Boot-nav could not reach the title in 381 s. Diagnosed rather than retried:
skip_intro.sh only runs the title test on a static frame, gated at
rmse <= 1500, and this run measured 1503 at 104 s and 1549 at 139 s — just
above the cut — so is_title.py was never called and the one allowed press
was never spent. Recorded in BACKLOG with the explicit instruction NOT to
raise the constant: the first step is to log rmse and the glyph count
through a whole boot and look at the two distributions, because tuning a
threshold to make one run pass is fitting to a single sample.
Also reaped a stale lock: a gdb orphaned 2h14m earlier was holding
/tmp/xenia-canary.lock with an already-defunct emulator child.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
All re-read twice — the handler, and the thing it calls — because each had
been named from its shape rather than its effect.
* id 11 `yield` -> `end_coroutine`. 0x82272624 is li r11,1 ; li r3,3 ;
stw r11,164(r31), and the dispatcher's r3==3 arm erases the thread from
the active list and returns it to the free list. It destroys the thread.
2945 sites game-wide, 372 in Stage 02 — the most-used built-in there was.
* id 5 `await_label` -> `kill_coroutine(label)`. sub_82273B08 kills the
thread parked at the target pc, or itself if the target is its own pc.
It waits for nothing.
* id 100 `push_trigger` -> `reset_phase_threads`. It clears the trigger
container and then frees every thread whose pc differs from the caller's
— the opposite of pushing a trigger. Corroborated by usage: its 12 Stage
02 sites all sit in the phase terminator, next to timer_stop,
clear_flag(-1) and MARK_LAST_PHASE.
One name recovered from the game's own text: opcode 992 prints
"RequestScriptMessage %s" at 0x820A5700, so id 64 is request_script_message
(2683 sites).
Return codes documented properly: 1 = restart the coroutine from its entry
(previously not recorded at all), 3 = terminate. And the blocking set was
wrong in two places — it is 102, 120, 137, 142, 143. Id 97 does NOT block;
its handler ends `b 0x822724F8`, so it always returns 0.
Unit-operand resolution settled from DATA over all 28 stages rather than by
reading 147 handlers: a slot qualifies only if every value is a valid
symtab-2 index, it takes >=15 distinct values, AND its maximum reaches most
of the table — that last clause is what discriminates, since every small
integer is trivially "in range". 31 built-ins at slot 4, 8 at slot 12, one
at slot 20. It also refutes set_flag's slot 0, whose maximum overruns the
table, and the resolver now declines rather than inventing a name.
New and unexplained: symtab-2 holds two types, 2 and 8, and built-ins 95 and
128 take type 8 at slot 12 in 100% of their sites.
A downstream inference is withdrawn with it: the note reading the live
trigger counter attributed it to "the script arming watches as it goes" via
built-in 100. The measurement stands; the attribution does not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Settled from the disassembly, no run needed. Built-in 69's tail maps the
lifecycle lookup into [phase+164]: handle == 0 takes the early exit at
0x8226AF44 and returns 0, while the destroyed states return 2, 3 or 4 (and two
of those also normalise the record's +16). Those are different values.
So poking rec+4 = 0 made the predicate report ABSENT -- the same answer an
undeployed unit gives -- and never the answer the script branches on. The
condition was polling at 5 Hz throughout and correctly saw 'not here'.
Both null results are now fully explained, and neither was evidence about the
condition: the first poke wrote a field nobody reads, the second wrote the wrong
value into the right field. Simulating a kill needs the handle to stay valid
while the lifecycle lookup returns 3/4/5, i.e. the write belongs in whatever
sub_82301240 reads, not in the script's own record.
Also names built-ins 46/47/48 as squadron_trace / squadron_attack /
squadron_escort in isl.py.
sub_822FE040 is a fully unrolled registration: 1023 slots filled with a default,
then 57 explicit writes, of which 48 are real handlers and nine are a shared
accept-and-discard stub. Return convention is nonzero = consumed, 0 = retry,
which is how the interpreter waits for a named unit to exist.
Opcode 995 is the ONLY handler touching the phase mirror [*(0x828F35F8)+236] --
the sole read and sole write in the table -- independently confirming why polling
that mirror saw nothing during phase 1. And no handler spawns or despawns a unit:
256 is the strongest deploy candidate but is unconfirmed because the message ids
are write-only in this image.
WITHDRAWN, verified wrong: I had recorded the writes to '+20' in sub_8226E7D8 /
sub_8226E930 as block initialisations by container constructors. At
0x8226E86C-0x8226E8E0 they do li r3,28 / bl 0x8230C160 then
lis r10,0xAB03 / ori r7,r10,0xE4BA / stw r7,4(r3): they build an INTERPRETER
COMMAND RECORD for opcode 996 and push it, i.e. AddSelector and RemoveSelector,
with a 32-entry cap. The stw to 20(r3) is the command record's +20, a different
object. Wrong twice: not constructors, and not that container.
Also flags that sub_8230C398 -- gated on *(0x82899CE0) == 16 at both call sites
-- looks like Stage 16's script compiled in C++, which 'debug defaults' does not
survive given the .ssb loader explicitly refuses mission 16.
The watchpoint plus Canary's source settle it. At the write, the guest context
(rsi, per x64_emitter.cc:881) holds 0x8226EAE0, inside sub_8226EAB8. That
function is a generic list-node insert: it reads the count at 8(r30), guards
against 0x3FFFFFFF overflow, does addi r11,r11,1 / stw r11,8(r30), then links the
node. It has 16 callers, so it is a shared container helper.
That explains why two static searches missed it. The trigger container at
phase+272 EMBEDS an inner list object at +12 -- which is why the push does
'addi r31, r30, 12' -- and the inner object keeps its count at its own +8. So
272 + 12 + 8 = 292 = the watched word, and the instruction is stw r11, 8(r30)
with r30 = phase+284. Searching for 'stw rN, 20(rM)' could never have found it.
Also resolves the earlier open item on +12: it is the embedded list object, not
a list head pointer, which is why it read 0x000A0009 instead of an address.
Method note kept: the static hunt assumed the field's offset in the OUTER object
would appear in the writing instruction. A watchpoint is indifferent to the
addressing form, which is why it was the right tool after two failed offset
searches.
ssb_watch.py polls the .ssb header, a symbol string and find_mission from the
moment flight is detected. On a normal run all three are present at the FIRST
sample (t=0.0, mission 0xBC79C960), so 'the script loads later than assumed' and
'the probe raced a load' are both out -- there is no window in which a healthy
mission flies without its script in memory.
A detail worth more than it looks: at that sample screen_id reported 'other',
not 'flight'. The script is loaded and the ScriptMission locatable BEFORE the
flight HUD appears, so residency is not gated on the HUD, and a run showing the
HUD without the script is in a state a healthy run never passes through.
That leaves the third candidate: the anomalous run's mission never loaded and
its IN FLIGHT was a misdetection -- consistent with it being frozen on a black
screen when checked afterwards. Not proven, but now the only surviving
explanation rather than one of three.
From Canary's own source (x64_emitter.cc:881) GetContextReg() returns rsi, so at
any JIT instruction %rsi is the PPCContext* -- which is also why the faulting
instruction read 0x110(%rsi), a guest register load. That is the way past the
watchpoint's ceiling: the guest register file is available at the write, and a
0x82xxxxxx word picked out of it resolves against sylpheed.db to name the caller.
trigger_watch.sh now dumps x/128wx instead of a useless host backtrace.
The re-run then failed for an unrelated and unexplained reason: it reached
flight, the pilot bound, the guest was animating, and find_mission returned
NOTFOUND. Narrowed: the .ssb header is absent from guest memory (0 hits where
earlier runs hit immediately), ADN110 is absent too, but the manifest string
'Stage02.ssb' IS present at 0xBDA6C50B. So memory is readable and the manifest
is loaded while the script is not, in a mission that is flying.
No explanation offered. The cheap discriminator for next time is to poll for the
header from the moment flight starts and record when it appears, instead of
sampling once.
All 28 StageNN.ssb decoded: 2,085,628 bytes, 25,705 call sites, 108 of 147
built-ins used. Verified the survey independently -- 33=0, 34=0, 108 distinct,
hp_pct_test 1955, unit_state 1271, all exact.
global_counter0/1 have ZERO call sites in all 28 stages. The handlers exist and
are wired; no mission calls them. So 'does a wave start after N kills?' is
answered for the whole game, not just Stage 02: no mission counts kills.
Two idioms change how the counts read. hp_pct_test(unit, 0.0) IS a destruction
test -- 1786 of 1955 calls (91%) pass 0.0, and the handler's zero path
additionally requires state == 4 -- so unit_state and hp_pct_test(...,0) are
interchangeable and stages just pick one. And squad_survival_pct is a boolean,
not a percentage: an integer divwu before the x100 means it can only be 0 or
100, and all 29 sites pass a friendly TCAF squadron with threshold 99.9, making
it 'has this escort lost anybody'. The game has no destroy-N%-of-a-squadron
objective.
Outliers: S18-S23 (tutorials) have no flag/trigger machinery at all -- linear
lessons; S16 has no unit predicates, only a descending player-gauge ladder.
Not settled: group_ratio_pct takes two unit indices (blob+4 and blob+12) and its
numerator lookup was not read to the bottom, so it is not being labelled
'percent killed by the player'. isl.py's UNIT_ARG omitted 71/72.
All 25 opcodes now have meanings. Ops 2/4/6/8 are integer compound assignment
(+= -= *= /=) and 3/5/7/9 the float versions; 10 and 11 are integer and float
compare writing three condition bits; 13-18 are je/jne/jl/jle/jg/jge; 21-24 are
push.i/push.f/pop.i/pop.f over deques at phase+44 and phase+64.
The shared-handler question is answered: the dispatcher leaves the opcode in r4
and the shared thunks never overwrite it, so those helpers take an extra opcode
argument and index a secondary table (0x82271448, 0x8227152C).
CORRECTION to my own tool and note: the branch/jump base is [phase+232], which
the phase initialiser sets to 0x24 + the phase's entry from the mission-level
stream -- 0xE4 / 0x14AA8 / 0x24B4C for Stage 02's three phases, not the file's
0x24. Measured on phase 1: base 0xE4 puts 525 of 525 branch targets on an
instruction boundary; base 0x24 manages 188. isl.py had been using 0x24 for
every phase, so its jump targets were wrong throughout. Fixed via
isl.phase_bases().
That also settles two things mission-script-ssb.md left open: offsets ARE
code-base-relative, and 0x1883's operand IS a code pointer -- the earlier worry
that some 'land on IEEE floats' was an artefact of adding the wrong base.
Hammering settles what a single write could not: hull 0x44BB8000 (1500.0f),
944,387 writes of 1 over 15s, and afterwards the value HELD at 1 -- the game
stopped rewriting it. The screen left 'flight', the HUD is gone, the ship is
burning, and Natalie radios 'I've lost contact with Rhino 3!', the player's own
callsign. The game read the poked value and killed the player.
Established: writes to /dev/shm reach the running guest; hull at pos+0x154 is
authoritative, not a readout; and a single write loses a race against the game's
own continuous writes.
This upgrades two earlier results from inconclusive to genuine negatives. The
unit-record pokes were downgraded because I could not tell 'ignored' from 'never
arrived'. The write arrives -- and those pokes persisted untouched for 60s, so
the game genuinely saw state=4 and handle=0 on all three objective squadrons and
did nothing. That is real evidence the phase-1 condition coroutine is not polling
and its checks run only when a trigger starts them.
Withdrawn: last iteration's claim that the pilot's hull= is a different field or
scale. I read 1000.0f at pos+0x154 and inferred a mismatch with the logged 1500;
this run reads 1500.0f at the same offset. Same field, different value per run.
poke_control.sh (self-retrying, succeeded on attempt 1 with no freeze) set the
player's hull at pos+0x154 to 1. Twelve seconds later the game had put
0x447A0000 back.
That establishes an asymmetry worth having: the hull word is continuously
rewritten by the game, while the unit-record fields held our value untouched for
60s. It separates fields the game maintains from fields nobody writes.
But it is not yet a control. The after-frame shows a red WARNING banner -- and
the before-frame already shows MISSILE ALERT, so the ship was under attack in
both and the banner is not attributable to the poke. A value being overwritten
proves the game writes that address, not that it read ours.
Correction: hull at pos+0x154 is 0x447A0000, a FLOAT 1000.0, not the 1500 the
pilot logs -- those are different fields or scales and should not be conflated.
The settling refinement: poke in a tight loop for several seconds so the value is
low whenever the game samples it. If hull is authoritative the ship dies and the
screen goes to GAME OVER; if it survives, the field is a readout.
The reusable part is the harness: boot -> verify animating -> locate -> act, with
a freeze at any step costing one retry rather than the iteration.
Ran the direct test instead of a seventh attempt at winning. All three objective
squadrons were live (state 2); the write to +16 sticks, and 60s later with all
three reading 4 -- the value a naturally-destroyed squadron takes, measured on
ADN111 -- [ScriptPhase+196] is still 0 and the ordinal still 1.
So 'phase 1 clears when ADN110/111/112 are destroyed' is not confirmed and its
simplest form is refuted. The bytecode reading (three unit_state polls then
set_flag(8)) stands; what does not follow is that flipping the field equals the
kill.
The persistence is the clue: built-in 69 normalises +16 when it polls, so a
running condition coroutine should have overwritten the poke within a frame. It
did not, which points at the condition being evaluated only when a trigger fires.
Also corrects the per-unit record layout: +4 is 26/27/28 for the three
squadrons -- small consecutive integers, NOT the 'live object pointer' the
built-in summary describes (an undeployed squadron has +4=0). +20 = 9 is exactly
their member count n from the roster, so the record is per-squadron and carries
its strength. My own probe printed 'obj=yes' by testing that word for non-zero
rather than pointer-ness, which made an index look like an object.
Bounding the pointer scan to 0xBC000000-0xBD000000 (with a full-sweep fallback)
drops find_mission from a ~371MB walk to 0.7s. The run then went 694s with the
probe attached and NO freeze, against 3-of-3 frozen inside ~4 minutes with the
unbounded version. n=1, but the first probe-attached run to survive.
State encoding pinned to three points: 1 = not yet deployed, 2 = active,
4 = destroyed. ADN111 caught going 2 -> 4 at 433s while the active count fell
36 -> 27.
The phase ended at 694.9s WITHOUT the ordinal advancing, and every field matches
the branch read statically from sub_82260710: [phase+300]=2 (last-phase flag),
[mission+20]=0 (mission-over state), [phase+196]=1 (finished), [mission+40]=1
(unchanged). The static state machine is confirmed on the live oracle for the
mission-over half.
But this was a LOSS, not a clear: GAME OVER on screen, escort at 35.7%, pilot
DEAD at 676s, and two of the three objective squadrons still at state 2. So the
'destroy all three clears phase 1' prediction remains untested. What is
established is that the else-branch is the only route to phase 2 and needs
[phase+300] != 2 when the phase ends.
Five attempts, still no phase advance observed -- the obstacle is now keeping the
escort alive, not the freeze or the instrument.
A pilot-only run froze at t~150.7s (frozen.py: max_pixel_delta=0), found by
accident when the sweep-free experiment aborted at startup and the run flew with
no script probe attached. The tally is now 3-of-3 frozen with the probe versus
1-of-3 without, not 3-versus-0. Still a lean, but not the clean separation the
previous entry claimed, and marked down accordingly -- the fourth time a freeze
conclusion here has had to be softened by one more run.
The sweep-free test itself could not run: ScriptMission is re-allocated per run,
so the address from an earlier run (0xBC7A2A20) read back all zeros. The cheap
self-consistency check ([m+44] must equal the phase's [+244]) rejected it
instead of reporting garbage, which is the part that worked.
Names the cheaper replacement: a BOUNDED pointer scan. Every ScriptMission seen
so far sits in 0xBC79xxxx-0xBC7Axxxx, so ~32MB instead of ~371MB would cut the
sweep cost roughly tenfold. Not yet implemented.
Watching [ScriptMission+40] and the three phase-1 objective squadrons together:
all of ADN110/111/112 flip state 1 -> 2 at ~143s, while records in state 2 climb
24 -> 35 over four minutes.
So state 1 means 'not yet deployed' for these, not 'gone'. The built-in table's
'1/3/4 = gone/dead/invalid' shorthand is incomplete, and reading state != 2 as
destroyed would have been wrong exactly as flagged last iteration.
This also answers a much older question: mission-arrival-watch.md and the wave
work recorded '0 confirmed arrivals' across many runs by watching the CRAFT
population. The script's own unit table shows arrivals plainly -- eleven records
enter state 2 within four minutes. The old negative measured the wrong
structure; craft counts conflate deployment with attrition, the per-unit state
field does not.
Both attempts froze (at ~70s and ~253s), so no phase advance was reached. The
freeze witness caught both immediately, which is why the truncation is visible
instead of a silently flat line.
New harness tools/re-capture/phase_watch.py.
Unblocks the phase experiment, which was stuck because '38 enemies died' could
not say whether the right ones did. Chasing craft->squadron was the wrong angle:
the script VM keeps that table itself, indexed by the .ssb symbol-table-2 index.
Route: find the .ssb header in guest memory (0xAB840010 for a Stage 02 run),
code base = filebase + 0x24, scan for a word equal to it, then VALIDATE
arithmetically -- [ScriptMission+44] must equal filebase + symtab1 offset + 4.
Measured 0xAB874C94, predicted 0xAB874C94, exact. A second candidate that also
pointed at the code base failed that check and was discarded; without it either
would have looked plausible.
ScriptPhase+324 -> +4 is an array of 122 per-unit records -- exactly the size of
Stage 02's symbol table 2, an independent confirmation of the index space.
[ScriptMission+40] reads 1 in a phase-1 mission. The mirror at
[*(0x828F35F8)+236] that three earlier runs polled reads 0, because ChangePhase
only posts once the ordinal exceeds 1 -- so +40 is the real counter and is
reachable from /dev/shm with no debugger.
Flagged rather than asserted: the three objective squadrons read state=1 with a
LIVE object pointer in a mission where nothing has been shot, which does not fit
the built-in table's '1 = gone'. Reading state != 2 as destroyed would be a
plausible-but-wrong inference; the encoding needs pinning first.
New tool tools/re-capture/squadron_state.py, verified end to end against the
manual reading.
SYLPH_PREFER=Turret, escort at 100%, 1070s, phase field 0 throughout, 4029
Turret-targeting pilot samples. It looked like a clean negative -- 'hunting the
objective squadrons does not advance the phase' -- and it is worthless:
frozen.py reports max_pixel_delta=0, and the pilot's first sample at t=0.0
already has spd=0 with the same yaw/pitch/target/d=7186 it still had at 1070s.
The mission froze on entry to flight and nothing was ever shot, while screen_id
said 'flight' the whole time.
I was one step from writing this up as a fact about the game; running the freeze
test rather than trusting a plausible log is what caught it.
phase_probe.py now calls frozen.py every 60s and prints a GUEST FROZEN banner
inline, so a dead-world reading is labelled in the data instead of discovered
later. Note frozen.frozen() returns a TUPLE (is_frozen, max_delta) -- testing it
directly is always truthy and would have made the witness fire on every check.
Verified against the frozen guest.
Also recorded: the pilot log is itself a freeze witness -- identical
yaw/pitch/target across thousands of samples is a dead world, not patience.
Resolving symbol-table-2 indices turns the bytecode into mission logic. At
0xF524 Stage02.ssb polls unit_state on ADN110, ADN111 and ADN112, updates each
one's objective marker, then latches set_flag(8) -- exactly the
trigger/predicate/set_flag/END_PHASE shape predicted from the disassembly, now
observed in the mission's own code with names the roster tables already gave.
The 12 END_PHASE sites are outro sequences (wait_cmds_drained / fade_sound(3) /
builtin85(3) / wait_s(3) / END_PHASE / yield) -- the terminator, not the
decision.
Fixes a decode bug that hid every argument: the tracker only followed
local[i] = special[0], but the common form is an immediate written straight into
local[i] (k=01,03), so every unit predicate printed with NO arguments. The
disassembly looked complete while being empty exactly where it mattered.
Also records the live probe result: the phase mirror at [*(0x828F35F8)+236]
stayed 0 for ~530s of actively-hunting flight, no advance observed -- which is
what the static analysis predicts for phase 1, since ChangePhase only posts once
the ordinal exceeds 1.
Table at 0x8227226C is 147 big-endian absolute VAs (verified structurally: it
ends exactly where the first handler begins, all targets inside sub_82272220).
Arguments are not in the instruction -- every handler does c_str() on
[phase+20], a packed blob, which is what the local[] staging fills. Return 2 =
yield; five built-ins block by skipping the pc advance.
Recovered the ScriptPhase state layout: 32-entry float and flag register files,
int/double result registers, the timer block, and the runtime unit array at
+324 indexed by symbol-table-2 index -- a direct hook from bytecode call sites
to the two .ssb symbol tables.
Spot-checked two claims against the disassembly rather than trusting them: id 4
loads a DOUBLE into the thread countdown and returns 2 (wait_s), and id 24 reads
current/initial squadron member counts (squad_survival_pct). Both exact.
Counting Stage02.ssb: unit_state 255, hp_pct_test 167, dist_lt 92, unit_alive
71, unit_relation 52 -- and squad_survival_pct, group_ratio_pct and the two
global counters are NOT called at all. So Stage 02's phases are gated on named
units (destroyed / HP / proximity), never on an aggregate count, even though the
kill-counter primitives exist in the VM. That answers the standing 'next wave
after N kills or after an event?' question for this stage: specific units, not a
number.
isl.py now names the built-ins, so the run-up to the first END_PHASE reads
wait_cmds_drained / fade_sound(3) / builtin85(3) / wait_s(3) / END_PHASE.
Not settled: 3 handlers unresolved (55, 75, 105); the 1024-slot interpreter
command table is only partly recovered.
Resolver table 0x82271D74 gives four kinds: 0 global[i], 1 immediate,
2 special[i] ([phase+164]/[phase+168]), 3 local[i] ([phase+20+i]). Byte[0] is
the rvalue kind, byte[1] the lvalue kind, so the recurring instruction pair is
argument staging -- values land in local[] at offsets 0,4,8,0xC and the next
call consumes them. A built-in's arguments are not in its own instruction.
Fixed a decode that would have been believed: immediates in set.f are DOUBLES
carried as two words (op 1 stores with stfd). Reading the high word as a float
gives 2.125 where the script means 3.0.
isl.py now tracks staging and prints call arguments, so the run-up to the first
END PHASE in Stage02 reads as builtin=64(0x42,2,1,9,1,-1) / 120 / 59(3) / 85(3)
/ 4(3) / 6. Three built-ins taking 3 just before the phase ends look like a
wait-seconds family -- flagged as unconfirmed until the built-in table is read.
Read the encoding off the interpreter rather than guessing: instruction is a
big-endian u32 whose LOW byte is the opcode (25 of them, table 0x822635FC),
byte[2] is the instruction length -- every handler advances the pc by it -- and
bytes[0..1] are operand kinds. Op 12 is a jump whose operand is relative to the
code base [phase+232], which settles that offsets are code-base-relative for
this opcode. Op 19 is the built-in call: id in word@+4, and word@+8 is a
monotonically increasing STATEMENT id (0x245, 0x248, 0x24A, ...).
Confirmed by disassembling Stage02.ssb: the stream decodes cleanly from the code
base and routines terminate on ret exactly where expected.
Scanning the code region on the call encoding: 2846 call sites, 73 of the 147
built-ins used. The phase-control ones are located -- built-in 6 (end phase) at
12 sites, 62 at 3, 39 (mark last phase) at 8 -- so a phase has several exit
paths, as a mission with win and lose branches should.
New tool tools/re-capture/isl.py with --calls and --to (resync-into-target,
needed because instructions are variable-length so you cannot walk backwards).
Not settled: the 147 built-ins are uncharacterised, so this is structure without
meaning -- we can see THAT a phase ends, not WHAT was tested.
Found the routines in the disassembly DB rather than guessing from data:
sub_82447DF0 IDXD tag hash (lbz+extsb, modulus 0x00FFFFDF, magic 0x2101)
sub_82447E70 IXUD tag hash (lhz, 64-bit, modulus 0xFFFFFF67 then 0x00FFFFDF)
Both transcribed instruction-for-instruction into Python and Rust.
IXUD SOLVED. It defeated every single-modulus search because it chains TWO
exact moduli -- the loop reduces mod 2^32-153 in 64-bit arithmetic and only the
result is folded mod 2^24-33. A polynomial mod M1 folded through M2 is not a
polynomial mod anything, which is exactly why the gcd test returned 1. Verified
independently: 86/86 record keys and 108,261/108,261 field tags in
GP_MAIN_GAME_E.pak, and NoRecord -> 0x1c6d9c96.
CORRECTION 1: tag_hash must SIGN-EXTEND each byte (extsb). My reconstruction
used unsigned bytes and matched all 1.27M disc names -- every one is ASCII --
while disagreeing on ~90% of random inputs with a byte >= 0x80 (verified:
18096/20000). The disc could never have caught this; only the disassembly did.
CORRECTION 2: name_hash's reduction is EXACT, not lossy. The module doc claimed
the missing conditional subtract made it something other than %. rlwinm r6,r6,
9,23,31 is just hi>>23, and with RECIP = floor(2^55/M)+1 that is Granlund-
Montgomery magic division -- 0 wrong at every quotient boundary across the full
32-bit domain. Retracted.
cargo test -p sylpheed-formats --lib hash: 10/10.
Closes the 4-byte record key. tag_hash is name_hash's shape -- byte-sum
checksum in the top byte over a 24-bit modular polynomial -- with two different
constants: modulus 0x00FFFFDF (2^24-33, prime) instead of 0x00FFF9D7, and no
lowercasing, so tags are case-sensitive. name_hash explains 0 of 8643.
Recovered from the tables rather than the executable: every inline field name
is a known (name -> tag) pair, and comparing names differing in one character
gives the per-position weights 1, 0x100, 0x10000, 0x21, 0x2100, ... -- a byte
leaving bit 24 re-enters as 33, i.e. reduction mod 2^24-33. Holds where it is
easy to get wrong (distance 8 and 9 carry correctly).
A record's key is the tag of its own name: FormationSet rosters 362/362,
UnitGroup rosters 281/281, S02 squadron names 111/111 -- so records can be
addressed by name without reading the roster first.
Implemented in Python (unitgroup.tag_hash) and Rust
(sylpheed_formats::hash::tag_hash) with 3 new unit tests carrying disc-derived
vectors; cargo test -p sylpheed-formats --lib hash is 8/8 green.
Not settled: the guest routine is unlocated, so this uses exact modular
arithmetic where the game may use a Barrett step without final fixup.
FormationSet_S<NN>.tbl records are slot lists -- 1 + 8*FrameCount fields,
exactly. Resolving every squadron's FormationID and comparing gives
sum(n) <= FrameCount holding 1159/1160 across all 28 stages, 0 unresolved, with
539 filling the formation exactly. The single violation is a debug leftover
(S20, AI_Test / MessageSet_test, Formation_1_only with n=2) and is recorded.
The old 'n is not the _NN suffix of FormationID' observation was right but drew
the wrong conclusion: the suffix IS FrameCount, so n=9 against _30 just means 9
units in 9 of 30 slots.
Also: FormationID does not hash into its table (0/16). FormationSet carries a
name roster record -- no FrameCount, fields are (tag, name, '') with the tags
being the record keys -- the same convention as Enumerate_Squadrons. Second
occurrence of 'keys are resolved by an in-table roster, not by hashing'.
Does not close the 387-vs-300 gap, and the key derivation stays open.
Caught the freeze by waiting for the event (frozen.py + in_flight) instead of
sleeping a guessed interval; freeze_waitobj.sh splits into boot/watch so the
wait is not capped by one Bash call. Verified hard: a frame minutes later is
byte-identical to the capture.
Healthy vs frozen, same run: 20 -> 24 wait frames, XEvent 19 -> 23,
XSemaphore 8 -> 7. The signature is per-thread -- 17 of 24 threads sit on the
exact object they were on, four previously-running threads park, and T74/T75
move off a semaphore onto an event. So the freeze is not a whole-emulator stall.
Also corrects the previous entry's test: screen_id reads 'flight' during a
freeze by design, which is why frozen.py exists. Re-testing the saved frames
says that run was genuinely healthy, but it was right by luck.
heavy_read.py added to test whether the instrument provokes the freeze: I/O is
free (371 MB in 0.1s, page cache), the cost is Python-level CPU. One data point
-- 670s clean, then frozen 54s after the inducer started -- recorded as n=1, not
as causation.
23 wait frames, 30 objects, nothing unresolved -- the second deref turns every
former miss into a resolved object, as predicted. XEvent 20 / XSemaphore 9 /
XTimer 1; every WaitMultiple thread waits on a pair, and 78/79/80 and 64/65 are
worker groups sharing a handle.
%ebp does not survive as the count -- WaitMultiple reuses it at 8fc158 -- so the
array is bounded by reading until an entry stops resolving instead.
The frozen capture is still not taken: screen_id reads 'flight' at the second
capture and out to ~470s, so the mission never black-screened. The diff in the
data file is two healthy captures and is recorded as such.
The 8 threads whose [rbx] did not resolve to a vtable were never in
XObject::Wait. The backtrace grep matched WaitMultiple as a substring, and
there %rbx is the XObject** array (mov %rsi,%rbx) with the count in %ebp, so
[rbx] is objects[0] -- an object pointer, needing a second deref -- not a
vtable. The unwind restored rbx correctly for all 18.
freeze_waitobj.sh now takes the function and frame index from the backtrace and
applies the matching read, and captures twice in one run (healthy and after the
~270s black-screen) so the comparison is within-run. waitobj_report.py tabulates
both and diffs them, discarding any value info symbol cannot resolve.
Re-extracting the same gdb capture per thread rather than by grep qualifies the
previous entry. Of eighteen threads whose frame 3 is XObject::Wait, eight have
[rbx] equal to the XEvent vtable plus sixteen, two equal to the XSemaphore
vtable plus sixteen, and eight hold a pointer into the mmap region that is not a
vtable at all. So the waits are on two distinct kernel types, XEvent and
XSemaphore, and the earlier claim that the object is an XEvent was right for the
majority but not the whole picture.
The eight non-vtable readings are the method checking itself rather than a
failure. A polymorphic object's first word is always a vtable pointer, so those
values are simply not this: rbx could not be restored for those frames and the
unwind returned whatever the register held. A reading counts only if [rbx] lands
in the binary's vtable range and resolves to a "vtable for" symbol; anything
else is discarded rather than interpreted. Ten of eighteen resolve and the rest
are honestly unknown.
That also settles the previous entry's worry that the 0x7ffc and 0x7ffd
addresses looked like stack. They are the shared mmap region, which holds thread
stacks and large allocations alike, so the vtable check rather than the address
range is what separates an object from a stack slot.
The follow-up run that would have added /proc/<pid>/maps classification and a
wider object dump never booted -- EMULATOR GONE at 0s, skip_intro exit 4 --
most likely a stale emulator or lockfile from the preceding gdb session, whose
process tree is parented differently and escaped the usual cleanup. So the map
classification, the multi-word object dump and the frozen-state capture are all
still unrun.
Executed the wait-object read on a live run under gdb. For every thread whose
frame 3 is XObject::Wait, rbx gives a pointer whose first quadword is
0x5555562db8f0, which is the PIE base plus 0xd878f0, and the symbol table has
vtable for xe::kernel::XEvent at 0xd878e0. The stored pointer is the vtable
symbol plus sixteen, since offset-to-top and RTTI come first, so it matches
exactly. The same vtable appears on every sampled waiting thread while the this
pointers differ, meaning many threads waiting on different XEvent instances. The
whole chain needs no DWARF and no rebuild, as the static groundwork predicted.
Two caveats, both recorded rather than smoothed over.
This is a healthy-play snapshot, not the freeze. The capture landed at 195 s of
flight with screen_id reporting flight and a non-black mean, because boot under
gdb costs about 300 s and the entire experiment has to fit inside one call --
a timeout kills the process group and takes the emulator with it, which lost an
earlier attempt outright. So this describes what threads wait on during normal
play, which is the control the frozen capture never had, but it is not the
frozen case.
And the this addresses look like host stack rather than heap. Either xenia
places these objects somewhere unusual, or rbx at frame 3 is not Wait's this
after the unwind and the vtable match is coincidence. An exact plus-sixteen
match on a known symbol is hard to get by accident, but the address range is not
what was expected. The settling check is to dump a few words at rbx and see
whether they look like an XEvent -- vtable, KernelState pointer, handle and type
fields -- or like saved registers.
A two-segment OB watch made the mechanism visible. Segment 1 ran clean, with the
witness reporting zero stalled samples of fifty and OB steady at 4 for 250
seconds. Segment 2, attaching to the same still-alive emulator, got "NOT IN A
MISSION (no unit definitions)".
The emulator was alive; the mission was gone. The pilot's hull reads 1500 in
every line, so the player never died. Its log stops at t=267 s with consecutive
byte-identical lines, the freeze signature. And the screen is now entirely
black, mean (0,0,0). This is the pre-existing mission-end black-screen freeze
already recorded in the corpus, caught in the act. Segment 1's clean witness is
not a contradiction, since it sampled to about 250 s and the freeze came at 267.
That corrects an earlier claim. mission-arrival-watch.md records the blocker as
removed because the emulator survives between calls, which is true, but it was
stated as though observation could be extended indefinitely within a turn. The
emulator surviving is not the mission surviving. The binding limit is the game's
own black-screen at roughly four and a half minutes and no amount of chaining
crosses it. The earlier chained run that reached 435 s of cumulative
verified-live observation did so across two segments of one mission, which fits:
its second segment froze at t=135 s.
The consequence is worth stating plainly. Any experiment needing more than about
four and a half minutes of one continuous mission is not doable on this build,
chained or not, and that includes clearing sixteen marked fighters at two kills
per five minutes. The freeze is therefore now the highest-value target in its own
right, since fixing it would unblock the arrival question, the phase-advance
question and the OB series together. It is also a long-standing known defect
rather than anything this work introduced.
ob_watch.py verifies the address against a confidence-gated HUD reading before
reporting anything, and two consecutive fresh launches settle the question:
0xbdb59668 held 3165285888 against a HUD of 4 on one, and exactly 4 on the next.
The address is not stable across launches, the old note that it recurs in about
five runs of seven was right, and the gate did its job by refusing to report a
series from an address that did not describe that run. The watcher now hunts the
address on the current run when confirmation fails, using the same intersection
method, so it no longer depends on a lucky launch.
The HUD reader is also gated now. ob_read returns a best and second score per
digit and those were printed but never checked, which is how one misread
poisoned an intersection and produced a wrong refutation of big-endian u32. A
reading is accepted only if every digit scores at least 0.80 with a margin of at
least 0.05, the rule ob_read's own docstring states.
The measurement itself is a negative. With a clean witness, zero stalled samples
of fifty, OB held at 4 for 250 seconds while the pilot targeted e010 for 1964
ticks and fired on 1635 of them. Constant fire at the marked attackers and not
one decrement, so it destroyed none, which matches the roughly two marked kills
per five minutes measured earlier. The fire rate itself rose from 4.6 % of ticks
in an earlier diagnosis to 83 % here without producing more kills.
Recorded as unreproduced rather than explained away: the run that found the
address saw the counter rise 4, 8, 12 over five minutes, and that reading was
confirmed against the HUD. This run was flat over a comparable window. Both
observations are sound and they disagree, so the rise is not a stable property
of the mission's first five minutes and presumably depends on progress this run
never reached.
Ported onto the shared harness, the HUD changed from 4 to 8 and the intersection
collapsed in a single step: one u32be survivor at 0xbdb59668, with the u16be and
u8 hits at 0xbdb5966a and 0xbdb5966b being the low half and low byte of that same
word.
Verified live rather than asserted. Reading screenshot and memory together three
times, the one legible HUD frame showed 012 against mem@0xbdb59668 = 12; the
other two frames were unreadable rather than mismatched. This independently
rediscovers the address the earliest sessions found by digit-transition hunting,
by a completely different method.
It also corrects an earlier conclusion. A previous run intersected HUD readings
of 4 and then 11, got zero survivors in every encoding, and that was written up
as eliminating big-endian u32 for the whole region. This run shows u32be holds
the counter, so the refutation was wrong. The likely cause is the input:
ob_digits.png has templates for 0 1 2 4 8 only, so values containing other
digits are misread rather than rejected, and "11" was probably one of those. A
single bad reading poisons an intersection permanently, because it removes the
true address and nothing later can restore it. The lesson is that an
intersection method needs individually verifiable inputs -- the reader's
confidence scores were printed but never gated on.
One observation reopens the arrival question in a useful way: the counter
increases, 4 then 8 then 12 across about five minutes, measured in memory so not
a digit misread. A count of remaining marked targets that rises means targets
are being added during the mission. That does not contradict the deployment
finding, since the roster is fixed at load, but it does mean the game marks new
objective targets as the mission proceeds. Watching this one address across a
whole mission is now the obvious next experiment and costs almost nothing.
Four probes were written from a blank file and each re-learned the same lessons
by losing a run: that a flat run cannot be told from a frozen guest without a
stall witness, that results held to the end of a run are destroyed by a turn
timeout, that a roster count which is not the stage's member count means a
different stage loaded and must be discarded, and that a run's witness state has
to be read before its numbers. Writing each lesson down did not stop the next
probe repeating it, because each probe started from nothing.
probeharness.py makes them structural. Probe(baseline=N) discovers the roster,
rescans up to five times and refuses to start if the count never reaches the
baseline. The witness is calibrated on construction, sampled by tick() and
reported by status() and summary(), so a probe cannot forget it, and when no
witness is found it reports UNVALIDATED rather than zero stalls. emit() flushes
on every line. craft(), strengths(), alive() and heap() supply the
roster-to-craft link, per-record liveness and the raw heap, so a new probe
writes only its own logic.
Verified rather than asserted: deploy_probe.py reimplements the per-record
deployment watch on top of it in about forty lines against wave7_probe's
hundred and fifty, and its first live run was clean -- 116 roster records, 32
witnesses at 10/s, zero stalled samples, seven losses tracked, and the TSV
written incrementally. Nothing about the result is new, which is the point: the
harness reproduces a known-good measurement.
The existing probes are deliberately not ported. They work, and rewriting them
would risk changing results other documents cite. New probes should use the
harness; old ones should be ported when they next need a change.
