Built-in 64's slot-0 operand is a symbol-table-1 type-6 message id, and every
one of them now has caption text: 2683 of 2683 call sites across the 28 stage
scripts, 1338 distinct names, no residue of any kind.
This only became reachable once build_caption_text was switched to the IXUD
field table (537 -> 8800 lines); before that most of these names had nothing
to resolve to.
Adds isl_dialogue.py plus a committed Stage 02 sample. Does not settle which
recording plays for a given line, multi-page captions, or the other five
languages.
ixud.rs now has an IdxdObject-shaped reader, IxudObject, and build_caption_text
reads captions as FIELDS instead of pairing them with whatever token follows in
the pool.
build_demo_text token adjacency 134 ids 537 lines
build_caption_text token adjacency 3721 8074
build_caption_text record fields 4085 8800 = all of them
Verified over the whole disc by tests/ixud_records_disc.rs: 1104/1104 objects
parse, 1476/1476 records and 628165/628165 named fields reproduce their
ixud_hash, 48 positional, zero failures. The header word at 0x08 is record 0's
hash, asserted per object -- there is no schema field, exactly as for IDXD. The
module doc described a 12-byte record directory and a "schema/type hash"; both
were wrong and are corrected.
I also have to correct my own number from the previous commit. "1.3% of the
game's text" counted OCCURRENCES: each family lives in 24-45 IXUD blocks and
the same key repeats across them. Distinct text-bearing MSG_* keys number 8800,
not 44579, and every one has the <id>_<page>_<line> shape. So the real coverage
was 537/8800 = 6.1%, and I overstated the gap about fivefold. Direction right,
magnitude wrong.
The DEMO control is the sharpest evidence for the change: token adjacency finds
537 lines there, the field reader 541. It was dropping lines even in the one
family it was written for -- which is why the test now asserts "must not lose
lines" rather than "must be identical".
Same lesson twice in one session: pool adjacency is a consequence of how
records are written, not a rule of the format.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
build_caption_text generalises the key parser from MSG_DEMO_* to all eight
families. The shapes are uniform and each family is 100% consistent with its
own: seven use MSG_<FAM>_<id>_<page>_<line>, and VOICE alone inserts a family
letter before the id.
ids lines
build_demo_text 134 537
build_caption_text 3721 8074
The DEMO family comes out identical through both readers -- 537 lines either
way -- which is the control that generalising changed nothing that already
worked. Pinned by tests/caption_families_disc.rs, along with VOICE ids keeping
their family letter.
But this does NOT close the gap, and the write-up says so: 8074 against the
44579 text-bearing fields the record-level scan counts is about 18%.
The reason is the same lesson this session already learned once.
build_caption_text pairs a value with the key that happens to follow it in the
raw UTF-16 token stream -- the adjacency heuristic that was wrong for IDXD and
is wrong here for the same reason. ixud.rs has no record/field reader at all.
The IXUD record table IS decoded and verified disc-wide (1104/1104 objects,
628165/628165 fields reproducing their key) and was simply never wired into
the crate.
Next step recorded: give ixud.rs an IdxdObject-shaped reader and read captions
as fields rather than adjacent tokens. The decode exists; only the plumbing is
missing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
movie_subtitle handles MSG_DEMO_*, the cutscene captions. Counting every IXUD
block in GP_MAIN_GAME_E.pak, that is the SMALLEST of eight families:
MSG_ADAN 23236 keys 9801 with text ADAN combat chatter
MSG_RHIN 21196 8509 Rhino squadron
MSG_TCAF 17148 6728 TCAF
MSG_VOICE 13060 6776 in-mission scripted dialogue
MSG_BIRD 14036 5834 Bird squadron
MSG_ADPL 12640 4127 ADAN pilots
MSG_ACRO 4804 2244 Acropolis
MSG_DEMO 1252 560 cutscene captions <- the only one read
total 107372 44579
560 of 44579 text-bearing keys = 1.3%. I report the text-bearing column rather
than raw keys because only 41.5% of keys carry text -- the rest are the empty
line slots this container pads with, and counting those would flatter the
denominator.
MSG_VOICE_* is the family the message tables reference -- the dialogue whose
voice bindings this file now analyses in detail -- and nothing in crates/
parses it. So the corpus knows which bank plays for a line it cannot read.
First step recorded: build_demo_text already pairs a text value with the
MSG_DEMO_<demo>_<page>_<line> key that follows it, and the other seven
families use the same <id>_<page>_<line> shape, so generalising the key parser
is most of the work. With a warning attached: do NOT assume the id spaces
relate, since the voice-bank id and the caption id within one message page are
different numbers.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The 60 tokens absent from BOTH voice languages, itemised:
45 VOICE_E_ family -- 44 numeric [0..43] plus the lettered VOICE_E_012B
13 VOICE_C_ at 421, 423-426, 430, 432, 447-450, 470, 471 -- INSIDE the
listed range [0..489], so interior gaps rather than a truncated tail
2 VOICE_D_182 and _183, adjacent
These lines were written and captioned; only the audio is missing. Their
caption keys resolve to real text in the IXUD blocks:
MSG_VOICE_C_355_000_00 "What are you doing? Quit wasting..."
MSG_VOICE_C_367_000_00 "The final defense weapon is..."
MSG_VOICE_C_347_000_00 (Japanese)
MSG_VOICE_D_152_000_00 (Japanese)
MSG_VOICE_E_044_000_00 (Japanese)
Three of the five sampled are still Japanese INSIDE the English pak --
captioned but never translated, matching the untranslated entries already
noted for the localised-text container.
Separately, a trap worth its own heading: within one message page the voice
bank token and the caption keys use DIFFERENT numbering.
Message_106 voice VOICE_C_468 lines MSG_VOICE_C_385_000_00..02
Message_129 voice VOICE_D_182 lines MSG_VOICE_D_152_000_00..02
Message_044 voice VOICE_E_012B ID MSG_VOICE_E_044
Same family letter, different index space. Deriving one id from the other
will silently mis-pair audio with text -- which is the same class of mistake
as the demo-id voice binding this corpus already had to reject in-game.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Last iteration I read the scattered missing runs as cut dialogue. Checking the
other languages refutes that for most of them.
Only two voice languages exist: tables.pak holds eng\sounds.tbl (4418 banks)
and jpn\sounds.tbl (5136), and NO fra/deu/esp/ita. The other four
localisations are subtitle-only, which is itself a fact the corpus did not
record.
The English set is a strict subset of the Japanese one:
eng jpn
listed 4418 5136
resolved 4175 4383
missing 268 60
lacked by
the other 0 718
Every id missing in Japanese is also missing in English, and NONE is missing
in Japanese but present in English. So the 268 split cleanly:
208 exist in Japanese, not in English -- recorded in the original and
never dubbed. A localisation gap, not cut content.
60 absent from both -- genuinely cut, including all 44 VOICE_E_ (listed
0 times in either language) plus 16 others.
Per family eng/jpn: A 683/771, B 383/423, C 366/474, D 387/546. The 718
Japanese-only banks include whole families English never had -- VOICE_ADAN_
(110), VOICE_ZZZZ_ (104), VOICE_TCAF_ (41), VOICE_RHIN_ (27).
So VOICE_E_ survives as the one genuinely-cut family, and my "short runs of
cut dialogue" reading was wrong: those lines are on the disc, in Japanese. The
previous text is kept above the correction.
This is why the "measured on English only" limit was worth writing down rather
than quietly generalising -- the second language changed the conclusion.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Following up the incidental gap from the DEMO_* work. Resolving every VOICE_*
token the way the engine does -- token, then eng\sounds.tbl for its directory,
then sound.pak -- gives 4175 resolved and 268 unresolved of 4443 distinct.
The gap is in the manifest, not the archive: the unresolved tokens are not
listed in sounds.tbl at all, and of the 4175 that ARE listed, every one is
present in the pak. Nothing is listed-then-missing.
By family, script-used ids against sounds.tbl-listed ids:
VOICE_A_ 696 [0..772] listed 683 missing 14
VOICE_B_ 391 [0..422] listed 383 missing 8
VOICE_C_ 429 [0..489] listed 366 missing 63
VOICE_D_ 508 [0..598] listed 387 missing 126
VOICE_E_ 44 [0..43] listed 0 missing 44 <- the whole family
Two different shapes, and the distinction matters. A/B/C/D lose short runs
scattered through an otherwise dense range -- 11,12,13 / 56,57 / 181,182,183
/ 389-392 -- which is what cut dialogue looks like. VOICE_E_ is absent
entirely: 44 ids used, none listed, no VOICE_E_* anywhere in the manifest. A
whole speaker or category, cut.
For the port: a missing voice bank is a RETAIL condition, not a decode bug. A
reimplementation must tolerate an unresolvable VOICE_* reference, the same way
it must tolerate SUBTITLE_S12B.tbl resolving in no language.
Limit stated: measured on GP_MAIN_GAME_E.pak and eng\sounds.tbl only; whether
the other five languages drop the same ids is untested.
An earlier probe of mine checked only three sound.pak directories and would
have mis-attributed this; sounds.tbl lists five, including Briefing. Resolving
through the manifest rather than guessing directories is what makes the 268
trustworthy.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
An open question from the game_data migration: 132 dialogue lines carry an
unprefixed DEMO_* token where a VOICE_* one is expected, and whether DEMO_*
names a bank was untested. It does not.
VOICE_* 4443 distinct, 4175 resolve in sound.pak
(2360 eng\Voice\, 1815 eng\etc\)
DEMO_* 296 distinct, 0 resolve
Zero resolve under any of the three directories, and looking them up through
the convention -- VOICE_DEMO_nnn -- resolves 0 of 296 as well. So the slot is
not "the voice bank"; it can hold either a bank reference or a demo one.
The tokens run DEMO_000 to DEMO_310, 287 distinct in [0, 310] -- a dense
sequential id space, which is the shape of the demo id this file already uses
in MSG_DEMO_<demo>_<page>_<line> caption keys. I did NOT confirm the overlap
and say so: those keys live in IXUD wide-string blocks while my scan walked
IDXD objects only, so it found 0 of them. That needs an IXUD-aware sweep.
Incidental and left open: 268 of the 4443 VOICE_* tokens do not resolve
either. That is a separate gap and the DEMO_* finding does not explain it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
unit-struct-runtime.md lists ~30 player-craft fields as defaulted on disc with
values "recovered from the running game". All of them are on the disc at
exactly those values. Verified myself for UN_f001_TCAF_DeltaSaber_T:
Size_Radius 10.0, FCSRange 500000.0, ResistanceToOptics/HQRatio/
ShieldRatio/ThrusterRatio/ResistanceToShell/ResistanceToExplosion 1.0,
MassScore 0 [Generic]
ChargeSpeed 25.0, ChargeDelay_Break 10.0 [Shield]
DryMass 100.0 [Mass]
LowerHPThresholdRatio 0.3 [SE]
The record column is the explanation: the values are spread across Generic,
Shield, Mass and SE. That is exactly why the old flat reader missed them --
it could not say which record it meant, so a per-record field read as absent.
Also wrong, and measured: "18 of the 23 vessel records are missing at least
one of Size_X/Y/Z/HP, nearly always Size_Y". Over every object carrying a
Generic.Type -- 43 Craft plus 71 Vessel, 114 objects -- ZERO are missing any
of the four.
live-unit-definitions.md's "Route B: 13 defaulted fields read out of the
running game" goes with it: all 13 Size_Y values are on disc, each exactly
equal to the value the runtime run "recovered". Nothing was recovered that
the disc did not hold; the run confirms only that the parser reads the disc.
As with the weapon sheet, the runtime captures stand as measurements. What is
withdrawn is the premise that they were reaching values the disc lacks.
Correction banners at the top of both; original text intact beneath.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The refutation used the wrong hash function. A record key is tag_hash of the
record's own name, not the pak-TOC name_hash which lowercases and uses a
different modulus:
tag_hash("Formation_4_Bird") = 0x22a5eeed <- exactly the record key
name_hash("Formation_4_Bird") = 0x6286edad <- the wrong hash, hence
the recorded "0 of 16"
Formation_4_Bird IS a record name in FormationSet_S02.tbl and every record in
that table keys by tag_hash of its name. Across all 28 UnitGroup_S<NN>.tbl the
same holds for field keys: 6960 of 6960 are tag_hash(field name), 0 are
name_hash. So FormationID resolves by a direct hash lookup and the roster
indirection is NOT required for it.
This is exactly the trap idxd-container.md warns about, and it cost this file
a spurious refutation. The original text is kept below the correction.
The squadron half of the claim still stands and is marked so: a squadron
record's key is not tag_hash of the squadron id either -- tag_hash("TCN001")
is 0x760cdeaa against keys running 0x659aff47 -- because the record's NAME is
not the squadron id. That roster indirection is real.
Also closes the file's standing "the key derivation stays open, a second hash
function is unidentified": that function is tag_hash, recovered since.
Found while checking a subagent's claim rather than taking it -- it reported
the hash correction, and verifying it turned up that the FormationID half was
a stronger result than reported.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
sub_822FE040 fills 1023 eight-byte slots at table+32 with a default and then
overwrites individual ones; slot = (N - 32) / 8 from each std r9, N(r31).
Symbolically executing it yields 57 populated slots, matching the count the
corpus recorded, now with the full opcode -> handler map committed as
docs/re/data/isl-command-table.txt and regenerable from
tools/re-capture/isl_cmdtab.py.
Nine opcodes point at 0x82391BA8, which is `li r3,1 ; blr` -- accept and
discard. 768, 769, 774, 775, 776, 791, 792, 793 and 805 are dead in this
build, which is why the built-ins posting them do nothing.
Opcodes 800-802's entries are thunks 8 bytes apart into 60-instruction
handlers that differ in exactly two words: a descriptor offset and a unit
message id.
800 builtin 26 0xED0802DE
801 builtin 28 0xED0803DE
802 builtin 29, 101 0xED0804DE
That fixes the id format as 0xED08 nn DE, and the ids known from other work
fit it: opcode 514 -> 00DE, 803 -> 07DE, 999 -> 0FDE.
Stopped one link short of the semantics, and saying so: the pump's arm for
0xED0802DE does not apply an effect. It walks the unit's child list at
[unit+320]/[unit+324] and REBROADCASTS to each child as 0xED0902DE. So 0xED08
is the to-unit family and 0xED09 the to-child one, and the terminal effect is
further on. 26/28/29 remain unnamed.
The command table is the reusable part -- it answers "what does this opcode
reach" for every future built-in question, not just this family.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Method-diffing settled the structure but not the semantics, so nothing is
named this time.
26 damage_unit vt76 sub_8226ACD0 (67) opcode 800 97 sites
28 vt84 sub_82268F98 (69) opcode 801 410 sites
29 vt88 sub_822690B0 (69) opcode 802 164 sites
101 vt276 sub_822691C8 (78) opcode 802 133 sites (broadcast)
28 and 29 differ in two words only -- the opcode (0x21BA vs 0x22BA) and a
descriptor pointer 8 bytes apart -- and are otherwise instruction-identical.
All three take (unit, double). 26 differs from both by one guard: it rejects
only state 3, while 28 and 29 reject states 1 and 3.
The operand distributions separate them:
26 97 sites 7 distinct [0, 100] 0 x69, 80 x10, 100 x6
28 410 sites 13 distinct [0, 2000] 200 x116, 120 x76, 300 x74
29 164 sites 6 distinct [0, 100] 0 x64, 100 x53, 50 x38
26 and 29 are percentage-shaped; 28 is an absolute quantity an order of
magnitude larger.
That casts doubt on an existing name. damage_unit (26) passes 0 in 69 of its
97 calls -- dealing zero damage is a no-op, so 71% of its call sites would do
nothing, while SETTING a percentage-valued property to 0 is a natural thing
to do 69 times. 29 has the same shape. The name predates this session and I
have not withdrawn it, but it should not be relied on.
Where this stopped: the three commands' descriptors at 0x820A8D10/+8/+16
point into 0x8210E5xx, which is BELOW the disassembly DB's range (it starts
at 0x82150000) and holds no code, so that route cannot reach an execute
method. Opcodes 800-802's semantics need the interpreter command table
instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
1146 sites in 22 stages, the second-largest unnamed built-in. Its method
sub_822646B8 (vtable slot 300) and built-in 2's sub_822642E0 (slot 12) are
190 and 199 instructions and differ in exactly one block. Diffed instruction
by instruction, 108 adds:
lwz r11, 16(r29) the blob's slot-16 int
cmpwi/blt/cmpwi/bgt range-check n to [0, 31]
slw r21, r25, r11 r21 = 1 << n <- a 32-bit selector
... default 1 when out of range
Everywhere built-in 2 passes its r21, 108 passes r20 and reserves r21 for the
mask, so the bit is an EXTRA argument to the same call rather than a
replacement. Both post the identical command word AB0100BA, opcode 256.
The operand is always a valid bit index: over all 1146 sites slot 16 is in
[0, 31], 1146 of 1146, none outside, so the out-of-range default never fires
in shipped content. Fifteen distinct values clustered at 16 (531x), 31
(165x), 20 (161x) and 2 (90x); 21 of 22 stages use more than one.
What the bit SELECTS is not established and the name does not claim it. There
is plainly a 32-bit space -- built-in 92 reportedly allocates a free bit by
OR-ing over live units, which would make 108 its "place in a named slot"
counterpart -- but I have not verified that, so the name says only what is
proven: the same deploy as built-in 2, with an extra selector.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The highest-traffic unnamed built-in: 1197 sites across all 28 stages.
sub_822659F0 read directly:
* indexes [phase+324]'s record array by the slot-4 symbol;
* returns 0 immediately when the live object [record+4] is NULL, so it
registers an object that already exists rather than spawning one;
* sets [record+16] = 2, the documented "active" state every unit predicate
tests;
* stores sub_82301118's packed result into [record+20] (low 16) and
[record+24] (high 16) -- member counts;
* posts opcode 513 (0xAB0201BA) either way; the slot-8 mode (1 in 999 sites,
0 in 198) only decides whether cmd+20 is also set to 1.
The ordering test: if this activates a unit for the script, no predicate
should test a unit before it. Over all 28 stages, for every (stage, unit)
pair having both, activate_unit comes first 517 times and a predicate first
0 times. Recorded as file order rather than proven execution order --
coroutines can interleave -- but 517 with no exceptions is not a coincidence.
344 units are tested without ever being activated (live from mission start)
and 203 activated without being tested.
Also recorded: a tidy closure that FAILS. squad_survival_pct reports current
over initial and activate_unit snapshots counts, so the snapshot looks like
the baseline. It is not -- built-in 24 reads [record+16] for the state then
calls sub_823011B0 and sub_82301118 on the LIVE object, never touching
+20/+24. What reads those two fields is unidentified.
Named coverage is now 57 of 108 distinct ids and ~80% of call-site traffic.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The member object sub_82348830 returns is the per-member unit DEFINITION, and
that identification is not a guess: the same spawn loop builds two aggregates
and each lands on a semantically apt field with the apt reducer.
group +192 min, seeded FLT_MAX member +164 = CruisingVelocity
group +472 sum member +84 = HP
A wrong struct would have to make both offsets land on apt fields AND pair
each with the apt reducer. Minimum of a speed, sum of hit points: a
formation's cruise limit and its total health.
The quantitative test over all 1360 sites, joining each to its craft's
definition:
value <= the craft's MaximumVelocity 1355 / 1360 = 99.6% (5 fail)
value <= the craft's CruisingVelocity 1042 / 1360 = 76.6% (318 fail)
The test discriminates -- the cruise bound breaks 318 times, the hull maximum
5 -- so the script sets a COMMANDED SPEED, free to exceed the cruise default
and bounded by what the hull can do.
The turret anomaly that stopped me naming this two iterations ago was my own
artefact. UN_e007_ADAN_Turret's definition carries MaximumVelocity 500 and
CruisingVelocity 280: the data models turrets as if mobile, so a script value
of 400 is legal and simply never manifests. I had assumed turrets have no
velocity fields and treated 13% of the traffic as a refutation.
Recorded as unsettled: the five overshoots are UN_e106_ADAN_Destroyer 200 vs
a 150 maximum (x2) and UN_e011_ADAN_Attacker_B_HF/_Wayne 500 vs 450 (x3).
Designer overrides or an engine clamp; not established.
Named set_group_speed. Default = the slowest member's CruisingVelocity;
mode 1 restores it, mode 3 sets it, mode 2 hands it a global constant.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The spawn routine sub_8232B538 settles what the field is without needing its
consumer:
8232B5B0 lfs f0, 25184(r11) -> 0x820B6260 = 3.4028235e38 (FLT_MAX)
8232B5B4 stfs f0, 192(r30) seed +192 with FLT_MAX
8232B674 lfs f13, 164(r3) each member's own value
8232B688 fsel f0, f11, f0, f13 f0 = min(f0, f13)
8232B68C stfs f0, 192(r30)
8232BA28 stfs f0, 196(r30) +196 starts equal to +192
The FLT_MAX seed is what makes it unambiguous: +192 is the MINIMUM of
[member+164] over the members.
So built-in 15's target is a GROUP, not an individual unit -- this file and
my own earlier notes both called it a unit. sub_82348830 hands back a
per-member object and the reduction runs over all of them.
A minimum over members is how a formation limit works: the group goes as fast
as its slowest ship. On that reading +192 is the natural cap and +196 the
effective one. And a CAP explains the turrets where a commanded speed could
not -- capping a static object at 400 is a no-op, so a designer can set it
uniformly from a template without consequence, and the asteroid's invariant 0
fits the same reading.
Recorded as unproven: mode 2's constant is [r27 + 13912] where r27 comes from
a runtime pointer at 0x8232C718, not a static base, so it cannot be resolved
from the image. If it is FLT_MAX then mode 2 is literally "uncapped" and the
three modes are set / uncap / restore. Tidy, and not shown.
The consumer is still not found. Also recorded: the image has NO RTTI at all
(0 of 1150 vtables), so class names are synthetic and the two constructors
writing vtable 0x820AF030 are 28 and 30 instructions touching neither field.
Bounding the group struct is the remaining prerequisite.
Built-in 15 stays unnamed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Read directly at 0x8232C7CC, the unit message pump switches on [msg+36]:
mode 1 [unit+196] = [unit+192] restore the unit's stored default
mode 2 [unit+196] = [r27+13912] a global constant
mode 3 [unit+196] = [msg+40] the value the script passed
So the field has a per-unit default at +192 and a distinguished global value.
The shipped scripts only ever use mode 3 -- slot 8 is the double tag in
1961/1961 call sites -- so the two defaulting modes are dead in this build.
The consumer I did not find, and the failed approaches are recorded so the
next attempt does not repeat them:
* searching the flight/AI range for 196(rN) gives 170 hits, almost all on
unrelated structs -- the N(rM) trap this file already warns about;
* narrowing to functions touching BOTH +192 and +196 still leaves 50;
* unit_definition_layout.txt names offsets 192/196 as AV_PitchMinus_Max/Min,
but that is the unit DEFINITION object (vtable 0x820af844), not the spawned
entity built-in 15 writes to. It does not apply, and taking it would have
been an easy wrong turn -- angular-velocity names would have "explained"
the turret anomaly while being about a different struct entirely.
What would settle it is pinning the spawned-entity struct first, then reading
+196's consumers within it.
Built-in 15 stays unnamed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
15 is the biggest unnamed built-in: 1360 sites across 27 of 28 stages, with
an invariant (unit, double) operand shape. Joining every call site's unit to
its craft type through stage\UnitGroup_S<NN>.tbl resolves 1360 of 1360, none
unknown, and the values stratify hard:
capital hulls (Destroyer, Carrier, CruiserEX, Cruiser, Acropolis,
BattleshipEX, ASFrigate) 0-100
mobile craft (Elan 500, ArrowHead 400, DeltaSaber_T 400,
DeltaSaber_W 600, Attacker_S 250-400,
Phantom 300) 250-600
asteroid (mn040_Asteroid_Big, 74 sites) 0, only 0
Capital ships crawl, fighters run, the player's craft is fastest at 600, and
a rock never moves. That is a speed's signature.
Except turrets break it. UN_e007_ADAN_Turret carries 400 x89 / 280 x17 /
380 x4 / 250 x2 and UN_e008_ADAN_TurretPlus 450 x62 / 500 x2 -- 176 of the
1360 sites, 13%, at fighter-class values on something that does not move.
The asteroid is what makes this sharp rather than dismissible: a genuinely
immobile object gets 0 every single time, so "immobile things get junk" does
not explain the turrets.
So the operand is a craft-class-dependent scalar with speed's range and
ordering, but set_speed is not a sound reading of it. Left UNNAMED. The
handler writes [unit+196], initialised from [unit+192] at spawn; reading its
consumer is what would settle it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
96, 97, 98 and 104 were unnamed. Measured over all 28 stages they form a
family that appears nowhere outside the six tutorials:
96 8 sites S18-S23 only
97 38 sites S18-S23 only, followed by start_coroutine 27/38
98 0 sites never called anywhere
104 39 sites S18-S23 only, followed by wait_s 39/39,
preceded by end_coroutine 37/39
104's adjacency is a textbook poll loop with no exceptions: coroutine starts,
tests the predicate, waits, repeats.
96's operand is the giveaway. Its eight payloads in stage order are
S18 -> 101, 102, 103; S19 -> 201; S20 -> 301; S21 -> 401; S22 -> 501;
S23 -> 601, i.e. (stage - 17) * 100 + section. Tutorial 1 has three sections
and the other five have one each, and the tutorial index is encoded in the
argument.
Names are deliberately NOT applied. tutorial_begin / tutorial_end /
tutorial_message_pending fit the shape, and [phase+340]/[phase+344] are said
to carry the state and payload, but I have not read that handler myself and
this file has already had to withdraw five names guessed from shape. The
distribution and the argument encoding are established and are what a port
needs; the labels can wait for someone to read the code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
77, 78, 81, 82 and 135 were unnamed. The engine has five contiguous strings
-- MISSION_START_PRT at 0x820A83F0, then _END_, _UPDATE_, _FAILED_,
_RESTART_ -- and five sequential ScriptPhase fields at +388/+392/+396/+400/
+404, stored in ascending order by one constructor region. Five names, five
fields, five unnamed built-ins.
Which is which is decided by call-site structure, measured over all 28
stages, and it is exact:
39 MARK_LAST_PHASE 89 sites -> 82 in 89/89
82 banner_mission_failed <- 39 in 89/89, then wait_s 89/89
40 mark_not_last 50 sites -> 78 (27) + 81 (17) + END_PHASE (6) = 50
78 banner_mission_complete <- 40 in 27/27
81 banner_objective_update <- 40 in 17/17
77 banner_mission_start 22 sites in 22 stages, one per stage,
after play_bgm
135 banner_mission_restart 16 sites, after play_bgm, phase >= 2
39 -> 82 is a perfect pairing and 40's sites partition exactly three ways.
Stated as inferred rather than read: the string-to-field pairing itself comes
from both sequences ascending in the same order; my operand tracker did not
catch the string loads in that constructor. The ROLES above do not depend on
it.
76 is left unnamed on purpose. It has 38 sites = 22 + 16, exactly 77's count
plus 135's, and precedes them; its body sets [phase+332] = 1 and nothing in
the image reads that field. Suggestive arithmetic is not a name.
Flagged as a consequence: MARK_LAST_PHASE is followed by the FAILED banner in
89 of 89 sites and mark_not_last by END or UPDATE. So [phase+300] = 2 reads
less like "this is the last phase" than "end the mission now,
unsuccessfully" -- the existing names for that pair may be mis-framing it.
Artifact regenerated: docs/re/data/isl-stage02.txt.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The corpus recorded that some unit fields the disc leaves defaulted inherit
from a sibling: Size_Y from Size_X, FCSRange from RadarRange, DefencePoint
from AttackVesselPoint. Size_Y was marked the one to trust, on 9/9 support
across 7 independent ships, and it is restated in INDEX.md.
The premise is false. These fields are not defaulted -- they are on disc for
113-114 of 114 unit tables -- and Size_Y DIFFERS from Size_X in 90 of them.
The mechanism, cross-tabulating "legacy reader missed it" against "equal on
disc":
pair seen+differ seen+equal miss+differ miss+equal
Size_Y / Size_X 90 0 0 24
FCSRange / RadarRange 54 0 1 58
DefencePoint / AttackVesselPoint 51 0 1 61
seen+equal is 0 for all three: a value shared with a sibling is ALWAYS
invisible to the string-pool reader, because the pool stores each distinct
string once. And the reader almost never misses a value that differs. So
"the missing value equals the sibling's" was true BY CONSTRUCTION -- the rule
re-derived the very condition that made the field go missing. That is why the
support looked perfect: it could not fail on the cases it was fitted to.
The two miss+differ cells are its real wrong predictions, both named:
UN_e104_ADAN_Carrier DefencePoint is 0.2 (rule says 0.003), and
UN_e011_ADAN_Attacker_B_HF_Wayne FCSRange is 3000.0 (rule says 6000.0).
Retracted in unit-struct-runtime.md (original reasoning kept below the
correction), live-unit-definitions.md and INDEX.md. Pinned by a disc test
that asserts the seen+equal cells stay zero, so the mechanism itself is
guarded, not just the counts. Artifact: examples/sibling_rule_check.rs.
This one was found by my own check after the subagent assigned to it stalled.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
movie_manifest::parse now reads BASE_INFO's positional field keys (the game's
own cutscene ids, stage*100 + slot) and follows each to its record, instead of
scraping the string pool. The pool stores each distinct string once, so a
REPEAT reference produced no token and read as "no binding".
That single cause explains every wrong cell: 13 later references to
VOICE_D_450..454, two to SUBTITLE_hokyu_LS_s11A.tbl, and MS01A's share of
pwterop_s01a.prt. All 18 hokyu movies are bound, not five.
Counts, verified independently by me against the disc before recording:
104 cutscene SLOTS binding 101 distinct MOVIES; 99 slots / 96 movies with a
voice track, 99 / 96 with a subtitle, 22 / 22 with a telop. The docs' old
94 / 83 / 21 are exactly the counts of DISTINCT POOL STRINGS -- not wrong
measurements, measurements of the wrong thing. Three denominators were being
conflated; the new test pins all three.
Two assertions in movie_manifest_disc.rs were false and are corrected:
hokyu_DS_s13A binds VOICE_D_452 and resolves to eng\etc\VOICE_D_452.slb. The
in-game verdict that rejected that value tested an INFERENCE from a shared
demo id, on a decoder that discards 85-87% of banks in this class -- see
voice-bank-leading-region.md, committed earlier today.
The ~104 script ids are no longer open: they are literal positional keys,
each naming its record, and all 104 resolve. The old "counts differ by three,
positional pairing does not work" has a concrete cause -- three resupply
movies are bound by TWO slots each.
Also corrected: the naming convention has 3 subtitle exceptions (s24A/s27A
borrow s11A's track) and 18 voice exceptions, not one and five.
The legacy scraper is kept as a fallback for blobs with no record table, so
the synthetic unit fixtures still exercise it.
Artifacts: examples/movie_map_csv.rs regenerates the CSV, now slot-keyed
(104 rows; the movie-keyed version silently dropped one slot of each
duplicate). Disc tests green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The record table gives a DIRECT binding hokyu_DS_s13A -> VOICE_D_452, where
the corpus records the movie as unbound and movie_manifest_disc.rs asserts
None, citing an in-game verdict that this exact value was "the wrong
recording". That is the only place on the disc where a runtime observation
disagrees with the record table, so it was worth settling.
First, shape: these banks are SHARED. Five slots bind VOICE_D_452, five bind
451, four 450, four 453, three 454 -- 21 hokyu slots over five banks, and the
movies repeat too. Generic resupply cutscenes, not per-stage recordings.
The recorded explanation for 453 decoding to 0.14 s and 454 to 0.43 s was
that the banks are "likely multi-subwave / not cleanly sliced". Refuted: the
count of RIFF magics EQUALS the number of sub-waves recovered in all five
banks, and the last data chunk ends exactly at EOF in four of them. Nothing
between or after sub-waves is being missed.
The real defect: slb::to_xma_riffs finds audio by searching for the RIFF
magic, and a large region PRECEDES it. 87% of VOICE_D_453 and 85% of
VOICE_D_454 sit in front of the first RIFF -- 21-27% zero over 256 distinct
byte values, i.e. content, not padding. VOICE_D_451 is the control: its
leading region is 100% zero, 1 distinct value, real padding.
So the in-game verdict listened to a decode that had discarded most of the
bank, for exactly this bank class. It is evidence about the decoder, not
about the mapping. Note also that what was rejected was a value INFERRED
from a shared demo id; the record table supplies the same value as a stored
field, and only the inference was ever tested.
This does NOT establish the binding is right -- it removes the only recorded
evidence against it. What the leading region actually holds is undecoded, and
confirming the binding needs a human listening.
Artifact: examples/voice_bank_shape.rs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Verified rather than adopted: a subagent proposed that every even operand
slot is a type tag. Measured, the strong form is false and a precise form is
true.
TRUE: a SYMBOL operand is two words, a tag holding the constant 1 followed
by the index. Slot 0 is the integer 1 in 19899/19899 calls whose slot 4 is a
unit; slot 8 is tag-shaped in 100% of calls for every built-in taking a
second unit; slot 16 is 1 in 152/152 for built-in 128, the only one taking a
third. The 24 built-ins whose slot 0 is NOT the constant are exactly those
taking no symbol there. This explains the unit slots 4/12/20 rather than
replacing them.
FALSE as stated: slot 8 is a bare double for built-ins 4, 20, 24, 26, 28,
29, 90, 106 and 127, and built-in 75 carries five bare indices at 0/4/8/12/16
with no tags at all. Each built-in has a fixed signature and is 100%
self-consistent; none of the 34 with >=20 sites mixes the two.
Symbol table 1 has three types -- 1 routes (1362), 6 messages (2247), 7
effects (81) -- and its operand slots are type-pure, measured the same way.
Resolving them makes listings say what the script means:
`request_script_message(MSG_VOICE_D_257, ...)`, a fourth independent
confirmation of that name. Slots 24@4, 46@12 and 114@4 resolve 100% but MIX
types 6 and 1, so they are left unresolved rather than guessed.
Two more names withdrawn, neither replaced:
* 88 `camera_at` -- ZERO call sites in all 28 stages; never testable.
* 90 `camera_at_route` -- 8 sites, all Stage 02 phase 3, first operand is
symtab-1 type 7 `eff_n0071`, an EFFECT name, in 8/8, with a per-missile
Route_ADT301..308_p3M at slot 20. Not aimed at a camera.
Left unnamed on purpose: replacing a guessed name with another guess is how
the three names corrected earlier today went wrong.
Also flagged: 115 `named_event`'s only symbol operand is an eff_* name in
84/84 sites, so that name is suspect too. Not renamed pending a handler read.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
Third independent line for yesterday's built-in 100 rename, from the callee
this time. sub_8226E3B8 was labelled "push", which is what made built-in 100
look like push_trigger. It reads the element count, returns immediately when
the container is EMPTY, and otherwise walks the node list splicing nodes out
until it is empty. A push links one node; this unlinks all of them. It is
clear(). The append is sub_8226E160, reached from built-ins 19 and 25.
So the rename now rests on the handler, the usage (all 12 Stage 02 sites sit
in the phase terminator next to timer_stop / clear_flag(-1) /
MARK_LAST_PHASE), and the callee.
The dynamic half did NOT run, and the write-up says so. phase_watch.py now
samples [phase+272+20] (triggers queued) and [phase+216+8] (coroutines
alive) so a phase terminator's effect on the VM is visible in one line —
written here, never yet exercised against a live guest.
Boot-nav could not reach the title in 381 s. Diagnosed rather than retried:
skip_intro.sh only runs the title test on a static frame, gated at
rmse <= 1500, and this run measured 1503 at 104 s and 1549 at 139 s — just
above the cut — so is_title.py was never called and the one allowed press
was never spent. Recorded in BACKLOG with the explicit instruction NOT to
raise the constant: the first step is to log rmse and the glyph count
through a whole boot and look at the two distributions, because tuning a
threshold to make one run pass is fitting to a single sample.
Also reaped a stale lock: a gdb orphaned 2h14m earlier was holding
/tmp/xenia-canary.lock with an already-defunct emulator child.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
All re-read twice — the handler, and the thing it calls — because each had
been named from its shape rather than its effect.
* id 11 `yield` -> `end_coroutine`. 0x82272624 is li r11,1 ; li r3,3 ;
stw r11,164(r31), and the dispatcher's r3==3 arm erases the thread from
the active list and returns it to the free list. It destroys the thread.
2945 sites game-wide, 372 in Stage 02 — the most-used built-in there was.
* id 5 `await_label` -> `kill_coroutine(label)`. sub_82273B08 kills the
thread parked at the target pc, or itself if the target is its own pc.
It waits for nothing.
* id 100 `push_trigger` -> `reset_phase_threads`. It clears the trigger
container and then frees every thread whose pc differs from the caller's
— the opposite of pushing a trigger. Corroborated by usage: its 12 Stage
02 sites all sit in the phase terminator, next to timer_stop,
clear_flag(-1) and MARK_LAST_PHASE.
One name recovered from the game's own text: opcode 992 prints
"RequestScriptMessage %s" at 0x820A5700, so id 64 is request_script_message
(2683 sites).
Return codes documented properly: 1 = restart the coroutine from its entry
(previously not recorded at all), 3 = terminate. And the blocking set was
wrong in two places — it is 102, 120, 137, 142, 143. Id 97 does NOT block;
its handler ends `b 0x822724F8`, so it always returns 0.
Unit-operand resolution settled from DATA over all 28 stages rather than by
reading 147 handlers: a slot qualifies only if every value is a valid
symtab-2 index, it takes >=15 distinct values, AND its maximum reaches most
of the table — that last clause is what discriminates, since every small
integer is trivially "in range". 31 built-ins at slot 4, 8 at slot 12, one
at slot 20. It also refutes set_flag's slot 0, whose maximum overruns the
table, and the resolver now declines rather than inventing a name.
New and unexplained: symtab-2 holds two types, 2 and 8, and built-ins 95 and
128 take type 8 at slot 12 in 100% of their sites.
A downstream inference is withdrawn with it: the note reading the live
trigger counter attributed it to "the script arming watches as it goes" via
built-in 100. The measurement stands; the attribution does not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
The binary region in front of the string pool was the parser's oldest open
note ("Not yet decoded"). It is a uniform 16-byte record array sorted by
name hash, a field count, a 12-byte field array sorted by key, a pool size,
and the pool. The trailing `pool_size == file_len - pool_base` identity makes
the layout self-checking, which is what caught the first wrong version.
Verified over the WHOLE disc with zero failures: 7750/7750 IDXD objects,
190782/190782 records reproducing their stored tag_hash, 1271462/1271462
named fields reproducing their key. IXUD is the same container with
ixud_hash, UTF-16BE and every offset in chars — 1104/1104 objects,
628165/628165 fields, checked with an independent parser.
Field names are stored on disc, so no preimage search is needed: a field's
middle word points at its own name. Only 504 fields disc-wide are hash-keyed
with no name; the other 1485073 nameless fields are positional, keyed by a
literal integer (line slots, movie ids).
Two long-held beliefs are WITHDRAWN:
* The word at 0x08 is not a schema hash. It is record 0's name_hash — the
format has no type field at all, and an object's kind is known only from
the caller that loads it. It survived as "schema" because tables of one
kind share their lowest-hashed record name. Caught by a test asserting
every movie id names a real record: 1005 -> STAGE10_PHASE01 failed because
tag_hash("STAGE10_PHASE01") IS 0x067025B9, that table's supposed schema id.
* The field's middle word is not an always-0xFFFFFFFF flags word. It is
0xFFFFFFFF for 54% of fields, enough to look constant in a small sample;
the tell was that it is constant per key ACROSS records, which a per-record
flag cannot be but a per-name pointer must.
`schema_hash` keeps its name rather than churn 33 call sites, with corrected
docs. The first sweep globbed dat/** and missed hidden/DefTables.pak (1425
objects); the test now walks the whole disc root.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
534 IXUD entries across 36 paks, 94 distinct keys, 92 resolved by name. Content
identified: mission objective/lose/hint panels, 91 bonus-objective banners,
scripted radio, cutscene dialogue, the reactive combat-chatter pool (MSG_ADAN
23,236 fields, MSG_RHIN 21,196, ...), resupply barks, speaker labels and the
hangar weapon Name/Desc/Condition.
Verified independently: language\MissionDialog_local_string.tbl decodes to 600
fields with the trailer identity holding, giving Stage 02's real objectives --
phase 1 'Shoot down all invading enemy fighters / while watching out for attacks
/ on the ACROPOLIS', phase 2 'Protect the Admiral's ship the CALIBAN', phase 3
'Destroy the interstellar cruise missiles'. That matches the script exactly: the
phase-1 condition polls three enemy squadrons and the escort's loss is the fail
branch, and phase 2 is positional, which is why no kill count appears there.
The .ssb symbol names (S02_P1_OBJECTIVE etc.) are NOT IXUD keys -- they are
records in an IDXD map, message\MissionDialogMessage.tbl, whose positional
fields list the lowercase per-line IXUD names; *_GRAPH holds a texture instead.
Structure: one record per object (except the six 63-record hangar files),
624,440 named fields, 55% of them empty line slots, no control characters, the
only escape a literal two-char \n in block-text tables, and records/fields in
ascending key order so lookup is a binary search. '[OB]' turns out to be a HUD
marker label in this text -- the same one the REMAINING OB work chased.
Six languages confirmed D/E/F/I/J/S; J omits the trailing empty line slot in 78
of 86 objects. Two keys remain unresolved, byte-identical across all six paks
(untranslated Japanese voice transcripts).
Also corrects mission-objectives-text.md: SUBOBJ_009 does have text, and
SUBOBJ_005 is two fields.
Three come from the game's own strings, verified present: 'Set Squadron order
attack/trace/escort' at 0x820AF17C/1A4/1CC, loaded once each at entry to the
unit message pump sub_8232C4C0 so the pairing is unambiguous. The printf is
compiled out (bl 0x82674028 = li r3,0; blr) but the format strings and arguments
survive, which is what makes them usable as labels. The same poster functions
serve both the message path and the interpreter-command path, so the identity
transfers to opcodes 517 trace / 784 attack / 790 escort, i.e. ISL built-ins
46/47/48, now named in isl.py.
The fourth, opcode 512, is recorded at LOWER confidence on purpose: those strings
name only three. Its label rests on the route-generation strings on its own
vtable, a symbol-table-1 (route) argument rather than the unit table, a route
point-count lookup, and the pre-existing move_order label -- four independent
lines, but none of them the game naming it.
Also records a trap: sub_82320B48's ORDER_{WINGMAN,SQUAD,SQUADRON}_{FORMATION,
ATTACK,COVER,EXTENDED} enum looks like it should map onto the order classes and
there is no evidence it does. It is a voice/comms axis; not connected.
sub_822FE040 is a fully unrolled registration: 1023 slots filled with a default,
then 57 explicit writes, of which 48 are real handlers and nine are a shared
accept-and-discard stub. Return convention is nonzero = consumed, 0 = retry,
which is how the interpreter waits for a named unit to exist.
Opcode 995 is the ONLY handler touching the phase mirror [*(0x828F35F8)+236] --
the sole read and sole write in the table -- independently confirming why polling
that mirror saw nothing during phase 1. And no handler spawns or despawns a unit:
256 is the strongest deploy candidate but is unconfirmed because the message ids
are write-only in this image.
WITHDRAWN, verified wrong: I had recorded the writes to '+20' in sub_8226E7D8 /
sub_8226E930 as block initialisations by container constructors. At
0x8226E86C-0x8226E8E0 they do li r3,28 / bl 0x8230C160 then
lis r10,0xAB03 / ori r7,r10,0xE4BA / stw r7,4(r3): they build an INTERPRETER
COMMAND RECORD for opcode 996 and push it, i.e. AddSelector and RemoveSelector,
with a 32-entry cap. The stw to 20(r3) is the command record's +20, a different
object. Wrong twice: not constructors, and not that container.
Also flags that sub_8230C398 -- gated on *(0x82899CE0) == 16 at both call sites
-- looks like Stage 16's script compiled in C++, which 'debug defaults' does not
survive given the .ssb loader explicitly refuses mission 16.
Chasing which of sub_8226EAB8's 16 callers grows the trigger count: sub_8226E458
decrements a count at 8(r30) then calls the insert helper -- a splice -- and has
exactly one caller, inside sub_8226D740, the per-frame engine->script drain. That
suggests the engine moves records into the phase queue each frame.
But the call site refutes the neat reading: at 0x8226D780 the argument is
lwz r4, 324(r29), the UNIT ARRAY, not the trigger container. So I have not shown
sub_8226E458 touches the trigger queue at all, and 'the engine feeds triggers
each frame' is my inference rather than the disassembly's.
Recorded unresolved. Taking a function's shape for its purpose is what produced
the 'push' mislabel on sub_8226E3B8 and the ADN110-for-null pretty-print, each of
which cost an iteration.
Unaffected and solid: sub_8226EAB8 increments a count at +8 of its container, the
trigger container embeds its list at +12, so the watched word at phase+272+20 is
that inner list's count, and the guest was inside sub_8226EAB8 at the write.
Next: instead of guessing among 16 callers, re-run the watchpoint and read the
guest LR from the context -- the technique that named the writer will name its
caller.
The watchpoint plus Canary's source settle it. At the write, the guest context
(rsi, per x64_emitter.cc:881) holds 0x8226EAE0, inside sub_8226EAB8. That
function is a generic list-node insert: it reads the count at 8(r30), guards
against 0x3FFFFFFF overflow, does addi r11,r11,1 / stw r11,8(r30), then links the
node. It has 16 callers, so it is a shared container helper.
That explains why two static searches missed it. The trigger container at
phase+272 EMBEDS an inner list object at +12 -- which is why the push does
'addi r31, r30, 12' -- and the inner object keeps its count at its own +8. So
272 + 12 + 8 = 292 = the watched word, and the instruction is stw r11, 8(r30)
with r30 = phase+284. Searching for 'stw rN, 20(rM)' could never have found it.
Also resolves the earlier open item on +12: it is the embedded list object, not
a list head pointer, which is why it read 0x000A0009 instead of an address.
Method note kept: the static hunt assumed the field's offset in the OUTER object
would appear in the writing instruction. A watchpoint is indifferent to the
addressing form, which is why it was the right tool after two failed offset
searches.
trigger_watch.sh + host_addr.py translate the guest VA to a host address
(0xBE14DEA4 -> host 0x1BE14DEA4) and watch it. It fires: Thread 50 'Main
XThread', old 0, new 16777216 -- which is big-endian 1 read little-endian, so
the count going 0 -> 1, independently confirming the field. The write happens on
the guest's own main thread, not an emulator worker.
But the writer cannot be named from the host stack: the faulting PC is
0xa0c65f23 with no symbol, executing 'mov 0x110(%rsi),%rbx', i.e. Xenia's
JIT-compiled guest code, and the frames above it are not host-unwindable. So the
watchpoint answers when and which thread, not which guest function.
Recorded as a ceiling of the method rather than retried blindly. The way past it
is that the JIT holds the guest context in a register (%rsi here), so the guest
PC is recoverable from the context block -- which needs Xenia's context layout
from the xenia-rs sources on this box, a separate tractable piece of work.
All 25 opcodes now have meanings. Ops 2/4/6/8 are integer compound assignment
(+= -= *= /=) and 3/5/7/9 the float versions; 10 and 11 are integer and float
compare writing three condition bits; 13-18 are je/jne/jl/jle/jg/jge; 21-24 are
push.i/push.f/pop.i/pop.f over deques at phase+44 and phase+64.
The shared-handler question is answered: the dispatcher leaves the opcode in r4
and the shared thunks never overwrite it, so those helpers take an extra opcode
argument and index a secondary table (0x82271448, 0x8227152C).
CORRECTION to my own tool and note: the branch/jump base is [phase+232], which
the phase initialiser sets to 0x24 + the phase's entry from the mission-level
stream -- 0xE4 / 0x14AA8 / 0x24B4C for Stage 02's three phases, not the file's
0x24. Measured on phase 1: base 0xE4 puts 525 of 525 branch targets on an
instruction boundary; base 0x24 manages 188. isl.py had been using 0x24 for
every phase, so its jump targets were wrong throughout. Fixed via
isl.phase_bases().
That also settles two things mission-script-ssb.md left open: offsets ARE
code-base-relative, and 0x1883's operand IS a code pointer -- the earlier worry
that some 'land on IEEE floats' was an artefact of adding the wrong base.
Three approaches, none successful. sub_8226E160, earlier flagged as 'enqueue a
pending trigger', has exactly one caller and is a specific operation rather than
the general append. Writes to the count at +20 inside the container code number
only four, and all four are part of a block initialisation (stw to 0/8/12/16/20/
24 in consecutive instructions) in sub_8226E7D8 and sub_8226E930 -- constructors,
called from 0x8226E560 and from ScriptMission's own constructor at 0x822608A0.
So the increment that takes the count 0 -> 1 -> 2, which is measured live, does
not appear as a plain stw to 20(rM) anywhere in the container's code. It is
inlined, uses another addressing form, or lives somewhere I have not looked.
Recorded as not found rather than guessed: inferring from the shape of nearby
functions is exactly what produced the 'push' mislabel last iteration.
Names the approach that would settle it: a gdb watchpoint on
ScriptPhase+272+20 during a live mission. The address is known at runtime, the
count demonstrably changes within ~2 minutes, and a watchpoint reports the
writing instruction directly instead of inferring it from static shape.
Walked the container's list from +16 for 200s. The structure holds: +20 tracks
the node count, nodes chain through their first word, and entries appear as the
mission runs (0 -> 1 -> 2, then stable).
The record layout does not. I expected node+8 to hold small symbol indices,
which the pop's out-parameters made natural. Every field is a guest heap pointer
(0xBC..), so trigger records reference objects rather than table indices, and
those objects are unidentified.
Also records a false resolution I introduced: a line printed 'f4=0(ADN101)'
because the raw value is 0 and my formatter mapped index 0 to symbol-table-2's
first entry. ADN101 is not in that record -- the pretty-printer invented a name
for a null. A resolver must refuse values that were never indices.
And corrects the previous section: sub_8226E3B8 is a CLEAR, not a push. Its tail
decrements a counter, calls an erase helper, and loops while [+20] != 0. So
built-in 100 clears the queue then rebuilds the thread list, matching the
built-in table's own wording; 'push' was my label, not the disassembly's. Two
callers: vt2 (script) and 0x8226D420 (an engine site). What appends a node is
still unidentified.
Read from a running Stage 02 mission: the container at ScriptPhase+272 has +20
moving 0 -> 1 -> 2 over the first two minutes while the phase ordinal stays 1.
So +20 is a real count of currently registered triggers -- the script arming
watches as it goes, and Stage 02 has 12 push_trigger sites -- and it is readable
from /dev/shm with no debugger. First direct view of what the script is waiting
for.
Corrects one field from the static reading: +12 is not (only) a list head. It
reads 0x000A0009, which is not a pointer, even though the push's
'addi r31, r30, 12' made list-head the obvious interpretation. Recorded as
unresolved rather than kept quietly.
Chasing what makes the phase-1 condition re-evaluate, since the polls do not run
continuously.
Two method corrections: searching the VM range for '272(rN)' mostly returns
VTABLE slot offsets -- 0x82273174 lwz r11,272(r11) is followed by mtctr/bctrl,
a virtual call through slot 68, not an access to the phase field. And
[phase+272] is not a pointer to a queue but an EMBEDDED container: vt2
(sub_82265DD0) is 'addi r3,r3,272 ; b 0x8226E3B8', passing phase+272 as this.
Layout from the push/pop pair (sub_8226E3B8 from built-in 100, sub_8226E220
called every frame from sub_8226D740): +12 list head, +16 current node, +20
element count (zero = empty, tested first by the pop), +24 scratch. The pop
returns the record through out-parameters read from node+8: three u32s, a
double at +16, another u32 at +24 -- matching the six pointers sub_8226D740
passes in.
The actionable part is [phase+272+20], a live pending-trigger count readable
from /dev/shm. Watching it alongside [ScriptMission+40] should show when the
engine hands the script an event, which is when condition coroutines start --
the thing every phase experiment so far has been blind to.
Layout is from disassembly only; not yet verified live.
Disassembling built-in 69's handler at 0x8226ADF0 instead of trusting the
summary: it loads rec = base[idx], tests rec+4 against zero (early exit =
absent), passes rec+4 to sub_82301240 for the lifecycle lookup, and compares the
byte at rec+104. rec+16 is never touched on that path.
That explains the null poke result exactly -- I wrote into a field the condition
does not consult. +16 remains a faithful OBSERVABLE (the arrival and death
transitions were real) but it is a readout, not the input. It also explains why
rec+4 holds small consecutive integers: it is a handle that sub_82301240
resolves, not a pointer.
The corrected simulation of 'this squadron is gone' is rec+4 = 0, taking the
documented early exit.
Flags a general caution: this built-in table came from a subagent's reading of
handler behaviour, and this is the second per-offset field description that did
not survive the disassembly. The identifications of which built-in does what
have held up; the offset meanings are leads to verify.
Last iteration ended with 'what reads the flag file is unknown' after an offset
search failed. The framing was wrong: I was hunting an engine-side reader, but
the consumer is the script itself, through built-in 9 (read_freg), which loads
[phase+88][i] into the double result register.
Stage02.ssb calls read_freg 12 times -- the same count as set_flag (12) and
clear_flag (12) -- so the latch is symmetric and entirely inside the VM. That
closes the middle of the set_flag -> ... -> END_PHASE chain: one coroutine
latches, another reads it back and branches.
Also records an unconfirmed observation: op10 + op13 pairs with consecutive
small immediates and their own code offsets look like a case/branch dispatch.
Flagged as a pattern, not named -- neither handler has been read.
Resolving symbol-table-2 indices turns the bytecode into mission logic. At
0xF524 Stage02.ssb polls unit_state on ADN110, ADN111 and ADN112, updates each
one's objective marker, then latches set_flag(8) -- exactly the
trigger/predicate/set_flag/END_PHASE shape predicted from the disassembly, now
observed in the mission's own code with names the roster tables already gave.
The 12 END_PHASE sites are outro sequences (wait_cmds_drained / fade_sound(3) /
builtin85(3) / wait_s(3) / END_PHASE / yield) -- the terminator, not the
decision.
Fixes a decode bug that hid every argument: the tracker only followed
local[i] = special[0], but the common form is an immediate written straight into
local[i] (k=01,03), so every unit predicate printed with NO arguments. The
disassembly looked complete while being empty exactly where it mattered.
Also records the live probe result: the phase mirror at [*(0x828F35F8)+236]
stayed 0 for ~530s of actively-hunting flight, no advance observed -- which is
what the static analysis predicts for phase 1, since ChangePhase only posts once
the ordinal exceeds 1.
Table at 0x8227226C is 147 big-endian absolute VAs (verified structurally: it
ends exactly where the first handler begins, all targets inside sub_82272220).
Arguments are not in the instruction -- every handler does c_str() on
[phase+20], a packed blob, which is what the local[] staging fills. Return 2 =
yield; five built-ins block by skipping the pc advance.
Recovered the ScriptPhase state layout: 32-entry float and flag register files,
int/double result registers, the timer block, and the runtime unit array at
+324 indexed by symbol-table-2 index -- a direct hook from bytecode call sites
to the two .ssb symbol tables.
Spot-checked two claims against the disassembly rather than trusting them: id 4
loads a DOUBLE into the thread countdown and returns 2 (wait_s), and id 24 reads
current/initial squadron member counts (squad_survival_pct). Both exact.
Counting Stage02.ssb: unit_state 255, hp_pct_test 167, dist_lt 92, unit_alive
71, unit_relation 52 -- and squad_survival_pct, group_ratio_pct and the two
global counters are NOT called at all. So Stage 02's phases are gated on named
units (destroyed / HP / proximity), never on an aggregate count, even though the
kill-counter primitives exist in the VM. That answers the standing 'next wave
after N kills or after an event?' question for this stage: specific units, not a
number.
isl.py now names the built-ins, so the run-up to the first END_PHASE reads
wait_cmds_drained / fade_sound(3) / builtin85(3) / wait_s(3) / END_PHASE.
Not settled: 3 handlers unresolved (55, 75, 105); the 1024-slot interpreter
command table is only partly recovered.
Resolver table 0x82271D74 gives four kinds: 0 global[i], 1 immediate,
2 special[i] ([phase+164]/[phase+168]), 3 local[i] ([phase+20+i]). Byte[0] is
the rvalue kind, byte[1] the lvalue kind, so the recurring instruction pair is
argument staging -- values land in local[] at offsets 0,4,8,0xC and the next
call consumes them. A built-in's arguments are not in its own instruction.
Fixed a decode that would have been believed: immediates in set.f are DOUBLES
carried as two words (op 1 stores with stfd). Reading the high word as a float
gives 2.125 where the script means 3.0.
isl.py now tracks staging and prints call arguments, so the run-up to the first
END PHASE in Stage02 reads as builtin=64(0x42,2,1,9,1,-1) / 120 / 59(3) / 85(3)
/ 4(3) / 6. Three built-ins taking 3 just before the phase ends look like a
wait-seconds family -- flagged as unconfirmed until the built-in table is read.
Read the encoding off the interpreter rather than guessing: instruction is a
big-endian u32 whose LOW byte is the opcode (25 of them, table 0x822635FC),
byte[2] is the instruction length -- every handler advances the pc by it -- and
bytes[0..1] are operand kinds. Op 12 is a jump whose operand is relative to the
code base [phase+232], which settles that offsets are code-base-relative for
this opcode. Op 19 is the built-in call: id in word@+4, and word@+8 is a
monotonically increasing STATEMENT id (0x245, 0x248, 0x24A, ...).
Confirmed by disassembling Stage02.ssb: the stream decodes cleanly from the code
base and routines terminate on ret exactly where expected.
Scanning the code region on the call encoding: 2846 call sites, 73 of the 147
built-ins used. The phase-control ones are located -- built-in 6 (end phase) at
12 sites, 62 at 3, 39 (mark last phase) at 8 -- so a phase has several exit
paths, as a mission with win and lose branches should.
New tool tools/re-capture/isl.py with --calls and --to (resync-into-target,
needed because instructions are variable-length so you cannot walk backwards).
Not settled: the 147 built-ins are uncharacterised, so this is structure without
meaning -- we can see THAT a phase ends, not WHAT was tested.
mission-phase-advance.md recorded the bytecode as not on the disc. It is; the
earlier grep failed because every pak entry is Z1+zlib, so no name is plaintext.
The loader resolves table KEYS, not a filename: GamePart name -> GP_SCRIPT ->
script.tbl (name_hash 0x75FE4656), whose SCRIPTS record is a 40-field manifest
mapping MISSION1..MISSION29 to StageNN.ssb. Verified: Stage02.ssb is 226,596
bytes, md5 aff69b5a..., byte-identical across all six language paks.
28 scripts exist; S17 is the missing one -- which agrees with the table sweep
(S17 in none of the five families) AND with the loader guard
sub_8225EC78 'if (n == 16 || n > 32) return', proving that mission number is
0-based. Three independent routes to the same conclusion.
Header decoded from ScriptMission::Load rather than guessed: version, code
offset 0x24, two symbol tables. Stage02 symtab1 = 326 symbols (197 message ids,
119 route names, 10 subobjectives), symtab2 = 122 (111 unit ids, 11 characters).
Two loops close: the 111 unit ids are exactly the 111 UnitGroup_S02 squadrons,
and the route names appear as SCRIPT SYMBOLS -- which is why nothing in the
executable parses Route_*_p<N>* names. The _pN convention is the script's.
Refuted: the .embsec_ sections are code (32,368 instructions, 108 functions),
not script; MiscBin.pak and DefTables.pak have zero hits.
Open: the 25 ISL opcodes and 147 built-ins are not decoded, and the
mission-level stream at +0x24 is only partly read.
Swept all 33 dat paks plus hidden/DefTables.pak: 7,750 IDXD objects, 190,782
records, 1,271,462 (name -> tag) pairs, 0 failures, 0 parse failures.
The key rule is stronger than first written: a record's key is tag_hash of its
own inline name, 190,782/190,782 -- no roster needed (roster-based recovery only
reaches ~30% of records). Verified independently here on UnitGroup_S02, 112/112.
The modulus is uniquely pinned, not just consistent: gcd of
poly256(name) - (tag & 0xFFFFFF) over all pairs is exactly 0x00FFFFDF, prime.
Case-sensitivity is load-bearing, not incidental: the disc has 17 name pairs
differing only in case (UNIT/Unit, TYPE/Type, STAGE/Stage, ...); name_hash
collides on all 17, tag_hash separates all 17.
Records the limit too: IXUD, the wide-string sibling, uses a DIFFERENT low-24
hash (0/19,808) though it keeps the same top-byte checksum (19,808/19,808). Its
offsets are counted in 16-bit chars (STR + 2*strsize == filesize) -- newly
decoded, all 534 parse. Its low bits are provably not a modular polynomial (gcd
= 1; exhaustive Barrett search over M in [2^20,2^25) finds nothing; fixed-position
deltas are non-constant), so it is lossy and needs the XEX.
Closes the 4-byte record key. tag_hash is name_hash's shape -- byte-sum
checksum in the top byte over a 24-bit modular polynomial -- with two different
constants: modulus 0x00FFFFDF (2^24-33, prime) instead of 0x00FFF9D7, and no
lowercasing, so tags are case-sensitive. name_hash explains 0 of 8643.
Recovered from the tables rather than the executable: every inline field name
is a known (name -> tag) pair, and comparing names differing in one character
gives the per-position weights 1, 0x100, 0x10000, 0x21, 0x2100, ... -- a byte
leaving bit 24 re-enters as 33, i.e. reduction mod 2^24-33. Holds where it is
easy to get wrong (distance 8 and 9 carry correctly).
A record's key is the tag of its own name: FormationSet rosters 362/362,
UnitGroup rosters 281/281, S02 squadron names 111/111 -- so records can be
addressed by name without reading the roster first.
Implemented in Python (unitgroup.tag_hash) and Rust
(sylpheed_formats::hash::tag_hash) with 3 new unit tests carrying disc-derived
vectors; cargo test -p sylpheed-formats --lib hash is 8/8 green.
Not settled: the guest routine is unlocated, so this uses exact modular
arithmetic where the game may use a Barrett step without final fixup.
FormationSet_S<NN>.tbl records are slot lists -- 1 + 8*FrameCount fields,
exactly. Resolving every squadron's FormationID and comparing gives
sum(n) <= FrameCount holding 1159/1160 across all 28 stages, 0 unresolved, with
539 filling the formation exactly. The single violation is a debug leftover
(S20, AI_Test / MessageSet_test, Formation_1_only with n=2) and is recorded.
The old 'n is not the _NN suffix of FormationID' observation was right but drew
the wrong conclusion: the suffix IS FrameCount, so n=9 against _30 just means 9
units in 9 of 30 slots.
Also: FormationID does not hash into its table (0/16). FormationSet carries a
name roster record -- no FrameCount, fields are (tag, name, '') with the tags
being the record keys -- the same convention as Enumerate_Squadrons. Second
occurrence of 'keys are resolved by an in-table roster, not by hashing'.
Does not close the 387-vs-300 gap, and the key derivation stays open.
The 'S17-S23 have no stage record' gap was an artefact of enumerating by the
literal 'Stage_S<NN>'. Tutorial records omit it -- they name no per-stage .xpr
and pull AIParams/weapons/strings/subobjectives/nameplate/collision from a
shared _Tutorial set -- so all six were skipped. stagetbl.py Stage_S18 returns a
full six-record definition and always would have.
Counting distinct *_S<NN> names across all 1119 decompressed entries: UnitGroup
and Route cover 28 stages (S01-S16, S18-S23, S24-S29); Stage literal and
AIParams cover 22; SUBObjectiveSettings 16 (story only). S17 appears in none --
it is not a stage that lost its data, it does not exist.
Tutorial records carry the same Phase_1/2/3 structure as story stages, so the
tutorial is not a special mission type at the data layer.
Refuted en route: GP_TUTORIAL.pak does not hold the config -- 2 RATC entries,
zero IDXD, like GP_CHALLENGE.pak.
