Commit Graph

6 Commits

Author SHA1 Message Date
sim
120621daca fix(tools): every sylpheed.db reader honours $SYLPHEED_DB, and none needs a retired checkout
`SYLPHEED_DB` is the documented contract for the static-analysis database —
`docker/decoder/sylph-decoder` sets it, and `docs/agents/CONTAINER-NOTES.md` and
`decoder-loop.md` list it — but no reader honoured it:

  isl_cmdtab.py, name_block_bases.py   hardcoded `/work/xenia-rs/sylpheed.db`, a
                                        path that has not existed anywhere since
                                        `/work` became a clone. Both failed on
                                        every machine, before and after the
                                        database moved.
  zq.py                                 used `$SYLPH_XEXDB`, a name invented in
                                        #39 without grepping for the existing one.

All three now resolve `$SYLPHEED_DB`, then `$SYLPH_XEXDB` (kept as an alias so
nothing already written against it breaks), then `<repo root>/sylpheed.db`, and
refuse with exit 1 naming both variables and the build command otherwise.

`grab_tutorial.sh` hardcoded its helpers into the retired `sylpheed-reborn`
checkout. That copy's `skip_intro.sh` still calls the removed `vgamepad` and
exits 0 having pressed nothing — the failure this repository's own copy was
rewritten to make loud. So the script was already running a silently broken
helper; it now resolves its helpers from its own directory. Nothing exists only
in reborn's `tools/re-capture` (checked: 0 reborn-only files).

Verified against the same database, output compared byte for byte with the
ORIGINAL scripts (path-substituted copies, sibling imports resolvable):

  resolution path    isl_cmdtab   name_block_bases
  default (root)     identical    identical
  $SYLPHEED_DB       identical    identical
  $SYLPH_XEXDB       identical    identical
  bad path           exit 1       exit 1          (zq.py: exit 1 too)

and `isl_cmdtab`'s output is byte-identical to the body of the committed
`docs/re/data/isl-command-table.txt`.

⚠️ `name_block_bases`'s output does NOT reproduce the committed
`docs/re/data/name-block-bases.txt` (2,609 lines differ, `strings in the image:
7366` vs `7140`). That is the database, not this change: the artefact was
generated from the agent box's older 586 MB database, and this machine's is the
current generator's. Two databases are in circulation. Not regenerated here.

⚠️ Also not fixed, because it is a different bug: `name_block_bases.py` globs
`/work/sylph_extract/**/*.pak`, another path that exists nowhere now; that part
of its report is silently empty. It should read `$SYLPHEED_DISC`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 20:07:19 +02:00
Sylpheed RE agent
4a042bda53 re: the base-solver's confidence axis was inverted
The remaining named false-positive mode - "107 rows solve to a 64K-boundary
base, a bare addis with no addi of its own, so any scatter of displacements
votes for it" - is refuted by its own measurement.

New positive test in the tool: simulate lis/addis rD,r0,HI + addi rD,rA,N +
or rD,rA,rA forward through each row's function and ask whether the solved
base lands in the solved register.

  64K-boundary bases ("low confidence") : 107 / 107 confirmed
  non-zero low half ("trustworthy")     :   8 / 154 confirmed

A round base is the case where the compiler needed no second instruction, so
`addis r11, r0, 0x820B` stands in the code in full. A miss on the other class
is silence (base built in the caller or loaded from memory), not refutation.

Control: every row the corpus independently validated against the disc has a
64K-boundary base - debriefing, career, save, leaderboard, the 205-name PG*
HUD roster, material slots, the S16 boss collision/frames/motions and its
loader. 13 rows over 10 functions. The dense-short-string false positives the
corpus did name (r31 = 0x8202xxxx) all sit in the "trustworthy" class.

The 0x820B0000 cluster is DUPLICATION, not error: 60 of its 82 rows are one
function emitted 60 times, exactly 491 instructions each, two instructions
differing (both global data pointers), identical 41-address string sequences.
40 resource names written into a per-copy global via sub_8217FA08 at 24-byte
strides. 38 of the 40 are disc GameResourceID values (480 distinct); rot_n001
and rou_e202 are not, and no disc GameResourceID uses the rot_ prefix.

Artefact diff 13/4, confined to the replaced section; the 261-row table and
the 64K histogram untouched; byte-identical on a second run. Fourteen other
artefacts byte-identical.
2026-08-28 00:42:46 +00:00
Sylpheed RE agent
198273110b re: r0 is not a base register - the last shortlisted row withdrawn
sub_825F2CF0 / sub_825F2F88 (r0, 30 names each, 97% disc names) are not
name-block readers at all: they are the compiler's VMX save/restore helper
pair. 145 instructions each, 72 differ, every difference stvx<->lvx over
v14..v127.

Root cause: in `addi rD, rA, N` the rA slot reads as literal zero when it
names r0, so `addi r11, r0, -N` is `li r11, -N` - vector spill offsets, not
displacements. The solver excluded r1 only; the bogus base landed inside the
stage-settings name block, dense enough to "resolve" 30 real field names.

name_block_bases.py now skips r0 as well. Rows 277 -> 261, functions 190 ->
176, non-64K bases 170 -> 154, data-table rows 53 -> 50; 154+107=261 and
277-16=261. Artefact diff -100/+3, every removed row line carries r0, control
sub_82341A20 r30 = 0x82088F94 217/226 untouched.

Nothing lost: all 30 names are a strict subset of sub_8230D1F8 r29 (129
names), the stage-settings loader already in the corpus. String-xref join
agrees - FinalPassBG, FogMinDistance, ScreenColorR, ExposureKey_BeginValue
each exist once with exactly one xref, from sub_8230D1F8.

Fourteen other artefacts byte-identical; name-block-bases.txt byte-identical
on a second run.
2026-08-28 00:21:26 +00:00
Claude (auto)
187b10418e re: all 277 base-solver rows classified objectively; 53 are data-table schemas
name_block_bases.py extended with a per-row data-table test; artefact +57/-0,
byte-identical across two runs (now ~2 min 12 s -- it adds a disc-wide pak scan).

The test is objective, not by eye: a row is a data-table schema if its names are
IDXD record/field names on the disc (13450 such names disc-wide).  53 of 277 rows
are >=50 % disc names with >=8 names; the other 224 are engine/XDK vocabulary,
compiled key lists, or noise.

The two axes are independent: against base confidence, solved bases split 34
table / 136 not, round bases 16 / 91.  "Round base" and "not a table" are
different questions.

The 53 contain every loader already known -- that is the control.  Five rows in
the 53 are unowned, each noun grepped and appearing in no docs/re/ file:
sub_823BDAA8 r11 (33) = the S16 boss's muzzle/attach frames (GN_MainGun_*_Muz*);
sub_823BDAA8 r10 (25) = motion names (Motion_stand, Motion_attackA_start), the
EnumMotions family DefTables declares; sub_82315AE8 r11 (20) = the Guardian
record's own fields, i.e. the S16 boss loader; sub_8219E560 r11 (18) = the
leaderboard screen keys; sub_825F2CF0 + sub_825F2F88 r0 (30 each, same base) =
post-processing (FinalPassBG, FogMin/MaxDistance).

Four rows that look new are not, and their disc-overlap says so -- 53-70 % rather
than ~100 %, because they mix arsenal fields the corpus owns (ConditionToDevelop,
WeaponDesc, SilhouetteModel) with literal screen coordinates as strings.

Not settled: none of the five was opened -- this iteration produced the shortlist,
not the findings.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 23:09:57 +00:00
Claude (auto)
5c672b11e0 re: mining the base-solver index -- a false-positive mode named, and the AI-table reader found
277 rows over 190 distinct functions (a function can read several blocks).
name_block_bases.py extended with a confidence split.

The tool's false-positive mode, measured and named: 107 of 277 rows solve to a
base on a 64K boundary -- a bare "addis rX, r0, 0xHHHH" with no addi, so any
scatter of displacements votes for it.  82 are 0x820B0000: about 60
near-identical functions in 0x8281xxxx-0x8284xxxx all "naming" the same rou_e0NN
list.  The 170 rows with a non-zero low half are the trustworthy set.  A round
base is not automatically wrong -- sub_822215D0 sits on 0x820A0000 and resolves
205/206 -- so read the ratio, not the base.

The index re-derives every loader we already knew (unit 217, stage settings 129,
PlayerParams 90, hangar 81, squadron orders, missile guidance, shell movement,
substructures, six camera/fog readers) -- that is the control.

The find: sub_8233C368 reads the AI behaviour table.  r28, base 0x8208583C, 20
names -- Enumerate_AIs, FiringLength, GuardLength, AutoGuardLength, CounterLength,
MusterLength.  stage-mission-tables.md owns those field names on the data side,
but Enumerate_AIs appears in no document and no reader was known; the corpus
carries the AI tail of Maneuver as NEEDS-HUMAN/runtime.  It is statically
reachable after all.  The same base also serves sub_82338EE0 (97 names, Weapon
TargetType SpecialWeaponType ReticleType IsCharging ...) -- the weapon datasheet
loader, also not previously named.

Five unowned blocks surfaced and NOT opened: PGHUD_*/PGREMAIN_NUM HUD part names
(205/206), STAGE_RESULT/stage_num_shoot_down_aircrafts/EX_OVERVIEW,
g_mWorldViewProjection/NormalMap/GlossinessMap engine material slots,
Boss16Collision* (cross-links the S16 Guardian object), and roh_n001_menu1_cam_pos
menu camera tags.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 21:41:03 +00:00
Claude (auto)
15e6a4c498 re: the base-solver -- 277 name-block loaders indexed, and the analog block is SOLVED
New reusable tool, tools/re-capture/name_block_bases.py -> docs/re/data/name-block-bases.txt
(2880 lines, ~65 s, byte-identical across two runs).

A loader that reads a table by field name keeps one base pointer and emits
"addi rX, rBASE, -N" per name, so no static xref sees the strings.  Solve the base
from the DISPLACEMENT SET alone: every (string address, displacement) pair implies
a candidate base, and the true base collects a vote from every name it explains,
so it wins outright.  My first cut took candidates from ONE displacement and
scored the unit loader at 52/226 against the right answer's 217/226 -- vote over
the whole set, not a probe.

Control passes with no prior knowledge: the tool recovers sub_82341A20 -> r30 =
0x82088F94 at 217/226, and independently recovers sub_8230D1F8 (129/132),
sub_822F9498 (90/91) and sub_822AE628 (81/108).  277 name-block-reading functions
image-wide, with the schema each names.

The analog block is SOLVED: sub_821A6CF0, r29 = 0x820A1630, 22/24.  In code order
it names ControlTweakName, YawMagForNormal, the 12 Tweak fields, the 8
AnalogRevice_* curves and GP_MAIN_GAME -- the whole schema in the object's own
order plus its pak.  r29 is built at 0x821A6D34 as addi r29, r11, 5680 =
0x820A0000 + 5680, matching the solved base exactly.  It is the same function that
reads PlayerParams.

Two of my own verdicts withdrawn: "referenced by nothing" and the softened "not
found by these routes".  The measurements behind them were right; the conclusions
were wrong.  The base was solvable from the data the whole time.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
2026-08-27 21:22:50 +00:00