diff --git a/docs/re/BACKLOG.md b/docs/re/BACKLOG.md index 52e100e..e998eae 100644 --- a/docs/re/BACKLOG.md +++ b/docs/re/BACKLOG.md @@ -5553,3 +5553,67 @@ was a second one. **The shortlist is now empty.** Still open: whether a `stvx`- style helper elsewhere fooled a *non*-`r0` row (the 64K-boundary cluster is the remaining known false-positive mode, unaddressed); `Detail_Window_Known/_Unknown`; whether `Extra01…04` map onto the challenge missions. + +--- + +## 🔴✅ 2026-08-28 — the base-solver's confidence axis was inverted + +Item (a): the remaining named false-positive mode, "**107 rows solve to a base on +a 64K boundary — a bare `addis rX, r0, 0xHHHH` with no `addi` of its own, so any +scatter of displacements votes for it**". **Refuted, and by its own measurement.** + +**The test, now in the tool.** Simulate `lis`/`addis rD, r0, HI`, +`addi rD, rA, N` and `or rD, rA, rA` forward through each row's own function and +ask whether the solved base ever lands in the solved register — positive +confirmation from the instruction stream, independent of the displacement vote. + +* bases on a **64K boundary** ("low confidence"): **107 / 107 confirmed** +* bases with a **non-zero low half** ("trustworthy"): **8 / 154 confirmed** + +A 64K-boundary base is the case where the compiler needed **no second +instruction**, so `addis r11, r0, 0x820B` stands in the code in full. A non-zero +low half usually means the base was built in the caller or loaded from memory, +which the simulation cannot see — a miss there is silence, not a refutation +(the control `sub_82341A20 r30` is one such silence; it stands on 217/226 and on +the disc). + +🔑 **The control that settles it: every row the corpus independently validated +against the disc has a 64K-boundary base** — debriefing `sub_822814D8`, career +`sub_8227A3A0`, save `sub_82286BC8`, leaderboard `sub_8219E560`, the 205-name +`PG*` HUD roster `sub_822215D0` (r11 and r10), material slots + shader constants +`sub_822E3EC8` (r11 and r10), the S16 boss collision/frames/motions +`sub_823AE908` / `sub_823BDAA8` (r11 and r10) / `sub_823C0260`, and the boss +loader `sub_82315AE8`. Thirteen rows over ten functions. Meanwhile the +dense-short-string false positives the corpus *did* name — the +`r31 = 0x8202xxxx` rows — all sit in the class that was called trustworthy. +**The axis pointed the wrong way in both directions.** + +✅ **What the `0x820B0000` cluster actually is: duplication, not error.** 82 rows +carry that base and **60 are one function emitted 60 times** — exactly **491 +instructions** each, and diffing any two, **two instructions differ**, both a +global data pointer (`addi r30, r11, 14272` vs `15552`; `addi r3, r11, -13024` vs +`-12984`). All 60 reference an **identical sequence of 41 string addresses**, +writing 40 resource names into a per-copy global through +`sub_8217FA08(dest, name, -1)` at 24-byte strides. A static array duplicated once +per translation unit. The resolution ratio cannot show that, and the confirmation +test does not care. + +The roster is 30 `rou_*`, 6 `mob_*`, 3 `rob_*`, 1 `rot_*`, and **38 of the 40 are +`GameResourceID` values on the disc** (480 distinct, each used once). +[[collisionset]] owns the prefix vocabulary and the `GameResourceID` link; what it +does not say is that the image compiles a fixed 40-name subset. 🟡 The two that +are not disc `GameResourceID`s: **`rot_n001`** — and no disc `GameResourceID` uses +the `rot_` prefix at all (`eff` 197, `rou` 172, `mob` 71, `wep` 23, `rob` 4, +`stg*` 9) — and **`rou_e202`**, whose siblings `rou_e201` and `rou_e901` are both +present. + +Artefact diff is 13 added / 4 removed, confined to the replaced section; the +261-row table and the 64K-base histogram are untouched, and the file is +byte-identical on a second run. Written up in +`docs/re/structures/player-tuning-tables.md`. + +🟡 Not settled: what reads the 60 duplicated globals; what selects 40 of the disc's +172 `rou_*` ids; whether `rot_n001` / `rou_e202` appear anywhere else on the disc. +**Both of the solver's named false-positive modes are now closed — one real +(`r0`), one refuted (the 64K flag).** What remains unmeasured is the 146 rows the +positive test is silent on. diff --git a/docs/re/data/name-block-bases.txt b/docs/re/data/name-block-bases.txt index 82a4f4f..9a7f562 100644 --- a/docs/re/data/name-block-bases.txt +++ b/docs/re/data/name-block-bases.txt @@ -268,16 +268,25 @@ sub_82766DB0 r11 0x82050000 12 / 29 sub_827D7C00 r11 0x82060000 12 / 14 -## Confidence split - bases with a non-zero low half (a real `addis`+`addi` pair): 154 - bases on a 64K boundary (LOW CONFIDENCE, see below) : 107 +## Is the base built by the function itself? (positive test only) + rows whose function materialises the solved base into the solved register: 115 / 261 + bases on a 64K boundary : 107 / 107 + bases with a non-zero low half : 8 / 154 + A 64K-boundary base is `addis rX, r0, 0xHHHH` written out in full, so it confirms + directly; it is NOT the low-confidence class it was first called. A non-zero low + half usually means the base was built in the caller or loaded from memory, which + the simulation cannot see -- a miss there is silence, not a refutation. + + 64K-boundary bases by value: 0x820B0000 x82 rows 0x82030000 x10 rows 0x82050000 x7 rows 0x820A0000 x5 rows 0x82040000 x2 rows 0x82060000 x1 rows - ⚠ the 0x820B0000 cluster is ~60 near-identical functions in 0x8281xxxx-0x8284xxxx that all name the same `rou_e0NN` list. + ⚠ 60 of the 0x820B0000 rows are one function emitted 60 times: exactly 491 + instructions each, an identical sequence of 41 string addresses, differing only + in two global data pointers. That is DUPLICATION, not error. ## Is the row a DATA-TABLE schema? (names that are IDXD record/field names on the disc) distinct IDXD record+field names disc-wide: 13450 diff --git a/docs/re/structures/player-tuning-tables.md b/docs/re/structures/player-tuning-tables.md index 00e4f9d..092672b 100644 --- a/docs/re/structures/player-tuning-tables.md +++ b/docs/re/structures/player-tuning-tables.md @@ -360,3 +360,78 @@ region, and by the same mechanism). 🟡 What this does **not** settle: the 🟡 stalled `SpaceSize` / `GlareType` / `Nebura_*` numerics are no closer — their reader was already known to be `sub_8230D1F8`, and the r0 row never was a second one. + +## 🔴 CORRECTION (2026-08-28) — the confidence axis was inverted + +Having killed the `r0` rows, the other named false-positive mode was "**107 rows +solve to a base on a 64K boundary — a bare `addis rX, r0, 0xHHHH` with no `addi` +of its own, so any scatter of displacements votes for it**". That reasoning is +wrong, and the measurement says so. + +**The test.** The solver now simulates `lis`/`addis rD, r0, HI`, `addi rD, rA, N` +and `or rD, rA, rA` forward through each row's own function and asks whether the +**solved base ever lands in the solved register**. That is positive confirmation +from the instruction stream, independent of the displacement vote. + +| | confirmed | +|---|---| +| bases on a **64K boundary** ("low confidence") | **107 / 107** | +| bases with a non-zero low half ("trustworthy") | **8 / 154** | + +A 64K-boundary base is not evidence of a weak vote — it is the case where the +compiler needed **no second instruction**, so `addis r11, r0, 0x820B` stands in +the code in full and the base can be read off directly. A non-zero low half +usually means the base was built in the **caller** or loaded from memory, which +the simulation cannot see; a miss there is silence, not a refutation. (The +control, `sub_82341A20 r30 = 0x82088F94`, is one of those silences — it is +validated by its 217/226 resolution and by the disc, not by this test.) + +🔑 **The control that settles it: every row the corpus independently validated +against the disc has a 64K-boundary base.** + +| row | base | what it is, verified elsewhere | +|---|---|---| +| `sub_822814D8` r11 | `0x820B0000` | debriefing keys, 21 `STAGE_RESULT` fields | +| `sub_8227A3A0` r11 | `0x820B0000` | career keys, `OVERVIEW` 9 / `EX_OVERVIEW` 7 | +| `sub_82286BC8` r11 | `0x820B0000` | save-screen keys ([[savegame-format]]) | +| `sub_8219E560` r11 | `0x820A0000` | leaderboard, 13 records + 5 fields | +| `sub_822215D0` r11 / r10 | `0x820A0000` | the 205-name `PG*` HUD roster ([[hud-config]]) | +| `sub_822E3EC8` r11 / r10 | `0x820B0000` | shader constants + material slots ([[xbg7-mesh]]) | +| `sub_823AE908` r11, `sub_823BDAA8` r11 / r10, `sub_823C0260` r11 | `0x820B0000` | the S16 boss collision, frames and motions ([[collisionset]], [[stage16-boss]]) | +| `sub_82315AE8` r11 | `0x820B0000` | the S16 boss loader, 19/19 field names | + +Thirteen rows over ten functions. Meanwhile the dense-short-string false +positives the corpus *did* name — the `r31 = 0x8202xxxx` rows on +`sub_82341A20`, `sub_8230D1F8`, `sub_822F9498`, `sub_823AE908` — all sit in the +class that was called trustworthy. **The axis pointed the wrong way in both +directions.** + +### ✅ What the 0x820B0000 cluster actually is: duplication, not error + +82 rows carry base `0x820B0000`, and **60 of them are one function emitted 60 +times**. Each is **exactly 491 instructions**; diffing any two, **two +instructions differ**, both of them a global data pointer: + +``` + 9 828110FC addi r30, r11, 14272 | 8281250C addi r30, r11, 15552 +487 82811874 addi r3, r11, -13024 | 82812C84 addi r3, r11, -12984 +``` + +Their sequences of string addresses are **identical, all 60** — the same 40 +resource names in the same order, written into a per-copy global through +`sub_8217FA08(dest, name, -1)` in 24-byte strides. That is a static array +duplicated once per translation unit, not sixty tables. The index is inflated by +**duplication**, which the resolution ratio cannot show and the confirmation test +does not care about. + +The roster itself: 30 `rou_*`, 6 `mob_*`, 3 `rob_*`, 1 `rot_*`. **38 of the 40 +are `GameResourceID` values on the disc** (480 distinct, each used once) — +[[collisionset]] owns that prefix vocabulary and the `GameResourceID` link; what +it does not say is that the image compiles a fixed 40-name subset of them. +🟡 The two that are not: **`rot_n001`** — and no disc `GameResourceID` uses the +`rot_` prefix at all (`eff` 197, `rou` 172, `mob` 71, `wep` 23, `rob` 4, `stg*` 9) +— and **`rou_e202`**, whose siblings `rou_e201` and `rou_e901` are both present. +Two names the image carries that this disc field never uses. + +🟡 Not settled: what the 60 globals are read by, and what selects 40 of the 172 +`rou_*` ids. diff --git a/tools/re-capture/name_block_bases.py b/tools/re-capture/name_block_bases.py index f88b6a4..2bff445 100644 --- a/tools/re-capture/name_block_bases.py +++ b/tools/re-capture/name_block_bases.py @@ -90,20 +90,81 @@ def main(): for name, reg, B, tot, n in rows: print(" %-18s %-5s 0x%08X %d / %d" % (name, reg, B, tot, n)) - # Confidence. A base whose low half is 0x0000 is a bare `addis` with no - # `addi` of its own -- any scatter of displacements votes for it, so those - # rows are the tool's false-positive mode and must be read with the - # resolution ratio, not on their own. + # Does the function itself build the solved base into the solved register? + # Simulate `lis/addis rD, r0, HI`, `addi rD, rA, N` and `or rD, rA, rA` + # forward through the body; a hit is POSITIVE confirmation of the base. + # (A miss is not a refutation: a callee-saved base is often materialised in + # the caller or loaded from memory, which is why the solver exists at all.) + want = collections.defaultdict(set) + for name, reg, B, tot, n in rows: + want[name].add((reg, B)) + sim = con.execute( + "SELECT address,mnemonic,operands FROM instructions " + "WHERE mnemonic IN ('addi','addis','lis','or') ORDER BY address").fetchall() + body = collections.defaultdict(list) + for a, m, o in sim: + i = bisect.bisect_right(starts, a) - 1 + if i < 0 or funcs[i][1] <= a: + continue + if funcs[i][2] in want: + body[funcs[i][2]].append((m, o)) + confirmed = set() + for name, pairs in sorted(body.items()): + targets = want[name] + val = {} + for m, o in pairs: + q = [x.strip() for x in o.split(',')] + if len(q) != 3: + continue + if m in ('addis', 'lis') and q[1] == 'r0': + try: + val[q[0]] = (int(q[2], 0) << 16) & 0xFFFFFFFF + except ValueError: + val.pop(q[0], None) + elif m == 'addi': + try: + d = int(q[2], 0) + except ValueError: + val.pop(q[0], None) + continue + if q[1] == 'r0': + val[q[0]] = d & 0xFFFFFFFF + elif q[1] in val: + val[q[0]] = (val[q[1]] + d) & 0xFFFFFFFF + else: + val.pop(q[0], None) + elif m == 'or' and q[1] == q[2]: + if q[1] in val: + val[q[0]] = val[q[1]] + else: + val.pop(q[0], None) + for reg, B in sorted(targets): + if val.get(reg) == B: + confirmed.add((name, reg, B)) + round_rows = [r for r in rows if (r[2] & 0xFFFF) == 0] solved_rows = [r for r in rows if (r[2] & 0xFFFF) != 0] - print("\n## Confidence split") - print(" bases with a non-zero low half (a real `addis`+`addi` pair): %d" % len(solved_rows)) - print(" bases on a 64K boundary (LOW CONFIDENCE, see below) : %d" % len(round_rows)) + ok = lambda rs: sum(1 for r in rs if (r[0], r[1], r[2]) in confirmed) + print("\n## Is the base built by the function itself? (positive test only)") + print(" rows whose function materialises the solved base into the solved" + " register: %d / %d" % (ok(rows), len(rows))) + print(" bases on a 64K boundary : %d / %d" + % (ok(round_rows), len(round_rows))) + print(" bases with a non-zero low half : %d / %d" + % (ok(solved_rows), len(solved_rows))) + print(" A 64K-boundary base is `addis rX, r0, 0xHHHH` written out in full, so it" + " confirms\n directly; it is NOT the low-confidence class it was first" + " called. A non-zero low\n half usually means the base was built in the" + " caller or loaded from memory, which\n the simulation cannot see -- a miss" + " there is silence, not a refutation.") rb = collections.Counter(r[2] for r in round_rows) + print("\n 64K-boundary bases by value:") for B, c in sorted(rb.items(), key=lambda kv: (-kv[1], kv[0])): print(" 0x%08X x%d rows" % (B, c)) - print(" ⚠ the 0x820B0000 cluster is ~60 near-identical functions in" - " 0x8281xxxx-0x8284xxxx that all name the same `rou_e0NN` list.") + print(" \u26a0 60 of the 0x820B0000 rows are one function emitted 60 times:" + " exactly 491\n instructions each, an identical sequence of 41 string" + " addresses, differing only\n in two global data pointers. That is" + " DUPLICATION, not error.") # Is a row a DATA-TABLE schema or engine/XDK vocabulary? The objective test # is whether its names are IDXD record/field names on the disc.