diff --git a/docs/re/BACKLOG.md b/docs/re/BACKLOG.md index 38236d0a..82296c63 100644 --- a/docs/re/BACKLOG.md +++ b/docs/re/BACKLOG.md @@ -168,7 +168,15 @@ the emulator-era work (`sub_821CC7A0`, item vtable `0x820b30b4`) submits with exactly the PS hash `E59B2B3D` this capture sees, and `xenia-rs/sylpheed.db` is available in the container. -**And a second screen is BLOCKED**, which is what this item wanted next: the +**And a second screen is NO LONGER BLOCKED, but it is not routine either.** The +main menu has been reached (screenshot in +[`canary-scripted-input-traps.md`](canary-scripted-input-traps.md)), so the +"Ⓐ is dead" reading is withdrawn; what remains is that Ⓐ advances the title only +intermittently — one success in about four tries, with the press verifiably +delivered each time. A reliable route to the menu is the next cheap step, and it +is what the second screen's capture needs. + +The earlier reading, kept because it is what the evidence looked like: the title's Ⓐ leads into a content/save path that crashes the guest with `--mem_watch=true` and stalls it with `--mem_watch=false`. Three separate traps had to be cleared to establish that much — see diff --git a/docs/re/INDEX.md b/docs/re/INDEX.md index 9b118e9f..e5c791c5 100644 --- a/docs/re/INDEX.md +++ b/docs/re/INDEX.md @@ -26,7 +26,7 @@ Promote to a prose `structures/…md` file when a format needs behavioural notes | Arsenal develop economy | ✅/❔ | [arsenal-develop-economy](arsenal-develop-economy.md) + [conditions](captures/arsenal-develop-conditions.csv) | The Arsenal reads `weapon.tbl` (item ids, in the 8-category display order) and `strings.tbl` (names, descriptions, and a **"Conditions to obtain"** block per item) out of `GP_HANGAR_ARSENAL.pak`. All **60** conditions are extracted: gates are stage completion, a predecessor item, or an **ace kill**; costs run 3 000–350 000 P and **20 items are free** once gated. `weapon.tbl`'s first record reproduces the in-game DATA SHEET exactly (Range D / Power E / Speed – / Weight 0.3 = Light / 4000 P) — later records are unreadable from the string pool alone because IDXD **dedupes repeated values**. Used to identify the save blob's index space, now **solved**: the blob follows **`strings.tbl`'s** order — the display order *plus* the cut items only the localisation file lists (`Adhesive Mine B2A`, `Ballista GSH`, …) — pinned by four hand-written probe saves (9 Stiletto, 21 Falcon, 39 Tomahawk, 48 Jamming System) and closing exactly at index 53. `weapon.tbl`'s id list is **not** the index space; that it is also 54 long is a coincidence, and the two agree only to index 32. The retail save's five unexplained owned entries are the cut items, shipped owned and never rendered | | UI screen layout (`.rat`) | ✅/🟡 | [ui-rat-layout](structures/ui-rat-layout.md) | One pak per UI screen; each RATC = one (context × language) build; every `.t32` sprite has a `.rat` **layout record** (BE u32; 1280×720 design space; scale/tint/X/Y, keyframes for animated elements, `opt ` link to the focused state). **The tutorial PAUSE menu and the title main menu both rebuild pixel-accurately from the disc.** `loop1.rat` is decoded — it is a **looping sprite animation**, not a composition. ⚠️ **DEMOTED 2026-08-18** — the declaration table is *not* the paint order: a per-draw capture of the running title screen ([ui-title-paint-order-capture](ui-title-paint-order-capture.md)) paints element 13 first and elements 0/1 late, and the visible screen composites two bundles. The rest of the table's reading stands. Previously claimed: the **screen's draw list is the RATC bundle's own declaration table** (elements in back-to-front order, including the `eff*`/`deli*`/`msg` sprites that have no `.rat`, and excluding focused button variants reached via `opt `); its entry also carries a **parent element index** at `+32`. **A screen is fully reconstructible from its bundle**: the placement region right after the declaration table gives every element a keyframe group (header = element index + keyframe count, then 40-byte blocks of scale/tint/X/Y), including the `.rat`-less sprites — verified 11/11 on the tutorial pause bundle, with `pgp_ttrl_btn10`'s inline (546,288) matching its own record exactly | | UI screen paint order | ✅/🟡 | [title paint order](ui-title-paint-order-capture.md) | **Measured from the guest's draw stream** (new Canary hook `log_ui_draws`, F10): the title screen paints 11 draws a frame through one sprite shader, and the submission order is background → `back2` glow → `ptlogo1`+`tm` → `ptlogo2` → `ptcopyright` → the `PRESS Ⓐ BUTTON` plate. That refutes declaration order, reverse order, keyframe-start order and resting-time order, and shows the screen composites **two bundles** (build 7 + the one-element build 2). 🟡 the rule that produces the order is still unknown | -| Scripted input / profile traps | ✅/🔴 | [canary-scripted-input-traps](canary-scripted-input-traps.md) | Three reasons a scripted run "cannot press Ⓐ": **F10 opens the emulator menu bar**, which makes `IsUIActive()` true and zeroes every guest keystroke; the file-pad is not the problem (it delivers, and Ⓐ skips movies); and the title needs a **signed-in profile**, which the container cannot type into ImGui — hence Canary's new `--create_profile_if_none`. 🔴 Past that, Ⓐ crashes the guest with `--mem_watch=true` and stalls it with `--mem_watch=false` — a cheap second reproduction of the Ready-Room crash, 40 s from a cold boot | +| Scripted input / profile traps | ✅/🟡 | [canary-scripted-input-traps](canary-scripted-input-traps.md) | Why a scripted run appears unable to press Ⓐ: **F10 opens the emulator menu bar**, and any Xenia UI makes `XamInputGetKeystrokeEx` return SUCCESS with an empty keystroke *before* any driver is asked (Canary now logs `[RE-INPUT] … swallowed by IsUIActive`); the title needs a **signed-in profile** (hence `--create_profile_if_none`); and a **FIFO trace consumer that exits stalls the emulator**, which reads exactly like a dead pad. 🟡 The main menu HAS been reached — Ⓐ works, but only intermittently (1 in ~4), which is the open question | | Title-screen guest crash | ✅ | [title-crash-stl-tree](title-crash-stl-tree.md) | The guest throws **`std::out_of_range`** from its cache-manager flush (`sub_823070B0`, an STL map/set erase that builds `'invalid map/set iterator'`); the access violation after it is only the throw **returning**, because this build does not unwind guest EH. Trigger found and controlled: an **incomplete on-disc cache** (`~/.local/share/Xenia/cache/aab216c3`) throws ~100 s into a boot, a complete one never does — 2 runs each way. ❌ `mem_watch`, the handoff's suspect #1, is **eliminated**: cold cache + `--mem_watch=false` throws anyway | | Save file (`savedata`) | ✅/❔ | [savegame-format](structures/savegame-format.md) + [`tools/re-capture/savegame.py`](../../tools/re-capture/savegame.py) | `GDHA` container, zlib payload, chunk stream (`GDAA` / phase name / `GHAD` 122 B progress block / 16×20 B slot table / trailer). **Container and layout read off the title's own serializer `0x822C00E8` and verified by a byte-identical round-trip**; the whole save is 545 B. Payload offsets are also the live save object's offsets (`save+8` GHAD, `save+136` slots). A second save made in-game names **Points** (+24), **flight time in ms** (+4) and **clear ratio %** (+8) off the game's own Details panel; the payload is a **pure function of game state** (same state saved twice = byte-identical, only the header FILETIME and its uninitialised pointer padding move), and the 16 `SHAB` records are **not** the UI's 20 save slots. Difficulty vs stage is undecided — three fields hold 2. **A third save, taken after developing exactly one Arsenal weapon** (Light Machine Gun MG I, 4000 P), moves exactly three things: `+24` Points 4101→101 (which **separates it from `+28`**, that did not move), `+8` clear ratio 5→6 (so the ratio counts *collection*, not only stages), and two entries of the 54-byte blob — `2→4` for the item bought and `0→2` for the successor the game announced as newly developable, giving the blob its alphabet ✅ *0 locked / 2 developable / 4 developed* (only the `4`s are stored — `2` is re-derived at load). **Saves can also be written back**: three derived header fields (length at `+0x30`, payload length at `+0x8c`, `adler32` at `+0x8e`) are all that stand between a parse and a hand-written save that the title loads, and [`savegame_edit.py`](../../tools/re-capture/savegame_edit.py) re-wraps a real save byte-identically. That turned the blob's index space from blocked-on-story-progress into four probe saves — see the [economy note](arsenal-develop-economy.md) | diff --git a/docs/re/canary-scripted-input-traps.md b/docs/re/canary-scripted-input-traps.md index b84506e6..0e2f2278 100644 --- a/docs/re/canary-scripted-input-traps.md +++ b/docs/re/canary-scripted-input-traps.md @@ -64,63 +64,55 @@ B13EBABEBABEBABE) to slot 0`, and the Profiles Menu panel shows the user The profile persists on disk, so later runs need only `--logged_profile_slot_0_xuid=B13EBABEBABEBABE`. -## The blocker: what happens *after* Ⓐ +## Withdrawn: Ⓐ is not blocked — it works, intermittently -With the profile created and signed in — Xenia's own **Profile ▸ Show Profile -Menu** confirms it, `User: SylphRE / XUID: B13EBABEBABEBABE / Assigned to slot: 1` -(the panel counts from 1; the log says slot 0) — Ⓐ at the title **still does not -advance the screen**. Measured across four boots, including one that was allowed -to play the whole intro untouched, with holds of 10 s and bursts of five taps. +**The main menu has been reached** +([`captures/main-menu-reached.png`](captures/main-menu-reached.png) — NEW GAME / +LOAD GAME / TUTORIAL / OPTIONS / EXTRAS). So the previous version of this entry, +which called the title's Ⓐ a hard blocker, is **withdrawn**. What is true is +narrower and more annoying: **Ⓐ advances the title only sometimes** — one success +in roughly four attempts, with the press verifiably delivered every time +(`[file-pad] keystroke vk=5800 down/up` in the log) and no Xenia UI active. -What is *not* the explanation, each checked: +The successful press landed on the **first** title after boot, before any attract +cycle; the failures were on a title that had come round again. That is a +hypothesis, not a rule — a later attempt on a first title also failed (the screen +flickered to something else for one sample and came back), so timing is involved +and the window has not been pinned down. -* the emulator menu bar being open (trap 1) — the grabs are clean; -* the pad (trap 2) — Ⓐ skips movies in the same runs; -* the profile being absent (trap 3) — it is present, signed in, and the Sign In - dialog no longer appears; -* an ImGui window being open — but note this generalises trap 1: **any** Xenia UI - window makes `IsUIActive()` true, the Profiles Menu included, so a panel left - open silently disables the guest's input for as long as it is up. +## What was eliminated on the way, so nobody re-walks it -**One observation that did not reproduce, recorded rather than dropped.** In the -run where the Profiles Menu had just been closed, a single Ⓐ **crashed the -guest** — PC `0x824578A0`, guest thread 6, distinct from the boot-path crash -below. Two later attempts to reproduce it (open the panel, close it, press Ⓐ) -produced neither a crash nor a transition. So it is one data point that Ⓐ *can* -reach code the title does not otherwise run, and nothing more. +* **`IsUIActive()` is now observable.** `XamInputGetKeystrokeEx` returns + `X_ERROR_SUCCESS` with a zeroed keystroke *before consulting any driver* when a + Xenia dialog is up — an invisible input blackout. Canary now logs + `[RE-INPUT] XamInputGetKeystrokeEx swallowed by IsUIActive (dialogs=N …)` when + that happens. On the runs where Ⓐ failed, **it never fires**, so the blackout + is not the cause there. (It *is* the cause after F10 — trap 1 above.) +* **The driver filter is fine.** The game polls with `flags = 3`, and + `InputSystem::FilterDrivers` keeps a driver when `flags & InputType` is + non-zero; the file pad reports `Controller = 1`, so it is selected. +* **The game does not query sign-in on the press.** Tracing every `Xam*` call + around a press shows only `XamInputGetCapabilities`, `XamInputGetKeystrokeEx` + and `XamInputGetState` — no content, user or signin call at all. -**The reproducible crash is the cache flush, and it is not what it looked like.** -With the default `--mem_watch=true`, in two runs, the guest died with an access -violation at PC `0x82307128` right after `RtlRaiseException(E06D7363)`. Chasing -that produced a correction and a cheap trigger, both in -[`title-crash-stl-tree.md`](title-crash-stl-tree.md): +### Two traps in the measuring rig itself, which cost more than the bug -* the access violation is a **consequence** of the guest's own `throw` returning - (this build does not unwind guest EH), so the event to study is the throw; -* the throw is `std::out_of_range` from the game's **cache-manager flush**, and - an **incomplete on-disc cache** triggers it ~100 s into a boot; -* the observation that started this — "crash with `--mem_watch=true`, none with - `--mem_watch=false`" — was **confounded** by cache warmth, and is withdrawn: - cold cache with the probe off throws just the same. +* **A FIFO trace consumer that exits stalls the emulator.** `--phase_a_event_log_path` + writes 1.5 GB a minute, so it was pointed at a named pipe with a `grep` on the + other end. When that reader goes away, the emulator blocks on the write — and + the symptom is *exactly* "the pad is dead": the guest stops polling, the driver + never refreshes, nothing is logged. Two runs' worth of "the driver never + reacts" was this, not the game. +* **The trace's `return_value` is a placeholder.** `phase_a_bridge::EmitReturn` + is called as `EmitReturn(name, 0)` — literally zero — so every `kernel.return` + event reads `"status": "0x00000000"`. "Every keystroke call returns SUCCESS" + looked like hard evidence for the `IsUIActive` theory and means nothing. The + theory had to be tested with a real log line instead. -**The first of those next steps is done**, and it moves the crash out of this -entry into its own: `0x82307128` is inside an MSVC `std::map`/`set` **erase** — -the function carries the string `'invalid map/set iterator'` and the node -offsets to prove it — which makes this a 40-second reproduction of the crash the -Canary handoff blames on the `mem_watch` probe. See -[`title-crash-stl-tree.md`](title-crash-stl-tree.md). +## Still open -**Next step, for whoever picks this up.** What is left: - -1. dump the tree nodes from guest memory at the throw, to separate "the guest's - tree is corrupt" from "the emulator left a stale high word in a register" — - the fault address `0x1_0000000C` is consistent with either; -2. the content exports this title imports — `XamContentCreateEnumerator`, - `XamContentGetDeviceState`, `XamUserReadProfileSettings` — logged around the - press, to see whether the guest is handed a bad enumerator or a device state - it does not accept. `\aab216c3\5\…` is the content directory in question. - -**What this blocks:** every dynamic experiment past the title — a second UI -screen's paint order, the main menu, mission captures. Everything up to and -including the title screen is unaffected, which is why the paint-order result -above could still be measured. +Why the acceptance is intermittent. The next step is not another guess: capture +the *first* title's Ⓐ under `--cache_throw_diag`-style logging of the guest's own +state machine, or simply script "tap Ⓐ once per second for 30 s on the first +title" and see whether the success rate goes to 1 — a cheap, honest experiment +that also gives a reliable route to the main menu for the UI paint-order work. diff --git a/docs/re/captures/main-menu-reached.png b/docs/re/captures/main-menu-reached.png new file mode 100644 index 00000000..36cb1e1b Binary files /dev/null and b/docs/re/captures/main-menu-reached.png differ