diff --git a/docs/re/BACKLOG.md b/docs/re/BACKLOG.md index 12259628..6711e59f 100644 --- a/docs/re/BACKLOG.md +++ b/docs/re/BACKLOG.md @@ -143,6 +143,38 @@ been checked for one against this ground truth, and nothing yet explains how the two bundles are sequenced. Both are static questions again — the oracle side is answered. +### 2026-08-18 (third pass) — the bundle does not carry the order at all + +Three more places checked, all empty, so the static avenue for this item is +**exhausted** (detail and evidence in +[`ui-title-paint-order-capture.md`](ui-title-paint-order-capture.md)): + +- **the geometry has no depth.** A UI quad's attribute 0 is `k_32_32_32_FLOAT`, + so it carries a Z — and every Z in the capture is 0.00000. Submission order is + the entire ordering. +- **the declaration table has no key.** Every word of every entry dumped for the + build the game actually runs: `+28` 0, `+32` `0xffffffff`, `+36` `0xffffffff` + (except an instance index on `kind = 0x4`), `+44` `0xffffffff`, `+56` 0. +- **the placement region has none either**, including its per-group lead word, + which is 0 for all 24 groups; and the region is followed straight by the RATC + child stream, so there is no table hiding behind it. + +Also corrected: the running screen is **build 4**, not the largest build 7 that +`screen info` defaults to — the two disagree on sprite sizes and the capture +matches build 4. The conclusions are unchanged, the indices are not. + +**So the next step is the guest code**, not the file: the splash draw path from +the emulator-era work (`sub_821CC7A0`, item vtable `0x820b30b4`) submits with +exactly the PS hash `E59B2B3D` this capture sees, and `xenia-rs/sylpheed.db` is +available in the container. + +**And a second screen is BLOCKED**, which is what this item wanted next: the +title's Ⓐ leads into a content/save path that crashes the guest with +`--mem_watch=true` and stalls it with `--mem_watch=false`. Three separate traps +had to be cleared to establish that much — see +[`canary-scripted-input-traps.md`](canary-scripted-input-traps.md), which also +carries the reproduction and the fix for two of them. + ## Capital ships assemble wrong in the viewer diff --git a/docs/re/INDEX.md b/docs/re/INDEX.md index ba555bcd..e992eb6a 100644 --- a/docs/re/INDEX.md +++ b/docs/re/INDEX.md @@ -26,6 +26,7 @@ Promote to a prose `structures/…md` file when a format needs behavioural notes | Arsenal develop economy | ✅/❔ | [arsenal-develop-economy](arsenal-develop-economy.md) + [conditions](captures/arsenal-develop-conditions.csv) | The Arsenal reads `weapon.tbl` (item ids, in the 8-category display order) and `strings.tbl` (names, descriptions, and a **"Conditions to obtain"** block per item) out of `GP_HANGAR_ARSENAL.pak`. All **60** conditions are extracted: gates are stage completion, a predecessor item, or an **ace kill**; costs run 3 000–350 000 P and **20 items are free** once gated. `weapon.tbl`'s first record reproduces the in-game DATA SHEET exactly (Range D / Power E / Speed – / Weight 0.3 = Light / 4000 P) — later records are unreadable from the string pool alone because IDXD **dedupes repeated values**. Used to identify the save blob's index space, now **solved**: the blob follows **`strings.tbl`'s** order — the display order *plus* the cut items only the localisation file lists (`Adhesive Mine B2A`, `Ballista GSH`, …) — pinned by four hand-written probe saves (9 Stiletto, 21 Falcon, 39 Tomahawk, 48 Jamming System) and closing exactly at index 53. `weapon.tbl`'s id list is **not** the index space; that it is also 54 long is a coincidence, and the two agree only to index 32. The retail save's five unexplained owned entries are the cut items, shipped owned and never rendered | | UI screen layout (`.rat`) | ✅/🟡 | [ui-rat-layout](structures/ui-rat-layout.md) | One pak per UI screen; each RATC = one (context × language) build; every `.t32` sprite has a `.rat` **layout record** (BE u32; 1280×720 design space; scale/tint/X/Y, keyframes for animated elements, `opt ` link to the focused state). **The tutorial PAUSE menu and the title main menu both rebuild pixel-accurately from the disc.** `loop1.rat` is decoded — it is a **looping sprite animation**, not a composition. ⚠️ **DEMOTED 2026-08-18** — the declaration table is *not* the paint order: a per-draw capture of the running title screen ([ui-title-paint-order-capture](ui-title-paint-order-capture.md)) paints element 13 first and elements 0/1 late, and the visible screen composites two bundles. The rest of the table's reading stands. Previously claimed: the **screen's draw list is the RATC bundle's own declaration table** (elements in back-to-front order, including the `eff*`/`deli*`/`msg` sprites that have no `.rat`, and excluding focused button variants reached via `opt `); its entry also carries a **parent element index** at `+32`. **A screen is fully reconstructible from its bundle**: the placement region right after the declaration table gives every element a keyframe group (header = element index + keyframe count, then 40-byte blocks of scale/tint/X/Y), including the `.rat`-less sprites — verified 11/11 on the tutorial pause bundle, with `pgp_ttrl_btn10`'s inline (546,288) matching its own record exactly | | UI screen paint order | ✅/🟡 | [title paint order](ui-title-paint-order-capture.md) | **Measured from the guest's draw stream** (new Canary hook `log_ui_draws`, F10): the title screen paints 11 draws a frame through one sprite shader, and the submission order is background → `back2` glow → `ptlogo1`+`tm` → `ptlogo2` → `ptcopyright` → the `PRESS Ⓐ BUTTON` plate. That refutes declaration order, reverse order, keyframe-start order and resting-time order, and shows the screen composites **two bundles** (build 7 + the one-element build 2). 🟡 the rule that produces the order is still unknown | +| Scripted input / profile traps | ✅/🔴 | [canary-scripted-input-traps](canary-scripted-input-traps.md) | Three reasons a scripted run "cannot press Ⓐ": **F10 opens the emulator menu bar**, which makes `IsUIActive()` true and zeroes every guest keystroke; the file-pad is not the problem (it delivers, and Ⓐ skips movies); and the title needs a **signed-in profile**, which the container cannot type into ImGui — hence Canary's new `--create_profile_if_none`. 🔴 Past that, Ⓐ crashes the guest with `--mem_watch=true` and stalls it with `--mem_watch=false` — a cheap second reproduction of the Ready-Room crash, 40 s from a cold boot | | Save file (`savedata`) | ✅/❔ | [savegame-format](structures/savegame-format.md) + [`tools/re-capture/savegame.py`](../../tools/re-capture/savegame.py) | `GDHA` container, zlib payload, chunk stream (`GDAA` / phase name / `GHAD` 122 B progress block / 16×20 B slot table / trailer). **Container and layout read off the title's own serializer `0x822C00E8` and verified by a byte-identical round-trip**; the whole save is 545 B. Payload offsets are also the live save object's offsets (`save+8` GHAD, `save+136` slots). A second save made in-game names **Points** (+24), **flight time in ms** (+4) and **clear ratio %** (+8) off the game's own Details panel; the payload is a **pure function of game state** (same state saved twice = byte-identical, only the header FILETIME and its uninitialised pointer padding move), and the 16 `SHAB` records are **not** the UI's 20 save slots. Difficulty vs stage is undecided — three fields hold 2. **A third save, taken after developing exactly one Arsenal weapon** (Light Machine Gun MG I, 4000 P), moves exactly three things: `+24` Points 4101→101 (which **separates it from `+28`**, that did not move), `+8` clear ratio 5→6 (so the ratio counts *collection*, not only stages), and two entries of the 54-byte blob — `2→4` for the item bought and `0→2` for the successor the game announced as newly developable, giving the blob its alphabet ✅ *0 locked / 2 developable / 4 developed* (only the `4`s are stored — `2` is re-derived at load). **Saves can also be written back**: three derived header fields (length at `+0x30`, payload length at `+0x8c`, `adler32` at `+0x8e`) are all that stand between a parse and a hand-written save that the title loads, and [`savegame_edit.py`](../../tools/re-capture/savegame_edit.py) re-wraps a real save byte-identically. That turned the blob's index space from blocked-on-story-progress into four probe saves — see the [economy note](arsenal-develop-economy.md) | ## Runtime / dynamic-capture technique diff --git a/docs/re/canary-scripted-input-traps.md b/docs/re/canary-scripted-input-traps.md new file mode 100644 index 00000000..f9bf97fc --- /dev/null +++ b/docs/re/canary-scripted-input-traps.md @@ -0,0 +1,120 @@ +# Getting past the title screen in the container — three traps and one blocker + +**Status:** ✅ `CONFIRMED` for the three traps (each reproduced, and two of them +fixed in the toolkit). 🔴 **blocked** on reaching the main menu: the guest reads +Ⓐ at the title, and the step *behind* Ⓐ either crashes it or stalls it. + +Written down because each of these presents as "the pad is dead" or "the game +ignores input", and each has a completely different cause. + +## 1. F10 opens the emulator's menu bar, and that swallows every guest keystroke + +F10 is the RE hotkey (ship capture, and now `log_ui_draws`). It is *also* the +toolkit key that opens the application menu bar, and it does both. With that menu +open, `XamState::IsUIActive()` is true, so `XamInputGetKeystrokeEx` returns +`X_ERROR_SUCCESS` with a **zeroed** keystroke before the driver is ever asked +(`xam_input.cc`). The guest polls, gets an empty event, and does nothing — +forever, and with no error anywhere. + +This cost most of an iteration. Symptom: a run presses F10 to arm a capture, +then every later Ⓐ does nothing; the title sits there while the pad log happily +prints `[file-pad] keystroke vk=5800 down`. Screenshot +[`captures/title-with-emulator-menu-open.png`](captures/title-with-emulator-menu-open.png) +shows the giveaway — `Open… / Open Recent / Install Content… / Exit` over the +title art. It is easy to miss because the menu is at the far left of the window +and the game is still animating behind it. + +**Fix:** after any F10, click once on the game surface to dismiss the menu. +`Escape` does *not* close it. `ui_draw_capture.sh` now does this. + +## 2. The pad is fine — the driver was never the problem + +Worth stating because the file-pad's own history (`--hid=file`, `GetKeystroke` +implemented specifically so "PRESS Ⓐ BUTTON" would work) makes it the natural +suspect. It delivers: the log shows `vk=5800 down`/`up` per tap, and Ⓐ during a +movie skips the movie, every time. When the title ignores Ⓐ, the pad is not why. + +## 3. Ⓐ at the title needs a signed-in profile — and this container cannot type one in + +With no profile, Ⓐ **is** handled: the guest calls `XamShowSigninUI` and Xenia +pops its Sign In dialog ([`captures/title-signin-dialog.png`](captures/title-signin-dialog.png)), +`ProfileManager: Found 0 Profiles`. That dialog is the proof the guest reads the +button. + +The dialog cannot be completed from here: "Create Profile" opens an **ImGui text +field** for a gamertag, and synthetic X key events do not reach it. Tried, all +with the window focused (`xdotool getwindowfocus` = the game window): +`xdotool type --window`, `xdotool type` via XTEST, single `xdotool key` presses, +clicking the field first. The field stays empty; `Create` with an empty gamertag +is refused. Mouse clicks *do* work (they opened the dialog and the sub-panel), so +it is text entry specifically. + +Xenia's own **Profile ▸ Show Profile Menu** panel does have a working `Create +Profile` button, and mouse clicks reach it — but the gamertag it then asks for is +the same ImGui field, so the UI route dead-ends in the same place. + +**Fix, landed in Canary** (`auto/re-ui-draw-order`): a `--create_profile_if_none=` +cvar that calls `ProfileManager::CreateProfile(tag, autologin, default_xuid)` at +startup when no profile exists. `default_xuid` is fixed at `B13EBABEBABEBABE`, so +a script can then pass `--logged_profile_slot_0_xuid=B13EBABEBABEBABE` and be +deterministic. Verified: `ProfileManager: RE bootstrap profile 'SylphRE' -> +created`, then on the next boot `Found 1 Profiles` / `Loaded SylphRE (GUID: +B13EBABEBABEBABE) to slot 0`, and the Profiles Menu panel shows the user +([`captures/xenia-profile-panel-signed-in.png`](captures/xenia-profile-panel-signed-in.png)). +The profile persists on disk, so later runs need only +`--logged_profile_slot_0_xuid=B13EBABEBABEBABE`. + +## The blocker: what happens *after* Ⓐ + +With the profile created and signed in — Xenia's own **Profile ▸ Show Profile +Menu** confirms it, `User: SylphRE / XUID: B13EBABEBABEBABE / Assigned to slot: 1` +(the panel counts from 1; the log says slot 0) — Ⓐ at the title **still does not +advance the screen**. Measured across four boots, including one that was allowed +to play the whole intro untouched, with holds of 10 s and bursts of five taps. + +What is *not* the explanation, each checked: + +* the emulator menu bar being open (trap 1) — the grabs are clean; +* the pad (trap 2) — Ⓐ skips movies in the same runs; +* the profile being absent (trap 3) — it is present, signed in, and the Sign In + dialog no longer appears; +* an ImGui window being open — but note this generalises trap 1: **any** Xenia UI + window makes `IsUIActive()` true, the Profiles Menu included, so a panel left + open silently disables the guest's input for as long as it is up. + +**One observation that did not reproduce, recorded rather than dropped.** In the +run where the Profiles Menu had just been closed, a single Ⓐ **crashed the +guest** — PC `0x824578A0`, guest thread 6, distinct from the boot-path crash +below. Two later attempts to reproduce it (open the panel, close it, press Ⓐ) +produced neither a crash nor a transition. So it is one data point that Ⓐ *can* +reach code the title does not otherwise run, and nothing more. + +**The reproducible crash is the boot-time content path.** With the default +`--mem_watch=true`, in two independent runs, the guest dies with an access +violation reading `0x00000001_0000000C` at PC `0x82307128` on guest thread 9, +right after `HostPathDevice::ResolvePath(\aab216c3\5\c10eae6)` and a +`RtlRaiseException(E06D7363)` (a C++ throw). Xenia pauses itself and stacks up +crash dialogs — 991 in one run. With `--mem_watch=false` that crash does not +happen at all. + +That pairing — crash with the memory-probe thread on, none with it off, both in a +save/content path — matches the suspect ranked #1 in +`xenia-canary/HANDOFF-crash-oracle-2026-07-16.md`, whose crash was also a guest +C++ exception in a cache/save path. This is a second, far cheaper reproduction of +it: at the **title screen**, ~40 s from a cold boot, instead of in the Ready Room +after a mission. + +**Next step, for whoever picks this up.** Two threads to pull, in this order: + +1. `0x82307128` and `0x824578A0` in `xenia-rs/sylpheed.db` (the container has it) + — what function is the title's Ⓐ path in, and what is at `+0xC` of the + structure it dereferences; +2. the content exports this title imports — `XamContentCreateEnumerator`, + `XamContentGetDeviceState`, `XamUserReadProfileSettings` — logged around the + press, to see whether the guest is handed a bad enumerator or a device state + it does not accept. `\aab216c3\5\…` is the content directory in question. + +**What this blocks:** every dynamic experiment past the title — a second UI +screen's paint order, the main menu, mission captures. Everything up to and +including the title screen is unaffected, which is why the paint-order result +above could still be measured. diff --git a/docs/re/captures/title-signin-dialog.png b/docs/re/captures/title-signin-dialog.png new file mode 100644 index 00000000..84695153 Binary files /dev/null and b/docs/re/captures/title-signin-dialog.png differ diff --git a/docs/re/captures/title-with-emulator-menu-open.png b/docs/re/captures/title-with-emulator-menu-open.png new file mode 100644 index 00000000..21133316 Binary files /dev/null and b/docs/re/captures/title-with-emulator-menu-open.png differ diff --git a/docs/re/captures/xenia-profile-panel-signed-in.png b/docs/re/captures/xenia-profile-panel-signed-in.png new file mode 100644 index 00000000..be8fbab0 Binary files /dev/null and b/docs/re/captures/xenia-profile-panel-signed-in.png differ