Syplheed-Reborn/docs/re/challenge-mission-gate.md

# Challenge / EX missions — the stage set, the GamePart graph, and the kind field

**Status:** ✅ for the static structure (stage set, GamePart ids, the config-section
switch) and for the **unlock mechanism** — a bit test against a *cleared-stage* mask,
now **read live off the running game and matching the profile's progress exactly**
(§5.6). 🟡 for the stage-field crash mechanism, for word B's writer, and for which
mission consumes which bit; ❔ how the challenge menu is actually entered.
**Method:** static analysis (`.pe` strings/pointers + DuckDB disassembly + disc
records), then confirmed on the running title via the container-safe
[`--hid=file` pad](../../../xenia-canary-native/src/xenia/hid/file/file_input_driver.h)
and a live guest-memory read.
**Evidence:** [`captures/challenge-map.txt`](captures/challenge-map.txt),
[`captures/challenge-screen-config.txt`](captures/challenge-screen-config.txt),
[`structures/achievements.md`](structures/achievements.md);
`examples/challenge_map.rs`, `examples/challenge_screen.rs`.

## Why this was worth doing

The Route-B unit harvest is complete for the **story** campaign (68 of 110 units,
7 115 defaulted-on-disc values) and stalled there: the remaining units are `*_EX4` /
`*_EX5` / `*EX` variants that only the challenge missions field. The save's stage
field addresses those stage records — but setting it to **27** boots to the title and
then the emulator **exits during the load**, while 1–16 all load normally. This note
answers *what the challenge missions are on disc* and *why the stage field alone is
not enough*, so the next runtime attempt is targeted rather than another probe sweep.

## 1. The disc has exactly 29 stage records, in three families ✅

`StageResource` (schema `0x3c9ae32e`) in `dat/GP_MAIN_GAME_E.pak`:

| ids | count | `BackGroundID` | reading |
|---|---|---|---|
| `S01`–`S16` | 16 | Lebendorf … PD (real locations) | the **story** campaign |
| `S18`–`S23` | 6 | `Original` (all six) | the **tutorial** missions |
| `S24`–`S29` | 6 | Anastasis, Hargenteen ×2, Planet_Lebendorf, Lebendorf, Earth | the **challenge** missions |
| `Test` | 1 | Earth | developer stage |

`S17` does not exist. This **matches `weapon.tbl`'s sprite-key set exactly** —
`stage01…16` + `tutorial01…06` + `challenge01…06` (recorded in
[static screen config](#)) — which is an independent confirmation of the three-way
split: 16 + 6 + 6 + Test = 29.

The `Original` six carry ~43 tokens each while the story and challenge records carry
51–59, consistent with tutorials having no squadron/route/formation tables.

## 2. `GP_CHALLENGE.pak` is a screen, not mission data ✅

151 entries, **0 IDXD objects** — sprites and layout only. So challenge missions are
*not* a separate data set: they are ordinary `StageResource` records in
`GP_MAIN_GAME_E.pak`, reached through a different menu. That is why the EX unit
rosters show up in the same `EnumUnit_S<NN>` tables the story stages use.

## 3. The GamePart id table — the game's whole screen graph ✅

`.rdata` pointer array at **`0x820A1630`**, 29 entries, index = GamePart id:

```
 0 GP_TITLE            10 GP_BUNK             20 GP_STAGE_CLEAR
 1 GP_ADVERTISE_DEMO   11 GP_READY_ROOM       21 GP_MISSION_LOG
 2 GP_SELECT_STORAGE   12 GP_HANGAR           22 GP_GAMEOVER
 3 GP_LOAD             13 GP_ARSENAL          23 GP_DEBRIEFING
 4 GP_SAVE             14 GP_PILOT_LOG        24 GP_DIALOG
 5 GP_EXTRAS           15 GP_SYSTEM           25 GP_TUTORIAL
 6 GP_MOVIE_THEATER    16 GP_DEMO            *26 GP_CHALLENGE
 7 GP_MISSION_SELECT   17 GP_MAIN_GAME        27 GP_LEADERBOARD
 8 GP_OPTIONS          18 GP_SELECTOR         28 GP_TEST
 9 GP_MOVIE            19 GP_PAUSE_MENU
```

The indices are **not inferred from position** — the image carries the factory
registration text, e.g.
`silph::GamePartTask::RegisterToFactory<26, class silph::GamePart_ChallengeMission>::RegisterToFactory is failed!`
and `<10, … GamePart_Bunk>`, both of which agree with this table. So `GP_CHALLENGE`
is GamePart **26** and `GP_TUTORIAL` is **25**.

## 4. A mission-**kind** field selects the stage config section ✅

Immediately before the array above, `0x820A1600`–`0x820A162C` holds the stage-config
key list: `BASE_INFO`, `StageResource`, `Background`, `PlayerUnit`,
`EQUIIP_LIMITATION` *(sic)*, `MODEL_PATH`, `NEW_ITEM`, `LOADING`, `CHALLENGE`,
`EXTRA`, `FILE`.

The stage loader picks **one of the last three** by a field at **`object + 144`**, and
the same three-way switch appears at two independent sites (`0x82184df0` inside the
all-keys config reader, and `0x82185ed0` in `sub_82185E80`):

```asm
lwz  r11, 144(r30)
cmpi r11, 3          ; == 3          -> "EXTRA"     (0x820A2160)
beq  extra
addi r11, r11, -5
cmpli r11, 1         ; == 5 or 6     -> "CHALLENGE" (0x820A2154)
bgt  file            ; otherwise     -> "FILE"      (0x820A2168)
```

Two further sites (`0x82186a60`, `0x82186cb8`) classify the same field as
`{3, 5, 6}` versus everything else — i.e. **EXTRA and CHALLENGE together are "not an
ordinary story load"**. The field is initialised to **0** in the constructor
(`sub_821783D8`, `0x8217851c`, alongside `+148 = 0` — the stage index — and
`+132 = 2`, `+136/+140 = -1`), and every write to it inside this class
(`0x82184b10`, `0x82185d48`, `0x82186ab4`) only *clears* it. Nothing in the image
stores an immediate 3/5/6 into it, so the kind is **supplied from outside the class**
— by whichever GamePart launches the mission — not derived from the stage number.

### 4.1 Why the stage-field probe crashes 🟡

That gives a mechanism for the observed failure: patching the save's stage field to
27 selects the `S27` **record** while the kind stays **0**, so the loader reads the
`FILE` section for a stage whose config lives under `CHALLENGE`. A missing section
then propagates into the load, and the title exits. It fits the evidence (1–16 fine,
every 24–29 the same failure, `/dev/shm` empty and disk free, so not the resource
trap) but it is **not proven** — proving it needs a run.

**Cheap untried discriminator, no new tooling:** patch the stage field to **18–23**
(the tutorials). If those also die, the failure tracks "record outside the story
range", and the kind field is the likely gate. It has never been tested — only
1–16 and 24–29 were.

## 5. The challenge screen, its six missions, and the unlock gate ✅

The factory registration sites (`0x8280C000`–`0x8280F800`) give **id → creator** for
22 of the 24 registered parts, and the creators bound each class's code block. So
`GamePart_ChallengeMission`'s methods are **`0x82187E60`–`0x8218CF10`** (between
`GP_BUNK`'s creator `0x82187E38` and its own `0x8218CF10`). Resolving every string
that range references gives the screen's whole config schema:

```
MISSIONS · MISSION_ID · RECORD_TYPE · REQUIREMENT · REQUIREMENT_DESC
NORMAL_BUTTON · GRAY_BUTTON · THUMBNAIL · STAGE_DESC · TEXT_STAGE
TEXT_RECORD · NEW_STAGE · "Always" · "Time" · BASE_INFO
```

**The record itself is on disc**, in `tables.pak` (schema `54a10697`, one copy per
language — English is entry #64), *not* in `GP_CHALLENGE.pak`. It names six missions:

| slot | `MISSION_ID` | buttons / thumbnail | record |
|---|---|---|---|
| 1 | `TimeAttack` | `Button_TimeAttack{,_Gray}`, `Thumbnail_TimeAttack` | `Time` |
| 2 | `ScoreAttack` | `Button_ScoreAttack{,_Gray}` | `Points` |
| 3–6 | `Extra01`…`Extra04` | `Button_Extra0N{,_Gray}` | — |

`GRAY_BUTTON` is the locked art, `NORMAL_BUTTON` the unlocked art — so the screen
always shows all six and greys out what you have not earned. Full dump:
[`captures/challenge-screen-config.txt`](captures/challenge-screen-config.txt).

### 5.1 The gate is a bit test against a progress bitfield ✅

`0x82189870` builds the list, and `0x82189970`–`0x821899D8` is the availability
decision for each mission:

```asm
lookup REQUIREMENT in the mission record        ; bl 0x82448C50
  absent            -> AVAILABLE
  == "Always"       -> AVAILABLE                ; strcmp, bl 0x825EDD20
  else n = atoi(v)                              ; bl 0x825EDCD0
     n == 0         -> LOCKED
     n <  24        -> bit n      of word A
     n >= 24        -> bit (n-24) of word B
     bit set ? AVAILABLE : LOCKED
```

Word A and word B are read from a **singleton** (`0x821707C0`; the object pointer
lives at the global `0x828F48B0`, with a construct-once flag at `0x828F48BC`) at
offsets **`+80`** and **`+1956`**. So challenge
availability is one bit in a progress bitfield, and `REQUIREMENT` is that bit's index
— **not** a stage number, a score, or a difficulty.

### 5.2 The bit space is CLEARED STAGES ✅

Word A has exactly **one** writer in the image, and finding it settles the question.
Scanning all 22 callers of the `+80` struct copier (`0x82175110`) for one that stores
to the copy's **word 0** gives a single hit, `0x821C1820`, inside
**`GamePart_StageClear`** (`0x821C09D8`–`0x821C29F0`):

```asm
if (this+1004 & 0x20000) skip            ; already recorded
this+80 = 1
copy   local = singleton->progress        ; bl 0x82175110, src = obj+80
x      = this+84
local.word0 |= 1 << x                     ; slw r10, r26, r10
if changed: singleton->set(local)         ; bl 0x8216FF70 (assign + async persist)
```

**`this+84` is the stage number**, and two independent uses prove it:

- `0x821C1760` indexes a **20-byte record array** with it — `this + (x+7)*20`, whose
  records are written as three words plus an 8-byte timestamp, i.e. the shape of the
  savegame's 16×20-byte `SHAB` table;
- `0x821C1EEC` and `0x821C1924` pass it as the **index into the config key `STAGE`**
  (`0x820A2540`) — the debriefing config's `px_deb_stage01…16` sprite list.

So **word A is a "stage cleared" bitmask**, and a challenge mission's `REQUIREMENT n`
means **"stage `n` has been cleared"**. The `< 24` / `>= 24` split then lines up with
the disc's own stage numbering from §1: story `1`–`16` and tutorial `18`–`23` sit in
word A, and the challenge stages `24`–`29` are exactly word B's bits `0`–`5`.

> ⚠️ **Third revision of this claim — the first two were wrong, and how they went
> wrong is worth keeping.** An earlier pass read the split at 24 as "the game's 24
> achievements" because `ACHIEVEMENTS_REQUIREMENTS` has exactly 24 entries. That is a
> **coincidence**: the achievement count and the first challenge stage id are both 24
> for unrelated reasons. The achievement work itself stands — the `XACH` table, the
> 1000G self-check, and the `XACHIEVEMENT_DETAILS`/XAM enumeration are all solid, and
> are documented in [structures/achievements.md](structures/achievements.md) — it just
> **does not gate the challenge missions**. The lesson: a numeric coincidence is not a
> join; find the writer.

### 5.3 What the six requirement values are ✅/🟡

The record's numeric tokens are `16`, `25`, `26`, `27`, `29`, with `24`/`28` already
in the pool earlier (they double as font metrics) and so **deduped away** if used.
IDXD dedup makes positional key/value pairing unsound in general — but `16` sits
*immediately* before `REQUIREMENT` (tokens 96 → 97), the documented value-before-key
adjacency, so the first mission's value is well-supported.

**`TimeAttack` requires stage 16 cleared** — the final story mission. That is the
natural gate, and it is what the *first* reading of these numbers suggested before the
achievement detour talked me out of it.

The other five values are `≥ 24`, i.e. **challenge stages 24–29**: the challenge
missions chain off each other. 🟡 on the exact pairing (which mission needs which),
which needs the record's binary index section rather than the string pool.

🟡 **Word B has no known writer.** `GamePart_StageClear` sets `1 << x` into word A
unconditionally, which for a challenge stage (`x ≥ 24`) would land on word A bits
24–29, *not* word B — so clearing a challenge mission must be recorded by a different
path, presumably one that also stores its `Time`/`Points` record. Not yet found.

**Negative worth keeping:** the requirement *text* (`TimeAttackRequirement`,
`Extra01Requirement`, …) is **not** in `GP_CHALLENGE.pak` — building a `TextIndex`
over it yields **0 entries** and its only wordy payload is embedded font copyright.
The config's `PATH` is `dat\GP_CHALLENGE.pak+eng\`, a per-language branch, so those
keys resolve through a naming scheme the current loader does not reproduce. Reading
them would say in plain English what each mission asks for — worth one more attempt
via `hash::TOC_NAME_SCHEMES`.

### 5.4 Where the mask lives, and one lead refuted

### 5.5 Both gate words are one record — and it is **not** the savegame ✅

`0x82175110` copies the record at singleton `+80` in full: two words, an 8-byte pair, a
184-byte `memcpy`, 8 words from `+200`, then **816 bytes at `+232`, 816 more at
`+1048`**, a sub-object at `+1864`, and a final word at **`+1876`**. So the record runs
`+0 … ~+1880`, i.e. singleton `+80 … +1960` — which means

- **word A** is record `+0` (singleton `+80`), and
- **word B is record `+1876`** (singleton `+1956`).

That answers §5.3's "word B has no known writer": there is no separate `stw` because
the whole record is copied out, modified and assigned back as a unit
(`0x8216FF70` → compare `0x822C3708`, assign `0x82170650`, then spawn a worker
`0x821700A8` under a lock which retries a commit `0x822C33B8` up to five times).
`GamePart_Debriefing` uses the same get/modify/set to accumulate saturating career
counters at `+200`…`+224` (the last 64-bit) — the quantities the achievement
requirements test.

**And the record is not what `savedata` holds.** Two independent checks:

- **Size.** Every real save on disk is a **276-byte** container deflating to a
  **545-byte** payload. The progress record is **~1 880 bytes**. It does not fit.
- **Files.** After many sessions the game's content tree holds only
  `game0N/savedata` + `game0N/__thumbnail.png` per slot and the three
  `Headers/*.header` — no second data file anywhere.

Also checked and negative: the savegame object is `*(*(this+4)) + 304`, and the
singleton's holder address (`0x828F48B0`) is referenced **nowhere but inside the
accessor itself**, so `this+4` is a different holder — the save block is not a window
into this record.

⚠️ **So hand-editing a save cannot unlock the challenge missions.** That kills the
operational hope this section previously carried; the earlier savegame-editing win
does not extend here.

### 5.6 CONFIRMED ON THE RUNNING GAME ✅

Read live from a booted title (`tools/re-capture/gpoke.py r32`):

```
0x828F40C0 = 0x00000002      word A
0x828F4814 = 0x00000000      word B
```

**Word A = 2 = bit 1 set.** The profile's save is *Stage 02, "At Standby"* — i.e.
**stage 01 cleared** — so the mask is exactly one bit, at the index of the one
cleared stage, **1-based**. Reproduced on two separate cold boots. That confirms, on
the running game and against a known progress state:

- the singleton really is the static object at `0x828F4070`;
- word A is a **cleared-stage bitmask**, not achievements, not a stage number;
- bit index = **stage id, 1-based** — so `TimeAttack`'s `REQUIREMENT 16` is
  "clear stage 16", the last story mission;
- word B is the challenge half and is `0` on a story-only profile, as expected.

Both words were then poked (`0xFFFFFFFF` / `0x3F`) and read back OK.

**MISSION SELECT shows the mask directly** ([capture](captures/mission-select-stage01-only.png)).
With word A = 2 the screen lists `Stage01` **selectable, with a High Score and Best
Time**, and `Stage02`–`Stage08` **greyed out**. One cleared stage, one selectable
entry, at the bit index that names it — the semantics are visible on screen, not
inferred.

**And the mask drives that screen.** Two runs, identical navigation, fresh boot each:

| run | word A | MISSION SELECT |
|---|---|---|
| control | `0x00000002` (untouched) | opens; Stage01 selectable, rest greyed |
| poked | `0xFFFFFFFF` | `MmAllocatePhysicalMemoryEx` fails on a 128 MB request, guest throws, Xenia shows "Disc Read Error" |

So the earlier failure was **caused by the poke**, and by a careless one: `0xFFFFFFFF`
claims stages that do not exist (`0`, `17`, `24`–`31` in word A). Poking only real
story ids (`0x0001FFFE` = stages 1–16) does **not** blow the heap. That the list
screen changes behaviour with the mask is itself confirmation that word A feeds it.

**Poking real stage ids unlocks the story campaign in the menu** ✅. With word A =
`0x0001FFFE` (stages 1–16) every entry `Stage01`…`Stage16` is selectable
([capture](captures/mission-select-all-story-unlocked.png)) where the control had only
`Stage01`; `Stage16` reads *"Lonely Blue Planet — NO RECORD"*. So **any story stage
can now be launched from the menu**, with no save editing, by poking one word.

**But MISSION SELECT is story-only** ❌. With word B = `0x3F` (challenge stages 24–29
marked cleared) the list still **saturates at `Stage16`**
([capture](captures/mission-select-ends-at-stage16.png)) — the cursor stops there and
further presses do nothing. That matches the data: the debriefing config declares
exactly `px_deb_stage01…16`, so the list length is capped by the disc, not by the
mask. **The challenge missions cannot be reached through this screen**, and word B
does not feed it.

**What the poke did not do (yet):** `EXTRAS` still shows only `MISSION SELECT /
MOVIE THEATER / BACK` — no challenge entry — although the menu was built 26 s
*after* the poke, so this is not staleness. Entering `MISSION SELECT` then failed,
and the log gives the real reason: **`MmAllocatePhysicalMemoryEx` could not satisfy
a 128 MB request** (`parent free 30633/131072 pages`), the guest threw a C++
exception, and Xenia surfaced it as its generic *"Disc Read Error"* dialog. It is
preceded by `BaseHeap::Release failed because address is not a region start`, a
failed release that leaks the range. So that is an emulator/heap problem on the way
into the screen, **not** evidence about the gate. Open: repeat without the poke to
see whether `MISSION SELECT` fails the same way regardless.

### 5.7 What *would* work — static addresses for a live write

The singleton is a **static object at `0x828F4070`** (`0x8216F650`:
`addis 0x828F` + `addi …, 16496`), with the holder at `0x828F48B0` pointing at it. So
the two gate words are at fixed guest addresses, no scanning required:

| word | guest VA | meaning |
|---|---|---|
| A | **`0x828F40C0`** | cleared stages, bit = stage id (`< 24`) |
| B | **`0x828F4814`** | cleared stages, bit = stage id − 24 (challenge 24–29) |

Canary maps guest RAM into `/dev/shm`, which the project already reads live
([`tools/re-capture/gmem.py`](../../tools/re-capture/gmem.py)). Writing
`0xFFFF` into word A and `0x3F` into word B while the title sits on a menu should
open all six challenge missions **without playing the campaign** — and that is the
route to the last 42 `*_EX4`/`*_EX5` units. Untested: it needs a run, and the run
needs gamepad input to reach the menu.

**The obvious lead is dead.** The three `stw`s to `+1956` in `0x822AF278` /
`sub_822C8748` looked promising because they sit in the save serializer's code region
— they are on a **different object**. That one is fetched through `0x822CEB30`, has a
`+2652` flag the code checks first, and stores **string pointers** at `+1956`/`+2024`
(built by `0x822D35F8`); a pointer `AND`ed with `1 << n` would be meaningless as a
gate. So **nothing in the image stores to this singleton's `+1956` field-wise**, which
means it is filled by a bulk copy or by a path not yet found.

Two candidates remain and they need different levers: the **save** (hand-write it) or
the **Xbox profile** (the emulator's profile data). Note that XEX imports are resolved
**by ordinal**, so the absence of `XamUser*` name strings in the `.pe` is not evidence
against the profile route. `+80` at least is handled as a small struct by address
(`addi r4, obj, 80` → copy helper `0x82175110`, written back via `0x8216FF70`), which
is what a serialised value object looks like.

## 6. The unlock condition, from the strings ❔

Three strings say challenge missions are *announced*, not menu-browsed:

- `AVSCRIPT_COMMAND_ATTAINMENT_CHALLENGE_MISSION_CARGO_SCORE` — a mission-script
  command that grants challenge-mission attainment from a **cargo score**;
- `DLG_CHALLENGE_MISSION_AVAILABLE` — the "a challenge mission is now available"
  dialog;
- `DLG_GO_CHALLENGE_MISSION_MENU` — the prompt that takes you to GamePart 26.

So the gate is plausibly an in-mission achievement, not a title-menu state — which is
consistent with `Game Status = GAME_CLEAR` unlocking nothing (measured, refuted).
Neither dialog key is reachable by xref: they are selected **by index** through a
config lookup, exactly like the save screen's sprite keys, so there is nothing to
grep back to. Naming the flag needs the challenge-availability check disassembled
from the screen side.

## 7. What this makes actionable

`roster_target` against the harvested CSV, with the stage families now named:

| stage | family | roster units not yet read |
|---|---|---|
| `S27` | challenge | `f003_ArrowHead_EX4`, `e107_AAFrigate_EX4`, `e010_Attacker_S_EX4`, `e011_Attacker_B_EX4`, `e007_Turret_EX4`, `e008_TurretPlus_EX4` |
| `S28` | challenge | `f004_DeltaSaber_A_Player`, `f001_DeltaSaber_T_EX5(_el)`, `f003_ArrowHead_EX5`, `f104_Battleship_EX5`, `f105_Cruiser_EX5` |
| `S25` | challenge | `e101_SDBattleshipEX`, `e102_BattleshipEX`, `e105_CruiserEX`, `e108_ASFrigateEX` |
| `S29` | challenge | `e101_SDBattleshipEX`, `e102_BattleshipEX`, `e105_CruiserEX` |
| `S24` | challenge | `e105_CruiserEX`, `e106_DestroyerEX` |
| `S10` | **story** | `e005_ADAN_ElanTypeQ_Margras` |

⚠️ **The `S10` row contradicts "the story campaign is complete."** One story stage
still fields a unit that has never been read. Either `S10` was never flown (it is the
smallest stage container on the disc — 7 XBG7 resources — so it may be a cutscene
stage that is not flyable), or the completeness claim was one unit optimistic. Flagged
rather than resolved: it costs one ordinary story run to settle.

Remaining coverage: 42 units missing, of which the rosters above account for ~23. The
rest are not in any stage roster table and need a different lever.

## Reproduce

```bash
cargo run --release -q -p sylpheed-formats --example challenge_map    -- <disc-root>
cargo run --release -q -p sylpheed-formats --example challenge_screen -- <disc-root>
python3 xenia-rs/zq.py dis 0x82184df0 0x82184e30      # the three-way section switch
python3 xenia-rs/zq.py dis 0x82185ed0 0x82185f10      # the same switch, second site
python3 xenia-rs/zq.py dis 0x82189860 0x821899f0      # the challenge availability gate
```