Syplheed-Reborn/docs/re/challenge-mission-gate.md

Challenge / EX missions — the stage set, the GamePart graph, and the kind field

Status: ✅ for the static structure (stage set, GamePart ids, the config-section switch) and for the unlock mechanism (a bit test against the achievement mask); 🟡 for the crash mechanism and for which mission consumes which bit; ❔ for where the mask persists. Method: static only — .pe string/pointer analysis + DuckDB disassembly + disc records. No emulator run, no gamepad input. Evidence: captures/challenge-map.txt, captures/challenge-screen-config.txt, structures/achievements.md; examples/challenge_map.rs, examples/challenge_screen.rs.

Why this was worth doing

The Route-B unit harvest is complete for the story campaign (68 of 110 units, 7 115 defaulted-on-disc values) and stalled there: the remaining units are *_EX4 / *_EX5 / *EX variants that only the challenge missions field. The save's stage field addresses those stage records — but setting it to 27 boots to the title and then the emulator exits during the load, while 1–16 all load normally. This note answers what the challenge missions are on disc and why the stage field alone is not enough, so the next runtime attempt is targeted rather than another probe sweep.

1. The disc has exactly 29 stage records, in three families ✅

StageResource (schema 0x3c9ae32e) in dat/GP_MAIN_GAME_E.pak:

ids	count	BackGroundID	reading
S01–S16	16	Lebendorf … PD (real locations)	the story campaign
S18–S23	6	Original (all six)	the tutorial missions
S24–S29	6	Anastasis, Hargenteen ×2, Planet_Lebendorf, Lebendorf, Earth	the challenge missions
Test	1	Earth	developer stage

S17 does not exist. This matches weapon.tbl's sprite-key set exactly — stage01…16 + tutorial01…06 + challenge01…06 (recorded in static screen config) — which is an independent confirmation of the three-way split: 16 + 6 + 6 + Test = 29.

The Original six carry ~43 tokens each while the story and challenge records carry 51–59, consistent with tutorials having no squadron/route/formation tables.

2. GP_CHALLENGE.pak is a screen, not mission data ✅

151 entries, 0 IDXD objects — sprites and layout only. So challenge missions are not a separate data set: they are ordinary StageResource records in GP_MAIN_GAME_E.pak, reached through a different menu. That is why the EX unit rosters show up in the same EnumUnit_S<NN> tables the story stages use.

3. The GamePart id table — the game's whole screen graph ✅

.rdata pointer array at 0x820A1630, 29 entries, index = GamePart id:

 0 GP_TITLE            10 GP_BUNK             20 GP_STAGE_CLEAR
 1 GP_ADVERTISE_DEMO   11 GP_READY_ROOM       21 GP_MISSION_LOG
 2 GP_SELECT_STORAGE   12 GP_HANGAR           22 GP_GAMEOVER
 3 GP_LOAD             13 GP_ARSENAL          23 GP_DEBRIEFING
 4 GP_SAVE             14 GP_PILOT_LOG        24 GP_DIALOG
 5 GP_EXTRAS           15 GP_SYSTEM           25 GP_TUTORIAL
 6 GP_MOVIE_THEATER    16 GP_DEMO            *26 GP_CHALLENGE
 7 GP_MISSION_SELECT   17 GP_MAIN_GAME        27 GP_LEADERBOARD
 8 GP_OPTIONS          18 GP_SELECTOR         28 GP_TEST
 9 GP_MOVIE            19 GP_PAUSE_MENU

The indices are not inferred from position — the image carries the factory registration text, e.g. silph::GamePartTask::RegisterToFactory<26, class silph::GamePart_ChallengeMission>::RegisterToFactory is failed! and <10, … GamePart_Bunk>, both of which agree with this table. So GP_CHALLENGE is GamePart 26 and GP_TUTORIAL is 25.

4. A mission-kind field selects the stage config section ✅

Immediately before the array above, 0x820A1600–0x820A162C holds the stage-config key list: BASE_INFO, StageResource, Background, PlayerUnit, EQUIIP_LIMITATION (sic), MODEL_PATH, NEW_ITEM, LOADING, CHALLENGE, EXTRA, FILE.

The stage loader picks one of the last three by a field at object + 144, and the same three-way switch appears at two independent sites (0x82184df0 inside the all-keys config reader, and 0x82185ed0 in sub_82185E80):

lwz  r11, 144(r30)
cmpi r11, 3          ; == 3          -> "EXTRA"     (0x820A2160)
beq  extra
addi r11, r11, -5
cmpli r11, 1         ; == 5 or 6     -> "CHALLENGE" (0x820A2154)
bgt  file            ; otherwise     -> "FILE"      (0x820A2168)

Two further sites (0x82186a60, 0x82186cb8) classify the same field as {3, 5, 6} versus everything else — i.e. EXTRA and CHALLENGE together are "not an ordinary story load". The field is initialised to 0 in the constructor (sub_821783D8, 0x8217851c, alongside +148 = 0 — the stage index — and +132 = 2, +136/+140 = -1), and every write to it inside this class (0x82184b10, 0x82185d48, 0x82186ab4) only clears it. Nothing in the image stores an immediate 3/5/6 into it, so the kind is supplied from outside the class — by whichever GamePart launches the mission — not derived from the stage number.

4.1 Why the stage-field probe crashes 🟡

That gives a mechanism for the observed failure: patching the save's stage field to 27 selects the S27 record while the kind stays 0, so the loader reads the FILE section for a stage whose config lives under CHALLENGE. A missing section then propagates into the load, and the title exits. It fits the evidence (1–16 fine, every 24–29 the same failure, /dev/shm empty and disk free, so not the resource trap) but it is not proven — proving it needs a run.

Cheap untried discriminator, no new tooling: patch the stage field to 18–23 (the tutorials). If those also die, the failure tracks "record outside the story range", and the kind field is the likely gate. It has never been tested — only 1–16 and 24–29 were.

5. The challenge screen, its six missions, and the unlock gate ✅

The factory registration sites (0x8280C000–0x8280F800) give id → creator for 22 of the 24 registered parts, and the creators bound each class's code block. So GamePart_ChallengeMission's methods are 0x82187E60–0x8218CF10 (between GP_BUNK's creator 0x82187E38 and its own 0x8218CF10). Resolving every string that range references gives the screen's whole config schema:

MISSIONS · MISSION_ID · RECORD_TYPE · REQUIREMENT · REQUIREMENT_DESC
NORMAL_BUTTON · GRAY_BUTTON · THUMBNAIL · STAGE_DESC · TEXT_STAGE
TEXT_RECORD · NEW_STAGE · "Always" · "Time" · BASE_INFO

The record itself is on disc, in tables.pak (schema 54a10697, one copy per language — English is entry #64), not in GP_CHALLENGE.pak. It names six missions:

slot	MISSION_ID	buttons / thumbnail	record
1	TimeAttack	Button_TimeAttack{,_Gray}, Thumbnail_TimeAttack	Time
2	ScoreAttack	Button_ScoreAttack{,_Gray}	Points
3–6	Extra01…Extra04	Button_Extra0N{,_Gray}	—

GRAY_BUTTON is the locked art, NORMAL_BUTTON the unlocked art — so the screen always shows all six and greys out what you have not earned. Full dump: captures/challenge-screen-config.txt.

5.1 The gate is a bit test against a progress bitfield ✅

0x82189870 builds the list, and 0x82189970–0x821899D8 is the availability decision for each mission:

lookup REQUIREMENT in the mission record        ; bl 0x82448C50
  absent            -> AVAILABLE
  == "Always"       -> AVAILABLE                ; strcmp, bl 0x825EDD20
  else n = atoi(v)                              ; bl 0x825EDCD0
     n == 0         -> LOCKED
     n <  24        -> bit n      of word A
     n >= 24        -> bit (n-24) of word B
     bit set ? AVAILABLE : LOCKED

Word A and word B are read from a singleton (0x821707C0; the object pointer lives at the global 0x828F48B0, with a construct-once flag at 0x828F48BC) at offsets +80 and +1956. So challenge availability is one bit in a progress bitfield, and REQUIREMENT is that bit's index — not a stage number, a score, or a difficulty.

5.2 The bit space is the game's 24 ACHIEVEMENTS ✅

The < 24 / >= 24 split is not arbitrary. GamePart_Debriefing (0x8218CF38–0x82191B18) walks a disc config list called ACHIEVEMENTS_REQUIREMENTS (tables.pak entry #16, schema 744c0519) and, for entry index n, tests and sets bit n of an awarded-mask — and that list is exactly ACHIEVEMENT01 … ACHIEVEMENT24, 24 entries. The XEX's own XACH resource holds the matching 24 achievement definitions, summing to 1000G, the retail total. Full table and record layout: structures/achievements.md.

So word A (+80) reads as the earned-achievement mask and word B (+1956) as a second, different flag space that requirement values ≥ 24 index as bit n-24.

⚠️ Corrected the same day: an earlier draft said "bit n = achievement n+1" and called the join ✅. Both were over-claimed. The Debriefing's masks live on the Debriefing object (+208/+736/+740), nothing observed copies them into the singleton's +80, and the one place the image is seen turning an achievement into a bit does 1 << dwId with 1-based ids. So the mapping is 🟡 and, if it holds, is REQUIREMENT n = achievement n — see achievements §3.1 and §4.

5.3 What the six requirement values are 🟡

The record's numeric tokens are 16, 25, 26, 27, 29, and 24/28 are already in the pool earlier (they double as font metrics), so they would be deduped away if used. IDXD dedupes the string pool, so positional key/value pairing is not sound here (the same trap the movie/subtitle map hit) — with one exception: 16 sits immediately before REQUIREMENT (tokens 96 → 97), which is the documented value-before-key adjacency, so the first mission (TimeAttack) requiring bit 16 is well-supported.

Under the 1-based reading that §5.2 now prefers, bit 16 is achievement 16, "Night Ravens Patch" — "for challenging and eradicating the Night Raven Squadron", a stage-15-era award. (The withdrawn 0-based reading gave achievement 17, "Solar System Defense Award"; both are plausible gates for a first challenge mission, which is precisely why the numbering has to be settled by evidence rather than by which story reads better.)

The remaining five values (25–29, all ≥ 24) therefore index word B, not achievements — most plausibly a challenge-clear chain, since there are six challenge missions and word B's bits 0–5 would be 24–29. Recorded as a hypothesis; resolving the per-mission pairing needs the record's binary index section, not the pool.

Negative worth keeping: the requirement text (TimeAttackRequirement, Extra01Requirement, …) is not in GP_CHALLENGE.pak — building a TextIndex over it yields 0 entries and its only wordy payload is embedded font copyright. The config's PATH is dat\GP_CHALLENGE.pak+eng\, a per-language branch, so those keys resolve through a naming scheme the current loader does not reproduce. Reading them would say in plain English what each mission asks for — worth one more attempt via hash::TOC_NAME_SCHEMES.

5.4 Why this matters operationally — and one lead already refuted

If word A / word B are restored from the save, a hand-written save with those bits set unlocks all six challenge missions — and the savegame round-trip is already solved. That would turn the last 42 units of the Route-B harvest into one run instead of an unreachable menu.

The obvious lead is dead. The three stws to +1956 in 0x822AF278 / sub_822C8748 looked promising because they sit in the save serializer's code region — they are on a different object. That one is fetched through 0x822CEB30, has a +2652 flag the code checks first, and stores string pointers at +1956/+2024 (built by 0x822D35F8); a pointer ANDed with 1 << n would be meaningless as a gate. So nothing in the image stores to this singleton's +1956 field-wise, which means it is filled by a bulk copy or by a path not yet found.

Two candidates remain and they need different levers: the save (hand-write it) or the Xbox profile (the emulator's profile data). Note that XEX imports are resolved by ordinal, so the absence of XamUser* name strings in the .pe is not evidence against the profile route. +80 at least is handled as a small struct by address (addi r4, obj, 80 → copy helper 0x82175110, written back via 0x8216FF70), which is what a serialised value object looks like.

6. The unlock condition, from the strings ❔

Three strings say challenge missions are announced, not menu-browsed:

• AVSCRIPT_COMMAND_ATTAINMENT_CHALLENGE_MISSION_CARGO_SCORE — a mission-script command that grants challenge-mission attainment from a cargo score;
• DLG_CHALLENGE_MISSION_AVAILABLE — the "a challenge mission is now available" dialog;
• DLG_GO_CHALLENGE_MISSION_MENU — the prompt that takes you to GamePart 26.

So the gate is plausibly an in-mission achievement, not a title-menu state — which is consistent with Game Status = GAME_CLEAR unlocking nothing (measured, refuted). Neither dialog key is reachable by xref: they are selected by index through a config lookup, exactly like the save screen's sprite keys, so there is nothing to grep back to. Naming the flag needs the challenge-availability check disassembled from the screen side.

7. What this makes actionable

roster_target against the harvested CSV, with the stage families now named:

stage	family	roster units not yet read
S27	challenge	f003_ArrowHead_EX4, e107_AAFrigate_EX4, e010_Attacker_S_EX4, e011_Attacker_B_EX4, e007_Turret_EX4, e008_TurretPlus_EX4
S28	challenge	f004_DeltaSaber_A_Player, f001_DeltaSaber_T_EX5(_el), f003_ArrowHead_EX5, f104_Battleship_EX5, f105_Cruiser_EX5
S25	challenge	e101_SDBattleshipEX, e102_BattleshipEX, e105_CruiserEX, e108_ASFrigateEX
S29	challenge	e101_SDBattleshipEX, e102_BattleshipEX, e105_CruiserEX
S24	challenge	e105_CruiserEX, e106_DestroyerEX
S10	story	e005_ADAN_ElanTypeQ_Margras

⚠️ The S10 row contradicts "the story campaign is complete." One story stage still fields a unit that has never been read. Either S10 was never flown (it is the smallest stage container on the disc — 7 XBG7 resources — so it may be a cutscene stage that is not flyable), or the completeness claim was one unit optimistic. Flagged rather than resolved: it costs one ordinary story run to settle.

Remaining coverage: 42 units missing, of which the rosters above account for ~23. The rest are not in any stage roster table and need a different lever.

Reproduce

cargo run --release -q -p sylpheed-formats --example challenge_map    -- <disc-root>
cargo run --release -q -p sylpheed-formats --example challenge_screen -- <disc-root>
python3 xenia-rs/zq.py dis 0x82184df0 0x82184e30      # the three-way section switch
python3 xenia-rs/zq.py dis 0x82185ed0 0x82185f10      # the same switch, second site
python3 xenia-rs/zq.py dis 0x82189860 0x821899f0      # the challenge availability gate