Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
1013 lines
62 KiB
Markdown
1013 lines
62 KiB
Markdown
# RE backlog
|
||
|
||
Open items that are *not* being worked right now. Each entry says what is wrong or
|
||
unknown, what evidence exists, and what the first step would be. Move an item into
|
||
`INDEX.md` (with a `structures/…md` or a parser + test) once it is actually settled.
|
||
|
||
---
|
||
|
||
## ✅ SOLVED (2026-08-19) — the paint order is a runtime child list, not a table in the file
|
||
|
||
The screen object the game builds at load time holds **two** lists of its
|
||
elements: the declaration-ordered array at `+0x08`, and a **reordered child array
|
||
at `+0x30`** — and the second is the paint order. Read live off the title screen
|
||
and checked against the draw capture: the seven nameable elements sit at child
|
||
slots 0, 6, 7, 13, 16, 17, 22, strictly ascending, in exactly the captured order.
|
||
See [`structures/ui-screen-runtime.md`](structures/ui-screen-runtime.md).
|
||
|
||
**Landed rather than left open** (2026-08-19): the compositor now paints in the
|
||
**measured** order for the two builds that have been read off the running game
|
||
(the title build and the GAME ARTS/SETA/anima splash) and falls back to
|
||
declaration order elsewhere. Rendering that exposed a second defect the same
|
||
capture settles — `kind = 0x4` elements are motion-trail ghosts, absent at rest —
|
||
and the title now composites correctly
|
||
([`captures/title-composited-measured-order.png`](captures/title-composited-measured-order.png)).
|
||
Disc-gated test, checked both ways. The Bevy viewer's UI Screens browser calls
|
||
the same `ui_layout::compose`, so the fix reaches what a person actually looks at
|
||
rather than only the CLI's `screen render` — checked in `iso_loader.rs`
|
||
(`compose_screen`), which also keeps its element table in declaration order, so
|
||
the per-element visibility toggles still line up.
|
||
|
||
**Derivation found (2026-08-19)**: the order sorts by the word at **`+0x08` of
|
||
the T8aD sprite header** — non-decreasing in paint order on both measured
|
||
screens, with no inversion, and on the splash it explains the whole permutation.
|
||
See [`structures/ui-paint-order-key.md`](structures/ui-paint-order-key.md).
|
||
|
||
**Wired into the compositor (2026-08-19)** and regression-checked. `compose`
|
||
sorts by the key for every build except the two whose measured order is hard
|
||
coded. It reorders **341 of 965 builds**, and a disc-gated test asserts every
|
||
composite's draw list is strictly increasing in `(key, declaration index)`.
|
||
Against the two screens the corpus had already verified against the running game
|
||
— the tutorial PAUSE menu and the title main menu — the new order changes 3.8 %
|
||
and 1.1 % of pixels, max delta 45/255, **with no layout change**: only blends
|
||
where translucent sprites overlap.
|
||
|
||
**Still open:**
|
||
|
||
* 🟡 Which order is more faithful on those two verified screens. The difference
|
||
is too small to decide against the committed side-by-side oracle and no fresh
|
||
framebuffer capture of either exists. First thing to check if one is taken.
|
||
* ❔ The tie-break. Two groups share a key and the game paints them in an order
|
||
that is not declaration order; the compositor keeps declaration order there.
|
||
* ❔ What the field's bits mean — `0x8000`/`0x80a0`/`0xa110` look like flag words
|
||
with a layer in some bits, not a plain depth. Sorting the whole word works on
|
||
both measured screens; which bits carry the layer is unknown.
|
||
* ❔ A third measured permutation, to promote "holds on two" to a rule. The
|
||
cheapest is a screen whose object is resident at the same time as the title's.
|
||
* ❔ 341 builds now composite in an order no capture has checked.
|
||
|
||
## ✅ SOLVED (2026-08-19) — `_eff` glows were being dropped as focused states
|
||
|
||
`compose` skips focused-state records, and the flag matched a trailing `f` in
|
||
the name. `_eff` — this UI's word for a glow layer — ends in one. 2 458 elements
|
||
matched; **54** have the base element they would be the focused version of, and
|
||
the other 2 404 across 864 bundles are glows. Requiring the pair recovers 587 of
|
||
them in composable builds; `GP_OPTIONS` went from two floating brackets to an
|
||
actual window. The `opt ` link was tried as a replacement and **refuted** — 221
|
||
targets, 2 suffix-matches, and the targets include `pjnet_bg.rat`.
|
||
See [`structures/ui-focus-and-effect-elements.md`](structures/ui-focus-and-effect-elements.md).
|
||
|
||
## ✅ SOLVED (2026-08-19) — the developer-logo splash can be rendered
|
||
|
||
`is_build` required a `.rat` layout child; the splash has none (its elements name
|
||
their sprites directly). New `is_composable` + opt-in `--all` on the screen
|
||
commands. The splash draws 6/7 elements, glows first, in the order measured off
|
||
the running game — so the second of the two measured paint orders is now
|
||
checkable instead of merely recorded.
|
||
See [`structures/ui-composable-bundles.md`](structures/ui-composable-bundles.md).
|
||
|
||
**Opened by those two:**
|
||
|
||
* ✅ **`.prm` primitives are decoded** (2026-08-19). Untextured full-screen
|
||
colour quads: 0 of 369 has a payload child, `kind & 0x10` ⟺ `.prm` with zero
|
||
exceptions in either direction, 361/369 are exactly 1280×720 at 100 % in the
|
||
corner, and the fill colour is the keyframe's `fade` ARGB — mostly black at
|
||
some alpha, i.e. the fade-to-black / dim / flash layers.
|
||
See [`structures/ui-prm-primitives.md`](structures/ui-prm-primitives.md).
|
||
**Still not composited**, for the reason below.
|
||
* ✅ **`Element::rest()` fixed** (2026-08-19): the resting pose is the **hold** —
|
||
the longest run of consecutive keyframes with an identical pose — not the
|
||
longest gap. A keyframe is the start of a ramp toward the next one, so a long
|
||
gap means the screen spends it *arriving at* the far end. Verified against the
|
||
title framebuffer capture by edge correlation: plateau **0.4597 at shift
|
||
(0,0)**, old rule 0.1511 and only after a (+3,+8) shift. Fixes six title
|
||
elements that rested invisible and the fade quad that rested opaque black.
|
||
See [`structures/ui-resting-pose.md`](structures/ui-resting-pose.md) and
|
||
`tools/re-capture/align_to_capture.py`.
|
||
* ✅ **The keyframe `fade` alpha is applied** (2026-08-19). ARGB, multiplied on
|
||
top of `tint`. Title composite vs the running-game capture: **0.4597 → 0.9538**
|
||
edge correlation at zero shift. No-op on 4 060 of 5 200 sprite elements, hides
|
||
687 transient HUD indicators, blanks **zero** builds. It also exposed a defect
|
||
in the resting rule — a keyframe group carries the screen's *exit* animation
|
||
too, and the tie-break was grabbing it, which erased the word PAUSE; a run
|
||
ending on the last keyframe is now excluded.
|
||
See [`structures/ui-resting-pose.md`](structures/ui-resting-pose.md).
|
||
* 🟡 **The `.prm` quads are drawn, opt-in** (2026-08-19).
|
||
`ComposeOptions::include_primitives` / `screen render --primitives`. On the
|
||
title — the one screen with ground truth — it takes mean luminance from **+18 %
|
||
to −1.3 %** of the capture (76.30 → 63.72 vs 64.58) and mean abs diff 16.07 →
|
||
13.08. Off by default because of the item below.
|
||
* 🟡 **Where a primitive paints — not in the file; measured and tabled**
|
||
(2026-08-19). 🔴 Refuted twice over: the declaration entry's four unread words
|
||
are **constant** (`+28`=0, `+36`=0xffffffff, `+56`=0, `+44` a button ordinal),
|
||
and the bundle carries **no data at all** for a primitive — the menu build
|
||
declares three and has zero RATC children for any of them. The layer comes
|
||
from the game's code. ✅ But it is consistent: `pteff02.prm` implies a key in
|
||
**(0x8010, 0x8040) on both** screens it appears on, `pteff00.prm` past the
|
||
maximum on both, `palogo_eff0.prm` below the minimum. `implied_layer_key`
|
||
records those, and `derived_paint_order` now reproduces the **layer-key
|
||
sequence of all three measured orders**, primitives included (element-for-
|
||
element on 4 of 5 bundle instances; the title differs only inside tied groups).
|
||
❔ **Still open:** primitives whose position has never been measured —
|
||
`pzeff00.prm` and `pceff00.prm` are what wipe the 36 builds, which is why
|
||
`include_primitives` stays off by default. A capture of any screen carrying one
|
||
would close it.
|
||
See [`structures/ui-prm-primitives.md`](structures/ui-prm-primitives.md).
|
||
* 🟡 **The tie-break — refuted six ways, and its cost measured** (2026-08-19).
|
||
Elements sharing a layer key: on the **menu and splash** every tied group comes
|
||
out in declaration order, which the stable sort already gives. The **title** is
|
||
the only screen that discriminates and nothing predicts it (`0x8083` ×5 paints
|
||
`eff1, eff2, eff5, eff3, eff4`). Refuted: declaration order, RATC child order,
|
||
first keyframe time, resting time, resting X/Y, and `T8aD` header words `+00`
|
||
`+04` `+0c` `+10`. RATC child order is a *strict improvement* (7 misplaced
|
||
positions instead of 9, and it recovers the logo grouping) and is exact on the
|
||
other two screens — **not adopted**, because on the one screen that can tell
|
||
them apart it is still wrong.
|
||
**What it costs, exactly:** of 3 disagreeing pairs of drawn elements across all
|
||
three screens, 2 share opaque pixels — `ptlogo_back2eff5` vs `eff3` (22 568 px)
|
||
and vs `eff4` (32 395 px). The residual is one element's blend on one screen,
|
||
and it is pinned by a test. The third pair (`ptlogo2` vs `ptlogo_tm`) overlaps
|
||
by bounding box but shares no opaque pixel; a box test called it a defect and
|
||
the alpha says otherwise.
|
||
✅ **Fourth and fifth screens measured (2026-08-19)**, from `GP_SAVE_LOAD`,
|
||
reachable now that the Canary threading fix makes the menu dependable. The
|
||
9-element slot-list header is **EXACT** under the derived rule — two tied
|
||
groups both in declaration order, unkeyed `.prm` last — and it is the first
|
||
screen outside `GP_TITLE.pak`, so it *confirms* the rule rather than being
|
||
fitted to it. The 13-element save/load frame differs in exactly the two known
|
||
ways: unkeyed `pfbase.tbm` backgrounds paint **first** (now covered by
|
||
`implied_layer_key`), and the `0xb100` group of four paints `10,11,8,12`.
|
||
🔴 **Refuted: the tie-break is not `kind`.** "Descending kind" reproduces
|
||
`10,11,8,12` exactly but fails both title groups. Seven candidates refuted now. 🔴 **Attempted 2026-08-19 and blocked:** advancing
|
||
past the title is intermittent — **1 success in 3 attempts**, same binary,
|
||
same profile, same procedure. ✅ **And now diagnosed one layer deeper:** the
|
||
title *does* act on Ⓐ — the press spawns a slot-`(1F)` loader thread (exactly
|
||
once per run, at the keydown, never in a run that got no press). In the
|
||
successful boot that thread immediately reads six paths from the on-disc cache
|
||
and the menu appears; in a failed boot it starts and issues **no file I/O
|
||
ever**. So "the title ignores Ⓐ" is **withdrawn** — the loader stalls.
|
||
🔴 Refuted as the cause: the cache-flush crash. All four of today's runs have
|
||
**zero** `GUEST-THROW`, `CRASH DUMP` and `Access Violation`; the guest stays
|
||
alive and polling.
|
||
✅ **And now measured to the bottom:** with kernel logging finally on
|
||
(`LOG_MASK=12 LOG_LEVEL=3` — the scripts' `log_mask=13` had Kernel *disabled*,
|
||
which is why no boot log ever held a kernel call), a captured failure shows the
|
||
handler doing everything right — `XamUserGetXUID`, `NtCreateEvent`,
|
||
`ExCreateThread(entry=0x821748F0, CREATE_SUSPENDED)`, `NtResumeThread` — and
|
||
the thread then **never executing**: zero kernel calls of its own, and
|
||
**`00:00:00` host CPU time** while the process runs at 546 %. A spinning thread
|
||
burns CPU; this one never ran. A lost resume is a race, which is the first
|
||
explanation that fits the ~1-in-3 success rate.
|
||
✅ **LOCATED AND FIXED** (canary `a60fe7d11`): `threading_posix.cc` publishes a
|
||
suspended thread's `state_` and its `suspend_count_` in **two separate lock
|
||
scopes**, and `Resume()` waits only for `state_` before testing
|
||
`if (suspend_count_ == 0) return false`. A resumer in that gap drops the
|
||
resume; the thread then waits on the count forever. The Linux `XThread::Resume`
|
||
discards the `false`, so the guest saw success. Fixed by publishing both under
|
||
one lock and waiting without releasing it. On the first clean boot after, the
|
||
loader thread is the **caller** on 20 kernel-call lines with 4 `ResolvePath`
|
||
reads — every failure before had **zero** of both.
|
||
🟡 **Still to show:** that boots now reach the menu *reliably*. The post-fix
|
||
boot is confounded — `skip_intro.sh`'s title test has been wrong twice (an
|
||
absolute pixel against the wrong surface size, then `screen_id.py` matching the
|
||
SQUARE ENIX logo). Now `tools/re-capture/is_title.py` counts the green Ⓐ glyph:
|
||
0 px on the logo, 1520 on a real title. A before/after reliability count over
|
||
several boots is the remaining work.
|
||
See [`canary-scripted-input-traps.md`](canary-scripted-input-traps.md).
|
||
* ❔ **Blend mode.** Everything is straight alpha-over. The near-white flash
|
||
quads (`0xf0ffffff`) and coloured ones (`0x60ff0000`) may be additive. The
|
||
title capture cannot separate the two — its resting elements are all
|
||
`0xffffff`. A screen with a coloured primitive, captured, would.
|
||
|
||
* 🔴 **What marks a focused state in the file — NOT the declaration entry**
|
||
(2026-08-24). Swept disc-wide and asserted: **54** name-paired focused/base
|
||
pairs, **all 54** with identical `kind` (all `0x0`), **no** bit ever set on the
|
||
focused entry and clear on its base, and the only words of the 60-byte entry
|
||
that ever differ are **`+48`/`+52`, the pivot**. The naming pairing is not
|
||
standing in for a field — there is no field.
|
||
❔ Still open: the `.rat` record, the RATC child stream, or the game's code.
|
||
See [`structures/ui-rat-layout.md`](structures/ui-rat-layout.md).
|
||
* 🔴 **What makes a bundle a screen rather than a fragment — not the header**
|
||
(2026-08-24). Swept over all **2 859** composable bundles: **no bit** of the
|
||
flags word at `+0x10` labels a screen (best is bit 13 at **44 %** full-screen
|
||
against a **12.8 %** base; the commonest bit is set on **91 %** of everything).
|
||
The population really is mostly fragments — element counts min 1, **median 2**,
|
||
p95 23, max 56, and only **365** carry a full-screen element — so the
|
||
separation is *shape*, or which bundle references which, and the PAK cannot
|
||
answer the latter directly because its entries are name-hashed.
|
||
✅ **By-product:** the header is not dead space. `+0x18`/`+0x1c` are the
|
||
**design resolution** (1280/720 on 98.7 %, asserted), and ✅ `+0x08` is the
|
||
**animation length** — checked against the keyframe times, which it bounds in
|
||
**2 313 of 2 313** bundles and is attained by **444**, with the ratio peaking
|
||
at 1.0 rather than near zero (that histogram is what rules out a vacuous
|
||
bound). 🟡 `+0x04` (`0x3C0000`/`0x1E0000` = 60.0/30.0 in 16.16) stays amber:
|
||
the only supporting evidence is that the twelve 30.0 bundles cap at `+0x08`=30
|
||
while the 2 843 60.0 ones reach 1 440. ❔ `+0x0c` and `+0x10` unexplained.
|
||
See [`structures/ui-rat-layout.md`](structures/ui-rat-layout.md).
|
||
* 🟡 **What `opt ` links — a record→record reference** (2026-08-24, measured
|
||
disc-wide and asserted). All **1 467** links reachable from a declaration table
|
||
resolve to a **RATC child of their own bundle**, all are `.rat → .rat`, none
|
||
dangle, none self-link. **1 076 (73 %)** are the `<stem>f` focus pattern; the
|
||
rest are **chains** between effect records (`px_bunk_eff01 → pjex_eff →
|
||
pjex_eff07`), which is also why only 227 targets are declared elements — the
|
||
middle of a chain is, the end is not. So focus is the commonest *use*, not the
|
||
meaning. ⚠️ Coverage: 18 718 raw `opt ` tags exist against 1 467 classified —
|
||
`opt_link` reads the first tag of a declared element's record, so ~92 % of
|
||
occurrences sit deeper in the chains and are untested.
|
||
|
||
The investigation that got here follows, kept in full because most of it is
|
||
refutations that were worth the cost.
|
||
|
||
## The dynamic-RE state is not in git, and it was gone
|
||
|
||
**Found 2026-08-23.** Everything the oracle runs on — the baseline emulator
|
||
binary, the Xbox profile, the Stage 02 save, the shader/code caches — lives
|
||
outside both repos and had been wiped. `sylph-doctor` says "all good" without
|
||
any of it; the first symptom is `NO PROFILE on disc` one second into a boot.
|
||
|
||
**Rebuilt and verified by driving it** (LOAD GAME lists the slot → READY ROOM →
|
||
Stage 02 flight): [`dynamic-re-state-restore.md`](dynamic-re-state-restore.md)
|
||
carries the recipe — incremental rebuild of `auto/upstream-baseline` in the
|
||
shared checkout (202 files, no submodule churn), profile bootstrapped with the
|
||
*instrumented* binary's `--create_profile_if_none`, and the committed
|
||
`savedata-stage02-5pct.bin` installed **without** an Xbox content header, which
|
||
`ContentManager::ListContent` does not need.
|
||
|
||
**Open, and cheap:**
|
||
|
||
* ✅ **`launch_mission.sh` finishes unattended again** (2026-08-23, later):
|
||
boot → title → LOAD GAME → slot 01 → READY ROOM → TAKE OFF → `IN FLIGHT at
|
||
34s`, pilot bound and engaging. Two defects, not one: the fixed `sleep 28` for
|
||
LOAD → READY ROOM (now `wait_screen.sh readyroom`), and the READY ROOM being
|
||
**drawn before it is usable** — `Preparing to Sortie`, TAKE OFF greyed, which
|
||
whole-image statistics cannot see (1.7 units of blue) so `take_off_armed.py`
|
||
tests the label. A third defect fell out of the same run: `wait_flight.sh` was
|
||
testing pixel (450,640) "inside the SHIELD bar" of a **1280×720** window, while
|
||
`screenshot` crops to the **1279×675** game surface — it lands between the
|
||
SHIELD and ARMOR bars. That is the long-standing "reported NEVER REACHED FLIGHT
|
||
while plainly in flight" note, now explained and fixed.
|
||
* ❔ **Nothing guarantees this state survives the next container.** If it is
|
||
meant to, the profile + save + `bin/` copies want a home inside a repo or a
|
||
named volume; that is a call for the user, not for an agent.
|
||
|
||
## ✅ SOLVED (2026-08-23) — the mission objective counter is at `0xbdb59668`, and the hunt is automated
|
||
|
||
🔴 **`0xbdb59668` is refuted as a durable address** (2026-08-23): 0 in two
|
||
independent Stage 02 runs while the HUD read `004`/`008`/`012`, on an allocated
|
||
(not sparse) page. The **method** stands; the number does not, and every session
|
||
must re-scan. Two candidates from the re-scan were themselves refuted by the
|
||
corpus's own "verify across a transition you did not select on" rule. Detail and
|
||
the corrected method note (the scan takes **0.9 s** — the trap is the counter
|
||
climbing `004 → 012` in four minutes, not scan duration) in
|
||
[`structures/mission-objective-counter.md`](structures/mission-objective-counter.md).
|
||
|
||
✅ **Settled the same day, once the HUD stopped costing a human round trip.**
|
||
`ob_read.py` reads the three digits by normalised template correlation and
|
||
`ob_hunt.py` runs the whole method unattended; run 4 then gave **one** survivor
|
||
from 35 897, selected on `004 → 008` and verified on the unselected `008 → 012`,
|
||
plus three live paired RAM/HUD readings. The blocker was never the pilot's
|
||
survival — the evidence lives in the first four minutes of the stage, and the
|
||
earlier runs simply could not look often enough to catch the `008` step.
|
||
|
||
🔴 Yesterday's refutation **stands, refined**: the address is not universal (runs
|
||
2 and 3 read a hard 0 there while the HUD counted), but it is not meaningless
|
||
either — it recurs exactly, and run 3's amber candidate sits one 64 KB page below
|
||
it at the same page offset `0x9668`. Rule: try `0xbdb59668`, check it against the
|
||
HUD, re-scan (~5 min, `ob_session.sh`) when it reads 0.
|
||
The follow-on that the autopilot actually needs is unchanged and untouched:
|
||
❔ what the counter counts, and whether an `OB`-badged entity carries a flag in
|
||
its entity object.
|
||
|
||
## What `REMAINING OB` counts — and an in-mission freeze in the way
|
||
|
||
**2026-08-23.** The address is settled (above); *what it counts* is not, and it
|
||
is what the autopilot needs in order to CHOOSE a target. One run in:
|
||
|
||
* 🔴 **Not a live class head-count.** Counter 4 against 8 attackers / 7 friendly
|
||
Delta Sabers / 7 turrets / 1 player — no class matches, no pair sums to it.
|
||
* 🔴 **The per-entity flag is REFUTED** (2026-08-23, final): sample A at counter
|
||
12 over 120 entities gave **2** candidates; the counter went **12 → 11** and
|
||
**neither survived**. Within ±0x400 of the position triple there is no 4-byte
|
||
word whose shared-value population tracks the counter. ❔ **Not** ruled out: a
|
||
single **bit** ORed into a word that also varies (the test needs an exact
|
||
shared value), anything outside that window, and anything on entities
|
||
`entities2` cannot see — it types by position *changing*, so stationary
|
||
objectives are invisible. A bit-level differential is the follow-on. 🔎 **Built**
|
||
(`ob_bitflag.py`, 2026-08-24) and run three times with **no verification
|
||
yet**: one window was spent on a mission that had already ended in GAME OVER,
|
||
one hit the same dead mission, and the third had the counter at a different
|
||
address and then froze after a single filter. Sample A alone gives ~187
|
||
set-polarity + 33 clear-polarity candidates at counter 4, so the second
|
||
transition is the whole test.
|
||
* ⚠️ **Attrition is now the dominant cost of every in-mission item.** Roughly
|
||
half the runs that reach flight end early — a freeze, or a GAME OVER when the
|
||
ACROPOLIS or the craft is lost — and a scan needs the run to survive **two**
|
||
counter transitions. `frozen.in_flight()` at least makes a dead run say so
|
||
immediately instead of waiting out its window.
|
||
* 🔴 **The counter is not "hostiles left" either.** It held at `012` for fifteen
|
||
minutes of live flight while the ADAN population fell 132 → 93.
|
||
* ✅ **But it decrements when the player kills**: `12 → 11` with 411 `fire=1`
|
||
samples and `YOU KILLED WARPLANES 0003` on the HUD — the first decrement seen,
|
||
and the first run where the player's guns were part of the experiment.
|
||
* ✅ **`pilot.py` never fires — ROOT CAUSE FIXED** (2026-08-23):
|
||
`flight_probe.Pad` was writing to the **vgamepad FIFO**, dead since the uinput
|
||
pad was replaced by `--hid=file`, so every axis, trigger and button from every
|
||
flight tool went into a file nothing reads while `/tmp/xenia_pad.txt` stayed
|
||
empty. The craft was never being flown. Verified: full stick went from `0.00°`
|
||
of heading change to `12.72°`, and the attitude matrix from `d 0.0000` to
|
||
`d 0.4438` — which also **refutes** the "stale attitude matrix" suspicion.
|
||
✅ **And the second half: the PITCH stick sign was inverted.** Measured on a
|
||
45° error, both sides, two pulse widths, with the opposite sign as control:
|
||
the pilot's sign grew the error every time, the opposite shrank it every time.
|
||
Fixed, and **the pilot fires** — `fire=1` in **43 of 1 732** samples against 0
|
||
of 13 521, aim down to 2.3°, range median 43 km → 6.3 km, and the HUD's own
|
||
ammunition counters moving. 🟡 Still open: `YOU KILLED` is `0000` after 250 s
|
||
of firing and `REMAINING OB` is still `012` — whether it *destroys* anything is
|
||
the next measurement, and the objective-counter item is waiting on it.
|
||
⚠️ `findrot_global.py`, `findself.py`, `findspeed.py`, `selfstate.py` still
|
||
write to the dead FIFO and `ctrl_probe.py`/`target_probe.py` still use
|
||
`pad.f`; all flagged in place, none repaired.
|
||
See [`pilot-never-fires.md`](pilot-never-fires.md).
|
||
* 🔴 **An in-mission freeze — the item in front of everything else.** Reproduced
|
||
with the Kernel channel on. 🔴 The resume-spin lead is **refuted by its own
|
||
control**: a still-flying run has *more* refused resumes (2 738) than a frozen
|
||
one, because the game runs a self-suspending worker and the host refusal is one
|
||
per cycle by design. ✅ What is established instead: the guest is **spinning,
|
||
not deadlocked** — over 10 s while frozen the main thread is in state `R`
|
||
gaining 409 ticks, guest threads ~680 in total, and **not one kernel call** is
|
||
made. So it is guest code waiting on guest memory. ✅ **Seen from inside** (2026-08-24, gdb): all **79** threads are in a
|
||
**wait** — guest threads in `KeWaitForSingleObject`/`NtWaitForSingleObjectEx`,
|
||
the GPU processor idle, the main thread in `poll()` — while the process still
|
||
burns **1 253 ticks / 10 s**, 403 of them in the **TimerQueue** thread and
|
||
~280 each in two guest threads the backtrace shows *blocked*. So they are
|
||
**cycling through a timed wait**, and the CPU burn is in the kernel layer's
|
||
wait path, not in guest code. No Canary build was needed: `XENIA_BIN` pointing
|
||
at a gdb wrapper keeps the lockfile and satisfies `ptrace_scope=1`.
|
||
🔴 **Reading the wait target from the log is blocked by cost**:
|
||
`KeWaitForSingleObject` is `kHighFrequency` and silent without
|
||
`--log_high_frequency_kernel_calls=true`, and *with* it the emulator is 17
|
||
minutes into a boot with a **black screen** and 175 MB of log. ✅ **Built** (canary `auto/re-wait-timeout-probe` `820696c11`,
|
||
`--log_stuck_waits=true`, binary at `/sylph-home/re/bin/waitprobe/`): counts
|
||
consecutive timeouts on the same object per thread and logs at 100 then every
|
||
500. **Healthy-run control measured** — 27 lines over 25 minutes, all one
|
||
thread polling one Event at guest VA `BE56BB5C` with a ~30 ms timeout, so the
|
||
freeze signal is a **new (thread, object) pair**, not the presence of output.
|
||
🔴 **A freeze WAS caught (2026-08-24) and the probe says nothing.** It froze
|
||
9 s into the watcher's window, in flight, and reported the healthy baseline
|
||
only — one pair, same object VA, no new (thread, object) pair — while the CPU
|
||
signature was unchanged (1 255 ticks/10 s, 401 in the TimerQueue thread). So
|
||
the freeze is **not** a thread looping on timeouts against one object.
|
||
**Two blind spots survive:** waits cycling over *different* objects (the streak
|
||
resets, so they are invisible), or waits that **succeed** rather than time out
|
||
(nothing for a timeout counter to count — which fits the self-suspending worker
|
||
seen cycling successfully in the kernel log).
|
||
✅ **v2 built and its baseline is itself a result** (canary `597740046`): it
|
||
counts every call per thread per second with the distinct-object count and the
|
||
return value. On a healthy 22-minute run the **main thread cleared 500 calls/s
|
||
in 224 windows, peaking at 1 235/s over up to 13 distinct objects**, and the
|
||
result was `X_STATUS_SUCCESS` in **all 314** windows — not one timeout. So the
|
||
game's normal mode is hundreds of *successful* waits a second across many
|
||
objects, which is exactly what v1 could not see.
|
||
🟡 **Consequence:** 500/s is not self-selecting, so the freeze signal must be a
|
||
*different shape* — far above 1 235/s, a new thread, or a non-SUCCESS result.
|
||
**Next:** a frozen sample to compare against; runs 5 and 6 did not freeze
|
||
(GAME OVER at ~22 min, and still healthy at 10 min). **Three** runs in a
|
||
row have now failed to freeze (the third ended in GAME OVER), and the probe's
|
||
healthy control is measured three times — 27, 24 and 36 lines, always the same
|
||
single pair. `frozen.py` detects the state in one call; `ob_hunt.py` /
|
||
`ob_flag.py` abort on it. **Roughly two runs in three.**
|
||
See [`mission-freeze-resume-spin.md`](mission-freeze-resume-spin.md).
|
||
|
||
**First step, revised:** make `pilot.py` shoot, then re-run `ob_flag.py`. The
|
||
freeze is no longer the blocker it looked like — a 25-minute run stayed
|
||
animating — and the actual obstacle is that nothing the pilot does moves the
|
||
counter, so there is never a second sample. If the counter still will not move
|
||
when the player is killing things, the next question is what *does* move it, and
|
||
the objective card's own wording ("shoot down all invading enemy fighters") is
|
||
the place to start. Second step, if that comes back empty: `entities2.typed` only
|
||
sees entities whose position *changes*, so a stationary objective is invisible to
|
||
it, and the enumeration itself would need widening before a null result means
|
||
anything.
|
||
|
||
See [`mission-freeze-and-ob-flag.md`](mission-freeze-and-ob-flag.md).
|
||
|
||
## The declaration table is not a paint order on every screen
|
||
|
||
**Found 2026-08-17**, building the Explorer's UI Screens browser on
|
||
[`ui_layout`](structures/ui-rat-layout.md). **Status: 🔎 open — the pause menu is
|
||
right, the title screen is not.**
|
||
|
||
`ui-rat-layout.md` says the bundle's element declaration table lists elements
|
||
"in back-to-front order", verified 11/11 on the tutorial pause bundle. That
|
||
holds — the tutorial and in-mission PAUSE builds both composite correctly, and
|
||
`pgpeff02a` → parent 3 / `pgp_ttrl_btn10` at (546,288) / the 70 px button pitch
|
||
all reproduce exactly.
|
||
|
||
**`GP_TITLE.pak` build 7 does not.** Painting in declaration order puts
|
||
`ptbase2.t32` (the full-screen background art, element **13**) *on top of* the
|
||
`ptlogo1`/`ptlogo2` wordmarks (elements **0–5**), which the real title screen
|
||
obviously does not do. The pause bundles never caught this because their
|
||
elements barely overlap.
|
||
|
||
**What has been ruled out:** there is no depth/layer key in the 60-byte
|
||
declaration entry. Dumping every word across the title build's 30 entries, the
|
||
unknown fields are constant — `+28` is 0 everywhere, `+44` is `0xffffffff`
|
||
everywhere, `+56` is 0 everywhere — and `+36`, which the doc lists as
|
||
`0xffffffff`, is not a depth either: it is `0`/`1` **only** on the `kind = 0x4`
|
||
repeated-instance entries (`ptlogo1`/`ptlogo2` copies), i.e. an instance index.
|
||
So the order is not recoverable by sorting the table on any field it carries.
|
||
|
||
**What that leaves.** The background group is contiguous — elements 12, 13, 14
|
||
are `pteff00.prm`, `ptbase2.t32`, `pteff04.t32`, and 12 carries `kind = 0x10`,
|
||
a flag no pause element has (theirs are `0x0` / `0x1` / `0x3002`). `pteff02.prm`
|
||
at 17 has it too. So `0x10` marking a `PRMD` primitive, and primitives opening a
|
||
layer that draws beneath what precedes them, is the cheapest hypothesis — but it
|
||
is a **hypothesis**, and "draw the `.prm` group first" would fit this one screen
|
||
without being evidence of anything.
|
||
|
||
**First step:** composite `GP_MISSION_SELECT` / `GP_READY_ROOM` / `GP_OPTIONS`,
|
||
which have both a background and overlapping foreground elements, and see
|
||
whether their background sits at a `0x10`-adjacent index too. Two more screens
|
||
agreeing turns the hypothesis into a rule; one disagreeing kills it. The
|
||
Explorer's `screen render`/`screen info` commands make that a minute's work per
|
||
screen, and the per-element visibility toggles isolate a suspect element.
|
||
|
||
**Meanwhile** the viewer paints in declaration order and does not pretend
|
||
otherwise — a screen whose background lands on top is showing you this bug, not
|
||
a decode failure.
|
||
|
||
### 2026-08-18 — measured against the running game; three orderings refuted, and half the symptom was a different bug
|
||
|
||
**The premise is confirmed by the oracle**, which this entry had not had: a
|
||
framebuffer capture of Canary on the title screen
|
||
([`captures/title-screen-oracle.png`](captures/title-screen-oracle.png)) shows
|
||
the `PROJECT SYLPHEED` wordmarks (elements 0–5) drawn **over** `ptbase2.t32`
|
||
(element 13), which is a full-screen background. Declaration order is therefore
|
||
not the paint order on this screen, and no reading of the element table changes
|
||
that.
|
||
|
||
**But part of what the render showed was not the paint order at all.** In the
|
||
capture `ptbase2` covers the whole screen; the compositor drew it as a
|
||
960×540-visible slab starting at (320,180), because a keyframe's `scale` was
|
||
being grown from the keyframe's corner instead of about the declared **pivot**.
|
||
Fixed, and pinned against the capture by cross-correlation (peak at (0,0)) — see
|
||
[`structures/ui-rat-layout.md`](structures/ui-rat-layout.md). That was a real
|
||
defect worth separating out: it moves **865** of the disc's 5 130 resting
|
||
placements, on every screen, independently of any ordering question.
|
||
|
||
**Three candidate orderings are now dead**, all cheaply:
|
||
|
||
- **The placement region is not a second ordering.** Its keyframe groups carry an
|
||
explicit element index, so they *could* be stored in a different order — they
|
||
are not, on **every** build on the disc (`placement_region_order_is_never_a_second_ordering`,
|
||
>500 builds, identity every time).
|
||
- **The RATC child order is not it either.** For the title build it is the
|
||
declaration order with the `.prm` elements absent — strictly less information,
|
||
and it has no place to put `ptbase2` other than where the table already puts it.
|
||
- **Reverse declaration order is refuted by the same capture**: it would draw
|
||
`ptbase2` (13) over `ptcopyright` (28), and the copyright line is visible.
|
||
|
||
**The `0x10`-adjacency first step was run, and it does not survive.** The
|
||
background *is* adjacent to a `kind = 0x10` `.prm` element on both screens that
|
||
have one — but on **opposite sides**. `GP_TITLE` build 7 is
|
||
`12 pteff00.prm (0x10)`, `13 ptbase2.t32`, `14 pteff04.t32`;
|
||
`GP_MISSION_SELECT` build 0 is `0 px_mission_base.tbm`, `1 px_mission_eff00.prm
|
||
(0x10)`. So "the `.prm` opens a layer that draws beneath what precedes it" cannot
|
||
place both, and no rule keyed on the `.prm`'s position orders the background.
|
||
`GP_READY_ROOM` and `GP_OPTIONS` turned out not to be the third and fourth
|
||
witnesses this entry hoped for: neither of their largest builds carries a `.prm`
|
||
or a full-screen background at all, so they cannot discriminate.
|
||
|
||
**What is still open, stated plainly:** nothing in the bundle has been found that
|
||
orders element 13 behind elements 0–5. Every ordering the file itself carries is
|
||
now either identical to the declaration table or refuted by the capture. The next
|
||
step is no longer static — it is either the guest code that walks this table, or a
|
||
per-draw capture of the title screen showing the order the game submits.
|
||
|
||
**Blocker, checked rather than assumed.** The obvious move is to reuse Canary's
|
||
existing RE instrumentation, which is already in the built binary on
|
||
`sylpheed-re`. Neither hook can answer this:
|
||
|
||
- **`--log_draws`** (`command_processor.cc`) de-dups by a *vertex-declaration
|
||
fingerprint* — shader hash + primitive type + per-stream element
|
||
formats/offsets + index-buffer guest base — and writes each distinct one once.
|
||
A screen's sprites share a declaration, so they collapse; and the record
|
||
carries no texture identity and no per-frame submission order, only first-seen
|
||
order. It is a mesh-format log, not a draw-order log.
|
||
- **The F10 ship capture** does preserve per-draw order within a frame and
|
||
de-dups on `(vertex base, WVP transform, index range)`, which would separate
|
||
the elements — but it **explicitly drops UI draws**:
|
||
`if (pos_off_bytes < 0 …) return; // no float-position stream (UI/effects) —
|
||
skip`. It requires an `f32x3` position attribute, which a 2D quad stream does
|
||
not have.
|
||
|
||
So this needs a **new hook in Canary** — log each draw in submission order with
|
||
its bound texture fetch (or its screen-space quad), gated behind a cvar the way
|
||
the other two are — and therefore a `build-canary` run. That is the cost to
|
||
state up front rather than discover halfway in; it is not a container
|
||
limitation, just a long build plus a title-screen run.
|
||
|
||
### 2026-08-18 (later) — the hook was built and run; the order is now measured
|
||
|
||
`log_ui_draws` exists (Canary branch `auto/re-ui-draw-order`), and the title
|
||
screen's paint order is **ground truth** rather than a candidate:
|
||
[`ui-title-paint-order-capture.md`](ui-title-paint-order-capture.md).
|
||
|
||
Background first, then the `back2` glow pair, then `ptlogo1` + `ptlogo_tm`, then
|
||
`ptlogo2`, then `ptcopyright`, then the `PRESS Ⓐ BUTTON` plate — i.e.
|
||
declaration indices `13, 22|24, 23, 0, 11, 1, 28` and then two elements that are
|
||
**not in build 7 at all**. Two more orderings die on that evidence (keyframe
|
||
start time, resting-keyframe time), and one structural fact reframes the whole
|
||
item: the screen composites **two bundles** (build 7 plus the one-element build
|
||
2 that is the button), so no single build's element table can be the paint order
|
||
whatever its order.
|
||
|
||
**Still open, and now sharper:** the rule. The bundle's 60-byte declaration entry
|
||
carries no depth field (dumped, above); the per-element `.rat` record has not
|
||
been checked for one against this ground truth, and nothing yet explains how the
|
||
two bundles are sequenced. Both are static questions again — the oracle side is
|
||
answered.
|
||
|
||
### 2026-08-18 (third pass) — the bundle does not carry the order at all
|
||
|
||
Three more places checked, all empty, so the static avenue for this item is
|
||
**exhausted** (detail and evidence in
|
||
[`ui-title-paint-order-capture.md`](ui-title-paint-order-capture.md)):
|
||
|
||
- **the geometry has no depth.** A UI quad's attribute 0 is `k_32_32_32_FLOAT`,
|
||
so it carries a Z — and every Z in the capture is 0.00000. Submission order is
|
||
the entire ordering.
|
||
- **the declaration table has no key.** Every word of every entry dumped for the
|
||
build the game actually runs: `+28` 0, `+32` `0xffffffff`, `+36` `0xffffffff`
|
||
(except an instance index on `kind = 0x4`), `+44` `0xffffffff`, `+56` 0.
|
||
- **the placement region has none either**, including its per-group lead word,
|
||
which is 0 for all 24 groups; and the region is followed straight by the RATC
|
||
child stream, so there is no table hiding behind it.
|
||
|
||
Also corrected: the running screen is **build 4**, not the largest build 7 that
|
||
`screen info` defaults to — the two disagree on sprite sizes and the capture
|
||
matches build 4. The conclusions are unchanged, the indices are not.
|
||
|
||
**So the next step is the guest code**, not the file: the splash draw path from
|
||
the emulator-era work (`sub_821CC7A0`, item vtable `0x820b30b4`) submits with
|
||
exactly the PS hash `E59B2B3D` this capture sees, and `xenia-rs/sylpheed.db` is
|
||
available in the container.
|
||
|
||
**And a second screen is NO LONGER BLOCKED, but it is not routine either.** The
|
||
main menu has been reached (screenshot in
|
||
[`canary-scripted-input-traps.md`](canary-scripted-input-traps.md)), so the
|
||
"Ⓐ is dead" reading is withdrawn. **Not routine after all** — see the 2026-08-19 tables in
|
||
[`canary-scripted-input-traps.md`](canary-scripted-input-traps.md): 4 of 5
|
||
successes without `--log_ui_draws`, 0 of 7 with it. An interleaved series
|
||
**refuted the boot-time confound** (the latest title of all, 268 s, accepted Ⓐ;
|
||
a 232 s title refused), and no mechanism exists for the flag — it is read only
|
||
when F10 arms a capture, and F10 was never pressed. The variable was removed rather than
|
||
believed — F10 now arms the capture unconditionally — and with it gone a fresh
|
||
run **still** failed, so the flag is not the cause either. Net: Ⓐ succeeds about
|
||
half the time and nothing measurable predicts which; five explanations are
|
||
eliminated. The input path is now mapped statically (`entry_point` →
|
||
`sub_8216EA68` main loop → `sub_822F1AA8` per-frame input → `sub_82457038` pad
|
||
poll → `XamInputGetKeystrokeEx`), and the poll itself is not state-gated, so the
|
||
gate is in a consumer further up. Until that
|
||
is separated, capturing a screen *and* navigating to it in the same run is not
|
||
dependable. The earlier claim, kept: the title that ends the boot sequence
|
||
accepts a single Ⓐ (2 of 2 at the time); the title the attract loop returns to
|
||
accepts nothing (Ⓐ, START, B,
|
||
BACK, X, Y — dozens of delivered presses). The proposed tell was refuted on the
|
||
way: the two states draw **13 identical quads**, `ptbtn00` included, so they
|
||
differ only to the guest. Recipe: first title after boot, one tap, and never tap
|
||
during the boot (88 presses over the intro ends on a permanent black screen).
|
||
|
||
**The second screen is captured** — the main menu, `GP_TITLE` build 5 — and it
|
||
does not discriminate: its background sits at declaration indices 1–2, so
|
||
"declaration order" and "background first" predict the same sequence. Same
|
||
failure mode as `GP_READY_ROOM`/`GP_OPTIONS`. The next screen worth capturing is
|
||
one whose background sits **late** in its table, as the title's does.
|
||
|
||
The earlier reading, kept because it is what the evidence looked like: the
|
||
title's Ⓐ leads into a content/save path that crashes the guest with
|
||
`--mem_watch=true` and stalls it with `--mem_watch=false`. Three separate traps
|
||
had to be cleared to establish that much — see
|
||
[`canary-scripted-input-traps.md`](canary-scripted-input-traps.md), which also
|
||
carries the reproduction and the fix for two of them.
|
||
|
||
### 2026-08-18 (fourth pass) — the crash is named, and the code avenue is scoped
|
||
|
||
The crash PC resolves to an MSVC `std::map`/`set` erase that throws
|
||
`std::out_of_range` from the game's cache-manager flush, and the trigger is now
|
||
controlled: an **incomplete on-disc cache** throws ~100 s into a boot, a complete
|
||
one never does. The access violation people have been chasing is only that throw
|
||
*returning*, because this build does not unwind guest EH. And the handoff's
|
||
suspect #1 is **eliminated** — cold cache with `--mem_watch=false` throws just
|
||
the same, which withdraws a claim made here yesterday. See
|
||
[`title-crash-stl-tree.md`](title-crash-stl-tree.md). That is a by-product of
|
||
this item and belongs to whoever picks up the crash bisection.
|
||
|
||
For the ordering itself, three more negatives, all recorded in
|
||
[`ui-title-paint-order-capture.md`](ui-title-paint-order-capture.md): the two
|
||
time-based orderings were re-checked against **build 4** (the previous pass used
|
||
build 7's numbers, and build 7 is not what runs) and both still fail on the same
|
||
element; and a fresh candidate — painter's order by resting **Y** — reproduces
|
||
the capture to within a single transposition but is refuted by `ptlogo_tm` and by
|
||
the background, so it is not the rule either.
|
||
|
||
The code avenue is scoped rather than walked: the splash item vtable
|
||
`0x820b30b4` is real (25 slots, three construction sites), RTTI carries **no**
|
||
class names disc-wide, and the format tags are fourcc immediates behind a virtual
|
||
call rather than strings — so this needs a deliberate read of the UI engine, not
|
||
a keyword search.
|
||
|
||
|
||
## Capital ships assemble wrong in the viewer
|
||
|
||
**Reported:** 2026-07-30, by the user. **Status:** ✅ **format-side cause found and
|
||
fixed 2026-08-12** — see below for the 2026-08-10 diagnosis this supersedes.
|
||
|
||
The remaining format-side defect this entry pointed at (a shared turret decoding
|
||
~100× too large in some containers) was real and is gone. `e303_wep_01` decoded
|
||
as a 1600×2100×4800 block in `Stage_S02`, swallowing the `e106` hull; requiring an
|
||
index buffer to cover its vertex pool **exactly** moved it to the block every
|
||
other container agrees on, and it now decodes 49×23×42 everywhere and places at
|
||
±179 on the hull. The same fix repaired `e106_bdy_03` (a 600×1600×998 slab) and
|
||
moved 29 anchors disc-wide, 22 of which had been carrying **another resource's
|
||
geometry under their own name**. See
|
||
[`structures/xbg7-mesh.md`](structures/xbg7-mesh.md).
|
||
|
||
Two things are worth carrying forward rather than closing:
|
||
- the assembler was **audited and exonerated** — every composite node carries
|
||
scale 1.0 and an orthonormal matrix, so nothing on that side inflates a part;
|
||
- **no metric caught this.** Coverage, cross-container consistency, the capture
|
||
oracle and the twin invariant were all green while a 1 600-unit slab sat through
|
||
the ship. It was found by *rendering the ship and looking at it*, and the
|
||
numeric screens written afterwards to automate that check both failed.
|
||
|
||
The 2026-08-10 diagnosis follows, and its viewer-side pointers still stand.
|
||
|
||
**Status (2026-08-10):** 🔎 **the format layer is exonerated.** Runtime captures of three classes (`f105`, `e105`,
|
||
`e106`) at controlled range reproduce `assemble_ship` to ≤0.43 units in translation
|
||
and to 0.000 in rotation for every part that does not move; see
|
||
[`ship-placement-capture-generalisation.md`](ship-placement-capture-generalisation.md)
|
||
§4. So look at **the viewer**: first that it passes `include_external = true`
|
||
(`iso_loader.rs:4012` — with `false` an e106 loses its bridge and both nacelles,
|
||
5 parts instead of 11), then its own transform stack.
|
||
|
||
One real format-side bug was found on the way and is **fixed**: index-less parts
|
||
(`e105_brg`) never matched their `GN_Bridge_01` hardpoint, so 34 (stage, ship) entries
|
||
— `e102`, `e104`, `e105` across Stages 02–29 — assembled without a bridge. The other
|
||
apparent exception (`e105_eng_01` rotation) was an aggregation artefact and is 0.000.
|
||
|
||
The original report and its reasoning follow.
|
||
|
||
The reborn viewer builds capital ships from the split XBG7 parts via
|
||
`sylpheed-formats::ship::assemble_ship`, and they come out **wrong** — parts in the
|
||
wrong place / wrong orientation.
|
||
|
||
**Why this is a real finding and not a known limitation:** the RE write-up
|
||
[`ship-placement-runtime-capture.md`](ship-placement-runtime-capture.md) declares
|
||
static assembly ✅ **exact** as of 2026-07-26 — 9-channel joint tables
|
||
`[TX TY TZ RY RX RZ SX SY SZ]`, Euler `Ry·Rx·Rz`, with
|
||
`ship::tests::static_assembly_matches_runtime_capture` asserting static == runtime
|
||
capture (T < 1.0, R < 0.02). So either the viewer is not using that path, or the
|
||
claim generalises worse than the test suggests.
|
||
|
||
**The likely gap:** that test is **one ship** — the `e106` destroyer, 8 parts plus
|
||
two nacelles, two turrets and the hull mirror. Nothing pins the other classes.
|
||
Rules that were derived from `e106` and could easily be `e106`-specific:
|
||
|
||
- the engine cluster rig mounted at `GN_Engine_01` (two mirrored nacelles + centre);
|
||
- "X-reflect the shared-geometry twin whose lateral offset opposes the geometry's
|
||
dominant side" — a heuristic, not a decoded flag;
|
||
- cross-id turret instancing (×2).
|
||
|
||
**First step (the oracle already exists):** re-run the runtime capture on a *different*
|
||
capital ship and diff static vs captured, exactly as `e106` was done — F10 in the
|
||
`capture-ship-placement` build of `xenia-canary-native` dumps the ship shader's
|
||
`c0..c2` WorldViewProjection rows per part; `WV_ref⁻¹ · WV_p` is the ship-space rigid
|
||
transform, which is ground truth. Pick a class whose rig differs from `e106`
|
||
(different engine count, a ship with no `sld`, a carrier). Then extend
|
||
`static_assembly_matches_runtime_capture` into a per-ship table so a regression in one
|
||
class cannot hide behind `e106` passing.
|
||
|
||
**Also worth ruling out first, cheaply:** that the viewer's own transform stack (scale,
|
||
handedness, node-instance recursion) is not re-breaking a correct assembly — compare
|
||
the viewer's placement against `assemble_ship`'s output directly before blaming the
|
||
format layer.
|
||
|
||
---
|
||
|
||
## Viewer: `include_external` is already on — that hypothesis is dead
|
||
|
||
**Checked 2026-08-11.** The item above names "first that it passes
|
||
`include_external = true` (`iso_loader.rs:4012`)" as the cheap first step. It
|
||
does: `ShipBrowser::show_external` defaults to `true`
|
||
(`iso_loader.rs:643`), the checkbox reads it (`ui.rs:1593`) and it is threaded
|
||
through `RequestShipRender` → `build_ship_model` → `assemble_ship` unchanged
|
||
(`ui.rs:1689`, `iso_loader.rs:4012`). So a ship rendered by the viewer is the
|
||
full external assembly, not the bare hull.
|
||
|
||
The viewer also does not have a transform stack of its own to blame: it bakes
|
||
`ScenePart::apply` straight into the vertices and rotates normals by the same
|
||
`p.m` (`iso_loader.rs:4030-4062`), so its placement is `assemble_ship`'s output
|
||
by construction. What remains unexcluded, in order of cheapness: the mirror
|
||
handling (`det < 0` reverses triangle winding only — a reflected part keeps its
|
||
reflected geometry), `Xbg7Model::models_named` resolving the wrong sub-model when
|
||
a resource name repeats, and the exhaust cones. **Next step is a visual**: the
|
||
diagnosis has run out of things it can settle by reading, so the viewer needs to
|
||
be run against a known-good class (`e106`) and its render compared with
|
||
`ship_render`'s.
|
||
|
||
---
|
||
|
||
## Viewer: the duplicate-resource-name hypothesis is dead too
|
||
|
||
**Checked 2026-08-11.** The diagnosis above left three candidates for why capital
|
||
ships assemble wrong in the viewer: mirror handling, `Xbg7Model::models_named`
|
||
resolving the wrong sub-model when a resource name repeats, and the exhaust
|
||
cones. The second is now **refuted**, and comprehensively.
|
||
|
||
`build_ship_model` resolves each placement with
|
||
`base.iter().find(|m| m.name == p.resource)` (`iso_loader.rs:4041`) — first match
|
||
wins — so a repeated resource name inside a container would silently draw the
|
||
wrong geometry. It cannot happen: decoding **every** XBG7 resource in **all 22
|
||
stage containers** gives **4 603 resources and zero repeated names**.
|
||
|
||
```
|
||
Stage_S01 62/62 Stage_S07 323/323 Stage_S13 290/290 Stage_S25 351/351
|
||
Stage_S02 304/304 Stage_S08 388/388 Stage_S14 22/22 Stage_S26 318/318
|
||
Stage_S03 214/214 Stage_S09 316/316 Stage_S15 386/386 Stage_S27 321/321
|
||
Stage_S04 179/179 Stage_S10 7/7 Stage_S16 65/65 Stage_S28 118/118
|
||
Stage_S05 92/92 Stage_S11 157/157 Stage_S24 162/162 Stage_S29 386/386
|
||
Stage_S06 266/266 Stage_S12 376/376
|
||
```
|
||
|
||
Per-ship it is tighter still: `e106` wants 9 distinct names and decodes exactly
|
||
9 models for 11 placements; `e105` 9 for 9; `f105` 5 for 6. Every placement
|
||
resolves to the one model it names.
|
||
|
||
**So two of the three candidates are gone** (this one and `include_external`),
|
||
leaving **mirror handling** and **the exhaust cones** — and the still-untried
|
||
visual comparison, which remains the right next step.
|
||
|
||
---
|
||
|
||
## Viewer: mirror handling and the exhaust cones are cleared too — the static avenue is exhausted
|
||
|
||
**Checked 2026-08-11.** Both remaining candidates were tested across every ship
|
||
on the disc, and neither shows the reported signature.
|
||
|
||
**Mirror handling.** The concern was that `ScenePart::apply` bakes `R·(S·v)+T`
|
||
while the viewer takes its winding-flip decision from `det(m)` alone and rotates
|
||
normals by `m` alone — both ignoring `s`. A mirror encoded as a *negative scale*
|
||
would then reflect geometry without flipping winding, drawing the part
|
||
inside-out. It never happens: across **1 485 assembled parts** in all 22
|
||
containers there are **22 mirrored parts, every one with `det(m) < 0`**, and
|
||
**zero** parts with a negative scale or a non-uniform one. `apply_twin_mirrors`
|
||
writes the reflection into `m` (negating its X column), so the viewer's flip
|
||
always fires, and ignoring `s` for normals is harmless because `s` is always
|
||
uniform.
|
||
|
||
**Exhaust cones.** These are the one piece of geometry the viewer *invents* — a
|
||
cone at each `GN_Jet`/`GN_SJet` frame, because the real engine geometry is
|
||
recessed and the game draws FX there instead. If they landed wrongly they would
|
||
read exactly as "a part in the wrong place". Across **335 assembled ships, 192 of
|
||
which have exhaust frames, not one cone sits outside its hull's bounding box**
|
||
(tolerance 10 % of the axis span).
|
||
|
||
**Caveat, stated rather than glossed:** "inside the hull box" does not prove a
|
||
cone is *right* — orientation and size are untested, and a cone could be wrong
|
||
while still inside. What it does rule out is the reported symptom for that part.
|
||
|
||
So every mechanism this diagnosis proposed is now eliminated: `include_external`,
|
||
duplicate resource names, mirror handling, and cones-in-the-wrong-place. The
|
||
format and assembly layers pass every static test available, and **the visual
|
||
comparison is no longer merely the next step — it is the only remaining one.**
|
||
Render `e106` in the viewer beside `ship_render`'s output of the same
|
||
`assemble_ship` result; if they agree, the bug is in neither and the original
|
||
report needs re-grounding against a specific ship and a specific expectation.
|
||
|
||
---
|
||
|
||
## ⚠️ DIAGNOSED 2026-08-12 — a mis-decode; the locality fix was written, then withdrawn
|
||
|
||
> Resolution at the end of this entry. Kept in full because the two wrong turns
|
||
> along the way (a "stray volume", then "monotonic anchoring") are the useful part.
|
||
|
||
## ⚠️ The format layer is NOT exonerated — but the cause is a MIS-DECODE, not a stray volume
|
||
|
||
**Found 2026-08-11 by finally doing the visual**, which the notes above kept
|
||
naming as the next step. It overturns their conclusion.
|
||
|
||
Render `e106` from the static assembly and from the baked runtime capture and
|
||
compare — `ship_render` does both:
|
||
|
||
| | placements | parts |
|
||
|---|---|---|
|
||
| runtime capture (ground truth) | **8** | `bdy_01…04`, `brg_01`, `eng_01`, `eng_02`, `wep_02_01` |
|
||
| `assemble_ship(--static)` | **11** | the same 8, **plus `e303_wep_01` ×2** and a second `e106_eng_01` |
|
||
|
||
The render makes it obvious: the destroyer sits inside a white slab that dwarfs
|
||
it ([capture](captures/e106-static-assembly-volume-bug.png)). That slab is
|
||
`e303_wep_01`, and its own geometry is:
|
||
|
||
```
|
||
e303_wep_01 172 verts, 110 tris bounds X[-1000, 600] Y[-1050, 1050] Z[-2400, 2400] 1600 x 2100 x 4800
|
||
e106_wep_02_01 1002 verts, 772 tris 269 x 179 x 417 ← what a real e106 turret looks like
|
||
e106_brg_01 202 verts, 202 tris 105 x 76 x 305
|
||
```
|
||
|
||
**110 triangles, perfectly round axis-aligned bounds, and bigger than the ship it
|
||
is mounted on.**
|
||
|
||
### CORRECTION (same day, one iteration later): it is not a volume — it is a bad decode
|
||
|
||
The first reading of this was that `e303_wep_01` is a collision/trigger volume
|
||
the assembler wrongly draws. **That is wrong, and the evidence that settles it is
|
||
decoding the same resource from every container that holds it:**
|
||
|
||
```
|
||
Stage_S01 172 verts 110 tris X[-24.5, 24.5] Y[0.0, 23.4] Z[-20.8, 20.8] ← 49 × 23 × 42, a turret
|
||
Stage_S02 172 verts 110 tris X[-1000, 600] Y[±1050] Z[±2400] ← 1600 × 2100 × 4800
|
||
Stage_S03… 172 verts 110 tris 49 × 23 × 42 (correct)
|
||
Stage_S08 … 1600 × 2100 × 4800
|
||
Stage_S26 … 1600 × 2100 × 4800
|
||
```
|
||
|
||
Same resource, same vertex and triangle count, **decoding correctly in eleven
|
||
containers and wrongly in exactly three** (`Stage_S02`, `S08`, `S26`). So:
|
||
|
||
- the **placement is legitimate** — `e303_wep_01` is a small shared turret,
|
||
cross-mounted on `e101` and `e106`, and at its true size it is unremarkable;
|
||
- the original author's explanation of the capture's silence (**vbase dedup**)
|
||
stands, and my "dedup would show one, not zero" objection does not survive:
|
||
with the correct decode the turret is small, ordinary geometry;
|
||
- **the defect is in the mesh decoder**, which resolved this resource's vertex
|
||
data differently in three containers.
|
||
|
||
The render and the symptom are real; the cause named in the first version of this
|
||
entry was not.
|
||
|
||
### The part that matters more than this one resource
|
||
|
||
**The decoder can produce wrong geometry without declining.** The
|
||
[XBG7 audit](structures/xbg7-mesh.md) counted 814 resources it *refuses* — a
|
||
visible, honest failure. This is the other kind: `e303_wep_01` decodes "fine" in
|
||
`Stage_S02` and is silently 100× too large. Screening for the signature (bounds
|
||
that are exact multiples of 50 with a span over 1000) flags 22–32 models in each
|
||
of `S02`, `S03`, `S08`, `S26`, `S27` — **but that screen also catches legitimate
|
||
`e_rou_*` composite proxies**, so it is a candidate list, not a count of bugs.
|
||
|
||
**Next:** diff the anchor scan's chosen `vb0` for `e303_wep_01` between
|
||
`Stage_S01` (correct) and `Stage_S02` (wrong) — same resource, two outcomes, so
|
||
the divergence is directly observable — then use whatever distinguishes them to
|
||
add a post-decode sanity check, so a silent 100× mis-decode becomes a decline.
|
||
|
||
### Why this was missed
|
||
|
||
`assemble_ship` treats **every** `rou_*` node in the composite as a drawable
|
||
part, and the doc comment states the cross-id mount as intended behaviour —
|
||
`"INCLUDING repeated instances and cross-id turret mounts (rou_e303_wep_01_root
|
||
×2 on the e106 hull)"` — with
|
||
`ship::tests::static_assembly_matches_runtime_capture` asserting
|
||
`count("e303_wep_01") == 2`. The absence from the capture was explained away as
|
||
vbase dedup, but **dedup would show one instance, not zero**.
|
||
|
||
The test cannot catch it either: it walks the capture's parts and looks each up
|
||
in the static output, so **extra** static placements are invisible to it. That is
|
||
the same shape of gap as the earlier `include_external` hypothesis — a test that
|
||
can only fail one way.
|
||
|
||
### Scope, stated carefully
|
||
|
||
Sweeping all 335 assembled ships for the signature *ship-scale span with under
|
||
400 triangles* flags **20 ships and 58 placements** over 28 distinct resources
|
||
(`e005_ant_*`, `f001_ant_*`, `f002_bdy_*`, `f301_barrel`, `f303_body`,
|
||
`e303_wep_01`, …). **Only the `e106`/`e303_wep_01` case is proven** — by render,
|
||
by capture absence, and by geometry. Some of the others may be legitimately large
|
||
low-poly parts, and each needs the same three checks before being called a bug.
|
||
|
||
**Still true, and independent of the correction above:**
|
||
`static_assembly_matches_runtime_capture` walks the capture's parts and looks each
|
||
up in the static output, so **extra static placements can never fail it**. That is
|
||
worth fixing regardless — it is the same one-way-test shape as the earlier
|
||
`include_external` hypothesis.
|
||
|
||
Also unchanged: only **two** cross-id placements exist fleet-wide (`e303_wep_01`
|
||
on `e101` ×24 and `e106` ×36, across 335 assembled ships), so cross-id mounting is
|
||
a narrow, real feature rather than a systemic guess.
|
||
|
||
---
|
||
|
||
## Resolution (2026-08-12)
|
||
|
||
`anchor_pool_mesh` took the **first** candidate in file order from a
|
||
container-global scan, so a resource could be handed another resource's block
|
||
whenever both shared `(stride, vertex count, index count)`. Fixed by anchoring
|
||
each resource near its **descriptor neighbours** (two-pass: learn, then re-anchor).
|
||
|
||
- it took inconsistency **125 → 51** with coverage unchanged, and made `e106`
|
||
render correctly ([after](captures/e106-static-assembly-fixed.png))
|
||
- **but it flipped the `e106` twin-mirror decision**, which
|
||
`static_assembly_matches_runtime_capture` (ISO-gated, so it skips in a plain
|
||
`cargo test`) catches against the runtime capture — so it was **reverted**
|
||
- the user-reported "capital ships assemble wrong" is therefore **diagnosed, not
|
||
yet fixed**; see [xbg7](structures/xbg7-mesh.md) for what the real fix needs
|
||
|
||
Still open from this entry: `static_assembly_matches_runtime_capture` walks only
|
||
the capture's parts, so **extra** static placements still cannot fail it.
|
||
|
||
### 2026-08-18 — that last line was stale, and the residual gap is now closed too
|
||
|
||
**The one-way-test complaint had already been fixed** when this entry was
|
||
written down: `64d372c` (the revert commit itself) added an extras check, so
|
||
"extra static placements cannot fail it" has not been true since. Checked rather
|
||
than assumed — perturbing the expectation makes the test fail with the real disc
|
||
behind it, so it runs and is live, not a `SYLPHEED_ISO`-less skip.
|
||
|
||
**But it compared a set of resource *names*, which leaves one direction open**: a
|
||
resource placed *twice* when the capture lists it once changes no set. That is
|
||
not hypothetical — a duplicated instance is exactly what a bad node walk emits,
|
||
and the two legitimate duplicates here (`e106_eng_01`, `e303_wep_01`) are the
|
||
reason the test had to special-case counts at all. Replaced with the full
|
||
**multiset**, pinned to the e106 ground truth:
|
||
|
||
```
|
||
e106_bdy_01 1 e106_bdy_02 1 e106_bdy_03 1 e106_bdy_04 1 e106_brg_01 1
|
||
e106_eng_01 2 e106_eng_02 1 e106_wep_02_01 1 e303_wep_01 2
|
||
```
|
||
|
||
— 9 resources, 11 placements, against the capture's 8 dedup'd parts. That
|
||
subsumes the two hand-written count assertions, and it now fails on an extra
|
||
resource, a missing one, **and** a duplicated one. Refuted before believing:
|
||
declaring `e106_bdy_01` twice makes it fail, with the real multiset on the left.
|
||
|
||
**Not closed by this**, and worth keeping separate: the multiset is `e106`'s
|
||
alone. The generalisation this entry originally asked for — a per-ship table so
|
||
a regression in one class cannot hide behind `e106` passing — still needs a
|
||
runtime capture of a *second* capital ship.
|
||
|
||
**That entry's stated blocker is stale** (checked 2026-08-19): the ship capture
|
||
is in the current build — `RequestShipCaptureFrame` / `CaptureShipDrawForRE` are
|
||
in `command_processor.cc` on `auto/re-ui-draw-order`, and F10 wrote a 2.9 MB
|
||
`xenia_ship_capture_01.log` from this session's binary. No separate
|
||
`capture-ship-placement` build is needed.
|
||
|
||
**Update 2026-08-19: the mission is now REACHABLE.** With the Canary threading
|
||
fix, `tutorial_launch.sh` drives boot → title → menu → TUTORIAL and the mission
|
||
**loads and renders** (flight HUD, "Go to the box on your screen"). It then
|
||
freezes under 13 243 crash dumps, all at `0x82307128`, preceded by exactly one
|
||
guest C++ throw — identical frames 6 s apart, no new dumps, 400 % CPU. So the
|
||
blocker moved from "cannot reach a mission" to "the mission freezes". 🔴 **The cache is REFUTED as the cure** (3 runs): the
|
||
missing entry `\aab216c3\6` was real and got written, and the run with a complete
|
||
cache stormed anyway — 11 497 dumps, all `0x82307128`. ✅ **But a usable window
|
||
exists:** both post-cache runs ran the mission with exactly **2 crashes for
|
||
56–80 s** before the storm, where the first run was at 641 by t+24 s. The ship
|
||
capture needs `F10` armed *inside* that window. ✅ **Done, and the mission ran
|
||
with ZERO crashes** — first clean mission run, fully rendered. 🔴 **But the
|
||
capture contains no ship geometry**: 181 deduped draws, 180 sharing one vertex
|
||
shader, all screen-space, none 3D — against a known-good 2.9 MB capture from an
|
||
earlier session. The 8 000-draw budget was not the limit and the scene *was*
|
||
rendering. ⚠️ The "cache refuted" claim above is **overstated**: this run used
|
||
the same complete cache as `tut4` and got 0 crashes vs 11 497, so variance
|
||
dominates. 🔴 **Corrected:** nothing is broken. Ship
|
||
geometry is `stride=24 prim=4` with a large vcount (`vcount=10891` for a real
|
||
one); this capture has one `prim=4 vcount=6` quad and the 2.9 MB "known-good"
|
||
file has **no `prim=4` at all** — it is a **UI** capture (1 303 of 1 582 draws
|
||
are `stride=24 prim=13`, the UI sprite shader). The earlier "3D draws" test
|
||
counted UI sprite coordinates as 3D. The capture recorded what was on screen, and
|
||
the tutorial's opening has **no capital ship**. **What remains** is what the
|
||
capture doc always said: play into a real mission and frame a ship side-on —
|
||
gameplay driving, not a menu step, and the original was taken on HW Vulkan where
|
||
this container has lavapipe. 🔴 The resume-refused lead (1 663 on one
|
||
thread) is **REFUTED**: that thread did execute, and `KeWaitForSingleObject` /
|
||
`NtWaitForSingleObjectEx` are `kHighFrequency`, which is unlogged unless
|
||
`--log_high_frequency_kernel_calls=true` — so a parked thread is invisible and
|
||
the refusals are just the guest kicking a worker blocked on an object. Method
|
||
note: the title-loader finding rested on **host CPU time**, not log silence,
|
||
which is why it stands and this did not.
|
||
|
||
What blocked it before was **the cache-flush crash**, not navigation — measured
|
||
2026-08-19. `tools/re-capture/tutorial_launch.sh` (which retries whole boots,
|
||
because re-pressing the same title never works) gets all the way from the title
|
||
through the main menu to **DIFFICULTY** and then **SELECT DATA**, and the guest
|
||
dies there at `0x82307128` — the same `std::map`/`set` erase as the boot-time
|
||
throw, 537 stacked dumps, with `--mem_watch=false`. See
|
||
[`title-crash-stl-tree.md`](title-crash-stl-tree.md).
|
||
|
||
So a second capital-ship capture needs that crash dealt with first. Everything up
|
||
to the save-slot screen is now scripted and works, and one run got *past* it —
|
||
`SELECT DATA` reached with zero crashes, slot chosen, the game proceeding into a
|
||
cinematic — before crashing at the same `0x82307128`. The crash is intermittent
|
||
in **where** it fires, not whether, so there is no menu route around it. See
|
||
[`title-crash-stl-tree.md`](title-crash-stl-tree.md) for the end-to-end
|
||
measurement and for what has been ruled out (`--mem_watch=false`, twice).
|