f481ccbb5f
The three sections recorded as undecoded are fixed-stride arrays and counts[0..2] are their record counts: 12, 96 and 48 bytes. Section 1's remainder is exactly 0 in 11/11 objects and section 2's exactly 96 in 11/11, which is what makes these strides rather than a coincidence of division. Section 0 is a point list: 13467 of 13467 records lie inside their object's own header bounding box. Section 2 is a plane list, 12 f32: four zeros, a unit normal (|n|=1 in 133573/133573), a signed distance, a point inside the bbox (133573/133573), and a trailing 1.0 (133573/133573). The decisive check is algebraic -- n.p + d must vanish for a real plane, and over all 133573 records the relative residual has a median of 2.29e-08 and a maximum of 2.15e-07. That is float round-off, not a fit. So a REGN object carries a point list and a plane list beside its uniform grid, which fits collision or region-boundary geometry and sits next to MCOL. Still open: section 1 (96 B, 60631 records), what queries the planes, the zeros at [0..3], and the constant 96-byte tail.
2166 lines
146 KiB
Markdown
2166 lines
146 KiB
Markdown
# RE backlog
|
||
|
||
Open items that are *not* being worked right now. Each entry says what is wrong or
|
||
unknown, what evidence exists, and what the first step would be. Move an item into
|
||
`INDEX.md` (with a `structures/…md` or a parser + test) once it is actually settled.
|
||
|
||
---
|
||
|
||
## ❌ WITHDRAWN (2026-08-26, same day) — the Japanese voice banks are NOT a separate format
|
||
|
||
The `sound.pak` census (see
|
||
[`structures/sound-pak-contents.md`](structures/sound-pak-contents.md)) shows
|
||
the archive is 5 100 `jpn\` banks + 4 382 `eng\` banks + 35 music banks, and
|
||
that **the leading-region rule holds for 0 of the 5 100 Japanese banks** — 4 301
|
||
have a first `RIFF` at an offset that is not `1392 + n·2048`, and 799 have no
|
||
`RIFF` at all. The rule was derived on English banks and reproduces those
|
||
exactly, so this is a limit of the finding, not a defect in the reader.
|
||
|
||
Since `FILES` names the **Japanese** set and the game substitutes the language
|
||
directory, the undecoded half is the one the table actually points at.
|
||
|
||
**Resolved by that first step.** Scanning instead of assuming showed the offset
|
||
is simply `first_riff % 2048`, which takes four values disc-wide (1392, 1468,
|
||
1600, 1728) by language and subdirectory. The Japanese banks decode fine at
|
||
their own offset — median **70×** more audio on a 140-bank FFmpeg-verified
|
||
sample — and the same bug was silently affecting 1 873 `eng\Voice` banks. I had
|
||
mistaken a constant derived from `eng\etc\` for a property of the format. See
|
||
[`structures/slb-data-offset.md`](structures/slb-data-offset.md).
|
||
|
||
**Closed 2026-08-26**: the 2 unaccounted TOC entries are `static.slb` and
|
||
`Pj_Silph.xgs`, named by the same table's `BANK_SE` and `SETTINGS` records —
|
||
`sound.pak` is now 9 519 of 9 519 accounted for.
|
||
|
||
**Closed**: the 55 "early `RIFF`" English banks are not an anomaly — all 55 sit
|
||
at *exactly* 1392 behind a zero-filled header, i.e. a zero-length leading
|
||
region, which both the old code and the new derivation already handle.
|
||
|
||
**Closed 2026-08-26**: the 28 offset-scan ties. A bank's `seek` chunk sits on a
|
||
packet boundary, so `seek_pos % 2048` is a second and structural derivation of
|
||
the offset (99.97 % on the labelled set). It resolves 26 of the 28 correctly and
|
||
none wrongly; the combined rule is 99.95 %.
|
||
|
||
**Both remaining questions closed the same day**, and both by correcting an
|
||
error of mine rather than by finding something new:
|
||
|
||
* ❌ **"69.8 % of banks declare more `data` than they store"** — the declared
|
||
sizes are **honest**. Every `RIFF`-bearing entry on the disc carries `seek`
|
||
magic at exactly `data_at + declared_size` with `packet_count × 2048 ==
|
||
declared`, **7 620 / 7 620**. The TOC window is simply not the wave boundary.
|
||
My supporting case, `VOICE_TCAF_608`, was not truncated either: it is **stereo**
|
||
(170 of 8 021 banks are) and I decoded it as mono, which yields one frame. Read
|
||
correctly it is 33.96 s, agreeing with both length signals inside the bank.
|
||
* ❌ **"the offset takes four values by directory"** — the *cause* was wrong.
|
||
`X = (cumulative start of the .pNN segment) mod 2048`; the per-directory table
|
||
was a correlation, because directories cluster into segments. There is no
|
||
header: those bytes are the previous bank's audio.
|
||
|
||
Both are written up in
|
||
[`structures/slb-data-offset.md`](structures/slb-data-offset.md), including the
|
||
`seek` layout (little-endian, packet count at `+12`) and the boundary recipe.
|
||
|
||
---
|
||
|
||
## ✅ SOLVED (2026-08-26) — the mission scripts are readable as dialogue
|
||
|
||
Built-in **64** (`request_script_message`) stages a symbol-table-1 reference in
|
||
slot 0; the symbol is a message id (`MSG_VOICE_D_257`), and the caption table
|
||
holds its text under `<id>_000_00`. **2683 of 2683** call sites across all 28
|
||
stage scripts resolve — every one stages slot 0, every operand is type 6, every
|
||
name has text. 1338 distinct names in six families.
|
||
|
||
This was only reachable after `build_caption_text` was switched to the IXUD
|
||
field table on 2026-08-25 (537 → 8800 lines); before that most of these names
|
||
had nothing to resolve to, which is why the item sat open.
|
||
|
||
Write-up + tool: [`structures/isl-message-dialogue-link.md`](structures/isl-message-dialogue-link.md),
|
||
`tools/re-capture/isl_dialogue.py`.
|
||
|
||
**Answered the same day**: what drives `MSG_DEMO_*` (its own IDXD tables in the
|
||
language packs — see [`structures/cutscene-message-table.md`](structures/cutscene-message-table.md)),
|
||
and which bank voices a line (`tables.pak`'s cue index — see
|
||
[`structures/sound-cue-table.md`](structures/sound-cue-table.md); 1 326 of 1 338
|
||
script message ids resolve to a `.slb`).
|
||
|
||
**Also closed (2026-08-26)**: the 17 cue-less cutscene pages and the 5
|
||
duplicated `MSG_DEMO` records are one and the same thing — the resupply lines,
|
||
whose audio binds through the movie path rather than a `DEMO_nnn` cue. Their
|
||
repeat counts match the movie-slot counts recorded for `VOICE_D_450`–`454`, 5 of
|
||
5, from a separately decoded container.
|
||
|
||
**Still open from this**: nothing about the wrong-recording case — that was
|
||
already settled in `voice-bank-leading-region.md` (generic line, correct
|
||
binding) and my first write-up wrongly reopened it; the other five languages; and play order, since the tool prints
|
||
script order without resolving phases or branches. Multi-page captions were
|
||
settled the same day — and in doing so refuted the first version of the tool,
|
||
which truncated 356 of the 1338 names to their opening utterance.
|
||
|
||
---
|
||
|
||
## ✅ SOLVED (2026-08-19) — the paint order is a runtime child list, not a table in the file
|
||
|
||
The screen object the game builds at load time holds **two** lists of its
|
||
elements: the declaration-ordered array at `+0x08`, and a **reordered child array
|
||
at `+0x30`** — and the second is the paint order. Read live off the title screen
|
||
and checked against the draw capture: the seven nameable elements sit at child
|
||
slots 0, 6, 7, 13, 16, 17, 22, strictly ascending, in exactly the captured order.
|
||
See [`structures/ui-screen-runtime.md`](structures/ui-screen-runtime.md).
|
||
|
||
**Landed rather than left open** (2026-08-19): the compositor now paints in the
|
||
**measured** order for the two builds that have been read off the running game
|
||
(the title build and the GAME ARTS/SETA/anima splash) and falls back to
|
||
declaration order elsewhere. Rendering that exposed a second defect the same
|
||
capture settles — `kind = 0x4` elements are motion-trail ghosts, absent at rest —
|
||
and the title now composites correctly
|
||
([`captures/title-composited-measured-order.png`](captures/title-composited-measured-order.png)).
|
||
Disc-gated test, checked both ways. The Bevy viewer's UI Screens browser calls
|
||
the same `ui_layout::compose`, so the fix reaches what a person actually looks at
|
||
rather than only the CLI's `screen render` — checked in `iso_loader.rs`
|
||
(`compose_screen`), which also keeps its element table in declaration order, so
|
||
the per-element visibility toggles still line up.
|
||
|
||
**Derivation found (2026-08-19)**: the order sorts by the word at **`+0x08` of
|
||
the T8aD sprite header** — non-decreasing in paint order on both measured
|
||
screens, with no inversion, and on the splash it explains the whole permutation.
|
||
See [`structures/ui-paint-order-key.md`](structures/ui-paint-order-key.md).
|
||
|
||
**Wired into the compositor (2026-08-19)** and regression-checked. `compose`
|
||
sorts by the key for every build except the two whose measured order is hard
|
||
coded. It reorders **341 of 965 builds**, and a disc-gated test asserts every
|
||
composite's draw list is strictly increasing in `(key, declaration index)`.
|
||
Against the two screens the corpus had already verified against the running game
|
||
— the tutorial PAUSE menu and the title main menu — the new order changes 3.8 %
|
||
and 1.1 % of pixels, max delta 45/255, **with no layout change**: only blends
|
||
where translucent sprites overlap.
|
||
|
||
**Still open:**
|
||
|
||
* 🟡 Which order is more faithful on those two verified screens. The difference
|
||
is too small to decide against the committed side-by-side oracle and no fresh
|
||
framebuffer capture of either exists. First thing to check if one is taken.
|
||
* ❔ The tie-break. Two groups share a key and the game paints them in an order
|
||
that is not declaration order; the compositor keeps declaration order there.
|
||
* ❔ What the field's bits mean — `0x8000`/`0x80a0`/`0xa110` look like flag words
|
||
with a layer in some bits, not a plain depth. Sorting the whole word works on
|
||
both measured screens; which bits carry the layer is unknown.
|
||
* ❔ A third measured permutation, to promote "holds on two" to a rule. The
|
||
cheapest is a screen whose object is resident at the same time as the title's.
|
||
* ❔ 341 builds now composite in an order no capture has checked.
|
||
|
||
## ✅ SOLVED (2026-08-19) — `_eff` glows were being dropped as focused states
|
||
|
||
`compose` skips focused-state records, and the flag matched a trailing `f` in
|
||
the name. `_eff` — this UI's word for a glow layer — ends in one. 2 458 elements
|
||
matched; **54** have the base element they would be the focused version of, and
|
||
the other 2 404 across 864 bundles are glows. Requiring the pair recovers 587 of
|
||
them in composable builds; `GP_OPTIONS` went from two floating brackets to an
|
||
actual window. The `opt ` link was tried as a replacement and **refuted** — 221
|
||
targets, 2 suffix-matches, and the targets include `pjnet_bg.rat`.
|
||
See [`structures/ui-focus-and-effect-elements.md`](structures/ui-focus-and-effect-elements.md).
|
||
|
||
## ✅ SOLVED (2026-08-19) — the developer-logo splash can be rendered
|
||
|
||
`is_build` required a `.rat` layout child; the splash has none (its elements name
|
||
their sprites directly). New `is_composable` + opt-in `--all` on the screen
|
||
commands. The splash draws 6/7 elements, glows first, in the order measured off
|
||
the running game — so the second of the two measured paint orders is now
|
||
checkable instead of merely recorded.
|
||
See [`structures/ui-composable-bundles.md`](structures/ui-composable-bundles.md).
|
||
|
||
**Opened by those two:**
|
||
|
||
* ✅ **`.prm` primitives are decoded** (2026-08-19). Untextured full-screen
|
||
colour quads: 0 of 369 has a payload child, `kind & 0x10` ⟺ `.prm` with zero
|
||
exceptions in either direction, 361/369 are exactly 1280×720 at 100 % in the
|
||
corner, and the fill colour is the keyframe's `fade` ARGB — mostly black at
|
||
some alpha, i.e. the fade-to-black / dim / flash layers.
|
||
See [`structures/ui-prm-primitives.md`](structures/ui-prm-primitives.md).
|
||
**Still not composited**, for the reason below.
|
||
* ✅ **`Element::rest()` fixed** (2026-08-19): the resting pose is the **hold** —
|
||
the longest run of consecutive keyframes with an identical pose — not the
|
||
longest gap. A keyframe is the start of a ramp toward the next one, so a long
|
||
gap means the screen spends it *arriving at* the far end. Verified against the
|
||
title framebuffer capture by edge correlation: plateau **0.4597 at shift
|
||
(0,0)**, old rule 0.1511 and only after a (+3,+8) shift. Fixes six title
|
||
elements that rested invisible and the fade quad that rested opaque black.
|
||
See [`structures/ui-resting-pose.md`](structures/ui-resting-pose.md) and
|
||
`tools/re-capture/align_to_capture.py`.
|
||
* ✅ **The keyframe `fade` alpha is applied** (2026-08-19). ARGB, multiplied on
|
||
top of `tint`. Title composite vs the running-game capture: **0.4597 → 0.9538**
|
||
edge correlation at zero shift. No-op on 4 060 of 5 200 sprite elements, hides
|
||
687 transient HUD indicators, blanks **zero** builds. It also exposed a defect
|
||
in the resting rule — a keyframe group carries the screen's *exit* animation
|
||
too, and the tie-break was grabbing it, which erased the word PAUSE; a run
|
||
ending on the last keyframe is now excluded.
|
||
See [`structures/ui-resting-pose.md`](structures/ui-resting-pose.md).
|
||
* 🟡 **The `.prm` quads are drawn, opt-in** (2026-08-19).
|
||
`ComposeOptions::include_primitives` / `screen render --primitives`. On the
|
||
title — the one screen with ground truth — it takes mean luminance from **+18 %
|
||
to −1.3 %** of the capture (76.30 → 63.72 vs 64.58) and mean abs diff 16.07 →
|
||
13.08. Off by default because of the item below.
|
||
* 🟡 **Where a primitive paints — not in the file; measured and tabled**
|
||
(2026-08-19). 🔴 Refuted twice over: the declaration entry's four unread words
|
||
are **constant** (`+28`=0, `+36`=0xffffffff, `+56`=0, `+44` a button ordinal),
|
||
and the bundle carries **no data at all** for a primitive — the menu build
|
||
declares three and has zero RATC children for any of them. The layer comes
|
||
from the game's code. ✅ But it is consistent: `pteff02.prm` implies a key in
|
||
**(0x8010, 0x8040) on both** screens it appears on, `pteff00.prm` past the
|
||
maximum on both, `palogo_eff0.prm` below the minimum. `implied_layer_key`
|
||
records those, and `derived_paint_order` now reproduces the **layer-key
|
||
sequence of all three measured orders**, primitives included (element-for-
|
||
element on 4 of 5 bundle instances; the title differs only inside tied groups).
|
||
❔ **Still open:** primitives whose position has never been measured —
|
||
`pzeff00.prm` and `pceff00.prm` are what wipe the 36 builds, which is why
|
||
`include_primitives` stays off by default. A capture of any screen carrying one
|
||
would close it.
|
||
See [`structures/ui-prm-primitives.md`](structures/ui-prm-primitives.md).
|
||
* 🟡 **The tie-break — refuted six ways, and its cost measured** (2026-08-19).
|
||
Elements sharing a layer key: on the **menu and splash** every tied group comes
|
||
out in declaration order, which the stable sort already gives. The **title** is
|
||
the only screen that discriminates and nothing predicts it (`0x8083` ×5 paints
|
||
`eff1, eff2, eff5, eff3, eff4`). Refuted: declaration order, RATC child order,
|
||
first keyframe time, resting time, resting X/Y, and `T8aD` header words `+00`
|
||
`+04` `+0c` `+10`. RATC child order is a *strict improvement* (7 misplaced
|
||
positions instead of 9, and it recovers the logo grouping) and is exact on the
|
||
other two screens — **not adopted**, because on the one screen that can tell
|
||
them apart it is still wrong.
|
||
**What it costs, exactly:** of 3 disagreeing pairs of drawn elements across all
|
||
three screens, 2 share opaque pixels — `ptlogo_back2eff5` vs `eff3` (22 568 px)
|
||
and vs `eff4` (32 395 px). The residual is one element's blend on one screen,
|
||
and it is pinned by a test. The third pair (`ptlogo2` vs `ptlogo_tm`) overlaps
|
||
by bounding box but shares no opaque pixel; a box test called it a defect and
|
||
the alpha says otherwise.
|
||
✅ **Fourth and fifth screens measured (2026-08-19)**, from `GP_SAVE_LOAD`,
|
||
reachable now that the Canary threading fix makes the menu dependable. The
|
||
9-element slot-list header is **EXACT** under the derived rule — two tied
|
||
groups both in declaration order, unkeyed `.prm` last — and it is the first
|
||
screen outside `GP_TITLE.pak`, so it *confirms* the rule rather than being
|
||
fitted to it. The 13-element save/load frame differs in exactly the two known
|
||
ways: unkeyed `pfbase.tbm` backgrounds paint **first** (now covered by
|
||
`implied_layer_key`), and the `0xb100` group of four paints `10,11,8,12`.
|
||
🔴 **Refuted: the tie-break is not `kind`.** "Descending kind" reproduces
|
||
`10,11,8,12` exactly but fails both title groups. Seven candidates refuted now. 🔴 **Attempted 2026-08-19 and blocked:** advancing
|
||
past the title is intermittent — **1 success in 3 attempts**, same binary,
|
||
same profile, same procedure. ✅ **And now diagnosed one layer deeper:** the
|
||
title *does* act on Ⓐ — the press spawns a slot-`(1F)` loader thread (exactly
|
||
once per run, at the keydown, never in a run that got no press). In the
|
||
successful boot that thread immediately reads six paths from the on-disc cache
|
||
and the menu appears; in a failed boot it starts and issues **no file I/O
|
||
ever**. So "the title ignores Ⓐ" is **withdrawn** — the loader stalls.
|
||
🔴 Refuted as the cause: the cache-flush crash. All four of today's runs have
|
||
**zero** `GUEST-THROW`, `CRASH DUMP` and `Access Violation`; the guest stays
|
||
alive and polling.
|
||
✅ **And now measured to the bottom:** with kernel logging finally on
|
||
(`LOG_MASK=12 LOG_LEVEL=3` — the scripts' `log_mask=13` had Kernel *disabled*,
|
||
which is why no boot log ever held a kernel call), a captured failure shows the
|
||
handler doing everything right — `XamUserGetXUID`, `NtCreateEvent`,
|
||
`ExCreateThread(entry=0x821748F0, CREATE_SUSPENDED)`, `NtResumeThread` — and
|
||
the thread then **never executing**: zero kernel calls of its own, and
|
||
**`00:00:00` host CPU time** while the process runs at 546 %. A spinning thread
|
||
burns CPU; this one never ran. A lost resume is a race, which is the first
|
||
explanation that fits the ~1-in-3 success rate.
|
||
✅ **LOCATED AND FIXED** (canary `a60fe7d11`): `threading_posix.cc` publishes a
|
||
suspended thread's `state_` and its `suspend_count_` in **two separate lock
|
||
scopes**, and `Resume()` waits only for `state_` before testing
|
||
`if (suspend_count_ == 0) return false`. A resumer in that gap drops the
|
||
resume; the thread then waits on the count forever. The Linux `XThread::Resume`
|
||
discards the `false`, so the guest saw success. Fixed by publishing both under
|
||
one lock and waiting without releasing it. On the first clean boot after, the
|
||
loader thread is the **caller** on 20 kernel-call lines with 4 `ResolvePath`
|
||
reads — every failure before had **zero** of both.
|
||
🟡 **Still to show:** that boots now reach the menu *reliably*. The post-fix
|
||
boot is confounded — `skip_intro.sh`'s title test has been wrong twice (an
|
||
absolute pixel against the wrong surface size, then `screen_id.py` matching the
|
||
SQUARE ENIX logo). Now `tools/re-capture/is_title.py` counts the green Ⓐ glyph:
|
||
0 px on the logo, 1520 on a real title. A before/after reliability count over
|
||
several boots is the remaining work.
|
||
See [`canary-scripted-input-traps.md`](canary-scripted-input-traps.md).
|
||
* ❔ **Blend mode.** Everything is straight alpha-over. The near-white flash
|
||
quads (`0xf0ffffff`) and coloured ones (`0x60ff0000`) may be additive. The
|
||
title capture cannot separate the two — its resting elements are all
|
||
`0xffffff`. A screen with a coloured primitive, captured, would.
|
||
|
||
* 🔴 **What marks a focused state in the file — NOT the declaration entry**
|
||
(2026-08-24). Swept disc-wide and asserted: **54** name-paired focused/base
|
||
pairs, **all 54** with identical `kind` (all `0x0`), **no** bit ever set on the
|
||
focused entry and clear on its base, and the only words of the 60-byte entry
|
||
that ever differ are **`+48`/`+52`, the pivot**. The naming pairing is not
|
||
standing in for a field — there is no field.
|
||
❔ Still open: the `.rat` record, the RATC child stream, or the game's code.
|
||
See [`structures/ui-rat-layout.md`](structures/ui-rat-layout.md).
|
||
* 🔴 **What makes a bundle a screen rather than a fragment — not the header**
|
||
(2026-08-24). Swept over all **2 859** composable bundles: **no bit** of the
|
||
flags word at `+0x10` labels a screen (best is bit 13 at **44 %** full-screen
|
||
against a **12.8 %** base; the commonest bit is set on **91 %** of everything).
|
||
The population really is mostly fragments — element counts min 1, **median 2**,
|
||
p95 23, max 56, and only **365** carry a full-screen element — so the
|
||
separation is *shape*, or which bundle references which, and the PAK cannot
|
||
answer the latter directly because its entries are name-hashed.
|
||
✅ **By-product:** the header is not dead space. `+0x18`/`+0x1c` are the
|
||
**design resolution** (1280/720 on 98.7 %, asserted), and ✅ `+0x08` is the
|
||
**animation length** — checked against the keyframe times, which it bounds in
|
||
**2 313 of 2 313** bundles and is attained by **444**, with the ratio peaking
|
||
at 1.0 rather than near zero (that histogram is what rules out a vacuous
|
||
bound). 🟡 `+0x04` (`0x3C0000`/`0x1E0000` = 60.0/30.0 in 16.16) stays amber:
|
||
the only supporting evidence is that the twelve 30.0 bundles cap at `+0x08`=30
|
||
while the 2 843 60.0 ones reach 1 440. ✅ `+0x0c` is **two u16s forming an ordered
|
||
interval** — `high < low` in **2 985/2 985**, both bounded by the animation
|
||
length; ✅ it is **authored, not the keyframe min/max** — that
|
||
reading holds in 6 of 2 985 (0.2 %), and the apparent 34 % match on `high` is a
|
||
coincidence of zeros. The window is narrow, a median 2 % of the keyframe span.
|
||
🟡 what the window means stays open. `+0x10` is ✅ a zero `u16` plus a 16-bit flag word at
|
||
`+0x12` (high half zero in 2 985/2 985); ❔ the bit meanings, with four
|
||
measurable predicates now excluded.
|
||
See [`structures/ui-rat-layout.md`](structures/ui-rat-layout.md).
|
||
* 🟡 **What `opt ` links — a record→record reference** (2026-08-24, measured
|
||
disc-wide and asserted). All **1 467** links reachable from a declaration table
|
||
resolve to a **RATC child of their own bundle**, all are `.rat → .rat`, none
|
||
dangle, none self-link. **1 076 (73 %)** are the `<stem>f` focus pattern; the
|
||
rest are **chains** between effect records (`px_bunk_eff01 → pjex_eff →
|
||
pjex_eff07`), which is also why only 227 targets are declared elements — the
|
||
middle of a chain is, the end is not. So focus is the commonest *use*, not the
|
||
meaning. ⚠️ Coverage: 18 718 raw `opt ` tags exist against 1 467 classified —
|
||
`opt_link` reads the first tag of a declared element's record, so ~92 % of
|
||
occurrences sit deeper in the chains and are untested.
|
||
|
||
The investigation that got here follows, kept in full because most of it is
|
||
refutations that were worth the cost.
|
||
|
||
## Where the wave / spawn scheduler lives — the counter is not a roster
|
||
|
||
**Raised by the user 2026-08-24, and it reframes the whole `REMAINING OB` line of
|
||
work.** A mission does **not** have a fixed enemy count: the number rises as
|
||
waves arrive and falls as they are destroyed. So the thing to find is not a list
|
||
of objectives but an **algorithm with parameters** — what spawns, where, when,
|
||
and on what trigger. That also explains every negative so far: a per-entity flag
|
||
search cannot find a *schedule*.
|
||
|
||
**Places searched so far:**
|
||
|
||
* ✅ **`hidden/MiscBin.pak` — new to the corpus.** 11 `REGN` + 11 `MCOL` objects,
|
||
none name-resolved. `REGN`'s header is now decoded: a per-map **uniform spatial
|
||
grid** (bbox, cell size, 5³ or 10³ cells), self-checked 11/11. Its four data
|
||
sections are unread — the 49 KB object is the cheapest way in.
|
||
✅ **Section 3 decoded**: one 8-byte `(count, offset)` record per cell followed
|
||
by 32-byte payload records, payload at `align16(index end)` — **11 of 11**, one
|
||
record per occupied cell. ✅ **And the container is a serialised object graph**:
|
||
every object carries a **`POF0` pointer-fixup table** at exactly
|
||
`header[0x04] + 16` (11 of 11), which is why the internal offsets are absolute
|
||
file offsets. 🔴 **Two payload readings refuted** by generalising from the one
|
||
object they were fitted to (a "bounding-sphere radius" float, and "leaf arrays
|
||
of `count × 4` bytes"). The payload is float-dominated and otherwise ❔.
|
||
✅ **2026-08-26: the other three sections have strides** — 12 / 96 / 48 bytes,
|
||
with `counts[0..2]` as their record counts (section 1's remainder is exactly 0
|
||
and section 2's exactly 96, in 11/11). **Section 0 is a point list**
|
||
(13 467/13 467 inside the bbox) and **section 2 is a plane list** — unit normal,
|
||
signed distance, and a point, with `n·p + d = 0` to float round-off in
|
||
**133 573/133 573**. Section 1 (96 B) still ❔.
|
||
See [`structures/regn-map-grid.md`](structures/regn-map-grid.md).
|
||
* 🔴 **`hidden/DefTables.pak` is NOT it** (checked 2026-08-24). The three
|
||
unnamed schemas are more **model/render** tables in the same vocabulary as the
|
||
named ones — `7e66225f` (283 objects) carries `RenderFrameModel`,
|
||
`FrameAnimLength1st`, `FrameAnimLoopLength`, `IsPlayerSE`; `634a80ae` (183)
|
||
adds `DissolveDistanceMin/Max`, `RootSEAttach`, `EnumNodes`; `a6d68fa1` (138)
|
||
has **no explicit fields at all** (defaults only).
|
||
* 🔴 **The 40 XML entries are XPR2 build manifests** with the developers' own
|
||
source tree — `machines\rou_e104\objects\*.xbg`, `…\images\*.bmp`,
|
||
`D3DFMT_DXT1_SRGB`. Tempting as a name source for the unresolved hashes, and it
|
||
**does not work**: of 82 names × 9 patterns, the only hits are 16 that were
|
||
already resolved under the known `LOD_Frame_*`/`LOD_Parts_*` convention, and
|
||
the raw source paths hash to **nothing** in any of the 16 514 TOC entries. The
|
||
on-disc keys are not the source paths.
|
||
* 🔴 **`STAGES = Static.slb` is a false lead.** It is a field of the *sound*
|
||
table; `.slb` is an XACT sound bank ([`structures/sound-slb.md`](structures/sound-slb.md)).
|
||
* ✅ **FOUND — it is the `GP_MAIN_GAME_<lang>.pak` unnamed objects.**
|
||
Sweeping the 811 unnamed entries by schema turned up **schema `3c9ae32e`, the
|
||
per-stage definition record**: one per stage (`Stage_S01.xpr` … `Stage_S29.xpr`),
|
||
naming that stage's background, resource package, collision set, message set,
|
||
nameplates, `MapMesh`/`MapPath`, and — the point — its
|
||
**`EnumerateSquadron = UnitGroup_S<NN>.tbl`**.
|
||
`stage\UnitGroup_S02.tbl` (`0x019fd129`) is the Stage 02 roster: **112 records**,
|
||
112 squadron IDs (`TCN001`, `ADN101`, `ADT102`, …), and a field vocabulary of
|
||
`FormationID` / `AIID` / `SideID` / `Count` / **`DisableInterval`** plus the unit
|
||
model (`UN_e010_ADAN_Attacker_S`, matching our XBG7 names), the `MessageSet_*`,
|
||
and the pilot character. `DisableInterval` is the first direct evidence of the
|
||
timing knob. Same pass also settled `MapPath = test.rgn` → the `REGN` objects.
|
||
See [`structures/stage-definition-table.md`](structures/stage-definition-table.md).
|
||
🔴 Refuted along the way: the 16-byte record key is **not** the squadron
|
||
ID's name hash (0 of 112).
|
||
|
||
* ✅ **`UnitGroup` is fully decoded** (2026-08-24). Container and field semantics
|
||
in [`structures/unit-group-table.md`](structures/unit-group-table.md), tool
|
||
`tools/re-capture/unitgroup.py`, Stage 02 dump committed at
|
||
[`data/unitgroup-s02.txt`](data/unitgroup-s02.txt). A squadron record is
|
||
`Count` member tuples — (unit model, message set, `n`, identity/nameplate) —
|
||
followed by five named fields `Count / SideID / AIID / FormationID /
|
||
DisableInterval`. Validated corpus-wide by two independent self-checks, each
|
||
**1160/1160** across all 28 stage tables: the `Count*4+5` length identity, and
|
||
agreement with the file's own `Enumerate_Squadrons` roster.
|
||
🔴 Two refutations recorded: the record key is **not** the squadron-ID name
|
||
hash, and an earlier "109/111" reading of the squadron-id string base was an
|
||
artefact of the uniform 7-byte id stride (it shifted every name by three).
|
||
|
||
* ✅ **The rest of the mission-parameter layer is decoded** (2026-08-24) —
|
||
[`structures/stage-mission-tables.md`](structures/stage-mission-tables.md),
|
||
tool `tools/re-capture/stagetbl.py --follow S02`. The *real* stage record (the
|
||
one dumped earlier was the `_Test` template) splits a stage into **`Phase_1..3`**
|
||
blocks and names `Route_S<NN>.tbl`, `SUBObjectiveSettings_S<NN>.tbl`,
|
||
`AIParams_S<NN>.tbl`, `FormationSet_S<NN>.tbl`, `nameplate_S<NN>.tbl` and more.
|
||
**`Route_S<NN>.tbl` is the arrival schedule**: records named
|
||
`Route_<squadron>_p<phase><kind>` holding time-stamped keyframes
|
||
`(time, quat x4, pos x3)`, tying a `UnitGroup` squadron to a phase and a path —
|
||
self-checked `FrameCount*8+1` on **1449/1449** route records across 28 stages.
|
||
`AIParams` carries directly portable combat tuning (firing/guard/muster ranges
|
||
and 14 manoeuvre weights per profile). 🔴 Refuted: the 8-value frame is *not*
|
||
universal — `Formation_Fleet_01/02` use 136 and 4 values per frame.
|
||
|
||
* ✅🔴 **The arrival timetable is found; the entity-count proxy is refuted**
|
||
(2026-08-24) — [`mission-wave-arrivals.md`](mission-wave-arrivals.md). Route
|
||
records' first-keyframe time is the **arrival schedule**, measured from the
|
||
start of that route's *phase* (phase 2 has all 37 routes at t=0). Stage 02
|
||
phase 1: 25 at t=0, then 3/3/3/2/1 at t=90/120/170/210/240. 🔴 A live count of
|
||
`0x820af030` entity records is FLAT at 116 for 210 s — no arrivals visible.
|
||
✅ **2026-08-26: the flat count is explained, not merely refuted.** Built-in 12
|
||
`activate_unit` returns immediately when the live object is NULL — it registers
|
||
an existing object and cannot spawn one, so arrivals are route positions, not
|
||
new records. Stage 02's roster totals 116, and **116 is the maximum across all
|
||
28 stages and unique to Stage 02**, so the match is not a common-number
|
||
coincidence. 🟡 still n=1 for the per-member identity. ❌ This cannot settle
|
||
timetable-vs-event — a position probe is needed, not a count.
|
||
✅ **Settled 2026-08-26: `t` is SECONDS.** Consecutive route keyframes give an
|
||
implied speed; checked against the live-measured player ceiling (~1530 world
|
||
units/s), **0 of 1 104** pairs exceed it under seconds while 89–94 % do under
|
||
frames at 30/60 fps. See [`mission-wave-arrivals.md`](mission-wave-arrivals.md).
|
||
* ✅🟡 **Motion-independent liveness probe works; `n` is probably craft-per-member**
|
||
(2026-08-24) — [`mission-liveness-probe.md`](mission-liveness-probe.md).
|
||
Enumerating by definition pointer instead of by motion removes the ±10 noise
|
||
(monotone 298→280). The hunting pilot **does** kill (hull crossing at t=57s).
|
||
🔴 "No births in 164 s" does NOT separate the wave models — everything is
|
||
pre-allocated, so an arrival must be a STATE CHANGE, not an allocation.
|
||
🔴 **`n` = craft-per-member is WITHDRAWN** (re-tested 2026-08-24): sites really
|
||
are entities 1:1 (all same-unit gaps ≥0x1000, hull plausible 298/298), so the
|
||
confound was not the explanation — and with it gone, `DeltaSaber_T`, `Player`
|
||
and `Acropolis` all come out at exactly **2×** `sum(n)`. An undershoot is
|
||
explainable by phases; an overshoot is not. `n` back to ❔. Formation slot count
|
||
also rejected (630 turret slots vs 214 live). ✅ Side result: `FormationSet`
|
||
`FrameCount` = slot count. 🔴 **A direct pointer link between them is REFUTED**
|
||
(2026-08-24, [`roster-to-craft-link.md`](roster-to-craft-link.md)): 0/116
|
||
roster records point at a craft base and 0/300 craft point at a roster base.
|
||
They sit in distinct regions (`0xbc372c00`–`0xbc9bc720` vs
|
||
`0xbdb2fd80`–`0xbdcd1d80`). Craft count varies run to run (296/298/300), so it
|
||
must only be compared WITHIN a run. ❔ **Still open: the expansion rule.** Four
|
||
candidates now dead — `Count`, `n`, formation slots, head pointers.
|
||
✅✅ **SOLVED (2026-08-24): the link is a pointer at `roster_base + 0x08`.**
|
||
The delta histogram spiked at `+0x08` with exactly 300 hits for 300 craft.
|
||
Verified on something a coincidence cannot survive — each side's unit type
|
||
resolved by a *different* chain (craft via def pointer `+0x130`, record via
|
||
`+0x04` name chain) — **agreeing 300/300, 0 disagreements**, fan-out `[(1,300)]`.
|
||
Fan-in: only **41 of 116** records have craft (2×24, 4×1, 8×4, 18×12 = 300).
|
||
⇒ **an arrival is craft appearing for a record that had none; a kill is that
|
||
count decreasing — both attributable to a named squadron**, hence to a route.
|
||
✅ **Blocker downgraded (2026-08-24): the baseline DOES reproduce** — two more
|
||
runs give 116 roster records flat from t=0 (not a load race) with craft
|
||
declining 300→288 and 296→280, so **losses are observable**. Raw hits ==
|
||
distinct VAs (116 == 116), so the VA-aliasing explanation is 🔴 refuted too.
|
||
🟡 The single 42/170 run stays unexplained; rule adopted: **discard a run that
|
||
disagrees with 116 rather than interpreting it, and reproduce any finding in
|
||
≥2 runs.**
|
||
* ✅🔴 **Six runs, no arrival — plus an accidental control**
|
||
([`mission-arrival-watch.md`](mission-arrival-watch.md), 2026-08-24). Deployment
|
||
reproduces byte-for-byte (116 records, 300 craft, 41 deployed, strengths
|
||
2×24/4×1/8×4/18×12). ✅ **Losses require the player**: an unpiloted run held at
|
||
exactly 300 craft for 240 s / 22 samples, vs 16–20 losses in each piloted run —
|
||
so NPC crossfire destroys nothing on its own. 🔴 **Zero `0→n` arrivals in either
|
||
condition**, ~15 min cumulative. 🔴 **Prime suspect REFUTED (2026-08-24):
|
||
the clock is running** — [`mission-clock-advances.md`](mission-clock-advances.md).
|
||
With no pilot, 286 heap words advance linearly, a large cluster in lockstep at
|
||
**16.5/s**, which is the emulator's known ~14–19 fps. 🟡 New leading
|
||
explanation: **the runs were far too short in GAME time** — at ~55% of
|
||
wall-clock, the longest 240 s run reached only t≈132, past the t=90/120 route
|
||
entries but nowhere near t=170/210/240. ✅🔴 **Long run done (2026-08-24)**: per-record tracking
|
||
works — watched ONE turret squadron fall **18→14→12→10→8→6→4→2** over seven
|
||
loss events while `deployed` held at 41. 🔴 **Still no arrival at 234 s wall
|
||
(≈129 game-s), past both t=90 and t=120**, so the "too short" explanation no
|
||
longer covers those (it still covers t=170/210/240; the run was cut at 240 s by
|
||
the turn timeout, not the planned 330 s). 🟡 **Sharper hypothesis:** the
|
||
squadron ended at **2, never 0** — no squadron has ever been eliminated in any
|
||
run, so the trigger may be *elimination*, not damage. 🔴 **Elimination test UNRUN (2026-08-24)**: the hunting
|
||
pilot died at t=83 s with the squadron at 14. ✅ But the player's own record hit
|
||
`2→0` — first `n→0` ever seen, so the signal does register elimination (no
|
||
arrival followed; weak, it was the player not an enemy squadron). ✅ **KEY:
|
||
after player death the mission is frozen** — 288 craft, zero losses, zero
|
||
arrivals for 220 s / 18 samples. **The usable window is player survival, not
|
||
probe duration.** 🔴 Harness bug found+fixed: a `sed` had stripped the probe
|
||
args, so every derived probe ran on defaults — the previous "cut by the turn
|
||
timeout" claim was wrong (it hit its own 240 s default).
|
||
✅🔴 **DONE + REFUTED (2026-08-24)**: `SYLPH_KEEPOUT=1400` gives a pilot that
|
||
kills and survives (hull 1500, asset 100 %, 8 losses). An enemy squadron was
|
||
**eliminated** — `e007 Turret 2→0` at t=163 s, `deployed` 41→40, first ever —
|
||
and **no arrival followed** in ~90 s of valid observation (two further losses in
|
||
that window prove the mission was live). ⇒ *"a wave is released when a squadron
|
||
is wiped out"* is **refuted**; event-gating generally is not (thresholds,
|
||
objectives, specific squadrons untested). 🔴 **Guest stalled at ~t=255 s** —
|
||
last 400 pilot lines have 1 distinct speed vs 236 in the first 400 — so trailing
|
||
flat samples in ANY earlier run may be a frozen guest, not a quiet mission.
|
||
Probe now carries a frame-rate tick witness printing `*** GUEST STALLED ***`.
|
||
* ✅🔴 **The stall witness WORKS and is now mandatory**
|
||
([`guest-stalls.md`](guest-stalls.md), 2026-08-24). Validated against the
|
||
pilot's independent telemetry (35 distinct speeds all run, 1 in the last 400
|
||
lines; witness flagged t=27 s). It immediately **discarded** a run that would
|
||
otherwise have read as "no arrivals across 313 s" — worthless, the game was
|
||
frozen for 90 % of it. **Rule: a stalled run is discarded, and every write-up
|
||
must state the witness result.** 🔴 Stalls are frequent and early — last three
|
||
long runs stalled at ~255 s, 83 s and **~27 s**. 🟡 **Leading suspect: the probe
|
||
itself** — it reads the whole 32 MB heap plus ~300 `pread`s every 12 s while
|
||
lavapipe renders, and AGENT.md warns a full scan competes for every core.
|
||
✅ **CONFIRMED (2026-08-24): the probe was causing the stalls.** The no-probe
|
||
control ran **300 s clean** — 211/257/250/241/200 distinct speeds per 60 s
|
||
window, no decline — against probed runs stalling at 27/83/255 s and ending on
|
||
**1** distinct speed. The instrument was degrading what it measured, exactly as
|
||
AGENT.md warned. Control is **n=1**: acted on, not proven.
|
||
⚠️ **Consequence: every "no arrival" result was collected under a struggling or
|
||
frozen guest** — none is as strong as written, and the arrival question needs
|
||
re-running with cheap sampling before silence means anything.
|
||
✅ **Fix VERIFIED (2026-08-24)**: `wave7_probe.py` (enumerate once, then poll
|
||
only the hull word per known craft, ~1.2 KB/sample, rescan every 90 s) ran
|
||
**0 stalled samples** and produced **19 losses vs 8** — starving the emulator
|
||
had been suppressing the activity the probe existed to watch.
|
||
🔴 **Its one apparent arrival (`0→2` at t=259 s) is FLICKER, not a wave** — it
|
||
reverted to 0 fifteen seconds later, and the same log shows a record reading 13
|
||
then 14 with no event printed because only decreases were surfaced. Count
|
||
stands at **0 confirmed arrivals in 11 runs**. Probe now prints every increase
|
||
and requires an arrival to **persist across 2 samples**.
|
||
* ✅🔴 **Run 12 (2026-08-24): persistence rule works; stall witness does not.**
|
||
16 losses / 290 s with a bound pilot, **0 confirmed arrivals** (twelve runs
|
||
now). One increase surfaced (`13→15`) and correctly not counted — exactly the
|
||
flicker that nearly became "first arrival". 🔴 **The single-word tick witness
|
||
gives FALSE POSITIVES**: 13 samples flagged `GUEST STALLED` *while recording
|
||
losses in the same samples*, which a frozen guest cannot do. Cause: it took the
|
||
first word in a 4 MB window with a plausible rate; intermittent counters pass
|
||
that test. `timer_probe` had already solved this (286 candidates, modal rate
|
||
~17/s) and the lesson was not carried over. **Fixed to a majority vote over the
|
||
modal-rate cluster, plus explicit `RUN UNVALIDATED` when no witness exists.**
|
||
* ✅🔴 **Run 13 (2026-08-24): the vote is better, the threshold was wrong, and
|
||
freezes persist.** Graded output (11→9→7→4→1→**0** of 31) is coherent, and the
|
||
drop to 0/31 at t=183 s coincides exactly with the last loss — a real freeze,
|
||
106 s of nothing after. 🔴 But "<half = stalled" marked the WHOLE run stalled
|
||
including samples where craft died: **11/31 advancing is healthy**. Cause: the
|
||
modal cluster was **93/s**, not the ~16.5/s frame rate — bursty subsystem
|
||
counters. **Fixed: prefer the 8–40/s frame-rate band, stall only when ZERO
|
||
advance.**
|
||
* 🔴 **Witness attempt 3 still unreliable (2026-08-24)**: only **5** candidates
|
||
in band, and `0/5 moved` in samples where craft died — a real 24/s counter
|
||
cannot miss 15 s. Cause was mine: the candidate search had been narrowed to
|
||
**4 MB** when the witness was bolted in, while `timer_probe` searched the full
|
||
**32 MB** and found 286 with a clean ~17/s cluster. **Fixed: full-region search
|
||
once at startup + warn when <8 witnesses.**
|
||
* ✅ **WITNESS VALIDATED (2026-08-24, 4th attempt)**: full-region search gives
|
||
6500 candidates → 32 witnesses at 21/s, and the flags have **zero
|
||
contradictions** with the loss data (the one loss is unflagged; every flagged
|
||
sample has no losses). That is the check the three previous versions failed.
|
||
* 🔴 **The freeze is now THE blocker.** Onset across runs: **27, 45, 83, 183,
|
||
255 s** (median ~83 s) — usable window is 1–4 min and unpredictable. This run
|
||
froze at 45 s with 1 loss, useless for arrivals despite a working instrument.
|
||
⚠️ Being honest: the "cheap" probe still reads 32 MB at startup + 32 MB every
|
||
90 s, and the new calibration added **two more 32 MB reads** — the 45 s freeze
|
||
came right after it. 🔴 **The trim BROKE the witness** (17 candidates, `0/17` on
|
||
every sample of a run with 13 losses) — reverted; two 32 MB reads once at
|
||
startup is the price of a working witness. ✅🟡 **Instead, disabling the periodic
|
||
rescan LARGELY removes the freeze** — corrected from "removes" after more runs:
|
||
clean at 210/240/300 s but **frozen at 60 s** on a fourth. Tally — heavy probe
|
||
froze at 27/45/83/183/255 s; cheap+rescan at 183 s; cheap, no rescan: 3 of 4
|
||
clean past 200 s. A large probability improvement, **not a fix**. Practical
|
||
rule: run, check the witness, discard frozen runs (**3 clean of 5** now: clean
|
||
210/240/300, frozen 60/90).
|
||
* ✅ **BLOCKER REMOVED (2026-08-24): the emulator survives BETWEEN Bash calls in
|
||
a turn** (checked: 611 s elapsed, still running after the launching call
|
||
returned), so observation is not capped by one call.
|
||
`tools/re-capture/attach_session.sh` attaches pilot+probe to a live mission and
|
||
attaches CHAIN. Cumulative **435 s verified-live on one mission** (300 s clean +
|
||
135 s of an attach before it froze), craft **300→258** (42 destroyed),
|
||
**0 confirmed arrivals** — covering route entries t=90/120/170/**210**; t=240
|
||
missed by one second. 🟡 Coverage assumes the 55 % wall→game factor; the witness
|
||
has measured 8–24/s across runs, and at the low end the same window covers only
|
||
~117 game-s (t=90 only). **Pinning the tick rate would make this sharp.**
|
||
* ✅🟡 **(2026-08-24) The 42-anomaly is STAGE 01, and it hands us two results.**
|
||
Rescans stuck at 42 (not a load race); live RAM holds `UN_S01_Asteroid_cmesh_*`
|
||
and `UnitGroup_S01.tbl` with S02 absent — **the launch sometimes loads Stage 01**
|
||
(the discard rule caught every such run). ✅ That supplies the refutation test
|
||
recorded as *blocked* for lack of a second stage: S01 static **42 members / 13
|
||
units** vs live **42 records / 13 definitions**, composition 20/20, 6/6, 5/5,
|
||
2/2 — **one record per member now holds on two independent stages.**
|
||
🟡 **Route times are almost certainly FRAMES**: S01 phase 2 has t=1500/1800/2100,
|
||
which as seconds is 25–35 min into one phase (implausible) but at 30 Hz is
|
||
50/60/70 s. ⇒ S02's t=90…240 would be **3–8 seconds**, all before the probe's
|
||
first sample (~25 s in) — explaining every null result, and consistent with
|
||
`deployed=41` already at t=0. **Test: sample at flight+2 s vs flight+30 s** —
|
||
a reordering of the probe, not new decoding.
|
||
* ✅🔴 **(2026-08-24) Early sampling done; still flat.** `early_probe.py` cuts
|
||
setup from ~25 s to **0.5–0.8 s** (no calibration, no labelling, `bytes.find`
|
||
scan). At flight+0.8 s **deployed=41 already**, flat for 252 s (only 41→40, one
|
||
squadron wiped). 🔴 **The `--wait` variant caught the READY ROOM, not flight** —
|
||
the roster is built before take-off, so waiting for it is not catching mission
|
||
start; that window (deployed=39, craft=276, flat 200 s) tests nothing.
|
||
🟡 **But ready room 39/276 vs flight 41/300 suggests deployment is a single step
|
||
at TAKE-OFF**, not a schedule unfolding in-mission (different runs, so
|
||
suggestive only). **Next: one ~400 s run spanning ready room → take-off →
|
||
flight** to catch the 39→41 step in one continuous series.
|
||
* ✅🔴 **(2026-08-24) DEPLOYMENT IS RESOLVED AT MISSION LOAD.** One continuous
|
||
series, 64 samples over ~380 s from roster-appearance through the ready room,
|
||
take-off (`IN FLIGHT` at +47 s) and flight: **`deployed=41, craft=292`, zero
|
||
changes throughout.** 🔴 **My own "deployment happens at take-off" is withdrawn**
|
||
— the 39/276 vs 41/300 gap was cross-run variance; within one run there is no
|
||
step. ⇒ **Stage 02 phase 1 has NO observable in-mission arrival**: 41 of 116
|
||
records are deployed before the ready room ends, and nothing changes in the
|
||
ready room, at take-off, or in up to 435 s of verified-live flight with 42
|
||
kills. Route times most likely encode **fly-in animation timing applied at
|
||
load**, not release times (frames reading: S01's t=2100 → 70 s at 30 Hz).
|
||
❔ **Phases 2 and 3 are entirely untested** — every run has stayed in phase 1,
|
||
and phase advance was never located; a phase transition is now the most likely
|
||
place an arrival could exist.
|
||
* 🟡 **(2026-08-24) ANSWER: enemies come into play PER PHASE, deployed at phase
|
||
start** — [`mission-phase-deployment.md`](mission-phase-deployment.md). Static:
|
||
phase 1 = 37 squadrons / **42 members**, phase 2 = 36, phase 3 = 49. Dynamic:
|
||
**`deployed` = 41** from the first instant, never changing. **41 vs 42 — the
|
||
deployed set IS the phase-1 roster.** That makes everything coherent: all 116
|
||
records exist at load but only the current phase holds craft; no arrival was
|
||
ever seen because phase 1 never completed; route times are within-phase fly-in
|
||
timings, not release times. ⚠️ The **off-by-one is unexplained** and recorded,
|
||
not rounded away. ❔ **Single remaining question: what advances a phase.** It is
|
||
directly observable (deployed would jump), did not happen in 435 s with 42
|
||
kills, and `SUBObjectiveSettings` names 9 objectives but carries no trigger.
|
||
Unsearched: `StageMessageSet_S02.tbl` (never resolved) and the PE in guest RAM.
|
||
* ✅🔴 **(2026-08-24) Message tables searched; trigger NOT there.**
|
||
✅ `StageMessageSet_S02.tbl` **resolved — prefix is `message\`**, not `stage\`
|
||
(closes a long-standing ❔). ✅ `message\UnitMessageSet_S02.tbl` has
|
||
`CrewCount` + **`PresetMessage_Phase1/2/3`** — a THIRD independent table family
|
||
organised around phases, after the stage record and route names.
|
||
🔴 **Refuted:** `ScriptMessage_S02_msg.tbl`'s promising third field is a radio
|
||
*delivery category* (`None` 105, `Emergency` 33, `Killed` 5, `Noise` 5; arg is
|
||
`1` in every record), not mission control flow.
|
||
❔ **No table anywhere carries a phase-advance condition** — everything found is
|
||
a consequence of the phase, never its cause. ⇒ the logic is in **code**, and
|
||
`default.xex` is encrypted on disc, so the decrypted image exists only in guest
|
||
RAM. That is the honest end of the static search.
|
||
* ✅🟡 **(2026-08-24) The mission script is READABLE — and phase 1 never advanced
|
||
because the pilot killed the wrong things.**
|
||
[`mission-objectives-text.md`](mission-objectives-text.md).
|
||
✅ `language\*_local_string.tbl` is **`IXUD`, UTF-16-BE** (prefix `language\`,
|
||
a third convention) — that encoding is why earlier dumps looked like garbage.
|
||
✅ Stage 02's nine sub-objectives in words (*"You destroyed all enemy
|
||
fighters!"*, *"You sunk all enemy warships!"*, *"You destroyed all enemy cruise
|
||
missiles!"*). ✅ **Phases confirmed twice over**: the guide script is three acts
|
||
— marked attackers → warship engine/weapons/shield → missiles — matching the
|
||
per-phase rosters, with phase 3's **`UN_e201_ADAN_ISCMissile` ×9** exactly
|
||
matching `SUBOBJ_013`. 🟡 **Phase-1 objective is stated outright: destroy the
|
||
MARKED ATTACKERS** (`UN_e010_ADAN_Attacker_S`, 4 squadrons) — but every loss
|
||
line in every run reads `UN_e007_ADAN_Turret`, because `SYLPH_HUNT`/`KEEPOUT`
|
||
were built for turrets. **42 kills were the wrong 42.** **Test: a pilot that
|
||
prioritises `e010` over `e007`**, watching `deployed` jump from 41.
|
||
* ✅🚧 **(2026-08-24) `SYLPH_PREFER` works; the blocker is now COMBAT, not RE.**
|
||
Preference knob added (matching units ×0.05, others ×4.0). One clean 320 s run
|
||
(**0 stalls**): 8 turret kills vs **2** `e010` — real effect (was ~1 across ALL
|
||
prior runs) but weak; commitment + proximity keep pulling back to turrets, which
|
||
outnumber attackers 108:16 in craft. `deployed` stayed **41**, no advance.
|
||
🚧 **Quantified blocker:** phase 1 fields **16** `Attacker_S` craft; at 2 per
|
||
320 s clearing them needs **~2560 s ≈ 43 min** of verified-live flight across
|
||
many chained attaches, against a ~2-in-5 freeze rate.
|
||
**Everything needed to OBSERVE the advance is built and validated** — roster
|
||
link, liveness, stall witness, chained attaches, discard rule. What is missing
|
||
is a pilot good enough to finish the objective. **User's choice:** (1) invest in
|
||
the pilot (game-playing, uncertain); (2) accept the static answer (structure,
|
||
rosters, routes, objective texts all decoded and cross-confirmed; only the
|
||
*trigger* is inferred); (3) one 40+ min chained run betting against freezes.
|
||
* ✅✅ **(2026-08-24) THE OBJECTIVES, FROM THE GAME'S OWN TEXT** — pinned by
|
||
`tests/phase_objectives_disc.rs`; `TextIndex::objectives(stage, phase)` already
|
||
existed. **S02 phase 1: "Shoot down all invading enemy fighters while watching
|
||
out for attacks on the ACROPOLIS"** (lose: ACROPOLIS sunk). **Phase 2: "Protect
|
||
the CALIBAN until it has entered the safe zone"** — *positional, not a kill
|
||
count*, so phase triggers are NOT all the same kind. **Phase 3: "Destroy the
|
||
interstellar cruise missiles headed for Schlos Base"** (matches the 9
|
||
`ISCMissile` + `SUBOBJ_013`). ✅ **`[OB]` = objective marker** — hints say *"Red
|
||
mission markers [OB] indicate your targets"*, so **`REMAINING OB` at
|
||
`0xbdb59668` is phase-1 progress** and is the right signal to watch, not
|
||
`deployed`. Closes a loop to the first session. 🔴 **Method lesson: the crate
|
||
already knew this** — several iterations reconstructed it the hard way;
|
||
`grep -rl TextIndex crates/` would have saved the detour. ❔ Minor: `ixud.rs`
|
||
says UTF-16BE, `localization.rs` says LE — one comment is wrong.
|
||
* ✅🔴 **(2026-08-24) REMAINING OB hunt: method works, run unfinished**
|
||
([`remaining-ob-hunt.md`](remaining-ob-hunt.md)). Correlate heap words against
|
||
*named* kill events instead of scanning for a value: **one `e010` event cut ~8 M
|
||
words to 1056**. 🔴 But the probe saved candidates only at the END and the turn
|
||
timeout discarded them — **the exact mistake already recorded in
|
||
`guest-stalls.md` four iterations earlier**. 🔴 And the follow-up attach logged
|
||
535 s of zero losses with **no stall witness**, so it cannot say whether the
|
||
guest was quiet or frozen. Both fixed (save per event + `SYLPH_OB_RESUME=1`
|
||
resume; witness carried over). **Pattern worth acting on: each new probe starts
|
||
from scratch and re-earns the same lessons — a shared probe harness would stick
|
||
where written-down lessons have not.** Needs 2–3 `e010` events, i.e. the same
|
||
combat limit.
|
||
🔴 `SYLPH_HZ=3` refuted as a lever: it gave the LOWEST frame rate (8/s) with the
|
||
most kills, so pilot polling is not the throttle.
|
||
* ✅🔴 **(2026-08-24) OB hunt, second attempt.** ✅ **Incremental saving verified**
|
||
— one `e010` event at t=241 s wrote **1187 candidates** to disk before the turn
|
||
timeout fired; session now clears the file at launch since offsets are only
|
||
valid within one emulator instance. 🔴 **The correlation had no value filter**:
|
||
survivors were float bit patterns (1044450858 ≈ 0.1f) whose integer forms
|
||
differed by the loss count. Fixed — candidates must be small non-negative
|
||
integers (`0 ≤ v < 1000`) in both samples. 🔴 **The attach was FROZEN**, not
|
||
merely unproductive: **25 of 26 samples flagged `GUEST STALLED`**. The witness
|
||
worked; the summary just quoted "0 events" before checking it. **Rule: read the
|
||
witness FIRST, before interpreting what a run showed.** ❔ Still no address;
|
||
needs 2–3 `e010` events in non-stalled samples.
|
||
* ✅🔴 **(2026-08-24) OB hunt: method PROVEN, turret tracking REFUTED.** With the
|
||
value filter, an attach watching frequent turret losses narrowed
|
||
**374 → 2 → 0** across three events — exactly how a correlation search should
|
||
behave, ending in a refutation rather than fizzling: **no plain `u32` in
|
||
`0xBD000000–0xBE000000` decrements with turret kills.** Witness checked first:
|
||
12/25 samples stalled but **all four events fall in the early non-stalled
|
||
stretch**. 🟡 The negative **fits the objective text** — phase 1 asks for
|
||
*fighters*, so a marked-target counter should ignore turrets; it also rules out
|
||
`REMAINING OB` being a general kill tally (it ignored ten turret deaths).
|
||
❔ Still needs 2–3 `e010` events; one clean 220 s run produced **zero**.
|
||
* ✅🔴 **(2026-08-24) Kill-free HUD route works; BE-`u32` assumption REFUTED.**
|
||
`ob_by_hud.py` reads the counter off screen (`ob_read.py`) and intersects heap
|
||
words equal to it — **no kills needed**. Four readings at HUD=4 narrowed
|
||
6156→4312, then HUD read **11** and the intersection collapsed to **0**. ⇒ within
|
||
`0xBD000000–0xBE000000` as **big-endian u32 the counter does not exist**; it may
|
||
be u16/u8/LE or outside the region. Both previous hunts assumed BE-u32, so this
|
||
eliminates the assumption rather than just failing. 🟡 **The value went UP, 4→11**,
|
||
which a pure countdown should not do — candidates: wrong HUD cell, misread digits
|
||
(template strip covers only **0 1 2 4 8**; most samples read `00?`/`???`), or a
|
||
counter that can rise. **Next: widen the scan to u16/u8 and LE, and beyond the
|
||
heap** — one function, no combat cost. Also extend `ob_digits.png`.
|
||
* 🔴 **(2026-08-24) Widened to 7 encodings; run inconclusive.** `ob_by_hud.py`
|
||
now keeps a candidate set per encoding (`u32be/le`, `u16be/le` at both
|
||
alignments, `u8`) as byte offsets. **`u32le` is tightest at 154** vs u32be's
|
||
4452 — a hint, not a result. 🔴 HUD read **4 at every sample**, so nothing
|
||
collapsed; and counts were **byte-identical across five samples in all seven
|
||
encodings** from t=136 s, which is what a freeze looks like. The probe had **no
|
||
witness** — now added. ⚠️ **Fourth probe written without one, third flat run
|
||
that cannot be told from a freeze**; the recurring fix is the shared harness
|
||
noted earlier. Needs two HUD readings at *different* values in non-stalled
|
||
samples.
|
||
* ✅ **(2026-08-25) SHARED PROBE HARNESS built and verified**
|
||
([`probe-harness.md`](probe-harness.md), `tools/re-capture/probeharness.py`).
|
||
Makes structural the four lessons that were re-learned in four separate probes:
|
||
**built-in stall witness** (says `UNVALIDATED` when absent rather than reporting
|
||
zero stalls), **`emit()` flushes every line** so a timeout cannot destroy
|
||
results, **baseline discard with rescan** (`Probe(baseline=116)` refuses to
|
||
start on a different stage), and `summary()` putting the witness first.
|
||
Also provides `craft()`/`strengths()`/`alive()`/`heap()` so a new probe writes
|
||
only its own logic. ✅ Verified: `deploy_probe.py` reimplements the deployment
|
||
watch in ~40 lines vs 150, first live run clean — 116 roster, 32 witnesses at
|
||
10/s, **0 stalled**, 7 losses, TSV written incrementally. ⚠️ Existing probes
|
||
deliberately **not** ported — they work and other docs cite their results.
|
||
* ✅✅ **(2026-08-25) `REMAINING OB` FOUND AND VERIFIED: big-endian u32 at
|
||
`0xbdb59668`.** On the new harness, the HUD changed **4→8** and the intersection
|
||
collapsed in one step to a single `u32be` survivor (with `u16be`/`u8` hits being
|
||
the low half/byte of the same word). **Verified live**: HUD `012` vs
|
||
`mem@0xbdb59668 = 12`. Independently rediscovers the address the earliest
|
||
sessions found by digit-transition hunting.
|
||
🔴 **Corrects the earlier "u32be refuted"** — that came from intersecting on a
|
||
HUD reading of "11" which was almost certainly a misread, since `ob_digits.png`
|
||
only has templates for **0 1 2 4 8** and misreads rather than rejects other
|
||
digits. **A single bad input permanently poisons an intersection**; the reader's
|
||
confidence scores were printed but never gated on.
|
||
🟡 **The counter INCREASES: 4 → 8 → 12** in ~5 min, measured in memory. Marked
|
||
objective targets are being **added during the mission** — the arrival question
|
||
again, now with a one-word signal instead of a 32 MB scan. **Next: watch this
|
||
address across a whole mission.**
|
||
* ✅🔴 **(2026-08-25) OB address is RUN-DEPENDENT; watcher now self-sufficient.**
|
||
Two fresh launches: `mem@0xbdb59668` = **3165285888 (MISMATCH)** then **4
|
||
(MATCH)** against HUD=4. The confirmation gate refused to report from the bad
|
||
one — the old "recurs in ~5 of 7 runs" note is right. `ob_watch.py` now **hunts
|
||
the address on the current run** when confirmation fails. ✅ HUD reader is now
|
||
**confidence-gated** (every digit ≥0.80, margin ≥0.05 — the rule `ob_read`'s own
|
||
docstring states), closing the hole that produced the wrong `u32be` refutation.
|
||
🔴 **OB flat at 4 for 250 s** (witness clean, 0/50 stalled) while the pilot
|
||
fired on **1635 of 1964 ticks** at marked attackers — constant fire, zero
|
||
decrements, so it destroyed none. Fire rate itself rose from 4.6% to 83% with
|
||
no more kills. ❔ **The earlier 4→8→12 rise is NOT reproduced** — both readings
|
||
were HUD-confirmed, so the rise is not a stable property of the first five
|
||
minutes; recorded as unreproduced rather than explained away.
|
||
* ✅🔴 **(2026-08-25) THE MISSION BLACK-SCREENS AT ~4.5 MIN — that is what ends
|
||
every run.** Two-segment OB watch: segment 1 clean (**0/50 stalled**, OB steady
|
||
at 4 for 250 s); segment 2 on the same live emulator got **`NOT IN A MISSION`**.
|
||
Player never died (**hull 1500 throughout**), pilot log stops at **t=267 s** with
|
||
byte-identical lines, screen now **entirely black** (`mean=(0,0,0)`). This is the
|
||
known pre-existing mission-end freeze, caught in the act.
|
||
🔴 **Corrects "chained attaches extend the window"** — the emulator surviving is
|
||
not the *mission* surviving; ~4.5 min is a hard per-mission bound and chaining
|
||
does not cross it. ⇒ **Anything needing >4.5 min of one mission is not doable on
|
||
this build**, including clearing 16 marked fighters at ~2 kills/5 min. The
|
||
freeze is now the highest-value target: fixing it unblocks arrivals, phase
|
||
advance and the OB series at once. ❔ The 4→8→12 OB rise is now unreproduced in
|
||
**3 of 4** runs.
|
||
* ✅🔴 **(2026-08-25) Freeze: one new data point, and the INDEX was the real bug.**
|
||
✅ New evidence closing the resume-spin lead from the other side: the 2026-08-25
|
||
freeze log has **0** `resume was refused` (vs **2447** in an older healthy-ish
|
||
log) — **a mission can freeze with none at all**, so the warning is not even
|
||
necessary, let alone sufficient. 🔴 Also refuted today: the `BaseHeap::Release
|
||
failed` burst is **not** a freeze signature (routine; spans lines 1044–5210 in a
|
||
log that runs 2 700 lines further). 🔴 **Both were already settled in
|
||
`mission-freeze-resume-spin.md`, as was `0xbdb59668` — that is twice in one
|
||
session that existing work was redone.** Cause is mechanical: **`INDEX.md`
|
||
listed 20 of 43 notes** and none of the recent ones, so searching it returned
|
||
nothing. ✅ **Fixed: `INDEX.md` now has a generated table of all 59 notes** with
|
||
title + status and says to search it before starting. Regenerate when notes are
|
||
added.
|
||
* ✅🔴 **(2026-08-25) Freeze frontier: wait confirmed, but `info args` won't work.**
|
||
✅ **Verified** the frozen frame's export by ordinal: the shim's
|
||
`(unsigned short)176` = `0xB0` = **`KeWaitForSingleObject`**
|
||
(`xboxkrnl_table.inc:190`) — previously read off frame names, now pinned.
|
||
🔴 **Corrected:** the doc's "the handle is one `info args` away" is wrong for
|
||
this binary — `readelf` shows **0 debug sections**, symtab only (27 807 entries).
|
||
Not stripped ≠ has debug info; gdb can name frames and nothing inside them.
|
||
**Two routes, costs recorded:** (1) prologue-guided stack read of `XObject::Wait`'s
|
||
spilled `this` — no rebuild, keeps the oracle identical, but per-frame
|
||
archaeology; (2) a `RelWithDebInfo` build via `build-canary` — makes this and all
|
||
future freeze questions easy, at a full compile and a binary differing from the
|
||
one every other measurement used.
|
||
* ✅ **(2026-08-25) Route 1 is viable and cheap — `this` is in `%rbx`.**
|
||
Static analysis, no run spent: `XObject::Wait`'s prologue does `mov %rdi,%rbx`
|
||
at `8fbc9c`, so `this` sits in a **callee-saved** register rather than a stack
|
||
slot. And the binary carries **`.eh_frame` with 127 231 FDEs** (Release builds
|
||
keep it for C++ exceptions), including one covering `8fbc90..8fbde2` that
|
||
tracks `rbx` explicitly. ⇒ from a frozen thread, **`frame 3` + `info registers
|
||
rbx` gives the `XObject*` being waited on**, and `x/gx $rbx` → vtable symbol
|
||
(in symtab) gives its concrete type — **no DWARF and no rebuild needed**.
|
||
**Revises the previous entry**, which called route 1 "per-frame archaeology"
|
||
and route 2 (RelWithDebInfo rebuild) the way to make it easy. **Next: execute on
|
||
a frozen run** — two gdb commands per thread.
|
||
* ✅🟡 **(2026-08-25) Wait-object read WORKS: the object is an `xe::kernel::XEvent`.**
|
||
Executed under gdb: `frame 3` → `rbx` → `x/1gx $rbx` = **`0x5555562db8f0`** =
|
||
PIE base + `0xd878f0` = **`vtable for xe::kernel::XEvent` (+16 for
|
||
offset-to-top/RTTI)** — exact. Same vtable on every sampled waiting thread,
|
||
different `this` each, so many threads waiting on *different XEvent instances*.
|
||
No DWARF, no rebuild, as predicted. 🟡 **Two caveats, both recorded not glossed:**
|
||
(1) this is a **healthy-play** snapshot (`screen_id` = `flight`), not the freeze
|
||
— boot under gdb costs ~300 s and the whole thing must fit one call (a `timeout`
|
||
kills the process group and took the emulator with it once), leaving too little
|
||
to reach the ~267 s freeze; (2) the `this` addresses (`0x7ffd…`, `0x7ffc…`) look
|
||
like **host stack**, not heap, so either xenia places them unusually or `rbx`
|
||
isn't `Wait`'s `this` after unwind. **Check: dump a few words at `$rbx` —
|
||
XEvent-like (vtable, KernelState*, handle) vs saved registers.**
|
||
* 🟡 **(2026-08-25) Refined: TWO wait types, and the read self-checks.**
|
||
Re-extracting per thread (not by grep): of **18** `XObject::Wait` frames, **8**
|
||
have `[rbx]` = `vtable for xe::kernel::XEvent`+16, **2** = `vtable for
|
||
xe::kernel::XSemaphore`+16, and **8** hold a non-vtable mmap pointer. ⇒ waits
|
||
are on **XEvent and XSemaphore**; "it's an XEvent" was the majority, not the
|
||
whole picture. ✅ **The 8 misses are the method validating itself** — a
|
||
polymorphic object's first word is always a vtable, so a value only counts if
|
||
`[rbx]` resolves to a `vtable for …` symbol; `rbx` simply wasn't restorable for
|
||
those frames. That also dissolves the "looks like stack" worry: `0x7ffc…` is the
|
||
shared mmap region (stacks *and* big allocations), and the **vtable check**, not
|
||
the address range, is the discriminator. 🔴 The follow-up run adding
|
||
`/proc/maps` classification + `x/8gx` **never booted** (`EMULATOR GONE at 0s`,
|
||
stale emulator/lock from the prior gdb session), so that check and the **frozen**
|
||
capture are still unrun.
|
||
* ✅ **(2026-08-25) WITHDRAWN: the 8 "unrestorable rbx" reads were `WaitMultiple`.**
|
||
`rbx` was restored fine on all 18. The backtrace grep matched `XObject::Wait`
|
||
as a **substring**, so `WaitMultiple` frames were pooled in and read with the
|
||
wrong rule — `Wait` keeps `this` in rbx (`8fbc90: mov %rdi,%rbx`) but
|
||
`WaitMultiple` keeps the **`XObject**` array** (`8fbfc0: mov %rsi,%rbx`), so
|
||
`[rbx]` there is `objects[0]`, an object pointer needing a **second** deref.
|
||
Confirmed live: re-reading with the matching rule per frame resolves
|
||
**30/30 objects across 23 wait frames, nothing unresolved** —
|
||
**XEvent 20 / XSemaphore 9 / XTimer 1** (`data/waitobj-s02.txt`). `XTimer` was
|
||
invisible before because its only thread is a `WaitMultiple` one. Every
|
||
`WaitMultiple` thread waits on a **pair**; T78/79/80 and T64/65 are worker
|
||
groups sharing a handle. 🟡 `%ebp` is **not** a usable count — `WaitMultiple`
|
||
reuses it at `8fc158` — so the array is bounded by reading until an entry stops
|
||
resolving. What survives: the object types and the self-validating read.
|
||
* 🔴 **(2026-08-25) The frozen capture is STILL unrun — the freeze did not happen.**
|
||
Two captures in one run (200 s and 367 s of mission), and `screen_id` reads
|
||
**`flight`** at both, plus at ~470 s with a drifting frame mean. So the labelled
|
||
`healthy -> frozen` diff is **two healthy captures**, and its `20 -> 18` is three
|
||
threads exiting, not a freeze. 🟡 **This also questions the "~270 s black-screen"
|
||
bound** these runs are planned around — confound not ruled out: gdb slows the
|
||
emulator (boot to title took 207 s), so 470 s wall-clock may be far short of
|
||
270 s *guest* time. **Next: reach the freeze by its actual trigger rather than a
|
||
clock, or measure guest time under gdb so the wait is set in the right units.**
|
||
Chaining Bash calls **within one turn** works and is no longer the limit — the
|
||
emulator survives between calls; this run spent ~900 s across three.
|
||
* ✅✅ **(2026-08-25) FROZEN CAPTURE TAKEN — and the per-thread diff is the result.**
|
||
`freeze_waitobj.sh` split into `boot`/`watch`; `watch` waits for the **event**
|
||
(`frozen.py` + `in_flight`) instead of sleeping a guessed interval, and caught
|
||
it first try. Hard stop, not a hitch: a frame minutes later is
|
||
`max_pixel_delta=0` against the capture. Same run, same mission
|
||
(`data/waitobj-s02.txt`): **20 → 24 wait frames**, XEvent **19 → 23**,
|
||
XSemaphore **8 → 7**, XTimer 1 → 1. 🔑 **17 of 24 threads sit on the EXACT
|
||
object they were already on** — so this is *not* a whole-emulator stall. What
|
||
moves: T105/T67/T68 park on `Wait(XEvent)` and T69 on `Wait(XSemaphore)` having
|
||
been running; **T74 and T75 move off a semaphore onto an event** (the only
|
||
threads that change what *kind* of object they wait for → chase these first);
|
||
T50 drops `WaitMultiple(XEvent,XEvent)` → `Wait(XEvent)`.
|
||
* 🔴 **(2026-08-25) WITHDRAWN: the T74/T75 signature.** It does **not** reproduce
|
||
— in run 2 they are on `Wait(XEvent)` while **healthy** and stay there. My
|
||
error: the healthy state is not fixed either, so a **one-sample-per-state**
|
||
diff cannot separate a freeze transition from ordinary variation between two
|
||
instants. ✅ **What survives two independent captures:** T68 and T69 go from
|
||
not-waiting to waiting in *both* runs (T69 on a semaphore, T68 on an event,
|
||
both times), and the bulk of threads are untouched (17/24 run 1, **21/24** run
|
||
2) ⇒ **not a whole-emulator stall** now has two captures behind it. **Next:
|
||
repeat the capture several times inside ONE healthy run** to learn which thread
|
||
states are stable before reading any frozen diff as meaningful.
|
||
* ✅ **(2026-08-25) CORRECTION: `screen_id == flight` is NOT a freeze test.**
|
||
The previous entry used it to rule out a freeze. `frozen.py` exists precisely
|
||
because a frozen mission still classifies as `flight` (its docstring: 724 s of
|
||
identical state, 212 % CPU, classifier happy). Re-testing the saved frames says
|
||
that run was genuinely healthy — right, but by luck. **And the "~270 s
|
||
black-screen" bound those runs were planned around is not a thing**: the freeze
|
||
does not black the screen and keeps no clock (onsets 27/45/83/183/255 s).
|
||
* 🟡 **(2026-08-25) One data point that OUR INSTRUMENT provokes the freeze.**
|
||
Run flew **~670 s clean** with only the pilot; a heavy-CPU inducer
|
||
(`heavy_read.py cpu`) started at 08:59:54 and it froze at **09:00:48 — 54 s
|
||
later**, inside the 27–255 s band. Consistent with the existing tally but
|
||
**n=1, not causal** (confounder: elapsed mission time). ⬆️ **UPGRADED to n=2
|
||
(2026-08-25)**: run 2 put the inducer **on from flight start** and froze
|
||
**~96 s** in (10:07:20 → 10:08:56). Both freezes sit inside the 27–255 s band;
|
||
the only long clean stretch is the only window with no inducer. Contrast is
|
||
sharp, confounder still untouched. ✅ **Refuted en route:
|
||
the I/O was never the cost** — a full uncapped walk of every allocated extent is
|
||
**371 MB in 0.1 s**, all page cache; the expense is Python-level CPU, ~4.2 s a
|
||
pass. **Settle it with:** alternating inducer-on/off windows within a run,
|
||
several runs, compared per unit of *mission* time — cheap now `watch` is
|
||
event-driven.
|
||
* ~~🚧 BLOCKER: t=210/240 unreachable in one turn~~ — **superseded, see above**;
|
||
it rested on an untested assumption that a turn is one shell call. 595 s shell cap − ~220 s boot (a ~190 s title movie that cannot be
|
||
tapped through) − ~25 s startup = **~350 s observation ≈ 193 game-seconds**.
|
||
t=90 needs 164 s wall ✅, t=120 → 218 s ✅, t=170 → 309 s ✅ (only on a
|
||
non-frozen run), **t=210 → 382 s ❌, t=240 → 436 s ❌**. No number of runs fixes
|
||
this. **Unblocking needs a decision I should not make alone:** (1) a safe way to
|
||
skip the title movie — it is >half the budget, and would roughly double the
|
||
window to ~297 game-s, covering everything; or (2) a longer shell timeout. The rescan
|
||
existed only to catch newly-allocated craft, which the roster work showed never
|
||
happens.
|
||
* ✅ **Run 16: the first TRUSTWORTHY negative.** Validated witness, no stall on
|
||
any sample, guest demonstrably live (8 losses) ⇒ **0 confirmed arrivals over
|
||
210 s of verified-live flight** ≈115 game-seconds. Establishes that nothing
|
||
arrives in the first ~115 game-s of Stage 02 phase 1 while the player kills 8
|
||
craft. Does NOT settle the question — t=170/210/240 route entries are still out
|
||
of reach.
|
||
* ✅ **Run 17 reproduces it (n=2)**: no stalls through **t=240 s** (~132 game-s),
|
||
9 losses, 0 confirmed arrivals — so nothing arrives past the t=90 and t=120
|
||
route entries either. 4 flicker `up`s correctly rejected (~1/min, which is why
|
||
the persistence rule matters). 🔴 **Startup eats ~100 s of a ~350 s budget**:
|
||
`enumerate_craft` iterated every 4-byte word of 32 MB in Python (8M steps) to
|
||
find 14 fixed needles. **Replaced with `bytes.find()` per definition VA — not
|
||
yet run**; if it recovers that time the window reaches ~340 s ≈ 187 game-s,
|
||
finally covering the **t=170** entry. Also: the 42-record anomaly recurred and the discard rule
|
||
correctly refused the run; a rescan-until-baseline retry is now in place to
|
||
test whether it climbs. All three witness
|
||
failures share one pattern: a shortcut in *selecting* the witness, each caught
|
||
only by the flagged samples contradicting losses in the same output.
|
||
* ✅ **Bind failure fixed + abort verified**: `entities2 self` finds the player by
|
||
MOTION, so a stationary craft at mission start is invisible; the session now
|
||
holds throttle 3 s before each attempt. One run correctly **aborted** after 3
|
||
failures rather than flying unattended; the next bound immediately (11 losses,
|
||
0 confirmed arrivals — 14 runs). 🔴 **The cheap probe froze too (t≈183 s)**, so the
|
||
earlier "0 stalled samples" validation is superseded; with n=1 per arm (control
|
||
clean 300 s, heavy 27–255 s, cheap 183 s) it is unresolved whether cheap
|
||
sampling helps or the freeze is stochastic. ⚠️ **Usable window ≈3 min per run
|
||
regardless** — design experiments to fit or to survive a freeze. ⚠️ The "0 stalled
|
||
samples" that validated the cheap probe came from the unreliable witness and
|
||
should be re-confirmed. Also fixed: bind now retries 3× and aborts rather than
|
||
flying an unattended craft (one run was wasted that way). ❔ Multi-squadron threshold test still not run (zero losses that run).
|
||
⚠️ The ~210 s title movie at boot is the binding constraint on observable game
|
||
time per turn.
|
||
Earlier framing:
|
||
[`mission-per-record-strength.md`](mission-per-record-strength.md) — one run
|
||
gives 116 records/300 craft, the next 42/170, same disc, save and script. Save
|
||
drift is REFUTED (savedata untouched since 2026-08-23) and the guest was not
|
||
frozen (pilot telemetry shows live engagement). Also: **zero arrivals and zero
|
||
losses in 190 s of hunting** — weak evidence against clock arrivals at
|
||
t=90/120/170 s, and no test of event-gating at all because nothing was killed
|
||
(`fc=0`, hull never moved). **Next: (1)** sample the roster-record count
|
||
repeatedly WITHIN one run to tell a load race from a run-specific constant;
|
||
**(2)** get a confirmed kill — the pilot reaches 259 m and still misses, so the
|
||
gap is accuracy, not engagement.
|
||
* 🟡 **COMPETING MODEL (user, 2026-08-24): waves are event-gated, not
|
||
clock-driven** — released by kills/objectives rather than at a fixed time, with
|
||
the route's start time being a post-trigger delay. Fits the evidence better:
|
||
both probe runs used the *survival* pilot (kills nothing) and saw no arrivals,
|
||
which a clock model does not explain. `DisableInterval` also gains a referent.
|
||
🟡 Supporting: no `DisableInterval=Yes` squadron has a delayed route (55/55 at
|
||
t=0, vs ~3.7 expected if independent, p≈0.02) — suggestive, NOT conclusive,
|
||
deliberately not promoted. **Test:** same stage twice, kill nothing vs kill
|
||
aggressively; cheapest precondition is whether `REMAINING OB` (`0xbdb59668`)
|
||
moves in one and not the other.
|
||
* 🔴 **Diffing inside the 116 records did not find the arrival flag**
|
||
(2026-08-24, same doc). 10 of 116 records are dynamic, 106 never change a byte
|
||
in 170 s — 🔴 **the "10 of 116 dynamic" figure is WITHDRAWN** — a hunting
|
||
run measured 41–56 records changing per tick; I had changed two variables at
|
||
once (record bound and pilot) so the discrepancy is unattributed. The roster
|
||
identity is unaffected: it now rests on the 10/10 unit-composition match.
|
||
Earlier note, kept for history: the "turrets don't move" hedge was withdrawn after the user
|
||
pointed out that early-mission "Turret" is a craft type, which the data
|
||
confirms (`UN_e007_ADAN_Turret` flies under `AI_ADAN_CraftSquadron_*`, never
|
||
`AI_Structure`). Lesson: check a unit's `AIID`, not its English name. No field transitions in groups of 3 at the
|
||
predicted times. **Blocked on two of my own defects**: the record→unit-ID
|
||
label resolved to `?` for all 116 (reuse `unit_discover.py`, do not re-derive),
|
||
and `RECLEN=0x200` was assumed, not measured — the busiest fields sit at the
|
||
very end of the window, which is what spilling into the next object looks like.
|
||
Fix both before diffing these records again.
|
||
* 🔴 **Probed 2026-08-24 and refuted the cheap hypothesis** — the phase state is
|
||
NOT adjacent to the loaded table strings; see
|
||
[`mission-phase-runtime.md`](mission-phase-runtime.md). The run did confirm
|
||
dynamically that every table the static decode predicts is resident in guest
|
||
RAM and findable by name, which validates the whole static layer against a
|
||
running mission. Next handles: watch `Route_ADN101_p1F` actually fire against
|
||
entity positions, or work back from the `SUBOBJ_*_Mes_L1` HUD strings; the
|
||
phase state is more likely near the known mutable `REMAINING OB` counter at
|
||
`0xbdb59668` than near the tables.
|
||
|
||
**Superseded first step:** find what *advances a phase* — the stage declares
|
||
`Phase_1..3` and routes are phase-tagged, but nothing static found so far says
|
||
what ends a phase. This is the point where the oracle should be measured rather
|
||
than reasoned about: fly Stage 02 and watch for the phase transition. Also open:
|
||
the route-name kind letters `F/S/A/M/B`, what activates a sub-objective, and
|
||
`StageMessageSet_S<NN>.tbl` which does not resolve in `GP_MAIN_GAME_E.pak`.
|
||
|
||
**Superseded first step:** find where the arrival *interval values* live.
|
||
`DisableInterval` is only a per-squadron flag (`Yes` for 31 of 1160); the
|
||
durations, spawn triggers and arrival positions are not in `UnitGroup`. The
|
||
stage record names two untouched candidates: `Formation_*.tbl` (formation
|
||
geometry, possibly its timing) and `EnumSquadron_Test.tbl`. Also still open: the
|
||
4-byte record key. ~~the member field `n`~~ — **SETTLED 2026-08-25, see below.**
|
||
~~the missing S17–S23 stage records~~ — **SETTLED 2026-08-25, and half the
|
||
premise was wrong.**
|
||
* ✅ **`n` is the number of units the member tuple instantiates**, filling slots
|
||
of the squadron's formation. `FormationSet_S<NN>.tbl` records are **slot
|
||
lists** — `1 + 8·FrameCount` fields, exactly (4→33, 14→113, 30→241, 32→257).
|
||
Resolving every squadron's `FormationID` and comparing: **`sum(n) ≤ FrameCount`
|
||
holds 1159/1160 across all 28 stages, 0 unresolved, 539 filling the formation
|
||
exactly.** The lone violation is a debug leftover (S20, `AI_Test` /
|
||
`MessageSet_test`, `Formation_1_only` with `n=2`) and is recorded, not swept
|
||
up. The old "`n` is not the `_NN` suffix of `FormationID`" was right but drew
|
||
the wrong conclusion — **the suffix IS `FrameCount`**, so `n=9` against `_30`
|
||
just means 9 units in 9 of 30 slots.
|
||
* ✅ **`FormationID` resolves by an in-table roster, not by hashing** — 0/16
|
||
resolve via `name_hash`; `FormationSet_S02.tbl` has 17 records for 16
|
||
formations and the extra one carries no `FrameCount`, its fields being
|
||
`(tag, name, "")` with the **tags equal to the record keys**. Same convention
|
||
as `Enumerate_Squadrons`. **Second sighting — treat it as the rule for the next
|
||
table.**
|
||
* ✅✅ **(2026-08-25) 387-vs-~300 SOLVED — the probe double-counts.** The
|
||
squadron→phase map was on the disc all along, **in the `Route_S<NN>.tbl` record
|
||
names** (`Route_<squadron>_p<N><kind>`, 120/120 for S02); 108/111 S02 squadrons
|
||
map, the 3 misses are route-table typos and all phase 2. Phase 1 Σ`n` = **151**,
|
||
and its member multiset `{1×25, 2×1, 4×4, 9×12}` matches the measured
|
||
craft-per-record fan-in `{2×24, 4×1, 8×4, 18×12}` **bucket for bucket at
|
||
exactly 2×** ⇒ `300 = 2 × (151 − 1)`, the −1 being the known 41-vs-42
|
||
off-by-one (probably the player). **The 2× is the instrument, not the game:** a
|
||
2× reading needs 16 Delta Sabers for 8 named pilots, and breaks
|
||
`Σn ≤ FrameCount` in **20 of 37** phase-1 squadrons (16 on `Formation_1_only`,
|
||
one slot) where 1× has **0**. All three measured totals being even is a further
|
||
tell. ⚠️ Static argument about a dynamic measurement — it says the number cannot
|
||
mean what we thought, **not** what is being double-counted; that needs a run
|
||
against the craft-scan counting rule. See `mission-phase-membership.md`.
|
||
* 🔴 **Refuted with it:** `UnitGroup` has **no** phase/spawn/delay field
|
||
(`1019 = ΣCount·4 + 5·111`, every slot accounted, positional tags are
|
||
sequential indices); `DisableInterval` is `No` for all 111 S02 squadrons (its
|
||
31 corpus `Yes` records are only S04 and S14, all `GNN***`);
|
||
**`stage\EnumSquadron_Test.tbl` does not exist** — that candidate is dead.
|
||
* 🔴 **(2026-08-25) NO freeze signature survives the control.** Six wait-object
|
||
captures across ONE healthy run: **12 thread states stable, 13 vary**
|
||
(`data/waitobj-healthy-stability.txt`). **Every** thread previously reported as
|
||
a signature is in the VARIES set — including **T68/T69**, which I had kept as
|
||
"what reproduces across both freezes": they park and unpark during ordinary
|
||
play, landing on the same objects they hold when frozen. Both frozen diffs are
|
||
consistent with healthy variation; one-sample-per-state could never separate
|
||
them. ✅ **Survives:** the 12 stable threads held the same object in all six
|
||
captures and none moved in either frozen capture, so "not a whole-emulator
|
||
stall" stands — now resting on the stable set being undisturbed rather than a
|
||
count. ⚠️ **gdb thread numbers are not comparable across runs** (this run had
|
||
T132–T142, earlier ones T104–T106) — key on object address or guest tid.
|
||
**Next: compare DISTRIBUTIONS** — N healthy vs N frozen captures, and call a
|
||
thread a signature only if its frozen distribution leaves its healthy range.
|
||
* 🐛 **(2026-08-25) Boot-nav bug found and worked around, not yet fixed.** Three
|
||
consecutive `BOOT FAILED (NO readyroom)` were **not** flakes and **not** input
|
||
loss — the d-pad and A both work (verified by moving the save cursor and
|
||
opening the dialog by hand). `wait_screen.sh readyroom 300 --tap A`
|
||
**blind-taps A**, which answers **NO** on the "Load game?" dialog, bounces back
|
||
to the save list, then reopens it — a 300 s oscillation. Driving `step up` →
|
||
`A` by hand reached the ready room in **18 s**. `launch_mission.sh` should not
|
||
pass `--tap A` while a YES/NO dialog can be on screen.
|
||
* ✅✅ **(2026-08-25) WHAT ADVANCES A PHASE — SOLVED: a compiled script VM.**
|
||
See `mission-phase-advance.md`. `[ScriptMission+40]` is the 1/2/3 ordinal,
|
||
init at `0x822606B0`, and **incremented at exactly one site** `0x822609F8`–
|
||
`0x82260A00` (checked: only one `stw` to `40(rN)` in the whole state machine
|
||
`sub_82260710`). Its guard is `[ScriptPhase+196] != 0`; that flag has only two
|
||
writers — vtable slots 0/1 at `0x82264058`/`0x822640F8` — reached ONLY from
|
||
built-ins **6** and **62** of the phase-script VM's 147-entry table
|
||
(`sub_82272220`, jump table `0x8227226C`). Built-in **39** sets
|
||
`[phase+300]=2` = "last phase", ending the mission instead of advancing.
|
||
🔴 **All four candidate triggers refuted as direct causes** — no kill counter,
|
||
timer, trigger volume or message event is on the path; those conditions live
|
||
*inside* the per-mission script, which is why the static sweep found nothing
|
||
and why three phases of one stage can differ. ✅ `sub_8230D1F8` reads
|
||
`"Phase_%1d"` for map/background only — confirms the executable never consults
|
||
`Phase_N` for a trigger, and **nothing parses `Route_*_p<N>*`** (the 3 such
|
||
literals at `0x820AEA38` are debug defaults) — the route-name phase map is our
|
||
convention, not the game's.
|
||
* 🎯 **PROBE TARGET:** `CScriptInterpreter::ChangePhase` (`sub_822FF330`, opcode
|
||
995) writes a runtime phase mirror at **`[*(0x828F35F8) + 236]`** — readable
|
||
from `/dev/shm` with **no gdb**.
|
||
* ✅✅ **(2026-08-25) SCRIPT BYTECODE FOUND — `Stage\StageNN.ssb`** in
|
||
`dat/GP_MAIN_GAME_<L>.pak`. See `structures/mission-script-ssb.md`. The earlier
|
||
grep failed only because every pak entry is `Z1`+zlib. The loader resolves
|
||
**table keys, not a filename**: GamePart name → `GP_SCRIPT` → `script.tbl`
|
||
(`name_hash 0x75FE4656`), whose `SCRIPTS` record is a 40-field manifest
|
||
`MISSION1..MISSION29 → StageNN.ssb`. **28 scripts; S17 is the missing one** —
|
||
agreeing with the table sweep *and* with the loader guard
|
||
`if (n==16 || n>32) return` (so mission numbers there are **0-based**): three
|
||
independent routes to the same conclusion. Header decoded from
|
||
`ScriptMission::Load` itself. Stage02 = 226,596 B, md5 `aff69b5a…`, identical
|
||
in all 6 language paks. Symtab1 326 syms (197 messages, **119 route names**,
|
||
10 subobjectives), symtab2 122 (**111 unit ids** = exactly the 111
|
||
`UnitGroup_S02` squadrons). 🔑 **The route names are SCRIPT SYMBOLS** — which is
|
||
why nothing in the executable parses `Route_*_p<N>*`; the `_pN` convention is
|
||
the script's, resolved at compile time.
|
||
* 🔴 **Refuted:** the 7 `.embsec_` sections are **code** (32,368 instructions,
|
||
108 functions with real prologues), not script; `MiscBin.pak` and
|
||
`DefTables.pak` have **zero** hits.
|
||
* ❔ **NEXT, and it is the one that pays:** decode the **25 ISL opcodes**
|
||
(dispatcher `0x822635D4`, low byte of a BE u32, jump table `0x822635FC`) and
|
||
the **147 built-ins** (`0x8227226C`) against `Stage02.ssb`, hunting calls to
|
||
built-ins **6**/**62** — the two that set `[ScriptPhase+196]` and end a phase.
|
||
That gives the **actual per-phase clear condition for every stage**. The
|
||
mission-level stream at `+0x24` is partly read (3 groups, one per phase, each
|
||
ending in a pair of plausible ISL entry offsets) but `1883`'s operand is not
|
||
uniformly a pointer — two land on IEEE floats.
|
||
* 🐛 **(2026-08-25) The nav fix is NOT fully reliable.** `dialog_up.py` works on
|
||
the saved frame (mean 34.3 vs threshold 45), and one boot went
|
||
readyroom-in-9 s — but a later boot logged "load dialog not up yet, retrying"
|
||
and still ended `NO readyroom within 300s`. So the retry loop can desynchronise
|
||
(a shot taken before the dialog renders reads as absent, and the *next* A then
|
||
answers NO). Needs a settle-and-recheck rather than one shot per attempt.
|
||
* ~~🔴 **Not settled: the script bytecode is not on the disc under any obvious
|
||
name.**~~ No `GP_SCRIPT.pak`; grepping the extraction for `MISSION_START_PRT`
|
||
returns nothing. Loader `sub_8225EE20` matches section names
|
||
`MISSION1..MISSION33` + 5 `*_PRT`; `sub_8225EC78` gates `if (n==16 || n>32)`.
|
||
Candidates: the **7 `.embsec_` sections** (VAs 0x84D0000–0x86AC000, ~129 KB
|
||
total, executable) or a hashed record in `hidden/MiscBin.pak`. **Finding it
|
||
gives the actual per-phase clear condition for every stage.**
|
||
* ❌ **(2026-08-26) RETRACTED — "the resupply banks are missing audio."** The
|
||
error was mine and it stood for three iterations across two write-ups that each
|
||
called the result proven: I treated a subtitle cue as a timestamp that must
|
||
fall *inside* the voice clip. **A cue is when the line STARTS** — the voice
|
||
plays from the cue to at most the movie's end. Under that reading all five
|
||
banks fit at plain **48 kHz** (3.31 s in a 5.30 s window, 2.26 s in 4.60 s, …),
|
||
and 2–3.5 s is the right length for the lines. Nothing is missing. The
|
||
17 091–20 563 Hz window from the previous entry is void with it.
|
||
✅ **Each shared bank is ONE generic line** — the 3–5 movies sharing a bank have
|
||
**identical subtitle text**, 5/5 banks (`examples/shared_bank_takes.rs`).
|
||
🎯 **That also explains the historical in-game rejection** of
|
||
`hokyu_DS_s13A → VOICE_D_452`: the line is the generic "Resupply complete. You
|
||
are cleared for take-off!", the same for s02A/s07A/s08A/s13A. Someone expecting
|
||
a stage-13-specific line would call the generic one wrong while the binding is
|
||
right.
|
||
✅ **(same day) The two parts are SEQUENTIAL SEGMENTS, not duplicates** — so the
|
||
totals do not double-count and the 48 kHz fit stands. Measured by RMS:
|
||
`450`/`451`/`452` have a **silent or near-silent** leading region (RMS 0–301
|
||
against ~9 000 for speech) with the line in the RIFF; `453`/`454` have the line
|
||
in the **leading** region with a short loud tail in the RIFF.
|
||
🎯 **That closes the original mystery.** The decoder skips everything before the
|
||
first `RIFF`. For the first three that discards only silence, so they looked
|
||
fine (2.8 / 1.6 / 2.2 s); for the last two it discards **the line itself**,
|
||
leaving 0.14 s and 0.43 s. One rule, two outcomes, depending on which segment
|
||
holds the speech. ✅ **(same day) LANDED.** `to_xma_riffs` emits the leading segment
|
||
wrapped **mono** when it is a whole number of packets and carries a non-zero
|
||
byte. Both reasons the first attempt was withdrawn are answered: it used the
|
||
stereo format (mono yields up to 113× more), and while the byte-level reach is
|
||
still 1524 entries the **audible** reach is not — across the 84 movie-bound
|
||
banks it adds >1 s to exactly **7**, the `hokyu_*_H` tankers on
|
||
`VOICE_D_453`/`454`, and ≤0.25 s to 66 of the rest. ⚠️ The safety oracle is
|
||
weak and says so: **8 of the 84 already exceed their movie duration before the
|
||
change**, by hundredths of a second, so it establishes scoping rather than
|
||
correctness. Pinned by `tests/slb_leading_segment_disc.rs`, including that the
|
||
all-zero `VOICE_D_451` region stays skipped. ❔ Not verified by ear — that
|
||
needs a human.
|
||
* ❌ **(2026-08-25) The `.slb` "multi-subwave" guess is REFUTED, and the voice
|
||
decoder is discarding up to 87 % of a bank.** The record table gives a
|
||
**direct** binding `hokyu_DS_s13A -> VOICE_D_452` where the corpus records the
|
||
movie as unbound and a test asserts `None`, citing an in-game verdict that the
|
||
same value was "the wrong recording". Measured: the RIFF-magic count equals the
|
||
sub-wave count in all five hokyu banks, so nothing between or after sub-waves
|
||
is missed — the recorded "likely multi-subwave / not cleanly sliced" is wrong.
|
||
The audio is lost because a **large region precedes the first RIFF** and
|
||
`slb::to_xma_riffs` finds audio by searching for that magic: **87 % of
|
||
`VOICE_D_453` and 85 % of `VOICE_D_454`** sit in front of it, 21–27 % zero over
|
||
256 distinct byte values — content, not padding. `VOICE_D_451` is the control,
|
||
its leading region being 100 % zero / 1 distinct value. 🟡 So the in-game
|
||
verdict tested a decode that had thrown away most of the bank and is **not**
|
||
evidence against the binding — though it does not confirm it either.
|
||
✅ **(same day) The region's SIZE is now exact**: the first `RIFF` sits at
|
||
`1392 + n*2048` in all five banks (n = 8, 1, 7, 22, 29) — 1392 being the
|
||
crate's own `HEADERLESS_DATA_OFFSET` and 2048 the XMA1 packet size. No free
|
||
parameter.
|
||
❌ **But my fix for it is WITHDRAWN.** Emitting that region as a sub-wave took
|
||
`VOICE_D_453` from 5.4 % to 89.9 % byte coverage — and the stream decodes to
|
||
**1792 PCM bytes**, silence, while the RIFF sub-waves decode to 150–270 KB.
|
||
Byte coverage was the wrong success metric. The rule also matches **1524 of
|
||
8021** RIFF-bearing `sound.pak` entries, including `RT*` banks that work today,
|
||
so it risked a wide regression to not-fix five banks.
|
||
✅ **(same day) Audio really IS missing — proven by the subtitle cue times**,
|
||
not by the "sounds too short" impression the docs recorded. A subtitle that
|
||
appears at *t* seconds cannot sit inside a clip shorter than *t*, and three of
|
||
five banks fail that: `D_450` cue 4.00 s vs 1.41 s decoded, `D_451` 3.70 vs
|
||
1.81, `D_453` **4.70 vs 0.07**. The other two have their only cue at 0.0 s and
|
||
give no signal. Artifact `examples/voice_len_vs_subs.rs`, FFmpeg-measured.
|
||
✅ **(same day) The leading region IS XMA1 — MONO, not stereo.** At
|
||
`channels = 2` every bank decodes to *exactly* 1792 bytes regardless of size
|
||
(one frame, then it stops); at `channels = 1` the same data yields up to
|
||
**113× more** — `VOICE_D_453` goes 1 792 → **203 648**. The bank's own RIFF
|
||
sub-wave is decoded through the same pipe as a control (13 568 bytes), so the
|
||
harness is sound, and the all-zero `VOICE_D_451` region is the control the
|
||
other way. The earlier 0-byte probe was my own error: I read
|
||
`synth_xma1_fmt`'s second argument as a stream count when it is a **channel
|
||
mask**. ❌ Solving for the sample rate from the subtitle cue **does not
|
||
converge** — 21 665 Hz for `D_453` (temptingly near 22 050, and I nearly wrote
|
||
it down) but **5 844 Hz** for `D_450`. The decodes are partial: samples per
|
||
input byte ranges 2.10–4.96 where a clean decode would be near-constant.
|
||
✅ **(same day) Why FFmpeg stops is captured** — I had been discarding its
|
||
stderr. It reports an unimplemented "Reserved bit" and a negative bit-skip, and
|
||
the failing frame is always the **last** one (44 of 45.7, 28 of 29.4, 198 of
|
||
198.9, 287 of 287.5). ❌ **That corrects my own previous claim**: "the decodes
|
||
are visibly partial" was wrong — the 2.10–4.96 samples-per-byte spread is
|
||
ordinary XMA1 variable bitrate (4.21–9.92 frames per packet), and only the
|
||
final frame is lost. ❌ The sample rate still does not converge, including
|
||
after counting leading region + RIFF together: 39 742 / 20 563 / 23 108 Hz for
|
||
the three banks with a usable cue. Two of them agreed at a tidy ~2.1× ratio and
|
||
the third refuted it.
|
||
✅ **(2026-08-26) The cue unit IS seconds** — checked against movie duration as
|
||
an independent oracle: **66 movies with subtitle tracks, 0 cues land after
|
||
their movie ends** (`examples/cue_unit_check.rs`). Centiseconds would have
|
||
overflowed essentially all 66.
|
||
❌ **And "the sample rate does not converge" was my own error.** The implied
|
||
rates are not competing point estimates — each is a **one-sided bound**
|
||
(`samples/cue` is an UPPER bound, `samples/movie` a LOWER one). Intersected,
|
||
they give a **non-empty window of 17 091 – 20 563 Hz**: a single rate *is*
|
||
consistent with all three banks.
|
||
❔ But that window holds **no standard XMA rate** (22 050 / 24 000 / 32 000 /
|
||
44 100 / 48 000 all fall outside). The lower bound assumes the whole bank plays
|
||
within one movie, and each bank is bound to **3–5 movie slots** — so if a bank
|
||
holds several takes the lower bound is void, leaving `rate <= 20 563`, which
|
||
22 050 nearly meets. ▶️ Next: establish whether a shared bank is one line or
|
||
several takes. See
|
||
[`voice-bank-leading-region.md`](voice-bank-leading-region.md).
|
||
* ❌ **(2026-08-25) My own boot-nav diagnosis, MEASURED AND WITHDRAWN.** I said
|
||
the run died because `skip_intro.sh` gates the title test at `rmse <= 1500`
|
||
and the run logged 1503/1549, just above the cut. Measured over a clean
|
||
no-press boot ([`boot-nav-title-gate.md`](boot-nav-title-gate.md),
|
||
`captures/boot-signal-trace.tsv`): the gate **opened eight times** in 29
|
||
samples, and at `t=145 s` the RMSE was **1205** — inside the threshold — with
|
||
`is_title.py` answering **0 glyph pixels**. The glyph count was 0 in *all* 29
|
||
samples over 484 s. A static frame is not the title; the intro movie has long
|
||
quiet stretches, three of them reading RMSE exactly 0. So the gate is not what
|
||
stopped it, and raising the constant would have admitted two more movie frames.
|
||
🟡 Narrowed: **the interactive title never appeared**, rather than appearing and
|
||
being missed. ❔ **But the run does not prove that** — the tracer intended 1 s
|
||
sampling and achieved **16.9 s** (two screenshots + ImageMagick `compare` + a
|
||
fresh Python per iteration), so a title window under ~17 s falls between
|
||
samples. ▶️ First step: make the tracer sample at the rate it claims — one
|
||
long-lived Python process computing both signals — then re-run. Second: check
|
||
whether the fast 2026-08-25 run had a warm shader cache that this one, started
|
||
after `rm -f /dev/shm/xenia_*`, did not.
|
||
* 🚧 **(2026-08-25) STILL UNRUN — the live test of the `reset_phase_threads`
|
||
rename.** Built-in 100 clears the trigger container and frees every thread but
|
||
the caller, so at a phase terminator **both** `[phase+272+20]` (triggers
|
||
queued) and `[phase+216+8]` (coroutines alive) should collapse; if they climb
|
||
straight through a phase boundary instead, the reading is wrong.
|
||
`tools/re-capture/phase_watch.py` prints both — **written, still never
|
||
exercised against a live guest.** Blocked behind the boot-nav item above.
|
||
* 🟡 **(2026-08-25) The legacy IDXD string-pool reader is wrong far more often
|
||
than assumed** — now measurable for the first time, since the record table gives
|
||
a ground truth. Verified by hand: `FCSRange` (the module doc's own canonical
|
||
"field with no value") is really `500000.0`; `ShieldRatio` is `1.0` where a
|
||
**committed test asserted `None` and called it defaulted**; and
|
||
`get_raw("Model")` on the hangar table returns the *first* record's model for
|
||
every record — silent corruption, not an absent value. Single-source disc-wide
|
||
rates: `get_raw` 52 % wrong, typed getters 38 % miss — but **100 % correct on the
|
||
548 single-record objects**, so all the damage is the flat API having no way to
|
||
say *which* record it means. ▶️ **Open work:** re-read every per-record number in
|
||
this corpus through `IdxdObject::record`; highest value first — hangar models,
|
||
weapon `Power`/`Acceleration`/`MinimumVelocity`, turret and subsystem stats.
|
||
See [`idxd-legacy-reader-audit.md`](idxd-legacy-reader-audit.md).
|
||
* ❌ **(2026-08-25) The 504 unnamed IDXD field keys were NOT recovered.** All sit
|
||
in `GP_READY_ROOM.pak`'s sound-bank table (6 identical objects × 2 records × 42
|
||
keys). A dictionary of 572 464 strings — every pool string disc-wide, PE ASCII
|
||
and UTF-16 runs, every identifier in this repo — plus 73 191 case/affix variants
|
||
gave **0/42**. The key deltas across `stage01…stage16` do prove the preimage
|
||
**ends with the two decimal digits**, and a meet-in-the-middle found nothing
|
||
word-like at ≤8 lowercase chars. 32 bits is not invertible without the right
|
||
wordlist; parked.
|
||
* ✅ **(2026-08-25) The IDXD/IXUD container is fully decoded** — the "binary
|
||
node/index region" in front of the string pool is a **uniform 16-byte record
|
||
array** `{name_hash, name_off, field_begin, field_end}` sorted by hash, then a
|
||
field count, **12-byte fields** `{key, name_off, value_off}` sorted by key, then
|
||
a pool size and the pool. Verified over the *whole* disc with zero failures:
|
||
IDXD **7 750/7 750** objects, **190 782/190 782** records, **1 271 462/1 271 462**
|
||
named fields; IXUD **1 104/1 104** objects, **628 165/628 165** fields (offsets in
|
||
chars). **Field names are stored on disc**, so no preimage search is needed —
|
||
only **504** fields disc-wide are hash-keyed with no name.
|
||
🔴 **Two corrections:** the header word at `0x08` is **not a schema hash**, it is
|
||
record 0's `name_hash` (7 750/7 750) — the format has no type field at all, so an
|
||
object's kind is known only from its loader; and the field's middle word is not
|
||
an `aux` flags word. See [`structures/idxd-container.md`](structures/idxd-container.md).
|
||
⚠️ My first disc sweep globbed `dat/**` and **missed `hidden/DefTables.pak`**
|
||
(1 425 objects); the test now walks the whole disc root.
|
||
▶️ **Follow-up now open:** the legacy value-before-key string-pool reader is an
|
||
*approximation* of the real table, and every number in this corpus that came out
|
||
of `get_f32`/`get_raw` is re-checkable against ground truth but **not yet
|
||
re-checked**. First step: diff the two readers across the disc and count
|
||
disagreements. Also open: recover the 504 unnamed hash keys.
|
||
* ✅ **(2026-08-25) Both guest hash routines located** — `sub_82447DF0` (IDXD)
|
||
and `sub_82447E70` (IXUD), transcribed instruction-for-instruction into Python
|
||
and Rust; `cargo test -p sylpheed-formats --lib hash` 10/10. **IXUD SOLVED:**
|
||
it chains **two** exact moduli (loop mod `2^32-153` in 64-bit, then fold mod
|
||
`2^24-33`), which is why no single-modulus search could ever find it —
|
||
86/86 keys and 108,261/108,261 tags verified. 🔴 **Two of my claims corrected:**
|
||
`tag_hash` must **sign-extend** (`extsb`) — the unsigned version matched all
|
||
1.27M disc names because every one is ASCII, but differs on 18,096/20,000
|
||
random high-byte inputs; and **`name_hash`'s reduction is EXACT, not lossy**
|
||
(0 wrong at every quotient boundary over the full 32-bit domain).
|
||
* ✅ **(2026-08-25) BOOT-NAV BUG FIXED and verified by artifact.** `dialog_up.py`
|
||
detects the dim the game draws behind a modal (mean 34 vs 59–62), and
|
||
`wait_screen.sh --tap-if-dialog` only presses while one is up;
|
||
`launch_mission.sh` now verifies the "Load game?" dialog is actually open
|
||
before selecting YES. Next boot: **readyroom at 9 s, IN FLIGHT at 37 s**
|
||
(against three prior 300 s failures).
|
||
* ✅ **Tooling:** `.pe` is **NOT stale** — it is a flat VA image
|
||
(offset = VA − 0x82000000), verified 7/7 against the DB. And
|
||
`instructions.raw` in `sylpheed.db` is an **INTEGER**, not a hex string —
|
||
decoding it as hex silently compares nothing and nearly recorded this
|
||
correction backwards.
|
||
* ~~❔ **What ADVANCES a phase is still unknown**~~ and is not in the data: swept the
|
||
stage record and every table it names plus the `message\` family for
|
||
`interval|time|phase|delay|wave|spawn|arrival|trigger|start|appear|event|condition`
|
||
— only `FrameCount` and `PresetMessage_Phase1/2/3` hit. It is in the executable.
|
||
* 🟡 ~~**Does NOT close the 387-vs-~300 gap**~~ in `roster-to-craft-link.md`. Σ`n`
|
||
over Stage 02 is 387 vs 296–300 live craft, but that was measured mid-mission
|
||
after kills and squadrons deploy across phases, so they are not comparable as
|
||
they stand. The earlier rejection of `n` assumed everything deploys at once —
|
||
**still untested**. The record-key derivation also stays ❔ (the tag is not
|
||
`name_hash`; a second hash function is unidentified).
|
||
* ✅ **S18–S23 stage records were never missing.** `stagetbl.py Stage_S18` returns
|
||
a full six-record definition and always would have; it was never run. They omit
|
||
the literal `Stage_S<NN>` (no per-stage `.xpr`; `AIParams`/weapons/strings/
|
||
subobjectives/nameplate/collision all come from a shared **`_Tutorial`** set),
|
||
so an enumeration keyed on that literal skipped all six. Counting distinct
|
||
`*_S<NN>` names across all **1119** decompressed entries: `UnitGroup` and
|
||
`Route` cover **28** stages (S01–S16, **S18–S23**, S24–S29); `Stage` literal and
|
||
`AIParams` **22**; `SUBObjectiveSettings` **16** (story only). Tutorial records
|
||
carry the same `Phase_1/2/3` shape, so the tutorial is not a special mission
|
||
type at the data layer. Dump in `data/stage-tutorial-records.txt`.
|
||
* ✅ **S17 alone is genuinely absent** — it appears in *none* of the five families.
|
||
Not a stage that lost its data; it does not exist. Matches the family split in
|
||
`challenge-mission-gate.md` (story 1–16, tutorial 18–23, challenge 24–29).
|
||
* 🔴 **Refuted:** `GP_TUTORIAL.pak` does **not** hold the tutorial stage config —
|
||
2 entries, both RATC, zero IDXD, exactly like `GP_CHALLENGE.pak`.
|
||
|
||
## The dynamic-RE state is not in git, and it was gone
|
||
|
||
**Found 2026-08-23.** Everything the oracle runs on — the baseline emulator
|
||
binary, the Xbox profile, the Stage 02 save, the shader/code caches — lives
|
||
outside both repos and had been wiped. `sylph-doctor` says "all good" without
|
||
any of it; the first symptom is `NO PROFILE on disc` one second into a boot.
|
||
|
||
**Rebuilt and verified by driving it** (LOAD GAME lists the slot → READY ROOM →
|
||
Stage 02 flight): [`dynamic-re-state-restore.md`](dynamic-re-state-restore.md)
|
||
carries the recipe — incremental rebuild of `auto/upstream-baseline` in the
|
||
shared checkout (202 files, no submodule churn), profile bootstrapped with the
|
||
*instrumented* binary's `--create_profile_if_none`, and the committed
|
||
`savedata-stage02-5pct.bin` installed **without** an Xbox content header, which
|
||
`ContentManager::ListContent` does not need.
|
||
|
||
**Open, and cheap:**
|
||
|
||
* ✅ **`launch_mission.sh` finishes unattended again** (2026-08-23, later):
|
||
boot → title → LOAD GAME → slot 01 → READY ROOM → TAKE OFF → `IN FLIGHT at
|
||
34s`, pilot bound and engaging. Two defects, not one: the fixed `sleep 28` for
|
||
LOAD → READY ROOM (now `wait_screen.sh readyroom`), and the READY ROOM being
|
||
**drawn before it is usable** — `Preparing to Sortie`, TAKE OFF greyed, which
|
||
whole-image statistics cannot see (1.7 units of blue) so `take_off_armed.py`
|
||
tests the label. A third defect fell out of the same run: `wait_flight.sh` was
|
||
testing pixel (450,640) "inside the SHIELD bar" of a **1280×720** window, while
|
||
`screenshot` crops to the **1279×675** game surface — it lands between the
|
||
SHIELD and ARMOR bars. That is the long-standing "reported NEVER REACHED FLIGHT
|
||
while plainly in flight" note, now explained and fixed.
|
||
* ❔ **Nothing guarantees this state survives the next container.** If it is
|
||
meant to, the profile + save + `bin/` copies want a home inside a repo or a
|
||
named volume; that is a call for the user, not for an agent.
|
||
|
||
## ✅ SOLVED (2026-08-23) — the mission objective counter is at `0xbdb59668`, and the hunt is automated
|
||
|
||
🔴 **`0xbdb59668` is refuted as a durable address** (2026-08-23): 0 in two
|
||
independent Stage 02 runs while the HUD read `004`/`008`/`012`, on an allocated
|
||
(not sparse) page. The **method** stands; the number does not, and every session
|
||
must re-scan. Two candidates from the re-scan were themselves refuted by the
|
||
corpus's own "verify across a transition you did not select on" rule. Detail and
|
||
the corrected method note (the scan takes **0.9 s** — the trap is the counter
|
||
climbing `004 → 012` in four minutes, not scan duration) in
|
||
[`structures/mission-objective-counter.md`](structures/mission-objective-counter.md).
|
||
|
||
✅ **Settled the same day, once the HUD stopped costing a human round trip.**
|
||
`ob_read.py` reads the three digits by normalised template correlation and
|
||
`ob_hunt.py` runs the whole method unattended; run 4 then gave **one** survivor
|
||
from 35 897, selected on `004 → 008` and verified on the unselected `008 → 012`,
|
||
plus three live paired RAM/HUD readings. The blocker was never the pilot's
|
||
survival — the evidence lives in the first four minutes of the stage, and the
|
||
earlier runs simply could not look often enough to catch the `008` step.
|
||
|
||
🔴 Yesterday's refutation **stands, refined**: the address is not universal (runs
|
||
2 and 3 read a hard 0 there while the HUD counted), but it is not meaningless
|
||
either — it recurs exactly, and run 3's amber candidate sits one 64 KB page below
|
||
it at the same page offset `0x9668`. Rule: try `0xbdb59668`, check it against the
|
||
HUD, re-scan (~5 min, `ob_session.sh`) when it reads 0.
|
||
The follow-on that the autopilot actually needs is unchanged and untouched:
|
||
❔ what the counter counts, and whether an `OB`-badged entity carries a flag in
|
||
its entity object.
|
||
|
||
## What `REMAINING OB` counts — and an in-mission freeze in the way
|
||
|
||
**2026-08-23.** The address is settled (above); *what it counts* is not, and it
|
||
is what the autopilot needs in order to CHOOSE a target. One run in:
|
||
|
||
* 🔴 **Not a live class head-count.** Counter 4 against 8 attackers / 7 friendly
|
||
Delta Sabers / 7 turrets / 1 player — no class matches, no pair sums to it.
|
||
* 🔴 **The per-entity flag is REFUTED** (2026-08-23, final): sample A at counter
|
||
12 over 120 entities gave **2** candidates; the counter went **12 → 11** and
|
||
**neither survived**. Within ±0x400 of the position triple there is no 4-byte
|
||
word whose shared-value population tracks the counter. ❔ **Not** ruled out: a
|
||
single **bit** ORed into a word that also varies (the test needs an exact
|
||
shared value), anything outside that window, and anything on entities
|
||
`entities2` cannot see — it types by position *changing*, so stationary
|
||
objectives are invisible. 🔴 **The bit-level differential is REFUTED too**
|
||
(2026-08-24): two independently selected transitions in one run — scan at 4
|
||
filtered on 4→8 (29 of 710 survive), scan at 8 filtered on 8→12 (2 of 197) —
|
||
and the **intersection is empty**. No per-entity bit in either polarity tracks
|
||
the counter; 16 of pass 1's survivors were the same word `+0x250` with
|
||
different bits, i.e. a *shared value*, not a flag. Earlier note: 🔎 **Built**
|
||
(`ob_bitflag.py`, 2026-08-24) and run three times with **no verification
|
||
yet**: one window was spent on a mission that had already ended in GAME OVER,
|
||
one hit the same dead mission, and the third had the counter at a different
|
||
address and then froze after a single filter. Sample A alone gives ~187
|
||
set-polarity + 33 clear-polarity candidates at counter 4, so the second
|
||
transition is the whole test.
|
||
* ⚠️ **Attrition is now the dominant cost of every in-mission item.** Roughly
|
||
half the runs that reach flight end early — a freeze, or a GAME OVER when the
|
||
ACROPOLIS or the craft is lost — and a scan needs the run to survive **two**
|
||
counter transitions. `frozen.in_flight()` at least makes a dead run say so
|
||
immediately instead of waiting out its window.
|
||
* ✅ **What the counter's neighbourhood IS — the HUD glyph quads**
|
||
(2026-08-24). The four pointers that move with it lead to objects whose vtable
|
||
is `0x820B2A64`: **32 slots**, methods `0x823c43b0…0x823c45a0`, **three**
|
||
construction sites in `sylpheed.db`. Each instance is a **textured quad** — a
|
||
pixel size (34×42 for a digit) and four vertices of `(colour, u, v)` — and the
|
||
UV rectangle × **1280×768** reproduces that pixel size to a rounding step, so
|
||
the font atlas size is measured, not guessed.
|
||
See [`structures/hud-glyph-quad.md`](structures/hud-glyph-quad.md).
|
||
* 🔴 **The counter is not "hostiles left" either.** It held at `012` for fifteen
|
||
minutes of live flight while the ADAN population fell 132 → 93.
|
||
* ✅ **But it decrements when the player kills**: `12 → 11` with 411 `fire=1`
|
||
samples and `YOU KILLED WARPLANES 0003` on the HUD — the first decrement seen,
|
||
and the first run where the player's guns were part of the experiment.
|
||
* ✅ **`pilot.py` never fires — ROOT CAUSE FIXED** (2026-08-23):
|
||
`flight_probe.Pad` was writing to the **vgamepad FIFO**, dead since the uinput
|
||
pad was replaced by `--hid=file`, so every axis, trigger and button from every
|
||
flight tool went into a file nothing reads while `/tmp/xenia_pad.txt` stayed
|
||
empty. The craft was never being flown. Verified: full stick went from `0.00°`
|
||
of heading change to `12.72°`, and the attitude matrix from `d 0.0000` to
|
||
`d 0.4438` — which also **refutes** the "stale attitude matrix" suspicion.
|
||
✅ **And the second half: the PITCH stick sign was inverted.** Measured on a
|
||
45° error, both sides, two pulse widths, with the opposite sign as control:
|
||
the pilot's sign grew the error every time, the opposite shrank it every time.
|
||
Fixed, and **the pilot fires** — `fire=1` in **43 of 1 732** samples against 0
|
||
of 13 521, aim down to 2.3°, range median 43 km → 6.3 km, and the HUD's own
|
||
ammunition counters moving. 🟡 Still open: `YOU KILLED` is `0000` after 250 s
|
||
of firing and `REMAINING OB` is still `012` — whether it *destroys* anything is
|
||
the next measurement, and the objective-counter item is waiting on it.
|
||
⚠️ `findrot_global.py`, `findself.py`, `findspeed.py`, `selfstate.py` still
|
||
write to the dead FIFO and `ctrl_probe.py`/`target_probe.py` still use
|
||
`pad.f`; all flagged in place, none repaired.
|
||
See [`pilot-never-fires.md`](pilot-never-fires.md).
|
||
* ✅ **`EMULATOR GONE` is SOLVED — it was this project's own `Stop` hook**
|
||
(2026-08-24), which `kill -9`s `xenia_canary` at the end of every agent turn.
|
||
Every "mysterious" death was a turn boundary. **Operational rule:** an emulator
|
||
experiment must complete **inside one turn** — nothing can be left running for
|
||
a later tick, and a watcher armed for 1 500 s only watches the rest of *this*
|
||
turn. Memory pressure was raised and refuted along the way; that measurement
|
||
stands, it just was not pointing at anything.
|
||
* 🔴 **An in-mission freeze — the item in front of everything else.** Reproduced
|
||
with the Kernel channel on. 🔴 The resume-spin lead is **refuted by its own
|
||
control**: a still-flying run has *more* refused resumes (2 738) than a frozen
|
||
one, because the game runs a self-suspending worker and the host refusal is one
|
||
per cycle by design. ✅ What is established instead: the guest is **spinning,
|
||
not deadlocked** — over 10 s while frozen the main thread is in state `R`
|
||
gaining 409 ticks, guest threads ~680 in total, and **not one kernel call** is
|
||
made. So it is guest code waiting on guest memory. ✅ **Seen from inside** (2026-08-24, gdb): all **79** threads are in a
|
||
**wait** — guest threads in `KeWaitForSingleObject`/`NtWaitForSingleObjectEx`,
|
||
the GPU processor idle, the main thread in `poll()` — while the process still
|
||
burns **1 253 ticks / 10 s**, 403 of them in the **TimerQueue** thread and
|
||
~280 each in two guest threads the backtrace shows *blocked*. So they are
|
||
**cycling through a timed wait**, and the CPU burn is in the kernel layer's
|
||
wait path, not in guest code. No Canary build was needed: `XENIA_BIN` pointing
|
||
at a gdb wrapper keeps the lockfile and satisfies `ptrace_scope=1`.
|
||
🔴 **Reading the wait target from the log is blocked by cost**:
|
||
`KeWaitForSingleObject` is `kHighFrequency` and silent without
|
||
`--log_high_frequency_kernel_calls=true`, and *with* it the emulator is 17
|
||
minutes into a boot with a **black screen** and 175 MB of log. ✅ **Built** (canary `auto/re-wait-timeout-probe` `820696c11`,
|
||
`--log_stuck_waits=true`, binary at `/sylph-home/re/bin/waitprobe/`): counts
|
||
consecutive timeouts on the same object per thread and logs at 100 then every
|
||
500. **Healthy-run control measured** — 27 lines over 25 minutes, all one
|
||
thread polling one Event at guest VA `BE56BB5C` with a ~30 ms timeout, so the
|
||
freeze signal is a **new (thread, object) pair**, not the presence of output.
|
||
🔴 **A freeze WAS caught (2026-08-24) and the probe says nothing.** It froze
|
||
9 s into the watcher's window, in flight, and reported the healthy baseline
|
||
only — one pair, same object VA, no new (thread, object) pair — while the CPU
|
||
signature was unchanged (1 255 ticks/10 s, 401 in the TimerQueue thread). So
|
||
the freeze is **not** a thread looping on timeouts against one object.
|
||
**Two blind spots survive:** waits cycling over *different* objects (the streak
|
||
resets, so they are invisible), or waits that **succeed** rather than time out
|
||
(nothing for a timeout counter to count — which fits the self-suspending worker
|
||
seen cycling successfully in the kernel log).
|
||
✅ **v2 built and its baseline is itself a result** (canary `597740046`): it
|
||
counts every call per thread per second with the distinct-object count and the
|
||
return value. On a healthy 22-minute run the **main thread cleared 500 calls/s
|
||
in 224 windows, peaking at 1 235/s over up to 13 distinct objects**, and the
|
||
result was `X_STATUS_SUCCESS` in **all 314** windows — not one timeout. So the
|
||
game's normal mode is hundreds of *successful* waits a second across many
|
||
objects, which is exactly what v1 could not see.
|
||
🟡 **Consequence:** 500/s is not self-selecting, so the freeze signal must be a
|
||
*different shape* — far above 1 235/s, a new thread, or a non-SUCCESS result.
|
||
**Next:** a frozen sample to compare against; runs 5 and 6 did not freeze
|
||
(GAME OVER at ~22 min, and still healthy at 10 min). **Three** runs in a
|
||
row have now failed to freeze (the third ended in GAME OVER), and the probe's
|
||
healthy control is measured three times — 27, 24 and 36 lines, always the same
|
||
single pair. `frozen.py` detects the state in one call; `ob_hunt.py` /
|
||
`ob_flag.py` abort on it. **Roughly two runs in three.**
|
||
See [`mission-freeze-resume-spin.md`](mission-freeze-resume-spin.md).
|
||
|
||
**First step, revised:** make `pilot.py` shoot, then re-run `ob_flag.py`. The
|
||
freeze is no longer the blocker it looked like — a 25-minute run stayed
|
||
animating — and the actual obstacle is that nothing the pilot does moves the
|
||
counter, so there is never a second sample. If the counter still will not move
|
||
when the player is killing things, the next question is what *does* move it, and
|
||
the objective card's own wording ("shoot down all invading enemy fighters") is
|
||
the place to start. Second step, if that comes back empty: `entities2.typed` only
|
||
sees entities whose position *changes*, so a stationary objective is invisible to
|
||
it, and the enumeration itself would need widening before a null result means
|
||
anything.
|
||
|
||
See [`mission-freeze-and-ob-flag.md`](mission-freeze-and-ob-flag.md).
|
||
|
||
## The declaration table is not a paint order on every screen
|
||
|
||
**Found 2026-08-17**, building the Explorer's UI Screens browser on
|
||
[`ui_layout`](structures/ui-rat-layout.md). **Status: 🔎 open — the pause menu is
|
||
right, the title screen is not.**
|
||
|
||
`ui-rat-layout.md` says the bundle's element declaration table lists elements
|
||
"in back-to-front order", verified 11/11 on the tutorial pause bundle. That
|
||
holds — the tutorial and in-mission PAUSE builds both composite correctly, and
|
||
`pgpeff02a` → parent 3 / `pgp_ttrl_btn10` at (546,288) / the 70 px button pitch
|
||
all reproduce exactly.
|
||
|
||
**`GP_TITLE.pak` build 7 does not.** Painting in declaration order puts
|
||
`ptbase2.t32` (the full-screen background art, element **13**) *on top of* the
|
||
`ptlogo1`/`ptlogo2` wordmarks (elements **0–5**), which the real title screen
|
||
obviously does not do. The pause bundles never caught this because their
|
||
elements barely overlap.
|
||
|
||
**What has been ruled out:** there is no depth/layer key in the 60-byte
|
||
declaration entry. Dumping every word across the title build's 30 entries, the
|
||
unknown fields are constant — `+28` is 0 everywhere, `+44` is `0xffffffff`
|
||
everywhere, `+56` is 0 everywhere — and `+36`, which the doc lists as
|
||
`0xffffffff`, is not a depth either: it is `0`/`1` **only** on the `kind = 0x4`
|
||
repeated-instance entries (`ptlogo1`/`ptlogo2` copies), i.e. an instance index.
|
||
So the order is not recoverable by sorting the table on any field it carries.
|
||
|
||
**What that leaves.** The background group is contiguous — elements 12, 13, 14
|
||
are `pteff00.prm`, `ptbase2.t32`, `pteff04.t32`, and 12 carries `kind = 0x10`,
|
||
a flag no pause element has (theirs are `0x0` / `0x1` / `0x3002`). `pteff02.prm`
|
||
at 17 has it too. So `0x10` marking a `PRMD` primitive, and primitives opening a
|
||
layer that draws beneath what precedes them, is the cheapest hypothesis — but it
|
||
is a **hypothesis**, and "draw the `.prm` group first" would fit this one screen
|
||
without being evidence of anything.
|
||
|
||
**First step:** composite `GP_MISSION_SELECT` / `GP_READY_ROOM` / `GP_OPTIONS`,
|
||
which have both a background and overlapping foreground elements, and see
|
||
whether their background sits at a `0x10`-adjacent index too. Two more screens
|
||
agreeing turns the hypothesis into a rule; one disagreeing kills it. The
|
||
Explorer's `screen render`/`screen info` commands make that a minute's work per
|
||
screen, and the per-element visibility toggles isolate a suspect element.
|
||
|
||
**Meanwhile** the viewer paints in declaration order and does not pretend
|
||
otherwise — a screen whose background lands on top is showing you this bug, not
|
||
a decode failure.
|
||
|
||
### 2026-08-18 — measured against the running game; three orderings refuted, and half the symptom was a different bug
|
||
|
||
**The premise is confirmed by the oracle**, which this entry had not had: a
|
||
framebuffer capture of Canary on the title screen
|
||
([`captures/title-screen-oracle.png`](captures/title-screen-oracle.png)) shows
|
||
the `PROJECT SYLPHEED` wordmarks (elements 0–5) drawn **over** `ptbase2.t32`
|
||
(element 13), which is a full-screen background. Declaration order is therefore
|
||
not the paint order on this screen, and no reading of the element table changes
|
||
that.
|
||
|
||
**But part of what the render showed was not the paint order at all.** In the
|
||
capture `ptbase2` covers the whole screen; the compositor drew it as a
|
||
960×540-visible slab starting at (320,180), because a keyframe's `scale` was
|
||
being grown from the keyframe's corner instead of about the declared **pivot**.
|
||
Fixed, and pinned against the capture by cross-correlation (peak at (0,0)) — see
|
||
[`structures/ui-rat-layout.md`](structures/ui-rat-layout.md). That was a real
|
||
defect worth separating out: it moves **865** of the disc's 5 130 resting
|
||
placements, on every screen, independently of any ordering question.
|
||
|
||
**Three candidate orderings are now dead**, all cheaply:
|
||
|
||
- **The placement region is not a second ordering.** Its keyframe groups carry an
|
||
explicit element index, so they *could* be stored in a different order — they
|
||
are not, on **every** build on the disc (`placement_region_order_is_never_a_second_ordering`,
|
||
>500 builds, identity every time).
|
||
- **The RATC child order is not it either.** For the title build it is the
|
||
declaration order with the `.prm` elements absent — strictly less information,
|
||
and it has no place to put `ptbase2` other than where the table already puts it.
|
||
- **Reverse declaration order is refuted by the same capture**: it would draw
|
||
`ptbase2` (13) over `ptcopyright` (28), and the copyright line is visible.
|
||
|
||
**The `0x10`-adjacency first step was run, and it does not survive.** The
|
||
background *is* adjacent to a `kind = 0x10` `.prm` element on both screens that
|
||
have one — but on **opposite sides**. `GP_TITLE` build 7 is
|
||
`12 pteff00.prm (0x10)`, `13 ptbase2.t32`, `14 pteff04.t32`;
|
||
`GP_MISSION_SELECT` build 0 is `0 px_mission_base.tbm`, `1 px_mission_eff00.prm
|
||
(0x10)`. So "the `.prm` opens a layer that draws beneath what precedes it" cannot
|
||
place both, and no rule keyed on the `.prm`'s position orders the background.
|
||
`GP_READY_ROOM` and `GP_OPTIONS` turned out not to be the third and fourth
|
||
witnesses this entry hoped for: neither of their largest builds carries a `.prm`
|
||
or a full-screen background at all, so they cannot discriminate.
|
||
|
||
**What is still open, stated plainly:** nothing in the bundle has been found that
|
||
orders element 13 behind elements 0–5. Every ordering the file itself carries is
|
||
now either identical to the declaration table or refuted by the capture. The next
|
||
step is no longer static — it is either the guest code that walks this table, or a
|
||
per-draw capture of the title screen showing the order the game submits.
|
||
|
||
**Blocker, checked rather than assumed.** The obvious move is to reuse Canary's
|
||
existing RE instrumentation, which is already in the built binary on
|
||
`sylpheed-re`. Neither hook can answer this:
|
||
|
||
- **`--log_draws`** (`command_processor.cc`) de-dups by a *vertex-declaration
|
||
fingerprint* — shader hash + primitive type + per-stream element
|
||
formats/offsets + index-buffer guest base — and writes each distinct one once.
|
||
A screen's sprites share a declaration, so they collapse; and the record
|
||
carries no texture identity and no per-frame submission order, only first-seen
|
||
order. It is a mesh-format log, not a draw-order log.
|
||
- **The F10 ship capture** does preserve per-draw order within a frame and
|
||
de-dups on `(vertex base, WVP transform, index range)`, which would separate
|
||
the elements — but it **explicitly drops UI draws**:
|
||
`if (pos_off_bytes < 0 …) return; // no float-position stream (UI/effects) —
|
||
skip`. It requires an `f32x3` position attribute, which a 2D quad stream does
|
||
not have.
|
||
|
||
So this needs a **new hook in Canary** — log each draw in submission order with
|
||
its bound texture fetch (or its screen-space quad), gated behind a cvar the way
|
||
the other two are — and therefore a `build-canary` run. That is the cost to
|
||
state up front rather than discover halfway in; it is not a container
|
||
limitation, just a long build plus a title-screen run.
|
||
|
||
### 2026-08-18 (later) — the hook was built and run; the order is now measured
|
||
|
||
`log_ui_draws` exists (Canary branch `auto/re-ui-draw-order`), and the title
|
||
screen's paint order is **ground truth** rather than a candidate:
|
||
[`ui-title-paint-order-capture.md`](ui-title-paint-order-capture.md).
|
||
|
||
Background first, then the `back2` glow pair, then `ptlogo1` + `ptlogo_tm`, then
|
||
`ptlogo2`, then `ptcopyright`, then the `PRESS Ⓐ BUTTON` plate — i.e.
|
||
declaration indices `13, 22|24, 23, 0, 11, 1, 28` and then two elements that are
|
||
**not in build 7 at all**. Two more orderings die on that evidence (keyframe
|
||
start time, resting-keyframe time), and one structural fact reframes the whole
|
||
item: the screen composites **two bundles** (build 7 plus the one-element build
|
||
2 that is the button), so no single build's element table can be the paint order
|
||
whatever its order.
|
||
|
||
**Still open, and now sharper:** the rule. The bundle's 60-byte declaration entry
|
||
carries no depth field (dumped, above); the per-element `.rat` record has not
|
||
been checked for one against this ground truth, and nothing yet explains how the
|
||
two bundles are sequenced. Both are static questions again — the oracle side is
|
||
answered.
|
||
|
||
### 2026-08-18 (third pass) — the bundle does not carry the order at all
|
||
|
||
Three more places checked, all empty, so the static avenue for this item is
|
||
**exhausted** (detail and evidence in
|
||
[`ui-title-paint-order-capture.md`](ui-title-paint-order-capture.md)):
|
||
|
||
- **the geometry has no depth.** A UI quad's attribute 0 is `k_32_32_32_FLOAT`,
|
||
so it carries a Z — and every Z in the capture is 0.00000. Submission order is
|
||
the entire ordering.
|
||
- **the declaration table has no key.** Every word of every entry dumped for the
|
||
build the game actually runs: `+28` 0, `+32` `0xffffffff`, `+36` `0xffffffff`
|
||
(except an instance index on `kind = 0x4`), `+44` `0xffffffff`, `+56` 0.
|
||
- **the placement region has none either**, including its per-group lead word,
|
||
which is 0 for all 24 groups; and the region is followed straight by the RATC
|
||
child stream, so there is no table hiding behind it.
|
||
|
||
Also corrected: the running screen is **build 4**, not the largest build 7 that
|
||
`screen info` defaults to — the two disagree on sprite sizes and the capture
|
||
matches build 4. The conclusions are unchanged, the indices are not.
|
||
|
||
**So the next step is the guest code**, not the file: the splash draw path from
|
||
the emulator-era work (`sub_821CC7A0`, item vtable `0x820b30b4`) submits with
|
||
exactly the PS hash `E59B2B3D` this capture sees, and `xenia-rs/sylpheed.db` is
|
||
available in the container.
|
||
|
||
**And a second screen is NO LONGER BLOCKED, but it is not routine either.** The
|
||
main menu has been reached (screenshot in
|
||
[`canary-scripted-input-traps.md`](canary-scripted-input-traps.md)), so the
|
||
"Ⓐ is dead" reading is withdrawn. **Not routine after all** — see the 2026-08-19 tables in
|
||
[`canary-scripted-input-traps.md`](canary-scripted-input-traps.md): 4 of 5
|
||
successes without `--log_ui_draws`, 0 of 7 with it. An interleaved series
|
||
**refuted the boot-time confound** (the latest title of all, 268 s, accepted Ⓐ;
|
||
a 232 s title refused), and no mechanism exists for the flag — it is read only
|
||
when F10 arms a capture, and F10 was never pressed. The variable was removed rather than
|
||
believed — F10 now arms the capture unconditionally — and with it gone a fresh
|
||
run **still** failed, so the flag is not the cause either. Net: Ⓐ succeeds about
|
||
half the time and nothing measurable predicts which; five explanations are
|
||
eliminated. The input path is now mapped statically (`entry_point` →
|
||
`sub_8216EA68` main loop → `sub_822F1AA8` per-frame input → `sub_82457038` pad
|
||
poll → `XamInputGetKeystrokeEx`), and the poll itself is not state-gated, so the
|
||
gate is in a consumer further up. Until that
|
||
is separated, capturing a screen *and* navigating to it in the same run is not
|
||
dependable. The earlier claim, kept: the title that ends the boot sequence
|
||
accepts a single Ⓐ (2 of 2 at the time); the title the attract loop returns to
|
||
accepts nothing (Ⓐ, START, B,
|
||
BACK, X, Y — dozens of delivered presses). The proposed tell was refuted on the
|
||
way: the two states draw **13 identical quads**, `ptbtn00` included, so they
|
||
differ only to the guest. Recipe: first title after boot, one tap, and never tap
|
||
during the boot (88 presses over the intro ends on a permanent black screen).
|
||
|
||
**The second screen is captured** — the main menu, `GP_TITLE` build 5 — and it
|
||
does not discriminate: its background sits at declaration indices 1–2, so
|
||
"declaration order" and "background first" predict the same sequence. Same
|
||
failure mode as `GP_READY_ROOM`/`GP_OPTIONS`. The next screen worth capturing is
|
||
one whose background sits **late** in its table, as the title's does.
|
||
|
||
The earlier reading, kept because it is what the evidence looked like: the
|
||
title's Ⓐ leads into a content/save path that crashes the guest with
|
||
`--mem_watch=true` and stalls it with `--mem_watch=false`. Three separate traps
|
||
had to be cleared to establish that much — see
|
||
[`canary-scripted-input-traps.md`](canary-scripted-input-traps.md), which also
|
||
carries the reproduction and the fix for two of them.
|
||
|
||
### 2026-08-18 (fourth pass) — the crash is named, and the code avenue is scoped
|
||
|
||
The crash PC resolves to an MSVC `std::map`/`set` erase that throws
|
||
`std::out_of_range` from the game's cache-manager flush, and the trigger is now
|
||
controlled: an **incomplete on-disc cache** throws ~100 s into a boot, a complete
|
||
one never does. The access violation people have been chasing is only that throw
|
||
*returning*, because this build does not unwind guest EH. And the handoff's
|
||
suspect #1 is **eliminated** — cold cache with `--mem_watch=false` throws just
|
||
the same, which withdraws a claim made here yesterday. See
|
||
[`title-crash-stl-tree.md`](title-crash-stl-tree.md). That is a by-product of
|
||
this item and belongs to whoever picks up the crash bisection.
|
||
|
||
For the ordering itself, three more negatives, all recorded in
|
||
[`ui-title-paint-order-capture.md`](ui-title-paint-order-capture.md): the two
|
||
time-based orderings were re-checked against **build 4** (the previous pass used
|
||
build 7's numbers, and build 7 is not what runs) and both still fail on the same
|
||
element; and a fresh candidate — painter's order by resting **Y** — reproduces
|
||
the capture to within a single transposition but is refuted by `ptlogo_tm` and by
|
||
the background, so it is not the rule either.
|
||
|
||
The code avenue is scoped rather than walked: the splash item vtable
|
||
`0x820b30b4` is real (25 slots, three construction sites), RTTI carries **no**
|
||
class names disc-wide, and the format tags are fourcc immediates behind a virtual
|
||
call rather than strings — so this needs a deliberate read of the UI engine, not
|
||
a keyword search.
|
||
|
||
|
||
## Capital ships assemble wrong in the viewer
|
||
|
||
**Reported:** 2026-07-30, by the user. **Status:** ✅ **format-side cause found and
|
||
fixed 2026-08-12** — see below for the 2026-08-10 diagnosis this supersedes.
|
||
|
||
The remaining format-side defect this entry pointed at (a shared turret decoding
|
||
~100× too large in some containers) was real and is gone. `e303_wep_01` decoded
|
||
as a 1600×2100×4800 block in `Stage_S02`, swallowing the `e106` hull; requiring an
|
||
index buffer to cover its vertex pool **exactly** moved it to the block every
|
||
other container agrees on, and it now decodes 49×23×42 everywhere and places at
|
||
±179 on the hull. The same fix repaired `e106_bdy_03` (a 600×1600×998 slab) and
|
||
moved 29 anchors disc-wide, 22 of which had been carrying **another resource's
|
||
geometry under their own name**. See
|
||
[`structures/xbg7-mesh.md`](structures/xbg7-mesh.md).
|
||
|
||
Two things are worth carrying forward rather than closing:
|
||
- the assembler was **audited and exonerated** — every composite node carries
|
||
scale 1.0 and an orthonormal matrix, so nothing on that side inflates a part;
|
||
- **no metric caught this.** Coverage, cross-container consistency, the capture
|
||
oracle and the twin invariant were all green while a 1 600-unit slab sat through
|
||
the ship. It was found by *rendering the ship and looking at it*, and the
|
||
numeric screens written afterwards to automate that check both failed.
|
||
|
||
The 2026-08-10 diagnosis follows, and its viewer-side pointers still stand.
|
||
|
||
**Status (2026-08-10):** 🔎 **the format layer is exonerated.** Runtime captures of three classes (`f105`, `e105`,
|
||
`e106`) at controlled range reproduce `assemble_ship` to ≤0.43 units in translation
|
||
and to 0.000 in rotation for every part that does not move; see
|
||
[`ship-placement-capture-generalisation.md`](ship-placement-capture-generalisation.md)
|
||
§4. So look at **the viewer**: first that it passes `include_external = true`
|
||
(`iso_loader.rs:4012` — with `false` an e106 loses its bridge and both nacelles,
|
||
5 parts instead of 11), then its own transform stack.
|
||
|
||
One real format-side bug was found on the way and is **fixed**: index-less parts
|
||
(`e105_brg`) never matched their `GN_Bridge_01` hardpoint, so 34 (stage, ship) entries
|
||
— `e102`, `e104`, `e105` across Stages 02–29 — assembled without a bridge. The other
|
||
apparent exception (`e105_eng_01` rotation) was an aggregation artefact and is 0.000.
|
||
|
||
The original report and its reasoning follow.
|
||
|
||
The reborn viewer builds capital ships from the split XBG7 parts via
|
||
`sylpheed-formats::ship::assemble_ship`, and they come out **wrong** — parts in the
|
||
wrong place / wrong orientation.
|
||
|
||
**Why this is a real finding and not a known limitation:** the RE write-up
|
||
[`ship-placement-runtime-capture.md`](ship-placement-runtime-capture.md) declares
|
||
static assembly ✅ **exact** as of 2026-07-26 — 9-channel joint tables
|
||
`[TX TY TZ RY RX RZ SX SY SZ]`, Euler `Ry·Rx·Rz`, with
|
||
`ship::tests::static_assembly_matches_runtime_capture` asserting static == runtime
|
||
capture (T < 1.0, R < 0.02). So either the viewer is not using that path, or the
|
||
claim generalises worse than the test suggests.
|
||
|
||
**The likely gap:** that test is **one ship** — the `e106` destroyer, 8 parts plus
|
||
two nacelles, two turrets and the hull mirror. Nothing pins the other classes.
|
||
Rules that were derived from `e106` and could easily be `e106`-specific:
|
||
|
||
- the engine cluster rig mounted at `GN_Engine_01` (two mirrored nacelles + centre);
|
||
- "X-reflect the shared-geometry twin whose lateral offset opposes the geometry's
|
||
dominant side" — a heuristic, not a decoded flag;
|
||
- cross-id turret instancing (×2).
|
||
|
||
**First step (the oracle already exists):** re-run the runtime capture on a *different*
|
||
capital ship and diff static vs captured, exactly as `e106` was done — F10 in the
|
||
`capture-ship-placement` build of `xenia-canary-native` dumps the ship shader's
|
||
`c0..c2` WorldViewProjection rows per part; `WV_ref⁻¹ · WV_p` is the ship-space rigid
|
||
transform, which is ground truth. Pick a class whose rig differs from `e106`
|
||
(different engine count, a ship with no `sld`, a carrier). Then extend
|
||
`static_assembly_matches_runtime_capture` into a per-ship table so a regression in one
|
||
class cannot hide behind `e106` passing.
|
||
|
||
**Also worth ruling out first, cheaply:** that the viewer's own transform stack (scale,
|
||
handedness, node-instance recursion) is not re-breaking a correct assembly — compare
|
||
the viewer's placement against `assemble_ship`'s output directly before blaming the
|
||
format layer.
|
||
|
||
---
|
||
|
||
## Viewer: `include_external` is already on — that hypothesis is dead
|
||
|
||
**Checked 2026-08-11.** The item above names "first that it passes
|
||
`include_external = true` (`iso_loader.rs:4012`)" as the cheap first step. It
|
||
does: `ShipBrowser::show_external` defaults to `true`
|
||
(`iso_loader.rs:643`), the checkbox reads it (`ui.rs:1593`) and it is threaded
|
||
through `RequestShipRender` → `build_ship_model` → `assemble_ship` unchanged
|
||
(`ui.rs:1689`, `iso_loader.rs:4012`). So a ship rendered by the viewer is the
|
||
full external assembly, not the bare hull.
|
||
|
||
The viewer also does not have a transform stack of its own to blame: it bakes
|
||
`ScenePart::apply` straight into the vertices and rotates normals by the same
|
||
`p.m` (`iso_loader.rs:4030-4062`), so its placement is `assemble_ship`'s output
|
||
by construction. What remains unexcluded, in order of cheapness: the mirror
|
||
handling (`det < 0` reverses triangle winding only — a reflected part keeps its
|
||
reflected geometry), `Xbg7Model::models_named` resolving the wrong sub-model when
|
||
a resource name repeats, and the exhaust cones. **Next step is a visual**: the
|
||
diagnosis has run out of things it can settle by reading, so the viewer needs to
|
||
be run against a known-good class (`e106`) and its render compared with
|
||
`ship_render`'s.
|
||
|
||
---
|
||
|
||
## Viewer: the duplicate-resource-name hypothesis is dead too
|
||
|
||
**Checked 2026-08-11.** The diagnosis above left three candidates for why capital
|
||
ships assemble wrong in the viewer: mirror handling, `Xbg7Model::models_named`
|
||
resolving the wrong sub-model when a resource name repeats, and the exhaust
|
||
cones. The second is now **refuted**, and comprehensively.
|
||
|
||
`build_ship_model` resolves each placement with
|
||
`base.iter().find(|m| m.name == p.resource)` (`iso_loader.rs:4041`) — first match
|
||
wins — so a repeated resource name inside a container would silently draw the
|
||
wrong geometry. It cannot happen: decoding **every** XBG7 resource in **all 22
|
||
stage containers** gives **4 603 resources and zero repeated names**.
|
||
|
||
```
|
||
Stage_S01 62/62 Stage_S07 323/323 Stage_S13 290/290 Stage_S25 351/351
|
||
Stage_S02 304/304 Stage_S08 388/388 Stage_S14 22/22 Stage_S26 318/318
|
||
Stage_S03 214/214 Stage_S09 316/316 Stage_S15 386/386 Stage_S27 321/321
|
||
Stage_S04 179/179 Stage_S10 7/7 Stage_S16 65/65 Stage_S28 118/118
|
||
Stage_S05 92/92 Stage_S11 157/157 Stage_S24 162/162 Stage_S29 386/386
|
||
Stage_S06 266/266 Stage_S12 376/376
|
||
```
|
||
|
||
Per-ship it is tighter still: `e106` wants 9 distinct names and decodes exactly
|
||
9 models for 11 placements; `e105` 9 for 9; `f105` 5 for 6. Every placement
|
||
resolves to the one model it names.
|
||
|
||
**So two of the three candidates are gone** (this one and `include_external`),
|
||
leaving **mirror handling** and **the exhaust cones** — and the still-untried
|
||
visual comparison, which remains the right next step.
|
||
|
||
---
|
||
|
||
## Viewer: mirror handling and the exhaust cones are cleared too — the static avenue is exhausted
|
||
|
||
**Checked 2026-08-11.** Both remaining candidates were tested across every ship
|
||
on the disc, and neither shows the reported signature.
|
||
|
||
**Mirror handling.** The concern was that `ScenePart::apply` bakes `R·(S·v)+T`
|
||
while the viewer takes its winding-flip decision from `det(m)` alone and rotates
|
||
normals by `m` alone — both ignoring `s`. A mirror encoded as a *negative scale*
|
||
would then reflect geometry without flipping winding, drawing the part
|
||
inside-out. It never happens: across **1 485 assembled parts** in all 22
|
||
containers there are **22 mirrored parts, every one with `det(m) < 0`**, and
|
||
**zero** parts with a negative scale or a non-uniform one. `apply_twin_mirrors`
|
||
writes the reflection into `m` (negating its X column), so the viewer's flip
|
||
always fires, and ignoring `s` for normals is harmless because `s` is always
|
||
uniform.
|
||
|
||
**Exhaust cones.** These are the one piece of geometry the viewer *invents* — a
|
||
cone at each `GN_Jet`/`GN_SJet` frame, because the real engine geometry is
|
||
recessed and the game draws FX there instead. If they landed wrongly they would
|
||
read exactly as "a part in the wrong place". Across **335 assembled ships, 192 of
|
||
which have exhaust frames, not one cone sits outside its hull's bounding box**
|
||
(tolerance 10 % of the axis span).
|
||
|
||
**Caveat, stated rather than glossed:** "inside the hull box" does not prove a
|
||
cone is *right* — orientation and size are untested, and a cone could be wrong
|
||
while still inside. What it does rule out is the reported symptom for that part.
|
||
|
||
So every mechanism this diagnosis proposed is now eliminated: `include_external`,
|
||
duplicate resource names, mirror handling, and cones-in-the-wrong-place. The
|
||
format and assembly layers pass every static test available, and **the visual
|
||
comparison is no longer merely the next step — it is the only remaining one.**
|
||
Render `e106` in the viewer beside `ship_render`'s output of the same
|
||
`assemble_ship` result; if they agree, the bug is in neither and the original
|
||
report needs re-grounding against a specific ship and a specific expectation.
|
||
|
||
---
|
||
|
||
## ⚠️ DIAGNOSED 2026-08-12 — a mis-decode; the locality fix was written, then withdrawn
|
||
|
||
> Resolution at the end of this entry. Kept in full because the two wrong turns
|
||
> along the way (a "stray volume", then "monotonic anchoring") are the useful part.
|
||
|
||
## ⚠️ The format layer is NOT exonerated — but the cause is a MIS-DECODE, not a stray volume
|
||
|
||
**Found 2026-08-11 by finally doing the visual**, which the notes above kept
|
||
naming as the next step. It overturns their conclusion.
|
||
|
||
Render `e106` from the static assembly and from the baked runtime capture and
|
||
compare — `ship_render` does both:
|
||
|
||
| | placements | parts |
|
||
|---|---|---|
|
||
| runtime capture (ground truth) | **8** | `bdy_01…04`, `brg_01`, `eng_01`, `eng_02`, `wep_02_01` |
|
||
| `assemble_ship(--static)` | **11** | the same 8, **plus `e303_wep_01` ×2** and a second `e106_eng_01` |
|
||
|
||
The render makes it obvious: the destroyer sits inside a white slab that dwarfs
|
||
it ([capture](captures/e106-static-assembly-volume-bug.png)). That slab is
|
||
`e303_wep_01`, and its own geometry is:
|
||
|
||
```
|
||
e303_wep_01 172 verts, 110 tris bounds X[-1000, 600] Y[-1050, 1050] Z[-2400, 2400] 1600 x 2100 x 4800
|
||
e106_wep_02_01 1002 verts, 772 tris 269 x 179 x 417 ← what a real e106 turret looks like
|
||
e106_brg_01 202 verts, 202 tris 105 x 76 x 305
|
||
```
|
||
|
||
**110 triangles, perfectly round axis-aligned bounds, and bigger than the ship it
|
||
is mounted on.**
|
||
|
||
### CORRECTION (same day, one iteration later): it is not a volume — it is a bad decode
|
||
|
||
The first reading of this was that `e303_wep_01` is a collision/trigger volume
|
||
the assembler wrongly draws. **That is wrong, and the evidence that settles it is
|
||
decoding the same resource from every container that holds it:**
|
||
|
||
```
|
||
Stage_S01 172 verts 110 tris X[-24.5, 24.5] Y[0.0, 23.4] Z[-20.8, 20.8] ← 49 × 23 × 42, a turret
|
||
Stage_S02 172 verts 110 tris X[-1000, 600] Y[±1050] Z[±2400] ← 1600 × 2100 × 4800
|
||
Stage_S03… 172 verts 110 tris 49 × 23 × 42 (correct)
|
||
Stage_S08 … 1600 × 2100 × 4800
|
||
Stage_S26 … 1600 × 2100 × 4800
|
||
```
|
||
|
||
Same resource, same vertex and triangle count, **decoding correctly in eleven
|
||
containers and wrongly in exactly three** (`Stage_S02`, `S08`, `S26`). So:
|
||
|
||
- the **placement is legitimate** — `e303_wep_01` is a small shared turret,
|
||
cross-mounted on `e101` and `e106`, and at its true size it is unremarkable;
|
||
- the original author's explanation of the capture's silence (**vbase dedup**)
|
||
stands, and my "dedup would show one, not zero" objection does not survive:
|
||
with the correct decode the turret is small, ordinary geometry;
|
||
- **the defect is in the mesh decoder**, which resolved this resource's vertex
|
||
data differently in three containers.
|
||
|
||
The render and the symptom are real; the cause named in the first version of this
|
||
entry was not.
|
||
|
||
### The part that matters more than this one resource
|
||
|
||
**The decoder can produce wrong geometry without declining.** The
|
||
[XBG7 audit](structures/xbg7-mesh.md) counted 814 resources it *refuses* — a
|
||
visible, honest failure. This is the other kind: `e303_wep_01` decodes "fine" in
|
||
`Stage_S02` and is silently 100× too large. Screening for the signature (bounds
|
||
that are exact multiples of 50 with a span over 1000) flags 22–32 models in each
|
||
of `S02`, `S03`, `S08`, `S26`, `S27` — **but that screen also catches legitimate
|
||
`e_rou_*` composite proxies**, so it is a candidate list, not a count of bugs.
|
||
|
||
**Next:** diff the anchor scan's chosen `vb0` for `e303_wep_01` between
|
||
`Stage_S01` (correct) and `Stage_S02` (wrong) — same resource, two outcomes, so
|
||
the divergence is directly observable — then use whatever distinguishes them to
|
||
add a post-decode sanity check, so a silent 100× mis-decode becomes a decline.
|
||
|
||
### Why this was missed
|
||
|
||
`assemble_ship` treats **every** `rou_*` node in the composite as a drawable
|
||
part, and the doc comment states the cross-id mount as intended behaviour —
|
||
`"INCLUDING repeated instances and cross-id turret mounts (rou_e303_wep_01_root
|
||
×2 on the e106 hull)"` — with
|
||
`ship::tests::static_assembly_matches_runtime_capture` asserting
|
||
`count("e303_wep_01") == 2`. The absence from the capture was explained away as
|
||
vbase dedup, but **dedup would show one instance, not zero**.
|
||
|
||
The test cannot catch it either: it walks the capture's parts and looks each up
|
||
in the static output, so **extra** static placements are invisible to it. That is
|
||
the same shape of gap as the earlier `include_external` hypothesis — a test that
|
||
can only fail one way.
|
||
|
||
### Scope, stated carefully
|
||
|
||
Sweeping all 335 assembled ships for the signature *ship-scale span with under
|
||
400 triangles* flags **20 ships and 58 placements** over 28 distinct resources
|
||
(`e005_ant_*`, `f001_ant_*`, `f002_bdy_*`, `f301_barrel`, `f303_body`,
|
||
`e303_wep_01`, …). **Only the `e106`/`e303_wep_01` case is proven** — by render,
|
||
by capture absence, and by geometry. Some of the others may be legitimately large
|
||
low-poly parts, and each needs the same three checks before being called a bug.
|
||
|
||
**Still true, and independent of the correction above:**
|
||
`static_assembly_matches_runtime_capture` walks the capture's parts and looks each
|
||
up in the static output, so **extra static placements can never fail it**. That is
|
||
worth fixing regardless — it is the same one-way-test shape as the earlier
|
||
`include_external` hypothesis.
|
||
|
||
Also unchanged: only **two** cross-id placements exist fleet-wide (`e303_wep_01`
|
||
on `e101` ×24 and `e106` ×36, across 335 assembled ships), so cross-id mounting is
|
||
a narrow, real feature rather than a systemic guess.
|
||
|
||
---
|
||
|
||
## Resolution (2026-08-12)
|
||
|
||
`anchor_pool_mesh` took the **first** candidate in file order from a
|
||
container-global scan, so a resource could be handed another resource's block
|
||
whenever both shared `(stride, vertex count, index count)`. Fixed by anchoring
|
||
each resource near its **descriptor neighbours** (two-pass: learn, then re-anchor).
|
||
|
||
- it took inconsistency **125 → 51** with coverage unchanged, and made `e106`
|
||
render correctly ([after](captures/e106-static-assembly-fixed.png))
|
||
- **but it flipped the `e106` twin-mirror decision**, which
|
||
`static_assembly_matches_runtime_capture` (ISO-gated, so it skips in a plain
|
||
`cargo test`) catches against the runtime capture — so it was **reverted**
|
||
- the user-reported "capital ships assemble wrong" is therefore **diagnosed, not
|
||
yet fixed**; see [xbg7](structures/xbg7-mesh.md) for what the real fix needs
|
||
|
||
Still open from this entry: `static_assembly_matches_runtime_capture` walks only
|
||
the capture's parts, so **extra** static placements still cannot fail it.
|
||
|
||
### 2026-08-18 — that last line was stale, and the residual gap is now closed too
|
||
|
||
**The one-way-test complaint had already been fixed** when this entry was
|
||
written down: `64d372c` (the revert commit itself) added an extras check, so
|
||
"extra static placements cannot fail it" has not been true since. Checked rather
|
||
than assumed — perturbing the expectation makes the test fail with the real disc
|
||
behind it, so it runs and is live, not a `SYLPHEED_ISO`-less skip.
|
||
|
||
**But it compared a set of resource *names*, which leaves one direction open**: a
|
||
resource placed *twice* when the capture lists it once changes no set. That is
|
||
not hypothetical — a duplicated instance is exactly what a bad node walk emits,
|
||
and the two legitimate duplicates here (`e106_eng_01`, `e303_wep_01`) are the
|
||
reason the test had to special-case counts at all. Replaced with the full
|
||
**multiset**, pinned to the e106 ground truth:
|
||
|
||
```
|
||
e106_bdy_01 1 e106_bdy_02 1 e106_bdy_03 1 e106_bdy_04 1 e106_brg_01 1
|
||
e106_eng_01 2 e106_eng_02 1 e106_wep_02_01 1 e303_wep_01 2
|
||
```
|
||
|
||
— 9 resources, 11 placements, against the capture's 8 dedup'd parts. That
|
||
subsumes the two hand-written count assertions, and it now fails on an extra
|
||
resource, a missing one, **and** a duplicated one. Refuted before believing:
|
||
declaring `e106_bdy_01` twice makes it fail, with the real multiset on the left.
|
||
|
||
**Not closed by this**, and worth keeping separate: the multiset is `e106`'s
|
||
alone. The generalisation this entry originally asked for — a per-ship table so
|
||
a regression in one class cannot hide behind `e106` passing — still needs a
|
||
runtime capture of a *second* capital ship.
|
||
|
||
**That entry's stated blocker is stale** (checked 2026-08-19): the ship capture
|
||
is in the current build — `RequestShipCaptureFrame` / `CaptureShipDrawForRE` are
|
||
in `command_processor.cc` on `auto/re-ui-draw-order`, and F10 wrote a 2.9 MB
|
||
`xenia_ship_capture_01.log` from this session's binary. No separate
|
||
`capture-ship-placement` build is needed.
|
||
|
||
**Update 2026-08-19: the mission is now REACHABLE.** With the Canary threading
|
||
fix, `tutorial_launch.sh` drives boot → title → menu → TUTORIAL and the mission
|
||
**loads and renders** (flight HUD, "Go to the box on your screen"). It then
|
||
freezes under 13 243 crash dumps, all at `0x82307128`, preceded by exactly one
|
||
guest C++ throw — identical frames 6 s apart, no new dumps, 400 % CPU. So the
|
||
blocker moved from "cannot reach a mission" to "the mission freezes". 🔴 **The cache is REFUTED as the cure** (3 runs): the
|
||
missing entry `\aab216c3\6` was real and got written, and the run with a complete
|
||
cache stormed anyway — 11 497 dumps, all `0x82307128`. ✅ **But a usable window
|
||
exists:** both post-cache runs ran the mission with exactly **2 crashes for
|
||
56–80 s** before the storm, where the first run was at 641 by t+24 s. The ship
|
||
capture needs `F10` armed *inside* that window. ✅ **Done, and the mission ran
|
||
with ZERO crashes** — first clean mission run, fully rendered. 🔴 **But the
|
||
capture contains no ship geometry**: 181 deduped draws, 180 sharing one vertex
|
||
shader, all screen-space, none 3D — against a known-good 2.9 MB capture from an
|
||
earlier session. The 8 000-draw budget was not the limit and the scene *was*
|
||
rendering. ⚠️ The "cache refuted" claim above is **overstated**: this run used
|
||
the same complete cache as `tut4` and got 0 crashes vs 11 497, so variance
|
||
dominates. 🔴 **Corrected:** nothing is broken. Ship
|
||
geometry is `stride=24 prim=4` with a large vcount (`vcount=10891` for a real
|
||
one); this capture has one `prim=4 vcount=6` quad and the 2.9 MB "known-good"
|
||
file has **no `prim=4` at all** — it is a **UI** capture (1 303 of 1 582 draws
|
||
are `stride=24 prim=13`, the UI sprite shader). The earlier "3D draws" test
|
||
counted UI sprite coordinates as 3D. The capture recorded what was on screen, and
|
||
the tutorial's opening has **no capital ship**. **What remains** is what the
|
||
capture doc always said: play into a real mission and frame a ship side-on —
|
||
gameplay driving, not a menu step, and the original was taken on HW Vulkan where
|
||
this container has lavapipe. 🔴 The resume-refused lead (1 663 on one
|
||
thread) is **REFUTED**: that thread did execute, and `KeWaitForSingleObject` /
|
||
`NtWaitForSingleObjectEx` are `kHighFrequency`, which is unlogged unless
|
||
`--log_high_frequency_kernel_calls=true` — so a parked thread is invisible and
|
||
the refusals are just the guest kicking a worker blocked on an object. Method
|
||
note: the title-loader finding rested on **host CPU time**, not log silence,
|
||
which is why it stands and this did not.
|
||
|
||
What blocked it before was **the cache-flush crash**, not navigation — measured
|
||
2026-08-19. `tools/re-capture/tutorial_launch.sh` (which retries whole boots,
|
||
because re-pressing the same title never works) gets all the way from the title
|
||
through the main menu to **DIFFICULTY** and then **SELECT DATA**, and the guest
|
||
dies there at `0x82307128` — the same `std::map`/`set` erase as the boot-time
|
||
throw, 537 stacked dumps, with `--mem_watch=false`. See
|
||
[`title-crash-stl-tree.md`](title-crash-stl-tree.md).
|
||
|
||
So a second capital-ship capture needs that crash dealt with first. Everything up
|
||
to the save-slot screen is now scripted and works, and one run got *past* it —
|
||
`SELECT DATA` reached with zero crashes, slot chosen, the game proceeding into a
|
||
cinematic — before crashing at the same `0x82307128`. The crash is intermittent
|
||
in **where** it fires, not whether, so there is no menu route around it. See
|
||
[`title-crash-stl-tree.md`](title-crash-stl-tree.md) for the end-to-end
|
||
measurement and for what has been ruled out (`--mem_watch=false`, twice).
|