fix(decoder): give the container the Canary it is supposed to run
Some checks failed
CI / Native — linux (pull_request) Failing after 1h1m35s
CI / WASM — Web (pull_request) Successful in 24m37s
CI / Formatting (pull_request) Successful in 26s

The database mount below was one of three ways the decoder could not reach its
own oracle. The other two are here.

`run-canary` never looked in `Checked/`. It tried `Release/` then `Debug/`, and
both of those exist on this box -- an Aug 28 binary and a Jul 19 one. They boot
the game perfectly well and carry NO `audit_61` branch probe, so a probe run
against either returns zero hits that read as a finding about the game rather
than as a stale binary. Configuration is now the outer loop and location the
inner one, so a `Checked` build anywhere beats a `Release` build anywhere;
`$XENIA_BIN` still overrides everything. Measured here: `Checked` has
`audit_61_branch_probe_pcs`, `Release` and `Debug` do not.

The launcher also now says which instrumentation is missing BEFORE the run,
because the alternative is reading an empty log afterwards and guessing.

`build-canary` built `$PROJECT_DIR/xenia-canary`, which does not exist in this
container -- the source is bind-mounted at `/canary` and the launcher already
exports `XENIA_SRC=/canary`. CONTAINER-NOTES has carried that defect since
2026-08-29 with a symlink workaround and a warning to remember to delete the
symlink afterwards. It now reads `$XENIA_SRC` first, so there is nothing to
remember. Its default configuration moves Release -> Checked to match what
`run-canary` picks; the old default spent a full build on a binary nothing ran.

Two documented blockers are refuted rather than deleted, since the sequence of
wrong readings is what makes the right one checkable: the CONTAINER-NOTES
symlink dance (the warm build volume it was configured against is gone too,
removed in the 2026-09-18 cleanup, so the next build configures cleanly against
`/canary`), and `upstream-baseline.md`'s "`version.h` is never generated" --
`CMakeLists.txt` generates it at configure time now, with a stub fallback.

`decoder-loop.md` claimed the oracle was at `Linux/Release/` and that the probe
was on two side branches; both were true when written and neither is now.

Verified: five pick_bin cases against the extracted function body, and `strings`
on all three real binaries.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-09-21 18:08:10 +02:00
parent 8108958a77
commit 1fdbb5f197
5 changed files with 69 additions and 26 deletions

View File

@@ -10,11 +10,22 @@
# AVAILABLE MEMORY as well as core count — a full-parallel build of this tree # AVAILABLE MEMORY as well as core count — a full-parallel build of this tree
# has OOM-killed the host outright. # has OOM-killed the host outright.
# #
# build-canary [Release|Debug] [extra cmake --build args] # `Checked` is the default, and should stay it: it is what the host builds and
# what `run-canary` picks first, so the two agree by construction. Building
# `Release` here instead leaves a second binary that `run-canary` will not use
# -- hours of CPU for something nothing runs. `Checked` is optimised, with
# assertions left in.
#
# build-canary [Checked|Release|Debug] [extra cmake --build args]
set -euo pipefail set -euo pipefail
CONFIG="${1:-Release}"; shift || true CONFIG="${1:-Checked}"; shift || true
SRC="${PROJECT_DIR:-/work}/xenia-canary" # $XENIA_SRC FIRST. In this container the Canary source is bind-mounted at
# `/canary`, and `$PROJECT_DIR/xenia-canary` does not exist -- so the old default
# made this script unusable here, and the documented workaround was to symlink
# `/canary` into the repository and remember to delete it again (CONTAINER-NOTES
# §"The toolchain is real"). The launcher already exports the right path; read it.
SRC="${XENIA_SRC:-${PROJECT_DIR:-/work}/xenia-canary}"
BUILD="${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}" BUILD="${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}"
JOBS="${SYLPH_JOBS:-2}" JOBS="${SYLPH_JOBS:-2}"

View File

@@ -48,14 +48,25 @@ fi
PROJECT_DIR="${PROJECT_DIR:-/work}" PROJECT_DIR="${PROJECT_DIR:-/work}"
# ── Binary ─────────────────────────────────────────────────────────────────── # ── Binary ───────────────────────────────────────────────────────────────────
# `Checked` FIRST. All three configurations can be built, but `Checked` is the
# one this project actually builds, so it is the one carrying our
# instrumentation; the `Release/` and `Debug/` binaries beside it are months-old
# leftovers that still run and still boot the game, which is exactly what makes
# them dangerous -- a probe run against one reports zero hits and reads as a
# finding about the game. Measured on this box 2026-09-21: `Checked` has
# `audit_61_branch_probe_pcs`, `Release` (Aug 28) and `Debug` (Jul 19) do not.
pick_bin() { pick_bin() {
[ -n "${XENIA_BIN:-}" ] && { echo "$XENIA_BIN"; return; } [ -n "${XENIA_BIN:-}" ] && { echo "$XENIA_BIN"; return; }
for c in \ # Configuration is the OUTER loop, location the inner one: a `Checked` build
"${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}/bin/Linux/Release/xenia_canary" \ # anywhere beats a `Release` build anywhere. The other order picks a stale
"${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}/bin/Linux/Debug/xenia_canary" \ # container `Release` over a fresh instrumented repo `Checked`, which is the
"$PROJECT_DIR/xenia-canary/build/bin/Linux/Release/xenia_canary" \ # exact mistake this is here to stop.
"$PROJECT_DIR/xenia-canary/build/bin/Linux/Debug/xenia_canary"; do for cfg in Checked Release Debug; do
[ -x "$c" ] && { echo "$c"; return; } for c in \
"${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}/bin/Linux/$cfg/xenia_canary" \
"$PROJECT_DIR/xenia-canary/build/bin/Linux/$cfg/xenia_canary"; do
[ -x "$c" ] && { echo "$c"; return; }
done
done done
} }
BIN="$(pick_bin)" BIN="$(pick_bin)"
@@ -64,6 +75,13 @@ if [ -z "${BIN:-}" ]; then
exit 1 exit 1
fi fi
# Say what is in the binary before the run, not after reading an empty log.
# A missing probe is a build that predates it, never a quiet game.
for sym in audit_61_branch_probe_pcs RE-DRAW; do
grep -aqm1 -- "$sym" "$BIN" \
|| echo "run-canary: ⚠ $BIN has NO '$sym' -- it predates that instrumentation. Rebuild with: build-canary" >&2
done
# ── ISO ────────────────────────────────────────────────────────────────────── # ── ISO ──────────────────────────────────────────────────────────────────────
ISO="${SYLPH_ISO:-}" ISO="${SYLPH_ISO:-}"
if [ -z "$ISO" ]; then if [ -z "$ISO" ]; then

View File

@@ -12,13 +12,20 @@ already went wrong once.
* **The toolchain is real.** `tools/re-capture/rebuild_canary.sh` exists because * **The toolchain is real.** `tools/re-capture/rebuild_canary.sh` exists because
the old box had no cmake/ninja/clang and only runtime sonames, so it hand- the old box had no cmake/ninja/clang and only runtime sonames, so it hand-
relinked object files. **Do not use it here.** Use `build-canary`. relinked object files. **Do not use it here.** Use `build-canary`.
🔴 **But `build-canary` does not work in this container as it stands **FIXED 2026-09-21 — `build-canary` now reads `$XENIA_SRC` first**, which the
launcher already sets to `/canary`, so there is no symlink to make and none to
forget. The two paragraphs below are kept because they are how the defect was
found and refuted, but **neither describes the script any more.**
🔴 ~~**`build-canary` does not work in this container as it stands
(2026-08-29).** It builds `${PROJECT_DIR:-/work}/xenia-canary`, which **does (2026-08-29).** It builds `${PROJECT_DIR:-/work}/xenia-canary`, which **does
not exist here** — the Canary source is at **`/canary`** (`$XENIA_SRC`). The not exist here** — the Canary source is at **`/canary`** (`$XENIA_SRC`). The
warm 235 MB tree at `/sylph-home/re/canary-build` is configured with warm 235 MB tree at `/sylph-home/re/canary-build` is configured with
`CMAKE_HOME_DIRECTORY=/work/xenia-canary`, also missing, and its `CMAKE_HOME_DIRECTORY=/work/xenia-canary`, also missing, and its
`build-Release.ninja` carries **no per-file rules** — it wants to re-run CMake `build-Release.ninja` carries **no per-file rules** — it wants to re-run CMake
first, which would fail on the absent source root. first, which would fail on the absent source root.~~ The warm tree is gone too:
every `sylph-*` volume was removed in the 2026-09-18 cleanup, so the next build
configures from scratch against `/canary` and caches the right source root.
**The conclusion drawn from all that is REFUTED (2026-08-31).** The note went **The conclusion drawn from all that is REFUTED (2026-08-31).** The note went
on to say *"any Canary change is a full reconfigure against `/canary` plus a full on to say *"any Canary change is a full reconfigure against `/canary` plus a full
@@ -38,9 +45,10 @@ already went wrong once.
warm 235 MB tree is otherwise intact and the ninja re-run resolves its rules warm 235 MB tree is otherwise intact and the ninja re-run resolves its rules
from the symlink. from the symlink.
⚠️ **Remove the symlink when you are done.** `/work` is the repository, and ⚠️ ~~**Remove the symlink when you are done.**~~ No longer needed — but if you
`xenia-canary` is not in `.gitignore`, so it shows up as untracked and can be ever make one by hand, note that `/work` is the repository and `xenia-canary`
swept into a `git add -A`. is not in `.gitignore`, so it shows up as untracked and can be swept into a
`git add -A`.
* **numpy and Pillow are installed.** `entities2.py`, `flight_probe.py` and the * **numpy and Pillow are installed.** `entities2.py`, `flight_probe.py` and the
image oracles work. Their absence used to look like a logic bug. image oracles work. Their absence used to look like a logic bug.

View File

@@ -57,7 +57,7 @@ rather than left in a document you might not reach.
|---|---|---| |---|---|---|
| `/image/sylpheed.pe` | the decompressed executable image | `SYLPHEED_PE` | | `/image/sylpheed.pe` | the decompressed executable image | `SYLPHEED_PE` |
| `sylpheed.db` at the repo root | the disassembly database, 337 MB | — (`zq.py` finds it; `$SYLPHEED_DB` only to override) | | `sylpheed.db` at the repo root | the disassembly database, 337 MB | — (`zq.py` finds it; `$SYLPHEED_DB` only to override) |
| `/canary/build/bin/Linux/Release/xenia_canary` | the built oracle, inside the `/canary` mount | — | | `/canary/build/bin/Linux/Checked/xenia_canary` | the built oracle, inside the `/canary` mount | — |
| `/disc` | the extracted disc | `SYLPHEED_DISC` | | `/disc` | the extracted disc | `SYLPHEED_DISC` |
| `/iso/game.iso` | the retail ISO Canary boots | `SYLPH_ISO` | | `/iso/game.iso` | the retail ISO Canary boots | `SYLPH_ISO` |
| `/canary` | the Canary source, read-write | `XENIA_SRC` | | `/canary` | the Canary source, read-write | `XENIA_SRC` |
@@ -67,11 +67,16 @@ rather than left in a document you might not reach.
says so and prints the command that builds it. Query the database with says so and prints the command that builds it. Query the database with
`python3 tools/zq.py …`, never by reading a disassembly dump. `python3 tools/zq.py …`, never by reading a disassembly dump.
⚠️ **The built Canary carries the `RE-INPUT`/`RE-DRAW` instrumentation but NOT ⚠️ **Take the oracle from `Checked/`, not `Release/`.** `Checked` is what this
the `audit_61` guest-PC branch probe** — that lives only on tree actually builds (optimised, with assertions); the `Release/` binary beside
`auto/canary-instrumentation-snapshot-2026-07-28` and `phase-a-tracing` it is a stale August build with no `audit_61` probe in it. `strings` on the two
(fork issue #1), so a probe run needs those two commits on `sylpheed-re` and a is how that was found, and is how to check any binary before quoting a run from
rebuild first. Verified by `strings` on the binary, not assumed. it.
The `Checked` build carries all three instrumentations — `RE-INPUT`, `RE-DRAW`
and the `audit_61` guest-PC branch probe (fork PR #2 brought the probe onto
`sylpheed-re`; before it, the probe lived only on `phase-a-tracing`). Verified
by `strings` on the binary, not assumed.
**The `.pe` is a flat VA dump**: file offset = `VA - 0x82000000`. Reading **The `.pe` is a flat VA dump**: file offset = `VA - 0x82000000`. Reading
`0x820A1630` is `seek(0xA1630)`. No XEX decrypt, no LZX, **no booted emulator** `0x820A1630` is `seek(0xA1630)`. No XEX decrypt, no LZX, **no booted emulator**

View File

@@ -19,12 +19,13 @@ The instrumented branch stops at the Stage 02 briefing under a storm of
behind `upstream/canary_experimental` (`a5a18f5c7`); our branch carries 50 of its behind `upstream/canary_experimental` (`a5a18f5c7`); our branch carries 50 of its
own. own.
* **`version.h` is never generated.** The build fails on * ~~**`version.h` is never generated.**~~ ✅ **FIXED — `CMakeLists.txt` now
`trace_writer.cc:17: fatal error: 'version.h' file not found`. Upstream's generates it at configure time**, calling `xenia-build.py`'s
`xenia-build.py` writes it from git HEAD; the container's `build-canary` `generate_version_h()` and falling back to a stub if that fails, so a
wrapper does not invoke it, and our tree only builds because a stale copy from CMake-direct build no longer depends on a stale copy in the build directory.
an old `sylpheed-re` build sits in the build directory. Regenerated by hand in It used to fail on `trace_writer.cc:17: fatal error: 'version.h' file not
exactly the format that script emits. found`, and the fix had to exist before the 2026-09-18 cleanup deleted the
build volume that was carrying that stale copy.
* **`build-canary` reports success on a failed build.** The harness recorded * **`build-canary` reports success on a failed build.** The harness recorded
"completed (exit code 0)" while ninja had stopped with `1 error generated`. "completed (exit code 0)" while ninja had stopped with `1 error generated`.
Only the missing binary gave it away. Only the missing binary gave it away.