From 1fdbb5f1970a7449ad56b200c6af6513122d66ce Mon Sep 17 00:00:00 2001 From: MechaCat02 Date: Mon, 21 Sep 2026 18:08:10 +0200 Subject: [PATCH] fix(decoder): give the container the Canary it is supposed to run The database mount below was one of three ways the decoder could not reach its own oracle. The other two are here. `run-canary` never looked in `Checked/`. It tried `Release/` then `Debug/`, and both of those exist on this box -- an Aug 28 binary and a Jul 19 one. They boot the game perfectly well and carry NO `audit_61` branch probe, so a probe run against either returns zero hits that read as a finding about the game rather than as a stale binary. Configuration is now the outer loop and location the inner one, so a `Checked` build anywhere beats a `Release` build anywhere; `$XENIA_BIN` still overrides everything. Measured here: `Checked` has `audit_61_branch_probe_pcs`, `Release` and `Debug` do not. The launcher also now says which instrumentation is missing BEFORE the run, because the alternative is reading an empty log afterwards and guessing. `build-canary` built `$PROJECT_DIR/xenia-canary`, which does not exist in this container -- the source is bind-mounted at `/canary` and the launcher already exports `XENIA_SRC=/canary`. CONTAINER-NOTES has carried that defect since 2026-08-29 with a symlink workaround and a warning to remember to delete the symlink afterwards. It now reads `$XENIA_SRC` first, so there is nothing to remember. Its default configuration moves Release -> Checked to match what `run-canary` picks; the old default spent a full build on a binary nothing ran. Two documented blockers are refuted rather than deleted, since the sequence of wrong readings is what makes the right one checkable: the CONTAINER-NOTES symlink dance (the warm build volume it was configured against is gone too, removed in the 2026-09-18 cleanup, so the next build configures cleanly against `/canary`), and `upstream-baseline.md`'s "`version.h` is never generated" -- `CMakeLists.txt` generates it at configure time now, with a stub fallback. `decoder-loop.md` claimed the oracle was at `Linux/Release/` and that the probe was on two side branches; both were true when written and neither is now. Verified: five pick_bin cases against the extracted function body, and `strings` on all three real binaries. Co-Authored-By: Claude Opus 5 --- docker/decoder/bin/build-canary | 17 ++++++++++++++--- docker/decoder/bin/run-canary | 30 ++++++++++++++++++++++++------ docs/agents/CONTAINER-NOTES.md | 18 +++++++++++++----- docs/agents/decoder-loop.md | 17 +++++++++++------ docs/re/upstream-baseline.md | 13 +++++++------ 5 files changed, 69 insertions(+), 26 deletions(-) diff --git a/docker/decoder/bin/build-canary b/docker/decoder/bin/build-canary index eeb796c1..f467d425 100755 --- a/docker/decoder/bin/build-canary +++ b/docker/decoder/bin/build-canary @@ -10,11 +10,22 @@ # AVAILABLE MEMORY as well as core count — a full-parallel build of this tree # has OOM-killed the host outright. # -# build-canary [Release|Debug] [extra cmake --build args] +# `Checked` is the default, and should stay it: it is what the host builds and +# what `run-canary` picks first, so the two agree by construction. Building +# `Release` here instead leaves a second binary that `run-canary` will not use +# -- hours of CPU for something nothing runs. `Checked` is optimised, with +# assertions left in. +# +# build-canary [Checked|Release|Debug] [extra cmake --build args] set -euo pipefail -CONFIG="${1:-Release}"; shift || true -SRC="${PROJECT_DIR:-/work}/xenia-canary" +CONFIG="${1:-Checked}"; shift || true +# $XENIA_SRC FIRST. In this container the Canary source is bind-mounted at +# `/canary`, and `$PROJECT_DIR/xenia-canary` does not exist -- so the old default +# made this script unusable here, and the documented workaround was to symlink +# `/canary` into the repository and remember to delete it again (CONTAINER-NOTES +# §"The toolchain is real"). The launcher already exports the right path; read it. +SRC="${XENIA_SRC:-${PROJECT_DIR:-/work}/xenia-canary}" BUILD="${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}" JOBS="${SYLPH_JOBS:-2}" diff --git a/docker/decoder/bin/run-canary b/docker/decoder/bin/run-canary index 4de2e120..4f236b66 100755 --- a/docker/decoder/bin/run-canary +++ b/docker/decoder/bin/run-canary @@ -48,14 +48,25 @@ fi PROJECT_DIR="${PROJECT_DIR:-/work}" # ── Binary ─────────────────────────────────────────────────────────────────── +# `Checked` FIRST. All three configurations can be built, but `Checked` is the +# one this project actually builds, so it is the one carrying our +# instrumentation; the `Release/` and `Debug/` binaries beside it are months-old +# leftovers that still run and still boot the game, which is exactly what makes +# them dangerous -- a probe run against one reports zero hits and reads as a +# finding about the game. Measured on this box 2026-09-21: `Checked` has +# `audit_61_branch_probe_pcs`, `Release` (Aug 28) and `Debug` (Jul 19) do not. pick_bin() { [ -n "${XENIA_BIN:-}" ] && { echo "$XENIA_BIN"; return; } - for c in \ - "${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}/bin/Linux/Release/xenia_canary" \ - "${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}/bin/Linux/Debug/xenia_canary" \ - "$PROJECT_DIR/xenia-canary/build/bin/Linux/Release/xenia_canary" \ - "$PROJECT_DIR/xenia-canary/build/bin/Linux/Debug/xenia_canary"; do - [ -x "$c" ] && { echo "$c"; return; } + # Configuration is the OUTER loop, location the inner one: a `Checked` build + # anywhere beats a `Release` build anywhere. The other order picks a stale + # container `Release` over a fresh instrumented repo `Checked`, which is the + # exact mistake this is here to stop. + for cfg in Checked Release Debug; do + for c in \ + "${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}/bin/Linux/$cfg/xenia_canary" \ + "$PROJECT_DIR/xenia-canary/build/bin/Linux/$cfg/xenia_canary"; do + [ -x "$c" ] && { echo "$c"; return; } + done done } BIN="$(pick_bin)" @@ -64,6 +75,13 @@ if [ -z "${BIN:-}" ]; then exit 1 fi +# Say what is in the binary before the run, not after reading an empty log. +# A missing probe is a build that predates it, never a quiet game. +for sym in audit_61_branch_probe_pcs RE-DRAW; do + grep -aqm1 -- "$sym" "$BIN" \ + || echo "run-canary: ⚠ $BIN has NO '$sym' -- it predates that instrumentation. Rebuild with: build-canary" >&2 +done + # ── ISO ────────────────────────────────────────────────────────────────────── ISO="${SYLPH_ISO:-}" if [ -z "$ISO" ]; then diff --git a/docs/agents/CONTAINER-NOTES.md b/docs/agents/CONTAINER-NOTES.md index cf4779bc..3710b870 100644 --- a/docs/agents/CONTAINER-NOTES.md +++ b/docs/agents/CONTAINER-NOTES.md @@ -12,13 +12,20 @@ already went wrong once. * **The toolchain is real.** `tools/re-capture/rebuild_canary.sh` exists because the old box had no cmake/ninja/clang and only runtime sonames, so it hand- relinked object files. **Do not use it here.** Use `build-canary`. - 🔴 **But `build-canary` does not work in this container as it stands + ✅ **FIXED 2026-09-21 — `build-canary` now reads `$XENIA_SRC` first**, which the + launcher already sets to `/canary`, so there is no symlink to make and none to + forget. The two paragraphs below are kept because they are how the defect was + found and refuted, but **neither describes the script any more.** + + 🔴 ~~**`build-canary` does not work in this container as it stands (2026-08-29).** It builds `${PROJECT_DIR:-/work}/xenia-canary`, which **does not exist here** — the Canary source is at **`/canary`** (`$XENIA_SRC`). The warm 235 MB tree at `/sylph-home/re/canary-build` is configured with `CMAKE_HOME_DIRECTORY=/work/xenia-canary`, also missing, and its `build-Release.ninja` carries **no per-file rules** — it wants to re-run CMake - first, which would fail on the absent source root. + first, which would fail on the absent source root.~~ The warm tree is gone too: + every `sylph-*` volume was removed in the 2026-09-18 cleanup, so the next build + configures from scratch against `/canary` and caches the right source root. ✅ **The conclusion drawn from all that is REFUTED (2026-08-31).** The note went on to say *"any Canary change is a full reconfigure against `/canary` plus a full @@ -38,9 +45,10 @@ already went wrong once. warm 235 MB tree is otherwise intact and the ninja re-run resolves its rules from the symlink. - ⚠️ **Remove the symlink when you are done.** `/work` is the repository, and - `xenia-canary` is not in `.gitignore`, so it shows up as untracked and can be - swept into a `git add -A`. + ⚠️ ~~**Remove the symlink when you are done.**~~ No longer needed — but if you + ever make one by hand, note that `/work` is the repository and `xenia-canary` + is not in `.gitignore`, so it shows up as untracked and can be swept into a + `git add -A`. * **numpy and Pillow are installed.** `entities2.py`, `flight_probe.py` and the image oracles work. Their absence used to look like a logic bug. diff --git a/docs/agents/decoder-loop.md b/docs/agents/decoder-loop.md index 50838b2b..5c9cca0c 100644 --- a/docs/agents/decoder-loop.md +++ b/docs/agents/decoder-loop.md @@ -57,7 +57,7 @@ rather than left in a document you might not reach. |---|---|---| | `/image/sylpheed.pe` | the decompressed executable image | `SYLPHEED_PE` | | `sylpheed.db` at the repo root | the disassembly database, 337 MB | — (`zq.py` finds it; `$SYLPHEED_DB` only to override) | -| `/canary/build/bin/Linux/Release/xenia_canary` | the built oracle, inside the `/canary` mount | — | +| `/canary/build/bin/Linux/Checked/xenia_canary` | the built oracle, inside the `/canary` mount | — | | `/disc` | the extracted disc | `SYLPHEED_DISC` | | `/iso/game.iso` | the retail ISO Canary boots | `SYLPH_ISO` | | `/canary` | the Canary source, read-write | `XENIA_SRC` | @@ -67,11 +67,16 @@ rather than left in a document you might not reach. says so and prints the command that builds it. Query the database with `python3 tools/zq.py …`, never by reading a disassembly dump. -⚠️ **The built Canary carries the `RE-INPUT`/`RE-DRAW` instrumentation but NOT -the `audit_61` guest-PC branch probe** — that lives only on -`auto/canary-instrumentation-snapshot-2026-07-28` and `phase-a-tracing` -(fork issue #1), so a probe run needs those two commits on `sylpheed-re` and a -rebuild first. Verified by `strings` on the binary, not assumed. +⚠️ **Take the oracle from `Checked/`, not `Release/`.** `Checked` is what this +tree actually builds (optimised, with assertions); the `Release/` binary beside +it is a stale August build with no `audit_61` probe in it. `strings` on the two +is how that was found, and is how to check any binary before quoting a run from +it. + +The `Checked` build carries all three instrumentations — `RE-INPUT`, `RE-DRAW` +and the `audit_61` guest-PC branch probe (fork PR #2 brought the probe onto +`sylpheed-re`; before it, the probe lived only on `phase-a-tracing`). Verified +by `strings` on the binary, not assumed. **The `.pe` is a flat VA dump**: file offset = `VA - 0x82000000`. Reading `0x820A1630` is `seek(0xA1630)`. No XEX decrypt, no LZX, **no booted emulator** — diff --git a/docs/re/upstream-baseline.md b/docs/re/upstream-baseline.md index 1354cd99..d01c9229 100644 --- a/docs/re/upstream-baseline.md +++ b/docs/re/upstream-baseline.md @@ -19,12 +19,13 @@ The instrumented branch stops at the Stage 02 briefing under a storm of behind `upstream/canary_experimental` (`a5a18f5c7`); our branch carries 50 of its own. -* **`version.h` is never generated.** The build fails on - `trace_writer.cc:17: fatal error: 'version.h' file not found`. Upstream's - `xenia-build.py` writes it from git HEAD; the container's `build-canary` - wrapper does not invoke it, and our tree only builds because a stale copy from - an old `sylpheed-re` build sits in the build directory. Regenerated by hand in - exactly the format that script emits. +* ~~**`version.h` is never generated.**~~ ✅ **FIXED — `CMakeLists.txt` now + generates it at configure time**, calling `xenia-build.py`'s + `generate_version_h()` and falling back to a stub if that fails, so a + CMake-direct build no longer depends on a stale copy in the build directory. + It used to fail on `trace_writer.cc:17: fatal error: 'version.h' file not + found`, and the fix had to exist before the 2026-09-18 cleanup deleted the + build volume that was carrying that stale copy. * **`build-canary` reports success on a failed build.** The harness recorded "completed (exit code 0)" while ninja had stopped with `1 error generated`. Only the missing binary gave it away.