fix(decoder): give the container the Canary it is supposed to run
The database mount below was one of three ways the decoder could not reach its own oracle. The other two are here. `run-canary` never looked in `Checked/`. It tried `Release/` then `Debug/`, and both of those exist on this box -- an Aug 28 binary and a Jul 19 one. They boot the game perfectly well and carry NO `audit_61` branch probe, so a probe run against either returns zero hits that read as a finding about the game rather than as a stale binary. Configuration is now the outer loop and location the inner one, so a `Checked` build anywhere beats a `Release` build anywhere; `$XENIA_BIN` still overrides everything. Measured here: `Checked` has `audit_61_branch_probe_pcs`, `Release` and `Debug` do not. The launcher also now says which instrumentation is missing BEFORE the run, because the alternative is reading an empty log afterwards and guessing. `build-canary` built `$PROJECT_DIR/xenia-canary`, which does not exist in this container -- the source is bind-mounted at `/canary` and the launcher already exports `XENIA_SRC=/canary`. CONTAINER-NOTES has carried that defect since 2026-08-29 with a symlink workaround and a warning to remember to delete the symlink afterwards. It now reads `$XENIA_SRC` first, so there is nothing to remember. Its default configuration moves Release -> Checked to match what `run-canary` picks; the old default spent a full build on a binary nothing ran. Two documented blockers are refuted rather than deleted, since the sequence of wrong readings is what makes the right one checkable: the CONTAINER-NOTES symlink dance (the warm build volume it was configured against is gone too, removed in the 2026-09-18 cleanup, so the next build configures cleanly against `/canary`), and `upstream-baseline.md`'s "`version.h` is never generated" -- `CMakeLists.txt` generates it at configure time now, with a stub fallback. `decoder-loop.md` claimed the oracle was at `Linux/Release/` and that the probe was on two side branches; both were true when written and neither is now. Verified: five pick_bin cases against the extracted function body, and `strings` on all three real binaries. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -10,11 +10,22 @@
|
|||||||
# AVAILABLE MEMORY as well as core count — a full-parallel build of this tree
|
# AVAILABLE MEMORY as well as core count — a full-parallel build of this tree
|
||||||
# has OOM-killed the host outright.
|
# has OOM-killed the host outright.
|
||||||
#
|
#
|
||||||
# build-canary [Release|Debug] [extra cmake --build args]
|
# `Checked` is the default, and should stay it: it is what the host builds and
|
||||||
|
# what `run-canary` picks first, so the two agree by construction. Building
|
||||||
|
# `Release` here instead leaves a second binary that `run-canary` will not use
|
||||||
|
# -- hours of CPU for something nothing runs. `Checked` is optimised, with
|
||||||
|
# assertions left in.
|
||||||
|
#
|
||||||
|
# build-canary [Checked|Release|Debug] [extra cmake --build args]
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
CONFIG="${1:-Release}"; shift || true
|
CONFIG="${1:-Checked}"; shift || true
|
||||||
SRC="${PROJECT_DIR:-/work}/xenia-canary"
|
# $XENIA_SRC FIRST. In this container the Canary source is bind-mounted at
|
||||||
|
# `/canary`, and `$PROJECT_DIR/xenia-canary` does not exist -- so the old default
|
||||||
|
# made this script unusable here, and the documented workaround was to symlink
|
||||||
|
# `/canary` into the repository and remember to delete it again (CONTAINER-NOTES
|
||||||
|
# §"The toolchain is real"). The launcher already exports the right path; read it.
|
||||||
|
SRC="${XENIA_SRC:-${PROJECT_DIR:-/work}/xenia-canary}"
|
||||||
BUILD="${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}"
|
BUILD="${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}"
|
||||||
JOBS="${SYLPH_JOBS:-2}"
|
JOBS="${SYLPH_JOBS:-2}"
|
||||||
|
|
||||||
|
|||||||
@@ -48,14 +48,25 @@ fi
|
|||||||
PROJECT_DIR="${PROJECT_DIR:-/work}"
|
PROJECT_DIR="${PROJECT_DIR:-/work}"
|
||||||
|
|
||||||
# ── Binary ───────────────────────────────────────────────────────────────────
|
# ── Binary ───────────────────────────────────────────────────────────────────
|
||||||
|
# `Checked` FIRST. All three configurations can be built, but `Checked` is the
|
||||||
|
# one this project actually builds, so it is the one carrying our
|
||||||
|
# instrumentation; the `Release/` and `Debug/` binaries beside it are months-old
|
||||||
|
# leftovers that still run and still boot the game, which is exactly what makes
|
||||||
|
# them dangerous -- a probe run against one reports zero hits and reads as a
|
||||||
|
# finding about the game. Measured on this box 2026-09-21: `Checked` has
|
||||||
|
# `audit_61_branch_probe_pcs`, `Release` (Aug 28) and `Debug` (Jul 19) do not.
|
||||||
pick_bin() {
|
pick_bin() {
|
||||||
[ -n "${XENIA_BIN:-}" ] && { echo "$XENIA_BIN"; return; }
|
[ -n "${XENIA_BIN:-}" ] && { echo "$XENIA_BIN"; return; }
|
||||||
for c in \
|
# Configuration is the OUTER loop, location the inner one: a `Checked` build
|
||||||
"${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}/bin/Linux/Release/xenia_canary" \
|
# anywhere beats a `Release` build anywhere. The other order picks a stale
|
||||||
"${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}/bin/Linux/Debug/xenia_canary" \
|
# container `Release` over a fresh instrumented repo `Checked`, which is the
|
||||||
"$PROJECT_DIR/xenia-canary/build/bin/Linux/Release/xenia_canary" \
|
# exact mistake this is here to stop.
|
||||||
"$PROJECT_DIR/xenia-canary/build/bin/Linux/Debug/xenia_canary"; do
|
for cfg in Checked Release Debug; do
|
||||||
[ -x "$c" ] && { echo "$c"; return; }
|
for c in \
|
||||||
|
"${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}/bin/Linux/$cfg/xenia_canary" \
|
||||||
|
"$PROJECT_DIR/xenia-canary/build/bin/Linux/$cfg/xenia_canary"; do
|
||||||
|
[ -x "$c" ] && { echo "$c"; return; }
|
||||||
|
done
|
||||||
done
|
done
|
||||||
}
|
}
|
||||||
BIN="$(pick_bin)"
|
BIN="$(pick_bin)"
|
||||||
@@ -64,6 +75,13 @@ if [ -z "${BIN:-}" ]; then
|
|||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
# Say what is in the binary before the run, not after reading an empty log.
|
||||||
|
# A missing probe is a build that predates it, never a quiet game.
|
||||||
|
for sym in audit_61_branch_probe_pcs RE-DRAW; do
|
||||||
|
grep -aqm1 -- "$sym" "$BIN" \
|
||||||
|
|| echo "run-canary: ⚠ $BIN has NO '$sym' -- it predates that instrumentation. Rebuild with: build-canary" >&2
|
||||||
|
done
|
||||||
|
|
||||||
# ── ISO ──────────────────────────────────────────────────────────────────────
|
# ── ISO ──────────────────────────────────────────────────────────────────────
|
||||||
ISO="${SYLPH_ISO:-}"
|
ISO="${SYLPH_ISO:-}"
|
||||||
if [ -z "$ISO" ]; then
|
if [ -z "$ISO" ]; then
|
||||||
|
|||||||
@@ -12,13 +12,20 @@ already went wrong once.
|
|||||||
* **The toolchain is real.** `tools/re-capture/rebuild_canary.sh` exists because
|
* **The toolchain is real.** `tools/re-capture/rebuild_canary.sh` exists because
|
||||||
the old box had no cmake/ninja/clang and only runtime sonames, so it hand-
|
the old box had no cmake/ninja/clang and only runtime sonames, so it hand-
|
||||||
relinked object files. **Do not use it here.** Use `build-canary`.
|
relinked object files. **Do not use it here.** Use `build-canary`.
|
||||||
🔴 **But `build-canary` does not work in this container as it stands
|
✅ **FIXED 2026-09-21 — `build-canary` now reads `$XENIA_SRC` first**, which the
|
||||||
|
launcher already sets to `/canary`, so there is no symlink to make and none to
|
||||||
|
forget. The two paragraphs below are kept because they are how the defect was
|
||||||
|
found and refuted, but **neither describes the script any more.**
|
||||||
|
|
||||||
|
🔴 ~~**`build-canary` does not work in this container as it stands
|
||||||
(2026-08-29).** It builds `${PROJECT_DIR:-/work}/xenia-canary`, which **does
|
(2026-08-29).** It builds `${PROJECT_DIR:-/work}/xenia-canary`, which **does
|
||||||
not exist here** — the Canary source is at **`/canary`** (`$XENIA_SRC`). The
|
not exist here** — the Canary source is at **`/canary`** (`$XENIA_SRC`). The
|
||||||
warm 235 MB tree at `/sylph-home/re/canary-build` is configured with
|
warm 235 MB tree at `/sylph-home/re/canary-build` is configured with
|
||||||
`CMAKE_HOME_DIRECTORY=/work/xenia-canary`, also missing, and its
|
`CMAKE_HOME_DIRECTORY=/work/xenia-canary`, also missing, and its
|
||||||
`build-Release.ninja` carries **no per-file rules** — it wants to re-run CMake
|
`build-Release.ninja` carries **no per-file rules** — it wants to re-run CMake
|
||||||
first, which would fail on the absent source root.
|
first, which would fail on the absent source root.~~ The warm tree is gone too:
|
||||||
|
every `sylph-*` volume was removed in the 2026-09-18 cleanup, so the next build
|
||||||
|
configures from scratch against `/canary` and caches the right source root.
|
||||||
|
|
||||||
✅ **The conclusion drawn from all that is REFUTED (2026-08-31).** The note went
|
✅ **The conclusion drawn from all that is REFUTED (2026-08-31).** The note went
|
||||||
on to say *"any Canary change is a full reconfigure against `/canary` plus a full
|
on to say *"any Canary change is a full reconfigure against `/canary` plus a full
|
||||||
@@ -38,9 +45,10 @@ already went wrong once.
|
|||||||
warm 235 MB tree is otherwise intact and the ninja re-run resolves its rules
|
warm 235 MB tree is otherwise intact and the ninja re-run resolves its rules
|
||||||
from the symlink.
|
from the symlink.
|
||||||
|
|
||||||
⚠️ **Remove the symlink when you are done.** `/work` is the repository, and
|
⚠️ ~~**Remove the symlink when you are done.**~~ No longer needed — but if you
|
||||||
`xenia-canary` is not in `.gitignore`, so it shows up as untracked and can be
|
ever make one by hand, note that `/work` is the repository and `xenia-canary`
|
||||||
swept into a `git add -A`.
|
is not in `.gitignore`, so it shows up as untracked and can be swept into a
|
||||||
|
`git add -A`.
|
||||||
* **numpy and Pillow are installed.** `entities2.py`, `flight_probe.py` and the
|
* **numpy and Pillow are installed.** `entities2.py`, `flight_probe.py` and the
|
||||||
image oracles work. Their absence used to look like a logic bug.
|
image oracles work. Their absence used to look like a logic bug.
|
||||||
|
|
||||||
|
|||||||
@@ -57,7 +57,7 @@ rather than left in a document you might not reach.
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `/image/sylpheed.pe` | the decompressed executable image | `SYLPHEED_PE` |
|
| `/image/sylpheed.pe` | the decompressed executable image | `SYLPHEED_PE` |
|
||||||
| `sylpheed.db` at the repo root | the disassembly database, 337 MB | — (`zq.py` finds it; `$SYLPHEED_DB` only to override) |
|
| `sylpheed.db` at the repo root | the disassembly database, 337 MB | — (`zq.py` finds it; `$SYLPHEED_DB` only to override) |
|
||||||
| `/canary/build/bin/Linux/Release/xenia_canary` | the built oracle, inside the `/canary` mount | — |
|
| `/canary/build/bin/Linux/Checked/xenia_canary` | the built oracle, inside the `/canary` mount | — |
|
||||||
| `/disc` | the extracted disc | `SYLPHEED_DISC` |
|
| `/disc` | the extracted disc | `SYLPHEED_DISC` |
|
||||||
| `/iso/game.iso` | the retail ISO Canary boots | `SYLPH_ISO` |
|
| `/iso/game.iso` | the retail ISO Canary boots | `SYLPH_ISO` |
|
||||||
| `/canary` | the Canary source, read-write | `XENIA_SRC` |
|
| `/canary` | the Canary source, read-write | `XENIA_SRC` |
|
||||||
@@ -67,11 +67,16 @@ rather than left in a document you might not reach.
|
|||||||
says so and prints the command that builds it. Query the database with
|
says so and prints the command that builds it. Query the database with
|
||||||
`python3 tools/zq.py …`, never by reading a disassembly dump.
|
`python3 tools/zq.py …`, never by reading a disassembly dump.
|
||||||
|
|
||||||
⚠️ **The built Canary carries the `RE-INPUT`/`RE-DRAW` instrumentation but NOT
|
⚠️ **Take the oracle from `Checked/`, not `Release/`.** `Checked` is what this
|
||||||
the `audit_61` guest-PC branch probe** — that lives only on
|
tree actually builds (optimised, with assertions); the `Release/` binary beside
|
||||||
`auto/canary-instrumentation-snapshot-2026-07-28` and `phase-a-tracing`
|
it is a stale August build with no `audit_61` probe in it. `strings` on the two
|
||||||
(fork issue #1), so a probe run needs those two commits on `sylpheed-re` and a
|
is how that was found, and is how to check any binary before quoting a run from
|
||||||
rebuild first. Verified by `strings` on the binary, not assumed.
|
it.
|
||||||
|
|
||||||
|
The `Checked` build carries all three instrumentations — `RE-INPUT`, `RE-DRAW`
|
||||||
|
and the `audit_61` guest-PC branch probe (fork PR #2 brought the probe onto
|
||||||
|
`sylpheed-re`; before it, the probe lived only on `phase-a-tracing`). Verified
|
||||||
|
by `strings` on the binary, not assumed.
|
||||||
|
|
||||||
**The `.pe` is a flat VA dump**: file offset = `VA - 0x82000000`. Reading
|
**The `.pe` is a flat VA dump**: file offset = `VA - 0x82000000`. Reading
|
||||||
`0x820A1630` is `seek(0xA1630)`. No XEX decrypt, no LZX, **no booted emulator** —
|
`0x820A1630` is `seek(0xA1630)`. No XEX decrypt, no LZX, **no booted emulator** —
|
||||||
|
|||||||
@@ -19,12 +19,13 @@ The instrumented branch stops at the Stage 02 briefing under a storm of
|
|||||||
behind `upstream/canary_experimental` (`a5a18f5c7`); our branch carries 50 of its
|
behind `upstream/canary_experimental` (`a5a18f5c7`); our branch carries 50 of its
|
||||||
own.
|
own.
|
||||||
|
|
||||||
* **`version.h` is never generated.** The build fails on
|
* ~~**`version.h` is never generated.**~~ ✅ **FIXED — `CMakeLists.txt` now
|
||||||
`trace_writer.cc:17: fatal error: 'version.h' file not found`. Upstream's
|
generates it at configure time**, calling `xenia-build.py`'s
|
||||||
`xenia-build.py` writes it from git HEAD; the container's `build-canary`
|
`generate_version_h()` and falling back to a stub if that fails, so a
|
||||||
wrapper does not invoke it, and our tree only builds because a stale copy from
|
CMake-direct build no longer depends on a stale copy in the build directory.
|
||||||
an old `sylpheed-re` build sits in the build directory. Regenerated by hand in
|
It used to fail on `trace_writer.cc:17: fatal error: 'version.h' file not
|
||||||
exactly the format that script emits.
|
found`, and the fix had to exist before the 2026-09-18 cleanup deleted the
|
||||||
|
build volume that was carrying that stale copy.
|
||||||
* **`build-canary` reports success on a failed build.** The harness recorded
|
* **`build-canary` reports success on a failed build.** The harness recorded
|
||||||
"completed (exit code 0)" while ninja had stopped with `1 error generated`.
|
"completed (exit code 0)" while ninja had stopped with `1 error generated`.
|
||||||
Only the missing binary gave it away.
|
Only the missing binary gave it away.
|
||||||
|
|||||||
Reference in New Issue
Block a user