Compare commits

..

2 Commits

Author SHA1 Message Date
MechaCat02
1fdbb5f197 fix(decoder): give the container the Canary it is supposed to run
Some checks failed
CI / Native — linux (pull_request) Failing after 1h1m35s
CI / WASM — Web (pull_request) Successful in 24m37s
CI / Formatting (pull_request) Successful in 26s
The database mount below was one of three ways the decoder could not reach its
own oracle. The other two are here.

`run-canary` never looked in `Checked/`. It tried `Release/` then `Debug/`, and
both of those exist on this box -- an Aug 28 binary and a Jul 19 one. They boot
the game perfectly well and carry NO `audit_61` branch probe, so a probe run
against either returns zero hits that read as a finding about the game rather
than as a stale binary. Configuration is now the outer loop and location the
inner one, so a `Checked` build anywhere beats a `Release` build anywhere;
`$XENIA_BIN` still overrides everything. Measured here: `Checked` has
`audit_61_branch_probe_pcs`, `Release` and `Debug` do not.

The launcher also now says which instrumentation is missing BEFORE the run,
because the alternative is reading an empty log afterwards and guessing.

`build-canary` built `$PROJECT_DIR/xenia-canary`, which does not exist in this
container -- the source is bind-mounted at `/canary` and the launcher already
exports `XENIA_SRC=/canary`. CONTAINER-NOTES has carried that defect since
2026-08-29 with a symlink workaround and a warning to remember to delete the
symlink afterwards. It now reads `$XENIA_SRC` first, so there is nothing to
remember. Its default configuration moves Release -> Checked to match what
`run-canary` picks; the old default spent a full build on a binary nothing ran.

Two documented blockers are refuted rather than deleted, since the sequence of
wrong readings is what makes the right one checkable: the CONTAINER-NOTES
symlink dance (the warm build volume it was configured against is gone too,
removed in the 2026-09-18 cleanup, so the next build configures cleanly against
`/canary`), and `upstream-baseline.md`'s "`version.h` is never generated" --
`CMakeLists.txt` generates it at configure time now, with a stub fallback.

`decoder-loop.md` claimed the oracle was at `Linux/Release/` and that the probe
was on two side branches; both were true when written and neither is now.

Verified: five pick_bin cases against the extracted function body, and `strings`
on all three real binaries.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 18:08:10 +02:00
8108958a77 fix(decoder): mount the disassembly database, which it had lost
`SYLPHEED_DB=/xenia-rs/sylpheed.db` named a path that no longer exists. The
xenia-rs repo was retired by the consolidation, its local clone was deleted, and
the `/xenia-rs` mount that served the database was removed in #61 because it
pointed at nothing. The env var stayed. So the decoder had NO database: `zq.py`
and `/sylph-dis` -- most of what a static-RE brief asks for -- could not run.

Mounted read-only at the container's repo root instead, which is where `zq.py`
looks when `$SYLPHEED_DB` is unset, so there is no variable left to drift out of
step with the mount. That drift is the whole bug: a path in an env var and a
path in a mount, maintained separately.

Read-only is deliberate. The host owns the file, DuckDB takes an exclusive lock
to write, and two agents plus the human sharing one database would corrupt it.
Regenerating means writing elsewhere and pointing $SYLPHEED_DB at it.

Missing-file cases now say so and print the command that builds one, rather than
starting an agent that discovers it mid-iteration.

Verified in the real agent image: DB mounted, no env var set,
`zq.py fn 0x824609C8` -> `Pak_FindEntryByName`, `zq.py classes` lists RTTI.

The brief's tooling table also claimed the old path, and said nothing about the
oracle binary; both corrected. It now records that the built Canary carries
RE-INPUT/RE-DRAW but NOT the audit_61 branch probe -- measured with `strings`
on both built binaries, zero hits; it is on two other branches (fork issue #1).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 17:14:06 +02:00
9 changed files with 102 additions and 61 deletions

View File

@@ -10,11 +10,22 @@
# AVAILABLE MEMORY as well as core count — a full-parallel build of this tree
# has OOM-killed the host outright.
#
# build-canary [Release|Debug] [extra cmake --build args]
# `Checked` is the default, and should stay it: it is what the host builds and
# what `run-canary` picks first, so the two agree by construction. Building
# `Release` here instead leaves a second binary that `run-canary` will not use
# -- hours of CPU for something nothing runs. `Checked` is optimised, with
# assertions left in.
#
# build-canary [Checked|Release|Debug] [extra cmake --build args]
set -euo pipefail
CONFIG="${1:-Release}"; shift || true
SRC="${PROJECT_DIR:-/work}/xenia-canary"
CONFIG="${1:-Checked}"; shift || true
# $XENIA_SRC FIRST. In this container the Canary source is bind-mounted at
# `/canary`, and `$PROJECT_DIR/xenia-canary` does not exist -- so the old default
# made this script unusable here, and the documented workaround was to symlink
# `/canary` into the repository and remember to delete it again (CONTAINER-NOTES
# §"The toolchain is real"). The launcher already exports the right path; read it.
SRC="${XENIA_SRC:-${PROJECT_DIR:-/work}/xenia-canary}"
BUILD="${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}"
JOBS="${SYLPH_JOBS:-2}"

View File

@@ -48,15 +48,26 @@ fi
PROJECT_DIR="${PROJECT_DIR:-/work}"
# ── Binary ───────────────────────────────────────────────────────────────────
# `Checked` FIRST. All three configurations can be built, but `Checked` is the
# one this project actually builds, so it is the one carrying our
# instrumentation; the `Release/` and `Debug/` binaries beside it are months-old
# leftovers that still run and still boot the game, which is exactly what makes
# them dangerous -- a probe run against one reports zero hits and reads as a
# finding about the game. Measured on this box 2026-09-21: `Checked` has
# `audit_61_branch_probe_pcs`, `Release` (Aug 28) and `Debug` (Jul 19) do not.
pick_bin() {
[ -n "${XENIA_BIN:-}" ] && { echo "$XENIA_BIN"; return; }
# Configuration is the OUTER loop, location the inner one: a `Checked` build
# anywhere beats a `Release` build anywhere. The other order picks a stale
# container `Release` over a fresh instrumented repo `Checked`, which is the
# exact mistake this is here to stop.
for cfg in Checked Release Debug; do
for c in \
"${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}/bin/Linux/Release/xenia_canary" \
"${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}/bin/Linux/Debug/xenia_canary" \
"$PROJECT_DIR/xenia-canary/build/bin/Linux/Release/xenia_canary" \
"$PROJECT_DIR/xenia-canary/build/bin/Linux/Debug/xenia_canary"; do
"${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}/bin/Linux/$cfg/xenia_canary" \
"$PROJECT_DIR/xenia-canary/build/bin/Linux/$cfg/xenia_canary"; do
[ -x "$c" ] && { echo "$c"; return; }
done
done
}
BIN="$(pick_bin)"
if [ -z "${BIN:-}" ]; then
@@ -64,6 +75,13 @@ if [ -z "${BIN:-}" ]; then
exit 1
fi
# Say what is in the binary before the run, not after reading an empty log.
# A missing probe is a build that predates it, never a quiet game.
for sym in audit_61_branch_probe_pcs RE-DRAW; do
grep -aqm1 -- "$sym" "$BIN" \
|| echo "run-canary: ⚠ $BIN has NO '$sym' -- it predates that instrumentation. Rebuild with: build-canary" >&2
done
# ── ISO ──────────────────────────────────────────────────────────────────────
ISO="${SYLPH_ISO:-}"
if [ -z "$ISO" ]; then

View File

@@ -123,7 +123,16 @@ docker_args() {
# emulator and scrape /dev/shm to get at it. An earlier belief that this
# file was STALE was tested and refuted -- it is current.
-v "${SYLPH_PE:-$PROJECT/Project Sylpheed - Arc of Deception (USA, Europe) (En,Ja).pe}:/image/sylpheed.pe:ro"
-e "SYLPHEED_DB=/xenia-rs/sylpheed.db"
# 🔴 THE DISASSEMBLY DATABASE. This used to be read from the xenia-rs clone
# (`SYLPHEED_DB=/xenia-rs/sylpheed.db`). That repo is retired, its directory
# is gone, and the mount that served it was removed -- so the variable named
# a path that did not exist and `zq.py` had nothing to open. The database now
# lives beside the repo as a build artefact; mounted at the container's repo
# root, which is where `zq.py` looks when `$SYLPHEED_DB` is unset.
#
# Read-only on purpose: the host owns it, DuckDB takes an exclusive lock to
# write, and two agents plus the human sharing one file would corrupt it.
# Regenerating means writing elsewhere and pointing `$SYLPHEED_DB` at it.
-e "SYLPHEED_PE=/image/sylpheed.pe"
-e "SYLPHEED_IMAGE_BASE=0x82000000"
# 🔴 EVIDENCE IS SHARED, NOT COMMITTED (issue #49). Each agent works in its
@@ -276,6 +285,20 @@ docker_args() {
echo "==> NOTE: no ISO at $_iso -- Canary has nothing to boot." >&2
fi
# The disassembly database. `zq.py` looks at the repo root when $SYLPHEED_DB
# is unset, so mounting it there needs no environment variable at all -- which
# is what the dead `/xenia-rs/sylpheed.db` variable taught: a path in an env
# var and a path in a mount drift apart silently.
_db="${SYLPH_DB:-$PROJECT/Sylpheed/sylpheed.db}"
if [ -f "$_db" ]; then
_out+=(-v "$_db:/work/sylpheed.db:ro")
else
echo "==> NOTE: no sylpheed.db at $_db -- /sylph-dis and zq.py cannot run." >&2
echo " build one: cargo run --release -p sylpheed-xexdb --bin sylph-xexdb -- \\" >&2
echo " dis \"\$SYLPH_ISO\" --db sylpheed.db --analyze sql --quiet" >&2
echo " then re-apply the RE names with tools/apply_re_symbols.sql" >&2
fi
# ── GPU ──
# Three distinct cases, and conflating them is how you end up believing you
# have hardware Vulkan while actually running llvmpipe:

View File

@@ -12,13 +12,20 @@ already went wrong once.
* **The toolchain is real.** `tools/re-capture/rebuild_canary.sh` exists because
the old box had no cmake/ninja/clang and only runtime sonames, so it hand-
relinked object files. **Do not use it here.** Use `build-canary`.
🔴 **But `build-canary` does not work in this container as it stands
**FIXED 2026-09-21 — `build-canary` now reads `$XENIA_SRC` first**, which the
launcher already sets to `/canary`, so there is no symlink to make and none to
forget. The two paragraphs below are kept because they are how the defect was
found and refuted, but **neither describes the script any more.**
🔴 ~~**`build-canary` does not work in this container as it stands
(2026-08-29).** It builds `${PROJECT_DIR:-/work}/xenia-canary`, which **does
not exist here** — the Canary source is at **`/canary`** (`$XENIA_SRC`). The
warm 235 MB tree at `/sylph-home/re/canary-build` is configured with
`CMAKE_HOME_DIRECTORY=/work/xenia-canary`, also missing, and its
`build-Release.ninja` carries **no per-file rules** — it wants to re-run CMake
first, which would fail on the absent source root.
first, which would fail on the absent source root.~~ The warm tree is gone too:
every `sylph-*` volume was removed in the 2026-09-18 cleanup, so the next build
configures from scratch against `/canary` and caches the right source root.
**The conclusion drawn from all that is REFUTED (2026-08-31).** The note went
on to say *"any Canary change is a full reconfigure against `/canary` plus a full
@@ -38,9 +45,10 @@ already went wrong once.
warm 235 MB tree is otherwise intact and the ninja re-run resolves its rules
from the symlink.
⚠️ **Remove the symlink when you are done.** `/work` is the repository, and
`xenia-canary` is not in `.gitignore`, so it shows up as untracked and can be
swept into a `git add -A`.
⚠️ ~~**Remove the symlink when you are done.**~~ No longer needed — but if you
ever make one by hand, note that `/work` is the repository and `xenia-canary`
is not in `.gitignore`, so it shows up as untracked and can be swept into a
`git add -A`.
* **numpy and Pillow are installed.** `entities2.py`, `flight_probe.py` and the
image oracles work. Their absence used to look like a logic bug.

View File

@@ -56,11 +56,28 @@ rather than left in a document you might not reach.
| path | what | env |
|---|---|---|
| `/image/sylpheed.pe` | the decompressed executable image | `SYLPHEED_PE` |
| `/xenia-rs/sylpheed.db` | a disassembly database, 586 MB | `SYLPHEED_DB` |
| `sylpheed.db` at the repo root | the disassembly database, 337 MB | — (`zq.py` finds it; `$SYLPHEED_DB` only to override) |
| `/canary/build/bin/Linux/Checked/xenia_canary` | the built oracle, inside the `/canary` mount | — |
| `/disc` | the extracted disc | `SYLPHEED_DISC` |
| `/iso/game.iso` | the retail ISO Canary boots | `SYLPH_ISO` |
| `/canary` | the Canary source, read-write | `XENIA_SRC` |
⚠️ **The database is read-only and is NOT in git** — it is a build artefact, and
`docs/re/captures/` is the same (issue #49). If either is missing the launcher
says so and prints the command that builds it. Query the database with
`python3 tools/zq.py …`, never by reading a disassembly dump.
⚠️ **Take the oracle from `Checked/`, not `Release/`.** `Checked` is what this
tree actually builds (optimised, with assertions); the `Release/` binary beside
it is a stale August build with no `audit_61` probe in it. `strings` on the two
is how that was found, and is how to check any binary before quoting a run from
it.
The `Checked` build carries all three instrumentations — `RE-INPUT`, `RE-DRAW`
and the `audit_61` guest-PC branch probe (fork PR #2 brought the probe onto
`sylpheed-re`; before it, the probe lived only on `phase-a-tracing`). Verified
by `strings` on the binary, not assumed.
**The `.pe` is a flat VA dump**: file offset = `VA - 0x82000000`. Reading
`0x820A1630` is `seek(0xA1630)`. No XEX decrypt, no LZX, **no booted emulator**
dumping guest memory works but makes the whole static corpus depend on a running

View File

@@ -19,12 +19,13 @@ The instrumented branch stops at the Stage 02 briefing under a storm of
behind `upstream/canary_experimental` (`a5a18f5c7`); our branch carries 50 of its
own.
* **`version.h` is never generated.** The build fails on
`trace_writer.cc:17: fatal error: 'version.h' file not found`. Upstream's
`xenia-build.py` writes it from git HEAD; the container's `build-canary`
wrapper does not invoke it, and our tree only builds because a stale copy from
an old `sylpheed-re` build sits in the build directory. Regenerated by hand in
exactly the format that script emits.
* ~~**`version.h` is never generated.**~~ ✅ **FIXED — `CMakeLists.txt` now
generates it at configure time**, calling `xenia-build.py`'s
`generate_version_h()` and falling back to a stub if that fails, so a
CMake-direct build no longer depends on a stale copy in the build directory.
It used to fail on `trace_writer.cc:17: fatal error: 'version.h' file not
found`, and the fix had to exist before the 2026-09-18 cleanup deleted the
build volume that was carrying that stale copy.
* **`build-canary` reports success on a failed build.** The harness recorded
"completed (exit code 0)" while ninja had stopped with `1 error generated`.
Only the missing binary gave it away.

View File

@@ -32,18 +32,7 @@ export ALSA_CONFIG_PATH="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/asound-nu
# Paths derive from where this script sits — see the note in run-canary-safe.sh.
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
WORKSPACE="$(cd "$HERE/../.." && pwd)"
# 🔴 THE INSTRUMENTED BINARY IS THE `Checked` ONE, and it is the only config this
# build tree can produce: `ninja -n bin/Linux/Release/xenia_canary` lists ZERO
# steps, so Release is not a target here and its binary still dates from
# 2026-08-28, before the audit_61 probe existed. Pointing at Release therefore
# runs an oracle with no probe and no way to refresh it.
#
# The old default named `xenia-canary-native/`, a directory that does not exist
# on this workspace at all -- the checkout is `xenia-canary/`.
#
# `Checked` is optimised WITH assertions, which is what you want from a
# behavioural reference: a bad state stops loudly instead of being sampled.
BIN_DEFAULT="$WORKSPACE/xenia-canary/build/bin/Linux/Checked/xenia_canary"
BIN_DEFAULT="$WORKSPACE/xenia-canary-native/build/bin/Linux/Release/xenia_canary"
BIN_EXE="${CANARY_BIN:-$BIN_DEFAULT}"
ISO="${SYLPHEED_ISO:-$WORKSPACE/Project Sylpheed - Arc of Deception (USA, Europe) (En,Ja).iso}"
[ -f "$ISO" ] || { echo "ABORT: no ISO at '$ISO' — set \$SYLPHEED_ISO"; exit 4; }

View File

@@ -24,18 +24,7 @@ set -u
# Paths derive from where this script sits — see the note in run-canary-safe.sh.
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
WORKSPACE="$(cd "$HERE/../.." && pwd)"
# 🔴 THE INSTRUMENTED BINARY IS THE `Checked` ONE, and it is the only config this
# build tree can produce: `ninja -n bin/Linux/Release/xenia_canary` lists ZERO
# steps, so Release is not a target here and its binary still dates from
# 2026-08-28, before the audit_61 probe existed. Pointing at Release therefore
# runs an oracle with no probe and no way to refresh it.
#
# The old default named `xenia-canary-native/`, a directory that does not exist
# on this workspace at all -- the checkout is `xenia-canary/`.
#
# `Checked` is optimised WITH assertions, which is what you want from a
# behavioural reference: a bad state stops loudly instead of being sampled.
BIN_DEFAULT="$WORKSPACE/xenia-canary/build/bin/Linux/Checked/xenia_canary"
BIN_DEFAULT="$WORKSPACE/xenia-canary-native/build/bin/Linux/Release/xenia_canary"
BIN_EXE="${CANARY_BIN:-$BIN_DEFAULT}"
ISO="${SYLPHEED_ISO:-$WORKSPACE/Project Sylpheed - Arc of Deception (USA, Europe) (En,Ja).iso}"
[ -f "$ISO" ] || { echo "ABORT: no ISO at '$ISO' — set \$SYLPHEED_ISO"; exit 4; }

View File

@@ -57,22 +57,7 @@ rm -f xenia.log
# Binary is overridable (default keeps the historical _i2d snapshot); extra
# cvars pass through via CANARY_EXTRA_ARGS (space-separated, values w/o spaces).
# ⚠️ THIS WINE BUILD HAS NO INSTRUMENTATION AND CANNOT BE REFRESHED HERE.
# Measured 2026-09-20: `xenia_canary_i2d.exe` does not exist at all, and the
# `xenia_canary.exe` that does (2026-06-19) has ZERO hits for both
# `audit_61_branch_probe_pcs` and `RE-INPUT`/`RE-DRAW`. Rebuilding it needs the
# clang-cl + xwin cross toolchain on the `cross-build-wine` branch.
#
# 🔴 FOR PROBE RUNS USE `run-canary-native-safe.sh` INSTEAD -- same hard Vulkan
# gate, same headless behaviour, and it runs the Checked native binary, which
# carries audit_61 and the RE-INPUT/RE-DRAW logging.
BIN_EXE="${CANARY_BIN:-xenia_canary_i2d.exe}"
if [ ! -f "$BIN/$BIN_EXE" ]; then
echo "ABORT: no Wine binary at $BIN/$BIN_EXE" >&2
echo " This build is stale and uninstrumented; use run-canary-native-safe.sh" >&2
echo " for anything needing audit_61 or the RE-* logging." >&2
exit 4
fi
args=(--log_level=3 --mute=true)
[ -n "$PROBES" ] && args+=("--audit_61_branch_probe_pcs=$PROBES")
if [ -n "${CANARY_EXTRA_ARGS:-}" ]; then