diff --git a/docker/agent/.dockerignore b/docker/agent/.dockerignore new file mode 100644 index 00000000..91ba5b68 --- /dev/null +++ b/docker/agent/.dockerignore @@ -0,0 +1,4 @@ +** +!bin +!bin/** +!entrypoint.sh diff --git a/docker/agent/AGENT.md b/docker/agent/AGENT.md new file mode 100644 index 00000000..695d6c75 --- /dev/null +++ b/docker/agent/AGENT.md @@ -0,0 +1,62 @@ +# Notes for an agent working inside this container + +Read this before starting a dynamic-RE run. Everything here is something that +already went wrong once. + +## The container fixes three old traps for you + +* **The display outlives the turn.** Xvfb and openbox are children of PID 1, not + of your shell. The old "Xvfb dies on its own every few minutes" note is gone β€” + you no longer have to wrap a whole session in one blocking foreground call to + keep it alive. +* **The toolchain is real.** `tools/re-capture/rebuild_canary.sh` exists because + the old box had no cmake/ninja/clang and only runtime sonames, so it hand- + relinked object files. **Do not use it here.** Use `build-canary`. +* **numpy and Pillow are installed.** `entities2.py`, `flight_probe.py` and the + image oracles work. Their absence used to look like a logic bug. + +## Method (the part that matters more than the tooling) + +* **Measure the oracle; never infer it.** A session with zero Canary runs is a + red flag. +* **Trace upstream to where data first goes wrong**, rather than patching the + symptom you can see. +* **Try to refute before believing.** Record demotions rather than editing them + away β€” `docs/re/README.md` has the βœ…/🟑/❔ convention, and a withdrawn result + is more useful than a quietly deleted one. +* **A probe that never performs the action will "prove" the action does not + exist.** The "targeting is automatic" conclusion came from a sweep that only + ever tapped once; target select is β’Ά pressed *twice*. +* **Do not poll faster than the guest updates** β€” it manufactures a clean curve + out of noise. `rate-curve-aliased-BAD.csv` is committed as the bad example. + +## Running the emulator + +```bash +run-canary # correct audio/pad/display flags baked in +pad.py tap A ; pad.py dpad down # scripted input (--hid=file, no uinput) +screenshot ~/shots/now.png # cropped to the GAME surface, not the window +python3 tools/re-capture/gmem.py find hex:820af844 400 +``` + +* **One emulator at a time.** `run-canary` enforces it with a lockfile. +* Boot is slow cold, ~25 s once the shader/code caches are warm β€” so a + launch-and-dump fits in a single call. +* **Screens: classify by whole-image statistics** (`screen_id.py`), not named + pixels. Named-pixel oracles are only valid while the game image sits at a + known place, and nothing errors when it moves. + +## Verifying your own work + +* Reborn's disc-gated tests **self-skip** without `SYLPHEED_DISC`. A green run + with it unset means almost nothing. `build-reborn test` wires it up for you. +* Prefer a headless self-verify over "it compiles": `sylpheed-cli mesh render`, + `screen render`, `save info` all produce checkable artifacts. +* A Bevy system-parameter conflict is invisible to the type checker and panics + at startup. If you touch viewer systems, *run the binary*, don't just build it. + +## Reporting + +State what you measured, what you assumed, and what you could not settle. If a +result is withdrawn, say so and keep the reasoning β€” that is the corpus's whole +convention, and the reason its numbers can be trusted. diff --git a/docker/agent/Dockerfile b/docker/agent/Dockerfile new file mode 100644 index 00000000..25ac0d9a --- /dev/null +++ b/docker/agent/Dockerfile @@ -0,0 +1,142 @@ +# Autonomous RE agent container for Project Sylpheed. +# +# Builds and runs BOTH halves of the project β€” Xenia Canary (C++/CMake/Ninja) +# as the behaviour oracle, and Sylpheed Reborn (Rust/Bevy) as the port β€” plus +# the dynamic-RE toolkit that drives the emulator and reads its guest memory. +# +# Three things here exist because their absence cost the previous agent real +# hours, and they are load-bearing rather than nice-to-have: +# +# 1. A REAL toolchain. The old box shipped runtime sonames only (libgtk-3.so.0 +# but no libgtk-3.so), no cmake/ninja/clang and no libstdc++fs, so a full +# build was impossible and `tools/re-capture/rebuild_canary.sh` had to +# hand-relink object files. With -dev packages present that script is +# obsolete; use `build-canary`. +# 2. numpy and Pillow. Their absence silently disabled entities2.py, +# flight_probe.py and every image oracle, and the failure looked like a +# logic bug rather than a missing package. +# 3. A display that outlives the turn. Xvfb kept dying "on its own every few +# minutes"; it was being reaped because nothing owned it. Here it is a +# child of PID 1 and lives exactly as long as the container. +# +# Clang is pinned to 19 to match the host that produced the checked-in build +# caches (Ubuntu clang 19.1.1). + +FROM ubuntu:24.04 + +ENV DEBIAN_FRONTEND=noninteractive \ + LANG=C.UTF-8 \ + TZ=Etc/UTC + +# ── System packages ────────────────────────────────────────────────────────── +RUN apt-get update && apt-get install -y --no-install-recommends \ + # toolchain + build-essential cmake ninja-build pkg-config git curl wget ca-certificates \ + clang-19 lld-19 llvm-19 libc++-19-dev libc++abi-19-dev \ + # Canary: GTK window, SDL input/audio, Vulkan, compression + libgtk-3-dev libsdl2-dev liblz4-dev libvulkan-dev libx11-xcb-dev \ + libxcb1-dev libxrandr-dev libssl-dev libfuse2t64 \ + # Shader toolchain: the GPU build shells out to `glslangValidator` and the + # SPIR-V tools to compile xenia's own shaders. Missing them does not fail + # configure β€” it fails ~500 objects in, as a Python FileNotFoundError. + glslang-tools spirv-tools spirv-headers \ + # Vulkan runtime β€” lavapipe (software) plus the real ICDs for /dev/dri + mesa-vulkan-drivers vulkan-tools libvulkan1 libgl1-mesa-dri libglx-mesa0 \ + # Reborn / Bevy: audio, input, windowing + libasound2-dev libudev-dev libwayland-dev libxkbcommon-dev \ + libx11-dev libxi-dev libxcursor-dev libxinerama-dev libxext-dev \ + # headless display + window manager + the screenshot path + xvfb x11-utils x11-xserver-utils openbox xdotool imagemagick ffmpeg \ + # dynamic RE + python3 python3-numpy python3-pil python3-pip \ + gdb strace ltrace binutils file xxd ripgrep jq unzip zip p7zip-full \ + procps psmisc lsof less nano tini sudo \ + && rm -rf /var/lib/apt/lists/* + +# Pin the unversioned tool names to 19 so CMake, and anything that shells out to +# `clang`, agree with what the caches were built by. +RUN for t in clang clang++ lld ld.lld llvm-ar llvm-ranlib llvm-nm clang-cpp; do \ + src="/usr/bin/${t}-19"; \ + [ -e "$src" ] && update-alternatives --install "/usr/bin/${t}" "$t" "$src" 200 || true; \ + done + +# duckdb reads the static-analysis database (sylpheed.db); it is not packaged. +# PEP 668 marks the system env externally-managed, and this image has no other +# Python consumer to protect, so installing into it is the honest simple option. +RUN pip3 install --no-cache-dir --break-system-packages duckdb + +# ── Node + Claude Code ─────────────────────────────────────────────────────── +RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \ + && apt-get install -y --no-install-recommends nodejs \ + && npm install -g @anthropic-ai/claude-code \ + && npm cache clean --force \ + && rm -rf /var/lib/apt/lists/* + +# ── The agent user ─────────────────────────────────────────────────────────── +# NOT root, and not negotiable: Claude Code refuses --dangerously-skip-permissions +# when it has root privileges. uid/gid 1000 matches the host account so files +# written into the bind-mounted repos keep the right ownership. +ARG AGENT_UID=1000 +ARG AGENT_GID=1000 +# Ubuntu 24.04 ships its own `ubuntu` account at uid/gid 1000, so the common +# case β€” matching a host user who is also 1000 β€” collides with it. Remove the +# stock account first; nothing in this image uses it. +RUN if getent passwd "${AGENT_UID}" >/dev/null; then \ + userdel -r "$(getent passwd "${AGENT_UID}" | cut -d: -f1)" 2>/dev/null || true; \ + fi; \ + if getent group "${AGENT_GID}" >/dev/null; then \ + groupdel "$(getent group "${AGENT_GID}" | cut -d: -f1)" 2>/dev/null || true; \ + fi; \ + groupadd -g "${AGENT_GID}" agent \ + && useradd -m -u "${AGENT_UID}" -g "${AGENT_GID}" -s /bin/bash -d /sylph-home/re agent \ + && mkdir -p /sylph-home/re /work \ + && chown -R "${AGENT_UID}:${AGENT_GID}" /sylph-home \ + && echo 'agent ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/agent + +COPY bin/ /usr/local/bin/ +COPY entrypoint.sh /usr/local/bin/entrypoint.sh +RUN chmod +x /usr/local/bin/* /usr/local/bin/entrypoint.sh + +USER agent +WORKDIR /work + +# ── Rust ───────────────────────────────────────────────────────────────────── +# CARGO_TARGET_DIR deliberately points OUTSIDE the bind-mounted repo: the host +# also builds Reborn, and sharing target/ makes the two invalidate each other's +# incremental state on every switch. +ENV RUSTUP_HOME=/sylph-home/re/.rustup \ + CARGO_HOME=/sylph-home/re/.cargo \ + CARGO_TARGET_DIR=/sylph-home/re/target-container \ + PATH=/sylph-home/re/.cargo/bin:/usr/local/bin:/usr/bin:/bin +RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \ + | sh -s -- -y --default-toolchain stable --profile minimal \ + --component clippy --component rustfmt \ + && rustup target add wasm32-unknown-unknown + +# trunk serves the Reborn viewer's wasm build; the release binary avoids a +# ten-minute `cargo install`. +RUN curl -fsSL https://github.com/trunk-rs/trunk/releases/download/v0.21.4/trunk-x86_64-unknown-linux-gnu.tar.gz \ + | tar -xz -C /sylph-home/re/.cargo/bin trunk + +# Create the volume mount points HERE, owned by `agent`. Docker seeds an empty +# named volume from whatever the image has at that path β€” including ownership β€” +# but if the path does not exist it creates a root-owned directory instead, and +# the first write fails with something as unhelpful as +# "CMake Error: Unable to (re)create the private pkgRedirects directory". +RUN mkdir -p /sylph-home/re/target-container /sylph-home/re/canary-build /sylph-home/re/.claude + +# ── Runtime environment ────────────────────────────────────────────────────── +# DISPLAY :98 and HOME /sylph-home/re are what tools/re-capture/*.sh already +# assume; keeping them means the existing toolkit runs unmodified. +ENV HOME=/sylph-home/re \ + DISPLAY=:98 \ + SCREEN_GEOMETRY=1280x720x24 \ + PROJECT_DIR=/work \ + XENIA_PAD_FILE=/tmp/xenia_pad.txt \ + XENIA_BUILD_DIR=/sylph-home/re/canary-build \ + SDL_AUDIODRIVER=dummy \ + LIBGL_ALWAYS_SOFTWARE=1 \ + PATH=/work/Syplheed-Reborn/tools/re-capture/bin:/work/Syplheed-Reborn/tools/re-capture:/sylph-home/re/.cargo/bin:/usr/local/bin:/usr/bin:/bin + +ENTRYPOINT ["/usr/bin/tini", "--", "/usr/local/bin/entrypoint.sh"] +CMD ["bash"] diff --git a/docker/agent/README.md b/docker/agent/README.md new file mode 100644 index 00000000..21c04cfd --- /dev/null +++ b/docker/agent/README.md @@ -0,0 +1,178 @@ +# The RE agent container + +A container an autonomous Claude Code agent can be turned loose in: it builds +and runs **both** halves of the project β€” Xenia Canary as the behaviour oracle +and Sylpheed Reborn as the port β€” and carries the dynamic-RE toolkit that drives +the emulator, reads its guest memory and photographs its screen. + +```bash +./sylph-agent build # build the image +./sylph-agent doctor # prove it can do the four things it exists for +./sylph-agent shell # poke around +./sylph-agent agent # Claude Code, --dangerously-skip-permissions +``` + +## The resource cap + +The container gets **half the machine**, computed at launch so it stays half on +any box: + +| | how | +|---|---| +| CPU | `--cpus $(nproc)/2` | +| memory | `--memory` = half `MemTotal`, **`--memory-swap` equal to it** | +| `/dev/shm` | a third of the memory cap, min 1 GiB, mounted **exec** | +| build jobs | derived *inside* the container from **available memory**, not cores | + +Two of those deserve a word. + +**No swap headroom.** `--memory-swap` is set equal to `--memory`, so the +container cannot swap. That is deliberate: a swapping build thrashes the whole +host, which is precisely the failure the cap exists to prevent. A build that +would have swapped gets OOM-killed inside the container instead, and the host +stays usable. + +**`/dev/shm` is not incidental.** Xenia backs the guest address space with +`/dev/shm/xenia_memory_*`, and the whole live-memory toolkit (`gmem.py`, +`gpoke.py`, `mission_state.py`) reads it from there. Docker's default is 64 MiB, +which is far too small for a 512 MiB console β€” and it fails as an obscure mmap +error rather than an out-of-space message. + +**Build parallelism is memory-bound.** A full-parallel build of this tree has +OOM-killed the host outright, so the entrypoint computes jobs from *available +memory* (β‰ˆ1.5 GiB per C++ TU) and exports it as `SYLPH_JOBS`, `CARGO_BUILD_JOBS` +and `CMAKE_BUILD_PARALLEL_LEVEL`. Override with `SYLPH_CPUS` / `SYLPH_MEM_GB`. + +## Inside + +| command | what | +|---|---| +| `build-canary [Release\|Debug]` | configure + build Canary | +| `build-reborn [build\|test\|ci]` | build/test Reborn, with the disc env wired up | +| `run-canary [flags…]` | launch Canary with the settings this title needs | +| `screenshot [out.png]` | grab the display | +| `sylph-doctor` | self-check | +| `tools/re-capture/*` | the RE toolkit, already on `PATH` | + +Layout: project at `/work`, `HOME=/sylph-home/re`, `DISPLAY=:98` β€” the values +`tools/re-capture/*.sh` already assume, so the existing toolkit runs unmodified. + +Build outputs live **outside** the bind mount (`CARGO_TARGET_DIR`, +`XENIA_BUILD_DIR`, both named Docker volumes). The host builds the same trees, +and sharing `target/` or `build/` makes host and container reconfigure and +relink everything the other just did. + +## Screenshots + +Two layers, and the distinction matters: + +* `/usr/local/bin/screenshot` β€” raw full-root PNG (ImageMagick, falling back to + ffmpeg's x11grab, then xwd). +* `tools/re-capture/bin/screenshot` β€” **first on `PATH`**, wraps the above and + crops to the *game surface*. + +The crop is not cosmetic. Xenia's window is a GTK window whose menu bar pushes +the 1280Γ—720 game image down ~25 px, and every pixel oracle in the toolkit was +measured against the bare game image. When that offset was unaccounted for, one +run sat 300 s in front of a plainly visible MAIN MENU reporting "no main menu". +The wrapper derives the offset from the window's own height rather than a +per-display constant. + +For finding a screen at all, prefer `screen_id.py`, which classifies by +whole-image statistics instead of named pixels. + +## Vulkan + +`mesa-vulkan-drivers` + `vulkan-tools` are installed, so Vulkan works with **no +host GPU** via lavapipe (software β€” correct, slow). When the host has +`/dev/dri`, the launcher passes the device through and adds the host's `render` +and `video` GIDs, and the entrypoint uses the hardware ICD. Force software with +`SYLPH_VULKAN=sw`. `vulkaninfo --summary` (or `sylph-doctor`) says which you got β€” and the +entrypoint reports the device that **actually enumerated**, not the one it asked +for, because "I passed `/dev/dri`" and "I have hardware Vulkan" are different +claims. + +⚠️ **On an NVIDIA host, `/dev/dri` alone does nothing** β€” Mesa cannot drive an +NVIDIA card and the proprietary userspace lives outside the image. You need the +NVIDIA Container Toolkit; the launcher detects the situation and tells you the +three commands. Until then Canary runs on lavapipe, which is correct but has not +been observed to reach a rendered frame in a couple of minutes β€” everything +*else* (guest memory, the JIT, the live-memory toolkit) works fine on it. + +## Input, and why there is no virtual gamepad + +`run-canary` passes `--hid=file --pad_file=/tmp/xenia_pad.txt`; drive it with +`tools/re-capture/pad.py`. There is deliberately **no `/dev/uinput`**: input +devices are not namespaced, so a virtual pad created in a container registers +with the *host's* input stack and every scripted press leaks onto the user's +desktop. + +The trap that wasted a session: 360 menus poll `XamInputGetKeystrokeEx`, not +`GetState` β€” with `GetKeystroke` stubbed the pad looks completely dead on a +title screen while its own log shows the press arriving. + +## Settings that are requirements, not preferences + +`run-canary` bakes these in; changing them will cost you an afternoon. + +* **`--apu=sdl` with `SDL_AUDIODRIVER=dummy`** β€” and **no `--audio` flag**, + which is not a cvar here (see below). There is no PulseAudio, so `--apu=nop` + looks like the safe muted choice. It is not: the log fills with + `CreateDriver failed for index=0`, the guest never gets past the intro movie, + and the window stays black for 8+ minutes. SDL against a dummy device is + silent *and* lets the title advance. +* **One emulator at a time**, enforced with a lockfile. Two at once perturbs + both and the box. +* Stale `/dev/shm/xenia_memory_*` from a killed run is removed at launch β€” + otherwise the memory readers find two candidates and pick the dead one. + +## Claude Code + +Runs as an unprivileged `agent` user, because `--dangerously-skip-permissions` +is **refused under root**. `./sylph-agent agent` sets `SYLPH_AUTONOMOUS=1` and +the entrypoint adds the flag. + +Auth comes from the host `~/.claude`, bind-mounted read-write (token refresh +needs to write). **That directory also holds your memory and project state**, so +the container agent and you share it. Point `SYLPH_CLAUDE_HOME` at a separate +directory to isolate it, or set `ANTHROPIC_API_KEY` instead. + +## Host prerequisites + +* **A Vulkan SDK** (LunarG), for *building* only. Canary's shader step calls + `spirv-opt --canonicalize-ids`, which Ubuntu's packaged SPIRV-Tools (v2025.1) + does not have β€” the build then dies ~500 objects in, and the error you see is + a Python `TypeError`, not the real message. The launcher mounts the host's SDK + read-only at its own path and sets `VULKAN_SDK`; that also guarantees the + container produces byte-identical shaders to a host build. +* **`nvidia-container-toolkit`**, for hardware Vulkan β€” see below. + +## Things that will waste your afternoon + +Each of these was hit while bringing this container up. + +* **An unknown xenia flag hangs; it does not error.** `ParseLaunchArguments` + calls `ShowSimpleMessageBox` *before logging is initialised*, and that SDL + dialog blocks on `XIfEvent` forever with nobody to click it. The symptom is a + 10Γ—10 window, a completely empty log and no guest memory β€” which reads like a + hang deep in the emulator. `--audio` is **not** a cvar in this tree despite + appearing in the RE notes; `--apu=sdl` is the real one. If Canary appears to + hang at startup, suspect a typo'd flag first. +* **`/dev/shm` must be `exec`.** Docker mounts it `noexec`, and xenia maps its + JIT code cache out of a shm file. With `noexec` it dies at startup with + "Unable to allocate code cache generated code storage / Cannot initalize + processor", which reads like an address-space clash. The launcher uses + `--tmpfs /dev/shm:rw,exec,…` rather than `--shm-size`. +* **gdb needs root inside the container.** `--cap-add SYS_PTRACE` is passed, but + the *host's* `kernel.yama.ptrace_scope=1` still blocks attaching to a + non-descendant. Use `sudo gdb -p ` (passwordless), or launch the target + under gdb so it is a child. +* **Named volumes need their mount points to exist in the image**, or Docker + creates them root-owned and the first write fails obscurely. + +## Known limitation + +`build-reborn ci` runs the native legs only. `just ci`'s wasm check does not +build, for a pre-existing reason unrelated to any change under test: the +workspace pins `tokio = { features = ["full"] }`, which pulls `mio`, which +refuses to compile for `wasm32-unknown-unknown`. diff --git a/docker/agent/bin/build-canary b/docker/agent/bin/build-canary new file mode 100755 index 00000000..eeb796c1 --- /dev/null +++ b/docker/agent/bin/build-canary @@ -0,0 +1,52 @@ +#!/usr/bin/env bash +# Configure + build Xenia Canary inside the container. +# +# The build directory is $XENIA_BUILD_DIR (outside the bind-mounted repo) on +# purpose. The host builds this same tree, and CMake caches an absolute compiler +# path and a generator: sharing repo/build between host and container makes each +# one reconfigure and relink everything the other just did. +# +# Parallelism comes from $SYLPH_JOBS, which the entrypoint derives from +# AVAILABLE MEMORY as well as core count β€” a full-parallel build of this tree +# has OOM-killed the host outright. +# +# build-canary [Release|Debug] [extra cmake --build args] +set -euo pipefail + +CONFIG="${1:-Release}"; shift || true +SRC="${PROJECT_DIR:-/work}/xenia-canary" +BUILD="${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}" +JOBS="${SYLPH_JOBS:-2}" + +[ -d "$SRC" ] || { echo "build-canary: no source at $SRC" >&2; exit 1; } + +# Submodules: this tree has drifted before, and a checkout that changes a +# gitlink fails silently into a half-built third_party. Report rather than fix, +# because one submodule here carries an in-tree cmake build whose untracked +# artifacts block an update. +if ! git -C "$SRC" submodule status --recursive 2>/dev/null | grep -qv '^ '; then + : +else + echo "build-canary: note β€” submodules are not all at their recorded commits:" >&2 + git -C "$SRC" submodule status 2>/dev/null | grep -v '^ ' | sed 's/^/ /' >&2 +fi + +if [ ! -f "$BUILD/CMakeCache.txt" ]; then + echo "==> configuring $BUILD ($CONFIG, Ninja Multi-Config, clang $(clang --version | head -1 | grep -oE '[0-9]+\.[0-9]+\.[0-9]+'))" + cmake -S "$SRC" -B "$BUILD" -G "Ninja Multi-Config" \ + -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \ + -DXENIA_BUILD_TESTS=OFF -DXENIA_BUILD_MISC=OFF \ + -DXENIA_ENABLE_LTO=OFF +fi + +echo "==> building $CONFIG with -j$JOBS" +cmake --build "$BUILD" --config "$CONFIG" --parallel "$JOBS" --target xenia_canary "$@" + +BIN="$BUILD/bin/Linux/$CONFIG/xenia_canary" +if [ -x "$BIN" ]; then + echo "==> $BIN" + echo " run it with: run-canary" +else + echo "build-canary: target did not produce $BIN" >&2 + exit 1 +fi diff --git a/docker/agent/bin/build-reborn b/docker/agent/bin/build-reborn new file mode 100755 index 00000000..8c36dc16 --- /dev/null +++ b/docker/agent/bin/build-reborn @@ -0,0 +1,47 @@ +#!/usr/bin/env bash +# Build / test Sylpheed Reborn inside the container. +# +# CARGO_TARGET_DIR points outside the bind-mounted repo (see the Dockerfile), so +# this never fights the host's incremental state. +# +# build-reborn cargo build --workspace +# build-reborn test cargo test --workspace, disc tests enabled +# build-reborn ci fmt + clippy + test +# build-reborn +set -euo pipefail + +SRC="${PROJECT_DIR:-/work}/Syplheed-Reborn" +JOBS="${SYLPH_JOBS:-2}" +cd "$SRC" + +# The disc-gated integration tests self-skip when this is unset, and a green run +# then means almost nothing β€” point them at the extracted disc if it is there. +if [ -z "${SYLPHEED_DISC:-}" ]; then + for c in "${PROJECT_DIR:-/work}/sylph_extract" "$SRC/../sylph_extract"; do + [ -d "$c/dat" ] && { export SYLPHEED_DISC="$(readlink -f "$c")"; break; } + done +fi +[ -n "${SYLPHEED_DISC:-}" ] && export SYLPHEED_RES3D="$SYLPHEED_DISC/hidden/resource3d" +if [ -z "${SYLPHEED_ISO:-}" ]; then + iso="$(find "${PROJECT_DIR:-/work}" -maxdepth 2 -iname '*.iso' -print -quit 2>/dev/null || true)" + [ -n "$iso" ] && export SYLPHEED_ISO="$iso" +fi +echo "==> SYLPHEED_DISC=${SYLPHEED_DISC:-}" >&2 + +export CARGO_BUILD_JOBS="$JOBS" + +case "${1:-build}" in + build) shift || true; exec cargo build --workspace "$@" ;; + test) shift || true; exec cargo test --workspace "$@" ;; + ci) + cargo fmt --all -- --check + cargo clippy --workspace -- -D warnings + cargo test --workspace + # NOTE: `just ci` also checks wasm32. That leg does not build, and not for + # any reason in this crate: the workspace pins tokio with features=["full"], + # which pulls mio, which refuses to compile for wasm32. Left out here rather + # than reported as a failure of the change under test. + echo "==> native CI green (wasm leg skipped β€” see the note in this script)" + ;; + *) exec cargo "$@" ;; +esac diff --git a/docker/agent/bin/run-canary b/docker/agent/bin/run-canary new file mode 100755 index 00000000..4de2e120 --- /dev/null +++ b/docker/agent/bin/run-canary @@ -0,0 +1,99 @@ +#!/usr/bin/env bash +# Launch Xenia Canary with the settings this title actually needs. +# +# Four of these are not preferences β€” they are measured requirements, and every +# one of them cost a debugging session before it was pinned down: +# +# --apu=sdl + SDL_AUDIODRIVER=dummy +# There is no PulseAudio here, so `--apu=nop` looks like the safe muted +# choice. It is not: the log then fills with "CreateDriver failed for +# index=0", the guest never gets past the intro movie, and the window +# stays black for 8+ minutes. The SDL driver against a dummy device is +# both silent AND lets the title advance. +# +# NO --audio flag +# The RE notes say "--audio --apu=sdl". `--audio` is NOT a cvar in this +# tree, and an unknown argument is not a friendly error: xenia calls +# ShowSimpleMessageBox from ParseLaunchArguments, BEFORE logging is +# initialised, and that SDL dialog blocks on XIfEvent forever. Headless, +# the symptom is a 10x10 window, an empty log, and no guest memory β€” +# which reads like a hang deep in the emulator rather than a typo. +# If this ever appears to hang at startup, suspect a bad flag first. +# +# --hid=file --pad_file=... +# The old vgamepad path made its device through /dev/uinput, which is NOT +# namespaced β€” a pad created inside a container registers with the HOST's +# input stack and every scripted press leaks to the user's desktop. This +# driver reads a text file instead. Drive it with tools/re-capture/pad.py. +# Trap worth remembering: 360 menus poll XamInputGetKeystrokeEx, not +# GetState, so a stubbed GetKeystroke looks like a completely dead pad. +# +# one instance at a time +# Two emulators (or ours + canary) at once perturbs both and the box. +# Enforced with a lockfile rather than left to discipline. +# +# Usage: run-canary [extra xenia flags...] +# ISO from $SYLPH_ISO, else the first *.iso under $PROJECT_DIR. +# Binary from $XENIA_BIN, else the container build, else the repo build. +set -u + +LOCK=/tmp/xenia-canary.lock +exec 9>"$LOCK" +if ! flock -n 9; then + echo "run-canary: an emulator is already running (lock $LOCK)." >&2 + echo " Only one at a time β€” kill it first: pkill -x xenia_canary" >&2 + exit 1 +fi + +PROJECT_DIR="${PROJECT_DIR:-/work}" + +# ── Binary ─────────────────────────────────────────────────────────────────── +pick_bin() { + [ -n "${XENIA_BIN:-}" ] && { echo "$XENIA_BIN"; return; } + for c in \ + "${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}/bin/Linux/Release/xenia_canary" \ + "${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}/bin/Linux/Debug/xenia_canary" \ + "$PROJECT_DIR/xenia-canary/build/bin/Linux/Release/xenia_canary" \ + "$PROJECT_DIR/xenia-canary/build/bin/Linux/Debug/xenia_canary"; do + [ -x "$c" ] && { echo "$c"; return; } + done +} +BIN="$(pick_bin)" +if [ -z "${BIN:-}" ]; then + echo "run-canary: no xenia_canary binary found. Build one with: build-canary" >&2 + exit 1 +fi + +# ── ISO ────────────────────────────────────────────────────────────────────── +ISO="${SYLPH_ISO:-}" +if [ -z "$ISO" ]; then + # Prefer a REAL file over a symlink and take the largest: the tree carries + # `xenia-rs/sylpheed.iso` as a symlink to the retail image, and a symlink has + # already cost a session once (Wine could not resolve it -> "path invalid"). + ISO="$(find "$PROJECT_DIR" -maxdepth 2 -type f -iname '*.iso' -printf '%s\t%p\n' 2>/dev/null \ + | sort -rn | head -1 | cut -f2-)" +fi +if [ -z "$ISO" ] || [ ! -f "$ISO" ]; then + echo "run-canary: no ISO. Set SYLPH_ISO=/path/to/game.iso" >&2 + exit 1 +fi +ISO="$(readlink -f "$ISO")" + +export SDL_AUDIODRIVER="${SDL_AUDIODRIVER:-dummy}" +export DISPLAY="${DISPLAY:-:98}" +PAD="${XENIA_PAD_FILE:-/tmp/xenia_pad.txt}" +: > "$PAD" + +# Guest memory is backed by /dev/shm; a stale file from a killed run confuses +# the memory readers (gmem.py finds two candidates and picks the dead one). +rm -f /dev/shm/xenia_memory_* /dev/shm/xenia_code_cache_* 2>/dev/null || true + +echo "run-canary: $BIN" >&2 +echo " iso: $ISO" >&2 +echo " pad: $PAD display: $DISPLAY shm: $(df -h /dev/shm | awk 'NR==2{print $2}')" >&2 + +exec "$BIN" "$ISO" \ + --apu=sdl \ + --hid=file --pad_file="$PAD" \ + --mute=true \ + "$@" diff --git a/docker/agent/bin/screenshot b/docker/agent/bin/screenshot new file mode 100755 index 00000000..098fac7e --- /dev/null +++ b/docker/agent/bin/screenshot @@ -0,0 +1,51 @@ +#!/usr/bin/env bash +# Raw full-root PNG grab of the headless display. +# +# This is deliberately the *uncropped* root window, because +# `tools/re-capture/bin/screenshot` is a wrapper that calls this one as its raw +# grabber and then crops to the game surface β€” xenia's window is a GTK window +# whose menu bar pushes the 1280x720 game image down ~25 px, and every pixel +# oracle in the toolkit was measured against the bare game image. That wrapper +# directory is first on PATH, so scripts calling `screenshot` get the cropped +# game surface and this stays the honest raw grab underneath it. +# +# screenshot [out.png] default: $HOME/shots/shot-NNNN.png +set -u + +OUT="${1:-}" +if [ -z "$OUT" ]; then + dir="${HOME:-/tmp}/shots"; mkdir -p "$dir" + n_file="$dir/.counter" + n=$(( $(cat "$n_file" 2>/dev/null || echo 0) + 1 )); echo "$n" > "$n_file" + OUT="$dir/shot-$(printf '%04d' "$n").png" +fi +mkdir -p "$(dirname "$OUT")" + +if ! xdpyinfo >/dev/null 2>&1; then + echo "screenshot: no display on ${DISPLAY:-}" >&2 + exit 1 +fi + +# ImageMagick first. `import` talks X11 directly and captures a window that is +# mid-redraw without tearing the way a video grabber can. +if command -v import >/dev/null 2>&1 && import -silent -window root "$OUT" 2>/dev/null; then + echo "$OUT"; exit 0 +fi + +# Fallback: ffmpeg's x11grab. Needs an explicit size, so read it off the server +# rather than assuming the geometry. +if command -v ffmpeg >/dev/null 2>&1; then + size=$(xdpyinfo | awk '/dimensions:/{print $2; exit}') + if ffmpeg -loglevel error -y -f x11grab -draw_mouse 0 \ + -video_size "$size" -i "$DISPLAY" -frames:v 1 "$OUT" 2>/dev/null; then + echo "$OUT"; exit 0 + fi +fi + +# Last resort: xwd, which is always present with x11-utils. +if command -v xwd >/dev/null 2>&1 && command -v convert >/dev/null 2>&1; then + xwd -root -silent | convert xwd:- "$OUT" && { echo "$OUT"; exit 0; } +fi + +echo "screenshot: no working capture backend" >&2 +exit 1 diff --git a/docker/agent/bin/sylph-doctor b/docker/agent/bin/sylph-doctor new file mode 100755 index 00000000..f1c87f11 --- /dev/null +++ b/docker/agent/bin/sylph-doctor @@ -0,0 +1,88 @@ +#!/usr/bin/env bash +# Prove the container can actually do the four things it exists for, before an +# unattended agent spends an hour discovering otherwise. +# +# Every check here stands for a failure that has already happened once: a +# display that was not there, a missing numpy that looked like a logic bug, a +# /dev/shm too small for guest memory, a Vulkan stack with no ICD. +set -u +fail=0 +ok() { printf ' \033[32mβœ”\033[0m %s\n' "$*"; } +bad() { printf ' \033[31mβœ–\033[0m %s\n' "$*"; fail=$((fail+1)); } +warn() { printf ' \033[33m!\033[0m %s\n' "$*"; } + +echo "── resources ──" +# nproc shows the HOST's cores: --cpus is a quota, not a mask. Report both so +# "12 cpus" is never mistaken for 12 cpus' worth of throughput. +quota="unlimited" +if [ -r /sys/fs/cgroup/cpu.max ]; then + read -r q p < /sys/fs/cgroup/cpu.max + [ "$q" != max ] && quota="$(( q / p )) (quota)" +fi +echo " cpus: $(nproc) visible, $quota" +if [ -r /sys/fs/cgroup/memory.max ]; then + m=$(cat /sys/fs/cgroup/memory.max) + [ "$m" = max ] && warn "memory: UNLIMITED β€” the half-the-box cap is not applied" \ + || ok "memory cap: $(( m / 1024 / 1024 / 1024 )) GiB" +fi +shm=$(df -BM /dev/shm | awk 'NR==2{print $2}' | tr -d M) +# Guest memory for a 512 MB console plus the code cache does not fit in +# Docker's 64 MB default, and the symptom is an mmap error, not a disk-full one. +[ "${shm:-0}" -ge 512 ] && ok "/dev/shm: ${shm} MiB" || bad "/dev/shm only ${shm:-?} MiB β€” need >=512; pass --shm-size" + +echo "── toolchain ──" +for t in clang clang++ cmake ninja cargo rustc python3 node claude; do + command -v "$t" >/dev/null && ok "$t ($("$t" --version 2>/dev/null | head -1))" || bad "$t missing" +done + +echo "── python (dynamic RE) ──" +# numpy and PIL missing is the specific hole that silently disabled entities2.py +# and every image oracle in the toolkit. +for m in numpy PIL duckdb; do + python3 -c "import $m" 2>/dev/null && ok "python: $m" || bad "python: $m MISSING" +done + +echo "── display ──" +if xdpyinfo >/dev/null 2>&1; then + ok "display $DISPLAY ($(xdpyinfo | awk '/dimensions:/{print $2; exit}'))" + pgrep -x openbox >/dev/null && ok "openbox running" || warn "no window manager β€” window geometry oracles will misread" + out=$(screenshot /tmp/_doctor.png 2>&1) && [ -s /tmp/_doctor.png ] \ + && ok "screenshot works -> $(identify -format '%wx%h' /tmp/_doctor.png 2>/dev/null || echo ok)" \ + || bad "screenshot failed: $out" + rm -f /tmp/_doctor.png +else + bad "no display on ${DISPLAY:-}" +fi + +echo "── vulkan ──" +if command -v vulkaninfo >/dev/null 2>&1; then + dev=$(vulkaninfo --summary 2>/dev/null | grep -m3 -E 'deviceName' | sed 's/^ *//') + [ -n "$dev" ] && { ok "Vulkan devices:"; echo "$dev" | sed 's/^/ /'; } \ + || bad "vulkaninfo found no device (ICD missing?)" +else + bad "vulkaninfo missing" +fi +# Judge by what enumerated, not by whether a device node is present: an NVIDIA +# card needs the NVIDIA Container Toolkit, and /dev/dri alone does nothing. +case "${dev:-}" in + *llvmpipe*|*lavapipe*) + warn "SOFTWARE Vulkan only β€” correct but slow." + command -v nvidia-smi >/dev/null 2>&1 \ + && warn " host has an NVIDIA GPU: install nvidia-container-toolkit for hardware" ;; + "") ;; + *) ok "hardware Vulkan" ;; +esac + +echo "── project ──" +[ -d /work/xenia-canary ] && ok "/work/xenia-canary" || bad "/work/xenia-canary not mounted" +[ -d /work/Syplheed-Reborn ] && ok "/work/Syplheed-Reborn" || bad "/work/Syplheed-Reborn not mounted" +iso=$(find /work -maxdepth 2 -type f -iname '*.iso' -printf '%s\t%p\n' 2>/dev/null | sort -rn | head -1 | cut -f2-) +[ -n "$iso" ] && ok "ISO: $iso" || warn "no ISO under /work β€” run-canary needs SYLPH_ISO" +[ -d /work/sylph_extract/dat ] && ok "extracted disc (disc-gated tests will run)" \ + || warn "no extracted disc β€” Reborn disc tests will SKIP" +[ -w /sylph-home/re/.claude ] && ok "~/.claude writable (token refresh works)" \ + || warn "~/.claude not writable β€” Claude Code may fail to refresh auth" + +echo +[ "$fail" -eq 0 ] && { echo "all good."; exit 0; } +echo "$fail check(s) failed."; exit 1 diff --git a/docker/agent/entrypoint.sh b/docker/agent/entrypoint.sh new file mode 100755 index 00000000..a2c97424 --- /dev/null +++ b/docker/agent/entrypoint.sh @@ -0,0 +1,103 @@ +#!/usr/bin/env bash +# Bring up the headless display, then hand over to the command. +# +# Xvfb and openbox are started HERE, as children of PID 1 (tini), rather than by +# the toolkit scripts. That is the fix for the long-standing "Xvfb and the +# emulator die on their own every few minutes" note: nothing owned those +# processes, so nothing kept them alive, and a run could sit for 300 s in front +# of a visible MAIN MENU reporting "no main menu" because the display had gone. +# A container-lifetime display makes that failure mode impossible. +set -euo pipefail + +log() { printf '[entrypoint] %s\n' "$*" >&2; } + +DISPLAY="${DISPLAY:-:98}" +GEOM="${SCREEN_GEOMETRY:-1280x720x24}" +export DISPLAY + +# ── Display ────────────────────────────────────────────────────────────────── +if ! xdpyinfo -display "$DISPLAY" >/dev/null 2>&1; then + rm -f "/tmp/.X${DISPLAY#:}-lock" "/tmp/.X11-unix/X${DISPLAY#:}" 2>/dev/null || true + # GLX and RANDR are both required: Canary's window is GTK+OpenGL even when the + # graphics backend is Vulkan, and xwininfo-based screen oracles need RANDR. + Xvfb "$DISPLAY" -screen 0 "$GEOM" -ac -nolisten tcp \ + +extension GLX +extension RANDR >/tmp/xvfb.log 2>&1 & + for _ in $(seq 1 50); do + xdpyinfo -display "$DISPLAY" >/dev/null 2>&1 && break + sleep 0.2 + done +fi +if ! xdpyinfo -display "$DISPLAY" >/dev/null 2>&1; then + log "FATAL: no display on $DISPLAY β€” see /tmp/xvfb.log" + exit 1 +fi + +# A window manager is not cosmetic here: without one the emulator window is +# never mapped at a known position, and every pixel oracle reads the wrong rows. +if ! pgrep -x openbox >/dev/null 2>&1; then + openbox >/tmp/openbox.log 2>&1 & + sleep 0.5 +fi +log "display $DISPLAY ready ($GEOM)" + +# ── Vulkan ─────────────────────────────────────────────────────────────────── +# Prefer the real GPU when /dev/dri was passed through; fall back to lavapipe, +# which is slow but correct and needs no host device. +LVP=$(ls /usr/share/vulkan/icd.d/lvp_icd*.json 2>/dev/null | head -1) +if [ "${SYLPH_VULKAN:-auto}" = "sw" ] || [ ! -e /dev/dri/renderD128 ]; then + [ -n "$LVP" ] && export VK_ICD_FILENAMES="$LVP" +else + unset LIBGL_ALWAYS_SOFTWARE +fi +# Report what Vulkan ACTUALLY enumerated, not what we asked for. Announcing +# "host GPU via /dev/dri" because the device node exists is how an agent ends up +# believing it has hardware while running llvmpipe β€” an NVIDIA card needs the +# NVIDIA Container Toolkit, and /dev/dri alone does nothing for it. +if command -v vulkaninfo >/dev/null 2>&1; then + vkdev=$(vulkaninfo --summary 2>/dev/null | awk -F= '/deviceName/{gsub(/^ +| +$/,"",$2); print $2; exit}') + case "${vkdev:-}" in + "") log "Vulkan: NO DEVICE β€” vulkaninfo enumerated nothing" ;; + llvmpipe*|lavapipe*) log "Vulkan: $vkdev (SOFTWARE β€” correct but slow)" ;; + *) log "Vulkan: $vkdev (hardware)" ;; + esac +fi + +# ── Build parallelism ──────────────────────────────────────────────────────── +# Bounded by MEMORY, not just cores. A full-parallel build of this tree has +# OOM-killed the host outright, and inside a half-the-box container the ceiling +# is lower still. ~1.5 GiB per C++ TU is the rule of thumb that has held. +# nproc reports the HOST's core count: --cpus is a CFS quota, not a mask. Using +# it would oversubscribe a half-the-box container by exactly 2x, so read the +# quota the cgroup actually grants. +cpus=$(nproc) +if [ -r /sys/fs/cgroup/cpu.max ]; then + read -r _q _p < /sys/fs/cgroup/cpu.max || true + if [ "${_q:-max}" != max ] && [ "${_p:-0}" -gt 0 ]; then + cpus=$(( (_q + _p - 1) / _p )) + [ "$cpus" -lt 1 ] && cpus=1 + fi +fi +mem_gib=$(awk '/MemAvailable/{printf "%d", $2/1048576}' /proc/meminfo) +# MemAvailable is the HOST's too under cgroup v2; prefer the container's cap. +if [ -r /sys/fs/cgroup/memory.max ]; then + _m=$(cat /sys/fs/cgroup/memory.max) + [ "$_m" != max ] && mem_gib=$(( _m / 1073741824 )) +fi +[ "${mem_gib:-0}" -lt 1 ] && mem_gib=1 +by_mem=$(( mem_gib * 2 / 3 )) +[ "$by_mem" -lt 1 ] && by_mem=1 +jobs=$(( cpus < by_mem ? cpus : by_mem )) +export SYLPH_JOBS="$jobs" CARGO_BUILD_JOBS="$jobs" CMAKE_BUILD_PARALLEL_LEVEL="$jobs" +log "build parallelism: $jobs (cpus=$cpus, mem=${mem_gib}GiB avail)" + +mkdir -p "$HOME/shots" "$HOME/logs" + +# ── Claude Code ────────────────────────────────────────────────────────────── +if [ "${SYLPH_AUTONOMOUS:-0}" = "1" ]; then + # The flag the user asked for. It is refused under root, which is why this + # image runs as `agent`. + set -- claude --dangerously-skip-permissions "$@" + log "starting Claude Code with --dangerously-skip-permissions" +fi + +exec "$@" diff --git a/docker/agent/sylph-agent b/docker/agent/sylph-agent new file mode 100755 index 00000000..f6e29341 --- /dev/null +++ b/docker/agent/sylph-agent @@ -0,0 +1,174 @@ +#!/usr/bin/env bash +# Host-side launcher for the Sylpheed RE agent container. +# +# Caps the container at HALF the machine's CPUs and memory, computed at run time +# so it stays half on whatever box it lands on. +# +# ./sylph-agent build build (or rebuild) the image +# ./sylph-agent shell interactive shell in the container +# ./sylph-agent agent [prompt] Claude Code, --dangerously-skip-permissions +# ./sylph-agent run one-shot command +# ./sylph-agent stop stop a detached container +# +# Environment: +# SYLPH_PROJECT host project root (default: three levels up from this file) +# SYLPH_CLAUDE_HOME host dir mounted as the agent's ~/.claude +# (default: $HOME/.claude β€” shares auth AND memory with you) +# SYLPH_VULKAN=sw force software Vulkan (lavapipe) even if /dev/dri exists +# SYLPH_CPUS / SYLPH_MEM_GB override the computed half +set -euo pipefail + +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +IMAGE="${SYLPH_IMAGE:-sylpheed-agent:latest}" +NAME="${SYLPH_NAME:-sylpheed-agent}" +PROJECT="${SYLPH_PROJECT:-$(cd "$HERE/../../.." && pwd)}" + +# ── Half the box ───────────────────────────────────────────────────────────── +# LC_ALL=C is required, not tidiness: under a locale with a comma decimal +# separator (de_DE and friends) awk prints "6,0" and docker rejects it as +# --cpus with "failed to parse as a rational number". +HOST_CPUS=$(nproc) +HOST_MEM_KB=$(awk '/MemTotal/{print $2}' /proc/meminfo) +CPUS="${SYLPH_CPUS:-$(LC_ALL=C awk -v c="$HOST_CPUS" 'BEGIN{printf "%.1f", c/2}')}" +MEM_GB="${SYLPH_MEM_GB:-$(LC_ALL=C awk -v m="$HOST_MEM_KB" 'BEGIN{printf "%d", m/1048576/2}')}" +[ "$MEM_GB" -lt 2 ] && MEM_GB=2 +# /dev/shm holds the emulator's guest memory (gmem.py reads it there). Docker's +# 64 MB default is far too small for a 512 MB console address space, and the +# failure is an obscure mmap error rather than an out-of-space message. tmpfs +# pages count against the memory cap, so take a third of it and no more. +SHM_GB=$(( MEM_GB / 3 )); [ "$SHM_GB" -lt 1 ] && SHM_GB=1 + +usage() { sed -n '2,20p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'; exit "${1:-0}"; } + +docker_args() { + local -n _out=$1 + _out=( + --name "$NAME" + --hostname sylph-agent + # ── the cap ── + --cpus "$CPUS" + --memory "${MEM_GB}g" + --memory-swap "${MEM_GB}g" # no swap escape hatch: a swapping build + # thrashes the whole host, which is the + # failure this cap exists to prevent + --pids-limit 4096 + # /dev/shm as an EXEC-capable tmpfs, not --shm-size. Docker's default mounts + # it `noexec`, and xenia maps its JIT code cache out of a shm file at a fixed + # address β€” so with noexec it dies at startup with "Unable to allocate code + # cache generated code storage / Cannot initalize processor", which reads + # like an address-space clash rather than a mount flag. + --tmpfs "/dev/shm:rw,exec,nosuid,nodev,size=${SHM_GB}g" + # Dynamic RE needs to attach to a live process: without SYS_PTRACE, gdb and + # strace are installed but inert ("Could not attach to process"), and the + # container's whole reason for existing is watching the emulator run. + # Docker's default seccomp profile also blocks calls the JIT and the guest + # memory mapper rely on. + --cap-add SYS_PTRACE + --security-opt seccomp=unconfined + --security-opt apparmor=unconfined + # ── project ── + -v "$PROJECT:/work" + -e "PROJECT_DIR=/work" + # ── claude ── + -v "${SYLPH_CLAUDE_HOME:-$HOME/.claude}:/sylph-home/re/.claude" + # persistent build caches, so a container restart is not a rebuild + -v sylph-agent-cargo:/sylph-home/re/.cargo + -v sylph-agent-target:/sylph-home/re/target-container + -v sylph-agent-canary-build:/sylph-home/re/canary-build + ) + # ── Vulkan SDK ── + # xenia's shader step calls `spirv-opt --canonicalize-ids`, which Ubuntu's + # packaged SPIRV-Tools (v2025.1) does not have β€” the build then dies ~500 + # objects in. The LunarG SDK has it. Mounting the host's copy at the same path + # is cheaper than baking a 200 MB SDK into the image AND guarantees the + # container produces byte-identical shaders to the host build. + SDK="${VULKAN_SDK:-}" + if [ -z "$SDK" ]; then + SDK=$(ls -d "$HOME"/vulkan-sdk/*/x86_64 2>/dev/null | sort -V | tail -1 || true) + fi + if [ -n "$SDK" ] && [ -x "$SDK/bin/spirv-opt" ]; then + _out+=(-v "$SDK:$SDK:ro" -e "VULKAN_SDK=$SDK") + else + echo "==> NOTE: no Vulkan SDK found on the host. Building Canary's shaders" >&2 + echo " needs spirv-opt with --canonicalize-ids (LunarG SDK); Ubuntu's" >&2 + echo " packaged SPIRV-Tools is too old. Running is unaffected." >&2 + fi + + [ -n "${ANTHROPIC_API_KEY:-}" ] && _out+=(-e "ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY") + [ -n "${SYLPH_VULKAN:-}" ] && _out+=(-e "SYLPH_VULKAN=$SYLPH_VULKAN") + [ -n "${SYLPH_ISO:-}" ] && _out+=(-e "SYLPH_ISO=$SYLPH_ISO") + + # ── GPU ── + # Three distinct cases, and conflating them is how you end up believing you + # have hardware Vulkan while actually running llvmpipe: + # + # NVIDIA needs the NVIDIA Container Toolkit (`--gpus all`). Passing + # /dev/dri alone does NOT work β€” Mesa cannot drive an NVIDIA card, + # and the proprietary userspace lives outside the image. + # Mesa (AMD/Intel) works with a plain /dev/dri passthrough plus the + # host's render/video GIDs. + # neither software Vulkan (lavapipe): correct, and slow. + if [ "${SYLPH_VULKAN:-auto}" = "sw" ]; then + _out+=(-e SYLPH_VULKAN=sw) + elif command -v nvidia-smi >/dev/null 2>&1 && nvidia-smi -L >/dev/null 2>&1; then + if docker info --format '{{json .Runtimes}}' 2>/dev/null | grep -q nvidia; then + _out+=(--gpus all) + else + echo "==> NOTE: NVIDIA GPU found but the NVIDIA Container Toolkit is not" >&2 + echo " installed, so hardware Vulkan is unavailable and the container" >&2 + echo " will use lavapipe (software β€” correct, slow). To enable it:" >&2 + echo " sudo apt install nvidia-container-toolkit \\" >&2 + echo " && sudo nvidia-ctk runtime configure --runtime=docker \\" >&2 + echo " && sudo systemctl restart docker" >&2 + _out+=(-e SYLPH_VULKAN=sw) + fi + elif [ -e /dev/dri/renderD128 ]; then + _out+=(--device /dev/dri) + for g in render video; do + gid=$(getent group "$g" | cut -d: -f3 || true) + [ -n "$gid" ] && _out+=(--group-add "$gid") + done + else + _out+=(-e SYLPH_VULKAN=sw) + fi +} + +case "${1:-}" in + build) + shift + echo "==> building $IMAGE (uid $(id -u), gid $(id -g))" + exec docker build -t "$IMAGE" \ + --build-arg "AGENT_UID=$(id -u)" --build-arg "AGENT_GID=$(id -g)" \ + "$@" "$HERE" + ;; + + shell|agent|run) + mode=$1; shift + declare -a ARGS; docker_args ARGS + echo "==> $mode | cpus=$CPUS mem=${MEM_GB}g shm=${SHM_GB}g (host: ${HOST_CPUS} cpus, $((HOST_MEM_KB/1048576))g)" + echo "==> project: $PROJECT -> /work" + docker rm -f "$NAME" >/dev/null 2>&1 || true + # Allocate a TTY only when stdin actually is one: `docker run -it` fails + # outright ("cannot attach stdin to a TTY-enabled container") under a + # pipeline or a CI runner, which is exactly where `run` gets used. + TTY=(-i); [ -t 0 ] && TTY=(-it) + case "$mode" in + shell) exec docker run --rm "${TTY[@]}" "${ARGS[@]}" "$IMAGE" bash ;; + agent) + # The flag the user asked for. Refused under root, which is why the + # image runs as an unprivileged `agent` user. + ARGS+=(-e SYLPH_AUTONOMOUS=1) + exec docker run --rm "${TTY[@]}" "${ARGS[@]}" "$IMAGE" "$@" + ;; + run) exec docker run --rm "${TTY[@]}" "${ARGS[@]}" "$IMAGE" "$@" ;; + esac + ;; + + stop) exec docker rm -f "$NAME" ;; + doctor) + declare -a ARGS; docker_args ARGS + exec docker run --rm "${ARGS[@]}" "$IMAGE" sylph-doctor + ;; + ""|-h|--help) usage 0 ;; + *) echo "unknown command: $1" >&2; usage 2 ;; +esac