docker: a container the autonomous RE agent can be turned loose in
Builds and runs both halves of the project -- Canary as the oracle, Reborn as
the port -- plus the dynamic-RE toolkit. Claude Code runs with
--dangerously-skip-permissions as an unprivileged `agent` user, because that
flag is refused under root.
Capped at half the machine, computed at launch: --cpus nproc/2, --memory half
of MemTotal with --memory-swap equal to it (no swap escape hatch -- a swapping
build thrashes the host, which is the failure the cap exists to prevent), and
build parallelism derived INSIDE the container from available memory rather
than core count, since a full-parallel build of this tree has OOM-killed the
host outright.
Three things the old box got wrong are fixed rather than reproduced: a real
toolchain (so rebuild_canary.sh's hand-relinking is obsolete), numpy and Pillow
(whose absence silently disabled every image oracle and looked like a logic
bug), and a display owned by PID 1 (so Xvfb no longer "dies on its own every
few minutes" -- it was being reaped because nothing owned it).
Verified end to end, not by inspection: the image builds, sylph-doctor is green,
`build-canary` links xenia_canary inside the container, and that binary then
runs -- guest memory and the JIT code cache appear in /dev/shm within 4 s,
1 205 log lines, gmem.py reads guest RAM, pad.py drives the file pad, and
screenshot captures the display.
Five environment defects found and fixed on the way, each of which fails in a
way that points somewhere else entirely:
* /dev/shm is `noexec` under Docker. Xenia maps its JIT code cache out of an
shm file, so it died with "Unable to allocate code cache generated code
storage / Cannot initalize processor" -- which reads as an address-space
clash, not a mount flag. Now `--tmpfs /dev/shm:rw,exec`.
* An unknown xenia flag HANGS rather than errors: ParseLaunchArguments calls
ShowSimpleMessageBox before logging is initialised, and that SDL dialog
blocks on XIfEvent forever. `--audio` (which the RE notes recommend) is not
a cvar in this tree; the symptom was a 10x10 window and an empty log.
* Named volumes come up root-owned unless their mount point exists in the
image, so the first cmake configure failed on pkgRedirects.
* Ubuntu 24.04 ships its own uid-1000 account, colliding with the host user.
* Ubuntu's spirv-opt has no --canonicalize-ids, so the shader step dies ~500
objects in; the launcher mounts the host's LunarG SDK instead of baking one
in, which also keeps shader output byte-identical to a host build.
Known limits, stated rather than papered over: on an NVIDIA host without the
NVIDIA Container Toolkit there is no hardware Vulkan (/dev/dri alone does
nothing for NVIDIA), and under lavapipe the emulator runs correctly but was not
observed to reach a rendered frame within a couple of minutes. gdb needs `sudo`
inside the container because the host's yama ptrace_scope outranks SYS_PTRACE.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
4
docker/agent/.dockerignore
Normal file
4
docker/agent/.dockerignore
Normal file
@@ -0,0 +1,4 @@
|
||||
**
|
||||
!bin
|
||||
!bin/**
|
||||
!entrypoint.sh
|
||||
62
docker/agent/AGENT.md
Normal file
62
docker/agent/AGENT.md
Normal file
@@ -0,0 +1,62 @@
|
||||
# Notes for an agent working inside this container
|
||||
|
||||
Read this before starting a dynamic-RE run. Everything here is something that
|
||||
already went wrong once.
|
||||
|
||||
## The container fixes three old traps for you
|
||||
|
||||
* **The display outlives the turn.** Xvfb and openbox are children of PID 1, not
|
||||
of your shell. The old "Xvfb dies on its own every few minutes" note is gone —
|
||||
you no longer have to wrap a whole session in one blocking foreground call to
|
||||
keep it alive.
|
||||
* **The toolchain is real.** `tools/re-capture/rebuild_canary.sh` exists because
|
||||
the old box had no cmake/ninja/clang and only runtime sonames, so it hand-
|
||||
relinked object files. **Do not use it here.** Use `build-canary`.
|
||||
* **numpy and Pillow are installed.** `entities2.py`, `flight_probe.py` and the
|
||||
image oracles work. Their absence used to look like a logic bug.
|
||||
|
||||
## Method (the part that matters more than the tooling)
|
||||
|
||||
* **Measure the oracle; never infer it.** A session with zero Canary runs is a
|
||||
red flag.
|
||||
* **Trace upstream to where data first goes wrong**, rather than patching the
|
||||
symptom you can see.
|
||||
* **Try to refute before believing.** Record demotions rather than editing them
|
||||
away — `docs/re/README.md` has the ✅/🟡/❔ convention, and a withdrawn result
|
||||
is more useful than a quietly deleted one.
|
||||
* **A probe that never performs the action will "prove" the action does not
|
||||
exist.** The "targeting is automatic" conclusion came from a sweep that only
|
||||
ever tapped once; target select is Ⓐ pressed *twice*.
|
||||
* **Do not poll faster than the guest updates** — it manufactures a clean curve
|
||||
out of noise. `rate-curve-aliased-BAD.csv` is committed as the bad example.
|
||||
|
||||
## Running the emulator
|
||||
|
||||
```bash
|
||||
run-canary # correct audio/pad/display flags baked in
|
||||
pad.py tap A ; pad.py dpad down # scripted input (--hid=file, no uinput)
|
||||
screenshot ~/shots/now.png # cropped to the GAME surface, not the window
|
||||
python3 tools/re-capture/gmem.py find hex:820af844 400
|
||||
```
|
||||
|
||||
* **One emulator at a time.** `run-canary` enforces it with a lockfile.
|
||||
* Boot is slow cold, ~25 s once the shader/code caches are warm — so a
|
||||
launch-and-dump fits in a single call.
|
||||
* **Screens: classify by whole-image statistics** (`screen_id.py`), not named
|
||||
pixels. Named-pixel oracles are only valid while the game image sits at a
|
||||
known place, and nothing errors when it moves.
|
||||
|
||||
## Verifying your own work
|
||||
|
||||
* Reborn's disc-gated tests **self-skip** without `SYLPHEED_DISC`. A green run
|
||||
with it unset means almost nothing. `build-reborn test` wires it up for you.
|
||||
* Prefer a headless self-verify over "it compiles": `sylpheed-cli mesh render`,
|
||||
`screen render`, `save info` all produce checkable artifacts.
|
||||
* A Bevy system-parameter conflict is invisible to the type checker and panics
|
||||
at startup. If you touch viewer systems, *run the binary*, don't just build it.
|
||||
|
||||
## Reporting
|
||||
|
||||
State what you measured, what you assumed, and what you could not settle. If a
|
||||
result is withdrawn, say so and keep the reasoning — that is the corpus's whole
|
||||
convention, and the reason its numbers can be trusted.
|
||||
142
docker/agent/Dockerfile
Normal file
142
docker/agent/Dockerfile
Normal file
@@ -0,0 +1,142 @@
|
||||
# Autonomous RE agent container for Project Sylpheed.
|
||||
#
|
||||
# Builds and runs BOTH halves of the project — Xenia Canary (C++/CMake/Ninja)
|
||||
# as the behaviour oracle, and Sylpheed Reborn (Rust/Bevy) as the port — plus
|
||||
# the dynamic-RE toolkit that drives the emulator and reads its guest memory.
|
||||
#
|
||||
# Three things here exist because their absence cost the previous agent real
|
||||
# hours, and they are load-bearing rather than nice-to-have:
|
||||
#
|
||||
# 1. A REAL toolchain. The old box shipped runtime sonames only (libgtk-3.so.0
|
||||
# but no libgtk-3.so), no cmake/ninja/clang and no libstdc++fs, so a full
|
||||
# build was impossible and `tools/re-capture/rebuild_canary.sh` had to
|
||||
# hand-relink object files. With -dev packages present that script is
|
||||
# obsolete; use `build-canary`.
|
||||
# 2. numpy and Pillow. Their absence silently disabled entities2.py,
|
||||
# flight_probe.py and every image oracle, and the failure looked like a
|
||||
# logic bug rather than a missing package.
|
||||
# 3. A display that outlives the turn. Xvfb kept dying "on its own every few
|
||||
# minutes"; it was being reaped because nothing owned it. Here it is a
|
||||
# child of PID 1 and lives exactly as long as the container.
|
||||
#
|
||||
# Clang is pinned to 19 to match the host that produced the checked-in build
|
||||
# caches (Ubuntu clang 19.1.1).
|
||||
|
||||
FROM ubuntu:24.04
|
||||
|
||||
ENV DEBIAN_FRONTEND=noninteractive \
|
||||
LANG=C.UTF-8 \
|
||||
TZ=Etc/UTC
|
||||
|
||||
# ── System packages ──────────────────────────────────────────────────────────
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
# toolchain
|
||||
build-essential cmake ninja-build pkg-config git curl wget ca-certificates \
|
||||
clang-19 lld-19 llvm-19 libc++-19-dev libc++abi-19-dev \
|
||||
# Canary: GTK window, SDL input/audio, Vulkan, compression
|
||||
libgtk-3-dev libsdl2-dev liblz4-dev libvulkan-dev libx11-xcb-dev \
|
||||
libxcb1-dev libxrandr-dev libssl-dev libfuse2t64 \
|
||||
# Shader toolchain: the GPU build shells out to `glslangValidator` and the
|
||||
# SPIR-V tools to compile xenia's own shaders. Missing them does not fail
|
||||
# configure — it fails ~500 objects in, as a Python FileNotFoundError.
|
||||
glslang-tools spirv-tools spirv-headers \
|
||||
# Vulkan runtime — lavapipe (software) plus the real ICDs for /dev/dri
|
||||
mesa-vulkan-drivers vulkan-tools libvulkan1 libgl1-mesa-dri libglx-mesa0 \
|
||||
# Reborn / Bevy: audio, input, windowing
|
||||
libasound2-dev libudev-dev libwayland-dev libxkbcommon-dev \
|
||||
libx11-dev libxi-dev libxcursor-dev libxinerama-dev libxext-dev \
|
||||
# headless display + window manager + the screenshot path
|
||||
xvfb x11-utils x11-xserver-utils openbox xdotool imagemagick ffmpeg \
|
||||
# dynamic RE
|
||||
python3 python3-numpy python3-pil python3-pip \
|
||||
gdb strace ltrace binutils file xxd ripgrep jq unzip zip p7zip-full \
|
||||
procps psmisc lsof less nano tini sudo \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Pin the unversioned tool names to 19 so CMake, and anything that shells out to
|
||||
# `clang`, agree with what the caches were built by.
|
||||
RUN for t in clang clang++ lld ld.lld llvm-ar llvm-ranlib llvm-nm clang-cpp; do \
|
||||
src="/usr/bin/${t}-19"; \
|
||||
[ -e "$src" ] && update-alternatives --install "/usr/bin/${t}" "$t" "$src" 200 || true; \
|
||||
done
|
||||
|
||||
# duckdb reads the static-analysis database (sylpheed.db); it is not packaged.
|
||||
# PEP 668 marks the system env externally-managed, and this image has no other
|
||||
# Python consumer to protect, so installing into it is the honest simple option.
|
||||
RUN pip3 install --no-cache-dir --break-system-packages duckdb
|
||||
|
||||
# ── Node + Claude Code ───────────────────────────────────────────────────────
|
||||
RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \
|
||||
&& apt-get install -y --no-install-recommends nodejs \
|
||||
&& npm install -g @anthropic-ai/claude-code \
|
||||
&& npm cache clean --force \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# ── The agent user ───────────────────────────────────────────────────────────
|
||||
# NOT root, and not negotiable: Claude Code refuses --dangerously-skip-permissions
|
||||
# when it has root privileges. uid/gid 1000 matches the host account so files
|
||||
# written into the bind-mounted repos keep the right ownership.
|
||||
ARG AGENT_UID=1000
|
||||
ARG AGENT_GID=1000
|
||||
# Ubuntu 24.04 ships its own `ubuntu` account at uid/gid 1000, so the common
|
||||
# case — matching a host user who is also 1000 — collides with it. Remove the
|
||||
# stock account first; nothing in this image uses it.
|
||||
RUN if getent passwd "${AGENT_UID}" >/dev/null; then \
|
||||
userdel -r "$(getent passwd "${AGENT_UID}" | cut -d: -f1)" 2>/dev/null || true; \
|
||||
fi; \
|
||||
if getent group "${AGENT_GID}" >/dev/null; then \
|
||||
groupdel "$(getent group "${AGENT_GID}" | cut -d: -f1)" 2>/dev/null || true; \
|
||||
fi; \
|
||||
groupadd -g "${AGENT_GID}" agent \
|
||||
&& useradd -m -u "${AGENT_UID}" -g "${AGENT_GID}" -s /bin/bash -d /sylph-home/re agent \
|
||||
&& mkdir -p /sylph-home/re /work \
|
||||
&& chown -R "${AGENT_UID}:${AGENT_GID}" /sylph-home \
|
||||
&& echo 'agent ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/agent
|
||||
|
||||
COPY bin/ /usr/local/bin/
|
||||
COPY entrypoint.sh /usr/local/bin/entrypoint.sh
|
||||
RUN chmod +x /usr/local/bin/* /usr/local/bin/entrypoint.sh
|
||||
|
||||
USER agent
|
||||
WORKDIR /work
|
||||
|
||||
# ── Rust ─────────────────────────────────────────────────────────────────────
|
||||
# CARGO_TARGET_DIR deliberately points OUTSIDE the bind-mounted repo: the host
|
||||
# also builds Reborn, and sharing target/ makes the two invalidate each other's
|
||||
# incremental state on every switch.
|
||||
ENV RUSTUP_HOME=/sylph-home/re/.rustup \
|
||||
CARGO_HOME=/sylph-home/re/.cargo \
|
||||
CARGO_TARGET_DIR=/sylph-home/re/target-container \
|
||||
PATH=/sylph-home/re/.cargo/bin:/usr/local/bin:/usr/bin:/bin
|
||||
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \
|
||||
| sh -s -- -y --default-toolchain stable --profile minimal \
|
||||
--component clippy --component rustfmt \
|
||||
&& rustup target add wasm32-unknown-unknown
|
||||
|
||||
# trunk serves the Reborn viewer's wasm build; the release binary avoids a
|
||||
# ten-minute `cargo install`.
|
||||
RUN curl -fsSL https://github.com/trunk-rs/trunk/releases/download/v0.21.4/trunk-x86_64-unknown-linux-gnu.tar.gz \
|
||||
| tar -xz -C /sylph-home/re/.cargo/bin trunk
|
||||
|
||||
# Create the volume mount points HERE, owned by `agent`. Docker seeds an empty
|
||||
# named volume from whatever the image has at that path — including ownership —
|
||||
# but if the path does not exist it creates a root-owned directory instead, and
|
||||
# the first write fails with something as unhelpful as
|
||||
# "CMake Error: Unable to (re)create the private pkgRedirects directory".
|
||||
RUN mkdir -p /sylph-home/re/target-container /sylph-home/re/canary-build /sylph-home/re/.claude
|
||||
|
||||
# ── Runtime environment ──────────────────────────────────────────────────────
|
||||
# DISPLAY :98 and HOME /sylph-home/re are what tools/re-capture/*.sh already
|
||||
# assume; keeping them means the existing toolkit runs unmodified.
|
||||
ENV HOME=/sylph-home/re \
|
||||
DISPLAY=:98 \
|
||||
SCREEN_GEOMETRY=1280x720x24 \
|
||||
PROJECT_DIR=/work \
|
||||
XENIA_PAD_FILE=/tmp/xenia_pad.txt \
|
||||
XENIA_BUILD_DIR=/sylph-home/re/canary-build \
|
||||
SDL_AUDIODRIVER=dummy \
|
||||
LIBGL_ALWAYS_SOFTWARE=1 \
|
||||
PATH=/work/Syplheed-Reborn/tools/re-capture/bin:/work/Syplheed-Reborn/tools/re-capture:/sylph-home/re/.cargo/bin:/usr/local/bin:/usr/bin:/bin
|
||||
|
||||
ENTRYPOINT ["/usr/bin/tini", "--", "/usr/local/bin/entrypoint.sh"]
|
||||
CMD ["bash"]
|
||||
178
docker/agent/README.md
Normal file
178
docker/agent/README.md
Normal file
@@ -0,0 +1,178 @@
|
||||
# The RE agent container
|
||||
|
||||
A container an autonomous Claude Code agent can be turned loose in: it builds
|
||||
and runs **both** halves of the project — Xenia Canary as the behaviour oracle
|
||||
and Sylpheed Reborn as the port — and carries the dynamic-RE toolkit that drives
|
||||
the emulator, reads its guest memory and photographs its screen.
|
||||
|
||||
```bash
|
||||
./sylph-agent build # build the image
|
||||
./sylph-agent doctor # prove it can do the four things it exists for
|
||||
./sylph-agent shell # poke around
|
||||
./sylph-agent agent # Claude Code, --dangerously-skip-permissions
|
||||
```
|
||||
|
||||
## The resource cap
|
||||
|
||||
The container gets **half the machine**, computed at launch so it stays half on
|
||||
any box:
|
||||
|
||||
| | how |
|
||||
|---|---|
|
||||
| CPU | `--cpus $(nproc)/2` |
|
||||
| memory | `--memory` = half `MemTotal`, **`--memory-swap` equal to it** |
|
||||
| `/dev/shm` | a third of the memory cap, min 1 GiB, mounted **exec** |
|
||||
| build jobs | derived *inside* the container from **available memory**, not cores |
|
||||
|
||||
Two of those deserve a word.
|
||||
|
||||
**No swap headroom.** `--memory-swap` is set equal to `--memory`, so the
|
||||
container cannot swap. That is deliberate: a swapping build thrashes the whole
|
||||
host, which is precisely the failure the cap exists to prevent. A build that
|
||||
would have swapped gets OOM-killed inside the container instead, and the host
|
||||
stays usable.
|
||||
|
||||
**`/dev/shm` is not incidental.** Xenia backs the guest address space with
|
||||
`/dev/shm/xenia_memory_*`, and the whole live-memory toolkit (`gmem.py`,
|
||||
`gpoke.py`, `mission_state.py`) reads it from there. Docker's default is 64 MiB,
|
||||
which is far too small for a 512 MiB console — and it fails as an obscure mmap
|
||||
error rather than an out-of-space message.
|
||||
|
||||
**Build parallelism is memory-bound.** A full-parallel build of this tree has
|
||||
OOM-killed the host outright, so the entrypoint computes jobs from *available
|
||||
memory* (≈1.5 GiB per C++ TU) and exports it as `SYLPH_JOBS`, `CARGO_BUILD_JOBS`
|
||||
and `CMAKE_BUILD_PARALLEL_LEVEL`. Override with `SYLPH_CPUS` / `SYLPH_MEM_GB`.
|
||||
|
||||
## Inside
|
||||
|
||||
| command | what |
|
||||
|---|---|
|
||||
| `build-canary [Release\|Debug]` | configure + build Canary |
|
||||
| `build-reborn [build\|test\|ci]` | build/test Reborn, with the disc env wired up |
|
||||
| `run-canary [flags…]` | launch Canary with the settings this title needs |
|
||||
| `screenshot [out.png]` | grab the display |
|
||||
| `sylph-doctor` | self-check |
|
||||
| `tools/re-capture/*` | the RE toolkit, already on `PATH` |
|
||||
|
||||
Layout: project at `/work`, `HOME=/sylph-home/re`, `DISPLAY=:98` — the values
|
||||
`tools/re-capture/*.sh` already assume, so the existing toolkit runs unmodified.
|
||||
|
||||
Build outputs live **outside** the bind mount (`CARGO_TARGET_DIR`,
|
||||
`XENIA_BUILD_DIR`, both named Docker volumes). The host builds the same trees,
|
||||
and sharing `target/` or `build/` makes host and container reconfigure and
|
||||
relink everything the other just did.
|
||||
|
||||
## Screenshots
|
||||
|
||||
Two layers, and the distinction matters:
|
||||
|
||||
* `/usr/local/bin/screenshot` — raw full-root PNG (ImageMagick, falling back to
|
||||
ffmpeg's x11grab, then xwd).
|
||||
* `tools/re-capture/bin/screenshot` — **first on `PATH`**, wraps the above and
|
||||
crops to the *game surface*.
|
||||
|
||||
The crop is not cosmetic. Xenia's window is a GTK window whose menu bar pushes
|
||||
the 1280×720 game image down ~25 px, and every pixel oracle in the toolkit was
|
||||
measured against the bare game image. When that offset was unaccounted for, one
|
||||
run sat 300 s in front of a plainly visible MAIN MENU reporting "no main menu".
|
||||
The wrapper derives the offset from the window's own height rather than a
|
||||
per-display constant.
|
||||
|
||||
For finding a screen at all, prefer `screen_id.py`, which classifies by
|
||||
whole-image statistics instead of named pixels.
|
||||
|
||||
## Vulkan
|
||||
|
||||
`mesa-vulkan-drivers` + `vulkan-tools` are installed, so Vulkan works with **no
|
||||
host GPU** via lavapipe (software — correct, slow). When the host has
|
||||
`/dev/dri`, the launcher passes the device through and adds the host's `render`
|
||||
and `video` GIDs, and the entrypoint uses the hardware ICD. Force software with
|
||||
`SYLPH_VULKAN=sw`. `vulkaninfo --summary` (or `sylph-doctor`) says which you got — and the
|
||||
entrypoint reports the device that **actually enumerated**, not the one it asked
|
||||
for, because "I passed `/dev/dri`" and "I have hardware Vulkan" are different
|
||||
claims.
|
||||
|
||||
⚠️ **On an NVIDIA host, `/dev/dri` alone does nothing** — Mesa cannot drive an
|
||||
NVIDIA card and the proprietary userspace lives outside the image. You need the
|
||||
NVIDIA Container Toolkit; the launcher detects the situation and tells you the
|
||||
three commands. Until then Canary runs on lavapipe, which is correct but has not
|
||||
been observed to reach a rendered frame in a couple of minutes — everything
|
||||
*else* (guest memory, the JIT, the live-memory toolkit) works fine on it.
|
||||
|
||||
## Input, and why there is no virtual gamepad
|
||||
|
||||
`run-canary` passes `--hid=file --pad_file=/tmp/xenia_pad.txt`; drive it with
|
||||
`tools/re-capture/pad.py`. There is deliberately **no `/dev/uinput`**: input
|
||||
devices are not namespaced, so a virtual pad created in a container registers
|
||||
with the *host's* input stack and every scripted press leaks onto the user's
|
||||
desktop.
|
||||
|
||||
The trap that wasted a session: 360 menus poll `XamInputGetKeystrokeEx`, not
|
||||
`GetState` — with `GetKeystroke` stubbed the pad looks completely dead on a
|
||||
title screen while its own log shows the press arriving.
|
||||
|
||||
## Settings that are requirements, not preferences
|
||||
|
||||
`run-canary` bakes these in; changing them will cost you an afternoon.
|
||||
|
||||
* **`--apu=sdl` with `SDL_AUDIODRIVER=dummy`** — and **no `--audio` flag**,
|
||||
which is not a cvar here (see below). There is no PulseAudio, so `--apu=nop`
|
||||
looks like the safe muted choice. It is not: the log fills with
|
||||
`CreateDriver failed for index=0`, the guest never gets past the intro movie,
|
||||
and the window stays black for 8+ minutes. SDL against a dummy device is
|
||||
silent *and* lets the title advance.
|
||||
* **One emulator at a time**, enforced with a lockfile. Two at once perturbs
|
||||
both and the box.
|
||||
* Stale `/dev/shm/xenia_memory_*` from a killed run is removed at launch —
|
||||
otherwise the memory readers find two candidates and pick the dead one.
|
||||
|
||||
## Claude Code
|
||||
|
||||
Runs as an unprivileged `agent` user, because `--dangerously-skip-permissions`
|
||||
is **refused under root**. `./sylph-agent agent` sets `SYLPH_AUTONOMOUS=1` and
|
||||
the entrypoint adds the flag.
|
||||
|
||||
Auth comes from the host `~/.claude`, bind-mounted read-write (token refresh
|
||||
needs to write). **That directory also holds your memory and project state**, so
|
||||
the container agent and you share it. Point `SYLPH_CLAUDE_HOME` at a separate
|
||||
directory to isolate it, or set `ANTHROPIC_API_KEY` instead.
|
||||
|
||||
## Host prerequisites
|
||||
|
||||
* **A Vulkan SDK** (LunarG), for *building* only. Canary's shader step calls
|
||||
`spirv-opt --canonicalize-ids`, which Ubuntu's packaged SPIRV-Tools (v2025.1)
|
||||
does not have — the build then dies ~500 objects in, and the error you see is
|
||||
a Python `TypeError`, not the real message. The launcher mounts the host's SDK
|
||||
read-only at its own path and sets `VULKAN_SDK`; that also guarantees the
|
||||
container produces byte-identical shaders to a host build.
|
||||
* **`nvidia-container-toolkit`**, for hardware Vulkan — see below.
|
||||
|
||||
## Things that will waste your afternoon
|
||||
|
||||
Each of these was hit while bringing this container up.
|
||||
|
||||
* **An unknown xenia flag hangs; it does not error.** `ParseLaunchArguments`
|
||||
calls `ShowSimpleMessageBox` *before logging is initialised*, and that SDL
|
||||
dialog blocks on `XIfEvent` forever with nobody to click it. The symptom is a
|
||||
10×10 window, a completely empty log and no guest memory — which reads like a
|
||||
hang deep in the emulator. `--audio` is **not** a cvar in this tree despite
|
||||
appearing in the RE notes; `--apu=sdl` is the real one. If Canary appears to
|
||||
hang at startup, suspect a typo'd flag first.
|
||||
* **`/dev/shm` must be `exec`.** Docker mounts it `noexec`, and xenia maps its
|
||||
JIT code cache out of a shm file. With `noexec` it dies at startup with
|
||||
"Unable to allocate code cache generated code storage / Cannot initalize
|
||||
processor", which reads like an address-space clash. The launcher uses
|
||||
`--tmpfs /dev/shm:rw,exec,…` rather than `--shm-size`.
|
||||
* **gdb needs root inside the container.** `--cap-add SYS_PTRACE` is passed, but
|
||||
the *host's* `kernel.yama.ptrace_scope=1` still blocks attaching to a
|
||||
non-descendant. Use `sudo gdb -p <pid>` (passwordless), or launch the target
|
||||
under gdb so it is a child.
|
||||
* **Named volumes need their mount points to exist in the image**, or Docker
|
||||
creates them root-owned and the first write fails obscurely.
|
||||
|
||||
## Known limitation
|
||||
|
||||
`build-reborn ci` runs the native legs only. `just ci`'s wasm check does not
|
||||
build, for a pre-existing reason unrelated to any change under test: the
|
||||
workspace pins `tokio = { features = ["full"] }`, which pulls `mio`, which
|
||||
refuses to compile for `wasm32-unknown-unknown`.
|
||||
52
docker/agent/bin/build-canary
Executable file
52
docker/agent/bin/build-canary
Executable file
@@ -0,0 +1,52 @@
|
||||
#!/usr/bin/env bash
|
||||
# Configure + build Xenia Canary inside the container.
|
||||
#
|
||||
# The build directory is $XENIA_BUILD_DIR (outside the bind-mounted repo) on
|
||||
# purpose. The host builds this same tree, and CMake caches an absolute compiler
|
||||
# path and a generator: sharing repo/build between host and container makes each
|
||||
# one reconfigure and relink everything the other just did.
|
||||
#
|
||||
# Parallelism comes from $SYLPH_JOBS, which the entrypoint derives from
|
||||
# AVAILABLE MEMORY as well as core count — a full-parallel build of this tree
|
||||
# has OOM-killed the host outright.
|
||||
#
|
||||
# build-canary [Release|Debug] [extra cmake --build args]
|
||||
set -euo pipefail
|
||||
|
||||
CONFIG="${1:-Release}"; shift || true
|
||||
SRC="${PROJECT_DIR:-/work}/xenia-canary"
|
||||
BUILD="${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}"
|
||||
JOBS="${SYLPH_JOBS:-2}"
|
||||
|
||||
[ -d "$SRC" ] || { echo "build-canary: no source at $SRC" >&2; exit 1; }
|
||||
|
||||
# Submodules: this tree has drifted before, and a checkout that changes a
|
||||
# gitlink fails silently into a half-built third_party. Report rather than fix,
|
||||
# because one submodule here carries an in-tree cmake build whose untracked
|
||||
# artifacts block an update.
|
||||
if ! git -C "$SRC" submodule status --recursive 2>/dev/null | grep -qv '^ '; then
|
||||
:
|
||||
else
|
||||
echo "build-canary: note — submodules are not all at their recorded commits:" >&2
|
||||
git -C "$SRC" submodule status 2>/dev/null | grep -v '^ ' | sed 's/^/ /' >&2
|
||||
fi
|
||||
|
||||
if [ ! -f "$BUILD/CMakeCache.txt" ]; then
|
||||
echo "==> configuring $BUILD ($CONFIG, Ninja Multi-Config, clang $(clang --version | head -1 | grep -oE '[0-9]+\.[0-9]+\.[0-9]+'))"
|
||||
cmake -S "$SRC" -B "$BUILD" -G "Ninja Multi-Config" \
|
||||
-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \
|
||||
-DXENIA_BUILD_TESTS=OFF -DXENIA_BUILD_MISC=OFF \
|
||||
-DXENIA_ENABLE_LTO=OFF
|
||||
fi
|
||||
|
||||
echo "==> building $CONFIG with -j$JOBS"
|
||||
cmake --build "$BUILD" --config "$CONFIG" --parallel "$JOBS" --target xenia_canary "$@"
|
||||
|
||||
BIN="$BUILD/bin/Linux/$CONFIG/xenia_canary"
|
||||
if [ -x "$BIN" ]; then
|
||||
echo "==> $BIN"
|
||||
echo " run it with: run-canary"
|
||||
else
|
||||
echo "build-canary: target did not produce $BIN" >&2
|
||||
exit 1
|
||||
fi
|
||||
47
docker/agent/bin/build-reborn
Executable file
47
docker/agent/bin/build-reborn
Executable file
@@ -0,0 +1,47 @@
|
||||
#!/usr/bin/env bash
|
||||
# Build / test Sylpheed Reborn inside the container.
|
||||
#
|
||||
# CARGO_TARGET_DIR points outside the bind-mounted repo (see the Dockerfile), so
|
||||
# this never fights the host's incremental state.
|
||||
#
|
||||
# build-reborn cargo build --workspace
|
||||
# build-reborn test cargo test --workspace, disc tests enabled
|
||||
# build-reborn ci fmt + clippy + test
|
||||
# build-reborn <cargo args...>
|
||||
set -euo pipefail
|
||||
|
||||
SRC="${PROJECT_DIR:-/work}/Syplheed-Reborn"
|
||||
JOBS="${SYLPH_JOBS:-2}"
|
||||
cd "$SRC"
|
||||
|
||||
# The disc-gated integration tests self-skip when this is unset, and a green run
|
||||
# then means almost nothing — point them at the extracted disc if it is there.
|
||||
if [ -z "${SYLPHEED_DISC:-}" ]; then
|
||||
for c in "${PROJECT_DIR:-/work}/sylph_extract" "$SRC/../sylph_extract"; do
|
||||
[ -d "$c/dat" ] && { export SYLPHEED_DISC="$(readlink -f "$c")"; break; }
|
||||
done
|
||||
fi
|
||||
[ -n "${SYLPHEED_DISC:-}" ] && export SYLPHEED_RES3D="$SYLPHEED_DISC/hidden/resource3d"
|
||||
if [ -z "${SYLPHEED_ISO:-}" ]; then
|
||||
iso="$(find "${PROJECT_DIR:-/work}" -maxdepth 2 -iname '*.iso' -print -quit 2>/dev/null || true)"
|
||||
[ -n "$iso" ] && export SYLPHEED_ISO="$iso"
|
||||
fi
|
||||
echo "==> SYLPHEED_DISC=${SYLPHEED_DISC:-<unset — disc tests will SKIP>}" >&2
|
||||
|
||||
export CARGO_BUILD_JOBS="$JOBS"
|
||||
|
||||
case "${1:-build}" in
|
||||
build) shift || true; exec cargo build --workspace "$@" ;;
|
||||
test) shift || true; exec cargo test --workspace "$@" ;;
|
||||
ci)
|
||||
cargo fmt --all -- --check
|
||||
cargo clippy --workspace -- -D warnings
|
||||
cargo test --workspace
|
||||
# NOTE: `just ci` also checks wasm32. That leg does not build, and not for
|
||||
# any reason in this crate: the workspace pins tokio with features=["full"],
|
||||
# which pulls mio, which refuses to compile for wasm32. Left out here rather
|
||||
# than reported as a failure of the change under test.
|
||||
echo "==> native CI green (wasm leg skipped — see the note in this script)"
|
||||
;;
|
||||
*) exec cargo "$@" ;;
|
||||
esac
|
||||
99
docker/agent/bin/run-canary
Executable file
99
docker/agent/bin/run-canary
Executable file
@@ -0,0 +1,99 @@
|
||||
#!/usr/bin/env bash
|
||||
# Launch Xenia Canary with the settings this title actually needs.
|
||||
#
|
||||
# Four of these are not preferences — they are measured requirements, and every
|
||||
# one of them cost a debugging session before it was pinned down:
|
||||
#
|
||||
# --apu=sdl + SDL_AUDIODRIVER=dummy
|
||||
# There is no PulseAudio here, so `--apu=nop` looks like the safe muted
|
||||
# choice. It is not: the log then fills with "CreateDriver failed for
|
||||
# index=0", the guest never gets past the intro movie, and the window
|
||||
# stays black for 8+ minutes. The SDL driver against a dummy device is
|
||||
# both silent AND lets the title advance.
|
||||
#
|
||||
# NO --audio flag
|
||||
# The RE notes say "--audio --apu=sdl". `--audio` is NOT a cvar in this
|
||||
# tree, and an unknown argument is not a friendly error: xenia calls
|
||||
# ShowSimpleMessageBox from ParseLaunchArguments, BEFORE logging is
|
||||
# initialised, and that SDL dialog blocks on XIfEvent forever. Headless,
|
||||
# the symptom is a 10x10 window, an empty log, and no guest memory —
|
||||
# which reads like a hang deep in the emulator rather than a typo.
|
||||
# If this ever appears to hang at startup, suspect a bad flag first.
|
||||
#
|
||||
# --hid=file --pad_file=...
|
||||
# The old vgamepad path made its device through /dev/uinput, which is NOT
|
||||
# namespaced — a pad created inside a container registers with the HOST's
|
||||
# input stack and every scripted press leaks to the user's desktop. This
|
||||
# driver reads a text file instead. Drive it with tools/re-capture/pad.py.
|
||||
# Trap worth remembering: 360 menus poll XamInputGetKeystrokeEx, not
|
||||
# GetState, so a stubbed GetKeystroke looks like a completely dead pad.
|
||||
#
|
||||
# one instance at a time
|
||||
# Two emulators (or ours + canary) at once perturbs both and the box.
|
||||
# Enforced with a lockfile rather than left to discipline.
|
||||
#
|
||||
# Usage: run-canary [extra xenia flags...]
|
||||
# ISO from $SYLPH_ISO, else the first *.iso under $PROJECT_DIR.
|
||||
# Binary from $XENIA_BIN, else the container build, else the repo build.
|
||||
set -u
|
||||
|
||||
LOCK=/tmp/xenia-canary.lock
|
||||
exec 9>"$LOCK"
|
||||
if ! flock -n 9; then
|
||||
echo "run-canary: an emulator is already running (lock $LOCK)." >&2
|
||||
echo " Only one at a time — kill it first: pkill -x xenia_canary" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
PROJECT_DIR="${PROJECT_DIR:-/work}"
|
||||
|
||||
# ── Binary ───────────────────────────────────────────────────────────────────
|
||||
pick_bin() {
|
||||
[ -n "${XENIA_BIN:-}" ] && { echo "$XENIA_BIN"; return; }
|
||||
for c in \
|
||||
"${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}/bin/Linux/Release/xenia_canary" \
|
||||
"${XENIA_BUILD_DIR:-/sylph-home/re/canary-build}/bin/Linux/Debug/xenia_canary" \
|
||||
"$PROJECT_DIR/xenia-canary/build/bin/Linux/Release/xenia_canary" \
|
||||
"$PROJECT_DIR/xenia-canary/build/bin/Linux/Debug/xenia_canary"; do
|
||||
[ -x "$c" ] && { echo "$c"; return; }
|
||||
done
|
||||
}
|
||||
BIN="$(pick_bin)"
|
||||
if [ -z "${BIN:-}" ]; then
|
||||
echo "run-canary: no xenia_canary binary found. Build one with: build-canary" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ── ISO ──────────────────────────────────────────────────────────────────────
|
||||
ISO="${SYLPH_ISO:-}"
|
||||
if [ -z "$ISO" ]; then
|
||||
# Prefer a REAL file over a symlink and take the largest: the tree carries
|
||||
# `xenia-rs/sylpheed.iso` as a symlink to the retail image, and a symlink has
|
||||
# already cost a session once (Wine could not resolve it -> "path invalid").
|
||||
ISO="$(find "$PROJECT_DIR" -maxdepth 2 -type f -iname '*.iso' -printf '%s\t%p\n' 2>/dev/null \
|
||||
| sort -rn | head -1 | cut -f2-)"
|
||||
fi
|
||||
if [ -z "$ISO" ] || [ ! -f "$ISO" ]; then
|
||||
echo "run-canary: no ISO. Set SYLPH_ISO=/path/to/game.iso" >&2
|
||||
exit 1
|
||||
fi
|
||||
ISO="$(readlink -f "$ISO")"
|
||||
|
||||
export SDL_AUDIODRIVER="${SDL_AUDIODRIVER:-dummy}"
|
||||
export DISPLAY="${DISPLAY:-:98}"
|
||||
PAD="${XENIA_PAD_FILE:-/tmp/xenia_pad.txt}"
|
||||
: > "$PAD"
|
||||
|
||||
# Guest memory is backed by /dev/shm; a stale file from a killed run confuses
|
||||
# the memory readers (gmem.py finds two candidates and picks the dead one).
|
||||
rm -f /dev/shm/xenia_memory_* /dev/shm/xenia_code_cache_* 2>/dev/null || true
|
||||
|
||||
echo "run-canary: $BIN" >&2
|
||||
echo " iso: $ISO" >&2
|
||||
echo " pad: $PAD display: $DISPLAY shm: $(df -h /dev/shm | awk 'NR==2{print $2}')" >&2
|
||||
|
||||
exec "$BIN" "$ISO" \
|
||||
--apu=sdl \
|
||||
--hid=file --pad_file="$PAD" \
|
||||
--mute=true \
|
||||
"$@"
|
||||
51
docker/agent/bin/screenshot
Executable file
51
docker/agent/bin/screenshot
Executable file
@@ -0,0 +1,51 @@
|
||||
#!/usr/bin/env bash
|
||||
# Raw full-root PNG grab of the headless display.
|
||||
#
|
||||
# This is deliberately the *uncropped* root window, because
|
||||
# `tools/re-capture/bin/screenshot` is a wrapper that calls this one as its raw
|
||||
# grabber and then crops to the game surface — xenia's window is a GTK window
|
||||
# whose menu bar pushes the 1280x720 game image down ~25 px, and every pixel
|
||||
# oracle in the toolkit was measured against the bare game image. That wrapper
|
||||
# directory is first on PATH, so scripts calling `screenshot` get the cropped
|
||||
# game surface and this stays the honest raw grab underneath it.
|
||||
#
|
||||
# screenshot [out.png] default: $HOME/shots/shot-NNNN.png
|
||||
set -u
|
||||
|
||||
OUT="${1:-}"
|
||||
if [ -z "$OUT" ]; then
|
||||
dir="${HOME:-/tmp}/shots"; mkdir -p "$dir"
|
||||
n_file="$dir/.counter"
|
||||
n=$(( $(cat "$n_file" 2>/dev/null || echo 0) + 1 )); echo "$n" > "$n_file"
|
||||
OUT="$dir/shot-$(printf '%04d' "$n").png"
|
||||
fi
|
||||
mkdir -p "$(dirname "$OUT")"
|
||||
|
||||
if ! xdpyinfo >/dev/null 2>&1; then
|
||||
echo "screenshot: no display on ${DISPLAY:-<unset>}" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ImageMagick first. `import` talks X11 directly and captures a window that is
|
||||
# mid-redraw without tearing the way a video grabber can.
|
||||
if command -v import >/dev/null 2>&1 && import -silent -window root "$OUT" 2>/dev/null; then
|
||||
echo "$OUT"; exit 0
|
||||
fi
|
||||
|
||||
# Fallback: ffmpeg's x11grab. Needs an explicit size, so read it off the server
|
||||
# rather than assuming the geometry.
|
||||
if command -v ffmpeg >/dev/null 2>&1; then
|
||||
size=$(xdpyinfo | awk '/dimensions:/{print $2; exit}')
|
||||
if ffmpeg -loglevel error -y -f x11grab -draw_mouse 0 \
|
||||
-video_size "$size" -i "$DISPLAY" -frames:v 1 "$OUT" 2>/dev/null; then
|
||||
echo "$OUT"; exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
# Last resort: xwd, which is always present with x11-utils.
|
||||
if command -v xwd >/dev/null 2>&1 && command -v convert >/dev/null 2>&1; then
|
||||
xwd -root -silent | convert xwd:- "$OUT" && { echo "$OUT"; exit 0; }
|
||||
fi
|
||||
|
||||
echo "screenshot: no working capture backend" >&2
|
||||
exit 1
|
||||
88
docker/agent/bin/sylph-doctor
Executable file
88
docker/agent/bin/sylph-doctor
Executable file
@@ -0,0 +1,88 @@
|
||||
#!/usr/bin/env bash
|
||||
# Prove the container can actually do the four things it exists for, before an
|
||||
# unattended agent spends an hour discovering otherwise.
|
||||
#
|
||||
# Every check here stands for a failure that has already happened once: a
|
||||
# display that was not there, a missing numpy that looked like a logic bug, a
|
||||
# /dev/shm too small for guest memory, a Vulkan stack with no ICD.
|
||||
set -u
|
||||
fail=0
|
||||
ok() { printf ' \033[32m✔\033[0m %s\n' "$*"; }
|
||||
bad() { printf ' \033[31m✖\033[0m %s\n' "$*"; fail=$((fail+1)); }
|
||||
warn() { printf ' \033[33m!\033[0m %s\n' "$*"; }
|
||||
|
||||
echo "── resources ──"
|
||||
# nproc shows the HOST's cores: --cpus is a quota, not a mask. Report both so
|
||||
# "12 cpus" is never mistaken for 12 cpus' worth of throughput.
|
||||
quota="unlimited"
|
||||
if [ -r /sys/fs/cgroup/cpu.max ]; then
|
||||
read -r q p < /sys/fs/cgroup/cpu.max
|
||||
[ "$q" != max ] && quota="$(( q / p )) (quota)"
|
||||
fi
|
||||
echo " cpus: $(nproc) visible, $quota"
|
||||
if [ -r /sys/fs/cgroup/memory.max ]; then
|
||||
m=$(cat /sys/fs/cgroup/memory.max)
|
||||
[ "$m" = max ] && warn "memory: UNLIMITED — the half-the-box cap is not applied" \
|
||||
|| ok "memory cap: $(( m / 1024 / 1024 / 1024 )) GiB"
|
||||
fi
|
||||
shm=$(df -BM /dev/shm | awk 'NR==2{print $2}' | tr -d M)
|
||||
# Guest memory for a 512 MB console plus the code cache does not fit in
|
||||
# Docker's 64 MB default, and the symptom is an mmap error, not a disk-full one.
|
||||
[ "${shm:-0}" -ge 512 ] && ok "/dev/shm: ${shm} MiB" || bad "/dev/shm only ${shm:-?} MiB — need >=512; pass --shm-size"
|
||||
|
||||
echo "── toolchain ──"
|
||||
for t in clang clang++ cmake ninja cargo rustc python3 node claude; do
|
||||
command -v "$t" >/dev/null && ok "$t ($("$t" --version 2>/dev/null | head -1))" || bad "$t missing"
|
||||
done
|
||||
|
||||
echo "── python (dynamic RE) ──"
|
||||
# numpy and PIL missing is the specific hole that silently disabled entities2.py
|
||||
# and every image oracle in the toolkit.
|
||||
for m in numpy PIL duckdb; do
|
||||
python3 -c "import $m" 2>/dev/null && ok "python: $m" || bad "python: $m MISSING"
|
||||
done
|
||||
|
||||
echo "── display ──"
|
||||
if xdpyinfo >/dev/null 2>&1; then
|
||||
ok "display $DISPLAY ($(xdpyinfo | awk '/dimensions:/{print $2; exit}'))"
|
||||
pgrep -x openbox >/dev/null && ok "openbox running" || warn "no window manager — window geometry oracles will misread"
|
||||
out=$(screenshot /tmp/_doctor.png 2>&1) && [ -s /tmp/_doctor.png ] \
|
||||
&& ok "screenshot works -> $(identify -format '%wx%h' /tmp/_doctor.png 2>/dev/null || echo ok)" \
|
||||
|| bad "screenshot failed: $out"
|
||||
rm -f /tmp/_doctor.png
|
||||
else
|
||||
bad "no display on ${DISPLAY:-<unset>}"
|
||||
fi
|
||||
|
||||
echo "── vulkan ──"
|
||||
if command -v vulkaninfo >/dev/null 2>&1; then
|
||||
dev=$(vulkaninfo --summary 2>/dev/null | grep -m3 -E 'deviceName' | sed 's/^ *//')
|
||||
[ -n "$dev" ] && { ok "Vulkan devices:"; echo "$dev" | sed 's/^/ /'; } \
|
||||
|| bad "vulkaninfo found no device (ICD missing?)"
|
||||
else
|
||||
bad "vulkaninfo missing"
|
||||
fi
|
||||
# Judge by what enumerated, not by whether a device node is present: an NVIDIA
|
||||
# card needs the NVIDIA Container Toolkit, and /dev/dri alone does nothing.
|
||||
case "${dev:-}" in
|
||||
*llvmpipe*|*lavapipe*)
|
||||
warn "SOFTWARE Vulkan only — correct but slow."
|
||||
command -v nvidia-smi >/dev/null 2>&1 \
|
||||
&& warn " host has an NVIDIA GPU: install nvidia-container-toolkit for hardware" ;;
|
||||
"") ;;
|
||||
*) ok "hardware Vulkan" ;;
|
||||
esac
|
||||
|
||||
echo "── project ──"
|
||||
[ -d /work/xenia-canary ] && ok "/work/xenia-canary" || bad "/work/xenia-canary not mounted"
|
||||
[ -d /work/Syplheed-Reborn ] && ok "/work/Syplheed-Reborn" || bad "/work/Syplheed-Reborn not mounted"
|
||||
iso=$(find /work -maxdepth 2 -type f -iname '*.iso' -printf '%s\t%p\n' 2>/dev/null | sort -rn | head -1 | cut -f2-)
|
||||
[ -n "$iso" ] && ok "ISO: $iso" || warn "no ISO under /work — run-canary needs SYLPH_ISO"
|
||||
[ -d /work/sylph_extract/dat ] && ok "extracted disc (disc-gated tests will run)" \
|
||||
|| warn "no extracted disc — Reborn disc tests will SKIP"
|
||||
[ -w /sylph-home/re/.claude ] && ok "~/.claude writable (token refresh works)" \
|
||||
|| warn "~/.claude not writable — Claude Code may fail to refresh auth"
|
||||
|
||||
echo
|
||||
[ "$fail" -eq 0 ] && { echo "all good."; exit 0; }
|
||||
echo "$fail check(s) failed."; exit 1
|
||||
103
docker/agent/entrypoint.sh
Executable file
103
docker/agent/entrypoint.sh
Executable file
@@ -0,0 +1,103 @@
|
||||
#!/usr/bin/env bash
|
||||
# Bring up the headless display, then hand over to the command.
|
||||
#
|
||||
# Xvfb and openbox are started HERE, as children of PID 1 (tini), rather than by
|
||||
# the toolkit scripts. That is the fix for the long-standing "Xvfb and the
|
||||
# emulator die on their own every few minutes" note: nothing owned those
|
||||
# processes, so nothing kept them alive, and a run could sit for 300 s in front
|
||||
# of a visible MAIN MENU reporting "no main menu" because the display had gone.
|
||||
# A container-lifetime display makes that failure mode impossible.
|
||||
set -euo pipefail
|
||||
|
||||
log() { printf '[entrypoint] %s\n' "$*" >&2; }
|
||||
|
||||
DISPLAY="${DISPLAY:-:98}"
|
||||
GEOM="${SCREEN_GEOMETRY:-1280x720x24}"
|
||||
export DISPLAY
|
||||
|
||||
# ── Display ──────────────────────────────────────────────────────────────────
|
||||
if ! xdpyinfo -display "$DISPLAY" >/dev/null 2>&1; then
|
||||
rm -f "/tmp/.X${DISPLAY#:}-lock" "/tmp/.X11-unix/X${DISPLAY#:}" 2>/dev/null || true
|
||||
# GLX and RANDR are both required: Canary's window is GTK+OpenGL even when the
|
||||
# graphics backend is Vulkan, and xwininfo-based screen oracles need RANDR.
|
||||
Xvfb "$DISPLAY" -screen 0 "$GEOM" -ac -nolisten tcp \
|
||||
+extension GLX +extension RANDR >/tmp/xvfb.log 2>&1 &
|
||||
for _ in $(seq 1 50); do
|
||||
xdpyinfo -display "$DISPLAY" >/dev/null 2>&1 && break
|
||||
sleep 0.2
|
||||
done
|
||||
fi
|
||||
if ! xdpyinfo -display "$DISPLAY" >/dev/null 2>&1; then
|
||||
log "FATAL: no display on $DISPLAY — see /tmp/xvfb.log"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# A window manager is not cosmetic here: without one the emulator window is
|
||||
# never mapped at a known position, and every pixel oracle reads the wrong rows.
|
||||
if ! pgrep -x openbox >/dev/null 2>&1; then
|
||||
openbox >/tmp/openbox.log 2>&1 &
|
||||
sleep 0.5
|
||||
fi
|
||||
log "display $DISPLAY ready ($GEOM)"
|
||||
|
||||
# ── Vulkan ───────────────────────────────────────────────────────────────────
|
||||
# Prefer the real GPU when /dev/dri was passed through; fall back to lavapipe,
|
||||
# which is slow but correct and needs no host device.
|
||||
LVP=$(ls /usr/share/vulkan/icd.d/lvp_icd*.json 2>/dev/null | head -1)
|
||||
if [ "${SYLPH_VULKAN:-auto}" = "sw" ] || [ ! -e /dev/dri/renderD128 ]; then
|
||||
[ -n "$LVP" ] && export VK_ICD_FILENAMES="$LVP"
|
||||
else
|
||||
unset LIBGL_ALWAYS_SOFTWARE
|
||||
fi
|
||||
# Report what Vulkan ACTUALLY enumerated, not what we asked for. Announcing
|
||||
# "host GPU via /dev/dri" because the device node exists is how an agent ends up
|
||||
# believing it has hardware while running llvmpipe — an NVIDIA card needs the
|
||||
# NVIDIA Container Toolkit, and /dev/dri alone does nothing for it.
|
||||
if command -v vulkaninfo >/dev/null 2>&1; then
|
||||
vkdev=$(vulkaninfo --summary 2>/dev/null | awk -F= '/deviceName/{gsub(/^ +| +$/,"",$2); print $2; exit}')
|
||||
case "${vkdev:-}" in
|
||||
"") log "Vulkan: NO DEVICE — vulkaninfo enumerated nothing" ;;
|
||||
llvmpipe*|lavapipe*) log "Vulkan: $vkdev (SOFTWARE — correct but slow)" ;;
|
||||
*) log "Vulkan: $vkdev (hardware)" ;;
|
||||
esac
|
||||
fi
|
||||
|
||||
# ── Build parallelism ────────────────────────────────────────────────────────
|
||||
# Bounded by MEMORY, not just cores. A full-parallel build of this tree has
|
||||
# OOM-killed the host outright, and inside a half-the-box container the ceiling
|
||||
# is lower still. ~1.5 GiB per C++ TU is the rule of thumb that has held.
|
||||
# nproc reports the HOST's core count: --cpus is a CFS quota, not a mask. Using
|
||||
# it would oversubscribe a half-the-box container by exactly 2x, so read the
|
||||
# quota the cgroup actually grants.
|
||||
cpus=$(nproc)
|
||||
if [ -r /sys/fs/cgroup/cpu.max ]; then
|
||||
read -r _q _p < /sys/fs/cgroup/cpu.max || true
|
||||
if [ "${_q:-max}" != max ] && [ "${_p:-0}" -gt 0 ]; then
|
||||
cpus=$(( (_q + _p - 1) / _p ))
|
||||
[ "$cpus" -lt 1 ] && cpus=1
|
||||
fi
|
||||
fi
|
||||
mem_gib=$(awk '/MemAvailable/{printf "%d", $2/1048576}' /proc/meminfo)
|
||||
# MemAvailable is the HOST's too under cgroup v2; prefer the container's cap.
|
||||
if [ -r /sys/fs/cgroup/memory.max ]; then
|
||||
_m=$(cat /sys/fs/cgroup/memory.max)
|
||||
[ "$_m" != max ] && mem_gib=$(( _m / 1073741824 ))
|
||||
fi
|
||||
[ "${mem_gib:-0}" -lt 1 ] && mem_gib=1
|
||||
by_mem=$(( mem_gib * 2 / 3 ))
|
||||
[ "$by_mem" -lt 1 ] && by_mem=1
|
||||
jobs=$(( cpus < by_mem ? cpus : by_mem ))
|
||||
export SYLPH_JOBS="$jobs" CARGO_BUILD_JOBS="$jobs" CMAKE_BUILD_PARALLEL_LEVEL="$jobs"
|
||||
log "build parallelism: $jobs (cpus=$cpus, mem=${mem_gib}GiB avail)"
|
||||
|
||||
mkdir -p "$HOME/shots" "$HOME/logs"
|
||||
|
||||
# ── Claude Code ──────────────────────────────────────────────────────────────
|
||||
if [ "${SYLPH_AUTONOMOUS:-0}" = "1" ]; then
|
||||
# The flag the user asked for. It is refused under root, which is why this
|
||||
# image runs as `agent`.
|
||||
set -- claude --dangerously-skip-permissions "$@"
|
||||
log "starting Claude Code with --dangerously-skip-permissions"
|
||||
fi
|
||||
|
||||
exec "$@"
|
||||
174
docker/agent/sylph-agent
Executable file
174
docker/agent/sylph-agent
Executable file
@@ -0,0 +1,174 @@
|
||||
#!/usr/bin/env bash
|
||||
# Host-side launcher for the Sylpheed RE agent container.
|
||||
#
|
||||
# Caps the container at HALF the machine's CPUs and memory, computed at run time
|
||||
# so it stays half on whatever box it lands on.
|
||||
#
|
||||
# ./sylph-agent build build (or rebuild) the image
|
||||
# ./sylph-agent shell interactive shell in the container
|
||||
# ./sylph-agent agent [prompt] Claude Code, --dangerously-skip-permissions
|
||||
# ./sylph-agent run <cmd...> one-shot command
|
||||
# ./sylph-agent stop stop a detached container
|
||||
#
|
||||
# Environment:
|
||||
# SYLPH_PROJECT host project root (default: three levels up from this file)
|
||||
# SYLPH_CLAUDE_HOME host dir mounted as the agent's ~/.claude
|
||||
# (default: $HOME/.claude — shares auth AND memory with you)
|
||||
# SYLPH_VULKAN=sw force software Vulkan (lavapipe) even if /dev/dri exists
|
||||
# SYLPH_CPUS / SYLPH_MEM_GB override the computed half
|
||||
set -euo pipefail
|
||||
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
IMAGE="${SYLPH_IMAGE:-sylpheed-agent:latest}"
|
||||
NAME="${SYLPH_NAME:-sylpheed-agent}"
|
||||
PROJECT="${SYLPH_PROJECT:-$(cd "$HERE/../../.." && pwd)}"
|
||||
|
||||
# ── Half the box ─────────────────────────────────────────────────────────────
|
||||
# LC_ALL=C is required, not tidiness: under a locale with a comma decimal
|
||||
# separator (de_DE and friends) awk prints "6,0" and docker rejects it as
|
||||
# --cpus with "failed to parse as a rational number".
|
||||
HOST_CPUS=$(nproc)
|
||||
HOST_MEM_KB=$(awk '/MemTotal/{print $2}' /proc/meminfo)
|
||||
CPUS="${SYLPH_CPUS:-$(LC_ALL=C awk -v c="$HOST_CPUS" 'BEGIN{printf "%.1f", c/2}')}"
|
||||
MEM_GB="${SYLPH_MEM_GB:-$(LC_ALL=C awk -v m="$HOST_MEM_KB" 'BEGIN{printf "%d", m/1048576/2}')}"
|
||||
[ "$MEM_GB" -lt 2 ] && MEM_GB=2
|
||||
# /dev/shm holds the emulator's guest memory (gmem.py reads it there). Docker's
|
||||
# 64 MB default is far too small for a 512 MB console address space, and the
|
||||
# failure is an obscure mmap error rather than an out-of-space message. tmpfs
|
||||
# pages count against the memory cap, so take a third of it and no more.
|
||||
SHM_GB=$(( MEM_GB / 3 )); [ "$SHM_GB" -lt 1 ] && SHM_GB=1
|
||||
|
||||
usage() { sed -n '2,20p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'; exit "${1:-0}"; }
|
||||
|
||||
docker_args() {
|
||||
local -n _out=$1
|
||||
_out=(
|
||||
--name "$NAME"
|
||||
--hostname sylph-agent
|
||||
# ── the cap ──
|
||||
--cpus "$CPUS"
|
||||
--memory "${MEM_GB}g"
|
||||
--memory-swap "${MEM_GB}g" # no swap escape hatch: a swapping build
|
||||
# thrashes the whole host, which is the
|
||||
# failure this cap exists to prevent
|
||||
--pids-limit 4096
|
||||
# /dev/shm as an EXEC-capable tmpfs, not --shm-size. Docker's default mounts
|
||||
# it `noexec`, and xenia maps its JIT code cache out of a shm file at a fixed
|
||||
# address — so with noexec it dies at startup with "Unable to allocate code
|
||||
# cache generated code storage / Cannot initalize processor", which reads
|
||||
# like an address-space clash rather than a mount flag.
|
||||
--tmpfs "/dev/shm:rw,exec,nosuid,nodev,size=${SHM_GB}g"
|
||||
# Dynamic RE needs to attach to a live process: without SYS_PTRACE, gdb and
|
||||
# strace are installed but inert ("Could not attach to process"), and the
|
||||
# container's whole reason for existing is watching the emulator run.
|
||||
# Docker's default seccomp profile also blocks calls the JIT and the guest
|
||||
# memory mapper rely on.
|
||||
--cap-add SYS_PTRACE
|
||||
--security-opt seccomp=unconfined
|
||||
--security-opt apparmor=unconfined
|
||||
# ── project ──
|
||||
-v "$PROJECT:/work"
|
||||
-e "PROJECT_DIR=/work"
|
||||
# ── claude ──
|
||||
-v "${SYLPH_CLAUDE_HOME:-$HOME/.claude}:/sylph-home/re/.claude"
|
||||
# persistent build caches, so a container restart is not a rebuild
|
||||
-v sylph-agent-cargo:/sylph-home/re/.cargo
|
||||
-v sylph-agent-target:/sylph-home/re/target-container
|
||||
-v sylph-agent-canary-build:/sylph-home/re/canary-build
|
||||
)
|
||||
# ── Vulkan SDK ──
|
||||
# xenia's shader step calls `spirv-opt --canonicalize-ids`, which Ubuntu's
|
||||
# packaged SPIRV-Tools (v2025.1) does not have — the build then dies ~500
|
||||
# objects in. The LunarG SDK has it. Mounting the host's copy at the same path
|
||||
# is cheaper than baking a 200 MB SDK into the image AND guarantees the
|
||||
# container produces byte-identical shaders to the host build.
|
||||
SDK="${VULKAN_SDK:-}"
|
||||
if [ -z "$SDK" ]; then
|
||||
SDK=$(ls -d "$HOME"/vulkan-sdk/*/x86_64 2>/dev/null | sort -V | tail -1 || true)
|
||||
fi
|
||||
if [ -n "$SDK" ] && [ -x "$SDK/bin/spirv-opt" ]; then
|
||||
_out+=(-v "$SDK:$SDK:ro" -e "VULKAN_SDK=$SDK")
|
||||
else
|
||||
echo "==> NOTE: no Vulkan SDK found on the host. Building Canary's shaders" >&2
|
||||
echo " needs spirv-opt with --canonicalize-ids (LunarG SDK); Ubuntu's" >&2
|
||||
echo " packaged SPIRV-Tools is too old. Running is unaffected." >&2
|
||||
fi
|
||||
|
||||
[ -n "${ANTHROPIC_API_KEY:-}" ] && _out+=(-e "ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY")
|
||||
[ -n "${SYLPH_VULKAN:-}" ] && _out+=(-e "SYLPH_VULKAN=$SYLPH_VULKAN")
|
||||
[ -n "${SYLPH_ISO:-}" ] && _out+=(-e "SYLPH_ISO=$SYLPH_ISO")
|
||||
|
||||
# ── GPU ──
|
||||
# Three distinct cases, and conflating them is how you end up believing you
|
||||
# have hardware Vulkan while actually running llvmpipe:
|
||||
#
|
||||
# NVIDIA needs the NVIDIA Container Toolkit (`--gpus all`). Passing
|
||||
# /dev/dri alone does NOT work — Mesa cannot drive an NVIDIA card,
|
||||
# and the proprietary userspace lives outside the image.
|
||||
# Mesa (AMD/Intel) works with a plain /dev/dri passthrough plus the
|
||||
# host's render/video GIDs.
|
||||
# neither software Vulkan (lavapipe): correct, and slow.
|
||||
if [ "${SYLPH_VULKAN:-auto}" = "sw" ]; then
|
||||
_out+=(-e SYLPH_VULKAN=sw)
|
||||
elif command -v nvidia-smi >/dev/null 2>&1 && nvidia-smi -L >/dev/null 2>&1; then
|
||||
if docker info --format '{{json .Runtimes}}' 2>/dev/null | grep -q nvidia; then
|
||||
_out+=(--gpus all)
|
||||
else
|
||||
echo "==> NOTE: NVIDIA GPU found but the NVIDIA Container Toolkit is not" >&2
|
||||
echo " installed, so hardware Vulkan is unavailable and the container" >&2
|
||||
echo " will use lavapipe (software — correct, slow). To enable it:" >&2
|
||||
echo " sudo apt install nvidia-container-toolkit \\" >&2
|
||||
echo " && sudo nvidia-ctk runtime configure --runtime=docker \\" >&2
|
||||
echo " && sudo systemctl restart docker" >&2
|
||||
_out+=(-e SYLPH_VULKAN=sw)
|
||||
fi
|
||||
elif [ -e /dev/dri/renderD128 ]; then
|
||||
_out+=(--device /dev/dri)
|
||||
for g in render video; do
|
||||
gid=$(getent group "$g" | cut -d: -f3 || true)
|
||||
[ -n "$gid" ] && _out+=(--group-add "$gid")
|
||||
done
|
||||
else
|
||||
_out+=(-e SYLPH_VULKAN=sw)
|
||||
fi
|
||||
}
|
||||
|
||||
case "${1:-}" in
|
||||
build)
|
||||
shift
|
||||
echo "==> building $IMAGE (uid $(id -u), gid $(id -g))"
|
||||
exec docker build -t "$IMAGE" \
|
||||
--build-arg "AGENT_UID=$(id -u)" --build-arg "AGENT_GID=$(id -g)" \
|
||||
"$@" "$HERE"
|
||||
;;
|
||||
|
||||
shell|agent|run)
|
||||
mode=$1; shift
|
||||
declare -a ARGS; docker_args ARGS
|
||||
echo "==> $mode | cpus=$CPUS mem=${MEM_GB}g shm=${SHM_GB}g (host: ${HOST_CPUS} cpus, $((HOST_MEM_KB/1048576))g)"
|
||||
echo "==> project: $PROJECT -> /work"
|
||||
docker rm -f "$NAME" >/dev/null 2>&1 || true
|
||||
# Allocate a TTY only when stdin actually is one: `docker run -it` fails
|
||||
# outright ("cannot attach stdin to a TTY-enabled container") under a
|
||||
# pipeline or a CI runner, which is exactly where `run` gets used.
|
||||
TTY=(-i); [ -t 0 ] && TTY=(-it)
|
||||
case "$mode" in
|
||||
shell) exec docker run --rm "${TTY[@]}" "${ARGS[@]}" "$IMAGE" bash ;;
|
||||
agent)
|
||||
# The flag the user asked for. Refused under root, which is why the
|
||||
# image runs as an unprivileged `agent` user.
|
||||
ARGS+=(-e SYLPH_AUTONOMOUS=1)
|
||||
exec docker run --rm "${TTY[@]}" "${ARGS[@]}" "$IMAGE" "$@"
|
||||
;;
|
||||
run) exec docker run --rm "${TTY[@]}" "${ARGS[@]}" "$IMAGE" "$@" ;;
|
||||
esac
|
||||
;;
|
||||
|
||||
stop) exec docker rm -f "$NAME" ;;
|
||||
doctor)
|
||||
declare -a ARGS; docker_args ARGS
|
||||
exec docker run --rm "${ARGS[@]}" "$IMAGE" sylph-doctor
|
||||
;;
|
||||
""|-h|--help) usage 0 ;;
|
||||
*) echo "unknown command: $1" >&2; usage 2 ;;
|
||||
esac
|
||||
Reference in New Issue
Block a user