64f9d15610
Builds and runs both halves of the project -- Canary as the oracle, Reborn as
the port -- plus the dynamic-RE toolkit. Claude Code runs with
--dangerously-skip-permissions as an unprivileged `agent` user, because that
flag is refused under root.
Capped at half the machine, computed at launch: --cpus nproc/2, --memory half
of MemTotal with --memory-swap equal to it (no swap escape hatch -- a swapping
build thrashes the host, which is the failure the cap exists to prevent), and
build parallelism derived INSIDE the container from available memory rather
than core count, since a full-parallel build of this tree has OOM-killed the
host outright.
Three things the old box got wrong are fixed rather than reproduced: a real
toolchain (so rebuild_canary.sh's hand-relinking is obsolete), numpy and Pillow
(whose absence silently disabled every image oracle and looked like a logic
bug), and a display owned by PID 1 (so Xvfb no longer "dies on its own every
few minutes" -- it was being reaped because nothing owned it).
Verified end to end, not by inspection: the image builds, sylph-doctor is green,
`build-canary` links xenia_canary inside the container, and that binary then
runs -- guest memory and the JIT code cache appear in /dev/shm within 4 s,
1 205 log lines, gmem.py reads guest RAM, pad.py drives the file pad, and
screenshot captures the display.
Five environment defects found and fixed on the way, each of which fails in a
way that points somewhere else entirely:
* /dev/shm is `noexec` under Docker. Xenia maps its JIT code cache out of an
shm file, so it died with "Unable to allocate code cache generated code
storage / Cannot initalize processor" -- which reads as an address-space
clash, not a mount flag. Now `--tmpfs /dev/shm:rw,exec`.
* An unknown xenia flag HANGS rather than errors: ParseLaunchArguments calls
ShowSimpleMessageBox before logging is initialised, and that SDL dialog
blocks on XIfEvent forever. `--audio` (which the RE notes recommend) is not
a cvar in this tree; the symptom was a 10x10 window and an empty log.
* Named volumes come up root-owned unless their mount point exists in the
image, so the first cmake configure failed on pkgRedirects.
* Ubuntu 24.04 ships its own uid-1000 account, colliding with the host user.
* Ubuntu's spirv-opt has no --canonicalize-ids, so the shader step dies ~500
objects in; the launcher mounts the host's LunarG SDK instead of baking one
in, which also keeps shader output byte-identical to a host build.
Known limits, stated rather than papered over: on an NVIDIA host without the
NVIDIA Container Toolkit there is no hardware Vulkan (/dev/dri alone does
nothing for NVIDIA), and under lavapipe the emulator runs correctly but was not
observed to reach a rendered frame within a couple of minutes. gdb needs `sudo`
inside the container because the host's yama ptrace_scope outranks SYS_PTRACE.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
175 lines
8.0 KiB
Bash
Executable File
175 lines
8.0 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Host-side launcher for the Sylpheed RE agent container.
|
|
#
|
|
# Caps the container at HALF the machine's CPUs and memory, computed at run time
|
|
# so it stays half on whatever box it lands on.
|
|
#
|
|
# ./sylph-agent build build (or rebuild) the image
|
|
# ./sylph-agent shell interactive shell in the container
|
|
# ./sylph-agent agent [prompt] Claude Code, --dangerously-skip-permissions
|
|
# ./sylph-agent run <cmd...> one-shot command
|
|
# ./sylph-agent stop stop a detached container
|
|
#
|
|
# Environment:
|
|
# SYLPH_PROJECT host project root (default: three levels up from this file)
|
|
# SYLPH_CLAUDE_HOME host dir mounted as the agent's ~/.claude
|
|
# (default: $HOME/.claude — shares auth AND memory with you)
|
|
# SYLPH_VULKAN=sw force software Vulkan (lavapipe) even if /dev/dri exists
|
|
# SYLPH_CPUS / SYLPH_MEM_GB override the computed half
|
|
set -euo pipefail
|
|
|
|
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
IMAGE="${SYLPH_IMAGE:-sylpheed-agent:latest}"
|
|
NAME="${SYLPH_NAME:-sylpheed-agent}"
|
|
PROJECT="${SYLPH_PROJECT:-$(cd "$HERE/../../.." && pwd)}"
|
|
|
|
# ── Half the box ─────────────────────────────────────────────────────────────
|
|
# LC_ALL=C is required, not tidiness: under a locale with a comma decimal
|
|
# separator (de_DE and friends) awk prints "6,0" and docker rejects it as
|
|
# --cpus with "failed to parse as a rational number".
|
|
HOST_CPUS=$(nproc)
|
|
HOST_MEM_KB=$(awk '/MemTotal/{print $2}' /proc/meminfo)
|
|
CPUS="${SYLPH_CPUS:-$(LC_ALL=C awk -v c="$HOST_CPUS" 'BEGIN{printf "%.1f", c/2}')}"
|
|
MEM_GB="${SYLPH_MEM_GB:-$(LC_ALL=C awk -v m="$HOST_MEM_KB" 'BEGIN{printf "%d", m/1048576/2}')}"
|
|
[ "$MEM_GB" -lt 2 ] && MEM_GB=2
|
|
# /dev/shm holds the emulator's guest memory (gmem.py reads it there). Docker's
|
|
# 64 MB default is far too small for a 512 MB console address space, and the
|
|
# failure is an obscure mmap error rather than an out-of-space message. tmpfs
|
|
# pages count against the memory cap, so take a third of it and no more.
|
|
SHM_GB=$(( MEM_GB / 3 )); [ "$SHM_GB" -lt 1 ] && SHM_GB=1
|
|
|
|
usage() { sed -n '2,20p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'; exit "${1:-0}"; }
|
|
|
|
docker_args() {
|
|
local -n _out=$1
|
|
_out=(
|
|
--name "$NAME"
|
|
--hostname sylph-agent
|
|
# ── the cap ──
|
|
--cpus "$CPUS"
|
|
--memory "${MEM_GB}g"
|
|
--memory-swap "${MEM_GB}g" # no swap escape hatch: a swapping build
|
|
# thrashes the whole host, which is the
|
|
# failure this cap exists to prevent
|
|
--pids-limit 4096
|
|
# /dev/shm as an EXEC-capable tmpfs, not --shm-size. Docker's default mounts
|
|
# it `noexec`, and xenia maps its JIT code cache out of a shm file at a fixed
|
|
# address — so with noexec it dies at startup with "Unable to allocate code
|
|
# cache generated code storage / Cannot initalize processor", which reads
|
|
# like an address-space clash rather than a mount flag.
|
|
--tmpfs "/dev/shm:rw,exec,nosuid,nodev,size=${SHM_GB}g"
|
|
# Dynamic RE needs to attach to a live process: without SYS_PTRACE, gdb and
|
|
# strace are installed but inert ("Could not attach to process"), and the
|
|
# container's whole reason for existing is watching the emulator run.
|
|
# Docker's default seccomp profile also blocks calls the JIT and the guest
|
|
# memory mapper rely on.
|
|
--cap-add SYS_PTRACE
|
|
--security-opt seccomp=unconfined
|
|
--security-opt apparmor=unconfined
|
|
# ── project ──
|
|
-v "$PROJECT:/work"
|
|
-e "PROJECT_DIR=/work"
|
|
# ── claude ──
|
|
-v "${SYLPH_CLAUDE_HOME:-$HOME/.claude}:/sylph-home/re/.claude"
|
|
# persistent build caches, so a container restart is not a rebuild
|
|
-v sylph-agent-cargo:/sylph-home/re/.cargo
|
|
-v sylph-agent-target:/sylph-home/re/target-container
|
|
-v sylph-agent-canary-build:/sylph-home/re/canary-build
|
|
)
|
|
# ── Vulkan SDK ──
|
|
# xenia's shader step calls `spirv-opt --canonicalize-ids`, which Ubuntu's
|
|
# packaged SPIRV-Tools (v2025.1) does not have — the build then dies ~500
|
|
# objects in. The LunarG SDK has it. Mounting the host's copy at the same path
|
|
# is cheaper than baking a 200 MB SDK into the image AND guarantees the
|
|
# container produces byte-identical shaders to the host build.
|
|
SDK="${VULKAN_SDK:-}"
|
|
if [ -z "$SDK" ]; then
|
|
SDK=$(ls -d "$HOME"/vulkan-sdk/*/x86_64 2>/dev/null | sort -V | tail -1 || true)
|
|
fi
|
|
if [ -n "$SDK" ] && [ -x "$SDK/bin/spirv-opt" ]; then
|
|
_out+=(-v "$SDK:$SDK:ro" -e "VULKAN_SDK=$SDK")
|
|
else
|
|
echo "==> NOTE: no Vulkan SDK found on the host. Building Canary's shaders" >&2
|
|
echo " needs spirv-opt with --canonicalize-ids (LunarG SDK); Ubuntu's" >&2
|
|
echo " packaged SPIRV-Tools is too old. Running is unaffected." >&2
|
|
fi
|
|
|
|
[ -n "${ANTHROPIC_API_KEY:-}" ] && _out+=(-e "ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY")
|
|
[ -n "${SYLPH_VULKAN:-}" ] && _out+=(-e "SYLPH_VULKAN=$SYLPH_VULKAN")
|
|
[ -n "${SYLPH_ISO:-}" ] && _out+=(-e "SYLPH_ISO=$SYLPH_ISO")
|
|
|
|
# ── GPU ──
|
|
# Three distinct cases, and conflating them is how you end up believing you
|
|
# have hardware Vulkan while actually running llvmpipe:
|
|
#
|
|
# NVIDIA needs the NVIDIA Container Toolkit (`--gpus all`). Passing
|
|
# /dev/dri alone does NOT work — Mesa cannot drive an NVIDIA card,
|
|
# and the proprietary userspace lives outside the image.
|
|
# Mesa (AMD/Intel) works with a plain /dev/dri passthrough plus the
|
|
# host's render/video GIDs.
|
|
# neither software Vulkan (lavapipe): correct, and slow.
|
|
if [ "${SYLPH_VULKAN:-auto}" = "sw" ]; then
|
|
_out+=(-e SYLPH_VULKAN=sw)
|
|
elif command -v nvidia-smi >/dev/null 2>&1 && nvidia-smi -L >/dev/null 2>&1; then
|
|
if docker info --format '{{json .Runtimes}}' 2>/dev/null | grep -q nvidia; then
|
|
_out+=(--gpus all)
|
|
else
|
|
echo "==> NOTE: NVIDIA GPU found but the NVIDIA Container Toolkit is not" >&2
|
|
echo " installed, so hardware Vulkan is unavailable and the container" >&2
|
|
echo " will use lavapipe (software — correct, slow). To enable it:" >&2
|
|
echo " sudo apt install nvidia-container-toolkit \\" >&2
|
|
echo " && sudo nvidia-ctk runtime configure --runtime=docker \\" >&2
|
|
echo " && sudo systemctl restart docker" >&2
|
|
_out+=(-e SYLPH_VULKAN=sw)
|
|
fi
|
|
elif [ -e /dev/dri/renderD128 ]; then
|
|
_out+=(--device /dev/dri)
|
|
for g in render video; do
|
|
gid=$(getent group "$g" | cut -d: -f3 || true)
|
|
[ -n "$gid" ] && _out+=(--group-add "$gid")
|
|
done
|
|
else
|
|
_out+=(-e SYLPH_VULKAN=sw)
|
|
fi
|
|
}
|
|
|
|
case "${1:-}" in
|
|
build)
|
|
shift
|
|
echo "==> building $IMAGE (uid $(id -u), gid $(id -g))"
|
|
exec docker build -t "$IMAGE" \
|
|
--build-arg "AGENT_UID=$(id -u)" --build-arg "AGENT_GID=$(id -g)" \
|
|
"$@" "$HERE"
|
|
;;
|
|
|
|
shell|agent|run)
|
|
mode=$1; shift
|
|
declare -a ARGS; docker_args ARGS
|
|
echo "==> $mode | cpus=$CPUS mem=${MEM_GB}g shm=${SHM_GB}g (host: ${HOST_CPUS} cpus, $((HOST_MEM_KB/1048576))g)"
|
|
echo "==> project: $PROJECT -> /work"
|
|
docker rm -f "$NAME" >/dev/null 2>&1 || true
|
|
# Allocate a TTY only when stdin actually is one: `docker run -it` fails
|
|
# outright ("cannot attach stdin to a TTY-enabled container") under a
|
|
# pipeline or a CI runner, which is exactly where `run` gets used.
|
|
TTY=(-i); [ -t 0 ] && TTY=(-it)
|
|
case "$mode" in
|
|
shell) exec docker run --rm "${TTY[@]}" "${ARGS[@]}" "$IMAGE" bash ;;
|
|
agent)
|
|
# The flag the user asked for. Refused under root, which is why the
|
|
# image runs as an unprivileged `agent` user.
|
|
ARGS+=(-e SYLPH_AUTONOMOUS=1)
|
|
exec docker run --rm "${TTY[@]}" "${ARGS[@]}" "$IMAGE" "$@"
|
|
;;
|
|
run) exec docker run --rm "${TTY[@]}" "${ARGS[@]}" "$IMAGE" "$@" ;;
|
|
esac
|
|
;;
|
|
|
|
stop) exec docker rm -f "$NAME" ;;
|
|
doctor)
|
|
declare -a ARGS; docker_args ARGS
|
|
exec docker run --rm "${ARGS[@]}" "$IMAGE" sylph-doctor
|
|
;;
|
|
""|-h|--help) usage 0 ;;
|
|
*) echo "unknown command: $1" >&2; usage 2 ;;
|
|
esac
|