Files
Sylpheed/docker/decoder/sylph-decoder
MechaCat02 e53d687755
Some checks failed
CI / Native — ubuntu-latest (push) Failing after 8m7s
CI / WASM — Web (push) Failing after 6m50s
CI / Formatting (push) Failing after 1m4s
CI / Native — macos-latest (push) Has been cancelled
CI / Native — windows-latest (push) Has been cancelled
docker: support a long-lived Claude token, and stop the seeding fighting it
The rotating OAuth credential file is why the agents kept parking, and a
long-lived token removes the failure by construction instead of recovering from
it after the fact.

MEASURED 2026-09-04. ~/.claude/.credentials.json holds a refresh token that
ROTATES ON USE. Seeding both containers from the host left three clients holding
one token; the first to refresh invalidated the other two, and on the failed
refresh Claude Code CLEARS the stored tokens -- writes empty strings, keeps the
metadata, and parks at "Login expired".

  decoder credentials emptied  13:04:28
  decoder last transcript      13:04:29   <- one second later

The emptying and the park are the same event, which is why it never self-heals:
not a stale token a retry could fix, but no token at all, with no browser in the
container to complete /login. A hollow file passes every "does it exist" check --
508 B healthy against 280 B emptied -- which is how three separate diagnoses
missed it. And recovery re-armed the bug: after re-seeding, host and decoder held
the IDENTICAL refresh token hash.

`claude setup-token` issues a long-lived token against the same Claude
subscription. Checked, not assumed: `claude auth login` defaults to --claudeai
and it is `--console` that means Console/API billing, so this is not the separate
API bill. `CLAUDE_CODE_OAUTH_TOKEN` is recognised by the installed binary.

Passed as an ENVIRONMENT VARIABLE, both halves of the failure are gone: nothing
rotates, so peers cannot invalidate each other, and there is no file for Claude
Code to empty on a failure.

Both launchers read $HOME/.sylph-claude-token if present -- same pattern as
SYLPH_GIT_CREDENTIALS -- and both entrypoints skip OAuth seeding entirely when
the variable is set, because copying the rotating file in would re-create the
exact collision the token exists to remove.

Inert until the file exists. Without it, nothing changes.

Also worth recording for the preflight work: `claude auth status` prints JSON
with loggedIn/authMethod/subscriptionType. That is a far better SessionStart
assertion than checking a file exists, and it would have caught this on the first
iteration rather than the third incident.
2026-09-04 15:20:31 +02:00

406 lines
20 KiB
Bash
Executable File

#!/usr/bin/env bash
# Host-side launcher for the Sylpheed RE agent container.
#
# Caps the container at HALF the machine's CPUs and memory, computed at run time
# so it stays half on whatever box it lands on.
#
# ./sylph-agent build build (or rebuild) the image
# ./sylph-agent shell interactive shell in the container
# ./sylph-agent agent [prompt] Claude Code, --dangerously-skip-permissions
# ./sylph-agent loose [task] turn it loose: detached, /loop, self-paced
# ./sylph-agent logs [-f] what the loose agent is doing
# ./sylph-agent remote print the Remote Control link (chat from anywhere)
# ./sylph-agent attach attach to the loose agent's session
# ./sylph-agent run <cmd...> one-shot command
# ./sylph-agent stop stop it
#
# Environment:
# SYLPH_PROJECT host project root (default: three levels up from this file)
# SYLPH_CLAUDE_HOME host dir mounted as the agent's ~/.claude
# (default: $HOME/.claude — shares auth AND memory with you)
# SYLPH_VULKAN=sw force software Vulkan (lavapipe) even if /dev/dri exists
# SYLPH_REMOTE=0 do NOT enable Remote Control (default: enabled for `loose`)
# SYLPH_REMOTE_NAME Remote Control session name (default: sylpheed-agent)
# SYLPH_GIT_CREDENTIALS file with `https://<user>:<token>@host` for push-work
# (default: $HOME/.sylph-git-credentials)
# SYLPH_LOOP_INTERVAL fixed loop cadence, e.g. 30m (default: 45m)
# SYLPH_CPUS / SYLPH_MEM_GB override the computed half
set -euo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
IMAGE="${SYLPH_IMAGE:-sylpheed-agent:latest}"
NAME="${SYLPH_NAME:-sylpheed-agent}"
PROJECT="${SYLPH_PROJECT:-$(cd "$HERE/../../.." && pwd)}"
# ── Half the box ─────────────────────────────────────────────────────────────
# LC_ALL=C is required, not tidiness: under a locale with a comma decimal
# separator (de_DE and friends) awk prints "6,0" and docker rejects it as
# --cpus with "failed to parse as a rational number".
HOST_CPUS=$(nproc)
HOST_MEM_KB=$(awk '/MemTotal/{print $2}' /proc/meminfo)
# Fixed, not "half the host": half was right when this was the only agent. There
# are now two, and a Referee is planned, so the budget is split deliberately
# instead of each container claiming half of a box it shares. The decoder gets
# the larger share because it builds and drives the emulator.
CPUS="${SYLPH_CPUS:-5}"
MEM_GB="${SYLPH_MEM_GB:-6}"
[ "$MEM_GB" -lt 2 ] && MEM_GB=2
# /dev/shm holds the emulator's guest memory (gmem.py reads it there). Docker's
# 64 MB default is far too small for a 512 MB console address space, and the
# failure is an obscure mmap error rather than an out-of-space message. tmpfs
# pages count against the memory cap, so take a third of it and no more.
SHM_GB=$(( MEM_GB / 3 )); [ "$SHM_GB" -lt 1 ] && SHM_GB=1
CLAUDE_JSON="${SYLPH_CLAUDE_JSON:-$HOME/.claude.json}"
if [ ! -f "$CLAUDE_JSON" ]; then
echo "==> WARNING: $CLAUDE_JSON does not exist." >&2
echo " Docker would create a DIRECTORY at that path inside the container," >&2
echo " and Claude Code would fail confusingly. Run \`claude\` once on the" >&2
echo " host first, or set SYLPH_CLAUDE_JSON." >&2
fi
usage() { sed -n '2,20p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'; exit "${1:-0}"; }
docker_args() {
local -n _out=$1
_out=(
--name "$NAME"
--hostname sylph-agent
# ── the cap ──
--cpus "$CPUS"
--memory "${MEM_GB}g"
--memory-swap "${MEM_GB}g" # no swap escape hatch: a swapping build
# thrashes the whole host, which is the
# failure this cap exists to prevent
--pids-limit 4096
# /dev/shm as an EXEC-capable tmpfs, not --shm-size. Docker's default mounts
# it `noexec`, and xenia maps its JIT code cache out of a shm file at a fixed
# address — so with noexec it dies at startup with "Unable to allocate code
# cache generated code storage / Cannot initalize processor", which reads
# like an address-space clash rather than a mount flag.
--tmpfs "/dev/shm:rw,exec,nosuid,nodev,size=${SHM_GB}g"
# Dynamic RE needs to attach to a live process: without SYS_PTRACE, gdb and
# strace are installed but inert ("Could not attach to process"), and the
# container's whole reason for existing is watching the emulator run.
# Docker's default seccomp profile also blocks calls the JIT and the guest
# memory mapper rely on.
--cap-add SYS_PTRACE
--security-opt seccomp=unconfined
--security-opt apparmor=unconfined
# ── the repository ──
# The agent's OWN clone, in its own volume -- not a bind mount of a human's
# working tree. That arrangement bit this project three times: an agent's
# `git config --local` captured a human's commits, a credential helper
# leaked a container-only path onto the host, and a `git add -A` swept an
# agent's in-flight files into someone else's commit. Separate checkouts
# make all three impossible rather than merely discouraged.
#
# The cost, accepted knowingly: Claude Code keys its per-project memory off
# the working directory, so moving from the host path to /work starts that
# memory empty. The corpus in docs/ is the memory that matters and it
# travels with the clone.
-v "sylpheed-decoder-repo:/work"
# Xenia Canary stays a separate repository -- it is a fork tracking upstream
# and carries our instrumentation. Read-write: building probes into it is
# real work, not a side effect.
-v "${SYLPH_CANARY:-$HOME/RE Project Sylpheed/xenia-canary}:/canary"
# The disc and the ISO. These used to arrive inside the project mount and
# silently stopped when /work became a clone -- silently because the
# disc-gated tests SELF-SKIP without SYLPHEED_DISC and report green, which
# is the most expensive kind of missing mount.
-v "${SYLPH_DISC:-$PROJECT/sylph_extract}:/disc:ro"
-e "SYLPHEED_DISC=/disc"
# ── the static-analysis corpus ──
# The xenia-rs era's disassembly database and the flat VA image. Neither is
# reproducible from anything in this repository yet -- the four scripts that
# READ sylpheed.db have no producer here -- so they are mounted read-only
# from the host as reference material.
#
# The `.pe` matters most: it is the decompressed image as a flat VA dump
# (file offset = VA - 0x82000000), which removes the need to boot the
# emulator and scrape /dev/shm to get at it. An earlier belief that this
# file was STALE was tested and refuted -- it is current.
-v "${SYLPH_XENIA_RS:-$PROJECT/xenia-rs}:/xenia-rs:ro"
-v "${SYLPH_PE:-$PROJECT/Project Sylpheed - Arc of Deception (USA, Europe) (En,Ja).pe}:/image/sylpheed.pe:ro"
-e "SYLPHEED_DB=/xenia-rs/sylpheed.db"
-e "SYLPHEED_PE=/image/sylpheed.pe"
-e "SYLPHEED_IMAGE_BASE=0x82000000"
# The shared exchange: transient files with provenance, outside git history.
-v "sylpheed-exchange:/exchange"
-e "PROJECT_DIR=/work"
-e "SYLPH_EXCHANGE=/exchange"
-e "SYLPH_AGENT=decoder"
# 🔴 THE JOB CAP LIVES IN THE ENVIRONMENT, NOT IN THE WRAPPER.
#
# `build-reborn` has always exported CARGO_BUILD_JOBS, and on 2026-09-01
# that was not enough: the agent ran a RAW `cargo test --release -p
# sylpheed-formats`, which never touches the wrapper, got one rustc per
# granted CPU, and the container was OOM-killed at its 6 GB cap mid-task.
# Docker reported ExitCode 0 with OOMKilled true, so it read as a clean
# exit and cost a diagnosis.
#
# A guardrail reachable only through a wrapper protects the calls that use
# the wrapper. This one is inherited by every process in the container, so
# bypassing it takes an explicit override rather than forgetting.
-e "CARGO_BUILD_JOBS=${SYLPH_JOBS:-2}"
-e "SYLPH_REPO_URL=https://git.mc02.dev/fabi/Sylpheed.git"
-e "XENIA_SRC=/canary"
# ── claude ──
# The state dir is shared read-write: credentials live in
# .claude/.credentials.json, so token refresh needs to write, and this is
# also what carries the project memory across.
-v "sylpheed-decoder-claude:/sylph-home/re/.claude"
-v "${SYLPH_CLAUDE_HOME:-$HOME/.claude}:/sylph-home/re/.claude.seed:ro"
# ~/.claude.json sits BESIDE that directory and holds `hasCompletedOnboarding`.
# Mounted READ-ONLY at a staging path: the entrypoint copies it to
# ~/.claude.json and stamps onboarding as done. Sharing the file directly
# would (a) re-run the first-run theme wizard whenever the container's
# Claude Code version differs from the host's — a silent hang an unattended
# agent never gets past — and (b) let the container rewrite your host config.
-v "${SYLPH_CLAUDE_JSON:-$HOME/.claude.json}:/sylph-home/re/.claude.host.json:ro"
# persistent build caches, so a container restart is not a rebuild
-v sylph-agent-cargo:/sylph-home/re/.cargo
-v sylph-agent-target:/sylph-home/re/target-container
-v sylph-agent-canary-build:/sylph-home/re/canary-build
)
# ── Vulkan SDK ──
# xenia's shader step calls `spirv-opt --canonicalize-ids`, which Ubuntu's
# packaged SPIRV-Tools (v2025.1) does not have — the build then dies ~500
# objects in. The LunarG SDK has it. Mounting the host's copy at the same path
# is cheaper than baking a 200 MB SDK into the image AND guarantees the
# container produces byte-identical shaders to the host build.
SDK="${VULKAN_SDK:-}"
if [ -z "$SDK" ]; then
SDK=$(ls -d "$HOME"/vulkan-sdk/*/x86_64 2>/dev/null | sort -V | tail -1 || true)
fi
if [ -n "$SDK" ] && [ -x "$SDK/bin/spirv-opt" ]; then
_out+=(-v "$SDK:$SDK:ro" -e "VULKAN_SDK=$SDK")
else
echo "==> NOTE: no Vulkan SDK found on the host. Building Canary's shaders" >&2
echo " needs spirv-opt with --canonicalize-ids (LunarG SDK); Ubuntu's" >&2
echo " packaged SPIRV-Tools is too old. Running is unaffected." >&2
fi
# ── git push ──
# Read-only, and only ever used by `push-work`, which refuses anything but an
# auto/* branch and never force-pushes. Without this the agent's work only
# exists inside the container and dies with it.
# ── Claude auth ──
#
# 🔴 THE ROTATING OAUTH FILE IS WHY THIS AGENT KEPT PARKING, and a long-lived
# token removes the failure by construction rather than recovering from it.
#
# Measured 2026-09-04: `~/.claude/.credentials.json` holds a REFRESH TOKEN THAT
# ROTATES ON USE. Seeding both containers from the host's copy left three
# clients holding one token; the first to refresh invalidated the other two,
# and on the failed refresh **Claude Code CLEARS the stored tokens** -- it
# writes empty strings, keeps the metadata, and parks at "Login expired". The
# decoder's file was caught emptied at 13:04:28 with its last work at 13:04:29.
# A hollow file passes every "does it exist" check, which is why three separate
# diagnoses missed it.
#
# `claude setup-token` issues a LONG-LIVED token against the same Claude
# subscription (not Console/API billing -- `claude auth login` defaults to
# `--claudeai`, and `--console` is the billed one). Passed as an environment
# variable it cannot be rotated out from under a peer and there is no file for
# Claude Code to empty, so both halves of the failure are gone.
#
# Inert until the file exists: without it the OAuth path below is unchanged.
CLAUDETOK="${SYLPH_CLAUDE_TOKEN:-$HOME/.sylph-claude-token}"
if [ -f "$CLAUDETOK" ]; then
_out+=(-e "CLAUDE_CODE_OAUTH_TOKEN=$(tr -d '[:space:]' < "$CLAUDETOK")")
echo "==> auth: long-lived token from $CLAUDETOK (no rotating credential file)" >&2
fi
GITCRED="${SYLPH_GIT_CREDENTIALS:-$HOME/.sylph-git-credentials}"
if [ -f "$GITCRED" ]; then
_out+=(-v "$GITCRED:/sylph-home/re/.git-credentials.host:ro")
else
echo "==> NOTE: no git credentials at $GITCRED — the agent cannot push," >&2
echo " so its work will be lost if the container is destroyed. Create it" >&2
echo " with a single line and chmod 600:" >&2
echo " https://<user>:<token>@git.mc02.dev" >&2
echo " or point SYLPH_GIT_CREDENTIALS elsewhere." >&2
fi
[ -n "${ANTHROPIC_API_KEY:-}" ] && _out+=(-e "ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY")
[ -n "${SYLPH_VULKAN:-}" ] && _out+=(-e "SYLPH_VULKAN=$SYLPH_VULKAN")
[ -n "${SYLPH_REMOTE:-}" ] && _out+=(-e "SYLPH_REMOTE=$SYLPH_REMOTE")
[ -n "${SYLPH_REMOTE_NAME:-}" ] && _out+=(-e "SYLPH_REMOTE_NAME=$SYLPH_REMOTE_NAME")
# The ISO Canary boots. Default resolves the retail image beside the project;
# mounted at a stable in-container path so run-canary does not depend on the
# host's directory names.
_iso="${SYLPH_ISO:-$PROJECT/Project Sylpheed - Arc of Deception (USA, Europe) (En,Ja).iso}"
if [ -f "$_iso" ]; then
_out+=(-v "$_iso:/iso/game.iso:ro" -e "SYLPH_ISO=/iso/game.iso")
else
echo "==> NOTE: no ISO at $_iso -- Canary has nothing to boot." >&2
fi
# ── GPU ──
# Three distinct cases, and conflating them is how you end up believing you
# have hardware Vulkan while actually running llvmpipe:
#
# NVIDIA needs the NVIDIA Container Toolkit (`--gpus all`). Passing
# /dev/dri alone does NOT work — Mesa cannot drive an NVIDIA card,
# and the proprietary userspace lives outside the image.
# Mesa (AMD/Intel) works with a plain /dev/dri passthrough plus the
# host's render/video GIDs.
# neither software Vulkan (lavapipe): correct, and slow.
if [ "${SYLPH_VULKAN:-auto}" = "sw" ]; then
_out+=(-e SYLPH_VULKAN=sw)
elif command -v nvidia-smi >/dev/null 2>&1 && nvidia-smi -L >/dev/null 2>&1; then
if docker info --format '{{json .Runtimes}}' 2>/dev/null | grep -q nvidia; then
_out+=(--gpus all)
else
echo "==> NOTE: NVIDIA GPU found but the NVIDIA Container Toolkit is not" >&2
echo " installed, so hardware Vulkan is unavailable and the container" >&2
echo " will use lavapipe (software — correct, slow). To enable it:" >&2
echo " sudo apt install nvidia-container-toolkit \\" >&2
echo " && sudo nvidia-ctk runtime configure --runtime=docker \\" >&2
echo " && sudo systemctl restart docker" >&2
_out+=(-e SYLPH_VULKAN=sw)
fi
elif [ -e /dev/dri/renderD128 ]; then
_out+=(--device /dev/dri)
for g in render video; do
gid=$(getent group "$g" | cut -d: -f3 || true)
[ -n "$gid" ] && _out+=(--group-add "$gid")
done
else
_out+=(-e SYLPH_VULKAN=sw)
fi
}
case "${1:-}" in
build)
shift
echo "==> building $IMAGE (uid $(id -u), gid $(id -g))"
exec docker build -t "$IMAGE" \
--build-arg "AGENT_UID=$(id -u)" --build-arg "AGENT_GID=$(id -g)" \
"$@" "$HERE"
;;
loose)
shift
# "On the loose": detached, self-paced, working the RE backlog until stopped.
#
# -d WITHOUT --rm so the transcript survives the container exiting; that is
# the only record of what an unattended run did. -t because /loop keeps the
# session alive and schedules its own wake-ups — a `-p`/print-mode run would
# answer once and exit, killing the loop on its first iteration.
TASK="${1:-}"
if [ -z "$TASK" ]; then
if [ -f "$HERE/../../docs/agents/decoder-loop.md" ]; then
TASK="$(cat "$HERE/../../docs/agents/decoder-loop.md")"
else
TASK="Work the RE backlog in Syplheed-Reborn/docs/re/BACKLOG.md."
fi
fi
# A FIXED interval by default, not self-pacing. Self-pacing requires the
# agent to call ScheduleWakeup itself at the end of every turn, and the one
# thing an agent deep in an experiment reliably forgets is the bookkeeping
# after it. With an interval the harness owns the cadence and a forgotten
# wakeup cannot end the run. Set SYLPH_LOOP_INTERVAL= (empty) to self-pace.
INTERVAL="${SYLPH_LOOP_INTERVAL-45m}"
declare -a ARGS; docker_args ARGS
ARGS+=(-e SYLPH_AUTONOMOUS=1 -w /work)
echo "==> loose | cpus=$CPUS mem=${MEM_GB}g shm=${SHM_GB}g"
echo "==> repo: own clone in volume sylpheed-decoder-repo -> /work"
echo "==> pacing: ${INTERVAL:-self-paced}"
docker rm -f "$NAME" >/dev/null 2>&1 || true
# 🔴 `unless-stopped`, NOT `on-failure` -- and the reason is a trap worth
# keeping. When this container was OOM-killed on 2026-09-01, Docker reported
# `OOMKilled: true` with **ExitCode 0**. `on-failure` keys off the exit code,
# so it would have treated a memory kill as a clean finish and left the agent
# down. `unless-stopped` restarts regardless, and still honours an explicit
# `./sylph-agent stop`.
#
# Restarting into the same death is handled at the other end: the entrypoint
# refuses to `--continue` if the last start was under two minutes ago.
docker run -d -i -t --restart unless-stopped "${ARGS[@]}" "$IMAGE" \
"/loop ${INTERVAL:+$INTERVAL }$TASK" >/dev/null
echo
echo " running detached as '$NAME'."
echo " ./sylph-agent remote link to chat with it from anywhere"
echo " ./sylph-agent logs -f follow it"
echo " ./sylph-agent attach chat with it locally (Ctrl-P Ctrl-Q to leave it running)"
echo " ./sylph-agent stop stop it"
echo
# Report what is actually true. This line used to claim unconditionally that
# the agent could not push, which was written before credentials were
# supported and then went stale — telling the operator their work was at risk
# when it was not, which is the exact failure the credential mount fixes.
if [ -f "${SYLPH_GIT_CREDENTIALS:-$HOME/.sylph-git-credentials}" ]; then
echo " It commits to auto/* branches and publishes them with push-work,"
echo " which refuses any other branch and never force-pushes. Review with:"
echo " git -C '$PROJECT/Syplheed-Reborn' fetch origin && git log --oneline origin/auto/..."
else
echo " It commits to auto/* branches and CANNOT PUSH — no git credentials"
echo " are mounted, so its work dies with the container. Review it with:"
echo " git -C '$PROJECT/Syplheed-Reborn' log --oneline auto/..."
fi
;;
logs) shift; exec docker logs "$@" "$NAME" ;;
remote)
# Fish the Remote Control link out of the session's own output. Claude Code
# prints it once, when the session registers with your account — which takes
# a minute or two after launch, so this waits rather than answering "not
# found" to a question that is really "not yet".
printf 'waiting for the session to register' >&2
for i in $(seq 1 90); do
url=$(docker logs "$NAME" 2>&1 \
| sed 's/\x1b\[[0-9;]*[a-zA-Z]//g; s/\r//g' \
| grep -oE 'https://claude\.ai/code/[A-Za-z0-9_-]+' | tail -1)
if [ -n "${url:-}" ]; then
printf '\n' >&2
echo "$url"
exit 0
fi
docker ps -q -f "name=$NAME" | grep -q . || {
printf '\n' >&2
echo "container is not running — start it with: ./sylph-agent loose" >&2
exit 1
}
printf '.' >&2
sleep 4
done
printf '\n' >&2
echo "no Remote Control URL after 6 minutes." >&2
echo " Launched with SYLPH_REMOTE=0? Or check: ./sylph-agent logs | tail" >&2
exit 1
;;
shell|agent|run)
mode=$1; shift
declare -a ARGS; docker_args ARGS
echo "==> $mode | cpus=$CPUS mem=${MEM_GB}g shm=${SHM_GB}g (host: ${HOST_CPUS} cpus, $((HOST_MEM_KB/1048576))g)"
echo "==> project: $PROJECT -> /work"
docker rm -f "$NAME" >/dev/null 2>&1 || true
# Allocate a TTY only when stdin actually is one: `docker run -it` fails
# outright ("cannot attach stdin to a TTY-enabled container") under a
# pipeline or a CI runner, which is exactly where `run` gets used.
TTY=(-i); [ -t 0 ] && TTY=(-it)
case "$mode" in
shell) exec docker run --rm "${TTY[@]}" "${ARGS[@]}" "$IMAGE" bash ;;
agent)
# The flag the user asked for. Refused under root, which is why the
# image runs as an unprivileged `agent` user.
ARGS+=(-e SYLPH_AUTONOMOUS=1)
exec docker run --rm "${TTY[@]}" "${ARGS[@]}" "$IMAGE" "$@"
;;
run) exec docker run --rm "${TTY[@]}" "${ARGS[@]}" "$IMAGE" "$@" ;;
esac
;;
stop) exec docker rm -f "$NAME" ;;
doctor)
declare -a ARGS; docker_args ARGS
exec docker run --rm "${ARGS[@]}" "$IMAGE" sylph-doctor
;;
""|-h|--help) usage 0 ;;
*) echo "unknown command: $1" >&2; usage 2 ;;
esac