docs(agents): how the containers and agents are actually set up
Written for the other machine, which asked. Everything in it was read off the running host rather than remembered. Also adds docker/ci/ -- the CI image recipe and a capped runner -- because the image existed on exactly one host and its Dockerfile was in a scratch directory under /tmp, which was swept. That is the same shape as every other thing this consolidation has turned up: something correct that exists in one place. docker/ci/Dockerfile rust 1.98.1 + the apt list copied from ci.yml docker/ci/Dockerfile.ffmpeg + ffmpeg, which sylpheed-export shells out to docker/ci/run 6 CPUs / 7 GB / NO SWAP, named cargo volumes The rule the runner exists to enforce: every heavy command goes in the capped container. CARGO_BUILD_JOBS caps codegen units, not rustc's threads, not the linker, not the test harness -- a bare host build is unbounded and has frozen this box repeatedly. The document also records the two agent defects the other machine will meet: a brief change does not reach a RESUMED session, and nothing brought an agent back to its own red PR (fixed in #24, which is itself subject to the first). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
MechaCat02
33
docker/ci/Dockerfile
Normal file
33
docker/ci/Dockerfile
Normal file
@@ -0,0 +1,33 @@
|
||||
# The CI image, reproduced locally.
|
||||
#
|
||||
# 🔴 WHY THIS FILE EXISTS. This image was built once, by hand, from a Dockerfile
|
||||
# in a scratch directory under /tmp — and /tmp was swept. The image survived on
|
||||
# one host and its recipe did not, which is the same shape as every other thing
|
||||
# this consolidation found: something correct that exists in exactly one place.
|
||||
#
|
||||
# The apt list below is COPIED from `.github/workflows/ci.yml`'s "Install Linux
|
||||
# system dependencies" step. If that list changes, this must change with it, or
|
||||
# a local run stops predicting the runner.
|
||||
#
|
||||
# docker build -t sylph-ci:local docker/ci
|
||||
#
|
||||
# ⚠️ The Rust version is PINNED and the runner's is NOT — `dtolnay/rust-toolchain@stable`
|
||||
# floats. That difference is issue #15: `collapsible_else_if` is `warn` on 1.92
|
||||
# and `allow` on 1.98.1, so a local clippy pass and a runner clippy pass can
|
||||
# legitimately disagree. When they do, the runner is the authority.
|
||||
FROM rust:1.98.1-bookworm
|
||||
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
libasound2-dev \
|
||||
libudev-dev \
|
||||
libwayland-dev \
|
||||
libxkbcommon-dev \
|
||||
libx11-dev \
|
||||
libxi-dev \
|
||||
pkg-config \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
RUN rustup component add clippy rustfmt \
|
||||
&& rustup target add wasm32-unknown-unknown
|
||||
|
||||
WORKDIR /work
|
||||
11
docker/ci/Dockerfile.ffmpeg
Normal file
11
docker/ci/Dockerfile.ffmpeg
Normal file
@@ -0,0 +1,11 @@
|
||||
# `sylph-ci:ffmpeg` — the CI image plus ffmpeg, for `sylpheed-export`.
|
||||
#
|
||||
# The exporter shells out to `ffprobe`/`ffmpeg` to transcode the two movies, so
|
||||
# `cargo run -p sylpheed-export -- export` fails in the plain CI image with
|
||||
# "run ffprobe -- is it on PATH?". CI itself never runs the exporter, which is
|
||||
# why its own image does not carry this.
|
||||
#
|
||||
# docker build -t sylph-ci:ffmpeg -f docker/ci/Dockerfile.ffmpeg docker/ci
|
||||
FROM sylph-ci:local
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
43
docker/ci/run
Executable file
43
docker/ci/run
Executable file
@@ -0,0 +1,43 @@
|
||||
#!/usr/bin/env bash
|
||||
# Run a command against this repository inside the capped CI container.
|
||||
#
|
||||
# docker/ci/run cargo check --workspace --all-targets
|
||||
# docker/ci/run cargo clippy --workspace --all-targets --keep-going -- -D warnings
|
||||
# docker/ci/run bash -c 'cargo fmt --all; cargo fmt --all -- --check | grep -c "^Diff in"'
|
||||
#
|
||||
# 🔴 EVERY HEAVY COMMAND GOES THROUGH HERE. A bare `cargo build` on the host is
|
||||
# unbounded: `CARGO_BUILD_JOBS` caps codegen units, not rustc's own threads, not
|
||||
# the linker, not the test harness. A full-parallel build has OOM-crashed this
|
||||
# box, and unbounded host runs have frozen it since. The container is the cap
|
||||
# that actually holds.
|
||||
#
|
||||
# Half the machine, no swap. `--memory-swap` equal to `--memory` means a build
|
||||
# that would swap is killed instead, which is a fast failure rather than an
|
||||
# hour of thrashing that takes the desktop with it.
|
||||
set -euo pipefail
|
||||
|
||||
REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
IMAGE="${SYLPH_CI_IMAGE:-sylph-ci:local}"
|
||||
CPUS="${SYLPH_CI_CPUS:-6}"
|
||||
MEM_GB="${SYLPH_CI_MEM_GB:-7}"
|
||||
|
||||
args=(
|
||||
--rm
|
||||
--cpus "$CPUS"
|
||||
--memory "${MEM_GB}g"
|
||||
--memory-swap "${MEM_GB}g"
|
||||
--pids-limit 2048
|
||||
-v "$REPO:/work"
|
||||
# Named volumes, not bind mounts: the host tree keeps a 32 GB `target/` from
|
||||
# earlier host-side builds, and mixing the two produces rebuilds that look
|
||||
# like cache misses and are actually two toolchains fighting over one directory.
|
||||
-v sylph-ci-cargo:/cargo -e CARGO_HOME=/cargo
|
||||
-v sylph-ci-target:/target -e CARGO_TARGET_DIR=/target
|
||||
-w /work
|
||||
)
|
||||
|
||||
# The disc, read-only, when a disc-backed test or the exporter needs it.
|
||||
DISC="${SYLPHEED_DISC:-$REPO/../sylph_extract}"
|
||||
[ -d "$DISC" ] && args+=(-v "$DISC:/disc:ro" -e SYLPHEED_DISC=/disc)
|
||||
|
||||
exec docker run "${args[@]}" "$IMAGE" "$@"
|
||||
157
docs/agents/CONTAINERS-AND-AGENTS.md
Normal file
157
docs/agents/CONTAINERS-AND-AGENTS.md
Normal file
@@ -0,0 +1,157 @@
|
||||
# Containers and agents — how this is actually set up
|
||||
|
||||
**Written 2026-09-13 for the other machine.** Everything below was read off the
|
||||
running host, not remembered. Paths are as they exist on the desktop; the Pi
|
||||
differs where it says so.
|
||||
|
||||
## 1. The rule that governs all of it
|
||||
|
||||
🔴 **Every heavy command runs in a capped container. No exceptions, including
|
||||
one-off checks.**
|
||||
|
||||
```bash
|
||||
docker/ci/run cargo check --workspace --all-targets
|
||||
```
|
||||
|
||||
A bare `cargo build` on the host is **unbounded**. `CARGO_BUILD_JOBS` caps
|
||||
codegen units — not rustc's own threads, not the linker, not the test harness.
|
||||
A full-parallel build has OOM-crashed this box, and unbounded host runs have
|
||||
frozen it repeatedly since. `--memory-swap` equal to `--memory` means a build
|
||||
that *would* swap is killed instead: a fast failure beats an hour of thrashing
|
||||
that takes the desktop with it.
|
||||
|
||||
What may run on the host: `git`, `grep`, `python3` over text, API calls. Anything
|
||||
that compiles, links, transcodes or boots an emulator does not.
|
||||
|
||||
## 2. Three kinds of container
|
||||
|
||||
| | image | what it is |
|
||||
|---|---|---|
|
||||
| **CI / build** | `sylph-ci:local` | the runner, reproduced locally — `docker/ci/Dockerfile` |
|
||||
| **Decoder agent** | `sylpheed-agent:latest` | an autonomous Claude Code session doing RE — `docker/decoder/` |
|
||||
| **Port agent** | `sylpheed-port:latest` | the same, building the Godot port — `docker/port/` |
|
||||
|
||||
### 2.1 The CI image
|
||||
|
||||
`docker/ci/Dockerfile` — `rust:1.98.1-bookworm`, **the apt list copied verbatim
|
||||
from `.github/workflows/ci.yml`**, plus `clippy`, `rustfmt` and the
|
||||
`wasm32-unknown-unknown` target.
|
||||
|
||||
```bash
|
||||
docker build -t sylph-ci:local docker/ci
|
||||
docker build -t sylph-ci:ffmpeg -f docker/ci/Dockerfile.ffmpeg docker/ci # exporter only
|
||||
```
|
||||
|
||||
⚠️ **The Rust version is pinned here and floats on the runner**
|
||||
(`dtolnay/rust-toolchain@stable`). That is issue #15, and it is not theoretical:
|
||||
`collapsible_else_if` is `warn` on 1.92 and `allow` on 1.98.1. When a local
|
||||
clippy pass and a runner pass disagree, **the runner is the authority.**
|
||||
|
||||
⚠️ **`cargo clippy` stops at the first failing compilation unit.** Without
|
||||
`--keep-going` the list looks short and is not. On the corpus branch the honest
|
||||
count was **80**, and the first run showed **14**.
|
||||
|
||||
`docker/ci/run` gives 6 CPUs / 7 GB / no swap, mounts the repo at `/work`, the
|
||||
extracted disc read-only at `/disc`, and uses **named volumes** for `CARGO_HOME`
|
||||
and `CARGO_TARGET_DIR` — never the host `target/`, which still holds 32 GB from
|
||||
older host-side builds and produces rebuilds that look like cache misses.
|
||||
|
||||
### 2.2 The agent containers
|
||||
|
||||
Launched from the host by `docker/decoder/sylph-decoder` and
|
||||
`docker/port/sylph-port`. Each starts a Claude Code session inside a container
|
||||
with the repo, the disc, Canary's source and a warm build tree.
|
||||
|
||||
**The budget is split deliberately, not halved per container** — there are two
|
||||
agents and a Referee is planned, so each claiming half a box it shares would
|
||||
oversubscribe it:
|
||||
|
||||
| | CPUs | memory | `/dev/shm` |
|
||||
|---|---|---|---|
|
||||
| decoder | 5 | 6 GB | 2 GB |
|
||||
| port | 3 | 4 GB | — |
|
||||
|
||||
Overridable with `SYLPH_CPUS` / `SYLPH_MEM_GB` and `SYLPH_PORT_CPUS` /
|
||||
`SYLPH_PORT_MEM_GB`. `/dev/shm` is an **exec-capable tmpfs**, not `--shm-size`:
|
||||
Docker mounts the default `noexec` and Xenia maps its JIT code cache out of a
|
||||
shm file. Its pages count against the memory cap, hence a third of it and no
|
||||
more.
|
||||
|
||||
**Volumes** (`docker volume ls`):
|
||||
|
||||
```
|
||||
sylpheed-decoder-repo sylpheed-port-repo the agent's own clone
|
||||
sylpheed-decoder-claude sylpheed-port-claude Claude Code state + transcripts
|
||||
sylph-agent-cargo sylpheed-port-cargo CARGO_HOME
|
||||
sylph-agent-target sylpheed-port-target CARGO_TARGET_DIR
|
||||
sylph-agent-canary-build the warm 235 MB Canary build tree
|
||||
sylpheed-exchange agent -> agent files, read-only in
|
||||
```
|
||||
|
||||
**Credentials** — three files on the host, `chmod 600`, mounted read-only:
|
||||
|
||||
```
|
||||
~/.sylph-claude-token Claude Code OAuth (a subscription, never the API)
|
||||
~/.sylph-git-credentials the agents' PUSH credential — write:repository ONLY
|
||||
~/.sylph-gitea-token-decoder the decoder's own Gitea account, for issues/PRs
|
||||
~/.sylph-gitea-token-port the port's
|
||||
~/.sylph-gitea-token-fabi the HUMAN's, full grant — mounted into NOTHING
|
||||
```
|
||||
|
||||
🔴 **The narrow scope on `~/.sylph-git-credentials` is load-bearing.** Every
|
||||
issue endpoint refuses it, which is what stops an agent reaching the Gitea API
|
||||
as the human. Putting a broader token there would hand both agents the human's
|
||||
identity, including merge. The human's own token lives in a path no launcher
|
||||
reads and no container mounts — verified, not assumed.
|
||||
|
||||
The Gitea MCP server is passed the token **by path**, not by value:
|
||||
`GITEA_TOKEN_FILE=/sylph-home/re/.sylph-gitea-token`.
|
||||
|
||||
## 3. How an agent iteration actually works
|
||||
|
||||
Both briefs — `decoder-loop.md`, `port-loop.md` — are read **from the host tree
|
||||
at container launch**. The loop is: read notifications → pick one approved issue
|
||||
→ do the smallest experiment → classify → refute something → write it down →
|
||||
`push-work`, open the PR, label `state/needs-human`, stop.
|
||||
|
||||
⚠️ **Notifications are POLLED. Nothing is pushed.** An `@mention` or a PR comment
|
||||
reaches an agent only on its next iteration.
|
||||
|
||||
### 🔴 Two defects the other machine should know about
|
||||
|
||||
**A brief change does not reach a running agent.** The entrypoint resumes the
|
||||
newest transcript in the container's Claude volume, and the only guard is a
|
||||
`<120 s` restart-loop test. Everything older resumes unconditionally — so a
|
||||
container restarted after a workflow change comes back **still following the old
|
||||
brief**, on the old branch. Seen: a decoder resumed a 7-day-old pre-migration
|
||||
session and had to be stopped. Until the entrypoint learns to test transcript
|
||||
age (or take a `SYLPH_FRESH=1`), adopting a brief change means **archiving the
|
||||
transcripts** in `sylpheed-decoder-claude/projects/` and starting fresh.
|
||||
|
||||
**Nothing brought an agent back to its own red PR.** It opened one, labelled the
|
||||
issue, stopped — correctly, per the brief — and the next firing started a new
|
||||
question on top. It pushed over a red CI six times. Fixed in both briefs (#24),
|
||||
which is itself subject to the defect above: the fix only reaches a *fresh*
|
||||
session.
|
||||
|
||||
## 4. The other machine's half
|
||||
|
||||
The Pi runs the Gitea instance (proxied to `git.mc02.dev` via a VPS, reachable
|
||||
from the LAN and the internet) and the Actions runner. Its
|
||||
`/var/lib/docker` is still on the **117 GB SD card** beside an idle 916 GB SSD;
|
||||
moving `data-root` is outstanding.
|
||||
|
||||
⚠️ **The runner serialises.** Three jobs per push, one at a time — a PR's checks
|
||||
sat *pending* for 40+ minutes behind a failed native job. A pending check there
|
||||
is not necessarily a slow one.
|
||||
|
||||
## 5. Standing constraints that are not about containers
|
||||
|
||||
* **One emulator process at a time**, ours or Canary, never both — a lockfile.
|
||||
* **Canary runs muted.** Point it at the real ISO, not the `sylpheed.iso` symlink
|
||||
(Wine cannot resolve it).
|
||||
* **Never judge a crash or a hang from a Bash-launched emulator run.** A SIGKILL
|
||||
that looked like our binary was the editor's process supervisor. Ask the human
|
||||
to run it natively.
|
||||
* **Never screenshot the port or the emulator yourself**; ask the human.
|
||||
* The oracle is the real game in Xenia Canary — not our renderer, not the port.
|
||||
Reference in New Issue
Block a user