docker: auto-restart, and resume the session the agent was actually in
The decoder died mid-task and it took four separate findings to explain, each of which read as something else: 1. OOM-KILLED, REPORTED AS A CLEAN EXIT. `OOMKilled: true` with **ExitCode 0**. So `--restart on-failure` would treat a memory kill as a successful finish and leave the agent down -- the policy has to be `unless-stopped`. 2. THE JOB CAP WAS SET AND THEN REMOVED THREE LINES LATER. build-reborn has always exported CARGO_BUILD_JOBS, but a raw `cargo test --release -p sylpheed-formats` never reaches the wrapper. Adding `-e CARGO_BUILD_JOBS` to the launcher did not help either: the entrypoint recomputes and exports over it unconditionally. An explicit value now wins, and says so in the log. 3. THE MEMORY CONSTANT WAS WRONG. `mem_gib * 2 / 3` assumes ~1.5 GB per job; release rustc on this workspace needs ~2 GB, and 4 jobs in 6 GB is what died. Divisor is now 2. 4. `--continue` CANNOT RESUME AN ABRUPT DEATH, which is the only kind we get. It resolves through ~/.claude.json's per-project `history`/`lastSessionId`, and MEASURED mid-session both are None -- they are written at a graceful shutdown. A killed container never writes them, so `--continue` answered "No conversation found to continue" with 33 MB of transcripts in the volume beside it. Persisting .claude.json did not help, because the fields were never populated in the first place; that attempt is removed rather than left in looking useful. The TRANSCRIPTS are durable and named by session id, so the entrypoint reads the id off the newest one for its cwd and passes `--resume <id>`. Verified on both agents: each reattached to its exact prior session and appended to the same file rather than opening a new one. The /loop prompt is still passed alongside `--resume`, so the loop is RE-ARMED rather than merely restored -- a resumed conversation with no wake-up scheduled answers once and stops, which looks like resuming and is not. Restarting into the same death is guarded at the other end: a start less than 120 s after the previous one begins FRESH instead of continuing back into whatever killed it. That fired correctly during this work. On resume the agent is told it was restarted, that its in-progress work is uncommitted in the tree, that any build or capture it had running did not finish and its absence is not a result, and which wrapper to prefer over a raw release build.
This commit is contained in:
@@ -130,6 +130,19 @@ docker_args() {
|
||||
-e "PROJECT_DIR=/work"
|
||||
-e "SYLPH_EXCHANGE=/exchange"
|
||||
-e "SYLPH_AGENT=decoder"
|
||||
# 🔴 THE JOB CAP LIVES IN THE ENVIRONMENT, NOT IN THE WRAPPER.
|
||||
#
|
||||
# `build-reborn` has always exported CARGO_BUILD_JOBS, and on 2026-09-01
|
||||
# that was not enough: the agent ran a RAW `cargo test --release -p
|
||||
# sylpheed-formats`, which never touches the wrapper, got one rustc per
|
||||
# granted CPU, and the container was OOM-killed at its 6 GB cap mid-task.
|
||||
# Docker reported ExitCode 0 with OOMKilled true, so it read as a clean
|
||||
# exit and cost a diagnosis.
|
||||
#
|
||||
# A guardrail reachable only through a wrapper protects the calls that use
|
||||
# the wrapper. This one is inherited by every process in the container, so
|
||||
# bypassing it takes an explicit override rather than forgetting.
|
||||
-e "CARGO_BUILD_JOBS=${SYLPH_JOBS:-2}"
|
||||
-e "SYLPH_REPO_URL=https://git.mc02.dev/fabi/Sylpheed.git"
|
||||
-e "XENIA_SRC=/canary"
|
||||
# ── claude ──
|
||||
@@ -269,7 +282,17 @@ case "${1:-}" in
|
||||
echo "==> repo: own clone in volume sylpheed-decoder-repo -> /work"
|
||||
echo "==> pacing: ${INTERVAL:-self-paced}"
|
||||
docker rm -f "$NAME" >/dev/null 2>&1 || true
|
||||
docker run -d -i -t "${ARGS[@]}" "$IMAGE" "/loop ${INTERVAL:+$INTERVAL }$TASK" >/dev/null
|
||||
# 🔴 `unless-stopped`, NOT `on-failure` -- and the reason is a trap worth
|
||||
# keeping. When this container was OOM-killed on 2026-09-01, Docker reported
|
||||
# `OOMKilled: true` with **ExitCode 0**. `on-failure` keys off the exit code,
|
||||
# so it would have treated a memory kill as a clean finish and left the agent
|
||||
# down. `unless-stopped` restarts regardless, and still honours an explicit
|
||||
# `./sylph-agent stop`.
|
||||
#
|
||||
# Restarting into the same death is handled at the other end: the entrypoint
|
||||
# refuses to `--continue` if the last start was under two minutes ago.
|
||||
docker run -d -i -t --restart unless-stopped "${ARGS[@]}" "$IMAGE" \
|
||||
"/loop ${INTERVAL:+$INTERVAL }$TASK" >/dev/null
|
||||
echo
|
||||
echo " running detached as '$NAME'."
|
||||
echo " ./sylph-agent remote link to chat with it from anywhere"
|
||||
|
||||
Reference in New Issue
Block a user