docker: auto-restart, and resume the session the agent was actually in

The decoder died mid-task and it took four separate findings to explain, each
of which read as something else:

1. OOM-KILLED, REPORTED AS A CLEAN EXIT. `OOMKilled: true` with **ExitCode 0**.
   So `--restart on-failure` would treat a memory kill as a successful finish
   and leave the agent down -- the policy has to be `unless-stopped`.

2. THE JOB CAP WAS SET AND THEN REMOVED THREE LINES LATER. build-reborn has
   always exported CARGO_BUILD_JOBS, but a raw `cargo test --release -p
   sylpheed-formats` never reaches the wrapper. Adding `-e CARGO_BUILD_JOBS` to
   the launcher did not help either: the entrypoint recomputes and exports over
   it unconditionally. An explicit value now wins, and says so in the log.

3. THE MEMORY CONSTANT WAS WRONG. `mem_gib * 2 / 3` assumes ~1.5 GB per job;
   release rustc on this workspace needs ~2 GB, and 4 jobs in 6 GB is what died.
   Divisor is now 2.

4. `--continue` CANNOT RESUME AN ABRUPT DEATH, which is the only kind we get.
   It resolves through ~/.claude.json's per-project `history`/`lastSessionId`,
   and MEASURED mid-session both are None -- they are written at a graceful
   shutdown. A killed container never writes them, so `--continue` answered
   "No conversation found to continue" with 33 MB of transcripts in the volume
   beside it. Persisting .claude.json did not help, because the fields were
   never populated in the first place; that attempt is removed rather than left
   in looking useful.

   The TRANSCRIPTS are durable and named by session id, so the entrypoint reads
   the id off the newest one for its cwd and passes `--resume <id>`. Verified
   on both agents: each reattached to its exact prior session and appended to
   the same file rather than opening a new one.

The /loop prompt is still passed alongside `--resume`, so the loop is RE-ARMED
rather than merely restored -- a resumed conversation with no wake-up scheduled
answers once and stops, which looks like resuming and is not.

Restarting into the same death is guarded at the other end: a start less than
120 s after the previous one begins FRESH instead of continuing back into
whatever killed it. That fired correctly during this work.

On resume the agent is told it was restarted, that its in-progress work is
uncommitted in the tree, that any build or capture it had running did not
finish and its absence is not a result, and which wrapper to prefer over a raw
release build.
This commit is contained in:
MechaCat02
2026-09-01 20:20:51 +02:00
parent 79783ff9ee
commit 4ac23b94dd
4 changed files with 196 additions and 4 deletions

View File

@@ -130,6 +130,19 @@ docker_args() {
-e "PROJECT_DIR=/work"
-e "SYLPH_EXCHANGE=/exchange"
-e "SYLPH_AGENT=decoder"
# 🔴 THE JOB CAP LIVES IN THE ENVIRONMENT, NOT IN THE WRAPPER.
#
# `build-reborn` has always exported CARGO_BUILD_JOBS, and on 2026-09-01
# that was not enough: the agent ran a RAW `cargo test --release -p
# sylpheed-formats`, which never touches the wrapper, got one rustc per
# granted CPU, and the container was OOM-killed at its 6 GB cap mid-task.
# Docker reported ExitCode 0 with OOMKilled true, so it read as a clean
# exit and cost a diagnosis.
#
# A guardrail reachable only through a wrapper protects the calls that use
# the wrapper. This one is inherited by every process in the container, so
# bypassing it takes an explicit override rather than forgetting.
-e "CARGO_BUILD_JOBS=${SYLPH_JOBS:-2}"
-e "SYLPH_REPO_URL=https://git.mc02.dev/fabi/Sylpheed.git"
-e "XENIA_SRC=/canary"
# ── claude ──
@@ -269,7 +282,17 @@ case "${1:-}" in
echo "==> repo: own clone in volume sylpheed-decoder-repo -> /work"
echo "==> pacing: ${INTERVAL:-self-paced}"
docker rm -f "$NAME" >/dev/null 2>&1 || true
docker run -d -i -t "${ARGS[@]}" "$IMAGE" "/loop ${INTERVAL:+$INTERVAL }$TASK" >/dev/null
# 🔴 `unless-stopped`, NOT `on-failure` -- and the reason is a trap worth
# keeping. When this container was OOM-killed on 2026-09-01, Docker reported
# `OOMKilled: true` with **ExitCode 0**. `on-failure` keys off the exit code,
# so it would have treated a memory kill as a clean finish and left the agent
# down. `unless-stopped` restarts regardless, and still honours an explicit
# `./sylph-agent stop`.
#
# Restarting into the same death is handled at the other end: the entrypoint
# refuses to `--continue` if the last start was under two minutes ago.
docker run -d -i -t --restart unless-stopped "${ARGS[@]}" "$IMAGE" \
"/loop ${INTERVAL:+$INTERVAL }$TASK" >/dev/null
echo
echo " running detached as '$NAME'."
echo " ./sylph-agent remote link to chat with it from anywhere"