From 01468b804a4859fa7ef6ba683affc84de617d565 Mon Sep 17 00:00:00 2001 From: Sylpheed RE agent Date: Mon, 24 Aug 2026 09:48:23 +0000 Subject: [PATCH] docs: the mystery SIGKILL was this project's own Stop hook /work/.claude/settings.json defines a Stop hook that kill -9s every xenia_canary at the end of each agent turn and prints "Stop hook killed N stale xenia process(es)". So every run that "died mysteriously" died at a turn boundary, which is why the timings looked random from inside the run (810 s, 54 s, 486 s) and why nothing in the guest, the cgroup or the host explained them. What survives is the measurement and not the story: the memory readings were real and did refute memory pressure - cgroup at 6.5 of 7.0 GB, host 12 GB free at the moment of a kill - but they were refuting a cause that was never in play. The method lesson is recorded because it cost three iterations: when a process dies at a SESSION boundary, check the harness before instrumenting the guest. A failure mode was documented, a hypothesis raised, a per-poll memory sampler written and committed, and host and cgroup counters read - all downstream of the assumption that the kill came from outside the agent. The hook's own message had been printed after every turn. The operational rule that follows: an emulator experiment must COMPLETE INSIDE A SINGLE TURN. Nothing survives the boundary, so runs cannot be left for the next tick, and experiments whose evidence arrives in the first minutes are the ones that work - which is exactly why the two-pass bit-level test succeeded where the long freeze-watches did not. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE --- docs/re/mission-freeze-resume-spin.md | 49 +++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/docs/re/mission-freeze-resume-spin.md b/docs/re/mission-freeze-resume-spin.md index c3ed128..8e1399e 100644 --- a/docs/re/mission-freeze-resume-spin.md +++ b/docs/re/mission-freeze-resume-spin.md @@ -583,3 +583,52 @@ in here attributes a SIGKILL to its sender (no `dmesg`, no audit, and `ptrace_scope=1` limits `strace` to my own descendants). The practical response is to stop designing experiments that need a *long* run: the counter transitions the OB work needs all happen in the first minutes of flight. + + +--- + +# ✅ 2026-08-24 — THE KILL IS SOLVED, and it was never the game: a Stop hook + +**Everything above about the `EMULATOR GONE` / SIGKILL failure mode has a +one-line cause**, and it is not in the emulator, the container runtime or the +kernel. It is in this project's own Claude Code configuration — +`/work/.claude/settings.json`: + +```json +"Stop": [{ "hooks": [{ "type": "command", "command": + "for name in xenia_canary xenia-rs; do pids=$(pgrep -x \"$name\"); ... + kill $pids; sleep 0.2; kill -9 $pids; ... 'Stop hook killed %d stale xenia process(es)'" +}]}] +``` + +A `Stop` hook fires at the **end of every agent turn** and `kill -9`s any running +`xenia_canary`. So every run that "died mysteriously" died at a turn boundary, +which is exactly why the timings looked random from inside the run (810 s, 54 s, +486 s) and why nothing in the guest, the cgroup or the host explained them. + +**What survives from the investigation** is only the measurement, not the story: +the memory readings were real and did refute memory pressure as a cause — the +cgroup was at 6.5 GB of 7.0 and the host had 12 GB free at the moment of a kill. +That refutation was correct; it just was not pointing at anything. + +## 🔑 The method lesson, which is the part worth keeping + +**When a process dies at a session boundary, check the harness before +instrumenting the guest.** Three iterations went into this: a failure mode +documented, a memory hypothesis raised, a per-poll memory sampler written and +committed, host and cgroup counters read — all of it downstream of an assumption +that the kill came from *outside the agent*. The one place not looked at was the +agent's own configuration, and that is where it was. The `systemMessage` the hook +prints ("Stop hook killed 1 stale xenia process(es)") is surfaced to the user +after each turn, so the answer had been on screen the whole time. + +## The rule that follows + +**An emulator experiment must complete inside a single turn.** Nothing survives +the turn boundary, so: + +* do not "leave a run going for the next tick" — it will be killed; +* prefer experiments whose evidence arrives in the first minutes of flight (the + `REMAINING OB` transitions all do — that is why the two-pass bit-level test + succeeded where the long freeze-watches did not); +* a watcher armed for 1 500 s can only ever watch for the rest of *this* turn.