docs: a third SIGKILL, traced live, refutes the memory explanation
Run 9 died the same way at t=486s, and the watcher's sampler had been running throughout: at the kill the cgroup held 6.5 GB of its 7.0 GB limit, the host had 12 GB free, and oom_kill was 0. So the memory-pressure suspicion recorded earlier is refuted - three kills, no OOM record on any, and the one with a full trace shows headroom on both the cgroup and the host. Also checked and clean: no stray process of mine was running, and the kills do not fall on a clock (08:31, 09:00, 09:14 - gaps of ~29 and ~14 minutes). Recorded as a container-level blocker rather than chased further, because nothing in here can attribute a SIGKILL to its sender: no dmesg, no audit, and ptrace_scope=1 limits strace to my own descendants. The practical response is to stop designing experiments that need a long run - the counter transitions the OB work needs all happen in the first minutes of flight. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
This commit is contained in:
@@ -551,3 +551,35 @@ usual single pair (`BE56BB5C`) and nothing else.
|
||||
|
||||
**Run tally since v2: three runs, no frozen sample** — GAME OVER at ~22 min
|
||||
(run 5), `EMULATOR GONE` at 810 s (run 6), and no freeze in 1 100 s (run 7).
|
||||
|
||||
|
||||
### 🔴 A third kill, with a contemporaneous trace — and it REFUTES memory pressure
|
||||
|
||||
Run 9 died the same way at t=486 s, and this time the watcher's own sampler had
|
||||
been running the whole time:
|
||||
|
||||
```
|
||||
EMULATOR GONE at 486s
|
||||
t=344s host_avail=12159MiB cgroup=6388MiB oom_kill=0
|
||||
t=382s host_avail=12061MiB cgroup=6490MiB oom_kill=0
|
||||
t=411s host_avail=12001MiB cgroup=6533MiB oom_kill=0
|
||||
t=453s host_avail=12066MiB cgroup=6489MiB oom_kill=0
|
||||
t=486s host_avail=13163MiB cgroup=5375MiB oom_kill=0 <- after the kill
|
||||
```
|
||||
|
||||
At the moment of death the cgroup held **6.5 GB of its 7.0 GB** — not at the
|
||||
limit — the host had **12 GB free**, and `oom_kill` was **0**. 🔴 So the memory
|
||||
suspicion recorded above is **refuted**: three kills, no OOM record on any of
|
||||
them, and the one with a full trace shows headroom on both the cgroup and the
|
||||
host.
|
||||
|
||||
**What was also checked and found clean:** no stray process of mine was running
|
||||
(only the pilot), and the kills do not fall on a clock — 08:31, 09:00, 09:14, so
|
||||
gaps of ~29 and ~14 minutes rather than a period.
|
||||
|
||||
**Stated as a container-level blocker rather than chased further:** something
|
||||
outside the emulator SIGKILLs it, roughly one run in three, and nothing available
|
||||
in here attributes a SIGKILL to its sender (no `dmesg`, no audit, and
|
||||
`ptrace_scope=1` limits `strace` to my own descendants). The practical response
|
||||
is to stop designing experiments that need a *long* run: the counter transitions
|
||||
the OB work needs all happen in the first minutes of flight.
|
||||
|
||||
Reference in New Issue
Block a user