docs+tools: the silent death is a SIGKILL from outside, and it is still not OOM

Run 8 died 54 s into its boot and this time bash named it: "176880 Killed" on the
run-canary line, which is SIGKILL. So the third failure mode is not an internal
fault - something outside the process is killing it.

And it is still not the OOM killer. Checked immediately after: oom_kill remained
0 and the allocation-stall counter did not move from 4421, so during run 8 the
cgroup never reached its limit (5.35 GB of 7 GiB), and the host had 13.8 GB
available. Two kills, no OOM evidence either time.

Rather than keep guessing after the fact, freeze_watch.sh now samples host
MemAvailable, cgroup memory.current and oom_kill on every poll and dumps the last
five samples when the process disappears - so the next occurrence carries its own
contemporaneous reading.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PMRJjbxLqZtsb5Vb7KunPE
This commit is contained in:
Sylpheed RE agent
2026-08-24 09:07:11 +00:00
parent bb81be7a02
commit 1b2bd3b22b
2 changed files with 37 additions and 1 deletions

View File

@@ -22,9 +22,24 @@ OUT="${2:-/sylph-home/re/logs/freeze-evidence.txt}"
DEADLINE=$(( SECONDS + ${3:-1800} ))
: > "$OUT"
# Sample memory every poll, so a run that DIES has a contemporaneous reading
# instead of a post-mortem guess. Two runs were SIGKILLed from outside with the
# cgroup's oom_kill counter reading 0 both times, and by the time anything was
# read the pressure (if that is what it was) had passed.
mem_line() {
local avail cur
avail=$(awk '/MemAvailable/{print int($2/1024)}' /proc/meminfo)
cur=$(( $(cat /sys/fs/cgroup/memory.current 2>/dev/null || echo 0) / 1048576 ))
local oom
oom=$(awk '/^oom_kill /{print $2}' /sys/fs/cgroup/memory.events 2>/dev/null)
echo "t=${SECONDS}s host_avail=${avail}MiB cgroup=${cur}MiB oom_kill=${oom}"
}
while [ $SECONDS -lt $DEADLINE ]; do
mem_line >> "${OUT%.txt}-mem.log"
if ! pgrep -x xenia_canary >/dev/null; then
echo "EMULATOR GONE at ${SECONDS}s" | tee -a "$OUT"; exit 4
{ echo "EMULATOR GONE at ${SECONDS}s"; echo "last memory samples:";
tail -5 "${OUT%.txt}-mem.log"; } | tee -a "$OUT"; exit 4
fi
if python3 "$SD/frozen.py" 5 >/dev/null; then
if python3 -c "import sys; sys.path.insert(0,'$SD'); import frozen; sys.exit(0 if frozen.in_flight() else 1)"; then