docker: the expect wrapper swallowed both the signal and the exit status
A tooling review predicted a PID-1 signal problem from two symptoms we could not
explain: `OOMKilled: true` with **ExitCode 0**, and `--continue` failing to find
a conversation that plainly existed. Traced it, and the prediction was right --
though the culprit is not PID 1, it is one level below.
The path is tini (PID 1) -> entrypoint.sh (exec'd) -> expect -> spawn -> claude
`spawn` CANNOT be an exec: expect has to stay alive to drive the pty. So expect
is the process Docker signals, and everything depends on it passing things on.
It did neither, in two lines:
1. NO SIGNAL FORWARDING, no trap of any kind. `docker stop` sent SIGTERM to
expect, which died and took the pty with it. Claude Code never got a SIGTERM,
so it never ran SessionEnd hooks and never wrote lastSessionId/history --
which are written ONLY at a graceful shutdown. That is the entire reason
`claude --continue` answered "No conversation found to continue" with 33 MB of
transcripts in the volume beside it, and why we resume by scraping a session
id off a transcript filename.
2. `eof { exit }` RETURNED 0 FOR EVERY DEATH. A bare `exit` in expect is exit
ZERO. When the OOM-killer took the child, expect saw EOF and reported a clean
exit. `OOMKilled: true` with `ExitCode 0` was never Docker being odd -- it was
this line. It also meant `--restart on-failure` would read a memory kill as
success, which is why the policy had to be `unless-stopped`.
Fixed and MEASURED, old against new, in a container:
child exits 7 old -> 0 (the bug) new -> 7
SIGTERM to wrapper old -> 143, child's trap NEVER RAN
new -> 42, child trapped and cleaned up
Same file in both images; they were byte-identical, so the port copy takes the
same change.
Consequences worth stating: a kill now reports 137 rather than 0, so exit codes
mean what they say; `docker stop` gives Claude Code a real SIGTERM, so it runs
SessionEnd and writes the session index -- which may make the transcript-filename
resume unnecessary. That is not assumed here: the resume path stays as it is
until it is verified redundant.
This commit is contained in:
@@ -39,6 +39,44 @@ set answered_trust 0
|
||||
set answered_bypass 0
|
||||
|
||||
spawn -noecho claude --dangerously-skip-permissions {*}$argv
|
||||
set child_pid [exp_pid]
|
||||
|
||||
# 🔴 THIS WRAPPER USED TO SWALLOW BOTH THE SIGNAL AND THE EXIT STATUS, and those
|
||||
# two omissions caused most of this project's multi-hour outages. Found
|
||||
# 2026-09-03 by tracing the signal path, after a tooling review predicted exactly
|
||||
# this from the symptoms.
|
||||
#
|
||||
# The path is: tini (PID 1) -> entrypoint.sh (exec'd) -> expect -> spawn -> claude
|
||||
#
|
||||
# `spawn` CANNOT be an exec: expect has to stay alive to drive the pty. So expect
|
||||
# is the process Docker signals, and everything below it depends on expect
|
||||
# passing things along. It did not.
|
||||
#
|
||||
# 1. NO SIGNAL FORWARDING. `docker stop` sent SIGTERM to expect, which died and
|
||||
# took the pty with it. Claude Code never received a SIGTERM, so it never ran
|
||||
# its `SessionEnd` hooks and never wrote `lastSessionId`/`history` to
|
||||
# `~/.claude.json` -- which are written only at a GRACEFUL shutdown. That is
|
||||
# the whole reason `claude --continue` answered "No conversation found to
|
||||
# continue" with 33 MB of transcripts sitting in the volume beside it, and why
|
||||
# we resume by scraping a session id off a transcript filename instead.
|
||||
#
|
||||
# 2. `eof { exit }` RETURNED 0 FOR EVERY DEATH. A bare `exit` in expect is exit
|
||||
# ZERO. So when the kernel OOM-killer took the child, expect saw EOF and
|
||||
# reported a clean exit -- `OOMKilled: true` with `ExitCode 0`, which is not
|
||||
# Docker being odd, it is this line. It also meant `--restart on-failure`
|
||||
# would have treated a memory kill as success, which is why the policy had to
|
||||
# be `unless-stopped`.
|
||||
#
|
||||
# Both are fixed here. Signals are forwarded to the child and its real status is
|
||||
# propagated, so a kill reads as 137, a clean stop lets Claude Code shut down
|
||||
# properly, and the exit code means what it says.
|
||||
proc forward {sig} {
|
||||
global child_pid
|
||||
catch { exec kill -$sig $child_pid }
|
||||
}
|
||||
trap { forward TERM } SIGTERM
|
||||
trap { forward INT } SIGINT
|
||||
trap { forward HUP } SIGHUP
|
||||
|
||||
expect {
|
||||
-re {Choose} {
|
||||
@@ -70,3 +108,18 @@ expect {
|
||||
|
||||
# Hand the terminal over for the rest of the run.
|
||||
interact
|
||||
|
||||
# Propagate the child's REAL exit status. `interact` returns when the child is
|
||||
# gone; `wait` then yields {pid spawnid os_error status}. Without this the script
|
||||
# simply ran off the end and returned 0 -- see the note at `spawn` above for what
|
||||
# that cost.
|
||||
catch wait result
|
||||
set status 0
|
||||
if {[info exists result] && [llength $result] >= 4} {
|
||||
# os_error_flag (index 2) is -1 for a normal exit; anything else means the
|
||||
# wait itself failed and the status field is not a status.
|
||||
if {[lindex $result 2] == 0} {
|
||||
set status [lindex $result 3]
|
||||
}
|
||||
}
|
||||
exit $status
|
||||
|
||||
Reference in New Issue
Block a user