feat(vision-manager): memory-pressure yield gate
Add a fourth gate to the vision autoscaler poll loop: when host memory gets tight, proactively SIGTERM-stop the mangalord-vision container so a transient spike elsewhere (a crawl, CI, a backend burst) can finish instead of the kernel OOM-killer shooting something stateful. - read_mem_used_pct: host-wide used% from /proc/meminfo MemAvailable (not MemFree), ERR sentinel when MemTotal or MemAvailable is missing. - mem_check_and_stop: active stop over MEM_HIGH_WATERMARK_PCT + arm a restart cooldown; only inspects docker once already over the mark. - mem_yield_active: start inhibit while cooldown active or used% >= MEM_LOW_WATERMARK_PCT (two watermarks = hysteresis). - Inhibit reuses the existing idle path via pending=0; flip-only logging; stop_vision gains a reason arg so idle/MAX_UPTIME/memory-yield stops are distinctly logged. - Faster MEM_POLL_INTERVAL sub-poll between backlog ticks to catch spikes. Backend verifications V1-V4 (see VISION-MEMORY-YIELD.md) all pass against the current queue, so no backend changes are needed. RESPECT_CRAWL_MUTEX is kept for now. New VISION_MEM_* knobs wired through docker-compose.yml and documented in .env.example. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
12
.env.example
12
.env.example
@@ -218,3 +218,15 @@ VISION_MANAGER_DATABASE_URL=
|
||||
# VISION_START_HEALTH_TIMEOUT Max seconds to wait for /health 200 after start. Default 300 (cold model load).
|
||||
# VISION_RESPECT_CRAWL_MUTEX 1 = don't start vision while a crawl runs (RAM mutex on the 8 GiB box). Default 1.
|
||||
# VISION_MAX_UPTIME >0 = force-stop vision after N seconds running (backstop). Default 0 (disabled).
|
||||
#
|
||||
# Memory-pressure yield — the dynamic generalization of the crawl mutex. The
|
||||
# manager reads host-wide used% (from /proc/meminfo MemAvailable) and stops a
|
||||
# RUNNING vision when memory gets tight, so a spike elsewhere (a crawl, CI, a
|
||||
# backend burst) can finish instead of the kernel OOM-killer shooting something
|
||||
# stateful. Two watermarks give hysteresis; the cooldown prevents stop/restart
|
||||
# thrash when vision itself is the hog. See VISION-MEMORY-YIELD.md.
|
||||
# VISION_MEM_YIELD_ENABLED 1 = enable the gate, 0 = disable entirely. Default 1.
|
||||
# VISION_MEM_HIGH_WATERMARK_PCT used% >= this → actively stop a running vision. Default 92.
|
||||
# VISION_MEM_LOW_WATERMARK_PCT used% >= this → inhibit starts (keep HIGH - LOW >= ~10 so the band beats jitter). Default 80.
|
||||
# VISION_MEM_YIELD_COOLDOWN Seconds after a pressure-stop during which restart is refused regardless of backlog. Default 300.
|
||||
# VISION_MEM_POLL_INTERVAL Mem sub-poll cadence, seconds (<= VISION_POLL_INTERVAL); catches spikes between backlog ticks. Default 5.
|
||||
|
||||
Reference in New Issue
Block a user