feat(vision-manager): memory-pressure yield gate

Add a fourth gate to the vision autoscaler poll loop: when host memory
gets tight, proactively SIGTERM-stop the mangalord-vision container so a
transient spike elsewhere (a crawl, CI, a backend burst) can finish
instead of the kernel OOM-killer shooting something stateful.

- read_mem_used_pct: host-wide used% from /proc/meminfo MemAvailable
  (not MemFree), ERR sentinel when MemTotal or MemAvailable is missing.
- mem_check_and_stop: active stop over MEM_HIGH_WATERMARK_PCT + arm a
  restart cooldown; only inspects docker once already over the mark.
- mem_yield_active: start inhibit while cooldown active or used% >=
  MEM_LOW_WATERMARK_PCT (two watermarks = hysteresis).
- Inhibit reuses the existing idle path via pending=0; flip-only logging;
  stop_vision gains a reason arg so idle/MAX_UPTIME/memory-yield stops are
  distinctly logged.
- Faster MEM_POLL_INTERVAL sub-poll between backlog ticks to catch spikes.

Backend verifications V1-V4 (see VISION-MEMORY-YIELD.md) all pass against
the current queue, so no backend changes are needed. RESPECT_CRAWL_MUTEX
is kept for now. New VISION_MEM_* knobs wired through docker-compose.yml
and documented in .env.example.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-06-16 14:33:34 +02:00
parent 790549636f
commit 0a9e727530
4 changed files with 342 additions and 9 deletions

View File

@@ -218,3 +218,15 @@ VISION_MANAGER_DATABASE_URL=
# VISION_START_HEALTH_TIMEOUT Max seconds to wait for /health 200 after start. Default 300 (cold model load).
# VISION_RESPECT_CRAWL_MUTEX 1 = don't start vision while a crawl runs (RAM mutex on the 8 GiB box). Default 1.
# VISION_MAX_UPTIME >0 = force-stop vision after N seconds running (backstop). Default 0 (disabled).
#
# Memory-pressure yield — the dynamic generalization of the crawl mutex. The
# manager reads host-wide used% (from /proc/meminfo MemAvailable) and stops a
# RUNNING vision when memory gets tight, so a spike elsewhere (a crawl, CI, a
# backend burst) can finish instead of the kernel OOM-killer shooting something
# stateful. Two watermarks give hysteresis; the cooldown prevents stop/restart
# thrash when vision itself is the hog. See VISION-MEMORY-YIELD.md.
# VISION_MEM_YIELD_ENABLED 1 = enable the gate, 0 = disable entirely. Default 1.
# VISION_MEM_HIGH_WATERMARK_PCT used% >= this → actively stop a running vision. Default 92.
# VISION_MEM_LOW_WATERMARK_PCT used% >= this → inhibit starts (keep HIGH - LOW >= ~10 so the band beats jitter). Default 80.
# VISION_MEM_YIELD_COOLDOWN Seconds after a pressure-stop during which restart is refused regardless of backlog. Default 300.
# VISION_MEM_POLL_INTERVAL Mem sub-poll cadence, seconds (<= VISION_POLL_INTERVAL); catches spikes between backlog ticks. Default 5.