feat(ops): vision-manager sidecar to autoscale the llama.cpp container

Add a tiny privileged sidecar (vision-manager/) that polls the analysis
backlog in Postgres and starts/stops the mangalord-vision container by name:
start when analyze_page jobs are pending, stop after STOP_DEBOUNCE idle.
It is the single owner of the vision lifecycle and the only component with
Docker access — scoped through tecnativa/docker-socket-proxy (CONTAINERS+POST
only) on an internal-only network, so the internet-facing backend never
touches the socket.

Both helpers sit behind `profiles: [ai]` (vanilla `compose up` is
unaffected). The manager debounces the stop, gates the first request on
GET /health==200 after a cold start, honours a crawl RAM-mutex on the 8 GiB
box, and owns its own idle timer (leak-safe if the backend dies). Backlog
query uses the real schema (state + payload->>'kind'); a read-only DB role
(readonly-role.sql) keeps it off the backend creds.

Bump 0.80.0 -> 0.81.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

.env.example

@@ -189,3 +189,32 @@ BACKEND_PROXY_TIMEOUT_MS=300000
 #   ANALYSIS_GROUNDING_PROMPT  Override the tags/scene/safety (pass B) prompt.
 # Leave the prompt vars unset to use the built-in defaults (also editable,
 # with a per-prompt "reset to default", in the dashboard).
+
+# ----- Vision autoscaling (the `ai` compose profile) -----
+# The mangalord-vision (llama.cpp) container pins ~4.4 GiB and has no
+# idle-unload, so the `vision-manager` sidecar starts it on demand and
+# idle-stops it. Bring the two helper containers up with:
+#   docker compose --profile ai up -d
+# The vision container itself is NOT defined in compose — it lives elsewhere
+# on the host and the manager drives it by name. See VISION-AUTOSCALE.md.
+#
+# ANALYSIS_VISION_HEALTH_URL — env-ONLY readiness gate for the analysis
+# worker. When set, the worker refuses to lease a page until this answers
+# 2xx, so the manager can stop vision mid-idle without jobs burning their
+# retries / landing `failed` rows. MUST point at the same vision instance the
+# worker analyses against. Leave empty to disable the gate (always-on setups).
+ANALYSIS_VISION_HEALTH_URL=http://mangalord-vision:8000/health
+#
+# vision-manager knobs:
+# VISION_MANAGER_DATABASE_URL — REQUIRED for the `ai` profile. A read-only
+#   role, NOT the backend creds: apply vision-manager/readonly-role.sql once,
+#   then set e.g.
+#   VISION_MANAGER_DATABASE_URL=postgres://vision_manager:<pw>@postgres:5432/mangalord
+VISION_MANAGER_DATABASE_URL=
+# VISION_CONTAINER          Container name the manager starts/stops. Default mangalord-vision.
+# VISION_HEALTH_URL         /health URL the manager polls after start. Default http://mangalord-vision:8000/health.
+# VISION_POLL_INTERVAL      Backlog poll cadence, seconds. Default 20.
+# VISION_STOP_DEBOUNCE      Idle seconds before stopping vision. Default 600 (debounces bursty enqueues).
+# VISION_START_HEALTH_TIMEOUT  Max seconds to wait for /health 200 after start. Default 300 (cold model load).
+# VISION_RESPECT_CRAWL_MUTEX   1 = don't start vision while a crawl runs (RAM mutex on the 8 GiB box). Default 1.
+# VISION_MAX_UPTIME         >0 = force-stop vision after N seconds running (backstop). Default 0 (disabled).

VISION-AUTOSCALE.md

@@ -0,0 +1,129 @@
+# Vision auto start/stop — design brief for the dev agent
+
+**Goal:** the `mangalord-vision` (llama.cpp) container should only run while there
+is analysis work, and stop (freeing its ~4 GiB) once the queue drains —
+*without* handing the internet-facing backend control of the host Docker daemon.
+
+`llama-server` has **no idle-unload** (unlike Ollama). The only lever is the
+container lifecycle: start it when work appears, stop it when work is gone.
+
+> **Status: implemented.** The sidecar lives in [vision-manager/](vision-manager/)
+> (`manager.sh` + `Dockerfile` + `readonly-role.sql`), wired into
+> [docker-compose.yml](docker-compose.yml) behind `profiles: [ai]` alongside a
+> scoped `docker-socket-proxy`. The companion **readiness gate** is in the
+> analysis worker: when `ANALYSIS_VISION_HEALTH_URL` is set the worker will not
+> lease a page until vision answers `GET /health` 2xx, so an idle-stopped or
+> still-loading vision never burns a job's retries or writes a `failed` row
+> (the gotcha called out below). Configure via the `ai`-profile vars in
+> [.env.example](.env.example).
+
+## Chosen approach — Option 2: a "vision-manager" sidecar
+
+A tiny, single-purpose container that:
+
+1. **Watches the analysis backlog** in Postgres (read-only DB user).
+2. **Starts** `mangalord-vision` when there is pending work.
+3. **Stops** it after the backlog has been empty for a debounce window.
+
+The backend (`mangalord-backend`) is **not modified and gets no Docker access** —
+all privilege is isolated in the manager. This is the whole point: a popped
+backend can't reach the Docker socket.
+
+```
+            ┌────────────────┐   reads pending count   ┌────────────┐
+            │ vision-manager │ ───────────────────────▶│  postgres  │
+            │  (has socket)  │                          └────────────┘
+            └──────┬─────────┘
+   start/stop one  │  (raw socket, OR scoped via docker-socket-proxy)
+   container only  ▼
+            ┌────────────────┐
+            │ mangalord-vision│  (profiles: [ai], started by name)
+            └────────────────┘
+```
+
+### What to poll (concrete)
+
+The analysis daemon consumes the **shared `crawler_jobs` queue, filtered by the
+analysis job kind**, and every unanalyzed page has a `page_analysis` row with
+`status='pending'`. Either is a valid "is there work?" signal:
+
+- **Queue-accurate (what the manager uses):** the queue column is `state`
+  (not `status`) and the kind lives in the JSONB payload — see
+  [backend/migrations/0012_crawler.sql](backend/migrations/0012_crawler.sql):
+  ```sql
+  SELECT count(*) FROM crawler_jobs
+   WHERE payload->>'kind' = 'analyze_page'
+     AND state IN ('pending','running');
+  ```
+- **Backlog-simple:** `SELECT count(*) FROM page_analysis WHERE status='pending'`.
+
+Prefer a **read-only** DB role scoped to those tables. Don't reuse the backend's
+DB credentials.
+
+### Lifecycle logic (sketch)
+
+```
+loop every POLL_INTERVAL (e.g. 20s):
+    pending = count_pending_work()
+    if pending > 0 and vision is stopped:
+        docker start mangalord-vision
+        wait for GET /health == 200 (up to a few minutes — cold model load)
+        reset idle timer
+    if pending == 0 and vision is running:
+        if idle for >= STOP_DEBOUNCE (e.g. 5–10 min):
+            docker stop mangalord-vision
+```
+
+## Recommendations
+
+- **Scope the Docker access.** Best: run `tecnativa/docker-socket-proxy` on an
+  internal-only network with everything disabled except container start/stop,
+  and point the manager at the proxy. Acceptable for a tiny trusted manager:
+  mount the raw socket *into the manager only* (never the backend).
+- **Keep the manager dumb and small.** ~100 lines. A shell loop with
+  `docker`/`curl`, or a small Go/Rust binary. No web surface.
+- **Make it the single source of truth** for the vision lifecycle. Don't also
+  have the backend or a cron poke the same container.
+- **Start by container name**, not `compose up` (the service is `profiles:[ai]`;
+  name-based `docker start/stop` works and won't fight a separate `compose up`).
+- Optionally expose the toggle as a runtime setting ("auto-manage vision: on/off")
+  so it can be disabled without redeploying.
+
+## Pitfalls / gotchas (flag all of these)
+
+- **Cold start is expensive (~minutes)** to mmap 3.2 GiB + warm up. So:
+  - **Debounce the STOP** (5–10 min idle) — bursty uploads/re-analysis enqueue
+    in waves; stopping the instant the queue hits zero thrashes the load cycle
+    and loses more time than it saves.
+  - **Gate the first request on `GET /health == 200`** after start, or the first
+    jobs fail with connection-refused. (The backend already has request/job
+    timeouts — 300s/1800s — but a multi-minute cold start can still exceed a
+    single request timeout if a job is dispatched too eagerly.)
+- **Idempotency / single-flight:** starting an already-running container must be
+  a no-op; never issue concurrent starts. One manager instance only.
+- **Leak safety / don't depend on a "drained" signal from the backend:** the
+  manager's own idle timer must stop the container even if the backend crashes
+  mid-batch. Conversely, if the manager dies, the container keeps running
+  (harmless, just RAM) — consider a max-uptime backstop.
+- **RAM headroom on the 8 GiB Pi:** vision sits ~4.4 GiB; `mem_limit: 6g` is set
+  on the service. The manager must not start vision while another big consumer
+  (a crawl with Chromium) is running, or the box OOMs. Consider a simple mutual
+  exclusion / total-RAM check.
+- **Health vs readiness:** `/health` returns 200 once the model is loaded; that
+  is the readiness signal. Don't treat "container running" as "ready".
+- **Re-analysis storms:** an admin "re-enqueue all" can flood the queue; the
+  manager will (correctly) start vision and keep it up for a long backfill —
+  expected, but make sure the STOP debounce doesn't bounce it mid-backfill if
+  the queue briefly empties between batches.
+- **Profile interaction:** `docker compose --profile ai up` / CI deploys name
+  only the two mangalord services, so they won't start/stop vision — but a
+  human running a full `compose --profile ai up -d` could. Document that the
+  manager owns the lifecycle.
+
+## Why not give the backend the socket directly
+
+`mangalord-backend` is internet-facing (behind Caddy at manga.mc02.dev). Mounting
+the raw Docker socket there makes any backend RCE a full host takeover
+(postgres, gitea, vaultwarden, …). If the backend *must* drive it, use the
+docker-socket-proxy scoped to start/stop of the single container — never the raw
+socket. Option 2 sidesteps the question entirely by keeping the backend clean.

backend/Cargo.lock

@@ -1517,7 +1517,7 @@ checksum = "c41e0c4fef86961ac6d6f8a82609f55f31b05e4fce149ac5710e439df7619ba4"
 
 [[package]]
 name = "mangalord"
-version = "0.80.0"
+version = "0.81.0"
 dependencies = [
  "anyhow",
  "argon2",

backend/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "mangalord"
-version = "0.80.0"
+version = "0.81.0"
 edition = "2021"
 default-run = "mangalord"
 

docker-compose.yml

@@ -91,6 +91,13 @@ services:
       CRAWLER_TOR_CONTROL_URL: ${CRAWLER_TOR_CONTROL_URL-tcp://tor:9051}
       CRAWLER_TOR_CONTROL_PASSWORD: ${TOR_CONTROL_PASSWORD:?TOR_CONTROL_PASSWORD must be set in .env}
       CRAWLER_TOR_RECIRCUIT_MAX_ATTEMPTS: ${CRAWLER_TOR_RECIRCUIT_MAX_ATTEMPTS:-3}
+      # Vision readiness gate (env-ONLY, not a dashboard setting). When set,
+      # the analysis worker refuses to lease a page until this answers 2xx, so
+      # the vision-manager autoscaler can idle-stop the vision container
+      # without jobs burning their retries / landing `failed` rows. Leave
+      # empty (the default) to disable the gate for an always-on endpoint.
+      # Pair with the `ai` profile services below. See VISION-AUTOSCALE.md.
+      ANALYSIS_VISION_HEALTH_URL: ${ANALYSIS_VISION_HEALTH_URL:-}
     volumes:
       - storage-data:/var/lib/mangalord/storage
     # No host port mapping in the default setup — the frontend proxies
@@ -111,6 +118,62 @@ services:
     ports:
       - "3000:3000"
 
+  # ----- Vision autoscaling (profile: ai) -----------------------------------
+  # Two extra containers that idle-stop the mangalord-vision (llama.cpp)
+  # container when there is no analysis work and start it back up on demand.
+  # Gated behind `profiles: [ai]` so a vanilla `docker compose up` is
+  # unaffected — bring them up with `docker compose --profile ai up -d`.
+  # The vision container itself is NOT defined here (it lives elsewhere on the
+  # host); the manager drives it by name. See VISION-AUTOSCALE.md.
+
+  # Scoped Docker access for the manager: exposes ONLY the /containers API and
+  # write methods (start/stop) over TCP on an internal-only network. The raw
+  # host socket is mounted HERE and nowhere else — never on the backend.
+  docker-socket-proxy:
+    image: tecnativa/docker-socket-proxy:latest
+    profiles: ["ai"]
+    environment:
+      CONTAINERS: 1   # allow /containers/* (inspect, start, stop)
+      POST: 1         # allow write methods (start/stop are POSTs)
+      # Everything else stays denied (images, exec, networks, volumes, ...).
+    volumes:
+      - /var/run/docker.sock:/var/run/docker.sock:ro
+    networks:
+      - vision-internal
+    restart: unless-stopped
+
+  vision-manager:
+    build: ./vision-manager
+    profiles: ["ai"]
+    depends_on:
+      postgres:
+        condition: service_healthy
+      docker-socket-proxy:
+        condition: service_started
+    environment:
+      # Read-only role — apply vision-manager/readonly-role.sql once, then set
+      # VISION_MANAGER_DATABASE_URL in .env. Do NOT reuse the backend creds.
+      DATABASE_URL: ${VISION_MANAGER_DATABASE_URL:?set VISION_MANAGER_DATABASE_URL in .env (vision_manager read-only role)}
+      VISION_CONTAINER: ${VISION_CONTAINER:-mangalord-vision}
+      VISION_HEALTH_URL: ${VISION_HEALTH_URL:-http://mangalord-vision:8000/health}
+      DOCKER_HOST: tcp://docker-socket-proxy:2375
+      POLL_INTERVAL: ${VISION_POLL_INTERVAL:-20}
+      STOP_DEBOUNCE: ${VISION_STOP_DEBOUNCE:-600}
+      START_HEALTH_TIMEOUT: ${VISION_START_HEALTH_TIMEOUT:-300}
+      RESPECT_CRAWL_MUTEX: ${VISION_RESPECT_CRAWL_MUTEX:-1}
+      MAX_UPTIME: ${VISION_MAX_UPTIME:-0}
+    networks:
+      - default          # reach postgres + the vision container by name
+      - vision-internal  # reach the socket-proxy
+    restart: unless-stopped
+
+networks:
+  default:
+  # Internal-only: no route to the outside world. Only the manager and the
+  # socket-proxy sit on it, so nothing else can reach the Docker API.
+  vision-internal:
+    internal: true
+
 volumes:
   postgres-data:
   storage-data:

frontend/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mangalord-frontend",
-  "version": "0.80.0",
+  "version": "0.81.0",
   "private": true,
   "type": "module",
   "scripts": {

vision-manager/Dockerfile

@@ -0,0 +1,17 @@
+# vision-manager — tiny sidecar that starts/stops mangalord-vision by name
+# according to the analysis backlog. See manager.sh and VISION-AUTOSCALE.md.
+FROM alpine:3.20
+
+# bash (the script uses arrays/arithmetic), curl (/health probe),
+# postgresql-client (psql backlog query), docker-cli (start/stop via the
+# socket-proxy). No daemon, no compiled build.
+RUN apk add --no-cache bash curl postgresql-client docker-cli
+
+COPY manager.sh /usr/local/bin/manager.sh
+RUN chmod +x /usr/local/bin/manager.sh
+
+# Run unprivileged: the container only needs to reach the socket-proxy over
+# TCP and Postgres — it never touches the host socket directly.
+USER nobody
+
+ENTRYPOINT ["bash", "/usr/local/bin/manager.sh"]

vision-manager/manager.sh

@@ -0,0 +1,146 @@
+#!/usr/bin/env bash
+# vision-manager — start/stop the mangalord-vision (llama.cpp) container by
+# name according to the analysis backlog in Postgres.
+#
+# Why this exists: llama-server has no idle-unload and pins ~4.4 GiB. On the
+# 8 GiB box vision must only run while there is analysis work. This sidecar is
+# the SINGLE owner of the vision lifecycle and the only component with Docker
+# access (scoped through docker-socket-proxy) — the internet-facing backend
+# never gets the socket. See VISION-AUTOSCALE.md.
+#
+# It is deliberately dumb: a poll loop with psql + curl + docker. No web
+# surface, no state beyond an in-memory idle timer. If it dies, vision keeps
+# running (harmless RAM); a MAX_UPTIME backstop bounds that.
+set -euo pipefail
+
+# ---- Config (all overridable via env) --------------------------------------
+: "${DATABASE_URL:?DATABASE_URL must be set (use the read-only vision_manager role)}"
+VISION_CONTAINER="${VISION_CONTAINER:-mangalord-vision}"
+VISION_HEALTH_URL="${VISION_HEALTH_URL:-http://mangalord-vision:8000/health}"
+# DOCKER_HOST points at the scoped docker-socket-proxy by default.
+export DOCKER_HOST="${DOCKER_HOST:-tcp://docker-socket-proxy:2375}"
+
+POLL_INTERVAL="${POLL_INTERVAL:-20}"        # seconds between backlog checks
+STOP_DEBOUNCE="${STOP_DEBOUNCE:-600}"       # idle seconds before stopping vision
+START_HEALTH_TIMEOUT="${START_HEALTH_TIMEOUT:-300}"  # max wait for /health 200
+HEALTH_POLL_INTERVAL="${HEALTH_POLL_INTERVAL:-5}"    # poll cadence while warming
+MAX_UPTIME="${MAX_UPTIME:-0}"               # >0: force-stop after N idle-or-not seconds running (backstop); 0 disables
+# When >0, refuse to start vision while a crawl is running (Chromium + vision
+# together OOM the 8 GiB box). Set to 0 on roomier hosts.
+RESPECT_CRAWL_MUTEX="${RESPECT_CRAWL_MUTEX:-1}"
+
+PSQL=(psql "$DATABASE_URL" -At -v ON_ERROR_STOP=1)
+
+log() { echo "[vision-manager] $(date -u +%FT%TZ) $*"; }
+
+# ---- Queries ---------------------------------------------------------------
+# Pending analysis work: pending OR currently-leased analyze_page jobs. Note
+# the queue column is `state` (not `status`) and the kind lives in the JSONB
+# payload — see backend/migrations/0012_crawler.sql.
+pending_analysis() {
+  "${PSQL[@]}" -c \
+    "SELECT count(*) FROM crawler_jobs
+      WHERE payload->>'kind' = 'analyze_page'
+        AND state IN ('pending','running');" 2>/dev/null || echo "ERR"
+}
+
+# A crawl is in-flight if any non-analyze job is currently running.
+crawl_running() {
+  "${PSQL[@]}" -c \
+    "SELECT count(*) FROM crawler_jobs
+      WHERE payload->>'kind' <> 'analyze_page'
+        AND state = 'running';" 2>/dev/null || echo "ERR"
+}
+
+# ---- Docker helpers --------------------------------------------------------
+# `docker inspect .State.Running` → true/false; empty if the container is
+# absent (defined elsewhere and not yet created) — treated as not-running.
+vision_running() {
+  docker inspect -f '{{.State.Running}}' "$VISION_CONTAINER" 2>/dev/null || echo "false"
+}
+
+vision_ready() {
+  curl -fsS -m "$HEALTH_POLL_INTERVAL" -o /dev/null "$VISION_HEALTH_URL" 2>/dev/null
+}
+
+start_vision() {
+  log "starting $VISION_CONTAINER"
+  # Idempotent: starting an already-running container is a docker no-op.
+  if ! docker start "$VISION_CONTAINER" >/dev/null 2>&1; then
+    log "ERROR: docker start failed (is $VISION_CONTAINER defined on this host?)"
+    return 1
+  fi
+  local waited=0
+  while ! vision_ready; do
+    if [ "$waited" -ge "$START_HEALTH_TIMEOUT" ]; then
+      log "WARNING: $VISION_CONTAINER not healthy after ${START_HEALTH_TIMEOUT}s; leaving it running"
+      return 1
+    fi
+    sleep "$HEALTH_POLL_INTERVAL"
+    waited=$((waited + HEALTH_POLL_INTERVAL))
+  done
+  log "$VISION_CONTAINER healthy after ~${waited}s"
+}
+
+stop_vision() {
+  log "stopping $VISION_CONTAINER (idle ${STOP_DEBOUNCE}s)"
+  docker stop "$VISION_CONTAINER" >/dev/null 2>&1 || log "WARNING: docker stop failed"
+}
+
+# ---- Main loop -------------------------------------------------------------
+log "started: container=$VISION_CONTAINER health=$VISION_HEALTH_URL poll=${POLL_INTERVAL}s debounce=${STOP_DEBOUNCE}s docker_host=$DOCKER_HOST"
+
+idle_for=0   # seconds the queue has been empty while vision is running
+up_for=0     # seconds vision has been running (for MAX_UPTIME backstop)
+
+while true; do
+  pending="$(pending_analysis)"
+  running="$(vision_running)"
+
+  if [ "$pending" = "ERR" ]; then
+    log "WARNING: backlog query failed; will retry next tick"
+    sleep "$POLL_INTERVAL"
+    continue
+  fi
+
+  if [ "$running" = "true" ]; then
+    up_for=$((up_for + POLL_INTERVAL))
+  else
+    up_for=0
+  fi
+
+  if [ "$pending" -gt 0 ]; then
+    idle_for=0
+    if [ "$running" != "true" ]; then
+      if [ "$RESPECT_CRAWL_MUTEX" != "0" ] && [ "$(crawl_running)" != "0" ]; then
+        log "deferring start: $pending analysis job(s) pending but a crawl is running (RAM mutex)"
+      else
+        start_vision || true
+      fi
+    fi
+  else
+    # No work. Debounce the stop so bursty enqueues don't thrash the load
+    # cycle, and own the stop on our OWN timer (never wait for a backend
+    # "drained" signal — leak safety).
+    if [ "$running" = "true" ]; then
+      idle_for=$((idle_for + POLL_INTERVAL))
+      if [ "$idle_for" -ge "$STOP_DEBOUNCE" ]; then
+        stop_vision
+        idle_for=0
+        up_for=0
+      fi
+    else
+      idle_for=0
+    fi
+  fi
+
+  # Backstop: bound how long vision can stay up regardless of debounce state.
+  if [ "$MAX_UPTIME" -gt 0 ] && [ "$running" = "true" ] && [ "$up_for" -ge "$MAX_UPTIME" ]; then
+    log "MAX_UPTIME ${MAX_UPTIME}s reached; force-stopping $VISION_CONTAINER"
+    stop_vision
+    idle_for=0
+    up_for=0
+  fi
+
+  sleep "$POLL_INTERVAL"
+done

vision-manager/readonly-role.sql

@@ -0,0 +1,32 @@
+-- Read-only DB role for vision-manager.
+--
+-- The sidecar only needs to count pending analysis work; give it SELECT on
+-- crawler_jobs and nothing else. It must NOT reuse the backend's credentials.
+--
+-- The Postgres service mounts no init dir and the data volume already exists,
+-- so this is applied ONCE by an operator (it is idempotent):
+--
+--   docker compose exec -T postgres \
+--     psql -U "$POSTGRES_USER" -d "$POSTGRES_DB" -v pw="<a-strong-password>" \
+--     -f - < vision-manager/readonly-role.sql
+--
+-- Then point the manager at it (VISION_MANAGER_DATABASE_URL in .env):
+--   postgres://vision_manager:<a-strong-password>@postgres:5432/<POSTGRES_DB>
+--
+-- The password is passed via psql's -v pw=... ; psql substitutes :'pw' as a
+-- quoted literal and :"DBNAME" (psql's built-in) as the current db identifier.
+-- These substitutions only happen in plain statements, NOT inside a
+-- dollar-quoted DO block — hence the \gexec form below.
+
+-- Create the LOGIN role only if it is absent (idempotent).
+SELECT 'CREATE ROLE vision_manager LOGIN'
+ WHERE NOT EXISTS (SELECT FROM pg_roles WHERE rolname = 'vision_manager')
+\gexec
+
+-- (Re)set the password every run so rotating it is just a re-apply.
+ALTER ROLE vision_manager LOGIN PASSWORD :'pw';
+
+-- CONNECT to the current database, and read-only on the one table we poll.
+GRANT CONNECT ON DATABASE :"DBNAME" TO vision_manager;
+GRANT USAGE ON SCHEMA public TO vision_manager;
+GRANT SELECT ON crawler_jobs TO vision_manager;