fix(analysis): harden readiness gate and vision-manager per self-review

Backend:
- Log on readiness state transitions (ready<->not-ready) so a misconfigured
  ANALYSIS_VISION_HEALTH_URL, which otherwise parks the worker silently
  forever, is diagnosable.
- Add unit tests for HttpVisionReadiness: 2xx->ready, non-2xx->not-ready,
  connection error->not-ready (the production status mapping had no test).

vision-manager:
- Guard the backlog count against non-numeric output before `-gt`, so a stray
  value can't exit-2 and kill the loop under `set -e`.
- Throttle the crawl-mutex "deferring start" log to once per episode.
- Only reset the idle/uptime timers when `docker stop` actually succeeds, so a
  failed stop retries next tick instead of waiting a full debounce window.
- Decouple the per-probe curl timeout (HEALTH_TIMEOUT) from the warm-up poll
  cadence.

Docs:
- Correct the docker-socket-proxy comment: CONTAINERS+POST permits the full
  container lifecycle (not just start/stop); state the real trust boundary.
- Document that the externally-defined mangalord-vision container must share
  the compose network for name resolution.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

.env.example

@@ -189,3 +189,32 @@ BACKEND_PROXY_TIMEOUT_MS=300000
 #   ANALYSIS_GROUNDING_PROMPT  Override the tags/scene/safety (pass B) prompt.
 # Leave the prompt vars unset to use the built-in defaults (also editable,
 # with a per-prompt "reset to default", in the dashboard).
+
+# ----- Vision autoscaling (the `ai` compose profile) -----
+# The mangalord-vision (llama.cpp) container pins ~4.4 GiB and has no
+# idle-unload, so the `vision-manager` sidecar starts it on demand and
+# idle-stops it. Bring the two helper containers up with:
+#   docker compose --profile ai up -d
+# The vision container itself is NOT defined in compose — it lives elsewhere
+# on the host and the manager drives it by name. See VISION-AUTOSCALE.md.
+#
+# ANALYSIS_VISION_HEALTH_URL — env-ONLY readiness gate for the analysis
+# worker. When set, the worker refuses to lease a page until this answers
+# 2xx, so the manager can stop vision mid-idle without jobs burning their
+# retries / landing `failed` rows. MUST point at the same vision instance the
+# worker analyses against. Leave empty to disable the gate (always-on setups).
+ANALYSIS_VISION_HEALTH_URL=http://mangalord-vision:8000/health
+#
+# vision-manager knobs:
+# VISION_MANAGER_DATABASE_URL — REQUIRED for the `ai` profile. A read-only
+#   role, NOT the backend creds: apply vision-manager/readonly-role.sql once,
+#   then set e.g.
+#   VISION_MANAGER_DATABASE_URL=postgres://vision_manager:<pw>@postgres:5432/mangalord
+VISION_MANAGER_DATABASE_URL=
+# VISION_CONTAINER          Container name the manager starts/stops. Default mangalord-vision.
+# VISION_HEALTH_URL         /health URL the manager polls after start. Default http://mangalord-vision:8000/health.
+# VISION_POLL_INTERVAL      Backlog poll cadence, seconds. Default 20.
+# VISION_STOP_DEBOUNCE      Idle seconds before stopping vision. Default 600 (debounces bursty enqueues).
+# VISION_START_HEALTH_TIMEOUT  Max seconds to wait for /health 200 after start. Default 300 (cold model load).
+# VISION_RESPECT_CRAWL_MUTEX   1 = don't start vision while a crawl runs (RAM mutex on the 8 GiB box). Default 1.
+# VISION_MAX_UPTIME         >0 = force-stop vision after N seconds running (backstop). Default 0 (disabled).

VISION-AUTOSCALE.md

@@ -0,0 +1,138 @@
+# Vision auto start/stop — design brief for the dev agent
+
+**Goal:** the `mangalord-vision` (llama.cpp) container should only run while there
+is analysis work, and stop (freeing its ~4 GiB) once the queue drains —
+*without* handing the internet-facing backend control of the host Docker daemon.
+
+`llama-server` has **no idle-unload** (unlike Ollama). The only lever is the
+container lifecycle: start it when work appears, stop it when work is gone.
+
+> **Status: implemented.** The sidecar lives in [vision-manager/](vision-manager/)
+> (`manager.sh` + `Dockerfile` + `readonly-role.sql`), wired into
+> [docker-compose.yml](docker-compose.yml) behind `profiles: [ai]` alongside a
+> scoped `docker-socket-proxy`. The companion **readiness gate** is in the
+> analysis worker: when `ANALYSIS_VISION_HEALTH_URL` is set the worker will not
+> lease a page until vision answers `GET /health` 2xx, so an idle-stopped or
+> still-loading vision never burns a job's retries or writes a `failed` row
+> (the gotcha called out below). Configure via the `ai`-profile vars in
+> [.env.example](.env.example).
+>
+> **Operator prerequisite:** the `mangalord-vision` container is defined
+> *outside* this compose project (the manager only drives it by name). For the
+> manager and backend to resolve it by name for the `/health` probe, attach it
+> to this project's default network (`<project>_default`, e.g.
+> `mangalord_default`) — or point `VISION_HEALTH_URL` /
+> `ANALYSIS_VISION_HEALTH_URL` at an address that resolves. If the container is
+> unreachable the manager silently treats it as "not running / not ready" and
+> will loop trying to start a container it cannot see.
+
+## Chosen approach — Option 2: a "vision-manager" sidecar
+
+A tiny, single-purpose container that:
+
+1. **Watches the analysis backlog** in Postgres (read-only DB user).
+2. **Starts** `mangalord-vision` when there is pending work.
+3. **Stops** it after the backlog has been empty for a debounce window.
+
+The backend (`mangalord-backend`) is **not modified and gets no Docker access** —
+all privilege is isolated in the manager. This is the whole point: a popped
+backend can't reach the Docker socket.
+
+```
+            ┌────────────────┐   reads pending count   ┌────────────┐
+            │ vision-manager │ ───────────────────────▶│  postgres  │
+            │  (has socket)  │                          └────────────┘
+            └──────┬─────────┘
+   start/stop one  │  (raw socket, OR scoped via docker-socket-proxy)
+   container only  ▼
+            ┌────────────────┐
+            │ mangalord-vision│  (profiles: [ai], started by name)
+            └────────────────┘
+```
+
+### What to poll (concrete)
+
+The analysis daemon consumes the **shared `crawler_jobs` queue, filtered by the
+analysis job kind**, and every unanalyzed page has a `page_analysis` row with
+`status='pending'`. Either is a valid "is there work?" signal:
+
+- **Queue-accurate (what the manager uses):** the queue column is `state`
+  (not `status`) and the kind lives in the JSONB payload — see
+  [backend/migrations/0012_crawler.sql](backend/migrations/0012_crawler.sql):
+  ```sql
+  SELECT count(*) FROM crawler_jobs
+   WHERE payload->>'kind' = 'analyze_page'
+     AND state IN ('pending','running');
+  ```
+- **Backlog-simple:** `SELECT count(*) FROM page_analysis WHERE status='pending'`.
+
+Prefer a **read-only** DB role scoped to those tables. Don't reuse the backend's
+DB credentials.
+
+### Lifecycle logic (sketch)
+
+```
+loop every POLL_INTERVAL (e.g. 20s):
+    pending = count_pending_work()
+    if pending > 0 and vision is stopped:
+        docker start mangalord-vision
+        wait for GET /health == 200 (up to a few minutes — cold model load)
+        reset idle timer
+    if pending == 0 and vision is running:
+        if idle for >= STOP_DEBOUNCE (e.g. 5–10 min):
+            docker stop mangalord-vision
+```
+
+## Recommendations
+
+- **Scope the Docker access.** Best: run `tecnativa/docker-socket-proxy` on an
+  internal-only network with everything disabled except container start/stop,
+  and point the manager at the proxy. Acceptable for a tiny trusted manager:
+  mount the raw socket *into the manager only* (never the backend).
+- **Keep the manager dumb and small.** ~100 lines. A shell loop with
+  `docker`/`curl`, or a small Go/Rust binary. No web surface.
+- **Make it the single source of truth** for the vision lifecycle. Don't also
+  have the backend or a cron poke the same container.
+- **Start by container name**, not `compose up` (the service is `profiles:[ai]`;
+  name-based `docker start/stop` works and won't fight a separate `compose up`).
+- Optionally expose the toggle as a runtime setting ("auto-manage vision: on/off")
+  so it can be disabled without redeploying.
+
+## Pitfalls / gotchas (flag all of these)
+
+- **Cold start is expensive (~minutes)** to mmap 3.2 GiB + warm up. So:
+  - **Debounce the STOP** (5–10 min idle) — bursty uploads/re-analysis enqueue
+    in waves; stopping the instant the queue hits zero thrashes the load cycle
+    and loses more time than it saves.
+  - **Gate the first request on `GET /health == 200`** after start, or the first
+    jobs fail with connection-refused. (The backend already has request/job
+    timeouts — 300s/1800s — but a multi-minute cold start can still exceed a
+    single request timeout if a job is dispatched too eagerly.)
+- **Idempotency / single-flight:** starting an already-running container must be
+  a no-op; never issue concurrent starts. One manager instance only.
+- **Leak safety / don't depend on a "drained" signal from the backend:** the
+  manager's own idle timer must stop the container even if the backend crashes
+  mid-batch. Conversely, if the manager dies, the container keeps running
+  (harmless, just RAM) — consider a max-uptime backstop.
+- **RAM headroom on the 8 GiB Pi:** vision sits ~4.4 GiB; `mem_limit: 6g` is set
+  on the service. The manager must not start vision while another big consumer
+  (a crawl with Chromium) is running, or the box OOMs. Consider a simple mutual
+  exclusion / total-RAM check.
+- **Health vs readiness:** `/health` returns 200 once the model is loaded; that
+  is the readiness signal. Don't treat "container running" as "ready".
+- **Re-analysis storms:** an admin "re-enqueue all" can flood the queue; the
+  manager will (correctly) start vision and keep it up for a long backfill —
+  expected, but make sure the STOP debounce doesn't bounce it mid-backfill if
+  the queue briefly empties between batches.
+- **Profile interaction:** `docker compose --profile ai up` / CI deploys name
+  only the two mangalord services, so they won't start/stop vision — but a
+  human running a full `compose --profile ai up -d` could. Document that the
+  manager owns the lifecycle.
+
+## Why not give the backend the socket directly
+
+`mangalord-backend` is internet-facing (behind Caddy at manga.mc02.dev). Mounting
+the raw Docker socket there makes any backend RCE a full host takeover
+(postgres, gitea, vaultwarden, …). If the backend *must* drive it, use the
+docker-socket-proxy scoped to start/stop of the single container — never the raw
+socket. Option 2 sidesteps the question entirely by keeping the backend clean.

backend/Cargo.lock

@@ -1517,7 +1517,7 @@ checksum = "c41e0c4fef86961ac6d6f8a82609f55f31b05e4fce149ac5710e439df7619ba4"
 
 [[package]]
 name = "mangalord"
-version = "0.80.0"
+version = "0.81.0"
 dependencies = [
  "anyhow",
  "argon2",

backend/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "mangalord"
-version = "0.80.0"
+version = "0.81.0"
 edition = "2021"
 default-run = "mangalord"
 

backend/src/analysis/daemon.rs

@@ -32,6 +32,10 @@ use crate::storage::Storage;
 const LEASE_DURATION: Duration = Duration::from_secs(60);
 /// Heartbeat cadence — a third of the lease window.
 const LEASE_HEARTBEAT: Duration = Duration::from_secs(20);
+/// How long to wait before re-probing vision when the readiness gate is
+/// closed. Short enough to resume promptly after an autoscaler cold start,
+/// long enough not to hammer `/health` while vision is down.
+const READINESS_POLL: Duration = Duration::from_secs(2);
 
 /// The unit of work: analyze one page. Implemented by
 /// [`RealAnalyzeDispatcher`] in production and stubbed in tests.
@@ -40,12 +44,45 @@ pub trait AnalyzeDispatcher: Send + Sync {
     async fn dispatch(&self, page_id: Uuid) -> anyhow::Result<()>;
 }
 
+/// Probe answering "is the vision backend up and the model loaded?". When a
+/// probe is wired in, the worker refuses to *lease* a job until it reports
+/// ready — so an idle-stopped vision (the autoscaler's normal state) never
+/// burns a job's retries or writes a `failed` row. Implemented by
+/// [`HttpVisionReadiness`] in production (a `GET /health`) and stubbed in
+/// tests.
+#[async_trait]
+pub trait VisionReadiness: Send + Sync {
+    async fn ready(&self) -> bool;
+}
+
+/// Production readiness probe: `GET {health_url}`, ready iff it answers 2xx.
+/// Any transport error (connection refused while stopped, 503 while the
+/// model loads) is treated as not-ready.
+pub struct HttpVisionReadiness {
+    pub http: reqwest::Client,
+    pub health_url: String,
+}
+
+#[async_trait]
+impl VisionReadiness for HttpVisionReadiness {
+    async fn ready(&self) -> bool {
+        match self.http.get(&self.health_url).send().await {
+            Ok(resp) => resp.status().is_success(),
+            Err(_) => false,
+        }
+    }
+}
+
 pub struct AnalysisDaemonConfig {
     pub dispatcher: Arc<dyn AnalyzeDispatcher>,
     pub workers: usize,
     pub job_timeout: Duration,
     /// Live-event sink (Started/Completed/Failed) for admin SSE.
     pub events: Arc<AnalysisEvents>,
+    /// Optional vision readiness gate. `None` ⇒ no gate (today's behavior,
+    /// for always-on endpoints). `Some` ⇒ the worker parks instead of
+    /// leasing while the probe is not ready.
+    pub readiness: Option<Arc<dyn VisionReadiness>>,
 }
 
 pub struct AnalysisDaemonHandle {
@@ -75,6 +112,7 @@ pub fn spawn(
             dispatcher: Arc::clone(&cfg.dispatcher),
             job_timeout: cfg.job_timeout,
             events: Arc::clone(&cfg.events),
+            readiness: cfg.readiness.clone(),
             id,
         };
         join.spawn(async move { ctx.run().await });
@@ -88,16 +126,46 @@ struct WorkerContext {
     dispatcher: Arc<dyn AnalyzeDispatcher>,
     job_timeout: Duration,
     events: Arc<AnalysisEvents>,
+    readiness: Option<Arc<dyn VisionReadiness>>,
     id: usize,
 }
 
 impl WorkerContext {
     async fn run(self) {
+        // Last observed readiness, so we log only on transitions (not every
+        // poll). `None` until the first probe.
+        let mut was_ready: Option<bool> = None;
         loop {
             if self.cancel.is_cancelled() {
                 tracing::info!(worker = self.id, "analysis worker: shutdown");
                 return;
             }
+            // Readiness gate: park (without leasing) while vision is down or
+            // still loading its model, so the autoscaler's idle-stop never
+            // costs a job its retries. Probe *before* the lease — leasing
+            // increments `attempts` in SQL and can't be undone.
+            if let Some(readiness) = &self.readiness {
+                let ready = readiness.ready().await;
+                // Log on flip so a typo'd health URL (which parks the worker
+                // forever) is diagnosable, without spamming every poll.
+                if was_ready != Some(ready) {
+                    if ready {
+                        tracing::info!(worker = self.id, "analysis worker: vision ready — resuming");
+                    } else {
+                        tracing::warn!(
+                            worker = self.id,
+                            "analysis worker: vision not ready — parking until /health is 2xx"
+                        );
+                    }
+                    was_ready = Some(ready);
+                }
+                if !ready {
+                    if self.sleep_or_cancel(READINESS_POLL).await {
+                        return;
+                    }
+                    continue;
+                }
+            }
             let leases =
                 match jobs::lease(&self.pool, Some(KIND_ANALYZE_PAGE), 1, LEASE_DURATION).await {
                     Ok(v) => v,
@@ -269,7 +337,29 @@ impl AnalyzeDispatcher for RealAnalyzeDispatcher {
 /// in the `tests/` dir (a separate crate).
 pub mod test_support {
     use super::*;
-    use std::sync::atomic::{AtomicUsize, Ordering};
+    use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
+
+    /// A readiness probe whose answer can be flipped from the test thread, to
+    /// drive the "vision is down, then comes up" transition.
+    pub struct ToggleReadiness {
+        ready: AtomicBool,
+    }
+
+    impl ToggleReadiness {
+        pub fn new(ready: bool) -> Arc<Self> {
+            Arc::new(Self { ready: AtomicBool::new(ready) })
+        }
+        pub fn set(&self, ready: bool) {
+            self.ready.store(ready, Ordering::Release);
+        }
+    }
+
+    #[async_trait]
+    impl VisionReadiness for ToggleReadiness {
+        async fn ready(&self) -> bool {
+            self.ready.load(Ordering::Acquire)
+        }
+    }
 
     /// Counts dispatch calls and returns a configurable result.
     pub struct CountingDispatcher {
@@ -307,3 +397,56 @@ pub mod test_support {
         }
     }
 }
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use tokio::io::{AsyncReadExt, AsyncWriteExt};
+    use tokio::net::TcpListener;
+
+    /// Bind an ephemeral port that answers every request with `status_line`
+    /// (e.g. `"200 OK"`), and return its `/health` URL. The probe under test
+    /// only inspects the status code, so a zero-length body is enough.
+    async fn serve(status_line: &'static str) -> String {
+        let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
+        let addr = listener.local_addr().unwrap();
+        tokio::spawn(async move {
+            while let Ok((mut sock, _)) = listener.accept().await {
+                let mut buf = [0u8; 1024];
+                let _ = sock.read(&mut buf).await;
+                let resp = format!(
+                    "HTTP/1.1 {status_line}\r\nContent-Length: 0\r\nConnection: close\r\n\r\n"
+                );
+                let _ = sock.write_all(resp.as_bytes()).await;
+            }
+        });
+        format!("http://{addr}/health")
+    }
+
+    fn probe(url: String) -> HttpVisionReadiness {
+        HttpVisionReadiness {
+            http: reqwest::Client::builder()
+                .timeout(Duration::from_millis(500))
+                .build()
+                .unwrap(),
+            health_url: url,
+        }
+    }
+
+    #[tokio::test]
+    async fn http_readiness_2xx_is_ready() {
+        assert!(probe(serve("200 OK").await).ready().await);
+    }
+
+    #[tokio::test]
+    async fn http_readiness_non_2xx_is_not_ready() {
+        // llama-server answers 503 while the model is still loading.
+        assert!(!probe(serve("503 Service Unavailable").await).ready().await);
+    }
+
+    #[tokio::test]
+    async fn http_readiness_connection_error_is_not_ready() {
+        // Nothing listening (vision stopped) → not ready, never an error.
+        assert!(!probe("http://127.0.0.1:1/health".to_string()).ready().await);
+    }
+}

backend/src/app.rs

@@ -420,6 +420,24 @@ fn spawn_analysis_daemon(
         model: cfg.model.clone(),
         max_image_bytes: cfg.max_image_bytes,
     });
+    // When a readiness URL is configured, gate leasing on it so an
+    // autoscaler that idle-stops the vision container never lets a job burn
+    // its retries. A dedicated short-timeout client keeps the probe snappy
+    // and independent of the (long) per-request analysis timeout.
+    let readiness: Option<Arc<dyn crate::analysis::daemon::VisionReadiness>> =
+        match &cfg.vision_health_url {
+            Some(url) if !url.is_empty() => {
+                let probe = reqwest::Client::builder()
+                    .timeout(std::time::Duration::from_secs(5))
+                    .build()
+                    .context("build vision readiness http client")?;
+                Some(Arc::new(crate::analysis::daemon::HttpVisionReadiness {
+                    http: probe,
+                    health_url: url.clone(),
+                }))
+            }
+            _ => None,
+        };
     let handle = crate::analysis::daemon::spawn(
         db,
         CancellationToken::new(),
@@ -428,6 +446,7 @@ fn spawn_analysis_daemon(
             workers: cfg.workers,
             job_timeout: cfg.job_timeout,
             events,
+            readiness,
         },
     );
     tracing::info!(workers = cfg.workers, model = %cfg.model, "analysis worker daemon started");

backend/src/config.rs

@@ -124,6 +124,13 @@ pub struct AnalysisConfig {
     pub workers: usize,
     /// OpenAI-compatible chat/completions URL (`ANALYSIS_VISION_URL`).
     pub endpoint: String,
+    /// Optional vision readiness probe URL (`ANALYSIS_VISION_HEALTH_URL`),
+    /// e.g. `http://mangalord-vision:8000/health`. When set, the worker
+    /// refuses to lease a job until this answers 2xx — so an autoscaler that
+    /// idle-stops the vision container never lets a job burn its retries or
+    /// land a `failed` row. `None` (the default) disables the gate, matching
+    /// the prior behavior for always-on endpoints. Env-only, like `api_key`.
+    pub vision_health_url: Option<String>,
     /// Model id to request (`ANALYSIS_MODEL`).
     pub model: String,
     /// Optional bearer token (`ANALYSIS_API_KEY`); local servers usually
@@ -190,6 +197,7 @@ impl Default for AnalysisConfig {
             enabled: false,
             workers: 1,
             endpoint: "http://localhost:8000/v1/chat/completions".to_string(),
+            vision_health_url: None,
             model: String::new(),
             api_key: None,
             request_timeout: Duration::from_secs(120),
@@ -221,6 +229,9 @@ impl AnalysisConfig {
             enabled: env_bool("ANALYSIS_ENABLED", d.enabled),
             workers: env_usize("ANALYSIS_WORKERS", d.workers).max(1),
             endpoint: std::env::var("ANALYSIS_VISION_URL").unwrap_or(d.endpoint),
+            vision_health_url: std::env::var("ANALYSIS_VISION_HEALTH_URL")
+                .ok()
+                .filter(|s| !s.is_empty()),
             model: std::env::var("ANALYSIS_MODEL").unwrap_or(d.model),
             api_key: std::env::var("ANALYSIS_API_KEY")
                 .ok()

backend/src/settings.rs

@@ -410,6 +410,8 @@ impl AnalysisSettings {
             grounding_prompt: prompt(&self.grounding_prompt, GROUNDING_PROMPT_DEFAULT),
             // Env-only secret preserved from the base.
             api_key: base.api_key.clone(),
+            // Env-only readiness probe URL preserved from the base.
+            vision_health_url: base.vision_health_url.clone(),
         })
     }
 }

backend/tests/analysis_readiness.rs

@@ -0,0 +1,149 @@
+//! Integration tests for the analysis worker's *vision readiness gate*.
+//!
+//! When a readiness probe is wired in and reports "not ready" (the vision
+//! container is stopped, or running but the model is still loading), the
+//! worker must NOT lease any `analyze_page` job. Leasing increments
+//! `attempts` in SQL and a down vision then burns all retries and writes a
+//! permanent `failed` row — poisoning the queue while the autoscaler is
+//! mid-cold-start. So a not-ready gate must leave jobs untouched
+//! (`state='pending'`, `attempts=0`, no `page_analysis` row) and resume the
+//! instant the probe flips to ready.
+
+use std::sync::Arc;
+use std::time::Duration;
+
+use mangalord::analysis::daemon::{
+    self,
+    test_support::{CountingDispatcher, ToggleReadiness},
+    AnalysisDaemonConfig,
+};
+use mangalord::repo::page_analysis;
+use sqlx::PgPool;
+use tokio_util::sync::CancellationToken;
+use uuid::Uuid;
+
+async fn seed_page(pool: &PgPool) -> Uuid {
+    let manga_id: Uuid =
+        sqlx::query_scalar("INSERT INTO mangas (title) VALUES ('M') RETURNING id")
+            .fetch_one(pool)
+            .await
+            .unwrap();
+    let chapter_id: Uuid =
+        sqlx::query_scalar("INSERT INTO chapters (manga_id, number) VALUES ($1, 1) RETURNING id")
+            .bind(manga_id)
+            .fetch_one(pool)
+            .await
+            .unwrap();
+    sqlx::query_scalar(
+        "INSERT INTO pages (chapter_id, page_number, storage_key, content_type) \
+         VALUES ($1, 1, 'k/1.png', 'image/png') RETURNING id",
+    )
+    .bind(chapter_id)
+    .fetch_one(pool)
+    .await
+    .unwrap()
+}
+
+async fn job_row(pool: &PgPool, page_id: Uuid) -> (String, i32) {
+    sqlx::query_as(
+        "SELECT state, attempts FROM crawler_jobs \
+          WHERE payload->>'kind' = 'analyze_page' AND payload->>'page_id' = $1",
+    )
+    .bind(page_id.to_string())
+    .fetch_one(pool)
+    .await
+    .unwrap()
+}
+
+async fn wait_for_state(pool: &PgPool, page_id: Uuid, want: &str) {
+    let result = tokio::time::timeout(Duration::from_secs(8), async {
+        loop {
+            if job_row(pool, page_id).await.0 == want {
+                return;
+            }
+            tokio::time::sleep(Duration::from_millis(50)).await;
+        }
+    })
+    .await;
+    assert!(result.is_ok(), "job for {page_id} never reached state {want}");
+}
+
+fn spawn_with(
+    pool: &PgPool,
+    dispatcher: Arc<CountingDispatcher>,
+    readiness: Option<Arc<dyn daemon::VisionReadiness>>,
+) -> daemon::AnalysisDaemonHandle {
+    daemon::spawn(
+        pool.clone(),
+        CancellationToken::new(),
+        AnalysisDaemonConfig {
+            dispatcher,
+            workers: 1,
+            job_timeout: Duration::from_secs(5),
+            events: Arc::new(mangalord::analysis::events::AnalysisEvents::new()),
+            readiness,
+        },
+    )
+}
+
+#[sqlx::test(migrations = "./migrations")]
+async fn not_ready_gate_never_leases_or_fails_the_page(pool: PgPool) {
+    let page_id = seed_page(&pool).await;
+    page_analysis::enqueue_for_page(&pool, page_id, false)
+        .await
+        .unwrap();
+
+    let dispatcher = CountingDispatcher::ok();
+    let readiness = ToggleReadiness::new(false);
+    let handle = spawn_with(&pool, dispatcher.clone(), Some(readiness.clone()));
+
+    // Give the worker several poll cycles to (wrongly) lease if the gate is
+    // broken.
+    tokio::time::sleep(Duration::from_millis(800)).await;
+    handle.shutdown().await;
+
+    assert_eq!(dispatcher.call_count(), 0, "must not dispatch while not ready");
+    let (state, attempts) = job_row(&pool, page_id).await;
+    assert_eq!(state, "pending", "job must stay pending while vision is down");
+    assert_eq!(attempts, 0, "job must not be leased (attempts must stay 0)");
+    assert!(
+        page_analysis::load(&pool, page_id).await.unwrap().is_none(),
+        "no failed/any page_analysis row may be written while not ready"
+    );
+}
+
+#[sqlx::test(migrations = "./migrations")]
+async fn resumes_when_readiness_flips_to_ready(pool: PgPool) {
+    let page_id = seed_page(&pool).await;
+    page_analysis::enqueue_for_page(&pool, page_id, false)
+        .await
+        .unwrap();
+
+    let dispatcher = CountingDispatcher::ok();
+    let readiness = ToggleReadiness::new(false);
+    let handle = spawn_with(&pool, dispatcher.clone(), Some(readiness.clone()));
+
+    // Confirm it is parked, then open the gate.
+    tokio::time::sleep(Duration::from_millis(300)).await;
+    assert_eq!(dispatcher.call_count(), 0);
+    readiness.set(true);
+
+    wait_for_state(&pool, page_id, "done").await;
+    handle.shutdown().await;
+    assert_eq!(dispatcher.call_count(), 1, "must process once ready");
+}
+
+#[sqlx::test(migrations = "./migrations")]
+async fn no_gate_processes_as_before(pool: PgPool) {
+    // readiness = None preserves today's behavior for always-on endpoints.
+    let page_id = seed_page(&pool).await;
+    page_analysis::enqueue_for_page(&pool, page_id, false)
+        .await
+        .unwrap();
+
+    let dispatcher = CountingDispatcher::ok();
+    let handle = spawn_with(&pool, dispatcher.clone(), None);
+    wait_for_state(&pool, page_id, "done").await;
+    handle.shutdown().await;
+    assert_eq!(dispatcher.call_count(), 1);
+}

backend/tests/analysis_worker.rs

@@ -78,6 +78,7 @@ fn spawn_with(
             events: std::sync::Arc::new(
                 mangalord::analysis::events::AnalysisEvents::new(),
             ),
+            readiness: None,
         },
     );
     (handle, cancel)
@@ -227,6 +228,7 @@ async fn worker_publishes_started_and_completed_events(pool: PgPool) {
             workers: 1,
             job_timeout: Duration::from_secs(5),
             events: events.clone(),
+            readiness: None,
         },
     );
 
@@ -277,6 +279,7 @@ async fn worker_publishes_failed_event_on_dispatch_error(pool: PgPool) {
             workers: 1,
             job_timeout: Duration::from_secs(5),
             events: events.clone(),
+            readiness: None,
         },
     );
 

docker-compose.yml

@@ -91,6 +91,13 @@ services:
       CRAWLER_TOR_CONTROL_URL: ${CRAWLER_TOR_CONTROL_URL-tcp://tor:9051}
       CRAWLER_TOR_CONTROL_PASSWORD: ${TOR_CONTROL_PASSWORD:?TOR_CONTROL_PASSWORD must be set in .env}
       CRAWLER_TOR_RECIRCUIT_MAX_ATTEMPTS: ${CRAWLER_TOR_RECIRCUIT_MAX_ATTEMPTS:-3}
+      # Vision readiness gate (env-ONLY, not a dashboard setting). When set,
+      # the analysis worker refuses to lease a page until this answers 2xx, so
+      # the vision-manager autoscaler can idle-stop the vision container
+      # without jobs burning their retries / landing `failed` rows. Leave
+      # empty (the default) to disable the gate for an always-on endpoint.
+      # Pair with the `ai` profile services below. See VISION-AUTOSCALE.md.
+      ANALYSIS_VISION_HEALTH_URL: ${ANALYSIS_VISION_HEALTH_URL:-}
     volumes:
       - storage-data:/var/lib/mangalord/storage
     # No host port mapping in the default setup — the frontend proxies
@@ -111,6 +118,68 @@ services:
     ports:
       - "3000:3000"
 
+  # ----- Vision autoscaling (profile: ai) -----------------------------------
+  # Two extra containers that idle-stop the mangalord-vision (llama.cpp)
+  # container when there is no analysis work and start it back up on demand.
+  # Gated behind `profiles: [ai]` so a vanilla `docker compose up` is
+  # unaffected — bring them up with `docker compose --profile ai up -d`.
+  # The vision container itself is NOT defined here (it lives elsewhere on the
+  # host); the manager drives it by name. See VISION-AUTOSCALE.md.
+
+  # Scoped Docker access for the manager. The proxy gates by API *section*
+  # (not per-method), so CONTAINERS=1 + POST=1 permits the full /containers
+  # lifecycle — inspect/start/stop, but also create/kill/restart/update/
+  # rename/remove. It does NOT expose exec, images, volumes, networks, swarm,
+  # etc. (all default-denied). The trust boundary is therefore: (a) this is an
+  # internal-only network reachable solely by vision-manager, and (b) the raw
+  # host socket is mounted HERE and nowhere else — never on the backend. A
+  # backend RCE still cannot reach the Docker API. If you need true start/stop-
+  # only granularity, front the socket with an allow-list reverse proxy instead.
+  docker-socket-proxy:
+    image: tecnativa/docker-socket-proxy:latest
+    profiles: ["ai"]
+    environment:
+      CONTAINERS: 1   # allow the /containers/* section
+      POST: 1         # allow write methods (start/stop are POSTs)
+      # Everything else stays at its default-deny (EXEC, IMAGES, NETWORKS, ...).
+    volumes:
+      - /var/run/docker.sock:/var/run/docker.sock:ro
+    networks:
+      - vision-internal
+    restart: unless-stopped
+
+  vision-manager:
+    build: ./vision-manager
+    profiles: ["ai"]
+    depends_on:
+      postgres:
+        condition: service_healthy
+      docker-socket-proxy:
+        condition: service_started
+    environment:
+      # Read-only role — apply vision-manager/readonly-role.sql once, then set
+      # VISION_MANAGER_DATABASE_URL in .env. Do NOT reuse the backend creds.
+      DATABASE_URL: ${VISION_MANAGER_DATABASE_URL:?set VISION_MANAGER_DATABASE_URL in .env (vision_manager read-only role)}
+      VISION_CONTAINER: ${VISION_CONTAINER:-mangalord-vision}
+      VISION_HEALTH_URL: ${VISION_HEALTH_URL:-http://mangalord-vision:8000/health}
+      DOCKER_HOST: tcp://docker-socket-proxy:2375
+      POLL_INTERVAL: ${VISION_POLL_INTERVAL:-20}
+      STOP_DEBOUNCE: ${VISION_STOP_DEBOUNCE:-600}
+      START_HEALTH_TIMEOUT: ${VISION_START_HEALTH_TIMEOUT:-300}
+      RESPECT_CRAWL_MUTEX: ${VISION_RESPECT_CRAWL_MUTEX:-1}
+      MAX_UPTIME: ${VISION_MAX_UPTIME:-0}
+    networks:
+      - default          # reach postgres + the vision container by name
+      - vision-internal  # reach the socket-proxy
+    restart: unless-stopped
+
+networks:
+  default:
+  # Internal-only: no route to the outside world. Only the manager and the
+  # socket-proxy sit on it, so nothing else can reach the Docker API.
+  vision-internal:
+    internal: true
+
 volumes:
   postgres-data:
   storage-data:

frontend/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mangalord-frontend",
-  "version": "0.80.0",
+  "version": "0.81.0",
   "private": true,
   "type": "module",
   "scripts": {

vision-manager/Dockerfile

@@ -0,0 +1,17 @@
+# vision-manager — tiny sidecar that starts/stops mangalord-vision by name
+# according to the analysis backlog. See manager.sh and VISION-AUTOSCALE.md.
+FROM alpine:3.20
+
+# bash (the script uses arrays/arithmetic), curl (/health probe),
+# postgresql-client (psql backlog query), docker-cli (start/stop via the
+# socket-proxy). No daemon, no compiled build.
+RUN apk add --no-cache bash curl postgresql-client docker-cli
+
+COPY manager.sh /usr/local/bin/manager.sh
+RUN chmod +x /usr/local/bin/manager.sh
+
+# Run unprivileged: the container only needs to reach the socket-proxy over
+# TCP and Postgres — it never touches the host socket directly.
+USER nobody
+
+ENTRYPOINT ["bash", "/usr/local/bin/manager.sh"]

vision-manager/manager.sh

@@ -0,0 +1,171 @@
+#!/usr/bin/env bash
+# vision-manager — start/stop the mangalord-vision (llama.cpp) container by
+# name according to the analysis backlog in Postgres.
+#
+# Why this exists: llama-server has no idle-unload and pins ~4.4 GiB. On the
+# 8 GiB box vision must only run while there is analysis work. This sidecar is
+# the SINGLE owner of the vision lifecycle and the only component with Docker
+# access (scoped through docker-socket-proxy) — the internet-facing backend
+# never gets the socket. See VISION-AUTOSCALE.md.
+#
+# It is deliberately dumb: a poll loop with psql + curl + docker. No web
+# surface, no state beyond an in-memory idle timer. If it dies, vision keeps
+# running (harmless RAM); a MAX_UPTIME backstop bounds that.
+set -euo pipefail
+
+# ---- Config (all overridable via env) --------------------------------------
+: "${DATABASE_URL:?DATABASE_URL must be set (use the read-only vision_manager role)}"
+VISION_CONTAINER="${VISION_CONTAINER:-mangalord-vision}"
+VISION_HEALTH_URL="${VISION_HEALTH_URL:-http://mangalord-vision:8000/health}"
+# DOCKER_HOST points at the scoped docker-socket-proxy by default.
+export DOCKER_HOST="${DOCKER_HOST:-tcp://docker-socket-proxy:2375}"
+
+POLL_INTERVAL="${POLL_INTERVAL:-20}"        # seconds between backlog checks
+STOP_DEBOUNCE="${STOP_DEBOUNCE:-600}"       # idle seconds before stopping vision
+START_HEALTH_TIMEOUT="${START_HEALTH_TIMEOUT:-300}"  # max wait for /health 200
+HEALTH_POLL_INTERVAL="${HEALTH_POLL_INTERVAL:-5}"    # poll cadence while warming
+HEALTH_TIMEOUT="${HEALTH_TIMEOUT:-5}"                # per-probe curl timeout, seconds
+MAX_UPTIME="${MAX_UPTIME:-0}"               # >0: force-stop after N idle-or-not seconds running (backstop); 0 disables
+# When >0, refuse to start vision while a crawl is running (Chromium + vision
+# together OOM the 8 GiB box). Set to 0 on roomier hosts.
+RESPECT_CRAWL_MUTEX="${RESPECT_CRAWL_MUTEX:-1}"
+
+PSQL=(psql "$DATABASE_URL" -At -v ON_ERROR_STOP=1)
+
+log() { echo "[vision-manager] $(date -u +%FT%TZ) $*"; }
+
+# ---- Queries ---------------------------------------------------------------
+# Pending analysis work: pending OR currently-leased analyze_page jobs. Note
+# the queue column is `state` (not `status`) and the kind lives in the JSONB
+# payload — see backend/migrations/0012_crawler.sql.
+pending_analysis() {
+  "${PSQL[@]}" -c \
+    "SELECT count(*) FROM crawler_jobs
+      WHERE payload->>'kind' = 'analyze_page'
+        AND state IN ('pending','running');" 2>/dev/null || echo "ERR"
+}
+
+# A crawl is in-flight if any non-analyze job is currently running.
+crawl_running() {
+  "${PSQL[@]}" -c \
+    "SELECT count(*) FROM crawler_jobs
+      WHERE payload->>'kind' <> 'analyze_page'
+        AND state = 'running';" 2>/dev/null || echo "ERR"
+}
+
+# ---- Docker helpers --------------------------------------------------------
+# `docker inspect .State.Running` → true/false; empty if the container is
+# absent (defined elsewhere and not yet created) — treated as not-running.
+vision_running() {
+  docker inspect -f '{{.State.Running}}' "$VISION_CONTAINER" 2>/dev/null || echo "false"
+}
+
+vision_ready() {
+  curl -fsS -m "$HEALTH_TIMEOUT" -o /dev/null "$VISION_HEALTH_URL" 2>/dev/null
+}
+
+start_vision() {
+  log "starting $VISION_CONTAINER"
+  # Idempotent: starting an already-running container is a docker no-op.
+  if ! docker start "$VISION_CONTAINER" >/dev/null 2>&1; then
+    log "ERROR: docker start failed (is $VISION_CONTAINER defined on this host?)"
+    return 1
+  fi
+  local waited=0
+  while ! vision_ready; do
+    if [ "$waited" -ge "$START_HEALTH_TIMEOUT" ]; then
+      log "WARNING: $VISION_CONTAINER not healthy after ${START_HEALTH_TIMEOUT}s; leaving it running"
+      return 1
+    fi
+    sleep "$HEALTH_POLL_INTERVAL"
+    waited=$((waited + HEALTH_POLL_INTERVAL))
+  done
+  log "$VISION_CONTAINER healthy after ~${waited}s"
+}
+
+stop_vision() {
+  log "stopping $VISION_CONTAINER (idle ${STOP_DEBOUNCE}s)"
+  if docker stop "$VISION_CONTAINER" >/dev/null 2>&1; then
+    return 0
+  fi
+  log "WARNING: docker stop failed; will retry next tick"
+  return 1
+}
+
+# ---- Main loop -------------------------------------------------------------
+log "started: container=$VISION_CONTAINER health=$VISION_HEALTH_URL poll=${POLL_INTERVAL}s debounce=${STOP_DEBOUNCE}s docker_host=$DOCKER_HOST"
+
+idle_for=0       # seconds the queue has been empty while vision is running
+up_for=0         # seconds vision has been running (for MAX_UPTIME backstop)
+crawl_deferred=0 # 1 while a start is held off by the crawl mutex (log on flip only)
+
+while true; do
+  pending="$(pending_analysis)"
+  running="$(vision_running)"
+
+  # Treat anything non-numeric (the "ERR" sentinel, or a stray psql notice on
+  # stdout) as "query failed" — never feed it to `-gt`, which under
+  # `set -e` would otherwise exit-2 and kill the loop.
+  case "$pending" in
+    ''|*[!0-9]*)
+      log "WARNING: backlog query failed (got '${pending}'); will retry next tick"
+      sleep "$POLL_INTERVAL"
+      continue
+      ;;
+  esac
+
+  if [ "$running" = "true" ]; then
+    up_for=$((up_for + POLL_INTERVAL))
+  else
+    up_for=0
+  fi
+
+  if [ "$pending" -gt 0 ]; then
+    idle_for=0
+    if [ "$running" != "true" ]; then
+      if [ "$RESPECT_CRAWL_MUTEX" != "0" ] && [ "$(crawl_running)" != "0" ]; then
+        # Log once per deferral episode, not every poll, so a long crawl
+        # doesn't flood the log.
+        if [ "$crawl_deferred" != "1" ]; then
+          log "deferring start: $pending analysis job(s) pending but a crawl is running (RAM mutex)"
+          crawl_deferred=1
+        fi
+      else
+        crawl_deferred=0
+        start_vision || true
+      fi
+    else
+      crawl_deferred=0
+    fi
+  else
+    crawl_deferred=0
+    # No work. Debounce the stop so bursty enqueues don't thrash the load
+    # cycle, and own the stop on our OWN timer (never wait for a backend
+    # "drained" signal — leak safety).
+    if [ "$running" = "true" ]; then
+      idle_for=$((idle_for + POLL_INTERVAL))
+      if [ "$idle_for" -ge "$STOP_DEBOUNCE" ]; then
+        # Only reset the timers if the stop actually took; otherwise let them
+        # keep counting so we retry on the next tick rather than waiting out
+        # another full debounce window.
+        if stop_vision; then
+          idle_for=0
+          up_for=0
+        fi
+      fi
+    else
+      idle_for=0
+    fi
+  fi
+
+  # Backstop: bound how long vision can stay up regardless of debounce state.
+  if [ "$MAX_UPTIME" -gt 0 ] && [ "$running" = "true" ] && [ "$up_for" -ge "$MAX_UPTIME" ]; then
+    log "MAX_UPTIME ${MAX_UPTIME}s reached; force-stopping $VISION_CONTAINER"
+    if stop_vision; then
+      idle_for=0
+      up_for=0
+    fi
+  fi
+
+  sleep "$POLL_INTERVAL"
+done

vision-manager/readonly-role.sql

@@ -0,0 +1,32 @@
+-- Read-only DB role for vision-manager.
+--
+-- The sidecar only needs to count pending analysis work; give it SELECT on
+-- crawler_jobs and nothing else. It must NOT reuse the backend's credentials.
+--
+-- The Postgres service mounts no init dir and the data volume already exists,
+-- so this is applied ONCE by an operator (it is idempotent):
+--
+--   docker compose exec -T postgres \
+--     psql -U "$POSTGRES_USER" -d "$POSTGRES_DB" -v pw="<a-strong-password>" \
+--     -f - < vision-manager/readonly-role.sql
+--
+-- Then point the manager at it (VISION_MANAGER_DATABASE_URL in .env):
+--   postgres://vision_manager:<a-strong-password>@postgres:5432/<POSTGRES_DB>
+--
+-- The password is passed via psql's -v pw=... ; psql substitutes :'pw' as a
+-- quoted literal and :"DBNAME" (psql's built-in) as the current db identifier.
+-- These substitutions only happen in plain statements, NOT inside a
+-- dollar-quoted DO block — hence the \gexec form below.
+
+-- Create the LOGIN role only if it is absent (idempotent).
+SELECT 'CREATE ROLE vision_manager LOGIN'
+ WHERE NOT EXISTS (SELECT FROM pg_roles WHERE rolname = 'vision_manager')
+\gexec
+
+-- (Re)set the password every run so rotating it is just a re-apply.
+ALTER ROLE vision_manager LOGIN PASSWORD :'pw';
+
+-- CONNECT to the current database, and read-only on the one table we poll.
+GRANT CONNECT ON DATABASE :"DBNAME" TO vision_manager;
+GRANT USAGE ON SCHEMA public TO vision_manager;
+GRANT SELECT ON crawler_jobs TO vision_manager;