feat(admin): crawler observability dashboard + reliability hardening (0.55.0)

Admin-only crawler dashboard backed by an SSE live-status stream,
coordinated browser restart, runtime PHPSESSID refresh, dead-letter
requeue, and a batch of reliability fixes. Closes everything from
the two-pass audit (10 commits' worth) and bumps 0.52.0 -> 0.55.0.

Backend:

- New /admin/crawler/* surface (cookie-auth, RequireAdmin) split
  into status / control / dead_jobs / backlog modules. SSE stream
  composes in-memory status with DB-derived queue counts, memoizes
  the counts for 1s and debounces watch pokes for 250ms (~10x QPS
  reduction per subscriber). One-shot GET /admin/crawler shares the
  same compose path.
- POST /admin/crawler/run gated by manual_pass_lock try_lock_owned
  (409 Conflict on overlapping click); browser restart goes through
  the coordinated_restart gate (drain + relaunch + auto-clear of the
  sticky session_expired flag on Ok).
- Runtime PHPSESSID refresh via SessionController (allow-list
  validation, never logged, audit row carries SHA-256 fingerprint).
  Storage layer is repo::crawler::runtime_session_{load,persist}.
- Dead-letter requeue with four scopes (all/manga/chapter/job);
  scope=all requires confirm:true; DISTINCT ON dedup keeps the
  partial unique index from rejecting requeues for chapters with
  multiple dead rows. SQL is four &'static str constants per scope.
- StatusHandle + ChapterGuard / CoverGuard RAII model survives
  panics; last-writer-wins on cover so concurrent dispatches don't
  clobber each other's slot. Pure functions (should_stop /
  should_mark_clean_exit / should_abort_pass) with named regression
  tests.
- Reliability bundle: per-lease heartbeat, jitter on retries,
  per-job timeout, circuit breaker on consecutive failures, BrowserManager
  coordinated restart gate, request fingerprint changes.
- Streaming page download: Storage::put_stream trait method,
  LocalStorage impl atomic via temp + fsync + UUID-suffixed rename.
  Pages stream through with peak memory ~one HTTP chunk + 64-byte
  sniff prefix instead of one full image per dispatch.
- New partial indexes (migration 0022): mangas_missing_cover_idx
  and crawler_jobs_dead_idx, both ordered by updated_at DESC to
  match the dashboard's LIMIT/OFFSET reads.
- Security hardening: admin_csrf_guard (Origin/Referer allowlist
  on /admin/* mutations, opt-in via ADMIN_ALLOWED_ORIGINS),
  admin_no_store_guard (Cache-Control: no-store on admin
  responses), audit rows carry per-scope target_id.

Frontend:

- /admin/crawler page decomposed into lib/components/crawler/
  (11 components: ProgressBar, SearchBar, CrawlerHero,
  CrawlerControls, ActiveChaptersCard, ActiveJobsTable,
  MissingCoversTable, DeadJobsTable, RestartConfirmModal,
  RequeueAllConfirmModal, SessionModal). Page is 532 LOC of
  orchestration; each component 22-148 LOC.
- EventSource lifecycle wired to visibilitychange / pagehide /
  pageshow (BFCache); after 5 consecutive errors probes the status
  endpoint so a 401 routes through the global on401Hook instead of
  infinite silent reconnects.
- Backlog $effect refetches debounced 500ms with per-loader
  AbortControllers; refresh after a control action only runs when
  the SSE stream is dead.
- Inline requeue button on /admin/mangas patches the affected row's
  sync_state locally (no full chapter-list refetch); proper
  aria-label. Requeue-all gets its own confirm modal; both confirm
  modals autofocus Cancel.
- SvelteKit reverse proxy bypasses its 5-minute AbortController
  for Accept: text/event-stream; pure shouldBypassProxyTimeout
  helper covered by unit tests.

Config / docs:

- New env vars (.env.example): ADMIN_ALLOWED_ORIGINS,
  CRAWLER_JOB_TIMEOUT_SECS, CRAWLER_METADATA_MAX_CONSECUTIVE_FAILURES,
  CRAWLER_BROWSER_RESTART_THRESHOLD.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

backend/src/config.rs

@@ -74,6 +74,20 @@ pub struct Config {
     pub auth: AuthConfig,
     pub upload: UploadConfig,
     pub cors_allowed_origins: Vec<String>,
+    /// Origins (scheme + host[:port]) that may issue browser-driven
+    /// mutating requests to `/api/v1/admin/*`. Defends against
+    /// SameSite=Lax CSRF on the admin cookie: a top-level form POST from
+    /// a malicious page still carries the cookie, but the middleware
+    /// rejects it when the `Origin` (or `Referer` fallback) is absent
+    /// from this list. Sourced from `ADMIN_ALLOWED_ORIGINS`
+    /// (comma-separated). Leave empty to skip the check entirely
+    /// (curl / server-to-server callers send neither header, so they
+    /// pass; same-origin browser requests don't have Origin set on
+    /// same-origin POSTs in some browsers either — operators on a
+    /// same-origin deploy can leave this empty, but doing so removes
+    /// the CSRF defence). Safe methods (GET/HEAD/OPTIONS) never trigger
+    /// the check.
+    pub admin_allowed_origins: Vec<String>,
     pub crawler: CrawlerConfig,
     /// `(username, password)` for the admin user provisioned at startup
     /// when both `ADMIN_USERNAME` and `ADMIN_PASSWORD` are set. `None`
@@ -132,6 +146,19 @@ pub struct CrawlerConfig {
     /// (full sweep up to the source's own bound). Sourced from
     /// `CRAWLER_LIMIT`, mirroring the CLI binary.
     pub manga_limit: usize,
+    /// Hard upper bound on a single chapter-content job dispatch. A job
+    /// exceeding this is acked failed (exponential backoff) instead of
+    /// wedging a worker. Defaults to 600s. `CRAWLER_JOB_TIMEOUT_SECS`.
+    pub job_timeout: Duration,
+    /// Consecutive `fetch_manga` failures that abort a metadata pass
+    /// (circuit-breaker for a source outage). The pass does NOT mark a
+    /// clean exit, so the next tick does a recovery sweep. Defaults to
+    /// 10. `CRAWLER_METADATA_MAX_CONSECUTIVE_FAILURES`.
+    pub metadata_max_consecutive_failures: u32,
+    /// Consecutive transient chapter failures (after TOR recircuit is
+    /// exhausted) that trigger an automatic coordinated browser restart.
+    /// Defaults to 3. `CRAWLER_BROWSER_RESTART_THRESHOLD`.
+    pub browser_restart_threshold: u32,
 }
 
 impl Default for CrawlerConfig {
@@ -159,6 +186,9 @@ impl Default for CrawlerConfig {
             download_allowlist: DownloadAllowlist::new(),
             max_image_bytes: DEFAULT_MAX_IMAGE_BYTES,
             manga_limit: 0,
+            job_timeout: Duration::from_secs(600),
+            metadata_max_consecutive_failures: 10,
+            browser_restart_threshold: 3,
         }
     }
 }
@@ -205,6 +235,15 @@ impl Config {
                         .collect()
                 })
                 .unwrap_or_default(),
+            admin_allowed_origins: std::env::var("ADMIN_ALLOWED_ORIGINS")
+                .ok()
+                .map(|s| {
+                    s.split(',')
+                        .map(|o| o.trim().to_string())
+                        .filter(|o| !o.is_empty())
+                        .collect()
+                })
+                .unwrap_or_default(),
             crawler: CrawlerConfig::from_env()?,
             admin_bootstrap: admin_bootstrap_from_env(),
         })
@@ -283,6 +322,13 @@ impl CrawlerConfig {
             download_allowlist,
             max_image_bytes: env_usize("CRAWLER_MAX_IMAGE_BYTES", DEFAULT_MAX_IMAGE_BYTES),
             manga_limit: env_usize("CRAWLER_LIMIT", 0),
+            job_timeout: Duration::from_secs(env_u64("CRAWLER_JOB_TIMEOUT_SECS", 600).max(1)),
+            metadata_max_consecutive_failures: env_u64(
+                "CRAWLER_METADATA_MAX_CONSECUTIVE_FAILURES",
+                10,
+            ) as u32,
+            browser_restart_threshold: env_u64("CRAWLER_BROWSER_RESTART_THRESHOLD", 3).max(1)
+                as u32,
         })
     }
 }
@@ -384,6 +430,33 @@ mod tests {
         assert_eq!(cfg.manga_limit, 0);
     }
 
+    #[test]
+    fn reliability_knobs_default_when_unset() {
+        let _g = ENV_GUARD.lock().unwrap_or_else(|p| p.into_inner());
+        std::env::remove_var("CRAWLER_JOB_TIMEOUT_SECS");
+        std::env::remove_var("CRAWLER_METADATA_MAX_CONSECUTIVE_FAILURES");
+        std::env::remove_var("CRAWLER_BROWSER_RESTART_THRESHOLD");
+        let cfg = CrawlerConfig::from_env().expect("from_env");
+        assert_eq!(cfg.job_timeout, Duration::from_secs(600));
+        assert_eq!(cfg.metadata_max_consecutive_failures, 10);
+        assert_eq!(cfg.browser_restart_threshold, 3);
+    }
+
+    #[test]
+    fn reliability_knobs_parse_from_env() {
+        let _g = ENV_GUARD.lock().unwrap_or_else(|p| p.into_inner());
+        std::env::set_var("CRAWLER_JOB_TIMEOUT_SECS", "120");
+        std::env::set_var("CRAWLER_METADATA_MAX_CONSECUTIVE_FAILURES", "5");
+        std::env::set_var("CRAWLER_BROWSER_RESTART_THRESHOLD", "7");
+        let cfg = CrawlerConfig::from_env().expect("from_env");
+        std::env::remove_var("CRAWLER_JOB_TIMEOUT_SECS");
+        std::env::remove_var("CRAWLER_METADATA_MAX_CONSECUTIVE_FAILURES");
+        std::env::remove_var("CRAWLER_BROWSER_RESTART_THRESHOLD");
+        assert_eq!(cfg.job_timeout, Duration::from_secs(120));
+        assert_eq!(cfg.metadata_max_consecutive_failures, 5);
+        assert_eq!(cfg.browser_restart_threshold, 7);
+    }
+
     #[test]
     fn private_mode_env_parses_true() {
         let _g = ENV_GUARD.lock().unwrap_or_else(|p| p.into_inner());