feat(admin): crawler observability dashboard + reliability hardening (0.55.0)

Admin-only crawler dashboard backed by an SSE live-status stream,
coordinated browser restart, runtime PHPSESSID refresh, dead-letter
requeue, and a batch of reliability fixes. Closes everything from
the two-pass audit (10 commits' worth) and bumps 0.52.0 -> 0.55.0.

Backend:

- New /admin/crawler/* surface (cookie-auth, RequireAdmin) split
  into status / control / dead_jobs / backlog modules. SSE stream
  composes in-memory status with DB-derived queue counts, memoizes
  the counts for 1s and debounces watch pokes for 250ms (~10x QPS
  reduction per subscriber). One-shot GET /admin/crawler shares the
  same compose path.
- POST /admin/crawler/run gated by manual_pass_lock try_lock_owned
  (409 Conflict on overlapping click); browser restart goes through
  the coordinated_restart gate (drain + relaunch + auto-clear of the
  sticky session_expired flag on Ok).
- Runtime PHPSESSID refresh via SessionController (allow-list
  validation, never logged, audit row carries SHA-256 fingerprint).
  Storage layer is repo::crawler::runtime_session_{load,persist}.
- Dead-letter requeue with four scopes (all/manga/chapter/job);
  scope=all requires confirm:true; DISTINCT ON dedup keeps the
  partial unique index from rejecting requeues for chapters with
  multiple dead rows. SQL is four &'static str constants per scope.
- StatusHandle + ChapterGuard / CoverGuard RAII model survives
  panics; last-writer-wins on cover so concurrent dispatches don't
  clobber each other's slot. Pure functions (should_stop /
  should_mark_clean_exit / should_abort_pass) with named regression
  tests.
- Reliability bundle: per-lease heartbeat, jitter on retries,
  per-job timeout, circuit breaker on consecutive failures, BrowserManager
  coordinated restart gate, request fingerprint changes.
- Streaming page download: Storage::put_stream trait method,
  LocalStorage impl atomic via temp + fsync + UUID-suffixed rename.
  Pages stream through with peak memory ~one HTTP chunk + 64-byte
  sniff prefix instead of one full image per dispatch.
- New partial indexes (migration 0022): mangas_missing_cover_idx
  and crawler_jobs_dead_idx, both ordered by updated_at DESC to
  match the dashboard's LIMIT/OFFSET reads.
- Security hardening: admin_csrf_guard (Origin/Referer allowlist
  on /admin/* mutations, opt-in via ADMIN_ALLOWED_ORIGINS),
  admin_no_store_guard (Cache-Control: no-store on admin
  responses), audit rows carry per-scope target_id.

Frontend:

- /admin/crawler page decomposed into lib/components/crawler/
  (11 components: ProgressBar, SearchBar, CrawlerHero,
  CrawlerControls, ActiveChaptersCard, ActiveJobsTable,
  MissingCoversTable, DeadJobsTable, RestartConfirmModal,
  RequeueAllConfirmModal, SessionModal). Page is 532 LOC of
  orchestration; each component 22-148 LOC.
- EventSource lifecycle wired to visibilitychange / pagehide /
  pageshow (BFCache); after 5 consecutive errors probes the status
  endpoint so a 401 routes through the global on401Hook instead of
  infinite silent reconnects.
- Backlog $effect refetches debounced 500ms with per-loader
  AbortControllers; refresh after a control action only runs when
  the SSE stream is dead.
- Inline requeue button on /admin/mangas patches the affected row's
  sync_state locally (no full chapter-list refetch); proper
  aria-label. Requeue-all gets its own confirm modal; both confirm
  modals autofocus Cancel.
- SvelteKit reverse proxy bypasses its 5-minute AbortController
  for Accept: text/event-stream; pure shouldBypassProxyTimeout
  helper covered by unit tests.

Config / docs:

- New env vars (.env.example): ADMIN_ALLOWED_ORIGINS,
  CRAWLER_JOB_TIMEOUT_SECS, CRAWLER_METADATA_MAX_CONSECUTIVE_FAILURES,
  CRAWLER_BROWSER_RESTART_THRESHOLD.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

frontend/src/hooks.server.test.ts

@@ -7,7 +7,7 @@ import {
     afterEach,
     type MockInstance
 } from 'vitest';
-import { handle } from './hooks.server';
+import { handle, shouldBypassProxyTimeout } from './hooks.server';
 
 // `BACKEND_URL` is read at module load time, so the values used in the
 // asserts below assume the test env didn't set it. `?? 'http://localhost:8080'`
@@ -192,3 +192,37 @@ describe('hooks.server proxy', () => {
         expect(init.signal?.aborted).toBe(false);
     });
 });
+
+// ----------------------------------------------------------------------------
+// T1 — SSE bypass for the wall-clock proxy timeout.
+// ----------------------------------------------------------------------------
+
+describe('shouldBypassProxyTimeout', () => {
+    it('returns true for an SSE Accept header', () => {
+        const h = new Headers({ accept: 'text/event-stream' });
+        expect(shouldBypassProxyTimeout(h)).toBe(true);
+    });
+
+    it('matches case-insensitively', () => {
+        const h = new Headers({ accept: 'TEXT/EVENT-STREAM' });
+        expect(shouldBypassProxyTimeout(h)).toBe(true);
+    });
+
+    it('matches when SSE is one of several Accept values', () => {
+        const h = new Headers({ accept: 'text/event-stream, application/json' });
+        expect(shouldBypassProxyTimeout(h)).toBe(true);
+    });
+
+    it('returns false for plain JSON / HTML requests', () => {
+        expect(
+            shouldBypassProxyTimeout(new Headers({ accept: 'application/json' }))
+        ).toBe(false);
+        expect(shouldBypassProxyTimeout(new Headers({ accept: 'text/html' }))).toBe(
+            false
+        );
+    });
+
+    it('returns false when Accept is absent', () => {
+        expect(shouldBypassProxyTimeout(new Headers())).toBe(false);
+    });
+});

frontend/src/hooks.server.ts

@@ -46,6 +46,11 @@ const HOP_BY_HOP_HEADERS = [
  * tighter upstream proxy may want to lower it. A future improvement
  * is an idle-based timeout (reset per chunk) instead of this
  * wall-clock budget — that's a fair bit more code, deferred.
+ *
+ * SSE streams (`Accept: text/event-stream`) are exempt — see
+ * `shouldBypassProxyTimeout`. A wall-clock abort on a long-lived
+ * stream would tear the connection down every 5 min and force the
+ * browser to reconnect, which flickers the admin dashboard.
  */
 const PROXY_TIMEOUT_MS = (() => {
     const raw = process.env.BACKEND_PROXY_TIMEOUT_MS;
@@ -53,6 +58,18 @@ const PROXY_TIMEOUT_MS = (() => {
     return Number.isFinite(n) && n > 0 ? n : 300_000;
 })();
 
+/**
+ * Whether the proxy should skip its wall-clock timeout for this
+ * request. SSE clients open a single connection and stay subscribed
+ * for the lifetime of the page; aborting on a wall-clock budget would
+ * tear down the live admin dashboard every 5 min. Exported for unit
+ * test coverage.
+ */
+export function shouldBypassProxyTimeout(headers: Headers): boolean {
+    const accept = headers.get('accept') ?? '';
+    return accept.toLowerCase().includes('text/event-stream');
+}
+
 export const handle: Handle = async ({ event, resolve }) => {
     if (event.url.pathname.startsWith('/api/')) {
         const target = `${BACKEND_URL}${event.url.pathname}${event.url.search}`;
@@ -63,9 +80,14 @@ export const handle: Handle = async ({ event, resolve }) => {
         // AbortController times the upstream fetch out so a backend
         // wedged on a slow DB query doesn't keep the browser request
         // hanging forever. The `signal` is also wired into the
-        // RequestInit so the body stream is cancelled cleanly.
+        // RequestInit so the body stream is cancelled cleanly. For
+        // SSE streams the timer is suppressed so a long-lived stream
+        // isn't torn down on the 5-minute mark — see T1 in the audit.
+        const bypassTimeout = shouldBypassProxyTimeout(event.request.headers);
         const ctrl = new AbortController();
-        const timeoutHandle = setTimeout(() => ctrl.abort(), PROXY_TIMEOUT_MS);
+        const timeoutHandle = bypassTimeout
+            ? null
+            : setTimeout(() => ctrl.abort(), PROXY_TIMEOUT_MS);
 
         const init: RequestInit & { duplex?: 'half' } = {
             method: event.request.method,
@@ -91,7 +113,7 @@ export const handle: Handle = async ({ event, resolve }) => {
             // the real cause. Emit the standard envelope with a
             // dedicated code instead.
             console.error('Proxy to backend failed:', e);
-            clearTimeout(timeoutHandle);
+            if (timeoutHandle) clearTimeout(timeoutHandle);
             return new Response(
                 JSON.stringify({
                     error: {
@@ -106,7 +128,7 @@ export const handle: Handle = async ({ event, resolve }) => {
             );
         }
 
-        clearTimeout(timeoutHandle);
+        if (timeoutHandle) clearTimeout(timeoutHandle);
         return new Response(upstream.body, {
             status: upstream.status,
             statusText: upstream.statusText,

frontend/src/lib/api/admin.test.ts

@@ -16,7 +16,17 @@ import {
     listAdminChapters,
     getSystemStats,
     resyncManga,
-    resyncChapter
+    resyncChapter,
+    getCrawlerStatus,
+    crawlerStatusStreamUrl,
+    runCrawlerPass,
+    restartCrawlerBrowser,
+    updateCrawlerSession,
+    clearCrawlerSessionExpired,
+    listDeadJobs,
+    requeueDeadJobs,
+    listActiveJobs,
+    listMissingCovers
 } from './admin';
 
 function ok(body: unknown, status = 200): Response {
@@ -329,3 +339,159 @@ describe('admin api client', () => {
         expect(got.pages).toBeNull();
     });
 });
+
+describe('admin crawler api client', () => {
+    let fetchSpy: MockInstance<typeof globalThis.fetch>;
+    beforeEach(() => {
+        fetchSpy = vi.spyOn(globalThis, 'fetch');
+    });
+    afterEach(() => {
+        vi.restoreAllMocks();
+    });
+
+    const statusFixture = {
+        daemon: 'running',
+        phase: { state: 'fetching_metadata', index: 3, total: 10, title: 'One Piece' },
+        worker_count: 2,
+        active_chapters: [
+            {
+                manga_id: 'm-1',
+                manga_title: 'Bleach',
+                chapter_id: 'c-1',
+                chapter_number: 12,
+                pages_done: 4,
+                pages_total: 20
+            }
+        ],
+        current_cover: { manga_id: 'm-2', manga_title: 'Naruto' },
+        covers_queued: 7,
+        last_pass: { at: null, discovered: 0, upserted: 0, covers_fetched: 0, mangas_failed: 0 },
+        session: { expired: false, configured: true },
+        browser: 'healthy',
+        queue: { pending: 2, running: 1, dead: 4 }
+    };
+
+    it('crawlerStatusStreamUrl points at the SSE endpoint under the API base', () => {
+        expect(crawlerStatusStreamUrl()).toMatch(/\/v1\/admin\/crawler\/stream$/);
+    });
+
+    it('getCrawlerStatus GETs /v1/admin/crawler with live chapter/cover fields', async () => {
+        fetchSpy.mockResolvedValueOnce(ok(statusFixture));
+        const s = await getCrawlerStatus();
+        expect(s.queue.dead).toBe(4);
+        expect(s.phase?.state).toBe('fetching_metadata');
+        expect(s.active_chapters[0].pages_done).toBe(4);
+        expect(s.active_chapters[0].pages_total).toBe(20);
+        expect(s.current_cover?.manga_title).toBe('Naruto');
+        expect(s.covers_queued).toBe(7);
+        const url = fetchSpy.mock.calls[0][0] as string;
+        expect(url).toMatch(/\/v1\/admin\/crawler$/);
+    });
+
+    it('listActiveJobs GETs /v1/admin/crawler/active-jobs with search', async () => {
+        fetchSpy.mockResolvedValueOnce(
+            ok({ items: [], page: { limit: 20, offset: 0, total: 0 } })
+        );
+        await listActiveJobs({ search: 'bleach' });
+        const url = fetchSpy.mock.calls[0][0] as string;
+        expect(url).toMatch(/\/v1\/admin\/crawler\/active-jobs\?/);
+        expect(url).toContain('search=bleach');
+    });
+
+    it('listMissingCovers GETs /v1/admin/crawler/covers', async () => {
+        fetchSpy.mockResolvedValueOnce(
+            ok({ items: [{ manga_id: 'm-1', manga_title: 'X' }], page: { limit: 20, offset: 0, total: 1 } })
+        );
+        const r = await listMissingCovers();
+        expect(r.items[0].manga_title).toBe('X');
+        expect(fetchSpy.mock.calls[0][0]).toMatch(/\/v1\/admin\/crawler\/covers$/);
+    });
+
+    it('runCrawlerPass POSTs /v1/admin/crawler/run', async () => {
+        fetchSpy.mockResolvedValueOnce(ok({ started: true }));
+        const r = await runCrawlerPass();
+        expect(r.started).toBe(true);
+        const init = fetchSpy.mock.calls[0][1] as RequestInit;
+        expect(init.method).toBe('POST');
+        expect(fetchSpy.mock.calls[0][0]).toMatch(/\/v1\/admin\/crawler\/run$/);
+    });
+
+    it('restartCrawlerBrowser POSTs the restart endpoint', async () => {
+        fetchSpy.mockResolvedValueOnce(ok({ ok: true, error: null }));
+        const r = await restartCrawlerBrowser();
+        expect(r.ok).toBe(true);
+        expect(fetchSpy.mock.calls[0][0]).toMatch(/\/v1\/admin\/crawler\/browser\/restart$/);
+    });
+
+    it('updateCrawlerSession POSTs the phpsessid body', async () => {
+        fetchSpy.mockResolvedValueOnce(ok({ valid: true, error: null }));
+        const r = await updateCrawlerSession('abc123');
+        expect(r.valid).toBe(true);
+        const init = fetchSpy.mock.calls[0][1] as RequestInit;
+        expect(init.method).toBe('POST');
+        expect(JSON.parse(init.body as string)).toEqual({ phpsessid: 'abc123' });
+    });
+
+    it('clearCrawlerSessionExpired POSTs clear-expired', async () => {
+        fetchSpy.mockResolvedValueOnce(ok({ cleared: true }));
+        const r = await clearCrawlerSessionExpired();
+        expect(r.cleared).toBe(true);
+        expect(fetchSpy.mock.calls[0][0]).toMatch(/\/v1\/admin\/crawler\/session\/clear-expired$/);
+    });
+
+    it('listDeadJobs forwards search + pagination', async () => {
+        fetchSpy.mockResolvedValueOnce(
+            ok({ items: [], page: { limit: 20, offset: 20, total: 0 } })
+        );
+        await listDeadJobs({ search: 'naruto', limit: 20, offset: 20 });
+        const url = fetchSpy.mock.calls[0][0] as string;
+        expect(url).toContain('search=naruto');
+        expect(url).toContain('offset=20');
+    });
+
+    it('requeueDeadJobs POSTs the scope body', async () => {
+        fetchSpy.mockResolvedValueOnce(ok({ requeued: 3 }));
+        const r = await requeueDeadJobs({ scope: 'manga', manga_id: 'm-9' });
+        expect(r.requeued).toBe(3);
+        const init = fetchSpy.mock.calls[0][1] as RequestInit;
+        expect(JSON.parse(init.body as string)).toEqual({ scope: 'manga', manga_id: 'm-9' });
+    });
+
+    it('requeueDeadJobs serialises chapter scope verbatim', async () => {
+        // F7 — the per-chapter requeue button in /admin/mangas sends this
+        // exact shape; pin it so a future rename of `chapter_id` doesn't
+        // silently break the inline button.
+        fetchSpy.mockResolvedValueOnce(ok({ requeued: 1 }));
+        const r = await requeueDeadJobs({ scope: 'chapter', chapter_id: 'c-7' });
+        expect(r.requeued).toBe(1);
+        const init = fetchSpy.mock.calls[0][1] as RequestInit;
+        expect(JSON.parse(init.body as string)).toEqual({
+            scope: 'chapter',
+            chapter_id: 'c-7'
+        });
+    });
+
+    it('requeueDeadJobs serialises job + all (with confirm) scopes', async () => {
+        // Round out the variant coverage: { scope: 'job', job_id } and
+        // { scope: 'all', confirm: true }. The audit (S1) requires the
+        // confirm flag on the wire for scope=all; this pins it so a
+        // future refactor can't drop it.
+        fetchSpy.mockResolvedValueOnce(ok({ requeued: 1 }));
+        await requeueDeadJobs({ scope: 'job', job_id: 'j-1' });
+        expect(JSON.parse(fetchSpy.mock.calls[0][1]!.body as string)).toEqual({
+            scope: 'job',
+            job_id: 'j-1'
+        });
+        fetchSpy.mockResolvedValueOnce(ok({ requeued: 99 }));
+        await requeueDeadJobs({ scope: 'all', confirm: true });
+        expect(JSON.parse(fetchSpy.mock.calls[1][1]!.body as string)).toEqual({
+            scope: 'all',
+            confirm: true
+        });
+    });
+
+    it('surfaces a 503 as ApiError', async () => {
+        fetchSpy.mockResolvedValueOnce(envelope(503, 'service_unavailable', 'disabled'));
+        await expect(runCrawlerPass()).rejects.toMatchObject({ status: 503 });
+    });
+});

frontend/src/lib/api/admin.ts

@@ -3,7 +3,7 @@
 // won't reach these routes). 403s thrown here propagate up to the
 // /admin layout, which renders the framework error page.
 
-import { request, type Page } from './client';
+import { request, apiUrl, type Page } from './client';
 import type { User } from './auth';
 import type { MangaDetail } from './mangas';
 import type { Chapter } from './chapters';
@@ -214,3 +214,192 @@ export async function resyncChapter(id: string): Promise<ChapterResyncResponse>
         { method: 'POST' }
     );
 }
+
+// ---- crawler observability + control ---------------------------------------
+
+/** Current daemon activity. Discriminated on `state`. */
+export type CrawlerPhase =
+    | { state: 'idle'; next_fire: string | null }
+    | { state: 'walking_list' }
+    | { state: 'fetching_metadata'; index: number; total: number | null; title: string }
+    | { state: 'cover_backfill'; index: number; total: number };
+
+/** A chapter being crawled right now, with a live page count. */
+export type ActiveChapter = {
+    manga_id: string;
+    manga_title: string;
+    chapter_id: string;
+    chapter_number: number;
+    pages_done: number;
+    pages_total: number | null;
+};
+
+export type CrawlerLastPass = {
+    at: string | null;
+    discovered: number;
+    upserted: number;
+    covers_fetched: number;
+    mangas_failed: number;
+};
+
+export type CrawlerStatus = {
+    daemon: 'running' | 'disabled';
+    phase: CrawlerPhase | null;
+    worker_count: number;
+    active_chapters: ActiveChapter[];
+    current_cover: { manga_id: string; manga_title: string } | null;
+    covers_queued: number;
+    last_pass: CrawlerLastPass;
+    session: { expired: boolean; configured: boolean };
+    browser: 'healthy' | 'draining' | 'restarting' | 'down';
+    queue: { pending: number; running: number; dead: number };
+};
+
+export async function getCrawlerStatus(): Promise<CrawlerStatus> {
+    return request<CrawlerStatus>('/v1/admin/crawler');
+}
+
+/** URL of the Server-Sent Events live-status stream. Open with
+ *  `new EventSource(...)` while the crawler page is mounted and close it on
+ *  navigate-away so the subscription is scoped to the active page. Each
+ *  message is a named `status` event whose `data` is a {@link CrawlerStatus}. */
+export function crawlerStatusStreamUrl(): string {
+    return apiUrl('/v1/admin/crawler/stream');
+}
+
+/** POST /v1/admin/crawler/run — trigger an out-of-cycle metadata pass. */
+export async function runCrawlerPass(): Promise<{ started: boolean }> {
+    return request('/v1/admin/crawler/run', { method: 'POST' });
+}
+
+/** POST /v1/admin/crawler/browser/restart — coordinated Chromium restart. */
+export async function restartCrawlerBrowser(): Promise<{ ok: boolean; error: string | null }> {
+    return request('/v1/admin/crawler/browser/restart', { method: 'POST' });
+}
+
+/** POST /v1/admin/crawler/session — refresh PHPSESSID and re-probe. */
+export async function updateCrawlerSession(
+    phpsessid: string
+): Promise<{ valid: boolean; error: string | null }> {
+    return request('/v1/admin/crawler/session', {
+        method: 'POST',
+        headers: { 'content-type': 'application/json' },
+        body: JSON.stringify({ phpsessid })
+    });
+}
+
+/** POST /v1/admin/crawler/session/clear-expired — resume idled workers. */
+export async function clearCrawlerSessionExpired(): Promise<{ cleared: boolean }> {
+    return request('/v1/admin/crawler/session/clear-expired', { method: 'POST' });
+}
+
+export type DeadJob = {
+    id: string;
+    kind: string;
+    chapter_id: string | null;
+    manga_id: string | null;
+    manga_title: string | null;
+    chapter_number: number | null;
+    attempts: number;
+    max_attempts: number;
+    last_error: string | null;
+    updated_at: string;
+};
+
+export type DeadJobsPage = { items: DeadJob[]; page: Page };
+
+export async function listDeadJobs(
+    opts?: {
+        search?: string;
+        limit?: number;
+        offset?: number;
+    },
+    init?: RequestInit
+): Promise<DeadJobsPage> {
+    const params = new URLSearchParams();
+    if (opts?.search) params.set('search', opts.search);
+    if (opts?.limit != null) params.set('limit', String(opts.limit));
+    if (opts?.offset != null) params.set('offset', String(opts.offset));
+    const qs = params.toString();
+    return request<DeadJobsPage>(
+        `/v1/admin/crawler/dead-jobs${qs ? `?${qs}` : ''}`,
+        init
+    );
+}
+
+/** Requeue scope: all dead jobs, one manga's, one chapter's, or a single job.
+ *  `scope: 'all'` requires an explicit `confirm: true` so a careless
+ *  click (or CSRF bait) can't flip the entire dead pile. */
+export type RequeueScope =
+    | { scope: 'all'; confirm: true }
+    | { scope: 'manga'; manga_id: string }
+    | { scope: 'chapter'; chapter_id: string }
+    | { scope: 'job'; job_id: string };
+
+export async function requeueDeadJobs(scope: RequeueScope): Promise<{ requeued: number }> {
+    return request('/v1/admin/crawler/dead-jobs/requeue', {
+        method: 'POST',
+        headers: { 'content-type': 'application/json' },
+        body: JSON.stringify(scope)
+    });
+}
+
+/** A queued/running chapter-content job (which chapters are queued). */
+export type ActiveJob = {
+    id: string;
+    chapter_id: string | null;
+    manga_id: string | null;
+    manga_title: string | null;
+    chapter_number: number | null;
+    state: 'pending' | 'running';
+    attempts: number;
+    max_attempts: number;
+    updated_at: string;
+};
+
+export type ActiveJobsPage = { items: ActiveJob[]; page: Page };
+
+/** GET /v1/admin/crawler/active-jobs — which chapters of which mangas are
+ *  queued or running now. */
+export async function listActiveJobs(
+    opts?: {
+        search?: string;
+        limit?: number;
+        offset?: number;
+    },
+    init?: RequestInit
+): Promise<ActiveJobsPage> {
+    const params = new URLSearchParams();
+    if (opts?.search) params.set('search', opts.search);
+    if (opts?.limit != null) params.set('limit', String(opts.limit));
+    if (opts?.offset != null) params.set('offset', String(opts.offset));
+    const qs = params.toString();
+    return request<ActiveJobsPage>(
+        `/v1/admin/crawler/active-jobs${qs ? `?${qs}` : ''}`,
+        init
+    );
+}
+
+/** A manga queued for a cover fetch (no cover yet + a live source). */
+export type MissingCover = { manga_id: string; manga_title: string };
+export type MissingCoversPage = { items: MissingCover[]; page: Page };
+
+/** GET /v1/admin/crawler/covers — which manga covers are queued. */
+export async function listMissingCovers(
+    opts?: {
+        search?: string;
+        limit?: number;
+        offset?: number;
+    },
+    init?: RequestInit
+): Promise<MissingCoversPage> {
+    const params = new URLSearchParams();
+    if (opts?.search) params.set('search', opts.search);
+    if (opts?.limit != null) params.set('limit', String(opts.limit));
+    if (opts?.offset != null) params.set('offset', String(opts.offset));
+    const qs = params.toString();
+    return request<MissingCoversPage>(
+        `/v1/admin/crawler/covers${qs ? `?${qs}` : ''}`,
+        init
+    );
+}

frontend/src/lib/api/client.ts

@@ -12,6 +12,15 @@ export function fileUrl(key: string): string {
     return `${BASE}/v1/files/${key}`;
 }
 
+/**
+ * Builds an API URL for non-`fetch` consumers (e.g. `EventSource` for SSE),
+ * applying the same `VITE_API_BASE` prefix as `request()`. `path` is the
+ * route after the base, e.g. `/v1/admin/crawler/stream`.
+ */
+export function apiUrl(path: string): string {
+    return `${BASE}${path}`;
+}
+
 export class ApiError extends Error {
     constructor(
         public readonly status: number,

frontend/src/lib/components/crawler/ActiveChaptersCard.svelte

@@ -0,0 +1,112 @@
+<script lang="ts">
+    import type { CrawlerStatus } from '$lib/api/admin';
+    import ProgressBar from './ProgressBar.svelte';
+
+    let { status }: { status: CrawlerStatus } = $props();
+
+    function chapterPercent(c: { pages_done: number; pages_total: number | null }): number | null {
+        return c.pages_total && c.pages_total > 0
+            ? Math.min(100, (c.pages_done / c.pages_total) * 100)
+            : null;
+    }
+</script>
+
+<section class="grid2">
+    <article>
+        <h2>Queue</h2>
+        <dl>
+            <dt>Pending</dt>
+            <dd>{status.queue.pending}</dd>
+            <dt>Running</dt>
+            <dd>{status.queue.running}</dd>
+            <dt>Dead</dt>
+            <dd>{status.queue.dead}</dd>
+            <dt>Covers queued</dt>
+            <dd>{status.covers_queued}</dd>
+        </dl>
+    </article>
+    <article>
+        <h2>Active chapters ({status.active_chapters.length}/{status.worker_count})</h2>
+        {#if status.active_chapters.length === 0}
+            <p class="muted">idle — no chapters downloading</p>
+        {:else}
+            <table class="active">
+                <tbody>
+                    {#each status.active_chapters as c (c.chapter_id)}
+                        <tr>
+                            <td>{c.manga_title} · ch.{c.chapter_number}</td>
+                            <td class="pagecount" data-testid="active-pages">
+                                {c.pages_done}/{c.pages_total ?? '?'}
+                            </td>
+                            <td class="pagebar">
+                                {#if chapterPercent(c) !== null}
+                                    <ProgressBar percent={chapterPercent(c) ?? 0} />
+                                {/if}
+                            </td>
+                        </tr>
+                    {/each}
+                </tbody>
+            </table>
+        {/if}
+    </article>
+</section>
+
+<style>
+    h2 {
+        margin: 0 0 var(--space-3) 0;
+        font-size: var(--font-sm);
+        color: var(--text-muted);
+        text-transform: uppercase;
+        letter-spacing: 0.04em;
+    }
+    .grid2 {
+        display: grid;
+        grid-template-columns: repeat(auto-fit, minmax(16rem, 1fr));
+        gap: var(--space-3);
+        margin-bottom: var(--space-4);
+    }
+    article {
+        padding: var(--space-3);
+        border: 1px solid var(--border);
+        border-radius: var(--radius-md);
+        background: var(--surface);
+    }
+    dl {
+        display: grid;
+        grid-template-columns: max-content 1fr;
+        gap: var(--space-1) var(--space-3);
+        margin: 0;
+        font-size: var(--font-sm);
+    }
+    dt {
+        color: var(--text-muted);
+    }
+    dd {
+        margin: 0;
+        font-family: var(--font-mono, monospace);
+    }
+    table {
+        width: 100%;
+        border-collapse: collapse;
+    }
+    td {
+        padding: var(--space-2);
+        text-align: left;
+        border-bottom: 1px solid var(--border);
+        font-size: var(--font-sm);
+    }
+    .muted {
+        color: var(--text-muted);
+    }
+    .pagecount {
+        font-family: var(--font-mono, monospace);
+        font-size: var(--font-xs);
+        white-space: nowrap;
+    }
+    .pagebar {
+        width: 8rem;
+    }
+    table.active td {
+        vertical-align: middle;
+    }
+</style>

frontend/src/lib/components/crawler/ActiveJobsTable.svelte

@@ -0,0 +1,101 @@
+<script lang="ts">
+    import Pager from '$lib/components/Pager.svelte';
+    import type { ActiveJob } from '$lib/api/admin';
+    import SearchBar from './SearchBar.svelte';
+
+    let {
+        jobs,
+        total,
+        page,
+        totalPages,
+        search = $bindable(''),
+        onSearch,
+        onPageChange
+    }: {
+        jobs: ActiveJob[];
+        total: number;
+        page: number;
+        totalPages: number;
+        search?: string;
+        onSearch: () => void;
+        onPageChange: (p: number) => void;
+    } = $props();
+</script>
+
+<section class="backlog">
+    <div class="deadhead">
+        <h2>Queued chapters ({total})</h2>
+        <div class="deadtools">
+            <SearchBar bind:value={search} {onSearch} />
+        </div>
+    </div>
+    {#if jobs.length === 0}
+        <p class="muted">No chapters queued.</p>
+    {:else}
+        <table class="dead">
+            <thead>
+                <tr>
+                    <th>Manga / Chapter</th>
+                    <th>State</th>
+                    <th>Att.</th>
+                </tr>
+            </thead>
+            <tbody>
+                {#each jobs as j (j.id)}
+                    <tr>
+                        <td>
+                            {j.manga_title ?? '(unknown)'}
+                            {#if j.chapter_number != null}· ch.{j.chapter_number}{/if}
+                        </td>
+                        <td>
+                            <span
+                                class={`badge ${j.state === 'running' ? 'badge-downloading' : 'badge-not_downloaded'}`}
+                                >{j.state}</span
+                            >
+                        </td>
+                        <td>{j.attempts}/{j.max_attempts}</td>
+                    </tr>
+                {/each}
+            </tbody>
+        </table>
+        <Pager {page} {totalPages} onChange={onPageChange} />
+    {/if}
+</section>
+
+<style>
+    h2 {
+        margin: 0 0 var(--space-3) 0;
+        font-size: var(--font-sm);
+        color: var(--text-muted);
+        text-transform: uppercase;
+        letter-spacing: 0.04em;
+    }
+    table {
+        width: 100%;
+        border-collapse: collapse;
+    }
+    th,
+    td {
+        padding: var(--space-2);
+        text-align: left;
+        border-bottom: 1px solid var(--border);
+        font-size: var(--font-sm);
+    }
+    .deadhead {
+        display: flex;
+        justify-content: space-between;
+        align-items: center;
+        gap: var(--space-3);
+        flex-wrap: wrap;
+    }
+    .deadtools {
+        display: flex;
+        gap: var(--space-2);
+    }
+    .muted {
+        color: var(--text-muted);
+    }
+    .backlog {
+        margin-top: var(--space-4);
+    }
+</style>

frontend/src/lib/components/crawler/CrawlerControls.svelte

@@ -0,0 +1,43 @@
+<script lang="ts">
+    import type { CrawlerStatus } from '$lib/api/admin';
+
+    let {
+        status,
+        busy,
+        onRunPass,
+        onOpenRestart,
+        onOpenSession,
+        onClearExpired
+    }: {
+        status: CrawlerStatus;
+        busy: boolean;
+        onRunPass: () => void;
+        onOpenRestart: () => void;
+        onOpenSession: () => void;
+        onClearExpired: () => void;
+    } = $props();
+</script>
+
+<section class="controls">
+    <button onclick={onRunPass} disabled={busy || status.daemon !== 'running'}
+        >Run metadata pass now</button
+    >
+    <button onclick={onOpenRestart} disabled={busy || status.daemon !== 'running'}
+        >Restart browser</button
+    >
+    <button onclick={onOpenSession} disabled={busy || status.daemon !== 'running'}
+        >Manage session…</button
+    >
+    {#if status.session.expired}
+        <button onclick={onClearExpired} disabled={busy}>Clear expired flag</button>
+    {/if}
+</section>
+
+<style>
+    .controls {
+        display: flex;
+        gap: var(--space-2);
+        flex-wrap: wrap;
+        margin-bottom: var(--space-4);
+    }
+</style>

frontend/src/lib/components/crawler/CrawlerHero.svelte

@@ -0,0 +1,148 @@
+<script lang="ts">
+    import type { CrawlerStatus, CrawlerPhase } from '$lib/api/admin';
+    import ProgressBar from './ProgressBar.svelte';
+
+    let { status }: { status: CrawlerStatus } = $props();
+
+    function phaseLabel(p: CrawlerPhase | null): string {
+        if (!p) return 'Daemon disabled';
+        switch (p.state) {
+            case 'idle':
+                return p.next_fire
+                    ? `Idle — next pass ${new Date(p.next_fire).toLocaleString()}`
+                    : 'Idle';
+            case 'walking_list':
+                return 'Walking source list';
+            case 'fetching_metadata':
+                return `Fetching metadata · ${p.index}/${p.total ?? '?'} · ${p.title}`;
+            case 'cover_backfill':
+                return `Backfilling covers · ${p.index + 1}/${p.total}`;
+            default: {
+                // Exhaustive default: if a new phase state ships
+                // without a label, TypeScript flags the missing case
+                // at compile time. Keeps the UI from showing
+                // `undefined` if the server gets ahead of the client.
+                const _exhaustive: never = p;
+                void _exhaustive;
+                return 'Unknown phase';
+            }
+        }
+    }
+
+    function phasePercent(p: CrawlerPhase | null): number | null {
+        if (p && p.state === 'fetching_metadata' && p.total && p.total > 0) {
+            return Math.min(100, (p.index / p.total) * 100);
+        }
+        return null;
+    }
+
+    function sessionPill(s: CrawlerStatus): { cls: string; text: string } {
+        if (s.daemon === 'disabled') return { cls: 'badge-not_downloaded', text: 'n/a' };
+        if (s.session.expired) return { cls: 'badge-in_progress', text: 'Expired' };
+        if (!s.session.configured) return { cls: 'badge-not_downloaded', text: 'Not set' };
+        return { cls: 'badge-synced', text: 'OK' };
+    }
+
+    function browserPill(s: CrawlerStatus): { cls: string; text: string } {
+        switch (s.browser) {
+            case 'healthy':
+                return { cls: 'badge-synced', text: 'Up' };
+            case 'draining':
+            case 'restarting':
+                return { cls: 'badge-in_progress', text: s.browser };
+            default:
+                return { cls: 'badge-not_downloaded', text: 'Down' };
+        }
+    }
+</script>
+
+<section class="hero" data-testid="crawler-hero">
+    <div class="pills">
+        <span class="pill"
+            >Daemon
+            <span
+                class={`badge ${status.daemon === 'running' ? 'badge-synced' : 'badge-not_downloaded'}`}
+                >{status.daemon}</span
+            ></span
+        >
+        <span class="pill"
+            >Session
+            <span class={`badge ${sessionPill(status).cls}`}>{sessionPill(status).text}</span></span
+        >
+        <span class="pill"
+            >Browser
+            <span class={`badge ${browserPill(status).cls}`}>{browserPill(status).text}</span></span
+        >
+    </div>
+
+    <p class="phase" data-testid="crawler-phase">{phaseLabel(status.phase)}</p>
+    {#if phasePercent(status.phase) !== null}
+        <ProgressBar percent={phasePercent(status.phase) ?? 0} />
+    {/if}
+
+    {#if status.session.expired}
+        <p class="warn">
+            ⚠ Chapter downloads paused — session expired. Metadata + list crawl continue.
+        </p>
+    {/if}
+
+    {#if status.current_cover}
+        <p class="cover" data-testid="current-cover">
+            🖼 Fetching cover: <strong>{status.current_cover.manga_title}</strong>
+        </p>
+    {/if}
+
+    <p class="lastpass">
+        Last pass:
+        {#if status.last_pass.at}
+            {new Date(status.last_pass.at).toLocaleString()} ·
+            {status.last_pass.discovered} seen · {status.last_pass.upserted} upserted ·
+            {status.last_pass.mangas_failed} failed
+        {:else}
+            — none yet this session
+        {/if}
+    </p>
+</section>
+
+<style>
+    .hero {
+        padding: var(--space-4);
+        border: 1px solid var(--border);
+        border-radius: var(--radius-md);
+        background: var(--surface);
+        margin-bottom: var(--space-4);
+    }
+    .pills {
+        display: flex;
+        gap: var(--space-4);
+        flex-wrap: wrap;
+        margin-bottom: var(--space-3);
+    }
+    .pill {
+        font-size: var(--font-sm);
+        color: var(--text-muted);
+        display: inline-flex;
+        align-items: center;
+        gap: var(--space-2);
+    }
+    .phase {
+        font-size: var(--font-lg);
+        font-weight: var(--weight-semibold);
+        margin: var(--space-2) 0;
+    }
+    .lastpass {
+        color: var(--text-muted);
+        font-size: var(--font-sm);
+    }
+    .warn {
+        color: #92400e;
+        background: #fef3c7;
+        border: 1px solid #fcd34d;
+        padding: var(--space-2) var(--space-3);
+        border-radius: var(--radius-md);
+        font-size: var(--font-sm);
+    }
+    .cover {
+        font-size: var(--font-sm);
+    }
+</style>

frontend/src/lib/components/crawler/DeadJobsTable.svelte

@@ -0,0 +1,132 @@
+<script lang="ts">
+    import Pager from '$lib/components/Pager.svelte';
+    import type { DeadJob, RequeueScope } from '$lib/api/admin';
+    import SearchBar from './SearchBar.svelte';
+
+    let {
+        jobs,
+        total,
+        page,
+        totalPages,
+        search = $bindable(''),
+        busy,
+        onSearch,
+        onPageChange,
+        onRequeue,
+        onRequeueAll
+    }: {
+        jobs: DeadJob[];
+        total: number;
+        page: number;
+        totalPages: number;
+        search?: string;
+        busy: boolean;
+        onSearch: () => void;
+        onPageChange: (p: number) => void;
+        onRequeue: (scope: RequeueScope) => void;
+        onRequeueAll: () => void;
+    } = $props();
+</script>
+
+<section class="deadjobs">
+    <div class="deadhead">
+        <h2>Dead jobs ({total})</h2>
+        <div class="deadtools">
+            <SearchBar bind:value={search} {onSearch} />
+            <button onclick={onRequeueAll} disabled={busy || total === 0}
+                >Requeue all ({total})</button
+            >
+        </div>
+    </div>
+
+    {#if jobs.length === 0}
+        <p class="muted">No dead jobs 🎉</p>
+    {:else}
+        <table class="dead">
+            <thead>
+                <tr>
+                    <th>Manga / Chapter</th>
+                    <th>Att.</th>
+                    <th>Failed</th>
+                    <th>Last error</th>
+                    <th class="actions">Action</th>
+                </tr>
+            </thead>
+            <tbody>
+                {#each jobs as j (j.id)}
+                    <tr>
+                        <td>
+                            {j.manga_title ?? '(unknown)'}
+                            {#if j.chapter_number != null}· ch.{j.chapter_number}{/if}
+                        </td>
+                        <td>{j.attempts}/{j.max_attempts}</td>
+                        <td>{new Date(j.updated_at).toLocaleDateString()}</td>
+                        <td class="err" title={j.last_error ?? ''}>{j.last_error ?? '—'}</td>
+                        <td class="actions">
+                            <button
+                                onclick={() => onRequeue({ scope: 'job', job_id: j.id })}
+                                disabled={busy}>Requeue</button
+                            >
+                            {#if j.manga_id}
+                                <button
+                                    class="secondary"
+                                    onclick={() => onRequeue({ scope: 'manga', manga_id: j.manga_id! })}
+                                    disabled={busy}>Manga</button
+                                >
+                            {/if}
+                        </td>
+                    </tr>
+                {/each}
+            </tbody>
+        </table>
+        <Pager {page} {totalPages} onChange={onPageChange} />
+    {/if}
+</section>
+
+<style>
+    h2 {
+        margin: 0 0 var(--space-3) 0;
+        font-size: var(--font-sm);
+        color: var(--text-muted);
+        text-transform: uppercase;
+        letter-spacing: 0.04em;
+    }
+    table {
+        width: 100%;
+        border-collapse: collapse;
+    }
+    th,
+    td {
+        padding: var(--space-2);
+        text-align: left;
+        border-bottom: 1px solid var(--border);
+        font-size: var(--font-sm);
+    }
+    .actions {
+        text-align: right;
+    }
+    .err {
+        max-width: 22rem;
+        overflow: hidden;
+        text-overflow: ellipsis;
+        white-space: nowrap;
+        color: var(--text-muted);
+    }
+    .deadhead {
+        display: flex;
+        justify-content: space-between;
+        align-items: center;
+        gap: var(--space-3);
+        flex-wrap: wrap;
+    }
+    .deadtools {
+        display: flex;
+        gap: var(--space-2);
+    }
+    button.secondary {
+        background: var(--surface-elevated);
+    }
+    .muted {
+        color: var(--text-muted);
+    }
+</style>

frontend/src/lib/components/crawler/MissingCoversTable.svelte

@@ -0,0 +1,85 @@
+<script lang="ts">
+    import Pager from '$lib/components/Pager.svelte';
+    import type { MissingCover } from '$lib/api/admin';
+    import SearchBar from './SearchBar.svelte';
+
+    let {
+        covers,
+        total,
+        page,
+        totalPages,
+        search = $bindable(''),
+        onSearch,
+        onPageChange
+    }: {
+        covers: MissingCover[];
+        total: number;
+        page: number;
+        totalPages: number;
+        search?: string;
+        onSearch: () => void;
+        onPageChange: (p: number) => void;
+    } = $props();
+</script>
+
+<section class="backlog">
+    <div class="deadhead">
+        <h2>Queued covers ({total})</h2>
+        <div class="deadtools">
+            <SearchBar bind:value={search} {onSearch} />
+        </div>
+    </div>
+    {#if covers.length === 0}
+        <p class="muted">No covers queued 🎉</p>
+    {:else}
+        <table class="dead">
+            <thead>
+                <tr><th>Manga</th></tr>
+            </thead>
+            <tbody>
+                {#each covers as c (c.manga_id)}
+                    <tr><td>{c.manga_title}</td></tr>
+                {/each}
+            </tbody>
+        </table>
+        <Pager {page} {totalPages} onChange={onPageChange} />
+    {/if}
+</section>
+
+<style>
+    h2 {
+        margin: 0 0 var(--space-3) 0;
+        font-size: var(--font-sm);
+        color: var(--text-muted);
+        text-transform: uppercase;
+        letter-spacing: 0.04em;
+    }
+    table {
+        width: 100%;
+        border-collapse: collapse;
+    }
+    th,
+    td {
+        padding: var(--space-2);
+        text-align: left;
+        border-bottom: 1px solid var(--border);
+        font-size: var(--font-sm);
+    }
+    .deadhead {
+        display: flex;
+        justify-content: space-between;
+        align-items: center;
+        gap: var(--space-3);
+        flex-wrap: wrap;
+    }
+    .deadtools {
+        display: flex;
+        gap: var(--space-2);
+    }
+    .muted {
+        color: var(--text-muted);
+    }
+    .backlog {
+        margin-top: var(--space-4);
+    }
+</style>

frontend/src/lib/components/crawler/ProgressBar.svelte

@@ -0,0 +1,35 @@
+<script lang="ts">
+    // Tiny percent bar reused by the phase progress + per-chapter page
+    // progress in the admin crawler dashboard. Was an inline `{#snippet}`
+    // before; extracted so the per-component tables can use it too.
+    let { percent }: { percent: number } = $props();
+</script>
+
+<div class="bar" role="progressbar" aria-valuenow={percent} aria-valuemin="0" aria-valuemax="100">
+    <div class="fill" style:width="{Math.min(100, Math.max(0, percent))}%"></div>
+    <span class="label">{percent.toFixed(0)}%</span>
+</div>
+
+<style>
+    .bar {
+        position: relative;
+        background: var(--surface-elevated);
+        border-radius: var(--radius-sm, 4px);
+        height: 1.5rem;
+        margin: var(--space-2) 0;
+        overflow: hidden;
+    }
+    .fill {
+        height: 100%;
+        background: #22c55e;
+        transition: width 0.3s ease;
+    }
+    .label {
+        position: absolute;
+        top: 50%;
+        left: 50%;
+        transform: translate(-50%, -50%);
+        font-size: var(--font-xs);
+        font-weight: var(--weight-semibold);
+    }
+</style>

frontend/src/lib/components/crawler/RequeueAllConfirmModal.svelte

@@ -0,0 +1,32 @@
+<script lang="ts">
+    import Modal from '$lib/components/Modal.svelte';
+
+    let {
+        open,
+        total,
+        busy,
+        onCancel,
+        onConfirm
+    }: {
+        open: boolean;
+        total: number;
+        busy: boolean;
+        onCancel: () => void;
+        onConfirm: () => void;
+    } = $props();
+</script>
+
+<Modal {open} title="Requeue all dead jobs" onClose={onCancel} size="sm">
+    {#snippet children()}
+        <p>
+            This will flip every dead job ({total}) back to <code>pending</code>.
+            Chapters with a live job are skipped automatically; the rest will be
+            picked up by the next chapter worker.
+        </p>
+    {/snippet}
+    {#snippet footer()}
+        <!-- svelte-ignore a11y_autofocus -->
+        <button autofocus onclick={onCancel}>Cancel</button>
+        <button class="primary" onclick={onConfirm} disabled={busy}>Requeue {total}</button>
+    {/snippet}
+</Modal>

frontend/src/lib/components/crawler/RestartConfirmModal.svelte

@@ -0,0 +1,42 @@
+<script lang="ts">
+    import Modal from '$lib/components/Modal.svelte';
+
+    let {
+        open,
+        busy,
+        onCancel,
+        onConfirm
+    }: {
+        open: boolean;
+        busy: boolean;
+        onCancel: () => void;
+        onConfirm: () => void;
+    } = $props();
+</script>
+
+<Modal {open} title="Restart browser" onClose={onCancel} size="sm">
+    {#snippet children()}
+        <p>This relaunches Chromium and re-injects the session cookie.</p>
+        <ul class="coord">
+            <li>In-flight jobs are allowed to finish (bounded), then forced.</li>
+            <li>New jobs pause until the relaunch completes.</li>
+            <li>The metadata pass yields at its next checkpoint.</li>
+        </ul>
+    {/snippet}
+    {#snippet footer()}
+        <!-- Cancel autofocused so an accidental Enter on the modal
+             dismisses rather than triggering the destructive action. -->
+        <!-- svelte-ignore a11y_autofocus -->
+        <button autofocus onclick={onCancel}>Cancel</button>
+        <button class="primary" onclick={onConfirm} disabled={busy}>Restart</button>
+    {/snippet}
+</Modal>
+
+<style>
+    .coord {
+        margin: var(--space-2) 0;
+        padding-left: var(--space-4);
+        font-size: var(--font-sm);
+        color: var(--text-muted);
+    }
+</style>

frontend/src/lib/components/crawler/SearchBar.svelte

@@ -0,0 +1,22 @@
+<script lang="ts">
+    // Search input + button used by all three backlog tables on the
+    // crawler dashboard. The three sites had identical markup before;
+    // the audit (F2) flagged the duplication. `bindable` lets the
+    // parent keep `value` in $state and read it after Enter / click.
+    let {
+        value = $bindable(''),
+        placeholder = 'Search manga…',
+        onSearch
+    }: {
+        value?: string;
+        placeholder?: string;
+        onSearch: () => void;
+    } = $props();
+</script>
+
+<input
+    {placeholder}
+    bind:value
+    onkeydown={(e) => e.key === 'Enter' && onSearch()}
+/>
+<button onclick={onSearch}>Search</button>

frontend/src/lib/components/crawler/SessionModal.svelte

@@ -0,0 +1,49 @@
+<script lang="ts">
+    import Modal from '$lib/components/Modal.svelte';
+
+    let {
+        open,
+        busy,
+        phpsessid = $bindable(''),
+        result,
+        onCancel,
+        onSave
+    }: {
+        open: boolean;
+        busy: boolean;
+        phpsessid?: string;
+        result: string | null;
+        onCancel: () => void;
+        onSave: () => void;
+    } = $props();
+</script>
+
+<Modal {open} title="Manage crawler session" onClose={onCancel} size="md">
+    {#snippet children()}
+        <label for="phpsessid">PHPSESSID</label>
+        <input id="phpsessid" type="password" bind:value={phpsessid} autocomplete="off" />
+        <p class="hint">
+            Saving rewrites the cookie everywhere, persists it, restarts the browser, and re-probes.
+        </p>
+        {#if result}
+            <p class="sessionresult">{result}</p>
+        {/if}
+    {/snippet}
+    {#snippet footer()}
+        <button onclick={onCancel}>Cancel</button>
+        <button class="primary" onclick={onSave} disabled={busy || phpsessid.trim() === ''}
+            >Save &amp; validate</button
+        >
+    {/snippet}
+</Modal>
+
+<style>
+    .hint {
+        color: var(--text-muted);
+        font-size: var(--font-sm);
+    }
+    .sessionresult {
+        margin-top: var(--space-2);
+        font-size: var(--font-sm);
+    }
+</style>

frontend/src/routes/admin/+layout.svelte

@@ -6,6 +6,7 @@
         { href: '/admin', label: 'Overview' },
         { href: '/admin/users', label: 'Users' },
         { href: '/admin/mangas', label: 'Mangas' },
+        { href: '/admin/crawler', label: 'Crawler' },
         { href: '/admin/system', label: 'System' }
     ];
 </script>

frontend/src/routes/admin/crawler/+page.svelte

@@ -0,0 +1,532 @@
+<script lang="ts">
+    import { onMount, onDestroy } from 'svelte';
+    import CrawlerHero from '$lib/components/crawler/CrawlerHero.svelte';
+    import CrawlerControls from '$lib/components/crawler/CrawlerControls.svelte';
+    import ActiveChaptersCard from '$lib/components/crawler/ActiveChaptersCard.svelte';
+    import ActiveJobsTable from '$lib/components/crawler/ActiveJobsTable.svelte';
+    import MissingCoversTable from '$lib/components/crawler/MissingCoversTable.svelte';
+    import DeadJobsTable from '$lib/components/crawler/DeadJobsTable.svelte';
+    import SessionModal from '$lib/components/crawler/SessionModal.svelte';
+    import RestartConfirmModal from '$lib/components/crawler/RestartConfirmModal.svelte';
+    import RequeueAllConfirmModal from '$lib/components/crawler/RequeueAllConfirmModal.svelte';
+    import {
+        getCrawlerStatus,
+        crawlerStatusStreamUrl,
+        runCrawlerPass,
+        restartCrawlerBrowser,
+        updateCrawlerSession,
+        clearCrawlerSessionExpired,
+        listDeadJobs,
+        requeueDeadJobs,
+        listActiveJobs,
+        listMissingCovers,
+        type CrawlerStatus,
+        type DeadJob,
+        type ActiveJob,
+        type MissingCover,
+        type RequeueScope
+    } from '$lib/api/admin';
+
+    let status: CrawlerStatus | null = $state(null);
+    let error: string | null = $state(null);
+    let notice: string | null = $state(null);
+    let live = $state(false);
+    let source: EventSource | null = null;
+    let busy = $state(false);
+    // SSE error tracking (Q3): EventSource swallows status codes, so an
+    // auth-loss looks identical to a network blip and would reconnect
+    // forever. After this many consecutive errors we probe the status
+    // endpoint, which routes through `on401Hook` and surfaces the
+    // logout if appropriate.
+    let consecutiveSseErrors = 0;
+    const SSE_ERROR_PROBE_THRESHOLD = 5;
+    // Backlog refetch AbortControllers (Q4): a burst of SSE frames or
+    // search-key changes would otherwise fire concurrent fetches whose
+    // resolution order is last-wins-flicker. Each loader cancels its
+    // predecessor.
+    let activeAbort: AbortController | null = null;
+    let coversAbort: AbortController | null = null;
+    let deadAbort: AbortController | null = null;
+    // P4: debounce timers for backlog refetches. The SSE frame
+    // (chapter page tick, etc.) fires ~once/sec; refetching the queue
+    // on every frame produced a thundering herd against the same DB
+    // queries the SSE handler is already running.
+    let activeDebounce: ReturnType<typeof setTimeout> | null = null;
+    let coversDebounce: ReturnType<typeof setTimeout> | null = null;
+    const BACKLOG_DEBOUNCE_MS = 500;
+
+    // Dead jobs
+    let deadJobs: DeadJob[] = $state([]);
+    let deadTotal = $state(0);
+    let deadSearch = $state('');
+    let deadPage = $state(1);
+    const DEAD_LIMIT = 20;
+
+    // Queued chapters (pending/running)
+    let activeJobs: ActiveJob[] = $state([]);
+    let activeTotal = $state(0);
+    let activeSearch = $state('');
+    let activePage = $state(1);
+    const ACTIVE_LIMIT = 20;
+
+    // Queued covers (mangas missing a cover)
+    let covers: MissingCover[] = $state([]);
+    let coversTotal = $state(0);
+    let coversSearch = $state('');
+    let coversPage = $state(1);
+    const COVERS_LIMIT = 20;
+
+    // Modals
+    let sessionModalOpen = $state(false);
+    let restartModalOpen = $state(false);
+    let requeueAllModalOpen = $state(false);
+    let phpsessid = $state('');
+    let sessionResult: string | null = $state(null);
+
+    async function refresh() {
+        try {
+            status = await getCrawlerStatus();
+            error = null;
+        } catch (e) {
+            error = e instanceof Error ? e.message : 'refresh failed';
+        }
+    }
+
+    async function loadDeadJobs() {
+        deadAbort?.abort();
+        const ctrl = new AbortController();
+        deadAbort = ctrl;
+        try {
+            const resp = await listDeadJobs(
+                {
+                    search: deadSearch.trim() || undefined,
+                    limit: DEAD_LIMIT,
+                    offset: (deadPage - 1) * DEAD_LIMIT
+                },
+                { signal: ctrl.signal }
+            );
+            deadJobs = resp.items;
+            deadTotal = resp.page.total ?? resp.items.length;
+        } catch (e) {
+            if (ctrl.signal.aborted) return; // superseded by a newer call
+            error = e instanceof Error ? e.message : 'failed to load dead jobs';
+        }
+    }
+
+    async function loadActiveJobs() {
+        activeAbort?.abort();
+        const ctrl = new AbortController();
+        activeAbort = ctrl;
+        try {
+            const resp = await listActiveJobs(
+                {
+                    search: activeSearch.trim() || undefined,
+                    limit: ACTIVE_LIMIT,
+                    offset: (activePage - 1) * ACTIVE_LIMIT
+                },
+                { signal: ctrl.signal }
+            );
+            activeJobs = resp.items;
+            activeTotal = resp.page.total ?? resp.items.length;
+        } catch (e) {
+            if (ctrl.signal.aborted) return;
+            error = e instanceof Error ? e.message : 'failed to load queued chapters';
+        }
+    }
+
+    async function loadCovers() {
+        coversAbort?.abort();
+        const ctrl = new AbortController();
+        coversAbort = ctrl;
+        try {
+            const resp = await listMissingCovers(
+                {
+                    search: coversSearch.trim() || undefined,
+                    limit: COVERS_LIMIT,
+                    offset: (coversPage - 1) * COVERS_LIMIT
+                },
+                { signal: ctrl.signal }
+            );
+            covers = resp.items;
+            coversTotal = resp.page.total ?? resp.items.length;
+        } catch (e) {
+            if (ctrl.signal.aborted) return;
+            error = e instanceof Error ? e.message : 'failed to load queued covers';
+        }
+    }
+
+    // Auto-refresh the (fetched, not streamed) backlog lists when the live
+    // status shows the relevant counts moved — keeps the lists feeling
+    // live without pushing big payloads over SSE. `$effect` re-runs when
+    // these tracked values change; the timers debounce a burst of
+    // adjacent counts (typical during a chapter download that emits one
+    // poke per stored page) into a single refetch.
+    let lastQueueKey = $state('');
+    let lastCoversKey = $state(-1);
+    $effect(() => {
+        const k = `${status?.queue.pending ?? 0}:${status?.queue.running ?? 0}`;
+        if (k !== lastQueueKey) {
+            lastQueueKey = k;
+            if (activeDebounce) clearTimeout(activeDebounce);
+            activeDebounce = setTimeout(() => loadActiveJobs(), BACKLOG_DEBOUNCE_MS);
+        }
+    });
+    $effect(() => {
+        const c = status?.covers_queued ?? -1;
+        if (c !== lastCoversKey) {
+            lastCoversKey = c;
+            if (coversDebounce) clearTimeout(coversDebounce);
+            coversDebounce = setTimeout(() => loadCovers(), BACKLOG_DEBOUNCE_MS);
+        }
+    });
+
+    // Live updates via Server-Sent Events instead of polling. The
+    // EventSource is opened on mount and closed on destroy, so the
+    // subscription exists only while this page is showing live data.
+    function openStream() {
+        if (source) return;
+        const es = new EventSource(crawlerStatusStreamUrl(), { withCredentials: true });
+        es.addEventListener('status', (e) => {
+            try {
+                status = JSON.parse((e as MessageEvent).data) as CrawlerStatus;
+                error = null;
+                live = true;
+                consecutiveSseErrors = 0;
+            } catch {
+                // ignore a malformed frame; the next one will replace it
+            }
+        });
+        es.onopen = () => {
+            live = true;
+            consecutiveSseErrors = 0;
+        };
+        es.onerror = () => {
+            // The browser auto-reconnects; reflect the gap in the UI.
+            live = false;
+            consecutiveSseErrors += 1;
+            // Q3: EventSource can't read the HTTP status, so an auth
+            // loss (401) looks identical to a network blip. After
+            // several consecutive failures, probe the status endpoint
+            // via the normal API client so on401Hook can fire if the
+            // session is gone.
+            if (consecutiveSseErrors >= SSE_ERROR_PROBE_THRESHOLD) {
+                consecutiveSseErrors = 0; // reset so we don't probe-storm
+                getCrawlerStatus().catch(() => {
+                    // The client's on401 hook (set globally) handles
+                    // the logout; any other error we ignore — the
+                    // EventSource will keep retrying and the UI flag
+                    // already reflects `live = false`.
+                });
+            }
+        };
+        source = es;
+    }
+
+    function closeStream() {
+        source?.close();
+        source = null;
+        live = false;
+    }
+
+    // Q2: close the stream when the tab is hidden so the browser
+    // doesn't accumulate a stale connection (mobile Safari throttles SSE
+    // aggressively in background tabs), and reopen on visible. Same for
+    // pagehide / pageshow to handle the BFCache.
+    function onVisibilityChange() {
+        if (typeof document === 'undefined') return;
+        if (document.visibilityState === 'hidden') {
+            closeStream();
+        } else if (document.visibilityState === 'visible') {
+            openStream();
+        }
+    }
+    function onPageHide() {
+        closeStream();
+    }
+    function onPageShow() {
+        openStream();
+    }
+
+    onMount(() => {
+        // One-shot fetch for instant initial paint + resilience if SSE is
+        // blocked; the stream then drives subsequent updates.
+        refresh();
+        loadDeadJobs();
+        openStream();
+        if (typeof document !== 'undefined') {
+            document.addEventListener('visibilitychange', onVisibilityChange);
+            window.addEventListener('pagehide', onPageHide);
+            window.addEventListener('pageshow', onPageShow);
+        }
+    });
+    onDestroy(() => {
+        closeStream();
+        if (typeof document !== 'undefined') {
+            document.removeEventListener('visibilitychange', onVisibilityChange);
+            window.removeEventListener('pagehide', onPageHide);
+            window.removeEventListener('pageshow', onPageShow);
+        }
+        if (activeDebounce) clearTimeout(activeDebounce);
+        if (coversDebounce) clearTimeout(coversDebounce);
+        activeAbort?.abort();
+        coversAbort?.abort();
+        deadAbort?.abort();
+    });
+
+    async function withBusy(label: string, fn: () => Promise<void>) {
+        busy = true;
+        notice = null;
+        error = null;
+        try {
+            await fn();
+        } catch (e) {
+            error = e instanceof Error ? e.message : `${label} failed`;
+        } finally {
+            busy = false;
+            // Q5: only fall back to a fetch refresh when the live SSE
+            // stream isn't pushing updates. With `live === true` the
+            // backend's `status.poke()` after each action already
+            // pushes a fresh status frame; the extra fetch would race
+            // it and produce flicker.
+            if (!live) {
+                await refresh();
+            }
+        }
+    }
+
+    async function onRunPass() {
+        await withBusy('run pass', async () => {
+            await runCrawlerPass();
+            notice = 'Metadata pass started.';
+        });
+    }
+
+    async function onConfirmRestart() {
+        restartModalOpen = false;
+        await withBusy('restart browser', async () => {
+            const r = await restartCrawlerBrowser();
+            notice = r.ok ? 'Browser restarted.' : `Restart failed: ${r.error ?? 'unknown'}`;
+        });
+    }
+
+    async function onSaveSession() {
+        sessionResult = null;
+        busy = true;
+        try {
+            const r = await updateCrawlerSession(phpsessid);
+            sessionResult = r.valid
+                ? '✓ Session valid — workers resumed.'
+                : `✕ Probe failed: ${r.error ?? 'unauthenticated'}`;
+            if (r.valid) {
+                sessionModalOpen = false;
+                phpsessid = '';
+                notice = 'Session updated.';
+            }
+        } catch (e) {
+            sessionResult = e instanceof Error ? e.message : 'update failed';
+        } finally {
+            busy = false;
+            if (!live) await refresh();
+        }
+    }
+
+    async function onClearExpired() {
+        await withBusy('clear expired', async () => {
+            await clearCrawlerSessionExpired();
+            notice = 'Session-expired flag cleared.';
+        });
+    }
+
+    async function requeue(scope: RequeueScope) {
+        await withBusy('requeue', async () => {
+            const r = await requeueDeadJobs(scope);
+            notice = `Requeued ${r.requeued} job(s).`;
+            await loadDeadJobs();
+        });
+    }
+
+    async function onConfirmRequeueAll() {
+        requeueAllModalOpen = false;
+        await requeue({ scope: 'all', confirm: true });
+    }
+
+    function onSearchDead() {
+        deadPage = 1;
+        loadDeadJobs();
+    }
+    function onDeadPageChange(p: number) {
+        deadPage = p;
+        loadDeadJobs();
+    }
+    function onSearchActive() {
+        activePage = 1;
+        loadActiveJobs();
+    }
+    function onActivePageChange(p: number) {
+        activePage = p;
+        loadActiveJobs();
+    }
+    function onSearchCovers() {
+        coversPage = 1;
+        loadCovers();
+    }
+    function onCoversPageChange(p: number) {
+        coversPage = p;
+        loadCovers();
+    }
+
+    const deadTotalPages = $derived(Math.max(1, Math.ceil(deadTotal / DEAD_LIMIT)));
+    const activeTotalPages = $derived(Math.max(1, Math.ceil(activeTotal / ACTIVE_LIMIT)));
+    const coversTotalPages = $derived(Math.max(1, Math.ceil(coversTotal / COVERS_LIMIT)));
+</script>
+
+<div class="titlebar">
+    <h1>Crawler</h1>
+    <span class="livedot" class:on={live} title={live ? 'Live (SSE)' : 'Reconnecting…'}>
+        {live ? '● live' : '○ reconnecting…'}
+    </span>
+</div>
+
+{#if error}
+    <p class="error" role="alert">{error}</p>
+{/if}
+{#if notice}
+    <p class="notice" role="status">{notice}</p>
+{/if}
+
+{#if status}
+    <CrawlerHero {status} />
+
+    <CrawlerControls
+        {status}
+        {busy}
+        {onRunPass}
+        onOpenRestart={() => (restartModalOpen = true)}
+        onOpenSession={() => {
+            sessionModalOpen = true;
+            sessionResult = null;
+        }}
+        {onClearExpired}
+    />
+
+    <ActiveChaptersCard {status} />
+{:else}
+    <p>Loading…</p>
+{/if}
+
+<ActiveJobsTable
+    jobs={activeJobs}
+    total={activeTotal}
+    page={activePage}
+    totalPages={activeTotalPages}
+    bind:search={activeSearch}
+    onSearch={onSearchActive}
+    onPageChange={onActivePageChange}
+/>
+
+<MissingCoversTable
+    {covers}
+    total={coversTotal}
+    page={coversPage}
+    totalPages={coversTotalPages}
+    bind:search={coversSearch}
+    onSearch={onSearchCovers}
+    onPageChange={onCoversPageChange}
+/>
+
+<DeadJobsTable
+    jobs={deadJobs}
+    total={deadTotal}
+    page={deadPage}
+    totalPages={deadTotalPages}
+    bind:search={deadSearch}
+    {busy}
+    onSearch={onSearchDead}
+    onPageChange={onDeadPageChange}
+    onRequeue={requeue}
+    onRequeueAll={() => (requeueAllModalOpen = true)}
+/>
+
+<RestartConfirmModal
+    open={restartModalOpen}
+    {busy}
+    onCancel={() => (restartModalOpen = false)}
+    onConfirm={onConfirmRestart}
+/>
+
+<RequeueAllConfirmModal
+    open={requeueAllModalOpen}
+    total={deadTotal}
+    {busy}
+    onCancel={() => (requeueAllModalOpen = false)}
+    onConfirm={onConfirmRequeueAll}
+/>
+
+<SessionModal
+    open={sessionModalOpen}
+    {busy}
+    bind:phpsessid
+    result={sessionResult}
+    onCancel={() => (sessionModalOpen = false)}
+    onSave={onSaveSession}
+/>
+
+<style>
+    h1 {
+        margin: 0;
+    }
+    .titlebar {
+        display: flex;
+        align-items: baseline;
+        gap: var(--space-3);
+        margin-bottom: var(--space-4);
+    }
+    .livedot {
+        font-size: var(--font-sm);
+        color: var(--text-muted);
+    }
+    .livedot.on {
+        color: var(--success, #0a7d2c);
+    }
+    .notice {
+        color: var(--success, #0a7d2c);
+        padding: var(--space-2) var(--space-3);
+        border: 1px solid var(--success, #0a7d2c);
+        border-radius: var(--radius-md);
+        margin-bottom: var(--space-3);
+    }
+    .error {
+        color: var(--danger, #dc2626);
+        padding: var(--space-2) var(--space-3);
+        border: 1px solid var(--danger, #dc2626);
+        border-radius: var(--radius-md);
+        margin-bottom: var(--space-3);
+    }
+    /* badges (shared convention with admin/mangas) */
+    :global(.badge) {
+        display: inline-block;
+        padding: 0 var(--space-2);
+        border-radius: var(--radius-sm, 4px);
+        font-size: var(--font-xs);
+        font-weight: var(--weight-semibold);
+        text-transform: uppercase;
+        letter-spacing: 0.04em;
+        border: 1px solid var(--border);
+        background: var(--surface);
+    }
+    :global(.badge-synced) {
+        background: #dcfce7;
+        color: #166534;
+        border-color: #86efac;
+    }
+    :global(.badge-in_progress),
+    :global(.badge-downloading) {
+        background: #fef3c7;
+        color: #92400e;
+        border-color: #fcd34d;
+    }
+    :global(.badge-not_downloaded) {
+        background: var(--surface-elevated);
+        color: var(--text-muted);
+    }
+</style>

frontend/src/routes/admin/mangas/+page.svelte

@@ -3,6 +3,7 @@
     import {
         listAdminMangas,
         listAdminChapters,
+        requeueDeadJobs,
         type AdminMangasPage,
         type AdminChapterRow,
         type MangaSyncState
@@ -59,6 +60,39 @@
     function badgeClass(state: string): string {
         return `badge badge-${state}`;
     }
+
+    let requeuingChapter: string | null = $state(null);
+
+    /** Requeue the dead job(s) for a single failed chapter, then patch
+     *  the local state instead of refetching the whole list. Refetching
+     *  500 chapters every time an operator clicks "Requeue" was wasteful;
+     *  the only field that changes is the per-chapter sync state of the
+     *  affected row. F4 in the audit. */
+    async function requeueChapter(mangaId: string, chapterId: string) {
+        requeuingChapter = chapterId;
+        error = null;
+        try {
+            await requeueDeadJobs({ scope: 'chapter', chapter_id: chapterId });
+            const existing = chaptersByManga[mangaId];
+            if (existing && existing !== 'loading') {
+                // After requeue the dead job is back to `pending`; the
+                // chapter itself stays "not_downloaded" until a worker
+                // picks it up and stores pages.
+                chaptersByManga[mangaId] = {
+                    items: existing.items.map((c) =>
+                        c.id === chapterId
+                            ? { ...c, sync_state: 'not_downloaded' as const }
+                            : c
+                    ),
+                    total: existing.total
+                };
+            }
+        } catch (e) {
+            error = e instanceof ApiError ? e.message : 'requeue failed';
+        } finally {
+            requeuingChapter = null;
+        }
+    }
 </script>
 
 <h1>Mangas</h1>
@@ -153,6 +187,17 @@
                                                         <span class={badgeClass(c.sync_state)}>
                                                             {c.sync_state}
                                                         </span>
+                                                        {#if c.sync_state === 'failed'}
+                                                            <button
+                                                                class="requeue"
+                                                                onclick={() => requeueChapter(m.id, c.id)}
+                                                                disabled={requeuingChapter === c.id}
+                                                                title="Requeue this chapter"
+                                                                aria-label={`Requeue chapter ${c.number}`}
+                                                            >
+                                                                ↻ requeue
+                                                            </button>
+                                                        {/if}
                                                     </td>
                                                 </tr>
                                             {/each}
@@ -272,6 +317,11 @@
         color: #991b1b;
         border-color: #fca5a5;
     }
+    .requeue {
+        margin-left: var(--space-2);
+        font-size: var(--font-xs);
+        padding: 0 var(--space-2);
+    }
     .badge-not_downloaded {
         background: var(--surface-elevated);
         color: var(--text-muted);