Extends the live dashboard so an operator can see exactly what's being
fetched, in realtime:
- Chapters being crawled now are tracked in the status as `active_chapters`
(manga title · ch.N) with a live page counter that climbs per stored page
(set_chapter_pages, pushed via the existing watch→SSE). The dispatcher
registers each via an RAII ChapterGuard (sync Mutex) that removes the
entry on completion, panic, or timeout-drop — replacing the old per-worker
slot model.
- Covers: status now carries the cover being fetched now (`current_cover`,
set around download_and_store_cover in both the metadata pass and backfill)
and a `covers_queued` backlog count; CoverBackfill phase gains index/total.
- Two paginated backlog endpoints (fetched on demand, auto-refreshed when the
live counts change): GET /admin/crawler/active-jobs (which chapters of which
mangas are queued/running) and GET /admin/crawler/covers (mangas missing a
cover). repo: list_active_jobs, list_missing_cover_mangas, count_missing_covers.
- dispatch_target now also returns manga title + chapter number.
Frontend: the crawler page replaces the Workers table with an Active-chapters
table (live page bars), adds a current-cover line + covers-queued figure, and
two backlog sections (Queued chapters / Queued covers) with search + Pager,
auto-refetched via $effect on the live counts.
Tests: status guard/page + cover unit tests; repo list/count tests; endpoint
tests; frontend api tests. Version 0.53.1 -> 0.54.0.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the dashboard's 5s polling with a Server-Sent Events stream:
- StatusHandle gains a tokio `watch` version bumped on every mutation;
GET /admin/crawler/stream subscribes and pushes a composed snapshot
immediately on connect, then on every status change (instant, no
lost-wakeup) plus a 5s backstop for DB queue counts / browser phase.
- Non-status signals poke the notifier so they push immediately too:
session-expired (worker), session update / clear-expired / browser
restart (endpoints).
- compose_status is shared by the one-shot GET and the stream; the stream
tolerates transient DB errors with a keep-alive comment instead of
tearing down.
Frontend: the crawler page opens an EventSource on mount and closes it on
destroy, so the subscription is scoped to the active page (no global
subscription). A one-shot fetch still paints initial state / serves as a
fallback if SSE is blocked; a live/reconnecting indicator reflects the
connection. The existing reverse proxy already streams SSE (its abort
timer is cleared once response headers arrive), so no proxy change needed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New /admin/crawler tab (5s-polled): status hero (daemon/session/browser
pills, phase line + progress bar, session-expired banner, last-pass),
controls (run pass, restart browser w/ confirm, manage session modal,
clear expired), queue gauges + worker table, and a dead-jobs table with
search, Pager, and per-job / per-manga / all requeue.
Adds inline "requeue" on failed chapters in the admin manga page, the
typed api-client functions in lib/api/admin.ts (+ tests), and the Crawler
nav tab. Version 0.52.0 -> 0.53.0.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a per-tick cover-backfill pass to the crawler daemon so mangas whose
cover download failed on first attempt get retried — the metadata pass's
early-stop optimisation otherwise prevents the walk from revisiting them.
Adds admin-only POST /admin/mangas/:id/resync and POST /admin/chapters/:id/resync
that refetch metadata + cover (or chapter content with force_refetch) from the
crawler source synchronously and return the refreshed row. Surfaced in the
UI as "Force resync" buttons on the manga detail and reader pages,
admin-only via session.user.is_admin.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pairs with the ALLOW_SELF_REGISTER toggle from 0.42.0: admins can mint
accounts regardless of the toggle state, so a closed-membership
deployment still has a working enrollment path. The endpoint accepts
{ username, password, is_admin? } so admins can mint co-admins in one
call (avoiding a separate promote + extra audit row for the common
"invite a co-admin" flow).
Implementation:
- POST /api/v1/admin/users guarded by RequireAdmin
- Reuses validate_username / validate_password from api::auth (made
pub(crate)) so the admin path can never produce an account self-
register would reject and vice versa
- repo::user::admin_create_user wraps INSERT + admin_audit insert in
a single tx — same "audit reflects what committed" semantics as the
existing admin_safe_* fns
- Audit row: action="create_user", payload={username, is_admin}
Frontend:
- createAdminUser() in lib/api/admin.ts
- /admin/users grows a collapsible "Create user" form above the table
(username, password, "Make admin" checkbox). Errors surface inline;
the list reloads on success.
Backend tests: 7 new, including the headline
`create_user_works_even_when_self_register_disabled` that pins the
admin-create path is NOT gated by the public toggle.
Addresses the security-audit findings on top of the admin feature stack:
M1: /admin/mangas/:id/chapters now paginates (default limit 200, max 500).
A long-runner with thousands of chapters would otherwise produce a multi-MB
response with that many scalar subqueries per row — admin-only but a real
stall risk on one expand-click. Adds explicit pagination tests for the cap
and offset; frontend renders a "Showing first N of M" hint when the cap
clips the result.
L1: repo::user::set_is_admin renamed to set_is_admin_unchecked with a
doc-comment pointing at admin_safe_set_is_admin for production use. The
short name was a footgun — a future contributor reaching for it would
silently bypass self-protection, the last-admin invariant, and the audit
log. Used only by integration-test setup; production code goes through
the admin_safe_* paths.
CSRF posture: build_session_cookie carries a comment that the
SameSite=Lax default is the project's CSRF defense for state-changing
mutations and breaks the instant anyone adds a side-effecting GET under
/admin/*. Spells out what to do then (Strict + explicit token check).
Test counts: 43 backend admin tests + 12 vitest admin tests all green;
svelte-check 0/0 across 446 files.
Adds the SvelteKit /admin route tree backed by the admin endpoints
landed in PR 1-4. Pages: Overview (alerts + summary cards), Users
(list / promote-demote / delete), Mangas (list with sync state +
expandable per-chapter state), System (live disk/mem/cpu bars,
refreshing every 5s).
Security model: the backend's RequireAdmin extractor is the actual
boundary. /admin/+layout.ts calls getSystemStats() at load and
translates the response — 401 → redirect to /login, 403 → throw
SvelteKit error(403) which renders the framework error page. The
header's "Admin" link is hidden unless `session.user?.is_admin`,
but that's UX only.
Carries `is_admin: boolean` through to the frontend User TS type so
the header check works and so admin tables can show role per row.
Vitest covers lib/api/admin.ts (10 tests: list/delete/PATCH for
users, sync-state filter for mangas, nested chapter route, system
disk-nullable case). Playwright is intentionally deferred until the
routes stabilise — admin UI is operator-only and changes shape often
in v0.
