The backend now boots an internal crawler daemon that runs a daily
metadata pass (CRAWLER_DAILY_AT in CRAWLER_TZ, advisory-lock guarded
for multi-replica safety) and drains SyncChapterContent jobs from
crawler_jobs through a worker pool. Chromium launches lazily on first
job and is torn down after CRAWLER_IDLE_TIMEOUT_S seconds of inactivity.
Modules:
- crawler::browser_manager — lazy-launch / idle-teardown wrapper
around browser::Handle, with an on_launch hook that re-injects
PHPSESSID on every fresh Chromium spawn.
- crawler::pipeline — run_metadata_pass (the shared discover/upsert
/cover/sync-chapters loop) and the enqueue_bookmarked_pending helper
used by the cron tick.
- crawler::daemon — cron task + worker pool, behind two trait seams
(MetadataPass, ChapterDispatcher) so tests can inject stubs without
standing up Chromium or a live source.
Behavior:
- CRAWLER_DAEMON=false skips daemon spawn entirely (default for tests).
- Catch-up tick fires on startup if the last persisted slot was missed.
- A SyncOutcome::SessionExpired sets a sticky AtomicBool; workers
idle until operator restart with a refreshed PHPSESSID.
- Worker dispatch wrapped in catch_unwind so a panicking handler
marks the job failed instead of taking down the worker.
- Migration 0015 adds a small crawler_state k-v table for the
last_metadata_tick_at watermark.
Dep additions: chrono-tz (IANA TZ parsing).
CLI (bin/crawler) reuses pipeline::run_metadata_pass and now holds
the browser via BrowserManager so the on_launch session injection
flow stays in one place. Inline chapter-content sync semantics are
unchanged — the queue is for the daemon, force-refetches and manual
backfills still bypass it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
POST /api/v1/mangas and POST /api/v1/mangas/{id}/chapters now accept
multipart/form-data, gated by CurrentUser:
- /mangas: required `metadata` part (NewManga JSON) + optional `cover`
image part.
- /mangas/{id}/chapters: required `metadata` (NewChapter JSON) + one or
more `page` parts ordered by arrival. Returns 404 if the parent manga
doesn't exist, 409 on duplicate (manga_id, number).
MIME is sniffed via the `infer` crate (magic bytes), not the
client-supplied filename or Content-Type. Whitelist:
jpeg / png / webp / gif / avif. Anything else → 415
unsupported_media_type. The stored key's extension is derived from the
sniffed type so a "page1.png" that's actually a JPEG lands as `.jpg`.
Size cap is two-layer:
- Request body cap (config.max_request_bytes, default 200 MiB) enforced
by axum's DefaultBodyLimit before the handler sees the request.
- Per-image-part cap (config.max_file_bytes, default 20 MiB) enforced
after reading the part, so a single oversized image can't pass even
if the total request fits.
Storage keys follow the layout documented in CLAUDE.md:
- mangas/{manga_id}/cover.{ext}
- mangas/{manga_id}/chapters/{chapter_id}/pages/{nnnn}.{ext} (1-indexed).
AppError grows PayloadTooLarge/UnsupportedMediaType/ValidationFailed
(413 / 415 / 422). ValidationFailed carries a `details` JSON object the
client can use to highlight bad fields (e.g. {"title":"required"}).
Top-level matching in code() stays exhaustive.
Backend coverage in tests/api_uploads.rs (10 cases):
- create_manga_with_cover_stores_image — file is reachable via
/api/v1/files/{key} with the right Content-Type.
- create_manga_without_cover_leaves_path_null.
- create_manga_rejects_non_image_cover_with_415 — PDF claimed as png.
- create_manga_rejects_oversized_cover_with_413.
- create_chapter_with_pages_stores_each — extension derived from
sniffed MIME, files reachable in arrival order.
- create_chapter_rejects_when_no_pages_with_422 — details.page set.
- create_chapter_rejects_renamed_non_image_page → 415.
- create_chapter_returns_409_on_duplicate_number.
- create_chapter_requires_authentication → 401.
- create_chapter_under_unknown_manga_is_404.
Existing tests/api_mangas.rs is migrated to multipart; the create
response is now 201 Created. tests/common::MultipartBuilder builds the
body by hand so the test crate stays free of HTTP-client deps.
Frontend lib/api/mangas.ts: createManga now sends FormData (metadata +
optional cover Blob). Browser fills in the boundary header automatically.
Vitest asserts the FormData structure via FileReader (jsdom doesn't
implement Blob.text()).
E2E tests wait for the post-hydration nav-login link before
interacting with the login form, fixing a flake where pre-hydration
clicks would submit via the browser default and bypass our handler.
Lockstep version bump to 0.5.0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the full auth flow. Reads stay public; writes (currently only POST
/api/v1/mangas) require a CurrentUser. Both browsers and bot scripts hit
the same endpoints — they just present credentials differently.
Migration 0002_auth.sql introduces users.password_hash, a sessions
table, and an api_tokens table. Sessions and api_tokens store only
sha256(raw_token) — the raw value lives in the cookie or the
Authorization header.
New endpoints under /api/v1/auth/:
- POST /register — argon2id hash, creates a session, sets cookie.
- POST /login — verifies, rotates to a fresh session (old ones expire
naturally so other devices stay signed in).
- POST /logout — deletes the server-side session row + clears the
cookie via Max-Age=0.
- GET /me — current user via the new CurrentUser extractor.
- POST /tokens — issue a bot bearer token; raw value returned exactly
once at creation.
- DELETE /tokens/{id} — owner-only: 404 if unknown, 403 if it exists
but belongs to another user, 204 on success.
The CurrentUser axum extractor resolves cookie first, then
Authorization: Bearer; failure → AppError::Unauthenticated (401). New
AppError variants Unauthenticated/Forbidden/Conflict carry the matching
envelope codes; the top-level match in `code()` stays exhaustive.
Backend integration coverage in tests/api_auth.rs: register sets a
HttpOnly SameSite=Lax cookie and never leaks password_hash; duplicate
username → 409; weak password → 400; login rotates the cookie; wrong
password / unknown user → 401; /me with vs without cookie; logout
invalidates the cookie; bot-token roundtrip via Bearer; user A cannot
delete user B's token (403); unknown delete → 404.
Frontend:
- lib/api/auth.ts — typed wrappers; me() returns null on 401.
- lib/session.svelte.ts — per-tab user state with a seq counter to
guard against an in-flight /me clobbering a fresh setUser.
- lib/api/client.ts — request<T> returns undefined for 204.
- routes/login + routes/register — forms with action="javascript:void(0)"
so the no-JS path is a no-op (avoids the hydration-race where a
pre-attach click would submit via the browser default).
- routes/+layout.svelte — session-aware nav: spinner → user + Logout,
or Login / Register.
- e2e/auth-flow.spec.ts — login flips the layout, logout flips back;
bad credentials surface the API error message.
Config grows AuthConfig (cookie_secure, cookie_domain, session_ttl_days)
and CORS_ALLOWED_ORIGINS. CORS middleware is mounted in app::build and
stays a no-op (same-origin) until origins are listed.
Lockstep version bump to 0.3.0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Set up Mangalord with a Rust/axum backend, SvelteKit frontend, Postgres,
and Docker Compose deployment. Establishes the architecture and TDD
patterns the project will extend:
- Hexagonal-ish backend layering (domain / repo / storage / api) with
a pluggable Storage trait (LocalStorage today, S3 as a future impl).
- Initial migration: users, mangas, chapters, bookmarks.
- Vertical slice for mangas (list, search, create, get) with
#[sqlx::test] integration coverage and storage unit tests.
- SvelteKit frontend using Svelte 5 runes, typed API client, Vitest
unit tests and Playwright e2e with route mocking.
- CLAUDE.md documenting layering, TDD/git/SemVer workflow rules, and
extension points (tags, fulltext search, OCR, S3, auth).
- Project-scoped .claude/settings.json with permission allowlist for
the toolchain (git, cargo, npm/vite, docker, psql, gh, doc fetches).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
