Commit Graph

23 Commits

Author SHA1 Message Date
MechaCat02
d0cd31c9a7 feat(analysis): live SSE event stream for the admin dashboard
Broadcasts analysis progress so the dashboard updates live:

- analysis::events: AnalysisEvents broadcaster + AnalysisEvent
  (Enqueued / Started / Completed / Failed), carrying the
  manga/chapter/page breadcrumb.
- The worker daemon resolves each page's breadcrumb (repo::page::locate)
  and publishes Started before dispatch and Completed/Failed after.
- The admin reenqueue publishes Enqueued (scoped by manga/chapter).
- GET /v1/admin/analysis/status/stream — SSE (RequireAdmin) forwarding
  each event as a named `analysis` frame; broadcast lag emits a `lagged`
  frame. AppState carries the always-present events bus.

Tests: worker publishes started+completed (with breadcrumb) and failed;
SSE route is admin-gated (403 non-admin) and returns text/event-stream.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 21:06:20 +02:00
MechaCat02
7d80a437bf feat(analysis): worker daemon + real dispatcher + app wiring
The background analysis worker that drains analyze_page jobs:

- analysis::daemon: a lean sibling of the crawler daemon — leases only
  KIND_ANALYZE_PAGE, skip-if-done-unless-force, lease heartbeat, panic +
  timeout isolation, ack done/failed, and a failed page_analysis row on
  terminal (dead-lettered) failure. AnalyzeDispatcher trait seam.
- RealAnalyzeDispatcher: load page → storage.get → VisionClient.analyze →
  persist_analysis (skips a deleted page; caps image bytes).
- app::build spawns the daemon (own plain reqwest client) when
  ANALYSIS_ENABLED; AppHandle/main shut it down alongside the crawler.
- repo::page::find_by_id.

Tests: dispatch+ack-done, skip-when-done, force re-dispatch, terminal
failure writes failed row, panic isolation, ignores non-analyze jobs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 18:51:53 +02:00
MechaCat02
2bbf1595ff feat(analysis): enqueue page-analysis on upload/crawl + admin backfill endpoints
Wires the analyze_page job kind into the page-create paths and adds admin
controls, all gated on a new ANALYSIS_ENABLED flag (AppState.analysis_enabled,
config::AnalysisConfig):

- Chapter upload (api::chapters) enqueues one analyze_page job per page
  after the tx commits (best-effort; failures logged, never fatal).
- Crawler content sync (persist_pages → RETURNING id) enqueues per
  persisted page when the daemon dispatcher's analysis_enabled flag is set;
  threaded through sync_chapter_content (CLI/resync pass false).
- repo::page_analysis::enqueue_all_pages: bulk backfill via INSERT..SELECT,
  skipping done pages (only_unanalyzed) and existing pending jobs.
- New admin endpoints (RequireAdmin, 503 when disabled, audited):
  POST /admin/analysis/reenqueue and POST /admin/pages/:id/analyze (force).

Tests: upload enqueues N jobs / no-op when disabled; crawler persist_pages
enqueue gate; admin reenqueue (backfill, idempotent, only-unanalyzed, 503,
non-admin 403) and force-analyze (force flag, 404).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 18:39:03 +02:00
MechaCat02
d6ac648ac9 feat(admin): crawler observability dashboard + reliability hardening (0.55.0)
Admin-only crawler dashboard backed by an SSE live-status stream,
coordinated browser restart, runtime PHPSESSID refresh, dead-letter
requeue, and a batch of reliability fixes. Closes everything from
the two-pass audit (10 commits' worth) and bumps 0.52.0 -> 0.55.0.

Backend:

- New /admin/crawler/* surface (cookie-auth, RequireAdmin) split
  into status / control / dead_jobs / backlog modules. SSE stream
  composes in-memory status with DB-derived queue counts, memoizes
  the counts for 1s and debounces watch pokes for 250ms (~10x QPS
  reduction per subscriber). One-shot GET /admin/crawler shares the
  same compose path.
- POST /admin/crawler/run gated by manual_pass_lock try_lock_owned
  (409 Conflict on overlapping click); browser restart goes through
  the coordinated_restart gate (drain + relaunch + auto-clear of the
  sticky session_expired flag on Ok).
- Runtime PHPSESSID refresh via SessionController (allow-list
  validation, never logged, audit row carries SHA-256 fingerprint).
  Storage layer is repo::crawler::runtime_session_{load,persist}.
- Dead-letter requeue with four scopes (all/manga/chapter/job);
  scope=all requires confirm:true; DISTINCT ON dedup keeps the
  partial unique index from rejecting requeues for chapters with
  multiple dead rows. SQL is four &'static str constants per scope.
- StatusHandle + ChapterGuard / CoverGuard RAII model survives
  panics; last-writer-wins on cover so concurrent dispatches don't
  clobber each other's slot. Pure functions (should_stop /
  should_mark_clean_exit / should_abort_pass) with named regression
  tests.
- Reliability bundle: per-lease heartbeat, jitter on retries,
  per-job timeout, circuit breaker on consecutive failures, BrowserManager
  coordinated restart gate, request fingerprint changes.
- Streaming page download: Storage::put_stream trait method,
  LocalStorage impl atomic via temp + fsync + UUID-suffixed rename.
  Pages stream through with peak memory ~one HTTP chunk + 64-byte
  sniff prefix instead of one full image per dispatch.
- New partial indexes (migration 0022): mangas_missing_cover_idx
  and crawler_jobs_dead_idx, both ordered by updated_at DESC to
  match the dashboard's LIMIT/OFFSET reads.
- Security hardening: admin_csrf_guard (Origin/Referer allowlist
  on /admin/* mutations, opt-in via ADMIN_ALLOWED_ORIGINS),
  admin_no_store_guard (Cache-Control: no-store on admin
  responses), audit rows carry per-scope target_id.

Frontend:

- /admin/crawler page decomposed into lib/components/crawler/
  (11 components: ProgressBar, SearchBar, CrawlerHero,
  CrawlerControls, ActiveChaptersCard, ActiveJobsTable,
  MissingCoversTable, DeadJobsTable, RestartConfirmModal,
  RequeueAllConfirmModal, SessionModal). Page is 532 LOC of
  orchestration; each component 22-148 LOC.
- EventSource lifecycle wired to visibilitychange / pagehide /
  pageshow (BFCache); after 5 consecutive errors probes the status
  endpoint so a 401 routes through the global on401Hook instead of
  infinite silent reconnects.
- Backlog $effect refetches debounced 500ms with per-loader
  AbortControllers; refresh after a control action only runs when
  the SSE stream is dead.
- Inline requeue button on /admin/mangas patches the affected row's
  sync_state locally (no full chapter-list refetch); proper
  aria-label. Requeue-all gets its own confirm modal; both confirm
  modals autofocus Cancel.
- SvelteKit reverse proxy bypasses its 5-minute AbortController
  for Accept: text/event-stream; pure shouldBypassProxyTimeout
  helper covered by unit tests.

Config / docs:

- New env vars (.env.example): ADMIN_ALLOWED_ORIGINS,
  CRAWLER_JOB_TIMEOUT_SECS, CRAWLER_METADATA_MAX_CONSECUTIVE_FAILURES,
  CRAWLER_BROWSER_RESTART_THRESHOLD.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 18:49:56 +02:00
MechaCat02
c134bdbbde feat: cover retry backfill + admin force-resync for manga & chapter (0.50.0)
Adds a per-tick cover-backfill pass to the crawler daemon so mangas whose
cover download failed on first attempt get retried — the metadata pass's
early-stop optimisation otherwise prevents the walk from revisiting them.

Adds admin-only POST /admin/mangas/:id/resync and POST /admin/chapters/:id/resync
that refetch metadata + cover (or chapter content with force_refetch) from the
crawler source synchronously and return the refreshed row. Surfaced in the
UI as "Force resync" buttons on the manga detail and reader pages,
admin-only via session.user.is_admin.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-01 22:00:09 +02:00
MechaCat02
e50fc093c3 feat: add PRIVATE_MODE site-wide auth gate (0.48.0)
When `PRIVATE_MODE=true`, every API path except a small allowlist
(`/health`, `/auth/{config,login,logout,register}`) requires a valid
session cookie or bearer token — anonymous reads are rejected with
401. Self-registration is force-disabled in private mode regardless
of `ALLOW_SELF_REGISTER`, so a locked-down instance flips with a
single switch (admins still mint accounts via `POST /admin/users`).

The backend gate is a tower middleware that reuses the existing
`CurrentUser` extractor, so the cookie + bearer paths cannot drift
from per-handler auth. `/auth/config` now exposes the flag plus the
effective `self_register_enabled` value so the frontend can render
the navbar correctly on the first paint.

On the frontend, a new universal root `+layout.ts` fetches the
config and redirects anonymous visitors to `/login?next=<path>`
before page-specific loads fire. The redirect is UX only — the
backend middleware is the source of truth, so crafted requests
still 401.

Defaults stay public (`PRIVATE_MODE=false`); existing deployments
need no env change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-01 20:11:22 +02:00
MechaCat02
72756cfef2 feat(crawler): honour CRAWLER_LIMIT in the in-process daemon (0.47.0)
The CLI binary already capped runs at CRAWLER_LIMIT mangas, but the
daemon's RealMetadataPass passed a hardcoded `0` (no cap) to
`pipeline::run_metadata_pass`, so the env var was silently ignored once
the daemon took over the metadata pass.

Adds `manga_limit` to `CrawlerConfig`, reads it from `CRAWLER_LIMIT`
(default 0 = no cap), and threads it through `RealMetadataPass::run`
so a daemon-driven sweep stops at the same boundary as a CLI run.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-01 20:07:01 +02:00
MechaCat02
4e20350645 fix(crawler): translate socks5h:// → socks5:// for Chromium --proxy-server
All checks were successful
deploy / test-backend (push) Successful in 19m30s
deploy / test-frontend (push) Successful in 9m42s
deploy / build-and-push (push) Successful in 8m10s
deploy / deploy (push) Successful in 15s
Chromium doesn't know the socks5h scheme (curl/reqwest convention)
and bails navigations with ERR_NO_SUPPORTED_PROXIES. It does, however,
send destination hostnames over SOCKS5 by default, so stripping the
`h` is a pure scheme rename — remote-DNS behaviour is preserved.

reqwest keeps the user's original CRAWLER_PROXY string (`socks5h://...`
remains valid and meaningful for it).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-31 20:56:45 +02:00
MechaCat02
8c6378b877 feat(crawler): recircuit TOR on transient pages and unauthenticated probes
- target.rs swaps retry_on_transient → retry_on_transient_with_hook,
  signaling NEWNYM via ctx.tor between attempts when configured.
- session.rs gains verify_session_with_recircuit; the bare
  verify_session is now a one-line wrapper passing tor=None,
  unauth_max_recircuit=0. The inner run_session_probe_loop is
  pure-over-IO and unit-tested with closure-based fakes.
- content.rs extracts fetch_chapter_html_once + the closure-driven
  fetch_chapter_html_with_recircuit, used by sync_chapter_content to
  retry on Transient or Unauthenticated up to a recircuit_budget.
  Budget = 0 (no TOR) preserves original behavior bit-for-bit.
- app.rs and bin/crawler.rs construct the controller before on_launch
  and pass it into verify_session_with_recircuit, so a transient
  hiccup at startup no longer requires PHPSESSID rotation.

Recircuit budget defaults to CRAWLER_TOR_RECIRCUIT_MAX_ATTEMPTS (3).
Errors from NEWNYM are logged and swallowed — failing to recircuit
should not take down the crawl.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-31 20:01:04 +02:00
MechaCat02
8557e432a2 feat(crawler): plumb TorController through FetchContext and pipelines
Adds CRAWLER_TOR_CONTROL_URL / _PASSWORD / _COOKIE_PATH /
_RECIRCUIT_MAX_ATTEMPTS to CrawlerConfig and to bin/crawler.rs's
env reads. Constructs an Option<Arc<TorController>> at daemon /
CLI startup and threads it through FetchContext,
pipeline::run_metadata_pass, and content::sync_chapter_content as
Option<&TorController>.

Pure scaffolding — the controller isn't used yet; behavior is
unchanged. Next commit wires the retry hooks and session-probe
recircuit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-31 19:59:47 +02:00
MechaCat02
ab8b7acc34 feat(auth): admin role with cookie-only RequireAdmin extractor (0.37.0)
Adds an `is_admin` flag on users plus the substrate every later PR in the
admin feature builds on:

- migration 0018 adds the column with default false
- `repo::user::bootstrap_admin` creates or promotes the user named by
  `ADMIN_USERNAME` at startup, hashing `ADMIN_PASSWORD` only when the row
  is new — never overwriting an existing hash, so an operator can rotate
  the admin password via the UI without env-var conflict
- `CurrentSessionUser` extractor accepts only the session cookie;
  `RequireAdmin` composes over it and additionally requires
  `user.is_admin`. Bearer tokens are intentionally excluded so an
  admin's bot token never inherits admin authority (privilege-escalation
  surface that bites every "API keys reuse user perms" auth design)
- demotion is instant: `RequireAdmin` re-reads the user row each request

`/api/v1/auth/me` now exposes `is_admin`; no other response embeds
`User`, so no privacy fanout to audit.
2026-05-30 21:26:26 +02:00
MechaCat02
70e8a7895c fix(crawler): relaunch chromium on CDP / nav-timeout errors (0.36.3)
BrowserManager only re-launched chromium when the cached handle was
None. A crash mid-pass left the handle Some pointing at a dead
process — every subsequent acquire returned the zombie Browser, and
every nav cascaded CDP errors until the idle reaper fired.

Add BrowserManager::invalidate(): take the inner mutex, drop the
handle (closing it if present), and signal the next acquire to
relaunch. Idempotent — invalidating an empty handle is a no-op.

Wire detection via NavError::is_likely_browser_dead and a
chain-walking anyhow_looks_browser_dead helper: substring-match
common channel/connection/transport/WebSocket markers and surface
NavError::Timeout as "presumed dead." Apply at both error
boundaries — RealChapterDispatcher::dispatch and
RealMetadataPass::run.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-30 18:39:19 +02:00
MechaCat02
9f56f283d4 feat(crawler): single-mode walker gated by recovery flag (0.36.0)
Collapses the crawler to a single newest-first walker and replaces the
N-consecutive-unchanged streak with a per-manga rule: stop on the first
manga where metadata is Unchanged AND chapter sync reports zero new
chapters. The early stop is gated by a per-source recovery flag stored
in `crawler_state` — set to `false` when a run starts, back to `true`
only on a clean exit (end-of-walk or intentional stop). A crashed run
leaves the flag `false` automatically (no shutdown code runs), so the
next tick walks the full catalog instead of bailing at the first
caught-up manga.

This means a crashed mid-walk run self-heals on the next tick: the
flag stays `false`, the next walk visits every page (recovering
anything the crash missed past its crash point), and steady state
resumes once the recovery sweep reaches end-of-walk.

Removed:
- DiscoverMode enum, Backfill mode, the boundary re-check +
  displaced-refs machinery in TargetSourceWalker.
- Drop-pass (mark_dropped_mangas) and seed-completion plumbing
  (mark_seed_completed / seed_completed_at). The recovery flag
  subsumes the seed-completion signal; drop detection was explicitly
  opted out.
- JobPayload::Discover (no production callers).
- CRAWLER_MODE / CRAWLER_INCREMENTAL_STOP_AFTER env vars and the
  CrawlerModePref config type.

`should_mark_clean_exit(walked_to_completion, hit_stop_condition)`
encodes the clean-exit truth table in its signature — `hit_limit` is
deliberately absent so a future edit cannot accidentally count a
caller-imposed cap as a clean exit.

Net -501 lines, 261 backend tests passing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 23:49:28 +02:00
MechaCat02
f57ca8e45c feat: harden auth, shutdown, and session bundle (0.35.0)
Some checks failed
deploy / test-backend (push) Failing after 1m37s
deploy / test-frontend (push) Failing after 16m31s
deploy / build-and-push (push) Has been skipped
deploy / deploy (push) Has been skipped
Three features bundled into one release:

- rate-limit /auth/login, /register, /me/password (token bucket,
  5 req/sec sustained with 10-request burst by default; 429 +
  Retry-After header on hit; tracing::warn! per hit so operators
  see attack patterns; AUTH_RATE_PER_SEC / AUTH_RATE_BURST env knobs)
- handle SIGTERM for graceful container stops (replaces bare
  ctrl_c() with a select over ctrl_c + SignalKind::terminate() so
  docker compose stop runs the daemon shutdown path instead of
  letting Chromium leak past SIGKILL)
- clear session.user on 401 from any API call (setOn401Hook in
  api/client.ts, registered from session.svelte.ts gated on
  $app/environment::browser so the SSR bundle never installs it;
  fixes "logged in but no bookmarks/collections" mid-session
  expiry state)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 20:27:21 +02:00
MechaCat02
8d34132883 bugfix: security & correctness bundle (0.34.1)
Five fixes bundled into one release:

- preserve user-attached tags across crawler upserts
  (repo::crawler::sync_tags now scopes to added_by IS NULL; orphaned
  attachments from deleted users are reaped as crawler-owned)
- gate manga PATCH and cover endpoints on uploaded_by (require_can_edit
  in api::mangas; non-NULL uploaded_by must match the caller)
- equalise login response time across user-existence branches
  (run argon2 against a OnceLock-cached dummy hash on the no-user
  branch so timing doesn't leak username existence)
- crawler download defences (SSRF allowlist of host literals
  including IPv4-mapped IPv6 ranges, 32 MiB streamed size cap,
  reject non-whitelisted image types, three-way chapter-probe
  classifier replaces the binary #avatar_menu check)
- tighten validation and clean up dead unload path
  (attach_tag + create_token enforce 64-char caps; LocalStorage
  rejects NUL bytes explicitly; reader flushFinalProgress drops
  the always-405 sendBeacon path)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 20:24:51 +02:00
MechaCat02
c320eda7cd chore: dedupe is_unique_violation, lift SQL into repo, centralise URL parsing
Three layering cleanups from REVIEW.md §5 / §3:

- Drop the three private `is_unique_violation` helpers in
  repo::{user,chapter,bookmark} in favour of sqlx 0.8's
  `DatabaseError::is_unique_violation()` method (already used by
  repo::collection).
- Remove the unreachable 23505 branch in repo::chapter::create — the
  (manga_id, number) UNIQUE was dropped in 0013, so the defensive arm
  could no longer fire. A doc note records what to do if uniqueness
  is re-added.
- Move three inline SQL queries out of handlers/daemon into repo
  functions: bookmarks' chapter-belongs-to-manga guard
  (`repo::chapter::belongs_to_manga`), the daemon's dispatch lookup
  (`repo::chapter::dispatch_target`), and the daemon's page_count
  safety net (`repo::chapter::page_count`). Restores the
  handlers→repo layering invariant in CLAUDE.md.
- New `crawler::url_utils` module consolidates host_of / origin_of /
  registrable_domain — they used to live in three crawler submodules
  with diverging edge-case behaviour. Tests moved with them.
- Doc cross-references on repo::author::set_for_manga and
  repo::genre::set_for_manga pointing to the crawler's name-keyed
  variants, so the intentional duplication is discoverable.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 20:24:05 +02:00
MechaCat02
45ce0d8f12 feat: incremental crawl mode with seed-completion gate (0.33.0)
Daemon now auto-detects mode per source: Backfill until the first
full walk records `seed_completed:<source>` in `crawler_state`, then
Incremental (newest-first, stops after N consecutive Unchanged
upserts). `CRAWLER_MODE` overrides to a fixed mode; CLI rejects
`auto` since it has no pre-run DB state.

`Source::discover` returns a lazy `DiscoverWalk` so Incremental can
break out mid-walk without prefetching pages. The drop pass and seed
marker are now gated on a true full walk — fixes a latent soft-drop
of the index tail under partial sweeps.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 06:41:26 +02:00
MechaCat02
9fe0f26d75 feat: in-process crawler daemon with cron and worker pool (0.28.0)
The backend now boots an internal crawler daemon that runs a daily
metadata pass (CRAWLER_DAILY_AT in CRAWLER_TZ, advisory-lock guarded
for multi-replica safety) and drains SyncChapterContent jobs from
crawler_jobs through a worker pool. Chromium launches lazily on first
job and is torn down after CRAWLER_IDLE_TIMEOUT_S seconds of inactivity.

Modules:
- crawler::browser_manager — lazy-launch / idle-teardown wrapper
  around browser::Handle, with an on_launch hook that re-injects
  PHPSESSID on every fresh Chromium spawn.
- crawler::pipeline — run_metadata_pass (the shared discover/upsert
  /cover/sync-chapters loop) and the enqueue_bookmarked_pending helper
  used by the cron tick.
- crawler::daemon — cron task + worker pool, behind two trait seams
  (MetadataPass, ChapterDispatcher) so tests can inject stubs without
  standing up Chromium or a live source.

Behavior:
- CRAWLER_DAEMON=false skips daemon spawn entirely (default for tests).
- Catch-up tick fires on startup if the last persisted slot was missed.
- A SyncOutcome::SessionExpired sets a sticky AtomicBool; workers
  idle until operator restart with a refreshed PHPSESSID.
- Worker dispatch wrapped in catch_unwind so a panicking handler
  marks the job failed instead of taking down the worker.
- Migration 0015 adds a small crawler_state k-v table for the
  last_metadata_tick_at watermark.

Dep additions: chrono-tz (IANA TZ parsing).

CLI (bin/crawler) reuses pipeline::run_metadata_pass and now holds
the browser via BrowserManager so the on_launch session injection
flow stays in one place. Inline chapter-content sync semantics are
unchanged — the queue is for the daemon, force-refetches and manual
backfills still bypass it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:32:02 +02:00
MechaCat02
2df4084c56 test: tighten audit-flagged tests and add missing coverage
Tightens three tests whose names overstated what they checked:

- `login_succeeds_and_rotates_session` now asserts the login cookie
  differs from the registration cookie, and that the registration
  cookie is still valid after login (the documented contract).
- `storage::local::rejects_path_traversal` exercises three extra
  rejection paths the existing implementation already handled but the
  tests didn't probe: `a/./b`, the single-segment `.`, and the empty
  segment `a//b`.
- `create_and_use_bot_token` asserts that `token_hash` is *absent*
  from the response (`get(...).is_none()`), not just `is_null()`,
  which would have accepted an explicit `"token_hash": null` payload
  too.

Adds four coverage cases that the audit flagged as missing:

- `me_rejects_expired_session` — hand-craft a session row with
  `expires_at = now() - 1h`, hit `/auth/me` with the matching cookie,
  expect 401 + `unauthenticated`. Proves the extractor's
  `expires_at > now()` filter is wired.
- `concurrent_manga_bookmarks_serialised_by_unique_index` — spawn two
  POSTs in parallel for the same `(user, manga, chapter=null)`,
  assert one wins (201) and one collides (409) via the partial unique
  index from migration 0004.
- `bookmark_create_accepts_bearer_token` — mint a bot token and POST
  /bookmarks with `Authorization: Bearer`, asserting `CurrentUser`
  resolves identically to the cookie path on a write endpoint (not
  just `/auth/me`).
- Three new unit tests on `app::cors_layer` covering the allowlist
  (origin reflected, credentials true), a foreign origin (no
  allow-origin header emitted), and the same-origin default (empty
  allowlist emits no CORS headers at all).

`cors_layer` is `pub(crate)` now so the tests in `app::tests` can
reach it; the function itself is unchanged.

No version bump.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 23:30:19 +02:00
MechaCat02
a92f6f70e2 feat: multipart manga + chapter uploads with magic-byte MIME sniff
POST /api/v1/mangas and POST /api/v1/mangas/{id}/chapters now accept
multipart/form-data, gated by CurrentUser:

- /mangas: required `metadata` part (NewManga JSON) + optional `cover`
  image part.
- /mangas/{id}/chapters: required `metadata` (NewChapter JSON) + one or
  more `page` parts ordered by arrival. Returns 404 if the parent manga
  doesn't exist, 409 on duplicate (manga_id, number).

MIME is sniffed via the `infer` crate (magic bytes), not the
client-supplied filename or Content-Type. Whitelist:
jpeg / png / webp / gif / avif. Anything else → 415
unsupported_media_type. The stored key's extension is derived from the
sniffed type so a "page1.png" that's actually a JPEG lands as `.jpg`.

Size cap is two-layer:
- Request body cap (config.max_request_bytes, default 200 MiB) enforced
  by axum's DefaultBodyLimit before the handler sees the request.
- Per-image-part cap (config.max_file_bytes, default 20 MiB) enforced
  after reading the part, so a single oversized image can't pass even
  if the total request fits.

Storage keys follow the layout documented in CLAUDE.md:
- mangas/{manga_id}/cover.{ext}
- mangas/{manga_id}/chapters/{chapter_id}/pages/{nnnn}.{ext} (1-indexed).

AppError grows PayloadTooLarge/UnsupportedMediaType/ValidationFailed
(413 / 415 / 422). ValidationFailed carries a `details` JSON object the
client can use to highlight bad fields (e.g. {"title":"required"}).
Top-level matching in code() stays exhaustive.

Backend coverage in tests/api_uploads.rs (10 cases):
- create_manga_with_cover_stores_image — file is reachable via
  /api/v1/files/{key} with the right Content-Type.
- create_manga_without_cover_leaves_path_null.
- create_manga_rejects_non_image_cover_with_415 — PDF claimed as png.
- create_manga_rejects_oversized_cover_with_413.
- create_chapter_with_pages_stores_each — extension derived from
  sniffed MIME, files reachable in arrival order.
- create_chapter_rejects_when_no_pages_with_422 — details.page set.
- create_chapter_rejects_renamed_non_image_page → 415.
- create_chapter_returns_409_on_duplicate_number.
- create_chapter_requires_authentication → 401.
- create_chapter_under_unknown_manga_is_404.

Existing tests/api_mangas.rs is migrated to multipart; the create
response is now 201 Created. tests/common::MultipartBuilder builds the
body by hand so the test crate stays free of HTTP-client deps.

Frontend lib/api/mangas.ts: createManga now sends FormData (metadata +
optional cover Blob). Browser fills in the boundary header automatically.
Vitest asserts the FormData structure via FileReader (jsdom doesn't
implement Blob.text()).

E2E tests wait for the post-hydration nav-login link before
interacting with the login form, fixing a flake where pre-hydration
clicks would submit via the browser default and bypass our handler.

Lockstep version bump to 0.5.0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 22:21:10 +02:00
MechaCat02
383cfbed3b feat: argon2id passwords, session cookies, bot bearer tokens
Adds the full auth flow. Reads stay public; writes (currently only POST
/api/v1/mangas) require a CurrentUser. Both browsers and bot scripts hit
the same endpoints — they just present credentials differently.

Migration 0002_auth.sql introduces users.password_hash, a sessions
table, and an api_tokens table. Sessions and api_tokens store only
sha256(raw_token) — the raw value lives in the cookie or the
Authorization header.

New endpoints under /api/v1/auth/:
- POST /register — argon2id hash, creates a session, sets cookie.
- POST /login — verifies, rotates to a fresh session (old ones expire
  naturally so other devices stay signed in).
- POST /logout — deletes the server-side session row + clears the
  cookie via Max-Age=0.
- GET  /me — current user via the new CurrentUser extractor.
- POST /tokens — issue a bot bearer token; raw value returned exactly
  once at creation.
- DELETE /tokens/{id} — owner-only: 404 if unknown, 403 if it exists
  but belongs to another user, 204 on success.

The CurrentUser axum extractor resolves cookie first, then
Authorization: Bearer; failure → AppError::Unauthenticated (401). New
AppError variants Unauthenticated/Forbidden/Conflict carry the matching
envelope codes; the top-level match in `code()` stays exhaustive.

Backend integration coverage in tests/api_auth.rs: register sets a
HttpOnly SameSite=Lax cookie and never leaks password_hash; duplicate
username → 409; weak password → 400; login rotates the cookie; wrong
password / unknown user → 401; /me with vs without cookie; logout
invalidates the cookie; bot-token roundtrip via Bearer; user A cannot
delete user B's token (403); unknown delete → 404.

Frontend:
- lib/api/auth.ts — typed wrappers; me() returns null on 401.
- lib/session.svelte.ts — per-tab user state with a seq counter to
  guard against an in-flight /me clobbering a fresh setUser.
- lib/api/client.ts — request<T> returns undefined for 204.
- routes/login + routes/register — forms with action="javascript:void(0)"
  so the no-JS path is a no-op (avoids the hydration-race where a
  pre-attach click would submit via the browser default).
- routes/+layout.svelte — session-aware nav: spinner → user + Logout,
  or Login / Register.
- e2e/auth-flow.spec.ts — login flips the layout, logout flips back;
  bad credentials surface the API error message.

Config grows AuthConfig (cookie_secure, cookie_domain, session_ttl_days)
and CORS_ALLOWED_ORIGINS. CORS middleware is mounted in app::build and
stays a no-op (same-origin) until origins are listed.

Lockstep version bump to 0.3.0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 22:04:25 +02:00
MechaCat02
ce9a01793f feat: nest API under /api/v1, structured error envelope, paged lists
Move every handler from /api/* to /api/v1/*. /api/* is now reserved for
future versioning.

Standardise the error response shape across the API as
{"error": {"code": "snake_case", "message": "..."}}. AppError gains a
`code()` whose top-level variants are matched exhaustively without a
wildcard — new variants are a compile error until coded. 500-class
responses always emit the fixed "internal error" string and log the
real cause via tracing only.

Lock in the list pagination envelope as {"items": [...], "page": {
"limit", "offset", "total"}} and apply it to GET /api/v1/mangas. `total`
serialises as null until feat/list-search-polish lands an indexed count.

The frontend client parses the envelope into ApiError.code with an
http_error fallback for non-JSON bodies. listMangas now returns the
paged shape; the root route consumes .items. New client.test.ts covers
envelope parsing and the fallback paths.

Lockstep version bump to 0.2.0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 21:41:20 +02:00
MechaCat02
6c1d04aaf4 chore: initial project scaffold
Set up Mangalord with a Rust/axum backend, SvelteKit frontend, Postgres,
and Docker Compose deployment. Establishes the architecture and TDD
patterns the project will extend:

- Hexagonal-ish backend layering (domain / repo / storage / api) with
  a pluggable Storage trait (LocalStorage today, S3 as a future impl).
- Initial migration: users, mangas, chapters, bookmarks.
- Vertical slice for mangas (list, search, create, get) with
  #[sqlx::test] integration coverage and storage unit tests.
- SvelteKit frontend using Svelte 5 runes, typed API client, Vitest
  unit tests and Playwright e2e with route mocking.
- CLAUDE.md documenting layering, TDD/git/SemVer workflow rules, and
  extension points (tags, fulltext search, OCR, S3, auth).
- Project-scoped .claude/settings.json with permission allowlist for
  the toolchain (git, cargo, npm/vite, docker, psql, gh, doc fetches).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 21:05:16 +02:00