Commit Graph

4 Commits

Author SHA1 Message Date
MechaCat02
cb34eeb82e feat(analysis): add in-process ocrs OCR backend
Add a CPU-only, English-first OCR engine (the `ocrs` crate on the `rten`
runtime) as an alternative to the local-LLM "vision" backend, which ran
too slowly on the target Raspberry Pi 5.

- `ANALYSIS_BACKEND` (default `ocr`) selects the engine; `vision` keeps the
  existing LLM path. Deploy-time, env-only — not admin-tunable.
- `analysis::ocr` adds an `OcrEngine` trait (testable seam), the `OcrsEngine`
  impl (models loaded once, inference on the blocking pool), and an
  `OcrAnalyzeDispatcher` that plugs into the existing `AnalyzeDispatcher`
  seam and reuses `persist_analysis`, so OCR text lands in `page_ocr_text`
  and the `search_doc` tsvector exactly as the vision path produces them.
- The backend image bakes the two `.rten` models into /models, where
  `OCRS_DETECTION_MODEL` / `OCRS_RECOGNITION_MODEL` default.

Because `/v1/me/page-search` ranks on `search_doc`, OCR text search works
the moment pages are processed. Auto-tagging, scene description and NSFW
flags remain the vision backend's job (deferred).

Bumps to 0.90.0 (minor) in both manifests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 07:12:01 +02:00
MechaCat02
d0cd31c9a7 feat(analysis): live SSE event stream for the admin dashboard
Broadcasts analysis progress so the dashboard updates live:

- analysis::events: AnalysisEvents broadcaster + AnalysisEvent
  (Enqueued / Started / Completed / Failed), carrying the
  manga/chapter/page breadcrumb.
- The worker daemon resolves each page's breadcrumb (repo::page::locate)
  and publishes Started before dispatch and Completed/Failed after.
- The admin reenqueue publishes Enqueued (scoped by manga/chapter).
- GET /v1/admin/analysis/status/stream — SSE (RequireAdmin) forwarding
  each event as a named `analysis` frame; broadcast lag emits a `lagged`
  frame. AppState carries the always-present events bus.

Tests: worker publishes started+completed (with breadcrumb) and failed;
SSE route is admin-gated (403 non-admin) and returns text/event-stream.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 21:06:20 +02:00
MechaCat02
7d80a437bf feat(analysis): worker daemon + real dispatcher + app wiring
The background analysis worker that drains analyze_page jobs:

- analysis::daemon: a lean sibling of the crawler daemon — leases only
  KIND_ANALYZE_PAGE, skip-if-done-unless-force, lease heartbeat, panic +
  timeout isolation, ack done/failed, and a failed page_analysis row on
  terminal (dead-lettered) failure. AnalyzeDispatcher trait seam.
- RealAnalyzeDispatcher: load page → storage.get → VisionClient.analyze →
  persist_analysis (skips a deleted page; caps image bytes).
- app::build spawns the daemon (own plain reqwest client) when
  ANALYSIS_ENABLED; AppHandle/main shut it down alongside the crawler.
- repo::page::find_by_id.

Tests: dispatch+ack-done, skip-when-done, force re-dispatch, terminal
failure writes failed row, panic isolation, ignores non-analyze jobs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 18:51:53 +02:00
MechaCat02
dbde6e02c4 feat(analysis): vision client + system prompt + AnalysisConfig
The OpenAI-compatible vision client and the bounded prompt/output handling
for the analysis worker (no DB, fully unit-tested):

- analysis::prompt: terse system prompt carrying the JSON schema, the OCR
  kind / content-warning vocabularies, and the output-size caps.
- analysis::vision: VisionClient (downscale via the new `image` dep →
  base64 data-URL → chat/completions with an image_url part), plus pure
  parse_chat_completion (fence/prose-tolerant) and sanitize (drop empty
  OCR, truncate, clamp tags to 10 via the shared page-tag normalizer,
  filter unknown warnings).
- config::AnalysisConfig extended with endpoint/model/api_key/timeouts/
  max_tokens/max_image_dim/max_image_bytes + from_env.
- Deps: add `image` (jpeg/png/webp), reqwest `json` feature. Expose
  api::page_tags::normalize_tag as pub(crate) for reuse.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 18:45:57 +02:00