Compare commits
3 Commits
main
...
2a978aa333
| Author | SHA1 | Date | |
|---|---|---|---|
|
2a978aa333 | ||
|
|
82264c74cd | ||
|
|
cb34eeb82e |
13
.env.example
13
.env.example
@@ -277,6 +277,19 @@ CRAWLER_TZ=UTC
|
||||
# ANALYSIS_ENABLED Turn the worker on at first boot. Toggleable live in
|
||||
# the dashboard. Default `false`.
|
||||
ANALYSIS_ENABLED=false
|
||||
# ANALYSIS_BACKEND Which engine the worker runs. env-ONLY (deploy-time).
|
||||
# `ocr` (default) = the in-process ocrs engine: fast,
|
||||
# CPU-only, text-only, ideal for a Pi. `vision` = the local
|
||||
# LLM at ANALYSIS_VISION_URL (full OCR + tags + scene +
|
||||
# safety, but heavy). The ANALYSIS_VISION_* / ANALYSIS_API_KEY
|
||||
# knobs below only apply to `vision`.
|
||||
ANALYSIS_BACKEND=ocr
|
||||
# OCRS_DETECTION_MODEL / OCRS_RECOGNITION_MODEL Paths to the ocrs `.rten`
|
||||
# text-detection / -recognition models (only read when ANALYSIS_BACKEND=ocr).
|
||||
# The backend image bakes both into /models, so the defaults work unchanged;
|
||||
# override only to point at custom-trained models.
|
||||
OCRS_DETECTION_MODEL=/models/text-detection.rten
|
||||
OCRS_RECOGNITION_MODEL=/models/text-recognition.rten
|
||||
# ANALYSIS_VISION_URL /v1/chat/completions endpoint. Required when enabled.
|
||||
# For the bundled vision container, use
|
||||
# http://mangalord-vision:8000/v1/chat/completions.
|
||||
|
||||
136
HYBRID-OCR-VISION.md
Normal file
136
HYBRID-OCR-VISION.md
Normal file
@@ -0,0 +1,136 @@
|
||||
# Hybrid OCR→Vision analysis pipeline — design brief for the dev agent
|
||||
|
||||
**Status: design only (not implemented).** Forward-looking model for a future
|
||||
`ANALYSIS_BACKEND=hybrid`, on top of the shipped `ocr` (default) and `vision`
|
||||
backends. Constraint that shapes it: `ocrs` and the vision LLM must **not** be
|
||||
resident at the same time (Pi RAM ceiling). OCR has priority; vision is a
|
||||
preemptible, only-when-OCR-is-drained background phase.
|
||||
|
||||
## The key realization: this is the crawl mutex, again
|
||||
|
||||
The repo already runs a **`vision-manager`** sidecar
|
||||
([vision-manager/manager.sh](vision-manager/manager.sh),
|
||||
[VISION-AUTOSCALE.md](VISION-AUTOSCALE.md)) that owns the `mangalord-vision`
|
||||
(llama.cpp, ~4 GiB) container lifecycle via a scoped docker-socket-proxy — the
|
||||
backend gets no Docker access. It already:
|
||||
|
||||
- **Starts** vision when `pending_analysis()` > 0 (counts `analyze_page` jobs).
|
||||
- **Idle-stops** it after `STOP_DEBOUNCE` (anti-thrash) once the queue drains.
|
||||
- **Defers starting** vision while `crawl_running()` > 0 — the **"RAM mutex"**:
|
||||
the crawler's browser and the LLM can't both fit, so vision waits for crawl.
|
||||
- **Memory-yield**: SIGTERMs a running vision at a HIGH host-RAM watermark,
|
||||
blocks starting at a LOW watermark (see [VISION-MEMORY-YIELD.md](VISION-MEMORY-YIELD.md)).
|
||||
|
||||
The OCR↔vision relationship is the relationship that already exists between
|
||||
crawl and vision. OCR is just a third RAM-competing, higher-priority workload
|
||||
that vision must defer to. Extend the existing arbiter rather than build a new one.
|
||||
|
||||
## Job model: split one queue into two
|
||||
|
||||
Today there is one job kind, `analyze_page`. Split it:
|
||||
|
||||
- **`ocr_page`** — the in-process ocrs pass (cheap, fast, no container).
|
||||
- **`ground_page`** — the vision grounding pass (needs the LLM container).
|
||||
|
||||
**Pipeline (hybrid mode):** upload → enqueue `ocr_page`. The OCR daemon runs
|
||||
ocrs, writes `page_ocr_text` + `search_doc` (text searchable *immediately*), then
|
||||
enqueues `ground_page` for that page. The grounding daemon later adds
|
||||
tags/scene/safety. Each kind gets its own dedup index (mirror migration 0031).
|
||||
|
||||
Two daemons, two queues.
|
||||
|
||||
## Arbitration: OCR priority + memory exclusivity
|
||||
|
||||
Responsibilities split cleanly between the in-backend daemons and the manager:
|
||||
|
||||
**Backend — OCR daemon** (already built): leases `ocr_page`, always allowed to
|
||||
run (ocrs is small/local). This is the priority phase.
|
||||
|
||||
**Backend — grounding daemon:** leases `ground_page`, but gates leasing on
|
||||
**both**:
|
||||
1. `VisionReadiness` (the existing `/health` gate — vision container up), AND
|
||||
2. **`ocr_backlog == 0`** (a cheap `count(ocr_page WHERE state IN pending,running)`).
|
||||
|
||||
Gate #2 is the whole priority mechanism, in-process, no aborts: the instant OCR
|
||||
work appears, the grounding daemon **finishes its current page** (the in-flight
|
||||
lease completes normally) and then **stops leasing** new pages and parks — "it
|
||||
finishes, then yields." It resumes only when OCR has fully drained.
|
||||
|
||||
**vision-manager:** two one-line changes to the existing logic:
|
||||
1. `pending_analysis()` counts **`ground_page`** (not `ocr_page`) — vision only
|
||||
starts when there is *grounding* work.
|
||||
2. Add an **OCR start-block mutex** identical to `RESPECT_CRAWL_MUTEX`: defer
|
||||
starting vision while `ocr_page` backlog > 0. (`RESPECT_OCR_MUTEX=1`.)
|
||||
|
||||
**Why this yields memory exclusivity:** while OCR backlog > 0, the grounding
|
||||
daemon won't lease → vision goes idle → the manager's idle-debounce stops the
|
||||
container (and never restarts it under the OCR mutex). Only ocrs (small) is
|
||||
resident. When OCR drains, grounding resumes, `pending_analysis()` > 0 again, the
|
||||
manager starts vision. Only the *big* consumer (the LLM container) is ever
|
||||
mutually exclusive with active OCR — which is the actual RAM constraint.
|
||||
|
||||
**The one overlap window:** if OCR work arrives mid-grounding-page, ocrs (small,
|
||||
in-process) briefly coexists with the one in-flight vision page before the
|
||||
grounding daemon parks. ocrs's footprint makes this a non-issue in practice. If a
|
||||
deployment needs *hard* exclusivity even there, the OCR daemon can additionally
|
||||
wait for `vision_running == false` before its first dispatch. No deadlock: OCR's
|
||||
wait is transient (one grounding page, bounded by `job_timeout`), while vision's
|
||||
deferral on OCR backlog is the persistent, priority-respecting side.
|
||||
|
||||
**Anti-thrash:** the manager's `STOP_DEBOUNCE` already prevents a stray OCR page
|
||||
from cold-cycling the 4 GiB model. Chapter uploads arrive as bursts, so OCR
|
||||
drains a whole batch in one phase before vision resumes — the natural good case.
|
||||
|
||||
## Data-model split (the real refactor)
|
||||
|
||||
`persist_analysis` currently writes OCR + tags + scene + safety in one
|
||||
transaction and derives `search_doc` from the OCR rows (+ scene weight C). Two
|
||||
passes means splitting it, sharing the tsvector builder:
|
||||
|
||||
- **`persist_ocr(page, lines)`** — writes `page_ocr_text`, sets
|
||||
`search_doc` from OCR (A/B/D buckets), marks the page text-searchable. Status
|
||||
reflects OCR completion.
|
||||
- **`persist_grounding(page, tags, scene, safety)`** — writes auto-tags,
|
||||
warnings, scene; **recomputes** `search_doc` to fold in the scene (weight C) on
|
||||
top of the existing OCR buckets. Records grounding completion (e.g. a
|
||||
`grounded_at` column or tags-present sentinel).
|
||||
|
||||
This keeps text search live after the cheap OCR phase, with semantic search/tags
|
||||
arriving after the (deferred) grounding phase.
|
||||
|
||||
## Reusable pieces
|
||||
|
||||
- OCR side: the shipped `OcrsEngine` + `OcrAnalyzeDispatcher`
|
||||
([backend/src/analysis/ocr.rs](backend/src/analysis/ocr.rs)) — retarget to
|
||||
`ocr_page`, and on success enqueue `ground_page`.
|
||||
- Vision side: factor `VisionClient::ground(image, mime, ocr_text)` out of
|
||||
`analyze()`'s Pass-B block
|
||||
([backend/src/analysis/vision.rs](backend/src/analysis/vision.rs) ~240–261) —
|
||||
pure refactor, existing tests cover it. The grounding daemon is the existing
|
||||
analysis daemon retargeted to `ground_page` with the extra `ocr_backlog==0`
|
||||
lease gate.
|
||||
- Manager: `pending_analysis()` kind swap + `RESPECT_OCR_MUTEX` block, mirroring
|
||||
the crawl-mutex branch already at [vision-manager/manager.sh](vision-manager/manager.sh).
|
||||
|
||||
## Scope / sequencing
|
||||
|
||||
Larger than an inline hybrid dispatcher: job-kind split + dedup migrations,
|
||||
`persist_*` split + search_doc-builder extraction, pipeline enqueue (OCR→ground),
|
||||
grounding daemon lease gate, and the manager mutex. No new privileged surface
|
||||
(reuses the docker-socket-proxy). Suggested order:
|
||||
1. Split `persist_analysis` → `persist_ocr` / `persist_grounding` (+ tests).
|
||||
2. Split job kinds + dedup migrations; retarget the OCR daemon to `ocr_page`.
|
||||
3. `VisionClient::ground()` refactor + grounding daemon (`ground_page`, gated on
|
||||
`ocr_backlog==0`).
|
||||
4. OCR→ground enqueue on OCR success (hybrid mode only).
|
||||
5. vision-manager: `ground_page` counting + `RESPECT_OCR_MUTEX`.
|
||||
|
||||
Pure-`ocr` (shipped default) and `vision` (legacy) backends are unaffected; this
|
||||
is the `hybrid` backend's runtime model.
|
||||
|
||||
## Open decision
|
||||
|
||||
Hard exclusivity in the one overlap window (OCR daemon waits for `vision_running
|
||||
== false` before its first dispatch) — needed only if ocrs's few-hundred-MB
|
||||
footprint can't coexist with a single in-flight grounding page on the target
|
||||
Pi's RAM. Default: don't gate (rely on the grounding daemon parking fast).
|
||||
258
backend/Cargo.lock
generated
258
backend/Cargo.lock
generated
@@ -214,6 +214,12 @@ version = "1.8.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06"
|
||||
|
||||
[[package]]
|
||||
name = "bitflags"
|
||||
version = "1.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a"
|
||||
|
||||
[[package]]
|
||||
name = "bitflags"
|
||||
version = "2.11.1"
|
||||
@@ -514,6 +520,25 @@ dependencies = [
|
||||
"cfg-if",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "crossbeam-deque"
|
||||
version = "0.8.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "9dd111b7b7f7d55b72c0a6ae361660ee5853c9af73f70c3c2ef6858b950e2e51"
|
||||
dependencies = [
|
||||
"crossbeam-epoch",
|
||||
"crossbeam-utils",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "crossbeam-epoch"
|
||||
version = "0.9.18"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "5b82ac4a3c2ca9c3460964f020e1402edd5753411d7737aa39c3714ad1b5420e"
|
||||
dependencies = [
|
||||
"crossbeam-utils",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "crossbeam-queue"
|
||||
version = "0.3.12"
|
||||
@@ -638,7 +663,7 @@ version = "0.3.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1e0e367e4e7da84520dedcac1901e4da967309406d1e51017ae1abfb97adbd38"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
"objc2",
|
||||
]
|
||||
|
||||
@@ -772,6 +797,16 @@ version = "0.1.9"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "5baebc0774151f905a1a2cc41989300b1e6fbb29aff0ceffa1064fdd3088d582"
|
||||
|
||||
[[package]]
|
||||
name = "flatbuffers"
|
||||
version = "24.12.23"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "4f1baf0dbf96932ec9a3038d57900329c015b0bfb7b63d904f3bc27e2b02a096"
|
||||
dependencies = [
|
||||
"bitflags 1.3.2",
|
||||
"rustc_version",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "flate2"
|
||||
version = "1.1.9"
|
||||
@@ -1059,6 +1094,12 @@ version = "0.5.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
|
||||
|
||||
[[package]]
|
||||
name = "hermit-abi"
|
||||
version = "0.5.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "fc0fef456e4baa96da950455cd02c081ca953b141298e41db3fc7e36b1da849c"
|
||||
|
||||
[[package]]
|
||||
name = "hex"
|
||||
version = "0.4.3"
|
||||
@@ -1448,7 +1489,7 @@ version = "0.1.16"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e02f3bb43d335493c96bf3fd3a321600bf6bd07ed34bc64118e9293bdffea46c"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
"libc",
|
||||
"plain",
|
||||
"redox_syscall 0.7.5",
|
||||
@@ -1517,7 +1558,7 @@ checksum = "c41e0c4fef86961ac6d6f8a82609f55f31b05e4fce149ac5710e439df7619ba4"
|
||||
|
||||
[[package]]
|
||||
name = "mangalord"
|
||||
version = "0.89.0"
|
||||
version = "0.90.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"argon2",
|
||||
@@ -1537,8 +1578,10 @@ dependencies = [
|
||||
"infer",
|
||||
"mime",
|
||||
"nix 0.29.0",
|
||||
"ocrs",
|
||||
"rand 0.8.6",
|
||||
"reqwest",
|
||||
"rten",
|
||||
"scraper",
|
||||
"serde",
|
||||
"serde_json",
|
||||
@@ -1669,7 +1712,7 @@ version = "0.29.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "71e2746dc3a24dd78b3cfcb7be93368c6de9963d30f43a6a73998a9cf4b17b46"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
"cfg-if",
|
||||
"cfg_aliases",
|
||||
"libc",
|
||||
@@ -1681,7 +1724,7 @@ version = "0.31.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "cf20d2fde8ff38632c426f1165ed7436270b44f199fc55284c38276f9db47c3d"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
"cfg-if",
|
||||
"cfg_aliases",
|
||||
"libc",
|
||||
@@ -1757,6 +1800,16 @@ dependencies = [
|
||||
"libm",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "num_cpus"
|
||||
version = "1.17.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "91df4bbde75afed763b708b7eee1e8e7651e02d97f6d5dd763e89367e957b23b"
|
||||
dependencies = [
|
||||
"hermit-abi",
|
||||
"libc",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "objc2"
|
||||
version = "0.6.4"
|
||||
@@ -1772,7 +1825,7 @@ version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "73ad74d880bb43877038da939b7427bba67e9dd42004a18b809ba7d87cee241c"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
"objc2",
|
||||
"objc2-foundation",
|
||||
]
|
||||
@@ -1793,7 +1846,7 @@ version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2a180dd8642fa45cdb7dd721cd4c11b1cadd4929ce112ebd8b9f5803cc79d536"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
"dispatch2",
|
||||
"objc2",
|
||||
]
|
||||
@@ -1804,7 +1857,7 @@ version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e022c9d066895efa1345f8e33e584b9f958da2fd4cd116792e15e07e4720a807"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
"dispatch2",
|
||||
"objc2",
|
||||
"objc2-core-foundation",
|
||||
@@ -1837,7 +1890,7 @@ version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "0cde0dfb48d25d2b4862161a4d5fcc0e3c24367869ad306b0c9ec0073bfed92d"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
"objc2",
|
||||
"objc2-core-foundation",
|
||||
"objc2-core-graphics",
|
||||
@@ -1855,7 +1908,7 @@ version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e3e0adef53c21f888deb4fa59fc59f7eb17404926ee8a6f59f5df0fd7f9f3272"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
"block2",
|
||||
"libc",
|
||||
"objc2",
|
||||
@@ -1868,7 +1921,7 @@ version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "180788110936d59bab6bd83b6060ffdfffb3b922ba1396b312ae795e1de9d81d"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
"objc2",
|
||||
"objc2-core-foundation",
|
||||
]
|
||||
@@ -1879,7 +1932,7 @@ version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "96c1358452b371bf9f104e21ec536d37a650eb10f7ee379fff67d2e08d537f1f"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
"objc2",
|
||||
"objc2-core-foundation",
|
||||
"objc2-foundation",
|
||||
@@ -1891,7 +1944,7 @@ version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d87d638e33c06f577498cbcc50491496a3ed4246998a7fbba7ccb98b1e7eab22"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
"block2",
|
||||
"objc2",
|
||||
"objc2-cloud-kit",
|
||||
@@ -1916,6 +1969,21 @@ dependencies = [
|
||||
"objc2-foundation",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ocrs"
|
||||
version = "0.12.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "a5379fdd3f11522b5a2ff53017a189463dabf5d0a9c915cb3eb97fabec4ea11c"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"rayon",
|
||||
"rten",
|
||||
"rten-imageproc",
|
||||
"rten-tensor",
|
||||
"thiserror 2.0.18",
|
||||
"wasm-bindgen",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "once_cell"
|
||||
version = "1.21.4"
|
||||
@@ -2142,7 +2210,7 @@ version = "0.18.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "60769b8b31b2a9f263dae2776c37b1b28ae246943cf719eb6946a1db05128a61"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
"crc32fast",
|
||||
"fdeflate",
|
||||
"flate2",
|
||||
@@ -2361,13 +2429,33 @@ dependencies = [
|
||||
"getrandom 0.3.4",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rayon"
|
||||
version = "1.12.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "fb39b166781f92d482534ef4b4b1b2568f42613b53e5b6c160e24cfbfa30926d"
|
||||
dependencies = [
|
||||
"either",
|
||||
"rayon-core",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rayon-core"
|
||||
version = "1.13.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "22e18b0f0062d30d4230b2e85ff77fdfe4326feb054b9783a3460d8435c8ab91"
|
||||
dependencies = [
|
||||
"crossbeam-deque",
|
||||
"crossbeam-utils",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "redox_syscall"
|
||||
version = "0.5.18"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -2376,7 +2464,7 @@ version = "0.7.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "4666a1a60d8412eab19d94f6d13dcc9cea0a5ef4fdf6a5db306537413c661b1b"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -2496,19 +2584,135 @@ dependencies = [
|
||||
"zeroize",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rten"
|
||||
version = "0.24.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "43c230fa4ade87c913f61dbd911b7eb0d49460ceff3f1e4fabc837fac191137c"
|
||||
dependencies = [
|
||||
"flatbuffers",
|
||||
"num_cpus",
|
||||
"rayon",
|
||||
"rten-base",
|
||||
"rten-gemm",
|
||||
"rten-model-file",
|
||||
"rten-onnx",
|
||||
"rten-shape-inference",
|
||||
"rten-simd",
|
||||
"rten-tensor",
|
||||
"rten-vecmath",
|
||||
"rustc-hash",
|
||||
"smallvec",
|
||||
"typeid",
|
||||
"wasm-bindgen",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rten-base"
|
||||
version = "0.24.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2738cf8bb4c27f828ac788d01ccf4e367e8e773cfec6851f81851b5211de6a79"
|
||||
dependencies = [
|
||||
"rayon",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rten-gemm"
|
||||
version = "0.24.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "330a81a0ca209fb5ce21bd17efa0bd287d5881c6cebfbff0b21c4294a1a14a9e"
|
||||
dependencies = [
|
||||
"rayon",
|
||||
"rten-base",
|
||||
"rten-simd",
|
||||
"rten-tensor",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rten-imageproc"
|
||||
version = "0.24.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d5f148e7e941fb5727b9046a5fa1b45525543d5105f14b384fd9261df0ee49bc"
|
||||
dependencies = [
|
||||
"rten-tensor",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rten-model-file"
|
||||
version = "0.24.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ed2f8d270f07ab1bbfff47250c6039f6caa5da59d6da7d74f66aa48559aa6fea"
|
||||
dependencies = [
|
||||
"flatbuffers",
|
||||
"rten-base",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rten-onnx"
|
||||
version = "0.24.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "23086eef75bfb55278cb0b45cf9f5a877d466d914914aafebee4ffca9b24d20c"
|
||||
|
||||
[[package]]
|
||||
name = "rten-shape-inference"
|
||||
version = "0.24.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8e8a913c7ca40e2bfbb2a0cd447cce56b33ab19435f56693271a2ef37cf58984"
|
||||
dependencies = [
|
||||
"rten-tensor",
|
||||
"smallvec",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rten-simd"
|
||||
version = "0.24.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b19a0032dfcb70dd20960c1c51a37674b237586cbc1ce586f45b46605d108e82"
|
||||
|
||||
[[package]]
|
||||
name = "rten-tensor"
|
||||
version = "0.24.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "05dc744a270aa32d154f1a3df8e48740ccc1be9dfbcf23295ada66d83aa98de6"
|
||||
dependencies = [
|
||||
"rayon",
|
||||
"rten-base",
|
||||
"smallvec",
|
||||
"typeid",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rten-vecmath"
|
||||
version = "0.24.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "9574ddebf5671bc08ceb76e2e1638fadc57fdeff318634eab2c29e9a803cff64"
|
||||
dependencies = [
|
||||
"rten-base",
|
||||
"rten-simd",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rustc-hash"
|
||||
version = "2.1.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "94300abf3f1ae2e2b8ffb7b58043de3d399c73fa6f4b73826402a5c457614dbe"
|
||||
|
||||
[[package]]
|
||||
name = "rustc_version"
|
||||
version = "0.4.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "cfcb3a22ef46e85b45de6ee7e79d063319ebb6594faafcf1c225ea92ab6e9b92"
|
||||
dependencies = [
|
||||
"semver",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rustix"
|
||||
version = "0.38.44"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "fdb5bc1ae2baa591800df16c9ca78619bf65c0488b41b96ccec5d11220d8c154"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
"errno",
|
||||
"libc",
|
||||
"linux-raw-sys 0.4.15",
|
||||
@@ -2521,7 +2725,7 @@ version = "1.1.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
"errno",
|
||||
"libc",
|
||||
"linux-raw-sys 0.12.1",
|
||||
@@ -2603,7 +2807,7 @@ version = "0.25.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "4eb30575f3638fc8f6815f448d50cb1a2e255b0897985c8c59f4d37b72a07b06"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
"cssparser",
|
||||
"derive_more",
|
||||
"fxhash",
|
||||
@@ -2911,7 +3115,7 @@ checksum = "aa003f0038df784eb8fecbbac13affe3da23b45194bd57dba231c8f48199c526"
|
||||
dependencies = [
|
||||
"atoi",
|
||||
"base64",
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
"byteorder",
|
||||
"bytes",
|
||||
"chrono",
|
||||
@@ -2955,7 +3159,7 @@ checksum = "db58fcd5a53cf07c184b154801ff91347e4c30d17a3562a635ff028ad5deda46"
|
||||
dependencies = [
|
||||
"atoi",
|
||||
"base64",
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
"byteorder",
|
||||
"chrono",
|
||||
"crc",
|
||||
@@ -3317,7 +3521,7 @@ version = "0.6.10"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "68d6fdd9f81c2819c9a8b0e0cd91660e7746a8e6ea2ba7c6b2b057985f6bcb51"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
"bytes",
|
||||
"futures-util",
|
||||
"http",
|
||||
@@ -3428,6 +3632,12 @@ dependencies = [
|
||||
"utf-8",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "typeid"
|
||||
version = "1.0.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "bc7d623258602320d5c55d1bc22793b57daff0ec7efc270ea7d55ce1d5f5471c"
|
||||
|
||||
[[package]]
|
||||
name = "typenum"
|
||||
version = "1.20.0"
|
||||
@@ -3668,7 +3878,7 @@ version = "0.244.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "47b807c72e1bac69382b3a6fb3dbe8ea4c0ed87ff5629b8685ae6b9a611028fe"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
"hashbrown 0.15.5",
|
||||
"indexmap",
|
||||
"semver",
|
||||
@@ -4096,7 +4306,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "9d66ea20e9553b30172b5e831994e35fbde2d165325bec84fc43dbf6f4eb9cb2"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"bitflags",
|
||||
"bitflags 2.11.1",
|
||||
"indexmap",
|
||||
"log",
|
||||
"serde",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "mangalord"
|
||||
version = "0.89.0"
|
||||
version = "0.90.0"
|
||||
edition = "2021"
|
||||
default-run = "mangalord"
|
||||
|
||||
@@ -52,6 +52,8 @@ sysinfo = { version = "0.32", default-features = false, features = ["system", "c
|
||||
nix = { version = "0.29", features = ["fs"] }
|
||||
scraper = "0.20"
|
||||
reqwest = { version = "0.12", default-features = false, features = ["rustls-tls", "socks", "cookies", "stream", "json"] }
|
||||
ocrs = "0.12"
|
||||
rten = "0.24"
|
||||
|
||||
[dev-dependencies]
|
||||
tempfile = "3"
|
||||
|
||||
@@ -58,6 +58,20 @@ WORKDIR /app
|
||||
COPY --from=builder /app/target/release/mangalord /usr/local/bin/mangalord
|
||||
COPY --from=builder /app/migrations /app/migrations
|
||||
|
||||
# OCR models for the default `ANALYSIS_BACKEND=ocr` (ocrs) engine. The two
|
||||
# `.rten` files are pulled at build time into /models, where the runtime's
|
||||
# `OCRS_DETECTION_MODEL` / `OCRS_RECOGNITION_MODEL` defaults point. They're a
|
||||
# few MB each and pure data (no native code), so they bake cleanly into the
|
||||
# image and need no manual setup on the Pi. Set INSTALL_OCR_MODELS=false to
|
||||
# skip (e.g. for a vision-only deploy that never runs ocrs).
|
||||
ARG INSTALL_OCR_MODELS=true
|
||||
ARG OCRS_MODEL_BASE_URL=https://ocrs-models.s3-accelerate.amazonaws.com
|
||||
RUN if [ "$INSTALL_OCR_MODELS" = "true" ]; then \
|
||||
mkdir -p /models \
|
||||
&& curl -fsSL "${OCRS_MODEL_BASE_URL}/text-detection.rten" -o /models/text-detection.rten \
|
||||
&& curl -fsSL "${OCRS_MODEL_BASE_URL}/text-recognition.rten" -o /models/text-recognition.rten; \
|
||||
fi
|
||||
|
||||
ENV STORAGE_DIR=/var/lib/mangalord/storage
|
||||
# Pre-create the storage dir so the entrypoint doesn't need to
|
||||
# mkdir-as-root and so the named volume mount inherits the right
|
||||
|
||||
@@ -9,5 +9,6 @@
|
||||
|
||||
pub mod daemon;
|
||||
pub mod events;
|
||||
pub mod ocr;
|
||||
pub mod prompt;
|
||||
pub mod vision;
|
||||
|
||||
201
backend/src/analysis/ocr.rs
Normal file
201
backend/src/analysis/ocr.rs
Normal file
@@ -0,0 +1,201 @@
|
||||
//! The in-process OCR analysis backend (the `ocrs` engine).
|
||||
//!
|
||||
//! A lightweight alternative to [`crate::analysis::vision`]: instead of a slow
|
||||
//! local LLM, each page is run through `ocrs` — a pure-Rust detect→recognize
|
||||
//! OCR pipeline on the `rten` runtime. It extracts **text only** (no tags,
|
||||
//! scene description or NSFW flags — those stay the vision backend's job), then
|
||||
//! reuses [`repo::page_analysis::persist_analysis`] so the OCR lines land in
|
||||
//! `page_ocr_text` and the weighted `search_doc` tsvector exactly as the vision
|
||||
//! path produces them. That makes the existing text-search surfaces
|
||||
//! (`/v1/me/page-search` and the tag aggregations) work with no further wiring.
|
||||
//!
|
||||
//! The engine is split behind the [`OcrEngine`] trait so the dispatcher is
|
||||
//! unit-testable without shipping the (multi-MB) `.rten` model files: tests use
|
||||
//! [`test_support::StubOcrEngine`], production uses [`OcrsEngine`].
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use async_trait::async_trait;
|
||||
use sqlx::PgPool;
|
||||
use uuid::Uuid;
|
||||
|
||||
use crate::analysis::daemon::AnalyzeDispatcher;
|
||||
use crate::domain::page_analysis::{OcrResult, SafetyFlag, VisionAnalysis};
|
||||
use crate::repo;
|
||||
use crate::storage::Storage;
|
||||
|
||||
/// The `model` label stamped onto `page_analysis` rows written by this backend.
|
||||
pub const OCR_MODEL_LABEL: &str = "ocrs";
|
||||
|
||||
/// Extracts text lines from a decoded-or-encoded page image. The production
|
||||
/// impl ([`OcrsEngine`]) decodes the bytes itself; the trait takes the raw
|
||||
/// stored image bytes so the dispatcher stays engine-agnostic.
|
||||
pub trait OcrEngine: Send + Sync {
|
||||
/// Run OCR over one page image (the bytes as stored, e.g. PNG/JPEG/WebP).
|
||||
/// Returns the recognized text lines in reading order (top→bottom).
|
||||
fn recognize(&self, image: &[u8]) -> anyhow::Result<Vec<String>>;
|
||||
}
|
||||
|
||||
/// Turn the OCR engine's ordered text lines into the [`VisionAnalysis`] shape
|
||||
/// that [`repo::page_analysis::persist_analysis`] consumes. OCR-only: the tag,
|
||||
/// scene and safety fields are left empty/default. The line `kind` is left
|
||||
/// blank — `persist_analysis` maps an empty kind to the neutral mid-weight
|
||||
/// `OcrKind::Narration` bucket (classifying speech/sfx/… is the deferred
|
||||
/// vision backend's job).
|
||||
pub fn lines_to_analysis(lines: Vec<String>) -> VisionAnalysis {
|
||||
let ocr_results = lines
|
||||
.into_iter()
|
||||
.map(|text| OcrResult { text, kind: String::new(), y: None })
|
||||
.collect();
|
||||
VisionAnalysis {
|
||||
ocr_results,
|
||||
tagging_results: Vec::new(),
|
||||
scene_description: String::new(),
|
||||
safety_flag: SafetyFlag::default(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Production OCR engine: an `ocrs` detect+recognize pipeline with the two
|
||||
/// `.rten` models loaded once at startup. Cheap to share across workers — the
|
||||
/// recognize path borrows `&self`.
|
||||
pub struct OcrsEngine {
|
||||
engine: ocrs::OcrEngine,
|
||||
}
|
||||
|
||||
impl OcrsEngine {
|
||||
/// Load the detection + recognition models from disk and build the engine.
|
||||
/// Fails (at startup) if either model file is missing or unreadable, so a
|
||||
/// misconfigured path is a loud boot error rather than a per-page failure.
|
||||
pub fn from_model_paths(detection: &str, recognition: &str) -> anyhow::Result<Self> {
|
||||
use anyhow::Context;
|
||||
let detection_model = rten::Model::load_file(detection)
|
||||
.with_context(|| format!("load ocrs detection model {detection}"))?;
|
||||
let recognition_model = rten::Model::load_file(recognition)
|
||||
.with_context(|| format!("load ocrs recognition model {recognition}"))?;
|
||||
let engine = ocrs::OcrEngine::new(ocrs::OcrEngineParams {
|
||||
detection_model: Some(detection_model),
|
||||
recognition_model: Some(recognition_model),
|
||||
..Default::default()
|
||||
})
|
||||
.context("construct ocrs engine")?;
|
||||
Ok(Self { engine })
|
||||
}
|
||||
}
|
||||
|
||||
impl OcrEngine for OcrsEngine {
|
||||
fn recognize(&self, image: &[u8]) -> anyhow::Result<Vec<String>> {
|
||||
use anyhow::Context;
|
||||
// Decode to RGB8 so `ImageSource` gets a known channel layout.
|
||||
let rgb = image::load_from_memory(image)
|
||||
.context("decode page image for OCR")?
|
||||
.into_rgb8();
|
||||
let source = ocrs::ImageSource::from_bytes(rgb.as_raw(), rgb.dimensions())
|
||||
.map_err(|e| anyhow::anyhow!("build OCR image source: {e}"))?;
|
||||
let input = self.engine.prepare_input(source)?;
|
||||
// detect words → group into lines → recognize each line. Mirrors
|
||||
// `OcrEngine::get_text`, but keeps the lines as a Vec instead of
|
||||
// joining them, so each becomes its own `page_ocr_text` row.
|
||||
let words = self.engine.detect_words(&input)?;
|
||||
let line_rects = self.engine.find_text_lines(&input, &words);
|
||||
let lines = self
|
||||
.engine
|
||||
.recognize_text(&input, &line_rects)?
|
||||
.into_iter()
|
||||
.filter_map(|line| line.map(|l| l.to_string()))
|
||||
.map(|s| s.trim().to_string())
|
||||
.filter(|s| !s.is_empty())
|
||||
.collect();
|
||||
Ok(lines)
|
||||
}
|
||||
}
|
||||
|
||||
/// Production dispatcher for the OCR backend: load the page, read its image
|
||||
/// from storage, run OCR on the blocking pool, and persist the lines. Mirrors
|
||||
/// [`crate::analysis::daemon::RealAnalyzeDispatcher`] but with no network I/O.
|
||||
pub struct OcrAnalyzeDispatcher {
|
||||
pub db: PgPool,
|
||||
pub storage: Arc<dyn Storage>,
|
||||
pub engine: Arc<dyn OcrEngine>,
|
||||
pub max_image_bytes: usize,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl AnalyzeDispatcher for OcrAnalyzeDispatcher {
|
||||
async fn dispatch(&self, page_id: Uuid) -> anyhow::Result<()> {
|
||||
let Some(page) = repo::page::find_by_id(&self.db, page_id).await? else {
|
||||
// Page was deleted between enqueue and dispatch — nothing to do.
|
||||
return Ok(());
|
||||
};
|
||||
let bytes = self
|
||||
.storage
|
||||
.get(&page.storage_key)
|
||||
.await
|
||||
.map_err(|e| anyhow::anyhow!("read page image {}: {e}", page.storage_key))?;
|
||||
if bytes.len() > self.max_image_bytes {
|
||||
anyhow::bail!(
|
||||
"page image {} is {} bytes, over the {} cap",
|
||||
page.storage_key,
|
||||
bytes.len(),
|
||||
self.max_image_bytes
|
||||
);
|
||||
}
|
||||
// OCR inference is CPU-bound and synchronous — keep it off the async
|
||||
// worker's runtime thread.
|
||||
let engine = Arc::clone(&self.engine);
|
||||
let lines = tokio::task::spawn_blocking(move || engine.recognize(&bytes))
|
||||
.await
|
||||
.map_err(|e| anyhow::anyhow!("OCR task join error: {e}"))??;
|
||||
let analysis = lines_to_analysis(lines);
|
||||
repo::page_analysis::persist_analysis(&self.db, page_id, &analysis, OCR_MODEL_LABEL).await?;
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// Stubs for the OCR dispatcher's integration tests. Public because the tests
|
||||
/// live in the `tests/` dir (a separate crate).
|
||||
pub mod test_support {
|
||||
use super::*;
|
||||
|
||||
/// An [`OcrEngine`] that returns a fixed set of lines regardless of input,
|
||||
/// so the dispatcher's storage→persist path can be tested without models.
|
||||
pub struct StubOcrEngine {
|
||||
pub lines: Vec<String>,
|
||||
}
|
||||
|
||||
impl StubOcrEngine {
|
||||
pub fn new(lines: &[&str]) -> Arc<Self> {
|
||||
Arc::new(Self { lines: lines.iter().map(|s| s.to_string()).collect() })
|
||||
}
|
||||
}
|
||||
|
||||
impl OcrEngine for StubOcrEngine {
|
||||
fn recognize(&self, _image: &[u8]) -> anyhow::Result<Vec<String>> {
|
||||
Ok(self.lines.clone())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn lines_to_analysis_maps_lines_in_order_and_leaves_rest_empty() {
|
||||
let v = lines_to_analysis(vec!["Hello".to_string(), "world!".to_string()]);
|
||||
assert_eq!(v.ocr_results.len(), 2);
|
||||
assert_eq!(v.ocr_results[0].text, "Hello");
|
||||
assert_eq!(v.ocr_results[1].text, "world!");
|
||||
// OCR-only: kind blank (→ Narration at persist), no tags/scene/safety.
|
||||
assert!(v.ocr_results.iter().all(|r| r.kind.is_empty()));
|
||||
assert!(v.tagging_results.is_empty());
|
||||
assert_eq!(v.scene_description, "");
|
||||
assert!(!v.safety_flag.is_nsfw);
|
||||
assert!(v.safety_flag.content_type.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn lines_to_analysis_handles_no_text() {
|
||||
let v = lines_to_analysis(Vec::new());
|
||||
assert!(v.ocr_results.is_empty());
|
||||
}
|
||||
}
|
||||
@@ -392,10 +392,9 @@ pub struct AggregateParams {
|
||||
pub limit: i64,
|
||||
#[serde(default)]
|
||||
pub offset: i64,
|
||||
/// Reserved for the planned OCR text-search input. Accepted on
|
||||
/// the wire so adding OCR later won't break the API shape, but
|
||||
/// rejected with 501 `text_search_not_yet_supported` if non-empty
|
||||
/// until the backend supports it.
|
||||
/// OCR text filter. When non-empty, only pages whose analysis
|
||||
/// `search_doc` matches the query (`plainto_tsquery`) are aggregated.
|
||||
/// Blank/absent ⇒ tag-only aggregation.
|
||||
#[serde(default)]
|
||||
pub text: Option<String>,
|
||||
}
|
||||
@@ -411,32 +410,17 @@ fn parse_order(raw: Option<&str>) -> AppResult<Order> {
|
||||
}
|
||||
}
|
||||
|
||||
fn ensure_text_unsupported(text: Option<&str>) -> AppResult<()> {
|
||||
// Future OCR search will plug in here. Until then, return a
|
||||
// distinct code (`text_search_not_yet_supported`) so clients can
|
||||
// detect "feature pending" vs. a generic 4xx — the code is the
|
||||
// wire contract, not the message.
|
||||
if text.is_some_and(|s| !s.trim().is_empty()) {
|
||||
return Err(AppError::NotImplemented {
|
||||
code: "text_search_not_yet_supported",
|
||||
message: "text search is reserved for the planned OCR input but not yet supported",
|
||||
});
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn list_chapters_for_tag(
|
||||
State(state): State<AppState>,
|
||||
CurrentUser(user): CurrentUser,
|
||||
Query(params): Query<AggregateParams>,
|
||||
) -> AppResult<Json<PagedResponse<TaggedChapterAggregate>>> {
|
||||
ensure_text_unsupported(params.text.as_deref())?;
|
||||
let tag = normalize_tag(¶ms.tag)?;
|
||||
let order = parse_order(params.order.as_deref())?;
|
||||
let limit = params.limit.clamp(1, 200);
|
||||
let offset = params.offset.max(0);
|
||||
let (items, total) = repo::page_tag::aggregate_chapters_for_tag(
|
||||
&state.db, user.id, &tag, order, limit, offset,
|
||||
&state.db, user.id, &tag, order, limit, offset, params.text.as_deref(),
|
||||
)
|
||||
.await?;
|
||||
Ok(Json(PagedResponse::with_total(items, limit, offset, total)))
|
||||
@@ -447,13 +431,12 @@ async fn list_mangas_for_tag(
|
||||
CurrentUser(user): CurrentUser,
|
||||
Query(params): Query<AggregateParams>,
|
||||
) -> AppResult<Json<PagedResponse<TaggedMangaAggregate>>> {
|
||||
ensure_text_unsupported(params.text.as_deref())?;
|
||||
let tag = normalize_tag(¶ms.tag)?;
|
||||
let order = parse_order(params.order.as_deref())?;
|
||||
let limit = params.limit.clamp(1, 200);
|
||||
let offset = params.offset.max(0);
|
||||
let (items, total) = repo::page_tag::aggregate_mangas_for_tag(
|
||||
&state.db, user.id, &tag, order, limit, offset,
|
||||
&state.db, user.id, &tag, order, limit, offset, params.text.as_deref(),
|
||||
)
|
||||
.await?;
|
||||
Ok(Json(PagedResponse::with_total(items, limit, offset, total)))
|
||||
|
||||
@@ -429,46 +429,74 @@ async fn spawn_analysis_daemon(
|
||||
),
|
||||
Err(e) => tracing::warn!(?e, "analysis: reclaim_orphaned at startup failed"),
|
||||
}
|
||||
let http = reqwest::Client::builder()
|
||||
.timeout(cfg.request_timeout)
|
||||
// Refuse to honour ambient HTTP_PROXY / HTTPS_PROXY container env.
|
||||
// The vision call carries an env-managed bearer token + page image
|
||||
// bytes; a stray upstream proxy would exfiltrate both. Mirrors the
|
||||
// crawler client's `.no_proxy()` (see `spawn_crawler_daemon`).
|
||||
.no_proxy()
|
||||
.build()
|
||||
.context("build analysis http client")?;
|
||||
let vision = crate::analysis::vision::VisionClient::new(http, cfg);
|
||||
let dispatcher = Arc::new(crate::analysis::daemon::RealAnalyzeDispatcher {
|
||||
db: db.clone(),
|
||||
storage,
|
||||
vision,
|
||||
model: cfg.model.clone(),
|
||||
max_image_bytes: cfg.max_image_bytes,
|
||||
});
|
||||
// When a readiness URL is configured, gate leasing on it so an
|
||||
// autoscaler that idle-stops the vision container never lets a job burn
|
||||
// its retries. A dedicated short-timeout client keeps the probe snappy
|
||||
// and independent of the (long) per-request analysis timeout.
|
||||
let readiness: Option<Arc<dyn crate::analysis::daemon::VisionReadiness>> =
|
||||
match &cfg.vision_health_url {
|
||||
Some(url) if !url.is_empty() => {
|
||||
let probe = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(5))
|
||||
// Same reasoning as the main analysis client: do not
|
||||
// honour ambient HTTP_PROXY env. The readiness probe is
|
||||
// unauthenticated but a hostile upstream still gets a
|
||||
// useful side-channel on backend uptime + vision health.
|
||||
.no_proxy()
|
||||
.build()
|
||||
.context("build vision readiness http client")?;
|
||||
Some(Arc::new(crate::analysis::daemon::HttpVisionReadiness {
|
||||
http: probe,
|
||||
health_url: url.clone(),
|
||||
}))
|
||||
}
|
||||
_ => None,
|
||||
};
|
||||
// Pick the engine. The OCR backend runs in-process (no network, no
|
||||
// readiness gate); the vision backend talks to a local LLM server.
|
||||
let (dispatcher, readiness): (
|
||||
Arc<dyn crate::analysis::daemon::AnalyzeDispatcher>,
|
||||
Option<Arc<dyn crate::analysis::daemon::VisionReadiness>>,
|
||||
) = match cfg.backend {
|
||||
crate::config::AnalysisBackend::Ocr => {
|
||||
// Load the `.rten` models once; a bad path is a loud boot error.
|
||||
let engine = crate::analysis::ocr::OcrsEngine::from_model_paths(
|
||||
&cfg.ocr_detection_model,
|
||||
&cfg.ocr_recognition_model,
|
||||
)
|
||||
.context("build ocrs engine")?;
|
||||
let dispatcher = Arc::new(crate::analysis::ocr::OcrAnalyzeDispatcher {
|
||||
db: db.clone(),
|
||||
storage,
|
||||
engine: Arc::new(engine),
|
||||
max_image_bytes: cfg.max_image_bytes,
|
||||
});
|
||||
// In-process engine is always ready — no gate.
|
||||
(dispatcher, None)
|
||||
}
|
||||
crate::config::AnalysisBackend::Vision => {
|
||||
let http = reqwest::Client::builder()
|
||||
.timeout(cfg.request_timeout)
|
||||
// Refuse to honour ambient HTTP_PROXY / HTTPS_PROXY container
|
||||
// env. The vision call carries an env-managed bearer token +
|
||||
// page image bytes; a stray upstream proxy would exfiltrate
|
||||
// both. Mirrors the crawler client's `.no_proxy()` (see
|
||||
// `spawn_crawler_daemon`).
|
||||
.no_proxy()
|
||||
.build()
|
||||
.context("build analysis http client")?;
|
||||
let vision = crate::analysis::vision::VisionClient::new(http, cfg);
|
||||
let dispatcher = Arc::new(crate::analysis::daemon::RealAnalyzeDispatcher {
|
||||
db: db.clone(),
|
||||
storage,
|
||||
vision,
|
||||
model: cfg.model.clone(),
|
||||
max_image_bytes: cfg.max_image_bytes,
|
||||
});
|
||||
// When a readiness URL is configured, gate leasing on it so an
|
||||
// autoscaler that idle-stops the vision container never lets a job
|
||||
// burn its retries. A dedicated short-timeout client keeps the
|
||||
// probe snappy and independent of the (long) per-request timeout.
|
||||
let readiness: Option<Arc<dyn crate::analysis::daemon::VisionReadiness>> =
|
||||
match &cfg.vision_health_url {
|
||||
Some(url) if !url.is_empty() => {
|
||||
let probe = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(5))
|
||||
// Same reasoning as the main analysis client: do
|
||||
// not honour ambient HTTP_PROXY env. The readiness
|
||||
// probe is unauthenticated but a hostile upstream
|
||||
// still gets a useful side-channel on backend
|
||||
// uptime + vision health.
|
||||
.no_proxy()
|
||||
.build()
|
||||
.context("build vision readiness http client")?;
|
||||
Some(Arc::new(crate::analysis::daemon::HttpVisionReadiness {
|
||||
http: probe,
|
||||
health_url: url.clone(),
|
||||
}))
|
||||
}
|
||||
_ => None,
|
||||
};
|
||||
(dispatcher, readiness)
|
||||
}
|
||||
};
|
||||
let handle = crate::analysis::daemon::spawn(
|
||||
db,
|
||||
CancellationToken::new(),
|
||||
@@ -480,7 +508,12 @@ async fn spawn_analysis_daemon(
|
||||
readiness,
|
||||
},
|
||||
);
|
||||
tracing::info!(workers = cfg.workers, model = %cfg.model, "analysis worker daemon started");
|
||||
tracing::info!(
|
||||
workers = cfg.workers,
|
||||
backend = ?cfg.backend,
|
||||
model = %cfg.model,
|
||||
"analysis worker daemon started"
|
||||
);
|
||||
Ok(handle)
|
||||
}
|
||||
|
||||
@@ -1447,6 +1480,10 @@ mod tests {
|
||||
let storage: Arc<dyn Storage> =
|
||||
Arc::new(LocalStorage::new(storage_dir.path()));
|
||||
let mut cfg = crate::config::AnalysisConfig::default();
|
||||
// Use the vision backend so the daemon doesn't try to load the ocrs
|
||||
// `.rten` models (absent in unit CI). This test only exercises the
|
||||
// pre-worker lease reclaim, which is engine-agnostic; no dispatch runs.
|
||||
cfg.backend = crate::config::AnalysisBackend::Vision;
|
||||
cfg.workers = 1;
|
||||
cfg.job_timeout = Duration::from_secs(1);
|
||||
let events = Arc::new(crate::analysis::events::AnalysisEvents::new());
|
||||
|
||||
@@ -113,6 +113,32 @@ impl ResponseFormat {
|
||||
}
|
||||
}
|
||||
|
||||
/// Which engine the analysis worker dispatches each page through. A
|
||||
/// deploy-time choice (the engine is either installed or not), so it lives in
|
||||
/// env only and is not part of the admin-editable `AnalysisSettings`.
|
||||
///
|
||||
/// * `Ocr` — in-process [`crate::analysis::ocr`] (the `ocrs` engine): fast,
|
||||
/// CPU-only, English text. Writes OCR text only (no tags/scene/safety).
|
||||
/// * `Vision` — the local OpenAI-compatible LLM in [`crate::analysis::vision`]:
|
||||
/// full OCR + tags + scene + safety, but heavy.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum AnalysisBackend {
|
||||
Ocr,
|
||||
Vision,
|
||||
}
|
||||
|
||||
impl AnalysisBackend {
|
||||
/// Lenient env parse. Defaults to `Ocr` (the Pi-friendly path) for any
|
||||
/// unset or unrecognized value; `vision` opts back into the LLM engine.
|
||||
fn from_str(s: &str) -> AnalysisBackend {
|
||||
match s.trim().to_lowercase().as_str() {
|
||||
"vision" | "llm" => AnalysisBackend::Vision,
|
||||
// Default (incl. "ocr", "ocrs", and anything unrecognized).
|
||||
_ => AnalysisBackend::Ocr,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// AI content-analysis worker configuration: the enable gate, the local
|
||||
/// OpenAI-compatible vision endpoint, and the worker / request knobs.
|
||||
#[derive(Clone, Debug)]
|
||||
@@ -120,6 +146,16 @@ pub struct AnalysisConfig {
|
||||
/// Master switch (`ANALYSIS_ENABLED`). When `false`, no analysis jobs
|
||||
/// are enqueued and no worker runs. Defaults to `false`.
|
||||
pub enabled: bool,
|
||||
/// Which engine the worker dispatches through (`ANALYSIS_BACKEND`):
|
||||
/// `ocr` (default, the in-process `ocrs` engine) or `vision` (the local
|
||||
/// LLM). Deploy-time, env-only — see [`AnalysisBackend`].
|
||||
pub backend: AnalysisBackend,
|
||||
/// Path to the ocrs text-*detection* `.rten` model
|
||||
/// (`OCRS_DETECTION_MODEL`). Only read when `backend == Ocr`.
|
||||
pub ocr_detection_model: String,
|
||||
/// Path to the ocrs text-*recognition* `.rten` model
|
||||
/// (`OCRS_RECOGNITION_MODEL`). Only read when `backend == Ocr`.
|
||||
pub ocr_recognition_model: String,
|
||||
/// Number of concurrent analysis workers (`ANALYSIS_WORKERS`).
|
||||
pub workers: usize,
|
||||
/// OpenAI-compatible chat/completions URL (`ANALYSIS_VISION_URL`).
|
||||
@@ -195,6 +231,9 @@ impl Default for AnalysisConfig {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
enabled: false,
|
||||
backend: AnalysisBackend::Ocr,
|
||||
ocr_detection_model: "/models/text-detection.rten".to_string(),
|
||||
ocr_recognition_model: "/models/text-recognition.rten".to_string(),
|
||||
workers: 1,
|
||||
endpoint: "http://localhost:8000/v1/chat/completions".to_string(),
|
||||
vision_health_url: None,
|
||||
@@ -227,6 +266,17 @@ impl AnalysisConfig {
|
||||
let d = AnalysisConfig::default();
|
||||
Self {
|
||||
enabled: env_bool("ANALYSIS_ENABLED", d.enabled),
|
||||
backend: std::env::var("ANALYSIS_BACKEND")
|
||||
.map(|s| AnalysisBackend::from_str(&s))
|
||||
.unwrap_or(d.backend),
|
||||
ocr_detection_model: std::env::var("OCRS_DETECTION_MODEL")
|
||||
.ok()
|
||||
.filter(|s| !s.is_empty())
|
||||
.unwrap_or(d.ocr_detection_model),
|
||||
ocr_recognition_model: std::env::var("OCRS_RECOGNITION_MODEL")
|
||||
.ok()
|
||||
.filter(|s| !s.is_empty())
|
||||
.unwrap_or(d.ocr_recognition_model),
|
||||
workers: env_usize("ANALYSIS_WORKERS", d.workers).max(1),
|
||||
endpoint: std::env::var("ANALYSIS_VISION_URL").unwrap_or(d.endpoint),
|
||||
vision_health_url: std::env::var("ANALYSIS_VISION_HEALTH_URL")
|
||||
@@ -734,11 +784,18 @@ mod tests {
|
||||
"ANALYSIS_MAX_SLICES",
|
||||
"ANALYSIS_RESPONSE_FORMAT",
|
||||
"ANALYSIS_FREQUENCY_PENALTY",
|
||||
"ANALYSIS_BACKEND",
|
||||
"OCRS_DETECTION_MODEL",
|
||||
"OCRS_RECOGNITION_MODEL",
|
||||
] {
|
||||
std::env::remove_var(k);
|
||||
}
|
||||
let cfg = AnalysisConfig::from_env();
|
||||
assert!(!cfg.enabled);
|
||||
// OCR is the default engine (the Pi-friendly path).
|
||||
assert_eq!(cfg.backend, AnalysisBackend::Ocr);
|
||||
assert_eq!(cfg.ocr_detection_model, "/models/text-detection.rten");
|
||||
assert_eq!(cfg.ocr_recognition_model, "/models/text-recognition.rten");
|
||||
assert_eq!(cfg.workers, 1);
|
||||
assert_eq!(cfg.max_pixels, 1_000_000);
|
||||
assert_eq!(cfg.min_slice_height, 640);
|
||||
@@ -765,6 +822,36 @@ mod tests {
|
||||
std::env::remove_var("ANALYSIS_RESPONSE_FORMAT");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn analysis_backend_parses_and_defaults_to_ocr() {
|
||||
let _g = ENV_GUARD.lock().unwrap_or_else(|p| p.into_inner());
|
||||
for (raw, want) in [
|
||||
("ocr", AnalysisBackend::Ocr),
|
||||
("ocrs", AnalysisBackend::Ocr),
|
||||
("vision", AnalysisBackend::Vision),
|
||||
("llm", AnalysisBackend::Vision),
|
||||
("anything-else", AnalysisBackend::Ocr),
|
||||
] {
|
||||
std::env::set_var("ANALYSIS_BACKEND", raw);
|
||||
assert_eq!(AnalysisConfig::from_env().backend, want, "raw={raw}");
|
||||
}
|
||||
// Unset → OCR.
|
||||
std::env::remove_var("ANALYSIS_BACKEND");
|
||||
assert_eq!(AnalysisConfig::from_env().backend, AnalysisBackend::Ocr);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ocr_model_paths_parse_from_env() {
|
||||
let _g = ENV_GUARD.lock().unwrap_or_else(|p| p.into_inner());
|
||||
std::env::set_var("OCRS_DETECTION_MODEL", "/opt/det.rten");
|
||||
std::env::set_var("OCRS_RECOGNITION_MODEL", "/opt/rec.rten");
|
||||
let cfg = AnalysisConfig::from_env();
|
||||
std::env::remove_var("OCRS_DETECTION_MODEL");
|
||||
std::env::remove_var("OCRS_RECOGNITION_MODEL");
|
||||
assert_eq!(cfg.ocr_detection_model, "/opt/det.rten");
|
||||
assert_eq!(cfg.ocr_recognition_model, "/opt/rec.rten");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn analysis_config_parses_from_env() {
|
||||
let _g = ENV_GUARD.lock().unwrap_or_else(|p| p.into_inner());
|
||||
|
||||
@@ -39,10 +39,10 @@ pub enum AppError {
|
||||
details: serde_json::Value,
|
||||
},
|
||||
/// 501 — the wire shape is accepted but the feature isn't built yet.
|
||||
/// Carries a `&'static str` snake_case code so clients can detect
|
||||
/// the specific pending feature (`text_search_not_yet_supported`,
|
||||
/// etc.) without parsing the message. Used today by the `?text=`
|
||||
/// reservation on the page-tag aggregation endpoints.
|
||||
/// Carries a `&'static str` snake_case code so clients can detect the
|
||||
/// specific pending feature without parsing the message. Generic; kept
|
||||
/// for future reservations (the `?text=` page-tag reservation that first
|
||||
/// used it now performs real OCR search).
|
||||
#[error("not implemented: {code}")]
|
||||
NotImplemented {
|
||||
code: &'static str,
|
||||
|
||||
@@ -247,6 +247,11 @@ pub async fn distinct_tags_for_user(
|
||||
/// storage keys (page-number ascending) so the row can render a
|
||||
/// thumbnail strip without a follow-up fetch.
|
||||
///
|
||||
/// When `text` is non-blank, results are further restricted to pages whose
|
||||
/// analysis `search_doc` matches the query (OCR text search), and both
|
||||
/// `match_count` and the sample thumbnails reflect that filtered set —
|
||||
/// i.e. `match_count` counts tagged pages whose OCR also matches `text`.
|
||||
///
|
||||
/// `order` is inlined via `format!()` — the enum value space is
|
||||
/// closed (`ASC` / `DESC`) so this is not a SQL-injection vector.
|
||||
pub async fn aggregate_chapters_for_tag(
|
||||
@@ -256,7 +261,30 @@ pub async fn aggregate_chapters_for_tag(
|
||||
order: Order,
|
||||
limit: i64,
|
||||
offset: i64,
|
||||
text: Option<&str>,
|
||||
) -> AppResult<(Vec<TaggedChapterAggregate>, i64)> {
|
||||
// OCR text filter: when present, additionally require the page's analysis
|
||||
// `search_doc` to match the query. Reuses the precomputed tsvector exactly
|
||||
// like `repo::page_analysis::page_search`. `None`/blank ⇒ tag-only.
|
||||
let text = text.map(str::trim).filter(|s| !s.is_empty());
|
||||
let (text_join, text_where) = match text {
|
||||
// `$5` in the main query, `$3` in the count query (see binds below).
|
||||
Some(_) => (
|
||||
"JOIN page_analysis pa ON pa.page_id = p.id",
|
||||
"AND pa.search_doc @@ plainto_tsquery('simple', {n})",
|
||||
),
|
||||
None => ("", ""),
|
||||
};
|
||||
// Same filter inside the correlated sample-thumbnail subquery (its page is
|
||||
// aliased `p`), so the thumbnails match what the text search matched. The
|
||||
// subquery lives in the main query, so it reuses the `$5` text bind.
|
||||
let (sample_join, sample_where) = match text {
|
||||
Some(_) => (
|
||||
"JOIN page_analysis pa2 ON pa2.page_id = p.id",
|
||||
"AND pa2.search_doc @@ plainto_tsquery('simple', $5)",
|
||||
),
|
||||
None => ("", ""),
|
||||
};
|
||||
let sql = format!(
|
||||
r#"
|
||||
SELECT
|
||||
@@ -274,9 +302,11 @@ pub async fn aggregate_chapters_for_tag(
|
||||
FROM pages p
|
||||
JOIN page_tags pt2 ON pt2.page_id = p.id
|
||||
JOIN tags t2 ON t2.id = pt2.tag_id
|
||||
{sample_join}
|
||||
WHERE p.chapter_id = ch.id
|
||||
AND pt2.user_id = $1
|
||||
AND lower(t2.name) = $2
|
||||
{sample_where}
|
||||
ORDER BY p.page_number ASC
|
||||
LIMIT 3
|
||||
) p2
|
||||
@@ -288,43 +318,60 @@ pub async fn aggregate_chapters_for_tag(
|
||||
JOIN pages p ON p.id = pt.page_id
|
||||
JOIN chapters ch ON ch.id = p.chapter_id
|
||||
JOIN mangas m ON m.id = ch.manga_id
|
||||
{text_join}
|
||||
WHERE pt.user_id = $1
|
||||
AND lower(t.name) = $2
|
||||
{text_where}
|
||||
GROUP BY ch.id, ch.manga_id, m.title, ch.number, ch.title
|
||||
ORDER BY match_count {dir}, ch.id
|
||||
LIMIT $3 OFFSET $4
|
||||
"#,
|
||||
dir = order.as_sql(),
|
||||
text_join = text_join,
|
||||
text_where = text_where.replace("{n}", "$5"),
|
||||
sample_join = sample_join,
|
||||
sample_where = sample_where,
|
||||
);
|
||||
let rows = sqlx::query_as::<_, TaggedChapterAggregate>(&sql)
|
||||
let mut q = sqlx::query_as::<_, TaggedChapterAggregate>(&sql)
|
||||
.bind(user_id)
|
||||
.bind(tag)
|
||||
.bind(limit)
|
||||
.bind(offset)
|
||||
.fetch_all(pool)
|
||||
.await?;
|
||||
.bind(offset);
|
||||
if let Some(text) = text {
|
||||
q = q.bind(text);
|
||||
}
|
||||
let rows = q.fetch_all(pool).await?;
|
||||
|
||||
let (total,): (i64,) = sqlx::query_as(
|
||||
let count_sql = format!(
|
||||
r#"
|
||||
SELECT count(*) FROM (
|
||||
SELECT 1
|
||||
FROM page_tags pt
|
||||
JOIN tags t ON t.id = pt.tag_id
|
||||
JOIN pages p ON p.id = pt.page_id
|
||||
{text_join}
|
||||
WHERE pt.user_id = $1 AND lower(t.name) = $2
|
||||
{text_where}
|
||||
GROUP BY p.chapter_id
|
||||
) c
|
||||
"#,
|
||||
)
|
||||
.bind(user_id)
|
||||
.bind(tag)
|
||||
.fetch_one(pool)
|
||||
.await?;
|
||||
text_join = text_join,
|
||||
text_where = text_where.replace("{n}", "$3"),
|
||||
);
|
||||
let mut cq = sqlx::query_as::<_, (i64,)>(&count_sql).bind(user_id).bind(tag);
|
||||
if let Some(text) = text {
|
||||
cq = cq.bind(text);
|
||||
}
|
||||
let (total,) = cq.fetch_one(pool).await?;
|
||||
Ok((rows, total))
|
||||
}
|
||||
|
||||
/// Paged list of mangas containing pages tagged `tag` for `user_id`,
|
||||
/// ranked by `match_count` summed across all their chapters.
|
||||
///
|
||||
/// `text` behaves as in [`aggregate_chapters_for_tag`]: non-blank restricts to
|
||||
/// pages whose OCR `search_doc` matches, and both `match_count` and the sample
|
||||
/// thumbnails reflect that filtered set.
|
||||
pub async fn aggregate_mangas_for_tag(
|
||||
pool: &PgPool,
|
||||
user_id: Uuid,
|
||||
@@ -332,7 +379,26 @@ pub async fn aggregate_mangas_for_tag(
|
||||
order: Order,
|
||||
limit: i64,
|
||||
offset: i64,
|
||||
text: Option<&str>,
|
||||
) -> AppResult<(Vec<TaggedMangaAggregate>, i64)> {
|
||||
// OCR text filter — see `aggregate_chapters_for_tag` for the rationale.
|
||||
let text = text.map(str::trim).filter(|s| !s.is_empty());
|
||||
let (text_join, text_where) = match text {
|
||||
Some(_) => (
|
||||
"JOIN page_analysis pa ON pa.page_id = p.id",
|
||||
"AND pa.search_doc @@ plainto_tsquery('simple', {n})",
|
||||
),
|
||||
None => ("", ""),
|
||||
};
|
||||
// Same filter inside the sample-thumbnail subquery (page aliased `p`),
|
||||
// reusing the main query's `$5` text bind.
|
||||
let (sample_join, sample_where) = match text {
|
||||
Some(_) => (
|
||||
"JOIN page_analysis pa2 ON pa2.page_id = p.id",
|
||||
"AND pa2.search_doc @@ plainto_tsquery('simple', $5)",
|
||||
),
|
||||
None => ("", ""),
|
||||
};
|
||||
let sql = format!(
|
||||
r#"
|
||||
SELECT
|
||||
@@ -349,9 +415,11 @@ pub async fn aggregate_mangas_for_tag(
|
||||
JOIN chapters ch2 ON ch2.id = p.chapter_id
|
||||
JOIN page_tags pt2 ON pt2.page_id = p.id
|
||||
JOIN tags t2 ON t2.id = pt2.tag_id
|
||||
{sample_join}
|
||||
WHERE ch2.manga_id = m.id
|
||||
AND pt2.user_id = $1
|
||||
AND lower(t2.name) = $2
|
||||
{sample_where}
|
||||
ORDER BY p.page_number ASC
|
||||
LIMIT 3
|
||||
) p2
|
||||
@@ -363,23 +431,31 @@ pub async fn aggregate_mangas_for_tag(
|
||||
JOIN pages p ON p.id = pt.page_id
|
||||
JOIN chapters ch ON ch.id = p.chapter_id
|
||||
JOIN mangas m ON m.id = ch.manga_id
|
||||
{text_join}
|
||||
WHERE pt.user_id = $1
|
||||
AND lower(t.name) = $2
|
||||
{text_where}
|
||||
GROUP BY m.id, m.title, m.cover_image_path
|
||||
ORDER BY match_count {dir}, m.id
|
||||
LIMIT $3 OFFSET $4
|
||||
"#,
|
||||
dir = order.as_sql(),
|
||||
text_join = text_join,
|
||||
text_where = text_where.replace("{n}", "$5"),
|
||||
sample_join = sample_join,
|
||||
sample_where = sample_where,
|
||||
);
|
||||
let rows = sqlx::query_as::<_, TaggedMangaAggregate>(&sql)
|
||||
let mut q = sqlx::query_as::<_, TaggedMangaAggregate>(&sql)
|
||||
.bind(user_id)
|
||||
.bind(tag)
|
||||
.bind(limit)
|
||||
.bind(offset)
|
||||
.fetch_all(pool)
|
||||
.await?;
|
||||
.bind(offset);
|
||||
if let Some(text) = text {
|
||||
q = q.bind(text);
|
||||
}
|
||||
let rows = q.fetch_all(pool).await?;
|
||||
|
||||
let (total,): (i64,) = sqlx::query_as(
|
||||
let count_sql = format!(
|
||||
r#"
|
||||
SELECT count(*) FROM (
|
||||
SELECT 1
|
||||
@@ -387,15 +463,20 @@ pub async fn aggregate_mangas_for_tag(
|
||||
JOIN tags t ON t.id = pt.tag_id
|
||||
JOIN pages p ON p.id = pt.page_id
|
||||
JOIN chapters ch ON ch.id = p.chapter_id
|
||||
{text_join}
|
||||
WHERE pt.user_id = $1 AND lower(t.name) = $2
|
||||
{text_where}
|
||||
GROUP BY ch.manga_id
|
||||
) m
|
||||
"#,
|
||||
)
|
||||
.bind(user_id)
|
||||
.bind(tag)
|
||||
.fetch_one(pool)
|
||||
.await?;
|
||||
text_join = text_join,
|
||||
text_where = text_where.replace("{n}", "$3"),
|
||||
);
|
||||
let mut cq = sqlx::query_as::<_, (i64,)>(&count_sql).bind(user_id).bind(tag);
|
||||
if let Some(text) = text {
|
||||
cq = cq.bind(text);
|
||||
}
|
||||
let (total,) = cq.fetch_one(pool).await?;
|
||||
Ok((rows, total))
|
||||
}
|
||||
|
||||
|
||||
@@ -459,6 +459,11 @@ impl AnalysisSettings {
|
||||
api_key: base.api_key.clone(),
|
||||
// Env-only readiness probe URL preserved from the base.
|
||||
vision_health_url: base.vision_health_url.clone(),
|
||||
// Deploy-time engine selection + ocrs model paths: env-only, not
|
||||
// admin-tunable, so carry them through from the base unchanged.
|
||||
backend: base.backend,
|
||||
ocr_detection_model: base.ocr_detection_model.clone(),
|
||||
ocr_recognition_model: base.ocr_recognition_model.clone(),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
109
backend/tests/analysis_ocr.rs
Normal file
109
backend/tests/analysis_ocr.rs
Normal file
@@ -0,0 +1,109 @@
|
||||
//! Integration tests for the OCR analysis backend
|
||||
//! (`analysis::ocr::OcrAnalyzeDispatcher`). A stub OCR engine stands in for
|
||||
//! `ocrs` (whose `.rten` models aren't shipped to CI), so these pin the
|
||||
//! storage→OCR→persist wiring: the dispatcher reads the page image, runs the
|
||||
//! engine, and persists the lines via the shared `persist_analysis` path —
|
||||
//! landing `page_ocr_text` rows and a populated `search_doc` exactly like the
|
||||
//! vision backend. Each `#[sqlx::test]` gets a fresh migrated DB.
|
||||
|
||||
mod common;
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use mangalord::analysis::daemon::AnalyzeDispatcher;
|
||||
use mangalord::analysis::ocr::test_support::StubOcrEngine;
|
||||
use mangalord::analysis::ocr::OcrAnalyzeDispatcher;
|
||||
use mangalord::domain::page_analysis::AnalysisStatus;
|
||||
use mangalord::repo;
|
||||
use mangalord::storage::{LocalStorage, Storage};
|
||||
use sqlx::PgPool;
|
||||
use tempfile::TempDir;
|
||||
use uuid::Uuid;
|
||||
|
||||
/// Seed a manga → chapter → page chain whose page points at `storage_key`,
|
||||
/// and return the page id.
|
||||
async fn seed_page(pool: &PgPool, storage_key: &str) -> Uuid {
|
||||
let manga_id: Uuid =
|
||||
sqlx::query_scalar("INSERT INTO mangas (title) VALUES ('M') RETURNING id")
|
||||
.fetch_one(pool)
|
||||
.await
|
||||
.unwrap();
|
||||
let chapter_id: Uuid = sqlx::query_scalar(
|
||||
"INSERT INTO chapters (manga_id, number) VALUES ($1, 1) RETURNING id",
|
||||
)
|
||||
.bind(manga_id)
|
||||
.fetch_one(pool)
|
||||
.await
|
||||
.unwrap();
|
||||
sqlx::query_scalar(
|
||||
"INSERT INTO pages (chapter_id, page_number, storage_key, content_type) \
|
||||
VALUES ($1, 1, $2, 'image/png') RETURNING id",
|
||||
)
|
||||
.bind(chapter_id)
|
||||
.bind(storage_key)
|
||||
.fetch_one(pool)
|
||||
.await
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
fn ocr_dispatcher(
|
||||
pool: &PgPool,
|
||||
storage: Arc<dyn Storage>,
|
||||
lines: &[&str],
|
||||
) -> OcrAnalyzeDispatcher {
|
||||
OcrAnalyzeDispatcher {
|
||||
db: pool.clone(),
|
||||
storage,
|
||||
engine: StubOcrEngine::new(lines),
|
||||
max_image_bytes: 8 * 1024 * 1024,
|
||||
}
|
||||
}
|
||||
|
||||
#[sqlx::test(migrations = "./migrations")]
|
||||
async fn dispatch_persists_ocr_lines_and_search_doc(pool: PgPool) {
|
||||
let dir = TempDir::new().unwrap();
|
||||
let storage: Arc<dyn Storage> = Arc::new(LocalStorage::new(dir.path()));
|
||||
let key = "mangas/x/p1.png";
|
||||
storage.put(key, &common::fake_png_bytes()).await.unwrap();
|
||||
let page_id = seed_page(&pool, key).await;
|
||||
|
||||
let dispatcher = ocr_dispatcher(&pool, Arc::clone(&storage), &["Hello there", "general"]);
|
||||
dispatcher.dispatch(page_id).await.unwrap();
|
||||
|
||||
// Two OCR rows, in order, with the recognized text.
|
||||
let rows: Vec<(String, i32)> = sqlx::query_as(
|
||||
"SELECT text, ord FROM page_ocr_text WHERE page_id = $1 ORDER BY ord",
|
||||
)
|
||||
.bind(page_id)
|
||||
.fetch_all(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(rows.len(), 2);
|
||||
assert_eq!(rows[0].0, "Hello there");
|
||||
assert_eq!(rows[1].0, "general");
|
||||
|
||||
// The analysis row is `done`, stamped with the ocrs model label, and has a
|
||||
// non-empty tsvector so text search works.
|
||||
let row = repo::page_analysis::load(&pool, page_id).await.unwrap().unwrap();
|
||||
assert_eq!(row.status, AnalysisStatus::Done);
|
||||
assert_eq!(row.model.as_deref(), Some("ocrs"));
|
||||
let has_doc: bool = sqlx::query_scalar(
|
||||
"SELECT search_doc IS NOT NULL AND search_doc != ''::tsvector \
|
||||
FROM page_analysis WHERE page_id = $1",
|
||||
)
|
||||
.bind(page_id)
|
||||
.fetch_one(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(has_doc, "search_doc must be populated from OCR text");
|
||||
}
|
||||
|
||||
#[sqlx::test(migrations = "./migrations")]
|
||||
async fn dispatch_missing_page_is_noop(pool: PgPool) {
|
||||
let dir = TempDir::new().unwrap();
|
||||
let storage: Arc<dyn Storage> = Arc::new(LocalStorage::new(dir.path()));
|
||||
// A page id that was never inserted — the dispatcher must treat it as a
|
||||
// deleted page and succeed without writing anything.
|
||||
let dispatcher = ocr_dispatcher(&pool, storage, &["whatever"]);
|
||||
dispatcher.dispatch(Uuid::new_v4()).await.unwrap();
|
||||
}
|
||||
@@ -680,24 +680,123 @@ async fn aggregate_rejects_invalid_order(pool: PgPool) {
|
||||
}
|
||||
|
||||
#[sqlx::test(migrations = "./migrations")]
|
||||
async fn aggregate_with_text_param_is_501_with_stable_code(pool: PgPool) {
|
||||
// OCR text search isn't built yet; the param is accepted so adding
|
||||
// OCR won't break the wire shape, but rejected with a distinct
|
||||
// status + code. The code is the wire contract — clients pin on
|
||||
// `text_search_not_yet_supported`, not the message.
|
||||
let h = common::harness(pool);
|
||||
async fn aggregate_with_text_param_filters_by_ocr(pool: PgPool) {
|
||||
// OCR text search: a page tagged `funny` whose OCR contains "guts" is
|
||||
// returned for `&text=guts` on both aggregation endpoints, and excluded
|
||||
// for a query its OCR doesn't contain. The filter runs against the same
|
||||
// precomputed `search_doc` the OCR worker writes via `persist_analysis`.
|
||||
let h = common::harness(pool.clone());
|
||||
let (_, cookie) = common::register_user(&h.app).await;
|
||||
let manga_id = common::seed_manga_via_api(&h.app, &cookie, "M").await;
|
||||
let (_, page_id) = seed_chapter_with_page(&h.app, &cookie, manga_id, 1).await;
|
||||
assert_eq!(add_tag(&h.app, &cookie, &page_id, "funny").await, StatusCode::CREATED);
|
||||
|
||||
// Persist OCR text exactly as the ocrs backend would (via the same mapper).
|
||||
let page_uuid = Uuid::parse_str(&page_id).unwrap();
|
||||
let analysis =
|
||||
mangalord::analysis::ocr::lines_to_analysis(vec!["spilling the guts here".to_string()]);
|
||||
mangalord::repo::page_analysis::persist_analysis(&pool, page_uuid, &analysis, "ocrs")
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
for endpoint in ["chapters", "mangas"] {
|
||||
// Matching text → the tagged+OCR'd row is returned.
|
||||
let resp = h
|
||||
.app
|
||||
.clone()
|
||||
.oneshot(common::get_with_cookie(
|
||||
&format!("/api/v1/me/page-tags/{endpoint}?tag=funny&text=guts"),
|
||||
&cookie,
|
||||
))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(resp.status(), StatusCode::OK, "{endpoint} matching");
|
||||
let body = common::body_json(resp).await;
|
||||
assert_eq!(body["items"].as_array().unwrap().len(), 1, "{endpoint} matching items");
|
||||
assert_eq!(body["page"]["total"], 1, "{endpoint} matching total");
|
||||
|
||||
// Non-matching text → excluded.
|
||||
let resp = h
|
||||
.app
|
||||
.clone()
|
||||
.oneshot(common::get_with_cookie(
|
||||
&format!("/api/v1/me/page-tags/{endpoint}?tag=funny&text=zzzznomatch"),
|
||||
&cookie,
|
||||
))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(resp.status(), StatusCode::OK, "{endpoint} non-matching");
|
||||
let body = common::body_json(resp).await;
|
||||
assert_eq!(body["items"].as_array().unwrap().len(), 0, "{endpoint} non-matching items");
|
||||
assert_eq!(body["page"]["total"], 0, "{endpoint} non-matching total");
|
||||
}
|
||||
}
|
||||
|
||||
#[sqlx::test(migrations = "./migrations")]
|
||||
async fn aggregate_text_param_counts_only_matching_pages(pool: PgPool) {
|
||||
// A chapter with TWO tagged pages where only ONE page's OCR matches the
|
||||
// text. `match_count` / `total` must reflect the filtered count (1), not
|
||||
// the tag-only count (2), and the sample thumbnails must contain only the
|
||||
// matching page. This is what a single-page test can't distinguish — it
|
||||
// pins the JOIN/placeholder wiring and the sample-subquery filter.
|
||||
let h = common::harness(pool.clone());
|
||||
let (_, cookie) = common::register_user(&h.app).await;
|
||||
let manga_id = common::seed_manga_via_api(&h.app, &cookie, "M").await;
|
||||
let (_, page_ids) = seed_chapter_with_n_pages(&h.app, &cookie, manga_id, 1, 2).await;
|
||||
for pid in &page_ids {
|
||||
assert_eq!(add_tag(&h.app, &cookie, pid, "funny").await, StatusCode::CREATED);
|
||||
}
|
||||
|
||||
// Page 0 OCR contains "guts"; page 1 OCR contains only "filler".
|
||||
let p0 = Uuid::parse_str(&page_ids[0]).unwrap();
|
||||
let p1 = Uuid::parse_str(&page_ids[1]).unwrap();
|
||||
let a0 = mangalord::analysis::ocr::lines_to_analysis(vec!["the guts spill out".to_string()]);
|
||||
let a1 = mangalord::analysis::ocr::lines_to_analysis(vec!["just filler text".to_string()]);
|
||||
mangalord::repo::page_analysis::persist_analysis(&pool, p0, &a0, "ocrs").await.unwrap();
|
||||
mangalord::repo::page_analysis::persist_analysis(&pool, p1, &a1, "ocrs").await.unwrap();
|
||||
|
||||
let resp = h
|
||||
.app
|
||||
.clone()
|
||||
.oneshot(common::get_with_cookie(
|
||||
"/api/v1/me/page-tags/chapters?tag=funny&text=guts",
|
||||
&cookie,
|
||||
))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(resp.status(), StatusCode::NOT_IMPLEMENTED);
|
||||
assert_eq!(resp.status(), StatusCode::OK);
|
||||
let body = common::body_json(resp).await;
|
||||
assert_eq!(body["error"]["code"], "text_search_not_yet_supported");
|
||||
let items = body["items"].as_array().unwrap();
|
||||
assert_eq!(items.len(), 1, "one chapter row");
|
||||
assert_eq!(body["page"]["total"], 1);
|
||||
// Filtered match_count is the matching-page count, not the tag count.
|
||||
assert_eq!(items[0]["match_count"], 1, "only one page's OCR matches");
|
||||
// Sample thumbnails reflect the filter: only the matching page's key.
|
||||
let samples = items[0]["sample_storage_keys"].as_array().unwrap();
|
||||
assert_eq!(samples.len(), 1, "thumbnails restricted to matching pages");
|
||||
}
|
||||
|
||||
#[sqlx::test(migrations = "./migrations")]
|
||||
async fn aggregate_blank_text_param_is_tag_only(pool: PgPool) {
|
||||
// A blank `text=` must not filter — it falls back to tag-only aggregation
|
||||
// (the page has a tag but no analysis row at all).
|
||||
let h = common::harness(pool);
|
||||
let (_, cookie) = common::register_user(&h.app).await;
|
||||
let manga_id = common::seed_manga_via_api(&h.app, &cookie, "M").await;
|
||||
let (_, page_id) = seed_chapter_with_page(&h.app, &cookie, manga_id, 1).await;
|
||||
assert_eq!(add_tag(&h.app, &cookie, &page_id, "funny").await, StatusCode::CREATED);
|
||||
|
||||
let resp = h
|
||||
.app
|
||||
.oneshot(common::get_with_cookie(
|
||||
"/api/v1/me/page-tags/chapters?tag=funny&text=",
|
||||
&cookie,
|
||||
))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(resp.status(), StatusCode::OK);
|
||||
let body = common::body_json(resp).await;
|
||||
assert_eq!(body["items"].as_array().unwrap().len(), 1);
|
||||
}
|
||||
|
||||
#[sqlx::test(migrations = "./migrations")]
|
||||
|
||||
@@ -143,8 +143,13 @@ services:
|
||||
# when both are present. Leave unset for the bundled
|
||||
# HashedControlPassword path above.
|
||||
CRAWLER_TOR_CONTROL_COOKIE_PATH: ${CRAWLER_TOR_CONTROL_COOKIE_PATH:-}
|
||||
# Analysis worker (vision) — boot seeds + env-only API key.
|
||||
# Analysis worker — boot seeds + env-only API key.
|
||||
ANALYSIS_ENABLED: ${ANALYSIS_ENABLED:-false}
|
||||
# Engine selection (deploy-time): `ocr` (in-process ocrs, default) or
|
||||
# `vision` (local LLM). Model paths default to the image-baked /models.
|
||||
ANALYSIS_BACKEND: ${ANALYSIS_BACKEND:-ocr}
|
||||
OCRS_DETECTION_MODEL: ${OCRS_DETECTION_MODEL:-/models/text-detection.rten}
|
||||
OCRS_RECOGNITION_MODEL: ${OCRS_RECOGNITION_MODEL:-/models/text-recognition.rten}
|
||||
ANALYSIS_VISION_URL: ${ANALYSIS_VISION_URL:-}
|
||||
ANALYSIS_VISION_MODEL: ${ANALYSIS_VISION_MODEL:-}
|
||||
ANALYSIS_WORKERS: ${ANALYSIS_WORKERS:-1}
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "mangalord-frontend",
|
||||
"version": "0.89.0",
|
||||
"version": "0.90.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
|
||||
Reference in New Issue
Block a user