Mangalord/HYBRID-OCR-VISION.md

Hybrid OCR→Vision analysis pipeline — design brief for the dev agent

Status: design only (not implemented). Forward-looking model for a future ANALYSIS_BACKEND=hybrid, on top of the shipped ocr (default) and vision backends. Constraint that shapes it: ocrs and the vision LLM must not be resident at the same time (Pi RAM ceiling). OCR has priority; vision is a preemptible, only-when-OCR-is-drained background phase.

The key realization: this is the crawl mutex, again

The repo already runs a vision-manager sidecar (vision-manager/manager.sh, VISION-AUTOSCALE.md) that owns the mangalord-vision (llama.cpp, ~4 GiB) container lifecycle via a scoped docker-socket-proxy — the backend gets no Docker access. It already:

• Starts vision when pending_analysis() > 0 (counts analyze_page jobs).
• Idle-stops it after STOP_DEBOUNCE (anti-thrash) once the queue drains.
• Defers starting vision while crawl_running() > 0 — the "RAM mutex": the crawler's browser and the LLM can't both fit, so vision waits for crawl.
• Memory-yield: SIGTERMs a running vision at a HIGH host-RAM watermark, blocks starting at a LOW watermark (see VISION-MEMORY-YIELD.md).

The OCR↔vision relationship is the relationship that already exists between crawl and vision. OCR is just a third RAM-competing, higher-priority workload that vision must defer to. Extend the existing arbiter rather than build a new one.

Job model: split one queue into two

Today there is one job kind, analyze_page. Split it:

• ocr_page — the in-process ocrs pass (cheap, fast, no container).
• ground_page — the vision grounding pass (needs the LLM container).

Pipeline (hybrid mode): upload → enqueue ocr_page. The OCR daemon runs ocrs, writes page_ocr_text + search_doc (text searchable immediately), then enqueues ground_page for that page. The grounding daemon later adds tags/scene/safety. Each kind gets its own dedup index (mirror migration 0031).

Two daemons, two queues.

Arbitration: OCR priority + memory exclusivity

Responsibilities split cleanly between the in-backend daemons and the manager:

Backend — OCR daemon (already built): leases ocr_page, always allowed to run (ocrs is small/local). This is the priority phase.

Backend — grounding daemon: leases ground_page, but gates leasing on both:

1. VisionReadiness (the existing /health gate — vision container up), AND
2. ocr_backlog == 0 (a cheap count(ocr_page WHERE state IN pending,running)).

Gate #2 is the whole priority mechanism, in-process, no aborts: the instant OCR work appears, the grounding daemon finishes its current page (the in-flight lease completes normally) and then stops leasing new pages and parks — "it finishes, then yields." It resumes only when OCR has fully drained.

vision-manager: two one-line changes to the existing logic:

1. pending_analysis() counts ground_page (not ocr_page) — vision only starts when there is grounding work.
2. Add an OCR start-block mutex identical to RESPECT_CRAWL_MUTEX: defer starting vision while ocr_page backlog > 0. (RESPECT_OCR_MUTEX=1.)

Why this yields memory exclusivity: while OCR backlog > 0, the grounding daemon won't lease → vision goes idle → the manager's idle-debounce stops the container (and never restarts it under the OCR mutex). Only ocrs (small) is resident. When OCR drains, grounding resumes, pending_analysis() > 0 again, the manager starts vision. Only the big consumer (the LLM container) is ever mutually exclusive with active OCR — which is the actual RAM constraint.

The one overlap window: if OCR work arrives mid-grounding-page, ocrs (small, in-process) briefly coexists with the one in-flight vision page before the grounding daemon parks. ocrs's footprint makes this a non-issue in practice. If a deployment needs hard exclusivity even there, the OCR daemon can additionally wait for vision_running == false before its first dispatch. No deadlock: OCR's wait is transient (one grounding page, bounded by job_timeout), while vision's deferral on OCR backlog is the persistent, priority-respecting side.

Anti-thrash: the manager's STOP_DEBOUNCE already prevents a stray OCR page from cold-cycling the 4 GiB model. Chapter uploads arrive as bursts, so OCR drains a whole batch in one phase before vision resumes — the natural good case.

Data-model split (the real refactor)

persist_analysis currently writes OCR + tags + scene + safety in one transaction and derives search_doc from the OCR rows (+ scene weight C). Two passes means splitting it, sharing the tsvector builder:

• persist_ocr(page, lines) — writes page_ocr_text, sets search_doc from OCR (A/B/D buckets), marks the page text-searchable. Status reflects OCR completion.
• persist_grounding(page, tags, scene, safety) — writes auto-tags, warnings, scene; recomputes search_doc to fold in the scene (weight C) on top of the existing OCR buckets. Records grounding completion (e.g. a grounded_at column or tags-present sentinel).

This keeps text search live after the cheap OCR phase, with semantic search/tags arriving after the (deferred) grounding phase.

Reusable pieces

• OCR side: the shipped OcrsEngine + OcrAnalyzeDispatcher (backend/src/analysis/ocr.rs) — retarget to ocr_page, and on success enqueue ground_page.
• Vision side: factor VisionClient::ground(image, mime, ocr_text) out of analyze()'s Pass-B block (backend/src/analysis/vision.rs ~240–261) — pure refactor, existing tests cover it. The grounding daemon is the existing analysis daemon retargeted to ground_page with the extra ocr_backlog==0 lease gate.
• Manager: pending_analysis() kind swap + RESPECT_OCR_MUTEX block, mirroring the crawl-mutex branch already at vision-manager/manager.sh.

Scope / sequencing

Larger than an inline hybrid dispatcher: job-kind split + dedup migrations, persist_* split + search_doc-builder extraction, pipeline enqueue (OCR→ground), grounding daemon lease gate, and the manager mutex. No new privileged surface (reuses the docker-socket-proxy). Suggested order:

1. Split persist_analysis → persist_ocr / persist_grounding (+ tests).
2. Split job kinds + dedup migrations; retarget the OCR daemon to ocr_page.
3. VisionClient::ground() refactor + grounding daemon (ground_page, gated on ocr_backlog==0).
4. OCR→ground enqueue on OCR success (hybrid mode only).
5. vision-manager: ground_page counting + RESPECT_OCR_MUTEX.

Pure-ocr (shipped default) and vision (legacy) backends are unaffected; this is the hybrid backend's runtime model.

Open decision

Hard exclusivity in the one overlap window (OCR daemon waits for vision_running == false before its first dispatch) — needed only if ocrs's few-hundred-MB footprint can't coexist with a single in-flight grounding page on the target Pi's RAM. Default: don't gate (rely on the grounding daemon parking fast).