Mangalord/HYBRID-OCR-VISION.md

# Hybrid OCR→Vision analysis pipeline — design brief for the dev agent

**Status: design only (not implemented).** Forward-looking model for a future
`ANALYSIS_BACKEND=hybrid`, on top of the shipped `ocr` (default) and `vision`
backends. Constraint that shapes it: `ocrs` and the vision LLM must **not** be
resident at the same time (Pi RAM ceiling). OCR has priority; vision is a
preemptible, only-when-OCR-is-drained background phase.

## The key realization: this is the crawl mutex, again

The repo already runs a **`vision-manager`** sidecar
([vision-manager/manager.sh](vision-manager/manager.sh),
[VISION-AUTOSCALE.md](VISION-AUTOSCALE.md)) that owns the `mangalord-vision`
(llama.cpp, ~4 GiB) container lifecycle via a scoped docker-socket-proxy — the
backend gets no Docker access. It already:

- **Starts** vision when `pending_analysis()` > 0 (counts `analyze_page` jobs).
- **Idle-stops** it after `STOP_DEBOUNCE` (anti-thrash) once the queue drains.
- **Defers starting** vision while `crawl_running()` > 0 — the **"RAM mutex"**:
  the crawler's browser and the LLM can't both fit, so vision waits for crawl.
- **Memory-yield**: SIGTERMs a running vision at a HIGH host-RAM watermark,
  blocks starting at a LOW watermark (see [VISION-MEMORY-YIELD.md](VISION-MEMORY-YIELD.md)).

The OCR↔vision relationship is the relationship that already exists between
crawl and vision. OCR is just a third RAM-competing, higher-priority workload
that vision must defer to. Extend the existing arbiter rather than build a new one.

## Job model: split one queue into two

Today there is one job kind, `analyze_page`. Split it:

- **`ocr_page`** — the in-process ocrs pass (cheap, fast, no container).
- **`ground_page`** — the vision grounding pass (needs the LLM container).

**Pipeline (hybrid mode):** upload → enqueue `ocr_page`. The OCR daemon runs
ocrs, writes `page_ocr_text` + `search_doc` (text searchable *immediately*), then
enqueues `ground_page` for that page. The grounding daemon later adds
tags/scene/safety. Each kind gets its own dedup index (mirror migration 0031).

Two daemons, two queues.

## Arbitration: OCR priority + memory exclusivity

Responsibilities split cleanly between the in-backend daemons and the manager:

**Backend — OCR daemon** (already built): leases `ocr_page`, always allowed to
run (ocrs is small/local). This is the priority phase.

**Backend — grounding daemon:** leases `ground_page`, but gates leasing on
**both**:
1. `VisionReadiness` (the existing `/health` gate — vision container up), AND
2. **`ocr_backlog == 0`** (a cheap `count(ocr_page WHERE state IN pending,running)`).

Gate #2 is the whole priority mechanism, in-process, no aborts: the instant OCR
work appears, the grounding daemon **finishes its current page** (the in-flight
lease completes normally) and then **stops leasing** new pages and parks — "it
finishes, then yields." It resumes only when OCR has fully drained.

**vision-manager:** two one-line changes to the existing logic:
1. `pending_analysis()` counts **`ground_page`** (not `ocr_page`) — vision only
   starts when there is *grounding* work.
2. Add an **OCR start-block mutex** identical to `RESPECT_CRAWL_MUTEX`: defer
   starting vision while `ocr_page` backlog > 0. (`RESPECT_OCR_MUTEX=1`.)

**Why this yields memory exclusivity:** while OCR backlog > 0, the grounding
daemon won't lease → vision goes idle → the manager's idle-debounce stops the
container (and never restarts it under the OCR mutex). Only ocrs (small) is
resident. When OCR drains, grounding resumes, `pending_analysis()` > 0 again, the
manager starts vision. Only the *big* consumer (the LLM container) is ever
mutually exclusive with active OCR — which is the actual RAM constraint.

**The one overlap window:** if OCR work arrives mid-grounding-page, ocrs (small,
in-process) briefly coexists with the one in-flight vision page before the
grounding daemon parks. ocrs's footprint makes this a non-issue in practice. If a
deployment needs *hard* exclusivity even there, the OCR daemon can additionally
wait for `vision_running == false` before its first dispatch. No deadlock: OCR's
wait is transient (one grounding page, bounded by `job_timeout`), while vision's
deferral on OCR backlog is the persistent, priority-respecting side.

**Anti-thrash:** the manager's `STOP_DEBOUNCE` already prevents a stray OCR page
from cold-cycling the 4 GiB model. Chapter uploads arrive as bursts, so OCR
drains a whole batch in one phase before vision resumes — the natural good case.

## Data-model split (the real refactor)

`persist_analysis` currently writes OCR + tags + scene + safety in one
transaction and derives `search_doc` from the OCR rows (+ scene weight C). Two
passes means splitting it, sharing the tsvector builder:

- **`persist_ocr(page, lines)`** — writes `page_ocr_text`, sets
  `search_doc` from OCR (A/B/D buckets), marks the page text-searchable. Status
  reflects OCR completion.
- **`persist_grounding(page, tags, scene, safety)`** — writes auto-tags,
  warnings, scene; **recomputes** `search_doc` to fold in the scene (weight C) on
  top of the existing OCR buckets. Records grounding completion (e.g. a
  `grounded_at` column or tags-present sentinel).

This keeps text search live after the cheap OCR phase, with semantic search/tags
arriving after the (deferred) grounding phase.

## Reusable pieces

- OCR side: the shipped `OcrsEngine` + `OcrAnalyzeDispatcher`
  ([backend/src/analysis/ocr.rs](backend/src/analysis/ocr.rs)) — retarget to
  `ocr_page`, and on success enqueue `ground_page`.
- Vision side: factor `VisionClient::ground(image, mime, ocr_text)` out of
  `analyze()`'s Pass-B block
  ([backend/src/analysis/vision.rs](backend/src/analysis/vision.rs) ~240–261) —
  pure refactor, existing tests cover it. The grounding daemon is the existing
  analysis daemon retargeted to `ground_page` with the extra `ocr_backlog==0`
  lease gate.
- Manager: `pending_analysis()` kind swap + `RESPECT_OCR_MUTEX` block, mirroring
  the crawl-mutex branch already at [vision-manager/manager.sh](vision-manager/manager.sh).

## Scope / sequencing

Larger than an inline hybrid dispatcher: job-kind split + dedup migrations,
`persist_*` split + search_doc-builder extraction, pipeline enqueue (OCR→ground),
grounding daemon lease gate, and the manager mutex. No new privileged surface
(reuses the docker-socket-proxy). Suggested order:
1. Split `persist_analysis` → `persist_ocr` / `persist_grounding` (+ tests).
2. Split job kinds + dedup migrations; retarget the OCR daemon to `ocr_page`.
3. `VisionClient::ground()` refactor + grounding daemon (`ground_page`, gated on
   `ocr_backlog==0`).
4. OCR→ground enqueue on OCR success (hybrid mode only).
5. vision-manager: `ground_page` counting + `RESPECT_OCR_MUTEX`.

Pure-`ocr` (shipped default) and `vision` (legacy) backends are unaffected; this
is the `hybrid` backend's runtime model.

## Open decision

Hard exclusivity in the one overlap window (OCR daemon waits for `vision_running
== false` before its first dispatch) — needed only if ocrs's few-hundred-MB
footprint can't coexist with a single in-flight grounding page on the target
Pi's RAM. Default: don't gate (rely on the grounding daemon parking fast).