cb34eeb82e
Add a CPU-only, English-first OCR engine (the `ocrs` crate on the `rten` runtime) as an alternative to the local-LLM "vision" backend, which ran too slowly on the target Raspberry Pi 5. - `ANALYSIS_BACKEND` (default `ocr`) selects the engine; `vision` keeps the existing LLM path. Deploy-time, env-only — not admin-tunable. - `analysis::ocr` adds an `OcrEngine` trait (testable seam), the `OcrsEngine` impl (models loaded once, inference on the blocking pool), and an `OcrAnalyzeDispatcher` that plugs into the existing `AnalyzeDispatcher` seam and reuses `persist_analysis`, so OCR text lands in `page_ocr_text` and the `search_doc` tsvector exactly as the vision path produces them. - The backend image bakes the two `.rten` models into /models, where `OCRS_DETECTION_MODEL` / `OCRS_RECOGNITION_MODEL` default. Because `/v1/me/page-search` ranks on `search_doc`, OCR text search works the moment pages are processed. Auto-tagging, scene description and NSFW flags remain the vision backend's job (deferred). Bumps to 0.90.0 (minor) in both manifests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
101 lines
4.4 KiB
Docker
101 lines
4.4 KiB
Docker
# Multi-stage build for the Rust backend.
|
|
FROM rust:1-slim AS builder
|
|
WORKDIR /app
|
|
RUN apt-get update \
|
|
&& apt-get install -y --no-install-recommends pkg-config libssl-dev ca-certificates \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
# Cache deps separately from sources. `--locked` makes cargo refuse to
|
|
# update the lockfile, so the production image is built against the
|
|
# exact crate versions CI tested. Without Cargo.lock + the flag, cargo
|
|
# would silently resolve fresh on every image build.
|
|
COPY Cargo.toml Cargo.lock ./
|
|
RUN mkdir -p src/bin && echo "fn main() {}" > src/main.rs && echo "" > src/lib.rs \
|
|
&& echo "fn main() {}" > src/bin/crawler.rs \
|
|
&& cargo build --locked --release \
|
|
&& rm -rf src
|
|
|
|
COPY src ./src
|
|
COPY migrations ./migrations
|
|
RUN touch src/main.rs src/lib.rs && cargo build --locked --release
|
|
|
|
FROM debian:trixie-slim
|
|
# Runtime base must match the builder's Debian release: `rust:1-slim` tracks
|
|
# trixie (glibc 2.41), so a bookworm runtime (glibc 2.36) can't run the
|
|
# binary ("GLIBC_2.39 not found"). Keep these two in lockstep on bumps.
|
|
# `curl` is for the container HEALTHCHECK; `ca-certificates` is for
|
|
# outbound HTTPS (crawler covers/pages).
|
|
#
|
|
# INSTALL_CHROMIUM is an opt-in for deployments that can't use the
|
|
# chromiumoxide fetcher path (notably Linux_arm64 / Raspberry Pi, where
|
|
# the upstream snapshot bucket has no usable build). When `true`, adds
|
|
# Debian's apt-packaged headless chromium plus a baseline font set —
|
|
# pair with `CRAWLER_CHROMIUM_BINARY=/usr/bin/chromium-headless-shell`
|
|
# at runtime so the launcher uses it. Default `false` keeps cloud/x86
|
|
# images slim.
|
|
#
|
|
# Build the Pi image with:
|
|
# docker compose build --build-arg INSTALL_CHROMIUM=true backend
|
|
ARG INSTALL_CHROMIUM=false
|
|
RUN apt-get update \
|
|
&& apt-get install -y --no-install-recommends ca-certificates curl \
|
|
&& if [ "$INSTALL_CHROMIUM" = "true" ]; then \
|
|
apt-get install -y --no-install-recommends chromium-headless-shell fonts-liberation; \
|
|
fi \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
# Non-root runtime user. The API binary doesn't need any root
|
|
# privilege; the crawler daemon's Chromium launcher uses --no-sandbox
|
|
# precisely because user-namespace sandboxing is fragile, so dropping
|
|
# privileges costs nothing operationally and shrinks the blast radius
|
|
# of any RCE.
|
|
ARG APP_UID=10001
|
|
ARG APP_GID=10001
|
|
RUN groupadd --system --gid ${APP_GID} app \
|
|
&& useradd --system --uid ${APP_UID} --gid app --home-dir /home/app --create-home --shell /usr/sbin/nologin app
|
|
|
|
WORKDIR /app
|
|
COPY --from=builder /app/target/release/mangalord /usr/local/bin/mangalord
|
|
COPY --from=builder /app/migrations /app/migrations
|
|
|
|
# OCR models for the default `ANALYSIS_BACKEND=ocr` (ocrs) engine. The two
|
|
# `.rten` files are pulled at build time into /models, where the runtime's
|
|
# `OCRS_DETECTION_MODEL` / `OCRS_RECOGNITION_MODEL` defaults point. They're a
|
|
# few MB each and pure data (no native code), so they bake cleanly into the
|
|
# image and need no manual setup on the Pi. Set INSTALL_OCR_MODELS=false to
|
|
# skip (e.g. for a vision-only deploy that never runs ocrs).
|
|
ARG INSTALL_OCR_MODELS=true
|
|
ARG OCRS_MODEL_BASE_URL=https://ocrs-models.s3-accelerate.amazonaws.com
|
|
RUN if [ "$INSTALL_OCR_MODELS" = "true" ]; then \
|
|
mkdir -p /models \
|
|
&& curl -fsSL "${OCRS_MODEL_BASE_URL}/text-detection.rten" -o /models/text-detection.rten \
|
|
&& curl -fsSL "${OCRS_MODEL_BASE_URL}/text-recognition.rten" -o /models/text-recognition.rten; \
|
|
fi
|
|
|
|
ENV STORAGE_DIR=/var/lib/mangalord/storage
|
|
# Pre-create the storage dir so the entrypoint doesn't need to
|
|
# mkdir-as-root and so the named volume mount inherits the right
|
|
# ownership.
|
|
#
|
|
# UPGRADE NOTE for operators: if you're moving from an older image
|
|
# that ran as root, the existing `storage-data` volume has files owned
|
|
# by UID 0 and the new UID-10001 user can't write them. Run once
|
|
# before the upgrade:
|
|
# docker compose run --rm --user 0 backend \
|
|
# chown -R 10001:10001 /var/lib/mangalord/storage
|
|
# (Postgres is unaffected — that image's `postgres` user UID hasn't
|
|
# changed.)
|
|
RUN mkdir -p ${STORAGE_DIR} \
|
|
&& chown -R app:app ${STORAGE_DIR} /app /home/app
|
|
|
|
USER app
|
|
EXPOSE 8080
|
|
|
|
# `--start-period` is generous because first boot runs sqlx::migrate
|
|
# against postgres which can take a few seconds; subsequent restarts
|
|
# are sub-second.
|
|
HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \
|
|
CMD curl -fsS http://localhost:8080/api/v1/health > /dev/null || exit 1
|
|
|
|
CMD ["mangalord"]
|