Off-by-default left headless-browser page JS/subresources/redirects able to reach internal IPs (Chromium bypasses the reqwest SafeResolver). The crawler runs in the backend container so it can't be network-isolated without breaking Postgres — the CDP Fetch interceptor is the control. Flip CRAWLER_SSRF_INTERCEPT default to true (config + .env.example + compose); off-path unchanged as break-glass. The #[ignore]'d smoke test validates against real Chromium (command documented). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
428 lines
23 KiB
Plaintext
428 lines
23 KiB
Plaintext
# Copy to .env for `docker compose up --build`. Local-dev runs (cargo run
|
|
# / npm run dev) read backend/.env if present, or pick up the variables
|
|
# from your shell.
|
|
#
|
|
# Production note: COOKIE_SECURE=true (the default below) makes browsers
|
|
# refuse to send the session cookie over plain HTTP. Run with a TLS-
|
|
# terminating reverse proxy (Caddy, Traefik, nginx) in front — the
|
|
# compose file here doesn't ship one. Local/dev runs without HTTPS
|
|
# should set COOKIE_SECURE=false.
|
|
|
|
# ----- Postgres -----
|
|
# These are read by the Postgres container *and* by DATABASE_URL below;
|
|
# changing them after the first boot won't migrate existing data, so set
|
|
# them up front for any new deployment.
|
|
#
|
|
# POSTGRES_PASSWORD is REQUIRED — docker-compose.yml fails fast if it
|
|
# isn't set in this file, to prevent a deploy without an .env booting
|
|
# Postgres with a publicly-known credential.
|
|
POSTGRES_USER=mangalord
|
|
POSTGRES_PASSWORD=change-me-to-a-strong-random-string
|
|
POSTGRES_DB=mangalord
|
|
|
|
# ----- Backend -----
|
|
DATABASE_URL=postgres://mangalord:mangalord@postgres:5432/mangalord
|
|
BIND_ADDRESS=0.0.0.0:8080
|
|
STORAGE_DIR=/var/lib/mangalord/storage
|
|
RUST_LOG=info,mangalord=debug,chromiumoxide::conn=off,chromiumoxide::handler=off
|
|
# Postgres connection-pool sizing. One pool serves HTTP handlers and the
|
|
# crawler/analysis daemons. DB_MAX_CONNECTIONS caps open connections;
|
|
# DB_ACQUIRE_TIMEOUT_SECS is how long a request waits for a free connection
|
|
# before failing fast (rather than hanging on the driver's 30s default).
|
|
DB_MAX_CONNECTIONS=20
|
|
DB_ACQUIRE_TIMEOUT_SECS=10
|
|
|
|
# ----- Auth / cookies -----
|
|
# COOKIE_SECURE controls whether the `Secure` flag is set on the session
|
|
# cookie. Keep `true` in production (HTTPS); set to `false` if you're
|
|
# serving over plain HTTP locally (e.g., behind a dev reverse proxy).
|
|
COOKIE_SECURE=true
|
|
# COOKIE_DOMAIN scopes the session cookie. Leave empty to default to the
|
|
# requesting host. Set when serving the API and frontend on subdomains of
|
|
# a shared parent (e.g., `.example.com`) so the cookie is shared.
|
|
COOKIE_DOMAIN=
|
|
# Session lifetime in days. Expired sessions are no longer accepted and
|
|
# get reaped lazily.
|
|
SESSION_TTL_DAYS=30
|
|
|
|
# ----- Auth brute-force rate limits -----
|
|
# Token-bucket budget shared across /auth/login, /auth/register, and
|
|
# /auth/me/password. Set per_sec=0 to disable (e.g. behind a
|
|
# rate-limiting reverse proxy that already enforces a budget).
|
|
AUTH_RATE_PER_SEC=5
|
|
AUTH_RATE_BURST=10
|
|
# Trust a proxy-supplied X-Forwarded-For as the client IP for per-IP auth
|
|
# rate limiting. Enable ONLY when the backend sits behind a trusted proxy
|
|
# that overrides the header (the compose deploy: SvelteKit forwards the real
|
|
# peer IP). When false, the header is ignored and a single shared bucket is
|
|
# used — a directly-exposed backend MUST keep this off or clients could spoof
|
|
# their IP to dodge the limit.
|
|
AUTH_TRUSTED_PROXY=false
|
|
|
|
# ----- CORS -----
|
|
# Comma-separated origins allowed to call the API with credentials.
|
|
# Default is empty: same-origin only. Set when frontend and backend live
|
|
# on different hosts. Example: https://app.example.com,https://app.example.de
|
|
CORS_ALLOWED_ORIGINS=
|
|
|
|
# ----- Admin CSRF allowlist -----
|
|
# Browser origins (scheme + host[:port]) permitted to POST to
|
|
# /api/v1/admin/* mutating endpoints. Defends the session-cookie-
|
|
# authenticated admin surface against SameSite=Lax form-POST CSRF.
|
|
# Same shape as CORS_ALLOWED_ORIGINS (comma-separated). Compare against
|
|
# the request's Origin header (falling back to Referer when absent);
|
|
# safe methods (GET/HEAD/OPTIONS) are not checked, and requests with
|
|
# neither Origin nor Referer (curl, server-to-server callers) are
|
|
# always allowed.
|
|
#
|
|
# Empty does NOT mean "off" for browsers: cookie-authenticated admin
|
|
# mutations FAIL CLOSED (403) when this is empty, since there's no
|
|
# allowlist to check the Origin against. Non-cookie callers (curl,
|
|
# bots with a Bearer token, no Origin/Referer) are still allowed.
|
|
# So set this to the SvelteKit origin for any browser-exposed deploy,
|
|
# e.g. https://app.example.com. For a same-origin docker-compose deploy
|
|
# set the same value the browser uses.
|
|
# Local dev (native `npm run dev`): the Vite origin is
|
|
# http://localhost:5173 — set ADMIN_ALLOWED_ORIGINS=http://localhost:5173
|
|
# or admin toggles (e.g. enabling the analysis worker) return 403.
|
|
ADMIN_ALLOWED_ORIGINS=
|
|
|
|
# ----- Admin bootstrap -----
|
|
# When BOTH are set and non-empty, the backend ensures a user with this
|
|
# username exists at startup, promotes it to admin, and (for a brand-new
|
|
# row only) sets the password. An existing user's password is NEVER
|
|
# overwritten by this — rotate via the dashboard or `/auth/me/password`.
|
|
# Leave BOTH empty in a fully provisioned deploy.
|
|
ADMIN_USERNAME=
|
|
ADMIN_PASSWORD=
|
|
|
|
# ----- Site-wide auth gates (env-ONLY) -----
|
|
# PRIVATE_MODE: when `true`, every endpoint except a small public allowlist
|
|
# (/health, /auth/config, /auth/login, /auth/logout) demands a valid
|
|
# session — anonymous reads return 401. Self-registration is also
|
|
# force-disabled regardless of ALLOW_SELF_REGISTER. Default `false`.
|
|
PRIVATE_MODE=false
|
|
# ALLOW_SELF_REGISTER: when `false`, /auth/register returns 403 and the
|
|
# frontend hides its register affordance. Admins can still mint accounts
|
|
# via /admin/users. Default `true` (open registration). Forced to `false`
|
|
# when PRIVATE_MODE=true.
|
|
ALLOW_SELF_REGISTER=true
|
|
|
|
# ----- Upload limits -----
|
|
# Per-request body cap. axum rejects oversized requests with 413 before
|
|
# our handlers run. Default 200 MiB.
|
|
MAX_REQUEST_BYTES=209715200
|
|
# Per-image-part cap. Enforced after reading each part, so a single
|
|
# oversized image is rejected even when the total request fits.
|
|
# Default 20 MiB.
|
|
MAX_FILE_BYTES=20971520
|
|
# Max page images accepted in one chapter upload. Bounds how many parts
|
|
# the handler will stage before rejecting the request with 413, so a
|
|
# client can't pin a worker with an unbounded page count. 0 disables the
|
|
# cap. Default 2000.
|
|
MAX_PAGES_PER_CHAPTER=2000
|
|
|
|
# ----- Crawler download safety -----
|
|
# Hosts the crawler is allowed to fetch images/covers from, in addition
|
|
# to CRAWLER_START_URL's host and CRAWLER_CDN_HOST. Comma-separated.
|
|
# Defends against SSRF via scraped <img src="http://10.0.0.1/...">.
|
|
CRAWLER_DOWNLOAD_ALLOWLIST=
|
|
# Bypass the host allowlist entirely. Intended for sources that shard
|
|
# images across numbered CDN subdomains (cdn1/cdn2/…) where enumerating
|
|
# every host upfront is impractical. The private-IP / localhost / non-
|
|
# http(s) scheme defenses STAY ON — a scraped <img src="http://10.0.0.1/">
|
|
# is still refused with this flag set.
|
|
CRAWLER_ALLOW_ANY_HOST=false
|
|
# Hard cap on a single image body. Default 32 MiB.
|
|
CRAWLER_MAX_IMAGE_BYTES=33554432
|
|
# Hard cap on the number of page images in one crawled chapter. The
|
|
# per-image byte cap doesn't stop a hostile reader page listing thousands
|
|
# of <img> tags; an over-cap chapter is acked failed instead of downloaded.
|
|
# 0 disables the cap. Default 2000.
|
|
CRAWLER_MAX_IMAGES_PER_CHAPTER=2000
|
|
# Max manga detail fetches per metadata pass (both the in-process daemon
|
|
# and the `bin/crawler` CLI). 0 means no cap — let the source walker run
|
|
# to completion. Useful for capped test runs against a new source.
|
|
CRAWLER_LIMIT=0
|
|
|
|
# ----- Crawler reliability knobs -----
|
|
# Hard upper bound on a single chapter-content job dispatch (seconds).
|
|
# A job that exceeds the budget is acked failed (with exponential
|
|
# backoff) instead of wedging a worker. Default 600s.
|
|
CRAWLER_JOB_TIMEOUT_SECS=600
|
|
# Idle seconds the daemon waits for new jobs before parking (between
|
|
# scheduled passes). Lower for snappier dev loops; higher for fewer
|
|
# wakeups in steady state. Default 600.
|
|
CRAWLER_IDLE_TIMEOUT_S=600
|
|
# Concurrent chapter-content workers. Each pulls one chapter at a time
|
|
# from the queue. Higher = faster on multi-core hosts BUT also more
|
|
# pressure on the source (raise CRAWLER_RATE_MS in tandem). Default 1.
|
|
CRAWLER_CHAPTER_WORKERS=1
|
|
# Days of completed `crawler_jobs` rows kept before vacuum (per-row
|
|
# audit trail; failed rows survive to make retries traceable). Default 7.
|
|
CRAWLER_JOB_RETENTION_DAYS=7
|
|
# Days of `crawl_metrics` rows kept (per-run aggregate dashboard
|
|
# data). Longer history → bigger dashboard charts; shorter → less DB.
|
|
# Default 90.
|
|
CRAWL_METRICS_RETENTION_DAYS=90
|
|
# Politeness rate limit between requests to the source (milliseconds).
|
|
# Pause inserted after each fetch. Raising slows the crawl but is
|
|
# friendlier to small sources. Default 1000ms.
|
|
CRAWLER_RATE_MS=1000
|
|
# Politeness rate for the CDN host (CRAWLER_CDN_HOST). Defaults to
|
|
# CRAWLER_RATE_MS when unset — set lower if the CDN tolerates faster
|
|
# pulls, or higher if it rate-limits aggressively. Default 1000ms.
|
|
CRAWLER_CDN_RATE_MS=1000
|
|
# User-Agent string for crawler HTTP requests. Leave unset for a
|
|
# generic default. Some sources fingerprint UA — set a realistic
|
|
# browser UA if the source 403s the default.
|
|
CRAWLER_USER_AGENT=
|
|
# Source session cookie value (PHPSESSID). Required when the source
|
|
# gates listings behind a logged-in session. Treat as a secret.
|
|
CRAWLER_PHPSESSID=
|
|
# Cookie domain the crawler scopes its session to. Usually the source
|
|
# host (e.g. `mangadex.org`). Leave unset to let reqwest infer it.
|
|
CRAWLER_COOKIE_DOMAIN=
|
|
# Consecutive metadata-pass `fetch_manga` failures that abort the pass
|
|
# (circuit breaker for a source outage). The pass does NOT mark a clean
|
|
# exit, so the next tick does a recovery sweep. Default 10.
|
|
CRAWLER_METADATA_MAX_CONSECUTIVE_FAILURES=10
|
|
# Consecutive transient chapter failures (after TOR recircuit is
|
|
# exhausted) that trigger an automatic coordinated browser restart.
|
|
# Default 3.
|
|
CRAWLER_BROWSER_RESTART_THRESHOLD=3
|
|
# CDP Fetch interception that re-validates every headless-browser
|
|
# navigation/redirect/subresource against the SSRF check (blocks a scraped page
|
|
# that drives the browser — via a redirect OR page JS/subresource — to an
|
|
# internal target such as the cloud metadata service or postgres). Default TRUE:
|
|
# with it off, only the top-level URL string is checked and page JS/subresources
|
|
# reach internal IPs (the reqwest SafeResolver does not cover Chromium's own
|
|
# stack). The CDP Fetch hook can't be exercised in CI (no Chromium); before
|
|
# relying on a fresh deploy, validate it doesn't wedge navigation:
|
|
# CRAWLER_CHROMIUM_BINARY=/usr/bin/chromium \
|
|
# cargo test -p mangalord --test crawler_browser_smoke -- --ignored \
|
|
# ssrf_interception_does_not_wedge_allowed_navigation
|
|
# Set to false only as a break-glass if the hook destabilizes a deployment.
|
|
CRAWLER_SSRF_INTERCEPT=true
|
|
# Path to a system Chromium binary. When set, the crawler skips the
|
|
# bundled-fetcher download. Required on platforms without a usable
|
|
# upstream Chromium build (notably Linux_arm64 / Raspberry Pi). On
|
|
# Debian: /usr/bin/chromium-headless-shell or /usr/bin/chromium. On
|
|
# Ubuntu the package is chromium-browser (different path). Pair with
|
|
# `docker compose build --build-arg INSTALL_CHROMIUM=true backend` so
|
|
# the image actually contains the binary.
|
|
CRAWLER_CHROMIUM_BINARY=
|
|
|
|
# ----- Crawler TOR proxy + recircuit -----
|
|
# The compose stack ships a `tor` service (dockurr/tor) and defaults
|
|
# CRAWLER_PROXY to it, so by default all crawler traffic exits via the
|
|
# TOR network. To opt out, set CRAWLER_PROXY= (empty) AND
|
|
# CRAWLER_TOR_CONTROL_URL= (empty) below — the tor service can stay
|
|
# running, it just won't be used.
|
|
#
|
|
# Going through TOR adds latency to every fetch; image downloads in
|
|
# particular slow noticeably. The win is on sites that rate-limit or
|
|
# fingerprint by exit IP — NEWNYM recirculation makes a fresh exit
|
|
# cheap to reach for.
|
|
#
|
|
# CRAWLER_PROXY: SOCKS5(h) URL. Use `socks5h://` (not `socks5://`) so
|
|
# DNS resolution also goes through TOR, avoiding leaks via the host's
|
|
# resolver. Leave unset to talk to the upstream directly.
|
|
CRAWLER_PROXY=socks5h://tor:9050
|
|
# Control-port URL for SIGNAL NEWNYM ("get a fresh circuit"). Triggered
|
|
# automatically on bad pages (broken-page body, missing #logo) and on
|
|
# the Unauthenticated session probe outcome. Leave unset to disable
|
|
# the recircuit feature (the SOCKS proxy still works).
|
|
CRAWLER_TOR_CONTROL_URL=tcp://tor:9051
|
|
# Max NEWNYM-and-retry cycles per recircuit-eligible failure. Default 3.
|
|
CRAWLER_TOR_RECIRCUIT_MAX_ATTEMPTS=3
|
|
# Path to a tor cookie-auth file. Alternative to TOR_CONTROL_PASSWORD —
|
|
# the TorController prefers cookie when both are present. Useful when
|
|
# running against an externally managed tor daemon that uses
|
|
# `CookieAuthentication 1` instead of HashedControlPassword. Default unset.
|
|
CRAWLER_TOR_CONTROL_COOKIE_PATH=
|
|
|
|
# ----- TOR control-port password -----
|
|
# Shared between the bundled dockurr/tor service (which hashes it into
|
|
# its HashedControlPassword) and the backend's
|
|
# CRAWLER_TOR_CONTROL_PASSWORD. REQUIRED — docker-compose.yml fails
|
|
# fast if absent. Generate a strong random string; rotate by setting
|
|
# a new value and restarting both `tor` and `backend`.
|
|
#
|
|
# Operators running their own non-dockurr tor daemon with cookie-file
|
|
# auth can ignore this var and instead set
|
|
# CRAWLER_TOR_CONTROL_COOKIE_PATH on the backend — the TorController
|
|
# prefers cookie when both are present.
|
|
TOR_CONTROL_PASSWORD=change-me-to-a-strong-random-string
|
|
|
|
# ----- Frontend -----
|
|
# The frontend container runs SvelteKit's Node adapter on :3000 and
|
|
# proxies /api/* to BACKEND_URL via src/hooks.server.ts. In compose the
|
|
# default `http://backend:8080` reaches the backend service over the
|
|
# internal docker network. Override only if you're running the
|
|
# frontend container against a backend somewhere else.
|
|
BACKEND_URL=http://backend:8080
|
|
# Per-request wall-clock cap for the /api/* reverse proxy (milliseconds).
|
|
# Default 300000 (5 min) covers a typical 200 MiB chapter upload over
|
|
# 25 Mbps; raise for users on slower upstream links or lower if a
|
|
# tighter front proxy already bounds the request lifetime.
|
|
BACKEND_PROXY_TIMEOUT_MS=300000
|
|
|
|
# ----- Runtime settings (crawler + analysis) -----
|
|
# The crawler and analysis subsystems are configured at runtime from the
|
|
# `app_settings` table and edited live in the admin dashboard
|
|
# (Admin → Settings) — no restart needed. The CRAWLER_* / ANALYSIS_* env
|
|
# vars below act as the BOOT SEED: on first boot (when the row is absent)
|
|
# the env values populate the DB; thereafter the DB is the source of
|
|
# truth and env changes are ignored for those fields.
|
|
#
|
|
# Host/infra and secret fields stay env-ONLY (never persisted, shown
|
|
# read-only in the dashboard): the chromium binary/dir, browser mode/args,
|
|
# CRAWLER_PROXY, all CRAWLER_TOR_CONTROL_*, the cookie domain, and the
|
|
# analysis ANALYSIS_API_KEY. The important site levers (PRIVATE_MODE,
|
|
# ALLOW_SELF_REGISTER) also remain env-only by design.
|
|
#
|
|
# Crawler boot seeds (effective only on first boot, then editable from the
|
|
# dashboard):
|
|
# CRAWLER_START_URL The catalog listing URL the crawler walks. Without
|
|
# it, the cron is disabled (no daemon-driven sweeps;
|
|
# force-resync from the dashboard still works).
|
|
# CRAWLER_CDN_HOST Distinct host the source serves images from. Used to
|
|
# set a slower rate limit (CRAWLER_CDN_RATE_MS) and
|
|
# seed the download allowlist.
|
|
CRAWLER_START_URL=
|
|
CRAWLER_CDN_HOST=
|
|
# CRAWLER_DAEMON / CRAWLER_DAILY_AT / CRAWLER_TZ — the daemon sweep
|
|
# schedule. CRAWLER_DAEMON=false keeps the in-process daemon off; clicks
|
|
# in the dashboard still work. DAILY_AT is `HH:MM` in the chosen TZ;
|
|
# TZ is an IANA name (e.g. `Europe/Berlin`). Default: daemon on, runs
|
|
# at 00:00 UTC.
|
|
CRAWLER_DAEMON=true
|
|
CRAWLER_DAILY_AT=00:00
|
|
CRAWLER_TZ=UTC
|
|
#
|
|
# New analysis knobs (all optional, all seed defaults):
|
|
# ANALYSIS_TEMPERATURE Sampling temperature. Default 0 (deterministic).
|
|
# ANALYSIS_SYSTEM_PROMPT Override the single-call vision system prompt.
|
|
# ANALYSIS_OCR_PROMPT Override the tall-page OCR (pass A) prompt.
|
|
# ANALYSIS_GROUNDING_PROMPT Override the tags/scene/safety (pass B) prompt.
|
|
# Leave the prompt vars unset to use the built-in defaults (also editable,
|
|
# with a per-prompt "reset to default", in the dashboard).
|
|
|
|
# ----- Analysis (vision) — boot seeds + env-only secret -----
|
|
# The page-analysis worker (off by default) POSTs each page image at a
|
|
# Chat-Completions-compatible vision endpoint and persists OCR + tags.
|
|
# All fields except ANALYSIS_API_KEY are boot seeds for the app_settings
|
|
# row and switch to dashboard-editable once persisted.
|
|
#
|
|
# ANALYSIS_ENABLED Turn the worker on at first boot. Toggleable live in
|
|
# the dashboard. Default `false`.
|
|
ANALYSIS_ENABLED=false
|
|
# ANALYSIS_BACKEND Which engine the worker runs. env-ONLY (deploy-time).
|
|
# `ocr` (default) = the in-process ocrs engine: fast,
|
|
# CPU-only, text-only, ideal for a Pi.
|
|
# NOTE: the `vision` backend (local LLM at ANALYSIS_VISION_URL,
|
|
# full OCR + tags + scene + safety) is TEMPORARILY DISABLED —
|
|
# its code is kept intact but the worker forces OCR regardless,
|
|
# logging a warning if `vision` is requested. So setting
|
|
# `vision` here currently has no effect; the ANALYSIS_VISION_* /
|
|
# ANALYSIS_API_KEY knobs below are dormant until it's re-enabled.
|
|
ANALYSIS_BACKEND=ocr
|
|
# OCRS_DETECTION_MODEL / OCRS_RECOGNITION_MODEL Paths to the ocrs `.rten`
|
|
# text-detection / -recognition models (only read when ANALYSIS_BACKEND=ocr).
|
|
# The backend image bakes both into /models, so the defaults work unchanged;
|
|
# override only to point at custom-trained models.
|
|
OCRS_DETECTION_MODEL=/models/text-detection.rten
|
|
OCRS_RECOGNITION_MODEL=/models/text-recognition.rten
|
|
# ANALYSIS_VISION_URL /v1/chat/completions endpoint. Required when enabled.
|
|
# For the bundled vision container, use
|
|
# http://mangalord-vision:8000/v1/chat/completions.
|
|
ANALYSIS_VISION_URL=
|
|
# ANALYSIS_VISION_MODEL Model id the endpoint expects (`model:` field).
|
|
ANALYSIS_VISION_MODEL=
|
|
# ANALYSIS_WORKERS Concurrent vision dispatches. Default 1.
|
|
ANALYSIS_WORKERS=1
|
|
# ANALYSIS_JOB_TIMEOUT_SECS Hard upper bound per vision call before the
|
|
# job is acked failed (backoff). Default 600.
|
|
ANALYSIS_JOB_TIMEOUT_SECS=600
|
|
# ANALYSIS_API_KEY env-ONLY. Bearer-attached to every vision call; never
|
|
# persisted, never logged.
|
|
ANALYSIS_API_KEY=
|
|
# Analysis tuning — boot seeds (live-editable from the dashboard once
|
|
# persisted). Defaults are tuned for a 4 GiB-class local vision model;
|
|
# raise/lower as the model and budget allow.
|
|
# ANALYSIS_MAX_TOKENS Output-token cap per call. Default 4096.
|
|
# ANALYSIS_MAX_PIXELS Per-image pixel budget; we resize to fit.
|
|
# Default 1_000_000 (~1 MP).
|
|
# ANALYSIS_MIN_SLICE_HEIGHT Slice grid height for tall pages, px.
|
|
# Default 640.
|
|
# ANALYSIS_SLICE_OVERLAP Fractional overlap between adjacent slices
|
|
# so text on a cut survives. Default 0.12.
|
|
# ANALYSIS_TALL_ASPECT Slice only when `height/width` exceeds this;
|
|
# normal pages take a single call. Default 1.6.
|
|
# ANALYSIS_MAX_SLICES Hard cap on slices per page (coarsens
|
|
# beyond cap). Default 16.
|
|
# ANALYSIS_MAX_IMAGE_BYTES Per-image byte cap before downscale.
|
|
# Default 8388608 (8 MiB).
|
|
# ANALYSIS_OCR_MAX_DECODE_PIXELS Decompression-bomb guard for the OCR
|
|
# backend: hard cap on a page's *decoded* pixel
|
|
# count (encoded size is bounded separately).
|
|
# Default 100000000 (100 MP).
|
|
# ANALYSIS_RESPONSE_FORMAT `json_schema` | `json_object` | `none`.
|
|
# Default `json_schema`.
|
|
# ANALYSIS_FREQUENCY_PENALTY Discourages repetition loops. Default 0.3.
|
|
# ANALYSIS_TEMPERATURE Sampling temperature; 0 = deterministic.
|
|
# Default 0.
|
|
ANALYSIS_MAX_TOKENS=4096
|
|
ANALYSIS_MAX_PIXELS=1000000
|
|
ANALYSIS_MIN_SLICE_HEIGHT=640
|
|
ANALYSIS_SLICE_OVERLAP=0.12
|
|
ANALYSIS_TALL_ASPECT=1.6
|
|
ANALYSIS_MAX_SLICES=16
|
|
ANALYSIS_MAX_IMAGE_BYTES=8388608
|
|
ANALYSIS_OCR_MAX_DECODE_PIXELS=100000000
|
|
ANALYSIS_RESPONSE_FORMAT=json_schema
|
|
ANALYSIS_FREQUENCY_PENALTY=0.3
|
|
ANALYSIS_TEMPERATURE=0.0
|
|
|
|
# ----- Vision autoscaling (the `ai` compose profile) -----
|
|
# The mangalord-vision (llama.cpp) container pins ~4.4 GiB and has no
|
|
# idle-unload, so the `vision-manager` sidecar starts it on demand and
|
|
# idle-stops it. Bring the two helper containers up with:
|
|
# docker compose --profile ai up -d
|
|
# The vision container itself is NOT defined in compose — it lives elsewhere
|
|
# on the host and the manager drives it by name. See VISION-AUTOSCALE.md.
|
|
#
|
|
# ANALYSIS_VISION_HEALTH_URL — env-ONLY readiness gate for the analysis
|
|
# worker. When set, the worker refuses to lease a page until this answers
|
|
# 2xx, so the manager can stop vision mid-idle without jobs burning their
|
|
# retries / landing `failed` rows. MUST point at the same vision instance the
|
|
# worker analyses against. Leave empty to disable the gate (always-on setups).
|
|
ANALYSIS_VISION_HEALTH_URL=http://mangalord-vision:8000/health
|
|
#
|
|
# vision-manager knobs:
|
|
# VISION_MANAGER_DATABASE_URL — REQUIRED for the `ai` profile. A read-only
|
|
# role, NOT the backend creds: apply vision-manager/readonly-role.sql once,
|
|
# then set e.g.
|
|
# VISION_MANAGER_DATABASE_URL=postgres://vision_manager:<pw>@postgres:5432/mangalord
|
|
VISION_MANAGER_DATABASE_URL=
|
|
# VISION_CONTAINER Container name the manager starts/stops. Default mangalord-vision.
|
|
# VISION_HEALTH_URL /health URL the manager polls after start. Default http://mangalord-vision:8000/health.
|
|
# VISION_POLL_INTERVAL Backlog poll cadence, seconds. Default 20.
|
|
# VISION_STOP_DEBOUNCE Idle seconds before stopping vision. Default 600 (debounces bursty enqueues).
|
|
# VISION_START_HEALTH_TIMEOUT Max seconds to wait for /health 200 after start. Default 300 (cold model load).
|
|
# VISION_RESPECT_CRAWL_MUTEX 1 = don't start vision while a crawl runs (RAM mutex on the 8 GiB box). Default 1.
|
|
# VISION_MAX_UPTIME >0 = force-stop vision after N seconds running (backstop). Default 0 (disabled).
|
|
#
|
|
# Memory-pressure yield — the dynamic generalization of the crawl mutex. The
|
|
# manager reads host-wide used% (from /proc/meminfo MemAvailable) and stops a
|
|
# RUNNING vision when memory gets tight, so a spike elsewhere (a crawl, CI, a
|
|
# backend burst) can finish instead of the kernel OOM-killer shooting something
|
|
# stateful. Two watermarks give hysteresis; the cooldown prevents stop/restart
|
|
# thrash when vision itself is the hog. See VISION-MEMORY-YIELD.md.
|
|
# VISION_MEM_YIELD_ENABLED 1 = enable the gate, 0 = disable entirely. Default 1.
|
|
# VISION_MEM_HIGH_WATERMARK_PCT used% >= this → actively stop a running vision. Default 92.
|
|
# VISION_MEM_LOW_WATERMARK_PCT used% >= this → inhibit starts (keep HIGH - LOW >= ~10 so the band beats jitter). Default 80.
|
|
# VISION_MEM_YIELD_COOLDOWN Seconds after a pressure-stop during which restart is refused regardless of backlog. Default 300.
|
|
# VISION_MEM_POLL_INTERVAL Mem sub-poll cadence, seconds (<= VISION_POLL_INTERVAL); catches spikes between backlog ticks. Default 5.
|