fabi/Mangalord
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(ops): vision-manager sidecar to autoscale the llama.cpp container

Browse Source 
Add a tiny privileged sidecar (vision-manager/) that polls the analysis
backlog in Postgres and starts/stops the mangalord-vision container by name:
start when analyze_page jobs are pending, stop after STOP_DEBOUNCE idle.
It is the single owner of the vision lifecycle and the only component with
Docker access — scoped through tecnativa/docker-socket-proxy (CONTAINERS+POST
only) on an internal-only network, so the internet-facing backend never
touches the socket.

Both helpers sit behind `profiles: [ai]` (vanilla `compose up` is
unaffected). The manager debounces the stop, gates the first request on
GET /health==200 after a cold start, honours a crawl RAM-mutex on the 8 GiB
box, and owns its own idle timer (leak-safe if the backend dies). Backlog
query uses the real schema (state + payload->>'kind'); a read-only DB role
(readonly-role.sql) keeps it off the backend creds.

Bump 0.80.0 -> 0.81.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-06-14 15:02:46 +02:00
parent b86aa80c87
commit 64a9dceb67
9 changed files with 419 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

29
.env.example
View File

@@ -189,3 +189,32 @@ BACKEND_PROXY_TIMEOUT_MS=300000
# ANALYSIS_GROUNDING_PROMPT Override the tags/scene/safety (pass B) prompt.
# Leave the prompt vars unset to use the built-in defaults (also editable,
# with a per-prompt "reset to default", in the dashboard).
# ----- Vision autoscaling (the `ai` compose profile) -----
# The mangalord-vision (llama.cpp) container pins ~4.4 GiB and has no
# idle-unload, so the `vision-manager` sidecar starts it on demand and
# idle-stops it. Bring the two helper containers up with:
# docker compose --profile ai up -d
# The vision container itself is NOT defined in compose — it lives elsewhere
# on the host and the manager drives it by name. See VISION-AUTOSCALE.md.
#
# ANALYSIS_VISION_HEALTH_URL — env-ONLY readiness gate for the analysis
# worker. When set, the worker refuses to lease a page until this answers
# 2xx, so the manager can stop vision mid-idle without jobs burning their
# retries / landing `failed` rows. MUST point at the same vision instance the
# worker analyses against. Leave empty to disable the gate (always-on setups).
ANALYSIS_VISION_HEALTH_URL=http://mangalord-vision:8000/health
#
# vision-manager knobs:
# VISION_MANAGER_DATABASE_URL — REQUIRED for the `ai` profile. A read-only
# role, NOT the backend creds: apply vision-manager/readonly-role.sql once,
# then set e.g.
# VISION_MANAGER_DATABASE_URL=postgres://vision_manager:<pw>@postgres:5432/mangalord
VISION_MANAGER_DATABASE_URL=
# VISION_CONTAINER Container name the manager starts/stops. Default mangalord-vision.
# VISION_HEALTH_URL /health URL the manager polls after start. Default http://mangalord-vision:8000/health.
# VISION_POLL_INTERVAL Backlog poll cadence, seconds. Default 20.
# VISION_STOP_DEBOUNCE Idle seconds before stopping vision. Default 600 (debounces bursty enqueues).
# VISION_START_HEALTH_TIMEOUT Max seconds to wait for /health 200 after start. Default 300 (cold model load).
# VISION_RESPECT_CRAWL_MUTEX 1 = don't start vision while a crawl runs (RAM mutex on the 8 GiB box). Default 1.
# VISION_MAX_UPTIME >0 = force-stop vision after N seconds running (backstop). Default 0 (disabled).