Files
EventSnap/docker-compose.yml
MechaCat02 6fd75adb27 fix(ops): bound logs, drain ffmpeg's stderr, and stop three silent hangs
Six independent operational defects, none of which needed a new feature to fix.

Log rotation. Docker's json-file driver is unbounded by default, and those files
land on the HOST filesystem — outside every deploy.resources.limits in the compose
file, and on the same disk as postgres_data and media_data. A full disk stops
Postgres writing WAL, which takes the event down. Capped at 10m x 3 per service.

Log level. RUST_LOG was set in neither .env.example nor docker-compose.yml, so the
code fallback WAS the production level — and it was `debug`, with tower_http=debug
emitting a line per request and per response into that unrotated file. Now info,
with tower_http=warn to state that those spans are diagnostics, not an access log.

ffmpeg pipe deadlock. run_ffmpeg piped stdout and stderr and then called wait(),
which drains neither. Once the ~64 KiB pipe buffer filled, ffmpeg blocked writing
and wait() never returned — burning the full 120s timeout, twice per seek position,
three times per compression attempt. And the timeout is an Err, so the end state was
a soft-deleted upload: a guest's playable video destroyed by a poster-frame failure.
Now stdout is null (nothing ever read it) and stderr is drained by wait_with_output,
whose tail is logged on a non-zero exit. Note wait_with_output consumes the child, so
the old kill-on-timeout is gone; kill_on_drop(true) already covers it.

Readiness probe. /health never touched the pool, so the disk-full endgame above
stayed green all the way down. Adds /health/ready (SELECT 1 under 2s) as a SECOND
route — the compose healthcheck deliberately keeps pointing at /health, because
caddy gates its startup on it and a DB-dependent probe would turn a Postgres blip
into the reverse proxy refusing to start.

api.ts request timeout. The abort timer was cleared in a finally around fetch(),
which resolves on the response HEAD — leaving res.text() uncovered and no longer
abortable. An upstream that sends headers then stalls the body hung the call
forever. The timer now lives until the body is read, including the 204 path (which
otherwise leaked a live 20s timer per no-content request).

Upload XHR watchdog. The XHR had no timeout while processQueue held the
isProcessing latch across it; on a half-open socket neither error nor abort ever
fires, so the latch pinned and the queue wedged. Bounds SILENCE rather than total
duration — a 500 MB video over a venue uplink legitimately runs 30+ minutes while
making steady progress. Rejects as NetworkError, which is already the retryable
branch, so a stalled upload now recovers like any network blip.

IndexedDB failures. addToQueue called getDb() unguarded and handleSubmit had no
catch, so a private-mode refusal or a QuotaExceededError on a large blob left a
permanent "Wird hochgeladen…" spinner, no toast, and — for an in-app camera
capture — the only copy of the photo gone. Now reported as 'failed', which keeps
the staged files on screen and stays on the page.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 22:25:23 +02:00

150 lines
5.4 KiB
YAML

# Docker's default json-file driver is UNBOUNDED. Those files land on the HOST
# filesystem, outside every `deploy.resources.limits` below — so the container memory
# caps do nothing to stop them. On a single-box deployment the host disk is also where
# the postgres_data and media_data volumes live, and a full disk stops Postgres writing
# WAL, which takes the whole event down. 4 services x 3 x 10m caps the worst case at
# ~120 MiB. Applied to every service via the anchor; a new service must opt in too.
x-logging: &default-logging
driver: json-file
options:
max-size: "10m"
max-file: "3"
services:
db:
image: postgres:16-alpine
restart: unless-stopped
logging: *default-logging
env_file: .env
environment:
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: ${POSTGRES_DB}
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"]
interval: 5s
timeout: 5s
retries: 10
deploy:
resources:
limits:
# 1G, not 512M. DATABASE_MAX_CONNECTIONS defaults to 30 for a ~100-guest event
# (feed polling + SSE + uploads at once), and 30 backends plus Postgres 16's
# default shared_buffers leaves very little headroom at 512M. An OOM here does
# not degrade one feature — it takes the event down, because every request
# path touches the database. Memory is the cheaper knob than shrinking the
# pool back and reintroducing the queueing it was raised to fix.
#
# Raising DATABASE_MAX_CONNECTIONS further means raising this too.
memory: 1G
app:
build:
context: ./backend
dockerfile: Dockerfile
restart: unless-stopped
logging: *default-logging
env_file: .env
environment:
# Default to info. The code fallback in main.rs is info too, but a stock deploy
# sets RUST_LOG nowhere, and this is the layer an operator will actually find when
# they need to raise it for a single event ("RUST_LOG=eventsnap_backend=debug").
RUST_LOG: ${RUST_LOG:-info}
# Activates the production secret guard in config.rs — refuses to boot with
# placeholder JWT_SECRET / ADMIN_PASSWORD_HASH.
APP_ENV: production
# The media volume is mounted at /media (below), so the app MUST write there.
# Pin it here rather than trusting .env: if MEDIA_PATH in .env points elsewhere
# (e.g. a host path used for running the backend natively) the container can't
# create it and every upload 500s with EACCES. `environment` overrides `env_file`,
# so this is authoritative for the container.
MEDIA_PATH: /media
depends_on:
db:
condition: service_healthy
volumes:
- media_data:/media
# Export archives live OUTSIDE /media so the public media ServeDir can't
# serve them — downloads go only through the ticket-gated handler.
- exports_data:/exports
expose:
- "3000"
healthcheck:
# Use 127.0.0.1, NOT localhost: the app binds IPv4 (0.0.0.0) but `localhost`
# resolves to ::1 (IPv6) first inside the container, so a localhost probe gets
# "connection refused" and the container never turns healthy — which would leave
# Caddy (gated on `condition: service_healthy` below) blocked forever on boot.
test: ["CMD-SHELL", "wget -q -O- http://127.0.0.1:3000/health || exit 1"]
interval: 10s
timeout: 5s
retries: 5
start_period: 20s
deploy:
resources:
limits:
# Bounds a runaway ffmpeg transcode (large uploads, 2 workers) so it can't
# OOM the single box and take down Postgres.
memory: 1G
frontend:
build:
context: ./frontend
dockerfile: Dockerfile
restart: unless-stopped
logging: *default-logging
env_file: .env
environment:
# adapter-node behind Caddy TLS needs the public origin for CSRF checks on
# POST form actions — without it they fail only in production.
ORIGIN: "https://${DOMAIN}"
depends_on:
- app
expose:
- "3001"
healthcheck:
# 127.0.0.1, not localhost — see the app healthcheck note above (IPv4 bind vs
# ::1 resolution would leave this container permanently unhealthy).
test: ["CMD-SHELL", "wget -q -O- http://127.0.0.1:3001/ >/dev/null 2>&1 || exit 1"]
interval: 10s
timeout: 5s
retries: 5
start_period: 15s
deploy:
resources:
limits:
memory: 256M
caddy:
image: caddy:2-alpine
restart: unless-stopped
logging: *default-logging
environment:
# The Caddyfile's site address is `{$DOMAIN}`, read from THIS container's env.
# Without it, `{$DOMAIN}` expands to empty, the site block collapses, and Caddy
# serves nothing / fails to obtain a TLS cert. `env_file` alone wouldn't help —
# Caddy needs it in `environment`, and this keeps the Caddyfile the single source.
DOMAIN: ${DOMAIN}
ports:
- "80:80"
- "443:443"
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
- caddy_data:/data
depends_on:
app:
condition: service_healthy
frontend:
condition: service_healthy
deploy:
resources:
limits:
memory: 256M
volumes:
postgres_data:
media_data:
exports_data:
caddy_data: