fix(ops): bound logs, drain ffmpeg's stderr, and stop three silent hangs

Six independent operational defects, none of which needed a new feature to fix.

Log rotation. Docker's json-file driver is unbounded by default, and those files
land on the HOST filesystem — outside every deploy.resources.limits in the compose
file, and on the same disk as postgres_data and media_data. A full disk stops
Postgres writing WAL, which takes the event down. Capped at 10m x 3 per service.

Log level. RUST_LOG was set in neither .env.example nor docker-compose.yml, so the
code fallback WAS the production level — and it was `debug`, with tower_http=debug
emitting a line per request and per response into that unrotated file. Now info,
with tower_http=warn to state that those spans are diagnostics, not an access log.

ffmpeg pipe deadlock. run_ffmpeg piped stdout and stderr and then called wait(),
which drains neither. Once the ~64 KiB pipe buffer filled, ffmpeg blocked writing
and wait() never returned — burning the full 120s timeout, twice per seek position,
three times per compression attempt. And the timeout is an Err, so the end state was
a soft-deleted upload: a guest's playable video destroyed by a poster-frame failure.
Now stdout is null (nothing ever read it) and stderr is drained by wait_with_output,
whose tail is logged on a non-zero exit. Note wait_with_output consumes the child, so
the old kill-on-timeout is gone; kill_on_drop(true) already covers it.

Readiness probe. /health never touched the pool, so the disk-full endgame above
stayed green all the way down. Adds /health/ready (SELECT 1 under 2s) as a SECOND
route — the compose healthcheck deliberately keeps pointing at /health, because
caddy gates its startup on it and a DB-dependent probe would turn a Postgres blip
into the reverse proxy refusing to start.

api.ts request timeout. The abort timer was cleared in a finally around fetch(),
which resolves on the response HEAD — leaving res.text() uncovered and no longer
abortable. An upstream that sends headers then stalls the body hung the call
forever. The timer now lives until the body is read, including the 204 path (which
otherwise leaked a live 20s timer per no-content request).

Upload XHR watchdog. The XHR had no timeout while processQueue held the
isProcessing latch across it; on a half-open socket neither error nor abort ever
fires, so the latch pinned and the queue wedged. Bounds SILENCE rather than total
duration — a 500 MB video over a venue uplink legitimately runs 30+ minutes while
making steady progress. Rejects as NetworkError, which is already the retryable
branch, so a stalled upload now recovers like any network blip.

IndexedDB failures. addToQueue called getDb() unguarded and handleSubmit had no
catch, so a private-mode refusal or a QuotaExceededError on a large blob left a
permanent "Wird hochgeladen…" spinner, no toast, and — for an in-app camera
capture — the only copy of the photo gone. Now reported as 'failed', which keeps
the staged files on screen and stays on the page.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-07-31 22:25:23 +02:00
parent 7d0334bf22
commit 6fd75adb27
11 changed files with 394 additions and 32 deletions

View File

@@ -1,7 +1,20 @@
# Docker's default json-file driver is UNBOUNDED. Those files land on the HOST
# filesystem, outside every `deploy.resources.limits` below — so the container memory
# caps do nothing to stop them. On a single-box deployment the host disk is also where
# the postgres_data and media_data volumes live, and a full disk stops Postgres writing
# WAL, which takes the whole event down. 4 services x 3 x 10m caps the worst case at
# ~120 MiB. Applied to every service via the anchor; a new service must opt in too.
x-logging: &default-logging
driver: json-file
options:
max-size: "10m"
max-file: "3"
services:
db:
image: postgres:16-alpine
restart: unless-stopped
logging: *default-logging
env_file: .env
environment:
POSTGRES_USER: ${POSTGRES_USER}
@@ -32,8 +45,13 @@ services:
context: ./backend
dockerfile: Dockerfile
restart: unless-stopped
logging: *default-logging
env_file: .env
environment:
# Default to info. The code fallback in main.rs is info too, but a stock deploy
# sets RUST_LOG nowhere, and this is the layer an operator will actually find when
# they need to raise it for a single event ("RUST_LOG=eventsnap_backend=debug").
RUST_LOG: ${RUST_LOG:-info}
# Activates the production secret guard in config.rs — refuses to boot with
# placeholder JWT_SECRET / ADMIN_PASSWORD_HASH.
APP_ENV: production
@@ -75,6 +93,7 @@ services:
context: ./frontend
dockerfile: Dockerfile
restart: unless-stopped
logging: *default-logging
env_file: .env
environment:
# adapter-node behind Caddy TLS needs the public origin for CSRF checks on
@@ -100,6 +119,7 @@ services:
caddy:
image: caddy:2-alpine
restart: unless-stopped
logging: *default-logging
environment:
# The Caddyfile's site address is `{$DOMAIN}`, read from THIS container's env.
# Without it, `{$DOMAIN}` expands to empty, the site block collapses, and Caddy