Files
EventSnap/docker-compose.yml
MechaCat02 eb0e405562 fix: gate uploads on keepsake headroom, and close five unattended-event gaps
The box is 2 vCPU / 4 GB / 40 GB, not the 4 vCPU / 8 GB / 80 GB that the audit,
the committed comments and README's sizing section all assumed. That correction
is what the first change is about; the rest are the remaining pre-event items.

THE ARCHIVE COULD BECOME UNBUILDABLE WHILE UPLOADS KEPT SUCCEEDING

`required_free_bytes` is `media × 1.1 × 2` — the ZIP and the HTML viewer are each
gallery-sized — and the export preflight also wants DISK_RESERVE_BYTES on top. The
upload gate, though, only refused below a FLAT 10 GB reserve. On 40 GB that let
uploads run to ~25 GB of media while a release needed `2.2 × 25 + 10` = 65 GB free.
Every upload in that band succeeded and the keepsake could then never be built: the
product's entire promise, failing silently at the end of the night with nobody there.

The gate now enforces the invariant that actually matters — never accept an upload
that would make the keepsake unbuildable — sharing `required_free_bytes` with the
preflight so the two cannot drift into disagreeing about the same question. Uploads
stop at ~8 GB of media on this disk, with a German message naming the cause.
Refusing the 1001st photo beats losing all 1000.

`media_total.rs` backs it: SUM(user.total_upload_bytes) over ~100 rows, cached 5s,
rather than `estimate_export_bytes`'s join across every upload. It counts hidden and
banned users' bytes, which the export excludes — skew in the SAFE direction, so the
gate closes marginally early rather than late. Fails open on a query error.

A test pins the gate against the preflight across the whole gallery-size range, and
a second asserts the per-user floor alone would over-commit the volume — i.e. that
the global gate is what must bind.

THE WATCHDOG ABORTED HEALTHY UPLOADS EVERY TIME A PHONE WAS POCKETED

`Date.now()` advances while a backgrounded phone is frozen but `setInterval` does
not, so the first tick after a screen lock read the whole sleep as silence and
aborted — re-sending a video from byte zero and burning one of five PERMANENT
auto-attempts. The interval is now its own suspension detector: a tick that arrives
125s late for a 5s schedule credits that window back, because a period the watchdog
could not observe is not evidence of silence.

Chosen over a `visibilitychange` listener, which only covers causes that fire that
event — a throttled-but-visible tab, a closed lid and an occluded window all freeze
timers without one — and which would have needed module state, an SSR guard and a
teardown for strictly less coverage. `performance.now()` was rejected because Safari
pauses it across system sleep on some paths and Chrome does not.

The credit buys one fresh window, not immunity: a socket iOS reaped while
backgrounded still aborts ~90s after resume rather than hanging for `xhr.timeout`
(5-60 min) with the queue's `processing` latch held.

Two latent leaks found while in there: `xhr.abort()` on a request already in
readyState DONE emits no `abort` event, so `settle()` never ran and the interval
re-aborted every 5s forever while `activeUploads` kept a stale entry (the ✕ button
silently stopped working); and a synchronous throw from `xhr.send` — a blob whose
backing store the OS purged — leaked the same way. Both closed.

OKLCH MADE THE DELETE BUTTON INVISIBLE ON SAMSUNG'S DEFAULT BROWSER

red/amber/green were never in the @theme block and fell through to Tailwind v4's
`oklch()` defaults, which Safari <15.4, Chrome <111 and Samsung Internet <22 cannot
parse: `var(--color-red-600)` is then invalid at computed-value time, `background-color`
falls back to transparent, and `.btn-danger` renders white text on nothing. Pinned to
Tailwind's own defaults gamut-mapped to sRGB by Lightning CSS — the converter already
in this pipeline — so modern browsers render exactly what they render today. Verified
against seven hex fallbacks it had already emitted for the /alpha forms. rose and teal
(avatar chips) had the same leak. The app CSS goes from 40 oklch declarations to 0.

Also fixes `--color-purple-950`, which was simply missing: `dark:bg-purple-950/50` on
the host dashboard was rendering default violet on EVERY browser, off-brand.

The keepsake viewer only picks this up on a rebuild, so its committed artefact is
rebuilt here too — still single-file, still zero external references.

A BRICKED BOOT LOOKED LIKE A SPINNER FOREVER

With `ssr = false` the page is empty until the bundle mounts, so a chunk 404 after a
redeploy or a dead uplink left the guest on the boot spinner with no message, no
reload control, and in a standalone PWA no URL bar. A 15s timeout in the existing
nonce'd IIFE (no CSP change) swaps in German copy and a reload button. Deliberately a
timeout rather than feature detection: a SyntaxError in the bundle is invisible to any
capability check. Plus a <noscript>, since there was nothing at all to see without JS.

EVERY 4xx WAS INVISIBLE AT ANY LOG LEVEL

tower_http counts 4xx as a success, so it logs at DEBUG while production runs at info.
If guests spend the evening hitting 429s or 413s, the post-event logs said nothing.
Now one WARN per client error; 5xx excluded because Internal already logs its source
chain and the pool-exhaustion 503 logs at construction.

A DEAD FRONTEND SERVED A BLANK 502

`handle_errors 5xx` with an inline German page (the caddy service mounts only the
Caddyfile, so there is no volume to ship a static file through). Verified empirically
against this config, not from documentation: an upstream 404 through `reverse_proxy`
still arrives as untouched `application/json`, and only a dial failure renders the
page. That mattered — the keepsake download navigates a hidden iframe and DEPENDS on a
real 404/429 arriving, and swallowing those would have been worse than the blank 502.

CONFIG CORRECTIONS FOR THE REAL HARDWARE

DATABASE_MAX_CONNECTIONS 30 → 15: sized to 2 vCPU rather than to the guest count.
Since migration 024 a feed page costs well under a millisecond, so connections are no
longer spent waiting, and 30 backends crowd the db container's 1 GB on a 4 GB host.
COMPRESSION_WORKER_CONCURRENCY stays at 2 — the merged heavy-image permit already
serialises anything over 150 MiB, so the "two 48 MP photos" worst case that number was
sized against is unreachable; dropping to 1 would halve light-path throughput and push
more feed tiles onto full-size originals. README's sizing section rewritten for the
actual disk.

Verified: 149/149 backend tests against a live Postgres, clippy clean, 57/57 vitest,
svelte-check 0 errors, eslint clean, vite build, export-viewer rebuild, caddy validate,
compose YAML parse.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-08 22:07:56 +02:00

187 lines
8.4 KiB
YAML

# Docker's default json-file driver has NO rotation at all, and every container writes to the
# same filesystem as postgres_data, media_data and exports_data. Filling that filesystem does
# not degrade one subsystem — Postgres stops being able to write and the whole event goes down
# (see README "Sizing the disk"). This caps logs at 30 MB per service, permanently.
#
# Paired with RUST_LOG in .env: without it the app falls back to `eventsnap_backend=debug,
# tower_http=debug` (main.rs), which is a debug line per HTTP request including every preview.
x-logging: &default-logging
driver: json-file
options:
max-size: "10m"
max-file: "3"
services:
db:
image: postgres:16-alpine
restart: unless-stopped
logging: *default-logging
env_file: .env
environment:
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: ${POSTGRES_DB}
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"]
interval: 5s
timeout: 5s
retries: 10
deploy:
resources:
limits:
# 1G, not 512M. Postgres 16's default shared_buffers plus a pool of backends
# leaves very little headroom at 512M, and an OOM here does not degrade one
# feature — it takes the event down, because every request path touches the
# database.
#
# `.env.example` now sets DATABASE_MAX_CONNECTIONS=15, sized to the 2 vCPU this
# box has rather than to the guest count: since migration 024 replaced the feed
# view's GROUP BY with scalar subqueries, a feed page costs well under a
# millisecond, so connections are no longer spent waiting. Raising it back
# toward 30 means raising this limit with it.
memory: 1G
app:
# Production PULLS a prebuilt image; it never compiles. A release build of this crate is
# fat-LTO over 427 dependencies (see backend/Cargo.toml [profile.release]) and peaks well
# above the RAM a 4 GB box has spare with the stack running — and a rollback would be a
# second build under pressure. Images are built on a workstation and pushed; see
# docker-compose.build.yml and DEPLOYMENT_RUNBOOK.md.
#
# There is deliberately NO `build:` key here: without one, a wrong tag fails instantly with
# "manifest unknown" instead of silently starting a 45-minute compile on the event server.
# The `:?` form fails loudly on an unset variable rather than resolving to an empty tag.
image: registry.mc02.dev/eventsnap/app:${EVENTSNAP_VERSION:?set EVENTSNAP_VERSION in .env}
restart: unless-stopped
logging: *default-logging
env_file: .env
environment:
# Default to info. Without this the code fallback in main.rs applies, which is
# `eventsnap_backend=debug,tower_http=debug` — a line per HTTP request AND per
# response, including every preview fetch, for a multi-day run. The x-logging cap
# above bounds the disk cost but not the CPU/IO one.
#
# Set here rather than only in `.env` because a stock deploy sets RUST_LOG nowhere,
# and this is the layer an operator will actually find when they need to raise it
# for a single event (`RUST_LOG=eventsnap_backend=debug docker compose up -d app`).
RUST_LOG: ${RUST_LOG:-info}
# Activates the production secret guard in config.rs — refuses to boot with
# placeholder JWT_SECRET / ADMIN_PASSWORD_HASH.
APP_ENV: production
# Pinned beside MEDIA_PATH for the same reason, and because nothing validates it:
# config.rs defaults it to /exports but never checks that it is a mount or that it
# differs from media_path. A stray EXPORT_PATH in .env builds the keepsake into the
# container's writable layer, where it passes every health check and disk preflight
# and then evaporates on the next `up -d`.
EXPORT_PATH: /exports
# The media volume is mounted at /media (below), so the app MUST write there.
# Pin it here rather than trusting .env: if MEDIA_PATH in .env points elsewhere
# (e.g. a host path used for running the backend natively) the container can't
# create it and every upload 500s with EACCES. `environment` overrides `env_file`,
# so this is authoritative for the container.
MEDIA_PATH: /media
# Pinned for the same reason as MEDIA_PATH: `environment` beats `env_file`, so this cannot
# be lost by an operator who copies `.env.example` and edits only the secrets — which is
# the likely path, and `.env.example` ships the generic default of `true`.
#
# This is a product decision for this event, not a technical one: guests should be present
# at the party, not in a comment thread. Likes and captions stay on and are unaffected.
# Boot-time only, so changing it means `docker compose up -d`, not an admin toggle.
# To re-enable comments, delete this line and set COMMENTS_ENABLED in .env.
COMMENTS_ENABLED: "false"
depends_on:
db:
condition: service_healthy
# Longer than the app's own 10s shutdown backstop (main.rs SHUTDOWN_GRACE), because Docker's
# default stop timeout is ALSO 10s — so a redeploy raced the graceful drain and could SIGKILL
# the process at the exact moment it was finishing, truncating the in-flight upload the
# graceful shutdown exists to protect. The app always exits well before 20s.
stop_grace_period: 20s
volumes:
- media_data:/media
# Export archives live OUTSIDE /media so the public media ServeDir can't
# serve them — downloads go only through the ticket-gated handler.
- exports_data:/exports
expose:
- "3000"
healthcheck:
# Use 127.0.0.1, NOT localhost: the app binds IPv4 (0.0.0.0) but `localhost`
# resolves to ::1 (IPv6) first inside the container, so a localhost probe gets
# "connection refused" and the container never turns healthy — which would leave
# Caddy (gated on `condition: service_healthy` below) blocked forever on boot.
test: ["CMD-SHELL", "wget -q -O- http://127.0.0.1:3000/health || exit 1"]
interval: 10s
timeout: 5s
retries: 5
start_period: 20s
deploy:
resources:
limits:
# Bounds a runaway ffmpeg transcode (large uploads, 2 workers) so it can't
# OOM the single box and take down Postgres.
memory: 1G
frontend:
# Pulled, not built — see the note on `app` above.
image: registry.mc02.dev/eventsnap/frontend:${EVENTSNAP_VERSION:?set EVENTSNAP_VERSION in .env}
restart: unless-stopped
logging: *default-logging
env_file: .env
environment:
# adapter-node behind Caddy TLS needs the public origin for CSRF checks on
# POST form actions — without it they fail only in production.
# `:?` for the same reason EVENTSNAP_VERSION uses it. A blank DOMAIN doesn't fail — it
# produces `https://` here and collapses the Caddyfile's site block below, so the stack
# comes up with no TLS and no site and the only symptom is a browser error.
ORIGIN: "https://${DOMAIN:?set DOMAIN in .env}"
depends_on:
- app
expose:
- "3001"
healthcheck:
# 127.0.0.1, not localhost — see the app healthcheck note above (IPv4 bind vs
# ::1 resolution would leave this container permanently unhealthy).
test: ["CMD-SHELL", "wget -q -O- http://127.0.0.1:3001/ >/dev/null 2>&1 || exit 1"]
interval: 10s
timeout: 5s
retries: 5
start_period: 15s
deploy:
resources:
limits:
memory: 256M
caddy:
image: caddy:2-alpine
restart: unless-stopped
logging: *default-logging
environment:
# The Caddyfile's site address is `{$DOMAIN}`, read from THIS container's env.
# Without it, `{$DOMAIN}` expands to empty, the site block collapses, and Caddy
# serves nothing / fails to obtain a TLS cert. `env_file` alone wouldn't help —
# Caddy needs it in `environment`, and this keeps the Caddyfile the single source.
DOMAIN: ${DOMAIN:?set DOMAIN in .env}
ports:
- "80:80"
- "443:443"
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
- caddy_data:/data
depends_on:
app:
condition: service_healthy
frontend:
condition: service_healthy
deploy:
resources:
limits:
memory: 256M
volumes:
postgres_data:
media_data:
exports_data:
caddy_data: