Two independent lines of production hardening diverged at7d0334band attacked overlapping problems. Neither was a superset, so this is a merge of substance rather than a fast-forward: every conflict was resolved on the merits, and the losing side's intent was re-checked against the winner rather than assumed. MIGRATIONS. The branch's 021/022/023 collided with main's already-DEPLOYED 021_hashtag_counts_respect_bans and 022_client_upload_idempotency. Renumbered to 023/024/025 in a prior commit — main's versions are applied in production, so their version numbers are immutable and the branch's had to move. Verified by running the full sqlx::test suite, which applies the whole chain from scratch. RESOLVED IN MAIN'S FAVOUR (the branch would have regressed these): * upload-queue.ts wholesale — the branch's copy has ZERO client_upload_id references, so taking it would have silently destroyed end-to-end upload idempotency, the one thing standing between a lost response and a duplicate photo charged twice against the guest's quota. * maintenance.rs supervisor — the branch replaced it with a bare tokio::spawn, where one panic silently stops session pruning, media reclaim, the temp sweep and both HashMap prunes, permanently and with no log line. * The decode-budget probe on spawn_blocking, not inline on the async runtime. * feed/+page.svelte's 8s debounce + jitter + max-wait + hidden-tab deferral, against the branch's naive 800ms — at 100 guests the branch's version walks straight into the per-user feed rate limit. * db.rs pool tuning, /uploaders, and the docker-compose deployment story. * ONE /health, still DB-backed. The branch's split (dependency-free liveness + DB-backed readiness) is defensible, but a constant-"ok" /health is the exact defectfaea555fixed and verified live, its motive (Caddy's boot gate) is already covered by app depends_on db: service_healthy, and the two handlers were the same SELECT 1 under two names. TAKEN FROM THE BRANCH: * The large-PNG OOM guard and its bounded-retry counter (023). Together these turn a single upload that can OOM-kill a 1G container into a bounded failure instead of an infinite restart loop under `restart: unless-stopped`. * 024_feed_scalar_counts — the feed no longer aggregates the whole event per page. Pure SQL; column names, order and types are unchanged by design. * The admin-lockout fix: look the admin up BY ROLE, never by name. 025 also frees any guest already squatting on a reserved name. * PIN lockout tier ordering, bounded caption/hashtag reads, SSE ticket caps, PoolTimedOut -> 503 + Retry-After, and the ffmpeg stderr drain. * backfill_video_posters, which main lacked entirely. * TempFileGuard, plus sweep_orphan_originals wired into main's SUPERVISED loop (not the branch's bare one) — it reclaims final-named originals whose commit never happened, a class main's .tmp-only sweep structurally cannot see. * shouldAbortForStall, hand-ported into main's upload-queue.ts since that file was resolved to main. Widens the watchdog at loadend instead of disarming it, bounding a half-open socket at 2 minutes rather than handing the window to xhr.timeout (5-60 min) with the whole queue's `processing` latch held. ALSO: RUST_LOG and EXPORT_PATH pinned in compose. The code fallback was `debug` (a line per request, all night) and EXPORT_PATH was the one path with a mount-shaped default that nothing validated. Verified: cargo check --all-targets, cargo clippy (clean), 144/144 backend tests against a live Postgres including upload_idempotency and upload_concurrency, 51/51 vitest, svelte-check 0 errors, eslint clean, vite build. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
185 lines
8.4 KiB
YAML
185 lines
8.4 KiB
YAML
# Docker's default json-file driver has NO rotation at all, and every container writes to the
|
|
# same filesystem as postgres_data, media_data and exports_data. Filling that filesystem does
|
|
# not degrade one subsystem — Postgres stops being able to write and the whole event goes down
|
|
# (see README "Sizing the disk"). This caps logs at 30 MB per service, permanently.
|
|
#
|
|
# Paired with RUST_LOG in .env: without it the app falls back to `eventsnap_backend=debug,
|
|
# tower_http=debug` (main.rs), which is a debug line per HTTP request including every preview.
|
|
x-logging: &default-logging
|
|
driver: json-file
|
|
options:
|
|
max-size: "10m"
|
|
max-file: "3"
|
|
|
|
services:
|
|
db:
|
|
image: postgres:16-alpine
|
|
restart: unless-stopped
|
|
logging: *default-logging
|
|
env_file: .env
|
|
environment:
|
|
POSTGRES_USER: ${POSTGRES_USER}
|
|
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
|
|
POSTGRES_DB: ${POSTGRES_DB}
|
|
volumes:
|
|
- postgres_data:/var/lib/postgresql/data
|
|
healthcheck:
|
|
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"]
|
|
interval: 5s
|
|
timeout: 5s
|
|
retries: 10
|
|
deploy:
|
|
resources:
|
|
limits:
|
|
# 1G, not 512M. DATABASE_MAX_CONNECTIONS defaults to 30 for a ~100-guest event
|
|
# (feed polling + SSE + uploads at once), and 30 backends plus Postgres 16's
|
|
# default shared_buffers leaves very little headroom at 512M. An OOM here does
|
|
# not degrade one feature — it takes the event down, because every request
|
|
# path touches the database. Memory is the cheaper knob than shrinking the
|
|
# pool back and reintroducing the queueing it was raised to fix.
|
|
#
|
|
# Raising DATABASE_MAX_CONNECTIONS further means raising this too.
|
|
memory: 1G
|
|
|
|
app:
|
|
# Production PULLS a prebuilt image; it never compiles. A release build of this crate is
|
|
# fat-LTO over 427 dependencies (see backend/Cargo.toml [profile.release]) and peaks well
|
|
# above the RAM a 4 GB box has spare with the stack running — and a rollback would be a
|
|
# second build under pressure. Images are built on a workstation and pushed; see
|
|
# docker-compose.build.yml and DEPLOYMENT_RUNBOOK.md.
|
|
#
|
|
# There is deliberately NO `build:` key here: without one, a wrong tag fails instantly with
|
|
# "manifest unknown" instead of silently starting a 45-minute compile on the event server.
|
|
# The `:?` form fails loudly on an unset variable rather than resolving to an empty tag.
|
|
image: registry.mc02.dev/eventsnap/app:${EVENTSNAP_VERSION:?set EVENTSNAP_VERSION in .env}
|
|
restart: unless-stopped
|
|
logging: *default-logging
|
|
env_file: .env
|
|
environment:
|
|
# Default to info. Without this the code fallback in main.rs applies, which is
|
|
# `eventsnap_backend=debug,tower_http=debug` — a line per HTTP request AND per
|
|
# response, including every preview fetch, for a multi-day run. The x-logging cap
|
|
# above bounds the disk cost but not the CPU/IO one.
|
|
#
|
|
# Set here rather than only in `.env` because a stock deploy sets RUST_LOG nowhere,
|
|
# and this is the layer an operator will actually find when they need to raise it
|
|
# for a single event (`RUST_LOG=eventsnap_backend=debug docker compose up -d app`).
|
|
RUST_LOG: ${RUST_LOG:-info}
|
|
# Activates the production secret guard in config.rs — refuses to boot with
|
|
# placeholder JWT_SECRET / ADMIN_PASSWORD_HASH.
|
|
APP_ENV: production
|
|
# Pinned beside MEDIA_PATH for the same reason, and because nothing validates it:
|
|
# config.rs defaults it to /exports but never checks that it is a mount or that it
|
|
# differs from media_path. A stray EXPORT_PATH in .env builds the keepsake into the
|
|
# container's writable layer, where it passes every health check and disk preflight
|
|
# and then evaporates on the next `up -d`.
|
|
EXPORT_PATH: /exports
|
|
# The media volume is mounted at /media (below), so the app MUST write there.
|
|
# Pin it here rather than trusting .env: if MEDIA_PATH in .env points elsewhere
|
|
# (e.g. a host path used for running the backend natively) the container can't
|
|
# create it and every upload 500s with EACCES. `environment` overrides `env_file`,
|
|
# so this is authoritative for the container.
|
|
MEDIA_PATH: /media
|
|
# Pinned for the same reason as MEDIA_PATH: `environment` beats `env_file`, so this cannot
|
|
# be lost by an operator who copies `.env.example` and edits only the secrets — which is
|
|
# the likely path, and `.env.example` ships the generic default of `true`.
|
|
#
|
|
# This is a product decision for this event, not a technical one: guests should be present
|
|
# at the party, not in a comment thread. Likes and captions stay on and are unaffected.
|
|
# Boot-time only, so changing it means `docker compose up -d`, not an admin toggle.
|
|
# To re-enable comments, delete this line and set COMMENTS_ENABLED in .env.
|
|
COMMENTS_ENABLED: "false"
|
|
depends_on:
|
|
db:
|
|
condition: service_healthy
|
|
# Longer than the app's own 10s shutdown backstop (main.rs SHUTDOWN_GRACE), because Docker's
|
|
# default stop timeout is ALSO 10s — so a redeploy raced the graceful drain and could SIGKILL
|
|
# the process at the exact moment it was finishing, truncating the in-flight upload the
|
|
# graceful shutdown exists to protect. The app always exits well before 20s.
|
|
stop_grace_period: 20s
|
|
volumes:
|
|
- media_data:/media
|
|
# Export archives live OUTSIDE /media so the public media ServeDir can't
|
|
# serve them — downloads go only through the ticket-gated handler.
|
|
- exports_data:/exports
|
|
expose:
|
|
- "3000"
|
|
healthcheck:
|
|
# Use 127.0.0.1, NOT localhost: the app binds IPv4 (0.0.0.0) but `localhost`
|
|
# resolves to ::1 (IPv6) first inside the container, so a localhost probe gets
|
|
# "connection refused" and the container never turns healthy — which would leave
|
|
# Caddy (gated on `condition: service_healthy` below) blocked forever on boot.
|
|
test: ["CMD-SHELL", "wget -q -O- http://127.0.0.1:3000/health || exit 1"]
|
|
interval: 10s
|
|
timeout: 5s
|
|
retries: 5
|
|
start_period: 20s
|
|
deploy:
|
|
resources:
|
|
limits:
|
|
# Bounds a runaway ffmpeg transcode (large uploads, 2 workers) so it can't
|
|
# OOM the single box and take down Postgres.
|
|
memory: 1G
|
|
|
|
frontend:
|
|
# Pulled, not built — see the note on `app` above.
|
|
image: registry.mc02.dev/eventsnap/frontend:${EVENTSNAP_VERSION:?set EVENTSNAP_VERSION in .env}
|
|
restart: unless-stopped
|
|
logging: *default-logging
|
|
env_file: .env
|
|
environment:
|
|
# adapter-node behind Caddy TLS needs the public origin for CSRF checks on
|
|
# POST form actions — without it they fail only in production.
|
|
# `:?` for the same reason EVENTSNAP_VERSION uses it. A blank DOMAIN doesn't fail — it
|
|
# produces `https://` here and collapses the Caddyfile's site block below, so the stack
|
|
# comes up with no TLS and no site and the only symptom is a browser error.
|
|
ORIGIN: "https://${DOMAIN:?set DOMAIN in .env}"
|
|
depends_on:
|
|
- app
|
|
expose:
|
|
- "3001"
|
|
healthcheck:
|
|
# 127.0.0.1, not localhost — see the app healthcheck note above (IPv4 bind vs
|
|
# ::1 resolution would leave this container permanently unhealthy).
|
|
test: ["CMD-SHELL", "wget -q -O- http://127.0.0.1:3001/ >/dev/null 2>&1 || exit 1"]
|
|
interval: 10s
|
|
timeout: 5s
|
|
retries: 5
|
|
start_period: 15s
|
|
deploy:
|
|
resources:
|
|
limits:
|
|
memory: 256M
|
|
|
|
caddy:
|
|
image: caddy:2-alpine
|
|
restart: unless-stopped
|
|
logging: *default-logging
|
|
environment:
|
|
# The Caddyfile's site address is `{$DOMAIN}`, read from THIS container's env.
|
|
# Without it, `{$DOMAIN}` expands to empty, the site block collapses, and Caddy
|
|
# serves nothing / fails to obtain a TLS cert. `env_file` alone wouldn't help —
|
|
# Caddy needs it in `environment`, and this keeps the Caddyfile the single source.
|
|
DOMAIN: ${DOMAIN:?set DOMAIN in .env}
|
|
ports:
|
|
- "80:80"
|
|
- "443:443"
|
|
volumes:
|
|
- ./Caddyfile:/etc/caddy/Caddyfile:ro
|
|
- caddy_data:/data
|
|
depends_on:
|
|
app:
|
|
condition: service_healthy
|
|
frontend:
|
|
condition: service_healthy
|
|
deploy:
|
|
resources:
|
|
limits:
|
|
memory: 256M
|
|
|
|
volumes:
|
|
postgres_data:
|
|
media_data:
|
|
exports_data:
|
|
caddy_data:
|