fix(deploy): give Postgres a CPU floor that Docker actually honours

`deploy.resources.reservations.cpus` was doing nothing. Outside Swarm, `docker compose up`
silently drops it — verified by inspecting a running container, where CpuShares, CpuQuota
and CpusetCpus were all unset while `limits.cpus` and `reservations.memory` came through as
NanoCpus and MemoryReservation. So the comment calling it "the piece that actually protects
the database" described a guarantee the box never had.

It matters on the CX22 the runbook targets: the ceilings sum to 1.2 + 0.6 + 0.5 = 2.3 on
2 vCPU, so the other services can oversubscribe the machine, and with every container on the
default weight Postgres competed on equal footing with two image resizes and an ffmpeg
poster. Replaced with `cpu_shares`, which does survive the translation — db 2048, caddy
1024, app 512, frontend 256 — so the weighting only binds when the CPU is actually
saturated, which is the moment the database must not lose.

The Caddyfile gains a 10s header-read timeout: there was no read timeout anywhere, so a
client could hold a connection, a tokio task and a `.tmp` file open indefinitely by sending
one byte a minute, and the upload sweeper is keyed on mtime precisely so a live upload never
ages out. Body reads stay unbounded — a 500 MB video over cellular legitimately takes
minutes, and a body timeout would fail exactly the uploads this product exists to collect.

.env.example documents that estimated_guest_count is a live input to the quota divisor
rather than the inert setting both it and the runbook previously implied.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
fabi
2026-08-11 22:44:48 +02:00
parent 32dfe6874a
commit 0c0d5d5981
6 changed files with 344 additions and 66 deletions

View File

@@ -42,6 +42,28 @@ services:
# millisecond, so connections are no longer spent waiting. Raising it back
# toward 30 means raising this limit with it.
memory: 1G
# CPU ceiling. Postgres is the one service that must never be starved: every request
# path touches it, so a CPU-bound image resize elsewhere degrades the whole event
# rather than one feature. 1.5 of 2 cores is a ceiling, not a reservation — it only
# binds when something else is competing.
cpus: '1.5'
reservations:
# Memory floor only. `reservations.cpus` USED TO BE HERE and did nothing: outside
# Swarm, `docker compose up` silently drops it — verified by inspecting a running
# container, where CpuShares, CpuQuota and CpusetCpus were all unset while
# `limits.cpus` and `reservations.memory` came through as NanoCpus and
# MemoryReservation. So the comment claiming it was "the piece that actually
# protects the database" described a guarantee the box never had.
#
# It matters on a CX22: the ceilings below sum to 1.2 + 0.6 + 0.5 = 2.3 on 2 vCPU,
# so the other services CAN oversubscribe the machine, and with every container on
# the default weight Postgres competes on equal footing with two image resizes and
# an ffmpeg poster. `cpu_shares` is the knob that survives the translation — see the
# weights on each service.
memory: 256M
# Relative CPU weight under contention (Docker default is 1024). Only consulted when the
# CPU is actually saturated, which is exactly the moment the database must not lose.
cpu_shares: 2048
app:
# Production PULLS a prebuilt image; it never compiles. A release build of this crate is
@@ -122,6 +144,15 @@ services:
# Bounds a runaway ffmpeg transcode (large uploads, 2 workers) so it can't
# OOM the single box and take down Postgres.
memory: 1G
# CPU ceiling for the two image workers + ffmpeg poster extraction. Bounded below
# 2.0 so the app can never take both cores on its own.
# COMPRESSION_WORKER_CONCURRENCY=2 is the memory bound; this is the CPU one.
cpus: '1.2'
# Half the default weight, and this is the ceiling's other half: the cap alone leaves
# 0.8 vCPU for db + frontend + caddy, which frontend and caddy can consume between them.
# Compression is throughput work with no guest waiting on it, so it yields to Postgres —
# which every request path, including the app's own, is blocked on.
cpu_shares: 512
frontend:
# Pulled, not built — see the note on `app` above.
@@ -161,6 +192,13 @@ services:
resources:
limits:
memory: 256M
# Node SSR is bursty and not latency-critical for guests (the app is CSR after the
# first paint), so it yields first under contention.
cpus: '0.6'
# Lowest weight of the four, for the reason above: `ssr = false`, so this serves the shell
# and then guests talk to `app` directly. A slow shell delays a reload; a slow database
# breaks the event.
cpu_shares: 256
caddy:
image: caddy:2-alpine
@@ -187,6 +225,12 @@ services:
resources:
limits:
memory: 256M
# TLS termination and static serving. Small but must stay responsive — a starved
# reverse proxy makes every service look down.
cpus: '0.5'
# Left at the Docker default (1024). Caddy is cheap but sits in front of everything, so
# it must not be the bottleneck; it is capped at 0.5 vCPU regardless.
cpu_shares: 1024
volumes:
postgres_data: