fix(deploy): give Postgres a CPU floor that Docker actually honours
`deploy.resources.reservations.cpus` was doing nothing. Outside Swarm, `docker compose up` silently drops it — verified by inspecting a running container, where CpuShares, CpuQuota and CpusetCpus were all unset while `limits.cpus` and `reservations.memory` came through as NanoCpus and MemoryReservation. So the comment calling it "the piece that actually protects the database" described a guarantee the box never had. It matters on the CX22 the runbook targets: the ceilings sum to 1.2 + 0.6 + 0.5 = 2.3 on 2 vCPU, so the other services can oversubscribe the machine, and with every container on the default weight Postgres competed on equal footing with two image resizes and an ffmpeg poster. Replaced with `cpu_shares`, which does survive the translation — db 2048, caddy 1024, app 512, frontend 256 — so the weighting only binds when the CPU is actually saturated, which is the moment the database must not lose. The Caddyfile gains a 10s header-read timeout: there was no read timeout anywhere, so a client could hold a connection, a tokio task and a `.tmp` file open indefinitely by sending one byte a minute, and the upload sweeper is keyed on mtime precisely so a live upload never ages out. Body reads stay unbounded — a 500 MB video over cellular legitimately takes minutes, and a body timeout would fail exactly the uploads this product exists to collect. .env.example documents that estimated_guest_count is a live input to the quota divisor rather than the inert setting both it and the runbook previously implied. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -42,6 +42,28 @@ services:
|
||||
# millisecond, so connections are no longer spent waiting. Raising it back
|
||||
# toward 30 means raising this limit with it.
|
||||
memory: 1G
|
||||
# CPU ceiling. Postgres is the one service that must never be starved: every request
|
||||
# path touches it, so a CPU-bound image resize elsewhere degrades the whole event
|
||||
# rather than one feature. 1.5 of 2 cores is a ceiling, not a reservation — it only
|
||||
# binds when something else is competing.
|
||||
cpus: '1.5'
|
||||
reservations:
|
||||
# Memory floor only. `reservations.cpus` USED TO BE HERE and did nothing: outside
|
||||
# Swarm, `docker compose up` silently drops it — verified by inspecting a running
|
||||
# container, where CpuShares, CpuQuota and CpusetCpus were all unset while
|
||||
# `limits.cpus` and `reservations.memory` came through as NanoCpus and
|
||||
# MemoryReservation. So the comment claiming it was "the piece that actually
|
||||
# protects the database" described a guarantee the box never had.
|
||||
#
|
||||
# It matters on a CX22: the ceilings below sum to 1.2 + 0.6 + 0.5 = 2.3 on 2 vCPU,
|
||||
# so the other services CAN oversubscribe the machine, and with every container on
|
||||
# the default weight Postgres competes on equal footing with two image resizes and
|
||||
# an ffmpeg poster. `cpu_shares` is the knob that survives the translation — see the
|
||||
# weights on each service.
|
||||
memory: 256M
|
||||
# Relative CPU weight under contention (Docker default is 1024). Only consulted when the
|
||||
# CPU is actually saturated, which is exactly the moment the database must not lose.
|
||||
cpu_shares: 2048
|
||||
|
||||
app:
|
||||
# Production PULLS a prebuilt image; it never compiles. A release build of this crate is
|
||||
@@ -122,6 +144,15 @@ services:
|
||||
# Bounds a runaway ffmpeg transcode (large uploads, 2 workers) so it can't
|
||||
# OOM the single box and take down Postgres.
|
||||
memory: 1G
|
||||
# CPU ceiling for the two image workers + ffmpeg poster extraction. Bounded below
|
||||
# 2.0 so the app can never take both cores on its own.
|
||||
# COMPRESSION_WORKER_CONCURRENCY=2 is the memory bound; this is the CPU one.
|
||||
cpus: '1.2'
|
||||
# Half the default weight, and this is the ceiling's other half: the cap alone leaves
|
||||
# 0.8 vCPU for db + frontend + caddy, which frontend and caddy can consume between them.
|
||||
# Compression is throughput work with no guest waiting on it, so it yields to Postgres —
|
||||
# which every request path, including the app's own, is blocked on.
|
||||
cpu_shares: 512
|
||||
|
||||
frontend:
|
||||
# Pulled, not built — see the note on `app` above.
|
||||
@@ -161,6 +192,13 @@ services:
|
||||
resources:
|
||||
limits:
|
||||
memory: 256M
|
||||
# Node SSR is bursty and not latency-critical for guests (the app is CSR after the
|
||||
# first paint), so it yields first under contention.
|
||||
cpus: '0.6'
|
||||
# Lowest weight of the four, for the reason above: `ssr = false`, so this serves the shell
|
||||
# and then guests talk to `app` directly. A slow shell delays a reload; a slow database
|
||||
# breaks the event.
|
||||
cpu_shares: 256
|
||||
|
||||
caddy:
|
||||
image: caddy:2-alpine
|
||||
@@ -187,6 +225,12 @@ services:
|
||||
resources:
|
||||
limits:
|
||||
memory: 256M
|
||||
# TLS termination and static serving. Small but must stay responsive — a starved
|
||||
# reverse proxy makes every service look down.
|
||||
cpus: '0.5'
|
||||
# Left at the Docker default (1024). Caddy is cheap but sits in front of everything, so
|
||||
# it must not be the bottleneck; it is capped at 0.5 vCPU regardless.
|
||||
cpu_shares: 1024
|
||||
|
||||
volumes:
|
||||
postgres_data:
|
||||
|
||||
Reference in New Issue
Block a user