Two smaller operational items.
POSTGRES 512M -> 1G. DATABASE_MAX_CONNECTIONS is 30 for a ~100-guest event (feed
polling + SSE + uploads at once), and 30 backends plus Postgres 16's default
shared_buffers leaves very little headroom at 512M. An OOM here doesn't degrade one
feature -- every request path touches the database, so it takes the event down.
Memory is the cheaper knob than shrinking the pool back and reintroducing the
queueing it was raised to fix. .env.example now names the pairing explicitly, the way
it already does for COMPRESSION_WORKER_CONCURRENCY.
SOCIAL WRITES WERE UNTHROTTLED. toggle_like, add_comment and delete_comment were the
only mutating endpoints in the app with no limit at all -- upload, join, recover,
export and admin login all carry one. Asymmetric coverage rather than a deliberate
decision.
Low severity, and honestly so: a like fans an SSE broadcast to every client, but the
export regeneration a comment deletion triggers is contained (REGEN_DEBOUNCE 20s,
workers born with their epoch, superseded ones inert). So the ceiling is 120/min --
far above anything a real guest produces. This bounds a script, not an enthusiastic
double-tapper.
ONE bucket across all three actions: separate buckets would let a caller triple the
aggregate write rate by alternating between them. Keyed per USER, matching the feed
and upload limits -- at a venue every guest is behind one NAT, and an IP key is what
made the /join and /feed limits turn guests away in the first place.
Migration 020 seeds both keys, and both are wired into the admin allowlist, the
config UI and the e2e reseed -- the step two earlier per-area toggles missed, which
left switches that existed in code and could never be flipped.
Tests: 4 e2e, including that the shared bucket really is shared (the part most likely
to be lost in a refactor) and that one guest hitting the ceiling doesn't block
another behind the same IP.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Round 1 gave /join a per-IP ceiling and left /recover with only its
`recover:{ip}:{name}` bucket. That key is right for the job it was written for —
stopping someone who knows a display name (they're listed on the feed) from
burning the victim's 3-strike PIN counter and locking them out on repeat. But the
name is ATTACKER-CHOSEN, so cycling names mints a fresh 5-attempt bucket every
time and the per-IP cost is unbounded.
What sits behind that limiter makes it worse than a normal flood: every call runs
a cost-12 bcrypt verify, including an UNCONDITIONAL throwaway verify for names
that don't exist — added deliberately to close a timing oracle. So an unknown name
is the single cheapest way to make the server do ~200ms of hashing.
Adds `recover_ip_rate_per_min` (default 30, migration 019), checked BEFORE the
per-name bucket so a name generator can't walk past it. 30/min is far above any
real recovery attempt while capping a flood. The per-name bucket is untouched and
remains the anti-guessing control.
The second half matters as much as the first: bcrypt was running inline on the
async runtime everywhere. At cost 12 that pins a tokio worker thread for ~200ms,
and there is only one per core — so a login flood stalled every other request on
the box, including the feed. There was no spawn_blocking anywhere in the auth
module, despite SECURITY-BACKLOG claiming bcrypt had been offloaded.
Route all of it through `verify_password` / `hash_password` on the blocking pool.
That covers /recover, /admin/login, the host PIN reset, and — the one most likely
to bite at a real event — the PIN hash minted on every single /join. Saturating
the blocking pool degrades logins; saturating the worker threads degrades
everything.
Tests: cycling distinct names from one IP now hits the ceiling with a Retry-After,
and — the assertion that keeps the fix honest — repeated wrong PINs against ONE
name are still throttled with the ceiling set generously high, so the ceiling
added protection rather than replacing it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
At a venue every guest is behind one NAT, so an IP-keyed limiter hands the
whole party a single bucket. On a fresh deploy 12 guests arriving together
meant 5 joined and 7 were turned away, with no Retry-After telling them when
to retry. `/feed` (60/min) and `/export` (3 per DAY — the fourth guest to
fetch their keepsake locked out until tomorrow) had the same defect.
`feed_delta` was already keyed per-user and its comment states the exact
rationale ("so one client can't starve others behind a shared NAT"); this
makes its siblings match.
- feed:{ip} -> feed:{user_id} (auth was already in scope)
- export:{ip} -> export:{user_id} (resolved from the download ticket's
session, which was previously looked up and discarded)
- join:{ip}: pre-auth, so there is no user to key on. Split in two — a loose
per-IP ceiling that only bounds raw volume (new `join_ip_rate_per_min`,
default 60, migration 017), plus the real 5/60s anti-spam bucket keyed
per (ip, name), mirroring the existing `recover:{ip}:{name}`.
admin_login / recover / pin_reset_req stay IP-keyed on purpose and are now
commented as such: they guard credential guessing, where a per-user or
per-name key would just hand an attacker a fresh bucket per guess.
Retry-After: the machinery existed but 7 of 8 sites called `check()` and
hard-coded `None`, so a throttled client was told to back off but never for
how long. Delete the bool `check()` wrapper entirely so `check_with_retry`
is the only entry point and the delay cannot be discarded by accident. Also
surface it for the PIN lockout, where the deadline was already known.
Fix the "unknown" fallback while here: every client_ip() caller passed that
literal, so any request without X-Forwarded-For — anything reaching the app
directly rather than through Caddy — shared ONE global bucket. Serve with
connect-info and use the peer address.
Tests: the reseed forces every limiter toggle off before each test, which is
why this whole class was invisible. Add 01-auth/rate-limit-shared-nat, which
enables them and asserts 12 guests share an IP without collision, that one
guest hammering their own name IS still throttled (so the fix re-keys rather
than removes the limit), and that feed/export buckets are per-user. Retarget
the ddos join test at the new per-IP ceiling — it asserted the defect.
Also seed `admin_login_rate_enabled` (read by the handler, seeded by no
migration and no reseed) and register `join_ip_rate_per_min` in the admin
config allowlist. Unrelated pre-existing red test fixed: 01-auth/join
asserted a "Willkommen!" heading the wedding redesign removed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
diashow only subscribed to SSE events but never called connectSse(), so a
kiosk/projector opening /diashow directly (not via /feed) never opened the
EventSource — the showcase display got a one-time /feed snapshot and no live
updates, showing "Noch keine Beiträge" forever when turned on before any
photos. Open the stream in onMount (idempotent) and close it in onDestroy,
mirroring the feed page.
Raise the default upload_rate_per_hour from 10 to 100 (migration 015, scoped
to installs still on the old default so admin overrides are preserved). Guests
routinely upload bursts of 10-20 photos; the old default throttled the first
burst. Also update the code fallback and the test-mode reseed.
Both verified end-to-end against the docker test stack.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
edit_upload updated the caption/hashtags and nothing else. A caption lives in the HTML
viewer keepsake (the ZIP holds media only — export.rs), so after release the downloadable
viewer kept showing the OLD caption forever while the live feed showed the new one.
Editing stays allowed while locked/released — like comments and likes, the lock freezes
new uploads only (USER_JOURNEYS §9.3) — so the fix is to regenerate, not forbid: the
edit and an invalidate_and_arm(ViewerOnly) share one transaction (same atomicity as
delete_upload), then start_regen after commit. ViewerOnly carries the finished ZIP forward
untouched since the media didn't change; when the gallery isn't released, invalidate_and_arm
returns None and this is a no-op. Mutation-verified: without it, an edit doesn't bump the
epoch and the caption test fails.
Also (test isolation): the compression worker now carries a generation counter that
TRUNCATE bumps. A worker queued on the concurrency semaphore when a truncate wiped media
would otherwise wake in the NEXT test, fail to find its file, and broadcast
upload-error/upload-deleted into that test's SSE stream — corrupting any test asserting on
toasts or feed. It now abandons itself when the generation moved. No-op in production
(TRUNCATE is the only caller). Export workers were already inert across a truncate (epoch
guard on a fresh-UUID event).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All 40 backend tests were pure-function tests — not one line of SQL ran under `cargo test`,
even though the riskiest code in the repo is SQL. Add 12 DB-backed `#[sqlx::test]` tests
(fresh throwaway DB per test, real migrations) pinning the invariants that code is
load-bearing for, each mutation-verified to fail on broken code:
export_epoch.rs (8): epoch is post-increment on release (a worker born pre-increment is
inert); epoch strictly monotonic across reopen; a retired-epoch worker's writes are
no-ops; export_current is EXACTLY the invariant (8-case table); the ViewerOnly
carry-forward carries a done ZIP forward AND matches nothing when the ZIP is unfinished
(the af997a8 strand bug); boot recovery doesn't clobber a live ZIP.
upload_concurrency.rs (4): the atomic quota UPDATE stops two stale-snapshot uploads from
both committing (sequential AND concurrent-tx via EPQ); the FOR SHARE upload lock
serializes against release, blocking a photo from committing after the export snapshot.
Deleting FOR SHARE, or the carry-forward's `status='done'`, each makes a test fail.
Test isolation: TRUNCATE now also resets disk_cache and sse_tickets. The stale disk
reading was harmless only while quotas were globally off in e2e (they no longer are — see
the new quota spec), i.e. two holes were masking each other.
clippy: `cargo clippy --all-targets -- -D warnings` now passes (it did not). Deleted the
dead jobs.rs (an unused BackgroundJob sketch) and unused model methods; `#[allow(dead_code)]`
+ comment on the sqlx FromRow field-sets that ARE populated by the DB. No SQL or logic
changed. `cargo test` requires DATABASE_URL by design.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Performance:
- Cache the runtime `config` table in-memory (ConfigCache) with synchronous
invalidation on every write (admin PATCH + test reseed). Was re-reading each
key from Postgres on every request (~8 round-trips per upload).
- Stream uploads chunk-by-chunk to a temp file instead of buffering the whole
body in RAM (peak was up to the per-class cap, e.g. 500 MB/video); only 512
sniff-bytes are kept for magic-byte detection, then atomic rename into place.
- Cache the media-filesystem disk snapshot (DiskCache, 15s TTL) shared by the
quota check and admin stats; drop the discarded System::refresh_all().
- HTML export streams video (and small-image) originals straight into the ZIP
via a manifest instead of copying them to a temp dir first (removed the
transient 2x disk usage) and drops the double directory scan.
- Auth extractor resolves session -> live user in one JOIN (was two queries),
touching last_seen_at by token hash.
Stability:
- SSE: on broadcast lag, emit a `resync` event so the client runs a delta
fetch instead of silently losing events; frontend reconciles adds, deletions,
and (via an in-place refresh) like/comment counts on visible cards.
- Storage quota fails OPEN when the disk can't be read (was a 0-byte limit that
locked out all uploads).
- Graceful shutdown drains in-flight requests on SIGTERM/SIGINT, bounded by a
10s backstop so open SSE streams can't stall a deploy.
- Upload removes the persisted file if the DB transaction fails (no orphaned
bytes with no row to reclaim them).
Tests:
- New pure select_disk() with 5 unit tests (longest-prefix, fallbacks, fail-open).
- New e2e export-video spec covering the HTML export's video-streaming branch.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The prior test asserted 404 on /media/exports/Gallery.zip against an empty
stack — it would pass even if exports were still written under /media, because
no archive was ever produced. Now it:
1. seeds an upload, releases the gallery, and polls until the real zip job
writes Gallery.zip to disk;
2. asserts the archive is NOT served from public /media (the CR2 leak); and
3. asserts it IS retrievable via the gated ticket endpoint (200 + PK zip
magic) — proving the 404 means "not public", not "no file".
Also fixes a test-isolation gap the CR2 relocation introduced: __truncate wiped
media_path but not export_path, so a real export would leave Gallery.zip on disk
and break export.spec's "ready-but-file-missing → 404" test. truncate_all now
purges export_path too. Full 06-export dir: 5/5 green, no contamination.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two bugs surfaced while running the new E2E suite, plus a small test hook:
- jwt.rs: add a per-token `jti: Uuid` claim. Without it, two `create_token`
calls in the same wall-clock second for the same (sub, role, event_id)
produced identical JWT bytes — and identical sha256(token) hashes —
which then collided on `session.token_hash UNIQUE` with a 500. Manifests
in real use when an admin clicks "Anmelden" twice fast.
- auth/handlers.rs: reject display names containing 0x00. Postgres rejects
NUL in TEXT columns with `invalid byte sequence for encoding "UTF8"` and
the request leaks back as a 500. Now returns 400 with a clean message.
- handlers/test_admin.rs + main.rs: new POST /api/v1/admin/__truncate route,
compiled in always but only **registered** when EVENTSNAP_TEST_MODE=1 is
set on startup. Truncates every event-scoped table, reseeds config from
migration defaults, wipes media on disk, and clears the in-memory rate
limiter. RequireAdmin-gated so it's not anonymous even in test mode. In
production builds (no env var) the route returns 404 — verified by the
startup log message.
- services/rate_limiter.rs: add `clear()` so the truncate handler can wipe
the in-memory window map between tests.
- Dockerfile: bump rust:1.87 → rust:1.88 (current dep tree needs it) and
COPY ./migrations into the build context so the `sqlx::migrate!()` macro
can resolve at compile time.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
