fabi/EventSnap
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: fabi:5546fb82e650c8a9ddf17901571913d8bd1d9203
Branches Tags
fabi:main fabi:fix/deploy-unattended-blockers fabi:fix/audit-2026-06-27-critical-medium
fabi:v0.12.0 fabi:v0.11.0 fabi:v0.10.0 fabi:v0.9.0 fabi:v0.8.0 fabi:v0.7.0 fabi:v0.6.0 fabi:v0.5.0 fabi:v0.4.0 fabi:v0.3.0 fabi:v0.2.0 fabi:v0.1.0
..
compare: fabi:fix/deploy-unattended-blockers
Branches Tags
fabi:fix/deploy-unattended-blockers fabi:main fabi:fix/audit-2026-06-27-critical-medium
fabi:v0.12.0 fabi:v0.11.0 fabi:v0.10.0 fabi:v0.9.0 fabi:v0.8.0 fabi:v0.7.0 fabi:v0.6.0 fabi:v0.5.0 fabi:v0.4.0 fabi:v0.3.0 fabi:v0.2.0 fabi:v0.1.0

42 Commits
5546fb82e6 ... fix/deploy

Author SHA1 Message Date
MechaCat02
9b90929269 test(e2e): re-point the PIN lockout specs at the property that matters 
Three specs asserted the OLD policy — that three wrong PINs lock an account —
which is exactly the behaviour the previous commit removed, because that threshold
sat below the per-(IP, name) throttle ceiling and so let any single IP lock any
guest whose display name is readable off the feed.

Rewritten to assert the distinction the fix introduces, which a status code alone
cannot show: both tiers answer 429, but only the account lock costs the VICTIM.
The new specs read the row via db.isPinLocked rather than the response, so:

- one IP hammering /recover is throttled and the account stays UNLOCKED;
- a distributed guesser (counter preloaded via db.setFailedPinAttempts, since no
  single source can reach the threshold any more) still trips the lock, and it
  holds even against the correct PIN;
- concurrent wrong PINs are all counted — the atomicity property the old parallel
  test was really about, now asserted on the counter instead of inferred from a
  429 that the throttle could equally have produced.

The UI spec asserts the user-visible half: after four wrong PINs Dave can still
get into his own account. It also now types the PIN digit by digit rather than
filling and clicking, because the 4th digit auto-submits (pin-auto-submit.spec.ts)
and doing both raced the button's disabled state.

The adversarial spec enables rate_limits_enabled for its own run — it is off by
default in this environment, so without that the throttle tier would silently not
be exercised — and restores it in afterEach so it cannot leak into other specs
sharing the stack.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-07-31 23:58:17 +02:00
MechaCat02
117e67fa80 fix(auth,upload): close the admin lockout and four unbounded-input paths 
ADMIN LOCKOUT. admin_login looked its user up BY NAME. Migration 007 makes
display_name unique per event case-insensitively and join had no reserved-name
guard, so any guest joining as "admin"/"Admin"/"ADMIN" before the operator's first
login made find(role == Admin) miss, the fallback create("Admin") violate that
index, and `?` return a 500 — permanently, with no in-app recovery. Moderation,
config and gallery release all gone; the fix was hand-editing the database.

The root cause is the lookup key, not the creation. The name was never the
identity. User::find_admin_for_event resolves by role, which makes the whole class
of name collisions irrelevant — including the homoglyph bypasses of the new
reserved-name list, which is now defence in depth rather than the control.

Promoting the squatting row would be the obvious fix and is a serious mistake: it
carries a recovery_pin_hash the guest knows, so it would hand them the admin
dashboard via /recover, permanently, through a path needing no password. A
separate row under a fallback name is worse UX and much better security. Verified
against the real schema — the guest keeps their uploads, PIN and session under a
freed name, and the role lookup then finds exactly one admin.

Second, independent bug in that block: create() followed by a SEPARATE UPDATE ...
SET role = 'admin' manufactures the same poisoned state if anything fails between
them. Collapsed into create_with_role.

UNBOUNDED INPUTS — one root cause, four places: validation ran after the
allocation.
- upload caption/hashtags used Field::text(), which buffers the whole field, on
  the one route whose DefaultBodyLimit is 576 MiB — so 576 MiB of heap per
  concurrent request in a 1 GiB container, with the length check running
  afterwards on a string already built. Now refused mid-read.
- the hashtag CSV was never length-checked at all and was upserted tag by tag
  INSIDE the commit transaction, which holds FOR SHARE on the event row — one
  request could stall every other upload behind tens of thousands of round trips.
  Capped at 30 tags of <=50 chars.
- /recover and /recover/request built rate-limiter keys by format!() from an
  unvalidated, unbounded display name, retained up to 24h in a map pruned hourly:
  the limiter itself became the memory-exhaustion primitive it exists to prevent.
  join validated first; that check is now shared by all three. /recover/request
  also had no per-IP ceiling at all — /join got one in 017, /recover in 019, and
  019's own comment describes exactly this attack. It returns 204 rather than 400
  on a bad name, because a 400 would be a new signal on an endpoint whose contract
  is that it cannot enumerate guests.
- the SSE ticket store had no size cap, no per-session cap and no rate limit on
  its endpoint, while prune ran hourly against a 30s TTL. Now pruned on issue,
  capped, and rate-limited. At capacity it REFUSES rather than evicting a
  stranger's ticket — evicting would let one client deny SSE to the venue. Not
  one-ticket-per-session either: two tabs open their EventSources concurrently.

PATCH /upload/{id} had no rate limit, no validation, and called
invalidate_and_arm unconditionally — outside both `if let Some` guards. So
PATCH {} bumped export_epoch and armed a fresh pair of full-gallery export workers
every call; REGEN_DEBOUNCE bounds the rate of that, not the total work, so a guest
could keep the keepsake permanently un-downloadable. All three fixed. The
validation also resolves a divergence: upload normalised tags while edit stored
them raw, so #Party via edit and party via upload became two hashtag rows.

PIN LOCKOUT was an ordering bug before a policy one: the account-lock threshold
(3) sat BELOW the per-(IP, name) ceiling (5), so three requests from one IP locked
any guest whose name is on the feed, every 15 minutes, forever. The tier meant to
protect a guest was the cheapest way to attack them. Ceiling drops to 4, threshold
rises to 12, so locking a victim now needs at least three distinct sources.
Brute-force cost is unchanged — 48 attempts/hour means 10k PINs still take ~208h
regardless of IP count — and increment_failed_pin now decays the streak after 15
minutes, since the counter previously only cleared on success and honest typos
accumulated across days. Both invariants are pinned by tests rather than comments.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-07-31 23:09:20 +02:00
MechaCat02
f275de5c8f perf(feed): stop every feed page from aggregating the whole event 
v_feed computed like_count/comment_count with LEFT JOINs and a GROUP BY. Postgres
CAN push `event_id = $1` and the keyset predicate through the view — verified with
EXPLAIN, it uses idx_upload_event_created_id — but it CANNOT push ORDER BY ...
LIMIT across a GroupAggregate. So every request aggregated every upload in the
event, times its likes and comments, and only then sorted and took 21 rows. Cost
grew with the event, not with the page.

Measured on a throwaway database seeded to a real reception (1000 uploads, 100
guests, 27k likes, 10k comments), same query, same data:

  before   GroupAggregate (actual rows=1001) -> Sort -> Limit    449 ms
  after    Index Scan (actual rows=21) -> Limit -> SubPlans      0.58 ms

migration 022 replaces the joins with correlated scalar subqueries, which puts the
counts ABOVE the Limit so they run 21 times instead of 1001. Exactly equivalent,
not merely close: "like" is keyed (upload_id, user_id) so COUNT(DISTINCT user_id)
== count(*), comment.id is the PK so COUNT(DISTINCT c.id) == count(*), and the
GROUP BY was on u.id so it was already one row per upload. Column names, order and
types are unchanged, so no Rust changes. No new index needed — idx_like_upload and
idx_comment_upload already match the subqueries.

Note the existing load harness cannot see any of this: e2e/loadtest/driver.mjs
creates no likes and no comments, so the expensive path had never been exercised.

The amplifier, feed/+page.svelte: every open feed subscribed to `upload-processed`
and refetched page 1 — the most expensive page — so ~100 open feeds each fired one
per completed upload. Now gated on whether this client actually shows the card
that changed, and the debounce is jittered, because a fixed delay just moves a
simultaneous herd 800 ms later. Nothing is lost by skipping: a client without the
card also missed its `new-upload`, and the reconnect `feed-delta` already
schedules a refresh.

Load shedding, because the above reduces the risk rather than removing it: db.rs
set no acquire_timeout, so sqlx's 30 s default applied — longer than the
frontend's own 20 s fetch timeout, meaning the browser gave up while the server
kept holding the slot and the work was done for nobody. Now 5 s, and PoolTimedOut
maps to 503 + Retry-After instead of a generic 500. That mattered because the
upload queue classifies 5xx as transient and retries: a 500 sent the retries
straight back into the saturated pool with nothing to pace them. PoolClosed stays
Internal — it only occurs during shutdown, where a 503 would invite a retry
against a server that is going away.

The Retry-After extraction in into_response matches on variants, so unlike
message() a missing arm is not a compile error — it would silently drop the
header. Pinned by a test covering both retry-carrying variants.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-07-31 23:01:42 +02:00
MechaCat02
3f0f9c098b fix(upload-queue): rehydrate the persisted queue app-wide, not only on /upload 
loadQueue() had exactly one call site in the entire frontend: the /upload route's
onMount. So after a reload, an iOS tab discard or a PWA relaunch, staged photos
sat in IndexedDB while the badge read 0 and nothing sent them — unless the guest
happened to navigate back to /upload, which they have no reason to do, having
already been shown a success. The photo never leaves the phone and the guest is
never told.

The root cause was narrower than "loadQueue isn't called enough".
requeueRetriable() read IndexedDB but only .map()'d over whatever the in-memory
store already held, so it could reset statuses and never ADD an entry — and
processQueue reads only that store. That is why the `online` listener and the SSE
resume hooks, which both call it, could not recover a cold start either. It now
REBUILDS the store from IndexedDB, which makes all three resume paths work.

Rebuilding needs one guard: entryToQueueItem downgrades `uploading` to `pending`
with progress 0, and this runs on every `online` event and every SSE reconnect,
so a blind rebuild would visibly reset the progress bar of a request still on the
wire. In-flight items are carried over by id.

Hydration is module-level, SSR-guarded and idempotent, re-armed via
onSetAuth/onClearAuth because login is a client-side goto() — no module
re-import, no onMount re-run — so a hydration that no-oped for lack of a token
gets a second chance. Module level rather than a layout onMount because
+layout.svelte already imports this module on every entry point, it matches the
file's own bindOnline()/bindSse() pattern, and a store owning its own persistence
keeps the layout free of a concern it cannot test. auth.ts does not import this
module, so no cycle.

The burst-queue e2e test no longer navigates to /upload after its reload — it now
asserts the resume happens wherever the reload lands, which is the actual
regression guard.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-07-31 22:58:26 +02:00
MechaCat02
e52b2f1cd1 fix(upload): reclaim the bytes of uploads that never finish 
Reclaim was a dozen explicit remove_file calls on the handler's return paths.
That covers every way the handler can FINISH and none of the ways it can simply
STOP. When a client disconnects mid-body — a phone leaving wifi, iOS killing a
backgrounded PWA, the user hitting back — axum drops the handler future at an
.await inside field.chunk() and no return path runs at all. The partial file then
survives forever: it has no upload row, so cleanup_deleted_media (row-driven)
can never see it, and no sweeper covered the originals directory. The triggers
are routine rather than adversarial, and the client keeps the blob and auto-
retries on every `online` event, so one large video over bad wifi leaves several
copies.

Those bytes are also invisible to the quota while still consuming the free disk
that compute_storage_quota divides among guests — so orphans silently shrink
every guest's ceiling while the admin widget under-reports. All three volumes
share one filesystem; the end state is Postgres unable to write WAL.

TempFileGuard is an RAII guard, because dropping the future is exactly what runs
Drop — it is the only construct that survives cancellation. Armed before the file
can exist, disarmed only after tx.commit() succeeds. The twelve explicit cleanups
are deleted so one owner holds the rule.

The subtler half is the rename. It happens BEFORE the commit, so between them the
file exists under its final name with no row pointing at it — an orphan that
looks legitimate. The guard is RETARGETED there rather than disarmed, and the
retarget sits on the same poll as the rename with no .await between, which is
what makes that window uncancellable.

sweep_orphan_originals is the backstop for the process that was killed, where no
Drop can run at all. Hourly, alongside the existing sweeps: .tmp files past the
window go unconditionally (a .tmp never has a row by construction), other files
are batched 500 at a time through a single NOT EXISTS query.

Two things that look like oversights and are not, both commented in place:
- the 6h window is what makes the sweep safe against the rename-before-commit
  ordering, since a committing upload is briefly indistinguishable from an
  orphan. It must not be shortened to speed up a test.
- the NOT EXISTS deliberately does NOT filter deleted_at IS NULL. A soft-deleted
  row still points at its file during its retention window, and reclaiming that
  is cleanup_deleted_media's job; filtering here would race the two sweeps and
  destroy the files the recovery window exists to preserve.

Verified against the real schema: given a live original, a soft-deleted one and a
true orphan, the query returns only the orphan.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-07-31 22:56:26 +02:00
MechaCat02
5969ec74ea fix(compression): bound derivative retries so one bad upload can't loop forever 
The OOM in the previous commit was survivable; what made it an outage was that it
repeated. The upload row is committed before compression starts, derivatives_rev
defaults to 0, and set_derivatives_rev only runs on success — so a row whose
processing killed the container survived at rev 0, and backfill_stale_derivatives
(called unconditionally at every boot) re-selected it and re-ran the identical
workload. With restart: unless-stopped that is an infinite kill loop, and every
cycle also drops every SSE stream and truncates every in-flight upload.

Verified end to end against the real schema in a scratch database: with the new
guard the backfill selects the row on boots 1-3 and zero rows from boot 4 on,
and a later success resets the counter.

migration 021 adds derivative_attempts and derivative_last_error.

The counter is incremented WRITE-AHEAD, before the work is attempted. This is the
whole design: the failure being bounded is a cgroup SIGKILL, so no Err is
returned, no error handler runs and no Drop fires. A counter bumped in a failure
path increments zero times per crash and the loop would be unchanged.
set_derivatives_rev clears it, so success is the only reset and both the live
path and the backfill get it without a new call site to forget.

Also in the backfill:
- one task walking the rows sequentially instead of one task per row. A large
  backlog used to spawn thousands of tasks, each holding a pool handle and
  queueing on the same two permits, competing with live uploads for a whole boot.
- LIMIT 200 per boot, and original_path <> '' replacing an IS NOT NULL that was
  dead (the column is NOT NULL; cleanup_deleted_media blanks it instead).
- a once-per-boot error log naming how many uploads have given up. Without it the
  give-up is invisible — the loop stops, which is the point, but the photos keep
  a stale derivative forever with nothing to notice.

Adds backfill_video_posters for the mirror-image gap: a video interrupted by a
restart has its compression_status flipped processing -> failed by
startup_recovery and is never re-enqueued, so thumbnail_path stays NULL for the
rest of the event while the clip itself plays fine. It shares the same attempt
budget, which means a genuinely posterless sub-second clip (Live Photo, mis-tap)
stops being re-ffmpeg'd after three boots. That is intended, not a bug to fix
later — Ok(false) is a normal permanent outcome there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-07-31 22:52:48 +02:00
MechaCat02
a4bac03628 fix(compression): stop a single large PNG from OOM-killing the container 
An 8000x8000 RGBA PNG passes admission — 256,000,000 bytes is just under the
256 MiB max_alloc, and smooth content is under 3 MB on disk, far below any size
cap. Processing it peaked at ~1250 MiB inside a 1 GiB cgroup. Measured, not
argued: the new (ignored) test builds exactly that image and reads VmHWM around
the pipeline, resetting the watermark via /proc/self/clear_refs so the number
covers only the code under test.

Three independent causes, all of which had to go:

1. The decode outlived everything. `resize` takes &self, and the no-downscale
   arm bound `img` into `display`, so the ~244 MiB buffer was still alive when
   oxipng ran — and oxipng decodes the PNG *again*, holding a full-size buffer
   per filter trial. The decode now lives in a block that yields the display
   derivative; the else arm moves `img` out, which is what makes "the block's
   value is the only survivor" true in both arms.

2. oxipng was unbounded in every dimension: preset 2 with timeout: None, and the
   default features pull in rayon, which evaluates filter trials concurrently
   with a full-size buffer each and has no Options knob to cap it. Now gated at
   8 MP, given a 20 s timeout, and built with default-features = false so
   oxipng's own sequential shim is used. "filetime" is kept — without it
   preserve_attrs silently no-ops. Dropping "binary" also stops compiling
   clap/glob/env_logger (a CLI's deps) into the server image.

3. Even with those fixed it still measured 516 MiB, and compression_concurrency
   defaults to 2 — so two guests uploading big photos at once was another OOM,
   1032 MiB against a 1 GiB limit. The cost is dominated by image's Lanczos3
   resize, which accumulates in f32: the intermediate is new_width * old_height
   * 16 bytes, i.e. 262 MiB for this image — larger than the decode itself, and
   invisible to max_alloc. Two changes: the preview now derives from the 2048px
   display instead of re-resizing the original (one full-size pass, not two),
   and a job whose header-estimated peak exceeds 150 MiB takes an exclusive
   permit so two giants can never overlap. Ordinary photos (a 12 MP JPEG
   estimates ~50 MiB) never touch that permit, so throughput is unchanged for
   everything except the case that must not run in parallel.

Chaining 8000 -> 2048 -> 800 for the preview is not a quality trade: a staged
Lanczos3 downscale is standard for large ratios and is visually
indistinguishable at 800px.

The blocking half is now a free function so its memory behaviour is testable —
the fix is a scoping property a future edit could silently undo.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-07-31 22:42:03 +02:00
MechaCat02
6fd75adb27 fix(ops): bound logs, drain ffmpeg's stderr, and stop three silent hangs 
Six independent operational defects, none of which needed a new feature to fix.

Log rotation. Docker's json-file driver is unbounded by default, and those files
land on the HOST filesystem — outside every deploy.resources.limits in the compose
file, and on the same disk as postgres_data and media_data. A full disk stops
Postgres writing WAL, which takes the event down. Capped at 10m x 3 per service.

Log level. RUST_LOG was set in neither .env.example nor docker-compose.yml, so the
code fallback WAS the production level — and it was `debug`, with tower_http=debug
emitting a line per request and per response into that unrotated file. Now info,
with tower_http=warn to state that those spans are diagnostics, not an access log.

ffmpeg pipe deadlock. run_ffmpeg piped stdout and stderr and then called wait(),
which drains neither. Once the ~64 KiB pipe buffer filled, ffmpeg blocked writing
and wait() never returned — burning the full 120s timeout, twice per seek position,
three times per compression attempt. And the timeout is an Err, so the end state was
a soft-deleted upload: a guest's playable video destroyed by a poster-frame failure.
Now stdout is null (nothing ever read it) and stderr is drained by wait_with_output,
whose tail is logged on a non-zero exit. Note wait_with_output consumes the child, so
the old kill-on-timeout is gone; kill_on_drop(true) already covers it.

Readiness probe. /health never touched the pool, so the disk-full endgame above
stayed green all the way down. Adds /health/ready (SELECT 1 under 2s) as a SECOND
route — the compose healthcheck deliberately keeps pointing at /health, because
caddy gates its startup on it and a DB-dependent probe would turn a Postgres blip
into the reverse proxy refusing to start.

api.ts request timeout. The abort timer was cleared in a finally around fetch(),
which resolves on the response HEAD — leaving res.text() uncovered and no longer
abortable. An upstream that sends headers then stalls the body hung the call
forever. The timer now lives until the body is read, including the 204 path (which
otherwise leaked a live 20s timer per no-content request).

Upload XHR watchdog. The XHR had no timeout while processQueue held the
isProcessing latch across it; on a half-open socket neither error nor abort ever
fires, so the latch pinned and the queue wedged. Bounds SILENCE rather than total
duration — a 500 MB video over a venue uplink legitimately runs 30+ minutes while
making steady progress. Rejects as NetworkError, which is already the retryable
branch, so a stalled upload now recovers like any network blip.

IndexedDB failures. addToQueue called getDb() unguarded and handleSubmit had no
catch, so a private-mode refusal or a QuotaExceededError on a large blob left a
permanent "Wird hochgeladen…" spinner, no toast, and — for an in-app camera
capture — the only copy of the photo gone. Now reported as 'failed', which keeps
the staged files on screen and stays on the page.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-07-31 22:25:23 +02:00
fabi
7d0334bf22 Merge branch 'fix/video-thumbnail-seek'
Some checks failed
Audit / cargo audit (backend) (push) Failing after 13m24s
Details
Audit / npm audit (frontend) (push) Successful in 54s
Details
Checks / Backend — cargo test + clippy + fmt (push) Failing after 45s
Details
Checks / Frontend — vitest + svelte-check (push) Failing after 6m31s
Details
Checks / E2E — typecheck + lint (push) Failing after 47s
Details
E2E / Playwright E2E (chromium + webkit) (push) Failing after 9m31s
Details
E2E / Cross-UA smoke matrix (push) Failing after 5m36s
Details
2026-07-30 19:40:27 +02:00
fabi
402215d405 fix(video): extract a real poster frame, and stop claiming one that isn't there 
Both the compression worker and the HTML export ran the same invocation:

    ffmpeg -i <src> -vframes 1 -ss 00:00:01 -vf scale=… -y <out>

`-ss` AFTER `-i` is an output-side seek. Against a clip of a second or less ffmpeg
exits 0 and writes nothing, and both call sites gated on the exit status:

- the worker wrote `thumbnail_path` and logged "thumbnail generated" for a file that
  was never created, so GET /upload/{id}/thumbnail 404s in the live feed;
- the export listed media/<id>_thumb.jpg in data.json while the ZIP writer skipped the
  unopenable file, so the keepsake drew a broken image tile.

Any clip at or under a second, which phones produce constantly — mis-taps, Live
Photos, boomerangs. Not data loss; the .mp4 is in both archives and plays. Every
server-side signal stayed green.

New `services/video.rs` owns the extraction for both callers, mirroring the imaging.rs
precedent (created for the same duplication, and it paid off when the max_alloc fix
landed in both workers at once). Three changes in it:

- `-ss` before `-i`, an input-side seek. NOT sufficient alone: verified against the
  production image, seeking to 1s in a 1.000s clip is still past the last frame and
  still exits 0 with no file. The 0s fallback is what actually fixes this, and 1s is
  tried first only because an opening frame makes a poor poster.
- Verify the artifact, not the exit status. This is the check both sites were missing.
- Carry compression.rs's 120s timeout. export.rs had NONE — a hung ffmpeg there would
  strand the job at `running` and the keepsake would never complete.

The worker's call used `?`. Tightening the check without also making a missing poster
non-fatal would have been far worse than the bug: every sub-second clip would fail
compression, exhaust its retries and be soft-deleted. It now logs a warning and leaves
`thumbnail_path` NULL, which FeedListCard, VirtualFeed and LightboxModal already
handle.

The export now sets `thumb: ""` and skips the manifest entry when there is no poster —
and does the same for the IMAGE branch, whose decode failure left the identical
dangling reference. No viewer change was needed: +page.svelte already guards
`{#if post.media.thumb}` and falls back to a video tile with a play glyph. The comment
claiming "viewer handles missing thumbs gracefully" was true about the viewer and false
about what the backend sent — the guard never fired because the string was never empty.

e2e/specs/06-export/export-video.spec.ts had DOCUMENTED this as intended behaviour
("the fixture clip is <1s, so ffmpeg extracts no thumbnail frame — but exits 0 …
that's the intended shape here"). sample.mp4 is exactly 1.000s, so every video test in
the suite ran at that boundary and none ever fetched the poster. That comment is now
corrected to say what it actually was.

Tests: 2 unit; a new sample-5s.mp4 fixture so the ordinary first-seek path is covered
at all; video-playback now FETCHES the poster rather than asserting the attribute (the
one extra request that nine rounds of green never made); a new spec covering both
fixtures plus the mirror that a posterless video still uploads and plays; and a
keepsake spec asserting every <img> in the opened viewer resolves — naturalWidth === 0
is exactly the broken-tile case, whatever produced it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-30 19:40:27 +02:00
fabi
2b313e67e0 Merge branch 'fix/track-e2e-fixtures' 2026-07-30 19:39:59 +02:00
fabi
2551c25436 fix(e2e): track the test fixtures the suite reads 
`.gitignore` carried an unanchored `media/`, commented "Media uploads (mounted volume in
production)". Production media lives in the `media_data` Docker volume and never
appears in the working tree, so that rule guarded nothing real — but unanchored it
matches a directory of that name at ANY depth, and the only one in the repo is
`e2e/fixtures/media/`.

Its entire practical effect was to keep every E2E fixture untracked. A fresh clone got
the specs and none of the images or videos they read, and
`.github/workflows/e2e.yml` does a plain `actions/checkout` and generates nothing — so
the committed CI job could not have run the upload, video, EXIF, quota, oversized-image
or export suites at all. It only ever looked green locally, where the fixtures survive
as untracked leftovers from whoever first created them.

The `origin` remote is git.mc02.dev rather than GitHub, so those workflows have most
likely never executed against this repo, which is consistent with nobody noticing. That
also makes this a live blocker for the standing "rotate the token and push" item: the
first time CI runs, it fails on ENOENT in a way that looks like a broken suite rather
than a missing file.

Anchors the pattern to `/media/` and commits the six fixtures (672 KB total). Verified
`e2e/fixtures/media` is the ONLY directory the old pattern matched, so nothing else
changes visibility.

Found while adding `sample-5s.mp4` for the video-poster work: the new fixture would
have been invisible to every other machine, which is how the existing ones got here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-30 19:39:59 +02:00
fabi
d51c6b8c4b Merge branch 'fix/postgres-password-deploy-trap' 2026-07-30 19:28:34 +02:00
fabi
3bcb7c6a76 fix(deploy): close the POSTGRES_PASSWORD trap that broke a fresh deploy either way 
README step 2 said to set "DOMAIN, JWT_SECRET, ADMIN_PASSWORD_HASH, EVENT_NAME, etc."
-- POSTGRES_PASSWORD was not in that list. .env.example said to set it and keep it in
sync with DATABASE_URL. The two documents disagreed, and both readings ended badly.

Branch A, README verbatim: the stack came up GREEN and healthy on
CHANGE_ME_use_a_strong_password -- a database credential published in the public repo.
The production secret guard covered JWT_SECRET and ADMIN_PASSWORD_HASH, and nothing
anywhere looked at the Postgres password.

Branch B, .env.example verbatim: a permanent restart loop, "password authentication
failed for user eventsnap".

The guard's own design made Branch B near-certain. It stops the APP on the first
`docker compose up -d` -- but not the `db` service in that same command, which
initialises its data directory and bakes in whatever password was in .env at that
moment. POSTGRES_PASSWORD is honoured ONLY at initdb. So the intended recovery -- see
the refusal, fix your secrets, boot again -- was exactly the sequence that broke it.
Nothing in the error named the cause, and the remedy (`down -v`) is both unguessable
and the one command you must never run once real data exists.

Three changes, which have to ship together: the guard alone would just move operators
out of Branch A and into Branch B.

- The guard now rejects a placeholder DATABASE_URL in production (the password rides
  in that URL, which is what the app actually reads). Branch A can no longer boot.
- It reports EVERY unset secret in one message instead of returning on the first.
  Fixing two secrets used to cost two boot cycles, on a stack where Caddy waits on the
  unhealthy app throughout, and each avoidable cycle is another chance to reach for -v.
- A 28P01 handler in db.rs turns the unguessable failure into a self-explaining one:
  it names the initdb semantics, gives `down -v` with an explicit "deletes db + media +
  exports, no undo", and gives the ALTER ROLE alternative for when data already exists.

Docs: step 2 now names POSTGRES_PASSWORD and says every secret must be set BEFORE the
first up; the troubleshooting block covers the auth-failure loop and both remedies.

Also replaces htpasswd (apache2-utils -- not on a stock VPS) with
`docker run --rm caddy:2-alpine caddy hash-password`, an image the stack already pulls,
in README, .env.example and the guard's own message. Verified the output ($2a$14)
against the shipped $2y$12 example.

Verified on a real Postgres, not just in tests: Branch A refuses and names DATABASE_URL;
all three placeholders report in one boot; a volume initialised with one password and
connected to with another prints the diagnostic; an unrelated connect failure (dead
port) stays silent. 8 unit tests, including that non-prod ignores all of it so the e2e
stack is unaffected.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-30 19:28:34 +02:00
fabi
08d92b7531 fix(audit-7): keepsake viewer integrity, readable archives, quota_tolerance floor 
Seventh round, both defects in the keepsake — the one artifact that leaves the
system entirely, and so the one where a green server-side signal carries no
information at all.

Squashed from 2 commits, original messages preserved below.

──────── fix(export): stop a caption bricking the viewer, and ship readable archives

Two defects in the keepsake, both silent server-side and both only visible by
extracting the real artifact and trying to use it.

1. A CAPTION COULD BRICK THE VIEWER.

The viewer's data is inlined as `<script>window.__EXPORT_DATA__={…}</script>` -- it has
to be, since guests open index.html over file:// where fetching a sibling data.json is
blocked. The escape was `</` -> `<\/`. Against XSS that holds; I fired
`</script><img src=x onerror=…>` through a real Chromium parser and it round-trips
inert.

It does not stop the caption steering the HTML TOKENIZER. `<!--<script` with no later
`-->` drives the parser into script-data-double-escaped state, where the template's own
`</script>` only steps back to script-data-escaped instead of closing the element.
Everything after it -- including the viewer bundle -- is swallowed as script data.
Nothing executes and nothing leaks: `__EXPORT_DATA__` is simply never assigned and the
keepsake opens blank. A denial of the deliverable, not an XSS.

Reproduced in Chromium before changing anything, and the near-miss is worth recording:
`<!--<script>alert(1)</script>-->` comes back CLEAN, because the trailing `-->` returns
the parser to script-data state. A probe using the terminated form quietly repairs the
thing it is testing for.

Fix: escape every `<` as `<`, not just `</`. `<` never appears in JSON structural
syntax -- only inside string values -- so a global replace is sound, and one rule covers
`</script`, `<!--` and `<script` together. That is the point: the old escape was named
for the single case it handled. Only the INLINED copy is escaped; data.json is written
separately, in no HTML context, and stays literal.

2. EVERY ENTRY IN BOTH ARCHIVES WAS STORED MODE 0000.

`ZipEntryBuilder::new` leaves the external file attribute at zero and async_zip's host
compatibility defaults to Unix, so `unzip -Z` showed `?---------` on every line of both
Gallery.zip and Memories.zip. Windows Explorer ignores Unix modes, which is why this
survived; on Linux and macOS `unzip` faithfully applies what the archive asks for and
the guest gets a folder of photos none of which they can open.

Unconditional -- every keepsake ever produced, no hostile input required -- and
invisible server-side: the export succeeds, the ZIP is well-formed, the job writes
`done`, /export/status is green.

Found by accident. The browser test for defect 1 failed with ERR_ACCESS_DENIED on
file://, which looked exactly like a Playwright sandbox quirk; I twice "worked around"
it (fresh context, then a separately launched browser) before checking the extracted
files and finding mode 000. The workaround was suppressing a real bug. Both workarounds
are gone -- the ordinary `page` fixture loads the archive fine now.

Fix: all six ZipEntryBuilder sites route through one `keepsake_entry` helper stamping
`S_IFREG | 0644`, so the mode cannot be forgotten at a call site.

Tests: 3 unit (no `<` survives; the payload still decodes to the original value, because
this is a transport encoding and not a sanitiser; a clean payload is untouched) and 2
e2e that release for real, download the real archives, and check them from outside the
app -- one opening index.html over file:// in Chromium and asserting the viewer booted,
the captions came back verbatim and nothing executed; one asserting every stored mode
and every extracted file is readable. Both assertions verified to FAIL against the
pre-fix artifacts.

──────── fix(admin): reject quota_tolerance = 0 instead of silently blocking every upload

Zero is inside the documented 0–1 range and catastrophic. The per-user limit is
`free_disk * tolerance / active_uploaders`, so a tolerance of 0 makes every limit 0 and
refuses EVERY upload -- mid-event, with "Du hast dein Upload-Limit für dieses Event
erreicht", an error naming the wrong cause entirely. An admin reaching for an off-switch
wants `storage_quota_enabled`; the rejection now says so.

Rejecting the value rather than raising the floor. A floor of 0.01 was the obvious fix
and it is wrong: very small tolerances are legitimate -- they are how a large disk is
throttled down to a sensible per-guest ceiling, and how the quota specs steer it
(tolerance = target * active / free lands around 1e-5 on the 174 GB volume this suite
runs on). A floor would forbid real configurations, and would have broken the entire
storage-quota describe block, to prevent one typo. Verified: those four tests still pass.

Tests: the rejection, that the stored value is untouched (validation fully precedes any
write), and the mirror -- 0.00001 still round-trips -- so the guard can't quietly become
a floor later.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-29 21:37:39 +02:00
fabi
06bc9ddcb3 refactor(export): share the visibility filter between the row query and the estimate 
`query_uploads` selects the rows the archives are built from; `estimate_export_bytes`
sizes them for the disk preflight. They stated the same WHERE clause separately, and
the direction of drift matters: an estimate that MISSES rows the archive writes
under-reserves, which is precisely the ENOSPC the preflight exists to prevent.

The integration test claimed to guard this and cannot. Both sides of
`the_estimate_sums_exactly_the_rows_the_archive_will_contain` are `SRC:`-marked
hand-copies in tests/common/mod.rs -- neither is production code -- so drift means
production moved while both copies sat still, and the test goes on passing. The
convention is sound for pinning behaviour; it is structurally incapable of detecting
divergence from the thing it copies.

So fix it where it can be fixed. One `export_visibility_where!()` fragment,
`concat!`-ed into both queries at compile time (still `&'static str`, no allocation),
with the `u`/`usr` alias contract stated. Divergence is now impossible by
construction rather than watched for.

The tests keep their value and lose the overclaim: the docstrings now say they pin
WHICH uploads may be counted -- each excluded row in the fixture is excluded by a
different predicate, so weakening any one of them still fails here -- and say plainly
that they do not detect drift, with a pointer to what does.

No behaviour change. The filters were verified identical before the hoist
(`u.event_id = $1 AND u.deleted_at IS NULL AND usr.uploads_hidden = FALSE AND
usr.is_banned = FALSE`); 99 backend tests still pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-29 20:50:04 +02:00
fabi
5f702f2b40 fix(audit-5): export disk preflight, media reclaim, low-disk warning, restore docs 
Fifth round, all storage. The keepsake could not fit on the documented hardware
and failed halfway through a multi-GB write, leaving the deliverable stuck; the
quota had stopped bounding the disk; and the backup had no restore procedure.

Squashed from 6 commits, original messages preserved below.

──────── fix(export): refuse an export that cannot fit, and stop peaking at two generations

Nothing in export.rs ever asked whether the keepsake would fit. Both archives write
their media `Compression::Stored`, so each is essentially a byte-for-byte second copy
of the originals -- Gallery.zip always, and Memories.zip for every video and every
image at or under 5 MB. On the documented CX33 (80 GB, all three volumes on one
filesystem) the upload quota's fixed point leaves ~40 GB free, and a release spawns
BOTH halves concurrently against it.

The failure is not "the export failed", it is "the deliverable is stuck":

  1. ENOSPC lands partway through a multi-GB write.
  2. The epoch has already moved, so the job row is `failed` at the CURRENT
     generation and readiness (epoch = event.export_epoch AND status = 'done') is
     false -- GET /export/zip 404s.
  3. The last good archive sits on disk, unreferenced and unreachable.
  4. POST /host/export/rebuild, the only escape, re-arms the same doomed write.

Three changes.

Reclaim before building. `prune_stale_export_files` ran only after the new archive
was written, renamed and finalised. That reads as durability but buys nothing: the
moment `invalidate_and_arm` bumps the epoch the old archive is ALREADY unreachable,
so keeping it reserves gigabytes for a download nobody can perform -- and for a
takedown it is content someone explicitly asked to have removed. Peak usage is now
one generation. Narrower than the post-finalize prune on purpose: final archives
only, never a `.tmp` or a `viewer_tmp_` dir, since a superseded worker can still be
streaming into those and at build START is far more likely to be alive.

Preflight the space. SUM(original_size_bytes) over exactly `query_uploads`'
visibility filter, +10% for ZIP overhead, multiplied by the number of armed jobs --
without that multiplier each of the two concurrent halves independently sees "it
fits" and together they don't. Runs AFTER claim_job, not before as reported: bailing
before the claim leaves the row `pending` with no worker and no error, the
spinner-forever state `mark_failed`'s status guard exists to prevent. Fails open when
the mount can't be read, exactly as the upload quota does.

Show the host the reason. /export/status returned {status, progress_pct} and nothing
else, so the host dashboard could only render "fehlgeschlagen" next to the retry
button. The message was written to the row and surfaced solely in the ADMIN job list
-- a different screen, possibly a different person. It now travels with the status,
and only on a failure, so a message left on a since-succeeded row can't appear beside
a green "ist bereit".

Tests: 10 unit (the u128 clamp caught a real bug in the first draft -- saturating_mul
then /100 turns an overflow into a number ~100x too small, the one direction that
authorises the write being guarded against; the carried-forward archive must survive
its own older epoch in the filename), 4 DB-backed (the estimate is asserted against
the row set the archive actually contains, not against a restatement of the WHERE
clause, so the two queries cannot drift), 3 e2e over the four-hop plumbing.

──────── fix(maintenance): reclaim the media of deliberately deleted uploads

The quota stopped bounding the disk. `soft_delete_in_event` stamps `deleted_at` and
refunds `total_upload_bytes`, but nothing ever removed the bytes, and the hourly
sweep reached only `compression_status = 'failed'`. Upload 500 MB, delete, quota back
to zero, upload another 500 MB. Not an attack -- a guest curating their camera roll,
which is what people do. The host then sees guests hitting "Du hast dein Upload-Limit
erreicht" while the admin widget shows a disk full of files no upload row points at,
and the quota message is actively misleading because the space really is gone, just
not to anyone the accounting can name.

Two retention windows, because the two deletes mean different things. A compression
failure keeps its 14 days: the guest didn't ask for it and may not be able to retake
the photo. A deliberate removal gets 24 hours -- 14 days outlives the whole event, so
a deliberate delete would never reclaim anything while it mattered, and a day still
covers a mis-tap.

Wider than reported: ALL FOUR paths are reclaimed, not just the original. Preview,
display and thumbnail are each a separate file, none counted in
`original_size_bytes`, and nothing ever removed them either. That was invisible while
the sweep only saw failed compressions (which produce no derivatives) and becomes
three leaked files per upload the moment it reaches a successful one. A row is
re-selected until every path is cleared, and the columns are cleared only once every
file for that upload is gone -- clearing after a partial success would strand the
survivors in exactly the unowned state this drains.

`backfill_stale_derivatives` selects on `display_path IS NULL AND preview_path IS NOT
NULL`, which is close enough to the post-sweep state to be worth pinning: it is
guarded on `deleted_at IS NULL`, so it cannot re-decode an original that is no longer
on disk. Covered.

Residual, deliberately: within the 24h window the bytes are still spent and still
unaccounted, so delete-and-re-upload through an eight-hour event can outrun the
sweep. Bounding that means holding the quota until the file is reclaimed rather than
refunding at `deleted_at`. The low-disk warning is the net under it.

Tests: 6 DB-backed, replacing 3. The one asserting an owner-deleted upload IS
reclaimed is the exact inverse of what this file used to assert.

──────── feat(host): warn about low disk before it becomes unrecoverable

Storage visibility existed in exactly one place: a passive Speicherauslastung widget
on the ADMIN dashboard. A host who isn't the admin had no view of it, and nothing
warned anyone. README carried "Low-disk alert (< 10 GB free)" under Planned since v1.

Two things make this a safety net rather than a nice-to-have. postgres_data,
media_data and exports_data are all Docker named volumes on ONE filesystem, so
running out doesn't degrade a subsystem -- Postgres stops being able to write and the
whole event goes down. And the keepsake needs room for two gallery-sized archives,
which the export preflight can only ever refuse AFTER the release, when the event is
over and every remedy is harder.

So the threshold is not a fixed number alone. It fires on the 10 GB floor the README
always named, OR on "you could not build the keepsake right now" -- the trigger a
host can still act on, computed with the same arithmetic the preflight uses. Unknown
free space is NOT low: it fails open like the upload quota and the preflight do,
because a banner that cries wolf on an unreadable mount is a banner nobody reads.

Carried on GET /host/event, which the dashboard already fetches on load and on every
reload -- no new endpoint, no new poll. Rendered above everything else including the
PIN-reset queue, and it names the consequence (the event, not just the download)
rather than only the number.

Also fixes the host page's formatBytes, which topped out at MB: 30 GB free would have
rendered as "30720.0 MB", and a guest with 2 GB of uploads was already being shown
that way in the user list.

Tests: 5 unit on the threshold (including that plenty of free space is still low when
the keepsake wouldn't fit -- the case a fixed threshold misses entirely), 3 e2e.
The e2e drives it through `original_size_bytes` rather than a genuinely full disk:
the estimate is pure SQL over that column, so overstating one row moves the
accounting without touching a byte on disk.

──────── docs: add a restore procedure, fix the backup cadence, and correct quota_tolerance

Four things, all found by the same question: what does an operator standing at the
venue actually need?

A RESTORE PROCEDURE. There was none anywhere, and a backup you have never restored
isn't a backup. Two hazards worth writing down: media must be extracted preserving
ownership (the app runs as uid 100 / gid 101, and a root-owned restore makes every
upload fail with EACCES surfacing as a generic 500), and the app must be STOPPED
first, because migrations run on boot and a live pool will fight the restore.

Both the backup and the restore commands were run against the real stack before being
written down, which caught two that would have failed:

  - The plain `pg_dump` did not restore: `psql` aborted on `ERROR: schema
    "_sqlx_test" already exists`. pg_dump emits no DROPs without --clean --if-exists,
    so the documented dump could only ever be restored into an empty database. Fixed
    at the source (the dump is now self-cleaning) and verified end to end: 16 tables
    back, exit 0.
  - `--same-owner` does not exist in BusyBox tar, which is what `alpine` ships, so
    the extract aborted before unpacking anything. `--numeric-owner` plus the
    explicit chown, verified to land 100:101.

BACKUP CADENCE. "Weekly offsite" is the wrong shape when every irreplaceable byte is
created in one eight-hour window and nobody can retake a wedding. The backup that
matters runs that night, and again after the release so the keepsake is captured.
Also: take the DB dump and the media tarball back to back, or you get rows pointing
at files the dump doesn't know about.

quota_tolerance WAS DOCUMENTED AS SOMETHING IT ISN'T. .env.example called it "fraction
of disk that triggers the low-storage warning". It is the multiplier in
`floor(free_disk * tolerance / active_uploaders)` -- so an operator who wants "warn me
later" and sets 0.95 is actually authorising guests to fill 95% of the disk, moving
the fixed point from 43% to ~49% and eating the export headroom. The admin UI labelled
it "Toleranz (0-1)" with no explanation at all, which invites exactly that reading;
it is now "Speicher-Anteil für Gäste" with the formula in the hint. Wrong docs on a
tuning knob are worse than no docs.

SIZING. New section with the arithmetic: three volumes on one filesystem, the quota
fixed point at tolerance/(1+tolerance), and the fact the 80 GB baseline does not cover
the keepsake -- both archives are built concurrently and each is roughly a second copy
of every original. Provision ~3x expected media, or give exports its own volume.

Also ticks the low-disk alert off the roadmap, since it now exists.

──────── chore: raise the db memory limit and rate-limit social writes

Two smaller operational items.

POSTGRES 512M -> 1G. DATABASE_MAX_CONNECTIONS is 30 for a ~100-guest event (feed
polling + SSE + uploads at once), and 30 backends plus Postgres 16's default
shared_buffers leaves very little headroom at 512M. An OOM here doesn't degrade one
feature -- every request path touches the database, so it takes the event down.
Memory is the cheaper knob than shrinking the pool back and reintroducing the
queueing it was raised to fix. .env.example now names the pairing explicitly, the way
it already does for COMPRESSION_WORKER_CONCURRENCY.

SOCIAL WRITES WERE UNTHROTTLED. toggle_like, add_comment and delete_comment were the
only mutating endpoints in the app with no limit at all -- upload, join, recover,
export and admin login all carry one. Asymmetric coverage rather than a deliberate
decision.

Low severity, and honestly so: a like fans an SSE broadcast to every client, but the
export regeneration a comment deletion triggers is contained (REGEN_DEBOUNCE 20s,
workers born with their epoch, superseded ones inert). So the ceiling is 120/min --
far above anything a real guest produces. This bounds a script, not an enthusiastic
double-tapper.

ONE bucket across all three actions: separate buckets would let a caller triple the
aggregate write rate by alternating between them. Keyed per USER, matching the feed
and upload limits -- at a venue every guest is behind one NAT, and an IP key is what
made the /join and /feed limits turn guests away in the first place.

Migration 020 seeds both keys, and both are wired into the admin allowlist, the
config UI and the e2e reseed -- the step two earlier per-area toggles missed, which
left switches that existed in code and could never be flipped.

Tests: 4 e2e, including that the shared bucket really is shared (the part most likely
to be lost in a refactor) and that one guest hitting the ceiling doesn't block
another behind the same IP.

──────── fix(e2e): stop the video poster assertion racing the ffmpeg thumbnail

Pre-existing, and it fired for real during the full-suite run on a cold stack.

The lightbox binds `poster={upload.thumbnail_url ?? undefined}`, so the attribute is
absent until compression produces the thumbnail. This test asserted on it immediately
after seeding, never waiting for the worker -- unlike the Range test further down the
same file, which does poll. Against a warm stack the worker usually wins; against a
freshly rebuilt one (`stack:down -v`, cold ffmpeg) it doesn't.

That is the worst possible time for a false failure: the first run after a rebuild is
exactly when you are trying to establish whether a change broke something. Poll for
`compression_status = 'done'` before the poster assertion. The `src` assertion needs
no wait and keeps none.

Verified with --repeat-each=3.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-29 20:10:54 +02:00
fabi
31faccfdf8 fix(audit-4): refuse undecodable images at admission, stop retrying permanent failures 
Fourth round: reject at the door what the worker can only fail on, and stop
retrying errors that cannot succeed — while keeping IoError retryable so ENOSPC
still gets another attempt.

Squashed from 2 commits, original messages preserved below.

──────── fix(upload): refuse undecodable images at the door, and stop retrying them

Two halves of the same complaint: an oversized photo was accepted with a 201 and
then silently soft-deleted minutes later, after the worker had burned six seconds
of backoff re-reaching a conclusion it could not change.

Admission. The compression budget now runs at upload time, against the header
only, so a guest is told immediately and told why:

  "Bild hat zu viele Bildpunkte (ca. 99 Megapixel) und kann nicht verarbeitet
   werden. Bitte verkleinere es und lade es erneut hoch."

instead of watching the photo vanish behind a vague "could not be processed" —
which arrived only if they happened to still be on the feed with that card
loaded. Nothing is stored, so there is no row to soft-delete and no orphan for
the sweep to reclaim.

Admission and the worker share ONE function (`decoder_within_budget`), so they
cannot drift apart and start disagreeing about what is acceptable — a photo
accepted at the door and rejected by the worker would be worse than either
behaviour alone. The worker keeps its own check: the backfill decodes files that
predate this check, and defence in depth is the whole reason the budget exists.

Retries. The loop retried every failure, including ones that are a property of
the input. An image over the budget, a corrupt file, an unsupported format: each
fails identically on all three attempts, so the only effect was 2s + 4s of sleep
and three near-identical warnings before the same outcome. `is_permanent_image_error`
classifies the `ImageError` variants that cannot change between attempts — Limits,
Unsupported, Decoding — and the loop gives up on those at once. `IoError` is
deliberately excluded: an ENOSPC while writing a derivative is exactly the
transient case the retry exists for, and misclassifying it would turn a blip back
into the data loss round 1 fixed. Measured: retry log lines went from 3 per
oversized upload to 0.

Tests: unit tests for both sides of the classifier (a Limits error is permanent, a
missing file is not) and for admission agreeing with the decoder on accept AND
reject. The e2e spec is rewritten for the new contract — 400 with an actionable
message, nothing stored, backend alive after a burst of four — plus a mirror
asserting an ordinary photo still uploads and processes, since a budget that
rejected everything would satisfy the other two.

──────── fix(upload): narrow the admission check to the memory budget only

The admission check I just added rejected ANY image the decoder couldn't build —
corrupt, truncated, or unsupported, not only over-budget. That broke two
adversarial tests, and they were right to break.

07-adversarial/file-upload-attacks pins, deliberately, that acceptance follows the
MAGIC BYTES: a payload whose first three bytes are a JPEG header is accepted
regardless of what follows, because the security property under test is that the
client-declared Content-Type has no influence. Both failing cases upload 1024
bytes of JPEG magic followed by zeros. Rejecting those at admission is a
different, broader contract than the one asked for, and rewriting an adversarial
test to match new behaviour is precisely the thing that needs justifying rather
than doing quietly.

So admission now checks only what it was meant to: `exceeds_decode_budget`
returns true solely for `ImageError::Limits`. A corrupt file goes to the
compression worker exactly as before — which handles it gracefully and, since the
retry classifier in the previous commit, no longer burns backoff on it. The
resource guard is the part that had to move earlier; nothing else did.

Tests: the size agreement between admission and the worker is still asserted in
both directions, plus a new one writing a magic-bytes-only stub and asserting
admission accepts it WHILE the worker still rejects it — pinning the boundary
between the two checks so a future widening fails here rather than in the
adversarial suite.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-29 07:55:56 +02:00
fabi
06ade4e158 fix(audit-3): restore the decode allocation guard, route /health in production 
Third round. The decode allocation guard is a regression from round 1: swapping
reader.decode() for into_decoder() silently dropped the max_alloc enforcement
while keeping the comment that claimed it held.

Squashed from 3 commits, original messages preserved below.

──────── fix(imaging): restore the decode allocation guard I removed in round 1

This is a regression I introduced, not a pre-existing gap. Before 05948d8 the
compression worker used `ImageReader::decode()`, which does:

    let mut decoder = Self::make_decoder(format, self.inner, limits.clone())?;
    limits.reserve(decoder.total_bytes())?;   // enforces max_alloc
    decoder.set_limits(limits)?;

Reading the EXIF orientation tag needs `into_decoder()` instead, and that skips
the reserve entirely — the crate's own FIXME concedes `from_decoder` doesn't
compensate. Nothing else enforces `max_alloc`: the JPEG decoder's `set_limits`
only checks support and dimensions. So the 256 MiB budget has been inert since
that commit, and round 2 then propagated the weakened path into export.rs through
the shared helper, in a commit whose message claimed the helper "carries" the
decompression-bomb cap. It didn't, and the comment saying max_alloc "hard-caps
the decode allocation" was simply false.

What was left was only the per-axis cap, which permits 12000x12000 — 412 MiB
decoded, 824 MiB for the two concurrent decodes the worker runs by default,
against a 1 GiB container. Deploy-blocking right now because bumping
DERIVATIVES_REV makes the first boot after a deploy re-decode the entire gallery
two at a time: an OOM kill there restarts the container, which re-runs the
backfill. A boot loop, on the first deploy of these fixes.

Re-add the reserve exactly as `decode()` does it. Per the budget decision it stays
at 256 MiB (~89 MP for RGB8, above any mainstream phone's real output); two
concurrent decodes now peak at 512 MiB. Oversized images take the graceful path
from round 1 — original retained, quota refunded, upload-error toast — and fail
after the header parse but BEFORE any pixels are read, so they cost a header read
rather than an allocation. Measured peak during a concurrent oversized burst: 3.0
MiB.

Test parity is the other half, and the reason this was invisible: the e2e app
container had NO memory limit while production is capped at 1 GiB, so a decode
that would OOM-kill production simply succeeded in CI. Mirror the 1 GiB cap in
docker-compose.test.yml. That is the third divergence of this shape, after WebKit
missing from CI and /health existing only in Caddyfile.test.

Tests: a fixture that is 568 KiB on disk and 283 MiB decoded (11000x9000 = 99 MP,
deliberately UNDER the per-axis cap so the axis check cannot be what rejects it).
A unit test asserts the refusal — it fails against the old code, which decoded it
into an 11000x9000 buffer — with a companion asserting an ordinary photo still
decodes AND still gets its orientation applied, so the guard didn't become a
blanket refusal. An e2e test uploads it singly and as a concurrent pair, asserting
compression lands in 'failed' and the backend is still serving and still
processing afterwards.

──────── fix(deploy): route /health in production, and actually apply Caddyfile changes

Two defects in the update procedure I wrote last round, both of which make a
successful-looking deploy a lie.

1. The documented health check could never pass.

`curl -fsS https://DOMAIN/health` 404s against a perfectly healthy production
stack. The backend registers /health on its ROOT router, not under /api/v1, and
the production Caddyfile proxies only /api/* and /media/* — so /health fell
through to the SvelteKit catch-all, which has no such route and returns its 404
page. With -f, curl exits 22 and the `&& echo` never runs. My own gloss
("Anything other than ok means check the logs") then sent the operator chasing a
phantom outage.

e2e/Caddyfile.test has carried `reverse_proxy /health app:3000` since it was
written — precisely because the catch-all would otherwise swallow it. Production
never did. Per the fix-the-gap-not-the-doc call, production gets the same line,
and /health joins the no-store matcher so a cached response can't report the last
known state instead of the current one. Verified by running the production
Caddyfile against the real backend: /health -> 200 "ok", Cache-Control: no-store,
with /api/v1/event and / unaffected.

2. The sequence never reloaded Caddy, so a Caddyfile-only change was dropped.

`--build` only rebuilds services with a `build:` section, and caddy is a pinned
upstream image. Compose decides whether to recreate a container from its config
hash, which covers the mount SPECIFICATION but not the mounted file's CONTENTS —
so a git pull that changes ./Caddyfile produces no delta, Compose reports
`Running`, and Caddy serves its old config indefinitely. Exit code 0 throughout.

Round 1's iOS download fix (137c4ee) is exactly this shape: Caddyfile plus four
e2e files, so 100% of its production effect is in that one file. Following the
README to the letter deployed it, showed both image IDs changing, and left iOS
downloads broken.

Demonstrated rather than assumed — added a probe header to a Caddyfile, ran the
old sequence (`up -d --build`): header absent, change silently dropped. Ran the
new step 4 (`up -d --force-recreate caddy`): header served.

`--force-recreate` rather than `restart` or `caddy reload` because the bind mount
is resolved to an inode at container-create time and git pull replaces the file
rather than editing in place, so a restart can re-read the stale content — the
exact failure I hit in round 1 when `caddy reload` didn't pick up an edit.

Also rewrites the "db and caddy are untouched … so data volumes survive" sentence.
I wrote it as reassurance; "caddy is untouched" was the bug.

──────── chore: take the Bash(*) permission change back out of the shared settings

`.claude/settings.json` is committed and applies to anyone who clones. Fabi's
local `allow: ["Bash(*)"]` plus deny list ended up in it, inside f0d69f1 — a
commit about the image decode guard, which has nothing to do with permissions.

That was my mistake, twice. The file was already modified when I started the
round: my `git status --short` check printed "(clean)" from an unconditional
`echo` rather than from the status output, so I read a dirty tree as clean. Then
`git add -A` swept it into an unrelated commit, and I reported afterwards that I
had left it untouched. Neither the check nor the claim was true.

Restores the shared file to its previous three narrow entries. The permission
setup itself is preserved, moved to `.claude/settings.local.json`, which
`.gitignore:34` covers precisely so per-user permissions stay per-user — the
existing 442 entries there are kept alongside it.

Not rewriting f0d69f1 to erase this: main is unpushed so it would be safe, but a
visible correction is worth more than a tidy history, and a rebase across the
merge commits carries more risk than the mistake does.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-29 07:18:08 +02:00
fabi
b601c062bd fix(audit-2): video playback, role identity, keepsake EXIF, recover ceiling, WebKit CI 
Second round: the two regressions the first round introduced, the remaining
gaps it left, and the CI change that makes the iOS guarantees actually gate a
change rather than being asserted.

Squashed from 9 commits, original messages preserved below.

──────── docs(deploy): document the update path — `up -d` alone ships nothing

The README only ever described a fresh install. There was no update section
anywhere, and `--build` appeared nowhere in the docs.

That matters because `app` and `frontend` are `build:` services with no published
image tag, and Compose has no source-change detection: if an image by that name
exists it is reused. So the natural `git pull && docker compose up -d` reports
"Container app-1 Running", rebuilds nothing, and exits 0. A deploy that shipped
none of the new code is indistinguishable from a successful one — which is how
eleven merged fixes can sit in the repo and never reach the box.

Verified both halves against a real stack rather than asserting them: with a
source change staged, `up -d` left the image ID untouched; `up -d --build`
produced a new image ID and a healthy /health.

Adds an "Updating an existing deployment" section covering backup-before-migrate,
pull, rebuild, health check, and an image-ID comparison to prove a build actually
happened. Also spells out the rollback trap: migrations run on boot and are not
undone by checking out an older commit, so rolling back code without restoring
the snapshot leaves the schema ahead of the binary and the app refusing to start.

──────── fix(video): play the actual video, and answer Range requests

Every video in the app was unplayable. Two independent defects, either one
sufficient on its own, and nothing in the suite covered either — no test
anywhere played media or asserted a `<video>` src.

1. The lightbox handed `<video>` a JPEG.

`pickMediaUrl` is mime-agnostic, and compression only ever produces a THUMBNAIL
for a video (one `ffmpeg -vframes 1` frame) — no preview, no display. So in the
DEFAULT saver mode the element's src resolved to `/api/v1/upload/{id}/thumbnail`,
served as `image/jpeg` with `nosniff` so the browser can't even sniff its way
out. Chromium reports DEMUXER_ERROR_COULD_NOT_OPEN.

Fixed in the lightbox rather than in `pickMediaUrl`: FeedListCard shares that
helper and legitimately wants the thumbnail for its `<img>` poster, so a central
mime branch would break the feed. This mirrors the rule the diashow already
applies ("videos play the original file directly"). Added `preload="none"` so
saver-mode guests on cellular still fetch nothing until they press play — there
is no smaller video derivative to offer them — plus `playsinline`, without which
iOS hijacks playback into fullscreen.

2. `stream_media_file` ignored Range entirely.

It took no request headers, so it could not see `Range`; it always returned 200
with the whole body and never sent Accept-Ranges or Content-Range. iOS Safari
opens every `<video>` with a `Range: bytes=0-1` probe and abandons the load
without a 206 — so video failed on the app's primary platform even in `original`
mode, where the src was already correct.

Adds single-range support (`bytes=N-`, `bytes=N-M`, `bytes=-S`) with 206 +
Content-Range, 416 + `bytes */len` past EOF, and Accept-Ranges advertised on
every response. Anything it won't handle — multi-range, non-bytes units, garbage
— falls back to a full 200, which RFC 9110 explicitly permits and which is safer
than guessing. All four media routes share the helper, so seeking works
uniformly.

`get_original` now serves `inline` instead of `attachment`. An attachment
disposition is hostile to a `<video>` element, and this route is the only source
of playable video bytes; it also matches what the UI promises, since the action
is labelled "Original anzeigen" — view, not download. `no-store` is deliberately
kept so a takedown still revokes access promptly; ranges work fine under it, the
client just re-fetches.

Tests: 11 unit tests pin the parser (the iOS `bytes=0-1` probe, inclusive ends,
suffix ranges, clamping past EOF, 416 vs 200, malformed fallbacks). A new
03-feed/video-playback spec asserts the src is the original and not the
thumbnail, that the browser accepts the bytes as media (readyState > 0, no
MediaError), that no video bytes are delivered before play, and that Range
returns the correct 206 slices and a 416 past EOF — verified on both Chromium
and WebKit.

The "not downloaded before play" test asserts no *delivered body* rather than no
request: WebKit opens a connection for a preload="none" video and immediately
aborts it (GET, no Range, status 0, nothing transferred) while Chromium issues
nothing at all. The portable guarantee is that no response carrying bytes
completes.

──────── fix(auth): bind the role store to the identity, not to the tab

The role store I added in the moderation work is a module-level singleton seeded
ONCE at import. `goto()` is a client-side navigation, so leaving and re-joining in
the same tab re-imports no module and re-runs no onMount — the previous user's
role simply stayed resident. Nothing reset it: not join, recover, admin login,
"Event verlassen", `clearAuth`, nor the api.ts 401 auto-clear.

So a host who left, followed by a guest joining on the same phone, left that guest
with `isStaff === true` and a "🚫 Beitrag entfernen" action on other people's
photos. The backend 403s the delete, so this was a false affordance rather than a
privilege escalation — but `/feed` never fetched `/me/context`, so unlike every
other route it never self-corrected either. It survived until a hard reload.

The mirror case was equally broken and easier to overlook: a guest who recovered
into a host account got NO host affordances.

`clearAuth` already had a hook registry for exactly this shape of problem, with a
comment explaining it exists to avoid circular imports. Add the missing mirror,
`onSetAuth`, fired by both `setAuth` and `setAdminAuth` after the new token is
resident, and have the role store register on both sides: clear to null on
logout, re-seed from the new token on login. That also gives
`syncRoleFromToken` — dead code with zero callers since I introduced it — its
intended purpose.

Seeding from the claim fixes the reported bug, but the claim is frozen for the
token's 30-day life, so a promotion or demotion still wouldn't reach the feed.
`/feed` now calls the existing `refreshEventState()` on mount, which fetches
`/me/context` and applies both the authoritative role and the lock/release state
in one request. The feed is the one route gating a destructive action on the role,
so it should not be the only route running on a stale claim.

Tests: 04-host/role-identity-reset drives the real flows. The first asserts the
host DOES see the action before asserting the newcomer does not — a negative
assertion alone would pass against a build that shipped no moderation at all. The
second covers the mirror, promoting a guest server-side while their resident token
still claims `role: guest`, so a fix that only cleared the role would fail it.

──────── fix(compression): reclaim failed originals instead of leaking them

Round 1 stopped the compression worker deleting an upload's original on failure —
a transient ENOSPC or a codec panic must never destroy the only copy of a photo a
guest cannot retake. But it left `Upload::soft_delete`'s quota refund in place, so
the bytes stayed on disk while the uploader was charged nothing for them.

That is worse than it first looks. The row is soft-deleted, so the file is
invisible and unowned; a guest hitting a reproducible codec failure can accumulate
orphans indefinitely at zero personal cost. And `active_uploaders` counts only
users with non-deleted uploads, so dropping out of that count RAISES everyone's
per-user ceiling — the leak loosens the very quota meant to contain it.

Keep the refund: the uploader didn't cause the failure and shouldn't silently lose
quota to it. Bound the leak instead, with an hourly sweep alongside the existing
session cleanup in `spawn_periodic_tasks`, reclaiming failed originals older than
14 days — comfortably longer than any single event, so an operator investigating a
failed upload still has the file.

The selection predicate is the entire safety argument, so it is deliberately
narrow: `compression_status = 'failed'` AND soft-deleted AND past the window AND
`original_path <> ''`. That is exactly the state the give-up path leaves behind,
and it cannot reach a live upload, an owner-deleted one, or a failure still inside
its recovery window. `original_path` is cleared after a successful reclaim, which
makes the sweep idempotent — otherwise a row whose file is already gone is
re-selected on every tick forever. The row itself is kept as the audit trail.

Tests reproduce the selection verbatim (same pattern as upload_concurrency) and
assert it against five near-misses that must survive, both sides of the retention
boundary, and the idempotence property.

Also fixes two comments in export.rs still claiming "the compression worker
hard-deletes an original when its transcode fails" — no longer true, and the
defensive handling they justify is now justified by this sweep and by ordinary
deletes instead.

──────── fix(export): apply EXIF orientation in the keepsake too

Round 1 fixed EXIF orientation in the compression worker, which corrected the live
app — feed preview and diashow display. The export worker was missed, and it does
not reuse those derivatives: it re-decodes the originals itself with `image::open`,
which ignores the orientation tag, then re-encodes to JPEG, which drops the tag —
so the viewer has no way to recover it.

The damage was oddly shaped, which is exactly why it reads as a viewer bug:

  Gallery.zip originals              correct  (byte-copied, EXIF intact)
  Memories viewer grid thumbnails    SIDEWAYS (always)
  Memories viewer full image >5 MB   SIDEWAYS (re-encoded at 2000px)
  Memories viewer full image ≤5 MB   correct  (streamed byte-for-byte)

So in the keepsake people actually keep, every portrait photo in the grid was on
its side, and clicking through silently "fixed" small photos but not large ones.

Rather than paste the decoder dance a third time, extract `services::imaging::
decode_oriented` and route both workers through it, so there is exactly one way to
turn a file on disk into a DynamicImage. It carries a second invariant that had
also drifted: `image::open` applies NO decode limits, so the export path was
decoding arbitrary user-supplied images unbounded — the decompression-bomb cap
existed only in the compression worker. Both now come as a pair, which is the
point of having one function.

Not done: switching export to consume the existing `display` derivative. It would
fix orientation and drop a redundant full-resolution decode per photo, but it
would also replace the pristine ≤5 MB originals in the keepsake with 2048px
re-encodes — a real quality regression in the one artefact people keep forever.

Test uploads the round-1 fixture (40x20 landscape tagged Orientation=6), runs a
real export, pulls the thumbnail out of Memories.zip and asserts it came back
portrait — with a sanity check that the source really is stored landscape, so the
test can't pass against a pipeline that does nothing.

──────── fix(recover): cap name cycling, and stop bcrypt blocking the runtime

Round 1 gave /join a per-IP ceiling and left /recover with only its
`recover:{ip}:{name}` bucket. That key is right for the job it was written for —
stopping someone who knows a display name (they're listed on the feed) from
burning the victim's 3-strike PIN counter and locking them out on repeat. But the
name is ATTACKER-CHOSEN, so cycling names mints a fresh 5-attempt bucket every
time and the per-IP cost is unbounded.

What sits behind that limiter makes it worse than a normal flood: every call runs
a cost-12 bcrypt verify, including an UNCONDITIONAL throwaway verify for names
that don't exist — added deliberately to close a timing oracle. So an unknown name
is the single cheapest way to make the server do ~200ms of hashing.

Adds `recover_ip_rate_per_min` (default 30, migration 019), checked BEFORE the
per-name bucket so a name generator can't walk past it. 30/min is far above any
real recovery attempt while capping a flood. The per-name bucket is untouched and
remains the anti-guessing control.

The second half matters as much as the first: bcrypt was running inline on the
async runtime everywhere. At cost 12 that pins a tokio worker thread for ~200ms,
and there is only one per core — so a login flood stalled every other request on
the box, including the feed. There was no spawn_blocking anywhere in the auth
module, despite SECURITY-BACKLOG claiming bcrypt had been offloaded.

Route all of it through `verify_password` / `hash_password` on the blocking pool.
That covers /recover, /admin/login, the host PIN reset, and — the one most likely
to bite at a real event — the PIN hash minted on every single /join. Saturating
the blocking pool degrades logins; saturating the worker threads degrades
everything.

Tests: cycling distinct names from one IP now hits the ceiling with a Retry-After,
and — the assertion that keeps the fix honest — repeated wrong PINs against ONE
name are still throttled with the ceiling set generously high, so the ceiling
added protection rather than replacing it.

──────── ci(e2e): run WebKit, so the iOS guarantees actually gate a PR

The workflow installed only Chromium and ran chromium-desktop + chromium-mobile.
iOS Safari is the app's stated primary user — a wedding guest opening a QR link —
and WebKit is the only engine in the matrix that reproduces two of its behaviours:

  - it enforces X-Frame-Options on the hidden download iframe, so a site-wide DENY
    makes the keepsake download silently do nothing. Blink hands attachments to
    the download manager before the frame check and never notices.
  - it abandons a <video> load unless its Range probe gets a 206.

Both of those shipped. Adding 06-export to the webkit project in the round-1 fix
bought nothing on a PR, because CI never ran that project at all — the regression
test written specifically to catch the blocker only ever executed locally.

Runs 71 tests (67 pass, 4 skip on the documented IndexedDB-blob harness
limitation) in ~1.5 minutes locally, using the exact command added here.

──────── chore(backend): satisfy cargo fmt

`checks.yml` runs `cargo fmt --check`, and it has been failing since the round-1
audit fixes: I gated those on `cargo build` and `cargo clippy` but never ran fmt,
so three files drifted then and eight more this round. Pure formatting — no
behaviour change; clippy stays at zero and all 70 backend tests still pass.

Worth noting for next time: clippy passing is not evidence fmt does.

──────── chore: satisfy prettier in frontend and e2e

`checks.yml` runs `npm run format:check` for both projects and both were failing.

- frontend/src/lib/ui-store.ts is mine, unformatted since the round-1 upload-queue
  badge fix — the same miss as the rustfmt one: I gated on svelte-check and eslint
  but never on format:check.
- e2e/loadtest/* and e2e/shots.mjs have been unformatted since 7758270 and are
  unrelated to the audit work. Fixed here because they block the same gate and the
  fix is mechanical; no behaviour change in either project.

Still red and deliberately NOT fixed here: `npm run lint` in the frontend reports
`svelte/prefer-svelte-reactivity` on routes/diashow/+page.svelte:208 (a mutable
`Set` where the rule wants `SvelteSet`), pre-existing since 5009590. That one is a
real reactivity change in code I have no test coverage for, so it belongs in its
own change rather than smuggled into a formatting commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-28 21:28:11 +02:00
fabi
0c0eed885a fix(audit-1): media gating, per-user limits, moderation UI, upload integrity 
First audit round: a blocker and eight high-severity findings across the media
access gate, the guest-facing rate limiters, host moderation, and the upload
pipeline — plus the backup commands and the e2e/production divergences that hid
several of them.

Squashed from 7 commits, original messages preserved below.

──────── fix(export): let the keepsake download through X-Frame-Options on iOS

The keepsake download navigates a hidden, same-origin iframe (deliberately:
a top-level navigation to a 404/429 would unload the PWA). Caddy stamped a
site-wide `X-Frame-Options: DENY` that also covered the proxied `/api/*`.

Blink hands a `Content-Disposition: attachment` response to the download
manager at the network layer, so Chromium never noticed. WebKit enforces XFO
on the frame navigation first and aborts the load — so on iOS Safari, the
app's primary platform, tapping Download did nothing at all, silently.

Carve the two export endpoints out to SAMEORIGIN, which still blocks
cross-origin framing. Implemented as two disjoint matchers rather than an
override: Caddy applies the FIRST `header` directive outermost, so it wins on
write and a later, more specific `header` is silently ignored (verified
against the running test stack).

Also close the test gap that let this ship:

- `06-export` ran on chromium-desktop only; add it to `webkit-iphone`, the
  only engine that enforces XFO on the download frame.
- No test in the suite ever clicked a download button — every archive
  assertion used Node `fetch`, which has no frame and no XFO enforcement.
  Add a spec that clicks it and awaits a real `download` event. Verified
  falsifiable: with the blanket DENY reinstated it fails and reports the
  WebKit refusal as the cause.
- Fix `ExportPage`'s card-scoped locators, which matched nothing: the cards
  carry `class="card p-5"` (a Tailwind `@apply` component class), never the
  `rounded-xl` the page object looked for. This had left the "shows enabled
  download buttons" test red on main.

──────── fix(media): close the percent-escape bypass of the media gate

`/media/%70reviews/{id}.jpg` served a taken-down photo to anyone,
unauthenticated. Verified against the running stack: the literal path 404s,
the escaped one returned 200 with the full image. Same for displays,
thumbnails and originals, and any escaped byte in any position works.

Cause: the block was four `nest_service("/media/previews", 404)` route
matches sitting above a `ServeDir` on `/media`. axum matches on the RAW path
(matchit does no percent-decoding), while `ServeDir` percent-decodes when it
resolves the file. So `%70reviews` missed every blocker, fell through to the
ServeDir, and was decoded back to `previews/` on disk — reaching the bytes
with no soft-delete and no ban-hide check. That defeats a host takedown,
which is the entire point of the gate.

Remove the `/media` route tree outright instead of racing the decoder.
Nothing needs it: every media URL the backend emits is already a gated
`/api/v1/upload/{id}/{original,preview,display,thumbnail}` alias
(handlers::feed), the frontend contains zero `/media/` references, and the
`/media` in config.rs/disk.rs is the filesystem path while `media/` in
export.rs is a path inside the zip. `/media/**` now 404s regardless of
encoding. The route's own comment already said it "serves nothing" — it
wasn't a backstop, it was the vector.

Caddy keeps proxying /media/* deliberately: the app 404s it, and forwarding
means the e2e gating specs exercise the app's refusal exactly as production
would rather than being masked by the SvelteKit 404 page.

Extend the gating spec with the encoded variants — asserting only the literal
spelling is what let this sit undetected.

──────── fix(rate-limit): key the guest-facing limiters per user, not per IP

At a venue every guest is behind one NAT, so an IP-keyed limiter hands the
whole party a single bucket. On a fresh deploy 12 guests arriving together
meant 5 joined and 7 were turned away, with no Retry-After telling them when
to retry. `/feed` (60/min) and `/export` (3 per DAY — the fourth guest to
fetch their keepsake locked out until tomorrow) had the same defect.

`feed_delta` was already keyed per-user and its comment states the exact
rationale ("so one client can't starve others behind a shared NAT"); this
makes its siblings match.

- feed:{ip}   -> feed:{user_id}    (auth was already in scope)
- export:{ip} -> export:{user_id}  (resolved from the download ticket's
  session, which was previously looked up and discarded)
- join:{ip}: pre-auth, so there is no user to key on. Split in two — a loose
  per-IP ceiling that only bounds raw volume (new `join_ip_rate_per_min`,
  default 60, migration 017), plus the real 5/60s anti-spam bucket keyed
  per (ip, name), mirroring the existing `recover:{ip}:{name}`.

admin_login / recover / pin_reset_req stay IP-keyed on purpose and are now
commented as such: they guard credential guessing, where a per-user or
per-name key would just hand an attacker a fresh bucket per guess.

Retry-After: the machinery existed but 7 of 8 sites called `check()` and
hard-coded `None`, so a throttled client was told to back off but never for
how long. Delete the bool `check()` wrapper entirely so `check_with_retry`
is the only entry point and the delay cannot be discarded by accident. Also
surface it for the PIN lockout, where the deadline was already known.

Fix the "unknown" fallback while here: every client_ip() caller passed that
literal, so any request without X-Forwarded-For — anything reaching the app
directly rather than through Caddy — shared ONE global bucket. Serve with
connect-info and use the peer address.

Tests: the reseed forces every limiter toggle off before each test, which is
why this whole class was invisible. Add 01-auth/rate-limit-shared-nat, which
enables them and asserts 12 guests share an IP without collision, that one
guest hammering their own name IS still throttled (so the fix re-keys rather
than removes the limit), and that feed/export buckets are per-user. Retarget
the ddos join test at the new per-IP ceiling — it asserted the defect.

Also seed `admin_login_rate_enabled` (read by the handler, seeded by no
migration and no reseed) and register `join_ip_rate_per_min` in the admin
config allowlist. Unrelated pre-existing red test fixed: 01-auth/join
asserted a "Willkommen!" heading the wedding redesign removed.

──────── feat(moderation): let a host remove a guest's photo or comment from the UI

`DELETE /host/upload/{id}` and `DELETE /host/comment/{id}` were complete on the
backend — transactional, SSE-broadcasting, audit-logged — and had zero frontend
callers. The feed context sheet offered "Löschen" only when
`target.user_id === myUserId`, so the only lever a host actually had against an
unwanted photo was banning the uploader.

That is both disproportionate and ineffective. A ban doesn't retract what was
already posted, and it makes things strictly worse for comments: the ban check
runs BEFORE the ownership check on the guest delete route, so banning an abusive
author leaves their comment on screen and permanently undeletable by them. With
no host affordance, nobody could remove it at all.

- feed: hosts/admins get "Beitrag entfernen" on other people's posts, routed to
  the host endpoint (the guest route 403s anything the caller doesn't own) with
  moderation-specific confirm copy. Own-post "Löschen" is unchanged.
- lightbox: same for comments, via /host/comment/{id}.
- Ban semantics are deliberately untouched (USER_JOURNEYS §10 — banned users keep
  read access and cannot write). The deadlock is broken by giving the host a way
  in, not by loosening the ban.

Live role (this had to come first). `getRole()` decodes the JWT claim, but the
token is never reissued — the backend slides the session row forward and treats
the DB row as authoritative. The claim is therefore frozen for the token's
lifetime: up to 30 days. A guest promoted at the party saw no Host-Dashboard and
no moderation actions until they signed out and back in, even though
`/me/context` had been returning their real role on every page load and 4 of its
6 call sites dropped the field on the floor.

Add `role-store.ts`: seeded from the claim so there's no flash of the wrong nav,
then corrected by every `/me/context` response. Point the ad-hoc `getRole()`
callers at it (account, upload, host, admin, and the new feed gate). The host and
admin dashboards now derive `myRole` reactively, so a demotion disables their
controls immediately instead of at next login.

Tests: 04-host/moderation-ui drives the real UI — host removes a guest photo and
it's gone from /feed server-side; a plain guest is offered nothing on someone
else's post (the mirror that keeps the first test honest); a promoted guest gains
the dashboard on reload while their token still carries `role: guest`; and a host
removes the comment of an already-banned guest, asserting first that the author's
own delete 403s so the deadlock is real.

──────── fix(upload): stop destroying originals, apply EXIF orientation, surface rejections

Three defects in the same pipeline, each of which loses a photo or misrepresents
one.

1. A transient error destroyed the guest's only copy.

`process`'s error arm unconditionally `remove_file`d the original. Every failure
routed there: `create_dir_all`, both derivative `save_with_format` calls (disk
full is the canonical case, and it arrives exactly when many guests upload at
once), a panic inside the image codec, or a momentary DB-pool exhaustion. The
row is only SOFT-deleted, so the bytes were the sole unrecoverable part — and
they were the part we deleted. The author already knew this was wrong next door:
`backfill_missing_display` says it "must NEVER soft-delete an upload that already
has a working preview".

Retry up to 3 times with backoff (re-checking the e2e generation guard after each
sleep), and on final failure keep the refund + soft-delete but leave the original
on disk, logging its path. A failed upload is now recoverable instead of gone.

2. Every portrait photo was stored sideways.

Phones don't rotate sensor data — they record the camera orientation in EXIF and
store the pixels as shot. `decode()` returns those raw pixels and the JPEG
re-encode writes no EXIF, so the 800px preview, the 2048px diashow display and
the keepsake were all rotated 90°, while "Original anzeigen" rendered upright
because the original keeps its tag. That asymmetry is why it reads as a viewer
bug. There was no EXIF handling anywhere in the repo and no exif crate.

Read the tag via `into_decoder()` (which carries the decode Limits through, so
the decompression-bomb cap is untouched) and apply it. Missing/malformed tags
fall back to NoTransforms — most images have none.

Existing derivatives are already baked wrong, so migration 018 adds
`derivatives_rev` and `backfill_missing_display` becomes
`backfill_stale_derivatives`: it now also picks up anything below the current rev
and regenerates it once from the original, which still carries its EXIF. Videos
are marked current in the migration — ffmpeg already honours the rotation matrix.
Bump DERIVATIVES_REV for any future change that invalidates derivatives.

3. A rejected upload vanished without a word.

`UploadQueue.svelte` — 162 lines holding the ONLY renderer of an item's error
text, the only "Erneut" retry button and the only rate-limit countdown — was
never imported anywhere, so `retryItem`, `removeItem` and `clearCompleted` were
unreachable at runtime. On a terminal rejection the store purged the blob and
wrote a clear German reason into `entry.error` "so the UI shows a clear reason".
There was no such UI. And `uploadBadgeCount` counted only pending/uploading, so
the badge decremented exactly as if the upload had succeeded.

Mount the queue on /upload, toast the reason immediately (the flow sends the user
to /feed straight after staging, so the list alone would still miss them), and
count blocked/error in the badge so a failure can't read as success.

Tests: 02-upload/exif-orientation uploads a 40x20 fixture tagged Orientation=6
and asserts both derivatives come back PORTRAIT, with a sanity check that the
source really is stored landscape. 02-upload/rejection-visible bans the uploader
between staging and sending, then asserts the toast, the queue row with the
server's reason, and that the item is still counted.

Note: 02-upload/quota's 4 failures are pre-existing and unrelated — see the next
commit.

──────── docs(backup): make the backup commands work; fix the e2e/prod divergences

Backup. Both documented commands failed on the shipped stack, and the sentence
explaining them was wrong too:

- `pg_dump $DATABASE_URL` — `DATABASE_URL` is only ever in the compose
  environment, never an operator's shell, and it points at `db:5432`, which is
  compose-internal DNS. The app image has no postgres client either.
- `> /media/backups/…` — `/media` is a named volume mounted inside the app
  container, not a host path, and nothing ever creates a `backups` subdirectory.
- `rsync /opt/eventsnap/media/` — that path does not exist anywhere.
- "a single path to back up" — false, and dangerously so: exports were moved to
  their own `exports_data` volume precisely so a keepsake (which contains every
  photo in the event) can't be served off the media tree. Backing up only
  `media_data` silently loses every generated keepsake.

Rewritten as three commands — db via `docker compose exec -T db pg_dump`, and one
`docker run … tar` per volume — all verified against the running stack. The
volume mounts use `/src`, not `/media`: I hit the footgun while testing this.
Docker pre-populates an EMPTY volume from the image's own directory and chowns it
to match, so `-v media_data:/media alpine` tars alpine's cdrom/floppy/usb, writes
them into the volume, and leaves it root-owned so the non-root app can no longer
write. Mounting where the image has nothing avoids all of it. Documented inline
so the next person doesn't rediscover it.

Also correct the architecture notes: `/media/*` no longer routes to the backend
(that static tree was removed as a gating bypass), and `exports_data` was missing
from the volume list — the one volume an operator most needs to know about.

e2e stack: add the `EXPORT_PATH` + `/exports` volume it was missing. The file
says "mirrors production layout"; without these, exports landed on the container's
writable layer at the default path, so export-leak and export-video wrote real
archives into ephemeral storage and the "exports live outside media" invariant
was never actually exercised.

Pre-existing red test, unrelated to the audit: all four 02-upload/quota tests
have been failing since 4464147 "stop /me/quota leaking raw disk to guests"
(2026-07-19), which post-dates the spec's last edit. `setLimitTo` calibrated
`quota_tolerance` from `free_disk_bytes` read through the GUEST's token — a field
that commit deliberately zeroes for non-staff. Dividing by it yields a NaN
tolerance, so every test in the block died in the helper. Read the calibration
inputs through a staff token and keep reading the ceiling back through the guest,
whose limit is the thing under test.

──────── test(e2e): document why WebKit can't run the client-queue upload tests

Three 02-upload tests have been failing on `webkit-iphone` on main — `02-upload`
was already in that project's testMatch, so this is pre-existing red, not
something the audit work introduced.

Root cause is the harness, not the app. Playwright's Linux WebKit build cannot
store Blobs in IndexedDB at all: `put()` fails with "UnknownError: Error
preparing Blob/File data to be stored in object store". Confirmed it is not
about how Playwright delivers files — a Blob constructed in-page with
`new Blob([bytes])` fails identically, while Chromium stores both that and a
`setInputFiles` File without complaint.

That breaks every test driving the composer (FAB → UploadSheet → /upload →
submit), because `handleSubmit` awaits `addToQueue`, which persists the file
before it can navigate. The symptom is a submit button stuck on "Wird
hochgeladen…" and a timeout waiting for /feed — which reads like an app hang and
cost real time to run down.

Skip those four (the three above plus the new rejection-visible) on WebKit only,
behind a named helper carrying the full explanation, so the next person gets the
answer instead of the investigation. Deliberately narrow: WebKit still runs every
API-driven upload test, all of 01-auth, 03-feed and 06-export — including the
keepsake download, which only WebKit can meaningfully verify. Chromium continues
to run all 14.

Worth being explicit, since these tests exist to protect iOS: real Safari
supports Blobs in IndexedDB, so this is NOT evidence that the offline upload
queue is broken on the platform. It does mean that guarantee is currently
unverifiable in CI and rests on Chromium coverage plus manual device testing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-28 19:14:37 +02:00
MechaCat02
a77c2ddc00 Merge branch 'fix/prod-readiness-healthcheck-domain'
Some checks failed
Checks / Backend — cargo test + clippy + fmt (push) Failing after 1m14s
Details
Checks / Frontend — vitest + svelte-check (push) Failing after 5m50s
Details
Checks / E2E — typecheck + lint (push) Failing after 48s
Details
E2E / Playwright E2E (chromium-desktop) (push) Failing after 6m44s
Details
E2E / Cross-UA smoke matrix (push) Failing after 4m32s
Details
Audit / cargo audit (backend) (push) Failing after 12m17s
Details
Audit / npm audit (frontend) (push) Successful in 43s
Details
2026-07-19 18:42:36 +02:00
MechaCat02
40c6fd2ccb fix(deploy): unblock production bring-up (healthchecks + Caddy DOMAIN) 
Two issues would each stop a clean production `docker compose up` for the event:

1. Healthchecks probed http://localhost:{3000,3001}, but the app and frontend
   bind IPv4 (0.0.0.0) while `localhost` resolves to ::1 (IPv6) first inside the
   container — so the probe got "connection refused" and neither container ever
   turned healthy. Caddy is gated on `condition: service_healthy` for both, so on
   a fresh boot it would block forever and nothing gets served. Switch both probes
   to 127.0.0.1. (Verified: both containers now report healthy.)

2. The prod caddy service never received DOMAIN, so the Caddyfile's `{$DOMAIN}`
   site address expanded to empty — malformed site block, no TLS, no serving. Add
   `environment: { DOMAIN: ${DOMAIN} }` to the caddy service.

Also make .env.example honest and event-ready:
- Add DATABASE_MAX_CONNECTIONS (real env lever; recommend 30 for ~100 guests).
- The DEFAULT_* upload/rate/capacity vars are NOT read from env — they are seeded
  into the DB config table and managed at runtime via the admin dashboard. Replace
  the misleading entries (e.g. upload rate showed 10; live value is 100 via
  migration 015) with a note pointing to the admin UI and the real seeded defaults.
- Document that raising COMPRESSION_WORKER_CONCURRENCY also needs the app memory
  limit raised (ffmpeg), so a video burst can't OOM the box.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-19 18:42:32 +02:00
MechaCat02
e69ec4d736 Merge branch 'feat/hide-comments-when-disabled' 2026-07-19 18:27:05 +02:00
MechaCat02
3fb1b5d80d feat(comments): hide every comment mention when COMMENTS_ENABLED=false 
The kill-switch previously left comment UI/text visible in several places.
Sweep the whole surface so a comments-off instance shows no trace:

Frontend (main app):
- VirtualFeed grid tile: gate the comment button/count on $commentsEnabled
  (was ungated — the only feed surface still showing it).
- admin stats: hide the "Kommentare" count card.
- UploadSheet "Uploads geschlossen" notice, host ban-modal description, and
  host/admin unban confirmations: drop the "…und kommentieren" wording.
- export page: drop "Kommentaren" from the keepsake description.

Keepsake export (had no concept of the flag):
- export.rs: thread comments_enabled into the exported data (ViewerEvent),
  wired through spawn_export_jobs/recover_exports and their call sites
  (host.rs, main.rs).
- export-viewer: gate comment counts (list + grid) and the lightbox comments
  section; older exports without the field default to enabled (?? true).

Backend still 403s comment writes when disabled (unchanged) — this is the UI
half so stale clients and archives match.

Verified on the running stack (COMMENTS_ENABLED=false): /event reports
comments_enabled=false, a regenerated keepsake embeds "comments_enabled": false
with no comment UI, and the uploads-closed notice renders "…ansehen und liken."

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-19 18:26:59 +02:00
MechaCat02
e1ca9d192f Merge branch 'feat/diashow-completeness-display' 2026-07-19 17:53:57 +02:00
MechaCat02
5009590882 feat(diashow): guarantee all eligible photos shown + 2048px display derivative 
Diashow completeness rewrite so every eligible upload is shown regardless of
bursts, disconnects, or library size:

- queue.ts: SlideQueue with live/shuffle queues, allKnown map, recentlyShown
  ring; merge(dedup, live-first), remove/removeByUser (prunes recentlyShown),
  knownIds for reconcile-eviction. Adds queue.test.ts (burst/completeness/race).
- diashow/+page.svelte: reconcile (full paginate + evict, pre-scan snapshot to
  spare concurrent uploads) on mount/reconnect/periodic; catchUpNew paginate-
  until-known for bursts with debounced maxWait; hard-cut removals; decode
  timeout + candidate fallback + bounded skip so a broken image never stalls.

New ~2048px "display" derivative for big-screen sharpness, decoupled from the
data-saver preview (800px) used on phones:

- migration 016: upload.display_path + v_feed rebuilt (DROP+CREATE, not REPLACE,
  to slot the column beside preview/thumbnail).
- compression: generate_image_derivatives emits preview+display (downscale-only
  guard, no upscaling); backfill_missing_display regenerates on startup (safe:
  logs on error, never soft-deletes).
- upload.rs/main.rs: GET /upload/{id}/display (mirrors preview auth/cache),
  /media/displays direct-serve blocked.
- feed.rs + types.ts: display_url in feed/delta DTOs.
- diashow candidate chain: display -> original -> preview.

Verified on the running stack: migration applied, 10/10 existing images
backfilled (2048px cap honoured, small images not upscaled), /display serves
200, /feed returns display_url, diashow cycles.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-19 17:53:53 +02:00
MechaCat02
d9738a4cb9 Merge branch 'fix/prod-env-hardening' 
Harden production env handling: single-quote the bcrypt ADMIN_PASSWORD_HASH so
Compose/dotenvy don't corrupt it, and pin MEDIA_PATH to the /media mount.
 2026-07-19 16:46:55 +02:00
MechaCat02
669a191968 fix(deploy): harden prod env for bcrypt hash and media path 
Two latent production pitfalls, both proven by probing Compose's env_file resolution:

- A bcrypt ADMIN_PASSWORD_HASH is full of $; Compose env_file interpolation AND
  dotenvy variable substitution both eat the $… segments (as unset vars), corrupting
  the hash so every admin login 401s. Single-quote it so both read it literally —
  verified correct for env_file (container) and dotenvy 0.15.7 strong-quote (native).
  Updated .env.example with the quotes + a warning comment.

- MEDIA_PATH: pin it to the /media mount in the app 'environment:' block so a stray
  host path in .env can't leak in and make every upload 500 with EACCES. environment
  overrides env_file, so the container is always correct.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-19 16:46:38 +02:00
MechaCat02
a1733b03d5 Merge branch 'feature/diashow-theming-config' 
Runtime colour theme + COMMENTS_ENABLED switch, diashow transition/fullscreen/wake-lock
work, quota disk-leak fix, and docker-compose.dev.yml container-env corrections.
 2026-07-19 15:23:04 +02:00
MechaCat02
4026648f98 fix(diashow): working transitions + slide/push, fullscreen, activity controls 
Transitions never actually animated: Svelte scopes @keyframes names but does NOT
rewrite animation references in inline style attributes, so the inline
'animation: crossfade-in ...' never matched its scoped keyframe — a hard cut, no fade
(Ken Burns' zoom was dead too). Move the animation into scoped classes and pass dynamic
values via CSS custom properties.

- Real two-layer crossfade: hold the outgoing frame opaque beneath the incoming one
  (which is decode-gated), so there's no black flash and no decode pop.
- New slide transitions (from below/left/right) as one reusable component; the previous
  frame is pushed off the opposite edge in step (a conveyor/push), easeInOutSine 1s.
- Fullscreen toggle button (Fullscreen API) + 'f' shortcut; Esc in fullscreen no longer
  also navigates away.
- Controls now reveal on pointer/keyboard activity and fade when idle (cursor hides too)
  instead of being always visible.
- wakelock: null the sentinel on the OS 'release' event so re-acquire-on-visible
  actually fires (previously the screen could sleep after the tab was first hidden).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-19 15:22:15 +02:00
MechaCat02
44641473ea fix(quota): stop /me/quota leaking raw disk to guests 
The per-user quota widget was shown to everyone and the /me/quota payload returned
free_disk_bytes (raw server free space) and active_uploaders to any authenticated
guest. Gate the widget to staff (host/admin) on the upload and account pages, and zero
the server-wide telemetry fields for non-staff in the handler. Guests still get their
own used/limit so enforcement stays transparent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-19 15:22:03 +02:00
MechaCat02
57a907eca5 feat(theme+comments): runtime colour theme and COMMENTS_ENABLED switch 
Colour theme is configurable at runtime from two seed colours (brand + accent);
neutrals stay fixed for contrast safety. Tailwind v4 var()-based tokens let a
:root:root override recolour everything with no rebuild; the 50->950 ramps are
derived via a color-mix ladder. Config lives in the DB config table (admin UI:
Config > Farbschema, presets + custom pickers + live preview), served on the public
/event endpoint with env defaults (THEME_PRESET/PRIMARY/ACCENT), propagated live via
event-updated SSE, and cached in localStorage for a no-flash boot. The keepsake export
mirrors the same ladder in Rust so offline archives match the event theme.

COMMENTS_ENABLED (env, default true) is a boot-time kill-switch: the backend rejects
new comments with 403 and the frontend hides the comment button (feed card) and
panel/composer (lightbox). Existing comments stay in the DB, hidden, and return when
re-enabled.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-19 15:21:55 +02:00
MechaCat02
6e0a760271 chore(dev): correct container env in docker-compose.dev.yml 
Running in a container picked up host-oriented values from .env via env_file:
- MEDIA_PATH pointed at a host path that doesn't exist in the container, so every
  upload 500'd with EACCES; point it at the /media volume mount.
- ADMIN_PASSWORD_HASH lost its $-delimited bcrypt segments to Compose interpolation
  (admin login 401'd); re-supply it with $$ escaping.
- COMMENTS_ENABLED=false to smoke-test the comment kill-switch.

NOTE: production compose + .env carry the same MEDIA_PATH / admin-hash pitfalls.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-19 15:21:44 +02:00
MechaCat02
0abf413693 Merge branch 'test/burst-queue' 
Add an e2e regression covering the client upload queue under a 10-20 photo
burst: serial drain, IndexedDB persistence, all-land, and reload-resume.
 2026-07-18 19:54:44 +02:00
MechaCat02
002355ba40 test(e2e): client upload-queue under a realistic burst 
Cover the scenario the server-side load test could not — a guest multi-selecting
10-20 photos at once — by driving the real client path (UploadSheet → /upload →
addToQueue → processQueue → XHR) rather than hitting POST /upload directly.

Two tests assert the properties that keep a guest's photos from being lost:
serial per-device draining (one upload in flight at a time), IndexedDB
persistence (blobs kept until upload, dropped after), every file landing
server-side, and a hard reload mid-burst resuming the rest via loadQueue()
with no photo lost. Injects small per-upload latency via route interception so
the serial drain is observable and the mid-burst reload window is reliable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-18 19:54:36 +02:00
MechaCat02
6155b4123d Merge branch 'release/v0.16.0' 
Wedding redesign, single-file offline keepsake, diashow SSE fix,
upload-rate 10→100, and a 100-guest load-test harness.
 2026-07-18 17:29:30 +02:00
MechaCat02
7758270cac chore: shared permission allowlist, screenshot script, gitignore 
Add project .claude/settings.json (read-only cargo check/clippy + git diff
allowlist; personal settings.local.json stays gitignored). Add e2e/shots.mjs
(one-off mobile screenshot seeder). Ignore load-test run artifacts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-18 17:29:20 +02:00
MechaCat02
9b8698f86b test(loadtest): 100-guest / 1000-image stress harness 
HTTP-level load driver simulating ~100 guests uploading ~1000 images in bursts
over a window, plus SSE viewers and one real browser on /diashow. Correlates
upload→upload-processed (pipeline latency), waits for the compression backlog
to drain against DB ground truth, and emits per-status/latency metrics with
pass/fail flags. Includes a realistic-JPEG generator and a diashow-SSE
regression check (confirm-diashow-fix.mjs). Run artifacts are gitignored.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-18 17:29:20 +02:00
MechaCat02
3c3a7d0082 fix(diashow): open SSE on mount; raise upload rate 10→100/hour 
diashow only subscribed to SSE events but never called connectSse(), so a
kiosk/projector opening /diashow directly (not via /feed) never opened the
EventSource — the showcase display got a one-time /feed snapshot and no live
updates, showing "Noch keine Beiträge" forever when turned on before any
photos. Open the stream in onMount (idempotent) and close it in onDestroy,
mirroring the feed page.

Raise the default upload_rate_per_hour from 10 to 100 (migration 015, scoped
to installs still on the old default so admin overrides are preserved). Guests
routinely upload bursts of 10-20 photos; the old default throttled the first
burst. Also update the code fallback and the test-mode reseed.

Both verified end-to-end against the docker test stack.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-18 17:29:08 +02:00
MechaCat02
3654aca18b feat(export): single-file offline keepsake viewer + guest download 
Rebuild the guest "keepsake" (offline HTML gallery) so it renders when the
extracted index.html is opened over file://. Browsers block external ES-module
scripts and fetch() at origin null, so a normal multi-file SvelteKit build shows
a blank window. Build the viewer as one self-contained index.html (inlined
JS/CSS via vite-plugin-singlefile, standalone non-SvelteKit entry) and inject
the data as window.__EXPORT_DATA__; the backend embeds the built viewer via
include_dir! and writes the data global into index.html when zipping.

Also surface the guest-facing download: a feed banner and the BottomNav Export
tab, both shown once the host releases the export.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-18 17:28:54 +02:00
MechaCat02
f243bfe89a feat(ui): elegant wedding white/silver/gold redesign 
Reskin the whole app via design tokens in tailwind-theme.css (remapped
color ramps) plus a define-once component layer in lib/styles/components.css
(.btn, .card, .input, .chip, .badge, .sheet, …). Buttons are muted/outlined
gold rather than flat fills. Self-host Inter + Fraunces (woff2) under the
existing font-src 'self' CSP. Restyle the shared components and the account,
admin, host, join, recover and upload screens against the new tokens.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-18 17:28:42 +02:00
173 changed files with 12481 additions and 1819 deletions
Expand all files Collapse all files

9
.claude/settings.json Normal file
View File

@@ -0,0 +1,9 @@
{
"permissions": {
"allow": [
"Bash(cargo check *)",
"Bash(cargo clippy *)",
"Bash(git --no-pager diff *)"
]
}
}

82
.env.example
View File

@@ -12,10 +12,32 @@ APP_ENV=production
# ── Database ──────────────────────────────────────────────────────────────────
# Set a strong password and keep it in sync between DATABASE_URL and
# POSTGRES_PASSWORD. Generate one with: openssl rand -hex 24
#
# SET THIS BEFORE THE FIRST `docker compose up -d`. Postgres reads POSTGRES_PASSWORD
# only when it initialises its data directory, on that very first boot. Change it
# afterwards and the app authenticates with the new password against a volume still
# holding the old one — a permanent restart loop ("password authentication failed").
# The only ways out are restoring the old password or `docker compose down -v`, which
# deletes the database, the media and the exports. In production the app refuses to
# boot while this is still the placeholder below, so it cannot be missed by accident.
DATABASE_URL=postgres://eventsnap:CHANGE_ME_use_a_strong_password@db:5432/eventsnap
POSTGRES_USER=eventsnap
POSTGRES_PASSWORD=CHANGE_ME_use_a_strong_password
POSTGRES_DB=eventsnap
# Connection pool size. Default 10. For a busy event (~100 guests polling the feed
# + SSE + uploads at once) raise to ~30 so requests don't queue on a pool permit.
# PAIRED WITH THE DB CONTAINER'S MEMORY LIMIT: 30 backends plus Postgres 16's default
# shared_buffers is already snug in the 1G that docker-compose.yml allots the `db`
# service. If you raise this, raise `db.deploy.resources.limits.memory` with it — an
# OOM in Postgres doesn't degrade one feature, it takes the whole event down.
DATABASE_MAX_CONNECTIONS=30
# Log level. `info` is the right production default: at `debug` the tower-http trace
# layer writes a line per request AND per response, which on a busy event is a large
# multiple of the useful output. Container logs are capped at 10m x 3 per service
# (docker-compose.yml), so a chatty level buys you a shorter history, not more of it.
# To debug a live event: RUST_LOG=eventsnap_backend=debug docker compose up -d app
RUST_LOG=info
# ── Authentication ────────────────────────────────────────────────────────────
# Generate with: openssl rand -hex 64
@@ -23,8 +45,14 @@ JWT_SECRET=change_me_to_a_random_64_byte_hex_string
SESSION_EXPIRY_DAYS=30
# Admin dashboard password (bcrypt hash).
# Generate with: htpasswd -bnBC 12 "" yourpassword | tr -d ':\n'
ADMIN_PASSWORD_HASH=$2y$12$placeholder_replace_me
# Generate with an image the stack already pulls (htpasswd needs apache2-utils, which
# a stock VPS does not have):
# docker run --rm caddy:2-alpine caddy hash-password --plaintext 'yourpassword'
# IMPORTANT: keep the SINGLE QUOTES. A bcrypt hash is full of `$` (e.g. $2b$12$…$…),
# and both Docker Compose's env_file interpolation and dotenvy's variable substitution
# would otherwise eat the `$…` segments (reading them as unset vars) and corrupt the
# hash — every admin login then 401s. Single quotes make both read it literally.
ADMIN_PASSWORD_HASH='$2y$12$placeholder_replace_me'
# ── Event ─────────────────────────────────────────────────────────────────────
EVENT_NAME=Max & Maria's Wedding
@@ -36,19 +64,43 @@ MEDIA_PATH=/media
# /media is publicly served, so exports here would be downloadable without auth.
EXPORT_PATH=/exports
# ── Upload limits ─────────────────────────────────────────────────────────────
DEFAULT_MAX_IMAGE_SIZE_MB=20
DEFAULT_MAX_VIDEO_SIZE_MB=500
# ── Rate limiting ─────────────────────────────────────────────────────────────
DEFAULT_UPLOAD_RATE_PER_HOUR=10
DEFAULT_FEED_RATE_PER_MIN=60
DEFAULT_EXPORT_RATE_PER_DAY=3
# ── Capacity ──────────────────────────────────────────────────────────────────
DEFAULT_ESTIMATED_GUEST_COUNT=100
# Fraction of total storage that triggers the "low storage" warning (0.0–1.0)
DEFAULT_QUOTA_TOLERANCE=0.75
# ── Runtime settings (upload limits, rate limits, capacity) ───────────────────
# NOTE: These are NOT environment variables. Upload size caps, rate limits, guest
# count and quota tolerance are stored in the database `config` table (seeded once
# at first boot) and changed at runtime from the ADMIN DASHBOARD — the backend does
# not read them from .env. Setting them here has no effect. Current seeded defaults:
# upload rate 100 / hour / guest (raised from 10 by migration 015)
# feed rate 60 / minute
# export rate 3 / day
# max image size 20 MB
# max video size 500 MB
# estimated guests 100
# quota tolerance 0.75 (see below — NOT a warning threshold)
# Adjust these in the admin UI before the event if needed.
#
# quota_tolerance is the MULTIPLIER IN THE PER-USER QUOTA FORMULA, not the point at
# which anything warns you:
#
# per_user_limit = floor(free_disk * quota_tolerance / active_uploaders)
#
# It is recomputed against LIVE free space on every upload, so it self-throttles: guests
# converge on a fixed point at tolerance/(1+tolerance) of the free space you started
# with — 43% at 0.75, i.e. ~30 GB of a fresh 70 GB.
#
# Raising it therefore AUTHORISES GUESTS TO FILL MORE OF THE DISK. Setting 0.95 in the
# belief that it means "warn me later" moves the fixed point to ~49% and eats the
# headroom the keepsake needs — and the keepsake needs a lot, because Gallery.zip and
# Memories.zip are each roughly a second copy of every original (both store media
# uncompressed). Budget for media + 2x media, or move exports to their own volume.
#
# 0.75 is the tested default. Lower it if the box is tight; raise it only if you have
# provisioned export headroom separately.
# ── Workers ───────────────────────────────────────────────────────────────────
# Number of parallel image/video compression workers. Default 2. This is the main
# throughput bottleneck: with 2 workers a burst of uploads can take ~5s to appear.
# For a large event (100+ guests) 4 is a good target — but each worker can run an
# ffmpeg transcode, so if you raise this ALSO raise the app container's memory limit
# in docker-compose.yml (`app.deploy.resources.limits.memory`) from 1G to ~2G, or a
# burst of large videos can OOM the box and take Postgres down with it.
COMPRESSION_WORKER_CONCURRENCY=2

20
.github/workflows/e2e.yml vendored
View File

@@ -7,7 +7,7 @@ on:
jobs:
e2e:
name: Playwright E2E (chromium-desktop)
name: Playwright E2E (chromium + webkit)
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
@@ -25,7 +25,7 @@ jobs:
- name: Install Playwright browsers
working-directory: ./e2e
run: npx playwright install --with-deps chromium
run: npx playwright install --with-deps chromium webkit
- name: Bring up the test stack
working-directory: ./e2e
@@ -54,6 +54,22 @@ jobs:
working-directory: ./e2e
run: npm run test:e2e -- --project=chromium-mobile
# iOS Safari is the app's stated primary user (a wedding guest opening a QR link), and
# WebKit is the ONLY engine here that reproduces two of its behaviours:
# - it enforces X-Frame-Options on the download iframe, so a site-wide `DENY` makes the
# keepsake download silently do nothing. Blink hands attachments to the download
# manager first and never notices. That shipped once already.
# - it abandons a <video> load without a 206 response to its Range probe.
# Both regressions are invisible to every Chromium project, so running WebKit is what
# actually gates them on a PR rather than on someone remembering to test locally.
#
# The project is scoped in playwright.config.ts to the journeys a guest walks
# (01-auth, 02-upload, 03-feed, 06-export); four IndexedDB-blob tests skip themselves
# there — see helpers/webkit.ts for why that is the harness and not the app.
- name: Run E2E tests (webkit / iOS)
working-directory: ./e2e
run: npm run test:e2e -- --project=webkit-iphone
- name: Upload Playwright report
if: failure()
uses: actions/upload-artifact@v4

15
.gitignore vendored
View File

@@ -13,8 +13,16 @@ frontend/build/
frontend/export-viewer/node_modules/
frontend/export-viewer/.svelte-kit/
# Media uploads (mounted volume in production)
media/
# Media uploads. In production these live in the `media_data` DOCKER VOLUME, never in the
# working tree — so this pattern is anchored to the repo root and exists only for a local
# bind-mount experiment.
#
# It used to read `media/`, unanchored, which matches a directory of that name at ANY depth.
# The only one in the repo is `e2e/fixtures/media/`, so the rule's entire practical effect was
# to keep every E2E fixture untracked: a fresh clone got the specs and none of the images or
# videos they read. `.github/workflows/e2e.yml` does a plain checkout and generates nothing, so
# the committed CI job could not have run the upload, video or export suites at all.
/media/
# Playwright E2E suite — runtime artifacts (the suite itself is committed)
e2e/node_modules/
@@ -29,3 +37,6 @@ e2e/.env.test
# OS
.DS_Store
Thumbs.db
# Claude Code personal (per-user) settings — shared settings.json IS committed
.claude/settings.local.json

43
Caddyfile
View File

@@ -9,34 +9,63 @@
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains"
X-Content-Type-Options "nosniff"
X-Frame-Options "DENY"
Referrer-Policy "strict-origin-when-cross-origin"
}
# X-Frame-Options: DENY everywhere EXCEPT the keepsake download endpoints, which
# are navigated in a HIDDEN, SAME-ORIGIN iframe so a 404/429 can't unload the PWA
# (see frontend/src/routes/export/+page.svelte). WebKit enforces XFO *before*
# honouring Content-Disposition, so a blanket DENY makes the download silently do
# nothing on iOS Safari — the app's primary platform. SAMEORIGIN still blocks
# cross-origin framing.
#
# Split into two disjoint matchers rather than an override: Caddy applies the
# FIRST header directive outermost, so it wins on write — a later, more specific
# `header` would be silently ignored.
@framable path /api/v1/export/zip /api/v1/export/html
@not_framable not path /api/v1/export/zip /api/v1/export/html
header @framable X-Frame-Options "SAMEORIGIN"
header @not_framable X-Frame-Options "DENY"
# SvelteKit frontend — static assets with long-lived cache (content-hashed filenames)
@hashed_assets path_regexp hashed /_app/immutable/.*\.[a-f0-9]{8,}\.(js|css|woff2)$
header @hashed_assets Cache-Control "public, max-age=31536000, immutable"
# Preview/thumbnail images. These are now served by the app through a
# visibility-checked alias (/api/v1/upload/{id}/{preview,thumbnail}) so moderation
# can revoke access; direct /media/previews|thumbnails is 404-blocked at the app.
# Preview/thumbnail images. These are served by the app through a visibility-checked
# alias (/api/v1/upload/{id}/{preview,thumbnail}) so moderation can revoke access;
# the app serves no /media route at all, so there is no direct path to the bytes.
# Privately cacheable for a short window (the app sets the same header; this is the
# edge carve-out from the blanket no-store below). Kept short so a moderated image
# stops being served to a direct-URL holder promptly.
@media_api path /api/v1/upload/*/preview /api/v1/upload/*/thumbnail
header @media_api Cache-Control "private, max-age=300"
# API — never cache, EXCEPT the gated image routes above.
# API and health — never cache, EXCEPT the gated image routes above. A cached health
# response would report the last known state rather than the current one.
@api {
path /api/*
path /api/* /health
not path /api/v1/upload/*/preview /api/v1/upload/*/thumbnail
}
header @api Cache-Control "no-store"
# Route API and media requests to the Rust backend
# Route API and media requests to the Rust backend.
#
# The app serves no /media route at all (see the note in backend/src/main.rs) — media
# bytes are reachable only through the visibility-checked /api/v1/upload aliases, so
# /media/* forwards to a plain 404. The proxy line is kept deliberately: it means the
# edge faithfully hands /media to the app, so if a future change ever re-introduces a
# static media route the e2e gating specs see it here exactly as production would,
# instead of being masked by the SvelteKit 404 page.
reverse_proxy /api/* app:3000
reverse_proxy /media/* app:3000
# The backend registers /health on its ROOT router, not under /api/v1, so it needs its
# own line — without it the catch-all below hands /health to SvelteKit, which has no
# such route and returns its 404 page. That made the documented post-deploy check
# (`curl -fsS https://DOMAIN/health`) fail 100% of the time on a perfectly healthy
# stack. e2e/Caddyfile.test has always carried this line; production never did.
reverse_proxy /health app:3000
# Everything else goes to SvelteKit frontend
reverse_proxy frontend:3001
}

21
PROJECT.md
View File

@@ -709,7 +709,7 @@ CREATE TABLE config (
INSERT INTO config (key, value) VALUES
('max_image_size_mb', '20'),
('max_video_size_mb', '500'),
('upload_rate_per_hour', '10'),
('upload_rate_per_hour', '100'), -- raised from 10 in migration 015 (guests upload bursts of 10-20)
('feed_rate_per_min', '60'),
('export_rate_per_day', '3'),
('quota_tolerance', '0.75'),
@@ -1133,16 +1133,19 @@ eventsnap/
### Backup Strategy
```bash
# Daily (e.g. as a separate Compose service or cron on the VPS)
pg_dump $DATABASE_URL | gzip > /media/backups/db_$(date +%Y-%m-%d).sql.gz
Three artefacts in three places: the database, the `media_data` volume
(originals + derivatives), and the **separate** `exports_data` volume. See
[README.md](README.md#backup) for the exact commands.
# Weekly: rsync /media volume to Hetzner Storage Box
rsync -az /opt/eventsnap/media/ \
user@u123456.your-storagebox.de:backup/eventsnap/
```
Everything runs through `docker compose` / `docker run`, because `DATABASE_URL`
and the `/media` and `/exports` paths only exist inside the compose network —
they are not host paths, and `DATABASE_URL` is never exported into an operator's
shell.
The `/media` volume contains originals, previews, thumbnails, generated exports, and DB backups — a single volume to back up.
Export archives are deliberately outside `MEDIA_PATH` (`EXPORT_PATH=/exports`): a
keepsake contains every photo in the event, and keeping it off the media tree is
what stops it being reachable except through the ticket-gated handler. A backup
of the media volume alone silently loses every generated keepsake.
---

284
README.md
View File

@@ -34,7 +34,6 @@ A guest scans the QR code on their way in, types their name, and is immediately
### Planned (v1.x)
- Individual file download button
- Low-disk alert (< 10 GB free)
- Event banner / cover image
- Chunked resumable upload for large videos
- Host-curated story highlights
@@ -98,33 +97,135 @@ eventsnap/
git clone https://git.mc02.dev/fabi/EventSnap.git eventsnap
cd eventsnap
# 2. Configure environment
# 2. Configure environment — set EVERY secret NOW, before step 3.
cp .env.example .env
nano .env # set DOMAIN, JWT_SECRET, ADMIN_PASSWORD_HASH, EVENT_NAME, etc.
nano .env # DOMAIN, EVENT_NAME, EVENT_SLUG,
# JWT_SECRET, ADMIN_PASSWORD_HASH,
# POSTGRES_PASSWORD *and* the same password inside DATABASE_URL
# (see "Generate required secrets" below)
# 3. Start the stack
docker compose up -d
```
> **Set every secret before step 3 — `POSTGRES_PASSWORD` especially.** Postgres reads it **only
> when it initialises its data directory**, which happens on the very first `docker compose up -d`.
> Changing it in `.env` afterwards does not change the stored password: the app then authenticates
> with the new one against a volume holding the old one, and you get a permanent restart loop with
> `password authentication failed for user "eventsnap"`. The only fixes are restoring the old
> password or `docker compose down -v`, which **deletes the database, the media and the exports**.
> Getting it right once, up front, costs nothing; getting it wrong costs the volume.
Caddy automatically obtains a Let's Encrypt certificate on first start. The app is live at `https://DOMAIN` within ~30 seconds.
> **If the site never comes up:** with `APP_ENV=production` the backend **refuses to boot** while `JWT_SECRET`/`ADMIN_PASSWORD_HASH` still hold the `.env.example` placeholders (this is deliberate — a publicly-known signing key is worse than downtime). Caddy then waits on the unhealthy `app` container and never serves. Check `docker compose logs app` — a "Refusing to start … placeholder …" line means you skipped step 2. Rotate the secrets (see below) and restart.
> **If the site never comes up:** with `APP_ENV=production` the backend **refuses to boot** while
> `JWT_SECRET`, `ADMIN_PASSWORD_HASH` or the password inside `DATABASE_URL` still hold the
> `.env.example` placeholders (this is deliberate — a publicly-known signing key or database
> password is worse than downtime). Caddy then waits on the unhealthy `app` container and never
> serves. Check `docker compose logs app` — a "Refusing to start … placeholder …" line lists
> **every** unset secret at once, so one edit fixes them all.
>
> **If it comes up but keeps restarting with `password authentication failed for user
> "eventsnap"`:** `POSTGRES_PASSWORD` was changed after the database volume was created. Postgres
> applies that variable only at initialisation, so `.env` and the stored password have drifted
> apart permanently. `docker compose logs app` spells this out. Before the event, with nothing
> worth keeping:
>
> ```bash
> docker compose down -v && docker compose up -d # -v DELETES db + media + exports. No undo.
> ```
>
> **Once the event has real data, never do that.** Put the original password back into
> `DATABASE_URL`, or change the stored one instead:
>
> ```bash
> docker compose exec db psql -U "$POSTGRES_USER" -c \
> "ALTER ROLE eventsnap WITH PASSWORD 'the-password-now-in-your-.env';"
> ```
> **Production note:** `docker compose up -d` does **not** expose the database — Postgres is reachable only on the internal Docker network. For local development where you need host access to Postgres, opt into the dev overlay explicitly:
> ```bash
> docker compose -f docker-compose.yml -f docker-compose.dev.yml up
> ```
### Updating an existing deployment
> **`docker compose up -d` alone will NOT deploy your changes.** `app` and `frontend` are
> `build:` services with no published image tag, and Compose has no source-change detection:
> if an image with that name already exists it is reused. After a `git pull` the command
> reports `Container … Running`, changes nothing, and **exits 0** — so a deploy that shipped
> nothing looks exactly like a successful one. `--build` is what makes it real.
```bash
cd /path/to/eventsnap
# 1. Back up first — migrations run automatically on boot and are not reversible in place.
# (See "Backup" below; the database dump is the one that matters here.)
# 2. Fetch the new code.
git pull
# 3. Rebuild and restart the application services. --build is NOT optional.
docker compose up -d --build
# 4. Apply any Caddyfile change. Step 3 does NOT do this — see the warning below.
docker compose up -d --force-recreate caddy
# 5. Confirm the app came back up. Anything other than "ok" means check the logs.
curl -fsS https://DOMAIN/health && echo
# 6. Confirm a NEW image was actually built. Note the IMAGE ID before you start and
# compare — it must have changed. (Ignore the CREATED column; it reports the base
# layer's age, not this build's.) An unchanged ID means step 3 ran without --build
# and you are still serving the old code.
docker compose images app frontend
```
Migrations are applied by the backend on startup, so step 3 covers them. If `app` stays
unhealthy afterwards, `docker compose logs app` will name the failing migration — and note
that a migration applied by a *newer* build is not removed by checking out an older commit,
so rolling back code without restoring the database snapshot from step 1 leaves the schema
ahead of the binary and the app refusing to boot.
> **Why step 4 exists.** `--build` only rebuilds services that have a `build:` section, and
> `caddy` is a pinned upstream image. Compose decides whether to recreate a container from its
> *config hash*, which covers the mount **specification** (`./Caddyfile:/etc/caddy/Caddyfile:ro`)
> but **not the file's contents** — so a `git pull` that changes `./Caddyfile` produces no
> delta, Compose reports `Running`, and Caddy keeps serving its old config indefinitely. Exit
> code 0 throughout.
>
> That is not hypothetical: the fix that made the keepsake download work on iOS
> (`137c4ee`) touched the Caddyfile and four e2e files and nothing else, so **all** of its
> production effect lives in that one file. Without step 4 you deploy it, watch both image IDs
> change, and iOS downloads stay broken.
>
> `--force-recreate` rather than `restart` or `caddy reload`: the bind mount is resolved to an
> **inode** when the container is created, and `git pull` replaces the file instead of editing
> it in place, so the container can still be bound to the old, now-unlinked inode. A restart
> then re-reads the stale content. Recreating the container re-resolves the path.
`db` is never touched, and recreating `caddy` does not disturb the `caddy_data` volume, so the
TLS certificate and all data volumes survive.
### Generate required secrets
```bash
# JWT secret (64 random bytes)
openssl rand -hex 64
# Admin password hash (bcrypt, cost 12)
htpasswd -bnBC 12 "" yourpassword | tr -d ':\n'
# Database password (goes in BOTH DATABASE_URL and POSTGRES_PASSWORD)
openssl rand -hex 24
# Admin password hash (bcrypt). Uses an image the stack already pulls, so it needs
# nothing installed on the host — `htpasswd` lives in apache2-utils, which a stock
# VPS does not have. Emits cost 14 rather than 12; that is fine (admin login is
# rate-limited and hashed off the async runtime), and any $2a/$2b/$2y hash verifies.
docker run --rm caddy:2-alpine caddy hash-password --plaintext 'yourpassword'
```
Wrap the resulting hash in **single quotes** in `.env` — see the note there; a bcrypt
hash is full of `$`, and both Compose and dotenvy would otherwise eat those segments.
### Environment Variables
See [.env.example](.env.example) for the full list with descriptions and defaults. Key variables:
@@ -162,23 +263,176 @@ See [.env.example](.env.example) for the full list with descriptions and default
└────────┘
```
- `/api/*` and `/media/*` → Rust backend
- `/api/*` → Rust backend
- Everything else → SvelteKit frontend (`adapter-node`)
- Named volumes: `postgres_data`, `media_data`, `caddy_data`
- Named volumes: `postgres_data`, `media_data`, `exports_data`, `caddy_data`
Media is **not** served as static files. Every image goes through a
visibility-checked alias (`/api/v1/upload/{id}/{preview,display,thumbnail,original}`)
so a host takedown or a ban actually revokes access to the bytes.
---
## Sizing the disk
`postgres_data`, `media_data` and `exports_data` are all Docker named volumes under
`/var/lib/docker/volumes`, so **they share one filesystem**. Filling it does not
degrade one subsystem — Postgres stops being able to write and the whole event goes
down.
Uploads are self-limiting. `per_user_limit = free_disk × quota_tolerance ÷
active_uploaders` is recomputed against live free space on every upload, so guests
converge on a fixed point at `tolerance / (1 + tolerance)` of the free space you
started with — **43%** at the default 0.75. On an 80 GB box with ~70 GB free after
the OS and images, media settles at ~30 GB and stops.
**The keepsake is what the 80 GB baseline does not cover.** `Gallery.zip` and
`Memories.zip` are built concurrently and each is roughly a second copy of every
original: both write their media `Compression::Stored`, and `Memories.zip` streams the
untouched original for every video and for every image at or under 5 MB. So a release
wants room for **two more copies of the gallery** on top of the gallery itself.
| Stage | Used | Free (80 GB box) |
|---|---|---|
| Fresh box (OS + images) | ~10 GB | ~70 GB |
| Guests reach the quota fixed point | ~40 GB | ~40 GB |
| Host releases → both archives | ~100 GB | **ENOSPC** |
Two ways to size for it:
- **Provision ~3× your expected media** on one volume (media + two archives), or
- **give `exports_data` its own volume** so a full export cannot reach Postgres, and
size that one at ~2× expected media.
This is no longer silent. The export refuses up front with the two numbers rather than
hitting ENOSPC halfway through a multi-GB write, a rebuild reclaims the superseded
generation before it starts (so peak is one generation, not two), and the host
dashboard warns as soon as the keepsake would not fit — which is the only point at
which anyone can still do something about it.
---
## Backup
```bash
# Database snapshot
pg_dump $DATABASE_URL | gzip > /media/backups/db_$(date +%Y-%m-%d).sql.gz
There are **three** things to back up, and they live in three different places.
`DATABASE_URL` and the container paths (`/media`, `/exports`) are meaningful only
*inside* the compose network — they are not host paths, and `DATABASE_URL` is
never exported into an operator's shell — so every command below runs through
`docker compose` from the repo directory.
# Weekly offsite sync (Hetzner Storage Box or similar)
rsync -az /opt/eventsnap/media/ user@storagebox.example.com:backup/eventsnap/
```bash
# 1. Database snapshot. Runs pg_dump inside the db container (the app image has no
# postgres client), reading credentials from the compose environment.
# --clean --if-exists makes the dump SELF-CLEANING: without it the restore below
# aborts on the first "already exists" against a database that has ever booted,
# which is every database you would actually want to restore over.
mkdir -p ./backups
docker compose exec -T db \
sh -c 'pg_dump --clean --if-exists -U "$POSTGRES_USER" "$POSTGRES_DB"' \
| gzip > ./backups/db_$(date +%Y-%m-%d).sql.gz
# 2. Uploaded media (originals + derivatives) out of the named volume.
# NOTE the mountpoint is /src, not /media: if the volume is ever empty, Docker
# pre-populates a fresh mount from the image's own directory, and alpine ships a
# /media containing cdrom/floppy/usb. Mounting somewhere the image has nothing
# avoids silently tarring (and polluting the volume with) those.
docker run --rm \
-v eventsnap_media_data:/src:ro -v "$PWD/backups":/backup \
alpine tar czf /backup/media_$(date +%Y-%m-%d).tar.gz -C /src .
# 3. Export archives — a SEPARATE volume (see the security note below).
docker run --rm \
-v eventsnap_exports_data:/src:ro -v "$PWD/backups":/backup \
alpine tar czf /backup/exports_$(date +%Y-%m-%d).tar.gz -C /src .
# Offsite sync of the three artefacts above.
rsync -az ./backups/ user@storagebox.example.com:backup/eventsnap/
```
The `/media` volume holds originals, previews, thumbnails, exports, and DB backups — a single path to back up.
Volume names are prefixed with the compose project name — `eventsnap_` if you run
from a directory called `eventsnap`. Confirm yours with `docker volume ls`.
> **Exports are deliberately NOT under `/media`.** They live on their own
> `exports_data` volume (`EXPORT_PATH=/exports`) because a keepsake archive
> contains every photo in the event; keeping it outside the media tree is what
> stops it being reachable except through the ticket-gated download handler.
> Backing up only the media volume therefore loses every generated keepsake.
### When to run it
**A nightly cron is the wrong shape for this app.** Every irreplaceable byte is
created inside one eight-hour window, and nobody can retake a wedding. Run the three
commands above:
1. **The night of the event**, once uploads have stopped. This is the backup that
matters; everything else is a formality.
2. **After the host releases the gallery**, so the generated keepsake is captured too.
3. Weekly thereafter, until the event is archived and torn down.
Take the DB dump and the media tarball **back to back**, without uploads in flight
between them. Upload rows reference files by path — a database from 22:00 and a media
volume from 23:00 gives you rows pointing at files the dump doesn't know about, and
rows whose files aren't in the tarball. Locking uploads from the host dashboard first
(**Uploads sperren**) makes the pair genuinely consistent.
---
## Restore
An untested backup is not a backup. Run this once against a scratch host **before**
the event — it is roughly ten minutes, and it is the only way to find out that your
tarball is empty or your dump is truncated while that is still a small problem.
```bash
# 0. Stop the app FIRST. Migrations run on boot and a live pool will fight the
# restore — a booting app against a half-restored schema can leave the migration
# table and the schema disagreeing, which is its own recovery problem.
# Leave `db` running: the dump is restored through it.
docker compose stop app caddy
# 1. Database. The dump carries its own DROPs (step 1 of Backup), so this replaces
# rather than collides. A dump taken WITHOUT --clean --if-exists will abort here
# on the first "already exists" — restore that one into a fresh empty database
# instead.
gunzip -c ./backups/db_2026-07-29.sql.gz \
| docker compose exec -T db \
sh -c 'psql -U "$POSTGRES_USER" -d "$POSTGRES_DB" --set ON_ERROR_STOP=1'
# 2. Media. NOTE the `--numeric-owner` and the chown: the app runs as a
# NON-ROOT user (uid 100, gid 101 — `addgroup -S app && adduser -S app`), and a
# restore that lands root-owned files makes every upload fail with EACCES deep in
# the write path, surfacing to the guest as a generic 500 with nothing in the UI
# to suggest permissions. The explicit chown is what guarantees it — BusyBox tar
# (which is what `alpine` ships) has no --same-owner, and restores ownership only
# because it runs as root here.
docker run --rm \
-v eventsnap_media_data:/dst -v "$PWD/backups":/backup:ro \
alpine sh -c 'tar xzf /backup/media_2026-07-29.tar.gz -C /dst \
--numeric-owner && chown -R 100:101 /dst'
# 3. Exports. Same volume-name caveat, same ownership rules.
docker run --rm \
-v eventsnap_exports_data:/dst -v "$PWD/backups":/backup:ro \
alpine sh -c 'tar xzf /backup/exports_2026-07-29.tar.gz -C /dst \
--numeric-owner && chown -R 100:101 /dst'
# 4. Back up. Migrations run, then export recovery re-arms any keepsake whose file
# didn't come back with the volume.
docker compose up -d app caddy
docker compose logs -f app # watch for "migrations applied"
# 5. Verify — all three, not just the first.
curl -fsS https://DOMAIN/health && echo # → ok
# … then sign in as host and confirm the feed renders images (proves the media
# volume restored AND is readable by uid 100), and that the keepsake downloads.
```
If the media volume restored but images 404 while the feed lists them, the paths are
there and the bytes aren't — check `docker compose exec app ls -ln /media/originals`
and confirm both the files and the `100:101` ownership.
The restore is deliberately **not** automated. It is rare, destructive, and the one
operation where a script that half-works is worse than a checklist someone reads.
---
@@ -254,7 +508,7 @@ Open:
- [ ] SSE delta-fetch on foreground reconnect (scaffolded in [sse.ts](frontend/src/lib/sse.ts), not wired)
- [ ] Live diashow / slideshow mode — see [docs/CONCEPT_DIASHOW.md](docs/CONCEPT_DIASHOW.md)
- [ ] Individual file download button per post
- [ ] Low-disk alert (< 10 GB free)
- [x] Low-disk alert — host dashboard warns below 10 GB free, or whenever the keepsake would not fit
- [ ] Event banner / cover image
- [ ] Chunked resumable upload for files > 100 MB
- [ ] Shared Tailwind config between main app and export-viewer

192
backend/Cargo.lock generated
View File

@@ -65,56 +65,6 @@ dependencies = [
"libc",
]
[[package]]
name = "anstream"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "824a212faf96e9acacdbd09febd34438f8f711fb84e09a8916013cd7815ca28d"
dependencies = [
"anstyle",
"anstyle-parse",
"anstyle-query",
"anstyle-wincon",
"colorchoice",
"is_terminal_polyfill",
"utf8parse",
]
[[package]]
name = "anstyle"
version = "1.0.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "940b3a0ca603d1eade50a4846a2afffd5ef57a9feac2c0e2ec2e14f9ead76000"
[[package]]
name = "anstyle-parse"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "52ce7f38b242319f7cabaa6813055467063ecdc9d355bbb4ce0c68908cd8130e"
dependencies = [
"utf8parse",
]
[[package]]
name = "anstyle-query"
version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc"
dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "anstyle-wincon"
version = "3.0.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d"
dependencies = [
"anstyle",
"once_cell_polyfill",
"windows-sys 0.61.2",
]
[[package]]
name = "anyhow"
version = "1.0.102"
@@ -554,46 +504,12 @@ dependencies = [
"inout",
]
[[package]]
name = "clap"
version = "4.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b193af5b67834b676abd72466a96c1024e6a6ad978a1f484bd90b85c94041351"
dependencies = [
"clap_builder",
]
[[package]]
name = "clap_builder"
version = "4.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "714a53001bf66416adb0e2ef5ac857140e7dc3a0c48fb28b2f10762fc4b5069f"
dependencies = [
"anstream",
"anstyle",
"clap_lex",
"strsim",
"terminal_size",
]
[[package]]
name = "clap_lex"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c8d4a3bb8b1e0c1050499d1815f5ab16d04f0959b233085fb31653fbfc9d98f9"
[[package]]
name = "color_quant"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3d7b894f5411737b7867f4827955924d7c254fc9f4d91a6aad6b097804b1018b"
[[package]]
name = "colorchoice"
version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
[[package]]
name = "compression-codecs"
version = "0.4.37"
@@ -677,15 +593,6 @@ dependencies = [
"cfg-if",
]
[[package]]
name = "crossbeam-channel"
version = "0.5.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "82b8f8f868b36967f9606790d1903570de9ceaf870a7bf9fbbd3016d636a2cb2"
dependencies = [
"crossbeam-utils",
]
[[package]]
name = "crossbeam-deque"
version = "0.8.6"
@@ -816,27 +723,6 @@ dependencies = [
"cfg-if",
]
[[package]]
name = "env_filter"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32e90c2accc4b07a8456ea0debdc2e7587bdd890680d71173a15d4ae604f6eef"
dependencies = [
"log",
]
[[package]]
name = "env_logger"
version = "0.11.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0621c04f2196ac3f488dd583365b9c09be011a4ab8b9f37248ffcc8f6198b56a"
dependencies = [
"anstream",
"anstyle",
"env_filter",
"log",
]
[[package]]
name = "equator"
version = "0.4.2"
@@ -1229,12 +1115,6 @@ dependencies = [
"weezl",
]
[[package]]
name = "glob"
version = "0.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0cc23270f6e1808e30a928bdc84dea0b9b4136a8bc82338574f23baf47bbd280"
[[package]]
name = "governor"
version = "0.6.3"
@@ -1622,7 +1502,6 @@ checksum = "7714e70437a7dc3ac8eb7e6f8df75fd8eb422675fc7678aff7364301092b1017"
dependencies = [
"equivalent",
"hashbrown 0.16.1",
"rayon",
"serde",
"serde_core",
]
@@ -1656,12 +1535,6 @@ dependencies = [
"syn",
]
[[package]]
name = "is_terminal_polyfill"
version = "1.70.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a6cb138bb79a146c1bd460005623e142ef0181e3d0219cb493e02f7d08a35695"
[[package]]
name = "itertools"
version = "0.14.0"
@@ -1795,12 +1668,6 @@ dependencies = [
"vcpkg",
]
[[package]]
name = "linux-raw-sys"
version = "0.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
[[package]]
name = "litemap"
version = "0.8.1"
@@ -2089,12 +1956,6 @@ version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
[[package]]
name = "once_cell_polyfill"
version = "1.70.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe"
[[package]]
name = "oxipng"
version = "9.1.5"
@@ -2102,18 +1963,12 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "26c613f0f566526a647c7473f6a8556dbce22c91b13485ee4b4ec7ab648e4973"
dependencies = [
"bitvec",
"clap",
"crossbeam-channel",
"env_logger",
"filetime",
"glob",
"indexmap",
"libdeflater",
"log",
"rayon",
"rgb",
"rustc-hash",
"zopfli",
]
[[package]]
@@ -2607,19 +2462,6 @@ version = "2.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "94300abf3f1ae2e2b8ffb7b58043de3d399c73fa6f4b73826402a5c457614dbe"
[[package]]
name = "rustix"
version = "1.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
dependencies = [
"bitflags",
"errno",
"libc",
"linux-raw-sys",
"windows-sys 0.61.2",
]
[[package]]
name = "rustversion"
version = "1.0.22"
@@ -3060,12 +2902,6 @@ dependencies = [
"unicode-properties",
]
[[package]]
name = "strsim"
version = "0.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f"
[[package]]
name = "subtle"
version = "2.6.1"
@@ -3120,16 +2956,6 @@ version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "55937e1799185b12863d447f42597ed69d9928686b8d88a1df17376a097d8369"
[[package]]
name = "terminal_size"
version = "0.4.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "230a1b821ccbd75b185820a1f1ff7b14d21da1e442e22c0863ea5f08771a8874"
dependencies = [
"rustix",
"windows-sys 0.61.2",
]
[[package]]
name = "thiserror"
version = "1.0.69"
@@ -3505,12 +3331,6 @@ version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be"
[[package]]
name = "utf8parse"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
[[package]]
name = "uuid"
version = "1.23.0"
@@ -4187,18 +4007,6 @@ version = "1.0.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa"
[[package]]
name = "zopfli"
version = "0.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f05cd8797d63865425ff89b5c4a48804f35ba0ce8d125800027ad6017d2b5249"
dependencies = [
"bumpalo",
"crc32fast",
"log",
"simd-adler32",
]
[[package]]
name = "zstd"
version = "0.13.3"

12
backend/Cargo.toml
View File

@@ -27,7 +27,17 @@ tracing-subscriber = { version = "0.3", features = ["env-filter"] }
dotenvy = "0.15"
sysinfo = "0.32"
image = "0.25"
oxipng = "9"
# default-features = false drops "parallel", which is what actually bounds oxipng's memory:
# with rayon it evaluates row filters concurrently, each trial holding its own full-size
# buffer, and there is no Options knob to cap that. Without the feature, lib.rs swaps in a
# sequential shim (oxipng's own supported path) so peak scales with ONE trial, not N.
# PNG optimisation gets slower; it is a background, best-effort, lossless size saving.
#
# "filetime" must be KEPT: without it OutFile::Path { preserve_attrs: true } silently no-ops.
# Dropping "binary" also removes clap/glob/env_logger — a CLI's dependencies that were being
# compiled into a server image — and "zopfli", which preset 2 does not use (it selects
# Deflaters::Libdeflater, which is not feature-gated).
oxipng = { version = "9", default-features = false, features = ["filetime"] }
async_zip = { version = "0.0.17", features = ["tokio", "deflate"] }
include_dir = "0.7"
infer = "0.15"

3
backend/migrations/015_raise_upload_rate.down.sql Normal file
View File

@@ -0,0 +1,3 @@
-- Revert the default upload rate to 10/hour for installs still on the raised
-- default (preserves any explicit admin override at another value).
UPDATE config SET value = '10' WHERE key = 'upload_rate_per_hour' AND value = '100';

10
backend/migrations/015_raise_upload_rate.up.sql Normal file
View File

@@ -0,0 +1,10 @@
-- Raise the default per-guest upload rate from 10/hour to 100/hour.
--
-- Rationale: guests routinely upload a burst of 10-20 photos at once (phone
-- multi-select). At the old default of 10/hour a real guest's first burst was
-- throttled — surfaced by the 2026-07-18 load test. 100/hour comfortably covers
-- several bursts across an event while still bounding abuse.
--
-- Only bump installs still on the old default; an admin who deliberately set a
-- different value keeps it (migration 005 seeded 10; this UPDATE is scoped to '10').
UPDATE config SET value = '100' WHERE key = 'upload_rate_per_hour' AND value = '10';

28
backend/migrations/016_display_derivative.down.sql Normal file
View File

@@ -0,0 +1,28 @@
-- Drop the view (frees the column dependency), remove the column, then restore the
-- pre-016 view definition (matches migration 011).
DROP VIEW IF EXISTS v_feed;
ALTER TABLE upload DROP COLUMN display_path;
CREATE VIEW v_feed AS
SELECT
u.id,
u.event_id,
u.user_id,
usr.display_name AS uploader_name,
usr.is_banned,
usr.uploads_hidden,
u.preview_path,
u.thumbnail_path,
u.mime_type,
u.caption,
u.created_at,
COUNT(DISTINCT l.user_id) AS like_count,
COUNT(DISTINCT c.id) AS comment_count
FROM upload u
JOIN "user" usr ON u.user_id = usr.id
LEFT JOIN "like" l ON l.upload_id = u.id
LEFT JOIN comment c ON c.upload_id = u.id AND c.deleted_at IS NULL
WHERE u.deleted_at IS NULL
AND usr.uploads_hidden = FALSE
AND usr.is_banned = FALSE
GROUP BY u.id, usr.display_name, usr.is_banned, usr.uploads_hidden;

34
backend/migrations/016_display_derivative.up.sql Normal file
View File

@@ -0,0 +1,34 @@
-- Display derivative: a big-screen-quality image (~2048px long edge) for the diashow.
-- The 800px `preview_path` is sized for phone feeds (data saver); upscaled on a projector
-- it looks soft. The diashow uses `display_path` instead — bounded in size (safe to decode
-- on weak kiosk hardware) yet sharp on 1080p/4K. NULL until the compression worker (or the
-- one-time backfill) generates it; consumers fall back to the original when absent.
ALTER TABLE upload ADD COLUMN display_path TEXT;
-- Recreate (not CREATE OR REPLACE, which only allows appending columns at the end) so the
-- new column can sit alongside preview_path/thumbnail_path.
DROP VIEW IF EXISTS v_feed;
CREATE VIEW v_feed AS
SELECT
u.id,
u.event_id,
u.user_id,
usr.display_name AS uploader_name,
usr.is_banned,
usr.uploads_hidden,
u.preview_path,
u.thumbnail_path,
u.display_path,
u.mime_type,
u.caption,
u.created_at,
COUNT(DISTINCT l.user_id) AS like_count,
COUNT(DISTINCT c.id) AS comment_count
FROM upload u
JOIN "user" usr ON u.user_id = usr.id
LEFT JOIN "like" l ON l.upload_id = u.id
LEFT JOIN comment c ON c.upload_id = u.id AND c.deleted_at IS NULL
WHERE u.deleted_at IS NULL
AND usr.uploads_hidden = FALSE
AND usr.is_banned = FALSE
GROUP BY u.id, usr.display_name, usr.is_banned, usr.uploads_hidden;

1
backend/migrations/017_join_ip_rate.down.sql Normal file
View File

@@ -0,0 +1 @@
DELETE FROM config WHERE key IN ('join_ip_rate_per_min', 'admin_login_rate_enabled');

18
backend/migrations/017_join_ip_rate.up.sql Normal file
View File

@@ -0,0 +1,18 @@
-- Per-IP flood ceiling for /join, and the `admin_login_rate_enabled` toggle that
-- every prior migration forgot to seed.
--
-- Rationale: /join was throttled at 5 requests per 60s keyed on the client IP. At a
-- venue every guest is behind one NAT, so the whole party shared a single bucket —
-- 12 guests scanning the QR code within a few seconds meant 5 got in and 7 were
-- turned away. The handler now keys the real anti-spam bucket per (ip, name), the
-- same shape as `recover:{ip}:{name}`, and keeps only a loose per-IP ceiling to bound
-- raw volume. 60/min comfortably covers a whole wedding arriving at once while still
-- capping a flood from a single source.
--
-- `admin_login_rate_enabled` is read by auth::handlers::admin_login with a code
-- default of `true`, but no migration ever inserted it, so it was invisible to the
-- admin config UI and to the e2e reseed. Seed it explicitly.
INSERT INTO config (key, value) VALUES
('join_ip_rate_per_min', '60'),
('admin_login_rate_enabled', 'true')
ON CONFLICT (key) DO NOTHING;

2
backend/migrations/018_derivatives_rev.down.sql Normal file
View File

@@ -0,0 +1,2 @@
DROP INDEX IF EXISTS idx_upload_derivatives_rev;
ALTER TABLE upload DROP COLUMN IF EXISTS derivatives_rev;

18
backend/migrations/018_derivatives_rev.up.sql Normal file
View File

@@ -0,0 +1,18 @@
-- Track which revision of the derivative pipeline produced an upload's preview/display.
--
-- Rev 1 applies the EXIF orientation tag. Everything generated before it decoded the raw
-- sensor pixels and re-encoded to JPEG (which writes no EXIF), so every portrait phone photo
-- was stored sideways in the feed preview, the diashow display and the keepsake — while the
-- untouched original still rendered upright.
--
-- Existing rows default to 0 so the startup backfill can find and re-generate them exactly
-- once; bump the constant in services/compression.rs if the pipeline ever changes again.
ALTER TABLE upload ADD COLUMN IF NOT EXISTS derivatives_rev SMALLINT NOT NULL DEFAULT 0;
-- Only image derivatives are affected — video thumbnails are extracted by ffmpeg, which
-- already honours the rotation matrix. Mark them current so the backfill skips them.
UPDATE upload SET derivatives_rev = 1 WHERE mime_type NOT LIKE 'image/%';
CREATE INDEX IF NOT EXISTS idx_upload_derivatives_rev
ON upload (derivatives_rev)
WHERE deleted_at IS NULL;

1
backend/migrations/019_recover_ip_rate.down.sql Normal file
View File

@@ -0,0 +1 @@
DELETE FROM config WHERE key = 'recover_ip_rate_per_min';

19
backend/migrations/019_recover_ip_rate.up.sql Normal file
View File

@@ -0,0 +1,19 @@
-- Per-IP flood ceiling for /recover, mirroring the one migration 017 added for /join.
--
-- Rationale: /recover is keyed `recover:{ip}:{name}` at 5 per 15 minutes. That is the
-- right shape for its actual job — stopping someone who knows a display name (they are
-- visible on the feed) from burning the victim's 3-strike PIN counter and locking them
-- out repeatedly. But the name is attacker-chosen, so cycling names mints a fresh bucket
-- every time and the per-IP cost is unbounded.
--
-- Behind that limiter sits a cost-12 bcrypt verify, including an UNCONDITIONAL throwaway
-- verify for names that don't exist — deliberately, to close a timing oracle. So an
-- unknown name is the cheapest possible way to make the server do ~200ms of hashing.
-- Without a ceiling, one client can saturate the box's CPU with a name generator.
--
-- 30/min is far above any real recovery attempt (a guest tries their PIN a handful of
-- times) while capping a name-cycling flood. The per-(ip, name) bucket is unchanged and
-- remains the anti-guessing control.
INSERT INTO config (key, value) VALUES
('recover_ip_rate_per_min', '30')
ON CONFLICT (key) DO NOTHING;

1
backend/migrations/020_social_rate.down.sql Normal file
View File

@@ -0,0 +1 @@
DELETE FROM config WHERE key IN ('social_rate_per_min', 'social_rate_enabled');

16
backend/migrations/020_social_rate.up.sql Normal file
View File

@@ -0,0 +1,16 @@
-- Per-user rate limit for social writes (likes, comments, comment deletions).
--
-- These were the only writes in the app with no limit at all. Every other mutating
-- path -- upload, join, recover, export, admin login -- carries one; social.rs
-- carried none, so the coverage was asymmetric rather than deliberately open.
--
-- Severity is genuinely low for an invited-guest event, and the amplification worry
-- turned out to be contained: a like fans an SSE broadcast to ~100 clients, but the
-- export regeneration it could otherwise trigger is debounced (REGEN_DEBOUNCE 20s)
-- and superseded workers are inert. So this closes the gap for symmetry, not urgency,
-- and the ceiling is set high enough that no real guest will ever meet it -- a
-- double-tapping enthusiast at a wedding is not the thing being defended against.
INSERT INTO config (key, value) VALUES
('social_rate_per_min', '120'),
('social_rate_enabled', 'true')
ON CONFLICT (key) DO NOTHING;

3
backend/migrations/021_derivative_attempts.down.sql Normal file
View File

@@ -0,0 +1,3 @@
DROP INDEX IF EXISTS idx_upload_derivative_backfill;
ALTER TABLE upload DROP COLUMN IF EXISTS derivative_last_error;
ALTER TABLE upload DROP COLUMN IF EXISTS derivative_attempts;

23
backend/migrations/021_derivative_attempts.up.sql Normal file
View File

@@ -0,0 +1,23 @@
-- Bound how many times a permanently-failing upload can be re-processed.
--
-- Without this, one poisoned row is an outage. The upload row is committed BEFORE compression
-- starts, `derivatives_rev` defaults to 0, and `set_derivatives_rev` only runs on success — so
-- a row whose processing kills the container survives at rev 0, the unconditional startup
-- backfill re-selects it on the next boot, and `restart: unless-stopped` turns that into an
-- infinite kill loop. Every restart also drops every SSE stream and truncates every in-flight
-- upload. That was reachable via a single large PNG (see services/compression.rs), but the
-- shape is general: any input that can kill or hang the worker repeats forever.
--
-- The counter is incremented WRITE-AHEAD, before the work is attempted, because the failure
-- mode being defended against is a SIGKILL — no error is returned, no handler runs, no Drop
-- fires. A counter bumped in an error path increments zero times per crash and changes nothing.
ALTER TABLE upload ADD COLUMN IF NOT EXISTS derivative_attempts SMALLINT NOT NULL DEFAULT 0;
-- Last failure text, so a row that has given up can be diagnosed without reproducing it.
-- Nothing reads this in code; it exists for the operator.
ALTER TABLE upload ADD COLUMN IF NOT EXISTS derivative_last_error TEXT;
-- Serves the backfill selection, which now filters on both columns.
CREATE INDEX IF NOT EXISTS idx_upload_derivative_backfill
ON upload (derivatives_rev, derivative_attempts)
WHERE deleted_at IS NULL;

26
backend/migrations/022_feed_scalar_counts.down.sql Normal file
View File

@@ -0,0 +1,26 @@
-- Restore the 016 definition verbatim.
DROP VIEW IF EXISTS v_feed;
CREATE VIEW v_feed AS
SELECT
u.id,
u.event_id,
u.user_id,
usr.display_name AS uploader_name,
usr.is_banned,
usr.uploads_hidden,
u.preview_path,
u.thumbnail_path,
u.display_path,
u.mime_type,
u.caption,
u.created_at,
COUNT(DISTINCT l.user_id) AS like_count,
COUNT(DISTINCT c.id) AS comment_count
FROM upload u
JOIN "user" usr ON u.user_id = usr.id
LEFT JOIN "like" l ON l.upload_id = u.id
LEFT JOIN comment c ON c.upload_id = u.id AND c.deleted_at IS NULL
WHERE u.deleted_at IS NULL
AND usr.uploads_hidden = FALSE
AND usr.is_banned = FALSE
GROUP BY u.id, usr.display_name, usr.is_banned, usr.uploads_hidden;

55
backend/migrations/022_feed_scalar_counts.up.sql Normal file
View File

@@ -0,0 +1,55 @@
-- Make a feed page cost a page, not the whole event.
--
-- The previous definition (016) computed like_count/comment_count with LEFT JOINs and a
-- GROUP BY. Postgres CAN push the `event_id = $1` qual and the keyset predicate through the
-- view — verified with EXPLAIN, it uses idx_upload_event_created_id — but it CANNOT push
-- ORDER BY ... LIMIT across a GroupAggregate. So every feed request aggregated every upload in
-- the event (times its likes and comments) and only then sorted and took 21 rows. The cost
-- grew with the event, not with the page, and page 1 — the most expensive one — is exactly
-- what refreshFeedInPlace refetches on every completed upload, from every open feed in the
-- venue.
--
-- Correlated scalar subqueries move the counts ABOVE the Limit in the plan: they are evaluated
-- once per returned row, so 21 index lookups instead of a full aggregation.
--
-- The rewrite is EXACTLY equivalent, not merely close:
-- * "like" is keyed (upload_id, user_id), so COUNT(DISTINCT l.user_id) == count(*).
-- * comment.id is the primary key, so COUNT(DISTINCT c.id) == count(*).
-- * one row per upload either way — the GROUP BY was on u.id.
-- Column names, order and types are unchanged (count(*) and COUNT(DISTINCT ...) are both
-- bigint), so no Rust code changes.
--
-- No new index needed: idx_like_upload plus the (upload_id, user_id) PK serve the like
-- subquery, and idx_comment_upload ... WHERE deleted_at IS NULL matches the comment
-- subquery's predicate exactly.
--
-- One thing a future editor needs to know: the hashtag-filtered feed joins upload_hashtag
-- against this view. That was safe before only because the GROUP BY collapsed the join
-- fan-out; it is safe now because the view is one row per upload and upload_hashtag is keyed
-- (upload_id, hashtag_id) with a single tag filtered. Adding a second tag filter would need
-- fresh thought.
-- Not CASCADE: if something ever comes to depend on this view, the migration should fail
-- loudly rather than silently drop it.
DROP VIEW IF EXISTS v_feed;
CREATE VIEW v_feed AS
SELECT
u.id,
u.event_id,
u.user_id,
usr.display_name AS uploader_name,
usr.is_banned,
usr.uploads_hidden,
u.preview_path,
u.thumbnail_path,
u.display_path,
u.mime_type,
u.caption,
u.created_at,
(SELECT count(*) FROM "like" l WHERE l.upload_id = u.id) AS like_count,
(SELECT count(*) FROM comment c WHERE c.upload_id = u.id AND c.deleted_at IS NULL) AS comment_count
FROM upload u
JOIN "user" usr ON u.user_id = usr.id
WHERE u.deleted_at IS NULL
AND usr.uploads_hidden = FALSE
AND usr.is_banned = FALSE;

11
backend/migrations/023_reserved_names_and_pin_decay.down.sql Normal file
View File

@@ -0,0 +1,11 @@
-- NOTE: the reserved-name rename in the up migration is NOT reversible. The original names
-- are not recorded anywhere, and reversing it would in any case re-create the state that
-- bricked admin login. Rolling back the schema does not roll back that data change.
DELETE FROM config WHERE key IN (
'recover_name_rate_per_15min',
'pin_reset_ip_rate_per_min',
'upload_edit_rate_per_min',
'upload_edit_rate_enabled'
);
ALTER TABLE "user" DROP COLUMN IF EXISTS last_failed_pin_at;

48
backend/migrations/023_reserved_names_and_pin_decay.up.sql Normal file
View File

@@ -0,0 +1,48 @@
-- Two independent auth defects that share a migration because they share a table.
-- 1. RESERVED NAMES — free any guest squatting on a name the admin path used to depend on.
--
-- Migration 007 made display_name unique per event case-insensitively, and `join` had no
-- reserved-name guard. So any guest could join as "admin"/"Admin"/"ADMIN" before the operator's
-- first admin login; admin_login then looked its user up BY NAME, missed (wrong role), fell
-- through to creating "Admin", violated that unique index, and returned a 500 — permanently,
-- with no in-app recovery. Moderation, config and gallery release all gone, fixed only by SQL.
--
-- The real fix is in code (look the admin up by role, never by name — see auth/handlers.rs).
-- This clears the state an already-deployed database may be carrying.
--
-- RENAMED, NEVER DELETED: the guest keeps their uploads, their PIN and their session. Only
-- non-admin rows are touched — a real admin row named "Admin" is the expected state.
UPDATE "user" u
SET display_name = u.display_name || ' (' || left(u.id::text, 8) || ')'
WHERE u.role <> 'admin'
AND lower(u.display_name) IN ('admin', 'administrator', 'host', 'eventsnap');
-- 2. PIN LOCKOUT DECAY.
--
-- failed_pin_attempts only ever cleared on a successful recovery or after a lockout expired, so
-- honest typos accumulated across days: a guest who fat-fingered their PIN twice last night
-- arrives today already two-thirds of the way to being locked out. With the threshold now
-- raised (see below) a decay window is what keeps that raise safe rather than merely lenient.
ALTER TABLE "user" ADD COLUMN IF NOT EXISTS last_failed_pin_at TIMESTAMPTZ;
-- Rate-limit knobs introduced with this release.
--
-- recover_name_rate_per_15min (4, was a hardcoded 5): the per-(IP, name) ceiling. It MUST stay
-- below the account-lock threshold, which is the whole defect — at 5-per-IP against a 3-strike
-- lock, three requests from one IP locked any guest whose name is visible on the feed, every 15
-- minutes, forever. The lock threshold moves to 12 in code, so locking a victim now needs at
-- least three distinct sources while an honest guest never comes close.
--
-- pin_reset_ip_rate_per_min (30): /recover/request was the one unauthenticated endpoint with no
-- per-IP ceiling at all — /join got one in 017 and /recover in 019, and this third one was
-- simply missed. Its per-name key is attacker-chosen, so cycling names minted a fresh bucket
-- every time and the per-IP cost was unbounded.
--
-- upload_edit_rate_per_min (30): PATCH /upload/{id} had no rate limit of any kind.
INSERT INTO config (key, value) VALUES
('recover_name_rate_per_15min', '4'),
('pin_reset_ip_rate_per_min', '30'),
('upload_edit_rate_per_min', '30'),
('upload_edit_rate_enabled', 'true')
ON CONFLICT (key) DO NOTHING;

413
backend/src/auth/handlers.rs
View File

@@ -1,11 +1,12 @@
use std::time::Duration;
use axum::Json;
use axum::extract::State;
use axum::extract::{ConnectInfo, State};
use axum::http::{HeaderMap, StatusCode};
use chrono::Utc;
use rand::Rng;
use serde::{Deserialize, Serialize};
use std::net::SocketAddr;
use uuid::Uuid;
use crate::auth::jwt;
@@ -18,6 +19,45 @@ use crate::services::config;
use crate::services::rate_limiter::client_ip;
use crate::state::AppState;
/// Names a guest may not take.
///
/// Defence in depth only. The real fix for the admin-lockout defect is that `admin_login` now
/// resolves its user by ROLE rather than by name (see `User::find_admin_for_event`), which is
/// why homoglyph and zero-width bypasses of this list are not a concern: the name is no longer
/// load-bearing for anything. What this buys is that a guest cannot impersonate the host in the
/// feed's byline, and that "Admin" stays available for the admin row.
const RESERVED_DISPLAY_NAMES: &[&str] = &["admin", "administrator", "host", "eventsnap"];
fn is_reserved_display_name(name: &str) -> bool {
let name = name.trim().to_lowercase();
RESERVED_DISPLAY_NAMES.contains(&name.as_str())
}
/// Trim and bounds-check a display name.
///
/// Shared by `join`, `recover` and `request_pin_reset` so the length check happens BEFORE the
/// name is used to build a rate-limiter key. It was inline in `join` only, so on the other two
/// endpoints `format!("...:{ip}:{name_key}")` allocated from an unbounded, attacker-chosen
/// string and stored it in a HashMap pruned once an hour with a 24 h ceiling — turning the
/// limiter itself into the memory-exhaustion primitive it exists to prevent.
fn validate_display_name(raw: &str) -> Result<&str, AppError> {
let name = raw.trim();
let chars = name.chars().count();
if chars == 0 || chars > 50 {
return Err(AppError::BadRequest(
"Name muss zwischen 1 und 50 Zeichen lang sein.".into(),
));
}
// Postgres rejects 0x00 in TEXT columns with a 500. Catch it here so callers see a clean
// 400 instead of an internal error.
if name.contains('\0') {
return Err(AppError::BadRequest(
"Name enthält ungültige Zeichen.".into(),
));
}
Ok(name)
}
#[derive(Deserialize)]
pub struct JoinRequest {
pub display_name: String,
@@ -33,37 +73,58 @@ pub struct JoinResponse {
pub async fn join(
State(state): State<AppState>,
ConnectInfo(peer): ConnectInfo<SocketAddr>,
headers: HeaderMap,
Json(body): Json<JoinRequest>,
) -> Result<(StatusCode, Json<JoinResponse>), AppError> {
let ip = client_ip(&headers, "unknown");
let ip = client_ip(&headers, &peer.ip().to_string());
let rate_limits_on = config::get_bool(&state.config_cache, "rate_limits_enabled", true).await;
let join_rate_on = config::get_bool(&state.config_cache, "join_rate_enabled", true).await;
if rate_limits_on
&& join_rate_on
&& !state
.rate_limiter
.check(format!("join:{ip}"), 5, Duration::from_secs(60))
{
return Err(AppError::TooManyRequests(
"Zu viele Anfragen. Bitte warte kurz und versuche es erneut.".into(),
None,
));
// Coarse per-IP flood ceiling. `/join` is pre-auth so there is no user to key on, and
// at a venue EVERY guest arrives from one public IP — a tight per-IP bucket meant the
// 6th person through the door was turned away by the 5 ahead of them. So the per-IP
// limit here only bounds raw volume; the real anti-spam bucket is per-name below.
// Cheap enough to run before validation, which keeps a flood of malformed bodies from
// being free.
if rate_limits_on && join_rate_on {
let ip_ceiling = config::get_usize(&state.config_cache, "join_ip_rate_per_min", 60).await;
if let Err(retry_after_secs) = state.rate_limiter.check_with_retry(
format!("join_ip:{ip}"),
ip_ceiling,
Duration::from_secs(60),
) {
return Err(AppError::TooManyRequests(
"Zu viele Anfragen. Bitte warte kurz und versuche es erneut.".into(),
Some(retry_after_secs),
));
}
}
let display_name = body.display_name.trim();
let name_chars = display_name.chars().count();
if name_chars == 0 || name_chars > 50 {
return Err(AppError::BadRequest(
"Name muss zwischen 1 und 50 Zeichen lang sein.".into(),
));
let display_name = validate_display_name(&body.display_name)?;
if is_reserved_display_name(display_name) {
// 409, matching the name-taken response below, so the frontend's existing handling
// works unchanged. See RESERVED_DISPLAY_NAMES for why this exists.
return Err(AppError::Conflict(format!(
"Der Name \"{display_name}\" ist reserviert. Bitte wähle einen anderen."
)));
}
// Postgres rejects 0x00 in TEXT columns with a 500. Catch it here so callers
// see a clean 400 instead of an internal error.
if display_name.contains('\0') {
return Err(AppError::BadRequest(
"Name enthält ungültige Zeichen.".into(),
));
// Per-guest bucket, keyed like the `recover:{ip}:{name}` limiter below. This carries
// the original 5/60s anti-spam intent, but one guest retrying can no longer consume
// the allowance of everyone else sharing the venue's NAT.
if rate_limits_on && join_rate_on {
let name_key = display_name.to_lowercase();
if let Err(retry_after_secs) = state.rate_limiter.check_with_retry(
format!("join:{ip}:{name_key}"),
5,
Duration::from_secs(60),
) {
return Err(AppError::TooManyRequests(
"Zu viele Anfragen. Bitte warte kurz und versuche es erneut.".into(),
Some(retry_after_secs),
));
}
}
let event = Event::find_or_create(
@@ -83,7 +144,7 @@ pub async fn join(
// Generate a 4-digit PIN
let pin: String = format!("{:04}", rand::rng().random_range(0..10000u32));
let pin_hash = bcrypt::hash(&pin, 12).map_err(|e| AppError::Internal(anyhow::anyhow!(e)))?;
let pin_hash = hash_password(pin.clone(), 12).await?;
// The pre-check above is racy: two simultaneous joins with the same name can both
// pass it, and the DB's unique index then rejects the loser. Map that unique
@@ -123,6 +184,28 @@ pub async fn join(
))
}
/// Default for `recover_name_rate_per_15min` — wrong PINs allowed per (IP, name) per 15 min.
/// Mirrors migration 023; kept here so the invariant below can be asserted in a test.
const RECOVER_NAME_CEILING_DEFAULT: usize = 4;
/// Wrong PINs, from ALL sources, before the ACCOUNT itself is locked for 15 minutes.
///
/// This was 3, which sat BELOW the per-(IP, name) ceiling of 5 — and that ordering, not the
/// number, was the defect. Display names are public on the feed, so three requests from a single
/// IP locked any guest out of their own account, repeatable every 15 minutes, indefinitely. The
/// tier meant to protect a guest was the easiest way to attack them.
///
/// Raised deliberately far above the per-IP tier so the two do different jobs. The per-(IP, name)
/// bucket is what stops a guesser, and it costs the ATTACKER. This tier is the last line against
/// a DISTRIBUTED guesser, and it is the only one an attacker can turn on a victim — so reaching
/// it must require at least three distinct sources inside the decay window.
///
/// Brute-force cost is unchanged: 12 attempts per 15 minutes is 48/hour against one account, so
/// 10 000 four-digit PINs still take ~208 hours no matter how many IPs are used. An honest guest
/// fat-fingering a 4-digit PIN never comes close, and `increment_failed_pin` now decays the
/// streak after 15 minutes so yesterday's typos don't count toward today's.
const PIN_LOCK_THRESHOLD: i16 = 12;
#[derive(Deserialize)]
pub struct RecoverRequest {
pub display_name: String,
@@ -147,31 +230,82 @@ fn dummy_pin_hash() -> &'static str {
})
}
/// Run a bcrypt verify on the blocking pool.
///
/// bcrypt at cost 12 is ~200ms of deliberate CPU. Called inline on an async task it pins a
/// tokio WORKER thread for that whole time, and the runtime only has one per core — so a
/// flood of `/recover` or `/admin/login` attempts stalls every other request on the box,
/// including the feed. Offloading moves that cost to the blocking pool, which is sized for
/// exactly this and whose saturation degrades logins rather than the whole app.
async fn verify_password(candidate: String, hash: String) -> bool {
tokio::task::spawn_blocking(move || bcrypt::verify(&candidate, &hash).unwrap_or(false))
.await
.unwrap_or(false)
}
/// Hash a secret on the blocking pool. Same reasoning as [`verify_password`] — and this one
/// runs on the busiest auth path there is, since every guest who joins gets a PIN hashed.
pub async fn hash_password(secret: String, cost: u32) -> Result<String, AppError> {
tokio::task::spawn_blocking(move || bcrypt::hash(&secret, cost))
.await
.map_err(|e| AppError::Internal(anyhow::anyhow!(e)))?
.map_err(|e| AppError::Internal(anyhow::anyhow!(e)))
}
pub async fn recover(
State(state): State<AppState>,
ConnectInfo(peer): ConnectInfo<SocketAddr>,
headers: HeaderMap,
Json(body): Json<RecoverRequest>,
) -> Result<Json<RecoverResponse>, AppError> {
let display_name = body.display_name.trim();
// Validated BEFORE it is used as a rate-limiter key — see `validate_display_name`. The
// per-IP ceiling below is keyed only on the IP, so it is safe to run either side of this;
// the per-NAME bucket is not.
let display_name = validate_display_name(&body.display_name)?;
// Per-IP+name throttle BEFORE the per-user 3-strike counter. Without this
// an attacker who knows a display name (they're visible on the feed) can
// burn through 3 wrong PINs and lock the victim for 15 minutes — repeated
// every 15 minutes, indefinitely. 5 attempts per 15 minutes per (IP, name)
// softens that into a real cost.
let ip = client_ip(&headers, "unknown");
// Per-IP+name throttle BEFORE the per-user lockout counter. Without this an attacker who
// knows a display name (they're visible on the feed) can burn through the victim's wrong-PIN
// budget and lock them out, repeatedly. The ceiling here MUST stay below
// PIN_LOCK_THRESHOLD — see the constant for why that ordering is the whole control.
let ip = client_ip(&headers, &peer.ip().to_string());
let rate_limits_on = config::get_bool(&state.config_cache, "rate_limits_enabled", true).await;
let recover_rate_on = config::get_bool(&state.config_cache, "recover_rate_enabled", true).await;
if rate_limits_on && recover_rate_on {
// Coarse per-IP ceiling FIRST. The per-(ip, name) bucket below is the anti-guessing
// control, but the name is attacker-chosen, so cycling names mints a fresh bucket
// every time and leaves the per-IP cost unbounded. That matters more here than
// anywhere else: every call runs a cost-12 bcrypt verify, including an
// unconditional throwaway one for names that don't exist (see below), so an unknown
// name is the CHEAPEST way to make the server do ~200ms of hashing. Checked before
// the per-name bucket so a name generator can't walk past it.
let ip_ceiling =
config::get_usize(&state.config_cache, "recover_ip_rate_per_min", 30).await;
if let Err(retry_after_secs) = state.rate_limiter.check_with_retry(
format!("recover_ip:{ip}"),
ip_ceiling,
Duration::from_secs(60),
) {
return Err(AppError::TooManyRequests(
"Zu viele Versuche. Bitte warte kurz und versuche es erneut.".into(),
Some(retry_after_secs),
));
}
let name_ceiling = config::get_usize(
&state.config_cache,
"recover_name_rate_per_15min",
RECOVER_NAME_CEILING_DEFAULT,
)
.await;
let name_key = display_name.to_lowercase();
if !state.rate_limiter.check(
if let Err(retry_after_secs) = state.rate_limiter.check_with_retry(
format!("recover:{ip}:{name_key}"),
5,
name_ceiling,
Duration::from_secs(15 * 60),
) {
return Err(AppError::TooManyRequests(
"Zu viele Versuche. Bitte warte kurz und versuche es erneut.".into(),
None,
Some(retry_after_secs),
));
}
}
@@ -188,7 +322,7 @@ pub async fn recover(
// PIN — so "no such name" and "wrong PIN" are indistinguishable by response or
// timing. Display names are already public on the feed, but this still closes
// the /recover enumeration + timing oracle.
let _ = bcrypt::verify(&body.pin, dummy_pin_hash());
let _ = verify_password(body.pin.clone(), dummy_pin_hash().to_string()).await;
return Err(AppError::Unauthorized("PIN ist falsch.".into()));
}
@@ -200,16 +334,19 @@ pub async fn recover(
// is effectively permanently fragile.
if let Some(locked_until) = user.pin_locked_until {
if Utc::now() < locked_until {
// The exact deadline is known, so surface it as Retry-After instead of
// making the client guess at the "15 Minuten" in the copy.
let retry_after_secs = (locked_until - Utc::now()).num_seconds().max(1) as u64;
return Err(AppError::TooManyRequests(
"Zu viele Versuche. Bitte warte 15 Minuten.".into(),
None,
Some(retry_after_secs),
));
}
// Lockout window expired — wipe the counter and the timestamp.
User::reset_pin_attempts(&state.pool, user.id).await?;
}
let pin_matches = bcrypt::verify(&body.pin, &user.recovery_pin_hash).unwrap_or(false);
let pin_matches = verify_password(body.pin.clone(), user.recovery_pin_hash.clone()).await;
if pin_matches {
// Reset failed attempts on success
@@ -243,7 +380,7 @@ pub async fn recover(
attempts,
"recover: wrong PIN"
);
if attempts >= 3 {
if attempts >= PIN_LOCK_THRESHOLD {
let lockout = Utc::now() + chrono::Duration::minutes(15);
User::lock_pin(&state.pool, user.id, lockout).await?;
tracing::warn!(
@@ -274,6 +411,7 @@ pub struct AdminLoginResponse {
pub async fn admin_login(
State(state): State<AppState>,
ConnectInfo(peer): ConnectInfo<SocketAddr>,
headers: HeaderMap,
Json(body): Json<AdminLoginRequest>,
) -> Result<Json<AdminLoginResponse>, AppError> {
@@ -287,23 +425,31 @@ pub async fn admin_login(
// verify) but with no IP-level limit a determined attacker can still mount
// a long-running guess campaign. 5 attempts / minute / IP is plenty for
// honest typos.
let ip = client_ip(&headers, "unknown");
let ip = client_ip(&headers, &peer.ip().to_string());
let rate_limits_on = config::get_bool(&state.config_cache, "rate_limits_enabled", true).await;
let admin_rate_on =
config::get_bool(&state.config_cache, "admin_login_rate_enabled", true).await;
// Stays keyed by IP on purpose: this guards a single shared credential, so a per-user
// or per-name key would just hand an attacker a fresh bucket per guess.
if rate_limits_on
&& admin_rate_on
&& !state
.rate_limiter
.check(format!("admin_login:{ip}"), 5, Duration::from_secs(60))
&& let Err(retry_after_secs) = state.rate_limiter.check_with_retry(
format!("admin_login:{ip}"),
5,
Duration::from_secs(60),
)
{
return Err(AppError::TooManyRequests(
"Zu viele Anmeldeversuche. Bitte warte kurz und versuche es erneut.".into(),
None,
Some(retry_after_secs),
));
}
let valid = bcrypt::verify(&body.password, &state.config.admin_password_hash).unwrap_or(false);
let valid = verify_password(
body.password.clone(),
state.config.admin_password_hash.clone(),
)
.await;
if !valid {
tracing::warn!(ip = %ip, "admin_login: wrong password");
@@ -317,28 +463,16 @@ pub async fn admin_login(
)
.await?;
// Find or create the admin user for this event
let admin_name = "Admin";
let users = User::find_by_event_and_name(&state.pool, event.id, admin_name).await?;
let admin_user = if let Some(u) = users.into_iter().find(|u| u.role == UserRole::Admin) {
u
} else {
// Admin authenticates via password, but the schema still requires a PIN
// hash. Generate a random unguessable PIN so the recovery path remains
// unusable as an escalation route even if the role flag ever got cleared.
let dummy_pin: String = (0..32)
.map(|_| rand::rng().random_range(b'a'..=b'z') as char)
.collect();
let dummy_hash =
bcrypt::hash(&dummy_pin, 4).map_err(|e| AppError::Internal(anyhow::anyhow!(e)))?;
let user = User::create(&state.pool, event.id, admin_name, &dummy_hash).await?;
sqlx::query("UPDATE \"user\" SET role = 'admin' WHERE id = $1")
.bind(user.id)
.execute(&state.pool)
.await?;
User::find_by_id(&state.pool, user.id)
.await?
.ok_or_else(|| AppError::Internal(anyhow::anyhow!("admin user creation failed")))?
// Find or create the admin user for this event — BY ROLE, never by name.
//
// The name lookup this replaces is what made admin login brickable. Migration 007 makes
// display_name unique per event case-insensitively and `join` had no reserved-name guard,
// so a guest joining as "admin" before the operator's first login made the lookup miss on
// role, the fallback `create("Admin")` violate that index, and `?` return a permanent 500 —
// taking out moderation, config and gallery release with no in-app recovery.
let admin_user = match User::find_admin_for_event(&state.pool, event.id).await? {
Some(u) => u,
None => create_admin_user(&state, event.id).await?,
};
tracing::info!(user_id = %admin_user.id, event_id = %event.id, ip = %ip, "admin_login: success");
@@ -363,6 +497,51 @@ pub async fn admin_login(
}))
}
/// Create this event's admin row on first successful admin login.
///
/// Prefers the name "Admin". If a legacy database has a guest squatting on it — the state
/// migration 023 renames away, but a row could also predate that or be created between
/// migrations — falls back to a suffixed name rather than failing the login.
///
/// PROMOTING THE SQUATTING ROW WOULD BE A SERIOUS MISTAKE, and is the obvious-looking fix, so
/// it is spelled out: that row carries a `recovery_pin_hash` the guest knows. Setting
/// `role = 'admin'` on it would hand them the admin dashboard through `/recover`, permanently,
/// via a path that needs no password. A separate row under an uglier name is worse UX and much
/// better security — and since the lookup is now by role, the fallback name never has to be
/// guessed again on a later login.
async fn create_admin_user(state: &AppState, event_id: Uuid) -> Result<User, AppError> {
// Admin authenticates via password, but the schema still requires a PIN hash. Generate a
// random unguessable one so the recovery path stays unusable as an escalation route even if
// the role flag were ever cleared.
let dummy_pin: String = (0..32)
.map(|_| rand::rng().random_range(b'a'..=b'z') as char)
.collect();
let dummy_hash = hash_password(dummy_pin, 4).await?;
match User::create_with_role(&state.pool, event_id, "Admin", &dummy_hash, UserRole::Admin).await
{
Ok(u) => Ok(u),
Err(sqlx::Error::Database(db)) if db.is_unique_violation() => {
let fallback = format!("Admin-{}", &Uuid::new_v4().to_string()[..8]);
tracing::warn!(
%event_id, %fallback,
"the name \"Admin\" is held by a non-admin user; creating the admin under a \
fallback name. Rename that guest to free it — do NOT promote their row, they \
know its recovery PIN."
);
Ok(User::create_with_role(
&state.pool,
event_id,
&fallback,
&dummy_hash,
UserRole::Admin,
)
.await?)
}
Err(e) => Err(e.into()),
}
}
pub async fn logout(State(state): State<AppState>, auth: AuthUser) -> Result<StatusCode, AppError> {
Session::delete_by_token_hash(&state.pool, &auth.token_hash).await?;
Ok(StatusCode::NO_CONTENT)
@@ -390,28 +569,55 @@ pub struct PinResetRequestBody {
/// feed already exposes.
pub async fn request_pin_reset(
State(state): State<AppState>,
ConnectInfo(peer): ConnectInfo<SocketAddr>,
headers: HeaderMap,
Json(body): Json<PinResetRequestBody>,
) -> Result<StatusCode, AppError> {
let display_name = body.display_name.trim();
let ip = client_ip(&headers, "unknown");
let ip = client_ip(&headers, &peer.ip().to_string());
let rate_limits_on = config::get_bool(&state.config_cache, "rate_limits_enabled", true).await;
// Coarse per-IP ceiling FIRST, keyed only on the IP so its key is bounded by construction.
// /join got one of these in migration 017 and /recover in 019; this third unauthenticated
// endpoint was simply missed — migration 019's own comment describes exactly this attack.
// Without it, the per-name bucket below is no ceiling at all: the name is attacker-chosen,
// so cycling names mints a fresh bucket every request.
if rate_limits_on {
let ip_ceiling =
config::get_usize(&state.config_cache, "pin_reset_ip_rate_per_min", 30).await;
if let Err(retry_after_secs) = state.rate_limiter.check_with_retry(
format!("pin_reset_ip:{ip}"),
ip_ceiling,
Duration::from_secs(60),
) {
return Err(AppError::TooManyRequests(
"Zu viele Anfragen. Bitte warte kurz und versuche es erneut.".into(),
Some(retry_after_secs),
));
}
}
// Validated BEFORE the per-name key is built, so an unbounded name can never be retained in
// the limiter map. NOTE the 204: this endpoint's contract is that it answers identically
// whether or not the name exists, so it cannot enumerate guests. A 400 here would be a new
// signal — it would distinguish a malformed name from a well-formed unknown one. Silence is
// the correct response, and matches what an empty name already did.
let Ok(display_name) = validate_display_name(&body.display_name) else {
return Ok(StatusCode::NO_CONTENT);
};
if rate_limits_on {
let name_key = display_name.to_lowercase();
if !state.rate_limiter.check(
if let Err(retry_after_secs) = state.rate_limiter.check_with_retry(
format!("pin_reset_req:{ip}:{name_key}"),
3,
Duration::from_secs(15 * 60),
) {
return Err(AppError::TooManyRequests(
"Zu viele Anfragen. Bitte warte kurz und versuche es erneut.".into(),
None,
Some(retry_after_secs),
));
}
}
if display_name.is_empty() {
return Ok(StatusCode::NO_CONTENT);
}
// Single statement so the existing-name and unknown-name paths do IDENTICAL work
// (same event+user index scans, an INSERT that matches 0 rows for an unknown name) —
@@ -441,3 +647,62 @@ pub async fn request_pin_reset(
Ok(StatusCode::NO_CONTENT)
}
#[cfg(test)]
mod tests {
use super::*;
/// THE defect, stated as arithmetic: the account-lock threshold sat BELOW the per-(IP, name)
/// attempt ceiling, so a single IP could exhaust it and lock any guest whose display name is
/// visible on the feed — every 15 minutes, indefinitely. The tier meant to protect a guest
/// was the cheapest way to attack them.
///
/// The fix is the ORDERING, not either number on its own, so that is what this pins.
#[test]
fn one_ip_cannot_reach_the_account_lock() {
assert!(
PIN_LOCK_THRESHOLD as usize >= RECOVER_NAME_CEILING_DEFAULT * 3,
"locking a victim must require at least three distinct sources; \
threshold {PIN_LOCK_THRESHOLD} vs per-IP ceiling {RECOVER_NAME_CEILING_DEFAULT}"
);
}
/// Raising the threshold must not quietly weaken brute-force resistance. 4-digit PINs, and
/// the lockout window is 15 minutes, so an attacker gets PIN_LOCK_THRESHOLD tries per window.
#[test]
fn the_raised_threshold_still_makes_guessing_a_four_digit_pin_impractical() {
let attempts_per_hour = PIN_LOCK_THRESHOLD as u64 * 4; // four 15-minute windows
let hours_for_full_keyspace = 10_000 / attempts_per_hour;
assert!(
hours_for_full_keyspace >= 168,
"exhausting 10k PINs would take {hours_for_full_keyspace}h — under a week is too fast"
);
}
#[test]
fn reserved_names_are_matched_case_insensitively_and_trimmed() {
for name in ["admin", "Admin", "ADMIN", " Host ", "EventSnap"] {
assert!(is_reserved_display_name(name), "{name} must be reserved");
}
}
/// A substring match here would reject perfectly ordinary names, which is a worse outcome
/// than the impersonation the list guards against.
#[test]
fn names_that_merely_contain_a_reserved_word_are_allowed() {
for name in ["Administrata", "Hostess", "Adminah", "Ghost", "hosting"] {
assert!(!is_reserved_display_name(name), "{name} must be allowed");
}
}
#[test]
fn display_names_are_bounded_before_they_can_become_a_rate_limit_key() {
assert!(validate_display_name(" Lena ").is_ok());
assert_eq!(validate_display_name(" Lena ").unwrap(), "Lena");
// The case that made the limiter itself the exhaustion primitive.
assert!(validate_display_name(&"a".repeat(51)).is_err());
assert!(validate_display_name(&"a".repeat(2_000_000)).is_err());
assert!(validate_display_name(" ").is_err());
assert!(validate_display_name("bad\0name").is_err());
}
}

211
backend/src/config.rs
View File

@@ -20,21 +20,53 @@ fn looks_placeholder(s: &str) -> bool {
/// Enforce secret hygiene. In production every guard is hard-fail: a booting app
/// with a publicly-known signing key is worse than one that refuses to start.
/// Outside production the dev sentinel is tolerated (warned) so local dev is frictionless.
fn validate_secrets(is_prod: bool, jwt_secret: &str, admin_password_hash: &str) -> Result<()> {
///
/// EVERY failure is collected and reported together. Returning on the first one made fixing two
/// secrets cost two boot cycles — the operator rotates JWT_SECRET, restarts, and only then learns
/// about ADMIN_PASSWORD_HASH. Restarting this stack is not free (Caddy waits on the unhealthy app),
/// and each avoidable cycle is another chance to reach for `down -v`.
fn validate_secrets(
is_prod: bool,
jwt_secret: &str,
admin_password_hash: &str,
database_url: &str,
) -> Result<()> {
if is_prod {
let mut problems: Vec<&str> = Vec::new();
if looks_placeholder(jwt_secret) {
return Err(anyhow!(
"Refusing to start in production with a placeholder JWT_SECRET — \
rotate it (openssl rand -hex 64)."
));
}
if jwt_secret.len() < 32 {
return Err(anyhow!("JWT_SECRET must be at least 32 characters."));
problems.push(
"JWT_SECRET is still the .env.example placeholder — rotate it \
(openssl rand -hex 64).",
);
} else if jwt_secret.len() < 32 {
problems.push("JWT_SECRET must be at least 32 characters.");
}
if admin_password_hash.is_empty() || looks_placeholder(admin_password_hash) {
problems.push(
"ADMIN_PASSWORD_HASH is unset or still the .env.example placeholder — generate one \
(docker run --rm caddy:2-alpine caddy hash-password --plaintext '<password>').",
);
}
// The DATABASE_URL carries the Postgres password, so a placeholder here means the stack is
// running on `CHANGE_ME_use_a_strong_password` — a credential published in the repo. The
// app used to boot green on it, because this guard only ever covered the two secrets it
// was written for and nothing else looked at POSTGRES_PASSWORD at all.
//
// Read POSTGRES_PASSWORD's docs before changing this: it is applied ONLY at initdb, so the
// remedy is not "edit .env and restart" — see the 28P01 diagnostic in db.rs.
if looks_placeholder(database_url) {
problems.push(
"DATABASE_URL still carries the .env.example placeholder password — set a strong \
one (openssl rand -hex 24) in BOTH DATABASE_URL and POSTGRES_PASSWORD.",
);
}
if !problems.is_empty() {
return Err(anyhow!(
"Refusing to start in production without a real ADMIN_PASSWORD_HASH — \
generate one (htpasswd -bnBC 12 '' <password> | tr -d ':\\n')."
"Refusing to start in production — {} secret(s) still unset or placeholder:\n - {}\n\
ALL secrets must be set BEFORE the first `docker compose up -d`: Postgres bakes \
POSTGRES_PASSWORD into its data directory on first boot and ignores later changes.",
problems.len(),
problems.join("\n - ")
));
}
} else if jwt_secret == DEV_JWT_SECRET_SENTINEL {
@@ -63,8 +95,25 @@ pub struct AppConfig {
pub app_port: u16,
/// Number of concurrent media compression workers (read once at boot).
pub compression_concurrency: usize,
/// Master switch for the comment feature (env `COMMENTS_ENABLED`, default true).
/// When false the backend rejects new comments and the frontend hides the whole
/// comment UI. Existing comments stay in the DB (hidden), so flipping it back
/// restores them. Boot-time immutable, like `compression_concurrency`.
pub comments_enabled: bool,
/// Default colour theme, used as the fallback when the DB config keys are unset.
/// Runtime overrides live in the `config` table (admin UI); these env vars only
/// seed the initial default. `preset` is an id the frontend knows (e.g.
/// "champagne-gold", "rose", … or "custom"); the two seeds are `#rrggbb` brand +
/// accent colours the whole palette is derived from.
pub default_theme_preset: String,
pub default_theme_primary: String,
pub default_theme_accent: String,
}
/// The shipped default brand/accent seed (champagne gold — matches the hand-tuned
/// ramp in tailwind-theme.css). Kept here so an unset env still yields the current look.
const DEFAULT_THEME_SEED: &str = "#8a6a2b";
impl AppConfig {
pub fn from_env() -> Result<Self> {
let app_env = std::env::var("APP_ENV").unwrap_or_else(|_| "development".to_string());
@@ -72,11 +121,12 @@ impl AppConfig {
let jwt_secret = std::env::var("JWT_SECRET").context("JWT_SECRET must be set")?;
let admin_password_hash = std::env::var("ADMIN_PASSWORD_HASH").unwrap_or_default();
let database_url = std::env::var("DATABASE_URL").context("DATABASE_URL must be set")?;
validate_secrets(is_prod, &jwt_secret, &admin_password_hash)?;
validate_secrets(is_prod, &jwt_secret, &admin_password_hash, &database_url)?;
Ok(Self {
database_url: std::env::var("DATABASE_URL").context("DATABASE_URL must be set")?,
database_url,
jwt_secret,
session_expiry_days: std::env::var("SESSION_EXPIRY_DAYS")
.unwrap_or_else(|_| "30".to_string())
@@ -100,6 +150,20 @@ impl AppConfig {
.and_then(|v| v.parse().ok())
.filter(|&n| n >= 1)
.unwrap_or(2),
comments_enabled: std::env::var("COMMENTS_ENABLED")
.map(|v| {
!matches!(
v.trim().to_ascii_lowercase().as_str(),
"false" | "0" | "no" | "off"
)
})
.unwrap_or(true),
default_theme_preset: std::env::var("THEME_PRESET")
.unwrap_or_else(|_| "champagne-gold".to_string()),
default_theme_primary: std::env::var("THEME_PRIMARY")
.unwrap_or_else(|_| DEFAULT_THEME_SEED.to_string()),
default_theme_accent: std::env::var("THEME_ACCENT")
.unwrap_or_else(|_| DEFAULT_THEME_SEED.to_string()),
})
}
}
@@ -110,12 +174,18 @@ mod tests {
const REAL_SECRET: &str = "a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2";
const REAL_HASH: &str = "$2y$12$abcdefghijklmnopqrstuv.wxyzABCDEFGHIJKLMNOPQRSTUVWXYZ012";
const REAL_DB_URL: &str = "postgres://eventsnap:7f3a9c1e5b2d8a4f@db:5432/eventsnap";
#[test]
fn prod_rejects_shipped_placeholder_secret() {
// The exact string shipped in `.env` — >32 chars, so it must be caught by
// the substring guard, not the length check.
let err = validate_secrets(true, "change_me_to_a_random_64_byte_hex_string", REAL_HASH);
let err = validate_secrets(
true,
"change_me_to_a_random_64_byte_hex_string",
REAL_HASH,
REAL_DB_URL,
);
assert!(
err.is_err(),
"placeholder JWT_SECRET must be rejected in prod"
@@ -124,29 +194,108 @@ mod tests {
#[test]
fn prod_rejects_dev_sentinel_and_short_secret() {
assert!(validate_secrets(true, DEV_JWT_SECRET_SENTINEL, REAL_HASH).is_err());
assert!(validate_secrets(true, "tooshort", REAL_HASH).is_err());
assert!(validate_secrets(true, DEV_JWT_SECRET_SENTINEL, REAL_HASH, REAL_DB_URL).is_err());
assert!(validate_secrets(true, "tooshort", REAL_HASH, REAL_DB_URL).is_err());
}
#[test]
fn prod_rejects_missing_or_placeholder_admin_hash() {
assert!(validate_secrets(true, REAL_SECRET, "").is_err());
assert!(validate_secrets(true, REAL_SECRET, "$2y$12$placeholder_replace_me").is_err());
assert!(validate_secrets(true, REAL_SECRET, "", REAL_DB_URL).is_err());
assert!(
validate_secrets(
true,
REAL_SECRET,
"$2y$12$placeholder_replace_me",
REAL_DB_URL
)
.is_err()
);
}
/// The stack used to come up GREEN on the database password published in the repo: this guard
/// covered the two secrets it was written for, and nothing anywhere looked at the Postgres
/// credential. README step 2 doesn't name POSTGRES_PASSWORD either, so following the
/// documented procedure verbatim shipped it.
#[test]
fn prod_rejects_the_shipped_placeholder_database_password() {
let shipped = "postgres://eventsnap:CHANGE_ME_use_a_strong_password@db:5432/eventsnap";
let err = validate_secrets(true, REAL_SECRET, REAL_HASH, shipped).unwrap_err();
assert!(
err.to_string().contains("DATABASE_URL"),
"the refusal must name DATABASE_URL, not just fail: {err}"
);
// And it must point at the initdb trap, or the operator edits .env, restarts, and lands
// in a permanent auth-failure loop instead.
assert!(
err.to_string().contains("POSTGRES_PASSWORD"),
"the refusal must name POSTGRES_PASSWORD as the other half: {err}"
);
}
/// Every problem in ONE message. Reporting them one per boot made fixing two secrets cost two
/// restart cycles, on a stack where Caddy waits on the unhealthy app the whole time.
#[test]
fn prod_reports_every_placeholder_at_once() {
let err = validate_secrets(
true,
"change_me_to_a_random_64_byte_hex_string",
"$2y$12$placeholder_replace_me",
"postgres://eventsnap:CHANGE_ME_use_a_strong_password@db:5432/eventsnap",
)
.unwrap_err()
.to_string();
for expected in ["JWT_SECRET", "ADMIN_PASSWORD_HASH", "DATABASE_URL"] {
assert!(err.contains(expected), "{expected} missing from: {err}");
}
assert!(
err.contains("3 secret(s)"),
"the count must match what is listed: {err}"
);
}
#[test]
fn prod_accepts_real_secrets() {
assert!(validate_secrets(true, REAL_SECRET, REAL_HASH).is_ok());
assert!(validate_secrets(true, REAL_SECRET, REAL_HASH, REAL_DB_URL).is_ok());
}
/// A real password that happens to contain no placeholder substring must pass — including one
/// with URL-ish punctuation, so the guard can't be mistaken for a URL validator.
#[test]
fn prod_accepts_a_real_database_url_with_awkward_punctuation() {
assert!(
validate_secrets(
true,
REAL_SECRET,
REAL_HASH,
"postgres://eventsnap:aB3%24xY9-_.qW@db:5432/eventsnap"
)
.is_ok()
);
}
/// The e2e stack runs without APP_ENV=production, so none of this applies there — but assert
/// it, because a guard that tripped in e2e would be found the hard way.
#[test]
fn non_prod_ignores_a_placeholder_database_url() {
assert!(
validate_secrets(
false,
REAL_SECRET,
"",
"postgres://eventsnap:CHANGE_ME_use_a_strong_password@db:5432/eventsnap"
)
.is_ok()
);
}
#[test]
fn non_prod_tolerates_dev_sentinel() {
assert!(validate_secrets(false, DEV_JWT_SECRET_SENTINEL, "").is_ok());
assert!(validate_secrets(false, DEV_JWT_SECRET_SENTINEL, "", REAL_DB_URL).is_ok());
}
#[test]
fn non_prod_still_rejects_short_non_sentinel_secret() {
assert!(validate_secrets(false, "tooshort", "").is_err());
assert!(validate_secrets(false, "tooshort", "", REAL_DB_URL).is_err());
}
#[test]
@@ -154,9 +303,23 @@ mod tests {
// looks_placeholder lowercases before matching — an upper/mixed-case
// placeholder must still be rejected in prod.
assert!(
validate_secrets(true, "CHANGE_ME_TO_A_RANDOM_64_BYTE_HEX_STRING", REAL_HASH).is_err()
validate_secrets(
true,
"CHANGE_ME_TO_A_RANDOM_64_BYTE_HEX_STRING",
REAL_HASH,
REAL_DB_URL
)
.is_err()
);
assert!(
validate_secrets(
true,
REAL_SECRET,
"$2Y$12$PLACEHOLDER_replace_me",
REAL_DB_URL
)
.is_err()
);
assert!(validate_secrets(true, REAL_SECRET, "$2Y$12$PLACEHOLDER_replace_me").is_err());
}
#[test]
@@ -166,7 +329,7 @@ mod tests {
const LEN_31: &str = "abcdefghijklmnopqrstuvwxyz01234";
assert_eq!(LEN_32.len(), 32);
assert_eq!(LEN_31.len(), 31);
assert!(validate_secrets(true, LEN_32, REAL_HASH).is_ok());
assert!(validate_secrets(true, LEN_31, REAL_HASH).is_err());
assert!(validate_secrets(true, LEN_32, REAL_HASH, REAL_DB_URL).is_ok());
assert!(validate_secrets(true, LEN_31, REAL_HASH, REAL_DB_URL).is_err());
}
}

62
backend/src/db.rs
View File

@@ -4,17 +4,75 @@ use sqlx::postgres::PgPoolOptions;
const DEFAULT_MAX_CONNECTIONS: u32 = 10;
/// How long a request waits for a pool connection before being shed.
///
/// sqlx's default is 30 s — longer than the frontend's own 20 s fetch timeout (api.ts
/// TIMEOUT_MS), so under saturation the browser gave up while the server kept holding the
/// slot: the client saw a timeout, the server saw a completed request, and the work was done
/// for nobody. Failing fast sheds load instead of compounding it, and `From<sqlx::Error>` in
/// error.rs turns the timeout into a 503 with a Retry-After rather than a bare 500.
const ACQUIRE_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(5);
/// SQLSTATE for `invalid_password`.
const PG_INVALID_PASSWORD: &str = "28P01";
/// Turn the one connect failure with an unguessable cause into a self-explaining one.
///
/// `POSTGRES_PASSWORD` is honoured ONLY when Postgres initialises its data directory. Change it in
/// `.env` afterwards and the app authenticates with the new password against a volume that still
/// holds the old one — a permanent restart loop whose only symptom is
/// `password authentication failed`.
///
/// The production secret guard makes that sequence NEARLY CERTAIN rather than rare: it stops the
/// app on the first `docker compose up -d`, but not the `db` service in that same command, which
/// initialises and bakes in whatever password was in `.env` at that moment. So the intended
/// recovery — see the refusal, fix your secrets, boot again — is exactly the sequence that breaks
/// it. Nothing in the error names the cause, and the remedy destroys data, so it is the last thing
/// an operator should guess at.
fn explain_auth_failure(err: &sqlx::Error) {
let is_auth_failure = match err {
sqlx::Error::Database(db) => db.code().as_deref() == Some(PG_INVALID_PASSWORD),
_ => false,
};
if !is_auth_failure {
return;
}
tracing::error!(
"Postgres rejected the credentials in DATABASE_URL (SQLSTATE {PG_INVALID_PASSWORD}).\n\
\n\
This almost always means POSTGRES_PASSWORD was changed AFTER the database volume was \
first created. Postgres applies that variable only when it initialises its data \
directory; editing .env and restarting does not change the stored password, so the two \
drift apart permanently.\n\
\n\
If the event has NOT started and you have no data worth keeping:\n\n \
docker compose down -v && docker compose up -d\n\n\
(-v DELETES the database, the uploaded media and the exports. There is no undo.)\n\
\n\
If you DO have data: restore the old password into DATABASE_URL instead, or change the \
stored one with ALTER ROLE inside the running db container. Never reach for -v to fix a \
login problem on a live event."
);
}
pub async fn create_pool(database_url: &str) -> Result<PgPool> {
let max_connections = std::env::var("DATABASE_MAX_CONNECTIONS")
.ok()
.and_then(|s| s.parse::<u32>().ok())
.unwrap_or(DEFAULT_MAX_CONNECTIONS);
let pool = PgPoolOptions::new()
let pool = match PgPoolOptions::new()
.max_connections(max_connections)
.acquire_timeout(ACQUIRE_TIMEOUT)
.connect(database_url)
.await
.context("failed to connect to database")?;
{
Ok(pool) => pool,
Err(e) => {
explain_auth_failure(&e);
return Err(e).context("failed to connect to database");
}
};
sqlx::migrate!()
.run(&pool)

101
backend/src/error.rs
View File

@@ -19,6 +19,12 @@ pub enum AppError {
/// the client can treat it as *terminal* (413, no retry) instead of backing off and
/// retrying a permanently-failing upload forever.
QuotaExceeded(String),
/// The server is temporarily unable to serve this request — currently only pool
/// saturation. Distinct from `Internal` because it is TRANSIENT and the client should be
/// told so: a 500 reads as "this request is broken", while a 503 + Retry-After reads as
/// "come back shortly", which is what the upload queue's retry classifier needs to make
/// the right call. Second field: optional retry-after seconds.
ServiceUnavailable(String, Option<u64>),
Internal(anyhow::Error),
}
@@ -33,6 +39,9 @@ impl AppError {
Self::Conflict(_) => (StatusCode::CONFLICT, "conflict"),
Self::TooManyRequests(..) => (StatusCode::TOO_MANY_REQUESTS, "too_many_requests"),
Self::QuotaExceeded(_) => (StatusCode::PAYLOAD_TOO_LARGE, "quota_exceeded"),
Self::ServiceUnavailable(..) => {
(StatusCode::SERVICE_UNAVAILABLE, "service_unavailable")
}
Self::Internal(_) => (StatusCode::INTERNAL_SERVER_ERROR, "internal_error"),
}
}
@@ -46,6 +55,7 @@ impl AppError {
| Self::NotFound(msg)
| Self::Conflict(msg) => msg.clone(),
Self::TooManyRequests(msg, _) => msg.clone(),
Self::ServiceUnavailable(msg, _) => msg.clone(),
Self::QuotaExceeded(msg) => msg.clone(),
Self::Internal(err) => {
tracing::error!("internal error: {err:#}");
@@ -58,10 +68,13 @@ impl AppError {
impl IntoResponse for AppError {
fn into_response(self) -> Response {
let (status, code) = self.status_and_code();
let retry_after_secs = if let Self::TooManyRequests(_, Some(secs)) = &self {
Some(*secs)
} else {
None
// BOTH retry-carrying variants must be matched here. `message()` would fail to
// compile on a missing arm; this one would not — it would silently drop the header and
// the `retry_after_secs` body field, which is exactly the sort of omission that only
// shows up under the load the 503 exists for.
let retry_after_secs = match &self {
Self::TooManyRequests(_, secs) | Self::ServiceUnavailable(_, secs) => *secs,
_ => None,
};
let message = self.message();
@@ -93,6 +106,84 @@ impl From<anyhow::Error> for AppError {
impl From<sqlx::Error> for AppError {
fn from(err: sqlx::Error) -> Self {
Self::Internal(err.into())
match err {
// Pool saturation is load, not a bug. Reporting it as a 500 was actively harmful:
// the frontend's upload-queue classifier treats 5xx as transient and retries, so
// the retries piled straight back into the saturated pool with no Retry-After to
// pace them. A 503 says the same thing honestly and carries the backoff.
//
// `PoolClosed` stays `Internal` — it only happens during shutdown, where a 503
// would invite a retry against a server that is going away.
sqlx::Error::PoolTimedOut => {
tracing::warn!("database pool exhausted; shedding a request with 503");
Self::ServiceUnavailable(
"Server ist gerade ausgelastet. Bitte versuche es in ein paar Sekunden erneut."
.into(),
Some(POOL_TIMEOUT_RETRY_AFTER_SECS),
)
}
other => Self::Internal(other.into()),
}
}
}
/// Retry-After for a shed request. Short: pool saturation clears in seconds once the queue
/// drains, and a long value would make a brief spike feel like an outage.
const POOL_TIMEOUT_RETRY_AFTER_SECS: u64 = 3;
#[cfg(test)]
mod tests {
use super::*;
/// `into_response` extracts `retry_after_secs` by MATCHING ON VARIANTS, so unlike
/// `message()` a missing arm is not a compile error — it silently drops the header. Pin the
/// behaviour for both retry-carrying variants.
#[test]
fn both_retry_carrying_variants_emit_retry_after() {
for err in [
AppError::TooManyRequests("slow down".into(), Some(42)),
AppError::ServiceUnavailable("busy".into(), Some(3)),
] {
let expected = match &err {
AppError::TooManyRequests(_, Some(s)) | AppError::ServiceUnavailable(_, Some(s)) => {
s.to_string()
}
_ => unreachable!(),
};
let resp = err.into_response();
assert_eq!(
resp.headers()
.get(axum::http::header::RETRY_AFTER)
.and_then(|v| v.to_str().ok()),
Some(expected.as_str()),
"a shed/throttled client must be told when to come back"
);
}
}
/// Pool saturation is load, not a bug. A 500 makes the frontend's retry classifier pile
/// straight back into the saturated pool with no backoff to pace it.
#[test]
fn pool_exhaustion_sheds_with_503_but_shutdown_does_not() {
let shed: AppError = sqlx::Error::PoolTimedOut.into();
assert_eq!(
shed.status_and_code(),
(StatusCode::SERVICE_UNAVAILABLE, "service_unavailable")
);
// PoolClosed only happens during shutdown; a 503 there would invite a retry against a
// server that is going away.
let closing: AppError = sqlx::Error::PoolClosed.into();
assert_eq!(
closing.status_and_code(),
(StatusCode::INTERNAL_SERVER_ERROR, "internal_error")
);
// Everything else must keep its existing mapping.
let missing: AppError = sqlx::Error::RowNotFound.into();
assert_eq!(
missing.status_and_code(),
(StatusCode::INTERNAL_SERVER_ERROR, "internal_error")
);
}
}

150
backend/src/handlers/admin.rs
View File

@@ -3,13 +3,13 @@ use std::time::Duration;
use axum::Json;
use axum::extract::{Query, State};
use axum::http::{HeaderMap, StatusCode};
use axum::http::StatusCode;
use serde::{Deserialize, Serialize};
use uuid::Uuid;
use crate::auth::middleware::RequireAdmin;
use crate::error::AppError;
use crate::services::config;
use crate::services::rate_limiter::client_ip;
use crate::state::AppState;
// ── DTOs ─────────────────────────────────────────────────────────────────────
@@ -120,6 +120,16 @@ pub async fn patch_config(
("upload_rate_per_hour", true, 1.0, 100_000.0),
("feed_rate_per_min", true, 1.0, 100_000.0),
("export_rate_per_day", true, 1.0, 100_000.0),
// Loose per-IP ceiling on /join. The real anti-spam bucket is per (ip, name); this
// only bounds raw volume from one source, so it must stay well above the size of a
// party arriving at once (see migration 017).
("join_ip_rate_per_min", true, 1.0, 100_000.0),
// Same shape for /recover: the per-(ip, name) bucket is the anti-guessing control,
// this only bounds a name-cycling flood in front of a cost-12 bcrypt (migration 019).
("recover_ip_rate_per_min", true, 1.0, 100_000.0),
// Aggregate ceiling on likes + comments + comment deletions, per user per minute.
// These were the only mutating endpoints with no limit at all (migration 020).
("social_rate_per_min", true, 1.0, 100_000.0),
("quota_tolerance", false, 0.0, 1.0),
("estimated_guest_count", true, 1.0, 1_000_000.0),
];
@@ -134,14 +144,32 @@ pub async fn patch_config(
// missing from this allowlist — so the switch existed in code and could never be flipped.
"admin_login_rate_enabled",
"recover_rate_enabled",
"social_rate_enabled",
"quota_enabled",
"storage_quota_enabled",
"upload_count_quota_enabled",
];
const TEXT_KEYS: &[&str] = &["privacy_note"];
const TEXT_KEYS: &[&str] = &[
"privacy_note",
"theme_preset",
"theme_primary",
"theme_accent",
];
const PRIVACY_NOTE_MAX_LEN: usize = 16 * 1024; // 16 KiB free text is plenty
// Preset ids the frontend knows how to render (mirror of PRESETS in
// frontend/src/lib/theme/palette.ts). "custom" means "use the theme_primary/accent
// seeds verbatim". Kept in sync by hand — a new preset must be added in both places.
const THEME_PRESETS: &[&str] = &[
"champagne-gold",
"rose",
"sage",
"dusk-blue",
"classic-silver",
"custom",
];
let mut privacy_note_changed = false;
let mut theme_changed = false;
// Validate every key first so a bad value in the batch can't leave a partial
// update behind — validation must fully precede any write.
@@ -169,6 +197,23 @@ pub async fn patch_config(
"Wert für {key} liegt außerhalb des zulässigen Bereichs ({min}–{max})."
)));
}
// Zero is in range and catastrophic. `quota_tolerance` is the multiplier in
// `free_disk * tolerance / active_uploaders`, so 0 makes every per-user limit 0 and
// refuses EVERY upload — mid-event, with "Du hast dein Upload-Limit für dieses Event
// erreicht", an error naming the wrong cause entirely. `storage_quota_enabled` is the
// intended off-switch.
//
// Rejecting the value rather than raising the floor: very small tolerances are
// legitimate (they are how a large disk is throttled down to a sensible per-guest
// ceiling, and how the e2e quota tests steer it — around 1e-5 on a 174 GB volume), so
// a floor of, say, 0.01 would forbid real configurations to prevent one typo.
if key_str == "quota_tolerance" && n == 0.0 {
return Err(AppError::BadRequest(
"quota_tolerance = 0 würde jeden Upload blockieren. Zum Abschalten der \
Speicher-Quote stattdessen „Speicher-Quote aktiv“ ausschalten."
.into(),
));
}
} else if BOOL_KEYS.contains(&key_str) {
match value.trim().to_ascii_lowercase().as_str() {
"true" | "false" | "1" | "0" | "yes" | "no" | "on" | "off" => {}
@@ -186,8 +231,23 @@ pub async fn patch_config(
"Wert für {key} ist zu lang (max. {PRIVACY_NOTE_MAX_LEN} Zeichen)."
)));
}
if key_str == "privacy_note" {
privacy_note_changed = true;
match key_str {
"privacy_note" => privacy_note_changed = true,
"theme_preset" => {
if !THEME_PRESETS.contains(&value.trim()) {
return Err(AppError::BadRequest(format!("Ungültiges Theme: {value}.")));
}
theme_changed = true;
}
"theme_primary" | "theme_accent" => {
if !is_hex_color(value.trim()) {
return Err(AppError::BadRequest(format!(
"Ungültige Farbe für {key}: muss #rrggbb sein."
)));
}
theme_changed = true;
}
_ => {}
}
} else {
return Err(AppError::BadRequest(format!(
@@ -217,16 +277,30 @@ pub async fn patch_config(
// Notify all clients that a publicly-readable config value changed so their stores
// (e.g. the privacy note in My Account) refresh without a manual reload.
if privacy_note_changed {
if privacy_note_changed || theme_changed {
let mut keys: Vec<&str> = Vec::new();
if privacy_note_changed {
keys.push("privacy_note");
}
if theme_changed {
keys.push("theme");
}
let _ = state.sse_tx.send(crate::state::SseEvent::new(
"event-updated",
serde_json::json!({ "keys": ["privacy_note"] }).to_string(),
serde_json::json!({ "keys": keys }).to_string(),
));
}
Ok(StatusCode::NO_CONTENT)
}
/// A strict `#rrggbb` hex-colour check (6 hex digits, leading `#`). Deliberately not
/// accepting shorthand/`#rgba` so the value is safe to drop straight into CSS.
fn is_hex_color(s: &str) -> bool {
let bytes = s.as_bytes();
bytes.len() == 7 && bytes[0] == b'#' && bytes[1..].iter().all(|b| b.is_ascii_hexdigit())
}
pub async fn get_export_jobs(
State(state): State<AppState>,
RequireAdmin(_auth): RequireAdmin,
@@ -274,25 +348,26 @@ pub async fn export_ticket(
}
/// Validate a download ticket (single-use) and confirm its session still exists.
async fn authenticate_download_ticket(state: &AppState, ticket: &str) -> Result<(), AppError> {
/// Resolve a single-use download ticket to the user who minted it. The caller needs the
/// id to key the export rate limit per-user (see `enforce_export_rate`).
async fn authenticate_download_ticket(state: &AppState, ticket: &str) -> Result<Uuid, AppError> {
let token_hash = state
.sse_tickets
.consume(ticket)
.ok_or_else(|| AppError::Unauthorized("Ticket ungültig oder abgelaufen.".into()))?;
crate::models::session::Session::find_by_token_hash(&state.pool, &token_hash)
let session = crate::models::session::Session::find_by_token_hash(&state.pool, &token_hash)
.await
.map_err(|e| AppError::Internal(e.into()))?
.ok_or_else(|| AppError::Unauthorized("Sitzung nicht gefunden.".into()))?;
Ok(())
Ok(session.user_id)
}
pub async fn download_zip(
State(state): State<AppState>,
Query(q): Query<DownloadQuery>,
headers: HeaderMap,
) -> Result<axum::response::Response, AppError> {
authenticate_download_ticket(&state, &q.ticket).await?;
enforce_export_rate(&state, &headers).await?;
let user_id = authenticate_download_ticket(&state, &q.ticket).await?;
enforce_export_rate(&state, user_id).await?;
let path =
resolve_export_file(&state, "zip", "Der ZIP-Export ist noch nicht verfügbar.").await?;
@@ -343,10 +418,9 @@ async fn resolve_export_file(
pub async fn download_html(
State(state): State<AppState>,
Query(q): Query<DownloadQuery>,
headers: HeaderMap,
) -> Result<axum::response::Response, AppError> {
authenticate_download_ticket(&state, &q.ticket).await?;
enforce_export_rate(&state, &headers).await?;
let user_id = authenticate_download_ticket(&state, &q.ticket).await?;
enforce_export_rate(&state, user_id).await?;
let path =
resolve_export_file(&state, "html", "Der HTML-Export ist noch nicht verfügbar.").await?;
@@ -403,8 +477,13 @@ pub async fn export_status(
// worker superseded mid-run) is meaningless — surfacing its frozen `running`/77% would show a
// progress bar that never moves for a keepsake nobody is building. It reads as "locked" (no
// current job), which is exactly what it is.
let jobs: Vec<(String, String, i16)> = sqlx::query_as(
"SELECT j.type::text, j.status::text, j.progress_pct
// `error_message` is carried here, not just on the admin dashboard's job list. The host is the
// one who releases the keepsake and the one who owns the "Erneut versuchen" button, but this
// endpoint used to hand them a bare `failed` — so a fully actionable reason (notably the disk
// preflight's "needs X GB, Y GB free") was written to the row and then shown to nobody who
// could act on it. An admin-only diagnostic is not a diagnostic for the person on the spot.
let jobs: Vec<(String, String, i16, Option<String>)> = sqlx::query_as(
"SELECT j.type::text, j.status::text, j.progress_pct, j.error_message
FROM export_job j
JOIN event e ON e.id = j.event_id
WHERE e.id = $1 AND j.epoch = e.export_epoch",
@@ -415,9 +494,21 @@ pub async fn export_status(
let job_status = |type_name: &str| {
jobs.iter()
.find(|(t, _, _)| t == type_name)
.map(|(_, status, pct)| serde_json::json!({ "status": status, "progress_pct": pct }))
.unwrap_or_else(|| serde_json::json!({ "status": "locked", "progress_pct": 0 }))
.find(|(t, _, _, _)| t == type_name)
.map(|(_, status, pct, err)| {
serde_json::json!({
"status": status,
"progress_pct": pct,
// Only on a failure. A stale message left on a row that has since been re-armed
// would otherwise show an error next to a running progress bar.
"error_message": if status == "failed" { err.clone() } else { None },
})
})
.unwrap_or_else(|| {
serde_json::json!({
"status": "locked", "progress_pct": 0, "error_message": null,
})
})
};
Ok(Json(serde_json::json!({
@@ -430,21 +521,24 @@ pub async fn export_status(
/// Centralised guard for the export rate limit. Same pattern as upload/feed: master
/// switch + per-endpoint switch + numeric value, all stored in `config` and read on
/// each request.
async fn enforce_export_rate(state: &AppState, headers: &HeaderMap) -> Result<(), AppError> {
async fn enforce_export_rate(state: &AppState, user_id: Uuid) -> Result<(), AppError> {
let rate_limits_on = config::get_bool(&state.config_cache, "rate_limits_enabled", true).await;
let export_rate_on = config::get_bool(&state.config_cache, "export_rate_enabled", true).await;
if !(rate_limits_on && export_rate_on) {
return Ok(());
}
let ip = client_ip(headers, "unknown");
let limit = config::get_usize(&state.config_cache, "export_rate_per_day", 3).await;
if !state
.rate_limiter
.check(format!("export:{ip}"), limit, Duration::from_secs(86400))
{
// Keyed per-user. This was the worst of the IP-keyed limiters: 3 downloads per DAY
// shared across every guest behind the venue's public IP, so the fourth person to
// fetch their keepsake was locked out until the next day.
if let Err(retry_after_secs) = state.rate_limiter.check_with_retry(
format!("export:{user_id}"),
limit,
Duration::from_secs(86400),
) {
return Err(AppError::TooManyRequests(
"Zu viele Anfragen. Bitte warte kurz und versuche es erneut.".into(),
None,
Some(retry_after_secs),
));
}
Ok(())

41
backend/src/handlers/feed.rs
View File

@@ -2,7 +2,6 @@ use std::time::Duration;
use axum::Json;
use axum::extract::{Query, State};
use axum::http::HeaderMap;
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use uuid::Uuid;
@@ -10,7 +9,6 @@ use uuid::Uuid;
use crate::auth::middleware::AuthUser;
use crate::error::AppError;
use crate::services::config;
use crate::services::rate_limiter::client_ip;
use crate::state::AppState;
#[derive(Deserialize)]
@@ -27,6 +25,8 @@ pub struct FeedUpload {
pub uploader_name: String,
pub preview_url: Option<String>,
pub thumbnail_url: Option<String>,
/// Big-screen (~2048px) variant for the diashow. Absent until the derivative exists.
pub display_url: Option<String>,
pub mime_type: String,
pub caption: Option<String>,
pub like_count: i64,
@@ -48,6 +48,7 @@ struct FeedRow {
uploader_name: String,
preview_path: Option<String>,
thumbnail_path: Option<String>,
display_path: Option<String>,
mime_type: String,
caption: Option<String>,
like_count: i64,
@@ -58,21 +59,23 @@ struct FeedRow {
pub async fn feed(
State(state): State<AppState>,
auth: AuthUser,
headers: HeaderMap,
Query(q): Query<FeedQuery>,
) -> Result<Json<FeedResponse>, AppError> {
let ip = client_ip(&headers, "unknown");
let rate_limits_on = config::get_bool(&state.config_cache, "rate_limits_enabled", true).await;
let feed_rate_on = config::get_bool(&state.config_cache, "feed_rate_enabled", true).await;
if rate_limits_on && feed_rate_on {
let rate_limit = config::get_usize(&state.config_cache, "feed_rate_per_min", 60).await;
if !state
.rate_limiter
.check(format!("feed:{ip}"), rate_limit, Duration::from_secs(60))
{
// Keyed per-user, exactly like `feed_delta` below: at a venue every guest shares
// one public IP, so an IP key gave the whole party a single 60/min bucket and the
// fastest scroller starved everyone else.
if let Err(retry_after_secs) = state.rate_limiter.check_with_retry(
format!("feed:{}", auth.user_id),
rate_limit,
Duration::from_secs(60),
) {
return Err(AppError::TooManyRequests(
"Zu viele Anfragen. Bitte warte kurz und versuche es erneut.".into(),
None,
Some(retry_after_secs),
));
}
}
@@ -94,7 +97,8 @@ pub async fn feed(
let tag = hashtag.trim().trim_start_matches('#').to_lowercase();
sqlx::query_as::<_, FeedRow>(
"SELECT v.id, v.user_id, v.uploader_name, v.preview_path, v.thumbnail_path,
v.mime_type, v.caption, v.like_count, v.comment_count, v.created_at
v.display_path, v.mime_type, v.caption, v.like_count, v.comment_count,
v.created_at
FROM v_feed v
JOIN upload_hashtag uh ON uh.upload_id = v.id
JOIN hashtag h ON h.id = uh.hashtag_id AND h.tag = $1
@@ -113,7 +117,7 @@ pub async fn feed(
} else {
sqlx::query_as::<_, FeedRow>(
"SELECT id, user_id, uploader_name, preview_path, thumbnail_path,
mime_type, caption, like_count, comment_count, created_at
display_path, mime_type, caption, like_count, comment_count, created_at
FROM v_feed
WHERE event_id = $1
AND ($2::timestamptz IS NULL OR (created_at, id) < ($2, $3))
@@ -154,6 +158,10 @@ pub async fn feed(
.thumbnail_path
.as_ref()
.map(|_| format!("/api/v1/upload/{}/thumbnail", r.id));
let display_url = r
.display_path
.as_ref()
.map(|_| format!("/api/v1/upload/{}/display", r.id));
FeedUpload {
liked_by_me: liked_set.contains(&r.id),
id: r.id,
@@ -161,6 +169,7 @@ pub async fn feed(
uploader_name: r.uploader_name,
preview_url,
thumbnail_url,
display_url,
mime_type: r.mime_type,
caption: r.caption,
like_count: r.like_count,
@@ -216,14 +225,14 @@ pub async fn feed_delta(
let feed_rate_on = config::get_bool(&state.config_cache, "feed_rate_enabled", true).await;
if rate_limits_on && feed_rate_on {
let rate_limit = config::get_usize(&state.config_cache, "feed_rate_per_min", 60).await;
if !state.rate_limiter.check(
if let Err(retry_after_secs) = state.rate_limiter.check_with_retry(
format!("feed_delta:{}", auth.user_id),
rate_limit,
Duration::from_secs(60),
) {
return Err(AppError::TooManyRequests(
"Zu viele Anfragen. Bitte warte kurz und versuche es erneut.".into(),
None,
Some(retry_after_secs),
));
}
}
@@ -247,7 +256,7 @@ pub async fn feed_delta(
// response's `server_time`, so this doesn't re-fetch on every subsequent delta.
let rows = sqlx::query_as::<_, FeedRow>(
"SELECT id, user_id, uploader_name, preview_path, thumbnail_path,
mime_type, caption, like_count, comment_count, created_at
display_path, mime_type, caption, like_count, comment_count, created_at
FROM v_feed
WHERE event_id = $1 AND created_at >= $2
ORDER BY created_at DESC, id DESC
@@ -304,6 +313,10 @@ pub async fn feed_delta(
.thumbnail_path
.as_ref()
.map(|_| format!("/api/v1/upload/{}/thumbnail", r.id)),
display_url: r
.display_path
.as_ref()
.map(|_| format!("/api/v1/upload/{}/display", r.id)),
mime_type: r.mime_type,
caption: r.caption,
like_count: r.like_count,

43
backend/src/handlers/health.rs Normal file
View File

@@ -0,0 +1,43 @@
use std::time::Duration;
use axum::extract::State;
use axum::http::StatusCode;
use crate::state::AppState;
/// How long a readiness probe waits for the database before calling the app not-ready.
///
/// Deliberately shorter than the pool's own `acquire_timeout`: a saturated pool IS a
/// not-ready condition, and reporting it as such is the point. Do not "fix" the two to
/// match — that would make the probe wait out the very saturation it exists to surface.
const READY_TIMEOUT: Duration = Duration::from_secs(2);
/// Readiness: can this instance actually serve a request end to end?
///
/// Separate from `/health` (liveness) ON PURPOSE, and the compose healthcheck must keep
/// pointing at `/health`. `caddy` declares `depends_on: app: {condition: service_healthy}`,
/// so a DB-dependent healthcheck would turn a transient Postgres hiccup during boot into
/// the reverse proxy refusing to start — converting a blip into a total outage. This route
/// is for an external monitor, which should page rather than restart.
///
/// The gap this closes: every request path touches the database, so with a dead or
/// saturated pool the app is useless while `/health` still answers "ok" — the disk-full
/// endgame (media volume fills, Postgres cannot write WAL) stayed green all the way down.
pub async fn ready(State(state): State<AppState>) -> (StatusCode, &'static str) {
let probe = sqlx::query_scalar::<_, i32>("SELECT 1").fetch_one(&state.pool);
match tokio::time::timeout(READY_TIMEOUT, probe).await {
Ok(Ok(_)) => (StatusCode::OK, "ready"),
Ok(Err(e)) => {
tracing::warn!(error = ?e, "readiness probe failed: database unreachable");
(StatusCode::SERVICE_UNAVAILABLE, "database unavailable")
}
Err(_) => {
tracing::warn!(
timeout_secs = READY_TIMEOUT.as_secs(),
"readiness probe timed out; the pool is saturated or the database is hung"
);
(StatusCode::SERVICE_UNAVAILABLE, "database timeout")
}
}
}

91
backend/src/handlers/host.rs
View File

@@ -35,6 +35,32 @@ pub struct EventStatus {
pub is_active: bool,
pub uploads_locked: bool,
pub export_released: bool,
/// Free space on the volume the keepsake is written to. `None` when the mount can't be
/// resolved — the UI hides the widget rather than rendering a confident zero.
pub disk_free_bytes: Option<u64>,
/// What a full keepsake build would need right now (both halves).
pub keepsake_required_bytes: u64,
/// Whether the host should be warned. See [`disk_is_low`].
pub disk_low: bool,
}
/// Absolute floor below which free space is worth surfacing regardless of gallery size — the
/// threshold the README has carried on the roadmap since v1.
const LOW_DISK_FLOOR_BYTES: u64 = 10_000_000_000;
/// Is free space low enough that the host needs to know?
///
/// Two triggers, because a fixed threshold answers the wrong question. `postgres_data`,
/// `media_data` and `exports_data` are all Docker named volumes on one filesystem, so a full disk
/// does not degrade one subsystem — it stops Postgres writing and takes the event down. That is
/// what the absolute floor is for.
///
/// The second trigger is the one that actually earns its place: the keepsake needs room for two
/// gallery-sized archives, and the only moment a host can do anything about that is BEFORE they
/// release. Warning at "you could not build the keepsake right now" turns a post-event dead end
/// into a decision someone can still make.
fn disk_is_low(free: u64, keepsake_required: u64) -> bool {
free < LOW_DISK_FLOOR_BYTES || free < keepsake_required
}
/// Count non-banned hosts/admins in the event OTHER than `excluding` — the operators
@@ -72,11 +98,29 @@ pub async fn get_event_status(
.await?
.ok_or_else(|| AppError::NotFound("Event nicht gefunden.".into()))?;
// Measured on the EXPORT volume, not the media one: that is where the cliff is, and it is a
// distinct mount point even when both are backed by the same filesystem. The cached reading is
// right here — this is advisory, polled on every dashboard load, and a 15s-stale number costs
// nothing (unlike the export preflight, which reads uncached because it is about to write).
let free = state
.disk_cache
.snapshot(&state.config.export_path)
.map(|d| d.free);
let keepsake_required_bytes =
crate::services::export::keepsake_space_required(&state.pool, event.id)
.await
.unwrap_or(0);
Ok(Json(EventStatus {
name: event.name,
is_active: event.is_active,
uploads_locked: event.uploads_locked_at.is_some(),
export_released: event.export_released_at.is_some(),
disk_free_bytes: free,
keepsake_required_bytes,
// Unknown free space is NOT low. Fails open, exactly as the upload quota and the export
// preflight do: a scary banner on an unreadable mount would train the host to ignore it.
disk_low: free.is_some_and(|f| disk_is_low(f, keepsake_required_bytes)),
}))
}
@@ -302,6 +346,7 @@ pub async fn rebuild_export(
r.event_id,
r.event_name,
r.epoch,
state.config.comments_enabled,
std::time::Duration::ZERO,
state.pool.clone(),
state.config.media_path.clone(),
@@ -432,7 +477,7 @@ pub async fn reset_user_pin(
}
let pin: String = format!("{:04}", rand::rng().random_range(0..10000u32));
let pin_hash = bcrypt::hash(&pin, 12).map_err(|e| AppError::Internal(anyhow::anyhow!(e)))?;
let pin_hash = crate::auth::handlers::hash_password(pin.clone(), 12).await?;
sqlx::query(
"UPDATE \"user\"
@@ -549,6 +594,7 @@ pub fn start_regen(state: &AppState, regen: crate::services::export::PendingRege
regen.event_id,
regen.event_name,
regen.epoch,
state.config.comments_enabled,
// Debounced: a takedown pass is a burst, and each request retires the last generation. The
// delay lets superseded workers fail their claim and do zero work instead of each building
// a full archive. See export::REGEN_DEBOUNCE.
@@ -753,6 +799,7 @@ pub async fn release_gallery(
event_id,
event_name,
epoch,
state.config.comments_enabled,
std::time::Duration::ZERO,
state.pool.clone(),
state.config.media_path.clone(),
@@ -762,3 +809,45 @@ pub async fn release_gallery(
Ok(StatusCode::NO_CONTENT)
}
#[cfg(test)]
mod tests {
use super::{LOW_DISK_FLOOR_BYTES, disk_is_low};
const GB: u64 = 1_000_000_000;
#[test]
fn a_healthy_disk_with_room_for_the_keepsake_is_not_low() {
assert!(!disk_is_low(40 * GB, 25 * GB));
}
#[test]
fn the_absolute_floor_fires_even_when_the_gallery_is_tiny() {
// All three volumes share one filesystem, so running out doesn't degrade one subsystem —
// Postgres stops being able to write and the event goes down. A 1 GB gallery would clear
// the keepsake test comfortably; the floor is what catches this.
assert!(disk_is_low(5 * GB, GB));
assert!(disk_is_low(LOW_DISK_FLOOR_BYTES - 1, 0));
assert!(!disk_is_low(LOW_DISK_FLOOR_BYTES, 0));
}
#[test]
fn plenty_of_space_is_still_low_when_the_keepsake_would_not_fit() {
// THE case the fixed threshold misses, and the one that matters: 30 GB free is nowhere near
// any floor, but a 30 GB gallery needs room for TWO archives. The host can act on this
// before releasing; after releasing, they cannot.
assert!(disk_is_low(30 * GB, 66 * GB));
}
#[test]
fn the_keepsake_trigger_is_exact_at_the_boundary() {
assert!(!disk_is_low(66 * GB, 66 * GB), "exactly enough is enough");
assert!(disk_is_low(66 * GB - 1, 66 * GB));
}
#[test]
fn an_empty_gallery_needs_nothing_and_only_the_floor_applies() {
assert!(!disk_is_low(11 * GB, 0));
assert!(disk_is_low(9 * GB, 0));
}
}

20
backend/src/handlers/me.rs
View File

@@ -14,7 +14,7 @@ use serde::Serialize;
use crate::auth::middleware::AuthUser;
use crate::error::AppError;
use crate::handlers::upload::compute_storage_quota;
use crate::models::user::User;
use crate::models::user::{User, UserRole};
use crate::services::config;
use crate::state::AppState;
@@ -37,12 +37,26 @@ pub async fn get_quota(
let estimate = compute_storage_quota(&state).await;
// Raw server telemetry (free disk, concurrent uploader count) is staff-only — it
// must never reach a guest, even though the guest upload UI no longer renders it.
// A guest still gets their own `used`/`limit` so enforcement stays transparent to
// the code paths that consume it; only the server-wide fields are zeroed.
let is_staff = matches!(auth.role, UserRole::Host | UserRole::Admin);
Ok(Json(QuotaDto {
enabled: estimate.limit_bytes.is_some(),
used_bytes: user.total_upload_bytes,
limit_bytes: estimate.limit_bytes,
active_uploaders: estimate.active_uploaders,
free_disk_bytes: estimate.free_disk_bytes,
active_uploaders: if is_staff {
estimate.active_uploaders
} else {
0
},
free_disk_bytes: if is_staff {
estimate.free_disk_bytes
} else {
0
},
}))
}

1
backend/src/handlers/mod.rs
View File

@@ -1,5 +1,6 @@
pub mod admin;
pub mod feed;
pub mod health;
pub mod host;
pub mod me;
pub mod public;

28
backend/src/handlers/public.rs
View File

@@ -4,21 +4,41 @@ use axum::Json;
use axum::extract::State;
use serde::Serialize;
use crate::services::config;
use crate::state::AppState;
#[derive(Serialize)]
pub struct PublicEventDto {
pub name: String,
pub slug: String,
/// Whether the comment feature is on (env `COMMENTS_ENABLED`). The frontend hides
/// the whole comment UI when false; exposed here so even the pre-auth shell knows.
pub comments_enabled: bool,
/// Active colour theme. `preset` is an id the frontend maps to a palette (or
/// "custom"); `primary`/`accent` are the `#rrggbb` seeds the ramps derive from.
/// Resolved as DB-config override → env default. Public so the theme applies on
/// the very first (pre-auth) paint without a flash.
pub theme_preset: String,
pub theme_primary: String,
pub theme_accent: String,
}
/// Public event identity, used by the pre-auth join/recover screens so a guest can
/// see *which* event they're joining. Only the display name and slug are exposed —
/// nothing user-scoped — so this is safe without a token. Served straight from the
/// instance config (no DB round-trip needed).
/// Public event identity + presentation config, used by the pre-auth join/recover
/// screens (which event am I joining, what does it look like). Only non-user-scoped
/// fields are exposed, so this is safe without a token. Identity comes straight from
/// instance config; the theme is resolved from the runtime `config` table (admin UI)
/// falling back to the env-seeded default.
pub async fn get_public_event(State(state): State<AppState>) -> Json<PublicEventDto> {
let cache = &state.config_cache;
Json(PublicEventDto {
name: state.config.event_name.clone(),
slug: state.config.event_slug.clone(),
comments_enabled: state.config.comments_enabled,
theme_preset: config::get_str(cache, "theme_preset", &state.config.default_theme_preset)
.await,
theme_primary: config::get_str(cache, "theme_primary", &state.config.default_theme_primary)
.await,
theme_accent: config::get_str(cache, "theme_accent", &state.config.default_theme_accent)
.await,
})
}

41
backend/src/handlers/social.rs
View File

@@ -10,8 +10,40 @@ use crate::error::AppError;
use crate::models::comment::{Comment, CommentDto};
use crate::models::hashtag::{self, Hashtag};
use crate::models::upload::Upload;
use crate::services::config;
use crate::state::AppState;
/// Throttle a social write. Keyed PER USER, like the feed and upload limits and for the same
/// reason: at a venue every guest sits behind one NAT, so an IP key hands the whole party a
/// single bucket and the most active guest starves everyone else.
///
/// These were the only mutating endpoints in the app with no limit at all — the coverage was
/// asymmetric, not deliberately open. The ceiling is set well above anything a real guest
/// produces; this bounds a script, not an enthusiastic double-tapper.
async fn check_social_rate(state: &AppState, user_id: Uuid) -> Result<(), AppError> {
let rate_limits_on = config::get_bool(&state.config_cache, "rate_limits_enabled", true).await;
let social_rate_on = config::get_bool(&state.config_cache, "social_rate_enabled", true).await;
if !(rate_limits_on && social_rate_on) {
return Ok(());
}
let rate_limit = config::get_usize(&state.config_cache, "social_rate_per_min", 120).await;
// ONE bucket across likes, comments and comment deletions. Separate buckets would let a
// caller triple the aggregate write rate just by alternating between them.
state
.rate_limiter
.check_with_retry(
format!("social:{user_id}"),
rate_limit,
std::time::Duration::from_secs(60),
)
.map_err(|retry_after_secs| {
AppError::TooManyRequests(
"Zu viele Aktionen. Bitte warte kurz und versuche es erneut.".into(),
Some(retry_after_secs),
)
})
}
#[derive(Serialize)]
pub struct LikeResponse {
/// The caller's like state *after* this toggle. The client sets `liked_by_me` from
@@ -35,6 +67,7 @@ pub async fn toggle_like(
if user.is_banned {
return Err(AppError::Forbidden("Du bist gesperrt.".into()));
}
check_social_rate(&state, auth.user_id).await?;
// Event-scope: the upload must belong to the caller's event (404 otherwise),
// matching the host handlers' find_by_id_and_event pattern.
@@ -129,12 +162,19 @@ pub async fn add_comment(
Path(upload_id): Path<Uuid>,
Json(body): Json<AddCommentRequest>,
) -> Result<(StatusCode, Json<CommentDto>), AppError> {
// Comments can be disabled instance-wide (env COMMENTS_ENABLED). The frontend hides
// the UI, but gate the API too so a stale client or direct call can't slip one in.
if !state.config.comments_enabled {
return Err(AppError::Forbidden("Kommentare sind deaktiviert.".into()));
}
let user = crate::models::user::User::find_by_id(&state.pool, auth.user_id)
.await?
.ok_or_else(|| AppError::NotFound("Benutzer nicht gefunden.".into()))?;
if user.is_banned {
return Err(AppError::Forbidden("Du bist gesperrt.".into()));
}
check_social_rate(&state, auth.user_id).await?;
// Event-scope: only comment on an upload that belongs to the caller's event.
Upload::find_by_id_and_event(&state.pool, upload_id, auth.event_id)
@@ -210,6 +250,7 @@ pub async fn delete_comment(
if auth.is_banned {
return Err(AppError::Forbidden("Du bist gesperrt.".into()));
}
check_social_rate(&state, auth.user_id).await?;
let comment = Comment::find_by_id(&state.pool, comment_id)
.await?
.ok_or_else(|| AppError::NotFound("Kommentar nicht gefunden.".into()))?;

21
backend/src/handlers/sse.rs
View File

@@ -38,7 +38,26 @@ pub async fn issue_ticket(
State(state): State<AppState>,
auth: AuthUser,
) -> Result<Json<StreamTicketResponse>, AppError> {
let ticket = state.sse_tickets.issue(auth.token_hash);
// The endpoint had no rate limit at all. Authentication is not a bound here: one valid
// session could loop it freely. 60/min is far above a real client (one ticket per SSE
// (re)connect, and reconnects are backed off) while capping a loop.
if let Err(retry_after_secs) = state.rate_limiter.check_with_retry(
format!("sse_ticket:{}", auth.user_id),
60,
Duration::from_secs(60),
) {
return Err(AppError::TooManyRequests(
"Zu viele Verbindungsversuche. Bitte warte kurz.".into(),
Some(retry_after_secs),
));
}
let ticket = state.sse_tickets.issue(auth.token_hash).ok_or_else(|| {
AppError::ServiceUnavailable(
"Server ist gerade ausgelastet. Live-Updates folgen in Kürze.".into(),
Some(30),
)
})?;
let server_time = sqlx::query_scalar("SELECT NOW()")
.fetch_one(&state.pool)
.await?;

29
backend/src/handlers/test_admin.rs
View File

@@ -13,9 +13,10 @@ use crate::auth::middleware::RequireAdmin;
use crate::error::AppError;
use crate::state::AppState;
/// Truncates every event-scoped table, wipes media on disk, and reseeds the
/// `config` table from migration defaults. Requires an admin JWT — even with
/// `EVENTSNAP_TEST_MODE=1` it cannot be hit anonymously.
/// Truncates every event-scoped table, wipes media on disk, and reseeds the `config`
/// table: numeric values from the migration defaults, but every feature toggle forced
/// OFF (production seeds them ON — see the note at the reseed below). Requires an admin
/// JWT — even with `EVENTSNAP_TEST_MODE=1` it cannot be hit anonymously.
pub async fn truncate_all(
State(state): State<AppState>,
RequireAdmin(_auth): RequireAdmin,
@@ -40,15 +41,29 @@ pub async fn truncate_all(
.execute(&state.pool)
.await?;
// Reseed config — mirrors migrations 005 and 009. Kept in sync by hand
// because pulling SQL out of the migration files at runtime is fragile.
// Reseed config. The NUMERIC values mirror migrations 005/015/016/017/019; the BOOLEAN
// toggles deliberately do NOT — migration 009 seeds every one of them `true`
// (production), and this forces them `false` so the suite isn't fighting rate limits
// and quotas it isn't testing.
//
// Be aware of what that costs: this runs as an auto-fixture before EVERY test, so no
// test starts from production's config unless it explicitly turns a toggle back on
// (02-upload/rate-limit, 07-adversarial/ddos, 01-auth/rate-limit-nat, …). That blind
// spot is exactly why an entire class of per-IP limiter bugs went unnoticed: the
// limiters were simply off. When adding a limiter or quota, add a spec that enables it.
//
// Kept in sync by hand because pulling SQL out of the migration files at runtime is
// fragile — if you add a config key in a migration, add it here too.
sqlx::query(
r#"INSERT INTO config (key, value) VALUES
('max_image_size_mb', '20'),
('max_video_size_mb', '500'),
('upload_rate_per_hour', '10'),
('upload_rate_per_hour', '100'),
('feed_rate_per_min', '60'),
('export_rate_per_day', '3'),
('join_ip_rate_per_min', '60'),
('recover_ip_rate_per_min', '30'),
('social_rate_per_min', '120'),
('quota_tolerance', '0.75'),
('estimated_guest_count', '100'),
('compression_concurrency', '2'),
@@ -57,6 +72,8 @@ pub async fn truncate_all(
('feed_rate_enabled', 'false'),
('export_rate_enabled', 'false'),
('join_rate_enabled', 'false'),
('social_rate_enabled', 'false'),
('admin_login_rate_enabled', 'false'),
('quota_enabled', 'false'),
('storage_quota_enabled', 'false'),
('upload_count_quota_enabled', 'false'),

675
backend/src/handlers/upload.rs
View File

@@ -17,6 +17,135 @@ use crate::state::AppState;
const MAX_CAPTION_LENGTH: usize = 2000;
/// Byte ceiling for the caption field, enforced WHILE reading it.
///
/// `Field::text()` buffers the entire field before returning, and this is the one route whose
/// `DefaultBodyLimit` is raised to 576 MiB (main.rs) — so `caption=<576 MiB of text>` allocated
/// 576 MiB of heap per concurrent request inside a 1 GiB container, and the
/// `MAX_CAPTION_LENGTH` check only ran afterwards, on a string that had already been built.
/// 4 bytes per code point is the worst case for UTF-8, so this can never reject a caption the
/// character limit would have accepted.
const MAX_CAPTION_BYTES: usize = MAX_CAPTION_LENGTH * 4;
/// Byte ceiling for the raw hashtag CSV. Generous next to what the tag caps below allow.
const MAX_HASHTAGS_BYTES: usize = 4 * 1024;
/// Hashtags stored per upload. The CSV was never length-checked at all and was split into an
/// unbounded `Vec`, then upserted TAG BY TAG inside the commit transaction — which holds a
/// `FOR SHARE` lock on the event row, so one request could stall every other upload behind
/// tens of thousands of round trips.
const MAX_HASHTAGS_PER_UPLOAD: usize = 30;
/// Characters per stored tag. `extract_hashtags` already self-bounds at 40; this covers the CSV
/// path, which had no bound of its own.
const MAX_HASHTAG_LENGTH: usize = 50;
/// Read a multipart text field, refusing it the moment it exceeds `max_bytes`.
///
/// The point is to fail DURING the read rather than after it — `Field::text()` cannot, because
/// it has already allocated the whole thing by the time it returns.
async fn read_text_field_bounded(
mut field: axum::extract::multipart::Field<'_>,
max_bytes: usize,
) -> Result<String, AppError> {
let mut buf: Vec<u8> = Vec::new();
while let Some(chunk) = field
.chunk()
.await
.map_err(|e| AppError::BadRequest(e.to_string()))?
{
if buf.len() + chunk.len() > max_bytes {
return Err(AppError::BadRequest(
"Eingabe ist zu lang.".to_string(),
));
}
buf.extend_from_slice(&chunk);
}
String::from_utf8(buf).map_err(|_| AppError::BadRequest("Ungültige Zeichenkodierung.".into()))
}
/// Normalise, dedupe and CAP the tags for one upload.
///
/// Extracted as a pure function so the caps are testable without standing up multipart, and
/// shared by the upload and edit paths — which previously disagreed: upload lowercased and
/// stripped `#`, while edit upserted raw strings, so `#Party` via edit and `party` via upload
/// became two different hashtag rows.
///
/// Truncates rather than rejecting. `extract_hashtags` legitimately derives tags from a
/// 2000-character caption, and 400-ing a guest for writing an enthusiastic caption would be a
/// worse outcome than silently keeping the first 30.
fn normalize_tags(caption_tags: Vec<String>, csv: Option<&str>) -> Vec<String> {
let mut tags = caption_tags;
if let Some(csv) = csv {
for tag in csv.split(',') {
let t = tag.trim().trim_start_matches('#').to_lowercase();
if !t.is_empty() {
tags.push(t);
}
}
}
tags.sort();
tags.dedup();
tags.retain(|t| t.chars().count() <= MAX_HASHTAG_LENGTH);
tags.truncate(MAX_HASHTAGS_PER_UPLOAD);
tags
}
/// Owns the bytes an in-flight upload has written to disk, and deletes them unless the
/// request reaches the point where a database row takes ownership.
///
/// Reclaim used to be a dozen explicit `remove_file` calls on the handler's return paths.
/// That covers every way the handler can FINISH, and none of the ways it can simply STOP:
/// when a client disconnects mid-body — a phone leaving wifi, iOS killing a backgrounded
/// PWA, the user hitting back — axum drops the handler future at a `.await` inside
/// `field.chunk()`, and no return path runs at all. The partial file then survives forever:
/// it has no upload row, so `cleanup_deleted_media` (which is row-driven) can never see it,
/// and no sweeper existed for the originals directory. Those bytes are also invisible to the
/// quota while still consuming the free disk that `quota_limit_bytes` divides among guests.
///
/// A drop guard is the only construct that survives cancellation, because dropping the future
/// is exactly what runs it.
struct TempFileGuard {
/// `None` once disarmed — a row now owns these bytes.
path: Option<std::path::PathBuf>,
}
impl TempFileGuard {
fn new(path: std::path::PathBuf) -> Self {
Self { path: Some(path) }
}
/// Follow the bytes to their new location after a rename.
///
/// NOT `disarm`. Between the rename and the commit the file exists under its FINAL name
/// with still no row pointing at it, so that window needs guarding just as much as the
/// `.tmp` did — arguably more, since a leftover final-named original looks legitimate.
fn retarget(&mut self, path: std::path::PathBuf) {
self.path = Some(path);
}
/// Hand ownership to the committed row. Only correct after `tx.commit()` succeeds.
fn disarm(&mut self) {
self.path = None;
}
}
impl Drop for TempFileGuard {
fn drop(&mut self) {
let Some(path) = self.path.take() else {
return;
};
// std::fs, not tokio::fs: `Drop` cannot await, and a runtime-dependent unlink is not
// guaranteed a live runtime here (shutdown drops in-flight tasks).
match std::fs::remove_file(&path) {
Ok(()) => tracing::debug!(path = %path.display(), "reclaimed an abandoned upload"),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
Err(e) => {
tracing::warn!(error = ?e, path = %path.display(), "failed to reclaim an abandoned upload")
}
}
}
}
/// Allowlist of accepted media types, keyed by the MIME that `infer` derives from
/// the file's magic bytes. The detected MIME (not the client-declared one) is what
/// we trust, store, and hand to the compression pipeline — so a text-based payload
@@ -48,7 +177,7 @@ pub async fn upload(
let upload_rate_on = config::get_bool(&state.config_cache, "upload_rate_enabled", true).await;
if rate_limits_on && upload_rate_on {
let upload_rate =
config::get_i64(&state.config_cache, "upload_rate_per_hour", 10).await as usize;
config::get_i64(&state.config_cache, "upload_rate_per_hour", 100).await as usize;
if let Err(retry_after_secs) = state.rate_limiter.check_with_retry(
format!("upload:{}", auth.user_id),
upload_rate,
@@ -107,13 +236,18 @@ pub async fn upload(
.media_path
.join(format!("originals/{event_slug}"));
let temp_abs = originals_dir.join(format!("{upload_id}.tmp"));
// Armed before anything can create the file, so there is no window in which bytes exist
// unowned. From here on, EVERY exit — return, `?`, panic, or the future being dropped
// mid-body by a client disconnect — reclaims them, and the explicit `remove_file` calls
// that used to be sprinkled over the return paths are gone. One owner, one rule.
let mut file_guard = TempFileGuard::new(temp_abs.clone());
let mut streamed: Option<(i64, Vec<u8>)> = None; // (size, head bytes for sniffing)
let mut caption: Option<String> = None;
let mut hashtags_csv: Option<String> = None;
// Wrap the multipart read so any error after the temp file is created still cleans
// it up (a mid-stream parse failure must not leave a stray `.tmp` on disk).
// The multipart read is wrapped so the field loop can use `?` freely; reclaiming the temp
// file on failure is `file_guard`'s job, not this block's.
let parse_result: Result<(), AppError> = async {
while let Some(field) = multipart
.next_field()
@@ -143,20 +277,10 @@ pub async fn upload(
streamed = Some(stream_field_to_file(field, &temp_abs, cap_bytes).await?);
}
"caption" => {
caption = Some(
field
.text()
.await
.map_err(|e| AppError::BadRequest(e.to_string()))?,
);
caption = Some(read_text_field_bounded(field, MAX_CAPTION_BYTES).await?);
}
"hashtags" => {
hashtags_csv = Some(
field
.text()
.await
.map_err(|e| AppError::BadRequest(e.to_string()))?,
);
hashtags_csv = Some(read_text_field_bounded(field, MAX_HASHTAGS_BYTES).await?);
}
_ => {}
}
@@ -165,13 +289,10 @@ pub async fn upload(
}
.await;
if let Err(e) = parse_result {
let _ = tokio::fs::remove_file(&temp_abs).await;
return Err(e);
}
parse_result?;
// From here on the temp file may exist; every validation failure removes it before
// returning so a rejected upload never leaves bytes behind.
// From here on the temp file may exist. Every exit reclaims it via `file_guard` — see
// TempFileGuard for why the explicit per-branch cleanup this replaced was not enough.
let (size, head) = match streamed {
Some(s) => s,
None => return Err(AppError::BadRequest("Keine Datei hochgeladen.".into())),
@@ -183,7 +304,6 @@ pub async fn upload(
if let Some(ref cap) = caption
&& cap.chars().count() > MAX_CAPTION_LENGTH
{
let _ = tokio::fs::remove_file(&temp_abs).await;
return Err(AppError::BadRequest(format!(
"Beschreibung ist zu lang. Maximum: {} Zeichen.",
MAX_CAPTION_LENGTH
@@ -198,7 +318,6 @@ pub async fn upload(
let kind = match infer::get(&head) {
Some(k) => k,
None => {
let _ = tokio::fs::remove_file(&temp_abs).await;
return Err(AppError::BadRequest(
"Dateityp nicht erkannt oder nicht unterstützt.".into(),
));
@@ -211,7 +330,6 @@ pub async fn upload(
{
Some(v) => v,
None => {
let _ = tokio::fs::remove_file(&temp_abs).await;
return Err(AppError::BadRequest(format!(
"Dateityp wird nicht unterstützt: {}.",
kind.mime_type()
@@ -226,13 +344,31 @@ pub async fn upload(
max_image_mb * 1024 * 1024
};
if size > max_bytes {
let _ = tokio::fs::remove_file(&temp_abs).await;
return Err(AppError::BadRequest(format!(
"Datei ist zu groß. Maximum: {} MB.",
max_bytes / (1024 * 1024)
)));
}
// Images only: refuse anything the compression worker could never decode, reading just
// the header. Without this the upload is accepted with a 201 and then silently
// soft-deleted minutes later when the worker gives up — the guest sees the photo
// vanish with, at best, a vague "could not be processed". Rejecting here gives them a
// reason at the door that they can act on, and it uses the SAME budget the worker
// enforces, so admission and processing cannot disagree.
if mime.starts_with("image/") && crate::services::imaging::exceeds_decode_budget(&temp_abs) {
let mp = crate::services::imaging::megapixels(&temp_abs);
tracing::info!(
%mime, megapixels = ?mp,
"rejecting an image that exceeds the decode budget at admission"
);
let detail = mp.map_or(String::new(), |mp| format!(" (ca. {mp:.0} Megapixel)"));
return Err(AppError::BadRequest(format!(
"Bild hat zu viele Bildpunkte{detail} und kann nicht verarbeitet werden. \
Bitte verkleinere es und lade es erneut hoch."
)));
}
// Per-user storage quota — dynamic formula based on available disk space and the
// number of active uploaders. Gated by master + per-area toggles so the admin can
// disable it on trusted instances.
@@ -251,7 +387,6 @@ pub async fn upload(
quota_limit = Some(limit);
let prospective_total = user.total_upload_bytes.saturating_add(size);
if prospective_total > limit {
let _ = tokio::fs::remove_file(&temp_abs).await;
return Err(AppError::QuotaExceeded(
"Du hast dein Upload-Limit für dieses Event erreicht.".into(),
));
@@ -266,22 +401,20 @@ pub async fn upload(
tokio::fs::rename(&temp_abs, &absolute_path)
.await
.map_err(|e| AppError::Internal(e.into()))?;
// THERE MUST BE NO `.await` BETWEEN THE RENAME AND THIS LINE. Both statements resolve on
// the same poll, so the future cannot be dropped between them and the guard is never
// pointing at a path that no longer holds the bytes. If the rename fails the guard still
// owns `temp_abs`, which is why retargeting comes after it rather than before.
file_guard.retarget(absolute_path.clone());
// Process hashtags from caption and explicit CSV
let mut tags: Vec<String> = Vec::new();
if let Some(ref cap) = caption {
tags.extend(hashtag::extract_hashtags(cap));
}
if let Some(ref csv) = hashtags_csv {
for tag in csv.split(',') {
let t = tag.trim().trim_start_matches('#').to_lowercase();
if !t.is_empty() {
tags.push(t);
}
}
}
tags.sort();
tags.dedup();
// Process hashtags from caption and explicit CSV, capped — see `normalize_tags`.
let tags = normalize_tags(
caption
.as_deref()
.map(hashtag::extract_hashtags)
.unwrap_or_default(),
hashtags_csv.as_deref(),
);
// Quota accounting, the upload row, and its hashtag links must be atomic: a
// crash between the bytes increment and the insert would permanently charge
@@ -366,15 +499,10 @@ pub async fn upload(
}
.await;
// The file is already on disk at `absolute_path`. If the transaction failed, no DB
// row will ever reference it, so remove it now rather than orphan bytes on disk.
let upload = match tx_result {
Ok(u) => u,
Err(e) => {
let _ = tokio::fs::remove_file(&absolute_path).await;
return Err(e);
}
};
// The committed row now references these bytes — hand ownership over. Anything other than
// a successful commit leaves the guard armed, so the file is reclaimed on the way out.
let upload = tx_result?;
file_guard.disarm();
// Spawn compression task
state
@@ -429,6 +557,57 @@ pub async fn edit_upload(
return Err(AppError::Forbidden("Nur eigene Uploads bearbeiten.".into()));
}
// This endpoint had no rate limit of any kind, while every other mutating route has one.
let rate_limits_on = config::get_bool(&state.config_cache, "rate_limits_enabled", true).await;
let edit_rate_on = config::get_bool(&state.config_cache, "upload_edit_rate_enabled", true).await;
if rate_limits_on && edit_rate_on {
let edit_rate =
config::get_i64(&state.config_cache, "upload_edit_rate_per_min", 30).await as usize;
if let Err(retry_after_secs) = state.rate_limiter.check_with_retry(
format!("upload_edit:{}", auth.user_id),
edit_rate,
Duration::from_secs(60),
) {
return Err(AppError::TooManyRequests(
"Zu viele Änderungen. Bitte warte kurz.".into(),
Some(retry_after_secs),
));
}
}
// Validate to the same limits as the upload path. This route had none at all, so a caption
// rejected at upload could be set here instead, and the tags went in raw — meaning `#Party`
// via edit and `party` via upload became two different hashtag rows.
if let Some(ref caption) = body.caption
&& caption.chars().count() > MAX_CAPTION_LENGTH
{
return Err(AppError::BadRequest(format!(
"Beschreibung ist zu lang. Maximum: {MAX_CAPTION_LENGTH} Zeichen."
)));
}
let normalized_tags = body
.hashtags
.as_ref()
.map(|tags| normalize_tags(tags.clone(), None));
// A PATCH that changes nothing must not retire the keepsake generation.
//
// `invalidate_and_arm` below ran unconditionally, outside both `if let Some(...)` guards, so
// `PATCH {}` — which any authenticated guest can send in a loop against their own upload —
// bumped export_epoch and armed a fresh pair of full-gallery export workers every time.
// REGEN_DEBOUNCE bounds the rate of that, not the total work, so the keepsake could be kept
// permanently un-downloadable.
//
// Residual, deliberately not fixed: re-sending an IDENTICAL hashtag list still counts as a
// change. Comparing would need another query, and unlike `PATCH {}` it is not a free loop.
let caption_changed = match (&body.caption, &upload.caption) {
(Some(new), existing) => Some(new.as_str()) != existing.as_deref(),
(None, _) => false,
};
if !caption_changed && normalized_tags.is_none() {
return Ok(StatusCode::OK);
}
// Caption update + hashtag wipe-then-relink in one transaction, so a crash
// mid-relink can't leave the upload with its hashtags stripped.
//
@@ -444,7 +623,7 @@ pub async fn edit_upload(
if let Some(ref caption) = body.caption {
Upload::update_caption(&mut *tx, upload_id, Some(caption)).await?;
}
if let Some(ref hashtags) = body.hashtags {
if let Some(ref hashtags) = normalized_tags {
Hashtag::unlink_all_from_upload(&mut *tx, upload_id).await?;
for tag in hashtags {
let h = Hashtag::upsert(&mut *tx, auth.event_id, tag).await?;
@@ -647,12 +826,95 @@ pub async fn compute_storage_quota(state: &AppState) -> QuotaEstimate {
}
}
/// Outcome of parsing a `Range` request header against a known file length.
#[derive(Debug, PartialEq, Eq)]
enum RangeSpec {
/// No `Range` header, or one we deliberately don't honour (multi-range, non-`bytes`
/// unit, malformed). RFC 9110 lets a server ignore a Range it can't process and reply
/// 200 with the full body, which is what every one of these cases does.
Full,
/// A single satisfiable range, resolved to inclusive absolute offsets.
Partial { start: u64, end: u64 },
/// Syntactically valid but starts beyond EOF — must be answered 416, not 200, or a
/// player can loop re-requesting it.
Unsatisfiable,
}
/// Parse a single-range `bytes=` header against `len`.
///
/// Deliberately supports only the three forms a media element actually sends —
/// `bytes=N-`, `bytes=N-M`, `bytes=-S` (suffix) — and treats everything else as `Full`.
/// Multi-range responses need `multipart/byteranges`, which no `<video>` requires.
fn parse_range(header: Option<&str>, len: u64) -> RangeSpec {
let Some(raw) = header else {
return RangeSpec::Full;
};
let Some(spec) = raw.trim().strip_prefix("bytes=") else {
return RangeSpec::Full;
};
// Multi-range → fall back to the whole body rather than lie about the content.
if spec.contains(',') {
return RangeSpec::Full;
}
let Some((from, to)) = spec.split_once('-') else {
return RangeSpec::Full;
};
let (from, to) = (from.trim(), to.trim());
// A zero-length file can satisfy no range at all.
if len == 0 {
return if from.is_empty() && to.is_empty() {
RangeSpec::Full
} else {
RangeSpec::Unsatisfiable
};
}
let (start, end) = if from.is_empty() {
// Suffix form: the last `to` bytes.
let Ok(suffix) = to.parse::<u64>() else {
return RangeSpec::Full;
};
if suffix == 0 {
return RangeSpec::Unsatisfiable;
}
(len.saturating_sub(suffix), len - 1)
} else {
let Ok(start) = from.parse::<u64>() else {
return RangeSpec::Full;
};
let end = if to.is_empty() {
len - 1
} else {
match to.parse::<u64>() {
// An end past EOF is clamped, not rejected (RFC 9110 §14.1.1).
Ok(end) => end.min(len - 1),
Err(_) => return RangeSpec::Full,
}
};
(start, end)
};
if start >= len || start > end {
RangeSpec::Unsatisfiable
} else {
RangeSpec::Partial { start, end }
}
}
/// Stream a media file from disk into an HTTP response with a fixed set of security
/// headers. Every media response (original, preview, thumbnail) goes through here so
/// they consistently carry `X-Content-Type-Options: nosniff` (defense-in-depth against
/// headers. Every media response (original, preview, display, thumbnail) goes through here
/// so they consistently carry `X-Content-Type-Options: nosniff` (defense-in-depth against
/// content-type confusion, even if the edge proxy is bypassed) plus an explicit
/// `Content-Disposition` and `Cache-Control`.
///
/// Honours a single `Range`. This is not an optimisation: iOS Safari opens every `<video>`
/// with a `Range: bytes=0-1` probe and abandons the load unless it gets a `206` with a
/// `Content-Range`. Without this, video is unplayable on the app's primary platform no
/// matter what `src` the element is given. `Accept-Ranges: bytes` is advertised on every
/// response so clients know seeking is available before they ask.
async fn stream_media_file(
req_headers: &axum::http::HeaderMap,
absolute: &std::path::Path,
content_type: String,
disposition: &str,
@@ -660,30 +922,60 @@ async fn stream_media_file(
) -> Result<axum::response::Response, AppError> {
use axum::body::Body;
use axum::http::{Response, StatusCode, header};
use tokio::io::{AsyncReadExt, AsyncSeekExt};
use tokio_util::io::ReaderStream;
if !absolute.exists() {
return Err(AppError::NotFound("Datei nicht gefunden.".into()));
}
let file = tokio::fs::File::open(absolute)
let mut file = tokio::fs::File::open(absolute)
.await
.map_err(|e| AppError::Internal(e.into()))?;
let metadata = file
let len = file
.metadata()
.await
.map_err(|e| AppError::Internal(e.into()))?;
let stream = ReaderStream::new(file);
.map_err(|e| AppError::Internal(e.into()))?
.len();
Response::builder()
.status(StatusCode::OK)
.header(header::CONTENT_TYPE, content_type)
.header(header::CONTENT_DISPOSITION, disposition)
.header(header::CONTENT_LENGTH, metadata.len())
.header(header::CACHE_CONTROL, cache_control)
.header(header::X_CONTENT_TYPE_OPTIONS, "nosniff")
.body(Body::from_stream(stream))
.map_err(|e| AppError::Internal(e.into()))
let range = parse_range(
req_headers.get(header::RANGE).and_then(|v| v.to_str().ok()),
len,
);
let base = |status: StatusCode| {
Response::builder()
.status(status)
.header(header::CONTENT_TYPE, content_type.clone())
.header(header::CONTENT_DISPOSITION, disposition)
.header(header::CACHE_CONTROL, cache_control)
.header(header::X_CONTENT_TYPE_OPTIONS, "nosniff")
.header(header::ACCEPT_RANGES, "bytes")
};
match range {
RangeSpec::Full => base(StatusCode::OK)
.header(header::CONTENT_LENGTH, len)
.body(Body::from_stream(ReaderStream::new(file)))
.map_err(|e| AppError::Internal(e.into())),
RangeSpec::Partial { start, end } => {
file.seek(std::io::SeekFrom::Start(start))
.await
.map_err(|e| AppError::Internal(e.into()))?;
let span = end - start + 1;
base(StatusCode::PARTIAL_CONTENT)
.header(header::CONTENT_LENGTH, span)
.header(header::CONTENT_RANGE, format!("bytes {start}-{end}/{len}"))
.body(Body::from_stream(ReaderStream::new(file.take(span))))
.map_err(|e| AppError::Internal(e.into()))
}
RangeSpec::Unsatisfiable => base(StatusCode::RANGE_NOT_SATISFIABLE)
.header(header::CONTENT_RANGE, format!("bytes */{len}"))
.body(Body::empty())
.map_err(|e| AppError::Internal(e.into())),
}
}
/// Streaming download of the original file behind an upload. Used by:
@@ -700,6 +992,7 @@ async fn stream_media_file(
/// [`get_preview`] / [`get_thumbnail`]).
pub async fn get_original(
State(state): State<AppState>,
headers: axum::http::HeaderMap,
Path(upload_id): Path<Uuid>,
) -> Result<axum::response::Response, AppError> {
let media = Upload::find_visible_media(&state.pool, upload_id)
@@ -711,10 +1004,22 @@ pub async fn get_original(
.file_name()
.and_then(|n| n.to_str())
.unwrap_or("original");
let disposition = format!("attachment; filename=\"{filename}\"");
// `inline`, not `attachment`. This route is the only source of playable video bytes
// (there is no video derivative), and an attachment disposition is hostile to a
// `<video>` element — Safari in particular. It also matches what the UI promises:
// the action is labelled "Original anzeigen", i.e. view, not download.
let disposition = format!("inline; filename=\"{filename}\"");
// Full-res original: force download, never cache at the edge.
stream_media_file(&absolute, media.mime_type, &disposition, "no-store").await
// Full-res original: never cache at the edge, so a takedown revokes access promptly.
// Range requests still work under no-store; the client simply re-fetches each range.
stream_media_file(
&headers,
&absolute,
media.mime_type,
&disposition,
"no-store",
)
.await
}
/// Streaming access to an upload's compressed **preview** image. Gated exactly like
@@ -729,6 +1034,7 @@ pub async fn get_original(
/// `upload-deleted` / `user-hidden` SSE events, so this only bounds the raw-URL edge case.
pub async fn get_preview(
State(state): State<AppState>,
headers: axum::http::HeaderMap,
Path(upload_id): Path<Uuid>,
) -> Result<axum::response::Response, AppError> {
let media = Upload::find_visible_media(&state.pool, upload_id)
@@ -739,6 +1045,33 @@ pub async fn get_preview(
.ok_or_else(|| AppError::NotFound("Vorschau nicht verfügbar.".into()))?;
let absolute = state.config.media_path.join(&rel);
stream_media_file(
&headers,
&absolute,
"image/jpeg".to_string(),
"inline",
"private, max-age=300",
)
.await
}
/// Streaming access to an upload's big-screen **display** derivative (~2048px), used by the
/// diashow. Gated identically to [`get_preview`]. 404s when the derivative doesn't exist yet
/// (still compressing, or an old upload the backfill hasn't reached) — the diashow then falls
/// back to the original.
pub async fn get_display(
State(state): State<AppState>,
headers: axum::http::HeaderMap,
Path(upload_id): Path<Uuid>,
) -> Result<axum::response::Response, AppError> {
let media = Upload::find_visible_media(&state.pool, upload_id)
.await?
.ok_or_else(|| AppError::NotFound("Upload nicht gefunden.".into()))?;
let rel = media
.display_path
.ok_or_else(|| AppError::NotFound("Anzeige nicht verfügbar.".into()))?;
let absolute = state.config.media_path.join(&rel);
stream_media_file(
&headers,
&absolute,
"image/jpeg".to_string(),
"inline",
@@ -751,6 +1084,7 @@ pub async fn get_preview(
/// [`get_preview`].
pub async fn get_thumbnail(
State(state): State<AppState>,
headers: axum::http::HeaderMap,
Path(upload_id): Path<Uuid>,
) -> Result<axum::response::Response, AppError> {
let media = Upload::find_visible_media(&state.pool, upload_id)
@@ -761,6 +1095,7 @@ pub async fn get_thumbnail(
.ok_or_else(|| AppError::NotFound("Thumbnail nicht verfügbar.".into()))?;
let absolute = state.config.media_path.join(&rel);
stream_media_file(
&headers,
&absolute,
"image/jpeg".to_string(),
"inline",
@@ -771,7 +1106,108 @@ pub async fn get_thumbnail(
#[cfg(test)]
mod tests {
use super::quota_limit_bytes;
use super::{RangeSpec, parse_range, quota_limit_bytes};
// `Range` handling exists because iOS Safari probes every `<video>` with
// `Range: bytes=0-1` and abandons the load without a 206. These pin the forms a
// media element actually sends, plus the edges that decide 206 vs 200 vs 416.
#[test]
fn no_range_header_is_a_full_response() {
assert_eq!(parse_range(None, 100), RangeSpec::Full);
}
#[test]
fn open_ended_range_runs_to_eof() {
assert_eq!(
parse_range(Some("bytes=10-"), 100),
RangeSpec::Partial { start: 10, end: 99 }
);
}
#[test]
fn closed_range_is_inclusive_on_both_ends() {
// The iOS probe. Two bytes, 0 and 1 — an exclusive end would return one.
assert_eq!(
parse_range(Some("bytes=0-1"), 100),
RangeSpec::Partial { start: 0, end: 1 }
);
}
#[test]
fn suffix_range_returns_the_last_n_bytes() {
assert_eq!(
parse_range(Some("bytes=-20"), 100),
RangeSpec::Partial { start: 80, end: 99 }
);
}
#[test]
fn suffix_longer_than_the_file_clamps_to_the_whole_file() {
assert_eq!(
parse_range(Some("bytes=-500"), 100),
RangeSpec::Partial { start: 0, end: 99 }
);
}
#[test]
fn end_past_eof_is_clamped_not_rejected() {
// RFC 9110 §14.1.1 — players routinely ask for more than is there.
assert_eq!(
parse_range(Some("bytes=90-999"), 100),
RangeSpec::Partial { start: 90, end: 99 }
);
}
#[test]
fn start_past_eof_is_416_not_a_full_body() {
// Answering 200 here makes a player re-request forever.
assert_eq!(
parse_range(Some("bytes=100-"), 100),
RangeSpec::Unsatisfiable
);
assert_eq!(
parse_range(Some("bytes=200-300"), 100),
RangeSpec::Unsatisfiable
);
}
#[test]
fn inverted_range_is_unsatisfiable() {
assert_eq!(
parse_range(Some("bytes=50-10"), 100),
RangeSpec::Unsatisfiable
);
}
#[test]
fn unsupported_or_malformed_forms_fall_back_to_the_full_body() {
// Ignoring a Range we can't process and sending 200 is explicitly allowed, and
// safer than guessing. Multi-range would need multipart/byteranges, which no
// <video> asks for.
for header in [
"bytes=0-10,20-30", // multi-range
"items=0-10", // non-bytes unit
"bytes=abc-def", // garbage
"bytes=", // empty spec
"nonsense", // no unit at all
] {
assert_eq!(parse_range(Some(header), 100), RangeSpec::Full, "{header}");
}
}
#[test]
fn last_byte_is_reachable() {
assert_eq!(
parse_range(Some("bytes=99-99"), 100),
RangeSpec::Partial { start: 99, end: 99 }
);
}
#[test]
fn empty_file_satisfies_no_range() {
assert_eq!(parse_range(Some("bytes=0-"), 0), RangeSpec::Unsatisfiable);
assert_eq!(parse_range(None, 0), RangeSpec::Full);
}
#[test]
fn divides_free_space_by_uploaders_with_tolerance() {
@@ -801,4 +1237,103 @@ mod tests {
fn full_tolerance_is_identity_for_a_single_uploader() {
assert_eq!(quota_limit_bytes(500, 1.0, 1), 500);
}
/// The guard is the only reclaim mechanism that survives a client disconnect, so its three
/// states have to be exactly right — a wrong `disarm` leaks bytes forever, a wrong `Drop`
/// deletes a committed guest's photo.
mod temp_file_guard {
use super::super::TempFileGuard;
fn scratch(name: &str) -> std::path::PathBuf {
let dir = std::env::temp_dir().join(format!("es-guard-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
let p = dir.join(name);
std::fs::write(&p, b"bytes").unwrap();
p
}
#[test]
fn dropping_an_armed_guard_reclaims_the_file() {
let p = scratch("armed.tmp");
drop(TempFileGuard::new(p.clone()));
assert!(!p.exists(), "an abandoned upload must not survive the request");
}
#[test]
fn a_disarmed_guard_leaves_the_file_alone() {
let p = scratch("committed.jpg");
let mut g = TempFileGuard::new(p.clone());
g.disarm();
drop(g);
assert!(p.exists(), "a committed upload's bytes must never be deleted");
}
#[test]
fn retarget_follows_the_rename_and_forgets_the_old_path() {
let old = scratch("old.tmp");
let new = scratch("new.jpg");
let mut g = TempFileGuard::new(old.clone());
// The rename already moved the bytes; only the new path is at risk now.
std::fs::remove_file(&old).unwrap();
g.retarget(new.clone());
drop(g);
assert!(!new.exists(), "the final-named original is orphaned too until the row commits");
}
#[test]
fn a_guard_whose_file_is_already_gone_is_harmless() {
let p = scratch("vanished.tmp");
std::fs::remove_file(&p).unwrap();
drop(TempFileGuard::new(p)); // must not panic
}
}
/// The CSV hashtag path had no length check at all and was upserted tag-by-tag INSIDE the
/// commit transaction — which holds a FOR SHARE lock on the event row, so one request could
/// stall every other upload behind tens of thousands of round trips.
mod hashtag_caps {
use super::super::{MAX_HASHTAGS_PER_UPLOAD, MAX_HASHTAG_LENGTH, normalize_tags};
#[test]
fn a_huge_csv_is_capped_not_upserted_in_full() {
let csv = (0..10_000)
.map(|i| format!("tag{i}"))
.collect::<Vec<_>>()
.join(",");
let tags = normalize_tags(vec![], Some(&csv));
assert_eq!(tags.len(), MAX_HASHTAGS_PER_UPLOAD);
}
#[test]
fn an_overlong_tag_is_dropped_rather_than_stored() {
let long = "a".repeat(MAX_HASHTAG_LENGTH + 1);
let tags = normalize_tags(vec![], Some(&format!("ok,{long}")));
assert_eq!(tags, vec!["ok"]);
}
#[test]
fn tags_are_normalised_and_deduped_across_both_sources() {
// The upload path lowercased and stripped `#` while the edit path did not, so
// `#Party` and `party` became two different hashtag rows. One helper, one rule.
let tags = normalize_tags(vec!["party".into()], Some("#Party, PARTY ,tanz"));
assert_eq!(tags, vec!["party", "tanz"]);
}
#[test]
fn truncation_keeps_a_stable_prefix_not_an_arbitrary_one() {
// Sorted before truncation, so the same input always yields the same tags —
// otherwise an edit could silently shuffle which 30 survived.
let csv = "zulu,alpha,mike,bravo";
assert_eq!(
normalize_tags(vec![], Some(csv)),
normalize_tags(vec![], Some("bravo,mike,alpha,zulu"))
);
}
#[test]
fn an_empty_or_absent_csv_yields_nothing() {
assert!(normalize_tags(vec![], None).is_empty());
assert!(normalize_tags(vec![], Some(",, ,")).is_empty());
}
}
}

85
backend/src/main.rs
View File

@@ -2,7 +2,6 @@ use anyhow::Result;
use axum::Router;
use axum::extract::DefaultBodyLimit;
use axum::routing::{delete, get, patch, post};
use tower_http::services::ServeDir;
use tower_http::trace::TraceLayer;
use tracing_subscriber::{layer::SubscriberExt, util::SubscriberInitExt};
@@ -28,8 +27,15 @@ async fn main() -> Result<()> {
tracing_subscriber::registry()
.with(
// `info`, not `debug`. A stock deploy sets RUST_LOG nowhere (it is absent from
// .env.example and was absent from docker-compose.yml), so this fallback IS the
// production level — and at `debug` the TraceLayer below emits a line per request
// AND per response, into a log file that had no rotation. `tower_http=warn`
// rather than `info` states the intent: those spans are diagnostics, not an
// access log, and a future `DefaultOnResponse::new().level(Level::INFO)` should
// not silently re-enable them.
tracing_subscriber::EnvFilter::try_from_default_env()
.unwrap_or_else(|_| "eventsnap_backend=debug,tower_http=debug".into()),
.unwrap_or_else(|_| "eventsnap_backend=info,tower_http=warn".into()),
)
.with(tracing_subscriber::fmt::layer())
.init();
@@ -45,6 +51,19 @@ async fn main() -> Result<()> {
let state = AppState::new(pool.clone(), config.clone());
// Regenerate image derivatives an older pipeline produced: the big-screen display for
// uploads processed before it existed (v0.17.x), and anything predating the current
// DERIVATIVES_REV (rev 1 applies the EXIF orientation, without which every portrait
// phone photo is stored sideways). Fire-and-forget behind the compression semaphore;
// originals are never touched, so a failure just retries on the next start.
state.compression.backfill_stale_derivatives().await;
// Re-extract poster frames for videos a restart interrupted. `startup_recovery` above
// marks their compression `failed` but nothing re-enqueued them, so `thumbnail_path`
// stayed NULL for the rest of the event. Shares the attempt budget with the image
// backfill, so a clip that genuinely yields no frame stops being retried.
state.compression.backfill_video_posters().await;
// Re-spawn exports for events that were released but whose keepsake never finished
// (crash mid-export). Needs the media/export paths + SSE sender, so it runs here
// rather than inside `startup_recovery`. Fire-and-forget: the workers run in the
@@ -53,6 +72,7 @@ async fn main() -> Result<()> {
pool.clone(),
config.media_path.clone(),
config.export_path.clone(),
config.comments_enabled,
state.sse_tx.clone(),
)
.await;
@@ -63,6 +83,7 @@ async fn main() -> Result<()> {
pool,
state.rate_limiter.clone(),
state.sse_tickets.clone(),
config.media_path.clone(),
);
// Ensure media directories exist
@@ -106,6 +127,10 @@ async fn main() -> Result<()> {
"/api/v1/upload/{id}/preview",
get(handlers::upload::get_preview),
)
.route(
"/api/v1/upload/{id}/display",
get(handlers::upload::get_display),
)
.route(
"/api/v1/upload/{id}/thumbnail",
get(handlers::upload::get_thumbnail),
@@ -219,41 +244,43 @@ async fn main() -> Result<()> {
api
};
// Serve media files from disk
let media_service = ServeDir::new(&config.media_path);
// NOTE: media is deliberately NOT served over HTTP.
//
// Files live under `media_path` so the compression worker and the export job can read
// them off disk, but nothing may pull them straight from `/media/**` — that bypasses
// the visibility checks (soft-delete + ban-hide) that make a host takedown stick.
// Every legitimate fetch goes through `/api/v1/upload/{id}/{original,preview,display,
// thumbnail}`, which filter via `find_visible_media`; those are the only media URLs the
// backend ever emits (see `handlers::feed`).
//
// This used to be a `ServeDir` on `/media` with four `nest_service` blockers on the
// subtrees above it. That was bypassable: axum routes on the RAW path while `ServeDir`
// percent-decodes afterwards, so `/media/%70reviews/{id}.jpg` missed every blocker,
// fell through to the `ServeDir`, and was decoded back to `previews/` on disk — serving
// a taken-down photo to anyone, unauthenticated. Any single escaped byte worked, in all
// four subtrees. Deleting the route removes the vector outright rather than racing the
// decoder; `/media/**` now 404s regardless of encoding.
let router = Router::new()
// Liveness. Stays dependency-free — the compose healthcheck gates Caddy's startup
// on it, so anything that can fail transiently must NOT be in here.
.route("/health", get(|| async { "ok" }))
// Readiness. Touches the pool; for an external monitor, not for the compose gate.
.route("/health/ready", get(handlers::health::ready))
.merge(api)
// Block direct HTTP access to ALL media subtrees. The files live under
// `media_path` (so the compression worker and export can read them off disk) but
// must NOT be pullable straight from `/media/**` — that bypasses the visibility
// checks (soft-delete + ban-hide) in the gated handlers. Every legitimate fetch
// goes through `/api/v1/upload/{id}/{original,preview,thumbnail}`, which filter
// via `find_visible_media`. The more specific nests take precedence over the
// `/media` ServeDir below (which, with all three subtrees blocked, now serves
// nothing — kept as a backstop).
.nest_service(
"/media/originals",
get(|| async { axum::http::StatusCode::NOT_FOUND }),
)
.nest_service(
"/media/previews",
get(|| async { axum::http::StatusCode::NOT_FOUND }),
)
.nest_service(
"/media/thumbnails",
get(|| async { axum::http::StatusCode::NOT_FOUND }),
)
.nest_service("/media", media_service)
.layer(TraceLayer::new_for_http())
.with_state(state);
let listener = tokio::net::TcpListener::bind(("0.0.0.0", config.app_port)).await?;
tracing::info!("listening on {}", listener.local_addr()?);
axum::serve(listener, router)
.with_graceful_shutdown(shutdown_signal())
.await?;
// `into_make_service_with_connect_info` is required by the pre-auth handlers, which
// extract `ConnectInfo<SocketAddr>` to use the peer address as the rate-limit key when
// X-Forwarded-For is absent. Without it those extractors fail at runtime.
axum::serve(
listener,
router.into_make_service_with_connect_info::<std::net::SocketAddr>(),
)
.with_graceful_shutdown(shutdown_signal())
.await?;
Ok(())
}

79
backend/src/models/upload.rs
View File

@@ -46,6 +46,7 @@ pub struct VisibleMedia {
pub original_path: String,
pub preview_path: Option<String>,
pub thumbnail_path: Option<String>,
pub display_path: Option<String>,
pub mime_type: String,
}
@@ -93,7 +94,7 @@ impl Upload {
id: Uuid,
) -> Result<Option<VisibleMedia>, sqlx::Error> {
sqlx::query_as::<_, VisibleMedia>(
"SELECT up.original_path, up.preview_path, up.thumbnail_path, up.mime_type
"SELECT up.original_path, up.preview_path, up.thumbnail_path, up.display_path, up.mime_type
FROM upload up
JOIN \"user\" u ON u.id = up.user_id
WHERE up.id = $1 AND up.deleted_at IS NULL
@@ -134,6 +135,82 @@ impl Upload {
Ok(())
}
pub async fn set_display_path(
pool: &PgPool,
id: Uuid,
display_path: &str,
) -> Result<(), sqlx::Error> {
sqlx::query("UPDATE upload SET display_path = $2 WHERE id = $1")
.bind(id)
.bind(display_path)
.execute(pool)
.await?;
Ok(())
}
/// Stamp which revision of the derivative pipeline produced this row's preview/display,
/// so the startup backfill can find rows generated by an older one exactly once.
///
/// Also clears the attempt counter: success is the only thing that resets it, and folding
/// the reset in here means both the live path and the backfill get it with no extra call
/// site to forget.
pub async fn set_derivatives_rev(pool: &PgPool, id: Uuid, rev: i16) -> Result<(), sqlx::Error> {
sqlx::query(
"UPDATE upload
SET derivatives_rev = $2, derivative_attempts = 0, derivative_last_error = NULL
WHERE id = $1",
)
.bind(id)
.bind(rev)
.execute(pool)
.await?;
Ok(())
}
/// Record that derivative processing is ABOUT to be attempted, returning the new count.
///
/// WRITE-AHEAD ON PURPOSE. The failure this bounds is a cgroup SIGKILL: the process
/// vanishes mid-work, so no `Err` is returned, no error handler runs and no `Drop` fires.
/// A counter incremented after a failure would increment zero times per crash and the
/// boot loop would be unchanged. Counting the ATTEMPT is the only thing that survives the
/// process dying. The cost is that a genuinely transient failure also burns an attempt —
/// acceptable, because the retry budget is per-boot-loop, not per-request, and success
/// resets it to zero.
/// `None` when the row no longer exists (hard-deleted, or an e2e TRUNCATE landed while the
/// task waited on the semaphore) — the caller should abandon quietly rather than treat a
/// missing row as a processing failure.
pub async fn begin_derivative_attempt(
pool: &PgPool,
id: Uuid,
) -> Result<Option<i16>, sqlx::Error> {
sqlx::query_scalar(
"UPDATE upload
SET derivative_attempts = derivative_attempts + 1
WHERE id = $1
RETURNING derivative_attempts",
)
.bind(id)
.fetch_optional(pool)
.await
}
/// Store why the last derivative attempt failed. Diagnostics only — nothing branches on it.
pub async fn record_derivative_failure(
pool: &PgPool,
id: Uuid,
error: &str,
) -> Result<(), sqlx::Error> {
// Bounded: an anyhow chain can be long, and this is written on a failure path that may
// repeat across every row of a bad batch.
let truncated: String = error.chars().take(500).collect();
sqlx::query("UPDATE upload SET derivative_last_error = $2 WHERE id = $1")
.bind(id)
.bind(truncated)
.execute(pool)
.await?;
Ok(())
}
pub async fn set_thumbnail_path(
pool: &PgPool,
id: Uuid,

71
backend/src/models/user.rs
View File

@@ -60,6 +60,54 @@ impl User {
.await
}
/// Create a user with an explicit role, in ONE statement.
///
/// `create` + a separate `UPDATE ... SET role` is not equivalent: a crash or a pool error
/// between the two leaves a GUEST row holding a reserved name, which is exactly the
/// poisoned state that bricked admin login — now self-inflicted, and invisible to a
/// role-based lookup, so the next login would create yet another.
pub async fn create_with_role(
pool: &PgPool,
event_id: Uuid,
display_name: &str,
pin_hash: &str,
role: UserRole,
) -> Result<Self, sqlx::Error> {
sqlx::query_as::<_, Self>(
"INSERT INTO \"user\" (event_id, display_name, recovery_pin_hash, role)
VALUES ($1, $2, $3, $4)
RETURNING *",
)
.bind(event_id)
.bind(display_name)
.bind(pin_hash)
.bind(role)
.fetch_one(pool)
.await
}
/// The event's admin, looked up BY ROLE.
///
/// The name is not the identity and never was. Looking the admin up by `display_name`
/// meant any guest who joined as "Admin" first made the lookup miss, and the fallback
/// `create` then violated the case-insensitive unique index from migration 007 — a
/// permanent 500 on admin login, recoverable only by hand-editing the database.
///
/// `ORDER BY created_at` so a database that somehow acquired two admin rows resolves to a
/// stable one rather than alternating between them.
pub async fn find_admin_for_event(
pool: &PgPool,
event_id: Uuid,
) -> Result<Option<Self>, sqlx::Error> {
sqlx::query_as::<_, Self>(
"SELECT * FROM \"user\" WHERE event_id = $1 AND role = 'admin'
ORDER BY created_at ASC LIMIT 1",
)
.bind(event_id)
.fetch_optional(pool)
.await
}
pub async fn find_by_id(pool: &PgPool, id: Uuid) -> Result<Option<Self>, sqlx::Error> {
sqlx::query_as::<_, Self>("SELECT * FROM \"user\" WHERE id = $1")
.bind(id)
@@ -96,14 +144,31 @@ impl User {
Ok(row.0)
}
/// Window after which a failed-PIN streak is forgotten. Matches the lockout duration, so
/// "wait out the cooldown" and "start clean" are the same interval to a guest.
const PIN_ATTEMPT_DECAY_MINUTES: i64 = 15;
/// Record a wrong PIN and return the CURRENT streak length.
///
/// The counter decays: before this, it only ever cleared on a successful recovery or after
/// a lockout expired, so ordinary typos accumulated across days and a guest could arrive at
/// an event already most of the way to being locked out by mistakes made the night before.
/// Decay is what makes the raised lock threshold safe rather than merely lenient.
pub async fn increment_failed_pin(pool: &PgPool, id: Uuid) -> Result<i16, sqlx::Error> {
let row: (i16,) = sqlx::query_as(
"UPDATE \"user\"
SET failed_pin_attempts = failed_pin_attempts + 1
SET failed_pin_attempts = CASE
WHEN last_failed_pin_at IS NULL
OR last_failed_pin_at < NOW() - ($2 || ' minutes')::interval
THEN 1
ELSE failed_pin_attempts + 1
END,
last_failed_pin_at = NOW()
WHERE id = $1
RETURNING failed_pin_attempts",
)
.bind(id)
.bind(Self::PIN_ATTEMPT_DECAY_MINUTES.to_string())
.fetch_one(pool)
.await?;
Ok(row.0)
@@ -124,7 +189,9 @@ impl User {
pub async fn reset_pin_attempts(pool: &PgPool, id: Uuid) -> Result<(), sqlx::Error> {
sqlx::query(
"UPDATE \"user\" SET failed_pin_attempts = 0, pin_locked_until = NULL WHERE id = $1",
"UPDATE \"user\"
SET failed_pin_attempts = 0, pin_locked_until = NULL, last_failed_pin_at = NULL
WHERE id = $1",
)
.bind(id)
.execute(pool)

775
backend/src/services/compression.rs
View File

@@ -13,6 +13,9 @@ use crate::state::SseEvent;
#[derive(Clone)]
pub struct CompressionWorker {
semaphore: Arc<Semaphore>,
/// Serialises the memory-heavy image jobs — see `HEAVY_IMAGE_BYTES`. Separate from
/// `semaphore` so ordinary photos keep full concurrency.
heavy: Arc<Semaphore>,
pool: PgPool,
media_path: PathBuf,
sse_tx: broadcast::Sender<SseEvent>,
@@ -31,6 +34,7 @@ impl CompressionWorker {
) -> Self {
Self {
semaphore: Arc::new(Semaphore::new(concurrency)),
heavy: Arc::new(Semaphore::new(1)),
pool,
media_path,
sse_tx,
@@ -48,6 +52,31 @@ impl CompressionWorker {
self.generation.fetch_add(1, Ordering::SeqCst);
}
/// How many times `do_process` is attempted before an upload is given up on. The
/// give-up path is user-visible (the photo disappears), so transient infrastructure
/// errors must not reach it.
const MAX_PROCESS_ATTEMPTS: u32 = 3;
/// Revision of the image-derivative pipeline. Bump this whenever a change makes existing
/// previews/displays wrong, so `backfill_stale_derivatives` regenerates them once on the
/// next start. Rev 1 = EXIF orientation is applied.
const DERIVATIVES_REV: i16 = 1;
/// How many times derivative generation may be ATTEMPTED for one upload before it is left
/// alone. Counted write-ahead and reset on success — see `Upload::begin_derivative_attempt`.
///
/// This is what turns a fatal input from an outage into a blemish. The startup backfill
/// runs unconditionally on every boot, so before this bound a row whose processing killed
/// the process was re-selected and re-run forever, and `restart: unless-stopped` made that
/// an infinite loop that also dropped every SSE stream and truncated every in-flight
/// upload on each cycle. Three attempts absorbs genuinely transient infrastructure
/// failures (an ENOSPC spike, a pool blip) without ever becoming unbounded.
const MAX_DERIVATIVE_ATTEMPTS: i16 = 3;
/// Rows regenerated per boot. Bounds both the query and the amount of work a single start
/// can queue; whatever is left is picked up on the next boot.
const BACKFILL_BATCH: i64 = 200;
/// Spawn a background task to process an uploaded file.
pub fn process(&self, upload_id: Uuid, original_path: String, mime_type: String) {
let worker = self.clone();
@@ -60,10 +89,42 @@ impl CompressionWorker {
if worker.generation.load(Ordering::SeqCst) != born_at {
return;
}
match worker
.do_process(upload_id, &original_path, &mime_type)
.await
{
// Retry before giving up. Most failures here are transient and self-clearing —
// an ENOSPC spike while several guests upload at once, a momentary DB-pool
// exhaustion, a panic inside the image codec — and the give-up path is
// user-visible data loss, so it is worth a few seconds to avoid entering it.
//
// But only for failures that CAN clear. An image that exceeds the decode budget,
// is corrupt, or is in an unsupported format fails identically on every attempt,
// so retrying it just burns 2s + 4s of backoff and writes three near-identical
// warnings before reaching the same conclusion. Give up on those immediately.
let mut attempt = 1u32;
let outcome = loop {
match worker
.do_process(upload_id, &original_path, &mime_type)
.await
{
Ok(v) => break Ok(v),
Err(e)
if attempt < Self::MAX_PROCESS_ATTEMPTS
&& !crate::services::imaging::is_permanent_image_error(&e) =>
{
tracing::warn!(
error = ?e, %upload_id, attempt,
"compression attempt failed; retrying"
);
tokio::time::sleep(std::time::Duration::from_secs(2u64.pow(attempt))).await;
attempt += 1;
// The data may have been reset while we slept (e2e TRUNCATE).
if worker.generation.load(Ordering::SeqCst) != born_at {
return;
}
}
Err(e) => break Err(e),
}
};
match outcome {
Ok(_) => {
tracing::info!("compression completed for upload {upload_id}");
let _ = worker.sse_tx.send(SseEvent {
@@ -72,21 +133,31 @@ impl CompressionWorker {
});
}
Err(e) => {
tracing::error!("compression failed for upload {upload_id}: {e:#}");
// Auto-cleanup: a failed transcode would otherwise leave a
// permanently broken feed card, silently charge the uploader's
// quota, and orphan the original on disk. Refund + soft-delete
// (one tx, so v_feed excludes it), remove the orphan file, then
// tell the uploader (upload-error toast) and evict the card
// everywhere (upload-deleted, already handled by the feed).
tracing::error!(
"compression failed for upload {upload_id} after {attempt} attempt(s): {e:#}"
);
// Refund + soft-delete (one tx, so v_feed excludes it) so a failed
// transcode doesn't leave a permanently broken feed card or silently
// charge the uploader's quota. Then tell the uploader (upload-error
// toast) and evict the card everywhere (upload-deleted).
//
// The ORIGINAL IS DELIBERATELY KEPT. This path used to `remove_file` it
// unconditionally, which meant any transient error — a disk-full blip
// while saving a derivative, a pool hiccup, a panic in the image codec —
// irreversibly destroyed the guest's only copy of a photo they can never
// retake. The row is only soft-deleted, so keeping the bytes makes the
// upload fully recoverable; the file is orphaned rather than lost, and
// the path is logged so it can be found. `backfill_stale_derivatives`
// already refuses to destroy data on error for exactly this reason.
let _ = Upload::set_compression_status(&worker.pool, upload_id, "failed").await;
if let Err(del) = Upload::soft_delete(&worker.pool, upload_id).await {
tracing::warn!(error = ?del, %upload_id, "failed to soft-delete after compression failure");
}
let orphan = worker.media_path.join(&original_path);
if let Err(rm) = tokio::fs::remove_file(&orphan).await {
tracing::warn!(error = ?rm, path = %orphan.display(), "failed to remove orphaned original");
}
tracing::warn!(
%upload_id,
path = %worker.media_path.join(&original_path).display(),
"original retained for recovery after compression failure"
);
let _ = worker.sse_tx.send(SseEvent {
event_type: "upload-error".to_string(),
data: serde_json::json!({ "upload_id": upload_id, "error": e.to_string() })
@@ -112,129 +183,623 @@ impl CompressionWorker {
let original = self.media_path.join(original_path);
if mime_type.starts_with("image/") {
let preview_rel = self
.generate_image_preview(upload_id, &original, mime_type)
// Count the attempt BEFORE doing the work — see `begin_derivative_attempt`. If this
// input is the one that kills the container, this write is the only record that
// survives, and it is what stops the boot backfill replaying it forever.
match Upload::begin_derivative_attempt(&self.pool, upload_id).await? {
Some(attempts) if attempts > Self::MAX_DERIVATIVE_ATTEMPTS => {
anyhow::bail!(
"derivative generation gave up after {} attempt(s)",
attempts - 1
);
}
Some(_) => {}
// The row vanished while this task waited on the semaphore. Nothing to do, and
// reporting a failure would broadcast into a stream that no longer has a card.
None => return Ok(()),
}
let (preview_rel, display_rel) = self
.generate_image_derivatives(upload_id, &original, mime_type)
.await?;
Upload::set_preview_path(&self.pool, upload_id, &preview_rel).await?;
tracing::info!("preview generated for upload {upload_id}");
Upload::set_display_path(&self.pool, upload_id, &display_rel).await?;
Upload::set_derivatives_rev(&self.pool, upload_id, Self::DERIVATIVES_REV).await?;
tracing::info!("preview + display generated for upload {upload_id}");
} else if mime_type.starts_with("video/") {
let thumb_rel = self.generate_video_thumbnail(upload_id, &original).await?;
Upload::set_thumbnail_path(&self.pool, upload_id, &thumb_rel).await?;
tracing::info!("thumbnail generated for upload {upload_id}");
// A missing poster must NOT fail the upload. `set_thumbnail_path` is only reached when
// a file really exists, so `thumbnail_path` stays NULL otherwise — which every consumer
// already handles (FeedListCard, VirtualFeed, LightboxModal are all null-safe).
//
// The `?` here used to hide the defect; making the check strict without also making
// this non-fatal would have been far worse than the bug. Every clip of a second or less
// would fail compression, exhaust its retries and be soft-deleted — a cosmetic defect
// turned into data loss, on exactly the mis-tap/Live-Photo clips guests produce most.
match self.generate_video_thumbnail(upload_id, &original).await? {
Some(thumb_rel) => {
Upload::set_thumbnail_path(&self.pool, upload_id, &thumb_rel).await?;
tracing::info!("thumbnail generated for upload {upload_id}");
}
None => {
tracing::warn!(
%upload_id,
"no poster frame could be extracted; the video keeps its own tile"
);
}
}
}
Upload::set_compression_status(&self.pool, upload_id, "done").await?;
Ok(())
}
async fn generate_image_preview(
/// Longest edge of the big-screen "display" derivative used by the diashow. Sized to be
/// sharp on 1080p/4K while staying bounded (a ~2048px JPEG decodes to ~16 MB — trivial
/// for any kiosk, unlike a raw multi-thousand-pixel original).
const DISPLAY_MAX_EDGE: u32 = 2048;
/// Longest edge of the phone-feed "preview" (data-saver default).
const PREVIEW_MAX_EDGE: u32 = 800;
/// Above this pixel count the PNG original is stored as uploaded, unoptimised.
///
/// oxipng's peak memory scales with PIXELS, not file size: it decodes the PNG itself and
/// then evaluates row filters, each trial holding a full-size buffer. That is why a 2.82
/// MiB file could measure 1250 MiB of peak RSS inside a 1 GiB container — smooth,
/// synthetic content compresses to almost nothing on disk while still being 8000x8000.
/// 8 MP covers every real phone photo; beyond it we decline the (lossless, cosmetic)
/// saving rather than risk the OOM kill.
const OXIPNG_MAX_PIXELS: u64 = 8_000_000;
/// Estimated peak heap above which an image job takes the exclusive `heavy` permit.
///
/// `compression_concurrency` (default 2) bounds how many jobs run at once, but says
/// nothing about how much memory each one costs, and the container gets 1 GiB total. A
/// single 8000x8000 original measures ~516 MiB peak even with the decode correctly scoped
/// — two of those overlapping is 1032 MiB and another OOM kill, from nothing more exotic
/// than two guests uploading big photos at the same moment.
///
/// 150 MiB sits far above a normal phone photo (a 12 MP JPEG costs ~50 MiB all-in) so the
/// common path never serialises, and far below the point where two jobs stop fitting.
/// Throughput is unaffected for everything except the rare giant, which is exactly the
/// case that must not run in parallel with another giant.
const HEAVY_IMAGE_BYTES: u64 = 150 * 1024 * 1024;
/// Wall-clock ceiling for one oxipng run.
///
/// Bounds TIME, NOT MEMORY — oxipng checks the deadline between trials, so a single trial
/// still allocates in full. The pixel gate above and the sequential build (see
/// `default-features = false` in Cargo.toml) are what bound memory. Do not treat this
/// constant as the OOM fix.
const OXIPNG_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(20);
/// Decode the image ONCE and emit both derivatives — the 800px `preview` (phone feed)
/// and the 2048px `display` (diashow). Returns `(preview_rel, display_rel)`.
async fn generate_image_derivatives(
&self,
upload_id: Uuid,
original: &Path,
mime_type: &str,
) -> Result<String> {
) -> Result<(String, String)> {
let previews_dir = self.media_path.join("previews");
let displays_dir = self.media_path.join("displays");
tokio::fs::create_dir_all(&previews_dir).await?;
tokio::fs::create_dir_all(&displays_dir).await?;
let preview_filename = format!("{upload_id}.jpg");
let preview_path = previews_dir.join(&preview_filename);
let filename = format!("{upload_id}.jpg");
let preview_path = previews_dir.join(&filename);
let display_path = displays_dir.join(&filename);
let original = original.to_path_buf();
let preview_path_clone = preview_path.clone();
let mime_owned = mime_type.to_string();
// Run blocking image operations in a spawn_blocking task
tokio::task::spawn_blocking(move || -> Result<()> {
// Reject decompression bombs *before* fully decoding: the upload body
// cap bounds the file size on disk, but a small file can still decode to
// enormous dimensions (e.g. a ~1 MB image expanding to 50k×50k px →
// gigabytes), OOM-ing the box during decode/resize. 12000×12000 covers
// any real phone photo; max_alloc hard-caps the decode allocation.
let mut reader = image::ImageReader::open(&original)
.context("failed to open image")?
.with_guessed_format()
.context("failed to read image header")?;
let mut limits = image::Limits::default();
limits.max_image_width = Some(12_000);
limits.max_image_height = Some(12_000);
limits.max_alloc = Some(256 * 1024 * 1024);
reader.limits(limits);
let img = reader.decode().context("failed to decode image")?;
// Resize to max 800px wide, preserving aspect ratio
let preview = img.resize(800, 800, image::imageops::FilterType::Lanczos3);
preview
.save_with_format(&preview_path_clone, image::ImageFormat::Jpeg)
.context("failed to save preview")?;
// If the original is PNG, try lossless compression in-place
if mime_owned == "image/png" {
let opts = oxipng::Options::from_preset(2);
let _ = oxipng::optimize(
&oxipng::InFile::Path(original),
&oxipng::OutFile::Path {
path: None,
preserve_attrs: true,
},
&opts,
// Estimate the peak from the HEADER (no pixels decoded — the same kind of cheap probe
// the upload handler already does via `exceeds_decode_budget`) and, if this job is a
// giant, take the exclusive permit so it cannot overlap another giant. Held for the
// whole blocking section, released on drop including on error.
let estimate =
crate::services::imaging::estimated_processing_peak_bytes(&original, Self::DISPLAY_MAX_EDGE);
let _heavy_permit = match estimate {
Some(bytes) if bytes > Self::HEAVY_IMAGE_BYTES => {
tracing::debug!(
%upload_id,
estimated_mib = bytes / (1024 * 1024),
"waiting for the heavy-image permit"
);
Some(self.heavy.acquire().await)
}
_ => None,
};
Ok(())
// Run blocking image operations in a spawn_blocking task
tokio::task::spawn_blocking(move || {
write_image_derivatives(upload_id, &original, &mime_owned, &preview_path, &display_path)
})
.await??;
Ok(format!("previews/{preview_filename}"))
Ok((
format!("previews/{filename}"),
format!("displays/{filename}"),
))
}
async fn generate_video_thumbnail(&self, upload_id: Uuid, original: &Path) -> Result<String> {
/// Regenerate image derivatives that an older pipeline produced. Fire-and-forget from
/// startup; picks up two cases, both of which leave the ORIGINAL untouched:
///
/// - uploads processed before the `display` derivative existed (preview but no
/// `display_path`), and
/// - uploads whose derivatives predate `DERIVATIVES_REV` — currently rev 1, which applies
/// the EXIF orientation. Everything generated before it is stored sideways for any
/// portrait phone photo.
///
/// Unlike the failure path in `process`, a backfill error is logged and skipped — it must
/// NEVER destroy or soft-delete an upload that already has a working preview.
///
/// Bounded in three ways, all of them load-bearing on a box that restarts itself:
/// `derivative_attempts` stops a fatal row being replayed on every boot, `BACKFILL_BATCH`
/// stops one start queueing unbounded work, and the whole thing runs as ONE task walking
/// the rows sequentially rather than N tasks racing for the same semaphore.
pub async fn backfill_stale_derivatives(&self) {
// `original_path IS NOT NULL` was dead — the column is NOT NULL. What actually needs
// excluding is the blanked path `cleanup_deleted_media` leaves behind.
let rows = sqlx::query_as::<_, (Uuid, String, String)>(
"SELECT id, original_path, mime_type FROM upload
WHERE deleted_at IS NULL AND mime_type LIKE 'image/%'
AND original_path <> ''
AND derivative_attempts < $2
AND (
(display_path IS NULL AND preview_path IS NOT NULL)
OR derivatives_rev < $1
)
ORDER BY created_at DESC
LIMIT $3",
)
.bind(Self::DERIVATIVES_REV)
.bind(Self::MAX_DERIVATIVE_ATTEMPTS)
.bind(Self::BACKFILL_BATCH)
.fetch_all(&self.pool)
.await;
let rows = match rows {
Ok(r) => r,
Err(e) => {
tracing::warn!(error = ?e, "derivative backfill query failed");
return;
}
};
self.report_exhausted_derivatives().await;
if rows.is_empty() {
return;
}
tracing::info!("regenerating derivatives for {} upload(s)", rows.len());
// ONE task for the whole batch. The previous shape spawned a task per row, so a large
// backlog created thousands of live tasks that each held a pool handle and queued on
// the same two semaphore permits, competing with live uploads for the entire boot.
let worker = self.clone();
tokio::spawn(async move {
for (id, original_path, mime_type) in rows {
let _permit = worker.semaphore.acquire().await;
// Write-ahead, exactly as in the live path: if this row is the one that kills
// the process, this increment is the only thing that outlives the SIGKILL.
match Upload::begin_derivative_attempt(&worker.pool, id).await {
Ok(Some(n)) if n > Self::MAX_DERIVATIVE_ATTEMPTS => continue,
Ok(Some(_)) => {}
Ok(None) => continue,
Err(e) => {
tracing::warn!(error = ?e, %id, "could not record a backfill attempt; skipping");
continue;
}
}
let original = worker.media_path.join(&original_path);
match worker
.generate_image_derivatives(id, &original, &mime_type)
.await
{
Ok((preview_rel, display_rel)) => {
let _ = Upload::set_preview_path(&worker.pool, id, &preview_rel).await;
let _ = Upload::set_display_path(&worker.pool, id, &display_rel).await;
// Clears derivative_attempts too, so a row that failed transiently is
// not one boot closer to being abandoned.
let _ =
Upload::set_derivatives_rev(&worker.pool, id, Self::DERIVATIVES_REV)
.await;
tracing::info!("derivatives regenerated for upload {id}");
}
Err(e) => {
// Leave the existing derivatives and the original intact; this row is
// retried on the next start until its attempt budget runs out. The rev
// stays behind, which is the marker that it still needs doing.
tracing::warn!(error = ?e, %id, "derivative backfill failed; leaving as-is");
let _ =
Upload::record_derivative_failure(&worker.pool, id, &format!("{e:#}"))
.await;
}
}
}
});
}
/// Re-extract poster frames for videos that never got one.
///
/// A video interrupted by a restart is stranded: `startup_recovery` flips its
/// `compression_status` from `processing` to `failed` and nothing re-enqueues it, so
/// `thumbnail_path` stays NULL forever while the clip itself plays fine. The feed shows a
/// posterless tile for the rest of the event, and after
/// `FAILED_ORIGINAL_RETENTION_DAYS` the reclaim sweep is entitled to the original.
///
/// Shares `derivative_attempts` with the image backfill on purpose. Note the consequence,
/// which is intended rather than a bug to fix later: `extract_poster_frame` returning
/// `Ok(false)` is a NORMAL, permanent outcome for a sub-second clip (Live Photos,
/// mis-taps), and since the counter is write-ahead and only cleared by a real success,
/// those clips stop being re-ffmpeg'd on every boot once the budget is spent.
pub async fn backfill_video_posters(&self) {
let rows = sqlx::query_as::<_, (Uuid, String)>(
"SELECT id, original_path FROM upload
WHERE deleted_at IS NULL AND mime_type LIKE 'video/%'
AND thumbnail_path IS NULL
AND original_path <> ''
AND derivative_attempts < $1
ORDER BY created_at DESC
LIMIT $2",
)
.bind(Self::MAX_DERIVATIVE_ATTEMPTS)
.bind(Self::BACKFILL_BATCH)
.fetch_all(&self.pool)
.await;
let rows = match rows {
Ok(r) => r,
Err(e) => {
tracing::warn!(error = ?e, "video poster backfill query failed");
return;
}
};
if rows.is_empty() {
return;
}
tracing::info!("re-extracting posters for {} video(s)", rows.len());
let worker = self.clone();
tokio::spawn(async move {
for (id, original_path) in rows {
let _permit = worker.semaphore.acquire().await;
match Upload::begin_derivative_attempt(&worker.pool, id).await {
Ok(Some(n)) if n > Self::MAX_DERIVATIVE_ATTEMPTS => continue,
Ok(Some(_)) => {}
Ok(None) => continue,
Err(e) => {
tracing::warn!(error = ?e, %id, "could not record a poster attempt; skipping");
continue;
}
}
let original = worker.media_path.join(&original_path);
match worker.generate_video_thumbnail(id, &original).await {
Ok(Some(thumb_rel)) => {
if Upload::set_thumbnail_path(&worker.pool, id, &thumb_rel)
.await
.is_ok()
{
// Clears the attempt counter: a video that eventually succeeded
// must not carry a budget scar into a future pipeline revision.
let _ = Upload::set_derivatives_rev(
&worker.pool,
id,
Self::DERIVATIVES_REV,
)
.await;
tracing::info!("poster regenerated for upload {id}");
}
}
// No frame at all — normal for a very short clip. The tile stays
// posterless and the attempt is spent, which is what stops the retry.
Ok(None) => {
tracing::debug!(%id, "still no poster frame; leaving the tile as-is");
}
Err(e) => {
tracing::warn!(error = ?e, %id, "poster backfill failed; leaving as-is");
let _ =
Upload::record_derivative_failure(&worker.pool, id, &format!("{e:#}"))
.await;
}
}
}
});
}
/// Say out loud, once per boot, that some uploads have stopped being retried.
///
/// Without this the give-up is invisible: the loop stops (which is the point) but the
/// affected photos keep a stale or missing derivative forever with nothing to notice. The
/// originals are untouched, so this is recoverable once the cause is fixed — reset
/// `derivative_attempts` to 0 and restart.
async fn report_exhausted_derivatives(&self) {
let exhausted: Result<i64, _> = sqlx::query_scalar(
"SELECT count(*) FROM upload
WHERE deleted_at IS NULL AND mime_type LIKE 'image/%'
AND derivative_attempts >= $2
AND (
(display_path IS NULL AND preview_path IS NOT NULL)
OR derivatives_rev < $1
)",
)
.bind(Self::DERIVATIVES_REV)
.bind(Self::MAX_DERIVATIVE_ATTEMPTS)
.fetch_one(&self.pool)
.await;
if let Ok(count) = exhausted
&& count > 0
{
tracing::error!(
count,
"{count} upload(s) exhausted derivative regeneration and will no longer be \
retried; their originals are intact — see upload.derivative_last_error, fix \
the cause, then reset derivative_attempts to 0 and restart"
);
}
}
/// Extract the feed poster for a video. `Ok(None)` when the clip yields no frame — see
/// [`crate::services::video::extract_poster_frame`], which owns the seek order, the timeout and
/// the artifact check that this function used to be missing.
async fn generate_video_thumbnail(
&self,
upload_id: Uuid,
original: &Path,
) -> Result<Option<String>> {
let thumbs_dir = self.media_path.join("thumbnails");
tokio::fs::create_dir_all(&thumbs_dir).await?;
let thumb_filename = format!("{upload_id}.jpg");
let thumb_path = thumbs_dir.join(&thumb_filename);
// Hard timeout — a malformed video can hang `ffmpeg` indefinitely. Without a
// cap, the held compression-worker semaphore permit is never released and the
// pool eventually deadlocks (no further uploads ever processed). 120s is well
// above the time to extract one frame from any sane input.
let mut child = tokio::process::Command::new("ffmpeg")
.args([
"-i",
original.to_str().unwrap_or_default(),
"-vframes",
"1",
"-ss",
"00:00:01",
"-vf",
"scale=800:-1",
"-y",
thumb_path.to_str().unwrap_or_default(),
])
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.kill_on_drop(true)
.spawn()
.context("failed to spawn ffmpeg")?;
let produced =
crate::services::video::extract_poster_frame(original, &thumb_path, 800).await?;
let status =
match tokio::time::timeout(std::time::Duration::from_secs(120), child.wait()).await {
Ok(res) => res.context("ffmpeg wait failed")?,
Err(_) => {
let _ = child.kill().await;
anyhow::bail!("ffmpeg timeout after 120s");
}
};
if !status.success() {
// Best-effort: drain stderr for the log.
let mut stderr = Vec::new();
if let Some(mut handle) = child.stderr.take() {
use tokio::io::AsyncReadExt;
let _ = handle.read_to_end(&mut stderr).await;
}
anyhow::bail!("ffmpeg failed: {}", String::from_utf8_lossy(&stderr));
}
Ok(format!("thumbnails/{thumb_filename}"))
Ok(produced.then(|| format!("thumbnails/{thumb_filename}")))
}
}
/// The blocking half of [`CompressionWorker::generate_image_derivatives`]: decode once, write
/// both derivatives, then optionally shrink a PNG original in place.
///
/// A free function rather than an inline closure so its memory behaviour is directly testable —
/// this is the code path that OOM-killed the container, and the fix is a scoping property that a
/// future edit could silently undo.
fn write_image_derivatives(
upload_id: Uuid,
original: &Path,
mime_type: &str,
preview_path: &Path,
display_path: &Path,
) -> Result<()> {
let preview_max = CompressionWorker::PREVIEW_MAX_EDGE;
let display_max = CompressionWorker::DISPLAY_MAX_EDGE;
// THE FULL-SIZE DECODE IS SCOPED TO THIS BLOCK ON PURPOSE, and the block yields the
// DISPLAY derivative rather than the original.
//
// `img` is up to 256 MiB (imaging::decode_limits max_alloc) and `resize` only BORROWS it,
// so it used to stay alive through both resizes AND the oxipng call below — which decodes
// the PNG a second time and holds a full-size buffer per filter trial. That measured
// ~1250 MiB of peak RSS for a 2.8 MiB input, inside a 1 GiB cgroup: the container was
// SIGKILLed, taking every SSE stream and every in-flight upload with it.
//
// A block rather than a bare `drop(img)` because a `drop` call is one careless edit away
// from being removed as redundant-looking — and note the `else` arm MOVES `img` out, which
// is what makes "the block's value is the only survivor" true in both arms.
let (display, width, height) = {
// Decompression-bomb limits + EXIF orientation, both in one place — see
// services::imaging for why neither may be skipped.
let img = crate::services::imaging::decode_oriented(original)?;
let (width, height) = (img.width(), img.height());
// Display: max 2048px for the diashow. Only DOWNSCALE — never upscale a smaller
// original (that adds bytes with no quality gain); re-encode it as JPEG as-is.
let display = if width > display_max || height > display_max {
img.resize(
display_max,
display_max,
image::imageops::FilterType::Lanczos3,
)
} else {
img
};
(display, width, height)
};
display
.save_with_format(display_path, image::ImageFormat::Jpeg)
.context("failed to save display")?;
// Preview: max 800px, derived from the DISPLAY, not from the original.
//
// Both derivatives used to resize the full-size decode independently, so a 8000x8000
// original paid for two full-size Lanczos passes and their intermediates — measured 520
// MiB peak even after the scoping fix above, which two concurrent workers cannot fit in a
// 1 GiB container. Chaining 8000 -> 2048 -> 800 makes the second pass operate on 2048px
// input, and the full-size buffer is already freed by the time it runs. Quality is not the
// trade-off here: a staged Lanczos3 downscale to 800px is visually indistinguishable from
// a single-step one (and is a standard technique for large ratios).
display
.resize(
preview_max,
preview_max,
image::imageops::FilterType::Lanczos3,
)
.save_with_format(preview_path, image::ImageFormat::Jpeg)
.context("failed to save preview")?;
drop(display);
let pixels = u64::from(width) * u64::from(height);
// If the original is PNG, try lossless compression in place — but only when its pixel count
// is inside the budget, and never for longer than OXIPNG_TIMEOUT. This is a best-effort size
// saving: declining it costs disk, while attempting it unbounded cost the whole container.
if mime_type == "image/png" {
if pixels <= CompressionWorker::OXIPNG_MAX_PIXELS {
let mut opts = oxipng::Options::from_preset(2);
opts.timeout = Some(CompressionWorker::OXIPNG_TIMEOUT);
let _ = oxipng::optimize(
&oxipng::InFile::Path(original.to_path_buf()),
&oxipng::OutFile::Path {
path: None,
preserve_attrs: true,
},
&opts,
);
} else {
tracing::info!(
%upload_id, pixels,
"skipping oxipng: above the pixel budget; the original is stored as uploaded"
);
}
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
/// Peak resident set of THIS process, in bytes, from `/proc/self/status`.
fn peak_rss_bytes() -> u64 {
let status = std::fs::read_to_string("/proc/self/status").expect("procfs");
let line = status
.lines()
.find(|l| l.starts_with("VmHWM:"))
.expect("VmHWM");
let kb: u64 = line
.split_whitespace()
.nth(1)
.and_then(|v| v.parse().ok())
.expect("VmHWM value");
kb * 1024
}
/// Reset the kernel's peak-RSS watermark so the measurement covers only what follows.
/// Linux 4.0+; writing "5" to `clear_refs` resets `VmHWM` to the current RSS.
fn reset_peak_rss() {
let _ = std::fs::write("/proc/self/clear_refs", "5");
}
/// The pixel gate has to sit below what the axis limits allow, or it can never fire.
#[test]
fn the_oxipng_gate_is_reachable_within_the_decode_limits() {
const _: () = {
// imaging::decode_limits permits 12_000 x 12_000 = 144 MP. A gate above that would
// never skip anything.
assert!(CompressionWorker::OXIPNG_MAX_PIXELS < 12_000 * 12_000);
// ...and it must stay above a 48 MP camera, so real photos still get optimised.
assert!(CompressionWorker::OXIPNG_MAX_PIXELS >= 8_000_000);
};
}
/// The heavy-image gate has to classify the two cases the way the sizing assumed:
/// an ordinary phone photo must NOT serialise, and the giant must.
#[test]
fn the_heavy_gate_separates_a_phone_photo_from_a_giant() {
let dir = std::env::temp_dir().join(format!("es-heavy-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
// 12 MP, the shape of a default phone capture.
let ordinary = dir.join("ordinary.jpg");
image::RgbImage::new(4032, 3024).save(&ordinary).unwrap();
let ordinary_peak = crate::services::imaging::estimated_processing_peak_bytes(
&ordinary,
CompressionWorker::DISPLAY_MAX_EDGE,
)
.expect("header readable");
assert!(
ordinary_peak <= CompressionWorker::HEAVY_IMAGE_BYTES,
"a 12 MP photo estimated at {} MiB would serialise the common path",
ordinary_peak / 1048576
);
// The 64 MP RGBA case that measured ~516 MiB peak.
let giant = dir.join("giant.png");
image::RgbaImage::new(8000, 8000).save(&giant).unwrap();
let giant_peak = crate::services::imaging::estimated_processing_peak_bytes(
&giant,
CompressionWorker::DISPLAY_MAX_EDGE,
)
.expect("header readable");
assert!(
giant_peak > CompressionWorker::HEAVY_IMAGE_BYTES,
"an 8000x8000 RGBA original estimated at only {} MiB would be allowed to run \
concurrently with another one — 2x its real ~516 MiB peak does not fit in 1 GiB",
giant_peak / 1048576
);
// The estimate must also be in the right ballpark, not merely on the right side of the
// threshold: 244 MiB decode + 262 MiB f32 resize intermediate.
assert!(
(400..700).contains(&(giant_peak / 1048576)),
"estimate {} MiB is far from the measured ~516 MiB peak",
giant_peak / 1048576
);
let _ = std::fs::remove_dir_all(&dir);
}
/// The OOM that took the container down, measured rather than argued.
///
/// An 8000x8000 RGBA PNG passes admission: 256,000,000 bytes is just under the 256 MiB
/// `max_alloc`, and smooth content is a few MB on disk, far under any size cap. The old
/// code kept that ~244 MiB decode alive across an unbounded, multi-threaded oxipng run and
/// peaked at ~1250 MiB — inside a 1 GiB cgroup. Being SIGKILLed there is not a blip: the
/// row was already committed, so the boot backfill replayed the identical workload on every
/// restart.
///
/// `#[ignore]` because it allocates ~250 MiB and takes a few seconds. Run explicitly:
/// cargo test --release oom -- --ignored --nocapture --test-threads=1
/// It must run ALONE — `VmHWM` is per process, so a concurrent test would pollute it.
#[test]
#[ignore = "heavy: allocates ~250 MiB; run with --ignored --test-threads=1"]
fn a_large_png_stays_far_below_the_container_limit() {
const EDGE: u32 = 8_000;
let dir = std::env::temp_dir().join(format!("es-oom-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
let original = dir.join("big.png");
// Smooth gradient: ~244 MiB decoded, a couple of MB on disk. That gap is the whole
// point — file size tells you nothing about what a PNG costs to process.
{
let mut buf = image::RgbaImage::new(EDGE, EDGE);
for (x, y, px) in buf.enumerate_pixels_mut() {
*px = image::Rgba([(x >> 5) as u8, (y >> 5) as u8, ((x + y) >> 6) as u8, 255]);
}
buf.save(&original).unwrap();
}
// Everything above is fixture setup, not the code under test.
reset_peak_rss();
let before = peak_rss_bytes();
write_image_derivatives(
Uuid::new_v4(),
&original,
"image/png",
&dir.join("preview.jpg"),
&dir.join("display.jpg"),
)
.expect("derivatives");
let peak = peak_rss_bytes();
let on_disk = std::fs::metadata(&original).unwrap().len();
eprintln!(
"input {:.2} MiB on disk ({EDGE}x{EDGE}); peak RSS {:.0} MiB (was {:.0} MiB before)",
on_disk as f64 / 1048576.0,
peak as f64 / 1048576.0,
before as f64 / 1048576.0
);
assert!(dir.join("preview.jpg").exists() && dir.join("display.jpg").exists());
// The container gets 1 GiB and runs two of these concurrently. 600 MiB is a generous
// ceiling that the old code (~1250 MiB) could not have met.
assert!(
peak < 600 * 1024 * 1024,
"peak RSS {} MiB — the decode is being held across oxipng again, or the pixel \
gate stopped firing",
peak / 1048576
);
let _ = std::fs::remove_dir_all(&dir);
}
}

11
backend/src/services/disk.rs
View File

@@ -76,6 +76,17 @@ impl Default for DiskCache {
}
}
/// UNCACHED free-space reading for the filesystem backing `path`.
///
/// Deliberately bypasses [`DiskCache`]. The cache exists for the quota poll, where a 15s-stale
/// number is fine because it is only ever advisory. The export preflight is the opposite case: it
/// decides whether to start writing a multi-GB archive, and the sibling export worker running
/// concurrently can move free space by tens of gigabytes well inside the TTL. A stale reading there
/// would authorise exactly the write that fills the disk.
pub fn free_bytes(path: &Path) -> Option<u64> {
read_disk_for_path(path).map(|d| d.free)
}
/// Resolve the filesystem backing `media_path` and read its total/free bytes.
///
/// Snapshots the mount table via `sysinfo`, then delegates the selection to the pure

776
backend/src/services/export.rs
View File

@@ -20,6 +20,29 @@ use crate::state::SseEvent;
static VIEWER_DIR: Dir<'_> = include_dir!("$CARGO_MANIFEST_DIR/static/export-viewer");
// ── Shared visibility filter ─────────────────────────────────────────────────
/// The predicate that decides what lands in a keepsake, as ONE definition.
///
/// Two queries have to agree on it: [`query_uploads`], which selects the rows the archives are
/// built from, and [`estimate_export_bytes`], which sizes them for the disk preflight. They used
/// to state it separately, and the direction of drift matters — an estimate that misses rows the
/// archive writes UNDER-reserves, which is the exact ENOSPC the preflight exists to prevent.
///
/// A `SRC:`-marked copy in the integration tests cannot catch that: drift means production moved
/// and the copy didn't, so both sides of such a test sit still and it keeps passing. Sharing the
/// fragment removes the failure by construction instead, and leaves the test doing what it is
/// actually good at — pinning the behaviour.
///
/// CONTRACT: callers must alias `upload` as `u` and join `"user"` as `usr`, and bind the event id
/// as `$1`.
macro_rules! export_visibility_where {
() => {
"WHERE u.event_id = $1 AND u.deleted_at IS NULL
AND usr.uploads_hidden = FALSE AND usr.is_banned = FALSE"
};
}
// ── DB query rows ────────────────────────────────────────────────────────────
#[derive(sqlx::FromRow)]
@@ -53,6 +76,9 @@ struct ViewerData {
struct ViewerEvent {
name: String,
exported_at: String,
// Mirrors the live COMMENTS_ENABLED flag so the offline keepsake hides all comment
// UI (buttons, counts, sections) when the feature was off for the event.
comments_enabled: bool,
}
#[derive(Serialize)]
@@ -238,6 +264,7 @@ pub async fn recover_exports(
pool: PgPool,
media_path: PathBuf,
export_path: PathBuf,
comments_enabled: bool,
sse_tx: broadcast::Sender<SseEvent>,
) {
let rows = match sqlx::query_as::<_, (Uuid, String, i64)>(
@@ -297,6 +324,7 @@ pub async fn recover_exports(
event_id,
event_name,
epoch,
comments_enabled,
Duration::ZERO,
pool.clone(),
media_path.clone(),
@@ -403,6 +431,7 @@ pub fn spawn_export_jobs(
event_id: Uuid,
event_name: String,
epoch: i64,
comments_enabled: bool,
delay: Duration,
pool: PgPool,
media_path: PathBuf,
@@ -439,6 +468,7 @@ pub fn spawn_export_jobs(
event_id,
epoch,
&event_name2,
comments_enabled,
&pool2,
&media_path2,
&export_path2,
@@ -469,6 +499,15 @@ async fn run_zip_export(
return Ok(());
}
// Reclaim BEFORE measuring: the superseded archive is already unreachable, and the space it
// holds is very often exactly the space this rebuild needs.
prune_superseded_archives(pool, export_path, "Gallery", event_id, epoch).await;
// AFTER the claim, not before. A preflight that bailed before claiming would leave the row
// `pending` with no worker and no error — the spinner-forever state `mark_failed`'s status
// guard was widened to prevent. Failing here goes through the caller's `mark_failed`, so the
// host gets the reason.
ensure_export_space(pool, event_id, export_path).await?;
// On error, mark THIS generation failed — a no-op if we've since been superseded (the
// caller in `spawn_export_jobs` does it, epoch-guarded). Temp artifacts are cleaned up
// here so a failing export can't leak them (which is what fills the disk in the first place).
@@ -541,7 +580,7 @@ async fn run_zip_export_inner(
};
let entry_name = format!("{folder}/{date}_{name_safe}_{}.{ext}", row.id);
let builder = ZipEntryBuilder::new(entry_name.into(), Compression::Stored);
let builder = keepsake_entry(entry_name, Compression::Stored);
// Open BEFORE writing the entry header. A missing source is skipped (the media file
// was deleted, or its processing failed) — but it must be skipped without aborting the
@@ -635,10 +674,12 @@ impl MediaSource {
}
}
#[allow(clippy::too_many_arguments)]
async fn run_html_export(
event_id: Uuid,
epoch: i64,
event_name: &str,
comments_enabled: bool,
pool: &PgPool,
media_path: &Path,
export_path: &Path,
@@ -649,10 +690,16 @@ async fn run_html_export(
return Ok(());
}
// See run_zip_export: reclaim the superseded generation first, then refuse at the door rather
// than ENOSPC mid-write.
prune_superseded_archives(pool, export_path, "Memories", event_id, epoch).await;
ensure_export_space(pool, event_id, export_path).await?;
let res = run_html_export_inner(
epoch,
event_id,
event_name,
comments_enabled,
pool,
media_path,
export_path,
@@ -672,10 +719,12 @@ async fn run_html_export(
abandon_if_superseded("HTML", event_id, epoch, res)
}
#[allow(clippy::too_many_arguments)]
async fn run_html_export_inner(
epoch: i64,
event_id: Uuid,
event_name: &str,
comments_enabled: bool,
pool: &PgPool,
media_path: &Path,
export_path: &Path,
@@ -708,10 +757,11 @@ async fn run_html_export_inner(
let src = media_path.join(&row.original_path);
// Stat ONCE, up front, and skip this upload if the source is gone. The old code probed with
// `exists()` here and then did `metadata(&src).await?` further down — a TOCTOU whose `?`
// aborted the ENTIRE keepsake if the file vanished in between. It genuinely can: the
// compression worker hard-deletes an original when its transcode fails, and it can still be
// running when the gallery is released. A missing source must degrade one entry, never the
// whole archive (which, once released, the host cannot rebuild without reopening uploads).
// aborted the ENTIRE keepsake if the file vanished in between. It can still happen: the
// compression worker no longer deletes originals on failure, but the hourly sweep reclaims
// them once past the retention window, and an owner or host delete can land mid-export. A
// missing source must degrade one entry, never the whole archive (which, once released, the
// host cannot rebuild without reopening uploads).
let src_meta = match tokio::fs::metadata(&src).await {
Ok(m) => m,
Err(e) => {
@@ -735,38 +785,32 @@ async fn run_html_export_inner(
let full_ext = ext_from_path(&row.original_path);
let full = format!("{id_str}.{full_ext}");
// Video thumbnail via ffmpeg
// Poster frame via the shared helper, which owns the seek order, the 120s timeout
// (this call site had NONE — a hung ffmpeg would strand the export at `running`
// forever) and the artifact check.
let thumb_path = media_tmp.join(&thumb);
let ffmpeg_result = tokio::process::Command::new("ffmpeg")
.args([
"-i",
src.to_str().unwrap_or_default(),
"-vframes",
"1",
"-ss",
"00:00:01",
"-vf",
"scale=400:-1",
"-y",
thumb_path.to_str().unwrap_or_default(),
])
.output()
.await;
match ffmpeg_result {
Ok(output) if output.status.success() => {}
_ => {
tracing::warn!(
"ffmpeg thumbnail failed for upload {}, skipping thumb",
row.id
);
// Missing thumb entry — viewer handles missing thumbs gracefully.
}
let produced =
match crate::services::video::extract_poster_frame(&src, &thumb_path, 400).await {
Ok(produced) => produced,
Err(e) => {
tracing::warn!("poster extraction errored for upload {}: {e:#}", row.id);
false
}
};
if !produced {
tracing::info!(
upload_id = %row.id,
"no poster frame for this video; exporting it without one"
);
}
// Stream the video full-res straight from the original at ZIP time — no
// copy to temp (that used to transiently double disk usage per video).
(thumb, full, MediaSource::Original(src.clone()))
(
produced.then(|| thumb.clone()),
full,
MediaSource::Original(src.clone()),
)
} else {
let thumb = format!("{id_str}_thumb.jpg");
let ext = ext_from_path(&row.original_path);
@@ -778,7 +822,12 @@ async fn run_html_export_inner(
let thumb_path_clone = thumb_path.clone();
let thumb_result = tokio::task::spawn_blocking(move || -> Result<()> {
let img = image::open(&src_clone).context("failed to open image for thumbnail")?;
// `decode_oriented`, not `image::open`: the latter ignores the EXIF
// orientation tag AND applies no decode limits. Using it here is why every
// portrait photo came out sideways in the keepsake's HTML viewer grid — the
// re-encode below drops the tag, so the viewer cannot recover it.
let img = crate::services::imaging::decode_oriented(&src_clone)
.context("failed to open image for thumbnail")?;
let resized = img.resize(400, 400, image::imageops::FilterType::Lanczos3);
resized
.save_with_format(&thumb_path_clone, image::ImageFormat::Jpeg)
@@ -787,9 +836,17 @@ async fn run_html_export_inner(
})
.await?;
if let Err(e) = thumb_result {
tracing::warn!("thumbnail generation failed for upload {}: {e:#}", row.id);
}
// Same dangling-reference hazard as the video branch: a failure here left `thumb`
// pointing at a file the ZIP writer would then skip, so `data.json` advertised an
// entry the archive didn't contain. An undecodable image is rarer than a sub-second
// clip, but the broken tile is identical.
let thumb_ok = match thumb_result {
Ok(()) => true,
Err(e) => {
tracing::warn!("thumbnail generation failed for upload {}: {e:#}", row.id);
false
}
};
// Full variant: compress to temp if >5MB, otherwise stream the original
// as-is (no temp copy). `src_meta` was stat'd once at the top of the loop.
@@ -799,8 +856,12 @@ async fn run_html_export_inner(
let full_path_clone = full_path.clone();
let compress_result = tokio::task::spawn_blocking(move || -> Result<()> {
let img =
image::open(&src_clone).context("failed to open image for compression")?;
// Same reason as the thumbnail above. This branch only runs for originals
// over 5 MB, which is why the viewer's full image looked correct for small
// photos and sideways for large ones — an inconsistency that reads as a
// viewer bug rather than an export one.
let img = crate::services::imaging::decode_oriented(&src_clone)
.context("failed to open image for compression")?;
let resized = img.resize(2000, 2000, image::imageops::FilterType::Lanczos3);
resized
.save_with_format(&full_path_clone, image::ImageFormat::Jpeg)
@@ -823,15 +884,16 @@ async fn run_html_export_inner(
MediaSource::Original(src.clone())
};
(thumb, full, full_source)
(thumb_ok.then_some(thumb), full, full_source)
};
// Register this post's two media entries. Thumbnails always come from temp
// (they're freshly generated); the full variant's source was decided above.
media_manifest.push((
thumb_name.clone(),
MediaSource::Temp(media_tmp.join(&thumb_name)),
));
// Register this post's media entries. The thumbnail is registered ONLY when one was
// actually produced: pushing a manifest entry for a file that doesn't exist made the ZIP
// writer skip it silently while `data.json` still advertised it — the viewer then drew a
// broken image tile for an entry the archive never contained.
if let Some(name) = &thumb_name {
media_manifest.push((name.clone(), MediaSource::Temp(media_tmp.join(name))));
}
media_manifest.push((full_name.clone(), full_source));
// Build comments for this upload
@@ -866,7 +928,15 @@ async fn run_html_export_inner(
} else {
"image".to_string()
},
thumb: format!("media/{thumb_name}"),
// Empty when there is no poster. The viewer already guards on this
// (`{#if post.media.thumb}` → a video tile with a play glyph, or the placeholder
// icon for an image), so telling it the truth is the entire fix — no schema
// change, no viewer rebuild. What was broken was the backend always claiming a
// thumbnail existed.
thumb: thumb_name
.as_ref()
.map(|n| format!("media/{n}"))
.unwrap_or_default(),
full: format!("media/{full_name}"),
},
});
@@ -882,12 +952,17 @@ async fn run_html_export_inner(
event: ViewerEvent {
name: event_name.to_string(),
exported_at: Utc::now().to_rfc3339(),
comments_enabled,
},
posts: viewer_posts,
};
let data_json =
serde_json::to_string_pretty(&viewer_data).context("failed to serialize data.json")?;
// Match the live app's colour theme in the offline keepsake.
let (theme_primary, theme_accent) = resolve_theme_seeds(pool).await;
let theme_css = theme_override_css(&theme_primary, &theme_accent);
let _ = update_progress(pool, event_id, "html", epoch, 72).await;
// 5. Create ZIP (per-generation paths — see run_zip_export)
@@ -899,14 +974,17 @@ async fn run_html_export_inner(
let file = tokio::fs::File::create(&tmp_path).await?;
let mut zip = ZipFileWriter::with_tokio(file);
// Write embedded viewer assets (index.html, _app/*, etc.)
write_dir_to_zip(&VIEWER_DIR, &mut zip).await?;
// Write the embedded single-file viewer, injecting the export data as a
// `window.__EXPORT_DATA__` global into index.html. Guests double-click
// index.html (file://), where a cross-origin fetch() of a sibling file is
// blocked — so the data must be inlined rather than fetched from data.json.
write_viewer_with_data(&VIEWER_DIR, &mut zip, &data_json, theme_css.as_deref()).await?;
let _ = update_progress(pool, event_id, "html", epoch, 75).await;
// Write data.json
{
let builder = ZipEntryBuilder::new("data.json".into(), Compression::Deflate);
let builder = keepsake_entry("data.json".into(), Compression::Deflate);
let mut entry = zip.write_entry_stream(builder).await?;
let mut cursor = AllowStdIo::new(std::io::Cursor::new(data_json.as_bytes()));
fcopy(&mut cursor, &mut entry).await?;
@@ -915,7 +993,7 @@ async fn run_html_export_inner(
// Write README.txt
{
let builder = ZipEntryBuilder::new("README.txt".into(), Compression::Deflate);
let builder = keepsake_entry("README.txt".into(), Compression::Deflate);
let mut entry = zip.write_entry_stream(builder).await?;
let mut cursor = AllowStdIo::new(std::io::Cursor::new(README_TEXT.as_bytes()));
fcopy(&mut cursor, &mut entry).await?;
@@ -933,9 +1011,9 @@ async fn run_html_export_inner(
for (name, source) in &media_manifest {
let path = source.path();
// Open-first: a source that disappeared between the manifest being built and now (the
// compression worker deletes originals on transcode failure) must skip this entry, not
// fail the whole viewer. Opening collapses the check and the use into one operation.
// Open-first: a source that disappeared between the manifest being built and now (a
// delete, or the hourly sweep reclaiming a long-failed original) must skip this entry,
// not fail the whole viewer. Opening collapses the check and the use into one operation.
let src_file = match tokio::fs::File::open(path).await {
Ok(f) => f,
Err(e) => {
@@ -948,7 +1026,7 @@ async fn run_html_export_inner(
};
let entry_name = format!("media/{name}");
let builder = ZipEntryBuilder::new(entry_name.into(), Compression::Stored);
let builder = keepsake_entry(entry_name, Compression::Stored);
let mut zip_entry = zip.write_entry_stream(builder).await?;
let mut f = src_file.compat();
fcopy(&mut f, &mut zip_entry).await?;
@@ -1007,7 +1085,7 @@ async fn run_html_export_inner(
// ── DB helpers ───────────────────────────────────────────────────────────────
async fn query_uploads(pool: &PgPool, event_id: Uuid) -> Result<Vec<ExportUploadRow>> {
Ok(sqlx::query_as::<_, ExportUploadRow>(
Ok(sqlx::query_as::<_, ExportUploadRow>(concat!(
"SELECT u.id, u.original_path, u.mime_type, u.caption,
usr.display_name AS uploader_name,
COUNT(DISTINCT l.user_id) AS like_count,
@@ -1015,11 +1093,12 @@ async fn query_uploads(pool: &PgPool, event_id: Uuid) -> Result<Vec<ExportUpload
FROM upload u
JOIN \"user\" usr ON usr.id = u.user_id
LEFT JOIN \"like\" l ON l.upload_id = u.id
WHERE u.event_id = $1 AND u.deleted_at IS NULL
AND usr.uploads_hidden = FALSE AND usr.is_banned = FALSE
",
export_visibility_where!(),
"
GROUP BY u.id, usr.display_name
ORDER BY u.created_at ASC",
)
))
.bind(event_id)
.fetch_all(pool)
.await?)
@@ -1131,6 +1210,197 @@ fn parse_gen_seq(name: &str, prefix: &str, suffix: &str) -> Option<i64> {
.ok()
}
/// Filenames a live (current-epoch, `done`) job row still points at — OFF LIMITS to every prune,
/// regardless of the epoch encoded in the name.
///
/// A ViewerOnly regeneration carries the finished ZIP forward by re-stamping its row to the new
/// epoch WITHOUT renaming the file, so `Gallery.<event>.<older>.zip` is still the served archive and
/// deleting it by filename-epoch would 404 the download.
async fn protected_files(pool: &PgPool, event_id: Uuid) -> Vec<String> {
sqlx::query_scalar::<_, String>(
"SELECT split_part(j.file_path, '/', -1) FROM export_job j
JOIN event e ON e.id = j.event_id
WHERE j.event_id = $1 AND j.epoch = e.export_epoch
AND j.status = 'done' AND j.file_path IS NOT NULL",
)
.bind(event_id)
.fetch_all(pool)
.await
.unwrap_or_default()
}
/// Reclaim superseded FINAL archives BEFORE this generation starts writing its own.
///
/// Peak disk usage used to be two full generations, because the only prune ran after the new archive
/// was written, renamed and finalised. That ordering reads as durability ("don't delete the good
/// keepsake before the replacement is safe") but it buys nothing: readiness is derived from
/// `job.epoch = event.export_epoch AND status = 'done'`, so the moment `invalidate_and_arm` bumps
/// the epoch the old archive is ALREADY unreachable — `GET /export/zip` 404s whether the file is on
/// disk or not. Keeping it only reserves gigabytes for a download nobody can perform, and for
/// `Affects::Both` (a takedown) it is content someone has explicitly asked to have removed. So a
/// rebuild reclaims first and peaks at one generation.
///
/// Narrower than [`prune_stale_export_files`] on purpose: FINAL archives only. Those are inert — a
/// superseded worker either already renamed its file (and will delete it itself when its guarded
/// `finalize_job` fails) or never will. `.tmp` files and `viewer_tmp_` staging dirs are NOT touched
/// here: a superseded worker can still be streaming into those, and at build START it is far more
/// likely to be alive than at finalize time.
async fn prune_superseded_archives(
pool: &PgPool,
exports_dir: &Path,
prefix: &str,
event_id: Uuid,
keep_seq: i64,
) {
let protected = protected_files(pool, event_id).await;
let final_prefix = format!("{prefix}.{event_id}.");
let mut rd = match tokio::fs::read_dir(exports_dir).await {
Ok(rd) => rd,
Err(_) => return,
};
let mut reclaimed = 0u64;
while let Ok(Some(entry)) = rd.next_entry().await {
let name = entry.file_name();
let name = name.to_string_lossy();
if !is_superseded_archive(&name, &final_prefix, keep_seq, &protected) {
continue;
}
let len = entry.metadata().await.map(|m| m.len()).unwrap_or(0);
if tokio::fs::remove_file(entry.path()).await.is_ok() {
reclaimed += len;
}
}
if reclaimed > 0 {
tracing::info!(
"reclaimed {reclaimed} bytes of superseded {prefix} archives before rebuilding \
event {event_id} @ epoch {keep_seq}"
);
}
}
/// Is `name` a FINAL archive of a strictly-older generation that no live row points at?
///
/// Pure so the two dangerous cases can be pinned without a filesystem: the carried-forward archive
/// (protected despite an older epoch in its name) and the in-flight `.tmp` (never matched at all).
fn is_superseded_archive(
name: &str,
final_prefix: &str,
keep_seq: i64,
protected: &[String],
) -> bool {
if protected.iter().any(|p| p == name) {
return false;
}
parse_gen_seq(name, final_prefix, ".zip").is_some_and(|n| n < keep_seq)
}
/// Bytes the media in this event's keepsake will occupy, as an UPPER BOUND per archive.
///
/// Both archives write their media entries `Compression::Stored`, so an archive is essentially a
/// byte-for-byte second copy of the originals: `Gallery.zip` always, and `Memories.zip` for every
/// video ([`MediaSource::Original`]) and every image at or under 5 MB. Images over 5 MB are
/// downscaled to 2000px first, which only makes this estimate more conservative — the direction we
/// want, since being wrong low means ENOSPC halfway through.
///
/// Shares [`query_uploads`]' visibility filter via [`export_visibility_where`], so hidden/banned
/// uploads can't be counted here but skipped there (or the reverse, which under-reserves).
pub async fn estimate_export_bytes(pool: &PgPool, event_id: Uuid) -> Result<u64> {
let (bytes,): (i64,) = sqlx::query_as(concat!(
"SELECT COALESCE(SUM(u.original_size_bytes), 0)::bigint
FROM upload u
JOIN \"user\" usr ON usr.id = u.user_id
",
export_visibility_where!(),
))
.bind(event_id)
.fetch_one(pool)
.await
.context("estimating the export size")?;
Ok(bytes.max(0) as u64)
}
/// Headroom multiplier over the raw media sum: ZIP central directory, per-entry headers, the
/// embedded viewer, and the HTML export's temp staging.
const EXPORT_SIZE_OVERHEAD_PCT: u64 = 110;
/// Bytes this export must have available, given the raw media sum and how many jobs are competing.
///
/// `armed` is the multiplier that keeps two concurrent workers honest. `spawn_export_jobs` starts
/// the ZIP and the HTML halves at the same instant and both are gallery-sized, so a worker that
/// reserved only for itself would see "it fits", its sibling would independently see the same, and
/// together they would not fit — which is precisely the ENOSPC this preflight exists to prevent.
/// Computed in `u128` and clamped, NOT with `saturating_mul`: saturating first and then dividing by
/// 100 quietly turns an overflow into a number ~100x too small, which is the one direction that
/// matters here — an under-estimate authorises the very write the preflight exists to refuse.
fn required_free_bytes(media_bytes: u64, armed: i64) -> u64 {
let needed = media_bytes as u128 * EXPORT_SIZE_OVERHEAD_PCT as u128 / 100
* armed.max(1).min(i64::from(u32::MAX)) as u128;
needed.min(u64::MAX as u128) as u64
}
/// Free bytes a full keepsake build would need RIGHT NOW, both halves included.
///
/// The same arithmetic the preflight uses, exposed so the host dashboard can warn BEFORE the
/// release rather than reporting a failure after it. The preflight can only ever say "this didn't
/// fit"; at that point the gallery is full, the event is over, and the remedies (ask guests to stop
/// uploading, grow the volume) are all much harder. Hard-codes both halves because that is what a
/// release arms.
pub async fn keepsake_space_required(pool: &PgPool, event_id: Uuid) -> Result<u64> {
Ok(required_free_bytes(
estimate_export_bytes(pool, event_id).await?,
2,
))
}
/// Refuse to start an export that cannot fit, with a reason the host can act on.
///
/// Without this the failure mode is ENOSPC halfway through a multi-GB write, and the wreckage
/// outlives it: the epoch has already moved, so the job row is `failed` at the CURRENT generation
/// and `GET /export/zip` 404s, while the last good archive sits on disk unreferenced. The host's
/// only escape (`POST /host/export/rebuild`) needs the very space that isn't there. Failing at the
/// door instead leaves the disk untouched and puts a number in front of the operator.
///
/// Both halves are spawned concurrently and both are gallery-sized, so a worker must reserve for
/// its live sibling too — otherwise each independently sees "it fits", and together they don't.
async fn ensure_export_space(pool: &PgPool, event_id: Uuid, export_path: &Path) -> Result<()> {
let media_bytes = estimate_export_bytes(pool, event_id).await?;
// Every job armed at any epoch for this event that hasn't finished is competing for this disk.
let (armed,): (i64,) = sqlx::query_as(
"SELECT COUNT(*) FROM export_job
WHERE event_id = $1 AND status IN ('pending', 'running')",
)
.bind(event_id)
.fetch_one(pool)
.await
.context("counting armed export jobs")?;
let needed = required_free_bytes(media_bytes, armed);
// `None` = the mount couldn't be resolved. Fail OPEN, exactly as the upload quota does: refusing
// to build the keepsake because we can't read a number would be a worse failure than trying.
let Some(free) = crate::services::disk::free_bytes(export_path) else {
tracing::warn!("export preflight: disk snapshot unavailable; proceeding without the check");
return Ok(());
};
if free < needed {
let gb = |b: u64| b as f64 / 1_000_000_000.0;
tracing::error!(
needed,
free,
armed,
"export preflight: not enough free space to build the keepsake for event {event_id}"
);
anyhow::bail!(
"Nicht genug Speicherplatz für das Keepsake: benötigt ca. {:.1} GB, frei sind {:.1} GB. \
Bitte Speicher freigeben und das Keepsake anschließend neu erstellen.",
gb(needed),
gb(free)
);
}
Ok(())
}
/// Best-effort removal of stale per-generation export artifacts for one export type. Deletes
/// ONLY strictly-older generations (`n < keep_seq`) — never `keep_seq`'s own current file,
/// and never a NEWER generation that a concurrent re-release may already be producing (that
@@ -1147,20 +1417,7 @@ async fn prune_stale_export_files(
event_id: Uuid,
keep_seq: i64,
) {
// Files still referenced by a live (current-epoch) job row are OFF LIMITS regardless of the
// epoch in their name. A ViewerOnly regeneration carries the finished ZIP forward by re-stamping
// its row to the new epoch WITHOUT renaming the file — so `Gallery.<event>.<older>.zip` is still
// the served archive, and deleting it by filename-epoch would 404 the download.
let protected: Vec<String> = sqlx::query_scalar::<_, String>(
"SELECT split_part(j.file_path, '/', -1) FROM export_job j
JOIN event e ON e.id = j.event_id
WHERE j.event_id = $1 AND j.epoch = e.export_epoch
AND j.status = 'done' AND j.file_path IS NOT NULL",
)
.bind(event_id)
.fetch_all(pool)
.await
.unwrap_or_default();
let protected = protected_files(pool, event_id).await;
// EVERY shape is event-scoped. All events share one exports volume, so a name keyed only by
// generation would let event A's prune delete event B's live keepsake (and let two events
@@ -1301,26 +1558,199 @@ async fn maybe_broadcast_complete(
}
}
/// Recursively write all files from an embedded `include_dir::Dir` into a ZIP.
async fn write_dir_to_zip(
/// Write the embedded viewer into the ZIP, injecting the export data as a
/// `window.__EXPORT_DATA__` global into `index.html`. The keepsake is opened by
/// double-clicking `index.html` (file://), where browsers block a cross-origin
/// `fetch()` of a sibling `data.json` — so the data is inlined into the page.
/// (`data.json` is still written separately for the http-served case.)
/// Permissions stamped on every entry in both archives: `rw-r--r--`.
///
/// `ZipEntryBuilder::new` leaves the external file attribute at zero, and the host compatibility
/// defaults to Unix — so every entry was written with a stored mode of **0000**. Windows Explorer
/// ignores Unix modes and was fine, which is exactly why this survived: on Linux and macOS
/// `unzip` faithfully applies what the archive asks for, and the guest gets a directory of files
/// none of which they can open. `?---------` on every line of `unzip -Z`.
///
/// That is the keepsake — the artifact the whole event exists to produce — arriving unreadable,
/// after distribution, with no server-side symptom at all.
/// `S_IFREG | 0644`. The type bits are included because the mode is written whole into the high
/// half of the external file attribute: without them extractors see a file of type "unknown"
/// (`unzip -Z` renders `?rw-r--r--`), which works but is not what the archive means to say.
const KEEPSAKE_ENTRY_MODE: u16 = 0o100_644;
/// Build a ZIP entry for the keepsake. ALL entries in both archives go through here so the mode
/// can't be forgotten at one of the six call sites.
fn keepsake_entry(name: String, compression: Compression) -> ZipEntryBuilder {
ZipEntryBuilder::new(name.into(), compression).unix_permissions(KEEPSAKE_ENTRY_MODE)
}
/// Escape a JSON payload for inlining inside a `<script>` element.
///
/// See the call site in [`write_viewer_with_data`] for why this is every `<` and not just `</`.
/// Kept separate so the property that matters — no `<` survives, and the value still decodes to
/// the original — can be asserted without building a ZIP.
fn escape_json_for_script(data_json: &str) -> String {
data_json.replace('<', "\\u003c")
}
async fn write_viewer_with_data(
dir: &include_dir::Dir<'_>,
zip: &mut ZipFileWriter<tokio::fs::File>,
data_json: &str,
theme_css: Option<&str>,
) -> Result<()> {
for file in dir.files() {
let path = file.path().to_string_lossy().to_string();
let contents = file.contents();
let builder = ZipEntryBuilder::new(path.into(), Compression::Deflate);
let mut entry = zip.write_entry_stream(builder).await?;
let mut cursor = AllowStdIo::new(std::io::Cursor::new(contents));
fcopy(&mut cursor, &mut entry).await?;
entry.close().await?;
if path == "index.html" {
let html = std::str::from_utf8(file.contents())
.context("export-viewer index.html is not valid UTF-8")?;
// Escape EVERY `<`, not just `</`.
//
// `</` -> `<\/` stops the obvious break-out (`</script><img onerror=…>`) and is inert
// against XSS. It does not stop the caption steering the HTML TOKENIZER. A caption
// containing `<!--<script` with no later `-->` puts the parser into
// script-data-double-escaped state; from there the template's own `</script>` only
// steps back to script-data-escaped instead of closing the element, and the rest of the
// document — including the viewer bundle — is swallowed as script data. Nothing
// executes and nothing leaks; `window.__EXPORT_DATA__` is simply never assigned and the
// keepsake opens blank.
//
// That failure is silent and POST-DISTRIBUTION: the export succeeds, the ZIP is
// well-formed, the job writes `done`, /export/status is green, and the host hands out a
// file that only fails when a guest double-clicks index.html — in every copy, with no
// way to fix it after the fact. Reachable from any guest-authored caption or comment,
// since both are embedded in the viewer.
//
// `<` never appears in JSON structural syntax — only inside string values — so a global
// replace is sound, and `<` is valid in both JSON and a JS string literal. One
// rule covers `</script`, `<!--` and `<script` together, which is the point: the
// previous escape was named for the single case it did handle.
//
// NOTE this is deliberately only for the INLINED copy. `data.json` is written
// separately, in no HTML context, and must stay literal.
let safe = escape_json_for_script(data_json);
// Match the live app's colour theme: inject the same `:root:root{…}` override
// the app builds at runtime so a rose/sage/custom event exports a rose/sage
// keepsake (not the embedded default gold). The CSS is generated purely from
// hex seeds (theme_override_css), so there's nothing to escape.
let mut head_inject = String::new();
if let Some(css) = theme_css {
head_inject.push_str(&format!("<style id=\"es-theme\">{css}</style>"));
}
head_inject.push_str(&format!("<script>window.__EXPORT_DATA__={safe};</script>"));
let injected = match html.find("</head>") {
Some(idx) => format!("{}{}{}", &html[..idx], head_inject, &html[idx..]),
None => format!("{head_inject}{html}"),
};
let builder = keepsake_entry(path, Compression::Deflate);
let mut entry = zip.write_entry_stream(builder).await?;
let mut cursor = AllowStdIo::new(std::io::Cursor::new(injected.as_bytes()));
fcopy(&mut cursor, &mut entry).await?;
entry.close().await?;
} else {
let builder = keepsake_entry(path, Compression::Deflate);
let mut entry = zip.write_entry_stream(builder).await?;
let mut cursor = AllowStdIo::new(std::io::Cursor::new(file.contents()));
fcopy(&mut cursor, &mut entry).await?;
entry.close().await?;
}
}
for sub_dir in dir.dirs() {
Box::pin(write_dir_to_zip(sub_dir, zip)).await?;
Box::pin(write_viewer_with_data(sub_dir, zip, data_json, theme_css)).await?;
}
Ok(())
}
/// Default champagne-gold seed — matches the hand-tuned ramp already embedded in the
/// viewer, so a default-themed event injects no override.
const KEEPSAKE_DEFAULT_SEED: &str = "#8a6a2b";
// stop → color-mix instruction. MIRRORS `LADDER` in frontend/src/lib/theme/palette.ts —
// keep the two in sync so an exported keepsake matches the live app pixel-for-pixel.
const THEME_LADDER: &[(u16, Option<&str>)] = &[
(50, Some("white 90%")),
(100, Some("white 80%")),
(200, Some("white 62%")),
(300, Some("white 42%")),
(400, Some("white 22%")),
(500, Some("white 9%")),
(600, None),
(700, Some("black 15%")),
(800, Some("black 30%")),
(900, Some("black 45%")),
(950, Some("black 63%")),
];
fn theme_stop_value(seed: &str, mix: Option<&str>) -> String {
match mix {
Some(m) => format!("color-mix(in oklab, {seed}, {m})"),
None => seed.to_string(),
}
}
fn theme_ramp(families: &[&str], seed: &str) -> String {
let mut out = String::new();
for (stop, mix) in THEME_LADDER {
let val = theme_stop_value(seed, *mix);
for fam in families {
out.push_str(&format!("--color-{fam}-{stop}:{val};"));
}
}
out
}
fn is_hex_color(s: &str) -> bool {
let b = s.as_bytes();
b.len() == 7 && b[0] == b'#' && b[1..].iter().all(|c| c.is_ascii_hexdigit())
}
/// Build the keepsake's `:root:root{…}` colour override from two seed colours, or None
/// for the default gold (viewer already carries that ramp) or an invalid seed (fall back
/// to the embedded default rather than emit unsafe CSS). MIRRORS `buildPaletteCss` in
/// frontend/src/lib/theme/palette.ts.
fn theme_override_css(primary: &str, accent: &str) -> Option<String> {
if !is_hex_color(primary) || !is_hex_color(accent) {
return None;
}
if primary.eq_ignore_ascii_case(KEEPSAKE_DEFAULT_SEED)
&& accent.eq_ignore_ascii_case(KEEPSAKE_DEFAULT_SEED)
{
return None;
}
let accent500 = theme_stop_value(accent, Some("white 9%"));
let mut css = String::from(":root:root{");
css.push_str(&theme_ramp(&["blue", "primary"], primary));
css.push_str(&theme_ramp(&["purple"], accent));
css.push_str(&format!(
"--color-violet-500:{accent500};--color-violet-600:{accent};"
));
css.push_str(&format!(
"--color-accent-500:{accent500};--color-accent-600:{accent};"
));
css.push('}');
Some(css)
}
/// Read the active theme seeds from the runtime `config` table (set by the admin UI),
/// falling back to the default gold. NOTE: an env-only default (THEME_PRIMARY set but
/// never saved via the admin UI) isn't stored in this table, so such a keepsake would
/// use gold; admin-set themes — the normal path — match the app exactly.
async fn resolve_theme_seeds(pool: &PgPool) -> (String, String) {
async fn read(pool: &PgPool, key: &str) -> String {
sqlx::query_scalar::<_, String>("SELECT value FROM config WHERE key = $1")
.bind(key)
.fetch_optional(pool)
.await
.ok()
.flatten()
.unwrap_or_else(|| KEEPSAKE_DEFAULT_SEED.to_string())
}
(
read(pool, "theme_primary").await,
read(pool, "theme_accent").await,
)
}
fn ext_from_path(path: &str) -> &str {
path.rsplit('.').next().unwrap_or("bin")
}
@@ -1354,3 +1784,187 @@ So geht's:\n\
Alles ist lokal auf deinem Gerät gespeichert.\n\
\n\
Viel Freude mit den Erinnerungen!\n";
// ── Tests ────────────────────────────────────────────────────────────────────
#[cfg(test)]
mod tests {
use super::*;
const EVT: &str = "11111111-1111-1111-1111-111111111111";
fn gallery_prefix() -> String {
format!("Gallery.{EVT}.")
}
#[test]
fn a_strictly_older_archive_is_reclaimed() {
// The whole point: at the start of a rebuild at epoch 5, generation 4's archive is dead
// weight — readiness is derived from `epoch = event.export_epoch`, so it is already
// unreachable — and its bytes are very often exactly the bytes the rebuild needs.
assert!(is_superseded_archive(
&format!("Gallery.{EVT}.4.zip"),
&gallery_prefix(),
5,
&[]
));
}
#[test]
fn our_own_and_newer_generations_are_never_touched() {
// `keep_seq` is OUR generation; a NEWER one belongs to a re-release that has already
// superseded us, and deleting it would let a lagging worker nuke a live keepsake.
for seq in [5, 6] {
assert!(
!is_superseded_archive(
&format!("Gallery.{EVT}.{seq}.zip"),
&gallery_prefix(),
5,
&[]
),
"generation {seq} must survive a prune keeping 5"
);
}
}
#[test]
fn a_carried_forward_archive_survives_despite_an_older_epoch_in_its_name() {
// THE dangerous case. A ViewerOnly regeneration (a moderated comment) re-stamps the
// finished ZIP's row to the new epoch WITHOUT renaming the file, so the SERVED archive
// legitimately carries an older generation number. Pruning it by filename-epoch would 404
// the photo download to change nothing in it. The protected set is what stops that, and
// moving the prune to build-start makes this case reachable far more often.
let carried = format!("Gallery.{EVT}.4.zip");
assert!(!is_superseded_archive(
&carried,
&gallery_prefix(),
5,
std::slice::from_ref(&carried)
));
}
#[test]
fn temps_and_staging_dirs_are_out_of_scope_for_the_early_prune() {
// A superseded worker may still be streaming into these, and at build START it is much
// more likely to be alive than at finalize time. Only inert FINAL archives are reclaimed
// here; `prune_stale_export_files` still handles the rest after we win.
for name in [
format!("Gallery.{EVT}.4.tmp"),
format!("viewer_tmp_{EVT}_4"),
format!("Memories.{EVT}.4.zip"),
] {
assert!(
!is_superseded_archive(&name, &gallery_prefix(), 5, &[]),
"{name} must not be reclaimed by the pre-build prune"
);
}
}
#[test]
fn another_events_archive_is_never_reclaimed() {
// All events share one exports volume, so the prefix carries the event id.
let other = "22222222-2222-2222-2222-222222222222";
assert!(!is_superseded_archive(
&format!("Gallery.{other}.4.zip"),
&gallery_prefix(),
5,
&[]
));
}
#[test]
fn unrelated_files_are_left_alone() {
for name in ["Gallery.zip", "notes.txt", "Gallery..4.zip"] {
assert!(!is_superseded_archive(name, &gallery_prefix(), 5, &[]));
}
}
/// Every `<` is escaped, whatever it is part of.
///
/// PREVENTS the regression to `</` -> `<\\/`, which is named for the one case it handles.
/// `<!--<script` with no later `-->` drives the HTML tokenizer into
/// script-data-double-escaped state, where the template's own `</script>` no longer closes
/// the element — the viewer bundle is swallowed as script data, `__EXPORT_DATA__` is never
/// assigned, and the keepsake opens blank in every copy the host has already handed out.
#[test]
fn no_left_angle_bracket_survives_inlining() {
for payload in [
r#"{"caption":"<!--<script"}"#,
r#"{"caption":"</script><img src=x onerror=alert(1)>"}"#,
r#"{"caption":"<!--"}"#,
r#"{"caption":"<script>"}"#,
r#"{"caption":"a < b"}"#,
] {
let escaped = escape_json_for_script(payload);
assert!(
!escaped.contains('<'),
"a surviving `<` can still steer the tokenizer: {escaped}"
);
}
}
/// The escape must not change what the viewer READS — it is a transport encoding, not a
/// sanitiser. A caption is guest-authored text that has to render back exactly.
#[test]
fn the_payload_still_decodes_to_the_original_value() {
// `<` appears only inside JSON string values, never in structural syntax, so a global
// replace is sound — this is the assertion that says so.
for caption in [
"<!--<script",
"</script><img src=x onerror=alert(1)>",
"a < b und c > d",
"ganz normale Bildunterschrift",
"Herz <3",
] {
let json = serde_json::json!({ "posts": [{ "caption": caption }] }).to_string();
let escaped = escape_json_for_script(&json);
let back: serde_json::Value =
serde_json::from_str(&escaped).expect("the escaped form must still be valid JSON");
assert_eq!(
back["posts"][0]["caption"].as_str(),
Some(caption),
"the caption must survive the round trip unchanged"
);
}
}
/// Nothing else in the document is touched.
#[test]
fn a_payload_with_no_angle_brackets_is_unchanged() {
let json = r#"{"posts":[{"caption":"schönes Foto"}]}"#;
assert_eq!(escape_json_for_script(json), json);
}
#[test]
fn a_lone_armed_job_reserves_for_one_archive() {
// A ViewerOnly regeneration re-arms only the HTML half — reserving for two would refuse
// a rebuild that fits perfectly well.
assert_eq!(required_free_bytes(1_000, 1), 1_100);
}
#[test]
fn two_concurrent_halves_reserve_for_both() {
// The bug this exists to prevent: each worker independently sees "it fits", and together
// they don't. Both halves are gallery-sized, so the reservation must be for the pair.
assert_eq!(required_free_bytes(1_000, 2), 2_200);
}
#[test]
fn a_zero_count_still_reserves_for_one() {
// Defensive: a racing status transition must never yield a zero requirement, which would
// wave through an export of any size onto a full disk.
assert_eq!(required_free_bytes(1_000, 0), 1_100);
}
#[test]
fn an_empty_gallery_needs_nothing() {
assert_eq!(required_free_bytes(0, 2), 0);
}
#[test]
fn a_pathological_size_saturates_instead_of_wrapping() {
// u64 overflow would wrap to a TINY requirement and authorise the exact write we're
// guarding against — the failure mode must be "refuse", never "wrap and allow".
assert_eq!(required_free_bytes(u64::MAX, 2), u64::MAX);
}
}

297
backend/src/services/imaging.rs Normal file
View File

@@ -0,0 +1,297 @@
//! Shared image decoding.
//!
//! Exists so there is exactly ONE way to turn a file on disk into a `DynamicImage` in this
//! codebase. Two properties have to hold everywhere an image is decoded, and both were
//! previously re-derived per call site — which is how they drifted apart:
//!
//! - **EXIF orientation must be applied.** Phones do not rotate sensor data; they record how
//! the camera was held in a tag and store the pixels as shot. `image::open` and
//! `ImageReader::decode` both hand back the raw pixels and ignore that tag, and re-encoding
//! to JPEG writes no EXIF, so the derivative is permanently sideways while the untouched
//! original still renders upright. The compression worker was fixed; the export worker was
//! not, so every portrait photo came out sideways in the keepsake's HTML viewer.
//! - **Decode limits must be set.** The upload body cap bounds the file on disk, but a small
//! file can decode to enormous dimensions (a ~1 MB image expanding to 50k×50k px), OOM-ing
//! the box. `image::open` applies NO limits at all, so the export path was also decoding
//! arbitrary user-supplied images unbounded.
use anyhow::{Context, Result};
use image::{DynamicImage, ImageDecoder};
use std::path::Path;
/// Bounds for any decode of user-supplied image data. The per-axis cap covers any real phone
/// photo; `max_alloc` bounds the decoded buffer — but only because `decode_oriented` reserves
/// against it explicitly, see there.
///
/// Sized against the deployment: the app container is capped at 1 GiB and the compression
/// worker runs `compression_concurrency` decodes at once (default 2), so 256 MiB per decode
/// leaves headroom for the resize buffers and the runtime.
fn decode_limits() -> image::Limits {
let mut limits = image::Limits::default();
limits.max_image_width = Some(12_000);
limits.max_image_height = Some(12_000);
limits.max_alloc = Some(256 * 1024 * 1024);
limits
}
/// True when re-running the exact same work on the exact same bytes cannot possibly
/// succeed, so retrying only burns wall-clock and log noise.
///
/// Deliberately narrow. Only the `ImageError` variants that are a property of the *input*
/// count: the file will not shrink, gain codec support, or un-corrupt itself between
/// attempts. `IoError` is excluded on purpose — an ENOSPC while writing a derivative, or
/// EMFILE under load, is exactly the transient case the retry exists for.
pub fn is_permanent_image_error(err: &anyhow::Error) -> bool {
err.chain().any(|cause| {
matches!(
cause.downcast_ref::<image::ImageError>(),
Some(
image::ImageError::Limits(_)
| image::ImageError::Unsupported(_)
| image::ImageError::Decoding(_)
)
)
})
}
/// Build a decoder for `path` with the budget enforced, WITHOUT reading any pixels.
///
/// Single source of truth for "may this image be decoded at all": both the upload
/// admission check and the compression worker go through here, so they cannot disagree
/// about what is acceptable.
fn decoder_within_budget(path: &Path) -> Result<impl image::ImageDecoder> {
let mut reader = image::ImageReader::open(path)
.context("failed to open image")?
.with_guessed_format()
.context("failed to read image header")?;
let mut limits = decode_limits();
reader.limits(limits.clone());
// We need `into_decoder` rather than `decode()` to read the EXIF orientation tag before
// the pixels are consumed. But the two are NOT equivalent on safety: `decode()` performs
//
// limits.reserve(decoder.total_bytes())?;
//
// between building the decoder and reading the image, and `into_decoder()` skips it (the
// crate's own FIXME concedes `from_decoder` doesn't compensate). Nothing else enforces
// `max_alloc` — the JPEG decoder's `set_limits` only checks support and dimensions — so
// without the line below the budget is inert and the ONLY bound is the per-axis cap. That
// leaves 12000x12000 decodable at 412 MiB, and two concurrent at 824 MiB against a 1 GiB
// container. Re-add it, exactly as `decode()` does.
let mut decoder = reader.into_decoder().context("failed to decode image")?;
limits
.reserve(decoder.total_bytes())
.context("image too large to decode within the memory budget")?;
decoder
.set_limits(limits)
.context("image too large to decode within the memory budget")?;
Ok(decoder)
}
/// Rough peak heap an image will cost to turn into derivatives, read from the HEADER only —
/// no pixels are decoded. `None` when the header can't be read or the image is over budget
/// (the caller is about to fail on it anyway).
///
/// Two terms, and the second is the one that surprises:
///
/// - the decoded buffer, `width * height * channels`; and
/// - the resize intermediate. `image`'s Lanczos3 path accumulates in `f32`, so the buffer
/// between the horizontal and vertical passes is `new_width * old_height * 4 channels * 4
/// bytes` — 16 bytes per pixel-row-slot, not the 4 the output uses. For an 8000x8000
/// original that is 262 MiB on top of a 244 MiB decode, measured. It is bigger than the
/// decode for any tall image, which is why "the decode is bounded by max_alloc" was never
/// the whole story.
///
/// Used to decide whether an image is heavy enough to need exclusive use of the box's memory
/// headroom, NOT to reject anything.
pub fn estimated_processing_peak_bytes(path: &Path, display_edge: u32) -> Option<u64> {
let decoder = decoder_within_budget(path).ok()?;
let (width, height) = decoder.dimensions();
let decoded = decoder.total_bytes();
// Aspect-preserving fit into `display_edge`, matching DynamicImage::resize. No downscale
// means no intermediate at all.
let intermediate = if width > display_edge || height > display_edge {
let ratio = f64::from(display_edge) / f64::from(width.max(height));
let new_width = (f64::from(width) * ratio).round().max(1.0) as u64;
new_width * u64::from(height) * 16
} else {
0
};
Some(decoded.saturating_add(intermediate))
}
/// Megapixels an image would decode to, or `None` if its header can't be read. Used only
/// to put a concrete number in the message the guest sees.
pub fn megapixels(path: &Path) -> Option<f64> {
let reader = image::ImageReader::open(path)
.ok()?
.with_guessed_format()
.ok()?;
let (w, h) = reader.into_dimensions().ok()?;
Some(f64::from(w) * f64::from(h) / 1_000_000.0)
}
/// True when an image cannot be decoded specifically because it would exceed the memory
/// budget — read from the header, no pixels touched.
///
/// Called at upload admission so a guest who sends a 100 MP photo is told at the door, with
/// a reason they can act on, instead of the upload being accepted with a 201 and then
/// silently soft-deleted minutes later when the worker gives up on it.
///
/// Deliberately narrow: ONLY the budget. A corrupt, truncated or unsupported file also
/// fails to build a decoder, but rejecting those here would change a contract the
/// adversarial suite pins on purpose — acceptance follows the magic bytes, and a payload
/// with a valid JPEG header is accepted regardless of what follows it. Those go to the
/// compression worker as before, which handles them gracefully and (since the retry
/// classifier) no longer burns backoff on them.
pub fn exceeds_decode_budget(path: &Path) -> bool {
match decoder_within_budget(path) {
Ok(_) => false,
Err(e) => e.chain().any(|cause| {
matches!(
cause.downcast_ref::<image::ImageError>(),
Some(image::ImageError::Limits(_))
)
}),
}
}
/// Decode an image from disk with decompression-bomb limits applied and its EXIF
/// orientation baked into the pixels.
///
/// Blocking — call inside `spawn_blocking`.
pub fn decode_oriented(path: &Path) -> Result<DynamicImage> {
let mut decoder = decoder_within_budget(path)?;
// Cheap, and it happens BEFORE any pixels are read: an oversized image costs a header
// parse, not an allocation.
let orientation = decoder
.orientation()
.unwrap_or(image::metadata::Orientation::NoTransforms);
let mut img = DynamicImage::from_decoder(decoder).context("failed to decode image")?;
img.apply_orientation(orientation);
Ok(img)
}
#[cfg(test)]
mod tests {
use super::*;
/// Shared with the e2e suite rather than duplicating 568 KiB of binary: the same file
/// drives `02-upload/oversized-image` so both layers assert on one artefact.
const HUGE: &str = concat!(
env!("CARGO_MANIFEST_DIR"),
"/../e2e/fixtures/media/huge-99mp.jpg"
);
#[test]
fn rejects_an_image_that_would_blow_the_allocation_budget() {
// 11000x9000 = 99 MP. Deliberately UNDER the 12000px per-axis cap, so the axis check
// cannot reject it — the allocation budget is the only thing that can, which is
// exactly what makes this a regression test rather than a restatement of the axis cap.
// 283 MiB decoded as RGB8 against a 256 MiB budget, from 568 KiB on disk.
//
// This failed before the guard was restored: `ImageReader::decode` performs
// `limits.reserve(decoder.total_bytes())`, and `into_decoder()` — which we need for
// the EXIF tag — skips it, so `max_alloc` was inert and this decoded happily.
// Map the Ok arm to its dimensions first: on failure `expect_err` Debug-prints the
// value, and Debug on a DynamicImage dumps every pixel — 283 MiB of output.
let err = decode_oriented(Path::new(HUGE))
.map(|img| (img.width(), img.height()))
.expect_err("a 99 MP image must be refused, not allocated");
let msg = format!("{err:#}");
assert!(
msg.to_lowercase().contains("limit") || msg.to_lowercase().contains("memory"),
"expected a limits error, got: {msg}"
);
}
#[test]
fn an_oversized_image_is_a_permanent_failure() {
// The retry loop must not burn 2s + 4s of backoff on this: the file will not shrink
// between attempts, so all three attempts reach the identical conclusion.
let err = decode_oriented(Path::new(HUGE))
.map(|img| (img.width(), img.height()))
.expect_err("fixture must exceed the budget");
assert!(
is_permanent_image_error(&err),
"a Limits error can never succeed on retry: {err:#}"
);
}
#[test]
fn a_plain_io_error_is_not_permanent() {
// The mirror that keeps the classifier honest. ENOSPC while writing a derivative, or
// EMFILE under load, is exactly what the retry exists for — misclassifying those as
// permanent would turn a transient blip back into the data loss round 1 fixed.
let err = decode_oriented(Path::new("/nonexistent/definitely-not-here.jpg"))
.map(|img| (img.width(), img.height()))
.expect_err("a missing file must error");
assert!(
!is_permanent_image_error(&err),
"an IO error must stay retryable: {err:#}"
);
}
#[test]
fn admission_rejects_only_the_over_budget_case() {
// Admission and processing must agree about SIZE — a photo accepted at the door and
// then rejected by the worker for being too big is the failure this pair prevents.
assert!(
exceeds_decode_budget(Path::new(HUGE)),
"admission must reject what the decoder rejects for size"
);
let ordinary = concat!(
env!("CARGO_MANIFEST_DIR"),
"/../e2e/fixtures/media/portrait-exif6.jpg"
);
assert!(
!exceeds_decode_budget(Path::new(ordinary)),
"admission must accept an ordinary photo"
);
}
#[test]
fn admission_does_not_reject_a_merely_undecodable_file() {
// The narrowing that keeps the adversarial contract intact: a payload with valid
// JPEG magic bytes and nothing behind them cannot be decoded, but acceptance follows
// the magic bytes by design (07-adversarial/file-upload-attacks). It is the worker's
// job to fail it, not admission's — admission is only the resource guard.
let dir = std::env::temp_dir().join("eventsnap-imaging-test");
std::fs::create_dir_all(&dir).expect("tmp dir");
let stub = dir.join("magic-only.jpg");
let mut bytes = vec![0u8; 1024];
bytes[..3].copy_from_slice(&[0xFF, 0xD8, 0xFF]);
std::fs::write(&stub, &bytes).expect("write stub");
assert!(
!exceeds_decode_budget(&stub),
"a corrupt file is not an over-budget file"
);
assert!(
decode_oriented(&stub)
.map(|i| (i.width(), i.height()))
.is_err(),
"...but it must still fail in the worker"
);
let _ = std::fs::remove_file(&stub);
}
#[test]
fn still_decodes_an_ordinary_photo_and_applies_orientation() {
// The guard must not have become a blanket refusal. This fixture is 40x20 stored with
// EXIF Orientation=6, so a correct decode returns it rotated to 20x40 portrait.
let path = concat!(
env!("CARGO_MANIFEST_DIR"),
"/../e2e/fixtures/media/portrait-exif6.jpg"
);
let img = decode_oriented(Path::new(path)).expect("an ordinary photo must decode");
assert_eq!(
(img.width(), img.height()),
(20, 40),
"EXIF orientation must still be applied after restoring the guard"
);
}
}

289
backend/src/services/maintenance.rs
View File

@@ -9,10 +9,13 @@
//! users staring at a spinner. Resetting them on startup recovers gracefully.
//!
//! 2. **Periodic tasks** — pruning that should happen "every hour" rather than per
//! request: expired sessions (otherwise the table grows unboundedly), and the
//! request: expired sessions (otherwise the table grows unboundedly), the
//! rate-limiter's in-memory windows (so keys for IPs that left long ago don't
//! accumulate).
//! accumulate), and the media of soft-deleted uploads — both the ones whose compression
//! permanently failed and the ones a guest or host deliberately removed — which are
//! retained for a recovery window and then reclaimed.
use std::path::PathBuf;
use std::time::Duration;
use sqlx::PgPool;
@@ -20,6 +23,42 @@ use sqlx::PgPool;
use crate::services::rate_limiter::RateLimiter;
use crate::services::sse_tickets::SseTicketStore;
/// How long a permanently-failed upload's original is kept on disk before it is
/// reclaimed.
///
/// The compression worker stops deleting originals on failure — a transient error must
/// never destroy the guest's only copy of a photo they can't retake. But the row is
/// soft-deleted and the uploader's quota IS refunded, so without a sweep those bytes are
/// invisible, unowned, and free: a reproducible codec failure lets one guest accumulate
/// orphans at no personal cost, and because `active_uploaders` counts only users with
/// non-deleted uploads, dropping out of that count actually RAISES everyone's per-user
/// ceiling while the disk gets fuller.
///
/// Two weeks is comfortably longer than any single event, so an operator investigating a
/// failed upload still has the file, while the leak stays bounded.
const FAILED_ORIGINAL_RETENTION_DAYS: i64 = 14;
/// How long a DELIBERATELY deleted upload's files are kept before they are reclaimed.
///
/// The same leak, reached by the ordinary path rather than the exceptional one.
/// `soft_delete_in_event` stamps `deleted_at` and refunds `total_upload_bytes`, but nothing ever
/// removed the bytes — so the quota stopped bounding the disk. Upload 500 MB, delete, quota is back
/// to zero, upload another 500 MB: not an attack, just a guest curating their camera roll, which is
/// what people do. The host then sees guests hitting "Du hast dein Upload-Limit erreicht" while the
/// admin widget shows a disk full of files no upload row points at, and the quota message is
/// actively misleading because the space really is gone — just not to anyone the accounting can
/// name.
///
/// Much shorter than the failure window on purpose. Fourteen days outlives the whole event, so a
/// deliberate delete would never reclaim anything while it mattered. A day still gives an operator
/// a recovery window for a mis-tap.
///
/// NOTE what this does NOT do: within the window the bytes are still spent and still unaccounted,
/// so a guest deleting and re-uploading through an eight-hour event can outrun the sweep. Bounding
/// that would mean holding the quota until the file is actually reclaimed rather than refunding at
/// `deleted_at` — a deliberate trade, and the reason the low-disk warning exists.
const DELETED_UPLOAD_RETENTION_HOURS: i64 = 24;
/// Reset rows left in flight by a previous crashed instance. Run once on startup,
/// before the HTTP server starts taking requests, so users never observe the
/// half-state.
@@ -81,13 +120,31 @@ pub async fn startup_recovery(pool: &PgPool) {
}
}
/// How long a file in `originals/` may exist without a database row before it is treated as
/// abandoned.
///
/// This window is the ONLY thing making the sweep safe, because the upload handler renames the
/// temp file into its final path BEFORE committing the row: for a short moment a perfectly
/// healthy upload legitimately looks exactly like an orphan. Six hours is far beyond any live
/// request (a 576 MiB body over a bad venue uplink is minutes, and the request itself is bounded
/// by the reverse proxy) while still reclaiming the leak inside a single event.
///
/// DO NOT SHORTEN THIS to make a test faster — a value below the longest possible in-flight
/// upload deletes photos out from under the request that is committing them.
const ORPHAN_UPLOAD_RETENTION_HOURS: u64 = 6;
/// Spawns a background task that periodically:
/// - deletes session rows whose `expires_at` is more than a day in the past
/// - prunes the in-memory rate-limiter HashMap of empty windows
/// - drops expired SSE tickets (30s TTL but the map keeps the slot until pruned)
///
/// Cadence is 1h — fine for both jobs at our scale.
pub fn spawn_periodic_tasks(pool: PgPool, rate_limiter: RateLimiter, sse_tickets: SseTicketStore) {
pub fn spawn_periodic_tasks(
pool: PgPool,
rate_limiter: RateLimiter,
sse_tickets: SseTicketStore,
media_path: PathBuf,
) {
tokio::spawn(async move {
let mut tick = tokio::time::interval(Duration::from_secs(3600));
// Fire the first tick immediately, then hourly.
@@ -95,12 +152,118 @@ pub fn spawn_periodic_tasks(pool: PgPool, rate_limiter: RateLimiter, sse_tickets
loop {
tick.tick().await;
cleanup_sessions(&pool).await;
cleanup_deleted_media(&pool, &media_path).await;
sweep_orphan_originals(&pool, &media_path).await;
rate_limiter.prune();
sse_tickets.prune();
}
});
}
/// Reclaim the media of soft-deleted uploads once they are past their retention window.
///
/// ONLY ever touches rows with `deleted_at IS NOT NULL`, so it can never reach a live upload. Two
/// classes, two windows, because the two deletes mean different things:
///
/// - a compression failure the guest didn't ask for and may want investigated —
/// [`FAILED_ORIGINAL_RETENTION_DAYS`];
/// - a deliberate removal by the guest or the host — [`DELETED_UPLOAD_RETENTION_HOURS`].
///
/// ALL FOUR paths are reclaimed, not just the original. The previous version cleared
/// `original_path` alone, which was right for its only case (a failed compression produces no
/// derivatives) but wrong the moment the sweep reaches a successfully processed upload: preview,
/// display and thumbnail are each a separate file on disk, none of them counted in
/// `original_size_bytes`, and nothing else ever removed them.
///
/// Every column is cleared in the same pass, which makes the sweep idempotent and stops a later run
/// re-reporting files that are already gone. The ROW is kept: it is the audit trail, it costs a few
/// hundred bytes, and `backfill_stale_derivatives` is guarded on `deleted_at IS NULL` so a nulled
/// `preview_path` can never make it regenerate what was just reclaimed.
async fn cleanup_deleted_media(pool: &PgPool, media_path: &std::path::Path) {
type Row = (
uuid::Uuid,
String,
Option<String>,
Option<String>,
Option<String>,
);
let rows = sqlx::query_as::<_, Row>(
"SELECT id, original_path, preview_path, display_path, thumbnail_path FROM upload
WHERE deleted_at IS NOT NULL
AND CASE WHEN compression_status = 'failed'
THEN deleted_at < NOW() - ($1 || ' days')::interval
ELSE deleted_at < NOW() - ($2 || ' hours')::interval
END
AND (original_path <> '' OR preview_path IS NOT NULL
OR display_path IS NOT NULL OR thumbnail_path IS NOT NULL)",
)
.bind(FAILED_ORIGINAL_RETENTION_DAYS.to_string())
.bind(DELETED_UPLOAD_RETENTION_HOURS.to_string())
.fetch_all(pool)
.await;
let rows = match rows {
Ok(r) => r,
Err(e) => {
tracing::warn!(error = ?e, "deleted-media sweep query failed");
return;
}
};
if rows.is_empty() {
return;
}
let mut reclaimed = 0usize;
for (id, original, preview, display, thumbnail) in rows {
let paths: Vec<String> = std::iter::once(original)
.filter(|p| !p.is_empty())
.chain([preview, display, thumbnail].into_iter().flatten())
.collect();
// All-or-nothing per row: the columns are only cleared once every file for that upload is
// gone. Clearing after a partial success would strand the survivors with nothing pointing
// at them — the same unowned-bytes state this sweep exists to drain.
let mut all_gone = true;
for rel in &paths {
let absolute = media_path.join(rel);
match tokio::fs::remove_file(&absolute).await {
Ok(()) => reclaimed += 1,
// Already gone (manual cleanup, restored backup) — still counts as reclaimed for
// the purpose of clearing the columns, or the row is re-selected every hour forever.
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
Err(e) => {
tracing::warn!(error = ?e, %id, path = %absolute.display(),
"could not reclaim deleted media; leaving the row for the next sweep");
all_gone = false;
}
}
}
if !all_gone {
continue;
}
if let Err(e) = sqlx::query(
"UPDATE upload SET original_path = '', preview_path = NULL,
display_path = NULL, thumbnail_path = NULL
WHERE id = $1",
)
.bind(id)
.execute(pool)
.await
{
tracing::warn!(error = ?e, %id, "reclaimed the files but could not clear the paths");
}
}
if reclaimed > 0 {
tracing::info!(
"reclaimed {reclaimed} file(s) from soft-deleted uploads (deliberate deletes after \
{DELETED_UPLOAD_RETENTION_HOURS}h, compression failures after \
{FAILED_ORIGINAL_RETENTION_DAYS}d)"
);
}
}
async fn cleanup_sessions(pool: &PgPool) {
match sqlx::query("DELETE FROM session WHERE expires_at < NOW() - INTERVAL '1 day'")
.execute(pool)
@@ -113,3 +276,123 @@ async fn cleanup_sessions(pool: &PgPool) {
Err(e) => tracing::warn!("session cleanup failed: {e:#}"),
}
}
/// Reclaim files in `originals/` that no upload row references.
///
/// The backstop behind [`TempFileGuard`](crate::handlers::upload). The guard covers the
/// process that is running; this covers the process that was killed — a SIGKILL, an OOM, or a
/// power cut leaves whatever bytes had been written with no `Drop` to reclaim them, and those
/// files are then permanently invisible: they have no row, so `cleanup_deleted_media` (which is
/// row-driven) can never see them, and they are not counted against any quota while still
/// consuming the free disk that `compute_storage_quota` divides among guests. On a single box
/// where all three volumes share a filesystem, that ends with Postgres unable to write WAL.
///
/// Two classes:
/// - `*.tmp` — an upload that never got as far as being renamed. Always safe past the window.
/// - everything else — a final-named original whose commit never happened.
async fn sweep_orphan_originals(pool: &PgPool, media_path: &std::path::Path) {
let originals = media_path.join("originals");
let cutoff = Duration::from_secs(ORPHAN_UPLOAD_RETENTION_HOURS * 3600);
// originals/{event_slug}/{uuid}.{ext} — one level of per-event directories.
let mut event_dirs = match tokio::fs::read_dir(&originals).await {
Ok(rd) => rd,
// Nothing uploaded yet; the directory is created lazily by the upload handler.
Err(_) => return,
};
let mut candidates: Vec<(String, std::path::PathBuf)> = Vec::new();
let mut temps_removed = 0u32;
while let Ok(Some(event_dir)) = event_dirs.next_entry().await {
if !event_dir
.file_type()
.await
.map(|t| t.is_dir())
.unwrap_or(false)
{
continue;
}
let slug = event_dir.file_name().to_string_lossy().to_string();
let Ok(mut files) = tokio::fs::read_dir(event_dir.path()).await else {
continue;
};
while let Ok(Some(entry)) = files.next_entry().await {
let Ok(meta) = entry.metadata().await else {
continue;
};
if !meta.is_file() {
continue;
}
// Too young to judge: an upload committing RIGHT NOW is indistinguishable from an
// orphan, because the rename precedes the commit.
let recent = meta
.modified()
.ok()
.and_then(|m| m.elapsed().ok())
.is_none_or(|age| age < cutoff);
if recent {
continue;
}
let name = entry.file_name().to_string_lossy().to_string();
if name.ends_with(".tmp") {
// A `.tmp` never has a row by construction — no DB check needed.
if tokio::fs::remove_file(entry.path()).await.is_ok() {
temps_removed += 1;
}
continue;
}
candidates.push((format!("originals/{slug}/{name}"), entry.path()));
}
}
if temps_removed > 0 {
tracing::warn!(
"reclaimed {temps_removed} abandoned upload temp file(s) older than \
{ORPHAN_UPLOAD_RETENTION_HOURS}h"
);
}
if candidates.is_empty() {
return;
}
// One query per batch, not one per file: a backlog of thousands of orphans must not turn
// into thousands of round trips on an hourly timer.
let mut orphans_removed = 0u32;
for chunk in candidates.chunks(500) {
let paths: Vec<String> = chunk.iter().map(|(rel, _)| rel.clone()).collect();
// NO `deleted_at IS NULL` FILTER HERE. A soft-deleted row still points at its file
// during its retention window, and reclaiming that file is `cleanup_deleted_media`'s
// job — filtering here would race the two sweeps and destroy the exact files the
// recovery window exists to preserve.
let unreferenced: Result<Vec<(String,)>, _> = sqlx::query_as(
"SELECT p FROM unnest($1::text[]) AS p
WHERE NOT EXISTS (SELECT 1 FROM upload u WHERE u.original_path = p)",
)
.bind(&paths)
.fetch_all(pool)
.await;
let unreferenced = match unreferenced {
Ok(rows) => rows,
Err(e) => {
tracing::warn!(error = ?e, "orphan-original sweep query failed");
return;
}
};
for (rel,) in unreferenced {
if let Some((_, abs)) = chunk.iter().find(|(r, _)| *r == rel)
&& tokio::fs::remove_file(abs).await.is_ok()
{
orphans_removed += 1;
}
}
}
if orphans_removed > 0 {
tracing::warn!(
"reclaimed {orphans_removed} original(s) with no upload row, older than \
{ORPHAN_UPLOAD_RETENTION_HOURS}h"
);
}
}

2
backend/src/services/mod.rs
View File

@@ -2,6 +2,8 @@ pub mod compression;
pub mod config;
pub mod disk;
pub mod export;
pub mod imaging;
pub mod maintenance;
pub mod rate_limiter;
pub mod sse_tickets;
pub mod video;

59
backend/src/services/rate_limiter.rs
View File

@@ -17,13 +17,14 @@ impl RateLimiter {
}
}
/// Returns `true` if the request is allowed, `false` if rate-limited.
pub fn check(&self, key: impl Into<String>, max: usize, window: Duration) -> bool {
self.check_with_retry(key, max, window).is_ok()
}
/// Returns `Ok(())` if allowed, `Err(retry_after_secs)` if rate-limited.
/// `retry_after_secs` is how long until the oldest slot in the window expires.
///
/// This is deliberately the ONLY entry point. There used to be a `check()` wrapper
/// returning a plain bool, and 7 of the 8 call sites used it and then hard-coded
/// `None` for the response's `Retry-After` — so a throttled client was told to back
/// off but never for how long. Forcing every caller through the `Result` makes the
/// retry delay impossible to discard by accident.
pub fn check_with_retry(
&self,
key: impl Into<String>,
@@ -84,6 +85,11 @@ impl RateLimiter {
/// appends is the real client. A client can prepend arbitrary spoofed values to
/// the left of XFF to dodge throttles — those are ignored here. This assumes
/// exactly one trusted proxy (Caddy); revisit if that changes.
///
/// Pass the peer address as `fallback`, never a constant. Every caller used to pass
/// the literal `"unknown"`, so any request that arrived without XFF — i.e. anything
/// reaching the app directly rather than through Caddy — shared ONE bucket with every
/// other such request, turning the limiter into a self-inflicted global throttle.
pub fn client_ip(headers: &axum::http::HeaderMap, fallback: &str) -> String {
headers
.get("x-forwarded-for")
@@ -104,29 +110,35 @@ mod tests {
#[test]
fn allows_up_to_max_then_blocks() {
let rl = RateLimiter::new();
assert!(rl.check("k", 3, MIN));
assert!(rl.check("k", 3, MIN));
assert!(rl.check("k", 3, MIN));
assert!(!rl.check("k", 3, MIN), "the 4th request must be blocked");
assert!(rl.check_with_retry("k", 3, MIN).is_ok());
assert!(rl.check_with_retry("k", 3, MIN).is_ok());
assert!(rl.check_with_retry("k", 3, MIN).is_ok());
assert!(
rl.check_with_retry("k", 3, MIN).is_err(),
"the 4th request must be blocked"
);
}
#[test]
fn keys_are_independent() {
let rl = RateLimiter::new();
assert!(rl.check("a", 1, MIN));
assert!(!rl.check("a", 1, MIN));
assert!(rl.check("b", 1, MIN), "a different key has its own window");
assert!(rl.check_with_retry("a", 1, MIN).is_ok());
assert!(rl.check_with_retry("a", 1, MIN).is_err());
assert!(
rl.check_with_retry("b", 1, MIN).is_ok(),
"a different key has its own window"
);
}
#[test]
fn window_slides_and_allows_again_after_expiry() {
let rl = RateLimiter::new();
let w = Duration::from_millis(40);
assert!(rl.check("k", 1, w));
assert!(!rl.check("k", 1, w));
assert!(rl.check_with_retry("k", 1, w).is_ok());
assert!(rl.check_with_retry("k", 1, w).is_err());
std::thread::sleep(Duration::from_millis(55));
assert!(
rl.check("k", 1, w),
rl.check_with_retry("k", 1, w).is_ok(),
"the slot should expire once the window passes"
);
}
@@ -191,10 +203,13 @@ mod tests {
#[test]
fn clear_resets_every_window() {
let rl = RateLimiter::new();
assert!(rl.check("k", 1, MIN));
assert!(!rl.check("k", 1, MIN));
assert!(rl.check_with_retry("k", 1, MIN).is_ok());
assert!(rl.check_with_retry("k", 1, MIN).is_err());
rl.clear();
assert!(rl.check("k", 1, MIN), "clear() must free the window");
assert!(
rl.check_with_retry("k", 1, MIN).is_ok(),
"clear() must free the window"
);
}
/// `prune()` is a memory-leak guard: without it a long-lived process keeps one HashMap
@@ -216,7 +231,7 @@ mod tests {
.insert("stale".to_string(), vec![ancient]);
// ...alongside a key that is still inside its window.
assert!(rl.check("live", 5, MIN));
assert!(rl.check_with_retry("live", 5, MIN).is_ok());
assert_eq!(rl.windows.lock().unwrap().len(), 2);
rl.prune();
@@ -239,13 +254,13 @@ mod tests {
// prune() dropped live keys, every background sweep would hand attackers a fresh
// budget.
let rl = RateLimiter::new();
assert!(rl.check("k", 1, MIN));
assert!(!rl.check("k", 1, MIN));
assert!(rl.check_with_retry("k", 1, MIN).is_ok());
assert!(rl.check_with_retry("k", 1, MIN).is_err());
rl.prune();
assert!(
!rl.check("k", 1, MIN),
rl.check_with_retry("k", 1, MIN).is_err(),
"prune() must not clear a window that is still active"
);
}

157
backend/src/services/sse_tickets.rs
View File

@@ -13,6 +13,16 @@ use rand::Rng;
/// stream open. Tickets are consumed on use and expire after `TTL`.
const TTL: Duration = Duration::from_secs(30);
/// Ceiling on outstanding tickets across the whole process.
///
/// Not really about the bytes (~120 each) — about `issue` having had no bound of any kind.
/// Sized well above a real event: ~1000 concurrent clients each holding one live 30 s ticket.
const MAX_TICKETS: usize = 4096;
/// Live tickets one session may hold. Above 1 because two tabs sharing a token open their
/// EventSources concurrently; 4 absorbs that without letting a reconnect loop accumulate.
const MAX_TICKETS_PER_SESSION: usize = 4;
#[derive(Clone)]
pub struct SseTicketStore {
inner: Arc<Mutex<HashMap<String, Entry>>>,
@@ -39,9 +49,47 @@ impl SseTicketStore {
}
/// Mint a new ticket bound to the caller's session (identified by token hash).
pub fn issue(&self, token_hash: String) -> String {
///
/// `None` when the store is at capacity — the caller should answer 503, not evict.
///
/// Three bounds, because `issue` had none: no size cap, no per-caller cap, and no rate
/// limit on the endpoint, while `prune` ran only hourly against a 30-second TTL. So any
/// authenticated session could loop the endpoint and grow the map for an hour.
pub fn issue(&self, token_hash: String) -> Option<String> {
let ticket = random_ticket();
let mut map = self.inner.lock().unwrap();
// Prune on issue rather than only hourly. This alone changes the bound from "tickets
// minted since the last maintenance tick" to "tickets live at once", which is what the
// 30 s TTL was always meant to express.
map.retain(|_, e| e.issued_at.elapsed() <= TTL);
// Cap the caller's own outstanding tickets, evicting their oldest. NOT one-per-session:
// two tabs sharing a token open their EventSources concurrently, and having tab B
// invalidate tab A's unconsumed ticket looks exactly like a flaky SSE connection.
let mut mine: Vec<(String, Instant)> = map
.iter()
.filter(|(_, e)| e.token_hash == token_hash)
.map(|(k, e)| (k.clone(), e.issued_at))
.collect();
if mine.len() >= MAX_TICKETS_PER_SESSION {
mine.sort_by_key(|(_, issued)| *issued);
for (key, _) in mine.iter().take(mine.len() - MAX_TICKETS_PER_SESSION + 1) {
map.remove(key);
}
}
// At capacity, REFUSE — never evict a stranger's ticket. Evicting would let one
// misbehaving client deny SSE to the whole venue, which is worse than failing the
// request that hit the ceiling.
if map.len() >= MAX_TICKETS {
tracing::warn!(
outstanding = map.len(),
"SSE ticket store at capacity; refusing to mint"
);
return None;
}
map.insert(
ticket.clone(),
Entry {
@@ -49,7 +97,7 @@ impl SseTicketStore {
issued_at: Instant::now(),
},
);
ticket
Some(ticket)
}
/// Consume a ticket. Returns `Some(token_hash)` if the ticket exists and is
@@ -84,10 +132,16 @@ fn random_ticket() -> String {
mod tests {
use super::*;
/// `issue` now returns `Option`; in every test below the store is far from capacity, so an
/// `expect` here documents that refusing is exceptional rather than routine.
fn issue(store: &SseTicketStore, hash: &str) -> String {
store.issue(hash.into()).expect("store has capacity")
}
#[test]
fn issue_then_consume_returns_the_hash_exactly_once() {
let store = SseTicketStore::new();
let ticket = store.issue("hash-1".into());
let ticket = issue(&store, "hash-1");
assert_eq!(store.consume(&ticket).as_deref(), Some("hash-1"));
// Single-use: a replay of the same ticket is rejected.
assert_eq!(
@@ -106,8 +160,8 @@ mod tests {
#[test]
fn issued_tickets_are_unique_and_hex() {
let store = SseTicketStore::new();
let a = store.issue("h".into());
let b = store.issue("h".into());
let a = issue(&store, "h");
let b = issue(&store, "h");
assert_ne!(a, b, "each ticket must be unique");
assert_eq!(a.len(), 48, "24 random bytes → 48 hex chars");
assert!(a.chars().all(|c| c.is_ascii_hexdigit()));
@@ -116,29 +170,104 @@ mod tests {
#[test]
fn fresh_ticket_survives_prune() {
let store = SseTicketStore::new();
let ticket = store.issue("h".into());
let ticket = issue(&store, "h");
store.prune(); // not expired → kept
assert_eq!(store.consume(&ticket).as_deref(), Some("h"));
}
#[test]
fn expired_ticket_consumes_to_none() {
// Construct an entry that is already past the TTL and confirm consume() rejects it.
let store = SseTicketStore::new();
let stale = "stale-ticket".to_string();
/// Build an entry that is already past the TTL.
fn insert_stale(store: &SseTicketStore, key: &str, token_hash: &str) {
store.inner.lock().unwrap().insert(
stale.clone(),
key.to_string(),
Entry {
token_hash: "h".into(),
token_hash: token_hash.into(),
issued_at: Instant::now()
.checked_sub(TTL + Duration::from_secs(1))
.expect("host uptime should exceed the ticket TTL"),
},
);
}
#[test]
fn expired_ticket_consumes_to_none() {
let store = SseTicketStore::new();
insert_stale(&store, "stale-ticket", "h");
assert_eq!(
store.consume(&stale),
store.consume("stale-ticket"),
None,
"an expired ticket must not authenticate"
);
}
/// The TTL is 30 s but `prune` only ran hourly, so the map was really bounded by "tickets
/// minted in the last hour" — which is unbounded for a client in a loop.
#[test]
fn issuing_prunes_expired_entries() {
let store = SseTicketStore::new();
insert_stale(&store, "stale-a", "someone-else");
insert_stale(&store, "stale-b", "someone-else");
issue(&store, "h");
assert_eq!(
store.inner.lock().unwrap().len(),
1,
"issue must reclaim expired slots, not merely add to them"
);
}
/// Two tabs sharing a token is normal, so the per-session cap must be above 1 — but a
/// reconnect loop must not accumulate. The caller's OWN oldest is what gets evicted.
#[test]
fn a_session_is_capped_and_evicts_only_its_own_oldest() {
let store = SseTicketStore::new();
let stranger = issue(&store, "other-session");
let mut mine: Vec<String> = Vec::new();
for _ in 0..MAX_TICKETS_PER_SESSION + 2 {
mine.push(issue(&store, "mine"));
}
let live = mine
.iter()
.filter(|t| store.inner.lock().unwrap().contains_key(*t))
.count();
assert_eq!(live, MAX_TICKETS_PER_SESSION, "one session, bounded");
assert!(
store.inner.lock().unwrap().contains_key(&mine[mine.len() - 1]),
"the newest ticket is the one the caller is about to use"
);
assert_eq!(
store.consume(&stranger).as_deref(),
Some("other-session"),
"another session's ticket must survive — evicting it would let one client deny \
SSE to the venue"
);
}
/// At capacity the store REFUSES rather than evicting a stranger. Refusing fails the one
/// request that hit the ceiling; evicting would break an unrelated client's live stream.
#[test]
fn at_capacity_the_store_refuses_instead_of_evicting() {
let store = SseTicketStore::new();
{
let mut map = store.inner.lock().unwrap();
for i in 0..MAX_TICKETS {
map.insert(
format!("filler-{i}"),
Entry {
token_hash: format!("session-{i}"),
issued_at: Instant::now(),
},
);
}
}
assert_eq!(
store.issue("newcomer".into()),
None,
"a full store must refuse, so the caller can answer 503"
);
assert!(
store.inner.lock().unwrap().contains_key("filler-0"),
"no existing ticket may be sacrificed to make room"
);
}
}

233
backend/src/services/video.rs Normal file
View File

@@ -0,0 +1,233 @@
//! Poster-frame extraction, shared by the compression worker and the HTML export.
//!
//! Both used to spawn `ffmpeg` themselves with the same broken invocation:
//!
//! ```text
//! ffmpeg -i <src> -vframes 1 -ss 00:00:01 -vf scale=… -y <out>
//! ```
//!
//! `-ss` AFTER `-i` is an output-side seek. Against a clip of a second or less ffmpeg exits **0 and
//! writes nothing** — and both call sites gated on the exit status, so neither noticed. The worker
//! then wrote `thumbnail_path` for a file that was never created (404 in the live feed) and the
//! export listed the entry in `data.json` while the ZIP writer skipped it (a broken image tile in
//! the keepsake). Every server-side signal stayed green. Phones produce such clips constantly:
//! mis-taps, Live Photos, boomerangs.
//!
//! This module exists for the same reason `imaging.rs` does — that one was created when compression
//! and export duplicated decode logic, and it paid off immediately when the `max_alloc` fix landed
//! in both workers at once. Same duplication, same fix.
use std::path::Path;
use std::time::Duration;
use anyhow::{Context, Result};
/// A malformed video can hang `ffmpeg` indefinitely. In the compression worker that never releases
/// the semaphore permit and the pool eventually deadlocks; in the export worker it strands the job
/// at `running` so the keepsake never completes. `export.rs` had NO timeout at all before this
/// module — sharing the spawn fixes that too.
const FFMPEG_TIMEOUT: Duration = Duration::from_secs(120);
/// Seek positions to try, in order.
///
/// One second first: the opening frame of a real video is often black, a fade-in, or motion-blurred
/// as the camera settles, so it makes a poor poster. Zero second as the fallback, which is what
/// makes short clips work — and it is genuinely required, not defensive. Moving `-ss` before `-i`
/// (an input-side seek) is necessary but NOT sufficient: seeking to 1 s in a 1.000 s clip is still
/// past the last frame, and ffmpeg still exits 0 having written nothing. Verified against the real
/// production image.
const SEEK_POSITIONS: &[&str] = &["00:00:01", "0"];
/// Extract one poster frame from `src` into `dest`, scaled to `width` px wide.
///
/// `Ok(false)` means the video yielded no frame — a normal outcome for a very short or unusual
/// clip, NOT an error. Callers must degrade (no poster) rather than fail the upload: treating this
/// as an error would soft-delete every sub-second video, turning a cosmetic defect into data loss.
///
/// `Err` is reserved for something genuinely wrong — a hang we had to kill, or a failure to spawn.
pub async fn extract_poster_frame(src: &Path, dest: &Path, width: u32) -> Result<bool> {
for seek in SEEK_POSITIONS {
// A stale file from a previous attempt would be indistinguishable from a fresh success.
let _ = tokio::fs::remove_file(dest).await;
run_ffmpeg(src, dest, width, seek).await?;
// THE CHECK BOTH CALL SITES WERE MISSING: ask the filesystem, not the exit status.
// Non-empty, because a zero-byte file is not a poster either.
if tokio::fs::metadata(dest)
.await
.map(|m| m.is_file() && m.len() > 0)
.unwrap_or(false)
{
return Ok(true);
}
}
// Leave nothing behind for a caller to mistake for a result.
let _ = tokio::fs::remove_file(dest).await;
Ok(false)
}
/// Run one ffmpeg attempt. A non-zero exit is NOT an error here — the artifact check above is the
/// authority, and a corrupt input that fails at 1 s may still yield a frame at 0.
async fn run_ffmpeg(src: &Path, dest: &Path, width: u32, seek: &str) -> Result<()> {
let child = tokio::process::Command::new("ffmpeg")
.args([
// BEFORE -i: an input-side seek. See SEEK_POSITIONS.
"-ss",
seek,
"-i",
src.to_str().unwrap_or_default(),
"-vframes",
"1",
"-vf",
&format!("scale={width}:-1"),
"-y",
dest.to_str().unwrap_or_default(),
])
// ffmpeg writes the poster to `dest` itself; nothing here ever reads stdout, so
// giving it a pipe only created something that could fill.
.stdout(std::process::Stdio::null())
// stderr IS piped — it is the only diagnostic when a clip yields no frame — but it
// must be DRAINED, which is the whole point of `wait_with_output` below.
.stderr(std::process::Stdio::piped())
.kill_on_drop(true)
.spawn()
.context("failed to spawn ffmpeg")?;
// `wait_with_output`, NOT `wait`. ffmpeg is verbose on stderr (banner, stream info,
// per-frame progress) and `wait()` reads neither pipe — so once the ~64 KiB pipe buffer
// filled, ffmpeg blocked writing, `wait()` never returned, and the call burned the full
// timeout. That is not merely slow: the timeout is an `Err`, so after 2 seek positions x
// 3 compression attempts the caller soft-deletes a perfectly playable video for a
// poster-frame failure. `wait_with_output` polls the pipe and the exit status together.
//
// It also CONSUMES the child, so the explicit `child.kill()` that used to sit on the
// timeout arm cannot exist here — and is not needed: `kill_on_drop(true)` is set above,
// and dropping the future on timeout drops the child with it.
let out = match tokio::time::timeout(FFMPEG_TIMEOUT, child.wait_with_output()).await {
Ok(res) => res.context("ffmpeg wait failed")?,
Err(_) => anyhow::bail!("ffmpeg timed out after {}s", FFMPEG_TIMEOUT.as_secs()),
};
// A non-zero exit is not an error (see the doc comment) — the artifact check in
// `extract_poster_frame` is the authority. Log the tail so a systematically failing
// format is diagnosable without turning it into data loss.
if !out.status.success() {
tracing::debug!(
seek,
status = ?out.status,
stderr = %tail_lines(&out.stderr, 10),
"ffmpeg exited non-zero; the artifact check decides"
);
}
Ok(())
}
/// Last `n` lines of a child's stderr, lossily decoded.
///
/// Bounded on purpose: ffmpeg's stderr is unbounded, and the reason we now drain it is that
/// unbounded output used to be a hazard. Emitting all of it into a log line — into container
/// logs that are themselves size-capped — would just move the problem.
fn tail_lines(bytes: &[u8], n: usize) -> String {
let text = String::from_utf8_lossy(bytes);
let lines: Vec<&str> = text.lines().filter(|l| !l.trim().is_empty()).collect();
lines[lines.len().saturating_sub(n)..].join(" | ")
}
#[cfg(test)]
mod tests {
use super::*;
/// The order is the whole fix. `-ss` must precede `-i`, and 0 must be tried after 1 s.
#[test]
fn the_fallback_seek_exists_and_comes_last() {
assert_eq!(
SEEK_POSITIONS,
&["00:00:01", "0"],
"1s first for a better poster, 0 as the fallback that makes short clips work"
);
}
/// A missing input yields no frame rather than an error: the caller must degrade to "no
/// poster", never fail the upload. `Err` is reserved for a hang or a spawn failure.
#[tokio::test]
async fn a_missing_source_yields_no_frame_rather_than_an_error() {
let dir = std::env::temp_dir().join(format!("es-video-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
let dest = dir.join("out.jpg");
let got = extract_poster_frame(Path::new("/nonexistent/clip.mp4"), &dest, 400).await;
match got {
Ok(false) => {}
other => panic!("expected Ok(false) for a missing input, got {other:?}"),
}
assert!(
!dest.exists(),
"a failed extraction must leave nothing a caller could mistake for a poster"
);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn the_stderr_tail_is_bounded_and_survives_invalid_utf8() {
let noisy: Vec<u8> = (0..500)
.map(|i| format!("line {i}\n"))
.collect::<String>()
.into_bytes();
let got = tail_lines(&noisy, 3);
assert_eq!(got, "line 497 | line 498 | line 499");
// ffmpeg emits filenames verbatim, so its stderr is not guaranteed to be UTF-8.
assert_eq!(tail_lines(&[b'o', b'k', 0xff], 5), "ok\u{fffd}");
assert_eq!(tail_lines(b"", 5), "");
}
/// A real extraction must finish in a small fraction of `FFMPEG_TIMEOUT`.
///
/// Wall-clock is the ONLY observable of the bug this guards: piping stderr and then
/// calling `wait()` (which drains nothing) blocks ffmpeg on a full pipe buffer until the
/// timeout fires, and the timeout is an `Err`, so the upload is soft-deleted. The
/// assertion is deliberately on elapsed time, not on the exit status.
///
/// Honest limitation: our fixture is quiet enough not to fill a 64 KiB pipe on its own,
/// so this catches a regression to `wait()` only in combination with a verbose input. It
/// is still worth pinning — a reverted drain plus any chatty clip is data loss.
#[tokio::test]
async fn a_real_clip_yields_a_poster_well_inside_the_timeout() {
if tokio::process::Command::new("ffmpeg")
.arg("-version")
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::null())
.status()
.await
.is_err()
{
eprintln!("skipping: ffmpeg not on PATH");
return;
}
let src = Path::new("../e2e/fixtures/media/sample.mp4");
if !src.exists() {
eprintln!("skipping: {} missing", src.display());
return;
}
let dir = std::env::temp_dir().join(format!("es-video-ok-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
let dest = dir.join("poster.jpg");
let started = std::time::Instant::now();
let got = extract_poster_frame(src, &dest, 400).await;
let elapsed = started.elapsed();
assert!(matches!(got, Ok(true)), "expected a poster, got {got:?}");
assert!(dest.metadata().unwrap().len() > 0);
assert!(
elapsed < FFMPEG_TIMEOUT / 4,
"extraction took {elapsed:?}; a drained stderr finishes in well under \
{FFMPEG_TIMEOUT:?} — this is the pipe-deadlock regression guard"
);
let _ = std::fs::remove_dir_all(&dir);
}
}

1
backend/static/export-viewer/_app/env.js
View File

@@ -1 +0,0 @@
export const env={}

1
backend/static/export-viewer/_app/immutable/assets/0.Cobcmike.css
View File

File diff suppressed because one or more lines are too long

1
backend/static/export-viewer/_app/immutable/chunks/BJ__EZ0W.js
View File

@@ -1 +0,0 @@
import{u as o,n as t,o as c}from"./CcONa1Mr.js";function u(e){throw new Error("https://svelte.dev/e/lifecycle_outside_component")}function r(e){t===null&&u(),o(()=>{const n=c(e);if(typeof n=="function")return n})}export{r as o};

1
backend/static/export-viewer/_app/immutable/chunks/BRDva_z9.js
View File

@@ -1 +0,0 @@
import{f as l,g as o,p as u,i as n,j as d,k as m,h as p,e as _,m as v,l as k}from"./CcONa1Mr.js";class w{anchor;#t=new Map;#s=new Map;#e=new Map;#i=new Set;#f=!0;constructor(t,s=!0){this.anchor=t,this.#f=s}#a=t=>{if(this.#t.has(t)){var s=this.#t.get(t),e=this.#s.get(s);if(e)l(e),this.#i.delete(s);else{var f=this.#e.get(s);f&&(this.#s.set(s,f.effect),this.#e.delete(s),f.fragment.lastChild.remove(),this.anchor.before(f.fragment),e=f.effect)}for(const[i,a]of this.#t){if(this.#t.delete(i),i===t)break;const r=this.#e.get(a);r&&(o(r.effect),this.#e.delete(a))}for(const[i,a]of this.#s){if(i===s||this.#i.has(i))continue;const r=()=>{if(Array.from(this.#t.values()).includes(i)){var c=document.createDocumentFragment();v(a,c),c.append(n()),this.#e.set(i,{effect:a,fragment:c})}else o(a);this.#i.delete(i),this.#s.delete(i)};this.#f||!e?(this.#i.add(i),u(a,r,!1)):r()}}};#r=t=>{this.#t.delete(t);const s=Array.from(this.#t.values());for(const[e,f]of this.#e)s.includes(e)||(o(f.effect),this.#e.delete(e))};ensure(t,s){var e=m,f=k();if(s&&!this.#s.has(t)&&!this.#e.has(t))if(f){var i=document.createDocumentFragment(),a=n();i.append(a),this.#e.set(t,{effect:d(()=>s(a)),fragment:i})}else this.#s.set(t,d(()=>s(this.anchor)));if(this.#t.set(e,t),f){for(const[r,h]of this.#s)r===t?e.unskip_effect(h):e.skip_effect(h);for(const[r,h]of this.#e)r===t?e.unskip_effect(h.effect):e.skip_effect(h.effect);e.oncommit(this.#a),e.ondiscard(this.#r)}else p&&(this.anchor=_),this.#a(e)}}export{w as B};

2
backend/static/export-viewer/_app/immutable/chunks/Bb9JxzU7.js
View File

File diff suppressed because one or more lines are too long

1
backend/static/export-viewer/_app/immutable/chunks/Bxn6SmGf.js
View File

@@ -1 +0,0 @@
import{b as c,h as o,a as l,E as b,r as p,s as v,c as g,d,e as m}from"./CcONa1Mr.js";import{B as y}from"./BRDva_z9.js";function k(f,h,_=!1){var n;o&&(n=m,l());var s=new y(f),u=_?b:0;function t(a,r){if(o){var e=p(n);if(a!==parseInt(e.substring(1))){var i=v();g(i),s.anchor=i,d(!1),s.ensure(a,r),d(!0);return}}s.ensure(a,r)}c(()=>{var a=!1;h((r,e=0)=>{a=!0,t(e,r)}),a||t(-1,null)},u)}export{k as i};

1
backend/static/export-viewer/_app/immutable/chunks/CcONa1Mr.js
View File

File diff suppressed because one or more lines are too long

1
backend/static/export-viewer/_app/immutable/chunks/RsTAN2PN.js
View File

@@ -1 +0,0 @@
import{A as v,i as d,B as l,C as u,D as T,T as p,F as h,h as i,e as s,R as E,a as y,G as g,c as w,H as N}from"./CcONa1Mr.js";const A=globalThis?.window?.trustedTypes&&globalThis.window.trustedTypes.createPolicy("svelte-trusted-html",{createHTML:t=>t});function M(t){return A?.createHTML(t)??t}function x(t){var r=v("template");return r.innerHTML=M(t.replaceAll("<!>","<!---->")),r.content}function n(t,r){var e=l;e.nodes===null&&(e.nodes={start:t,end:r,a:null,t:null})}function b(t,r){var e=(r&p)!==0,f=(r&h)!==0,a,_=!t.startsWith("<!>");return()=>{if(i)return n(s,null),s;a===void 0&&(a=x(_?t:"<!>"+t),e||(a=u(a)));var o=f||T?document.importNode(a,!0):a.cloneNode(!0);if(e){var c=u(o),m=o.lastChild;n(c,m)}else n(o,o);return o}}function C(t=""){if(!i){var r=d(t+"");return n(r,r),r}var e=s;return e.nodeType!==g?(e.before(e=d()),w(e)):N(e),n(e,e),e}function O(){if(i)return n(s,null),s;var t=document.createDocumentFragment(),r=document.createComment(""),e=d();return t.append(r,e),n(r,e),t}function P(t,r){if(i){var e=l;((e.f&E)===0||e.nodes.end===null)&&(e.nodes.end=s),y();return}t!==null&&t.before(r)}const L="5";typeof window<"u"&&((window.__svelte??={}).v??=new Set).add(L);export{P as a,n as b,O as c,b as f,C as t};

1
backend/static/export-viewer/_app/immutable/chunks/eAGLaJx1.js
View File

File diff suppressed because one or more lines are too long

2
backend/static/export-viewer/_app/immutable/entry/app.BTH3knpg.js
View File

File diff suppressed because one or more lines are too long

1
backend/static/export-viewer/_app/immutable/entry/start.YjNZv4co.js
View File

@@ -1 +0,0 @@
import{l as o,a as r}from"../chunks/eAGLaJx1.js";export{o as load_css,r as start};

1
backend/static/export-viewer/_app/immutable/nodes/0.DLesTSIe.js
View File

@@ -1 +0,0 @@
import{c as s,a as c}from"../chunks/RsTAN2PN.js";import{b as l,E as p,t as i}from"../chunks/CcONa1Mr.js";import{B as m}from"../chunks/BRDva_z9.js";function u(n,r,...e){var o=new m(n);l(()=>{const t=r()??null;o.ensure(t,t&&(a=>t(a,...e)))},p)}const f=!0,_=!1,g=Object.freeze(Object.defineProperty({__proto__:null,prerender:f,ssr:_},Symbol.toStringTag,{value:"Module"}));function h(n,r){var e=s(),o=i(e);u(o,()=>r.children),c(n,e)}export{h as component,g as universal};

1
backend/static/export-viewer/_app/immutable/nodes/1.D6Kk4lDA.js
View File

@@ -1 +0,0 @@
import{a as i,f as h}from"../chunks/RsTAN2PN.js";import{q as g,t as v,v as d,w as l,x as s,y as a,z as x}from"../chunks/CcONa1Mr.js";import{s as o}from"../chunks/Bb9JxzU7.js";import{s as _,p}from"../chunks/eAGLaJx1.js";const $={get error(){return p.error},get status(){return p.status}};_.updated.check;const m=$;var k=h("<h1> </h1> <p> </p>",1);function z(c,f){g(f,!0);var t=k(),r=v(t),n=s(r,!0);a(r);var e=x(r,2),u=s(e,!0);a(e),d(()=>{o(n,m.status),o(u,m.error?.message)}),i(c,t),l()}export{z as component};

2
backend/static/export-viewer/_app/immutable/nodes/2.CkuIG73r.js
View File

File diff suppressed because one or more lines are too long

1
backend/static/export-viewer/_app/version.json
View File

@@ -1 +0,0 @@
{"version":"1778876725548"}

33
backend/static/export-viewer/index.html
View File

File diff suppressed because one or more lines are too long

88
backend/tests/common/mod.rs
View File

@@ -255,3 +255,91 @@ pub async fn downloadable(pool: &PgPool, event_id: Uuid, export_type: &str) -> O
.expect("downloadable")
.flatten()
}
/// Insert an upload of `size` bytes, optionally already soft-deleted.
pub async fn seed_upload(
pool: &PgPool,
event_id: Uuid,
user_id: Uuid,
size: i64,
deleted: bool,
) -> Uuid {
sqlx::query_scalar(
"INSERT INTO upload (event_id, user_id, original_path, mime_type,
original_size_bytes, deleted_at)
VALUES ($1, $2, 'originals/x.jpg', 'image/jpeg', $3,
CASE WHEN $4 THEN NOW() ELSE NULL END)
RETURNING id",
)
.bind(event_id)
.bind(user_id)
.bind(size)
.bind(deleted)
.fetch_one(pool)
.await
.expect("seed upload")
}
/// Flip the moderation flags a ban sets.
pub async fn set_user_moderation(pool: &PgPool, user_id: Uuid, banned: bool, hidden: bool) {
sqlx::query("UPDATE \"user\" SET is_banned = $2, uploads_hidden = $3 WHERE id = $1")
.bind(user_id)
.bind(banned)
.bind(hidden)
.execute(pool)
.await
.expect("set moderation");
}
/// SRC: `services/export.rs::query_uploads` — the visibility filter, verbatim, projected down to
/// `(id, original_size_bytes)`. This is the row set that ACTUALLY lands in the archives.
///
/// Production builds this WHERE from `export_visibility_where!()`, shared with
/// `estimate_export_bytes`. A copy here can pin the behaviour but CANNOT detect production moving
/// away from it — that is what sharing the fragment is for, not this.
pub async fn export_visible_uploads(pool: &PgPool, event_id: Uuid) -> Vec<(Uuid, i64)> {
sqlx::query_as(
"SELECT u.id, u.original_size_bytes
FROM upload u
JOIN \"user\" usr ON usr.id = u.user_id
WHERE u.event_id = $1 AND u.deleted_at IS NULL
AND usr.uploads_hidden = FALSE AND usr.is_banned = FALSE
GROUP BY u.id, usr.display_name
ORDER BY u.created_at ASC",
)
.bind(event_id)
.fetch_all(pool)
.await
.expect("export_visible_uploads")
}
/// SRC: `services/export.rs::estimate_export_bytes` — verbatim. Same caveat as above: production
/// shares its WHERE with `query_uploads` via `export_visibility_where!()`, so these two copies
/// agreeing proves the behaviour, not the absence of drift.
pub async fn estimate_export_bytes(pool: &PgPool, event_id: Uuid) -> i64 {
let (bytes,): (i64,) = sqlx::query_as(
"SELECT COALESCE(SUM(u.original_size_bytes), 0)::bigint
FROM upload u
JOIN \"user\" usr ON usr.id = u.user_id
WHERE u.event_id = $1 AND u.deleted_at IS NULL
AND usr.uploads_hidden = FALSE AND usr.is_banned = FALSE",
)
.bind(event_id)
.fetch_one(pool)
.await
.expect("estimate_export_bytes");
bytes
}
/// SRC: `services/export.rs::ensure_export_space` — the armed-job count, verbatim.
pub async fn armed_job_count(pool: &PgPool, event_id: Uuid) -> i64 {
let (n,): (i64,) = sqlx::query_as(
"SELECT COUNT(*) FROM export_job
WHERE event_id = $1 AND status IN ('pending', 'running')",
)
.bind(event_id)
.fetch_one(pool)
.await
.expect("armed_job_count");
n
}

163
backend/tests/export_preflight.rs Normal file
View File

@@ -0,0 +1,163 @@
//! DB-backed tests for the export disk preflight.
//!
//! The keepsake used to be built with NO free-space check at all, and the failure that produced was
//! not "the export failed" but "the deliverable is stuck and the escape hatch needs the space that
//! isn't there":
//!
//! 1. A takedown bumps the epoch and re-arms both halves.
//! 2. The ZIP hits ENOSPC partway through a multi-GB write.
//! 3. The job row is now `failed` at the CURRENT epoch, so readiness
//! (`epoch = event.export_epoch AND status = 'done'`) is false and `GET /export/zip` 404s —
//! while the last good archive sits on disk, unreferenced and unreachable.
//! 4. `POST /host/export/rebuild` re-arms the same doomed write.
//!
//! Two changes close it: reclaim the superseded generation BEFORE building (so peak usage is one
//! generation, not two) and refuse up front with a number the host can act on.
//!
//! What these tests pin is the ESTIMATE — the part that decides. The arithmetic on top of it lives
//! in `services/export.rs`'s unit tests; the filesystem selection lives in `is_superseded_archive`.
//!
//! ON DRIFT, precisely, because it is easy to overclaim here. The hazard is that `query_uploads`
//! (which selects the rows the archives are built from) and `estimate_export_bytes` (which sizes
//! them) could disagree — and an estimate missing rows the archive writes UNDER-reserves, the one
//! direction that reintroduces the ENOSPC. **These tests cannot catch that**, and neither can any
//! test in this harness: both sides here are `SRC:`-marked hand-copies in `tests/common/mod.rs`,
//! so if production moved and the copies didn't, they would sit still and keep passing.
//!
//! That is fixed where it can be — the two queries now share one `export_visibility_where!()`
//! fragment in `services/export.rs`, so they cannot diverge by construction. What is left for
//! these tests is what the convention is genuinely good at: pinning the BEHAVIOUR, so a change
//! that deliberately alters the filter has to come here and say so.
mod common;
use common::*;
use sqlx::PgPool;
/// The estimate must equal the sum over EXACTLY the rows `query_uploads` returns — computed from
/// that row set, not from a restatement of its WHERE clause.
///
/// PINS: which uploads the preflight is allowed to count. Each excluded row below is excluded by a
/// DIFFERENT predicate, so a change that drops or weakens any one of them fails here and has to be
/// argued for. (It does not detect production drifting away from these copies — see the file
/// header; `export_visibility_where!()` is what makes that impossible.)
#[sqlx::test]
async fn the_estimate_sums_exactly_the_rows_the_archive_will_contain(pool: PgPool) {
let event_id = seed_event(&pool, "wedding").await;
let visible = seed_user(&pool, event_id, "Anna").await;
let banned = seed_user(&pool, event_id, "Ben").await;
let hidden = seed_user(&pool, event_id, "Cara").await;
seed_upload(&pool, event_id, visible, 1_000, false).await;
seed_upload(&pool, event_id, visible, 2_500, false).await;
// Each of these is excluded from the archive by a DIFFERENT predicate.
seed_upload(&pool, event_id, visible, 9_000, true).await; // soft-deleted
seed_upload(&pool, event_id, banned, 9_000, false).await; // uploader banned
seed_upload(&pool, event_id, hidden, 9_000, false).await; // uploads hidden
set_user_moderation(&pool, banned, true, true).await;
set_user_moderation(&pool, hidden, false, true).await;
let rows = export_visible_uploads(&pool, event_id).await;
let expected: i64 = rows.iter().map(|(_, bytes)| bytes).sum();
assert_eq!(rows.len(), 2, "only Anna's two live uploads are archived");
assert_eq!(
estimate_export_bytes(&pool, event_id).await,
expected,
"the preflight must size the gallery the export will actually write"
);
assert_eq!(expected, 3_500);
}
/// An event with nothing to archive estimates zero rather than NULL.
///
/// PREVENTS: `SUM()` over no rows returning NULL and the decode blowing up — which would abort the
/// export with a type error instead of building an (entirely legitimate) empty keepsake.
#[sqlx::test]
async fn an_empty_gallery_estimates_zero_not_null(pool: PgPool) {
let event_id = seed_event(&pool, "wedding").await;
assert_eq!(estimate_export_bytes(&pool, event_id).await, 0);
// And with a user who has uploaded nothing.
seed_user(&pool, event_id, "Anna").await;
assert_eq!(estimate_export_bytes(&pool, event_id).await, 0);
}
/// A release arms both halves, so the preflight sees a count of 2 and reserves for the pair.
///
/// PREVENTS: the concurrency under-reservation. `spawn_export_jobs` starts the ZIP and HTML workers
/// at the same instant, and BOTH are gallery-sized (`Memories.zip` streams the original for every
/// video and every image at or under 5 MB, all `Compression::Stored`). A worker reserving only for
/// itself would see "it fits", its sibling would independently see the same, and together they
/// would ENOSPC — which is why `required_free_bytes` multiplies by this count.
#[sqlx::test]
async fn a_release_arms_both_halves_so_the_preflight_reserves_for_two(pool: PgPool) {
let event_id = seed_event(&pool, "wedding").await;
let user = seed_user(&pool, event_id, "Anna").await;
seed_upload(&pool, event_id, user, 1_000, false).await;
assert_eq!(
armed_job_count(&pool, event_id).await,
0,
"nothing is armed before the release"
);
let epoch = release_gallery(&pool, "wedding").await.expect("released");
assert_eq!(
armed_job_count(&pool, event_id).await,
2,
"a release arms zip AND html — both compete for the same disk"
);
// A worker that has claimed its half is still competing; `running` must keep counting.
assert!(claim_job(&pool, event_id, "zip", epoch).await);
assert_eq!(
armed_job_count(&pool, event_id).await,
2,
"claiming moves pending -> running, which must not drop out of the reservation"
);
// Only a FINISHED half stops competing.
assert!(finalize_job(&pool, event_id, "zip", epoch, "exports/Gallery.zip").await);
assert_eq!(
armed_job_count(&pool, event_id).await,
1,
"a done half no longer needs space reserved for it"
);
}
/// A ViewerOnly regeneration re-arms only the HTML half, so the preflight reserves for one.
///
/// PREVENTS: over-reservation refusing a rebuild that fits perfectly well. Moderating a comment
/// carries the finished ZIP forward untouched; demanding room for a second copy of it would fail
/// the one operation that needs no new gallery-sized write at all.
#[sqlx::test]
async fn a_viewer_only_regeneration_reserves_for_one_half(pool: PgPool) {
let event_id = seed_event(&pool, "wedding").await;
let user = seed_user(&pool, event_id, "Anna").await;
seed_upload(&pool, event_id, user, 1_000, false).await;
let epoch = release_gallery(&pool, "wedding").await.expect("released");
for t in ["zip", "html"] {
assert!(claim_job(&pool, event_id, t, epoch).await);
assert!(finalize_job(&pool, event_id, t, epoch, &format!("exports/{t}")).await);
}
assert_eq!(armed_job_count(&pool, event_id).await, 0);
// A moderated comment: bump the epoch, carry the ZIP forward, re-arm only the viewer.
let (_, _, next) = bump_epoch(&pool, "wedding").await.expect("bumped");
assert!(
carry_zip_forward(&pool, event_id, next).await,
"the finished ZIP is re-stamped, not rebuilt"
);
let mut conn = pool.acquire().await.expect("acquire");
enqueue_types_at_epoch(&mut conn, event_id, next, &["html"]).await;
assert_eq!(
armed_job_count(&pool, event_id).await,
1,
"only the viewer is being rebuilt, so only one archive's worth of space is needed"
);
}

352
backend/tests/failed_original_sweep.rs Normal file
View File

@@ -0,0 +1,352 @@
//! DB-backed tests for the deleted-media sweep (`services/maintenance.rs`).
//!
//! Context, in two halves.
//!
//! The compression worker deliberately no longer deletes an upload's original when its transcode
//! fails — a transient ENOSPC or a codec panic must never destroy the only copy of a photo a guest
//! cannot retake. But the row is soft-deleted and the uploader's quota IS refunded, so those bytes
//! become invisible, unowned and free.
//!
//! The SAME hole was reachable by the ordinary path, and that one is not an edge case at all:
//! `soft_delete_in_event` refunds `total_upload_bytes` on every guest or host delete and nothing
//! removed the files, so the quota stopped bounding the disk. Upload 500 MB, delete, quota back to
//! zero, upload another 500 MB — a guest curating their camera roll, which is what people do. The
//! sweep used to reach only `compression_status = 'failed'`, so it never touched this case; the
//! test below that now asserts an owner-deleted upload IS reclaimed is the one that used to assert
//! the opposite.
//!
//! Two windows, because the two deletes mean different things: 14 days for a failure an operator
//! may want to investigate, 24 hours for a removal someone asked for (14 days outlives the whole
//! event, so a deliberate delete would never reclaim anything while it mattered).
//!
//! The selection predicate is the whole safety argument — it must reach both leftovers and never a
//! live upload — so that is what these pin, following the same "reproduce the SQL verbatim" pattern
//! as `upload_concurrency.rs`. `#[sqlx::test]` gives each test a fresh, migrated database.
mod common;
use common::*;
use sqlx::PgPool;
use uuid::Uuid;
const FAILED_DAYS: i64 = 14;
const DELETED_HOURS: i64 = 24;
/// SRC: `services/maintenance.rs::cleanup_deleted_media` — the selection, verbatim.
async fn sweep_selects(pool: &PgPool, failed_days: i64, deleted_hours: i64) -> Vec<Uuid> {
type Row = (Uuid, String, Option<String>, Option<String>, Option<String>);
sqlx::query_as::<_, Row>(
"SELECT id, original_path, preview_path, display_path, thumbnail_path FROM upload
WHERE deleted_at IS NOT NULL
AND CASE WHEN compression_status = 'failed'
THEN deleted_at < NOW() - ($1 || ' days')::interval
ELSE deleted_at < NOW() - ($2 || ' hours')::interval
END
AND (original_path <> '' OR preview_path IS NOT NULL
OR display_path IS NOT NULL OR thumbnail_path IS NOT NULL)",
)
.bind(failed_days.to_string())
.bind(deleted_hours.to_string())
.fetch_all(pool)
.await
.expect("sweep query")
.into_iter()
.map(|(id, ..)| id)
.collect()
}
/// Seed an upload aged `deleted_hours_ago` (None = live), with optional derivative paths.
async fn seed_aged_upload(
pool: &PgPool,
event_id: Uuid,
user_id: Uuid,
status: &str,
deleted_hours_ago: Option<i64>,
original_path: &str,
derivatives: bool,
) -> Uuid {
sqlx::query_scalar(
"INSERT INTO upload (event_id, user_id, original_path, mime_type, original_size_bytes,
compression_status, deleted_at,
preview_path, display_path, thumbnail_path)
VALUES ($1, $2, $3, 'image/jpeg', 1000, $4,
CASE WHEN $5::bigint IS NULL THEN NULL
ELSE NOW() - ($5::text || ' hours')::interval END,
CASE WHEN $6 THEN 'previews/p.jpg' END,
CASE WHEN $6 THEN 'displays/d.jpg' END,
CASE WHEN $6 THEN 'thumbs/t.jpg' END)
RETURNING id",
)
.bind(event_id)
.bind(user_id)
.bind(original_path)
.bind(status)
.bind(deleted_hours_ago)
.bind(derivatives)
.fetch_one(pool)
.await
.expect("seed upload")
}
/// A live upload is untouchable no matter how the windows are configured.
///
/// PREVENTS: the catastrophic loosening. Everything else here is about reclaiming more; this is the
/// one assertion that must never bend.
#[sqlx::test]
async fn a_live_upload_is_never_selected(pool: PgPool) {
let event_id = seed_event(&pool, "sweep-live").await;
let user_id = seed_user(&pool, event_id, "Sweeper").await;
for status in ["done", "failed", "processing", "pending"] {
let live = seed_aged_upload(
&pool,
event_id,
user_id,
status,
None,
"originals/e/live.jpg",
true,
)
.await;
assert!(
!sweep_selects(&pool, FAILED_DAYS, DELETED_HOURS)
.await
.contains(&live),
"a non-deleted upload with status {status} must never be swept"
);
}
}
/// THE FIX. An upload a guest or host deliberately deleted is reclaimed once past 24 hours.
///
/// PREVENTS: the regression back to a sweep scoped to `compression_status = 'failed'`, which is
/// what let the quota stop bounding the disk. This assertion is the inverse of the one this file
/// used to make.
#[sqlx::test]
async fn a_deliberately_deleted_upload_is_reclaimed_after_a_day(pool: PgPool) {
let event_id = seed_event(&pool, "sweep-deleted").await;
let user_id = seed_user(&pool, event_id, "Curator").await;
let deleted = seed_aged_upload(
&pool,
event_id,
user_id,
"done",
Some(48),
"originals/e/owner.jpg",
true,
)
.await;
// Still inside the window — a mis-tap is recoverable for a day.
let recent = seed_aged_upload(
&pool,
event_id,
user_id,
"done",
Some(2),
"originals/e/recent.jpg",
true,
)
.await;
let selected = sweep_selects(&pool, FAILED_DAYS, DELETED_HOURS).await;
assert!(
selected.contains(&deleted),
"a deliberate delete past the window must be reclaimed — this is the leak"
);
assert!(
!selected.contains(&recent),
"a delete inside the window keeps its recovery grace"
);
}
/// The two windows are independent: a failure is retained far longer than a deliberate delete.
///
/// PREVENTS: collapsing them into one. Applying 24h to failures would destroy the recovery window
/// the retained-original fix exists to provide; applying 14 days to deliberate deletes would mean
/// nothing is ever reclaimed during an event.
#[sqlx::test]
async fn the_two_retention_windows_do_not_bleed_into_each_other(pool: PgPool) {
let event_id = seed_event(&pool, "sweep-windows").await;
let user_id = seed_user(&pool, event_id, "Windows").await;
// 48h old: past the deliberate window, nowhere near the failure window.
let failed_recent = seed_aged_upload(
&pool,
event_id,
user_id,
"failed",
Some(48),
"originals/e/f-recent.jpg",
false,
)
.await;
let deleted_same_age = seed_aged_upload(
&pool,
event_id,
user_id,
"done",
Some(48),
"originals/e/d-same.jpg",
false,
)
.await;
// 30 days old: past both.
let failed_old = seed_aged_upload(
&pool,
event_id,
user_id,
"failed",
Some(30 * 24),
"originals/e/f-old.jpg",
false,
)
.await;
let selected = sweep_selects(&pool, FAILED_DAYS, DELETED_HOURS).await;
assert!(
!selected.contains(&failed_recent),
"a 2-day-old compression failure is still inside its 14-day recovery window"
);
assert!(
selected.contains(&deleted_same_age),
"a deliberate delete of the same age is past its 24-hour window"
);
assert!(
selected.contains(&failed_old),
"a 30-day-old failure is past both windows"
);
}
/// Boundary behaviour on both windows.
#[sqlx::test]
async fn retention_windows_are_honoured_at_the_boundary(pool: PgPool) {
let event_id = seed_event(&pool, "sweep-boundary").await;
let user_id = seed_user(&pool, event_id, "Boundary").await;
let cases = [
("failed", 13 * 24, false, "13 days"),
("failed", 15 * 24, true, "15 days"),
("done", 23, false, "23 hours"),
("done", 25, true, "25 hours"),
];
for (status, hours, expected, label) in cases {
let id = seed_aged_upload(
&pool,
event_id,
user_id,
status,
Some(hours),
"originals/e/b.jpg",
false,
)
.await;
assert_eq!(
sweep_selects(&pool, FAILED_DAYS, DELETED_HOURS)
.await
.contains(&id),
expected,
"a {status} upload deleted {label} ago: expected swept={expected}"
);
sqlx::query("DELETE FROM upload WHERE id = $1")
.bind(id)
.execute(&pool)
.await
.expect("clean up");
}
}
/// A row is re-selected until EVERY one of its paths is cleared.
///
/// PREVENTS: two failures at once. The sweep used to clear `original_path` alone, which was right
/// for its only case (a failed compression produces no derivatives) but leaves preview, display and
/// thumbnail on disk the moment it reaches a successfully processed upload — three files per
/// upload, none of them counted in `original_size_bytes`, that nothing else ever removes. And a row
/// whose paths are all cleared must stop coming back, or every hourly tick logs a phantom reclaim
/// forever.
#[sqlx::test]
async fn a_row_is_reselected_until_every_path_is_cleared(pool: PgPool) {
let event_id = seed_event(&pool, "sweep-idempotent").await;
let user_id = seed_user(&pool, event_id, "Idem").await;
let id = seed_aged_upload(
&pool,
event_id,
user_id,
"done",
Some(48),
"originals/e/once.jpg",
true,
)
.await;
assert_eq!(sweep_selects(&pool, FAILED_DAYS, DELETED_HOURS).await, [id]);
// Clearing only the original is NOT enough — the derivatives are still on disk.
sqlx::query("UPDATE upload SET original_path = '' WHERE id = $1")
.bind(id)
.execute(&pool)
.await
.expect("clear original");
assert_eq!(
sweep_selects(&pool, FAILED_DAYS, DELETED_HOURS).await,
[id],
"derivatives left behind must keep the row selected"
);
sqlx::query(
"UPDATE upload SET preview_path = NULL, display_path = NULL, thumbnail_path = NULL
WHERE id = $1",
)
.bind(id)
.execute(&pool)
.await
.expect("clear derivatives");
assert!(
sweep_selects(&pool, FAILED_DAYS, DELETED_HOURS)
.await
.is_empty(),
"a fully swept row must not come back"
);
}
/// The derivative backfill must never resurrect what the sweep just reclaimed.
///
/// PREVENTS: an interaction, not a bug in either piece. The sweep nulls `preview_path`, and
/// `backfill_stale_derivatives` selects on `display_path IS NULL AND preview_path IS NOT NULL` —
/// close enough that a future edit to either could have the backfill re-decode an original that is
/// no longer on disk, on every boot. `deleted_at IS NULL` is what keeps them apart.
#[sqlx::test]
async fn the_backfill_ignores_swept_rows(pool: PgPool) {
let event_id = seed_event(&pool, "sweep-backfill").await;
let user_id = seed_user(&pool, event_id, "Backfill").await;
seed_aged_upload(
&pool,
event_id,
user_id,
"done",
Some(48),
"originals/e/gone.jpg",
true,
)
.await;
// SRC: `services/compression.rs::backfill_stale_derivatives` — the selection, verbatim.
let backfilled: Vec<(Uuid, String, String)> = sqlx::query_as(
"SELECT id, original_path, mime_type FROM upload
WHERE deleted_at IS NULL AND mime_type LIKE 'image/%'
AND original_path IS NOT NULL
AND (
(display_path IS NULL AND preview_path IS NOT NULL)
OR derivatives_rev < $1
)",
)
.bind(1i16)
.fetch_all(&pool)
.await
.expect("backfill query");
assert!(
backfilled.is_empty(),
"a soft-deleted row must be invisible to the backfill, before or after sweeping"
);
}

28
docker-compose.dev.yml
View File

@@ -11,3 +11,31 @@ services:
# is tolerated (warned) rather than rejected.
environment:
APP_ENV: development
# `.env` sets DATABASE_URL to @localhost for the run-backend-natively workflow
# (the db port is published above for that). When the app runs IN a container,
# localhost is the app itself — point it at the `db` service instead. Creds are
# interpolated from .env so nothing is hardcoded.
DATABASE_URL: postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db:5432/${POSTGRES_DB}
# `.env` sets MEDIA_PATH to a HOST path (/home/fabi/EventSnap/media) for the
# run-backend-natively workflow. In a container that path doesn't exist and the
# app user can't create it from `/`, so every upload 500s with EACCES. The media
# volume is mounted at /media (see docker-compose.yml) — point the app there.
MEDIA_PATH: /media
# Recent Docker Compose interpolates env_file values, so the `$` segments of the
# bcrypt ADMIN_PASSWORD_HASH in .env get eaten (the salt reads as an unset var) —
# every admin login then 401s. Re-supply it here with `$` doubled to `$$` so Compose
# passes the literal hash. NOTE: production (docker-compose.yml + .env) has the SAME
# bug — escape the hash as `$$` in .env, or set it via `environment:` there too.
ADMIN_PASSWORD_HASH: "$$2b$$12$$PAteqCNpsbm6d0HTJcywfOaUovjAU.iNVlsL7EDYaRC/z4P/xv7ye"
# Smoke-testing the comment kill-switch: boot-time flag, so it needs a restart
# (not an admin-UI toggle). Backend rejects new comments (403) and the frontend
# hides the whole comment UI. Flip back to true (or drop this line) to restore.
COMMENTS_ENABLED: "false"
caddy:
# The prod caddy service has no env_file, so the Caddyfile's `{$DOMAIN}` expands
# to empty and the site block collapses into a malformed global block. Supply it
# for local dev (from .env → localhost, which Caddy serves with a local self-signed
# cert). NOTE: the prod compose likely needs DOMAIN wired to caddy too.
environment:
DOMAIN: ${DOMAIN}

52
docker-compose.yml
View File

@@ -1,7 +1,20 @@
# Docker's default json-file driver is UNBOUNDED. Those files land on the HOST
# filesystem, outside every `deploy.resources.limits` below — so the container memory
# caps do nothing to stop them. On a single-box deployment the host disk is also where
# the postgres_data and media_data volumes live, and a full disk stops Postgres writing
# WAL, which takes the whole event down. 4 services x 3 x 10m caps the worst case at
# ~120 MiB. Applied to every service via the anchor; a new service must opt in too.
x-logging: &default-logging
driver: json-file
options:
max-size: "10m"
max-file: "3"
services:
db:
image: postgres:16-alpine
restart: unless-stopped
logging: *default-logging
env_file: .env
environment:
POSTGRES_USER: ${POSTGRES_USER}
@@ -17,18 +30,37 @@ services:
deploy:
resources:
limits:
memory: 512M
# 1G, not 512M. DATABASE_MAX_CONNECTIONS defaults to 30 for a ~100-guest event
# (feed polling + SSE + uploads at once), and 30 backends plus Postgres 16's
# default shared_buffers leaves very little headroom at 512M. An OOM here does
# not degrade one feature — it takes the event down, because every request
# path touches the database. Memory is the cheaper knob than shrinking the
# pool back and reintroducing the queueing it was raised to fix.
#
# Raising DATABASE_MAX_CONNECTIONS further means raising this too.
memory: 1G
app:
build:
context: ./backend
dockerfile: Dockerfile
restart: unless-stopped
logging: *default-logging
env_file: .env
environment:
# Default to info. The code fallback in main.rs is info too, but a stock deploy
# sets RUST_LOG nowhere, and this is the layer an operator will actually find when
# they need to raise it for a single event ("RUST_LOG=eventsnap_backend=debug").
RUST_LOG: ${RUST_LOG:-info}
# Activates the production secret guard in config.rs — refuses to boot with
# placeholder JWT_SECRET / ADMIN_PASSWORD_HASH.
APP_ENV: production
# The media volume is mounted at /media (below), so the app MUST write there.
# Pin it here rather than trusting .env: if MEDIA_PATH in .env points elsewhere
# (e.g. a host path used for running the backend natively) the container can't
# create it and every upload 500s with EACCES. `environment` overrides `env_file`,
# so this is authoritative for the container.
MEDIA_PATH: /media
depends_on:
db:
condition: service_healthy
@@ -40,7 +72,11 @@ services:
expose:
- "3000"
healthcheck:
test: ["CMD-SHELL", "wget -q -O- http://localhost:3000/health || exit 1"]
# Use 127.0.0.1, NOT localhost: the app binds IPv4 (0.0.0.0) but `localhost`
# resolves to ::1 (IPv6) first inside the container, so a localhost probe gets
# "connection refused" and the container never turns healthy — which would leave
# Caddy (gated on `condition: service_healthy` below) blocked forever on boot.
test: ["CMD-SHELL", "wget -q -O- http://127.0.0.1:3000/health || exit 1"]
interval: 10s
timeout: 5s
retries: 5
@@ -57,6 +93,7 @@ services:
context: ./frontend
dockerfile: Dockerfile
restart: unless-stopped
logging: *default-logging
env_file: .env
environment:
# adapter-node behind Caddy TLS needs the public origin for CSRF checks on
@@ -67,7 +104,9 @@ services:
expose:
- "3001"
healthcheck:
test: ["CMD-SHELL", "wget -q -O- http://localhost:3001/ >/dev/null 2>&1 || exit 1"]
# 127.0.0.1, not localhost — see the app healthcheck note above (IPv4 bind vs
# ::1 resolution would leave this container permanently unhealthy).
test: ["CMD-SHELL", "wget -q -O- http://127.0.0.1:3001/ >/dev/null 2>&1 || exit 1"]
interval: 10s
timeout: 5s
retries: 5
@@ -80,6 +119,13 @@ services:
caddy:
image: caddy:2-alpine
restart: unless-stopped
logging: *default-logging
environment:
# The Caddyfile's site address is `{$DOMAIN}`, read from THIS container's env.
# Without it, `{$DOMAIN}` expands to empty, the site block collapses, and Caddy
# serves nothing / fails to obtain a TLS cert. `env_file` alone wouldn't help —
# Caddy needs it in `environment`, and this keeps the Caddyfile the single source.
DOMAIN: ${DOMAIN}
ports:
- "80:80"
- "443:443"

9
e2e/Caddyfile.test
View File

@@ -12,10 +12,17 @@
# Mirror prod's security headers (minus HSTS, which is HTTPS-only).
header {
X-Content-Type-Options "nosniff"
X-Frame-Options "DENY"
Referrer-Policy "strict-origin-when-cross-origin"
}
# Mirror prod's export carve-out: the keepsake download targets a hidden
# same-origin iframe, and WebKit enforces XFO before Content-Disposition.
# Two disjoint matchers, not an override — see the comment in ../Caddyfile.
@framable path /api/v1/export/zip /api/v1/export/html
@not_framable not path /api/v1/export/zip /api/v1/export/html
header @framable X-Frame-Options "SAMEORIGIN"
header @not_framable X-Frame-Options "DENY"
reverse_proxy /api/* app:3000
reverse_proxy /media/* app:3000
reverse_proxy /health app:3000

19
e2e/docker-compose.test.yml
View File

@@ -42,11 +42,29 @@ services:
EVENT_NAME: E2E Test Event
APP_PORT: '3000'
MEDIA_PATH: /media
# Exports MUST live outside MEDIA_PATH — see the note on the volume below and
# config.rs::validate. Omitting this left exports on the container's writable
# layer at the /exports default, so the test stack diverged from the prod layout
# it claims to mirror, and export-leak/export-video wrote real archives into
# ephemeral storage.
EXPORT_PATH: /exports
SESSION_EXPIRY_DAYS: '30'
EVENTSNAP_TEST_MODE: '1' # ENABLES /admin/__truncate — never set in prod
RUST_LOG: eventsnap_backend=info,tower_http=warn
volumes:
- media_data:/media
# Separate volume, exactly as in production: a keepsake archive contains every
# photo in the event, so it is kept off the media tree.
- exports_data:/exports
# Mirror production's cap (docker-compose.yml). The test stack having NO memory limit is
# why an unbounded image decode was invisible here: a 99 MP upload that would OOM-kill the
# 1 GiB production container simply succeeded in CI. A test environment more generous than
# production cannot catch a resource bug — the same shape as WebKit being absent from CI
# and /health existing only in Caddyfile.test.
deploy:
resources:
limits:
memory: 1G
expose:
- '3000'
@@ -75,3 +93,4 @@ services:
volumes:
media_data:
exports_data:

92
e2e/fixtures/db.ts
View File

@@ -37,6 +37,52 @@ export const db = {
);
},
/**
* Is this user's account currently PIN-locked?
*
* Distinguishes the two ways /recover can answer 429 — the per-(IP, name) throttle, which
* costs the attacker, and the account lock, which costs the VICTIM. Only the second one is
* weaponizable, so a test asserting "a single IP cannot lock a guest out" has to look at the
* row, not at the status code.
*/
async isPinLocked(userId: string): Promise<boolean> {
return withClient(async (c) => {
const r = await c.query<{ locked: boolean }>(
`SELECT (pin_locked_until IS NOT NULL AND pin_locked_until > NOW()) AS locked
FROM "user" WHERE id = $1`,
[userId]
);
return r.rows[0]?.locked ?? false;
});
},
/**
* Preload the wrong-PIN streak, standing in for failures that arrived from other IPs.
*
* The account lock is deliberately out of reach of any single source, so a test that wants to
* exercise it has to simulate the distributed case rather than hammer from one address.
* `last_failed_pin_at` is set to now so the 15-minute decay does not immediately reset it.
*/
async setFailedPinAttempts(userId: string, attempts: number) {
await withClient((c) =>
c.query(
`UPDATE "user" SET failed_pin_attempts = $2, last_failed_pin_at = NOW() WHERE id = $1`,
[userId, attempts]
)
);
},
/** Current wrong-PIN streak. Decays after 15 minutes — see User::increment_failed_pin. */
async failedPinAttempts(userId: string): Promise<number> {
return withClient(async (c) => {
const r = await c.query<{ failed_pin_attempts: number }>(
`SELECT failed_pin_attempts FROM "user" WHERE id = $1`,
[userId]
);
return r.rows[0]?.failed_pin_attempts ?? 0;
});
},
async expireSession(userId: string) {
await withClient((c) =>
c.query(`UPDATE session SET expires_at = NOW() - interval '1 hour' WHERE user_id = $1`, [
@@ -54,6 +100,27 @@ export const db = {
);
},
async compressionStatus(uploadId: string): Promise<string | null> {
return withClient(async (c) => {
const r = await c.query<{ compression_status: string }>(
`SELECT compression_status FROM upload WHERE id = $1`,
[uploadId]
);
return r.rows[0]?.compression_status ?? null;
});
},
/** Which revision of the derivative pipeline produced this row's preview/display. */
async derivativesRev(uploadId: string): Promise<number | null> {
return withClient(async (c) => {
const r = await c.query<{ derivatives_rev: number }>(
`SELECT derivatives_rev FROM upload WHERE id = $1`,
[uploadId]
);
return r.rows[0]?.derivatives_rev ?? null;
});
},
async countUploadsForUser(userId: string): Promise<number> {
return withClient(async (c) => {
const r = await c.query<{ count: string }>(
@@ -84,6 +151,20 @@ export const db = {
});
},
/**
* Overstate an upload's recorded size.
*
* The keepsake size estimate and the low-disk threshold are pure SQL over
* `original_size_bytes` — no file is read — so this is the lever for driving "the keepsake
* would not fit" without a genuinely full disk. The bytes on disk are unchanged; only the
* accounting the warning reads from moves.
*/
async setUploadSizeBytes(uploadId: string, bytes: number) {
await withClient((c) =>
c.query(`UPDATE upload SET original_size_bytes = $2 WHERE id = $1`, [uploadId, bytes])
);
},
async setExportReleased(slug: string, released: boolean) {
await withClient((c) =>
c.query(`UPDATE event SET export_released_at = $2 WHERE slug = $1`, [
@@ -121,7 +202,8 @@ export const db = {
async fakeExportJob(
eventSlug: string,
type: 'zip' | 'html',
status: 'pending' | 'running' | 'done'
status: 'pending' | 'running' | 'done' | 'failed',
errorMessage: string | null = null
) {
await withClient(async (c) => {
const ev = await c.query<{ id: string; export_epoch: string }>(
@@ -130,11 +212,12 @@ export const db = {
);
if (ev.rows.length === 0) throw new Error(`No event with slug ${eventSlug}`);
await c.query(
`INSERT INTO export_job (event_id, type, status, progress_pct, completed_at, epoch)
VALUES ($1, $2::export_type, $3::export_status, $4, $5, $6)
`INSERT INTO export_job (event_id, type, status, progress_pct, completed_at, epoch,
error_message)
VALUES ($1, $2::export_type, $3::export_status, $4, $5, $6, $7)
ON CONFLICT (event_id, type) DO UPDATE
SET status = EXCLUDED.status, progress_pct = EXCLUDED.progress_pct,
epoch = EXCLUDED.epoch`,
epoch = EXCLUDED.epoch, error_message = EXCLUDED.error_message`,
[
ev.rows[0].id,
type,
@@ -142,6 +225,7 @@ export const db = {
status === 'done' ? 100 : 0,
status === 'done' ? new Date() : null,
ev.rows[0].export_epoch,
errorMessage,
]
);
});

BIN
e2e/fixtures/media/huge-99mp.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 568 KiB

4
e2e/fixtures/media/not-an-image.jpg Normal file
View File

@@ -0,0 +1,4 @@
This is plain text, not an image at all.
This is plain text, not an image at all.
This is plain text, not an image at all.
This is plain text, not an image at all.

BIN
e2e/fixtures/media/portrait-exif6.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 807 B

BIN
e2e/fixtures/media/sample-5s.mp4 Normal file
View File

Binary file not shown.

BIN
e2e/fixtures/media/sample.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 27 KiB

BIN
e2e/fixtures/media/sample.mp4 Normal file
View File

Binary file not shown.

BIN
e2e/fixtures/media/sample2.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 35 KiB

29
e2e/helpers/webkit.ts Normal file
View File

@@ -0,0 +1,29 @@
import { test } from '@playwright/test';
/**
* Skip a test that depends on persisting a Blob/File in IndexedDB when running on
* Playwright's WebKit.
*
* The client upload queue (`frontend/src/lib/upload-queue.ts`) stores the file itself in
* IndexedDB so a backgrounded or reloaded phone can resume the upload. Playwright's Linux
* WebKit build cannot store Blobs in IndexedDB at all — `put()` fails with
* "UnknownError: Error preparing Blob/File data to be stored in object store". Verified to
* be the harness, not the app: a Blob constructed in-page with `new Blob([bytes])` fails
* exactly the same way, while Chromium stores both that and a `setInputFiles` File fine.
* Real iOS Safari supports Blobs in IndexedDB, so this is NOT evidence of a bug on the
* platform these tests exist to protect.
*
* Any test that drives the composer (FAB → UploadSheet → /upload → submit) hits this,
* because `handleSubmit` awaits `addToQueue`, which throws before it can navigate.
*
* This is deliberately narrow. WebKit still runs every API-driven upload test, the whole of
* 01-auth, 03-feed and 06-export — including the keepsake download, which only WebKit can
* meaningfully verify. If Playwright's WebKit ever gains IndexedDB Blob support, delete this
* helper and the four call sites.
*/
export function skipIfNoIdbBlobs(browserName: string) {
test.skip(
browserName === 'webkit',
"Playwright's Linux WebKit cannot store Blobs in IndexedDB (harness limitation, not an iOS one) — the client upload queue can't be exercised there"
);
}

4
e2e/loadtest/.gitignore vendored Normal file
View File

@@ -0,0 +1,4 @@
results/
__pycache__/
*.pyc
*.log

112
e2e/loadtest/README.md Normal file
View File

@@ -0,0 +1,112 @@
# EventSnap load / stress test
Simulates a real event: **~100 guests** joining and uploading **~1000 images** in
bursts (10–20 at a time) spread across a compressed time window, a pool of
**viewers** holding live SSE connections, and **one real browser** on `/diashow`
acting as the showcase display.
## Goal (this harness is tuned for it)
**Validate the shipping config.** We run at the real production defaults —
compression concurrency (`COMPRESSION_WORKER_CONCURRENCY`, default **2**), DB
pool (default **10**), quotas **on** — and answer: _does the app survive the
event, and how far behind real-time does the diashow fall?_
The headline metric is **pipeline latency**: time from an upload succeeding to
its preview being ready (`upload-processed` SSE event) — i.e. _how long until the
photo appears on the diashow_. A backlog that builds is fine; a backlog that
**never drains** is a fail for a live event.
## Methodology: what we change vs. shipping
We **only disable rate limits**. They're per-IP / per-user anti-abuse guards; a
synthetic test from one IP trips them in a way real guests (distinct IPs, phones)
never would — leaving them on would measure the limiter, not the pipeline.
Everything else (compression concurrency, DB pool, quotas) stays at the real
default so the result is honest.
> **Standalone finding to remember:** the shipping `upload_rate_per_hour` default
> is **10**. A real guest uploading a burst of 10–20 photos would be throttled by
> the shipping config too. That's a genuine event-day issue worth surfacing
> separately from this pipeline test.
## Prereqs
- The isolated test stack up: `cd e2e && npm run stack:up` (Caddy on `:3101`,
`EVENTSNAP_TEST_MODE=1`, `/admin/__truncate` live).
- Node 24+ (global `fetch`/`FormData`/`Blob`), Python 3 + Pillow, Docker CLI
access (used for `docker stats` + `docker exec psql` ground-truth sampling).
- `@playwright/test` (already an e2e dep) for the diashow watcher.
## 1. Generate the image pool (once)
Realistic phone-sized JPEGs (~2–4 MB, 12 MP, high entropy). The driver reuses
this pool at random across all 1000 uploads — real load is byte size + decode
cost, not file uniqueness.
```bash
python3 e2e/loadtest/gen-images.py 40 # → /tmp/eventsnap-loadtest/photos
```
~40 images ≈ 120 MB pool; projects to **~3–4 GB** of originals for 1000 uploads
(previews/thumbnails add more). The generator prints the projection; check disk.
## 2. Smoke run first (~1 min)
Proves the wiring — join, upload, SSE correlation, drain, metrics — before the
real thing:
```bash
LT_GUESTS=5 LT_IMAGES=50 LT_WINDOW_SEC=60 node e2e/loadtest/driver.mjs
```
## 3. Full run (~15 min + drain)
Two terminals. Start the showcase display first, then the driver:
```bash
# terminal A — the showcase device
node e2e/loadtest/diashow-watch.mjs
# terminal B — 100 guests / 1000 images / 15-min window (defaults)
node e2e/loadtest/driver.mjs
```
The driver truncates event data first (`LT_TRUNCATE=0` to keep), disables rate
limits, joins guests, opens SSE, runs the burst schedule, then **waits for the
compression backlog to drain** before reporting.
## Output
- Console: live progress every 10 s, then a RESULTS block with pass/fail flags.
- `e2e/loadtest/results/run-<timestamp>.json`: full metrics — upload latency
percentiles, pipeline latency percentiles, drain time, per-status counts, SSE
reconnect/resync counts, and a `docker stats` + DB-connection time series.
- `e2e/loadtest/results/diashow/`: periodic screenshots of the live display.
## What the flags mean
| Flag | Meaning |
| ------------------------- | ---------------------------------------------------------------------------- |
| `✗ 5xx` | server errored under load — hard fail |
| `✗ 507` | quota rejected uploads — disk/quota misconfig for the event size |
| `✗ backlog did not drain` | compression can't keep up even after uploads stop — diashow never catches up |
| `⚠ pipeline p95 > 60s` | photos take >1 min to appear on the diashow at peak |
| `⚠ SSE resyncs` | live consumers lagged the broadcast channel |
## Knobs
All via env (see header of `driver.mjs`): `LT_GUESTS`, `LT_IMAGES`,
`LT_WINDOW_SEC`, `LT_BURST_MIN/MAX`, `LT_BURST_CONC`, `LT_VIEWERS`,
`LT_TRUNCATE`, `LT_DRAIN_TIMEOUT_SEC`, `LT_KEEP_RATELIMITS`, `LT_BASE`,
`LT_APP_CONTAINER`, `LT_DB_CONTAINER`.
To later answer _"what config should I deploy?"_, re-run with a rebuilt stack
that sets `COMPRESSION_WORKER_CONCURRENCY` higher (boot-time env var in
`docker-compose.test.yml`) and compare the pipeline-latency / drain numbers.
## Teardown
```bash
cd e2e && npm run stack:down # wipes volumes (media + db)
```

109
e2e/loadtest/confirm-diashow-fix.mjs Normal file
View File

@@ -0,0 +1,109 @@
// End-to-end confirmation of the diashow SSE fix, reproducing the ORIGINAL failure:
// kiosk opens /diashow directly BEFORE any photos exist, then a guest uploads.
// Pre-fix: stream never opens, display stuck on "Noch keine Beiträge" forever.
// Post-fix: stream opens on mount, the uploaded photo appears live.
import { chromium } from '@playwright/test';
import { readFileSync } from 'node:fs';
const BASE = 'http://localhost:3101';
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const j = (r) => r.json();
const adminLogin = () =>
fetch(`${BASE}/api/v1/admin/login`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ password: 'admin-test-pw' }),
})
.then(j)
.then((b) => b.jwt);
const join = (name) =>
fetch(`${BASE}/api/v1/join`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ display_name: name }),
}).then(j);
async function truncate(admin) {
await fetch(`${BASE}/api/v1/admin/__truncate`, {
method: 'POST',
headers: { Authorization: `Bearer ${admin}` },
});
}
async function upload(jwt) {
const form = new FormData();
form.append(
'file',
new Blob([readFileSync('/tmp/eventsnap-loadtest/photos/photo_000.jpg')], {
type: 'image/jpeg',
}),
'live.jpg'
);
form.append('caption', 'LIVE-PROBE');
const r = await fetch(`${BASE}/api/v1/upload`, {
method: 'POST',
headers: { Authorization: `Bearer ${jwt}` },
body: form,
});
return (await r.json()).id;
}
// Reproduce the failing scenario: empty event, then open /diashow directly.
let admin = await adminLogin();
await truncate(admin);
admin = await adminLogin();
const showcase = await join('Showcase Display');
const guest = await join('Uploading Guest');
const browser = await chromium.launch();
const ctx = await browser.newContext({ viewport: { width: 1280, height: 800 } });
const page = await ctx.newPage();
const streamReqs = [];
page.on('request', (req) => {
if (req.url().includes('/stream')) streamReqs.push(req.url().replace(BASE, ''));
});
await page.goto(`${BASE}/join`, { waitUntil: 'domcontentloaded' });
await page.evaluate((g) => {
localStorage.setItem('eventsnap_jwt', g.jwt);
localStorage.setItem('eventsnap_pin', g.pin);
localStorage.setItem('eventsnap_user_id', g.user_id);
localStorage.setItem('eventsnap_display_name', 'Showcase');
localStorage.setItem('eventsnap_guide_seen', '1');
}, showcase);
// Open /diashow DIRECTLY (never via /feed) on an EMPTY event.
await page.goto(`${BASE}/diashow`, { waitUntil: 'domcontentloaded' });
await sleep(2500);
const emptyBefore = (await page.locator('text=Noch keine Beiträge').count()) > 0;
const streamOpened = streamReqs.length > 0;
console.log(`\n1) direct /diashow on empty event:`);
console.log(` "Noch keine Beiträge" shown: ${emptyBefore} (expected: true)`);
console.log(
` SSE stream opened: ${streamOpened} ${streamOpened ? '(' + streamReqs.join(', ') + ')' : ''} (expected: true — this is the fix)`
);
// Now a guest uploads a photo while the display is open.
console.log(`\n2) guest uploads a photo (display already open)…`);
await upload(guest.jwt);
// Wait for compression → upload-processed SSE → debounced refresh → slide appears.
let appeared = false;
for (let i = 0; i < 15; i++) {
await sleep(1000);
const stillEmpty = (await page.locator('text=Noch keine Beiträge').count()) > 0;
const imgs = await page.locator('img').count();
if (!stillEmpty && imgs > 0) {
appeared = true;
console.log(` photo appeared live after ~${i + 1}s (img rendered, placeholder gone)`);
break;
}
}
if (!appeared) console.log(` ✗ photo did NOT appear within 15s`);
await page.screenshot({ path: 'results/diashow-fix-confirm.png' });
await browser.close();
console.log(`\n════ VERDICT ════`);
console.log(`SSE opens on direct /diashow: ${streamOpened ? 'YES ✓' : 'NO ✗'}`);
console.log(`Live photo appears on showcase: ${appeared ? 'YES ✓' : 'NO ✗'}`);
console.log(streamOpened && appeared ? 'FIX CONFIRMED' : 'FIX NOT confirmed');

105
e2e/loadtest/diashow-watch.mjs Normal file
View File

@@ -0,0 +1,105 @@
#!/usr/bin/env node
/**
* The showcase display. Opens ONE real Chromium browser on /diashow (as a
* logged-in guest) for the duration of a load run, so we validate the real
* SSE → render path a projector/TV would use — not just the protocol.
*
* Run it alongside driver.mjs (separate terminal), started first:
* node diashow-watch.mjs
* # then in another terminal:
* node driver.mjs
*
* It captures:
* - periodic screenshots (so you can eyeball that new photos actually appear)
* - browser console errors / page crashes / SSE disconnects surfaced in console
* - a final count of slides rendered
*
* Env:
* LT_BASE http://localhost:3101
* LT_WATCH_SEC 960 how long to keep the display open
* LT_SHOT_EVERY_SEC 30 screenshot cadence
* LT_OUT_DIR ./results/diashow
*/
import { chromium, devices } from '@playwright/test';
import { mkdirSync } from 'node:fs';
import { fileURLToPath } from 'node:url';
import { dirname, join } from 'node:path';
const __dirname = dirname(fileURLToPath(import.meta.url));
const BASE = process.env.LT_BASE ?? 'http://localhost:3101';
const WATCH_SEC = parseInt(process.env.LT_WATCH_SEC ?? '960', 10);
const SHOT_EVERY = parseInt(process.env.LT_SHOT_EVERY_SEC ?? '30', 10);
const OUT = process.env.LT_OUT_DIR ?? join(__dirname, 'results', 'diashow');
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
async function joinGuest(name) {
const res = await fetch(`${BASE}/api/v1/join`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ display_name: name }),
});
if (res.status !== 201) throw new Error(`join failed ${res.status}`);
return res.json();
}
async function main() {
mkdirSync(OUT, { recursive: true });
const guest = await joinGuest('Showcase Display');
console.log('[diashow] joined showcase guest, launching browser…');
const browser = await chromium.launch();
// A TV/projector is landscape; use a desktop-ish large viewport.
const ctx = await browser.newContext({ viewport: { width: 1920, height: 1080 } });
const page = await ctx.newPage();
let consoleErrors = 0;
page.on('console', (msg) => {
if (msg.type() === 'error') {
consoleErrors++;
console.log('[diashow][console.error]', msg.text().slice(0, 200));
}
});
page.on('pageerror', (e) => console.log('[diashow][pageerror]', String(e).slice(0, 200)));
page.on('crash', () => console.log('[diashow][CRASH] page crashed'));
// Seed auth in localStorage on the origin, then open /diashow.
await page.goto(`${BASE}/join`, { waitUntil: 'domcontentloaded' });
await page.evaluate((g) => {
localStorage.setItem('eventsnap_theme', 'dark');
localStorage.setItem('eventsnap_jwt', g.jwt);
localStorage.setItem('eventsnap_pin', g.pin);
localStorage.setItem('eventsnap_user_id', g.user_id);
localStorage.setItem('eventsnap_display_name', 'Showcase Display');
localStorage.setItem('eventsnap_guide_seen', '1');
}, guest);
await page.goto(`${BASE}/diashow`, { waitUntil: 'domcontentloaded' });
console.log(`[diashow] on /diashow, watching for ${WATCH_SEC}s (shots every ${SHOT_EVERY}s)`);
const start = Date.now();
let shot = 0;
while ((Date.now() - start) / 1000 < WATCH_SEC) {
await sleep(SHOT_EVERY * 1000);
const elapsed = Math.round((Date.now() - start) / 1000);
// best-effort slide count (diashow renders <img>; adjust selector if markup changes)
let imgs = -1;
try {
imgs = await page.locator('img').count();
} catch {
/* ignore */
}
const path = join(OUT, `diashow-t${String(elapsed).padStart(4, '0')}.png`);
await page.screenshot({ path }).catch(() => {});
console.log(`[diashow][t+${elapsed}s] imgs≈${imgs} consoleErrors=${consoleErrors} → ${path}`);
shot++;
}
console.log(`[diashow] done. ${shot} screenshots, ${consoleErrors} console errors total.`);
await browser.close();
}
main().catch((e) => {
console.error('[diashow] failed:', e);
process.exit(1);
});

654
e2e/loadtest/driver.mjs Normal file
View File

@@ -0,0 +1,654 @@
#!/usr/bin/env node
/**
* EventSnap load / stress driver.
*
* Simulates ~100 guests joining an event and uploading ~1000 images in bursts
* (10-20 at a time) spread across a compressed time window, plus a pool of
* viewers holding live SSE connections (like phones with the feed open) and one
* "diashow" connection (the showcase display). Measures the metrics that matter
* for a live event — especially the *pipeline backlog*: how long between an
* upload succeeding and its preview being ready (SSE `upload-processed`), which
* is exactly "how long until the photo shows up on the diashow".
*
* This is an HTTP-level driver on purpose: 100 real browsers would bottleneck
* the test box, not the server. One real browser watches /diashow separately
* (see diashow-watch.mjs).
*
* METHODOLOGY NOTE — what we change vs. the shipping config:
* We ONLY disable rate limits. They are per-IP/per-user anti-abuse guards; a
* synthetic test from one IP would trip them in a way real guests (distinct
* IPs, phones) never would, so leaving them on would measure the rate limiter
* instead of the pipeline. We DELIBERATELY leave compression concurrency,
* DB pool size and quotas at their real defaults — validating those is the
* whole point. (Runtime can't change compression concurrency anyway; it's a
* boot-time env var.)
*
* Related real-world finding to keep in mind: the default upload_rate_per_hour
* is 10, so a real guest uploading a burst of 10-20 would be throttled by the
* SHIPPING config too. That's a genuine event-day issue worth its own report,
* independent of this pipeline test.
*
* Usage:
* node driver.mjs # full run (100 guests / 1000 imgs / 15 min)
* LT_GUESTS=5 LT_IMAGES=50 LT_WINDOW_SEC=60 node driver.mjs # smoke
*
* Env knobs (all optional; defaults target the full run):
* LT_BASE http://localhost:3101 frontend/caddy base URL
* LT_GUESTS 100 number of virtual guests
* LT_IMAGES 1000 total uploads to perform
* LT_WINDOW_SEC 900 spread uploads over this window
* LT_BURST_MIN/MAX 10 / 20 images per burst
* LT_BURST_CONC 3 parallel uploads within one burst (phone-like)
* LT_VIEWERS 20 extra guests holding SSE + polling feed
* LT_PHOTOS_DIR /tmp/eventsnap-loadtest/photos
* LT_TRUNCATE 1 wipe event data before the run
* LT_DRAIN_TIMEOUT_SEC 600 max wait for compression backlog to drain
* LT_APP_CONTAINER e2e-app-1 docker container for CPU/mem sampling
* LT_DB_CONTAINER e2e-db-1 docker container for psql ground-truth
* LT_ADMIN_PW admin-test-pw
* LT_KEEP_RATELIMITS 0 set 1 to NOT disable rate limits
* LT_OUT_DIR ./results
*/
import { readFileSync, readdirSync, writeFileSync, mkdirSync, statSync } from 'node:fs';
import { execFile } from 'node:child_process';
import { promisify } from 'node:util';
import { fileURLToPath } from 'node:url';
import { dirname, join } from 'node:path';
const execFileAsync = promisify(execFile);
const __dirname = dirname(fileURLToPath(import.meta.url));
// ── Config ──────────────────────────────────────────────────────────────────
const cfg = {
base: process.env.LT_BASE ?? 'http://localhost:3101',
guests: int('LT_GUESTS', 100),
images: int('LT_IMAGES', 1000),
windowSec: int('LT_WINDOW_SEC', 900),
burstMin: int('LT_BURST_MIN', 10),
burstMax: int('LT_BURST_MAX', 20),
burstConc: int('LT_BURST_CONC', 3),
viewers: int('LT_VIEWERS', 20),
photosDir: process.env.LT_PHOTOS_DIR ?? '/tmp/eventsnap-loadtest/photos',
truncate: process.env.LT_TRUNCATE !== '0',
drainTimeoutSec: int('LT_DRAIN_TIMEOUT_SEC', 600),
appContainer: process.env.LT_APP_CONTAINER ?? 'e2e-app-1',
dbContainer: process.env.LT_DB_CONTAINER ?? 'e2e-db-1',
adminPw: process.env.LT_ADMIN_PW ?? 'admin-test-pw',
keepRateLimits: process.env.LT_KEEP_RATELIMITS === '1',
outDir: process.env.LT_OUT_DIR ?? join(__dirname, 'results'),
};
function int(name, def) {
const v = process.env[name];
return v === undefined ? def : parseInt(v, 10);
}
const API = `${cfg.base}/api/v1`;
const now = () => Date.now();
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const rand = (min, max) => min + Math.floor(Math.random() * (max - min + 1));
const pick = (arr) => arr[Math.floor(Math.random() * arr.length)];
// ── HTTP helpers ──────────────────────────────────────────────────────────────
async function api(path, { method = 'GET', token, json, expect } = {}) {
const headers = {};
if (token) headers.Authorization = `Bearer ${token}`;
if (json !== undefined) headers['Content-Type'] = 'application/json';
const res = await fetch(`${API}${path}`, {
method,
headers,
body: json !== undefined ? JSON.stringify(json) : undefined,
});
let body;
if (res.status !== 204) {
const text = await res.text();
try {
body = text.length ? JSON.parse(text) : undefined;
} catch {
body = text;
}
}
if (expect && !expect.includes(res.status)) {
throw new Error(`${method} ${path} → ${res.status}: ${JSON.stringify(body)}`);
}
return { status: res.status, body };
}
const adminLogin = () =>
api('/admin/login', { method: 'POST', json: { password: cfg.adminPw }, expect: [200] }).then(
(r) => r.body.jwt
);
const joinGuest = (name) =>
api('/join', { method: 'POST', json: { display_name: name }, expect: [201] }).then((r) => r.body);
const patchConfig = (adminJwt, patch) =>
api('/admin/config', { method: 'PATCH', token: adminJwt, json: patch, expect: [204] });
const getConfig = (adminJwt) => api('/admin/config', { token: adminJwt }).then((r) => r.body);
const truncate = (adminJwt) =>
api('/admin/__truncate', { method: 'POST', token: adminJwt, expect: [204] });
const CAPTIONS = [
'Was für ein magischer Tag 💍',
'Der erste Tanz 🕺',
'Prost! 🥂',
'Die Torte 🍰',
'Feuerwerk 🎆',
'Beste Freunde 💕',
'Was für eine Stimmung! 🎉',
'Details 🌸',
'Sonnenuntergang 🌅',
'Tanzfläche brennt 🔥',
null,
null,
null,
];
const TAGS = ['hochzeit', 'liebe', 'party', 'tanzen', 'natur', 'feier', 'freunde', 'dessert'];
async function uploadPhoto(jwt, file, buf) {
const form = new FormData();
form.append('file', new Blob([buf], { type: 'image/jpeg' }), 'photo.jpg');
const cap = pick(CAPTIONS);
if (cap) form.append('caption', cap);
form.append('hashtags', `${pick(TAGS)},${pick(TAGS)}`);
const t0 = now();
const res = await fetch(`${API}/upload`, {
method: 'POST',
headers: { Authorization: `Bearer ${jwt}` },
body: form,
});
const t1 = now();
let id, errText;
if (res.status === 201) {
id = (await res.json()).id;
} else {
errText = (await res.text()).slice(0, 200);
}
return { status: res.status, id, ms: t1 - t0, endTs: t1, bytes: buf.length, errText };
}
// ── SSE client (auto-reconnecting) ────────────────────────────────────────────
class SseClient {
constructor(jwt, label, onEvent) {
this.jwt = jwt;
this.label = label;
this.onEvent = onEvent;
this.stop = false;
this.reconnects = 0;
this.resyncs = 0;
this.controller = null;
}
start() {
this._loop();
return this;
}
async _loop() {
while (!this.stop) {
try {
const tkt = await api('/stream/ticket', { method: 'POST', token: this.jwt, expect: [200] });
this.controller = new AbortController();
const res = await fetch(`${API}/stream?ticket=${tkt.body.ticket}`, {
headers: { Accept: 'text/event-stream' },
signal: this.controller.signal,
});
if (!res.ok || !res.body) throw new Error(`stream ${res.status}`);
const reader = res.body.getReader();
const dec = new TextDecoder();
let buf = '';
while (!this.stop) {
const { value, done } = await reader.read();
if (done) break;
buf += dec.decode(value, { stream: true });
let idx;
while ((idx = buf.indexOf('\n\n')) !== -1) {
const frame = buf.slice(0, idx);
buf = buf.slice(idx + 2);
this._parseFrame(frame);
}
}
} catch (e) {
if (this.stop) return;
this.reconnects++;
await sleep(500 + Math.random() * 500);
}
}
}
_parseFrame(frame) {
let event = 'message';
let data = '';
for (const line of frame.split('\n')) {
if (line.startsWith('event:')) event = line.slice(6).trim();
else if (line.startsWith('data:')) data += line.slice(5).trim();
}
if (event === 'resync') this.resyncs++;
if (!data) return;
let payload;
try {
payload = JSON.parse(data);
} catch {
payload = data;
}
this.onEvent(event, payload, this.label);
}
close() {
this.stop = true;
try {
this.controller?.abort();
} catch {
/* noop */
}
}
}
// ── Resource sampler (docker stats + psql ground truth) ───────────────────────
async function sampleResources() {
const out = { ts: now() };
try {
const { stdout } = await execFileAsync('docker', [
'stats',
'--no-stream',
'--format',
'{{.Name}}|{{.CPUPerc}}|{{.MemUsage}}',
cfg.appContainer,
cfg.dbContainer,
]);
out.docker = stdout.trim();
} catch (e) {
out.dockerErr = String(e).slice(0, 120);
}
try {
const { stdout } = await psql(
`select count(*) from pg_stat_activity where datname='eventsnap_test'`
);
out.dbConns = parseInt(stdout.trim(), 10);
} catch {
/* ignore */
}
return out;
}
function psql(sql) {
return execFileAsync('docker', [
'exec',
cfg.dbContainer,
'psql',
'-U',
'eventsnap_test',
'-d',
'eventsnap_test',
'-tAc',
sql,
]);
}
async function compressionCounts() {
try {
const { stdout } = await psql(
`select compression_status, count(*) from upload where deleted_at is null group by 1`
);
const map = {};
for (const line of stdout.trim().split('\n')) {
if (!line) continue;
const [status, n] = line.split('|');
map[status] = parseInt(n, 10);
}
return map;
} catch (e) {
return { error: String(e).slice(0, 120) };
}
}
// ── Stats helpers ─────────────────────────────────────────────────────────────
function pct(sorted, p) {
if (!sorted.length) return null;
const i = Math.min(sorted.length - 1, Math.floor((p / 100) * sorted.length));
return sorted[i];
}
function summarize(nums) {
if (!nums.length) return null;
const s = [...nums].sort((a, b) => a - b);
const sum = s.reduce((a, b) => a + b, 0);
return {
n: s.length,
min: s[0],
max: s[s.length - 1],
mean: Math.round(sum / s.length),
p50: pct(s, 50),
p95: pct(s, 95),
p99: pct(s, 99),
};
}
// ── Image pool ────────────────────────────────────────────────────────────────
function loadPool() {
let files;
try {
files = readdirSync(cfg.photosDir).filter((f) => f.endsWith('.jpg'));
} catch {
files = [];
}
if (!files.length) {
console.error(
`\n✗ No images in ${cfg.photosDir}. Run first:\n` +
` LT_PHOTOS_DIR=${cfg.photosDir} python3 ${join(__dirname, 'gen-images.py')}\n`
);
process.exit(1);
}
const pool = files.map((f) => {
const p = join(cfg.photosDir, f);
return { buf: readFileSync(p), size: statSync(p).size, name: f };
});
return pool;
}
// ── Burst schedule ────────────────────────────────────────────────────────────
// Build bursts until we reach the image target, assign each to a random guest at
// a random time in the window. Naturally gives some guests several bursts and
// some none — like a real event. Guests all join before their first burst.
function buildSchedule() {
const bursts = [];
let remaining = cfg.images;
while (remaining > 0) {
const size = Math.min(remaining, rand(cfg.burstMin, cfg.burstMax));
bursts.push({
guest: rand(0, cfg.guests - 1),
atMs: Math.floor(Math.random() * cfg.windowSec * 1000),
size,
});
remaining -= size;
}
bursts.sort((a, b) => a.atMs - b.atMs);
return bursts;
}
// ── Main ──────────────────────────────────────────────────────────────────────
async function main() {
console.log('━'.repeat(72));
console.log('EventSnap load driver');
console.log(
` target: ${cfg.guests} guests, ${cfg.images} images, ` +
`bursts ${cfg.burstMin}-${cfg.burstMax}, window ${cfg.windowSec}s, ${cfg.viewers} viewers`
);
console.log(` base: ${cfg.base}`);
console.log('━'.repeat(72));
const pool = loadPool();
const avgMb = pool.reduce((a, p) => a + p.size, 0) / pool.length / 1e6;
console.log(
`[pool] ${pool.length} images, avg ${avgMb.toFixed(2)} MB, ` +
`projected originals ~${((avgMb * cfg.images) / 1000).toFixed(1)} GB`
);
// Preflight: disk headroom
try {
const { stdout } = await execFileAsync('df', ['-BG', '--output=avail', '/']);
console.log(`[preflight] disk avail:${stdout.trim().split('\n').pop().trim()}`);
} catch {
/* ignore */
}
// Admin: reset + config
const admin = await adminLogin();
const cfgBefore = await getConfig(admin);
if (cfg.truncate) {
console.log('[setup] truncating event data…');
await truncate(admin);
}
const admin2 = await adminLogin();
if (!cfg.keepRateLimits) {
console.log('[setup] disabling rate limits (methodology: isolate pipeline, not limiter)');
await patchConfig(admin2, {
rate_limits_enabled: 'false',
upload_rate_enabled: 'false',
feed_rate_enabled: 'false',
join_rate_enabled: 'false',
recover_rate_enabled: 'false',
});
}
console.log(
`[setup] compression concurrency / quotas left at SHIPPING defaults ` +
`(compression_worker_concurrency is a boot env var, not runtime)`
);
// Metrics stores
const uploads = []; // {status, ms, endTs, bytes, id, guest}
const processedAt = new Map(); // upload_id -> ts of upload-processed
const uploadEndTs = new Map(); // upload_id -> endTs
const errorEvents = []; // upload-error payloads
const resources = [];
let newUploadEvents = 0;
// Join guests (staggered, small concurrency)
console.log(`[join] joining ${cfg.guests} guests…`);
const accounts = new Array(cfg.guests);
const joinConc = 10;
for (let i = 0; i < cfg.guests; i += joinConc) {
await Promise.all(
Array.from({ length: Math.min(joinConc, cfg.guests - i) }, (_, k) =>
joinGuest(`LoadGuest ${i + k}`).then((a) => (accounts[i + k] = a))
)
);
}
console.log('[join] done');
// Live connections: 1 diashow + N viewers
const onEvent = (event, payload) => {
if (event === 'new-upload') newUploadEvents++;
else if (event === 'upload-processed' && payload?.upload_id) {
if (!processedAt.has(payload.upload_id)) processedAt.set(payload.upload_id, now());
} else if (event === 'upload-error' && payload?.upload_id) {
errorEvents.push(payload);
}
};
const sseClients = [];
sseClients.push(new SseClient(accounts[0].jwt, 'diashow', onEvent).start());
for (let i = 0; i < Math.min(cfg.viewers, cfg.guests); i++) {
sseClients.push(new SseClient(accounts[i].jwt, `viewer${i}`, onEvent).start());
}
console.log(`[sse] opened ${sseClients.length} live connections (1 diashow + viewers)`);
// Viewers also poll the feed periodically (viewing load)
let feedPolls = 0;
const feedPoller = setInterval(async () => {
const g = accounts[rand(0, Math.min(cfg.viewers, cfg.guests) - 1)];
try {
await api('/feed', { token: g.jwt });
feedPolls++;
} catch {
/* ignore */
}
}, 1500);
// Resource sampler
const sampler = setInterval(async () => resources.push(await sampleResources()), 5000);
resources.push(await sampleResources());
// Run the burst schedule
const schedule = buildSchedule();
console.log(
`[run] ${schedule.length} bursts scheduled over ${cfg.windowSec}s ` +
`(≈${(cfg.images / (cfg.windowSec / 60)).toFixed(0)} img/min avg). Starting…`
);
const startTs = now();
let done = 0;
const runBurst = async (burst) => {
const jwt = accounts[burst.guest].jwt;
const items = Array.from({ length: burst.size }, () => pick(pool));
// upload with phone-like small concurrency inside the burst
for (let i = 0; i < items.length; i += cfg.burstConc) {
const chunk = items.slice(i, i + cfg.burstConc);
const results = await Promise.all(chunk.map((it) => uploadPhoto(jwt, it.name, it.buf)));
for (const r of results) {
uploads.push({ ...r, guest: burst.guest });
if (r.id) uploadEndTs.set(r.id, r.endTs);
done++;
}
}
};
const timers = [];
const burstPromises = [];
for (const burst of schedule) {
timers.push(
setTimeout(() => {
burstPromises.push(runBurst(burst));
}, burst.atMs)
);
}
// progress ticker
const ticker = setInterval(() => {
const elapsed = ((now() - startTs) / 1000).toFixed(0);
const ok = uploads.filter((u) => u.status === 201).length;
console.log(
`[t+${elapsed}s] uploads ${done}/${cfg.images} (ok ${ok}), ` +
`processed ${processedAt.size}, new-upload evts ${newUploadEvents}, ` +
`feedPolls ${feedPolls}`
);
}, 10000);
// wait for the window to elapse, then for all scheduled bursts to finish
await sleep(cfg.windowSec * 1000 + 500);
await Promise.all(burstPromises);
clearInterval(ticker);
console.log(
`[run] all bursts issued. uploaded ${done}, ok ${uploads.filter((u) => u.status === 201).length}`
);
// Drain: wait for compression backlog to clear (SSE processed ⊇ successful ids)
console.log('[drain] waiting for compression backlog to clear…');
const drainStart = now();
const successIds = new Set(uploads.filter((u) => u.id).map((u) => u.id));
let lastLog = 0;
let drainReason = 'timeout';
while (now() - drainStart < cfg.drainTimeoutSec * 1000) {
// SSE view (what the diashow actually "sees")
const pendingSse = [...successIds].filter((id) => !processedAt.has(id)).length;
if (pendingSse === 0) {
console.log('[drain] backlog cleared (all upload-processed events received)');
drainReason = 'sse-complete';
break;
}
// DB ground truth — authoritative even if an SSE reconnect dropped events
const counts = await compressionCounts();
const dbPending = Object.entries(counts)
.filter(([s]) => s !== 'done' && s !== 'error')
.reduce((a, [, n]) => a + n, 0);
if (!counts.error && dbPending === 0) {
console.log(`[drain] backlog cleared per DB (db status: ${JSON.stringify(counts)})`);
drainReason = 'db-complete';
break;
}
if (now() - lastLog > 5000) {
console.log(
`[drain] pending(sse) ${pendingSse}, pending(db) ${dbPending} — db: ${JSON.stringify(counts)}`
);
lastLog = now();
}
await sleep(1000);
}
const drainMs = now() - drainStart;
// Stop background work
clearInterval(sampler);
clearInterval(feedPoller);
timers.forEach(clearTimeout);
resources.push(await sampleResources());
const finalCounts = await compressionCounts();
await sleep(200);
sseClients.forEach((c) => c.close());
// ── Build report ────────────────────────────────────────────────────────────
const byStatus = {};
for (const u of uploads) byStatus[u.status] = (byStatus[u.status] ?? 0) + 1;
const okUploads = uploads.filter((u) => u.status === 201);
const uploadLatency = summarize(okUploads.map((u) => u.ms));
const pipelineLatency = summarize(
[...processedAt.entries()]
.filter(([id]) => uploadEndTs.has(id))
.map(([id, ts]) => ts - uploadEndTs.get(id))
);
const totalReconnects = sseClients.reduce((a, c) => a + c.reconnects, 0);
const totalResyncs = sseClients.reduce((a, c) => a + c.resyncs, 0);
const report = {
config: cfg,
startedAt: new Date(startTs).toISOString(),
durationSec: Math.round((now() - startTs) / 1000),
totals: {
uploadsAttempted: uploads.length,
uploadsOk: okUploads.length,
byStatus,
newUploadEvents,
processedEvents: processedAt.size,
uploadErrorEvents: errorEvents.length,
feedPolls,
},
uploadLatencyMs: uploadLatency,
pipelineLatencyMs: pipelineLatency,
drain: {
ms: drainMs,
cleared: drainReason !== 'timeout',
reason: drainReason,
ssePending: [...successIds].filter((id) => !processedAt.has(id)).length,
finalDbCounts: finalCounts,
},
sse: {
connections: sseClients.length,
reconnects: totalReconnects,
resyncs: totalResyncs,
},
resources,
rateLimitsDisabled: !cfg.keepRateLimits,
configBefore: cfgBefore,
};
mkdirSync(cfg.outDir, { recursive: true });
const stamp = new Date(startTs).toISOString().replace(/[:.]/g, '-');
const outPath = join(cfg.outDir, `run-${stamp}.json`);
writeFileSync(outPath, JSON.stringify(report, null, 2));
// ── Print verdict ─────────────────────────────────────────────────────────
console.log('\n' + '━'.repeat(72));
console.log('RESULTS');
console.log('━'.repeat(72));
console.log(
`uploads: ${okUploads.length}/${uploads.length} ok — byStatus ${JSON.stringify(byStatus)}`
);
console.log(`upload latency ms: ${JSON.stringify(uploadLatency)}`);
console.log(`pipeline latency ms (upload→preview ready): ${JSON.stringify(pipelineLatency)}`);
console.log(`backlog drain: ${(drainMs / 1000).toFixed(1)}s, cleared=${report.drain.cleared}`);
console.log(`final db compression status: ${JSON.stringify(finalCounts)}`);
console.log(
`sse: ${sseClients.length} conns, ${totalReconnects} reconnects, ${totalResyncs} resyncs`
);
console.log(`\nfull report → ${outPath}`);
// Heuristic pass/fail flags (validate shipping config)
const flags = [];
const err5xx = Object.entries(byStatus)
.filter(([s]) => +s >= 500)
.reduce((a, [, n]) => a + n, 0);
if (err5xx > 0) flags.push(`✗ ${err5xx} server errors (5xx)`);
if (byStatus['507']) flags.push(`✗ ${byStatus['507']} quota rejections (507)`);
if (byStatus['413']) flags.push(`⚠ ${byStatus['413']} too-large (413)`);
if (byStatus['429'])
flags.push(`⚠ ${byStatus['429']} rate-limited (429) — unexpected with limits off`);
if (!report.drain.cleared) flags.push(`✗ backlog did NOT drain within ${cfg.drainTimeoutSec}s`);
if (totalResyncs > sseClients.length) flags.push(`⚠ ${totalResyncs} SSE resyncs (consumer lag)`);
if (pipelineLatency && pipelineLatency.p95 > 60000)
flags.push(`⚠ pipeline p95 ${(pipelineLatency.p95 / 1000).toFixed(0)}s (diashow lags >1min)`);
console.log('\nflags:');
if (flags.length) flags.forEach((f) => console.log(' ' + f));
else console.log(' ✓ no red flags — default config handled the load');
console.log('━'.repeat(72));
}
main().catch((e) => {
console.error('\n✗ driver failed:', e);
process.exit(1);
});

146
e2e/loadtest/gen-images.py Normal file
View File

@@ -0,0 +1,146 @@
#!/usr/bin/env python3
"""
Generate a pool of realistic phone-sized JPEGs for the EventSnap load test.
The stress test uploads ~1000 images, but they don't need to be 1000 unique
files — real load comes from realistic *byte size* and *decode cost*, which
drive bandwidth, the compression/preview pipeline (decode + 800x800 resize),
disk usage and the dynamic storage quota. So we generate a modest POOL of
distinct, high-entropy images (~2-4 MB, 12 MP, like a phone camera) and the
driver reuses them at random across the 1000 uploads.
High entropy matters: a flat gradient compresses to almost nothing and would
under-stress both the network and the JPEG decoder. We blend random noise over
a colorful gradient + big shapes so the files land in a realistic size band,
then tune JPEG quality per-image to hit the target size.
Output: $LT_PHOTOS_DIR (default /tmp/eventsnap-loadtest/photos)
Usage: python3 gen-images.py [COUNT] (default 40)
"""
import os
import sys
import math
import random
from PIL import Image, ImageDraw, ImageFont
OUT_DIR = os.environ.get("LT_PHOTOS_DIR", "/tmp/eventsnap-loadtest/photos")
COUNT = int(sys.argv[1]) if len(sys.argv) > 1 else 40
# Target JPEG size band (bytes). Typical modern phone photo.
TARGET_MIN = 2_000_000
TARGET_MAX = 4_500_000
# 12 MP-ish, both orientations (phones shoot portrait and landscape).
SIZES = [(4032, 3024), (3024, 4032)]
PALETTES = [
[(250, 245, 235), (212, 175, 55), (120, 90, 30)], # champagne / gold
[(245, 244, 242), (190, 190, 198), (90, 92, 100)], # silver / pearl
[(255, 250, 240), (240, 200, 160), (170, 110, 70)], # warm sunset
[(235, 240, 248), (140, 170, 210), (40, 70, 120)], # cool blue hour
[(248, 240, 245), (210, 150, 180), (110, 50, 90)], # rose dusk
[(240, 248, 242), (150, 200, 170), (40, 110, 80)], # garden green
]
def lerp(a, b, t):
return tuple(int(a[i] + (b[i] - a[i]) * t) for i in range(3))
def gradient(w, h, palette, rng):
"""Diagonal 3-stop gradient base."""
base = Image.new("RGB", (w, h))
px = base.load()
ang = rng.uniform(0, math.pi)
dx, dy = math.cos(ang), math.sin(ang)
# precompute per-column/row projection for speed
maxproj = abs(dx) * w + abs(dy) * h
for y in range(h):
for x in range(0, w, 4): # step 4 then fill — good enough, much faster
t = (dx * x + dy * y) / maxproj
t = min(1.0, max(0.0, t + rng.uniform(-0.02, 0.02)))
if t < 0.5:
c = lerp(palette[0], palette[1], t * 2)
else:
c = lerp(palette[1], palette[2], (t - 0.5) * 2)
for k in range(4):
if x + k < w:
px[x + k, y] = c
return base
def add_shapes(img, rng):
d = ImageDraw.Draw(img, "RGBA")
w, h = img.size
for _ in range(rng.randint(6, 14)):
x0 = rng.randint(-w // 5, w)
y0 = rng.randint(-h // 5, h)
r = rng.randint(w // 12, w // 3)
col = (rng.randint(0, 255), rng.randint(0, 255), rng.randint(0, 255), rng.randint(20, 90))
if rng.random() < 0.5:
d.ellipse([x0, y0, x0 + r, y0 + r], fill=col)
else:
d.rectangle([x0, y0, x0 + r, y0 + int(r * rng.uniform(0.4, 1.6))], fill=col)
return img
def noisy(w, h, rng):
"""Full-resolution RGB noise from urandom — maximum entropy."""
return Image.frombytes("RGB", (w, h), os.urandom(w * h * 3))
def label(img, idx, rng):
d = ImageDraw.Draw(img)
txt = f"EventSnap load #{idx:03d}"
try:
font = ImageFont.load_default(size=64)
except TypeError:
font = ImageFont.load_default()
d.text((60, 60), txt, fill=(255, 255, 255), font=font)
d.text((62, 62), txt, fill=(0, 0, 0), font=font) # cheap shadow offset
def encode_to_band(img, path, rng):
"""Try qualities high→low until the file lands under TARGET_MAX; keep the
first that also clears TARGET_MIN if possible."""
best = None
for q in (92, 88, 84, 80, 76, 72):
img.save(path, "JPEG", quality=q, optimize=False)
size = os.path.getsize(path)
best = (q, size)
if size <= TARGET_MAX:
if size >= TARGET_MIN:
return q, size
# under the band — noise alpha likely too low; accept anyway at high q
return q, size
return best # even q72 too big; accept the smallest we made
def main():
os.makedirs(OUT_DIR, exist_ok=True)
rng = random.Random(20260718) # deterministic pool
total_bytes = 0
print(f"[gen] writing {COUNT} images to {OUT_DIR}")
for i in range(COUNT):
w, h = rng.choice(SIZES)
palette = rng.choice(PALETTES)
base = gradient(w, h, palette, rng)
base = add_shapes(base, rng)
# blend noise to inject entropy -> realistic JPEG size
alpha = rng.uniform(0.28, 0.42)
base = Image.blend(base, noisy(w, h, rng), alpha)
label(base, i, rng)
path = os.path.join(OUT_DIR, f"photo_{i:03d}.jpg")
q, size = encode_to_band(base, path, rng)
total_bytes += size
print(f" photo_{i:03d}.jpg {w}x{h} q{q} {size/1_000_000:.2f} MB")
avg = total_bytes / COUNT
print(f"[gen] done. {COUNT} images, avg {avg/1_000_000:.2f} MB, "
f"pool total {total_bytes/1_000_000:.1f} MB")
print(f"[gen] projected for 1000 uploads (originals only): "
f"~{avg*1000/1_000_000_000:.1f} GB")
if __name__ == "__main__":
main()

13
e2e/page-objects/export-page.ts
View File

@@ -37,12 +37,19 @@ export class ExportPage {
/**
* The "Download" button inside the card whose heading is `heading`.
*
* Scoped to the card element (`div.rounded-xl`) rather than "any div containing the
* heading" — the latter also matches the page wrapper, which contains BOTH cards' buttons.
* Scoped to the card element rather than "any div containing the heading" — the latter
* also matches the page wrapper, which contains BOTH cards' buttons.
*
* The scope class is `div.card` (see the ZIP/HTML cards in
* frontend/src/routes/export/+page.svelte). It was previously `div.rounded-xl`, which
* matched NOTHING: `.card` is a Tailwind `@apply` component class
* (frontend/src/lib/styles/components.css) so the DOM only ever carries `class="card p-5"`
* — and the utility it applies is `rounded-2xl` anyway. Both card-scoped locators were
* therefore dead, which is why the "shows enabled download buttons" test was red.
*/
private cardButton(heading: string): Locator {
return this.page
.locator('div.rounded-xl')
.locator('div.card')
.filter({ has: this.page.getByRole('heading', { name: heading, exact: true }) })
.getByRole('button', { name: 'Download', exact: true });
}

14
e2e/playwright.config.ts
View File

@@ -146,8 +146,18 @@ export default defineConfig({
// that on `@smoke` — which exists on exactly two specs — meant the entire iOS guarantee was
// one happy path and one join test. Every other UA here is a secondary browser and a smoke
// check is proportionate; WebKit is not. Give it the core journeys the guest actually walks:
// join/recover, upload, and browse the feed.
testMatch: ['**/__smoke/**', '**/01-auth/**', '**/02-upload/**', '**/03-feed/**'],
// join/recover, upload, browse the feed — and take the keepsake home.
//
// 06-export is here because WebKit is the ONLY engine that enforces X-Frame-Options on the
// hidden download iframe. Excluding it is what let a site-wide `XFO: DENY` ship a keepsake
// download that silently did nothing on iOS. See 06-export/download-iframe.spec.ts.
testMatch: [
'**/__smoke/**',
'**/01-auth/**',
'**/02-upload/**',
'**/03-feed/**',
'**/06-export/**',
],
},
{
name: 'firefox-android',

255
e2e/shots.mjs Normal file
View File

@@ -0,0 +1,255 @@
// One-off: seed a realistic feed, then capture mobile screenshots (light + dark).
// Run from the e2e dir so @playwright/test and pg resolve.
import { chromium, devices } from '@playwright/test';
import { Client } from 'pg';
import { readFileSync, mkdirSync } from 'node:fs';
const BASE = 'http://localhost:3101';
const PHOTOS = '/tmp/eventsnap-shots/photos';
const OUT = '/tmp/eventsnap-shots';
mkdirSync(OUT, { recursive: true });
const api = (path, opts = {}) =>
fetch(`${BASE}/api/v1${path}`, opts).then(async (r) => ({
status: r.status,
body: await r.text().then((t) => {
try {
return JSON.parse(t);
} catch {
return t;
}
}),
}));
const authHeaders = (jwt, json = true) => ({
Authorization: `Bearer ${jwt}`,
...(json ? { 'Content-Type': 'application/json' } : {}),
});
async function adminLogin() {
const r = await api('/admin/login', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ password: 'admin-test-pw' }),
});
return r.body.jwt;
}
async function joinGuest(name) {
const r = await api('/join', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ display_name: name }),
});
if (r.status !== 201)
throw new Error('join failed ' + name + ' ' + r.status + ' ' + JSON.stringify(r.body));
return r.body; // {jwt,pin,user_id}
}
async function upload(jwt, file, caption, hashtags) {
const form = new FormData();
form.append('file', new Blob([readFileSync(file)], { type: 'image/jpeg' }), 'photo.jpg');
if (caption) form.append('caption', caption);
if (hashtags) form.append('hashtags', hashtags);
const r = await fetch(`${BASE}/api/v1/upload`, {
method: 'POST',
headers: authHeaders(jwt, false),
body: form,
});
if (r.status !== 201) throw new Error('upload failed ' + r.status + ' ' + (await r.text()));
return (await r.json()).id;
}
async function like(jwt, id) {
await fetch(`${BASE}/api/v1/upload/${id}/like`, { method: 'POST', headers: authHeaders(jwt) });
}
async function comment(jwt, id, body) {
await fetch(`${BASE}/api/v1/upload/${id}/comments`, {
method: 'POST',
headers: authHeaders(jwt),
body: JSON.stringify({ body }),
});
}
const pg = () =>
new Client({
host: 'localhost',
port: 55432,
user: 'eventsnap_test',
password: 'eventsnap_test',
database: 'eventsnap_test',
});
async function truncate(adminJwt) {
await fetch(`${BASE}/api/v1/admin/__truncate`, {
method: 'POST',
headers: authHeaders(adminJwt),
});
}
async function patchConfig(adminJwt, patch) {
await fetch(`${BASE}/api/v1/admin/config`, {
method: 'PATCH',
headers: authHeaders(adminJwt),
body: JSON.stringify(patch),
});
}
// ---- SEED ----
console.log('[seed] admin login + reset');
let admin = await adminLogin();
await truncate(admin);
admin = await adminLogin();
await patchConfig(admin, {
rate_limits_enabled: 'false',
upload_rate_enabled: 'false',
feed_rate_enabled: 'false',
export_rate_enabled: 'false',
join_rate_enabled: 'false',
quota_enabled: 'false',
storage_quota_enabled: 'false',
upload_count_quota_enabled: 'false',
});
const guests = [
'Anna Bauer',
'Lukas Weber',
'Mia Schulz',
'Jonas Fischer',
'Emma Wagner',
'Ben Hoffmann',
'Sophie Klein',
'Paul Richter',
];
const accounts = {};
for (const g of guests) accounts[g] = await joinGuest(g);
console.log('[seed] joined', guests.length, 'guests');
const posts = [
['Anna Bauer', 'photo_00.jpg', 'Was für ein magischer Tag 💍✨', 'hochzeit,liebe'],
['Lukas Weber', 'photo_01.jpg', 'Der erste Tanz 🕺💃', 'party,tanzen'],
['Mia Schulz', 'photo_02.jpg', 'Sonnenuntergang am See 🌅', 'natur,abend'],
['Jonas Fischer', 'photo_03.jpg', 'Prost auf das Brautpaar! 🥂', 'feier,freunde'],
['Emma Wagner', 'photo_04.jpg', 'Die Torte war ein Traum 🍰', 'dessert,liebe'],
['Ben Hoffmann', 'photo_05.jpg', 'Feuerwerk zum Abschluss 🎆', 'party,abend'],
['Sophie Klein', 'photo_06.jpg', 'Beste Freunde für immer 💕', 'freunde,feier'],
['Paul Richter', 'photo_07.jpg', 'Was für eine Stimmung! 🎉', 'party'],
['Anna Bauer', 'photo_08.jpg', 'Details, die zählen 🌸', 'hochzeit'],
['Mia Schulz', 'photo_09.jpg', 'Tanzfläche brennt 🔥', 'tanzen,party'],
];
const ids = [];
for (const [who, f, cap, tags] of posts) {
const id = await upload(accounts[who].jwt, `${PHOTOS}/${f}`, cap, tags);
ids.push(id);
}
console.log('[seed] uploaded', ids.length, 'photos');
// Likes: distribute varied counts
const likeMatrix = [7, 3, 12, 5, 9, 4, 6, 2, 8, 5];
for (let i = 0; i < ids.length; i++) {
const n = Math.min(likeMatrix[i], guests.length);
for (let j = 0; j < n; j++) await like(accounts[guests[j]].jwt, ids[i]);
}
// Comments on a few
await comment(accounts['Lukas Weber'].jwt, ids[0], 'Wunderschön! 😍');
await comment(accounts['Mia Schulz'].jwt, ids[0], 'Der Moment war perfekt.');
await comment(accounts['Anna Bauer'].jwt, ids[2], 'Traumhaft 🌅');
await comment(accounts['Ben Hoffmann'].jwt, ids[1], 'Was ein Abend!');
console.log('[seed] likes + comments done');
// Make one guest a host so /host renders populated
await fetch(`${BASE}/api/v1/host/users/${accounts['Anna Bauer'].user_id}/role`, {
method: 'PATCH',
headers: authHeaders(admin),
body: JSON.stringify({ role: 'host' }),
});
// Wait for compression to finish so previews render
const c = pg();
await c.connect();
for (let t = 0; t < 40; t++) {
const r = await c.query(
`SELECT COUNT(*)::int AS n FROM upload WHERE compression_status <> 'done' AND deleted_at IS NULL`
);
if (r.rows[0].n === 0) {
console.log('[seed] compression done');
break;
}
await new Promise((res) => setTimeout(res, 500));
}
await c.end();
// ---- SCREENSHOTS ----
const device = devices['Pixel 7'];
const anna = accounts['Anna Bauer']; // host
const shot = async (label, theme, who, route, prep) => {
const ctx = await browser.newContext({ ...device });
const page = await ctx.newPage();
// Seed localStorage on the origin
await page.goto(`${BASE}/join`, { waitUntil: 'domcontentloaded' });
await page.evaluate(
({ jwt, pin, uid, name, theme, mode }) => {
localStorage.setItem('eventsnap_theme', theme);
localStorage.setItem('eventsnap_data_mode', mode);
if (jwt) {
localStorage.setItem('eventsnap_jwt', jwt);
localStorage.setItem('eventsnap_pin', pin);
localStorage.setItem('eventsnap_user_id', uid);
localStorage.setItem('eventsnap_display_name', name);
}
localStorage.setItem('eventsnap_guide_seen', '1');
},
{ jwt: who?.jwt, pin: who?.pin, uid: who?.user_id, name: who?.name, theme, mode: 'saver' }
);
await page.goto(`${BASE}${route}`, { waitUntil: 'domcontentloaded' });
await page.waitForTimeout(1200);
if (prep) await prep(page);
await page.waitForTimeout(600);
await page.screenshot({ path: `${OUT}/${label}-${theme}.png`, fullPage: false });
await ctx.close();
console.log('[shot]', label, theme);
};
const browser = await chromium.launch();
const hostWho = { ...anna, name: 'Anna Bauer' };
const adminWho = { jwt: admin, pin: '', user_id: '', name: 'Admin' };
for (const theme of ['light', 'dark']) {
await shot('01-join', theme, null, '/join');
await shot('02-feed-list', theme, hostWho, '/feed');
await shot('03-feed-grid', theme, hostWho, '/feed', async (p) => {
await p
.getByLabel('Rasteransicht')
.click()
.catch(() => {});
});
await shot('04-lightbox', theme, hostWho, '/feed', async (p) => {
await p
.locator('img')
.first()
.click()
.catch(() => {});
await p.waitForTimeout(500);
});
await shot('05-account', theme, hostWho, '/account');
await shot('06-host', theme, hostWho, '/host');
await shot('07-admin', theme, adminWho, '/admin');
await shot('08-upload', theme, hostWho, '/upload');
await shot('09-diashow', theme, hostWho, '/diashow');
}
// Onboarding: fresh guest, guide not seen
{
const ctx = await browser.newContext({ ...device });
const page = await ctx.newPage();
await page.goto(`${BASE}/join`, { waitUntil: 'domcontentloaded' });
await page.evaluate((w) => {
localStorage.setItem('eventsnap_theme', 'light');
localStorage.setItem('eventsnap_jwt', w.jwt);
localStorage.setItem('eventsnap_pin', w.pin);
localStorage.setItem('eventsnap_user_id', w.user_id);
localStorage.setItem('eventsnap_display_name', 'Emma Wagner');
}, accounts['Emma Wagner']);
await page.goto(`${BASE}/feed`, { waitUntil: 'domcontentloaded' });
await page.waitForTimeout(1500);
await page.screenshot({ path: `${OUT}/10-onboarding-light.png` });
await ctx.close();
console.log('[shot] onboarding');
}
await browser.close();
console.log('DONE');

45
e2e/specs/01-auth/join.spec.ts
View File

@@ -13,7 +13,11 @@ test.describe('Auth — join flow', () => {
const join = new JoinPage(page);
await join.goto();
await expect(page.getByRole('heading', { name: 'Willkommen!' })).toBeVisible();
// The join form's landing state. There is no "Willkommen!" heading — the wedding
// redesign (f243bfe) split it into a "Willkommen bei" lead-in plus the event name as
// the <h1>, and this assertion was never updated, so it had been failing since.
// Anchor on the testid the markup provides rather than on copy.
await expect(page.getByTestId('join-event-name')).toBeVisible();
const { pin } = await join.joinAs('Alice');
expect(pin).toMatch(/^\d{4}$/);
@@ -71,7 +75,11 @@ test.describe('Auth — join flow', () => {
expect(storage.pin).toBe(original.pin);
});
test('wrong PIN three times locks the account for 15 minutes', async ({ page, guest, db }) => {
test('repeated wrong PINs are throttled without locking the guest out', async ({
page,
guest,
db,
}) => {
const dave = await guest('Dave');
await clearAllStorage(page);
@@ -81,22 +89,33 @@ test.describe('Auth — join flow', () => {
await join.submit();
await expect(join.recoveryPinInput).toBeVisible();
// Wrong PIN (real one is dave.pin)
// Wrong PIN (real one is dave.pin), four times — one more than the OLD lock threshold of 3.
// Typed digit by digit so the 4th character auto-submits (see pin-auto-submit.spec.ts);
// clicking as well would double-submit and race the disabled state of the button.
const wrong = dave.pin === '0000' ? '1111' : '0000';
for (let i = 0; i < 3; i++) {
await join.recoveryPinInput.fill(wrong);
await join.recoverySubmit.click();
for (let i = 0; i < 4; i++) {
await join.recoveryPinInput.fill('');
await join.recoveryPinInput.pressSequentially(wrong, { delay: 30 });
await expect(join.recoveryError).toBeVisible();
await expect(join.recoverySubmit).toBeEnabled();
}
// Fourth attempt should hit the 429 lockout (even with the correct PIN now)
await join.recoveryPinInput.fill(dave.pin);
await join.recoverySubmit.click();
await expect(join.recoveryError).toContainText(/15 Minuten/);
// THE PROPERTY THIS TEST EXISTS FOR, stated the way a guest experiences it: Dave can still
// get into his own account.
//
// The lock threshold used to be 3, BELOW the per-(IP, name) ceiling — so these very
// keystrokes locked Dave out for 15 minutes, and anyone who can read his name off the feed
// could do it to him on repeat. Rate limits are disabled in this environment (see
// config `rate_limits_enabled`), so what is exercised here is purely the account-lock tier;
// the throttle tier is covered in 07-adversarial/auth-tampering.spec.ts.
expect(
await db.isPinLocked(dave.userId),
'four wrong PINs from one device must not lock a guest out of their own account'
).toBe(false);
// Sanity: DB row reflects the lock
// (The handler sets pin_locked_until directly — verify via API "recover" returning 429)
void db; // unused for now, documenting that db.lockUserPin exists if we want shortcut path
await join.recoveryPinInput.fill('');
await join.recoveryPinInput.pressSequentially(dave.pin, { delay: 30 });
await page.waitForURL('**/feed');
});
test('"Anderen Namen wählen" returns to the normal join form', async ({ page, guest }) => {

206
e2e/specs/01-auth/rate-limit-shared-nat.spec.ts Normal file
View File

@@ -0,0 +1,206 @@
/**
* Regression guard — the door must not close on a venue behind one NAT.
*
* `/join` was throttled 5 per 60s keyed purely on the client IP. Every guest at a venue
* arrives from the same public IP (that is what a NAT is), so the whole party shared one
* bucket: 12 guests scanning the QR code within a few seconds meant 5 got in and 7 were
* turned away — with no Retry-After to tell them when to try again. `/feed` (60/min) and
* `/export` (3/DAY) had the identical defect.
*
* These ran green for the same structural reason every time: the e2e reseed forces every
* limiter toggle OFF before each test, so nothing here was ever exercised. Enable them
* explicitly, exactly as 02-upload/rate-limit does.
*/
import { test, expect } from '../../fixtures/test';
import { BASE } from '../../helpers/env';
test.describe('Rate limits — guests behind a shared NAT', () => {
test('a dozen guests can all join from one IP, and 429s carry Retry-After', async ({
api,
adminToken,
}) => {
await api.patchConfig(adminToken, {
rate_limits_enabled: 'true',
join_rate_enabled: 'true',
});
// Twelve DISTINCT guests, same source IP — the arrival burst at a real party.
const names = Array.from({ length: 12 }, (_, i) => `NatGuest${i}`);
const results = await Promise.all(
names.map((display_name) =>
fetch(`${BASE}/api/v1/join`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ display_name }),
})
)
);
const rejected = results.filter((r) => r.status === 429);
expect(
rejected.length,
`all 12 guests must get in from one IP; ${rejected.length} were turned away`
).toBe(0);
expect(results.every((r) => r.status === 201)).toBe(true);
});
test('one guest retrying their own name is still throttled, and told for how long', async ({
api,
adminToken,
}) => {
// The per-name bucket must still bite — otherwise the NAT fix would have simply
// removed the anti-spam limit rather than re-keyed it.
await api.patchConfig(adminToken, {
rate_limits_enabled: 'true',
join_rate_enabled: 'true',
});
const attempt = () =>
fetch(`${BASE}/api/v1/join`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ display_name: 'RepeatOffender' }),
});
// 5 per 60s for the same (ip, name): the first succeeds (201), the next four collide
// with the taken name (409), and the sixth exhausts the bucket.
const codes: number[] = [];
for (let i = 0; i < 6; i++) codes.push((await attempt()).status);
expect(codes[0], 'the first join should succeed').toBe(201);
expect(codes.at(-1), 'the 6th attempt on one name must be throttled').toBe(429);
const throttled = await attempt();
expect(throttled.status).toBe(429);
const retryAfter = throttled.headers.get('retry-after');
expect(retryAfter, '429 must tell the client when to come back').toBeTruthy();
expect(Number(retryAfter)).toBeGreaterThan(0);
expect(Number(retryAfter)).toBeLessThanOrEqual(60);
});
test('the feed limit is per-user, not per-IP', async ({ api, adminToken, guest }) => {
// Two guests, one IP. With a limit of 3/min an IP key would let the first guest's
// three reads starve the second entirely.
await api.patchConfig(adminToken, {
rate_limits_enabled: 'true',
feed_rate_enabled: 'true',
feed_rate_per_min: '3',
});
const a = await guest('FeedHog');
const b = await guest('FeedVictim');
const read = (jwt: string) =>
fetch(`${BASE}/api/v1/feed`, { headers: { Authorization: `Bearer ${jwt}` } });
// Guest A burns their whole allowance.
for (let i = 0; i < 3; i++) expect((await read(a.jwt)).status).toBe(200);
expect((await read(a.jwt)).status, "A's own 4th read is throttled").toBe(429);
// Guest B must be entirely unaffected.
expect((await read(b.jwt)).status, 'B must not inherit A’s exhausted bucket').toBe(200);
});
test('the export limit is per-user — one guest cannot spend the whole venue’s quota', async ({
api,
adminToken,
guest,
host,
db,
}) => {
// The sharpest case: 3 downloads per DAY on an IP key meant the 4th guest to fetch
// their keepsake was locked out until tomorrow.
await db.setExportReleased('e2e-test-event', true);
await api.patchConfig(adminToken, {
rate_limits_enabled: 'true',
export_rate_enabled: 'true',
export_rate_per_day: '1',
});
const mintAndFetch = async (jwt: string) => {
const res = await fetch(`${BASE}/api/v1/export/ticket`, {
method: 'POST',
headers: { Authorization: `Bearer ${jwt}` },
});
const { ticket } = await res.json();
return fetch(`${BASE}/api/v1/export/zip?ticket=${encodeURIComponent(ticket)}`);
};
const a = await guest('ExportFirst');
const b = await guest('ExportSecond');
// A spends their single daily allowance. The archive itself may not exist (404) —
// what matters is that the limiter admitted the request rather than 429ing it.
expect((await mintAndFetch(a.jwt)).status).not.toBe(429);
expect((await mintAndFetch(a.jwt)).status, 'A’s second download is throttled').toBe(429);
// B shares A's IP and must still get their keepsake.
expect((await mintAndFetch(b.jwt)).status, 'B must not be locked out by A’s download').not.toBe(
429
);
// And the host too, for good measure.
expect((await mintAndFetch(host.jwt)).status).not.toBe(429);
});
});
test.describe('Rate limits — /recover name cycling', () => {
test('cycling names from one IP hits the ceiling, while one name is still throttled', async ({
api,
adminToken,
}) => {
// /recover is keyed `recover:{ip}:{name}` — right for its job (stopping someone who
// knows a display name from burning the victim's 3-strike PIN counter), but the name is
// ATTACKER-CHOSEN, so cycling names minted a fresh bucket every time. Behind it sits a
// cost-12 bcrypt verify, including an unconditional throwaway one for unknown names, so
// a name generator was the cheapest way to make the server hash forever.
//
// Squeeze the ceiling so the flood is reproducible without firing 30+ requests.
await api.patchConfig(adminToken, {
rate_limits_enabled: 'true',
recover_rate_enabled: 'true',
recover_ip_rate_per_min: '5',
});
const attempt = (name: string) =>
fetch(`${BASE}/api/v1/recover`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ display_name: name, pin: '0000' }),
});
// Every name is distinct, so the per-name bucket can never fire — only the ceiling can.
const codes: number[] = [];
for (let i = 0; i < 12; i++) codes.push((await attempt(`Unbekannt${i}_${Date.now()}`)).status);
expect(
codes.filter((c) => c === 429).length,
'name cycling must be capped by the per-IP ceiling'
).toBeGreaterThan(0);
const throttled = await attempt(`Unbekannt99_${Date.now()}`);
expect(throttled.status).toBe(429);
expect(Number(throttled.headers.get('retry-after'))).toBeGreaterThan(0);
});
test('the per-name bucket still protects a real account', async ({ api, adminToken, guest }) => {
// The ceiling must not have REPLACED the anti-guessing control. With a generous ceiling,
// repeated wrong PINs against ONE name must still be shut down by the per-name bucket.
const victim = await guest('PinVictim');
await api.patchConfig(adminToken, {
rate_limits_enabled: 'true',
recover_rate_enabled: 'true',
recover_ip_rate_per_min: '1000',
});
const attempt = () =>
fetch(`${BASE}/api/v1/recover`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ display_name: victim.displayName, pin: '9999' }),
});
const codes: number[] = [];
for (let i = 0; i < 7; i++) codes.push((await attempt()).status);
expect(codes.at(-1), 'guessing one name must still be throttled').toBe(429);
});
});

222
e2e/specs/02-upload/burst-queue.spec.ts Normal file
View File

@@ -0,0 +1,222 @@
/**
* Client upload-queue under a realistic burst — the scenario an event actually
* produces (a guest multi-selects 10-20 photos at once) and the one the
* server-side load test could NOT cover, because that test hit POST /upload
* directly and bypassed the browser queue entirely.
*
* Drives the REAL client path (UploadSheet → /upload → addToQueue →
* processQueue → XHR) and asserts the four properties that matter for not
* losing a guest's photos:
*
* 1. SERIAL drain — the queue uploads ONE file at a time per device
* (processQueue's find-next-pending loop), never a
* parallel fan-out. Proven by counting how many upload
* requests sit in the intercept handler at once.
* 2. PERSISTENCE — every staged file is written to IndexedDB with its
* blob before it's sent, and the blob is dropped only
* after the upload succeeds. Read straight out of IDB
* mid-burst.
* 3. ALL LAND — every file in the burst is created server-side.
* 4. RESUME ON RELOAD — a hard reload mid-burst (tab closed / PWA killed)
* resumes the remaining uploads from IndexedDB via
* loadQueue(), losing nothing.
*
* A tiny per-upload latency is injected via route interception so the serial
* drain is observable and there's a window to reload mid-burst — no need for
* large fixture files.
*/
import { test, expect } from '../../fixtures/test';
import { skipIfNoIdbBlobs } from '../../helpers/webkit';
import { FeedPage, UploadSheet } from '../../page-objects';
import { mkdtempSync, copyFileSync, rmSync } from 'node:fs';
import { join } from 'node:path';
import { tmpdir } from 'node:os';
import type { Page } from '@playwright/test';
const SAMPLE_JPG = join(process.cwd(), 'fixtures', 'media', 'sample.jpg');
const BURST = 16; // in the 10-20 "multi-select" range
// Make BURST distinct files. The queue dedupes on name+size+lastModified, so
// distinct NAMES are enough to keep them from collapsing into one item — content
// can be identical (we only assert rows are created, not what they decode to).
function makeBurstFiles(): { dir: string; paths: string[] } {
const dir = mkdtempSync(join(tmpdir(), 'eventsnap-burst-'));
const paths = Array.from({ length: BURST }, (_, i) => {
const p = join(dir, `burst_${String(i).padStart(2, '0')}.jpg`);
copyFileSync(SAMPLE_JPG, p);
return p;
});
return { dir, paths };
}
// White-box: read the queue rows straight from IndexedDB, including whether the
// blob is still attached — so we can prove blobs are kept until upload and
// dropped after. Mirrors the reader in offline-resume.spec.
async function queueRows(page: Page): Promise<Array<{ status: string; hasBlob: boolean }>> {
return page.evaluate(async () => {
return new Promise<Array<{ status: string; hasBlob: boolean }>>((resolve, reject) => {
const req = indexedDB.open('eventsnap-uploads', 3);
req.onerror = () => reject(req.error);
req.onsuccess = () => {
const dbh = req.result;
const tx = dbh.transaction('queue', 'readonly');
const all = tx.objectStore('queue').getAll();
all.onsuccess = () =>
resolve(all.result.map((r: any) => ({ status: r.status, hasBlob: r.blob != null })));
all.onerror = () => reject(all.error);
};
});
});
}
/**
* Intercept POST /upload to (a) inject latency so the serial drain is observable
* and (b) count how many uploads are in the handler simultaneously. A serial
* client never has more than one in flight; a parallel fan-out would spike.
* Returns a `peak()` accessor for the max concurrency seen.
*/
async function instrumentUploads(page: Page, delayMs: number) {
let active = 0;
let peak = 0;
await page.route('**/api/v1/upload', async (route) => {
active++;
peak = Math.max(peak, active);
await new Promise((r) => setTimeout(r, delayMs));
active--;
await route.continue();
});
return { peak: () => peak };
}
async function warmUploadChunk(page: Page) {
// Load the code-split /upload route while online so a later reload/navigation
// doesn't fetch the chunk at a bad moment (mirrors offline-resume.spec).
await page.goto('/upload');
await page.goto('/feed');
}
test.describe('Upload — client queue under a burst', () => {
test('a 16-file burst drains serially, persists to IndexedDB, and all land', async ({
page,
guest,
signIn,
db,
browserName,
}) => {
skipIfNoIdbBlobs(browserName);
const g = await guest('BurstSerial');
await signIn(page, g);
await warmUploadChunk(page);
const uploads = await instrumentUploads(page, 200);
// Stage the whole burst through the real UI.
const { dir, paths } = makeBurstFiles();
try {
const feed = new FeedPage(page);
const sheet = new UploadSheet(page);
await feed.openUploadSheet();
await sheet.stageFiles(paths);
await sheet.captionInput.waitFor({ state: 'visible', timeout: 10_000 });
await sheet.fillCaption('burst of 16 #hochzeit');
await sheet.submit();
await page.waitForURL('**/feed', { timeout: 15_000 });
// (2) PERSISTENCE: all 16 are written to IndexedDB with blobs essentially
// immediately — before most have been sent. Catch the burst while the
// queue still holds most of them.
await expect
.poll(async () => (await queueRows(page)).length, { timeout: 10_000 })
.toBe(BURST);
// While draining, pending rows keep their blob; done rows have dropped it.
// Poll for a mid-burst moment that shows both invariants at once.
await expect
.poll(
async () => {
const rows = await queueRows(page);
const pendingKeepBlob = rows
.filter((r) => r.status === 'pending' || r.status === 'uploading')
.every((r) => r.hasBlob);
const doneDropBlob = rows.filter((r) => r.status === 'done').every((r) => !r.hasBlob);
return pendingKeepBlob && doneDropBlob;
},
{ timeout: 10_000 }
)
.toBe(true);
// (3) ALL LAND: every file is created server-side.
await expect.poll(() => db.countUploadsForUser(g.userId), { timeout: 30_000 }).toBe(BURST);
// (1) SERIAL: never more than one upload in flight at a time.
expect(uploads.peak()).toBe(1);
// Clean end — the queue settles all items to done (no stuck error/blocked).
await expect
.poll(async () => (await queueRows(page)).every((r) => r.status === 'done'), {
timeout: 10_000,
})
.toBe(true);
} finally {
rmSync(dir, { recursive: true, force: true });
}
});
test('a hard reload mid-burst resumes the remaining uploads — nothing lost', async ({
page,
guest,
signIn,
db,
browserName,
}) => {
skipIfNoIdbBlobs(browserName);
const g = await guest('BurstResume');
await signIn(page, g);
await warmUploadChunk(page);
// Bigger delay → a wide, reliable mid-burst window to reload inside. (We don't
// assert seriality here — test 1 already proves it, and a reload aborts the
// in-flight request mid-intercept, which would overcount concurrency.)
await instrumentUploads(page, 300);
const { dir, paths } = makeBurstFiles();
try {
const feed = new FeedPage(page);
const sheet = new UploadSheet(page);
await feed.openUploadSheet();
await sheet.stageFiles(paths);
await sheet.captionInput.waitFor({ state: 'visible', timeout: 10_000 });
await sheet.fillCaption('burst then reload');
await sheet.submit();
await page.waitForURL('**/feed', { timeout: 15_000 });
// Wait until we're genuinely mid-burst: a few landed, but far from all.
await expect
.poll(() => db.countUploadsForUser(g.userId), { timeout: 20_000 })
.toBeGreaterThanOrEqual(3);
const landedBeforeReload = await db.countUploadsForUser(g.userId);
expect(landedBeforeReload).toBeLessThan(BURST);
// Hard reload — wipes the JS module (and its in-flight drain), exactly like
// a closed tab / killed PWA. The remaining pending items live only in
// IndexedDB now.
await page.reload();
// Deliberately NOT navigating to /upload. Rehydration is now module-level and
// auth-gated (upload-queue.ts `hydrateQueue`), so the queue resumes wherever the
// reload lands. This assertion is the regression guard for the defect it replaced:
// `loadQueue()` used to have a single call site in the whole app — the /upload
// route's onMount — so a guest who reloaded anywhere else saw a 0 badge and their
// staged photos never left the phone, having already been shown a success.
// (4) RESUME: every file ends up server-side without re-staging anything.
// `>=` not `===`: the only imperfection possible is a DUPLICATE (an upload
// that succeeded server-side in the ~1ms between the XHR load event and the
// IndexedDB 'done' write, caught by the reload, then re-sent on resume).
// That's at worst one extra row, never a lost photo — the property we care
// about. A shortfall (< BURST) would be a real data-loss regression.
await expect
.poll(() => db.countUploadsForUser(g.userId), { timeout: 30_000 })
.toBeGreaterThanOrEqual(BURST);
} finally {
rmSync(dir, { recursive: true, force: true });
}
});
});

80
e2e/specs/02-upload/exif-orientation.spec.ts Normal file
View File

@@ -0,0 +1,80 @@
/**
* Regression guard — EXIF orientation must be applied when generating derivatives.
*
* Phones do not rotate sensor data. They shoot in the sensor's native landscape and record
* how the camera was held in an EXIF `Orientation` tag. `image`'s `decode()` returns the raw
* pixels and ignores that tag, and the JPEG re-encode writes no EXIF at all — so every
* portrait photo was stored SIDEWAYS in the 800px feed preview, the 2048px diashow display
* and the keepsake, while "Original anzeigen" still rendered it upright (the original keeps
* its tag). That asymmetry is why it reads as a viewer bug instead of a pipeline one.
*
* The fixture is 40x20 landscape pixels tagged Orientation=6 ("rotate 90° CW to display"),
* so a correctly-processed derivative is PORTRAIT (20x40). Asserting on the aspect ratio
* rather than the bytes keeps this robust across encoder changes.
*/
import { test, expect } from '../../fixtures/test';
import { uploadRaw } from '../../helpers/upload-client';
import { BASE } from '../../helpers/env';
import { readFileSync } from 'node:fs';
import { join } from 'node:path';
const EXIF_FIXTURE = join(process.cwd(), 'fixtures', 'media', 'portrait-exif6.jpg');
/**
* Read a baseline/progressive JPEG's pixel dimensions from its SOF marker.
* Avoids pulling an image dependency into the suite for one assertion.
*/
function jpegSize(buf: Buffer): { width: number; height: number } {
let i = 2; // skip SOI
while (i < buf.length) {
if (buf[i] !== 0xff) {
i++;
continue;
}
const marker = buf[i + 1];
// SOF0..SOF15, excluding DHT (c4), JPGA (c8) and DAC (cc)
if (marker >= 0xc0 && marker <= 0xcf && marker !== 0xc4 && marker !== 0xc8 && marker !== 0xcc) {
return { height: buf.readUInt16BE(i + 5), width: buf.readUInt16BE(i + 7) };
}
i += 2 + buf.readUInt16BE(i + 2);
}
throw new Error('no SOF marker found — not a JPEG?');
}
test.describe('Upload — EXIF orientation', () => {
test('a rotated photo is upright in the preview and the display derivative', async ({
guest,
db,
}) => {
const g = await guest('SidewaysShooter');
const res = await uploadRaw(g.jwt, readFileSync(EXIF_FIXTURE), {
filename: 'portrait-exif6.jpg',
contentType: 'image/jpeg',
caption: 'hochkant',
});
expect(res.status).toBe(201);
const { id } = (await res.json()) as { id: string };
// Sanity: the SOURCE really is stored landscape with the tag, otherwise this test
// could pass against a pipeline that does nothing.
const source = jpegSize(readFileSync(EXIF_FIXTURE));
expect(source.width).toBeGreaterThan(source.height);
await expect
.poll(() => db.compressionStatus(id), { timeout: 30_000, intervals: [250] })
.toBe('done');
for (const variant of ['preview', 'display'] as const) {
const r = await fetch(`${BASE}/api/v1/upload/${id}/${variant}`, {
headers: { Authorization: `Bearer ${g.jwt}` },
});
expect(r.status, `${variant} must be served`).toBe(200);
const { width, height } = jpegSize(Buffer.from(await r.arrayBuffer()));
expect(
height,
`${variant} must be portrait (${width}x${height}) — EXIF orientation was not applied`
).toBeGreaterThan(width);
}
});
});

3
e2e/specs/02-upload/gallery-path.spec.ts
View File

@@ -4,6 +4,7 @@
* IndexedDB queue resumption after refresh, and SSE `upload-processed`.
*/
import { test, expect } from '../../fixtures/test';
import { skipIfNoIdbBlobs } from '../../helpers/webkit';
import { FeedPage, UploadSheet } from '../../page-objects';
import { SseListener } from '../../helpers/sse-listener';
import { join } from 'node:path';
@@ -49,7 +50,9 @@ test.describe('Upload — gallery path', () => {
guest,
signIn,
db,
browserName,
}) => {
skipIfNoIdbBlobs(browserName);
// Previously fixme'd: the UI queue never fired a POST. Root cause was NOT a
// navigation/blob timing quirk but an IndexedDB upgrade bug — the v1→v2
// `upgrade` callback opened a *new* transaction, which throws during a

Some files were not shown because too many files have changed in this diff Show More