0ae5a64e7734b701b8695ff2abcd7431ec90e9f9
12 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Fabian Hamm (Privat)
|
2f952494c2 |
fix(media): stop a poster-frame failure from deleting the guest's video
Reproduced live, by accident, while smoke-testing on a machine with no ffmpeg: the clip uploaded fine, returned 201, and roughly six seconds later had `deleted_at` set and was gone from the feed. The `Ok(None)` "this clip yields no frame" case was already handled — that fix landed when sub-second clips were being destroyed. But the `?` on the call itself still routed every OTHER failure into the same give-up path, which soft-deletes: ffmpeg missing from the image, ffmpeg hanging on a truncated `.mov` and tripping the timeout, an ENOSPC on `thumbnails/`, or a DB blip in `set_thumbnail_path`. None of those says anything about the video, and `get_original` serves the file byte-for-byte, so a post that merely lacks a poster is fully watchable. No failure in the video branch may fail the upload. iPhone `.mov` is exactly the input most likely to trip it, and a wedding clip is not retakeable. ENOSPC gets its own classifier. It was the one failure the retry loop actively made worse: a disk does not drain during six seconds of backoff, so all three attempts failed identically while holding a compression permit that photos were queued behind — and the give-up path then refunded the quota and soft-deleted the row while deliberately KEEPING the original. That freed nothing, removed the photo seconds after a 201, and handed the guest the allowance to upload it again into the same full disk. Now: no retry, no refund, no delete. The row stays live and the photo is served from its original, and `backfill_stale_derivatives` regenerates the derivatives on the next start once there is room. `is_storage_full_error` has to look inside `ImageError::IoError` as well as at bare io errors, because `image` wraps rather than sources it and a plain chain walk would miss every derivative-write failure. FFMPEG_TIMEOUT drops 120s -> 45s. It was never a budget for honest work — a poster from a phone clip takes well under a second, and `-ss` before `-i` means even a 500 MB file seeks rather than scans. It is the ceiling on how long a pathological input holds a permit that guests' photos are waiting behind, so it should be as tight as it can be without cutting off real work. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
fabi
|
402215d405 |
fix(video): extract a real poster frame, and stop claiming one that isn't there
Both the compression worker and the HTML export ran the same invocation:
ffmpeg -i <src> -vframes 1 -ss 00:00:01 -vf scale=… -y <out>
`-ss` AFTER `-i` is an output-side seek. Against a clip of a second or less ffmpeg
exits 0 and writes nothing, and both call sites gated on the exit status:
- the worker wrote `thumbnail_path` and logged "thumbnail generated" for a file that
was never created, so GET /upload/{id}/thumbnail 404s in the live feed;
- the export listed media/<id>_thumb.jpg in data.json while the ZIP writer skipped the
unopenable file, so the keepsake drew a broken image tile.
Any clip at or under a second, which phones produce constantly — mis-taps, Live
Photos, boomerangs. Not data loss; the .mp4 is in both archives and plays. Every
server-side signal stayed green.
New `services/video.rs` owns the extraction for both callers, mirroring the imaging.rs
precedent (created for the same duplication, and it paid off when the max_alloc fix
landed in both workers at once). Three changes in it:
- `-ss` before `-i`, an input-side seek. NOT sufficient alone: verified against the
production image, seeking to 1s in a 1.000s clip is still past the last frame and
still exits 0 with no file. The 0s fallback is what actually fixes this, and 1s is
tried first only because an opening frame makes a poor poster.
- Verify the artifact, not the exit status. This is the check both sites were missing.
- Carry compression.rs's 120s timeout. export.rs had NONE — a hung ffmpeg there would
strand the job at `running` and the keepsake would never complete.
The worker's call used `?`. Tightening the check without also making a missing poster
non-fatal would have been far worse than the bug: every sub-second clip would fail
compression, exhaust its retries and be soft-deleted. It now logs a warning and leaves
`thumbnail_path` NULL, which FeedListCard, VirtualFeed and LightboxModal already
handle.
The export now sets `thumb: ""` and skips the manifest entry when there is no poster —
and does the same for the IMAGE branch, whose decode failure left the identical
dangling reference. No viewer change was needed: +page.svelte already guards
`{#if post.media.thumb}` and falls back to a video tile with a play glyph. The comment
claiming "viewer handles missing thumbs gracefully" was true about the viewer and false
about what the backend sent — the guard never fired because the string was never empty.
e2e/specs/06-export/export-video.spec.ts had DOCUMENTED this as intended behaviour
("the fixture clip is <1s, so ffmpeg extracts no thumbnail frame — but exits 0 …
that's the intended shape here"). sample.mp4 is exactly 1.000s, so every video test in
the suite ran at that boundary and none ever fetched the poster. That comment is now
corrected to say what it actually was.
Tests: 2 unit; a new sample-5s.mp4 fixture so the ordinary first-seek path is covered
at all; video-playback now FETCHES the poster rather than asserting the attribute (the
one extra request that nine rounds of green never made); a new spec covering both
fixtures plus the mirror that a posterless video still uploads and plays; and a
keepsake spec asserting every <img> in the opened viewer resolves — naturalWidth === 0
is exactly the broken-tile case, whatever produced it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
fabi
|
31faccfdf8 |
fix(audit-4): refuse undecodable images at admission, stop retrying permanent failures
Fourth round: reject at the door what the worker can only fail on, and stop retrying errors that cannot succeed — while keeping IoError retryable so ENOSPC still gets another attempt. Squashed from 2 commits, original messages preserved below. ──────── fix(upload): refuse undecodable images at the door, and stop retrying them Two halves of the same complaint: an oversized photo was accepted with a 201 and then silently soft-deleted minutes later, after the worker had burned six seconds of backoff re-reaching a conclusion it could not change. Admission. The compression budget now runs at upload time, against the header only, so a guest is told immediately and told why: "Bild hat zu viele Bildpunkte (ca. 99 Megapixel) und kann nicht verarbeitet werden. Bitte verkleinere es und lade es erneut hoch." instead of watching the photo vanish behind a vague "could not be processed" — which arrived only if they happened to still be on the feed with that card loaded. Nothing is stored, so there is no row to soft-delete and no orphan for the sweep to reclaim. Admission and the worker share ONE function (`decoder_within_budget`), so they cannot drift apart and start disagreeing about what is acceptable — a photo accepted at the door and rejected by the worker would be worse than either behaviour alone. The worker keeps its own check: the backfill decodes files that predate this check, and defence in depth is the whole reason the budget exists. Retries. The loop retried every failure, including ones that are a property of the input. An image over the budget, a corrupt file, an unsupported format: each fails identically on all three attempts, so the only effect was 2s + 4s of sleep and three near-identical warnings before the same outcome. `is_permanent_image_error` classifies the `ImageError` variants that cannot change between attempts — Limits, Unsupported, Decoding — and the loop gives up on those at once. `IoError` is deliberately excluded: an ENOSPC while writing a derivative is exactly the transient case the retry exists for, and misclassifying it would turn a blip back into the data loss round 1 fixed. Measured: retry log lines went from 3 per oversized upload to 0. Tests: unit tests for both sides of the classifier (a Limits error is permanent, a missing file is not) and for admission agreeing with the decoder on accept AND reject. The e2e spec is rewritten for the new contract — 400 with an actionable message, nothing stored, backend alive after a burst of four — plus a mirror asserting an ordinary photo still uploads and processes, since a budget that rejected everything would satisfy the other two. ──────── fix(upload): narrow the admission check to the memory budget only The admission check I just added rejected ANY image the decoder couldn't build — corrupt, truncated, or unsupported, not only over-budget. That broke two adversarial tests, and they were right to break. 07-adversarial/file-upload-attacks pins, deliberately, that acceptance follows the MAGIC BYTES: a payload whose first three bytes are a JPEG header is accepted regardless of what follows, because the security property under test is that the client-declared Content-Type has no influence. Both failing cases upload 1024 bytes of JPEG magic followed by zeros. Rejecting those at admission is a different, broader contract than the one asked for, and rewriting an adversarial test to match new behaviour is precisely the thing that needs justifying rather than doing quietly. So admission now checks only what it was meant to: `exceeds_decode_budget` returns true solely for `ImageError::Limits`. A corrupt file goes to the compression worker exactly as before — which handles it gracefully and, since the retry classifier in the previous commit, no longer burns backoff on it. The resource guard is the part that had to move earlier; nothing else did. Tests: the size agreement between admission and the worker is still asserted in both directions, plus a new one writing a magic-bytes-only stub and asserting admission accepts it WHILE the worker still rejects it — pinning the boundary between the two checks so a future widening fails here rather than in the adversarial suite. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
fabi
|
b601c062bd |
fix(audit-2): video playback, role identity, keepsake EXIF, recover ceiling, WebKit CI
Second round: the two regressions the first round introduced, the remaining
gaps it left, and the CI change that makes the iOS guarantees actually gate a
change rather than being asserted.
Squashed from 9 commits, original messages preserved below.
──────── docs(deploy): document the update path — `up -d` alone ships nothing
The README only ever described a fresh install. There was no update section
anywhere, and `--build` appeared nowhere in the docs.
That matters because `app` and `frontend` are `build:` services with no published
image tag, and Compose has no source-change detection: if an image by that name
exists it is reused. So the natural `git pull && docker compose up -d` reports
"Container app-1 Running", rebuilds nothing, and exits 0. A deploy that shipped
none of the new code is indistinguishable from a successful one — which is how
eleven merged fixes can sit in the repo and never reach the box.
Verified both halves against a real stack rather than asserting them: with a
source change staged, `up -d` left the image ID untouched; `up -d --build`
produced a new image ID and a healthy /health.
Adds an "Updating an existing deployment" section covering backup-before-migrate,
pull, rebuild, health check, and an image-ID comparison to prove a build actually
happened. Also spells out the rollback trap: migrations run on boot and are not
undone by checking out an older commit, so rolling back code without restoring
the snapshot leaves the schema ahead of the binary and the app refusing to start.
──────── fix(video): play the actual video, and answer Range requests
Every video in the app was unplayable. Two independent defects, either one
sufficient on its own, and nothing in the suite covered either — no test
anywhere played media or asserted a `<video>` src.
1. The lightbox handed `<video>` a JPEG.
`pickMediaUrl` is mime-agnostic, and compression only ever produces a THUMBNAIL
for a video (one `ffmpeg -vframes 1` frame) — no preview, no display. So in the
DEFAULT saver mode the element's src resolved to `/api/v1/upload/{id}/thumbnail`,
served as `image/jpeg` with `nosniff` so the browser can't even sniff its way
out. Chromium reports DEMUXER_ERROR_COULD_NOT_OPEN.
Fixed in the lightbox rather than in `pickMediaUrl`: FeedListCard shares that
helper and legitimately wants the thumbnail for its `<img>` poster, so a central
mime branch would break the feed. This mirrors the rule the diashow already
applies ("videos play the original file directly"). Added `preload="none"` so
saver-mode guests on cellular still fetch nothing until they press play — there
is no smaller video derivative to offer them — plus `playsinline`, without which
iOS hijacks playback into fullscreen.
2. `stream_media_file` ignored Range entirely.
It took no request headers, so it could not see `Range`; it always returned 200
with the whole body and never sent Accept-Ranges or Content-Range. iOS Safari
opens every `<video>` with a `Range: bytes=0-1` probe and abandons the load
without a 206 — so video failed on the app's primary platform even in `original`
mode, where the src was already correct.
Adds single-range support (`bytes=N-`, `bytes=N-M`, `bytes=-S`) with 206 +
Content-Range, 416 + `bytes */len` past EOF, and Accept-Ranges advertised on
every response. Anything it won't handle — multi-range, non-bytes units, garbage
— falls back to a full 200, which RFC 9110 explicitly permits and which is safer
than guessing. All four media routes share the helper, so seeking works
uniformly.
`get_original` now serves `inline` instead of `attachment`. An attachment
disposition is hostile to a `<video>` element, and this route is the only source
of playable video bytes; it also matches what the UI promises, since the action
is labelled "Original anzeigen" — view, not download. `no-store` is deliberately
kept so a takedown still revokes access promptly; ranges work fine under it, the
client just re-fetches.
Tests: 11 unit tests pin the parser (the iOS `bytes=0-1` probe, inclusive ends,
suffix ranges, clamping past EOF, 416 vs 200, malformed fallbacks). A new
03-feed/video-playback spec asserts the src is the original and not the
thumbnail, that the browser accepts the bytes as media (readyState > 0, no
MediaError), that no video bytes are delivered before play, and that Range
returns the correct 206 slices and a 416 past EOF — verified on both Chromium
and WebKit.
The "not downloaded before play" test asserts no *delivered body* rather than no
request: WebKit opens a connection for a preload="none" video and immediately
aborts it (GET, no Range, status 0, nothing transferred) while Chromium issues
nothing at all. The portable guarantee is that no response carrying bytes
completes.
──────── fix(auth): bind the role store to the identity, not to the tab
The role store I added in the moderation work is a module-level singleton seeded
ONCE at import. `goto()` is a client-side navigation, so leaving and re-joining in
the same tab re-imports no module and re-runs no onMount — the previous user's
role simply stayed resident. Nothing reset it: not join, recover, admin login,
"Event verlassen", `clearAuth`, nor the api.ts 401 auto-clear.
So a host who left, followed by a guest joining on the same phone, left that guest
with `isStaff === true` and a "🚫 Beitrag entfernen" action on other people's
photos. The backend 403s the delete, so this was a false affordance rather than a
privilege escalation — but `/feed` never fetched `/me/context`, so unlike every
other route it never self-corrected either. It survived until a hard reload.
The mirror case was equally broken and easier to overlook: a guest who recovered
into a host account got NO host affordances.
`clearAuth` already had a hook registry for exactly this shape of problem, with a
comment explaining it exists to avoid circular imports. Add the missing mirror,
`onSetAuth`, fired by both `setAuth` and `setAdminAuth` after the new token is
resident, and have the role store register on both sides: clear to null on
logout, re-seed from the new token on login. That also gives
`syncRoleFromToken` — dead code with zero callers since I introduced it — its
intended purpose.
Seeding from the claim fixes the reported bug, but the claim is frozen for the
token's 30-day life, so a promotion or demotion still wouldn't reach the feed.
`/feed` now calls the existing `refreshEventState()` on mount, which fetches
`/me/context` and applies both the authoritative role and the lock/release state
in one request. The feed is the one route gating a destructive action on the role,
so it should not be the only route running on a stale claim.
Tests: 04-host/role-identity-reset drives the real flows. The first asserts the
host DOES see the action before asserting the newcomer does not — a negative
assertion alone would pass against a build that shipped no moderation at all. The
second covers the mirror, promoting a guest server-side while their resident token
still claims `role: guest`, so a fix that only cleared the role would fail it.
──────── fix(compression): reclaim failed originals instead of leaking them
Round 1 stopped the compression worker deleting an upload's original on failure —
a transient ENOSPC or a codec panic must never destroy the only copy of a photo a
guest cannot retake. But it left `Upload::soft_delete`'s quota refund in place, so
the bytes stayed on disk while the uploader was charged nothing for them.
That is worse than it first looks. The row is soft-deleted, so the file is
invisible and unowned; a guest hitting a reproducible codec failure can accumulate
orphans indefinitely at zero personal cost. And `active_uploaders` counts only
users with non-deleted uploads, so dropping out of that count RAISES everyone's
per-user ceiling — the leak loosens the very quota meant to contain it.
Keep the refund: the uploader didn't cause the failure and shouldn't silently lose
quota to it. Bound the leak instead, with an hourly sweep alongside the existing
session cleanup in `spawn_periodic_tasks`, reclaiming failed originals older than
14 days — comfortably longer than any single event, so an operator investigating a
failed upload still has the file.
The selection predicate is the entire safety argument, so it is deliberately
narrow: `compression_status = 'failed'` AND soft-deleted AND past the window AND
`original_path <> ''`. That is exactly the state the give-up path leaves behind,
and it cannot reach a live upload, an owner-deleted one, or a failure still inside
its recovery window. `original_path` is cleared after a successful reclaim, which
makes the sweep idempotent — otherwise a row whose file is already gone is
re-selected on every tick forever. The row itself is kept as the audit trail.
Tests reproduce the selection verbatim (same pattern as upload_concurrency) and
assert it against five near-misses that must survive, both sides of the retention
boundary, and the idempotence property.
Also fixes two comments in export.rs still claiming "the compression worker
hard-deletes an original when its transcode fails" — no longer true, and the
defensive handling they justify is now justified by this sweep and by ordinary
deletes instead.
──────── fix(export): apply EXIF orientation in the keepsake too
Round 1 fixed EXIF orientation in the compression worker, which corrected the live
app — feed preview and diashow display. The export worker was missed, and it does
not reuse those derivatives: it re-decodes the originals itself with `image::open`,
which ignores the orientation tag, then re-encodes to JPEG, which drops the tag —
so the viewer has no way to recover it.
The damage was oddly shaped, which is exactly why it reads as a viewer bug:
Gallery.zip originals correct (byte-copied, EXIF intact)
Memories viewer grid thumbnails SIDEWAYS (always)
Memories viewer full image >5 MB SIDEWAYS (re-encoded at 2000px)
Memories viewer full image ≤5 MB correct (streamed byte-for-byte)
So in the keepsake people actually keep, every portrait photo in the grid was on
its side, and clicking through silently "fixed" small photos but not large ones.
Rather than paste the decoder dance a third time, extract `services::imaging::
decode_oriented` and route both workers through it, so there is exactly one way to
turn a file on disk into a DynamicImage. It carries a second invariant that had
also drifted: `image::open` applies NO decode limits, so the export path was
decoding arbitrary user-supplied images unbounded — the decompression-bomb cap
existed only in the compression worker. Both now come as a pair, which is the
point of having one function.
Not done: switching export to consume the existing `display` derivative. It would
fix orientation and drop a redundant full-resolution decode per photo, but it
would also replace the pristine ≤5 MB originals in the keepsake with 2048px
re-encodes — a real quality regression in the one artefact people keep forever.
Test uploads the round-1 fixture (40x20 landscape tagged Orientation=6), runs a
real export, pulls the thumbnail out of Memories.zip and asserts it came back
portrait — with a sanity check that the source really is stored landscape, so the
test can't pass against a pipeline that does nothing.
──────── fix(recover): cap name cycling, and stop bcrypt blocking the runtime
Round 1 gave /join a per-IP ceiling and left /recover with only its
`recover:{ip}:{name}` bucket. That key is right for the job it was written for —
stopping someone who knows a display name (they're listed on the feed) from
burning the victim's 3-strike PIN counter and locking them out on repeat. But the
name is ATTACKER-CHOSEN, so cycling names mints a fresh 5-attempt bucket every
time and the per-IP cost is unbounded.
What sits behind that limiter makes it worse than a normal flood: every call runs
a cost-12 bcrypt verify, including an UNCONDITIONAL throwaway verify for names
that don't exist — added deliberately to close a timing oracle. So an unknown name
is the single cheapest way to make the server do ~200ms of hashing.
Adds `recover_ip_rate_per_min` (default 30, migration 019), checked BEFORE the
per-name bucket so a name generator can't walk past it. 30/min is far above any
real recovery attempt while capping a flood. The per-name bucket is untouched and
remains the anti-guessing control.
The second half matters as much as the first: bcrypt was running inline on the
async runtime everywhere. At cost 12 that pins a tokio worker thread for ~200ms,
and there is only one per core — so a login flood stalled every other request on
the box, including the feed. There was no spawn_blocking anywhere in the auth
module, despite SECURITY-BACKLOG claiming bcrypt had been offloaded.
Route all of it through `verify_password` / `hash_password` on the blocking pool.
That covers /recover, /admin/login, the host PIN reset, and — the one most likely
to bite at a real event — the PIN hash minted on every single /join. Saturating
the blocking pool degrades logins; saturating the worker threads degrades
everything.
Tests: cycling distinct names from one IP now hits the ceiling with a Retry-After,
and — the assertion that keeps the fix honest — repeated wrong PINs against ONE
name are still throttled with the ceiling set generously high, so the ceiling
added protection rather than replacing it.
──────── ci(e2e): run WebKit, so the iOS guarantees actually gate a PR
The workflow installed only Chromium and ran chromium-desktop + chromium-mobile.
iOS Safari is the app's stated primary user — a wedding guest opening a QR link —
and WebKit is the only engine in the matrix that reproduces two of its behaviours:
- it enforces X-Frame-Options on the hidden download iframe, so a site-wide DENY
makes the keepsake download silently do nothing. Blink hands attachments to
the download manager before the frame check and never notices.
- it abandons a <video> load unless its Range probe gets a 206.
Both of those shipped. Adding 06-export to the webkit project in the round-1 fix
bought nothing on a PR, because CI never ran that project at all — the regression
test written specifically to catch the blocker only ever executed locally.
Runs 71 tests (67 pass, 4 skip on the documented IndexedDB-blob harness
limitation) in ~1.5 minutes locally, using the exact command added here.
──────── chore(backend): satisfy cargo fmt
`checks.yml` runs `cargo fmt --check`, and it has been failing since the round-1
audit fixes: I gated those on `cargo build` and `cargo clippy` but never ran fmt,
so three files drifted then and eight more this round. Pure formatting — no
behaviour change; clippy stays at zero and all 70 backend tests still pass.
Worth noting for next time: clippy passing is not evidence fmt does.
──────── chore: satisfy prettier in frontend and e2e
`checks.yml` runs `npm run format:check` for both projects and both were failing.
- frontend/src/lib/ui-store.ts is mine, unformatted since the round-1 upload-queue
badge fix — the same miss as the rustfmt one: I gated on svelte-check and eslint
but never on format:check.
- e2e/loadtest/* and e2e/shots.mjs have been unformatted since
|
||
|
fabi
|
0c0eed885a |
fix(audit-1): media gating, per-user limits, moderation UI, upload integrity
First audit round: a blocker and eight high-severity findings across the media
access gate, the guest-facing rate limiters, host moderation, and the upload
pipeline — plus the backup commands and the e2e/production divergences that hid
several of them.
Squashed from 7 commits, original messages preserved below.
──────── fix(export): let the keepsake download through X-Frame-Options on iOS
The keepsake download navigates a hidden, same-origin iframe (deliberately:
a top-level navigation to a 404/429 would unload the PWA). Caddy stamped a
site-wide `X-Frame-Options: DENY` that also covered the proxied `/api/*`.
Blink hands a `Content-Disposition: attachment` response to the download
manager at the network layer, so Chromium never noticed. WebKit enforces XFO
on the frame navigation first and aborts the load — so on iOS Safari, the
app's primary platform, tapping Download did nothing at all, silently.
Carve the two export endpoints out to SAMEORIGIN, which still blocks
cross-origin framing. Implemented as two disjoint matchers rather than an
override: Caddy applies the FIRST `header` directive outermost, so it wins on
write and a later, more specific `header` is silently ignored (verified
against the running test stack).
Also close the test gap that let this ship:
- `06-export` ran on chromium-desktop only; add it to `webkit-iphone`, the
only engine that enforces XFO on the download frame.
- No test in the suite ever clicked a download button — every archive
assertion used Node `fetch`, which has no frame and no XFO enforcement.
Add a spec that clicks it and awaits a real `download` event. Verified
falsifiable: with the blanket DENY reinstated it fails and reports the
WebKit refusal as the cause.
- Fix `ExportPage`'s card-scoped locators, which matched nothing: the cards
carry `class="card p-5"` (a Tailwind `@apply` component class), never the
`rounded-xl` the page object looked for. This had left the "shows enabled
download buttons" test red on main.
──────── fix(media): close the percent-escape bypass of the media gate
`/media/%70reviews/{id}.jpg` served a taken-down photo to anyone,
unauthenticated. Verified against the running stack: the literal path 404s,
the escaped one returned 200 with the full image. Same for displays,
thumbnails and originals, and any escaped byte in any position works.
Cause: the block was four `nest_service("/media/previews", 404)` route
matches sitting above a `ServeDir` on `/media`. axum matches on the RAW path
(matchit does no percent-decoding), while `ServeDir` percent-decodes when it
resolves the file. So `%70reviews` missed every blocker, fell through to the
ServeDir, and was decoded back to `previews/` on disk — reaching the bytes
with no soft-delete and no ban-hide check. That defeats a host takedown,
which is the entire point of the gate.
Remove the `/media` route tree outright instead of racing the decoder.
Nothing needs it: every media URL the backend emits is already a gated
`/api/v1/upload/{id}/{original,preview,display,thumbnail}` alias
(handlers::feed), the frontend contains zero `/media/` references, and the
`/media` in config.rs/disk.rs is the filesystem path while `media/` in
export.rs is a path inside the zip. `/media/**` now 404s regardless of
encoding. The route's own comment already said it "serves nothing" — it
wasn't a backstop, it was the vector.
Caddy keeps proxying /media/* deliberately: the app 404s it, and forwarding
means the e2e gating specs exercise the app's refusal exactly as production
would rather than being masked by the SvelteKit 404 page.
Extend the gating spec with the encoded variants — asserting only the literal
spelling is what let this sit undetected.
──────── fix(rate-limit): key the guest-facing limiters per user, not per IP
At a venue every guest is behind one NAT, so an IP-keyed limiter hands the
whole party a single bucket. On a fresh deploy 12 guests arriving together
meant 5 joined and 7 were turned away, with no Retry-After telling them when
to retry. `/feed` (60/min) and `/export` (3 per DAY — the fourth guest to
fetch their keepsake locked out until tomorrow) had the same defect.
`feed_delta` was already keyed per-user and its comment states the exact
rationale ("so one client can't starve others behind a shared NAT"); this
makes its siblings match.
- feed:{ip} -> feed:{user_id} (auth was already in scope)
- export:{ip} -> export:{user_id} (resolved from the download ticket's
session, which was previously looked up and discarded)
- join:{ip}: pre-auth, so there is no user to key on. Split in two — a loose
per-IP ceiling that only bounds raw volume (new `join_ip_rate_per_min`,
default 60, migration 017), plus the real 5/60s anti-spam bucket keyed
per (ip, name), mirroring the existing `recover:{ip}:{name}`.
admin_login / recover / pin_reset_req stay IP-keyed on purpose and are now
commented as such: they guard credential guessing, where a per-user or
per-name key would just hand an attacker a fresh bucket per guess.
Retry-After: the machinery existed but 7 of 8 sites called `check()` and
hard-coded `None`, so a throttled client was told to back off but never for
how long. Delete the bool `check()` wrapper entirely so `check_with_retry`
is the only entry point and the delay cannot be discarded by accident. Also
surface it for the PIN lockout, where the deadline was already known.
Fix the "unknown" fallback while here: every client_ip() caller passed that
literal, so any request without X-Forwarded-For — anything reaching the app
directly rather than through Caddy — shared ONE global bucket. Serve with
connect-info and use the peer address.
Tests: the reseed forces every limiter toggle off before each test, which is
why this whole class was invisible. Add 01-auth/rate-limit-shared-nat, which
enables them and asserts 12 guests share an IP without collision, that one
guest hammering their own name IS still throttled (so the fix re-keys rather
than removes the limit), and that feed/export buckets are per-user. Retarget
the ddos join test at the new per-IP ceiling — it asserted the defect.
Also seed `admin_login_rate_enabled` (read by the handler, seeded by no
migration and no reseed) and register `join_ip_rate_per_min` in the admin
config allowlist. Unrelated pre-existing red test fixed: 01-auth/join
asserted a "Willkommen!" heading the wedding redesign removed.
──────── feat(moderation): let a host remove a guest's photo or comment from the UI
`DELETE /host/upload/{id}` and `DELETE /host/comment/{id}` were complete on the
backend — transactional, SSE-broadcasting, audit-logged — and had zero frontend
callers. The feed context sheet offered "Löschen" only when
`target.user_id === myUserId`, so the only lever a host actually had against an
unwanted photo was banning the uploader.
That is both disproportionate and ineffective. A ban doesn't retract what was
already posted, and it makes things strictly worse for comments: the ban check
runs BEFORE the ownership check on the guest delete route, so banning an abusive
author leaves their comment on screen and permanently undeletable by them. With
no host affordance, nobody could remove it at all.
- feed: hosts/admins get "Beitrag entfernen" on other people's posts, routed to
the host endpoint (the guest route 403s anything the caller doesn't own) with
moderation-specific confirm copy. Own-post "Löschen" is unchanged.
- lightbox: same for comments, via /host/comment/{id}.
- Ban semantics are deliberately untouched (USER_JOURNEYS §10 — banned users keep
read access and cannot write). The deadlock is broken by giving the host a way
in, not by loosening the ban.
Live role (this had to come first). `getRole()` decodes the JWT claim, but the
token is never reissued — the backend slides the session row forward and treats
the DB row as authoritative. The claim is therefore frozen for the token's
lifetime: up to 30 days. A guest promoted at the party saw no Host-Dashboard and
no moderation actions until they signed out and back in, even though
`/me/context` had been returning their real role on every page load and 4 of its
6 call sites dropped the field on the floor.
Add `role-store.ts`: seeded from the claim so there's no flash of the wrong nav,
then corrected by every `/me/context` response. Point the ad-hoc `getRole()`
callers at it (account, upload, host, admin, and the new feed gate). The host and
admin dashboards now derive `myRole` reactively, so a demotion disables their
controls immediately instead of at next login.
Tests: 04-host/moderation-ui drives the real UI — host removes a guest photo and
it's gone from /feed server-side; a plain guest is offered nothing on someone
else's post (the mirror that keeps the first test honest); a promoted guest gains
the dashboard on reload while their token still carries `role: guest`; and a host
removes the comment of an already-banned guest, asserting first that the author's
own delete 403s so the deadlock is real.
──────── fix(upload): stop destroying originals, apply EXIF orientation, surface rejections
Three defects in the same pipeline, each of which loses a photo or misrepresents
one.
1. A transient error destroyed the guest's only copy.
`process`'s error arm unconditionally `remove_file`d the original. Every failure
routed there: `create_dir_all`, both derivative `save_with_format` calls (disk
full is the canonical case, and it arrives exactly when many guests upload at
once), a panic inside the image codec, or a momentary DB-pool exhaustion. The
row is only SOFT-deleted, so the bytes were the sole unrecoverable part — and
they were the part we deleted. The author already knew this was wrong next door:
`backfill_missing_display` says it "must NEVER soft-delete an upload that already
has a working preview".
Retry up to 3 times with backoff (re-checking the e2e generation guard after each
sleep), and on final failure keep the refund + soft-delete but leave the original
on disk, logging its path. A failed upload is now recoverable instead of gone.
2. Every portrait photo was stored sideways.
Phones don't rotate sensor data — they record the camera orientation in EXIF and
store the pixels as shot. `decode()` returns those raw pixels and the JPEG
re-encode writes no EXIF, so the 800px preview, the 2048px diashow display and
the keepsake were all rotated 90°, while "Original anzeigen" rendered upright
because the original keeps its tag. That asymmetry is why it reads as a viewer
bug. There was no EXIF handling anywhere in the repo and no exif crate.
Read the tag via `into_decoder()` (which carries the decode Limits through, so
the decompression-bomb cap is untouched) and apply it. Missing/malformed tags
fall back to NoTransforms — most images have none.
Existing derivatives are already baked wrong, so migration 018 adds
`derivatives_rev` and `backfill_missing_display` becomes
`backfill_stale_derivatives`: it now also picks up anything below the current rev
and regenerates it once from the original, which still carries its EXIF. Videos
are marked current in the migration — ffmpeg already honours the rotation matrix.
Bump DERIVATIVES_REV for any future change that invalidates derivatives.
3. A rejected upload vanished without a word.
`UploadQueue.svelte` — 162 lines holding the ONLY renderer of an item's error
text, the only "Erneut" retry button and the only rate-limit countdown — was
never imported anywhere, so `retryItem`, `removeItem` and `clearCompleted` were
unreachable at runtime. On a terminal rejection the store purged the blob and
wrote a clear German reason into `entry.error` "so the UI shows a clear reason".
There was no such UI. And `uploadBadgeCount` counted only pending/uploading, so
the badge decremented exactly as if the upload had succeeded.
Mount the queue on /upload, toast the reason immediately (the flow sends the user
to /feed straight after staging, so the list alone would still miss them), and
count blocked/error in the badge so a failure can't read as success.
Tests: 02-upload/exif-orientation uploads a 40x20 fixture tagged Orientation=6
and asserts both derivatives come back PORTRAIT, with a sanity check that the
source really is stored landscape. 02-upload/rejection-visible bans the uploader
between staging and sending, then asserts the toast, the queue row with the
server's reason, and that the item is still counted.
Note: 02-upload/quota's 4 failures are pre-existing and unrelated — see the next
commit.
──────── docs(backup): make the backup commands work; fix the e2e/prod divergences
Backup. Both documented commands failed on the shipped stack, and the sentence
explaining them was wrong too:
- `pg_dump $DATABASE_URL` — `DATABASE_URL` is only ever in the compose
environment, never an operator's shell, and it points at `db:5432`, which is
compose-internal DNS. The app image has no postgres client either.
- `> /media/backups/…` — `/media` is a named volume mounted inside the app
container, not a host path, and nothing ever creates a `backups` subdirectory.
- `rsync /opt/eventsnap/media/` — that path does not exist anywhere.
- "a single path to back up" — false, and dangerously so: exports were moved to
their own `exports_data` volume precisely so a keepsake (which contains every
photo in the event) can't be served off the media tree. Backing up only
`media_data` silently loses every generated keepsake.
Rewritten as three commands — db via `docker compose exec -T db pg_dump`, and one
`docker run … tar` per volume — all verified against the running stack. The
volume mounts use `/src`, not `/media`: I hit the footgun while testing this.
Docker pre-populates an EMPTY volume from the image's own directory and chowns it
to match, so `-v media_data:/media alpine` tars alpine's cdrom/floppy/usb, writes
them into the volume, and leaves it root-owned so the non-root app can no longer
write. Mounting where the image has nothing avoids all of it. Documented inline
so the next person doesn't rediscover it.
Also correct the architecture notes: `/media/*` no longer routes to the backend
(that static tree was removed as a gating bypass), and `exports_data` was missing
from the volume list — the one volume an operator most needs to know about.
e2e stack: add the `EXPORT_PATH` + `/exports` volume it was missing. The file
says "mirrors production layout"; without these, exports landed on the container's
writable layer at the default path, so export-leak and export-video wrote real
archives into ephemeral storage and the "exports live outside media" invariant
was never actually exercised.
Pre-existing red test, unrelated to the audit: all four 02-upload/quota tests
have been failing since
|
||
|
MechaCat02
|
5009590882 |
feat(diashow): guarantee all eligible photos shown + 2048px display derivative
Diashow completeness rewrite so every eligible upload is shown regardless of
bursts, disconnects, or library size:
- queue.ts: SlideQueue with live/shuffle queues, allKnown map, recentlyShown
ring; merge(dedup, live-first), remove/removeByUser (prunes recentlyShown),
knownIds for reconcile-eviction. Adds queue.test.ts (burst/completeness/race).
- diashow/+page.svelte: reconcile (full paginate + evict, pre-scan snapshot to
spare concurrent uploads) on mount/reconnect/periodic; catchUpNew paginate-
until-known for bursts with debounced maxWait; hard-cut removals; decode
timeout + candidate fallback + bounded skip so a broken image never stalls.
New ~2048px "display" derivative for big-screen sharpness, decoupled from the
data-saver preview (800px) used on phones:
- migration 016: upload.display_path + v_feed rebuilt (DROP+CREATE, not REPLACE,
to slot the column beside preview/thumbnail).
- compression: generate_image_derivatives emits preview+display (downscale-only
guard, no upscaling); backfill_missing_display regenerates on startup (safe:
logs on error, never soft-deletes).
- upload.rs/main.rs: GET /upload/{id}/display (mirrors preview auth/cache),
/media/displays direct-serve blocked.
- feed.rs + types.ts: display_url in feed/delta DTOs.
- diashow candidate chain: display -> original -> preview.
Verified on the running stack: migration applied, 10/10 existing images
backfilled (2048px cap honoured, small images not upscaled), /display serves
200, /feed returns display_url, diashow cycles.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
fabi
|
ee554e7f38 |
style(backend): rustfmt the whole tree; gate cargo fmt --check in CI
The backend had never been run through rustfmt. Doing it in one mechanical pass (134 files) so no future functional diff is buried under formatting churn, then gating `cargo fmt --check` in checks.yml so it stays clean. Formatting only — no logic, SQL, or behaviour changed. Verified after the reformat: cargo test 56 passed, clippy --all-targets -D warnings clean, cargo fmt --check clean. This is the deferred cleanup noted when CI's Format step was first left out. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
fabi
|
db7c4459d7 |
fix(upload): editing a caption after release now regenerates the keepsake
edit_upload updated the caption/hashtags and nothing else. A caption lives in the HTML viewer keepsake (the ZIP holds media only — export.rs), so after release the downloadable viewer kept showing the OLD caption forever while the live feed showed the new one. Editing stays allowed while locked/released — like comments and likes, the lock freezes new uploads only (USER_JOURNEYS §9.3) — so the fix is to regenerate, not forbid: the edit and an invalidate_and_arm(ViewerOnly) share one transaction (same atomicity as delete_upload), then start_regen after commit. ViewerOnly carries the finished ZIP forward untouched since the media didn't change; when the gallery isn't released, invalidate_and_arm returns None and this is a no-op. Mutation-verified: without it, an edit doesn't bump the epoch and the caption test fails. Also (test isolation): the compression worker now carries a generation counter that TRUNCATE bumps. A worker queued on the concurrency semaphore when a truncate wiped media would otherwise wake in the NEXT test, fail to find its file, and broadcast upload-error/upload-deleted into that test's SSE stream — corrupting any test asserting on toasts or feed. It now abandons itself when the generation moved. No-op in production (TRUNCATE is the only caller). Export workers were already inert across a truncate (epoch guard on a fresh-UUID event). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
fabi
|
80179357a7 |
fix(review-2): high — read-only ban model, failed-compression cleanup, live SSE eviction
H1: corrected a round-1 over-reach. Bans are read-only per USER_JOURNEYS §10:
the base AuthUser extractor no longer rejects banned users (they keep read
access); Require{Host,Admin} and write handlers enforce the ban via
auth.is_banned → 403 (incl. self-unban). Demoted hosts still lose powers
immediately (live DB role) and are bounced off the dashboard. Also fixed the
login/recover redirect regression on unauthenticated 401.
H2: failed compression now self-heals instead of leaving a permanent broken
card — refund the quota, delete the orphaned original, soft-delete the row;
the frontend toasts the uploader and evicts the card on upload-error.
H3: self-delete, ban-with-hide (user-hidden), and comment-deletes now broadcast
SSE; feed, diashow, and lightbox evict the affected content live instead of
waiting for a reload. sse-eviction.spec.ts covers it.
Riders: feed/+page.svelte also carries the medium truncated-delta pill; host.rs
also carries the low atomic release_gallery claim; admin/+page.svelte also drops
the dead compression_concurrency control (medium).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
fabi
|
d9025f783a |
fix(security): port image-decode DoS cap + audit backlog/media-auth docs
From the 2026-06-27 audit branch (fix/audit-2026-06-27-critical-medium), whose work was ~80% absorbed into main via the batch branches. This ports the pieces that were NOT in main: - compression.rs: cap image decode with image::Limits (12000x12000, 256 MiB max_alloc) via ImageReader instead of image::open(). The upload body-size cap bounds the file on disk but not the *decoded* dimensions, so a small decompression-bomb image could OOM the box during decode/resize. Surgical port of the audit's decode guard only (not its image/video semaphore split). - docs/SECURITY-BACKLOG.md: the audit's triaged backlog, reconciled against main (each item tagged done / open / contingent-on-signed-media). Records the still- open items: host moderation UI gap, owner-delete SSE broadcast, quota mount-detection (starts_with) + low-disk guard. - docs/DECISION-media-auth.md: writes up the one real architectural divergence — main serves media unauthenticated (ServeDir + UUID-capability) while the audit built a signed/TTL'd gateway. Lays out the tradeoff (leaked URL = permanent vs ~24h access; banned-user access) and a recommendation, for a human decision. Verified: cargo build clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
141c918dd5 |
backend(infra): shared config helper, startup recovery, periodic maintenance
Foundations for the v0.16 features. No new endpoints here — those land in
the next commit on top of these.
- migrations 008 + 009: commit the load-bearing compression_status column
that was uncommitted on disk; add 009_feature_toggles seeding the master
+ per-endpoint rate-limit switches, the master + per-area quota switches,
and the admin-editable privacy_note.
- services/config.rs (new): get_str / get_i64 / get_usize / get_f64 / get_bool
consolidating the scattered helpers that lived in three handlers.
- services/maintenance.rs (new):
- startup_recovery() — resets compression_status='processing' and
export_job.status='running' rows orphaned by a previous crashed
instance, so users never see permanent "Wird vorbereitet…" spinners.
- spawn_periodic_tasks() — hourly cleanup of expired sessions (rows
were never pruned) + rate-limiter HashMap pruning (windows kept one
entry per IP forever).
- services/jobs.rs (new sketch): BackgroundJob trait + JobContext for
future jobs to plug into the same progress + SSE pipeline as
compression/export. Not wired yet — codifies the convention.
- services/compression.rs: 120s hard timeout + kill_on_drop on ffmpeg
so a malformed video can't hang and leak a worker semaphore permit.
- services/rate_limiter.rs: new prune() called from the periodic task.
- state.rs: SseEvent::new() constructor so event-type strings stay
consistent instead of being typed inline at every emit site.
- models/user.rs: UserRole::as_str() for /me/context serialization.
- models/upload.rs: soft_delete() now runs in a transaction and
decrements the uploader's total_upload_bytes (GREATEST(0, …) guard) —
fixes a quota drift where deleting reclaimed no quota.
- Cargo.toml + Cargo.lock: add `infer = "0.15"` (multipart MIME sniffing
used by the upload handler).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
fabi
|
3f052a4f91 |
feat: implement upload pipeline with compression and SSE
Backend:
- POST /api/v1/upload: multipart file upload with caption + hashtags
- Validates file size against DB config limits (image/video separate)
- Checks user ban status and event upload lock
- Saves original to disk under {media_path}/originals/{slug}/
- Tracks user total_upload_bytes for quota enforcement
- Extracts hashtags from caption text and explicit CSV field
- Upserts hashtags and links them to uploads
- PATCH /api/v1/upload/{id}: edit caption and hashtags (owner only)
- DELETE /api/v1/upload/{id}: soft-delete (owner only)
- GET /api/v1/stream: SSE endpoint with 30s keepalive
- Broadcasts new-upload events to all connected clients
- Uses tokio broadcast channel for fan-out
Services:
- CompressionWorker: Tokio semaphore-bounded (concurrency=2) background processor
- Images: resize to 800px wide JPEG preview via image crate
- PNG originals: lossless compression via oxipng
- Videos: ffmpeg thumbnail extraction (1 frame at 1s, scaled to 800px)
- Updates upload record with preview_path/thumbnail_path on completion
Models:
- Upload with full CRUD (create, find, update caption, soft delete, set paths)
- Hashtag with upsert, link/unlink, extract_hashtags() text parser
- UploadDto for API serialization with like/comment counts
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|