182e712a0e396bc3d114362767521a780d61fe66
38 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
fabi
|
9b38d31f97 |
fix(upload): stop a late retry from undoing a host takedown
Migration 026 freed the idempotency key as soon as deleted_at was set, so a
retry after a delete uploads afresh instead of 409ing forever. That rationale
only considered the GUEST deleting. deleted_at is also set by
host_delete_upload, and there the same rule reverses a moderation decision:
1. Guest uploads; the row commits and the photo appears, but the response
is lost on the way back — the flaky-wifi case the key exists for — so
the phone keeps the queue item.
2. The host takes the photo down. Epoch bumped, keepsake rebuilt without it.
3. The phone reconnects ten minutes later and retries. The key is free, the
INSERT succeeds, and the photo is back — in the feed and in the next
keepsake, under a NEW uuid that matches nothing in the host's moderation
history, with nothing logged to say a takedown was reversed.
Migration 031 keeps the key claimed for a host takedown and releases it only
for a guest's own delete, so the retry resolves to the duplicate path and is
refused. The refusal now says why ("von den Gastgebern entfernt") rather than
"already processed", which invites another try.
The index predicate and the ON CONFLICT arbiter are changed in lockstep;
these queries are not compile-checked, so a drift between them is a 500 on
exactly the retries the index exists to serve. Verified against a real
Postgres: live retry suppressed, host takedown holds the key, guest delete
releases it. The integration test's copy of the insert is updated too — it is
verbatim by design, and a stale copy would have kept passing.
|
||
|
fabi
|
7154b3a810 |
fix(upload): remove the /original rate limit that would have broken the feed
The limiter added here was justified as bounding "100 guests occasionally tapping Original anzeigen". That is not what this route is. `pickMediaUrl` resolves to `preview_url ?? thumbnail_url ?? /original`, and a freshly committed upload has BOTH derivatives null until the compression worker reaches it — at COMPRESSION_WORKER_CONCURRENCY=2 that is minutes during a post-ceremony burst. So /original is the feed's hot path for exactly the newest photos, in a newest-first grid, at the busiest moment. With every guest behind one NAT the 600/min bucket is venue-wide: six new photos fanned out by `upload-new` to ~100 open feeds exhausts it, and then every original fetch from anyone 429s for the rest of the window. The tiles' own 4-second retry uses a fresh `?r=` nonce, so the clients hold the bucket saturated themselves — the whole venue watching the newest photos render as broken tiles while the projector skips slides. A per-IP bucket cannot separate one scraper from the entire party when they share an address, and these media routes are unauthenticated by design (an `<img>` cannot send a bearer token), so there is no per-user key to move to. Bandwidth abuse belongs at the proxy. Also here: the release/lock check order. `release ⇒ lock`, so testing the lock first made the `GalleryReleased` arm unreachable dead code and every post-release upload answered `uploads_locked`. The codes are not interchangeable to the client — `uploads_locked` charges a retry attempt and re-pushes the whole photo on the backoff ladder against an answer that cannot change, while `gallery_released` parks it and says the photo is safe but the hosts must reopen. Both sites now test release first, so the fast path and the commit-time re-check agree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
MechaCat02
|
ef6d3a077a |
fix: close what nine adversarial reviews found, most of it mine
Some checks failed
Checks / Backend — cargo test + clippy + fmt (push) Failing after 1m5s
Checks / Frontend — vitest + svelte-check (push) Failing after 5m55s
Checks / E2E — typecheck + lint (push) Failing after 49s
E2E / Playwright E2E (chromium + webkit) (push) Failing after 10m42s
E2E / Cross-UA smoke matrix (push) Failing after 7m57s
Audit / cargo audit (backend) (push) Failing after 11m12s
Audit / npm audit (frontend) (push) Successful in 44s
Nine focused reviews (export state machine, upload path, auth/abuse, client queue, guest UI, database, deploy/ops, regression hunt, test honesty). Every finding below was re-verified against the code before being acted on; several plausible-sounding ones were checked and rejected. ## Data loss and denial of service **One request could OOM-kill the app container.** `client_upload_id` was read with `Field::text()` — axum builds its multipart reader with no SizeLimit, so the only bound was the route's 576 MiB body limit, then decoded into a second full String. `caption` and `hashtags` go through `read_text_field_bounded` for exactly this reason; this field arrived later and missed it. Any guest, one request, and every SSE stream drops and every in-flight temp file is stranded. **Nothing bounded concurrent upload bodies.** The headroom gate can only refuse to COMMIT — the body is already streamed to a temp file by the time it runs, and neither axum, the tower stack nor Caddy limits how many stream at once. ~100 guests tapping "upload all" after the ceremony puts 10-20 GB of .tmp on a 40 GB volume, invisible to the gate, eating the reserve that keeps Postgres able to write WAL. New `UploadAdmission` budgets bytes (not requests, so one video and two hundred photos coexist) via a permit that releases on drop, so every exit path returns it. **The export decode bypassed the memory permit the compression path takes.** Same class of work — decode + resize every image in the gallery — in a bare spawn_blocking. A release fired while the last photos were still compressing put both in the same 1 GiB cgroup; the OOM kill marks the export failed and `recover_exports` re-spawns it into the same conditions on the next boot. The permit is now process-wide in `imaging`, because the constraint it expresses is the container's memory, not one worker's. **`MediaTotalCache` cached its own failure as 0.** For the whole TTL the gate then saw an empty event and collapsed to the flat reserve — the behaviour the two-halves design replaced — with no log line. And the trigger correlates with the danger: with max_connections 10 the query fails exactly during a burst. Now falls back to the last good reading and says so. **V8's heap ceiling sat above the frontend container's entire budget** (measured: 259 MB inside a 256M limit), so GC could never intervene and the only backpressure was SIGKILL under an arrival burst. ## Guest-visible **The feed stopped being newest-first after the first reconcile.** It fetches whole 100-item server pages while `uploads` grows in 20s, so everything in the gap was absent from `present`, classified as new, and prepended — ~80 photos from earlier in the evening above the newest ones. It also stalled infinite scroll, since the cursor still pointed at item 20 and the observer only re-fires on a change. The union is now sorted on the server's own (created_at, id) key, which additionally places an SSE arrival correctly. **A stale `loadMoreError` outlived every refresh and filter change**, leaving a false error above a button that returns immediately on `!nextCursor`. **A failed derivative toasted "Ein Upload konnte nicht verarbeitet werden."** for a photo sitting right there on screen — the handler still assumed 1d9fb11's pre-fix behaviour (row deleted, quota refunded, card evicted), none of which is true any more. It was the last surviving route for the "your photo is gone" signal that fix set out to remove. ## Enforcement that existed only in comments `recover_name_rate_per_15min` is clamped at the point of use: the ordering `3 x ceiling <= PIN_LOCK_THRESHOLD` is the whole control against one source locking any guest whose name is on the feed, it was asserted in a comment, and `patch_config` accepted 1..100_000. The test pinned the default constant rather than the enforced bound; it now pins the bound. ## Tests that could not fail - The gate test asserted only its own premise (`500MB x 100 > 35GB`) and never touched the gate. It now checks both controls against the same state and requires them to disagree in the right direction. - `the_banner_always_fires_before_the_upload_gate_closes` reduced to `G < G + G/4` — true for any margin, including zero, so it could not detect the banner moving to exactly the gate. It now pins the gap. - `disk_is_low`'s `free < LOW_DISK_FLOOR_BYTES` clause was unreachable (warn_at is always >= 12.5 GB against a 10 GB floor). Two tests were named after it and neither could fail if it were deleted. Clause and constant removed. - The suspension test I added last commit hard-coded the credit cap instead of importing it, so changing STALL_TIMEOUT_MS would leave it passing against a system that no longer exists. Now imports MAX_SUSPEND_CREDIT_MS. ## Stale comments corrected The prune doc still argued at length for the pre-build ordering that |
||
|
MechaCat02
|
214f9e3062 |
fix: close four confirmed defects an adversarial review found
Findings from a multi-angle review, most of them in code I wrote in the last
few commits. Each was verified against the code before being acted on.
## Backend
**The export daily limit was bypassable ~60x/minute.** `SseTicketStore` is
untyped, and the export download quietly started reusing it. `POST
/stream/ticket` is free and rate-limited at 60/min per user; `POST
/export/ticket` charges one of three PER-DAY downloads. So a guest could mint
at the cheap endpoint and redeem at the expensive one, each redemption
streaming the whole multi-GB keepsake, `no-store`, off the same filesystem
Postgres writes WAL to. Tickets now carry a `TicketKind` and `consume` requires
it to match, asserted in both directions. The comment claiming "one mint is at
most one download" was simply false.
**`export_ticket` answered 200 `{"ticket": null}` when the store was full** —
after charging a daily slot. `issue` returns `Option`; `sse.rs` handles the
None with a 503 and this call site unwrapped it into the JSON body. The page
toasted success, the iframe navigated to `?ticket=null`, and one of three
downloads was gone. That is the phantom-success failure the pre-validation in
|
||
|
MechaCat02
|
05063694d2 |
fix: close eight regressions the audit pass found, five of them mine
Two adversarial reviews over |
||
|
MechaCat02
|
23e2f485dd |
fix(upload): correct two errors in the keepsake headroom gate
Both found reviewing my own change rather than by a test, which is the point. DOUBLE-SUBTRACTION. The gate computed `free - size`, but the body is streamed to its temp file during multipart parsing — far above the gate — so the free-space reading already excludes those bytes. Subtracting again refused uploads a full file-size early; with max_video_size_mb at 500 that is half a gigabyte of phantom pressure. `media_total` genuinely does need `+ size` (its row is not committed yet), which is what made the asymmetry easy to miss. BLOCKING SCAN ON THE HOTTEST PATH. It called `disk::free_bytes`, whose doc comment says it deliberately bypasses DiskCache — but that rationale is the export preflight's: a rare, high-stakes decision where a sibling worker can move free space by tens of GB inside the TTL. Per upload it means sysinfo re-scanning every mount, synchronously, on the async runtime, on a 2 vCPU box with two worker threads. Now uses the cached snapshot, the same 15s staleness the quota check immediately below already accepts for the same question. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
eb0e405562 |
fix: gate uploads on keepsake headroom, and close five unattended-event gaps
The box is 2 vCPU / 4 GB / 40 GB, not the 4 vCPU / 8 GB / 80 GB that the audit, the committed comments and README's sizing section all assumed. That correction is what the first change is about; the rest are the remaining pre-event items. THE ARCHIVE COULD BECOME UNBUILDABLE WHILE UPLOADS KEPT SUCCEEDING `required_free_bytes` is `media × 1.1 × 2` — the ZIP and the HTML viewer are each gallery-sized — and the export preflight also wants DISK_RESERVE_BYTES on top. The upload gate, though, only refused below a FLAT 10 GB reserve. On 40 GB that let uploads run to ~25 GB of media while a release needed `2.2 × 25 + 10` = 65 GB free. Every upload in that band succeeded and the keepsake could then never be built: the product's entire promise, failing silently at the end of the night with nobody there. The gate now enforces the invariant that actually matters — never accept an upload that would make the keepsake unbuildable — sharing `required_free_bytes` with the preflight so the two cannot drift into disagreeing about the same question. Uploads stop at ~8 GB of media on this disk, with a German message naming the cause. Refusing the 1001st photo beats losing all 1000. `media_total.rs` backs it: SUM(user.total_upload_bytes) over ~100 rows, cached 5s, rather than `estimate_export_bytes`'s join across every upload. It counts hidden and banned users' bytes, which the export excludes — skew in the SAFE direction, so the gate closes marginally early rather than late. Fails open on a query error. A test pins the gate against the preflight across the whole gallery-size range, and a second asserts the per-user floor alone would over-commit the volume — i.e. that the global gate is what must bind. THE WATCHDOG ABORTED HEALTHY UPLOADS EVERY TIME A PHONE WAS POCKETED `Date.now()` advances while a backgrounded phone is frozen but `setInterval` does not, so the first tick after a screen lock read the whole sleep as silence and aborted — re-sending a video from byte zero and burning one of five PERMANENT auto-attempts. The interval is now its own suspension detector: a tick that arrives 125s late for a 5s schedule credits that window back, because a period the watchdog could not observe is not evidence of silence. Chosen over a `visibilitychange` listener, which only covers causes that fire that event — a throttled-but-visible tab, a closed lid and an occluded window all freeze timers without one — and which would have needed module state, an SSR guard and a teardown for strictly less coverage. `performance.now()` was rejected because Safari pauses it across system sleep on some paths and Chrome does not. The credit buys one fresh window, not immunity: a socket iOS reaped while backgrounded still aborts ~90s after resume rather than hanging for `xhr.timeout` (5-60 min) with the queue's `processing` latch held. Two latent leaks found while in there: `xhr.abort()` on a request already in readyState DONE emits no `abort` event, so `settle()` never ran and the interval re-aborted every 5s forever while `activeUploads` kept a stale entry (the ✕ button silently stopped working); and a synchronous throw from `xhr.send` — a blob whose backing store the OS purged — leaked the same way. Both closed. OKLCH MADE THE DELETE BUTTON INVISIBLE ON SAMSUNG'S DEFAULT BROWSER red/amber/green were never in the @theme block and fell through to Tailwind v4's `oklch()` defaults, which Safari <15.4, Chrome <111 and Samsung Internet <22 cannot parse: `var(--color-red-600)` is then invalid at computed-value time, `background-color` falls back to transparent, and `.btn-danger` renders white text on nothing. Pinned to Tailwind's own defaults gamut-mapped to sRGB by Lightning CSS — the converter already in this pipeline — so modern browsers render exactly what they render today. Verified against seven hex fallbacks it had already emitted for the /alpha forms. rose and teal (avatar chips) had the same leak. The app CSS goes from 40 oklch declarations to 0. Also fixes `--color-purple-950`, which was simply missing: `dark:bg-purple-950/50` on the host dashboard was rendering default violet on EVERY browser, off-brand. The keepsake viewer only picks this up on a rebuild, so its committed artefact is rebuilt here too — still single-file, still zero external references. A BRICKED BOOT LOOKED LIKE A SPINNER FOREVER With `ssr = false` the page is empty until the bundle mounts, so a chunk 404 after a redeploy or a dead uplink left the guest on the boot spinner with no message, no reload control, and in a standalone PWA no URL bar. A 15s timeout in the existing nonce'd IIFE (no CSP change) swaps in German copy and a reload button. Deliberately a timeout rather than feature detection: a SyntaxError in the bundle is invisible to any capability check. Plus a <noscript>, since there was nothing at all to see without JS. EVERY 4xx WAS INVISIBLE AT ANY LOG LEVEL tower_http counts 4xx as a success, so it logs at DEBUG while production runs at info. If guests spend the evening hitting 429s or 413s, the post-event logs said nothing. Now one WARN per client error; 5xx excluded because Internal already logs its source chain and the pool-exhaustion 503 logs at construction. A DEAD FRONTEND SERVED A BLANK 502 `handle_errors 5xx` with an inline German page (the caddy service mounts only the Caddyfile, so there is no volume to ship a static file through). Verified empirically against this config, not from documentation: an upstream 404 through `reverse_proxy` still arrives as untouched `application/json`, and only a dial failure renders the page. That mattered — the keepsake download navigates a hidden iframe and DEPENDS on a real 404/429 arriving, and swallowing those would have been worse than the blank 502. CONFIG CORRECTIONS FOR THE REAL HARDWARE DATABASE_MAX_CONNECTIONS 30 → 15: sized to 2 vCPU rather than to the guest count. Since migration 024 a feed page costs well under a millisecond, so connections are no longer spent waiting, and 30 backends crowd the db container's 1 GB on a 4 GB host. COMPRESSION_WORKER_CONCURRENCY stays at 2 — the merged heavy-image permit already serialises anything over 150 MiB, so the "two 48 MP photos" worst case that number was sized against is unreachable; dropping to 1 would halve light-path throughput and push more feed tiles onto full-size originals. README's sizing section rewritten for the actual disk. Verified: 149/149 backend tests against a live Postgres, clippy clean, 57/57 vitest, svelte-check 0 errors, eslint clean, vite build, export-viewer rebuild, caddy validate, compose YAML parse. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
1d9fb11c7b |
fix: close the nine ways an unattended event loses photos or dies
Every one of these was found in the pre-event audit, verified against source, and
survives to production on the current main. Grouped by what actually goes wrong.
PHOTOS DISAPPEAR
* compression.rs no longer soft-deletes on a failed derivative. The guest got a
201, watched the card appear, then watched it vanish — the row left v_feed,
find_visible_media and BOTH keepsakes, while its bytes sat on disk for 14 days
waiting for a cleanup nothing announced. No screen anywhere lists compression
failures, so recovery meant hand-written SQL that also had to re-add the
refunded quota. Now it does exactly what the ENOSPC arm beside it already did
and documented as correct: keep the row, serve the original, retry on the next
boot (bounded by derivative_attempts). `upload-deleted` is no longer emitted;
`upload-processed` is, so the card re-renders instead of sitting on a
placeholder.
* A 413 is now a reversible lock, so the blob survives. The quota moves — free
disk falls, uploader count rises — so a guest goes over it having done nothing,
and treating that as permanent meant a 400 MB video was pushed across cellular
in full and THEN deleted from IndexedDB. Gone on both sides, and unrecoverable
for an in-app camera capture that exists nowhere else.
* quota_limit_bytes gained a floor and a stable divisor. The ceiling used to
decrease monotonically all evening; it now settles at max(uploaders,
estimated_guest_count) — a config key that was seeded, validated in the admin
whitelist, and read by no code at all. The floor is clamped to what the disk
can actually back, so a full volume still yields zero rather than handing out
an allowance it cannot honour.
* Because that floor gives up the aggregate guarantee the formula used to imply,
uploads now check a hard 10 GB reserve first, independent of every quota
toggle. postgres_data, media_data and exports_data share one filesystem: the
end state was not a degraded feature, it was Postgres unable to write WAL.
THE ARCHIVE DISAPPEARS
* prune_superseded_archives runs only after the new generation lands. It ran
before the preflight, reasoning the old archive was already unreachable — true
of reachability, false of recoverability. An epoch is a value that can be
rolled back; deleted bytes cannot. Any failed rebuild left the event with NO
keepsake at all.
* The export preflight reserves the same 10 GB. `free < needed` authorised an
export sized at exactly free, which ran for half an hour and landed the box at
zero with the keepsake still unfinished.
THE APP DIES
* The feed reconcile re-reads the id set after its awaits instead of reusing one
captured up to three round-trips earlier. The new-upload SSE handler prepends
during exactly that window, so the row was both already present and absent from
the stale set — prepended twice, and a duplicate key in a keyed {#each} throws
in production, not just dev. The SSE handler and loadMore now dedupe too.
* Added routes/+error.svelte. Without it any uncaught error fell through to
SvelteKit's unstyled English 500 with no reload control — inside a chromeless
standalone PWA with no URL bar, for the rest of the evening.
THE OPERATOR IS LOCKED OUT
* admin_login verifies the password BEFORE charging the rate bucket, and a
correct password is never throttled. The old order made this a denial of
service against its own operator: every guest shares one NAT IP, the check ran
first, so five requests a minute from any phone in the room kept the bucket
full — and the escape hatch needed the admin session being blocked. A generous
separate ceiling still bounds bcrypt CPU.
THE PROJECTOR DIES
* The preload budget is now strictly inside the dwell. At the 3s option the 4s
budget could never land a commit on a slow uplink, so the wall froze on one
photo while the queue drained silently behind it.
* The wake lock retries every 30s while visible, and the page says so on screen
when the browser has no wake lock API. visibilitychange was the only retry
trigger and a kiosk never changes visibility, so one refusal — iOS in Low Power
Mode, say — was permanent.
* Caddy: /api/v1/upload/*/display joins the cacheable carve-out. The backend set
max-age=300 on it and the blanket no-store silently replaced it, so a projector
re-fetched a full-size JPEG per slide, ~2-4 GB over an evening on the uplink
the guests are uploading over.
Also removes Upload::soft_delete, now unreferenced and an unscoped footgun next
to soft_delete_in_event.
Verified: 146/146 backend tests against a live Postgres, clippy clean, 51/51
vitest, svelte-check 0 errors, eslint clean, vite build, caddy validate, compose
YAML parse.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
61119be817 |
Merge branch 'fix/deploy-unattended-blockers' into main
Two independent lines of production hardening diverged at |
||
|
Fabian Hamm (Privat)
|
1d0df3ebf6 |
feat(upload): make uploads idempotent so a lost response cannot duplicate a photo
The ordinary mobile failure, not an exotic one: the server receives the body, validates it, commits the row — and the response is lost on the way back because the guest walked out of range or the AP dropped the connection. The client sees a network error with the blob still in hand and re-sends it, both when the guest taps "Erneut" and automatically when the queue requeues on reconnect. Every attempt minted a fresh `Uuid::new_v4()` server-side, so the same photo landed in the gallery two or three times and was charged against the guest's storage quota each time. The client already has a stable per-queue-item UUID, so it costs nothing to send. Migration 022 adds `client_upload_id` with a partial unique index — partial so the NULLs of every pre-022 upload, and of any caller that doesn't send one, keep working untouched. Two paths, because there are two races: - Sequential retry: a lookup before the transaction finds the stored row, deletes the re-sent bytes and replays the original response as 200. The body has necessarily already been streamed, since the key arrives as a multipart field — re-sending is the client's cost and is already paid by the time we see it. What must be prevented is a second ROW. - Concurrent retry: two attempts in flight at once. `ON CONFLICT DO NOTHING` returns no row to the loser, which abandons its transaction (quota increment included) and replays the winner. Letting the unique index raise instead would only surface after the transaction had aborted, as an opaque error the caller would have to string-match. The replay reads live state rather than assuming a fresh row: a reconnect can be minutes later, by which time the derivatives may exist and the photo may have been liked. Every read there fails soft — the upload is already safely stored, so a sparser response is fine and failing the request is not. Verified live: the same photo sent three times returns 201, 200, 200 with one id, one row, and the quota charged exactly once. Also in this file: the two image-header probes at admission now run on `spawn_blocking`. Both open the file and run the codec's header parse synchronously, and `#[tokio::main]` gives two worker threads on a 2-vCPU box — so every upload stalled half the runtime's request-serving capacity. Everything else that blocks here (image encode, bcrypt) was already offloaded; this was the one that wasn't. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
MechaCat02
|
117e67fa80 |
fix(auth,upload): close the admin lockout and four unbounded-input paths
ADMIN LOCKOUT. admin_login looked its user up BY NAME. Migration 007 makes
display_name unique per event case-insensitively and join had no reserved-name
guard, so any guest joining as "admin"/"Admin"/"ADMIN" before the operator's first
login made find(role == Admin) miss, the fallback create("Admin") violate that
index, and `?` return a 500 — permanently, with no in-app recovery. Moderation,
config and gallery release all gone; the fix was hand-editing the database.
The root cause is the lookup key, not the creation. The name was never the
identity. User::find_admin_for_event resolves by role, which makes the whole class
of name collisions irrelevant — including the homoglyph bypasses of the new
reserved-name list, which is now defence in depth rather than the control.
Promoting the squatting row would be the obvious fix and is a serious mistake: it
carries a recovery_pin_hash the guest knows, so it would hand them the admin
dashboard via /recover, permanently, through a path needing no password. A
separate row under a fallback name is worse UX and much better security. Verified
against the real schema — the guest keeps their uploads, PIN and session under a
freed name, and the role lookup then finds exactly one admin.
Second, independent bug in that block: create() followed by a SEPARATE UPDATE ...
SET role = 'admin' manufactures the same poisoned state if anything fails between
them. Collapsed into create_with_role.
UNBOUNDED INPUTS — one root cause, four places: validation ran after the
allocation.
- upload caption/hashtags used Field::text(), which buffers the whole field, on
the one route whose DefaultBodyLimit is 576 MiB — so 576 MiB of heap per
concurrent request in a 1 GiB container, with the length check running
afterwards on a string already built. Now refused mid-read.
- the hashtag CSV was never length-checked at all and was upserted tag by tag
INSIDE the commit transaction, which holds FOR SHARE on the event row — one
request could stall every other upload behind tens of thousands of round trips.
Capped at 30 tags of <=50 chars.
- /recover and /recover/request built rate-limiter keys by format!() from an
unvalidated, unbounded display name, retained up to 24h in a map pruned hourly:
the limiter itself became the memory-exhaustion primitive it exists to prevent.
join validated first; that check is now shared by all three. /recover/request
also had no per-IP ceiling at all — /join got one in 017, /recover in 019, and
019's own comment describes exactly this attack. It returns 204 rather than 400
on a bad name, because a 400 would be a new signal on an endpoint whose contract
is that it cannot enumerate guests.
- the SSE ticket store had no size cap, no per-session cap and no rate limit on
its endpoint, while prune ran hourly against a 30s TTL. Now pruned on issue,
capped, and rate-limited. At capacity it REFUSES rather than evicting a
stranger's ticket — evicting would let one client deny SSE to the venue. Not
one-ticket-per-session either: two tabs open their EventSources concurrently.
PATCH /upload/{id} had no rate limit, no validation, and called
invalidate_and_arm unconditionally — outside both `if let Some` guards. So
PATCH {} bumped export_epoch and armed a fresh pair of full-gallery export workers
every call; REGEN_DEBOUNCE bounds the rate of that, not the total work, so a guest
could keep the keepsake permanently un-downloadable. All three fixed. The
validation also resolves a divergence: upload normalised tags while edit stored
them raw, so #Party via edit and party via upload became two hashtag rows.
PIN LOCKOUT was an ordering bug before a policy one: the account-lock threshold
(3) sat BELOW the per-(IP, name) ceiling (5), so three requests from one IP locked
any guest whose name is on the feed, every 15 minutes, forever. The tier meant to
protect a guest was the cheapest way to attack them. Ceiling drops to 4, threshold
rises to 12, so locking a victim now needs at least three distinct sources.
Brute-force cost is unchanged — 48 attempts/hour means 10k PINs still take ~208h
regardless of IP count — and increment_failed_pin now decays the streak after 15
minutes, since the counter previously only cleared on success and honest typos
accumulated across days. Both invariants are pinned by tests rather than comments.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
e52b2f1cd1 |
fix(upload): reclaim the bytes of uploads that never finish
Reclaim was a dozen explicit remove_file calls on the handler's return paths. That covers every way the handler can FINISH and none of the ways it can simply STOP. When a client disconnects mid-body — a phone leaving wifi, iOS killing a backgrounded PWA, the user hitting back — axum drops the handler future at an .await inside field.chunk() and no return path runs at all. The partial file then survives forever: it has no upload row, so cleanup_deleted_media (row-driven) can never see it, and no sweeper covered the originals directory. The triggers are routine rather than adversarial, and the client keeps the blob and auto- retries on every `online` event, so one large video over bad wifi leaves several copies. Those bytes are also invisible to the quota while still consuming the free disk that compute_storage_quota divides among guests — so orphans silently shrink every guest's ceiling while the admin widget under-reports. All three volumes share one filesystem; the end state is Postgres unable to write WAL. TempFileGuard is an RAII guard, because dropping the future is exactly what runs Drop — it is the only construct that survives cancellation. Armed before the file can exist, disarmed only after tx.commit() succeeds. The twelve explicit cleanups are deleted so one owner holds the rule. The subtler half is the rename. It happens BEFORE the commit, so between them the file exists under its final name with no row pointing at it — an orphan that looks legitimate. The guard is RETARGETED there rather than disarmed, and the retarget sits on the same poll as the rename with no .await between, which is what makes that window uncancellable. sweep_orphan_originals is the backstop for the process that was killed, where no Drop can run at all. Hourly, alongside the existing sweeps: .tmp files past the window go unconditionally (a .tmp never has a row by construction), other files are batched 500 at a time through a single NOT EXISTS query. Two things that look like oversights and are not, both commented in place: - the 6h window is what makes the sweep safe against the rename-before-commit ordering, since a committing upload is briefly indistinguishable from an orphan. It must not be shortened to speed up a test. - the NOT EXISTS deliberately does NOT filter deleted_at IS NULL. A soft-deleted row still points at its file during its retention window, and reclaiming that is cleanup_deleted_media's job; filtering here would race the two sweeps and destroy the files the recovery window exists to preserve. Verified against the real schema: given a live original, a soft-deleted one and a true orphan, the query returns only the orphan. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
fabi
|
31faccfdf8 |
fix(audit-4): refuse undecodable images at admission, stop retrying permanent failures
Fourth round: reject at the door what the worker can only fail on, and stop retrying errors that cannot succeed — while keeping IoError retryable so ENOSPC still gets another attempt. Squashed from 2 commits, original messages preserved below. ──────── fix(upload): refuse undecodable images at the door, and stop retrying them Two halves of the same complaint: an oversized photo was accepted with a 201 and then silently soft-deleted minutes later, after the worker had burned six seconds of backoff re-reaching a conclusion it could not change. Admission. The compression budget now runs at upload time, against the header only, so a guest is told immediately and told why: "Bild hat zu viele Bildpunkte (ca. 99 Megapixel) und kann nicht verarbeitet werden. Bitte verkleinere es und lade es erneut hoch." instead of watching the photo vanish behind a vague "could not be processed" — which arrived only if they happened to still be on the feed with that card loaded. Nothing is stored, so there is no row to soft-delete and no orphan for the sweep to reclaim. Admission and the worker share ONE function (`decoder_within_budget`), so they cannot drift apart and start disagreeing about what is acceptable — a photo accepted at the door and rejected by the worker would be worse than either behaviour alone. The worker keeps its own check: the backfill decodes files that predate this check, and defence in depth is the whole reason the budget exists. Retries. The loop retried every failure, including ones that are a property of the input. An image over the budget, a corrupt file, an unsupported format: each fails identically on all three attempts, so the only effect was 2s + 4s of sleep and three near-identical warnings before the same outcome. `is_permanent_image_error` classifies the `ImageError` variants that cannot change between attempts — Limits, Unsupported, Decoding — and the loop gives up on those at once. `IoError` is deliberately excluded: an ENOSPC while writing a derivative is exactly the transient case the retry exists for, and misclassifying it would turn a blip back into the data loss round 1 fixed. Measured: retry log lines went from 3 per oversized upload to 0. Tests: unit tests for both sides of the classifier (a Limits error is permanent, a missing file is not) and for admission agreeing with the decoder on accept AND reject. The e2e spec is rewritten for the new contract — 400 with an actionable message, nothing stored, backend alive after a burst of four — plus a mirror asserting an ordinary photo still uploads and processes, since a budget that rejected everything would satisfy the other two. ──────── fix(upload): narrow the admission check to the memory budget only The admission check I just added rejected ANY image the decoder couldn't build — corrupt, truncated, or unsupported, not only over-budget. That broke two adversarial tests, and they were right to break. 07-adversarial/file-upload-attacks pins, deliberately, that acceptance follows the MAGIC BYTES: a payload whose first three bytes are a JPEG header is accepted regardless of what follows, because the security property under test is that the client-declared Content-Type has no influence. Both failing cases upload 1024 bytes of JPEG magic followed by zeros. Rejecting those at admission is a different, broader contract than the one asked for, and rewriting an adversarial test to match new behaviour is precisely the thing that needs justifying rather than doing quietly. So admission now checks only what it was meant to: `exceeds_decode_budget` returns true solely for `ImageError::Limits`. A corrupt file goes to the compression worker exactly as before — which handles it gracefully and, since the retry classifier in the previous commit, no longer burns backoff on it. The resource guard is the part that had to move earlier; nothing else did. Tests: the size agreement between admission and the worker is still asserted in both directions, plus a new one writing a magic-bytes-only stub and asserting admission accepts it WHILE the worker still rejects it — pinning the boundary between the two checks so a future widening fails here rather than in the adversarial suite. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
fabi
|
b601c062bd |
fix(audit-2): video playback, role identity, keepsake EXIF, recover ceiling, WebKit CI
Second round: the two regressions the first round introduced, the remaining
gaps it left, and the CI change that makes the iOS guarantees actually gate a
change rather than being asserted.
Squashed from 9 commits, original messages preserved below.
──────── docs(deploy): document the update path — `up -d` alone ships nothing
The README only ever described a fresh install. There was no update section
anywhere, and `--build` appeared nowhere in the docs.
That matters because `app` and `frontend` are `build:` services with no published
image tag, and Compose has no source-change detection: if an image by that name
exists it is reused. So the natural `git pull && docker compose up -d` reports
"Container app-1 Running", rebuilds nothing, and exits 0. A deploy that shipped
none of the new code is indistinguishable from a successful one — which is how
eleven merged fixes can sit in the repo and never reach the box.
Verified both halves against a real stack rather than asserting them: with a
source change staged, `up -d` left the image ID untouched; `up -d --build`
produced a new image ID and a healthy /health.
Adds an "Updating an existing deployment" section covering backup-before-migrate,
pull, rebuild, health check, and an image-ID comparison to prove a build actually
happened. Also spells out the rollback trap: migrations run on boot and are not
undone by checking out an older commit, so rolling back code without restoring
the snapshot leaves the schema ahead of the binary and the app refusing to start.
──────── fix(video): play the actual video, and answer Range requests
Every video in the app was unplayable. Two independent defects, either one
sufficient on its own, and nothing in the suite covered either — no test
anywhere played media or asserted a `<video>` src.
1. The lightbox handed `<video>` a JPEG.
`pickMediaUrl` is mime-agnostic, and compression only ever produces a THUMBNAIL
for a video (one `ffmpeg -vframes 1` frame) — no preview, no display. So in the
DEFAULT saver mode the element's src resolved to `/api/v1/upload/{id}/thumbnail`,
served as `image/jpeg` with `nosniff` so the browser can't even sniff its way
out. Chromium reports DEMUXER_ERROR_COULD_NOT_OPEN.
Fixed in the lightbox rather than in `pickMediaUrl`: FeedListCard shares that
helper and legitimately wants the thumbnail for its `<img>` poster, so a central
mime branch would break the feed. This mirrors the rule the diashow already
applies ("videos play the original file directly"). Added `preload="none"` so
saver-mode guests on cellular still fetch nothing until they press play — there
is no smaller video derivative to offer them — plus `playsinline`, without which
iOS hijacks playback into fullscreen.
2. `stream_media_file` ignored Range entirely.
It took no request headers, so it could not see `Range`; it always returned 200
with the whole body and never sent Accept-Ranges or Content-Range. iOS Safari
opens every `<video>` with a `Range: bytes=0-1` probe and abandons the load
without a 206 — so video failed on the app's primary platform even in `original`
mode, where the src was already correct.
Adds single-range support (`bytes=N-`, `bytes=N-M`, `bytes=-S`) with 206 +
Content-Range, 416 + `bytes */len` past EOF, and Accept-Ranges advertised on
every response. Anything it won't handle — multi-range, non-bytes units, garbage
— falls back to a full 200, which RFC 9110 explicitly permits and which is safer
than guessing. All four media routes share the helper, so seeking works
uniformly.
`get_original` now serves `inline` instead of `attachment`. An attachment
disposition is hostile to a `<video>` element, and this route is the only source
of playable video bytes; it also matches what the UI promises, since the action
is labelled "Original anzeigen" — view, not download. `no-store` is deliberately
kept so a takedown still revokes access promptly; ranges work fine under it, the
client just re-fetches.
Tests: 11 unit tests pin the parser (the iOS `bytes=0-1` probe, inclusive ends,
suffix ranges, clamping past EOF, 416 vs 200, malformed fallbacks). A new
03-feed/video-playback spec asserts the src is the original and not the
thumbnail, that the browser accepts the bytes as media (readyState > 0, no
MediaError), that no video bytes are delivered before play, and that Range
returns the correct 206 slices and a 416 past EOF — verified on both Chromium
and WebKit.
The "not downloaded before play" test asserts no *delivered body* rather than no
request: WebKit opens a connection for a preload="none" video and immediately
aborts it (GET, no Range, status 0, nothing transferred) while Chromium issues
nothing at all. The portable guarantee is that no response carrying bytes
completes.
──────── fix(auth): bind the role store to the identity, not to the tab
The role store I added in the moderation work is a module-level singleton seeded
ONCE at import. `goto()` is a client-side navigation, so leaving and re-joining in
the same tab re-imports no module and re-runs no onMount — the previous user's
role simply stayed resident. Nothing reset it: not join, recover, admin login,
"Event verlassen", `clearAuth`, nor the api.ts 401 auto-clear.
So a host who left, followed by a guest joining on the same phone, left that guest
with `isStaff === true` and a "🚫 Beitrag entfernen" action on other people's
photos. The backend 403s the delete, so this was a false affordance rather than a
privilege escalation — but `/feed` never fetched `/me/context`, so unlike every
other route it never self-corrected either. It survived until a hard reload.
The mirror case was equally broken and easier to overlook: a guest who recovered
into a host account got NO host affordances.
`clearAuth` already had a hook registry for exactly this shape of problem, with a
comment explaining it exists to avoid circular imports. Add the missing mirror,
`onSetAuth`, fired by both `setAuth` and `setAdminAuth` after the new token is
resident, and have the role store register on both sides: clear to null on
logout, re-seed from the new token on login. That also gives
`syncRoleFromToken` — dead code with zero callers since I introduced it — its
intended purpose.
Seeding from the claim fixes the reported bug, but the claim is frozen for the
token's 30-day life, so a promotion or demotion still wouldn't reach the feed.
`/feed` now calls the existing `refreshEventState()` on mount, which fetches
`/me/context` and applies both the authoritative role and the lock/release state
in one request. The feed is the one route gating a destructive action on the role,
so it should not be the only route running on a stale claim.
Tests: 04-host/role-identity-reset drives the real flows. The first asserts the
host DOES see the action before asserting the newcomer does not — a negative
assertion alone would pass against a build that shipped no moderation at all. The
second covers the mirror, promoting a guest server-side while their resident token
still claims `role: guest`, so a fix that only cleared the role would fail it.
──────── fix(compression): reclaim failed originals instead of leaking them
Round 1 stopped the compression worker deleting an upload's original on failure —
a transient ENOSPC or a codec panic must never destroy the only copy of a photo a
guest cannot retake. But it left `Upload::soft_delete`'s quota refund in place, so
the bytes stayed on disk while the uploader was charged nothing for them.
That is worse than it first looks. The row is soft-deleted, so the file is
invisible and unowned; a guest hitting a reproducible codec failure can accumulate
orphans indefinitely at zero personal cost. And `active_uploaders` counts only
users with non-deleted uploads, so dropping out of that count RAISES everyone's
per-user ceiling — the leak loosens the very quota meant to contain it.
Keep the refund: the uploader didn't cause the failure and shouldn't silently lose
quota to it. Bound the leak instead, with an hourly sweep alongside the existing
session cleanup in `spawn_periodic_tasks`, reclaiming failed originals older than
14 days — comfortably longer than any single event, so an operator investigating a
failed upload still has the file.
The selection predicate is the entire safety argument, so it is deliberately
narrow: `compression_status = 'failed'` AND soft-deleted AND past the window AND
`original_path <> ''`. That is exactly the state the give-up path leaves behind,
and it cannot reach a live upload, an owner-deleted one, or a failure still inside
its recovery window. `original_path` is cleared after a successful reclaim, which
makes the sweep idempotent — otherwise a row whose file is already gone is
re-selected on every tick forever. The row itself is kept as the audit trail.
Tests reproduce the selection verbatim (same pattern as upload_concurrency) and
assert it against five near-misses that must survive, both sides of the retention
boundary, and the idempotence property.
Also fixes two comments in export.rs still claiming "the compression worker
hard-deletes an original when its transcode fails" — no longer true, and the
defensive handling they justify is now justified by this sweep and by ordinary
deletes instead.
──────── fix(export): apply EXIF orientation in the keepsake too
Round 1 fixed EXIF orientation in the compression worker, which corrected the live
app — feed preview and diashow display. The export worker was missed, and it does
not reuse those derivatives: it re-decodes the originals itself with `image::open`,
which ignores the orientation tag, then re-encodes to JPEG, which drops the tag —
so the viewer has no way to recover it.
The damage was oddly shaped, which is exactly why it reads as a viewer bug:
Gallery.zip originals correct (byte-copied, EXIF intact)
Memories viewer grid thumbnails SIDEWAYS (always)
Memories viewer full image >5 MB SIDEWAYS (re-encoded at 2000px)
Memories viewer full image ≤5 MB correct (streamed byte-for-byte)
So in the keepsake people actually keep, every portrait photo in the grid was on
its side, and clicking through silently "fixed" small photos but not large ones.
Rather than paste the decoder dance a third time, extract `services::imaging::
decode_oriented` and route both workers through it, so there is exactly one way to
turn a file on disk into a DynamicImage. It carries a second invariant that had
also drifted: `image::open` applies NO decode limits, so the export path was
decoding arbitrary user-supplied images unbounded — the decompression-bomb cap
existed only in the compression worker. Both now come as a pair, which is the
point of having one function.
Not done: switching export to consume the existing `display` derivative. It would
fix orientation and drop a redundant full-resolution decode per photo, but it
would also replace the pristine ≤5 MB originals in the keepsake with 2048px
re-encodes — a real quality regression in the one artefact people keep forever.
Test uploads the round-1 fixture (40x20 landscape tagged Orientation=6), runs a
real export, pulls the thumbnail out of Memories.zip and asserts it came back
portrait — with a sanity check that the source really is stored landscape, so the
test can't pass against a pipeline that does nothing.
──────── fix(recover): cap name cycling, and stop bcrypt blocking the runtime
Round 1 gave /join a per-IP ceiling and left /recover with only its
`recover:{ip}:{name}` bucket. That key is right for the job it was written for —
stopping someone who knows a display name (they're listed on the feed) from
burning the victim's 3-strike PIN counter and locking them out on repeat. But the
name is ATTACKER-CHOSEN, so cycling names mints a fresh 5-attempt bucket every
time and the per-IP cost is unbounded.
What sits behind that limiter makes it worse than a normal flood: every call runs
a cost-12 bcrypt verify, including an UNCONDITIONAL throwaway verify for names
that don't exist — added deliberately to close a timing oracle. So an unknown name
is the single cheapest way to make the server do ~200ms of hashing.
Adds `recover_ip_rate_per_min` (default 30, migration 019), checked BEFORE the
per-name bucket so a name generator can't walk past it. 30/min is far above any
real recovery attempt while capping a flood. The per-name bucket is untouched and
remains the anti-guessing control.
The second half matters as much as the first: bcrypt was running inline on the
async runtime everywhere. At cost 12 that pins a tokio worker thread for ~200ms,
and there is only one per core — so a login flood stalled every other request on
the box, including the feed. There was no spawn_blocking anywhere in the auth
module, despite SECURITY-BACKLOG claiming bcrypt had been offloaded.
Route all of it through `verify_password` / `hash_password` on the blocking pool.
That covers /recover, /admin/login, the host PIN reset, and — the one most likely
to bite at a real event — the PIN hash minted on every single /join. Saturating
the blocking pool degrades logins; saturating the worker threads degrades
everything.
Tests: cycling distinct names from one IP now hits the ceiling with a Retry-After,
and — the assertion that keeps the fix honest — repeated wrong PINs against ONE
name are still throttled with the ceiling set generously high, so the ceiling
added protection rather than replacing it.
──────── ci(e2e): run WebKit, so the iOS guarantees actually gate a PR
The workflow installed only Chromium and ran chromium-desktop + chromium-mobile.
iOS Safari is the app's stated primary user — a wedding guest opening a QR link —
and WebKit is the only engine in the matrix that reproduces two of its behaviours:
- it enforces X-Frame-Options on the hidden download iframe, so a site-wide DENY
makes the keepsake download silently do nothing. Blink hands attachments to
the download manager before the frame check and never notices.
- it abandons a <video> load unless its Range probe gets a 206.
Both of those shipped. Adding 06-export to the webkit project in the round-1 fix
bought nothing on a PR, because CI never ran that project at all — the regression
test written specifically to catch the blocker only ever executed locally.
Runs 71 tests (67 pass, 4 skip on the documented IndexedDB-blob harness
limitation) in ~1.5 minutes locally, using the exact command added here.
──────── chore(backend): satisfy cargo fmt
`checks.yml` runs `cargo fmt --check`, and it has been failing since the round-1
audit fixes: I gated those on `cargo build` and `cargo clippy` but never ran fmt,
so three files drifted then and eight more this round. Pure formatting — no
behaviour change; clippy stays at zero and all 70 backend tests still pass.
Worth noting for next time: clippy passing is not evidence fmt does.
──────── chore: satisfy prettier in frontend and e2e
`checks.yml` runs `npm run format:check` for both projects and both were failing.
- frontend/src/lib/ui-store.ts is mine, unformatted since the round-1 upload-queue
badge fix — the same miss as the rustfmt one: I gated on svelte-check and eslint
but never on format:check.
- e2e/loadtest/* and e2e/shots.mjs have been unformatted since
|
||
|
MechaCat02
|
5009590882 |
feat(diashow): guarantee all eligible photos shown + 2048px display derivative
Diashow completeness rewrite so every eligible upload is shown regardless of
bursts, disconnects, or library size:
- queue.ts: SlideQueue with live/shuffle queues, allKnown map, recentlyShown
ring; merge(dedup, live-first), remove/removeByUser (prunes recentlyShown),
knownIds for reconcile-eviction. Adds queue.test.ts (burst/completeness/race).
- diashow/+page.svelte: reconcile (full paginate + evict, pre-scan snapshot to
spare concurrent uploads) on mount/reconnect/periodic; catchUpNew paginate-
until-known for bursts with debounced maxWait; hard-cut removals; decode
timeout + candidate fallback + bounded skip so a broken image never stalls.
New ~2048px "display" derivative for big-screen sharpness, decoupled from the
data-saver preview (800px) used on phones:
- migration 016: upload.display_path + v_feed rebuilt (DROP+CREATE, not REPLACE,
to slot the column beside preview/thumbnail).
- compression: generate_image_derivatives emits preview+display (downscale-only
guard, no upscaling); backfill_missing_display regenerates on startup (safe:
logs on error, never soft-deletes).
- upload.rs/main.rs: GET /upload/{id}/display (mirrors preview auth/cache),
/media/displays direct-serve blocked.
- feed.rs + types.ts: display_url in feed/delta DTOs.
- diashow candidate chain: display -> original -> preview.
Verified on the running stack: migration applied, 10/10 existing images
backfilled (2048px cap honoured, small images not upscaled), /display serves
200, /feed returns display_url, diashow cycles.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
MechaCat02
|
3c3a7d0082 |
fix(diashow): open SSE on mount; raise upload rate 10→100/hour
diashow only subscribed to SSE events but never called connectSse(), so a kiosk/projector opening /diashow directly (not via /feed) never opened the EventSource — the showcase display got a one-time /feed snapshot and no live updates, showing "Noch keine Beiträge" forever when turned on before any photos. Open the stream in onMount (idempotent) and close it in onDestroy, mirroring the feed page. Raise the default upload_rate_per_hour from 10 to 100 (migration 015, scoped to installs still on the old default so admin overrides are preserved). Guests routinely upload bursts of 10-20 photos; the old default throttled the first burst. Also update the code fallback and the test-mode reseed. Both verified end-to-end against the docker test stack. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
fabi
|
ee554e7f38 |
style(backend): rustfmt the whole tree; gate cargo fmt --check in CI
The backend had never been run through rustfmt. Doing it in one mechanical pass (134 files) so no future functional diff is buried under formatting churn, then gating `cargo fmt --check` in checks.yml so it stays clean. Formatting only — no logic, SQL, or behaviour changed. Verified after the reformat: cargo test 56 passed, clippy --all-targets -D warnings clean, cargo fmt --check clean. This is the deferred cleanup noted when CI's Format step was first left out. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
fabi
|
db7c4459d7 |
fix(upload): editing a caption after release now regenerates the keepsake
edit_upload updated the caption/hashtags and nothing else. A caption lives in the HTML viewer keepsake (the ZIP holds media only — export.rs), so after release the downloadable viewer kept showing the OLD caption forever while the live feed showed the new one. Editing stays allowed while locked/released — like comments and likes, the lock freezes new uploads only (USER_JOURNEYS §9.3) — so the fix is to regenerate, not forbid: the edit and an invalidate_and_arm(ViewerOnly) share one transaction (same atomicity as delete_upload), then start_regen after commit. ViewerOnly carries the finished ZIP forward untouched since the media didn't change; when the gallery isn't released, invalidate_and_arm returns None and this is a no-op. Mutation-verified: without it, an edit doesn't bump the epoch and the caption test fails. Also (test isolation): the compression worker now carries a generation counter that TRUNCATE bumps. A worker queued on the concurrency semaphore when a truncate wiped media would otherwise wake in the NEXT test, fail to find its file, and broadcast upload-error/upload-deleted into that test's SSE stream — corrupting any test asserting on toasts or feed. It now abandons itself when the generation moved. No-op in production (TRUNCATE is the only caller). Export workers were already inert across a truncate (epoch guard on a fresh-UUID event). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
fabi
|
e5201a9889 |
test(backend): DB-backed tests for the risky SQL; test isolation; clippy cleanup
All 40 backend tests were pure-function tests — not one line of SQL ran under `cargo test`,
even though the riskiest code in the repo is SQL. Add 12 DB-backed `#[sqlx::test]` tests
(fresh throwaway DB per test, real migrations) pinning the invariants that code is
load-bearing for, each mutation-verified to fail on broken code:
export_epoch.rs (8): epoch is post-increment on release (a worker born pre-increment is
inert); epoch strictly monotonic across reopen; a retired-epoch worker's writes are
no-ops; export_current is EXACTLY the invariant (8-case table); the ViewerOnly
carry-forward carries a done ZIP forward AND matches nothing when the ZIP is unfinished
(the
|
||
|
fabi
|
9666d74a46 |
fix(export): close all 13 findings from the multi-agent review
A 6-lens multi-agent review (103 agents; every finding adversarially verified by three
refute-by-default skeptics) confirmed the epoch redesign is sound — the core invariant
holds — and found 13 real defects around it. All are fixed here.
The redesign's invariant was re-verified and stands: an accepted upload is always in the
keepsake, and a downloadable keepsake always reflects the most recent release.
But a WEAKER adjacent invariant did NOT hold — "a downloadable keepsake reflects the
current content" — and that is the theme of the biggest fixes.
CONTENT REMOVAL NOW ALWAYS REACHES THE KEEPSAKE
Regeneration was wired only to host deletes. Ban, unban, guest self-delete and guest
comment-delete did not trigger it — yet the export query filters `is_banned = FALSE`,
so the pipeline already agreed that content must not be there. Ban someone for abusive
content after release and every guest kept downloading an archive containing it.
All five removal paths now invalidate and rebuild. `query_comments` also gained the
banned/hidden filter it was missing entirely (the ZIP had it; the viewer did not).
Each removal is now ATOMIC with its invalidation (one tx, models made executor-generic).
Previously the delete committed in its own tx and the regeneration in a second: a dropped
handler future between them left the taken-down photo permanently downloadable, and
nothing could notice — the keepsake still looked complete and the host could no longer
even find the upload to retry.
NO MORE TAKEDOWN STORM
Every removal spawned two uncancellable full exports. A superseded worker was INERT, not
STOPPED: it still ran every ffmpeg spawn and image resize and wrote a whole archive before
discovering it had lost. Five deletes left ten workers alive, each holding a
full-gallery-sized temp, in a 1 GB container.
Now: a debounce before `claim_job` (a superseded worker fails its claim and does ZERO
work, collapsing a burst into one export), plus `update_progress` — which already ran
exactly the right liveness predicate and threw the answer away — returning it so both file
loops bail the instant they are retired. Moderating a comment no longer rebuilds the
multi-GB ZIP either: the ZIP is media-only, so it is carried forward, with prune taught to
protect any file a live row still references.
AN ESCAPE HATCH
A failed export was terminal at runtime: release_gallery refuses an already-released event,
recovery only runs at boot, and there was no retry route — the only outs were restarting the
container or reopening uploads to every guest. Added POST /host/export/rebuild. Also widened
mark_failed's guard to `status IN ('running','pending')`: when claim_job itself ERRORED, the
row was still `pending`, so the failure was never recorded and the job sat at 0% forever.
SILENT INVALIDATION
Retiring the epoch 404s the download instantly, but nothing told the clients — guests kept
seeing an enabled download button through the whole rebuild. Now broadcasts an invalidation.
And the download was a top-level <a href>: a 404 (or a 429 from the per-IP export limit that
everyone on the venue WiFi shares) NAVIGATED THE USER OUT OF THE PWA onto raw JSON. It now
targets a hidden iframe, so an error response is harmless.
ALSO
- The HTML export still had the exists()-then-open TOCTOU the ZIP path was fixed to remove,
at three sites, two with a hard `?` that failed the ENTIRE viewer. It is reachable: the
compression worker hard-deletes an original when a transcode fails and can still be running
when the gallery is released.
- Crash-orphaned .tmp files were immortal (prune deliberately skips .tmp). Swept at boot,
where it is unambiguously safe.
- export_status read the epoch in one statement and used it in the next; now one query.
- DiskCache::snapshot IGNORED its path argument on a cache hit, returning whatever filesystem
was measured last. Latent only because both callers pass media_path — it would have silently
misreported the moment anyone measured the exports volume. Now keyed by path.
- Corrected two comments that asserted safety properties the code does not have (claim_job
does NOT prevent a retired-epoch claim — retirement is enforced at READ time; and a
multi-replica reap is NOT contained, it can corrupt the keepsake). Added a rollback runbook
to migration 014, the repo's first destructive migration.
TESTS
The regression guard for the headline FOR SHARE fix was probabilistic — it could pass on
broken code if the interleaving fell the wrong way. It is now STRUCTURAL: the upload is held
open mid-body with its pre-flight check already passed, so the release provably lands inside
the exact window. Verified 5/5 FAILING (201 instead of 403) with the fix reverted, 5/5
passing with it.
Verified: backend 40 tests, svelte-check 0 errors, e2e typecheck clean,
e2e 160 passed / 1 skipped on chromium-desktop.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
fabi
|
8e906d7866 |
refactor(export): single epoch authority; fix upload-path TOCTOU losing photos
A deep investigative review of the export concurrency (four parallel adversarial
reviewers) found that three rounds of race fixes had been chasing the wrong bug, and
that the model itself was the root cause. This replaces the model and fixes the real
data loss.
THE ACTUAL BUG (upload path, not the state machine)
`upload` checked `uploads_locked_at` BEFORE streaming the body — minutes, for a large
video — then committed the row without re-checking. A release landing mid-upload let the
row commit AFTER the export snapshot: the photo appeared in the live feed and was
PERMANENTLY missing from the keepsake, with nothing to regenerate it. release_gallery's
comment claims to prevent exactly this; it never did. Every prior round fixed the DB
state machine, and the leak was upstream of it.
Fix: re-assert the lock under `SELECT ... FOR SHARE` INSIDE the commit tx. That row lock
conflicts with release_gallery's `UPDATE event`, so either the upload commits first (and
the release — hence the snapshot — is ordered after it) or the release wins and the
upload is rejected as `uploads_locked` (reversible; the client keeps the blob).
Regression test verified NON-VACUOUS: with the fix reverted it fails, reporting accepted
uploads missing from the keepsake.
THE MODEL (migration 014)
"Which generation is current?" had no single answer: it was assembled from three
separately-written pieces across two tables (`export_released_at`, a per-job
`release_seq`, and cached `export_{zip,html}_ready` flags). Keeping them in agreement
took ~10 hand-written guards; each review round found one more path that slipped through.
Now: ONE monotonic `event.export_epoch`, bumped in the same UPDATE as any change to
`export_released_at`. Each job carries a copy. Downloadable IFF
`released AND job.epoch = event.export_epoch AND status = 'done'` — readiness DERIVED,
never stored. A retired-epoch worker is INERT BY CONSTRUCTION: losing a race can no
longer corrupt state, only waste work.
Deleted: both ready-flag columns, both ready-flip blocks, `if ready { continue }`, the
reopen seq-bump, open_event's transaction, and the cross-table claim guard.
That last one was also UNSOUND: under READ COMMITTED, a blocked UPDATE re-evaluates its
WHERE against the updated target row but answers subqueries on OTHER tables from the
original snapshot — so the previous `EXISTS (SELECT ... FROM event ...)` claim guard
could admit a claim against an already-reopened event while RETURNING the post-bump seq.
Every guard is now same-row (`epoch = $n`), which EPQ re-evaluates correctly.
OTHER REAL DEFECTS FIXED
- release_gallery committed the release and THEN awaited the enqueue inside the handler
future. Axum drops that future on client disconnect → event released, uploads locked,
ZERO export_job rows, host unable to retry ("bereits freigegeben"), restart-only
recovery. Now one tx; workers spawn (detached) after commit.
- No flush/fsync before the tmp→final rename. async_zip's close() never flushes the inner
file and tokio's File drop DISCARDS write errors — so a full disk silently produced a
truncated archive that was then marked done and published, after which the last good
generation was pruned. Now flush + sync_all, which also surfaces ENOSPC.
- Download read the ready flag and the file path in TWO queries; a reopen between them
served a keepsake that should 404. Now one read through the `export_current` view.
- `if !src.exists() { continue }` then `open()?` — a TOCTOU that turned a tolerated gap
into a hard failure of the ENTIRE keepsake (unretryable while released). Now open-first,
warn, and skip.
- Recovery was flag-driven and never checked the disk: a `done` row whose archive was gone
was a permanent dead end needing manual DB surgery. Now verifies the file and re-arms.
- Temp files/dirs leaked on every error path (the leak is what fills the disk that
corrupts the next archive). Now cleaned up.
- Export archives were not event-scoped (`viewer_tmp_` already was), so a second event
would collide on paths and one event's prune would delete another's live keepsake.
- Host deletes after release never regenerated the keepsake — a photo removed on request
lived on in the archive forever. Now bumps the epoch and rebuilds without it.
- claim_job swallowed DB errors as "someone else won", stranding a job pending at 0%.
- export_status reported retired-epoch rows (a progress bar that never moves).
- The startup sweep's single-instance assumption is now documented, not hidden.
Verified: backend 40 tests, svelte-check 0 errors, e2e typecheck clean,
e2e 160 passed / 1 skipped on chromium-desktop (incl. 2 new regressions; the
upload-acceptance one confirmed to fail without the fix).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
fabi
|
36fe59caa5 |
fix(user-flow): persona-audit fixes — ban replay, locked-upload data loss, host UX, admin session
Follows the perf + security + user-flow work with a role/persona audit (guest, host, admin, projector) and fixes across three review rounds. Highlights: HIGH - Ban now replays on reconnect. A ban isn't a soft-delete, and the `user-hidden` SSE has no replay, so a client that missed it (esp. the unattended diashow) kept cycling a banned user's slides. New `uploads_hidden_at` (migration 013) + `hidden_user_ids` in /feed/delta; feed + diashow evict those users. Applied even on a truncated delta. MEDIUM - Locked-upload data loss: a photo staged offline during a lock/release was purged as a terminal 4xx and lost when the host reopened. New reversible `uploads_locked` error code; the queue keeps the blob and auto-resumes on the `event-opened` SSE. - Reopen after release now warns (ConfirmSheet) that it revokes the published keepsake. - Host "forgotten-PIN" badge updates live (`pin-reset-requested` was broadcast but never in KNOWN_EVENTS / subscribed); host page also refetches on `pin-reset` so a two-host race can't hand out a conflicting PIN. - Ban modal copy fixed (read-only ban, not "session ended"); Degradieren/Sperren/Entsperren hidden on peer-host rows for non-admins (they always 403'd). - Host dashboard shows live keepsake generation progress / ready state + link to /export. - Admin JWT moved to sessionStorage (§11.1) to bound exposure on shared devices. Export generation guard (H1 from the prior round, hardened): per-(event,type) `release_seq` (migration 012) with seq-guarded claim/finalize/mark_failed/update_progress, per-generation temp/final paths, download follows `file_path`, prune only strictly-older generations. LOW: diashow coalesces upload-processed (avoids self-rate-limit); event-closed reconciles galleryReleased; /recover gains a forgot-PIN request + drops a stale cached PIN on 401; delta `>=` tie-break + 429 retry; misc copy/labels. Adds e2e: ban-replay, upload-lock-code, and rewrites export-reopen-rerelease with a data-completeness test. Reconciles USER_JOURNEYS §9/§11. Verified: cargo build clean, 40 unit tests, svelte-check 0 errors, 33 frontend unit tests, 155 e2e passing on chromium-desktop. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
641174717c |
fix(user-flow): close offline-queue, export, session, ban & realtime gaps
Comprehensive user-flow review across guest/host/admin roles, then fixes with
e2e regression guards. Highlights:
- Offline upload queue auto-resumes on reconnect: network errors keep items
pending (not error), 4xx are terminal (no infinite retry), quota exhaustion
returns a distinct 413; queue cap + dedup.
- Export lifecycle: release atomically locks uploads; reopen invalidates and
re-release regenerates the keepsake; workers claim their job atomically so a
reopen->re-release can't corrupt the ZIP; startup re-spawns interrupted
exports.
- Sessions slide on activity (no 30-day cliff); JWT expiry deferred to the
revocable session row; sign-out-everywhere + revoke-on-PIN-reset.
- Ban always hides content (v_feed / find_visible_media / export filter
is_banned) but stays a read-only ban per USER_JOURNEYS §10 — sessions are
not revoked, read access + keepsake download preserved.
- Realtime: server-clock SSE delta cursor; event-closed/opened drive the UI
live; feed_delta rate-limited; like returns {liked, like_count} to fix
multi-device drift; lightbox live comments; diashow delta backfill.
- Forgotten-PIN in-app request flow; simultaneous same-name join returns 409;
quota increment is transactional; operator floor.
Adds e2e/specs/10-flow-review/ (offline resume, export integrity, deterministic
anti-race guard) and updates existing specs for the new contracts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
fabi
|
a4b2c5bd1c |
fix(security): harden authz, media gating, recover, SSE, deps
Security audit follow-up. No Critical/High existed; these close the
Medium/Low findings.
F1 [Med] Host privilege boundary: a host could demote a peer host to
guest and then ban / PIN-reset (→ account takeover via /recover) them,
because the ban/pin-reset peer guards key off the target's *current*
role. set_role now blocks a non-admin from demoting a host.
F2 [Med] Moderation now revokes preview/thumbnail access: they are served
through visibility-checked aliases (/api/v1/upload/{id}/{preview,
thumbnail}) that filter soft-deleted / ban-hidden uploads, and direct
/media/previews|thumbnails is 404-blocked. Feed emits the gated URLs;
Caddy keeps them privately cacheable (max-age=300, short so moderation
revokes promptly — the live feed already evicts cards via SSE). Uses a
lean find_visible_media() (paths+mime only) on the media hot path.
F3 [Med/Low] npm audit fix: svelte 5.55.1→5.56.4 (SSR-XSS advisories),
devalue 5.6.4→5.8.1 (DoS). Prod deps: 0 vulnerabilities.
F4 [Low] /recover no longer leaks account existence: unknown display name
returns the same 401 as a wrong PIN, and runs a dummy bcrypt verify so
timing matches — closing the enumeration + timing oracle.
F5 [Low] SSE streams re-validate the session every ~60s and close once it
is logged out / expired, instead of running until client disconnect.
F6 [Info] X-Content-Type-Options: nosniff set at the app layer on all
media responses (defense-in-depth alongside the edge).
Tests: new e2e specs for F1 (host cannot demote peer host), F2 (preview
gated + 404 after delete + direct /media blocked), F4 (uniform 401).
40 backend unit tests + 182 e2e passing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
fabi
|
d6c91974eb |
perf(backend): close config/upload/export hot paths + stability fixes
Performance: - Cache the runtime `config` table in-memory (ConfigCache) with synchronous invalidation on every write (admin PATCH + test reseed). Was re-reading each key from Postgres on every request (~8 round-trips per upload). - Stream uploads chunk-by-chunk to a temp file instead of buffering the whole body in RAM (peak was up to the per-class cap, e.g. 500 MB/video); only 512 sniff-bytes are kept for magic-byte detection, then atomic rename into place. - Cache the media-filesystem disk snapshot (DiskCache, 15s TTL) shared by the quota check and admin stats; drop the discarded System::refresh_all(). - HTML export streams video (and small-image) originals straight into the ZIP via a manifest instead of copying them to a temp dir first (removed the transient 2x disk usage) and drops the double directory scan. - Auth extractor resolves session -> live user in one JOIN (was two queries), touching last_seen_at by token hash. Stability: - SSE: on broadcast lag, emit a `resync` event so the client runs a delta fetch instead of silently losing events; frontend reconciles adds, deletions, and (via an in-place refresh) like/comment counts on visible cards. - Storage quota fails OPEN when the disk can't be read (was a 0-byte limit that locked out all uploads). - Graceful shutdown drains in-flight requests on SIGTERM/SIGINT, bounded by a 10s backstop so open SSE streams can't stall a deploy. - Upload removes the persisted file if the DB transaction fails (no orphaned bytes with no row to reclaim them). Tests: - New pure select_disk() with 5 unit tests (longest-prefix, fallbacks, fail-open). - New e2e export-video spec covering the HTML export's video-streaming branch. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
fabi
|
faf7a2504a |
fix(audit): restore broken upload pipeline + role-based E2E audit fixes
A comprehensive role-based E2E audit (guest/host/admin, across browser sessions) surfaced one critical and several smaller issues; this addresses them and hardens the tests that missed them. Critical - The client upload pipeline was fully broken: the IndexedDB v1->v2 upgrade opened a *new* transaction inside the upgrade callback, which throws during a version-change transaction and aborted the whole upgrade, leaving the queue object store uncreated -- so no UI upload ever fired. Reuse the version-change transaction the callback provides, and bump the DB to v3 with a contains() guard so installs already corrupted by the shipped bug self-heal on next load. Re-enabled the previously-fixme'd UI upload E2E test. High / Medium - Event lock is uploads-only again: likes, comments and browsing stay open while the event is locked (USER_JOURNEYS 9.3 / FEATURES) -- it was wrongly freezing social interaction. Updated the event-lock spec accordingly. - get_original now excludes soft-deleted and ban-hidden uploads, and direct /media/originals/** serving is blocked, so a hidden user's originals can no longer be pulled by UUID (all originals go through the checked alias). - The upload handler reads the file field with an early-abort size cap chosen from the declared content-type, instead of buffering the entire body before the size check. Low - unban_user mirrors the ban role guard (a host can no longer unban a host/admin banned by an admin). - reset_user_pin's UPDATE is event-scoped. - Admin login returns and stores a real identity (user_id + display name) instead of a blank session. - The host user list no longer renders target-actions (ban/promote/demote/PIN) on the caller's own row, where the backend always rejected them. - /diashow gains a client-side auth guard like the other protected routes. - The join page shows the event name via a new public GET /api/v1/event. Verified: backend cargo build clean, frontend svelte-check 0 errors, full Playwright E2E suite 144 passed / 1 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
fabi
|
0ed97f45cf |
fix(review-2): address re-review — close two blocker regressions + harden tests
The round-2 re-review (full suite green) found two real defects the passing
tests missed, both now fixed and covered:
B1 — banned users could still edit/delete their OWN content: edit_upload,
delete_upload, delete_comment skipped the is_banned guard that upload/
like/comment got (round-2 removed the global extractor block and missed
these three). Added the guard (using auth.is_banned — no extra query).
The moderation ban test now asserts 403 on DELETE upload, PATCH upload,
and DELETE comment for a banned owner (+ upload survives) — it previously
only exercised POST /upload, which is why the gap shipped green.
B2 — H3 live-eviction was dead code: 'user-hidden' and 'comment-deleted' were
missing from KNOWN_EVENTS, so EventSource never listened and the handlers
never fired. Added both. New browser-driven sse-eviction test asserts a
hidden user's card actually evicts from an open feed with no reload — the
backend-only SseListener checks couldn't catch this.
Minor items folded in:
- admin dashboard bounces a mid-session-demoted admin to /feed via live
/me/context (mirrors the host page) instead of a dead error screen.
- feed "new posts" pill cleared on any full refresh (pull-to-refresh no longer
strands it).
- corrected the stale sse.rs comment (banned users may hold a read-only stream).
Verified: backend 35 unit tests; svelte-check 0 errors; e2e 04-host + comment-ui
+ export-leak 13/13 on chromium (incl. the two new regression tests).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
fabi
|
80179357a7 |
fix(review-2): high — read-only ban model, failed-compression cleanup, live SSE eviction
H1: corrected a round-1 over-reach. Bans are read-only per USER_JOURNEYS §10:
the base AuthUser extractor no longer rejects banned users (they keep read
access); Require{Host,Admin} and write handlers enforce the ban via
auth.is_banned → 403 (incl. self-unban). Demoted hosts still lose powers
immediately (live DB role) and are bounced off the dashboard. Also fixed the
login/recover redirect regression on unauthenticated 401.
H2: failed compression now self-heals instead of leaving a permanent broken
card — refund the quota, delete the orphaned original, soft-delete the row;
the frontend toasts the uploader and evicts the card on upload-error.
H3: self-delete, ban-with-hide (user-hidden), and comment-deletes now broadcast
SSE; feed, diashow, and lightbox evict the affected content live instead of
waiting for a reload. sse-eviction.spec.ts covers it.
Riders: feed/+page.svelte also carries the medium truncated-delta pill; host.rs
also carries the low atomic release_gallery claim; admin/+page.svelte also drops
the dead compression_concurrency control (medium).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
fabi
|
f08d858281 |
fix(security): high — live authz, XFF, SSE buffering, atomicity, pagination
H1: auth extractor re-reads the live user row — trusts the DB role (not the
JWT claim) and rejects banned users mid-session. e2e proves a demoted
host loses powers and a banned host is locked out and can't self-unban.
H2: client_ip takes the right-most XFF hop (the one Caddy appends); spoofed
left-most entries are ignored, restoring IP-based throttles.
H3: Caddy excludes /api/v1/stream from `encode` so SSE isn't buffered.
H4: upload — quota increment + row insert + hashtag links now one txn.
H5: feed keyset pagination tiebroken on (created_at, id) + composite index
(migration 010); feed_delta bounded with LIMIT.
H7: LightboxModal keys its comment load off upload.id, ending the
SSE-driven refetch storm.
H8: CSP via SvelteKit kit.csp (svelte.config.js) hardens the localStorage
token model against injection.
Migration 010 also adds the latent comment/comment_hashtag indexes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
fabi
|
226e455328 |
test(unit): broaden coverage — quota formula + auth token/JWT decode
Backend: extract the per-user quota formula from compute_storage_quota into a pure `quota_limit_bytes(free_disk, tolerance, active)` (behavior-preserving) and unit-test it: tolerance scaling, floor of fractional results, active<1 clamped to 1 (divide-by-zero guard), zero free disk, single-uploader identity. Frontend (jsdom + browser:true mock): auth.ts token storage and JWT claim decode — setAuth/getToken/getPin round-trip, clearAuth keeps the PIN (for recovery), clearPin, getExpiry (exp seconds→ms Date; null on missing/malformed), getRole (role claim; null on absent/malformed). Adds jsdom devDependency. Backend: 25 passing. Frontend: 18 passing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
fabi
|
1bb58b59ad |
fix(security): re-review follow-ups — drop HEIC/HEIF, fix stale comment
Address the two items from the adversarial re-review of batch-2. - upload: remove image/heic + image/heif from ALLOWED_MEDIA. Neither the `image` crate nor the bundled ffmpeg 6.1 (Alpine 3.21 — HEIF demuxer only landed in ffmpeg 7.0) can decode them, so accepting them stored posts that never got a thumbnail. iOS Safari transcodes HEIC->JPEG on file-input selection, so this rejects only the rare HEIC-preserving path, now with a clear error instead of a silently broken post. - data-mode-store: correct the stale "auth-gated" comment — the /original route is intentionally unauthenticated (UUID-as-capability) so it works from plain <img src> / <video src>. Re-verified: cargo build (only the pre-existing middleware.rs warning). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
fabi
|
edcef0258c |
fix(security): batch-2 hardening — XSS allowlist, event-scoping, deploy
Address the seven findings from the security & deployment review. High: - upload: make infer authoritative — reject files it can't identify (SVG/HTML/JS) and require the detected MIME to be on an ALLOWED_MEDIA allowlist; derive the stored MIME and on-disk extension from the detected type, ignoring client filename/Content-Type. Closes the stored-XSS vector via media served on-origin. - deploy: rename docker-compose.override.yml -> docker-compose.dev.yml so the default `docker compose up -d` no longer publishes Postgres 5432 to the host; the port map is now opt-in via -f. README updated. Medium: - upload: DefaultBodyLimit::disable() -> max(576 MiB) as an HTTP-level OOM backstop; handler still enforces precise per-class size limits. - docker: run backend and frontend as non-root users. Low: - social/upload: event-scope toggle_like, list_comments, add_comment, delete_comment, edit_upload, delete_upload via find_by_id_and_event / soft_delete_in_event — cross-event IDs now resolve to 404. - Caddy: site-wide HSTS / nosniff / X-Frame-Options / Referrer-Policy, plus Content-Disposition: attachment on /media/originals/*. - .env.example: replace default Postgres password with a CHANGE_ME hint. Out of scope: localStorage JWT (root cause fixed; httpOnly cookies are a larger change tracked separately). Verified: cargo build (no new warnings), cargo test (3 passed), caddy validate, docker compose config (no 5432 published by default). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
d228676a56 |
fix(security): cross-event authz, SSE ticket flow, account hardening, audit logs
Follow-up to the comprehensive code review. Five batches: 1. Cross-event authorization: host_delete_upload, unban_user, and host_delete_comment now scope by auth.event_id. Adds Upload::find_by_id_and_event / soft_delete_in_event and a Comment::soft_delete_in_event variant that joins through upload. 2. Token exposure: SSE auth no longer puts the JWT in the URL. New /api/v1/stream/ticket endpoint mints a short-lived single-use ticket bound to the session; the EventSource passes ?ticket=... instead. Refuse to start in APP_ENV=production with the dev JWT sentinel; warn loudly otherwise. 3. Account hardening: per-IP+name rate limit on /recover (mitigates targeted lockout DoS), per-IP rate limit on /admin/login, random 32-char admin recovery PIN (replaces "0000"), structured tracing events for wrong PIN, lockout, failed admin login, ban/unban/role change/pin-reset/host-delete. 4. DoS / correctness: comment listing paginated (LIMIT 50 + ?before= cursor), hashtag extraction whitelisted to ASCII alnum+underscore (≤40 chars) with unit tests, display_name / caption / comment body length validated in chars rather than bytes. 5. Cleanup: session-touch failures now logged, DATABASE_MAX_CONNECTIONS env var (default 10). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
2e98f5ddf5 |
backend(features): quota enforcement, PIN reset, /me, original download, toggles
- handlers/me.rs (new): GET /api/v1/me/context (profile + role + privacy_note
+ quota toggle state, fetched once on app bootstrap) and GET /api/v1/me/quota
(live used / limit / active uploaders / free disk).
- handlers/upload.rs:
- quota enforcement via the dynamic formula
floor((free_disk * tolerance) / max(active_uploaders, 1)),
gated by quota_enabled + storage_quota_enabled toggles
- new GET /api/v1/upload/{id}/original — unauthed by design
(matches /media/previews/* — URL is the secret) so it works as
<img src> / <video src> / window.open
- rate-limit toggle wiring (rate_limits_enabled + upload_rate_enabled)
- handlers/host.rs:
- POST /api/v1/host/users/{id}/pin-reset — Host may reset guest PINs,
Admin may reset guest + host PINs (never another admin or self).
Returns the freshly-generated plaintext PIN once; emits a global
pin-reset SSE so the affected user's device can clear its localStorage.
- set_role guard expanded so hosts can demote other hosts (not self,
never admins) — backend match for the doc'd permission model.
- handlers/admin.rs: ALLOWED_KEYS split into NUMERIC_KEYS / BOOL_KEYS /
TEXT_KEYS with per-kind validation; saving privacy_note broadcasts an
event-updated SSE so other clients refresh live.
- handlers/feed.rs, handlers/admin.rs (export), auth/handlers.rs:
rate-limit toggle wiring at every limiter call site.
- auth/handlers.rs: when an expired PIN lockout is detected on /recover,
reset failed_pin_attempts to zero before the bcrypt check — without
this every wrong PIN re-locked the user after the cooldown.
- main.rs: wire startup_recovery + spawn_periodic_tasks, register the
new /me/context, /me/quota, /upload/{id}/original, and
/host/users/{id}/pin-reset routes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
de0e395a9e |
feat: auto-retry uploads when rate limited (v0.13.0)
Backend: rate limiter gains check_with_retry() returning seconds until the next slot opens. Upload 429 responses include retry_after_secs in JSON and a Retry-After header. Frontend: upload queue catches 429 as RateLimitError, resets affected item to pending, schedules processQueue() for the server-reported delay, and shows a live countdown banner in the queue UI. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
MechaCat02
|
3dc69e6c6d |
fix: upload body limit, role case, and connection drain (v0.12.1)
- Disable Axum's 2 MB default body limit on the upload route so large
photos/videos are accepted without HTTP 400
- Serialize UserRole as lowercase in JWT so the frontend role checks
('guest'/'host'/'admin') match correctly
- Drain multipart body before returning early upload errors (rate-limit,
ban, event-lock) to keep the HTTP keep-alive connection clean and
prevent cascading Netzwerkfehler / empty-500 responses
- Add TraceLayer for request logging and Vite dev proxy config
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
||
|
MechaCat02
|
989d88022a |
feat: implement rate limiting across all API endpoints
Add sliding-window in-memory RateLimiter service (Arc<Mutex<HashMap>>) with per-IP and per-user-id limits on all public endpoint classes: - POST /api/v1/join: 5/min per IP - GET /api/v1/feed: configurable per IP (feed_rate_per_min, default 60) - POST /api/v1/upload: configurable per user (upload_rate_per_hour, default 10) - GET /api/v1/export/zip|html: configurable per IP (export_rate_per_day, default 3) Limits are hot-reloadable via the config table. All 429 responses use German error messages. Client IP is read from X-Forwarded-For (Caddy). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
fabi
|
3f052a4f91 |
feat: implement upload pipeline with compression and SSE
Backend:
- POST /api/v1/upload: multipart file upload with caption + hashtags
- Validates file size against DB config limits (image/video separate)
- Checks user ban status and event upload lock
- Saves original to disk under {media_path}/originals/{slug}/
- Tracks user total_upload_bytes for quota enforcement
- Extracts hashtags from caption text and explicit CSV field
- Upserts hashtags and links them to uploads
- PATCH /api/v1/upload/{id}: edit caption and hashtags (owner only)
- DELETE /api/v1/upload/{id}: soft-delete (owner only)
- GET /api/v1/stream: SSE endpoint with 30s keepalive
- Broadcasts new-upload events to all connected clients
- Uses tokio broadcast channel for fan-out
Services:
- CompressionWorker: Tokio semaphore-bounded (concurrency=2) background processor
- Images: resize to 800px wide JPEG preview via image crate
- PNG originals: lossless compression via oxipng
- Videos: ffmpeg thumbnail extraction (1 frame at 1s, scaled to 800px)
- Updates upload record with preview_path/thumbnail_path on completion
Models:
- Upload with full CRUD (create, find, update caption, soft delete, set paths)
- Hashtag with upsert, link/unlink, extract_hashtags() text parser
- UploadDto for API serialization with like/comment counts
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|