fabi/EventSnap
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
109 Commits 2 Branches 12 Tags
3c683247c06ad4014169e9db8650682c3caf3f84
Commit Graph

7 Commits

Author SHA1 Message Date
fabi
8e906d7866 refactor(export): single epoch authority; fix upload-path TOCTOU losing photos 
A deep investigative review of the export concurrency (four parallel adversarial
reviewers) found that three rounds of race fixes had been chasing the wrong bug, and
that the model itself was the root cause. This replaces the model and fixes the real
data loss.

THE ACTUAL BUG (upload path, not the state machine)
  `upload` checked `uploads_locked_at` BEFORE streaming the body — minutes, for a large
  video — then committed the row without re-checking. A release landing mid-upload let the
  row commit AFTER the export snapshot: the photo appeared in the live feed and was
  PERMANENTLY missing from the keepsake, with nothing to regenerate it. release_gallery's
  comment claims to prevent exactly this; it never did. Every prior round fixed the DB
  state machine, and the leak was upstream of it.
  Fix: re-assert the lock under `SELECT ... FOR SHARE` INSIDE the commit tx. That row lock
  conflicts with release_gallery's `UPDATE event`, so either the upload commits first (and
  the release — hence the snapshot — is ordered after it) or the release wins and the
  upload is rejected as `uploads_locked` (reversible; the client keeps the blob).
  Regression test verified NON-VACUOUS: with the fix reverted it fails, reporting accepted
  uploads missing from the keepsake.

THE MODEL (migration 014)
  "Which generation is current?" had no single answer: it was assembled from three
  separately-written pieces across two tables (`export_released_at`, a per-job
  `release_seq`, and cached `export_{zip,html}_ready` flags). Keeping them in agreement
  took ~10 hand-written guards; each review round found one more path that slipped through.
  Now: ONE monotonic `event.export_epoch`, bumped in the same UPDATE as any change to
  `export_released_at`. Each job carries a copy. Downloadable IFF
  `released AND job.epoch = event.export_epoch AND status = 'done'` — readiness DERIVED,
  never stored. A retired-epoch worker is INERT BY CONSTRUCTION: losing a race can no
  longer corrupt state, only waste work.
  Deleted: both ready-flag columns, both ready-flip blocks, `if ready { continue }`, the
  reopen seq-bump, open_event's transaction, and the cross-table claim guard.

  That last one was also UNSOUND: under READ COMMITTED, a blocked UPDATE re-evaluates its
  WHERE against the updated target row but answers subqueries on OTHER tables from the
  original snapshot — so the previous `EXISTS (SELECT ... FROM event ...)` claim guard
  could admit a claim against an already-reopened event while RETURNING the post-bump seq.
  Every guard is now same-row (`epoch = $n`), which EPQ re-evaluates correctly.

OTHER REAL DEFECTS FIXED
  - release_gallery committed the release and THEN awaited the enqueue inside the handler
    future. Axum drops that future on client disconnect → event released, uploads locked,
    ZERO export_job rows, host unable to retry ("bereits freigegeben"), restart-only
    recovery. Now one tx; workers spawn (detached) after commit.
  - No flush/fsync before the tmp→final rename. async_zip's close() never flushes the inner
    file and tokio's File drop DISCARDS write errors — so a full disk silently produced a
    truncated archive that was then marked done and published, after which the last good
    generation was pruned. Now flush + sync_all, which also surfaces ENOSPC.
  - Download read the ready flag and the file path in TWO queries; a reopen between them
    served a keepsake that should 404. Now one read through the `export_current` view.
  - `if !src.exists() { continue }` then `open()?` — a TOCTOU that turned a tolerated gap
    into a hard failure of the ENTIRE keepsake (unretryable while released). Now open-first,
    warn, and skip.
  - Recovery was flag-driven and never checked the disk: a `done` row whose archive was gone
    was a permanent dead end needing manual DB surgery. Now verifies the file and re-arms.
  - Temp files/dirs leaked on every error path (the leak is what fills the disk that
    corrupts the next archive). Now cleaned up.
  - Export archives were not event-scoped (`viewer_tmp_` already was), so a second event
    would collide on paths and one event's prune would delete another's live keepsake.
  - Host deletes after release never regenerated the keepsake — a photo removed on request
    lived on in the archive forever. Now bumps the epoch and rebuilds without it.
  - claim_job swallowed DB errors as "someone else won", stranding a job pending at 0%.
  - export_status reported retired-epoch rows (a progress bar that never moves).
  - The startup sweep's single-instance assumption is now documented, not hidden.

Verified: backend 40 tests, svelte-check 0 errors, e2e typecheck clean,
e2e 160 passed / 1 skipped on chromium-desktop (incl. 2 new regressions; the
upload-acceptance one confirmed to fail without the fix).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-14 19:48:38 +02:00
fabi
d6c91974eb perf(backend): close config/upload/export hot paths + stability fixes 
Performance:
- Cache the runtime `config` table in-memory (ConfigCache) with synchronous
  invalidation on every write (admin PATCH + test reseed). Was re-reading each
  key from Postgres on every request (~8 round-trips per upload).
- Stream uploads chunk-by-chunk to a temp file instead of buffering the whole
  body in RAM (peak was up to the per-class cap, e.g. 500 MB/video); only 512
  sniff-bytes are kept for magic-byte detection, then atomic rename into place.
- Cache the media-filesystem disk snapshot (DiskCache, 15s TTL) shared by the
  quota check and admin stats; drop the discarded System::refresh_all().
- HTML export streams video (and small-image) originals straight into the ZIP
  via a manifest instead of copying them to a temp dir first (removed the
  transient 2x disk usage) and drops the double directory scan.
- Auth extractor resolves session -> live user in one JOIN (was two queries),
  touching last_seen_at by token hash.

Stability:
- SSE: on broadcast lag, emit a `resync` event so the client runs a delta
  fetch instead of silently losing events; frontend reconciles adds, deletions,
  and (via an in-place refresh) like/comment counts on visible cards.
- Storage quota fails OPEN when the disk can't be read (was a 0-byte limit that
  locked out all uploads).
- Graceful shutdown drains in-flight requests on SIGTERM/SIGINT, bounded by a
  10s backstop so open SSE streams can't stall a deploy.
- Upload removes the persisted file if the DB transaction fails (no orphaned
  bytes with no row to reclaim them).

Tests:
- New pure select_disk() with 5 unit tests (longest-prefix, fallbacks, fail-open).
- New e2e export-video spec covering the HTML export's video-streaming branch.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-07 20:26:45 +02:00
fabi
3f6dafba05 test(review-2): strengthen CR2 export-leak test to drive a real export 
The prior test asserted 404 on /media/exports/Gallery.zip against an empty
stack — it would pass even if exports were still written under /media, because
no archive was ever produced. Now it:
  1. seeds an upload, releases the gallery, and polls until the real zip job
     writes Gallery.zip to disk;
  2. asserts the archive is NOT served from public /media (the CR2 leak); and
  3. asserts it IS retrievable via the gated ticket endpoint (200 + PK zip
     magic) — proving the 404 means "not public", not "no file".

Also fixes a test-isolation gap the CR2 relocation introduced: __truncate wiped
media_path but not export_path, so a real export would leave Gallery.zip on disk
and break export.spec's "ready-but-file-missing → 404" test. truncate_all now
purges export_path too. Full 06-export dir: 5/5 green, no contamination.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-03 07:19:09 +02:00
fabi
dba4d3f932 fix(review-2): critical — repair comment posting + close export data leak 
CR1: the lightbox comment button POSTed to /upload/{id}/comment (singular);
     no such route exists, so every UI-submitted comment 404'd and was lost.
     Fixed to /comments (plural). The prior e2e passed because its seed helper
     POSTs the API directly — added comment-ui.spec.ts which drives the real
     component so this can't regress silently again.

CR2: export archives (Gallery.zip / Memories.zip / HTML viewer) were written
     under media_path, which is a public ServeDir — so GET /media/exports/
     Gallery.zip served the entire event (every photo, caption, comment,
     uploader name) to any anonymous visitor at a guessable URL, bypassing the
     ticket + release gate. Moved exports to a separate EXPORT_PATH (=/exports)
     on its own volume (Dockerfile chown, compose volume, gated handler reads
     the new path). export-leak.spec.ts asserts /media/exports/Gallery.zip → 404.

Riders (git can't split hunks): LightboxModal also gains H3 live-eviction on
comment-deleted/upload-deleted; config.rs also wires compression_concurrency
from boot config (medium).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-02 22:19:32 +02:00
fabi
20125fb713 test(e2e): pin security assertions + real cross-user IDOR coverage 
Turn "accept-both-outcomes" documentation tests — which pass whether the app is
secure or vulnerable — into assertions that pin the secure behavior, and replace
fake-UUID authz tests that 404'd before ever reaching the ownership guard with
real cross-user resources.

file-upload-attacks:
- SVG-with-<script> → pin 400 (infer returns None for text → stored-XSS defense);
  was [201,400], which accepted the vulnerable outcome.
- zero-byte → pin 400; path-traversal → pin 201 with UUID-derived storage and a
  no-path-echo check (both were [201,400]).
- Fix the application/octet-stream rationale (no "application bypass" exists — the
  handler ignores the declared type and keys off magic bytes).

authorization-deep (IDOR):
- B deleting A's comment: seed a real upload+comment as A, assert 403, assert the
  comment survives, and assert the owner (A) still gets 204 — proving the 403 is
  about identity, not a broken route. (Was a DELETE on the all-zeros UUID → 404
  before the user_id guard, so authorization was never exercised.)
- Add B-deletes-A's-upload (403, countUploads unchanged) and B-edits-A's-caption
  (403, caption intact) — the find_by_id_and_event + ownership guards.

export: pin the ZIP-download 404 (was [404,200] — a 200 is a data-exposure
regression) and split into the two real branches: not-yet-ready, and
ready-but-file-missing (new db.setExportZipReady helper).

All 21 assertions verified green against the live secure backend.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-01 07:27:22 +02:00
fabi
e79e020566 feat(eventsnap): UX review batch-3 — feed virtualization, a11y/UX fixes, shared primitives 
Frontend
- Feed: DOM-windowing via @tanstack/svelte-virtual (new VirtualFeed). The window
  virtualizer keeps the sticky header, pull-to-refresh, infinite-scroll sentinel and
  bottom nav working. List = dynamic measured heights keyed by upload id +
  anchorTo:start so an SSE prepend doesn't jump a scrolled reader; grid = measured
  square rows. Drops the content-visibility band-aid and the old FeedGrid.
  measureElement(null) on row unmount prevents ResizeObserver retention; stable
  option callbacks + guarded setOptions avoid O(n) re-measure on like/comment patches.
- Global: viewport-fit=cover (activates safe-area insets), lang=de, PWA manifest +
  maskable icon + apple-touch metas, prefers-reduced-motion, safe-area-top headers.
- Feed reactivity: like/comment SSE patch the single card in place; upload-processed
  debounced in-place merge; IntersectionObserver leak fixed; shared 60s clock.
- CLS: list cards reserve the skeleton aspect box.
- Destructive actions (promote/demote/unban, gallery release) routed through
  ConfirmSheet (host + admin).
- Export OOM: streamed download via single-use ticket instead of res.blob().
- Shared primitives: IconButton (44px hit area), scrollLock action; UploadSheet a11y.
- Polish: api timeout + non-JSON guard, focus-trap offsetParent fix, pull-to-refresh
  passive:false + guards, diashow keyboard a11y, Toaster assertive errors, dark-mode
  gaps, aria-pressed on like/chip state, CameraCapture mic-on-video + retry,
  ConfirmSheet spinner, upload beforeunload warning, emoji->SVG icons.

Backend
- social.rs: like/comment SSE broadcasts now carry the fresh count so feed clients
  patch one card in place instead of refetching page 1.
- admin.rs / main.rs: supporting changes for the above.

Verified: svelte-check 0 errors, frontend build clean, /feed SSR 200. Runtime feed
scroll-feel validation still owed (documented in FOLLOWUPS.md).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-30 19:16:48 +02:00
MechaCat02
e42d8a92a1 feat(e2e): Playwright suite — 134 tests across 9 spec areas + UA matrix 
Adds an end-to-end Playwright test suite under e2e/ that spins up an
isolated docker-compose stack (Postgres :55432, Caddy :3101, backend with
EVENTSNAP_TEST_MODE=1, SvelteKit adapter-node frontend) and exercises the
SvelteKit app against the real Rust backend.

Phase 1 — happy paths covering every documented USER_JOURNEYS.md flow:
  01-auth/      join, recover, admin login, leave event, PIN lockout
  02-upload/    gallery picker (API path), rate-limit + admin toggle
  03-feed/      like/comment SSE, filters, SSE reconnect on visibility
  04-host/      event lock API, ban/unban, promote
  05-admin/     config validation, foundational authz guards, stats
  06-export/    /export status + download stub
  __smoke/      cross-UA happy-path (runs on every UA project)

Phase 2 — adversarial + browser chaos:
  07-adversarial/  XSS payloads (6 × display name path), SQLi shapes,
                   length / encoding / RTL override / NUL byte;
                   file-upload boundaries (ELF body claimed as JPEG,
                   oversize vs max_image_size_mb, zero-byte, NUL
                   filename, path-traversal, SVG-with-script);
                   JWT alg:none, signature/payload tamper, expired
                   session, PIN brute-force (serial + parallel),
                   admin password brute-force; deep authz (cross-user
                   delete, banned user across like/comment/feed-read,
                   host→admin escalation); small-scale DDoS (20× /join,
                   10MB comment body, 10 concurrent SSE).
  08-browser-chaos/ localStorage / sessionStorage / cookie purge,
                    IndexedDB drop mid-session, offline → reconnect,
                    slow-3G, 503 flakes, 429 with no retry storm,
                    multi-tab same/different user, no-JS, hostile CSS,
                    clock skew ±1h / -2d, localStorage quota exhausted.

Phase 3 — mobile gestures (runs only on chromium-mobile / Pixel 7):
  09-mobile/    touch-target ≥44px audit, env(safe-area-inset-bottom)
                structural check, long-press (FeedListCard → ContextSheet,
                quick-tap negation, click-suppression), double-tap
                (feed card like + lightbox heart-burst, via synthetic
                pointer events to bypass the first-tap-fires-click trap),
                viewport reflow (portrait/landscape/narrow/phablet),
                plus fixme stubs documenting planned gestures (swipe
                lightbox L/R, swipe-down dismiss, pull-to-refresh,
                long-press-comment).

Cross-UA matrix (chromium-engine projects run @smoke only):
  chromium-pixel7, chromium-galaxy-s22, samsung-internet (Samsung UA
  emulation on Galaxy viewport), edge-android, plus webkit-iphone,
  chrome-ios, firefox-android, firefox-desktop — the latter four need
  libavif16 on the host (Playwright dep) but the configs are in place.

Infrastructure:
  - fixtures/test.ts central test.extend (api, db, adminToken, guest,
    host, signIn). Per-test DB truncate via the dev-only POST
    /admin/__truncate route, gated by EVENTSNAP_TEST_MODE=1.
  - helpers/sse-listener.ts, helpers/upload-client.ts (Node-side
    multipart for adversarial file-upload tests + JPEG/PNG/ELF magic
    constants), helpers/touch.ts (longPress / doubleTap / swipe /
    inlineStyle / computedStyle).
  - 10 page objects covering every route + UploadSheet/Lightbox.
  - global-setup waits for /health, logs in admin, disables every
    rate-limit and quota toggle.
  - .github/workflows/e2e.yml: PR check runs chromium-desktop + the
    smoke matrix in parallel, uploads playwright-report/ and traces on
    failure.

Findings the suite surfaces as live `[finding]` warnings (not silenced):
  1. /admin/login has no rate-limit or lockout (bcrypt cost only).
  2. PIN-attempt counter races under parallel /recover requests.
  3. Zero-byte uploads pass /api/v1/upload.
  4. SVG-with-script can pass the magic-byte check (consider CSP +
     X-Content-Type-Options on /media/*).

Stack-internal docs live in e2e/README.md (UA tier table, Samsung
Internet escalation tiers A/B/C, debugging tips, roadmap).

Final tally: 134 passed / 0 failed / 9 skipped (test.fixme stubs for
not-yet-shipped gestures and one UI-upload-flow investigation).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-16 19:37:11 +02:00