fabi/EventSnap
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(export): close all 13 findings from the multi-agent review

Browse Source 
A 6-lens multi-agent review (103 agents; every finding adversarially verified by three
refute-by-default skeptics) confirmed the epoch redesign is sound — the core invariant
holds — and found 13 real defects around it. All are fixed here.

The redesign's invariant was re-verified and stands: an accepted upload is always in the
keepsake, and a downloadable keepsake always reflects the most recent release.

But a WEAKER adjacent invariant did NOT hold — "a downloadable keepsake reflects the
current content" — and that is the theme of the biggest fixes.

CONTENT REMOVAL NOW ALWAYS REACHES THE KEEPSAKE
  Regeneration was wired only to host deletes. Ban, unban, guest self-delete and guest
  comment-delete did not trigger it — yet the export query filters `is_banned = FALSE`,
  so the pipeline already agreed that content must not be there. Ban someone for abusive
  content after release and every guest kept downloading an archive containing it.
  All five removal paths now invalidate and rebuild. `query_comments` also gained the
  banned/hidden filter it was missing entirely (the ZIP had it; the viewer did not).

  Each removal is now ATOMIC with its invalidation (one tx, models made executor-generic).
  Previously the delete committed in its own tx and the regeneration in a second: a dropped
  handler future between them left the taken-down photo permanently downloadable, and
  nothing could notice — the keepsake still looked complete and the host could no longer
  even find the upload to retry.

NO MORE TAKEDOWN STORM
  Every removal spawned two uncancellable full exports. A superseded worker was INERT, not
  STOPPED: it still ran every ffmpeg spawn and image resize and wrote a whole archive before
  discovering it had lost. Five deletes left ten workers alive, each holding a
  full-gallery-sized temp, in a 1 GB container.
  Now: a debounce before `claim_job` (a superseded worker fails its claim and does ZERO
  work, collapsing a burst into one export), plus `update_progress` — which already ran
  exactly the right liveness predicate and threw the answer away — returning it so both file
  loops bail the instant they are retired. Moderating a comment no longer rebuilds the
  multi-GB ZIP either: the ZIP is media-only, so it is carried forward, with prune taught to
  protect any file a live row still references.

AN ESCAPE HATCH
  A failed export was terminal at runtime: release_gallery refuses an already-released event,
  recovery only runs at boot, and there was no retry route — the only outs were restarting the
  container or reopening uploads to every guest. Added POST /host/export/rebuild. Also widened
  mark_failed's guard to `status IN ('running','pending')`: when claim_job itself ERRORED, the
  row was still `pending`, so the failure was never recorded and the job sat at 0% forever.

SILENT INVALIDATION
  Retiring the epoch 404s the download instantly, but nothing told the clients — guests kept
  seeing an enabled download button through the whole rebuild. Now broadcasts an invalidation.
  And the download was a top-level <a href>: a 404 (or a 429 from the per-IP export limit that
  everyone on the venue WiFi shares) NAVIGATED THE USER OUT OF THE PWA onto raw JSON. It now
  targets a hidden iframe, so an error response is harmless.

ALSO
  - The HTML export still had the exists()-then-open TOCTOU the ZIP path was fixed to remove,
    at three sites, two with a hard `?` that failed the ENTIRE viewer. It is reachable: the
    compression worker hard-deletes an original when a transcode fails and can still be running
    when the gallery is released.
  - Crash-orphaned .tmp files were immortal (prune deliberately skips .tmp). Swept at boot,
    where it is unambiguously safe.
  - export_status read the epoch in one statement and used it in the next; now one query.
  - DiskCache::snapshot IGNORED its path argument on a cache hit, returning whatever filesystem
    was measured last. Latent only because both callers pass media_path — it would have silently
    misreported the moment anyone measured the exports volume. Now keyed by path.
  - Corrected two comments that asserted safety properties the code does not have (claim_job
    does NOT prevent a retired-epoch claim — retirement is enforced at READ time; and a
    multi-replica reap is NOT contained, it can corrupt the keepsake). Added a rollback runbook
    to migration 014, the repo's first destructive migration.

TESTS
  The regression guard for the headline FOR SHARE fix was probabilistic — it could pass on
  broken code if the interleaving fell the wrong way. It is now STRUCTURAL: the upload is held
  open mid-body with its pre-flight check already passed, so the release provably lands inside
  the exact window. Verified 5/5 FAILING (201 instead of 403) with the fix reverted, 5/5
  passing with it.

Verified: backend 40 tests, svelte-check 0 errors, e2e typecheck clean,
e2e 160 passed / 1 skipped on chromium-desktop.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
fabi
2026-07-14 20:35:47 +02:00
parent 5affef47cf
commit 9666d74a46
14 changed files with 632 additions and 147 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
frontend/src/routes/export/+page.svelte
View File

@@ -81,21 +81,28 @@
}
}
// Stream the (potentially multi-GB) archive straight to disk via a top-level
// download navigation instead of fetch()+blob() — buffering the whole ZIP in
// memory crashes mobile Safari/Chrome. The navigation can't send the Bearer
// header, so we first exchange the token for a single-use ticket and pass it
// as a query param; the server responds with Content-Disposition: attachment,
// so the browser downloads it and we stay on the page.
// Stream the (potentially multi-GB) archive straight to disk instead of fetch()+blob() —
// buffering the whole ZIP in memory crashes mobile Safari/Chrome. The request can't carry the
// Bearer header, so we exchange the token for a single-use ticket and pass it as a query param.
//
// The navigation targets a HIDDEN IFRAME, not the top-level page. On success the server sends
// `Content-Disposition: attachment` and the browser downloads it either way — but on an ERROR
// (404 because a takedown/ban just retired this generation and it's rebuilding, or 429 from the
// per-IP export rate limit, which everyone on the venue WiFi shares) there is no attachment
// header, so a top-level navigation would UNLOAD THE APP and dump the raw `{"error":…}` JSON on
// the guest. Inside an iframe the error is swallowed harmlessly and the PWA stays put.
//
// We deliberately don't probe with HEAD first: the ticket is single-use and axum serves HEAD from
// the GET route, so a probe would consume the ticket and burn a rate-limit slot.
let downloadFrame: HTMLIFrameElement;
async function downloadFile(endpoint: string) {
try {
const { ticket } = await api.post<{ ticket: string }>('/export/ticket');
const a = document.createElement('a');
a.href = `${endpoint}?ticket=${encodeURIComponent(ticket)}`;
a.rel = 'noopener';
document.body.appendChild(a);
a.click();
a.remove();
downloadFrame.src = `${endpoint}?ticket=${encodeURIComponent(ticket)}`;
// If the archive was retired between the page render and this click, the download
// silently does nothing — so re-sync the status to flip the UI to "generating".
setTimeout(() => void loadStatus(), 1500);
} catch (e) {
toastError(e);
}
@@ -120,6 +127,11 @@
}
</script>
<!-- Download sink. Archive downloads are navigations (they can be multi-GB, so fetch()+blob() would
OOM mobile browsers), and a navigation to an error response would unload the whole PWA. Pointing
them at this hidden iframe keeps a 404/429 harmless. -->
<iframe bind:this={downloadFrame} title="Download" class="hidden" aria-hidden="true"></iframe>
<!-- HTML guide modal -->
{#if showHtmlGuide}
<div class="fixed inset-0 z-50 flex items-center justify-center bg-black/50 p-4">