fabi/EventSnap
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(export): close all 13 findings from the multi-agent review

Browse Source 
A 6-lens multi-agent review (103 agents; every finding adversarially verified by three
refute-by-default skeptics) confirmed the epoch redesign is sound — the core invariant
holds — and found 13 real defects around it. All are fixed here.

The redesign's invariant was re-verified and stands: an accepted upload is always in the
keepsake, and a downloadable keepsake always reflects the most recent release.

But a WEAKER adjacent invariant did NOT hold — "a downloadable keepsake reflects the
current content" — and that is the theme of the biggest fixes.

CONTENT REMOVAL NOW ALWAYS REACHES THE KEEPSAKE
  Regeneration was wired only to host deletes. Ban, unban, guest self-delete and guest
  comment-delete did not trigger it — yet the export query filters `is_banned = FALSE`,
  so the pipeline already agreed that content must not be there. Ban someone for abusive
  content after release and every guest kept downloading an archive containing it.
  All five removal paths now invalidate and rebuild. `query_comments` also gained the
  banned/hidden filter it was missing entirely (the ZIP had it; the viewer did not).

  Each removal is now ATOMIC with its invalidation (one tx, models made executor-generic).
  Previously the delete committed in its own tx and the regeneration in a second: a dropped
  handler future between them left the taken-down photo permanently downloadable, and
  nothing could notice — the keepsake still looked complete and the host could no longer
  even find the upload to retry.

NO MORE TAKEDOWN STORM
  Every removal spawned two uncancellable full exports. A superseded worker was INERT, not
  STOPPED: it still ran every ffmpeg spawn and image resize and wrote a whole archive before
  discovering it had lost. Five deletes left ten workers alive, each holding a
  full-gallery-sized temp, in a 1 GB container.
  Now: a debounce before `claim_job` (a superseded worker fails its claim and does ZERO
  work, collapsing a burst into one export), plus `update_progress` — which already ran
  exactly the right liveness predicate and threw the answer away — returning it so both file
  loops bail the instant they are retired. Moderating a comment no longer rebuilds the
  multi-GB ZIP either: the ZIP is media-only, so it is carried forward, with prune taught to
  protect any file a live row still references.

AN ESCAPE HATCH
  A failed export was terminal at runtime: release_gallery refuses an already-released event,
  recovery only runs at boot, and there was no retry route — the only outs were restarting the
  container or reopening uploads to every guest. Added POST /host/export/rebuild. Also widened
  mark_failed's guard to `status IN ('running','pending')`: when claim_job itself ERRORED, the
  row was still `pending`, so the failure was never recorded and the job sat at 0% forever.

SILENT INVALIDATION
  Retiring the epoch 404s the download instantly, but nothing told the clients — guests kept
  seeing an enabled download button through the whole rebuild. Now broadcasts an invalidation.
  And the download was a top-level <a href>: a 404 (or a 429 from the per-IP export limit that
  everyone on the venue WiFi shares) NAVIGATED THE USER OUT OF THE PWA onto raw JSON. It now
  targets a hidden iframe, so an error response is harmless.

ALSO
  - The HTML export still had the exists()-then-open TOCTOU the ZIP path was fixed to remove,
    at three sites, two with a hard `?` that failed the ENTIRE viewer. It is reachable: the
    compression worker hard-deletes an original when a transcode fails and can still be running
    when the gallery is released.
  - Crash-orphaned .tmp files were immortal (prune deliberately skips .tmp). Swept at boot,
    where it is unambiguously safe.
  - export_status read the epoch in one statement and used it in the next; now one query.
  - DiskCache::snapshot IGNORED its path argument on a cache hit, returning whatever filesystem
    was measured last. Latent only because both callers pass media_path — it would have silently
    misreported the moment anyone measured the exports volume. Now keyed by path.
  - Corrected two comments that asserted safety properties the code does not have (claim_job
    does NOT prevent a retired-epoch claim — retirement is enforced at READ time; and a
    multi-replica reap is NOT contained, it can corrupt the keepsake). Added a rollback runbook
    to migration 014, the repo's first destructive migration.

TESTS
  The regression guard for the headline FOR SHARE fix was probabilistic — it could pass on
  broken code if the interleaving fell the wrong way. It is now STRUCTURAL: the upload is held
  open mid-body with its pre-flight check already passed, so the release provably lands inside
  the exact window. Verified 5/5 FAILING (201 instead of 403) with the fix reverted, 5/5
  passing with it.

Verified: backend 40 tests, svelte-check 0 errors, e2e typecheck clean,
e2e 160 passed / 1 skipped on chromium-desktop.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
fabi
2026-07-14 20:35:47 +02:00
parent 5affef47cf
commit 9666d74a46
14 changed files with 632 additions and 147 deletions
Download Patch File Download Diff File Expand all files Collapse all files

159
backend/src/handlers/host.rs
View File

@@ -12,6 +12,7 @@ use crate::models::event::Event;
use crate::models::session::Session;
use crate::models::upload::Upload;
use crate::models::user::UserRole;
use crate::services::export::Affects;
use crate::state::{AppState, SseEvent};
// ── DTOs ─────────────────────────────────────────────────────────────────────
@@ -148,6 +149,9 @@ pub async fn ban_user(
// (enforced live on the write handlers + Require{Host,Admin}). Revoking sessions would
// contradict that model, break the documented "banned guest can still download the
// keepsake" flow, and be ineffective anyway (the user could just /recover a new session).
//
// The ban and the keepsake invalidation are ONE transaction — see `host_delete_upload`.
let mut tx = state.pool.begin().await?;
sqlx::query(
"UPDATE \"user\"
SET is_banned = TRUE, uploads_hidden = TRUE, uploads_hidden_at = NOW()
@@ -155,9 +159,21 @@ pub async fn ban_user(
)
.bind(user_id)
.bind(auth.event_id)
.execute(&state.pool)
.execute(&mut *tx)
.await?;
// A ban hides the user's uploads EVERYWHERE — and the keepsake is the place that matters most,
// because it is the copy people keep. The export already filters `is_banned = FALSE`, so a
// FUTURE export excludes them; without this, an ALREADY-RELEASED archive would keep serving a
// banned user's photos forever. Same class as a takedown, so same treatment.
let regen =
crate::services::export::invalidate_and_arm(&mut tx, &state.config.event_slug, Affects::Both)
.await?;
tx.commit().await?;
if let Some(r) = regen {
start_regen(&state, r);
}
// Evict their content live from every feed + the diashow so it disappears without
// each viewer having to reload. (Their own SSE stream is separately dropped by the
// is_banned revalidation in `sse::stream`, so they stop receiving live pushes while
@@ -205,6 +221,7 @@ pub async fn unban_user(
// Unban restores visibility too: ban set `uploads_hidden = TRUE`, so clearing only
// `is_banned` would leave their content invisible. Clear all three (the timestamp too,
// so a future ban stamps a fresh `uploads_hidden_at` the reconnect delta will replay).
let mut tx = state.pool.begin().await?;
let result = sqlx::query(
"UPDATE \"user\"
SET is_banned = FALSE, uploads_hidden = FALSE, uploads_hidden_at = NULL
@@ -212,11 +229,22 @@ pub async fn unban_user(
)
.bind(user_id)
.bind(auth.event_id)
.execute(&state.pool)
.execute(&mut *tx)
.await?;
if result.rows_affected() == 0 {
return Err(AppError::NotFound("Benutzer nicht gefunden.".into()));
}
// The mirror of the ban case: an unban RESTORES their uploads to the export query, so an
// already-released keepsake is now missing content it should contain. Rebuild it.
let regen =
crate::services::export::invalidate_and_arm(&mut tx, &state.config.event_slug, Affects::Both)
.await?;
tx.commit().await?;
if let Some(r) = regen {
start_regen(&state, r);
}
tracing::info!(
actor_user_id = %auth.user_id,
target_user_id = %user_id,
@@ -226,6 +254,51 @@ pub async fn unban_user(
Ok(StatusCode::NO_CONTENT)
}
/// Force a keepsake rebuild. The ESCAPE HATCH.
///
/// Without this, a failed or stranded export is terminal at runtime: `release_gallery` refuses an
/// already-released event ("bereits freigegeben"), `recover_exports` only runs at boot, and there is
/// no other retry path — so the host's only options were restarting the container or reopening the
/// event (which unlocks uploads to every guest and discards the release). This is also the recovery
/// path for a keepsake that went stale for any reason we haven't thought of.
pub async fn rebuild_export(
State(state): State<AppState>,
RequireHost(auth): RequireHost,
) -> Result<StatusCode, AppError> {
let mut tx = state.pool.begin().await?;
let regen =
crate::services::export::invalidate_and_arm(&mut tx, &state.config.event_slug, Affects::Both)
.await?;
tx.commit().await?;
let Some(r) = regen else {
return Err(AppError::BadRequest(
"Die Galerie ist nicht freigegeben — es gibt nichts neu zu erzeugen.".into(),
));
};
// No debounce: this is an explicit, deliberate host action, not a burst.
for export_type in ["zip", "html"] {
let _ = state.sse_tx.send(SseEvent::new(
"export-progress",
serde_json::json!({ "type": export_type, "progress_pct": 0 }).to_string(),
));
}
crate::services::export::spawn_export_jobs(
r.event_id,
r.event_name,
r.epoch,
std::time::Duration::ZERO,
state.pool.clone(),
state.config.media_path.clone(),
state.config.export_path.clone(),
state.sse_tx.clone(),
);
tracing::info!(actor_user_id = %auth.user_id, "host: rebuild_export");
Ok(StatusCode::NO_CONTENT)
}
pub async fn set_role(
State(state): State<AppState>,
RequireHost(auth): RequireHost,
@@ -445,36 +518,33 @@ pub async fn dismiss_pin_reset_request(
/// stops being downloadable the instant the delete commits, and a fresh worker rebuilds it without
/// the removed content. The download 404s in the meantime, which is the correct answer — serving
/// the old archive would serve the deleted photo.
async fn regenerate_keepsake_if_released(state: &AppState) -> Result<(), AppError> {
let mut tx = state.pool.begin().await?;
let bumped: Option<(Uuid, String, i64)> = sqlx::query_as(
"UPDATE event SET export_epoch = export_epoch + 1
WHERE slug = $1 AND export_released_at IS NOT NULL
RETURNING id, name, export_epoch",
)
.bind(&state.config.event_slug)
.fetch_optional(&mut *tx)
.await?;
// Not released → nothing to regenerate; the export will be built fresh at release time.
let Some((event_id, event_name, epoch)) = bumped else {
return Ok(());
};
crate::services::export::enqueue_jobs_at_epoch(&mut tx, event_id, epoch).await?;
tx.commit().await?;
/// Start the workers for a regeneration that was armed inside a (now-committed) transaction, and
/// tell every client the current keepsake just became undownloadable.
///
/// The SSE matters: bumping the epoch retires the archive INSTANTLY, so `/export/zip` starts 404ing
/// the moment the change commits. Without a nudge, a guest sitting on `/export` keeps rendering an
/// enabled "download" button for the whole rebuild. Both the nav badge and the export page already
/// refetch `/export/status` on `export-progress`, so a 0% tick is the cheapest correct signal.
pub fn start_regen(state: &AppState, regen: crate::services::export::PendingRegen) {
for export_type in ["zip", "html"] {
let _ = state.sse_tx.send(SseEvent::new(
"export-progress",
serde_json::json!({ "type": export_type, "progress_pct": 0 }).to_string(),
));
}
crate::services::export::spawn_export_jobs(
event_id,
event_name,
epoch,
regen.event_id,
regen.event_name,
regen.epoch,
// Debounced: a takedown pass is a burst, and each request retires the last generation. The
// delay lets superseded workers fail their claim and do zero work instead of each building
// a full archive. See export::REGEN_DEBOUNCE.
crate::services::export::REGEN_DEBOUNCE,
state.pool.clone(),
state.config.media_path.clone(),
state.config.export_path.clone(),
state.sse_tx.clone(),
);
Ok(())
}
pub async fn host_delete_upload(
@@ -486,18 +556,26 @@ pub async fn host_delete_upload(
.await?
.ok_or_else(|| AppError::NotFound("Upload nicht gefunden.".into()))?;
let deleted = Upload::soft_delete_in_event(&state.pool, upload_id, auth.event_id).await?;
// The delete and the keepsake invalidation are ONE transaction: if the delete committed and the
// invalidation didn't, the taken-down photo would stay downloadable forever and nothing would
// notice (the keepsake still looks complete, and the host can no longer find the upload to retry).
let mut tx = state.pool.begin().await?;
let deleted = Upload::soft_delete_in_event(&mut tx, upload_id, auth.event_id).await?;
if !deleted {
return Err(AppError::NotFound("Upload nicht gefunden.".into()));
}
let regen =
crate::services::export::invalidate_and_arm(&mut tx, &state.config.event_slug, Affects::Both)
.await?;
tx.commit().await?;
let _ = state.sse_tx.send(SseEvent::new(
"upload-deleted",
serde_json::json!({ "upload_id": upload.id }).to_string(),
));
// A released keepsake must not keep serving a photo the host just took down.
regenerate_keepsake_if_released(&state).await?;
if let Some(r) = regen {
start_regen(&state, r);
}
tracing::info!(
actor_user_id = %auth.user_id,
@@ -514,17 +592,29 @@ pub async fn host_delete_comment(
RequireHost(auth): RequireHost,
Path(comment_id): Path<Uuid>,
) -> Result<StatusCode, AppError> {
let deleted =
Comment::soft_delete_in_event(&state.pool, comment_id, auth.event_id).await?;
let mut tx = state.pool.begin().await?;
let deleted = Comment::soft_delete_in_event(&mut tx, comment_id, auth.event_id).await?;
if !deleted {
return Err(AppError::NotFound("Kommentar nicht gefunden.".into()));
}
// Only the HTML viewer embeds comments — the ZIP is media-only, so it is carried forward rather
// than rebuilt. Otherwise moderating one comment would 404 the photo download for minutes to
// change nothing in it.
let regen = crate::services::export::invalidate_and_arm(
&mut tx,
&state.config.event_slug,
Affects::ViewerOnly,
)
.await?;
tx.commit().await?;
let _ = state.sse_tx.send(SseEvent::new(
"comment-deleted",
serde_json::json!({ "comment_id": comment_id }).to_string(),
));
// The HTML viewer embeds comments — rebuild it so a deleted comment doesn't live on there.
regenerate_keepsake_if_released(&state).await?;
if let Some(r) = regen {
start_regen(&state, r);
}
tracing::info!(
actor_user_id = %auth.user_id,
event_id = %auth.event_id,
@@ -647,6 +737,7 @@ pub async fn release_gallery(
event_id,
event_name,
epoch,
std::time::Duration::ZERO,
state.pool.clone(),
state.config.media_path.clone(),
state.config.export_path.clone(),