fix: close the nine ways an unattended event loses photos or dies

Every one of these was found in the pre-event audit, verified against source, and
survives to production on the current main. Grouped by what actually goes wrong.

PHOTOS DISAPPEAR

* compression.rs no longer soft-deletes on a failed derivative. The guest got a
  201, watched the card appear, then watched it vanish — the row left v_feed,
  find_visible_media and BOTH keepsakes, while its bytes sat on disk for 14 days
  waiting for a cleanup nothing announced. No screen anywhere lists compression
  failures, so recovery meant hand-written SQL that also had to re-add the
  refunded quota. Now it does exactly what the ENOSPC arm beside it already did
  and documented as correct: keep the row, serve the original, retry on the next
  boot (bounded by derivative_attempts). `upload-deleted` is no longer emitted;
  `upload-processed` is, so the card re-renders instead of sitting on a
  placeholder.

* A 413 is now a reversible lock, so the blob survives. The quota moves — free
  disk falls, uploader count rises — so a guest goes over it having done nothing,
  and treating that as permanent meant a 400 MB video was pushed across cellular
  in full and THEN deleted from IndexedDB. Gone on both sides, and unrecoverable
  for an in-app camera capture that exists nowhere else.

* quota_limit_bytes gained a floor and a stable divisor. The ceiling used to
  decrease monotonically all evening; it now settles at max(uploaders,
  estimated_guest_count) — a config key that was seeded, validated in the admin
  whitelist, and read by no code at all. The floor is clamped to what the disk
  can actually back, so a full volume still yields zero rather than handing out
  an allowance it cannot honour.

* Because that floor gives up the aggregate guarantee the formula used to imply,
  uploads now check a hard 10 GB reserve first, independent of every quota
  toggle. postgres_data, media_data and exports_data share one filesystem: the
  end state was not a degraded feature, it was Postgres unable to write WAL.

THE ARCHIVE DISAPPEARS

* prune_superseded_archives runs only after the new generation lands. It ran
  before the preflight, reasoning the old archive was already unreachable — true
  of reachability, false of recoverability. An epoch is a value that can be
  rolled back; deleted bytes cannot. Any failed rebuild left the event with NO
  keepsake at all.

* The export preflight reserves the same 10 GB. `free < needed` authorised an
  export sized at exactly free, which ran for half an hour and landed the box at
  zero with the keepsake still unfinished.

THE APP DIES

* The feed reconcile re-reads the id set after its awaits instead of reusing one
  captured up to three round-trips earlier. The new-upload SSE handler prepends
  during exactly that window, so the row was both already present and absent from
  the stale set — prepended twice, and a duplicate key in a keyed {#each} throws
  in production, not just dev. The SSE handler and loadMore now dedupe too.

* Added routes/+error.svelte. Without it any uncaught error fell through to
  SvelteKit's unstyled English 500 with no reload control — inside a chromeless
  standalone PWA with no URL bar, for the rest of the evening.

THE OPERATOR IS LOCKED OUT

* admin_login verifies the password BEFORE charging the rate bucket, and a
  correct password is never throttled. The old order made this a denial of
  service against its own operator: every guest shares one NAT IP, the check ran
  first, so five requests a minute from any phone in the room kept the bucket
  full — and the escape hatch needed the admin session being blocked. A generous
  separate ceiling still bounds bcrypt CPU.

THE PROJECTOR DIES

* The preload budget is now strictly inside the dwell. At the 3s option the 4s
  budget could never land a commit on a slow uplink, so the wall froze on one
  photo while the queue drained silently behind it.

* The wake lock retries every 30s while visible, and the page says so on screen
  when the browser has no wake lock API. visibilitychange was the only retry
  trigger and a kiosk never changes visibility, so one refusal — iOS in Low Power
  Mode, say — was permanent.

* Caddy: /api/v1/upload/*/display joins the cacheable carve-out. The backend set
  max-age=300 on it and the blanket no-store silently replaced it, so a projector
  re-fetched a full-size JPEG per slide, ~2-4 GB over an evening on the uplink
  the guests are uploading over.

Also removes Upload::soft_delete, now unreferenced and an unscoped footgun next
to soft_delete_in_event.

Verified: 146/146 backend tests against a live Postgres, clippy clean, 51/51
vitest, svelte-check 0 errors, eslint clean, vite build, caddy validate, compose
YAML parse.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-08-08 21:32:52 +02:00
parent 61119be817
commit 1d9fb11c7b
12 changed files with 448 additions and 106 deletions

View File

@@ -409,6 +409,14 @@ pub struct AdminLoginResponse {
pub display_name: String,
}
/// Requests per minute per IP that may reach `verify_password` at all.
///
/// Not a security control — the failure bucket below is. This exists solely so an unauthenticated
/// endpoint cannot burn the box's CPU on cost-12 bcrypt (~250 ms each) at line rate. Set far above
/// anything a person typing a password can produce, because on venue NAT every guest shares the
/// operator's IP and this ceiling, unlike the failure bucket, can still refuse a correct password.
const ADMIN_LOGIN_CPU_CEILING: usize = 120;
pub async fn admin_login(
State(state): State<AppState>,
ConnectInfo(peer): ConnectInfo<SocketAddr>,
@@ -421,21 +429,29 @@ pub async fn admin_login(
));
}
// Throttle password attempts. The admin password is bcrypt-hashed (slow to
// verify) but with no IP-level limit a determined attacker can still mount
// a long-running guess campaign. 5 attempts / minute / IP is plenty for
// honest typos.
// Throttling here is in two parts, and the ORDER is the whole point.
//
// A single tight IP-keyed bucket checked before the password was verified made this
// endpoint a denial-of-service against its own operator. Every guest at the venue shares
// one public IP behind NAT, `/admin/login` is a public linkable page, and the check ran
// BEFORE `verify_password` — so five requests a minute from any phone in the room kept the
// bucket permanently full and the admin, on that same IP, could never spend a slot.
// Successful logins consumed budget too, so a typo plus a retry on two devices did it by
// accident. And the escape hatch was circular: `admin_login_rate_enabled` can only be
// flipped through `PATCH /admin/config`, which needs the session being blocked.
let ip = client_ip(&headers, &peer.ip().to_string());
let rate_limits_on = config::get_bool(&state.config_cache, "rate_limits_enabled", true).await;
let admin_rate_on =
config::get_bool(&state.config_cache, "admin_login_rate_enabled", true).await;
// Stays keyed by IP on purpose: this guards a single shared credential, so a per-user
// or per-name key would just hand an attacker a fresh bucket per guess.
// Part 1: a deliberately GENEROUS ceiling whose only job is to bound bcrypt CPU. Cost-12
// verification is ~250 ms of a core, so an unbounded endpoint is a CPU exhaustion vector
// regardless of whether anyone guesses right. No human typing a password reaches this.
if rate_limits_on
&& admin_rate_on
&& let Err(retry_after_secs) = state.rate_limiter.check_with_retry(
format!("admin_login:{ip}"),
5,
format!("admin_login_cpu:{ip}"),
ADMIN_LOGIN_CPU_CEILING,
Duration::from_secs(60),
)
{
@@ -452,6 +468,24 @@ pub async fn admin_login(
.await;
if !valid {
// Part 2: the tight bucket, charged ONLY on a wrong password. A correct password is
// never rate-limited, so no amount of guessing by anyone else can lock the operator
// out — which also dissolves the circular escape hatch above. Brute force is still
// bounded: every wrong guess costs a slot, and slots are per-IP.
if rate_limits_on
&& admin_rate_on
&& let Err(retry_after_secs) = state.rate_limiter.check_with_retry(
format!("admin_login_fail:{ip}"),
5,
Duration::from_secs(60),
)
{
tracing::warn!(ip = %ip, "admin_login: wrong password, failure bucket exhausted");
return Err(AppError::TooManyRequests(
"Zu viele Anmeldeversuche. Bitte warte kurz und versuche es erneut.".into(),
Some(retry_after_secs),
));
}
tracing::warn!(ip = %ip, "admin_login: wrong password");
return Err(AppError::Unauthorized("Falsches Passwort.".into()));
}

View File

@@ -443,6 +443,38 @@ pub async fn upload(
// concurrent uploads from the same user (e.g. phone + laptop) both pass this stale
// pre-check and both increment, blowing past the quota. The pre-check stays as a
// fast path that avoids the disk write when the user is already clearly over.
// GLOBAL RESERVE, checked before the per-user ceiling and independent of every quota
// toggle. The per-user quota is a fairness mechanism, not a disk guarantee — and since it
// now carries a floor (MIN_QUOTA_LIMIT_BYTES) so a guest's allowance stops shrinking as
// the party fills up, the aggregate ceiling it used to imply is gone entirely. Something
// has to own "do not fill the volume", because `postgres_data`, `media_data` and
// `exports_data` share one filesystem: the end state is not a degraded feature, it is
// Postgres unable to write WAL and the whole event down with nobody watching.
//
// Deliberately NOT gated behind `quota_enabled`. That switch exists so an operator can
// stop rationing space between guests; it was never meant to authorise running the disk
// to zero, and an operator flipping it at 23:00 to unblock a guest should not silently
// disarm the last thing standing between the party and a dead database.
if let Some(free) = crate::services::disk::free_bytes(&state.config.media_path) {
let remaining = (free as i64).saturating_sub(size);
if remaining < DISK_RESERVE_BYTES {
tracing::error!(
free_bytes = free,
upload_size = size,
reserve = DISK_RESERVE_BYTES,
"refusing upload: it would take the media volume below the reserve"
);
return Err(AppError::QuotaExceeded(
"Der Speicher des Events ist voll. Bitte sag einem Host Bescheid — neue \
Uploads sind vorübergehend nicht möglich."
.into(),
));
}
}
// Failing OPEN when the disk can't be read is deliberate and matches the per-user quota
// above: refusing every upload because a `statfs` failed would be a worse outage than the
// one being guarded against.
let mut quota_limit: Option<i64> = None;
if quota_on && storage_quota_on {
let estimate = compute_storage_quota(&state).await;
@@ -956,11 +988,52 @@ pub struct QuotaEstimate {
pub tolerance: f64,
}
/// Pure per-user quota formula: `floor((free_disk * tolerance) / max(active, 1))`.
/// The smallest per-user ceiling this formula is ever allowed to produce.
///
/// Without a floor the quota is not a limit, it is a moving target: the numerator (free disk)
/// only falls and the denominator (uploaders who have posted) only rises, so the ceiling
/// decreases monotonically across the event. A guest comfortably under it at 20:00 is over it
/// at 22:00 having done nothing, and because a delete refunds the quota but does not free the
/// bytes for 24h, the remedy the error message names ("delete older posts") cannot move it
/// back either.
///
/// 500 MB is chosen to clear `max_video_size_mb` (500, seeded in 005) — below that the ceiling
/// could refuse a single legal video outright, which is the worst version of this: the guest
/// pushes 500 MB across cellular and is rejected on arrival, every time, with no way to comply.
///
/// This deliberately trades the quota's disk guarantee for a usability floor. The disk is now
/// bounded by the low-disk warning and the export preflight rather than by this formula alone —
/// see the reserve check in `ensure_export_space`.
const MIN_QUOTA_LIMIT_BYTES: i64 = 500 * 1024 * 1024;
/// Free space on the media volume that uploads may never consume, whatever any quota says.
///
/// Matches the host dashboard's low-disk threshold (`handlers::host`), so the banner the host
/// sees and the wall guests hit are the same number rather than two unrelated opinions. 10 GB
/// is chosen to leave Postgres, its WAL and a rotation of container logs comfortable room on
/// the shared filesystem long after new uploads have been refused.
pub const DISK_RESERVE_BYTES: i64 = 10_000_000_000;
/// Pure per-user quota formula: `max(floor((free_disk * tolerance) / divisor), MIN)`.
///
/// `divisor` is the LARGER of the observed uploader count and the operator's
/// `estimated_guest_count`, so the ceiling settles at its final value early instead of sliding
/// down all evening as guests arrive. (Before this, `estimated_guest_count` was seeded and
/// validated in the admin whitelist but read by no code at all — an operator who set it
/// expecting a stable divisor changed nothing.) It also blunts the abuse case, where the
/// divisor was attacker-controlled: ~1000 throwaway accounts drove every real guest's ceiling
/// to ~52 MB.
///
/// Extracted from `compute_storage_quota` so it's unit-testable without a DB or disk.
fn quota_limit_bytes(free_disk: i64, tolerance: f64, active_uploaders: i64) -> i64 {
let active = active_uploaders.max(1);
((free_disk as f64 * tolerance) / active as f64).floor() as i64
fn quota_limit_bytes(free_disk: i64, tolerance: f64, active_uploaders: i64, expected: i64) -> i64 {
let divisor = active_uploaders.max(expected).max(1);
let budget = (free_disk as f64 * tolerance).max(0.0);
let computed = (budget / divisor as f64).floor() as i64;
// The floor may never exceed what the disk can actually back. Raising a ceiling the volume
// cannot honour would hand out an allowance on a full disk — turning the quota from a
// usability floor into a way to finish filling the filesystem that Postgres writes WAL to.
let backed_floor = MIN_QUOTA_LIMIT_BYTES.min(budget as i64);
computed.max(backed_floor)
}
/// Computes the per-user storage quota using
@@ -979,6 +1052,9 @@ pub async fn compute_storage_quota(state: &AppState) -> QuotaEstimate {
.await
.unwrap_or((0,));
let active = active_count.max(1);
// The operator's expected headcount, used as a FLOOR on the divisor so the ceiling doesn't
// slide down as guests arrive — see `quota_limit_bytes`. Admin-editable at runtime.
let expected_guests = config::get_i64(&state.config_cache, "estimated_guest_count", 100).await;
// Cached disk reading. `None` means we couldn't resolve the media filesystem.
let disk = state.disk_cache.snapshot(&state.config.media_path);
@@ -986,7 +1062,12 @@ pub async fn compute_storage_quota(state: &AppState) -> QuotaEstimate {
let limit_bytes = if quota_on && storage_quota_on {
match disk {
Some(d) => Some(quota_limit_bytes(d.free as i64, tolerance, active)),
Some(d) => Some(quota_limit_bytes(
d.free as i64,
tolerance,
active,
expected_guests,
)),
// Fail OPEN, not closed: if the disk can't be read we don't know the real
// free space, and enforcing a 0-byte limit would reject every upload with a
// spurious "quota reached". Skip enforcement this round and warn instead.
@@ -1289,7 +1370,7 @@ pub async fn get_thumbnail(
#[cfg(test)]
mod tests {
use super::{RangeSpec, parse_range, quota_limit_bytes};
use super::{MIN_QUOTA_LIMIT_BYTES, RangeSpec, parse_range, quota_limit_bytes};
// `Range` handling exists because iOS Safari probes every `<video>` with
// `Range: bytes=0-1` and abandons the load without a 206. These pin the forms a
@@ -1392,33 +1473,70 @@ mod tests {
assert_eq!(parse_range(None, 0), RangeSpec::Full);
}
const GB: i64 = 1024 * 1024 * 1024;
#[test]
fn divides_free_space_by_uploaders_with_tolerance() {
// 1000 * 0.75 / 3 = 250
assert_eq!(quota_limit_bytes(1000, 0.75, 3), 250);
// 100 GB * 0.75 / 3 uploaders, well above the floor so the formula shows through.
assert_eq!(quota_limit_bytes(100 * GB, 0.75, 3, 1), 26_843_545_600);
}
#[test]
fn floors_fractional_results() {
// 1000 * 0.75 / 7 = 107.14… → 107
assert_eq!(quota_limit_bytes(1000, 0.75, 7), 107);
// 100 GB * 0.75 / 7 = 11_504_376_685.71… → truncated, not rounded.
assert_eq!(quota_limit_bytes(100 * GB, 0.75, 7, 1), 11_504_376_685);
}
#[test]
fn active_uploaders_below_one_is_clamped_to_one() {
fn divisor_below_one_is_clamped_to_one() {
// Guards against divide-by-zero when no one has uploaded yet.
assert_eq!(quota_limit_bytes(1000, 1.0, 0), 1000);
assert_eq!(quota_limit_bytes(1000, 1.0, -5), 1000);
assert_eq!(quota_limit_bytes(10 * GB, 1.0, 0, 0), 10 * GB);
assert_eq!(quota_limit_bytes(10 * GB, 1.0, -5, 0), 10 * GB);
}
/// The property the floor exists for: a guest's ceiling must not keep shrinking as more
/// guests arrive. Same disk, 10 uploaders vs 1000 — the second must not be starved.
#[test]
fn zero_free_disk_yields_zero() {
assert_eq!(quota_limit_bytes(0, 0.75, 3), 0);
fn the_ceiling_stops_falling_once_it_reaches_the_floor() {
let ten = quota_limit_bytes(70 * GB, 0.75, 10, 1);
let thousand = quota_limit_bytes(70 * GB, 0.75, 1000, 1);
assert!(ten > MIN_QUOTA_LIMIT_BYTES, "10 uploaders should be roomy");
assert_eq!(
thousand, MIN_QUOTA_LIMIT_BYTES,
"1000 uploaders (or 1000 fake accounts) must not drive the ceiling below the floor"
);
assert!(
thousand >= 500 * 1024 * 1024,
"the floor must still clear a single max-size video"
);
}
/// `estimated_guest_count` is a FLOOR on the divisor, so the ceiling settles early instead
/// of sliding down all evening as guests arrive.
#[test]
fn expected_headcount_holds_the_divisor_steady_while_guests_arrive() {
let early = quota_limit_bytes(70 * GB, 0.75, 5, 100);
let late = quota_limit_bytes(70 * GB, 0.75, 100, 100);
assert_eq!(
early, late,
"the 5th guest and the 100th must see the same ceiling"
);
}
/// The floor must never write a cheque the volume cannot cash — otherwise a full disk
/// still hands out a 500 MB allowance and the filesystem Postgres needs fills up.
#[test]
fn the_floor_never_exceeds_what_the_disk_can_back() {
assert_eq!(quota_limit_bytes(0, 0.75, 3, 1), 0, "no disk, no allowance");
// 400 MB free * 0.75 = 300 MB — below the floor, so the disk wins.
let tight = quota_limit_bytes(400 * 1024 * 1024, 0.75, 3, 1);
assert_eq!(tight, 300 * 1024 * 1024);
assert!(tight < MIN_QUOTA_LIMIT_BYTES);
}
#[test]
fn full_tolerance_is_identity_for_a_single_uploader() {
assert_eq!(quota_limit_bytes(500, 1.0, 1), 500);
assert_eq!(quota_limit_bytes(50 * GB, 1.0, 1, 1), 50 * GB);
}
/// The guard is the only reclaim mechanism that survives a client disconnect, so its three

View File

@@ -259,40 +259,8 @@ impl Upload {
Ok(())
}
/// Soft-deletes the upload and decrements the uploader's `total_upload_bytes`.
/// Done in a single transaction so a crash between the two writes can't leave
/// the quota counter pointing at bytes the user has already deleted (which would
/// silently lock them out of future uploads).
///
/// No-op if the row is already deleted — protects against a double-tap on the
/// delete action double-decrementing the counter.
pub async fn soft_delete(pool: &PgPool, id: Uuid) -> Result<(), sqlx::Error> {
let mut tx = pool.begin().await?;
let row: Option<(Uuid, i64)> = sqlx::query_as(
"UPDATE upload
SET deleted_at = NOW()
WHERE id = $1 AND deleted_at IS NULL
RETURNING user_id, original_size_bytes",
)
.bind(id)
.fetch_optional(&mut *tx)
.await?;
if let Some((user_id, bytes)) = row {
sqlx::query(
"UPDATE \"user\"
SET total_upload_bytes = GREATEST(0, total_upload_bytes - $2)
WHERE id = $1",
)
.bind(user_id)
.bind(bytes)
.execute(&mut *tx)
.await?;
}
tx.commit().await?;
Ok(())
}
/// Event-scoped variant of [`Self::soft_delete`]. Returns `false` if no row
/// Soft-deletes an upload within its event and refunds the uploader's
/// `total_upload_bytes`, in one transaction. Returns `false` if no row
/// matched (already deleted, wrong event, or unknown id) so host handlers
/// can return a clean 404 instead of silently no-op'ing.
/// Executor-generic so a caller can run the delete and the keepsake regeneration in ONE

View File

@@ -165,35 +165,43 @@ impl CompressionWorker {
tracing::error!(
"compression failed for upload {upload_id} after {attempt} attempt(s): {e:#}"
);
// Refund + soft-delete (one tx, so v_feed excludes it) so a failed
// transcode doesn't leave a permanently broken feed card or silently
// charge the uploader's quota. Then tell the uploader (upload-error
// toast) and evict the card everywhere (upload-deleted).
// KEEP THE ROW. This used to soft-delete, which made a derivative failure
// indistinguishable — to the guest — from their photo being deleted: they
// got a `201 Created`, watched the card appear, and then watched it vanish.
// The row left `v_feed`, `find_visible_media` and BOTH keepsake archives,
// so the photo was gone from the product's core promise while its bytes sat
// on disk for 14 days waiting for a `cleanup_deleted_media` that nothing
// told anyone about. There is no host or admin screen listing compression
// failures, so recovery meant hand-written SQL that also had to re-add the
// refunded quota bytes. Against "0 lost uploads", that was silent per-photo
// loss on any error the ENOSPC arm above doesn't catch — a HEIC that slipped
// the allowlist, a truncated frame, an ffmpeg hiccup, a pool blip.
//
// The ORIGINAL IS DELIBERATELY KEPT. This path used to `remove_file` it
// unconditionally, which meant any transient error — a disk-full blip
// while saving a derivative, a pool hiccup, a panic in the image codec
// irreversibly destroyed the guest's only copy of a photo they can never
// retake. The row is only soft-deleted, so keeping the bytes makes the
// upload fully recoverable; the file is orphaned rather than lost, and
// the path is logged so it can be found. `backfill_stale_derivatives`
// already refuses to destroy data on error for exactly this reason.
// This is exactly what the ENOSPC arm already does and documents as correct:
// every client falls back to the original when `preview_url` and
// `thumbnail_url` are NULL, so the photo stays visible and downloadable
// just uncompressed — and `backfill_stale_derivatives` retries it on the
// next boot, now bounded by `derivative_attempts` so a poisoned row cannot
// loop. The quota stays charged, which is correct: the bytes are still on
// disk and still the guest's.
let _ = Upload::set_compression_status(&worker.pool, upload_id, "failed").await;
if let Err(del) = Upload::soft_delete(&worker.pool, upload_id).await {
tracing::warn!(error = ?del, %upload_id, "failed to soft-delete after compression failure");
}
tracing::warn!(
%upload_id,
path = %worker.media_path.join(&original_path).display(),
"original retained for recovery after compression failure"
"derivatives failed; the upload is kept and served from its original"
);
// `upload-error` still fires so the uploader learns the photo will look
// uncompressed. `upload-deleted` deliberately does NOT — nothing was
// deleted, and evicting the card was the visible half of the data loss.
let _ = worker.sse_tx.send(SseEvent {
event_type: "upload-error".to_string(),
data: serde_json::json!({ "upload_id": upload_id, "error": e.to_string() })
.to_string(),
});
// Tell every client to refetch, so the card re-renders from the original
// instead of sitting on a stale "processing" placeholder forever.
let _ = worker.sse_tx.send(SseEvent {
event_type: "upload-deleted".to_string(),
event_type: "upload-processed".to_string(),
data: serde_json::json!({ "upload_id": upload_id }).to_string(),
});
}

View File

@@ -499,9 +499,6 @@ async fn run_zip_export(
return Ok(());
}
// Reclaim BEFORE measuring: the superseded archive is already unreachable, and the space it
// holds is very often exactly the space this rebuild needs.
prune_superseded_archives(pool, export_path, "Gallery", event_id, epoch).await;
// AFTER the claim, not before. A preflight that bailed before claiming would leave the row
// `pending` with no worker and no error — the spinner-forever state `mark_failed`'s status
// guard was widened to prevent. Failing here goes through the caller's `mark_failed`, so the
@@ -517,6 +514,25 @@ async fn run_zip_export(
tokio::fs::remove_file(export_path.join(gen_name(event_id, "Gallery", epoch, ".tmp")))
.await;
}
// Reclaim the PREVIOUS generation only once this one has actually landed.
//
// This used to run before the preflight, reasoning that the superseded archive is already
// unreachable and its space is usually exactly what the rebuild needs. That is true about
// REACHABILITY and false about RECOVERABILITY: an epoch is a database value that can be
// rolled back, deleted bytes cannot. Any rebuild that then failed — ENOSPC mid-write, an
// OOM, a hung ffmpeg, a host tapping "Neu erzeugen" on a bad day — left the event with NO
// archive at all, which is the one outcome the whole product exists to prevent, at the one
// moment nobody is watching.
//
// The cost of deferring is that a rebuild now needs room for both generations at once, and
// `ensure_export_space` above no longer gets to count the old archive's bytes as available.
// That is the correct trade: it converts "silently destroyed the only copy" into "refused
// to start, and said why".
if res.is_ok() {
prune_superseded_archives(pool, export_path, "Gallery", event_id, epoch).await;
}
abandon_if_superseded("ZIP", event_id, epoch, res)
}
@@ -690,9 +706,8 @@ async fn run_html_export(
return Ok(());
}
// See run_zip_export: reclaim the superseded generation first, then refuse at the door rather
// than ENOSPC mid-write.
prune_superseded_archives(pool, export_path, "Memories", event_id, epoch).await;
// See run_zip_export: refuse at the door rather than ENOSPC mid-write, and reclaim the
// superseded generation only AFTER this one lands.
ensure_export_space(pool, event_id, export_path).await?;
let res = run_html_export_inner(
@@ -716,6 +731,13 @@ async fn run_html_export(
tokio::fs::remove_dir_all(export_path.join(format!("viewer_tmp_{event_id}_{epoch}")))
.await;
}
// Only once the new keepsake exists — see the reasoning in run_zip_export. A failed rebuild
// must never be the reason the previous one is gone.
if res.is_ok() {
prune_superseded_archives(pool, export_path, "Memories", event_id, epoch).await;
}
abandon_if_superseded("HTML", event_id, epoch, res)
}
@@ -1383,18 +1405,32 @@ async fn ensure_export_space(pool: &PgPool, event_id: Uuid, export_path: &Path)
return Ok(());
};
if free < needed {
// The archive may not consume the last byte of the volume. `needed` alone authorised an
// export sized at exactly `free`: it would pass the check, run for half an hour, and land
// the box at zero — at which point Postgres cannot write WAL and the event is over, with
// the keepsake still unfinished. `postgres_data`, `media_data` and `exports_data` share one
// filesystem, so "enough room for the archive" was never the same question as "enough room
// for the archive AND a working database".
//
// Same reserve the upload path and the host dashboard's low-disk banner use, so all three
// agree on what "full" means.
let required = needed.saturating_add(crate::handlers::upload::DISK_RESERVE_BYTES as u64);
if free < required {
let gb = |b: u64| b as f64 / 1_000_000_000.0;
tracing::error!(
needed,
required,
free,
armed,
"export preflight: not enough free space to build the keepsake for event {event_id}"
);
anyhow::bail!(
"Nicht genug Speicherplatz für das Keepsake: benötigt ca. {:.1} GB, frei sind {:.1} GB. \
Bitte Speicher freigeben und das Keepsake anschließend neu erstellen.",
"Nicht genug Speicherplatz für das Keepsake: benötigt ca. {:.1} GB (plus {:.0} GB \
Reserve), frei sind {:.1} GB. Bitte Speicher freigeben und das Keepsake \
anschließend neu erstellen.",
gb(needed),
gb(crate::handlers::upload::DISK_RESERVE_BYTES as u64),
gb(free)
);
}