fix(audit-5): export disk preflight, media reclaim, low-disk warning, restore docs
Fifth round, all storage. The keepsake could not fit on the documented hardware
and failed halfway through a multi-GB write, leaving the deliverable stuck; the
quota had stopped bounding the disk; and the backup had no restore procedure.
Squashed from 6 commits, original messages preserved below.
──────── fix(export): refuse an export that cannot fit, and stop peaking at two generations
Nothing in export.rs ever asked whether the keepsake would fit. Both archives write
their media `Compression::Stored`, so each is essentially a byte-for-byte second copy
of the originals -- Gallery.zip always, and Memories.zip for every video and every
image at or under 5 MB. On the documented CX33 (80 GB, all three volumes on one
filesystem) the upload quota's fixed point leaves ~40 GB free, and a release spawns
BOTH halves concurrently against it.
The failure is not "the export failed", it is "the deliverable is stuck":
1. ENOSPC lands partway through a multi-GB write.
2. The epoch has already moved, so the job row is `failed` at the CURRENT
generation and readiness (epoch = event.export_epoch AND status = 'done') is
false -- GET /export/zip 404s.
3. The last good archive sits on disk, unreferenced and unreachable.
4. POST /host/export/rebuild, the only escape, re-arms the same doomed write.
Three changes.
Reclaim before building. `prune_stale_export_files` ran only after the new archive
was written, renamed and finalised. That reads as durability but buys nothing: the
moment `invalidate_and_arm` bumps the epoch the old archive is ALREADY unreachable,
so keeping it reserves gigabytes for a download nobody can perform -- and for a
takedown it is content someone explicitly asked to have removed. Peak usage is now
one generation. Narrower than the post-finalize prune on purpose: final archives
only, never a `.tmp` or a `viewer_tmp_` dir, since a superseded worker can still be
streaming into those and at build START is far more likely to be alive.
Preflight the space. SUM(original_size_bytes) over exactly `query_uploads`'
visibility filter, +10% for ZIP overhead, multiplied by the number of armed jobs --
without that multiplier each of the two concurrent halves independently sees "it
fits" and together they don't. Runs AFTER claim_job, not before as reported: bailing
before the claim leaves the row `pending` with no worker and no error, the
spinner-forever state `mark_failed`'s status guard exists to prevent. Fails open when
the mount can't be read, exactly as the upload quota does.
Show the host the reason. /export/status returned {status, progress_pct} and nothing
else, so the host dashboard could only render "fehlgeschlagen" next to the retry
button. The message was written to the row and surfaced solely in the ADMIN job list
-- a different screen, possibly a different person. It now travels with the status,
and only on a failure, so a message left on a since-succeeded row can't appear beside
a green "ist bereit".
Tests: 10 unit (the u128 clamp caught a real bug in the first draft -- saturating_mul
then /100 turns an overflow into a number ~100x too small, the one direction that
authorises the write being guarded against; the carried-forward archive must survive
its own older epoch in the filename), 4 DB-backed (the estimate is asserted against
the row set the archive actually contains, not against a restatement of the WHERE
clause, so the two queries cannot drift), 3 e2e over the four-hop plumbing.
──────── fix(maintenance): reclaim the media of deliberately deleted uploads
The quota stopped bounding the disk. `soft_delete_in_event` stamps `deleted_at` and
refunds `total_upload_bytes`, but nothing ever removed the bytes, and the hourly
sweep reached only `compression_status = 'failed'`. Upload 500 MB, delete, quota back
to zero, upload another 500 MB. Not an attack -- a guest curating their camera roll,
which is what people do. The host then sees guests hitting "Du hast dein Upload-Limit
erreicht" while the admin widget shows a disk full of files no upload row points at,
and the quota message is actively misleading because the space really is gone, just
not to anyone the accounting can name.
Two retention windows, because the two deletes mean different things. A compression
failure keeps its 14 days: the guest didn't ask for it and may not be able to retake
the photo. A deliberate removal gets 24 hours -- 14 days outlives the whole event, so
a deliberate delete would never reclaim anything while it mattered, and a day still
covers a mis-tap.
Wider than reported: ALL FOUR paths are reclaimed, not just the original. Preview,
display and thumbnail are each a separate file, none counted in
`original_size_bytes`, and nothing ever removed them either. That was invisible while
the sweep only saw failed compressions (which produce no derivatives) and becomes
three leaked files per upload the moment it reaches a successful one. A row is
re-selected until every path is cleared, and the columns are cleared only once every
file for that upload is gone -- clearing after a partial success would strand the
survivors in exactly the unowned state this drains.
`backfill_stale_derivatives` selects on `display_path IS NULL AND preview_path IS NOT
NULL`, which is close enough to the post-sweep state to be worth pinning: it is
guarded on `deleted_at IS NULL`, so it cannot re-decode an original that is no longer
on disk. Covered.
Residual, deliberately: within the 24h window the bytes are still spent and still
unaccounted, so delete-and-re-upload through an eight-hour event can outrun the
sweep. Bounding that means holding the quota until the file is reclaimed rather than
refunding at `deleted_at`. The low-disk warning is the net under it.
Tests: 6 DB-backed, replacing 3. The one asserting an owner-deleted upload IS
reclaimed is the exact inverse of what this file used to assert.
──────── feat(host): warn about low disk before it becomes unrecoverable
Storage visibility existed in exactly one place: a passive Speicherauslastung widget
on the ADMIN dashboard. A host who isn't the admin had no view of it, and nothing
warned anyone. README carried "Low-disk alert (< 10 GB free)" under Planned since v1.
Two things make this a safety net rather than a nice-to-have. postgres_data,
media_data and exports_data are all Docker named volumes on ONE filesystem, so
running out doesn't degrade a subsystem -- Postgres stops being able to write and the
whole event goes down. And the keepsake needs room for two gallery-sized archives,
which the export preflight can only ever refuse AFTER the release, when the event is
over and every remedy is harder.
So the threshold is not a fixed number alone. It fires on the 10 GB floor the README
always named, OR on "you could not build the keepsake right now" -- the trigger a
host can still act on, computed with the same arithmetic the preflight uses. Unknown
free space is NOT low: it fails open like the upload quota and the preflight do,
because a banner that cries wolf on an unreadable mount is a banner nobody reads.
Carried on GET /host/event, which the dashboard already fetches on load and on every
reload -- no new endpoint, no new poll. Rendered above everything else including the
PIN-reset queue, and it names the consequence (the event, not just the download)
rather than only the number.
Also fixes the host page's formatBytes, which topped out at MB: 30 GB free would have
rendered as "30720.0 MB", and a guest with 2 GB of uploads was already being shown
that way in the user list.
Tests: 5 unit on the threshold (including that plenty of free space is still low when
the keepsake wouldn't fit -- the case a fixed threshold misses entirely), 3 e2e.
The e2e drives it through `original_size_bytes` rather than a genuinely full disk:
the estimate is pure SQL over that column, so overstating one row moves the
accounting without touching a byte on disk.
──────── docs: add a restore procedure, fix the backup cadence, and correct quota_tolerance
Four things, all found by the same question: what does an operator standing at the
venue actually need?
A RESTORE PROCEDURE. There was none anywhere, and a backup you have never restored
isn't a backup. Two hazards worth writing down: media must be extracted preserving
ownership (the app runs as uid 100 / gid 101, and a root-owned restore makes every
upload fail with EACCES surfacing as a generic 500), and the app must be STOPPED
first, because migrations run on boot and a live pool will fight the restore.
Both the backup and the restore commands were run against the real stack before being
written down, which caught two that would have failed:
- The plain `pg_dump` did not restore: `psql` aborted on `ERROR: schema
"_sqlx_test" already exists`. pg_dump emits no DROPs without --clean --if-exists,
so the documented dump could only ever be restored into an empty database. Fixed
at the source (the dump is now self-cleaning) and verified end to end: 16 tables
back, exit 0.
- `--same-owner` does not exist in BusyBox tar, which is what `alpine` ships, so
the extract aborted before unpacking anything. `--numeric-owner` plus the
explicit chown, verified to land 100:101.
BACKUP CADENCE. "Weekly offsite" is the wrong shape when every irreplaceable byte is
created in one eight-hour window and nobody can retake a wedding. The backup that
matters runs that night, and again after the release so the keepsake is captured.
Also: take the DB dump and the media tarball back to back, or you get rows pointing
at files the dump doesn't know about.
quota_tolerance WAS DOCUMENTED AS SOMETHING IT ISN'T. .env.example called it "fraction
of disk that triggers the low-storage warning". It is the multiplier in
`floor(free_disk * tolerance / active_uploaders)` -- so an operator who wants "warn me
later" and sets 0.95 is actually authorising guests to fill 95% of the disk, moving
the fixed point from 43% to ~49% and eating the export headroom. The admin UI labelled
it "Toleranz (0-1)" with no explanation at all, which invites exactly that reading;
it is now "Speicher-Anteil für Gäste" with the formula in the hint. Wrong docs on a
tuning knob are worse than no docs.
SIZING. New section with the arithmetic: three volumes on one filesystem, the quota
fixed point at tolerance/(1+tolerance), and the fact the 80 GB baseline does not cover
the keepsake -- both archives are built concurrently and each is roughly a second copy
of every original. Provision ~3x expected media, or give exports its own volume.
Also ticks the low-disk alert off the roadmap, since it now exists.
──────── chore: raise the db memory limit and rate-limit social writes
Two smaller operational items.
POSTGRES 512M -> 1G. DATABASE_MAX_CONNECTIONS is 30 for a ~100-guest event (feed
polling + SSE + uploads at once), and 30 backends plus Postgres 16's default
shared_buffers leaves very little headroom at 512M. An OOM here doesn't degrade one
feature -- every request path touches the database, so it takes the event down.
Memory is the cheaper knob than shrinking the pool back and reintroducing the
queueing it was raised to fix. .env.example now names the pairing explicitly, the way
it already does for COMPRESSION_WORKER_CONCURRENCY.
SOCIAL WRITES WERE UNTHROTTLED. toggle_like, add_comment and delete_comment were the
only mutating endpoints in the app with no limit at all -- upload, join, recover,
export and admin login all carry one. Asymmetric coverage rather than a deliberate
decision.
Low severity, and honestly so: a like fans an SSE broadcast to every client, but the
export regeneration a comment deletion triggers is contained (REGEN_DEBOUNCE 20s,
workers born with their epoch, superseded ones inert). So the ceiling is 120/min --
far above anything a real guest produces. This bounds a script, not an enthusiastic
double-tapper.
ONE bucket across all three actions: separate buckets would let a caller triple the
aggregate write rate by alternating between them. Keyed per USER, matching the feed
and upload limits -- at a venue every guest is behind one NAT, and an IP key is what
made the /join and /feed limits turn guests away in the first place.
Migration 020 seeds both keys, and both are wired into the admin allowlist, the
config UI and the e2e reseed -- the step two earlier per-area toggles missed, which
left switches that existed in code and could never be flipped.
Tests: 4 e2e, including that the shared bucket really is shared (the part most likely
to be lost in a refactor) and that one guest hitting the ceiling doesn't block
another behind the same IP.
──────── fix(e2e): stop the video poster assertion racing the ffmpeg thumbnail
Pre-existing, and it fired for real during the full-suite run on a cold stack.
The lightbox binds `poster={upload.thumbnail_url ?? undefined}`, so the attribute is
absent until compression produces the thumbnail. This test asserted on it immediately
after seeding, never waiting for the worker -- unlike the Range test further down the
same file, which does poll. Against a warm stack the worker usually wins; against a
freshly rebuilt one (`stack:down -v`, cold ffmpeg) it doesn't.
That is the worst possible time for a false failure: the first run after a rebuild is
exactly when you are trying to establish whether a change broke something. Poll for
`compression_status = 'done'` before the poster assertion. The `src` assertion needs
no wait and keeps none.
Verified with --repeat-each=3.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
fabi
@@ -476,6 +476,15 @@ async fn run_zip_export(
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
// Reclaim BEFORE measuring: the superseded archive is already unreachable, and the space it
|
||||
// holds is very often exactly the space this rebuild needs.
|
||||
prune_superseded_archives(pool, export_path, "Gallery", event_id, epoch).await;
|
||||
// AFTER the claim, not before. A preflight that bailed before claiming would leave the row
|
||||
// `pending` with no worker and no error — the spinner-forever state `mark_failed`'s status
|
||||
// guard was widened to prevent. Failing here goes through the caller's `mark_failed`, so the
|
||||
// host gets the reason.
|
||||
ensure_export_space(pool, event_id, export_path).await?;
|
||||
|
||||
// On error, mark THIS generation failed — a no-op if we've since been superseded (the
|
||||
// caller in `spawn_export_jobs` does it, epoch-guarded). Temp artifacts are cleaned up
|
||||
// here so a failing export can't leak them (which is what fills the disk in the first place).
|
||||
@@ -658,6 +667,11 @@ async fn run_html_export(
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
// See run_zip_export: reclaim the superseded generation first, then refuse at the door rather
|
||||
// than ENOSPC mid-write.
|
||||
prune_superseded_archives(pool, export_path, "Memories", event_id, epoch).await;
|
||||
ensure_export_space(pool, event_id, export_path).await?;
|
||||
|
||||
let res = run_html_export_inner(
|
||||
epoch,
|
||||
event_id,
|
||||
@@ -1161,6 +1175,196 @@ fn parse_gen_seq(name: &str, prefix: &str, suffix: &str) -> Option<i64> {
|
||||
.ok()
|
||||
}
|
||||
|
||||
/// Filenames a live (current-epoch, `done`) job row still points at — OFF LIMITS to every prune,
|
||||
/// regardless of the epoch encoded in the name.
|
||||
///
|
||||
/// A ViewerOnly regeneration carries the finished ZIP forward by re-stamping its row to the new
|
||||
/// epoch WITHOUT renaming the file, so `Gallery.<event>.<older>.zip` is still the served archive and
|
||||
/// deleting it by filename-epoch would 404 the download.
|
||||
async fn protected_files(pool: &PgPool, event_id: Uuid) -> Vec<String> {
|
||||
sqlx::query_scalar::<_, String>(
|
||||
"SELECT split_part(j.file_path, '/', -1) FROM export_job j
|
||||
JOIN event e ON e.id = j.event_id
|
||||
WHERE j.event_id = $1 AND j.epoch = e.export_epoch
|
||||
AND j.status = 'done' AND j.file_path IS NOT NULL",
|
||||
)
|
||||
.bind(event_id)
|
||||
.fetch_all(pool)
|
||||
.await
|
||||
.unwrap_or_default()
|
||||
}
|
||||
|
||||
/// Reclaim superseded FINAL archives BEFORE this generation starts writing its own.
|
||||
///
|
||||
/// Peak disk usage used to be two full generations, because the only prune ran after the new archive
|
||||
/// was written, renamed and finalised. That ordering reads as durability ("don't delete the good
|
||||
/// keepsake before the replacement is safe") but it buys nothing: readiness is derived from
|
||||
/// `job.epoch = event.export_epoch AND status = 'done'`, so the moment `invalidate_and_arm` bumps
|
||||
/// the epoch the old archive is ALREADY unreachable — `GET /export/zip` 404s whether the file is on
|
||||
/// disk or not. Keeping it only reserves gigabytes for a download nobody can perform, and for
|
||||
/// `Affects::Both` (a takedown) it is content someone has explicitly asked to have removed. So a
|
||||
/// rebuild reclaims first and peaks at one generation.
|
||||
///
|
||||
/// Narrower than [`prune_stale_export_files`] on purpose: FINAL archives only. Those are inert — a
|
||||
/// superseded worker either already renamed its file (and will delete it itself when its guarded
|
||||
/// `finalize_job` fails) or never will. `.tmp` files and `viewer_tmp_` staging dirs are NOT touched
|
||||
/// here: a superseded worker can still be streaming into those, and at build START it is far more
|
||||
/// likely to be alive than at finalize time.
|
||||
async fn prune_superseded_archives(
|
||||
pool: &PgPool,
|
||||
exports_dir: &Path,
|
||||
prefix: &str,
|
||||
event_id: Uuid,
|
||||
keep_seq: i64,
|
||||
) {
|
||||
let protected = protected_files(pool, event_id).await;
|
||||
let final_prefix = format!("{prefix}.{event_id}.");
|
||||
let mut rd = match tokio::fs::read_dir(exports_dir).await {
|
||||
Ok(rd) => rd,
|
||||
Err(_) => return,
|
||||
};
|
||||
let mut reclaimed = 0u64;
|
||||
while let Ok(Some(entry)) = rd.next_entry().await {
|
||||
let name = entry.file_name();
|
||||
let name = name.to_string_lossy();
|
||||
if !is_superseded_archive(&name, &final_prefix, keep_seq, &protected) {
|
||||
continue;
|
||||
}
|
||||
let len = entry.metadata().await.map(|m| m.len()).unwrap_or(0);
|
||||
if tokio::fs::remove_file(entry.path()).await.is_ok() {
|
||||
reclaimed += len;
|
||||
}
|
||||
}
|
||||
if reclaimed > 0 {
|
||||
tracing::info!(
|
||||
"reclaimed {reclaimed} bytes of superseded {prefix} archives before rebuilding \
|
||||
event {event_id} @ epoch {keep_seq}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Is `name` a FINAL archive of a strictly-older generation that no live row points at?
|
||||
///
|
||||
/// Pure so the two dangerous cases can be pinned without a filesystem: the carried-forward archive
|
||||
/// (protected despite an older epoch in its name) and the in-flight `.tmp` (never matched at all).
|
||||
fn is_superseded_archive(
|
||||
name: &str,
|
||||
final_prefix: &str,
|
||||
keep_seq: i64,
|
||||
protected: &[String],
|
||||
) -> bool {
|
||||
if protected.iter().any(|p| p == name) {
|
||||
return false;
|
||||
}
|
||||
parse_gen_seq(name, final_prefix, ".zip").is_some_and(|n| n < keep_seq)
|
||||
}
|
||||
|
||||
/// Bytes the media in this event's keepsake will occupy, as an UPPER BOUND per archive.
|
||||
///
|
||||
/// Both archives write their media entries `Compression::Stored`, so an archive is essentially a
|
||||
/// byte-for-byte second copy of the originals: `Gallery.zip` always, and `Memories.zip` for every
|
||||
/// video ([`MediaSource::Original`]) and every image at or under 5 MB. Images over 5 MB are
|
||||
/// downscaled to 2000px first, which only makes this estimate more conservative — the direction we
|
||||
/// want, since being wrong low means ENOSPC halfway through.
|
||||
///
|
||||
/// Matches [`query_uploads`]' visibility filter exactly, so hidden/banned uploads aren't counted.
|
||||
pub async fn estimate_export_bytes(pool: &PgPool, event_id: Uuid) -> Result<u64> {
|
||||
let (bytes,): (i64,) = sqlx::query_as(
|
||||
"SELECT COALESCE(SUM(u.original_size_bytes), 0)::bigint
|
||||
FROM upload u
|
||||
JOIN \"user\" usr ON usr.id = u.user_id
|
||||
WHERE u.event_id = $1 AND u.deleted_at IS NULL
|
||||
AND usr.uploads_hidden = FALSE AND usr.is_banned = FALSE",
|
||||
)
|
||||
.bind(event_id)
|
||||
.fetch_one(pool)
|
||||
.await
|
||||
.context("estimating the export size")?;
|
||||
Ok(bytes.max(0) as u64)
|
||||
}
|
||||
|
||||
/// Headroom multiplier over the raw media sum: ZIP central directory, per-entry headers, the
|
||||
/// embedded viewer, and the HTML export's temp staging.
|
||||
const EXPORT_SIZE_OVERHEAD_PCT: u64 = 110;
|
||||
|
||||
/// Bytes this export must have available, given the raw media sum and how many jobs are competing.
|
||||
///
|
||||
/// `armed` is the multiplier that keeps two concurrent workers honest. `spawn_export_jobs` starts
|
||||
/// the ZIP and the HTML halves at the same instant and both are gallery-sized, so a worker that
|
||||
/// reserved only for itself would see "it fits", its sibling would independently see the same, and
|
||||
/// together they would not fit — which is precisely the ENOSPC this preflight exists to prevent.
|
||||
/// Computed in `u128` and clamped, NOT with `saturating_mul`: saturating first and then dividing by
|
||||
/// 100 quietly turns an overflow into a number ~100x too small, which is the one direction that
|
||||
/// matters here — an under-estimate authorises the very write the preflight exists to refuse.
|
||||
fn required_free_bytes(media_bytes: u64, armed: i64) -> u64 {
|
||||
let needed = media_bytes as u128 * EXPORT_SIZE_OVERHEAD_PCT as u128 / 100
|
||||
* armed.max(1).min(i64::from(u32::MAX)) as u128;
|
||||
needed.min(u64::MAX as u128) as u64
|
||||
}
|
||||
|
||||
/// Free bytes a full keepsake build would need RIGHT NOW, both halves included.
|
||||
///
|
||||
/// The same arithmetic the preflight uses, exposed so the host dashboard can warn BEFORE the
|
||||
/// release rather than reporting a failure after it. The preflight can only ever say "this didn't
|
||||
/// fit"; at that point the gallery is full, the event is over, and the remedies (ask guests to stop
|
||||
/// uploading, grow the volume) are all much harder. Hard-codes both halves because that is what a
|
||||
/// release arms.
|
||||
pub async fn keepsake_space_required(pool: &PgPool, event_id: Uuid) -> Result<u64> {
|
||||
Ok(required_free_bytes(
|
||||
estimate_export_bytes(pool, event_id).await?,
|
||||
2,
|
||||
))
|
||||
}
|
||||
|
||||
/// Refuse to start an export that cannot fit, with a reason the host can act on.
|
||||
///
|
||||
/// Without this the failure mode is ENOSPC halfway through a multi-GB write, and the wreckage
|
||||
/// outlives it: the epoch has already moved, so the job row is `failed` at the CURRENT generation
|
||||
/// and `GET /export/zip` 404s, while the last good archive sits on disk unreferenced. The host's
|
||||
/// only escape (`POST /host/export/rebuild`) needs the very space that isn't there. Failing at the
|
||||
/// door instead leaves the disk untouched and puts a number in front of the operator.
|
||||
///
|
||||
/// Both halves are spawned concurrently and both are gallery-sized, so a worker must reserve for
|
||||
/// its live sibling too — otherwise each independently sees "it fits", and together they don't.
|
||||
async fn ensure_export_space(pool: &PgPool, event_id: Uuid, export_path: &Path) -> Result<()> {
|
||||
let media_bytes = estimate_export_bytes(pool, event_id).await?;
|
||||
|
||||
// Every job armed at any epoch for this event that hasn't finished is competing for this disk.
|
||||
let (armed,): (i64,) = sqlx::query_as(
|
||||
"SELECT COUNT(*) FROM export_job
|
||||
WHERE event_id = $1 AND status IN ('pending', 'running')",
|
||||
)
|
||||
.bind(event_id)
|
||||
.fetch_one(pool)
|
||||
.await
|
||||
.context("counting armed export jobs")?;
|
||||
let needed = required_free_bytes(media_bytes, armed);
|
||||
|
||||
// `None` = the mount couldn't be resolved. Fail OPEN, exactly as the upload quota does: refusing
|
||||
// to build the keepsake because we can't read a number would be a worse failure than trying.
|
||||
let Some(free) = crate::services::disk::free_bytes(export_path) else {
|
||||
tracing::warn!("export preflight: disk snapshot unavailable; proceeding without the check");
|
||||
return Ok(());
|
||||
};
|
||||
|
||||
if free < needed {
|
||||
let gb = |b: u64| b as f64 / 1_000_000_000.0;
|
||||
tracing::error!(
|
||||
needed,
|
||||
free,
|
||||
armed,
|
||||
"export preflight: not enough free space to build the keepsake for event {event_id}"
|
||||
);
|
||||
anyhow::bail!(
|
||||
"Nicht genug Speicherplatz für das Keepsake: benötigt ca. {:.1} GB, frei sind {:.1} GB. \
|
||||
Bitte Speicher freigeben und das Keepsake anschließend neu erstellen.",
|
||||
gb(needed),
|
||||
gb(free)
|
||||
);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Best-effort removal of stale per-generation export artifacts for one export type. Deletes
|
||||
/// ONLY strictly-older generations (`n < keep_seq`) — never `keep_seq`'s own current file,
|
||||
/// and never a NEWER generation that a concurrent re-release may already be producing (that
|
||||
@@ -1177,20 +1381,7 @@ async fn prune_stale_export_files(
|
||||
event_id: Uuid,
|
||||
keep_seq: i64,
|
||||
) {
|
||||
// Files still referenced by a live (current-epoch) job row are OFF LIMITS regardless of the
|
||||
// epoch in their name. A ViewerOnly regeneration carries the finished ZIP forward by re-stamping
|
||||
// its row to the new epoch WITHOUT renaming the file — so `Gallery.<event>.<older>.zip` is still
|
||||
// the served archive, and deleting it by filename-epoch would 404 the download.
|
||||
let protected: Vec<String> = sqlx::query_scalar::<_, String>(
|
||||
"SELECT split_part(j.file_path, '/', -1) FROM export_job j
|
||||
JOIN event e ON e.id = j.event_id
|
||||
WHERE j.event_id = $1 AND j.epoch = e.export_epoch
|
||||
AND j.status = 'done' AND j.file_path IS NOT NULL",
|
||||
)
|
||||
.bind(event_id)
|
||||
.fetch_all(pool)
|
||||
.await
|
||||
.unwrap_or_default();
|
||||
let protected = protected_files(pool, event_id).await;
|
||||
|
||||
// EVERY shape is event-scoped. All events share one exports volume, so a name keyed only by
|
||||
// generation would let event A's prune delete event B's live keepsake (and let two events
|
||||
@@ -1504,3 +1695,131 @@ So geht's:\n\
|
||||
Alles ist lokal auf deinem Gerät gespeichert.\n\
|
||||
\n\
|
||||
Viel Freude mit den Erinnerungen!\n";
|
||||
|
||||
// ── Tests ────────────────────────────────────────────────────────────────────
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
const EVT: &str = "11111111-1111-1111-1111-111111111111";
|
||||
|
||||
fn gallery_prefix() -> String {
|
||||
format!("Gallery.{EVT}.")
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_strictly_older_archive_is_reclaimed() {
|
||||
// The whole point: at the start of a rebuild at epoch 5, generation 4's archive is dead
|
||||
// weight — readiness is derived from `epoch = event.export_epoch`, so it is already
|
||||
// unreachable — and its bytes are very often exactly the bytes the rebuild needs.
|
||||
assert!(is_superseded_archive(
|
||||
&format!("Gallery.{EVT}.4.zip"),
|
||||
&gallery_prefix(),
|
||||
5,
|
||||
&[]
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn our_own_and_newer_generations_are_never_touched() {
|
||||
// `keep_seq` is OUR generation; a NEWER one belongs to a re-release that has already
|
||||
// superseded us, and deleting it would let a lagging worker nuke a live keepsake.
|
||||
for seq in [5, 6] {
|
||||
assert!(
|
||||
!is_superseded_archive(
|
||||
&format!("Gallery.{EVT}.{seq}.zip"),
|
||||
&gallery_prefix(),
|
||||
5,
|
||||
&[]
|
||||
),
|
||||
"generation {seq} must survive a prune keeping 5"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_carried_forward_archive_survives_despite_an_older_epoch_in_its_name() {
|
||||
// THE dangerous case. A ViewerOnly regeneration (a moderated comment) re-stamps the
|
||||
// finished ZIP's row to the new epoch WITHOUT renaming the file, so the SERVED archive
|
||||
// legitimately carries an older generation number. Pruning it by filename-epoch would 404
|
||||
// the photo download to change nothing in it. The protected set is what stops that, and
|
||||
// moving the prune to build-start makes this case reachable far more often.
|
||||
let carried = format!("Gallery.{EVT}.4.zip");
|
||||
assert!(!is_superseded_archive(
|
||||
&carried,
|
||||
&gallery_prefix(),
|
||||
5,
|
||||
std::slice::from_ref(&carried)
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn temps_and_staging_dirs_are_out_of_scope_for_the_early_prune() {
|
||||
// A superseded worker may still be streaming into these, and at build START it is much
|
||||
// more likely to be alive than at finalize time. Only inert FINAL archives are reclaimed
|
||||
// here; `prune_stale_export_files` still handles the rest after we win.
|
||||
for name in [
|
||||
format!("Gallery.{EVT}.4.tmp"),
|
||||
format!("viewer_tmp_{EVT}_4"),
|
||||
format!("Memories.{EVT}.4.zip"),
|
||||
] {
|
||||
assert!(
|
||||
!is_superseded_archive(&name, &gallery_prefix(), 5, &[]),
|
||||
"{name} must not be reclaimed by the pre-build prune"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn another_events_archive_is_never_reclaimed() {
|
||||
// All events share one exports volume, so the prefix carries the event id.
|
||||
let other = "22222222-2222-2222-2222-222222222222";
|
||||
assert!(!is_superseded_archive(
|
||||
&format!("Gallery.{other}.4.zip"),
|
||||
&gallery_prefix(),
|
||||
5,
|
||||
&[]
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unrelated_files_are_left_alone() {
|
||||
for name in ["Gallery.zip", "notes.txt", "Gallery..4.zip"] {
|
||||
assert!(!is_superseded_archive(name, &gallery_prefix(), 5, &[]));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_lone_armed_job_reserves_for_one_archive() {
|
||||
// A ViewerOnly regeneration re-arms only the HTML half — reserving for two would refuse
|
||||
// a rebuild that fits perfectly well.
|
||||
assert_eq!(required_free_bytes(1_000, 1), 1_100);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn two_concurrent_halves_reserve_for_both() {
|
||||
// The bug this exists to prevent: each worker independently sees "it fits", and together
|
||||
// they don't. Both halves are gallery-sized, so the reservation must be for the pair.
|
||||
assert_eq!(required_free_bytes(1_000, 2), 2_200);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_zero_count_still_reserves_for_one() {
|
||||
// Defensive: a racing status transition must never yield a zero requirement, which would
|
||||
// wave through an export of any size onto a full disk.
|
||||
assert_eq!(required_free_bytes(1_000, 0), 1_100);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_empty_gallery_needs_nothing() {
|
||||
assert_eq!(required_free_bytes(0, 2), 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_pathological_size_saturates_instead_of_wrapping() {
|
||||
// u64 overflow would wrap to a TINY requirement and authorise the exact write we're
|
||||
// guarding against — the failure mode must be "refuse", never "wrap and allow".
|
||||
assert_eq!(required_free_bytes(u64::MAX, 2), u64::MAX);
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user