fabi/EventSnap
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: close eight regressions the audit pass found, five of them mine

Browse Source 
Two adversarial reviews over 61119be, 1d9fb11 and eb0e405. The merge itself came
back clean — client_upload_id end to end, TempFileGuard's arm/retarget/disarm, the
supervised sweep wiring and v_feed's column parity were all verified sound. What
follows is what my own three commits broke.

BLOCKER — a post-release rebuild was permanently impossible, and it 404'd the keepsake

1d9fb11 deferred prune_superseded_archives to run only on success, so a failed
rebuild could no longer destroy the last good archive. It did not follow that
through: ensure_export_space runs BEFORE the prune, so at rebuild time the previous
generation is still on disk and counted against free. That halves the gallery a
rebuild can survive (~4.6 GB) relative to what the upload gate accepts (~7.8 GB) —
and it self-locks, because invalidate_and_arm bumps the epoch on COMMIT, which 404s
both download routes immediately, while the only code that could free the space now
runs only after a success that can never happen. A guest deleting their own photo is
enough to trigger it. Recovery needed `docker exec rm`.

Now two-phase: try to build while preserving the old generation; if that genuinely
does not fit, reclaim it and try once more. Strictly better than both the original
ordering and my change — the old archive is sacrificed only when it is the only way
to get a new one.

BLOCKER — the deferred prune could delete the last archive when a worker LOST the race

run_*_export_inner returned Ok(()) on the superseded/discard path, so `res.is_ok()`
fired the prune with the worker's own RETIRED epoch as keep_seq. At that moment the
winning generation is still `pending` with no file, so protected_files is empty and
the last good archive was deleted with no replacement. Exactly the invariant
deferring the prune was meant to establish. Returns Err(Superseded) now, which
abandon_if_superseded already swallows for the caller.

BLOCKER — the low-disk banner could never fire before the wall

eb0e405's gate refuses at `free < keepsake + DISK_RESERVE`, while disk_is_low warned
at `free < keepsake`. The two differ by the whole reserve, so the wall always came
first: every guest blocked from uploading while the host dashboard showed ~27 GB free
and no banner, with nobody on site. disk_is_low now shares the gate's expression plus
a 25% margin, and a test asserts the banner fires at the gate threshold across the
whole gallery-size range.

BLOCKER — I raised the unauthenticated bcrypt ceiling 24x on a 2 vCPU box

1d9fb11 moved admin_login's tight bucket after verify_password (correct — that is what
stops a guest locking the operator out) but replaced the incidental 5/min bound on
bcrypt with 120/min and nothing global. bcrypt is on spawn_blocking, but tokio's
blocking pool is 512 threads, so "off the runtime" is not "bounded": enough concurrent
verifies preempt both async workers and uploads, feed and SSE stall. Three
unauthenticated endpoints reach bcrypt and every guest shares one NAT IP, so per-IP
limits bound nothing globally. Adds a process-wide semaphore of `cores - 1` around both
verify and hash, and drops the ceiling to 30.

Also correcting my own claim: "a correct password is never throttled" was wrong. The
failure bucket cannot block it, but the CPU ceiling still can. The code comment said so;
the commit message did not.

BLOCKER — migration 025 could crash-loop the app on boot

Its UPDATE derives `Name (8hex)` with no guard against idx_user_event_name_ci. A guest
who had already joined as exactly that string makes the migration fail, which
propagates out of create_pool, exits main, and `restart: unless-stopped` turns it into
a permanent loop — a worse version of the lockout the migration exists to clean up.
Now skips colliding rows (create_admin_user already falls back to Admin-<8hex>, so the
cleanup is convenience, not load-bearing). Also `role = 'guest'` rather than
`<> 'admin'`, which was renaming legitimately promoted hosts named "Host".

DEGRADATION — the watchdog's suspension credit was unbounded

Background tabs are throttled to ~1 tick/min WITHOUT the network stack pausing, and the
tick gap cannot tell that from a freeze. Crediting every late tick grew the observed
silence by only one interval per real minute, so a dead socket took ~18 minutes to
detect while holding the queue's processing latch. Credit is now capped at one stall
window and REFILLS on real progress: an upload that is moving survives any number of
screen locks, while one that is silent and suspended is detected within ~3 minutes.

DEGRADATION — the 4xx log line was an unauthenticated log-injection vector

validate_display_name allowed newlines, several 4xx messages interpolate the name, and
%message wrote it unescaped. Two unauthenticated /join requests could forge arbitrary
lines in the only forensic record an unattended event has. Fixed at both ends: control
characters rejected at the door, and `detail = ?message` escapes on the way out (which
also stops colliding with tracing's reserved `message` field). 401/404 drop to DEBUG —
they carry no operator signal and were the cheapest lines for a scanner to use to roll
the 30 MB log window in minutes.

DEGRADATION — the quota floor was inverted exactly where it mattered

`computed.max(MIN.min(budget))`: `budget` is the whole disk's share, so below 500 MiB
the "floor" became the entire remaining budget and EVERY uploader was authorised all of
it — 400 MB free, 3 uploaders, 300 MB each. A test pinned that as correct under the name
`the_floor_never_exceeds_what_the_disk_can_back`. Both fixed.

Also replaces the headline gate test, which asserted its own precondition inside an `if`
on that precondition and could not fail. It now pins what actually binds the gate to the
preflight — that required_free_bytes charges for both halves — plus the ceiling band.

Verified: 151/151 backend tests against a live Postgres, clippy clean, 58/58 vitest,
svelte-check 0 errors, eslint clean, both builds, caddy validate, and the migration
collision reproduced against Postgres 16 before and after.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-08-08 22:33:43 +02:00
parent 23e2f485dd
commit 05063694d2
8 changed files with 350 additions and 59 deletions
Download Patch File Download Diff File Expand all files Collapse all files

57
backend/src/handlers/host.rs
View File

@@ -59,8 +59,20 @@ const LOW_DISK_FLOOR_BYTES: u64 = 10_000_000_000;
/// gallery-sized archives, and the only moment a host can do anything about that is BEFORE they
/// release. Warning at "you could not build the keepsake right now" turns a post-event dead end
/// into a decision someone can still make.
///
/// IT MUST FIRE BEFORE THE UPLOAD GATE CLOSES, and that is why the reserve and the margin are
/// here. The gate in `handlers::upload` refuses at `free < keepsake_required + DISK_RESERVE_BYTES`;
/// warning at `free < keepsake_required` alone meant the two differed by the whole reserve, so
/// the wall was always hit FIRST. Every guest would be blocked from uploading while this
/// dashboard showed a comfortable disk and no banner at all — on the shipped 40 GB box, uploads
/// stopping with ~27 GB free and nothing on screen to explain it, with no operator present.
///
/// The 25% margin makes it a warning rather than an obituary: the host sees it while there is
/// still room to act (delete a few large videos, which refunds immediately and reopens the gate).
fn disk_is_low(free: u64, keepsake_required: u64) -> bool {
free < LOW_DISK_FLOOR_BYTES || free < keepsake_required
let gate_closes_at = keepsake_required.saturating_add(crate::handlers::upload::DISK_RESERVE_BYTES as u64);
let warn_at = gate_closes_at.saturating_add(gate_closes_at / 4);
free < LOW_DISK_FLOOR_BYTES || free < warn_at
}
/// Count non-banned hosts/admins in the event OTHER than `excluding` — the operators
@@ -813,12 +825,15 @@ pub async fn release_gallery(
#[cfg(test)]
mod tests {
use super::{LOW_DISK_FLOOR_BYTES, disk_is_low};
use crate::handlers::upload::DISK_RESERVE_BYTES;
use crate::services::export::required_free_bytes;
const GB: u64 = 1_000_000_000;
#[test]
fn a_healthy_disk_with_room_for_the_keepsake_is_not_low() {
assert!(!disk_is_low(40 * GB, 25 * GB));
// Room for the keepsake AND the reserve the upload gate holds back, with margin.
assert!(!disk_is_low(60 * GB, 25 * GB));
}
#[test]
@@ -828,7 +843,29 @@ mod tests {
// the keepsake test comfortably; the floor is what catches this.
assert!(disk_is_low(5 * GB, GB));
assert!(disk_is_low(LOW_DISK_FLOOR_BYTES - 1, 0));
assert!(!disk_is_low(LOW_DISK_FLOOR_BYTES, 0));
// NOT `!disk_is_low(LOW_DISK_FLOOR_BYTES, 0)` any more. The floor is a lower bound, not
// the boundary: the gate-aware trigger now dominates it (at an empty gallery it warns
// below 1.25 x the reserve, i.e. 12.5 GB). Pinning equality here asserted the very
// behaviour that let the wall arrive before the warning.
assert!(disk_is_low(LOW_DISK_FLOOR_BYTES, 0));
assert!(!disk_is_low(20 * GB, 0), "a roomy empty disk is not low");
}
/// THE PROPERTY THIS EXISTS FOR: the host must be warned BEFORE guests are blocked.
///
/// `handlers::upload` refuses at `free < keepsake_required + DISK_RESERVE_BYTES`. If the
/// banner fires only at or below that, the host's first signal is 100 guests being unable
/// to upload while the dashboard shows a comfortable disk — with nobody on site to ask.
#[test]
fn the_banner_always_fires_before_the_upload_gate_closes() {
for media_gb in [0u64, 1, 4, 8, 16, 32] {
let required = required_free_bytes(media_gb * GB, 2);
let gate_closes_at = required + DISK_RESERVE_BYTES as u64;
assert!(
disk_is_low(gate_closes_at, required),
"at media={media_gb}GB the gate is about to close but no banner is shown"
);
}
}
#[test]
@@ -841,13 +878,21 @@ mod tests {
#[test]
fn the_keepsake_trigger_is_exact_at_the_boundary() {
assert!(!disk_is_low(66 * GB, 66 * GB), "exactly enough is enough");
assert!(disk_is_low(66 * GB - 1, 66 * GB));
// The boundary is the UPLOAD GATE's threshold plus a 25% margin, not the bare keepsake
// size — see `disk_is_low`. Warning at the bare size fired only after the gate had
// already blocked every guest.
let required = 20 * GB;
let gate = required + DISK_RESERVE_BYTES as u64;
let warn_at = gate + gate / 4;
assert!(!disk_is_low(warn_at, required), "exactly enough is enough");
assert!(disk_is_low(warn_at - 1, required));
}
#[test]
fn an_empty_gallery_needs_nothing_and_only_the_floor_applies() {
assert!(!disk_is_low(11 * GB, 0));
// With no gallery the keepsake term is 0, so the warn threshold collapses to
// 1.25 x DISK_RESERVE_BYTES (12.5 GB), which dominates the 10 GB absolute floor.
assert!(!disk_is_low(13 * GB, 0));
assert!(disk_is_low(9 * GB, 0));
}
}

106
backend/src/handlers/upload.rs
View File

@@ -1064,8 +1064,18 @@ fn quota_limit_bytes(free_disk: i64, tolerance: f64, active_uploaders: i64, expe
// The floor may never exceed what the disk can actually back. Raising a ceiling the volume
// cannot honour would hand out an allowance on a full disk — turning the quota from a
// usability floor into a way to finish filling the filesystem that Postgres writes WAL to.
let backed_floor = MIN_QUOTA_LIMIT_BYTES.min(budget as i64);
computed.max(backed_floor)
// Only raise to the floor when the disk can back a floor-sized allowance for real.
//
// The earlier form was `computed.max(MIN.min(budget))`, which inverts exactly where it
// matters: `budget` is the WHOLE disk's share, not one user's, so once budget < 500 MiB the
// "floor" became the entire remaining budget and every uploader was authorised all of it —
// 400 MB free with 3 uploaders promised 300 MB each. Below the floor, fall through to the
// divided value, which is the only number that still shares the space out.
if budget < MIN_QUOTA_LIMIT_BYTES as f64 {
computed
} else {
computed.max(MIN_QUOTA_LIMIT_BYTES)
}
}
/// Computes the per-user storage quota using
@@ -1560,50 +1570,60 @@ mod tests {
/// THE INVARIANT THE UPLOAD GATE EXISTS FOR: if an upload is accepted, the keepsake must
/// still be buildable afterwards.
///
/// The gate and `ensure_export_space` answer the same question at different times, from the
/// same `required_free_bytes`. If they ever drift, the failure is silent and terminal — every
/// upload succeeds and the archive can never be built, discovered only when the host taps
/// release and there is nobody left to fix it. This pins the two together.
/// An earlier version of this test computed the gate's threshold and the preflight's
/// threshold with the SAME expression and then asserted one against the other inside an
/// `if` on that expression — a tautology that could not fail and would not have noticed a
/// term being added to `ensure_export_space`. What actually binds the two together is that
/// both call `required_free_bytes`, so what is worth pinning is the SHAPE of that function
/// and the ceiling it produces on the real volume.
///
/// Models the real box: 40 GB volume, ~5 GB consumed by OS, images and Postgres.
/// Models the shipped box: 40 GB, ~5 GB consumed by OS, images and Postgres.
#[test]
fn an_accepted_upload_always_leaves_room_to_build_the_keepsake() {
fn the_gate_ceiling_keeps_both_keepsake_halves_and_the_reserve_affordable() {
const USABLE: i64 = 35 * GB;
let reserve = DISK_RESERVE_BYTES;
// Walk the gallery upward in 250 MB steps and assert the two agree at every point.
let mut media: i64 = 0;
let step: i64 = 250 * 1024 * 1024;
let mut last_accepted = 0i64;
// Walk the gallery upward and find the last size the gate would accept.
let mut ceiling = 0i64;
let step = 250 * 1024 * 1024;
let mut media = 0i64;
while media < USABLE {
// `free` already excludes the bytes just streamed to the temp file, matching the
// handler: the gate compares live free space against what the keepsake will need.
let media_after = media + step;
let free = USABLE - media_after;
let required =
crate::services::export::required_free_bytes(media_after as u64, 2) as i64 + reserve;
let gate_accepts = free >= required;
if gate_accepts {
// The export preflight must agree, using the SAME arithmetic it will run later.
let preflight_needs =
crate::services::export::required_free_bytes(media_after as u64, 2) as i64
+ reserve;
assert!(
free >= preflight_needs,
"gate accepted at media={media_after} but the preflight would refuse"
);
last_accepted = media_after;
media += step;
let free = USABLE - media;
if free >= crate::services::export::required_free_bytes(media as u64, 2) as i64 + reserve
{
ceiling = media;
}
media = media_after;
}
// Sanity-check the ceiling is where the arithmetic says: 35 = 2.2·M + 10 ⇒ M ≈ 7.8 GB.
// Pinned loosely (6–9 GB) so a deliberate change to the overhead multiplier or the
// reserve fails this test loudly rather than silently moving the cliff.
// At the ceiling, BOTH archives and the reserve must genuinely fit in what is left.
let free_at_ceiling = USABLE - ceiling;
let both_halves = crate::services::export::required_free_bytes(ceiling as u64, 2) as i64;
assert!(
(6 * GB..=9 * GB).contains(&last_accepted),
"expected the gallery ceiling near 7.8 GB on a 35 GB volume, got {last_accepted} bytes"
free_at_ceiling >= both_halves + reserve,
"at the ceiling the keepsake ({both_halves}) + reserve ({reserve}) must fit in \
{free_at_ceiling}"
);
// And one byte more must NOT fit — i.e. the ceiling is where the gate actually closes,
// not somewhere short of it.
let over = ceiling + step;
let free_over = USABLE - over;
assert!(
free_over < crate::services::export::required_free_bytes(over as u64, 2) as i64 + reserve,
"the gate should already be closed one step past the ceiling"
);
// Independently: `required_free_bytes` must charge for TWO gallery-sized archives.
// If someone changes `armed` or the overhead, this is the line that notices.
let one = crate::services::export::required_free_bytes(ceiling as u64, 1) as i64;
assert_eq!(both_halves, one * 2, "a release arms both halves");
// Pinned loosely so a deliberate change to the overhead or the reserve fails loudly
// rather than silently moving the cliff.
assert!(
(6 * GB..=9 * GB).contains(&ceiling),
"expected a gallery ceiling near 7.8 GiB on a 35 GB volume, got {ceiling} bytes"
);
}
@@ -1621,13 +1641,23 @@ mod tests {
/// The floor must never write a cheque the volume cannot cash — otherwise a full disk
/// still hands out a 500 MB allowance and the filesystem Postgres needs fills up.
/// The floor must never write a cheque the volume cannot cash, and — the subtler half —
/// must never hand EVERY uploader the whole remaining budget.
#[test]
fn the_floor_never_exceeds_what_the_disk_can_back() {
assert_eq!(quota_limit_bytes(0, 0.75, 3, 1), 0, "no disk, no allowance");
// 400 MB free * 0.75 = 300 MB — below the floor, so the disk wins.
// 400 MB free x 0.75 = 300 MB of budget, below the 500 MiB floor. The floor must NOT
// apply: with 3 uploaders the answer is the divided share, not the whole budget.
let tight = quota_limit_bytes(400 * 1024 * 1024, 0.75, 3, 1);
assert_eq!(tight, 300 * 1024 * 1024);
assert_eq!(tight, 100 * 1024 * 1024, "a scarce budget is still divided");
assert!(tight < MIN_QUOTA_LIMIT_BYTES);
// The promise across all uploaders must stay inside the budget.
assert!(
tight * 3 <= (400.0 * 1024.0 * 1024.0 * 0.75) as i64,
"the sum of per-user allowances must not exceed the disk's share"
);
}
#[test]