fix: close eight regressions the audit pass found, five of them mine
Two adversarial reviews over61119be,1d9fb11andeb0e405. The merge itself came back clean — client_upload_id end to end, TempFileGuard's arm/retarget/disarm, the supervised sweep wiring and v_feed's column parity were all verified sound. What follows is what my own three commits broke. BLOCKER — a post-release rebuild was permanently impossible, and it 404'd the keepsake1d9fb11deferred prune_superseded_archives to run only on success, so a failed rebuild could no longer destroy the last good archive. It did not follow that through: ensure_export_space runs BEFORE the prune, so at rebuild time the previous generation is still on disk and counted against free. That halves the gallery a rebuild can survive (~4.6 GB) relative to what the upload gate accepts (~7.8 GB) — and it self-locks, because invalidate_and_arm bumps the epoch on COMMIT, which 404s both download routes immediately, while the only code that could free the space now runs only after a success that can never happen. A guest deleting their own photo is enough to trigger it. Recovery needed `docker exec rm`. Now two-phase: try to build while preserving the old generation; if that genuinely does not fit, reclaim it and try once more. Strictly better than both the original ordering and my change — the old archive is sacrificed only when it is the only way to get a new one. BLOCKER — the deferred prune could delete the last archive when a worker LOST the race run_*_export_inner returned Ok(()) on the superseded/discard path, so `res.is_ok()` fired the prune with the worker's own RETIRED epoch as keep_seq. At that moment the winning generation is still `pending` with no file, so protected_files is empty and the last good archive was deleted with no replacement. Exactly the invariant deferring the prune was meant to establish. Returns Err(Superseded) now, which abandon_if_superseded already swallows for the caller. BLOCKER — the low-disk banner could never fire before the wall eb0e405's gate refuses at `free < keepsake + DISK_RESERVE`, while disk_is_low warned at `free < keepsake`. The two differ by the whole reserve, so the wall always came first: every guest blocked from uploading while the host dashboard showed ~27 GB free and no banner, with nobody on site. disk_is_low now shares the gate's expression plus a 25% margin, and a test asserts the banner fires at the gate threshold across the whole gallery-size range. BLOCKER — I raised the unauthenticated bcrypt ceiling 24x on a 2 vCPU box1d9fb11moved admin_login's tight bucket after verify_password (correct — that is what stops a guest locking the operator out) but replaced the incidental 5/min bound on bcrypt with 120/min and nothing global. bcrypt is on spawn_blocking, but tokio's blocking pool is 512 threads, so "off the runtime" is not "bounded": enough concurrent verifies preempt both async workers and uploads, feed and SSE stall. Three unauthenticated endpoints reach bcrypt and every guest shares one NAT IP, so per-IP limits bound nothing globally. Adds a process-wide semaphore of `cores - 1` around both verify and hash, and drops the ceiling to 30. Also correcting my own claim: "a correct password is never throttled" was wrong. The failure bucket cannot block it, but the CPU ceiling still can. The code comment said so; the commit message did not. BLOCKER — migration 025 could crash-loop the app on boot Its UPDATE derives `Name (8hex)` with no guard against idx_user_event_name_ci. A guest who had already joined as exactly that string makes the migration fail, which propagates out of create_pool, exits main, and `restart: unless-stopped` turns it into a permanent loop — a worse version of the lockout the migration exists to clean up. Now skips colliding rows (create_admin_user already falls back to Admin-<8hex>, so the cleanup is convenience, not load-bearing). Also `role = 'guest'` rather than `<> 'admin'`, which was renaming legitimately promoted hosts named "Host". DEGRADATION — the watchdog's suspension credit was unbounded Background tabs are throttled to ~1 tick/min WITHOUT the network stack pausing, and the tick gap cannot tell that from a freeze. Crediting every late tick grew the observed silence by only one interval per real minute, so a dead socket took ~18 minutes to detect while holding the queue's processing latch. Credit is now capped at one stall window and REFILLS on real progress: an upload that is moving survives any number of screen locks, while one that is silent and suspended is detected within ~3 minutes. DEGRADATION — the 4xx log line was an unauthenticated log-injection vector validate_display_name allowed newlines, several 4xx messages interpolate the name, and %message wrote it unescaped. Two unauthenticated /join requests could forge arbitrary lines in the only forensic record an unattended event has. Fixed at both ends: control characters rejected at the door, and `detail = ?message` escapes on the way out (which also stops colliding with tracing's reserved `message` field). 401/404 drop to DEBUG — they carry no operator signal and were the cheapest lines for a scanner to use to roll the 30 MB log window in minutes. DEGRADATION — the quota floor was inverted exactly where it mattered `computed.max(MIN.min(budget))`: `budget` is the whole disk's share, so below 500 MiB the "floor" became the entire remaining budget and EVERY uploader was authorised all of it — 400 MB free, 3 uploaders, 300 MB each. A test pinned that as correct under the name `the_floor_never_exceeds_what_the_disk_can_back`. Both fixed. Also replaces the headline gate test, which asserted its own precondition inside an `if` on that precondition and could not fail. It now pins what actually binds the gate to the preflight — that required_free_bytes charges for both halves — plus the ceiling band. Verified: 151/151 backend tests against a live Postgres, clippy clean, 58/58 vitest, svelte-check 0 errors, eslint clean, both builds, caddy validate, and the migration collision reproduced against Postgres 16 before and after. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -166,6 +166,34 @@ describe('suspendedSinceLastTick', () => {
|
||||
expect(suspendedSinceLastTick(now - 6_900, now, 5_000)).toBe(0);
|
||||
});
|
||||
|
||||
/**
|
||||
* The bound that matters: however the page is throttled or frozen, a genuinely dead socket
|
||||
* must be detected in a time a guest would tolerate — not left to `xhr.timeout` (5-60 min).
|
||||
*/
|
||||
it('detects a dead socket in bounded time even under 1-tick-per-minute throttling', () => {
|
||||
// Background tabs are throttled to ~1 tick/min WITHOUT the network stack pausing, so a
|
||||
// socket can be dead while ticks keep arriving. Unbounded crediting made this take ~18
|
||||
// minutes, holding the queue's latch the whole time.
|
||||
const CAP = 90_000; // MAX_SUSPEND_CREDIT_MS
|
||||
let lastProgressAt = 0;
|
||||
let lastTickAt = 0;
|
||||
let creditSpent = 0;
|
||||
let clock = 0;
|
||||
let ticks = 0;
|
||||
while (!shouldAbortForStall(lastProgressAt, clock, false) && ticks < 100) {
|
||||
clock += 60_000; // throttled tick
|
||||
const credit = Math.min(
|
||||
suspendedSinceLastTick(lastTickAt, clock, 5_000),
|
||||
Math.max(0, CAP - creditSpent)
|
||||
);
|
||||
creditSpent += credit;
|
||||
lastProgressAt = Math.min(clock, lastProgressAt + credit);
|
||||
lastTickAt = clock;
|
||||
ticks += 1;
|
||||
}
|
||||
expect(clock).toBeLessThanOrEqual(240_000);
|
||||
});
|
||||
|
||||
it('credits the whole frozen window when the interval did not run', () => {
|
||||
// Screen locked ~2 minutes: a 5s interval arriving 130s late.
|
||||
expect(suspendedSinceLastTick(now - 130_000, now, 5_000)).toBe(125_000);
|
||||
|
||||
@@ -100,6 +100,24 @@ export function shouldAbortForStall(
|
||||
*/
|
||||
const SUSPEND_TOLERANCE_MS = 2_000;
|
||||
|
||||
/**
|
||||
* Total suspension credit a single silent stretch may ever be granted.
|
||||
*
|
||||
* A tick gap cannot distinguish "the page was frozen" from "the page is being throttled to one
|
||||
* tick per minute while the network stack keeps running" — and in the second case the socket can
|
||||
* be genuinely dead. Crediting every late tick meant the observed silence grew by only one
|
||||
* interval per real minute, so a dead upload took ~18 minutes to detect while holding the
|
||||
* queue's `processing` latch and blocking every other photo the guest had queued.
|
||||
*
|
||||
* Capping the total at one stall window bounds detection at roughly 2x the ceiling (~3 min)
|
||||
* whatever the throttling pattern, while still absorbing the pocket-length freezes this exists
|
||||
* for. Crucially the budget RESETS on real progress (see the `progress` handler), so a long
|
||||
* upload that is actually moving survives any number of screen locks — only an upload that is
|
||||
* silent AND suspended spends it, and that is exactly the case where being wrong is cheap: one
|
||||
* re-send, bounded by MAX_AUTO_ATTEMPTS.
|
||||
*/
|
||||
const MAX_SUSPEND_CREDIT_MS = STALL_TIMEOUT_MS;
|
||||
|
||||
/**
|
||||
* Wall-clock the watchdog interval FAILED to cover because the page was suspended.
|
||||
*
|
||||
@@ -945,6 +963,7 @@ async function uploadItem(id: string): Promise<void> {
|
||||
// that without also punishing a healthy slow link.
|
||||
let lastProgressAt = Date.now();
|
||||
let lastTickAt = Date.now();
|
||||
let creditSpent = 0;
|
||||
let bodySent = false;
|
||||
let stalled = false;
|
||||
const stallTimer = setInterval(() => {
|
||||
@@ -958,7 +977,15 @@ async function uploadItem(id: string): Promise<void> {
|
||||
// a period in which it could not observe anything is not evidence of silence.
|
||||
// Clamped to `now` so a progress event delivered right at resume cannot push the
|
||||
// timestamp into the future.
|
||||
lastProgressAt = Math.min(now, lastProgressAt + suspendedSinceLastTick(lastTickAt, now));
|
||||
// Bounded: see MAX_SUSPEND_CREDIT_MS. Without the cap, sustained background
|
||||
// throttling credited back all but one interval on every tick, so a dead socket
|
||||
// took ~18 minutes to notice while the queue's latch stayed held.
|
||||
const credit = Math.min(
|
||||
suspendedSinceLastTick(lastTickAt, now),
|
||||
Math.max(0, MAX_SUSPEND_CREDIT_MS - creditSpent)
|
||||
);
|
||||
creditSpent += credit;
|
||||
lastProgressAt = Math.min(now, lastProgressAt + credit);
|
||||
lastTickAt = now;
|
||||
if (!shouldAbortForStall(lastProgressAt, now, bodySent)) return;
|
||||
stalled = true;
|
||||
@@ -974,6 +1001,8 @@ async function uploadItem(id: string): Promise<void> {
|
||||
|
||||
xhr.upload.addEventListener('progress', (e) => {
|
||||
lastProgressAt = Date.now();
|
||||
// Bytes moved: this stretch of silence is over, so the suspension budget refills.
|
||||
creditSpent = 0;
|
||||
if (e.lengthComputable) {
|
||||
const pct = Math.round((e.loaded / e.total) * 100);
|
||||
queueItems.update((items) =>
|
||||
|
||||
Reference in New Issue
Block a user