Merge branch 'fix/deploy-unattended-blockers' into main

Two independent lines of production hardening diverged at 7d0334b and attacked
overlapping problems. Neither was a superset, so this is a merge of substance
rather than a fast-forward: every conflict was resolved on the merits, and the
losing side's intent was re-checked against the winner rather than assumed.

MIGRATIONS. The branch's 021/022/023 collided with main's already-DEPLOYED
021_hashtag_counts_respect_bans and 022_client_upload_idempotency. Renumbered to
023/024/025 in a prior commit — main's versions are applied in production, so
their version numbers are immutable and the branch's had to move. Verified by
running the full sqlx::test suite, which applies the whole chain from scratch.

RESOLVED IN MAIN'S FAVOUR (the branch would have regressed these):
  * upload-queue.ts wholesale — the branch's copy has ZERO client_upload_id
    references, so taking it would have silently destroyed end-to-end upload
    idempotency, the one thing standing between a lost response and a duplicate
    photo charged twice against the guest's quota.
  * maintenance.rs supervisor — the branch replaced it with a bare tokio::spawn,
    where one panic silently stops session pruning, media reclaim, the temp
    sweep and both HashMap prunes, permanently and with no log line.
  * The decode-budget probe on spawn_blocking, not inline on the async runtime.
  * feed/+page.svelte's 8s debounce + jitter + max-wait + hidden-tab deferral,
    against the branch's naive 800ms — at 100 guests the branch's version walks
    straight into the per-user feed rate limit.
  * db.rs pool tuning, /uploaders, and the docker-compose deployment story.
  * ONE /health, still DB-backed. The branch's split (dependency-free liveness +
    DB-backed readiness) is defensible, but a constant-"ok" /health is the exact
    defect faea555 fixed and verified live, its motive (Caddy's boot gate) is
    already covered by app depends_on db: service_healthy, and the two handlers
    were the same SELECT 1 under two names.

TAKEN FROM THE BRANCH:
  * The large-PNG OOM guard and its bounded-retry counter (023). Together these
    turn a single upload that can OOM-kill a 1G container into a bounded failure
    instead of an infinite restart loop under `restart: unless-stopped`.
  * 024_feed_scalar_counts — the feed no longer aggregates the whole event per
    page. Pure SQL; column names, order and types are unchanged by design.
  * The admin-lockout fix: look the admin up BY ROLE, never by name. 025 also
    frees any guest already squatting on a reserved name.
  * PIN lockout tier ordering, bounded caption/hashtag reads, SSE ticket caps,
    PoolTimedOut -> 503 + Retry-After, and the ffmpeg stderr drain.
  * backfill_video_posters, which main lacked entirely.
  * TempFileGuard, plus sweep_orphan_originals wired into main's SUPERVISED loop
    (not the branch's bare one) — it reclaims final-named originals whose commit
    never happened, a class main's .tmp-only sweep structurally cannot see.
  * shouldAbortForStall, hand-ported into main's upload-queue.ts since that file
    was resolved to main. Widens the watchdog at loadend instead of disarming it,
    bounding a half-open socket at 2 minutes rather than handing the window to
    xhr.timeout (5-60 min) with the whole queue's `processing` latch held.

ALSO: RUST_LOG and EXPORT_PATH pinned in compose. The code fallback was
`debug` (a line per request, all night) and EXPORT_PATH was the one path with a
mount-shaped default that nothing validated.

Verified: cargo check --all-targets, cargo clippy (clean), 144/144 backend tests
against a live Postgres including upload_idempotency and upload_concurrency,
51/51 vitest, svelte-check 0 errors, eslint clean, vite build.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-08-08 21:16:31 +02:00
30 changed files with 2192 additions and 439 deletions

View File

@@ -65,6 +65,35 @@ const MAX_RETRY_DELAY_MS = 5 * 60_000;
const STALL_TIMEOUT_MS = 90_000;
const STALL_CHECK_INTERVAL_MS = 5_000;
/**
* Ceiling for the window AFTER the last byte is sent.
*
* `upload.progress` is silent there BY DEFINITION — the server is sniffing the magic bytes,
* committing the row and writing the response — so the bytes-moved signal above has nothing
* to measure and the watchdog used to simply switch off at `loadend`. That left the wall-clock
* `xhr.timeout` as the only remaining bound: 5 minutes for a photo, up to 60 for a video. A
* half-open socket in that window (phone roams wifi→LTE while the server is still writing a
* 500 MB video) pinned `processing` for that entire time, so NOTHING else in the guest's queue
* drained either.
*
* Sized above the backend's worst-case commit path, not above compression — compression is
* spawned after the response and does not hold it open.
*/
const RESPONSE_TIMEOUT_MS = 120_000;
/**
* Pure predicate behind the watchdog, extracted so the policy is unit-testable without
* standing up an XHR harness.
*/
export function shouldAbortForStall(
lastActivityAt: number,
now: number,
bodySent: boolean
): boolean {
const ceiling = bodySent ? RESPONSE_TIMEOUT_MS : STALL_TIMEOUT_MS;
return now - lastActivityAt > ceiling;
}
/**
* Wall-clock cap for one attempt, scaled by file size assuming a floor of ~8 kB/s — a
* deliberately pessimistic rate, because killing a slow-but-progressing upload would lose
@@ -865,9 +894,10 @@ async function uploadItem(id: string): Promise<void> {
// connection that never errors and never completes. Only "no bytes moved" catches
// that without also punishing a healthy slow link.
let lastProgressAt = Date.now();
let bodySent = false;
let stalled = false;
const stallTimer = setInterval(() => {
if (Date.now() - lastProgressAt < STALL_TIMEOUT_MS) return;
if (!shouldAbortForStall(lastProgressAt, Date.now(), bodySent)) return;
stalled = true;
xhr.abort();
}, STALL_CHECK_INTERVAL_MS);
@@ -889,9 +919,19 @@ async function uploadItem(id: string): Promise<void> {
});
// Once the last byte is out the watchdog has nothing left to measure: the server may
// legitimately sit on the request while it validates and stores the file, and no
// progress event fires in that window. Aborting there would re-send a whole video
// the server had already accepted, so hand over to `xhr.timeout` instead.
xhr.upload.addEventListener('loadend', () => clearInterval(stallTimer));
// progress event fires in that window. Aborting on the 90s no-progress ceiling there
// would re-send a whole video the server had already accepted.
//
// But switching the watchdog OFF here (which is what this used to do) handed the
// window to `xhr.timeout` alone — 5 to 60 minutes, during which a half-open socket
// holds `processing` and the guest's whole queue stops draining. So instead of
// disarming, widen: `shouldAbortForStall` switches to RESPONSE_TIMEOUT_MS once
// `bodySent` flips, which bounds the wedge at 2 minutes without ever firing on a
// server that is legitimately still working.
xhr.upload.addEventListener('loadend', () => {
bodySent = true;
lastProgressAt = Date.now();
});
xhr.addEventListener('load', () => {
const body = (() => {