perf(backend): close config/upload/export hot paths + stability fixes
Performance: - Cache the runtime `config` table in-memory (ConfigCache) with synchronous invalidation on every write (admin PATCH + test reseed). Was re-reading each key from Postgres on every request (~8 round-trips per upload). - Stream uploads chunk-by-chunk to a temp file instead of buffering the whole body in RAM (peak was up to the per-class cap, e.g. 500 MB/video); only 512 sniff-bytes are kept for magic-byte detection, then atomic rename into place. - Cache the media-filesystem disk snapshot (DiskCache, 15s TTL) shared by the quota check and admin stats; drop the discarded System::refresh_all(). - HTML export streams video (and small-image) originals straight into the ZIP via a manifest instead of copying them to a temp dir first (removed the transient 2x disk usage) and drops the double directory scan. - Auth extractor resolves session -> live user in one JOIN (was two queries), touching last_seen_at by token hash. Stability: - SSE: on broadcast lag, emit a `resync` event so the client runs a delta fetch instead of silently losing events; frontend reconciles adds, deletions, and (via an in-place refresh) like/comment counts on visible cards. - Storage quota fails OPEN when the disk can't be read (was a 0-byte limit that locked out all uploads). - Graceful shutdown drains in-flight requests on SIGTERM/SIGINT, bounded by a 10s backstop so open SSE streams can't stall a deploy. - Upload removes the persisted file if the DB transaction fails (no orphaned bytes with no row to reclaim them). Tests: - New pure select_disk() with 5 unit tests (longest-prefix, fallbacks, fail-open). - New e2e export-video spec covering the HTML export's video-streaming branch. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -109,6 +109,17 @@ export function connectSse(): void {
|
||||
);
|
||||
}
|
||||
|
||||
// `resync` is emitted by the server when our broadcast subscription fell
|
||||
// behind and events were dropped. Rather than let those losses leave the feed
|
||||
// stale, fetch the gap since the last event we actually saw and fan it out as
|
||||
// a feed-delta (which reconciles new uploads AND deletions). Handled with its
|
||||
// own listener — not via `dispatch` — so reading `lastEventTime` as the gap
|
||||
// start isn't clobbered by dispatch bumping it to "now".
|
||||
eventSource.addEventListener('resync', () => {
|
||||
const since = lastEventTime;
|
||||
if (since) void deltaFetchAndFan(since);
|
||||
});
|
||||
|
||||
eventSource.onerror = () => {
|
||||
// EventSource auto-reconnects but the connection state can stay broken; close
|
||||
// and try again ourselves with exponential backoff capped at 60s. Prevents
|
||||
|
||||
@@ -249,6 +249,11 @@
|
||||
const dead = new Set(delta.deleted_ids);
|
||||
uploads = uploads.filter((u) => !dead.has(u.id));
|
||||
}
|
||||
// A delta reconciles new uploads and deletions, but not like/comment
|
||||
// counts that changed on already-visible cards while we were
|
||||
// disconnected or lagged (a `resync`). Debounced page-1 refresh merges
|
||||
// those fresh counts in place without disturbing scroll.
|
||||
scheduleInPlaceRefresh();
|
||||
} catch { /* ignore */ }
|
||||
})
|
||||
);
|
||||
|
||||
Reference in New Issue
Block a user