perf(backend): close config/upload/export hot paths + stability fixes

Performance:
- Cache the runtime `config` table in-memory (ConfigCache) with synchronous
  invalidation on every write (admin PATCH + test reseed). Was re-reading each
  key from Postgres on every request (~8 round-trips per upload).
- Stream uploads chunk-by-chunk to a temp file instead of buffering the whole
  body in RAM (peak was up to the per-class cap, e.g. 500 MB/video); only 512
  sniff-bytes are kept for magic-byte detection, then atomic rename into place.
- Cache the media-filesystem disk snapshot (DiskCache, 15s TTL) shared by the
  quota check and admin stats; drop the discarded System::refresh_all().
- HTML export streams video (and small-image) originals straight into the ZIP
  via a manifest instead of copying them to a temp dir first (removed the
  transient 2x disk usage) and drops the double directory scan.
- Auth extractor resolves session -> live user in one JOIN (was two queries),
  touching last_seen_at by token hash.

Stability:
- SSE: on broadcast lag, emit a `resync` event so the client runs a delta
  fetch instead of silently losing events; frontend reconciles adds, deletions,
  and (via an in-place refresh) like/comment counts on visible cards.
- Storage quota fails OPEN when the disk can't be read (was a 0-byte limit that
  locked out all uploads).
- Graceful shutdown drains in-flight requests on SIGTERM/SIGINT, bounded by a
  10s backstop so open SSE streams can't stall a deploy.
- Upload removes the persisted file if the DB transaction fails (no orphaned
  bytes with no row to reclaim them).

Tests:
- New pure select_disk() with 5 unit tests (longest-prefix, fallbacks, fail-open).
- New e2e export-video spec covering the HTML export's video-streaming branch.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
fabi
2026-07-07 20:26:45 +02:00
parent 4cdb3ae14a
commit d6c91974eb
18 changed files with 745 additions and 214 deletions

View File

@@ -3,6 +3,8 @@ use tokio::sync::broadcast;
use crate::config::AppConfig;
use crate::services::compression::CompressionWorker;
use crate::services::config::ConfigCache;
use crate::services::disk::DiskCache;
use crate::services::rate_limiter::RateLimiter;
use crate::services::sse_tickets::SseTicketStore;
@@ -31,6 +33,11 @@ pub struct AppState {
pub compression: CompressionWorker,
pub rate_limiter: RateLimiter,
pub sse_tickets: SseTicketStore,
/// In-memory cache in front of the `config` table. Reads go through here; the
/// admin PATCH handler and the test reseed invalidate it after committing.
pub config_cache: ConfigCache,
/// Cached total/free bytes for the media filesystem (quota + admin stats).
pub disk_cache: DiskCache,
}
impl AppState {
@@ -42,6 +49,7 @@ impl AppState {
config.compression_concurrency,
sse_tx.clone(),
);
let config_cache = ConfigCache::new(pool.clone());
Self {
pool,
config,
@@ -49,6 +57,8 @@ impl AppState {
compression,
rate_limiter: RateLimiter::new(),
sse_tickets: SseTicketStore::new(),
config_cache,
disk_cache: DiskCache::new(),
}
}
}