Files
Mangalord/backend/src/storage/local.rs
MechaCat02 d6ac648ac9 feat(admin): crawler observability dashboard + reliability hardening (0.55.0)
Admin-only crawler dashboard backed by an SSE live-status stream,
coordinated browser restart, runtime PHPSESSID refresh, dead-letter
requeue, and a batch of reliability fixes. Closes everything from
the two-pass audit (10 commits' worth) and bumps 0.52.0 -> 0.55.0.

Backend:

- New /admin/crawler/* surface (cookie-auth, RequireAdmin) split
  into status / control / dead_jobs / backlog modules. SSE stream
  composes in-memory status with DB-derived queue counts, memoizes
  the counts for 1s and debounces watch pokes for 250ms (~10x QPS
  reduction per subscriber). One-shot GET /admin/crawler shares the
  same compose path.
- POST /admin/crawler/run gated by manual_pass_lock try_lock_owned
  (409 Conflict on overlapping click); browser restart goes through
  the coordinated_restart gate (drain + relaunch + auto-clear of the
  sticky session_expired flag on Ok).
- Runtime PHPSESSID refresh via SessionController (allow-list
  validation, never logged, audit row carries SHA-256 fingerprint).
  Storage layer is repo::crawler::runtime_session_{load,persist}.
- Dead-letter requeue with four scopes (all/manga/chapter/job);
  scope=all requires confirm:true; DISTINCT ON dedup keeps the
  partial unique index from rejecting requeues for chapters with
  multiple dead rows. SQL is four &'static str constants per scope.
- StatusHandle + ChapterGuard / CoverGuard RAII model survives
  panics; last-writer-wins on cover so concurrent dispatches don't
  clobber each other's slot. Pure functions (should_stop /
  should_mark_clean_exit / should_abort_pass) with named regression
  tests.
- Reliability bundle: per-lease heartbeat, jitter on retries,
  per-job timeout, circuit breaker on consecutive failures, BrowserManager
  coordinated restart gate, request fingerprint changes.
- Streaming page download: Storage::put_stream trait method,
  LocalStorage impl atomic via temp + fsync + UUID-suffixed rename.
  Pages stream through with peak memory ~one HTTP chunk + 64-byte
  sniff prefix instead of one full image per dispatch.
- New partial indexes (migration 0022): mangas_missing_cover_idx
  and crawler_jobs_dead_idx, both ordered by updated_at DESC to
  match the dashboard's LIMIT/OFFSET reads.
- Security hardening: admin_csrf_guard (Origin/Referer allowlist
  on /admin/* mutations, opt-in via ADMIN_ALLOWED_ORIGINS),
  admin_no_store_guard (Cache-Control: no-store on admin
  responses), audit rows carry per-scope target_id.

Frontend:

- /admin/crawler page decomposed into lib/components/crawler/
  (11 components: ProgressBar, SearchBar, CrawlerHero,
  CrawlerControls, ActiveChaptersCard, ActiveJobsTable,
  MissingCoversTable, DeadJobsTable, RestartConfirmModal,
  RequeueAllConfirmModal, SessionModal). Page is 532 LOC of
  orchestration; each component 22-148 LOC.
- EventSource lifecycle wired to visibilitychange / pagehide /
  pageshow (BFCache); after 5 consecutive errors probes the status
  endpoint so a 401 routes through the global on401Hook instead of
  infinite silent reconnects.
- Backlog $effect refetches debounced 500ms with per-loader
  AbortControllers; refresh after a control action only runs when
  the SSE stream is dead.
- Inline requeue button on /admin/mangas patches the affected row's
  sync_state locally (no full chapter-list refetch); proper
  aria-label. Requeue-all gets its own confirm modal; both confirm
  modals autofocus Cancel.
- SvelteKit reverse proxy bypasses its 5-minute AbortController
  for Accept: text/event-stream; pure shouldBypassProxyTimeout
  helper covered by unit tests.

Config / docs:

- New env vars (.env.example): ADMIN_ALLOWED_ORIGINS,
  CRAWLER_JOB_TIMEOUT_SECS, CRAWLER_METADATA_MAX_CONSECUTIVE_FAILURES,
  CRAWLER_BROWSER_RESTART_THRESHOLD.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 18:49:56 +02:00

260 lines
9.3 KiB
Rust

use std::path::{Path, PathBuf};
use async_trait::async_trait;
use futures_util::StreamExt as _;
use tokio::fs;
use tokio::io::AsyncWriteExt as _;
use tokio_util::io::ReaderStream;
use super::{PutByteStream, Storage, StorageError, StreamingFile};
pub struct LocalStorage {
root: PathBuf,
}
impl LocalStorage {
pub fn new(root: impl Into<PathBuf>) -> Self {
Self { root: root.into() }
}
fn resolve(&self, key: &str) -> Result<PathBuf, StorageError> {
// NUL bytes are rejected by the Linux syscall layer, but the
// error surfaces as an opaque IO failure rather than the
// explicit `BadKey` the rest of the contract uses. Catch it
// here so the error path is consistent.
if key.contains('\0') {
return Err(StorageError::BadKey);
}
let key = key.trim_start_matches('/');
if key.is_empty() {
return Err(StorageError::BadKey);
}
if key.split('/').any(|seg| seg.is_empty() || seg == "." || seg == "..") {
return Err(StorageError::BadKey);
}
Ok(self.root.join(key))
}
}
#[async_trait]
impl Storage for LocalStorage {
async fn put(&self, key: &str, bytes: &[u8]) -> Result<(), StorageError> {
let path = self.resolve(key)?;
if let Some(parent) = path.parent() {
fs::create_dir_all(parent).await?;
}
fs::write(path, bytes).await?;
Ok(())
}
async fn put_stream(
&self,
key: &str,
mut stream: PutByteStream<'_>,
) -> Result<u64, StorageError> {
let path = self.resolve(key)?;
if let Some(parent) = path.parent() {
fs::create_dir_all(parent).await?;
}
// Atomic install via temp + rename. A failure mid-stream
// removes the temp so nothing is visible at `key`. The temp
// name uses a UUID suffix so concurrent puts of the same key
// (e.g. two workers racing a retry) don't clobber each
// other's in-progress file before the rename.
let tmp = path.with_extension(format!(
"{}.tmp.{}",
path.extension().and_then(|e| e.to_str()).unwrap_or(""),
uuid::Uuid::new_v4().simple()
));
let mut written: u64 = 0;
let result: Result<(), StorageError> = async {
let mut f = fs::File::create(&tmp).await?;
while let Some(chunk) = stream.next().await {
let chunk = chunk?;
f.write_all(&chunk).await?;
written = written.saturating_add(chunk.len() as u64);
}
// fsync before rename so a power-loss can't leave a
// zero-byte file at the destination.
f.sync_all().await?;
fs::rename(&tmp, &path).await?;
Ok(())
}
.await;
if let Err(e) = result {
// Best-effort cleanup; ignore "no such file" if the temp
// was never created.
let _ = fs::remove_file(&tmp).await;
return Err(e);
}
Ok(written)
}
async fn get(&self, key: &str) -> Result<Vec<u8>, StorageError> {
let path = self.resolve(key)?;
match fs::read(&path).await {
Ok(b) => Ok(b),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Err(StorageError::NotFound),
Err(e) => Err(e.into()),
}
}
async fn get_stream(&self, key: &str) -> Result<StreamingFile, StorageError> {
let path = self.resolve(key)?;
let file = match fs::File::open(&path).await {
Ok(f) => f,
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
return Err(StorageError::NotFound)
}
Err(e) => return Err(e.into()),
};
let size_bytes = file.metadata().await?.len();
// 64 KiB chunks: small enough that a few-MB page emits many frames
// (so streaming is observable), large enough to keep syscalls cheap.
let stream = ReaderStream::with_capacity(file, 64 * 1024);
Ok(StreamingFile {
stream: Box::pin(stream),
size_bytes,
})
}
async fn delete(&self, key: &str) -> Result<(), StorageError> {
let path = self.resolve(key)?;
match fs::remove_file(&path).await {
Ok(()) => Ok(()),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Err(StorageError::NotFound),
Err(e) => Err(e.into()),
}
}
async fn exists(&self, key: &str) -> Result<bool, StorageError> {
let path: &Path = &self.resolve(key)?;
Ok(fs::try_exists(path).await?)
}
fn local_root(&self) -> Option<&Path> {
Some(&self.root)
}
}
#[cfg(test)]
mod tests {
use super::*;
use tempfile::tempdir;
#[tokio::test]
async fn put_get_delete_roundtrip() {
let dir = tempdir().unwrap();
let s = LocalStorage::new(dir.path());
s.put("mangas/abc/cover.jpg", b"hello").await.unwrap();
assert!(s.exists("mangas/abc/cover.jpg").await.unwrap());
assert_eq!(s.get("mangas/abc/cover.jpg").await.unwrap(), b"hello");
s.delete("mangas/abc/cover.jpg").await.unwrap();
assert!(!s.exists("mangas/abc/cover.jpg").await.unwrap());
}
#[tokio::test]
async fn rejects_path_traversal() {
let dir = tempdir().unwrap();
let s = LocalStorage::new(dir.path());
// Parent-dir reference at the start.
assert!(matches!(s.put("../escape", b"x").await, Err(StorageError::BadKey)));
// Parent-dir reference mid-path.
assert!(matches!(s.get("a/../../b").await, Err(StorageError::BadKey)));
// Empty key.
assert!(matches!(s.exists("").await, Err(StorageError::BadKey)));
// Current-dir reference (the implementation rejects `.` segments
// alongside `..`; this exercises that arm).
assert!(matches!(s.get("a/./b").await, Err(StorageError::BadKey)));
assert!(matches!(s.get(".").await, Err(StorageError::BadKey)));
// Empty segment via doubled slash.
assert!(matches!(s.get("a//b").await, Err(StorageError::BadKey)));
// NUL byte (rejected explicitly so callers see BadKey rather
// than an opaque IO error from the kernel).
assert!(matches!(s.put("a\0b", b"x").await, Err(StorageError::BadKey)));
}
#[tokio::test]
async fn missing_key_is_not_found() {
let dir = tempdir().unwrap();
let s = LocalStorage::new(dir.path());
assert!(matches!(s.get("nope").await, Err(StorageError::NotFound)));
assert!(matches!(s.delete("nope").await, Err(StorageError::NotFound)));
assert!(matches!(
s.get_stream("nope").await.err(),
Some(StorageError::NotFound)
));
}
#[tokio::test]
async fn put_stream_writes_full_body_and_removes_temp_on_error() {
use bytes::Bytes;
use futures_util::stream;
let dir = tempdir().unwrap();
let s = LocalStorage::new(dir.path());
// Success: a 3-chunk stream writes the concatenation and
// returns the right byte count.
let chunks: Vec<Result<Bytes, StorageError>> = vec![
Ok(Bytes::from_static(b"alpha-")),
Ok(Bytes::from_static(b"beta-")),
Ok(Bytes::from_static(b"gamma")),
];
let bytes_written = s
.put_stream("streamed/ok.bin", Box::pin(stream::iter(chunks)))
.await
.unwrap();
assert_eq!(bytes_written, b"alpha-beta-gamma".len() as u64);
assert_eq!(s.get("streamed/ok.bin").await.unwrap(), b"alpha-beta-gamma");
// Failure mid-stream: nothing is visible at the destination
// and no .tmp file is left behind.
let chunks: Vec<Result<Bytes, StorageError>> = vec![
Ok(Bytes::from_static(b"good")),
Err(StorageError::Io(std::io::Error::other("boom"))),
];
let err = s
.put_stream("streamed/bad.bin", Box::pin(stream::iter(chunks)))
.await
.unwrap_err();
assert!(matches!(err, StorageError::Io(_)));
assert!(matches!(
s.get("streamed/bad.bin").await,
Err(StorageError::NotFound)
));
// No stray temp file in the streamed/ directory.
let entries: Vec<_> = std::fs::read_dir(dir.path().join("streamed"))
.unwrap()
.filter_map(|e| e.ok())
.map(|e| e.file_name().to_string_lossy().to_string())
.collect();
assert_eq!(entries, vec!["ok.bin"]);
}
#[tokio::test]
async fn get_stream_emits_multiple_chunks_for_large_files() {
use futures_util::StreamExt as _;
let dir = tempdir().unwrap();
let s = LocalStorage::new(dir.path());
// 256 KiB blob → at 64 KiB chunks should emit ~4 chunks.
let big = vec![7u8; 256 * 1024];
s.put("big.bin", &big).await.unwrap();
let StreamingFile { mut stream, size_bytes } = s.get_stream("big.bin").await.unwrap();
assert_eq!(size_bytes, big.len() as u64);
let mut chunks = 0usize;
let mut total = 0usize;
while let Some(frame) = stream.next().await {
let bytes = frame.unwrap();
chunks += 1;
total += bytes.len();
}
assert_eq!(total, big.len());
assert!(chunks > 1, "expected >1 chunk, got {chunks}");
}
}