feat(storage): admin storage-usage stats + per-manga/chapter sizes
Some checks failed
deploy / test-backend (push) Waiting to run
deploy / build-and-push (push) Has been cancelled
deploy / deploy (push) Has been cancelled
deploy / test-frontend (push) Has been cancelled

Persist blob sizes (covers + chapter pages) and surface upload storage
usage to admins in two places:

- Admin System tab: total/covers/chapters totals, ratio of disk, average
  image sizes, and top-5 largest mangas/chapters leaderboards, behind a
  new GET /api/v1/admin/storage endpoint (separate from /admin/system so
  the cheap DB aggregates aren't coupled to its 250ms CPU sample).
- Manga detail page: total chapter-content size and per-chapter size,
  with an em-dash for uncrawled or not-yet-measured chapters.

Sizes are captured at write time (crawler put_stream return, uploaded
page/cover byte length) into nullable pages.size_bytes /
mangas.cover_size_bytes columns; NULL means "not yet measured", distinct
from a real 0. A one-shot, idempotent admin "Backfill sizes" action
(POST /api/v1/admin/storage/backfill) stats pre-existing blobs in
keyset-paginated, capped batches and reports more_remaining so a large
legacy library can be drained over multiple runs.

Aggregates and leaderboards treat NULL honestly (only fully-measured
entities are ranked); a dashboard banner flags unmeasured rows so partial
figures aren't read as complete. persist_pages now prunes stale page rows
on a shrinking re-crawl so totals and page_count stay consistent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-06-15 20:14:49 +02:00
parent 4b6c19979a
commit 790549636f
35 changed files with 1804 additions and 40 deletions

View File

@@ -11,6 +11,13 @@ pub struct Chapter {
pub title: Option<String>,
pub page_count: i32,
pub created_at: DateTime<Utc>,
/// Total bytes of this chapter's stored page images. `Some(0)` for an
/// uncrawled chapter (no page rows). `None` when the size is unknown —
/// at least one page predates the size backfill — so the frontend can
/// show an em-dash instead of a misleading "0 B". `Some(n)` once every
/// page in the chapter has a measured size.
#[serde(default)]
pub size_bytes: Option<i64>,
}
#[derive(Debug, Clone, Deserialize)]

View File

@@ -43,6 +43,12 @@ pub struct MangaDetail {
pub genres: Vec<GenreRef>,
pub tags: Vec<TagRef>,
pub content_warnings: Vec<ContentWarning>,
/// Total bytes of all this manga's stored chapter pages (cover
/// excluded), over every chapter. `None` when any page is unmeasured
/// (frontend shows an em-dash); `Some(0)` when there are no pages.
/// Computed in the DB rather than summed from the chapter list, which
/// is paginated and would undercount mangas with many chapters.
pub chapter_storage_bytes: Option<i64>,
}
#[derive(Debug, Clone, Deserialize, Default)]

View File

@@ -12,6 +12,7 @@ pub mod page_tag;
pub mod patch;
pub mod read_progress;
pub mod session;
pub mod storage_stats;
pub mod sync_state;
pub mod tag;
pub mod upload_entry;
@@ -38,6 +39,7 @@ pub use page_tag::{
pub use patch::Patch;
pub use read_progress::{ReadProgress, ReadProgressForManga, ReadProgressSummary};
pub use session::Session;
pub use storage_stats::{StorageStats, TopChapter, TopManga};
pub use sync_state::{ChapterSyncState, MangaSyncState};
pub use tag::{Tag, TagRef};
pub use upload_entry::UploadEntry;

View File

@@ -0,0 +1,57 @@
//! Admin storage-usage stats. Pure data; assembled by
//! `crate::api::admin::storage` from `repo::storage_stats` aggregates
//! plus the storage backend's volume size.
use serde::Serialize;
use uuid::Uuid;
/// Storage usage of uploaded content (covers + chapter pages), for the
/// admin dashboard System tab. Sizes are bytes; the frontend formats
/// them with the binary (MiB/GiB) helper.
#[derive(Debug, Clone, Serialize)]
pub struct StorageStats {
/// covers + chapter pages.
pub total_bytes: i64,
pub covers_bytes: i64,
pub chapters_bytes: i64,
/// Volume size of the storage root (`statvfs`). `None` when the
/// backend has no local path (e.g. a future S3 backend).
pub disk_total_bytes: Option<u64>,
/// `total_bytes / disk_total_bytes`. `None` when the disk total is
/// unknown — the frontend hides the ratio rather than fabricate one.
pub ratio_of_disk: Option<f64>,
/// Average image size across covers + chapter pages. `None` for an
/// empty library (no division by zero).
pub avg_image_bytes: Option<f64>,
pub avg_cover_bytes: Option<f64>,
pub avg_chapter_page_bytes: Option<f64>,
/// Largest mangas by total stored bytes (cover + all chapter pages).
/// Only fully-measured mangas are ranked.
pub top_mangas: Vec<TopManga>,
/// Largest chapters by total stored page bytes. Only fully-measured
/// chapters are ranked.
pub top_chapters: Vec<TopChapter>,
/// Page rows still awaiting a size backfill. When > 0, every figure
/// above is a partial (lower-bound) measurement and the leaderboards
/// omit not-yet-measured content — the dashboard surfaces this so the
/// numbers aren't read as complete.
pub unmeasured_pages: i64,
/// Cover blobs still awaiting a size backfill.
pub unmeasured_covers: i64,
}
#[derive(Debug, Clone, Serialize)]
pub struct TopManga {
pub id: Uuid,
pub title: String,
pub total_bytes: i64,
}
#[derive(Debug, Clone, Serialize)]
pub struct TopChapter {
pub id: Uuid,
pub manga_id: Uuid,
pub number: i32,
pub title: Option<String>,
pub total_bytes: i64,
}