feat(storage): admin storage-usage stats + per-manga/chapter sizes
Some checks failed
deploy / test-backend (push) Waiting to run
deploy / build-and-push (push) Has been cancelled
deploy / deploy (push) Has been cancelled
deploy / test-frontend (push) Has been cancelled

Persist blob sizes (covers + chapter pages) and surface upload storage
usage to admins in two places:

- Admin System tab: total/covers/chapters totals, ratio of disk, average
  image sizes, and top-5 largest mangas/chapters leaderboards, behind a
  new GET /api/v1/admin/storage endpoint (separate from /admin/system so
  the cheap DB aggregates aren't coupled to its 250ms CPU sample).
- Manga detail page: total chapter-content size and per-chapter size,
  with an em-dash for uncrawled or not-yet-measured chapters.

Sizes are captured at write time (crawler put_stream return, uploaded
page/cover byte length) into nullable pages.size_bytes /
mangas.cover_size_bytes columns; NULL means "not yet measured", distinct
from a real 0. A one-shot, idempotent admin "Backfill sizes" action
(POST /api/v1/admin/storage/backfill) stats pre-existing blobs in
keyset-paginated, capped batches and reports more_remaining so a large
legacy library can be drained over multiple runs.

Aggregates and leaderboards treat NULL honestly (only fully-measured
entities are ranked); a dashboard banner flags unmeasured rows so partial
figures aren't read as complete. persist_pages now prunes stale page rows
on a shrinking re-crawl so totals and page_count stay consistent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-06-15 20:14:49 +02:00
parent 4b6c19979a
commit 790549636f
35 changed files with 1804 additions and 40 deletions

View File

@@ -22,7 +22,13 @@ pub async fn list_for_manga(
// (number, created_at) tail then orders them deterministically.
let rows = sqlx::query_as::<_, Chapter>(
r#"
SELECT id, manga_id, number, title, page_count, created_at
SELECT id, manga_id, number, title, page_count, created_at,
(SELECT CASE
WHEN count(*) = 0 THEN 0
WHEN bool_or(p.size_bytes IS NULL) THEN NULL
ELSE sum(p.size_bytes)
END
FROM pages p WHERE p.chapter_id = chapters.id)::bigint AS size_bytes
FROM chapters
WHERE manga_id = $1
ORDER BY source_index DESC NULLS LAST, number ASC, created_at ASC
@@ -46,7 +52,13 @@ pub async fn find_by_id_in_manga(
) -> AppResult<Option<Chapter>> {
let row = sqlx::query_as::<_, Chapter>(
r#"
SELECT id, manga_id, number, title, page_count, created_at
SELECT id, manga_id, number, title, page_count, created_at,
(SELECT CASE
WHEN count(*) = 0 THEN 0
WHEN bool_or(p.size_bytes IS NULL) THEN NULL
ELSE sum(p.size_bytes)
END
FROM pages p WHERE p.chapter_id = chapters.id)::bigint AS size_bytes
FROM chapters
WHERE manga_id = $1 AND id = $2
"#,
@@ -81,7 +93,8 @@ pub async fn create<'e, E: PgExecutor<'e>>(
r#"
INSERT INTO chapters (manga_id, number, title, uploaded_by)
VALUES ($1, $2, $3, $4)
RETURNING id, manga_id, number, title, page_count, created_at
RETURNING id, manga_id, number, title, page_count, created_at, 0::bigint AS size_bytes
-- A freshly created chapter has no pages yet → size 0 (known).
"#,
)
.bind(manga_id)