perf(admin): fix O(mangas x jobs) overview query that pinned Postgres (0.85.1)
The admin overview dashboard polls /admin/overview every 30s. Its `manga_stats` aggregate evaluated MANGA_SYNC_STATE_CASE over the WHOLE library (14.5k mangas) with no LIMIT, and that CASE runs a correlated EXISTS over crawler_jobs. With no index on the `sync_chapter_list` manga_id path, Postgres seqscanned all in-flight jobs *per manga* — O(mangas x jobs). The disabled-analysis backlog (9.5k pending `analyze_page` rows that can never match the sync-kind filter) inflated every scan, so each call took 6-7 min. The 30s poll stacked ~10 of them concurrently → load 11, Postgres at ~100%. EXPLAIN ANALYZE on the live DB: the rewrite drops the query from 6-7 min to ~1.5s (per-manga crawler_jobs seqscan → single index-backed pass). Fixes: - Rewrite `manga_stats` to collect the in-flight manga-id set in ONE pass over the sync jobs (index-backed), then classify via a hash semi-join — O(mangas + jobs), immune to the analyze_page backlog size. - Add partial index crawler_jobs_sync_chapter_list_manga_idx (migration 0029) covering the sync_chapter_list manga_id path; mirrors the existing sync_manga index (0020). Also speeds the per-row Mangas tab listing. - statement_timeout backstop (5s) on the aggregate so a pathological plan can never pin a backend for minutes again. - Single-flight + 10s TTL cache on the /admin/overview handler so concurrent pollers coalesce instead of stacking. - Slow the dashboard poll 30s -> 60s (server-cached now anyway). The in_progress rule in the rewrite is kept in lockstep with the first arm of MANGA_SYNC_STATE_CASE (documented in both places). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -9,11 +9,15 @@
|
||||
//!
|
||||
//! Admin-only (`RequireAdmin`, cookie-only).
|
||||
|
||||
use std::sync::OnceLock;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use axum::extract::State;
|
||||
use axum::routing::get;
|
||||
use axum::{Json, Router};
|
||||
use chrono::{DateTime, Utc};
|
||||
use serde::Serialize;
|
||||
use tokio::sync::Mutex;
|
||||
|
||||
use crate::app::AppState;
|
||||
use crate::auth::extractor::RequireAdmin;
|
||||
@@ -24,14 +28,14 @@ pub fn routes() -> Router<AppState> {
|
||||
Router::new().route("/admin/overview", get(overview))
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
#[derive(Debug, Clone, Serialize)]
|
||||
pub struct OverviewStats {
|
||||
pub users: UsersOverview,
|
||||
pub mangas: repo::admin_view::MangaStats,
|
||||
pub analysis: AnalysisOverview,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
#[derive(Debug, Clone, Serialize)]
|
||||
pub struct UsersOverview {
|
||||
pub total: i64,
|
||||
pub admins: i64,
|
||||
@@ -39,16 +43,42 @@ pub struct UsersOverview {
|
||||
pub newest_created_at: Option<DateTime<Utc>>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
#[derive(Debug, Clone, Serialize)]
|
||||
pub struct AnalysisOverview {
|
||||
pub analyzed_pages: i64,
|
||||
pub total_pages: i64,
|
||||
}
|
||||
|
||||
/// Process-wide single-flight cache for the overview aggregate.
|
||||
///
|
||||
/// The admin dashboard polls this endpoint on a timer; without coalescing, a
|
||||
/// slow DB (e.g. a deep crawl/analysis backlog) lets identical requests stack
|
||||
/// and pin Postgres. Holding the async mutex across the recompute gives BOTH a
|
||||
/// short-TTL cache and single-flight: concurrent pollers block on the lock,
|
||||
/// then read the freshly cached value instead of each launching their own
|
||||
/// aggregate. Resets on process restart, which is fine for a dashboard stat.
|
||||
///
|
||||
/// Unlike per-`AppState` shared state (e.g. `auth_limiter`), this is a single
|
||||
/// production-instance cache, so a process global is adequate. It is
|
||||
/// test-safe because the cache lives in the handler *body*: the two auth tests
|
||||
/// are rejected at the `RequireAdmin` extractor before reaching it, and the
|
||||
/// one aggregate test issues a single request. A future test that mutates data
|
||||
/// and re-reads within the TTL would need to account for this.
|
||||
static OVERVIEW_CACHE: OnceLock<Mutex<Option<(Instant, OverviewStats)>>> = OnceLock::new();
|
||||
const OVERVIEW_TTL: Duration = Duration::from_secs(10);
|
||||
|
||||
async fn overview(
|
||||
State(state): State<AppState>,
|
||||
_admin: RequireAdmin,
|
||||
) -> AppResult<Json<OverviewStats>> {
|
||||
let cache = OVERVIEW_CACHE.get_or_init(|| Mutex::new(None));
|
||||
let mut guard = cache.lock().await;
|
||||
if let Some((at, cached)) = guard.as_ref() {
|
||||
if at.elapsed() < OVERVIEW_TTL {
|
||||
return Ok(Json(cached.clone()));
|
||||
}
|
||||
}
|
||||
|
||||
// Independent reads — run concurrently so latency is the slowest
|
||||
// query, not their sum (mirrors the storage handler).
|
||||
let (user_counts, user_newest, mangas, coverage) = tokio::try_join!(
|
||||
@@ -65,7 +95,7 @@ async fn overview(
|
||||
};
|
||||
let (analyzed_pages, total_pages) = coverage;
|
||||
|
||||
Ok(Json(OverviewStats {
|
||||
let stats = OverviewStats {
|
||||
users: UsersOverview {
|
||||
total,
|
||||
admins,
|
||||
@@ -77,5 +107,7 @@ async fn overview(
|
||||
analyzed_pages,
|
||||
total_pages,
|
||||
},
|
||||
}))
|
||||
};
|
||||
*guard = Some((Instant::now(), stats.clone()));
|
||||
Ok(Json(stats))
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user