c6a6d1690d
Turn the dormant timing data in crawl_metrics / page_analysis into "is it
healthy over time" views. New bucketed series queries (GROUP BY date_trunc,
hour|day via a closed Bucket enum so the unit can't be attacker-controlled)
behind GET /v1/admin/{crawler,analysis}/metrics/series, with a shared
SeriesParams/resolve_bucket helper (bad bucket → 400) and migration 0030
indexing page_analysis(analyzed_at) for the analysis scan.
Frontend: a dependency-free SVG TrendChart (line+area, null buckets render as
gaps, empty-state, role=img) embedded above the per-op tables in Crawler and
Analysis → Metrics, driven by each panel's existing window selector with
AbortController-cancelled fetches. A buildSeries() util fills the continuous
bucket axis (throughput 0 for empty buckets, success/duration null) — unit
tested alongside the chart and the series API client.
Closes the Phase-1 observability set (audit log · health checks · trends).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
58 lines
2.0 KiB
Rust
58 lines
2.0 KiB
Rust
//! Timing metrics for crawler operations (see `crawl_metrics` table, 0028).
|
|
//!
|
|
//! One [`OpRow`] per completed operation (manga list walk, manga detail,
|
|
//! cover, whole chapter); [`OpSummary`] is the per-type average roll-up that
|
|
//! drives the admin Metrics tab. Per-page crawl timing is derived on the
|
|
//! client from the `chapter` summary's `avg_items` (pages/chapter), not
|
|
//! stored per image. Analysis duration lives on `page_analysis`, not here.
|
|
|
|
use chrono::{DateTime, Utc};
|
|
use serde::Serialize;
|
|
use sqlx::FromRow;
|
|
use uuid::Uuid;
|
|
|
|
/// One time bucket of a metrics series — count, ok/failed split, and mean
|
|
/// duration over the bucket. Shared by the crawl-ops and analysis trend
|
|
/// charts (both are `{t, n, ok, failed, avg_ms}` over a `date_trunc` window).
|
|
/// Empty intervals are simply absent; the client fills the continuous axis.
|
|
#[derive(Debug, Clone, Serialize, FromRow)]
|
|
pub struct MetricsBucket {
|
|
/// Bucket start (`date_trunc(unit, finished_at|analyzed_at)`).
|
|
pub t: DateTime<Utc>,
|
|
pub n: i64,
|
|
pub ok: i64,
|
|
pub failed: i64,
|
|
pub avg_ms: Option<f64>,
|
|
}
|
|
|
|
/// Per-operation-type average roll-up over a time window.
|
|
#[derive(Debug, Clone, Serialize, FromRow)]
|
|
pub struct OpSummary {
|
|
/// `manga_list` | `manga_detail` | `manga_cover` | `chapter`.
|
|
pub op: String,
|
|
/// Mean duration in milliseconds (`NULL`→absent only when `n = 0`).
|
|
pub avg_ms: Option<f64>,
|
|
/// Total operations in the window.
|
|
pub n: i64,
|
|
pub ok: i64,
|
|
pub failed: i64,
|
|
/// Mean `items` (pages/chapter, mangas/list-walk); `None` when unused.
|
|
pub avg_items: Option<f64>,
|
|
}
|
|
|
|
/// One timed operation, resolved to manga/chapter labels for the recent-ops log.
|
|
#[derive(Debug, Clone, Serialize, FromRow)]
|
|
pub struct OpRow {
|
|
pub id: Uuid,
|
|
pub op: String,
|
|
pub manga_id: Option<Uuid>,
|
|
pub manga_title: Option<String>,
|
|
pub chapter_id: Option<Uuid>,
|
|
pub chapter_number: Option<i32>,
|
|
pub outcome: String,
|
|
pub duration_ms: i64,
|
|
pub items: Option<i32>,
|
|
pub error: Option<String>,
|
|
pub finished_at: DateTime<Utc>,
|
|
}
|