feat(admin): observability — job history, live now-analyzing, durations & metrics (0.84.0) #6

fabi · 2026-06-16T12:20:03Z

fabi commented

2026-06-16 12:20:03 +00:00

Adds an admin observability surface: crawler + analysis job history, a live "now analyzing" banner, per-operation durations and metric roll-ups.

Backend

Migration 0028: durable crawl_metrics timing table (op/outcome/duration_ms/items, manga/chapter FK ON DELETE SET NULL) + page_analysis.duration_ms.
New crawl_metrics repo (record/summary/list_ops/reap) + history/metrics endpoints under admin crawler & analysis.
Analysis daemon resolves a labeled breadcrumb (manga title + chapter number) and stamps vision dispatch duration (best-effort, after row write).
CRAWL_METRICS_RETENTION_DAYS (default 90) + reaper, wired through config/settings/daemon.

Frontend

Crawler + Analysis history tables and metrics panels; fmtDuration helper; live event labels.

Review: static review (no CI build) cross-checked every runtime SQL string against the migrations — columns/tables resolve, bind arity matches, CHECK literals constrained to consts, FromRow mappings aligned, tests + frontend types consistent. No correctness issues found.

Adds an admin observability surface: crawler + analysis job history, a live "now analyzing" banner, per-operation durations and metric roll-ups. **Backend** - Migration 0028: durable `crawl_metrics` timing table (op/outcome/duration_ms/items, manga/chapter FK ON DELETE SET NULL) + `page_analysis.duration_ms`. - New `crawl_metrics` repo (record/summary/list_ops/reap) + history/metrics endpoints under admin crawler & analysis. - Analysis daemon resolves a labeled breadcrumb (manga title + chapter number) and stamps vision dispatch duration (best-effort, after row write). - `CRAWL_METRICS_RETENTION_DAYS` (default 90) + reaper, wired through config/settings/daemon. **Frontend** - Crawler + Analysis history tables and metrics panels; `fmtDuration` helper; live event labels. **Review:** static review (no CI build) cross-checked every runtime SQL string against the migrations — columns/tables resolve, bind arity matches, CHECK literals constrained to consts, FromRow mappings aligned, tests + frontend types consistent. No correctness issues found.

fabi added 1 commit 2026-06-16 12:20:03 +00:00

feat(admin): observability — job history, live "now analyzing", durations & metrics

deploy / test-frontend (pull_request) Waiting to run

Details

deploy / build-and-push (pull_request) Blocked by required conditions

Details

deploy / deploy (pull_request) Blocked by required conditions

Details

deploy / test-backend (pull_request) Has been cancelled

Details

5fa4442904

Crawler + analysis admin dashboards gain a Live / History / Metrics
segmented view.

History: searchable, filterable, paginated job log per subsystem
(crawler_jobs across all states/kinds; page_analysis terminal outcomes),
with inline dead-job requeue and a Duration column.

Live: enrich analysis SSE events with manga title + chapter number and add
a sticky "Now analyzing" banner that jumps to and highlights the page.

Metrics: new durable crawl_metrics table (migration 0028) timing every
crawl op (manga list walk, manga detail, cover, whole chapter; per-page
derived from chapter) plus page_analysis.duration_ms for analysis. New
endpoints serve per-type average durations + success rates and a recent-ops
log; a cron reaper (CRAWL_METRICS_RETENTION_DAYS) bounds growth.

Tested: repo + API integration tests, vitest for the API client and
fmtDuration, and Playwright for the History/Metrics tabs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

fabi merged commit d51ab2a049 into main

2026-06-16 12:21:13 +00:00

fabi referenced this issue from a commit

2026-06-16 12:21:14 +00:00

feat(admin): observability — job history, live now-analyzing, durations & metrics (0.84.0) (#6)

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: fabi/Mangalord#6