fix(crawler): reap dead jobs alongside done ones

The retention reaper deleted only `state = 'done'` rows, so terminal
`dead` jobs (retries exhausted) accumulated in crawler_jobs forever —
slow unbounded table growth. Both states are end-of-life and never
leased again.

Rename `reap_done` → `reap_terminal` and broaden the filter to
`state IN ('done','dead')`; `pending`/`running` stay untouched. Test
extended to assert old done + old dead both reap while fresh terminal and
old-but-active rows survive.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-07-03 21:16:56 +02:00
parent a141d65db1
commit 83a9ab40cd
8 changed files with 54 additions and 45 deletions

View File

@@ -187,7 +187,7 @@ pub async fn list_ops(
}
/// Delete metric rows older than `retention_days`. `0` disables the reaper
/// (returns 0 without touching the table). Mirrors `jobs::reap_done`.
/// (returns 0 without touching the table). Mirrors `jobs::reap_terminal`.
pub async fn reap(pool: &PgPool, retention_days: u32) -> sqlx::Result<u64> {
if retention_days == 0 {
return Ok(0);

View File

@@ -903,9 +903,8 @@ pub struct JobHistoryFilter<'a> {
/// manga/chapter/page context (best-effort) so the table can label rows.
/// Returns the page slice plus the filtered total for pagination.
///
/// History depth is bounded by the done-job reaper (`reap_done`): completed
/// jobs older than the retention window are gone. Terminal `dead` jobs
/// persist until requeued.
/// History depth is bounded by the terminal-job reaper (`reap_terminal`):
/// `done` and `dead` jobs older than the retention window are gone.
pub async fn list_job_history(
pool: &PgPool,
filter: JobHistoryFilter<'_>,