fix: batch and index the crawler_jobs retention reaper

reap_terminal was one unbatched DELETE with no updated_at index — a full scan and
long-held lock over a large backlog (audit M5). Migration 0040 adds a partial
index on (updated_at) WHERE state IN ('done','dead'); reap_terminal now deletes in
bounded 5k batches. Existing tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

backend/migrations/0040_crawler_jobs_terminal_reap_idx.sql

@@ -0,0 +1,9 @@
+-- Index the retention reaper's scan. reap_terminal deletes
+--   WHERE state IN ('done','dead') AND updated_at < now() - interval
+-- which had no supporting index, so each daily sweep sequential-scanned the whole
+-- crawler_jobs table to find expired terminal rows (audit M5). A partial index on
+-- updated_at over just the terminal states makes the batched reaper's per-batch
+-- SELECT index-backed.
+CREATE INDEX crawler_jobs_terminal_reap_idx
+    ON crawler_jobs (updated_at)
+    WHERE state IN ('done', 'dead');