fix: index the running/leased_until arm of the job-lease query
The lease predicate's `state='running' AND leased_until < now()` arm had no index, so every lease poll seq-scanned the whole crawler_jobs table (audit H2). migration 0039 adds a partial index on (leased_until) WHERE state='running'. Behavior-neutral; 29 lease tests green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
18
backend/migrations/0039_crawler_jobs_running_lease_idx.sql
Normal file
18
backend/migrations/0039_crawler_jobs_running_lease_idx.sql
Normal file
@@ -0,0 +1,18 @@
|
||||
-- Index the crashed-worker-recovery arm of the job lease predicate.
|
||||
--
|
||||
-- lease/lease_kinds match:
|
||||
-- WHERE (state = 'pending' OR (state = 'running' AND leased_until < now()))
|
||||
-- crawler_jobs_ready_idx (0016) is `ON (scheduled_at) WHERE state = 'pending'`,
|
||||
-- so it covers only the pending arm. The `state = 'running' AND leased_until`
|
||||
-- arm had no usable index, so Postgres could not BitmapOr the two arms and
|
||||
-- degraded to a sequential scan of the ENTIRE crawler_jobs table — including all
|
||||
-- done/dead rows not yet reaped — on every lease poll, by every worker,
|
||||
-- continuously (audit H2, the headline performance finding).
|
||||
--
|
||||
-- A partial index on leased_until over just the running rows makes the recovery
|
||||
-- arm index-backed. `state = 'running'` is an immutable predicate (now() stays
|
||||
-- in the query, not the index). The running set is tiny (in-flight jobs only),
|
||||
-- so the index is cheap to maintain.
|
||||
CREATE INDEX crawler_jobs_running_lease_idx
|
||||
ON crawler_jobs (leased_until)
|
||||
WHERE state = 'running';
|
||||
Reference in New Issue
Block a user