fix(analyze-dedup): migration pre-dedup, force-collision upgrade, gate test (0.87.11)
Four 0.87.6 follow-ups from the adversarial review: 1. **Migration 0031 pre-dedup.** Demote all-but-lowest-id duplicate `analyze_page` rows in `(pending|running)` to `dead` before creating the unique index, with a curator-recoverable last_error marker. Without this, `sqlx::migrate!` would refuse to boot on any dirty production DB. 2. **`enqueue_for_page(force=true)` collision.** The partial unique index used to silently swallow force requests when a `force=false` job was already pending. Repo function now upgrades the pending row's `force` flag in place (or falls through to re-INSERT if the sibling drained mid-call), and reports an `EnqueueForPageOutcome` for accurate auditing. 3. **`record_duration` gate test.** New test seeds a `done` row with known duration, force-re-analyzes with failing dispatcher + max_attempts=3 (non-terminal), asserts duration_ms wasn't overwritten. 4. **`bookmark.rs` PK comment correction.** Use `b.id DESC` instead of the wrong `manga_id`; PK is actually `id`, and migration 0004 allows both chapter-level and manga-level bookmarks on the same manga. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -200,6 +200,80 @@ async fn worker_marks_failed_row_on_terminal_failure(pool: PgPool) {
|
||||
assert!(row.error.is_some());
|
||||
}
|
||||
|
||||
/// Non-terminal failure of a force-re-analyze must NOT overwrite the
|
||||
/// prior `done` row's `duration_ms`. Before the 0.87.6 gate change,
|
||||
/// `record_duration` ran unconditionally — a transient failure-retry
|
||||
/// of a force-re-analyze would clobber the legitimately-measured
|
||||
/// previous duration with the duration of an attempt that wrote
|
||||
/// nothing. Test pins the gate.
|
||||
#[sqlx::test(migrations = "./migrations")]
|
||||
async fn worker_non_terminal_force_failure_does_not_overwrite_done_duration(pool: PgPool) {
|
||||
let page_id = seed_page(&pool).await;
|
||||
// 1) Pre-seed a done analysis row with a meaningful duration so we
|
||||
// have something to be "overwritten".
|
||||
page_analysis::persist_analysis(
|
||||
&pool,
|
||||
page_id,
|
||||
&VisionAnalysis {
|
||||
ocr_results: vec![],
|
||||
tagging_results: vec![],
|
||||
scene_description: String::new(),
|
||||
safety_flag: SafetyFlag::default(),
|
||||
},
|
||||
"m",
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
sqlx::query("UPDATE page_analysis SET duration_ms = $1 WHERE page_id = $2")
|
||||
.bind(12345_i64)
|
||||
.bind(page_id)
|
||||
.execute(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
// 2) Force re-analyze (force=true) — but with a failing dispatcher
|
||||
// AND max_attempts=3 so the first failure is NON-terminal.
|
||||
page_analysis::enqueue_for_page(&pool, page_id, true)
|
||||
.await
|
||||
.unwrap();
|
||||
sqlx::query("UPDATE crawler_jobs SET max_attempts = 3 WHERE payload->>'page_id' = $1")
|
||||
.bind(page_id.to_string())
|
||||
.execute(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let dispatcher = CountingDispatcher::failing();
|
||||
let (handle, cancel) = spawn_with(&pool, dispatcher.clone());
|
||||
|
||||
// 3) Wait until the worker has dispatched once — the job goes back
|
||||
// to `pending` because the retry is non-terminal (attempts < max).
|
||||
let deadline = std::time::Instant::now() + Duration::from_secs(5);
|
||||
while dispatcher.call_count() == 0 && std::time::Instant::now() < deadline {
|
||||
tokio::time::sleep(Duration::from_millis(50)).await;
|
||||
}
|
||||
assert!(dispatcher.call_count() >= 1, "dispatcher must have run");
|
||||
|
||||
// Wait briefly for `record_duration` to either run or get gated.
|
||||
// Then cancel before the next backoff fires (it'd burn 60s otherwise).
|
||||
tokio::time::sleep(Duration::from_millis(200)).await;
|
||||
cancel.cancel();
|
||||
handle.shutdown().await;
|
||||
|
||||
// 4) The prior `done` row's `duration_ms` must NOT have been
|
||||
// overwritten by the failed retry's duration.
|
||||
let dur: Option<i64> =
|
||||
sqlx::query_scalar("SELECT duration_ms FROM page_analysis WHERE page_id = $1")
|
||||
.bind(page_id)
|
||||
.fetch_one(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
dur,
|
||||
Some(12345),
|
||||
"non-terminal failure must not overwrite the prior done row's duration"
|
||||
);
|
||||
}
|
||||
|
||||
#[sqlx::test(migrations = "./migrations")]
|
||||
async fn worker_isolates_dispatcher_panics(pool: PgPool) {
|
||||
let page_id = seed_page(&pool).await;
|
||||
|
||||
Reference in New Issue
Block a user