fix(crawler): wrap wait_for_navigation in 30s timeout (0.36.1)

A hung TLS handshake or a page that never fires load could wedge a
worker (or the cron metadata pass) indefinitely — chromiumoxide
imposes no navigation timeout of its own.

New crawler::nav::wait_for_nav caps each navigation at NAV_TIMEOUT
(30s) and returns a typed NavError so timeouts surface as transient
(retryable) errors. Wired at the three navigation sites:
- source::target::navigate (catalog/detail/pagination)
- content::sync_chapter_content (chapter reader)
- session::fetch_probe_html (session probe)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-05-30 18:10:51 +02:00
parent 9f56f283d4
commit e2bd1462ba
8 changed files with 90 additions and 8 deletions

View File

@@ -203,7 +203,9 @@ async fn fetch_probe_html(browser: &Browser, probe_url: &str) -> anyhow::Result<
.new_page(probe_url)
.await
.with_context(|| format!("open probe page {probe_url}"))?;
page.wait_for_navigation().await.context("wait for nav on probe")?;
crate::crawler::nav::wait_for_nav(&page)
.await
.context("wait for nav on probe")?;
let html = page.content().await.context("read probe html")?;
page.close().await.ok();
Ok(html)