feat(analysis): enqueue page-analysis on upload/crawl + admin backfill endpoints
Wires the analyze_page job kind into the page-create paths and adds admin controls, all gated on a new ANALYSIS_ENABLED flag (AppState.analysis_enabled, config::AnalysisConfig): - Chapter upload (api::chapters) enqueues one analyze_page job per page after the tx commits (best-effort; failures logged, never fatal). - Crawler content sync (persist_pages → RETURNING id) enqueues per persisted page when the daemon dispatcher's analysis_enabled flag is set; threaded through sync_chapter_content (CLI/resync pass false). - repo::page_analysis::enqueue_all_pages: bulk backfill via INSERT..SELECT, skipping done pages (only_unanalyzed) and existing pending jobs. - New admin endpoints (RequireAdmin, 503 when disabled, audited): POST /admin/analysis/reenqueue and POST /admin/pages/:id/analyze (force). Tests: upload enqueues N jobs / no-op when disabled; crawler persist_pages enqueue gate; admin reenqueue (backfill, idempotent, only-unanalyzed, 503, non-admin 403) and force-analyze (force flag, 404). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -66,6 +66,31 @@ impl Default for UploadConfig {
|
||||
}
|
||||
}
|
||||
|
||||
/// AI content-analysis worker configuration. Phase 2 only needs the
|
||||
/// `enabled` gate (so page-create paths know whether to enqueue analysis
|
||||
/// jobs); later phases extend this with the vision endpoint, model, and
|
||||
/// worker knobs.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct AnalysisConfig {
|
||||
/// Master switch (`ANALYSIS_ENABLED`). When `false`, no analysis jobs
|
||||
/// are enqueued and no worker runs. Defaults to `false`.
|
||||
pub enabled: bool,
|
||||
}
|
||||
|
||||
impl Default for AnalysisConfig {
|
||||
fn default() -> Self {
|
||||
Self { enabled: false }
|
||||
}
|
||||
}
|
||||
|
||||
impl AnalysisConfig {
|
||||
pub fn from_env() -> Self {
|
||||
Self {
|
||||
enabled: env_bool("ANALYSIS_ENABLED", false),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct Config {
|
||||
pub database_url: String,
|
||||
@@ -89,6 +114,7 @@ pub struct Config {
|
||||
/// the check.
|
||||
pub admin_allowed_origins: Vec<String>,
|
||||
pub crawler: CrawlerConfig,
|
||||
pub analysis: AnalysisConfig,
|
||||
/// `(username, password)` for the admin user provisioned at startup
|
||||
/// when both `ADMIN_USERNAME` and `ADMIN_PASSWORD` are set. `None`
|
||||
/// skips the bootstrap entirely. See `repo::user::bootstrap_admin`
|
||||
@@ -245,6 +271,7 @@ impl Config {
|
||||
})
|
||||
.unwrap_or_default(),
|
||||
crawler: CrawlerConfig::from_env()?,
|
||||
analysis: AnalysisConfig::from_env(),
|
||||
admin_bootstrap: admin_bootstrap_from_env(),
|
||||
})
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user