feat(analysis): page-analysis schema, domain types, and analyze_page job kind

Adds the persistence foundation for the AI content-analysis worker:

- Migration 0025: page_analysis (status + scene + is_nsfw + weighted
  search_doc tsvector), page_ocr_text (kind-tagged pieces), global
  page_auto_tags (shared tags vocabulary, separate from per-user
  page_tags), and page_content_warnings (closed vocabulary).
- domain::page_analysis: AnalysisStatus/OcrKind/ContentWarning enums with
  kind->tsvector-weight mapping and lenient model-string parsing, plus the
  VisionAnalysis response DTOs.
- JobPayload::AnalyzePage { page_id, force } + KIND_ANALYZE_PAGE, reusing
  the existing crawler_jobs queue.
- repo::page_analysis: enqueue_for_page, load, mark_failed, and the single
  transactional persist_analysis (delete+reinsert; idempotent; never
  touches user page_tags; computes the A/B/C/D-weighted search_doc).

Tests: repo integration (persist/idempotency/user-tags-untouched/enqueue/
mark_failed/speech>sfx ranking) + unit (weights, parsing, DTO, job serde).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-06-13 18:27:08 +02:00
parent f30600162e
commit 63e1aa5484
10 changed files with 814 additions and 3 deletions

View File

@@ -34,6 +34,15 @@ pub enum JobPayload {
chapter_id: Uuid,
source_chapter_key: String,
},
/// Run AI content-analysis (OCR, auto-tags, scene description, NSFW
/// moderation) on a single page image via the local vision model.
/// `force` re-analyzes a page that is already `done` (manual admin
/// re-trigger); otherwise the worker skips it.
AnalyzePage {
page_id: Uuid,
#[serde(default)]
force: bool,
},
}
#[derive(Clone, Copy, Debug, sqlx::Type, Serialize, Deserialize)]
@@ -52,6 +61,10 @@ pub enum JobState {
/// without re-spelling the literal.
pub const KIND_SYNC_CHAPTER_CONTENT: &str = "sync_chapter_content";
/// Kind discriminator for AI page-analysis jobs. The analysis daemon
/// leases with this filter so it never contends with crawl jobs.
pub const KIND_ANALYZE_PAGE: &str = "analyze_page";
#[derive(Debug)]
pub enum EnqueueResult {
Inserted(Uuid),
@@ -323,6 +336,29 @@ pub async fn reap_done(pool: &PgPool, retention_days: u32) -> sqlx::Result<u64>
mod tests {
use super::*;
#[test]
fn analyze_page_payload_round_trips_through_tagged_json() {
let page_id = Uuid::new_v4();
let payload = JobPayload::AnalyzePage {
page_id,
force: true,
};
let json = serde_json::to_value(&payload).unwrap();
assert_eq!(json["kind"], "analyze_page");
assert_eq!(json["page_id"], page_id.to_string());
assert_eq!(json["force"], true);
// `force` defaults to false when an older enqueue omitted it.
let legacy = serde_json::json!({ "kind": "analyze_page", "page_id": page_id });
match serde_json::from_value::<JobPayload>(legacy).unwrap() {
JobPayload::AnalyzePage { page_id: pid, force } => {
assert_eq!(pid, page_id);
assert!(!force);
}
other => panic!("expected AnalyzePage, got {other:?}"),
}
}
#[test]
fn backoff_base_grows_exponentially_and_caps_at_one_hour() {
// attempts == 1 → 60s, doubling each step.