Files
Mangalord/backend/migrations/0025_page_analysis.sql
MechaCat02 63e1aa5484 feat(analysis): page-analysis schema, domain types, and analyze_page job kind
Adds the persistence foundation for the AI content-analysis worker:

- Migration 0025: page_analysis (status + scene + is_nsfw + weighted
  search_doc tsvector), page_ocr_text (kind-tagged pieces), global
  page_auto_tags (shared tags vocabulary, separate from per-user
  page_tags), and page_content_warnings (closed vocabulary).
- domain::page_analysis: AnalysisStatus/OcrKind/ContentWarning enums with
  kind->tsvector-weight mapping and lenient model-string parsing, plus the
  VisionAnalysis response DTOs.
- JobPayload::AnalyzePage { page_id, force } + KIND_ANALYZE_PAGE, reusing
  the existing crawler_jobs queue.
- repo::page_analysis: enqueue_for_page, load, mark_failed, and the single
  transactional persist_analysis (delete+reinsert; idempotent; never
  touches user page_tags; computes the A/B/C/D-weighted search_doc).

Tests: repo integration (persist/idempotency/user-tags-untouched/enqueue/
mark_failed/speech>sfx ranking) + unit (weights, parsing, DTO, job serde).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 18:27:08 +02:00

82 lines
3.9 KiB
SQL

-- AI content-analysis / enrichment / moderation results, one analysis
-- pass per page image. A background worker calls a local vision model
-- and writes four kinds of output here: OCR text (weighted by kind),
-- global auto-tags, a scene description, and content-warning flags.
--
-- Everything cascades with `pages` (like page_tags / collection_pages in
-- 0023): re-uploading a chapter recreates its page rows, intentionally
-- dropping the stale analysis so it gets re-enqueued and re-derived.
--
-- The auto-tags live in their OWN table (`page_auto_tags`), NOT in the
-- per-user `page_tags`, so re-analysis is a clean delete+reinsert that
-- never touches a user's personal tags. They reference the SAME shared
-- `tags` vocabulary (0009 / 0024) that manga tags and page tags use, so
-- model-proposed tags and human tags share one global namespace.
-- One row per analyzed page. `search_doc` is the worker-computed,
-- kind-weighted tsvector that the page text-search ranks against; it is
-- filled in the same transaction that writes the OCR rows (a generated
-- column can't aggregate the child `page_ocr_text` rows, and a trigger
-- would re-fire on every child insert during the delete+reinsert).
CREATE TABLE page_analysis (
page_id uuid PRIMARY KEY REFERENCES pages(id) ON DELETE CASCADE,
status text NOT NULL DEFAULT 'pending'
CHECK (status IN ('pending', 'done', 'failed')),
scene_description text,
is_nsfw boolean NOT NULL DEFAULT false,
model text,
error text,
analyzed_at timestamptz,
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
search_doc tsvector
);
-- Full-text ranking over the weighted document.
CREATE INDEX page_analysis_search_idx ON page_analysis USING gin (search_doc);
-- Partial index for "is this page flagged?" lookups (content-warning
-- joins and NSFW filters touch only the flagged minority).
CREATE INDEX page_analysis_nsfw_idx ON page_analysis (page_id) WHERE is_nsfw;
-- Extracted text pieces, each tagged with its kind. Kind drives the
-- tsvector weight (speech/title=A, narration/thought/caption=B,
-- sfx=D; scene_description is weighted C from page_analysis). `ord`
-- preserves reading order within the page.
CREATE TABLE page_ocr_text (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
page_id uuid NOT NULL REFERENCES pages(id) ON DELETE CASCADE,
kind text NOT NULL
CHECK (kind IN ('speech', 'thought', 'narration', 'sfx', 'title', 'caption')),
text text NOT NULL,
ord integer NOT NULL DEFAULT 0
);
CREATE INDEX page_ocr_text_page_idx ON page_ocr_text (page_id);
-- Global, model-derived page tags — visible to every user and searchable
-- alongside personal page tags. Keyed to the shared `tags` table by id.
CREATE TABLE page_auto_tags (
page_id uuid NOT NULL REFERENCES pages(id) ON DELETE CASCADE,
tag_id uuid NOT NULL REFERENCES tags(id) ON DELETE CASCADE,
created_at timestamptz NOT NULL DEFAULT now(),
PRIMARY KEY (page_id, tag_id)
);
-- tag -> pages (search by auto-tag) and page -> tags (render a page's
-- auto-tags); the PK already covers (page_id, tag_id) but a standalone
-- tag_id index serves the reverse direction.
CREATE INDEX page_auto_tags_tag_idx ON page_auto_tags (tag_id);
-- Per-page content warnings from the closed moderation vocabulary. The
-- manga detail page shows the deduplicated union across all its pages;
-- search filters include/exclude by these.
CREATE TABLE page_content_warnings (
page_id uuid NOT NULL REFERENCES pages(id) ON DELETE CASCADE,
warning text NOT NULL
CHECK (warning IN ('sexual', 'nudity', 'gore', 'violence', 'disturbing')),
PRIMARY KEY (page_id, warning)
);
-- warning -> pages, for the manga/page content-warning filters.
CREATE INDEX page_content_warnings_warning_idx ON page_content_warnings (warning);