Compare commits

...

4 Commits

Author SHA1 Message Date
MechaCat02
227b66cb4f docs(env): clarify ADMIN_ALLOWED_ORIGINS fails closed for browsers
The comment said an empty value means "CSRF check disabled", but
cookie-authenticated admin mutations fail closed (403) when the
allowlist is empty — which silently blocks enabling the analysis worker
from the browser. Correct the note and add the local-dev hint to set it
to the Vite origin (http://localhost:5173).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 20:03:30 +02:00
MechaCat02
89300605a4 feat(admin): present OCR-only analysis UI, hide vision surfaces
The analysis admin now reflects the OCR backend instead of the vision
LLM. Backend response types are unchanged (vision fields ride along,
ignored) so this is fully reversible.

- Settings → Analysis: keep only the OCR-relevant knobs (workers, job
  timeout, max image bytes) plus a read-only "Backend: OCR (ocrs)"
  line; drop the endpoint/model/API-key, sampling, slicing, and prompt
  fieldsets.
- Analysis tab: reword the lede to OCR text extraction; the page-detail
  modal drops the NSFW/content-warning, scene, and auto-tag sections and
  shows only OCR text (the per-line kind label renders only when set,
  since ocrs leaves it blank).
- History table: remove the NSFW-only filter and the per-row NSFW tag,
  and stop sending the `nsfw` query param.
- Metrics panel: remove the by-model table (OCR has a single engine);
  keep the tiles and trend charts.

Add component tests asserting the history table omits the NSFW filter
and never tags a row, and the metrics panel renders tiles without the
by-model table.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 20:03:17 +02:00
MechaCat02
808f4205d9 fix(settings): don't require vision endpoint/model when OCR is active
Enabling the analysis worker ran the vision validators: the SSRF gate
rejected the default `localhost` endpoint and a model id was required.
With the OCR backend active those fields are never used, and the
OCR-only settings UI no longer exposes them — so an operator could not
turn the worker on (a 422 with no fixable field).

Gate both checks on `base.backend == Vision`. The basic malformed-URL
sanity check stays unconditional. Existing vision-validation tests now
use a Vision base; a new test pins that enabling on the OCR backend with
the default localhost endpoint and an empty model validates cleanly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 20:03:07 +02:00
MechaCat02
de6eaf9f8b feat(analysis): make OCR the active backend, keep vision dormant
The worker now always dispatches through the in-process ocrs engine.
`AnalysisConfig::effective_backend()` returns Ocr regardless of the
parsed `ANALYSIS_BACKEND` (warning if `vision` was requested), and
`spawn_analysis_daemon` selects on it. The vision dispatcher, client,
and readiness probe stay compiled and intact behind the unreachable
match arm, so re-enabling is a one-line change.

The startup-reclaim regression test no longer leaned on the vision arm
to skip model loading; it now points the engine at a missing model path
and asserts the orphaned lease is still reclaimed before the (failing)
engine build — proving reclaim runs independently of engine readiness.

Also gitignore backend/models/*.rten so locally-downloaded ocrs models
for native `cargo run` aren't committed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 20:02:59 +02:00
11 changed files with 221 additions and 326 deletions

View File

@@ -62,11 +62,16 @@ CORS_ALLOWED_ORIGINS=
# neither Origin nor Referer (curl, server-to-server callers) are # neither Origin nor Referer (curl, server-to-server callers) are
# always allowed. # always allowed.
# #
# Default is empty: CSRF check disabled (operator opt-out). For a # Empty does NOT mean "off" for browsers: cookie-authenticated admin
# browser-exposed deployment this should be set to the SvelteKit # mutations FAIL CLOSED (403) when this is empty, since there's no
# origin, e.g. https://app.example.com. For a same-origin # allowlist to check the Origin against. Non-cookie callers (curl,
# docker-compose deploy where only one origin exists, set the same # bots with a Bearer token, no Origin/Referer) are still allowed.
# value the browser uses. # So set this to the SvelteKit origin for any browser-exposed deploy,
# e.g. https://app.example.com. For a same-origin docker-compose deploy
# set the same value the browser uses.
# Local dev (native `npm run dev`): the Vite origin is
# http://localhost:5173 — set ADMIN_ALLOWED_ORIGINS=http://localhost:5173
# or admin toggles (e.g. enabling the analysis worker) return 403.
ADMIN_ALLOWED_ORIGINS= ADMIN_ALLOWED_ORIGINS=
# ----- Admin bootstrap ----- # ----- Admin bootstrap -----

4
backend/.gitignore vendored
View File

@@ -1,3 +1,7 @@
/target /target
/.sqlx /.sqlx
.env .env
# Local OCR models for native dev (downloaded, not source)
models/
*.rten

View File

@@ -434,7 +434,7 @@ async fn spawn_analysis_daemon(
let (dispatcher, readiness): ( let (dispatcher, readiness): (
Arc<dyn crate::analysis::daemon::AnalyzeDispatcher>, Arc<dyn crate::analysis::daemon::AnalyzeDispatcher>,
Option<Arc<dyn crate::analysis::daemon::VisionReadiness>>, Option<Arc<dyn crate::analysis::daemon::VisionReadiness>>,
) = match cfg.backend { ) = match cfg.effective_backend() {
crate::config::AnalysisBackend::Ocr => { crate::config::AnalysisBackend::Ocr => {
// Load the `.rten` models once; a bad path is a loud boot error. // Load the `.rten` models once; a bad path is a loud boot error.
let engine = crate::analysis::ocr::OcrsEngine::from_model_paths( let engine = crate::analysis::ocr::OcrsEngine::from_model_paths(
@@ -1473,29 +1473,30 @@ mod tests {
// Bind to a local so the TempDir lives for the rest of the test. // Bind to a local so the TempDir lives for the rest of the test.
// `tempfile::tempdir().unwrap().path()` would drop the TempDir // `tempfile::tempdir().unwrap().path()` would drop the TempDir
// at end-of-expression and `LocalStorage` would hold a path to // at end-of-expression and `LocalStorage` would hold a path to
// a deleted directory. (Today this is fine because the dispatch // a deleted directory.
// fails before storage is touched, but it makes the test fragile
// to any future code rearrangement.)
let storage_dir = tempfile::tempdir().unwrap(); let storage_dir = tempfile::tempdir().unwrap();
let storage: Arc<dyn Storage> = let storage: Arc<dyn Storage> =
Arc::new(LocalStorage::new(storage_dir.path())); Arc::new(LocalStorage::new(storage_dir.path()));
let mut cfg = crate::config::AnalysisConfig::default(); let mut cfg = crate::config::AnalysisConfig::default();
// Use the vision backend so the daemon doesn't try to load the ocrs // The worker always runs OCR now (vision is dormant — see
// `.rten` models (absent in unit CI). This test only exercises the // `effective_backend`), and the `.rten` models aren't shipped to unit
// pre-worker lease reclaim, which is engine-agnostic; no dispatch runs. // CI. Point the engine at a path that can't exist so the *engine build*
cfg.backend = crate::config::AnalysisBackend::Vision; // fails deterministically. Reclaim runs at the very top of
// `spawn_analysis_daemon`, before — and independently of — engine
// readiness, so the row must still be reclaimed even though spawn
// returns Err. That's exactly the regression this test guards (an
// analysis-only deploy must reclaim orphaned leases at startup).
cfg.ocr_detection_model = "/nonexistent/text-detection.rten".to_string();
cfg.ocr_recognition_model = "/nonexistent/text-recognition.rten".to_string();
cfg.workers = 1; cfg.workers = 1;
cfg.job_timeout = Duration::from_secs(1); cfg.job_timeout = Duration::from_secs(1);
let events = Arc::new(crate::analysis::events::AnalysisEvents::new()); let events = Arc::new(crate::analysis::events::AnalysisEvents::new());
let handle = spawn_analysis_daemon(pool.clone(), storage, &cfg, events) let spawned = spawn_analysis_daemon(pool.clone(), storage, &cfg, events).await;
.await assert!(
.expect("spawn"); spawned.is_err(),
// Immediately shut down — we're only here to prove reclaim ran. "engine build must fail with a missing model path"
// The workers may briefly pick up the now-pending row; that's );
// fine, but we cancel before any real dispatch (the LocalStorage
// key doesn't exist, so a dispatch would fail anyway).
handle.shutdown().await;
// The reclaim must have moved the row back to pending with the // The reclaim must have moved the row back to pending with the
// attempt refunded (attempts goes from 1 → 0). Reaching it on the // attempt refunded (attempts goes from 1 → 0). Reaching it on the

View File

@@ -262,6 +262,24 @@ impl Default for AnalysisConfig {
} }
impl AnalysisConfig { impl AnalysisConfig {
/// The backend the worker actually dispatches through.
///
/// Vision is **temporarily disabled**: the engine code (`analysis::vision`,
/// `RealAnalyzeDispatcher`, the readiness probe) is kept intact but never
/// selected. Until it's re-enabled, this returns [`AnalysisBackend::Ocr`]
/// regardless of the parsed `backend`, logging a warning if `vision` was
/// requested so an env override isn't silently ignored. Re-enabling vision
/// is then a one-line change here (return `self.backend`).
pub fn effective_backend(&self) -> AnalysisBackend {
if self.backend == AnalysisBackend::Vision {
tracing::warn!(
"ANALYSIS_BACKEND=vision requested but the vision backend is temporarily \
disabled; running OCR instead"
);
}
AnalysisBackend::Ocr
}
pub fn from_env() -> Self { pub fn from_env() -> Self {
let d = AnalysisConfig::default(); let d = AnalysisConfig::default();
Self { Self {
@@ -840,6 +858,19 @@ mod tests {
assert_eq!(AnalysisConfig::from_env().backend, AnalysisBackend::Ocr); assert_eq!(AnalysisConfig::from_env().backend, AnalysisBackend::Ocr);
} }
#[test]
fn effective_backend_is_ocr_even_when_vision_requested() {
// Vision is temporarily disabled: the worker always runs OCR no matter
// what `ANALYSIS_BACKEND` parsed to. `backend` still reflects the raw
// request (so the override is visible/loggable), but `effective_backend`
// is the value the daemon actually dispatches through.
let mut cfg = AnalysisConfig::default();
cfg.backend = AnalysisBackend::Vision;
assert_eq!(cfg.effective_backend(), AnalysisBackend::Ocr);
cfg.backend = AnalysisBackend::Ocr;
assert_eq!(cfg.effective_backend(), AnalysisBackend::Ocr);
}
#[test] #[test]
fn ocr_model_paths_parse_from_env() { fn ocr_model_paths_parse_from_env() {
let _g = ENV_GUARD.lock().unwrap_or_else(|p| p.into_inner()); let _g = ENV_GUARD.lock().unwrap_or_else(|p| p.into_inner());

View File

@@ -25,7 +25,7 @@ use serde::{Deserialize, Serialize};
use crate::analysis::prompt::{ use crate::analysis::prompt::{
GROUNDING_PROMPT_DEFAULT, OCR_PROMPT_DEFAULT, SYSTEM_PROMPT_DEFAULT, GROUNDING_PROMPT_DEFAULT, OCR_PROMPT_DEFAULT, SYSTEM_PROMPT_DEFAULT,
}; };
use crate::config::{AnalysisConfig, CrawlerConfig, ResponseFormat}; use crate::config::{AnalysisBackend, AnalysisConfig, CrawlerConfig, ResponseFormat};
use crate::crawler::safety::DownloadAllowlist; use crate::crawler::safety::DownloadAllowlist;
/// `app_settings.key` for the crawler group. /// `app_settings.key` for the crawler group.
@@ -355,6 +355,15 @@ impl AnalysisSettings {
if self.workers < 1 { if self.workers < 1 {
errs.push("workers", "must be at least 1"); errs.push("workers", "must be at least 1");
} }
// The endpoint/model are vision-only knobs — the OCR backend never
// dials a URL or sends a model id. While vision is dormant
// (`base.backend == Ocr`), skip the live-worker SSRF gate and the
// model-required check so an OCR operator can enable the worker
// without a vision endpoint/model (the OCR settings UI doesn't even
// expose them). The basic malformed-URL sanity check below stays
// unconditional. Re-enabling vision restores the full gate via the
// `base.backend == Vision` predicate.
let vision_active = base.backend == AnalysisBackend::Vision;
let trimmed_endpoint = self.endpoint.trim(); let trimmed_endpoint = self.endpoint.trim();
if trimmed_endpoint.is_empty() { if trimmed_endpoint.is_empty() {
errs.push("endpoint", "must be a valid absolute URL"); errs.push("endpoint", "must be a valid absolute URL");
@@ -362,7 +371,7 @@ impl AnalysisSettings {
// Always reject obviously-malformed URLs (matches prior behaviour). // Always reject obviously-malformed URLs (matches prior behaviour).
let _ = e; let _ = e;
errs.push("endpoint", "must be a valid absolute URL"); errs.push("endpoint", "must be a valid absolute URL");
} else if self.enabled { } else if self.enabled && vision_active {
// SSRF + API-key exfiltration defence — only enforced when the // SSRF + API-key exfiltration defence — only enforced when the
// worker is actually live. Worker attaches an env-managed bearer // worker is actually live. Worker attaches an env-managed bearer
// token to every call; without this check, an admin (or one // token to every call; without this check, an admin (or one
@@ -379,7 +388,7 @@ impl AnalysisSettings {
errs.push("endpoint", url_safety_message(&e)); errs.push("endpoint", url_safety_message(&e));
} }
} }
if self.enabled && self.model.trim().is_empty() { if self.enabled && vision_active && self.model.trim().is_empty() {
errs.push("model", "required when analysis is enabled"); errs.push("model", "required when analysis is enabled");
} }
if self.max_tokens < 1 { if self.max_tokens < 1 {
@@ -701,7 +710,8 @@ mod tests {
// a hostile/CSRF-able admin must NOT be able to point endpoint at // a hostile/CSRF-able admin must NOT be able to point endpoint at
// cloud metadata, loopback services, or RFC1918 hosts — when the // cloud metadata, loopback services, or RFC1918 hosts — when the
// worker is enabled (toggling enabled=true later re-runs this gate). // worker is enabled (toggling enabled=true later re-runs this gate).
let base = AnalysisConfig::default(); let mut base = AnalysisConfig::default();
base.backend = AnalysisBackend::Vision;
for url in [ for url in [
"http://169.254.169.254/v1/chat/completions", "http://169.254.169.254/v1/chat/completions",
"http://127.0.0.1:5432/", "http://127.0.0.1:5432/",
@@ -725,7 +735,8 @@ mod tests {
// The documented default — docker DNS name resolving to a private IP // The documented default — docker DNS name resolving to a private IP
// at runtime — must still validate, because the bearer recipient // at runtime — must still validate, because the bearer recipient
// identity is the operator-chosen hostname, not the underlying IP. // identity is the operator-chosen hostname, not the underlying IP.
let base = AnalysisConfig::default(); let mut base = AnalysisConfig::default();
base.backend = AnalysisBackend::Vision;
for url in [ for url in [
"http://mangalord-vision:8000/v1/chat/completions", "http://mangalord-vision:8000/v1/chat/completions",
"https://api.openai.com/v1/chat/completions", "https://api.openai.com/v1/chat/completions",
@@ -756,7 +767,10 @@ mod tests {
#[test] #[test]
fn analysis_requires_model_only_when_enabled() { fn analysis_requires_model_only_when_enabled() {
let base = AnalysisConfig::default(); // Vision base: the model id is required only when the vision worker
// is actually live (see the OCR carve-out below).
let mut base = AnalysisConfig::default();
base.backend = AnalysisBackend::Vision;
let disabled = AnalysisSettings { let disabled = AnalysisSettings {
enabled: false, enabled: false,
model: "".to_string(), model: "".to_string(),
@@ -772,6 +786,28 @@ mod tests {
assert!(errs.errors.iter().any(|e| e.field == "model")); assert!(errs.errors.iter().any(|e| e.field == "model"));
} }
#[test]
fn analysis_ocr_backend_enable_skips_vision_endpoint_and_model_validation() {
// With the OCR backend active the worker never dials the vision
// endpoint nor sends a model id, so enabling analysis must NOT
// validate those vision-only fields. The default base carries the
// dev-localhost endpoint (which the SSRF gate would otherwise reject)
// and an empty model — under OCR, enabling is still valid. Without
// this carve-out an OCR operator can't turn the worker on, since the
// OCR settings UI doesn't expose endpoint/model to fix.
let base = AnalysisConfig::default(); // backend = Ocr
let dto = AnalysisSettings {
enabled: true,
endpoint: "http://localhost:8000/v1/chat/completions".to_string(),
model: "".to_string(),
..AnalysisSettings::from_config(&base)
};
assert!(
dto.to_config(&base).is_ok(),
"OCR-enabled config must not fail on vision endpoint/model"
);
}
#[test] #[test]
fn dtos_serialize_to_json_and_back() { fn dtos_serialize_to_json_and_back() {
let c = CrawlerSettings::default(); let c = CrawlerSettings::default();

View File

@@ -21,7 +21,6 @@
let total = $state(0); let total = $state(0);
let page = $state(1); let page = $state(1);
let statusFilter = $state<'' | 'done' | 'failed'>(''); let statusFilter = $state<'' | 'done' | 'failed'>('');
let nsfwOnly = $state(false);
let search = $state(''); let search = $state('');
let loading = $state(true); let loading = $state(true);
let error = $state<string | null>(null); let error = $state<string | null>(null);
@@ -34,7 +33,6 @@
try { try {
const resp = await listAnalysisHistory({ const resp = await listAnalysisHistory({
status: statusFilter || undefined, status: statusFilter || undefined,
nsfw: nsfwOnly,
search: search.trim() || undefined, search: search.trim() || undefined,
limit: LIMIT, limit: LIMIT,
offset: (page - 1) * LIMIT offset: (page - 1) * LIMIT
@@ -86,15 +84,6 @@
<option value="done">done</option> <option value="done">done</option>
<option value="failed">failed</option> <option value="failed">failed</option>
</select> </select>
<label class="nsfw">
<input
type="checkbox"
bind:checked={nsfwOnly}
onchange={applyFilters}
data-testid="analysis-history-nsfw"
/>
NSFW only
</label>
<input <input
class="search" class="search"
type="text" type="text"
@@ -136,7 +125,6 @@
<td><span class="status-pill {r.status}">{r.status}</span></td> <td><span class="status-pill {r.status}">{r.status}</span></td>
<td> <td>
{r.manga_title} · Ch {r.chapter_number} · p{r.page_number} {r.manga_title} · Ch {r.chapter_number} · p{r.page_number}
{#if r.is_nsfw}<span class="nsfw-tag">⚠ NSFW</span>{/if}
</td> </td>
<td class="muted">{r.model ?? '—'}</td> <td class="muted">{r.model ?? '—'}</td>
<td class="num">{fmtDuration(r.duration_ms)}</td> <td class="num">{fmtDuration(r.duration_ms)}</td>
@@ -169,13 +157,6 @@
color: var(--text); color: var(--text);
font-size: var(--font-sm); font-size: var(--font-sm);
} }
.nsfw {
display: inline-flex;
align-items: center;
gap: var(--space-1);
font-size: var(--font-sm);
color: var(--text-muted);
}
.muted { .muted {
color: var(--text-muted); color: var(--text-muted);
} }
@@ -218,11 +199,6 @@
background: color-mix(in srgb, var(--danger) 16%, transparent); background: color-mix(in srgb, var(--danger) 16%, transparent);
color: var(--danger); color: var(--danger);
} }
.nsfw-tag {
font-size: var(--font-xs);
color: #b85e1a;
margin-left: var(--space-1);
}
.error { .error {
color: var(--danger); color: var(--danger);
} }

View File

@@ -0,0 +1,64 @@
import { describe, it, expect, vi, afterEach } from 'vitest';
import { render, screen, cleanup, waitFor } from '@testing-library/svelte';
// Mock the admin API so the component's onMount load() is deterministic.
const listAnalysisHistory = vi.fn();
vi.mock('$lib/api/admin', () => ({
listAnalysisHistory: (...args: unknown[]) => listAnalysisHistory(...args)
}));
import AnalysisHistoryTable from './AnalysisHistoryTable.svelte';
afterEach(() => {
cleanup();
listAnalysisHistory.mockReset();
});
function row(over: Record<string, unknown> = {}) {
return {
page_id: 'p1',
page_number: 1,
chapter_id: 'c1',
chapter_number: 5,
manga_id: 'm1',
manga_title: 'Berserk',
status: 'done',
is_nsfw: true, // even when the backend reports nsfw, the OCR UI must not surface it
model: 'ocrs',
error: null,
analyzed_at: '2026-01-02T00:00:00Z',
duration_ms: 1234,
...over
};
}
function resolveOnce(rows: ReturnType<typeof row>[]) {
listAnalysisHistory.mockResolvedValue({
items: rows,
page: { total: rows.length }
});
}
describe('AnalysisHistoryTable (OCR)', () => {
it('does not render the NSFW-only filter', async () => {
resolveOnce([row()]);
render(AnalysisHistoryTable, { props: { onOpenDetail: () => {} } });
await waitFor(() => expect(listAnalysisHistory).toHaveBeenCalled());
expect(screen.queryByTestId('analysis-history-nsfw')).toBeNull();
});
it('does not pass an nsfw filter to the history query', async () => {
resolveOnce([row()]);
render(AnalysisHistoryTable, { props: { onOpenDetail: () => {} } });
await waitFor(() => expect(listAnalysisHistory).toHaveBeenCalled());
const arg = listAnalysisHistory.mock.calls[0][0] as Record<string, unknown>;
expect('nsfw' in arg).toBe(false);
});
it('never shows an NSFW tag on a row, even if the row is flagged', async () => {
resolveOnce([row({ is_nsfw: true })]);
render(AnalysisHistoryTable, { props: { onOpenDetail: () => {} } });
await waitFor(() => expect(screen.getByText(/Berserk/)).toBeTruthy());
expect(screen.queryByText(/NSFW/i)).toBeNull();
});
});

View File

@@ -105,30 +105,6 @@
<span class="value">{metrics.failed}</span> <span class="value">{metrics.failed}</span>
</div> </div>
</div> </div>
<h2>By model</h2>
{#if metrics.by_model.length === 0}
<p class="muted">No completed analyses in this window.</p>
{:else}
<table>
<thead>
<tr>
<th>Model</th>
<th class="num">Avg</th>
<th class="num">N</th>
</tr>
</thead>
<tbody>
{#each metrics.by_model as m (m.model)}
<tr>
<td>{m.model ?? '(unknown)'}</td>
<td class="num">{fmtDuration(m.avg_ms)}</td>
<td class="num">{m.n}</td>
</tr>
{/each}
</tbody>
</table>
{/if}
{/if} {/if}
</section> </section>
@@ -186,29 +162,6 @@
font-weight: var(--weight-semibold); font-weight: var(--weight-semibold);
font-variant-numeric: tabular-nums; font-variant-numeric: tabular-nums;
} }
h2 {
margin: 0 0 var(--space-2);
font-size: var(--font-sm);
color: var(--text-muted);
text-transform: uppercase;
letter-spacing: 0.04em;
}
table {
width: 100%;
border-collapse: collapse;
max-width: 32rem;
}
th,
td {
padding: var(--space-2);
text-align: left;
border-bottom: 1px solid var(--border);
font-size: var(--font-sm);
}
.num {
text-align: right;
font-variant-numeric: tabular-nums;
}
.muted { .muted {
color: var(--text-muted); color: var(--text-muted);
} }

View File

@@ -0,0 +1,40 @@
import { describe, it, expect, vi, afterEach } from 'vitest';
import { render, screen, cleanup, waitFor } from '@testing-library/svelte';
const getAnalysisMetrics = vi.fn();
const getAnalysisMetricsSeries = vi.fn();
vi.mock('$lib/api/admin', () => ({
getAnalysisMetrics: (...a: unknown[]) => getAnalysisMetrics(...a),
getAnalysisMetricsSeries: (...a: unknown[]) => getAnalysisMetricsSeries(...a)
}));
import AnalysisMetricsPanel from './AnalysisMetricsPanel.svelte';
afterEach(() => {
cleanup();
getAnalysisMetrics.mockReset();
getAnalysisMetricsSeries.mockReset();
});
describe('AnalysisMetricsPanel (OCR)', () => {
it('renders the aggregate tiles but not the by-model table', async () => {
getAnalysisMetrics.mockResolvedValue({
n: 10,
ok: 9,
failed: 1,
avg_ms: 1200,
// Even when the API returns a by-model breakdown, OCR has a single
// engine so the panel must not surface the table.
by_model: [{ model: 'ocrs', avg_ms: 1200, n: 10 }]
});
getAnalysisMetricsSeries.mockResolvedValue({ buckets: [] });
render(AnalysisMetricsPanel);
await waitFor(() =>
expect(screen.getByTestId('analysis-metrics-n').textContent).toContain('10')
);
expect(screen.queryByText(/By model/i)).toBeNull();
expect(screen.queryByRole('table')).toBeNull();
});
});

View File

@@ -533,9 +533,9 @@
</span> </span>
</div> </div>
<p class="lede"> <p class="lede">
Coverage of the AI analysis worker (OCR, auto-tags, scene description, Coverage of the OCR text-extraction worker. Browse which pages have had
NSFW moderation). Browse what's analyzed, queue more, and inspect any their text extracted, queue more, and inspect any page's OCR result.
page's result. Updates stream live as pages are queued and processed. Updates stream live as pages are queued and processed.
</p> </p>
<div class="viewtabs"> <div class="viewtabs">
@@ -818,37 +818,12 @@
<p class="error">{detail.error ?? 'Analysis failed.'}</p> <p class="error">{detail.error ?? 'Analysis failed.'}</p>
{/if} {/if}
{#if detail.is_nsfw || detail.content_warnings.length > 0}
<div class="warnings" data-testid="admin-analysis-detail-warnings">
<span class="cw-label"> NSFW</span>
{#each detail.content_warnings as w (w)}
<span class="cw-chip">{w}</span>
{/each}
</div>
{/if}
{#if detail.scene_description}
<section class="block">
<h3>Scene</h3>
<p>{detail.scene_description}</p>
</section>
{/if}
{#if detail.tags.length > 0}
<section class="block">
<h3>Tags</h3>
<div class="tag-row">
{#each detail.tags as t (t)}<span class="tag">{t}</span>{/each}
</div>
</section>
{/if}
{#if detail.ocr.length > 0} {#if detail.ocr.length > 0}
<section class="block"> <section class="block">
<h3>OCR text</h3> <h3>OCR text</h3>
<ul class="ocr"> <ul class="ocr">
{#each detail.ocr as line, i (i)} {#each detail.ocr as line, i (i)}
<li><span class="ocr-kind">{line.kind}</span> {line.text}</li> <li>{#if line.kind}<span class="ocr-kind">{line.kind}</span> {/if}{line.text}</li>
{/each} {/each}
</ul> </ul>
</section> </section>
@@ -1186,28 +1161,6 @@
background: color-mix(in srgb, var(--danger) 16%, transparent); background: color-mix(in srgb, var(--danger) 16%, transparent);
color: var(--danger); color: var(--danger);
} }
.warnings {
display: flex;
flex-wrap: wrap;
align-items: center;
gap: var(--space-2);
padding: var(--space-2) var(--space-3);
border-radius: var(--radius-md);
background: color-mix(in srgb, #d4762a 12%, transparent);
margin-bottom: var(--space-3);
}
.cw-label {
font-weight: 600;
color: #b85e1a;
font-size: var(--font-sm);
}
.cw-chip {
font-size: var(--font-sm);
padding: 1px var(--space-2);
border-radius: var(--radius-sm);
background: color-mix(in srgb, #d4762a 22%, transparent);
text-transform: capitalize;
}
.block { .block {
margin-bottom: var(--space-3); margin-bottom: var(--space-3);
} }
@@ -1218,17 +1171,6 @@
letter-spacing: 0.04em; letter-spacing: 0.04em;
margin: 0 0 var(--space-1); margin: 0 0 var(--space-1);
} }
.tag-row {
display: flex;
flex-wrap: wrap;
gap: var(--space-1);
}
.tag {
font-size: var(--font-sm);
padding: 1px var(--space-2);
border-radius: var(--radius-pill);
background: var(--surface-elevated);
}
.ocr { .ocr {
list-style: none; list-style: none;
margin: 0; margin: 0;

View File

@@ -9,8 +9,7 @@
updateAnalysisSettings, updateAnalysisSettings,
type CrawlerSettings, type CrawlerSettings,
type CrawlerEnvOnly, type CrawlerEnvOnly,
type AnalysisSettings, type AnalysisSettings
type PromptDefaults
} from '$lib/api/admin'; } from '$lib/api/admin';
type Tab = 'crawler' | 'analysis'; type Tab = 'crawler' | 'analysis';
@@ -21,10 +20,11 @@
let crawlerEnv = $state<CrawlerEnvOnly | null>(null); let crawlerEnv = $state<CrawlerEnvOnly | null>(null);
let allowlistText = $state(''); // textarea mirror of download_allowlist let allowlistText = $state(''); // textarea mirror of download_allowlist
// Analysis state // Analysis state. The full settings object is loaded and saved as-is;
// vision-only fields (endpoint, model, prompts, slicing, sampling) ride
// along unchanged while the OCR backend is active — only the OCR-relevant
// knobs are surfaced below.
let analysis = $state<AnalysisSettings | null>(null); let analysis = $state<AnalysisSettings | null>(null);
let promptDefaults = $state<PromptDefaults | null>(null);
let apiKeyConfigured = $state(false);
let loading = $state(true); let loading = $state(true);
let loadError = $state<string | null>(null); let loadError = $state<string | null>(null);
@@ -46,8 +46,6 @@
crawlerEnv = c.env_only; crawlerEnv = c.env_only;
allowlistText = c.editable.download_allowlist.join('\n'); allowlistText = c.editable.download_allowlist.join('\n');
analysis = a.editable; analysis = a.editable;
promptDefaults = a.prompt_defaults;
apiKeyConfigured = a.env_only.api_key_configured;
} catch (e) { } catch (e) {
loadError = e instanceof Error ? e.message : 'Failed to load settings.'; loadError = e instanceof Error ? e.message : 'Failed to load settings.';
} finally { } finally {
@@ -96,8 +94,6 @@
} else if (tab === 'analysis' && analysis) { } else if (tab === 'analysis' && analysis) {
const r = await updateAnalysisSettings(analysis); const r = await updateAnalysisSettings(analysis);
analysis = r.editable; analysis = r.editable;
promptDefaults = r.prompt_defaults;
apiKeyConfigured = r.env_only.api_key_configured;
} }
success = 'Saved and applied. The subsystem was restarted with the new settings.'; success = 'Saved and applied. The subsystem was restarted with the new settings.';
} catch (e) { } catch (e) {
@@ -113,9 +109,6 @@
} }
} }
function resetPrompt(which: 'system_prompt' | 'ocr_prompt' | 'grounding_prompt') {
if (analysis) analysis[which] = null;
}
</script> </script>
<h1>Settings</h1> <h1>Settings</h1>
@@ -322,145 +315,29 @@
{:else if tab === 'analysis' && analysis} {:else if tab === 'analysis' && analysis}
<div class="form"> <div class="form">
<fieldset> <fieldset>
<legend>Endpoint &amp; model</legend> <legend>Engine</legend>
<label class="field">
<span>Vision endpoint URL</span>
<input type="text" bind:value={analysis.endpoint} />
{#if fieldErrors.endpoint}<small class="fe">{fieldErrors.endpoint}</small>{/if}
</label>
<label class="field">
<span>Model</span>
<input type="text" bind:value={analysis.model} />
{#if fieldErrors.model}<small class="fe">{fieldErrors.model}</small>{/if}
</label>
<p class="env-note muted"> <p class="env-note muted">
API key: {apiKeyConfigured Backend: OCR (ocrs) — in-process text extraction. Selected via
? 'configured via environment' environment; vision is temporarily disabled.
: 'not set'} (managed via environment).
</p> </p>
</fieldset> </fieldset>
<fieldset> <fieldset>
<legend>Workers &amp; timeouts</legend> <legend>Worker</legend>
<label class="field"> <label class="field">
<span>Workers</span> <span>Workers</span>
<input type="number" min="1" bind:value={analysis.workers} /> <input type="number" min="1" bind:value={analysis.workers} />
{#if fieldErrors.workers}<small class="fe">{fieldErrors.workers}</small>{/if} {#if fieldErrors.workers}<small class="fe">{fieldErrors.workers}</small>{/if}
</label> </label>
<label class="field">
<span>Request timeout (seconds)</span>
<input type="number" min="1" bind:value={analysis.request_timeout_secs} />
</label>
<label class="field"> <label class="field">
<span>Job timeout (seconds)</span> <span>Job timeout (seconds)</span>
<input type="number" min="1" bind:value={analysis.job_timeout_secs} /> <input type="number" min="1" bind:value={analysis.job_timeout_secs} />
</label> </label>
</fieldset>
<fieldset>
<legend>Sampling &amp; output</legend>
<label class="field">
<span>Temperature</span>
<input type="number" min="0" step="0.05" bind:value={analysis.temperature} />
{#if fieldErrors.temperature}<small class="fe"
>{fieldErrors.temperature}</small
>{/if}
</label>
<label class="field">
<span>Frequency penalty</span>
<input type="number" step="0.05" bind:value={analysis.frequency_penalty} />
</label>
<label class="field">
<span>Max output tokens</span>
<input type="number" min="1" bind:value={analysis.max_tokens} />
</label>
<label class="field">
<span>Response format</span>
<select bind:value={analysis.response_format}>
<option value="json_schema">json_schema</option>
<option value="json_object">json_object</option>
<option value="none">none</option>
</select>
</label>
</fieldset>
<fieldset>
<legend>Image slicing</legend>
<label class="field">
<span>Max pixels per slice</span>
<input type="number" min="1" bind:value={analysis.max_pixels} />
</label>
<label class="field">
<span>Min slice height (px)</span>
<input type="number" min="1" bind:value={analysis.min_slice_height} />
</label>
<label class="field">
<span>Slice overlap (00.9)</span>
<input
type="number"
min="0"
max="0.9"
step="0.01"
bind:value={analysis.slice_overlap}
/>
{#if fieldErrors.slice_overlap}<small class="fe"
>{fieldErrors.slice_overlap}</small
>{/if}
</label>
<label class="field">
<span>Tall aspect threshold (≥1.0)</span>
<input
type="number"
min="1"
step="0.1"
bind:value={analysis.tall_aspect_threshold}
/>
{#if fieldErrors.tall_aspect_threshold}<small class="fe"
>{fieldErrors.tall_aspect_threshold}</small
>{/if}
</label>
<label class="field">
<span>Max slices</span>
<input type="number" min="1" bind:value={analysis.max_slices} />
</label>
<label class="field"> <label class="field">
<span>Max image bytes</span> <span>Max image bytes</span>
<input type="number" min="0" bind:value={analysis.max_image_bytes} /> <input type="number" min="0" bind:value={analysis.max_image_bytes} />
</label> </label>
</fieldset> </fieldset>
<fieldset>
<legend>Prompts</legend>
{#each [['system_prompt', 'System prompt'], ['ocr_prompt', 'OCR prompt (slices)'], ['grounding_prompt', 'Grounding prompt']] as [key, label] (key)}
{@const k = key as 'system_prompt' | 'ocr_prompt' | 'grounding_prompt'}
<div class="prompt">
<div class="prompt-head">
<span>{label}</span>
<button
type="button"
class="link"
disabled={analysis[k] == null}
onclick={() => resetPrompt(k)}>Reset to default</button
>
</div>
<textarea
rows="6"
class="mono"
value={analysis[k] ?? ''}
placeholder={promptDefaults?.[k] ?? ''}
oninput={(e) => {
const v = (e.currentTarget as HTMLTextAreaElement).value;
if (analysis) analysis[k] = v.trim() === '' ? null : v;
}}
></textarea>
<small class="muted">
{analysis[k] == null
? 'Using compiled default (shown as placeholder).'
: `${analysis[k]?.length ?? 0} chars (override).`}
</small>
</div>
{/each}
</fieldset>
</div> </div>
{/if} {/if}
{/if} {/if}
@@ -564,7 +441,6 @@
color: var(--text-muted); color: var(--text-muted);
} }
.field input, .field input,
.field select,
.field textarea { .field textarea {
padding: var(--space-2) var(--space-3); padding: var(--space-2) var(--space-3);
border: 1px solid var(--border); border: 1px solid var(--border);
@@ -584,39 +460,6 @@
color: var(--danger, #dc2626); color: var(--danger, #dc2626);
font-size: var(--font-xs); font-size: var(--font-xs);
} }
.mono {
font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
font-size: var(--font-xs);
width: 100%;
box-sizing: border-box;
padding: var(--space-2);
border: 1px solid var(--border);
border-radius: var(--radius-md);
background: var(--surface);
color: var(--text);
}
.prompt {
display: grid;
gap: var(--space-1);
}
.prompt-head {
display: flex;
justify-content: space-between;
align-items: center;
font-size: var(--font-sm);
}
button.link {
background: none;
border: none;
color: var(--primary);
cursor: pointer;
font-size: var(--font-xs);
padding: 0;
}
button.link:disabled {
color: var(--text-muted);
cursor: not-allowed;
}
.env dl { .env dl {
display: grid; display: grid;
grid-template-columns: max-content 1fr; grid-template-columns: max-content 1fr;