Admin-only crawler dashboard backed by an SSE live-status stream,
coordinated browser restart, runtime PHPSESSID refresh, dead-letter
requeue, and a batch of reliability fixes. Closes everything from
the two-pass audit (10 commits' worth) and bumps 0.52.0 -> 0.55.0.
Backend:
- New /admin/crawler/* surface (cookie-auth, RequireAdmin) split
into status / control / dead_jobs / backlog modules. SSE stream
composes in-memory status with DB-derived queue counts, memoizes
the counts for 1s and debounces watch pokes for 250ms (~10x QPS
reduction per subscriber). One-shot GET /admin/crawler shares the
same compose path.
- POST /admin/crawler/run gated by manual_pass_lock try_lock_owned
(409 Conflict on overlapping click); browser restart goes through
the coordinated_restart gate (drain + relaunch + auto-clear of the
sticky session_expired flag on Ok).
- Runtime PHPSESSID refresh via SessionController (allow-list
validation, never logged, audit row carries SHA-256 fingerprint).
Storage layer is repo::crawler::runtime_session_{load,persist}.
- Dead-letter requeue with four scopes (all/manga/chapter/job);
scope=all requires confirm:true; DISTINCT ON dedup keeps the
partial unique index from rejecting requeues for chapters with
multiple dead rows. SQL is four &'static str constants per scope.
- StatusHandle + ChapterGuard / CoverGuard RAII model survives
panics; last-writer-wins on cover so concurrent dispatches don't
clobber each other's slot. Pure functions (should_stop /
should_mark_clean_exit / should_abort_pass) with named regression
tests.
- Reliability bundle: per-lease heartbeat, jitter on retries,
per-job timeout, circuit breaker on consecutive failures, BrowserManager
coordinated restart gate, request fingerprint changes.
- Streaming page download: Storage::put_stream trait method,
LocalStorage impl atomic via temp + fsync + UUID-suffixed rename.
Pages stream through with peak memory ~one HTTP chunk + 64-byte
sniff prefix instead of one full image per dispatch.
- New partial indexes (migration 0022): mangas_missing_cover_idx
and crawler_jobs_dead_idx, both ordered by updated_at DESC to
match the dashboard's LIMIT/OFFSET reads.
- Security hardening: admin_csrf_guard (Origin/Referer allowlist
on /admin/* mutations, opt-in via ADMIN_ALLOWED_ORIGINS),
admin_no_store_guard (Cache-Control: no-store on admin
responses), audit rows carry per-scope target_id.
Frontend:
- /admin/crawler page decomposed into lib/components/crawler/
(11 components: ProgressBar, SearchBar, CrawlerHero,
CrawlerControls, ActiveChaptersCard, ActiveJobsTable,
MissingCoversTable, DeadJobsTable, RestartConfirmModal,
RequeueAllConfirmModal, SessionModal). Page is 532 LOC of
orchestration; each component 22-148 LOC.
- EventSource lifecycle wired to visibilitychange / pagehide /
pageshow (BFCache); after 5 consecutive errors probes the status
endpoint so a 401 routes through the global on401Hook instead of
infinite silent reconnects.
- Backlog $effect refetches debounced 500ms with per-loader
AbortControllers; refresh after a control action only runs when
the SSE stream is dead.
- Inline requeue button on /admin/mangas patches the affected row's
sync_state locally (no full chapter-list refetch); proper
aria-label. Requeue-all gets its own confirm modal; both confirm
modals autofocus Cancel.
- SvelteKit reverse proxy bypasses its 5-minute AbortController
for Accept: text/event-stream; pure shouldBypassProxyTimeout
helper covered by unit tests.
Config / docs:
- New env vars (.env.example): ADMIN_ALLOWED_ORIGINS,
CRAWLER_JOB_TIMEOUT_SECS, CRAWLER_METADATA_MAX_CONSECUTIVE_FAILURES,
CRAWLER_BROWSER_RESTART_THRESHOLD.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
261 lines
9.2 KiB
Rust
261 lines
9.2 KiB
Rust
//! PR 1 (feat/admin-role) integration tests.
|
|
//!
|
|
//! Covers: `bootstrap_admin` semantics, `is_admin` exposed on /auth/me,
|
|
//! and the `RequireAdmin` extractor's 401/403/200 matrix — including the
|
|
//! load-bearing decision that Bearer-authed callers can NEVER reach an
|
|
//! admin-guarded route, even when the underlying user IS admin.
|
|
|
|
mod common;
|
|
|
|
use std::sync::Arc;
|
|
|
|
use axum::http::StatusCode;
|
|
use axum::routing::get;
|
|
use axum::{Json, Router};
|
|
use serde_json::json;
|
|
use sqlx::PgPool;
|
|
use tempfile::TempDir;
|
|
use tower::ServiceExt;
|
|
|
|
use mangalord::api;
|
|
use mangalord::app::AppState;
|
|
use mangalord::auth::extractor::RequireAdmin;
|
|
use mangalord::auth::rate_limit::AuthRateLimiter;
|
|
use mangalord::config::{AuthConfig, UploadConfig};
|
|
use mangalord::repo;
|
|
use mangalord::storage::{LocalStorage, Storage};
|
|
|
|
/// Test-only handler guarded by `RequireAdmin`. Lets the test suite assert
|
|
/// the extractor's behaviour end-to-end without depending on an admin
|
|
/// endpoint existing yet (those land in PR 2+).
|
|
async fn admin_only_handler(RequireAdmin(user): RequireAdmin) -> Json<serde_json::Value> {
|
|
Json(json!({ "username": user.username, "is_admin": user.is_admin }))
|
|
}
|
|
|
|
/// Build a router that exposes the production /api/v1/* AND a test-only
|
|
/// `/_test/admin_only` route guarded by `RequireAdmin`. Pool is consumed;
|
|
/// callers that want to inspect the DB after a request should clone it.
|
|
fn admin_test_router(pool: PgPool) -> (Router, TempDir) {
|
|
let storage_dir = tempfile::tempdir().expect("tempdir");
|
|
let storage: Arc<dyn Storage> = Arc::new(LocalStorage::new(storage_dir.path()));
|
|
let auth = AuthConfig {
|
|
cookie_secure: false,
|
|
..AuthConfig::default()
|
|
};
|
|
let auth_limiter = Arc::new(AuthRateLimiter::new(auth.rate_limit));
|
|
let state = AppState {
|
|
db: pool,
|
|
storage,
|
|
auth,
|
|
upload: UploadConfig::default(),
|
|
auth_limiter,
|
|
resync: None,
|
|
crawler: None,
|
|
admin_allowed_origins: Arc::new(Vec::new()),
|
|
};
|
|
let app = Router::new()
|
|
.nest("/api/v1", api::routes())
|
|
.route("/_test/admin_only", get(admin_only_handler))
|
|
.with_state(state);
|
|
(app, storage_dir)
|
|
}
|
|
|
|
// ---- bootstrap_admin -------------------------------------------------------
|
|
|
|
#[sqlx::test(migrations = "./migrations")]
|
|
async fn bootstrap_creates_admin_when_user_missing(pool: PgPool) {
|
|
repo::user::bootstrap_admin(&pool, "root", "hunter2hunter2")
|
|
.await
|
|
.expect("bootstrap on empty DB");
|
|
|
|
let user = repo::user::find_by_username(&pool, "root")
|
|
.await
|
|
.unwrap()
|
|
.expect("root user exists after bootstrap");
|
|
assert!(user.is_admin, "bootstrap must set is_admin = true on creation");
|
|
|
|
// Password hash must verify the env-supplied password (and not be empty).
|
|
assert!(
|
|
mangalord::auth::password::verify_password("hunter2hunter2", &user.password_hash),
|
|
"bootstrap-created user must accept the env-supplied password"
|
|
);
|
|
}
|
|
|
|
#[sqlx::test(migrations = "./migrations")]
|
|
async fn bootstrap_promotes_existing_user_without_touching_password(pool: PgPool) {
|
|
// Pre-existing user, not admin. Use the real register path so the
|
|
// hash format matches production exactly.
|
|
let (app, _td) = admin_test_router(pool.clone());
|
|
let resp = app
|
|
.oneshot(common::post_json(
|
|
"/api/v1/auth/register",
|
|
json!({ "username": "preexisting", "password": "originalpw1234" }),
|
|
))
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), StatusCode::CREATED);
|
|
|
|
let before = repo::user::find_by_username(&pool, "preexisting")
|
|
.await
|
|
.unwrap()
|
|
.unwrap();
|
|
assert!(!before.is_admin);
|
|
let original_hash = before.password_hash.clone();
|
|
|
|
// Bootstrap with a DIFFERENT password — must not overwrite the hash.
|
|
repo::user::bootstrap_admin(&pool, "preexisting", "envpw_should_be_ignored")
|
|
.await
|
|
.expect("bootstrap on existing user");
|
|
|
|
let after = repo::user::find_by_username(&pool, "preexisting")
|
|
.await
|
|
.unwrap()
|
|
.unwrap();
|
|
assert!(after.is_admin, "bootstrap must promote existing user");
|
|
assert_eq!(
|
|
after.password_hash, original_hash,
|
|
"bootstrap must NOT overwrite the existing password hash"
|
|
);
|
|
assert!(
|
|
mangalord::auth::password::verify_password("originalpw1234", &after.password_hash),
|
|
"original password must still verify after bootstrap"
|
|
);
|
|
}
|
|
|
|
#[sqlx::test(migrations = "./migrations")]
|
|
async fn bootstrap_is_idempotent(pool: PgPool) {
|
|
repo::user::bootstrap_admin(&pool, "root", "hunter2hunter2")
|
|
.await
|
|
.expect("first bootstrap");
|
|
repo::user::bootstrap_admin(&pool, "root", "hunter2hunter2")
|
|
.await
|
|
.expect("second bootstrap is no-op");
|
|
|
|
// Exactly one row, still admin.
|
|
let (count,): (i64,) = sqlx::query_as("SELECT COUNT(*) FROM users WHERE username = $1")
|
|
.bind("root")
|
|
.fetch_one(&pool)
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(count, 1);
|
|
}
|
|
|
|
// ---- /api/v1/auth/me exposes is_admin --------------------------------------
|
|
|
|
#[sqlx::test(migrations = "./migrations")]
|
|
async fn auth_me_response_includes_is_admin(pool: PgPool) {
|
|
let (app, _td) = admin_test_router(pool.clone());
|
|
let (_username, cookie) = common::register_user(&app).await;
|
|
let resp = app
|
|
.oneshot(common::get_with_cookie("/api/v1/auth/me", &cookie))
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), StatusCode::OK);
|
|
let body = common::body_json(resp).await;
|
|
assert_eq!(
|
|
body["user"]["is_admin"], false,
|
|
"freshly-registered users default to is_admin=false"
|
|
);
|
|
}
|
|
|
|
// ---- RequireAdmin: 401 / 403 / 200 matrix ----------------------------------
|
|
|
|
#[sqlx::test(migrations = "./migrations")]
|
|
async fn require_admin_rejects_unauthenticated(pool: PgPool) {
|
|
let (app, _td) = admin_test_router(pool);
|
|
let resp = app
|
|
.oneshot(common::get("/_test/admin_only"))
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
|
|
}
|
|
|
|
#[sqlx::test(migrations = "./migrations")]
|
|
async fn require_admin_rejects_non_admin_cookie(pool: PgPool) {
|
|
let (app, _td) = admin_test_router(pool);
|
|
let (_username, cookie) = common::register_user(&app).await;
|
|
let resp = app
|
|
.oneshot(common::get_with_cookie("/_test/admin_only", &cookie))
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), StatusCode::FORBIDDEN);
|
|
let body = common::body_json(resp).await;
|
|
assert_eq!(body["error"]["code"], "forbidden");
|
|
}
|
|
|
|
#[sqlx::test(migrations = "./migrations")]
|
|
async fn require_admin_accepts_admin_cookie(pool: PgPool) {
|
|
let (app, _td) = admin_test_router(pool.clone());
|
|
let (username, cookie) = common::register_user(&app).await;
|
|
// Promote via the repo (the admin-users API doesn't exist yet).
|
|
let u = repo::user::find_by_username(&pool, &username)
|
|
.await
|
|
.unwrap()
|
|
.unwrap();
|
|
repo::user::set_is_admin_unchecked(&pool, u.id, true).await.unwrap();
|
|
|
|
let resp = app
|
|
.oneshot(common::get_with_cookie("/_test/admin_only", &cookie))
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), StatusCode::OK);
|
|
let body = common::body_json(resp).await;
|
|
assert_eq!(body["username"], username);
|
|
assert_eq!(body["is_admin"], true);
|
|
}
|
|
|
|
#[sqlx::test(migrations = "./migrations")]
|
|
async fn require_admin_rejects_bearer_token_even_for_admin_user(pool: PgPool) {
|
|
// Key privilege-escalation test: an API token belonging to an admin user
|
|
// must NOT grant admin authority. Bot tokens are excluded from admin
|
|
// routes by design (the RequireAdmin extractor only accepts session
|
|
// cookies). See cross-cutting decision #1 in the PR plan.
|
|
let (app, _td) = admin_test_router(pool.clone());
|
|
let (username, cookie) = common::register_user(&app).await;
|
|
|
|
// Promote to admin and mint an API token (the existing /auth/tokens
|
|
// endpoint authenticates via the same cookie).
|
|
let u = repo::user::find_by_username(&pool, &username)
|
|
.await
|
|
.unwrap()
|
|
.unwrap();
|
|
repo::user::set_is_admin_unchecked(&pool, u.id, true).await.unwrap();
|
|
|
|
let resp = app
|
|
.clone()
|
|
.oneshot(common::post_json_with_cookie(
|
|
"/api/v1/auth/tokens",
|
|
json!({ "name": "test-bot" }),
|
|
&cookie,
|
|
))
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), StatusCode::CREATED);
|
|
let body = common::body_json(resp).await;
|
|
let token = body["bearer"]
|
|
.as_str()
|
|
.expect("raw bearer token in response")
|
|
.to_string();
|
|
|
|
// Sanity: the bearer DOES work on a non-admin endpoint (proves the
|
|
// token is valid, isolating the failure below to the admin guard).
|
|
let resp = app
|
|
.clone()
|
|
.oneshot(common::get_with_bearer("/api/v1/auth/me", &token))
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(resp.status(), StatusCode::OK);
|
|
|
|
// Same token, same admin user, but on the admin-guarded route → 401
|
|
// (no session cookie present at all from the extractor's POV).
|
|
let resp = app
|
|
.oneshot(common::get_with_bearer("/_test/admin_only", &token))
|
|
.await
|
|
.unwrap();
|
|
assert_eq!(
|
|
resp.status(),
|
|
StatusCode::UNAUTHORIZED,
|
|
"Bearer-authed admin must NOT pass the RequireAdmin guard"
|
|
);
|
|
}
|