8d34132883
Five fixes bundled into one release: - preserve user-attached tags across crawler upserts (repo::crawler::sync_tags now scopes to added_by IS NULL; orphaned attachments from deleted users are reaped as crawler-owned) - gate manga PATCH and cover endpoints on uploaded_by (require_can_edit in api::mangas; non-NULL uploaded_by must match the caller) - equalise login response time across user-existence branches (run argon2 against a OnceLock-cached dummy hash on the no-user branch so timing doesn't leak username existence) - crawler download defences (SSRF allowlist of host literals including IPv4-mapped IPv6 ranges, 32 MiB streamed size cap, reject non-whitelisted image types, three-way chapter-probe classifier replaces the binary #avatar_menu check) - tighten validation and clean up dead unload path (attach_tag + create_token enforce 64-char caps; LocalStorage rejects NUL bytes explicitly; reader flushFinalProgress drops the always-405 sendBeacon path) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
29 lines
934 B
Rust
29 lines
934 B
Rust
//! Crawler subsystem.
|
|
//!
|
|
//! Runs as its own binary (`src/bin/crawler.rs`) and shares `domain`,
|
|
//! `repo`, and `storage` with the API binary. Layering mirrors the
|
|
//! `Storage` trait pattern: callers depend on the `source::Source`
|
|
//! trait, not on a concrete site; new sites plug in as additional
|
|
//! impls without touching the job runner.
|
|
//!
|
|
//! Submodules:
|
|
//! - [`browser`]: launches and pools Chromium via `chromiumoxide`.
|
|
//! First run downloads a known-good build via the `fetcher` feature.
|
|
//! - [`source`]: the `Source` trait. Per-site impls live alongside it.
|
|
//! - [`jobs`]: job kinds, queue wrapper, handler dispatch.
|
|
//! - [`diff`]: change detection — new / updated / dropped semantics.
|
|
|
|
pub mod browser;
|
|
pub mod browser_manager;
|
|
pub mod content;
|
|
pub mod daemon;
|
|
pub mod detect;
|
|
pub mod diff;
|
|
pub mod jobs;
|
|
pub mod pipeline;
|
|
pub mod rate_limit;
|
|
pub mod safety;
|
|
pub mod session;
|
|
pub mod source;
|
|
pub mod url_utils;
|