fix(upload): remove the /original rate limit that would have broken the feed
The limiter added here was justified as bounding "100 guests occasionally tapping Original anzeigen". That is not what this route is. `pickMediaUrl` resolves to `preview_url ?? thumbnail_url ?? /original`, and a freshly committed upload has BOTH derivatives null until the compression worker reaches it — at COMPRESSION_WORKER_CONCURRENCY=2 that is minutes during a post-ceremony burst. So /original is the feed's hot path for exactly the newest photos, in a newest-first grid, at the busiest moment. With every guest behind one NAT the 600/min bucket is venue-wide: six new photos fanned out by `upload-new` to ~100 open feeds exhausts it, and then every original fetch from anyone 429s for the rest of the window. The tiles' own 4-second retry uses a fresh `?r=` nonce, so the clients hold the bucket saturated themselves — the whole venue watching the newest photos render as broken tiles while the projector skips slides. A per-IP bucket cannot separate one scraper from the entire party when they share an address, and these media routes are unauthenticated by design (an `<img>` cannot send a bearer token), so there is no per-user key to move to. Bandwidth abuse belongs at the proxy. Also here: the release/lock check order. `release ⇒ lock`, so testing the lock first made the `GalleryReleased` arm unreachable dead code and every post-release upload answered `uploads_locked`. The codes are not interchangeable to the client — `uploads_locked` charges a retry attempt and re-pushes the whole photo on the backoff ladder against an answer that cannot change, while `gallery_released` parks it and says the photo is safe but the hosts must reopen. Both sites now test release first, so the fast path and the commit-time re-check agree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -150,6 +150,49 @@ pub struct AppConfig {
|
|||||||
/// ramp in tailwind-theme.css). Kept here so an unset env still yields the current look.
|
/// ramp in tailwind-theme.css). Kept here so an unset env still yields the current look.
|
||||||
const DEFAULT_THEME_SEED: &str = "#8a6a2b";
|
const DEFAULT_THEME_SEED: &str = "#8a6a2b";
|
||||||
|
|
||||||
|
/// Upper bound on `SESSION_EXPIRY_DAYS`. ~10 years — absurdly generous for a one-evening event,
|
||||||
|
/// and low enough that `chrono::Duration::days` cannot overflow downstream.
|
||||||
|
const MAX_SESSION_EXPIRY_DAYS: i64 = 3650;
|
||||||
|
|
||||||
|
/// Parse and RANGE-CHECK `SESSION_EXPIRY_DAYS`. Refusing to boot is the whole point.
|
||||||
|
///
|
||||||
|
/// This was `.parse().context(...)` with no bounds, and both ends of the range were live faults
|
||||||
|
/// that a green health check hid completely (H7):
|
||||||
|
///
|
||||||
|
/// * A huge value made `chrono::Duration::days` PANIC on every `/join`, `/recover` and
|
||||||
|
/// `/admin/login`. There is no `CatchPanicLayer`, so the client got a connection reset with no
|
||||||
|
/// HTTP response at all — the app was up, healthy, and unable to authenticate anybody.
|
||||||
|
/// * Zero or negative created every session already-expired: `/join` returns 201 with a token,
|
||||||
|
/// and then every authenticated request 401s. A guest joins successfully and the app
|
||||||
|
/// immediately behaves as though they never did.
|
||||||
|
///
|
||||||
|
/// Both booted green because `/health` only probes the database. A bad value must stop the
|
||||||
|
/// container instead, where the operator sees it.
|
||||||
|
fn parse_session_expiry_days(raw: Option<&str>) -> Result<i64> {
|
||||||
|
let Some(raw) = raw else { return Ok(30) };
|
||||||
|
let trimmed = raw.trim();
|
||||||
|
if trimmed.is_empty() {
|
||||||
|
return Ok(30);
|
||||||
|
}
|
||||||
|
let days: i64 = trimmed
|
||||||
|
.parse()
|
||||||
|
.with_context(|| format!("SESSION_EXPIRY_DAYS must be a whole number (got {trimmed:?})"))?;
|
||||||
|
if days < 1 {
|
||||||
|
return Err(anyhow!(
|
||||||
|
"SESSION_EXPIRY_DAYS must be at least 1 (got {days}). Zero or negative makes every \
|
||||||
|
session expire the moment it is created: /join succeeds and every request after it \
|
||||||
|
returns 401."
|
||||||
|
));
|
||||||
|
}
|
||||||
|
if days > MAX_SESSION_EXPIRY_DAYS {
|
||||||
|
return Err(anyhow!(
|
||||||
|
"SESSION_EXPIRY_DAYS must be at most {MAX_SESSION_EXPIRY_DAYS} (got {days}). Larger \
|
||||||
|
values overflow the token-expiry arithmetic and panic on every auth request."
|
||||||
|
));
|
||||||
|
}
|
||||||
|
Ok(days)
|
||||||
|
}
|
||||||
|
|
||||||
impl AppConfig {
|
impl AppConfig {
|
||||||
pub fn from_env() -> Result<Self> {
|
pub fn from_env() -> Result<Self> {
|
||||||
let app_env = std::env::var("APP_ENV").unwrap_or_else(|_| "development".to_string());
|
let app_env = std::env::var("APP_ENV").unwrap_or_else(|_| "development".to_string());
|
||||||
@@ -164,10 +207,9 @@ impl AppConfig {
|
|||||||
Ok(Self {
|
Ok(Self {
|
||||||
database_url,
|
database_url,
|
||||||
jwt_secret,
|
jwt_secret,
|
||||||
session_expiry_days: std::env::var("SESSION_EXPIRY_DAYS")
|
session_expiry_days: parse_session_expiry_days(
|
||||||
.unwrap_or_else(|_| "30".to_string())
|
std::env::var("SESSION_EXPIRY_DAYS").ok().as_deref(),
|
||||||
.parse()
|
)?,
|
||||||
.context("SESSION_EXPIRY_DAYS must be a number")?,
|
|
||||||
admin_password_hash,
|
admin_password_hash,
|
||||||
event_name: std::env::var("EVENT_NAME").unwrap_or_else(|_| "EventSnap".to_string()),
|
event_name: std::env::var("EVENT_NAME").unwrap_or_else(|_| "EventSnap".to_string()),
|
||||||
event_slug: std::env::var("EVENT_SLUG").context("EVENT_SLUG must be set")?,
|
event_slug: std::env::var("EVENT_SLUG").context("EVENT_SLUG must be set")?,
|
||||||
|
|||||||
@@ -121,3 +121,139 @@ pub async fn get_context(
|
|||||||
is_banned: user.is_banned,
|
is_banned: user.is_banned,
|
||||||
}))
|
}))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// `(original_path, preview_path, thumbnail_path, display_path)` for one upload.
|
||||||
|
type UploadFilePaths = (String, Option<String>, Option<String>, Option<String>);
|
||||||
|
|
||||||
|
/// Delete the caller's own account and everything attached to it.
|
||||||
|
///
|
||||||
|
/// The erasure path (H18). There was no user-deletion route at ANY role, so honouring a "please
|
||||||
|
/// remove my photos and my name" request meant hand-written SQL against production — during or
|
||||||
|
/// after a wedding, by whoever happened to have psql access. Deletion also never removed text:
|
||||||
|
/// captions, comment bodies and hashtag links survived indefinitely by design, so even the
|
||||||
|
/// existing per-photo delete left the guest's words in the database and in the keepsake.
|
||||||
|
///
|
||||||
|
/// Self-service on purpose. The alternative (host-initiated only) puts a guest's erasure request
|
||||||
|
/// through a third party who is at a party, and the join page's data notice now promises this.
|
||||||
|
///
|
||||||
|
/// ORDER MATTERS. `upload.user_id` and `comment.user_id` are plain FKs with NO `ON DELETE CASCADE`
|
||||||
|
/// (migration 002), so deleting the user first fails on a constraint violation. Children first,
|
||||||
|
/// then the row itself — at which point `session`, `like` and `pin_reset_request` do cascade.
|
||||||
|
pub async fn delete_account(
|
||||||
|
State(state): State<AppState>,
|
||||||
|
auth: AuthUser,
|
||||||
|
) -> Result<axum::http::StatusCode, AppError> {
|
||||||
|
// The last host/admin may not erase themselves: it would leave the event with no operator and
|
||||||
|
// no way to appoint one. Mirrors the floor `set_role` and `ban_user` already enforce.
|
||||||
|
let user = User::find_by_id(&state.pool, auth.user_id)
|
||||||
|
.await?
|
||||||
|
.ok_or_else(|| AppError::NotFound("Benutzer nicht gefunden.".into()))?;
|
||||||
|
if matches!(user.role, UserRole::Host | UserRole::Admin) {
|
||||||
|
let others = sqlx::query_scalar::<_, i64>(
|
||||||
|
"SELECT COUNT(*) FROM \"user\"
|
||||||
|
WHERE event_id = $1 AND id != $2
|
||||||
|
AND role IN ('host', 'admin') AND is_banned = FALSE",
|
||||||
|
)
|
||||||
|
.bind(auth.event_id)
|
||||||
|
.bind(auth.user_id)
|
||||||
|
.fetch_one(&state.pool)
|
||||||
|
.await?;
|
||||||
|
if others == 0 {
|
||||||
|
return Err(AppError::BadRequest(
|
||||||
|
"Du bist der letzte Gastgeber. Ernenne zuerst einen anderen Gastgeber, bevor du \
|
||||||
|
dein Konto löschst."
|
||||||
|
.into(),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Collect the file paths BEFORE the rows go, or they are unrecoverable. Every derivative, not
|
||||||
|
// just the original: a preview left behind is still the guest's photo.
|
||||||
|
let files: Vec<UploadFilePaths> = sqlx::query_as(
|
||||||
|
"SELECT original_path, preview_path, thumbnail_path, display_path
|
||||||
|
FROM upload WHERE user_id = $1",
|
||||||
|
)
|
||||||
|
.bind(auth.user_id)
|
||||||
|
.fetch_all(&state.pool)
|
||||||
|
.await?;
|
||||||
|
|
||||||
|
let mut tx = state.pool.begin().await?;
|
||||||
|
// Comments the guest wrote on OTHER people's photos. Hard delete, not `deleted_at`: this is
|
||||||
|
// erasure, and a soft delete leaves the body in the table and in the keepsake's data.json.
|
||||||
|
sqlx::query("DELETE FROM comment WHERE user_id = $1")
|
||||||
|
.bind(auth.user_id)
|
||||||
|
.execute(&mut *tx)
|
||||||
|
.await?;
|
||||||
|
// Their uploads. Cascades comments and likes ON those uploads, plus upload_hashtag links.
|
||||||
|
sqlx::query("DELETE FROM upload WHERE user_id = $1")
|
||||||
|
.bind(auth.user_id)
|
||||||
|
.execute(&mut *tx)
|
||||||
|
.await?;
|
||||||
|
// Invalidate the keepsake inside the same transaction — an already-released archive still
|
||||||
|
// contains this guest's photos and captions, and erasure that leaves them in the downloadable
|
||||||
|
// ZIP has not happened. Returns None when the event isn't released, in which case there is
|
||||||
|
// nothing to rebuild.
|
||||||
|
let regen = crate::services::export::invalidate_and_arm(
|
||||||
|
&mut tx,
|
||||||
|
&state.config.event_slug,
|
||||||
|
crate::services::export::Affects::Both,
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
// And the account. `session`, `like` and `pin_reset_request` cascade from here.
|
||||||
|
sqlx::query("DELETE FROM \"user\" WHERE id = $1")
|
||||||
|
.bind(auth.user_id)
|
||||||
|
.execute(&mut *tx)
|
||||||
|
.await?;
|
||||||
|
tx.commit().await?;
|
||||||
|
|
||||||
|
// Best effort, after the commit. Anything missed here is an orphan with no row pointing at it,
|
||||||
|
// which `sweep_orphan_originals` reclaims on its next pass — so a failure delays reclamation
|
||||||
|
// rather than leaving the file referenced.
|
||||||
|
for (original, preview, thumbnail, display) in &files {
|
||||||
|
for rel in [
|
||||||
|
Some(original),
|
||||||
|
preview.as_ref(),
|
||||||
|
thumbnail.as_ref(),
|
||||||
|
display.as_ref(),
|
||||||
|
]
|
||||||
|
.into_iter()
|
||||||
|
.flatten()
|
||||||
|
{
|
||||||
|
let abs = state.config.media_path.join(rel);
|
||||||
|
if let Err(e) = tokio::fs::remove_file(&abs).await
|
||||||
|
&& e.kind() != std::io::ErrorKind::NotFound
|
||||||
|
{
|
||||||
|
tracing::warn!(error = ?e, path = %abs.display(), "account deletion: could not remove media file");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if let Some(r) = regen {
|
||||||
|
crate::handlers::host::start_regen(&state, r);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Evict their content from every open feed and the projector. `user-hidden` is exactly the
|
||||||
|
// right signal — it already means "this user's cards must go" — and reusing it means every
|
||||||
|
// client already handles this with no new event type.
|
||||||
|
let _ = state.sse_tx.send(crate::state::SseEvent::new(
|
||||||
|
"user-hidden",
|
||||||
|
serde_json::json!({ "user_id": auth.user_id }).to_string(),
|
||||||
|
));
|
||||||
|
|
||||||
|
// Audited like the host actions it resembles, with the actor and target being the same person.
|
||||||
|
crate::services::audit::record(
|
||||||
|
&state.pool,
|
||||||
|
auth.event_id,
|
||||||
|
auth.user_id,
|
||||||
|
None,
|
||||||
|
user.role.clone(),
|
||||||
|
"delete_account",
|
||||||
|
Some(auth.user_id),
|
||||||
|
None,
|
||||||
|
Some(serde_json::json!({ "uploads_removed": files.len() })),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
|
||||||
|
tracing::info!(user_id = %auth.user_id, uploads = files.len(), "account deleted by its owner");
|
||||||
|
Ok(axum::http::StatusCode::NO_CONTENT)
|
||||||
|
}
|
||||||
|
|||||||
@@ -21,6 +21,15 @@ pub struct PublicEventDto {
|
|||||||
pub theme_preset: String,
|
pub theme_preset: String,
|
||||||
pub theme_primary: String,
|
pub theme_primary: String,
|
||||||
pub theme_accent: String,
|
pub theme_accent: String,
|
||||||
|
/// The operator's data notice, if they set one. Empty string when unset (migration 009
|
||||||
|
/// defaults it to `''`).
|
||||||
|
///
|
||||||
|
/// Exposed PUBLICLY — it was only on `/me/context`, which requires a token, so the one place a
|
||||||
|
/// notice actually has to appear (before a name is collected) could not read it. The join page
|
||||||
|
/// pairs this with a baseline notice of its own, precisely because this can be empty: relying
|
||||||
|
/// on an operator-supplied string meant a stock deploy collected ~100 EU guests' photos of
|
||||||
|
/// identifiable people, including children, with no notice at the point of collection at all.
|
||||||
|
pub privacy_note: String,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Public event identity + presentation config, used by the pre-auth join/recover
|
/// Public event identity + presentation config, used by the pre-auth join/recover
|
||||||
@@ -40,5 +49,6 @@ pub async fn get_public_event(State(state): State<AppState>) -> Json<PublicEvent
|
|||||||
.await,
|
.await,
|
||||||
theme_accent: config::get_str(cache, "theme_accent", &state.config.default_theme_accent)
|
theme_accent: config::get_str(cache, "theme_accent", &state.config.default_theme_accent)
|
||||||
.await,
|
.await,
|
||||||
|
privacy_note: config::get_str(cache, "privacy_note", "").await,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -54,9 +54,7 @@ async fn read_text_field_bounded(
|
|||||||
.map_err(|e| AppError::BadRequest(e.to_string()))?
|
.map_err(|e| AppError::BadRequest(e.to_string()))?
|
||||||
{
|
{
|
||||||
if buf.len() + chunk.len() > max_bytes {
|
if buf.len() + chunk.len() > max_bytes {
|
||||||
return Err(AppError::BadRequest(
|
return Err(AppError::BadRequest("Eingabe ist zu lang.".to_string()));
|
||||||
"Eingabe ist zu lang.".to_string(),
|
|
||||||
));
|
|
||||||
}
|
}
|
||||||
buf.extend_from_slice(&chunk);
|
buf.extend_from_slice(&chunk);
|
||||||
}
|
}
|
||||||
@@ -197,29 +195,47 @@ pub async fn upload(
|
|||||||
.ok_or_else(|| AppError::NotFound("Benutzer nicht gefunden.".into()))?;
|
.ok_or_else(|| AppError::NotFound("Benutzer nicht gefunden.".into()))?;
|
||||||
if user.is_banned {
|
if user.is_banned {
|
||||||
drain_multipart(multipart).await;
|
drain_multipart(multipart).await;
|
||||||
return Err(AppError::Forbidden("Du bist gesperrt.".into()));
|
// `UserBanned`, not `Forbidden`: a ban is reversible, so the client must KEEP the queued
|
||||||
|
// blob and park it until `user-shown` arrives. Under the generic `forbidden` code it
|
||||||
|
// purged the photo from IndexedDB and moved the row to `blocked`, which has no retry
|
||||||
|
// button — so an unban restored everything except whatever was in flight.
|
||||||
|
return Err(AppError::UserBanned("Du bist gesperrt.".into()));
|
||||||
}
|
}
|
||||||
|
|
||||||
// Check if uploads are locked
|
// Check if uploads are locked
|
||||||
let event = crate::models::event::Event::find_by_slug(&state.pool, &state.config.event_slug)
|
let event = crate::models::event::Event::find_by_slug(&state.pool, &state.config.event_slug)
|
||||||
.await?
|
.await?
|
||||||
.ok_or_else(|| AppError::NotFound("Event nicht gefunden.".into()))?;
|
.ok_or_else(|| AppError::NotFound("Event nicht gefunden.".into()))?;
|
||||||
if event.uploads_locked_at.is_some() {
|
// RELEASE IS CHECKED FIRST, AND THE ORDER IS THE WHOLE POINT.
|
||||||
drain_multipart(multipart).await;
|
//
|
||||||
// Reversible: a host can reopen the event, so the client keeps the queued blob and
|
// `release ⇒ lock`, so a released gallery satisfies BOTH conditions. Testing the lock first
|
||||||
// retries on `event-opened` rather than purging it (UploadsLocked, not Forbidden).
|
// made this branch unreachable: every post-release upload — the overwhelmingly common case,
|
||||||
return Err(AppError::UploadsLocked("Uploads sind gesperrt.".into()));
|
// since release is the end-of-event action every guest's queue runs into — answered
|
||||||
}
|
// `uploads_locked`, and the `GalleryReleased` arm below was dead code that read as if it
|
||||||
// Belt-and-suspenders on top of the lock (release ⇒ lock): once the gallery is
|
// worked. The commit-time re-check further down splits the two correctly, so the two paths
|
||||||
// released the export has been snapshotted, so a late upload could never make it into
|
// also disagreed about the same event state depending on where the upload was intercepted.
|
||||||
// the keepsake. Reject it explicitly rather than silently diverging the live feed.
|
//
|
||||||
// Also reversible (reopen clears `export_released_at`), so likewise UploadsLocked.
|
// The codes are not interchangeable to the client (see upload-queue.ts): `uploads_locked`
|
||||||
|
// charges an attempt and re-pushes the whole photo on the backoff ladder, and tells the guest
|
||||||
|
// to find it via the camera button. `gallery_released` PARKS it — no attempt charged, no
|
||||||
|
// re-push — and says the photo is safe but needs the hosts to reopen the gallery. Against an
|
||||||
|
// answer that cannot change on its own, the first is a cellular data leak with a misleading
|
||||||
|
// message attached.
|
||||||
|
//
|
||||||
|
// Both keep the blob; both are cleared by `event-opened`. Only the retry behaviour differs.
|
||||||
if event.export_released_at.is_some() {
|
if event.export_released_at.is_some() {
|
||||||
drain_multipart(multipart).await;
|
drain_multipart(multipart).await;
|
||||||
return Err(AppError::UploadsLocked(
|
return Err(AppError::GalleryReleased(
|
||||||
"Galerie wurde bereits freigegeben.".into(),
|
"Die Galerie ist abgeschlossen — es können keine neuen Fotos mehr hinzugefügt werden."
|
||||||
|
.into(),
|
||||||
));
|
));
|
||||||
}
|
}
|
||||||
|
if event.uploads_locked_at.is_some() {
|
||||||
|
drain_multipart(multipart).await;
|
||||||
|
// A PLAIN lock (the host paused uploads mid-event) is the reversible-and-likely-soon case,
|
||||||
|
// so auto-retry is right here: the client keeps the blob and resumes on `event-opened`.
|
||||||
|
return Err(AppError::UploadsLocked("Uploads sind gesperrt.".into()));
|
||||||
|
}
|
||||||
|
|
||||||
// Read config limits from DB
|
// Read config limits from DB
|
||||||
let max_image_mb: i64 = config::get_i64(&state.config_cache, "max_image_size_mb", 20).await;
|
let max_image_mb: i64 = config::get_i64(&state.config_cache, "max_image_size_mb", 20).await;
|
||||||
@@ -287,20 +303,16 @@ pub async fn upload(
|
|||||||
// 10-20 GB of `.tmp` on a 40 GB volume that the gate cannot see, eating the
|
// 10-20 GB of `.tmp` on a 40 GB volume that the gate cannot see, eating the
|
||||||
// reserve that keeps Postgres able to write WAL. The permit is held until the
|
// reserve that keeps Postgres able to write WAL. The permit is held until the
|
||||||
// handler returns, which is exactly as long as the temp file can exist.
|
// handler returns, which is exactly as long as the temp file can exist.
|
||||||
_admission = Some(
|
_admission = Some(state.upload_admission.reserve(cap_bytes).await.ok_or_else(
|
||||||
state
|
|| {
|
||||||
.upload_admission
|
AppError::ServiceUnavailable(
|
||||||
.reserve(cap_bytes)
|
"Gerade laden sehr viele Gäste hoch. Dein Foto bleibt in der \
|
||||||
.await
|
|
||||||
.ok_or_else(|| {
|
|
||||||
AppError::ServiceUnavailable(
|
|
||||||
"Gerade laden sehr viele Gäste hoch. Dein Foto bleibt in der \
|
|
||||||
Warteschlange und wird gleich automatisch gesendet."
|
Warteschlange und wird gleich automatisch gesendet."
|
||||||
.into(),
|
.into(),
|
||||||
Some(30),
|
Some(30),
|
||||||
)
|
)
|
||||||
})?,
|
},
|
||||||
);
|
)?);
|
||||||
tokio::fs::create_dir_all(&originals_dir)
|
tokio::fs::create_dir_all(&originals_dir)
|
||||||
.await
|
.await
|
||||||
.map_err(|e| AppError::Internal(e.into()))?;
|
.map_err(|e| AppError::Internal(e.into()))?;
|
||||||
@@ -513,7 +525,11 @@ pub async fn upload(
|
|||||||
// `media_total` is the opposite: it is a sum of committed DB rows, and this upload's row
|
// `media_total` is the opposite: it is a sum of committed DB rows, and this upload's row
|
||||||
// does not exist yet, so the prospective total does need `+ size`.
|
// does not exist yet, so the prospective total does need `+ size`.
|
||||||
let free = disk.free as i64;
|
let free = disk.free as i64;
|
||||||
let media_after = state.media_total.get(&state.pool).await.saturating_add(size);
|
let media_after = state
|
||||||
|
.media_total
|
||||||
|
.get(&state.pool, &state.config.event_slug)
|
||||||
|
.await
|
||||||
|
.saturating_add(size);
|
||||||
let keepsake_needs =
|
let keepsake_needs =
|
||||||
crate::services::export::required_free_bytes(media_after.max(0) as u64, 2) as i64;
|
crate::services::export::required_free_bytes(media_after.max(0) as u64, 2) as i64;
|
||||||
let required = keepsake_needs.saturating_add(DISK_RESERVE_BYTES);
|
let required = keepsake_needs.saturating_add(DISK_RESERVE_BYTES);
|
||||||
@@ -616,7 +632,19 @@ pub async fn upload(
|
|||||||
.bind(auth.event_id)
|
.bind(auth.event_id)
|
||||||
.fetch_one(&mut *tx)
|
.fetch_one(&mut *tx)
|
||||||
.await?;
|
.await?;
|
||||||
if locked_at.is_some() || released_at.is_some() {
|
// Same order as the fast-path check above, and for the same reason: `release ⇒ lock`, so
|
||||||
|
// testing the lock first would collapse a release into `uploads_locked` and set the client
|
||||||
|
// auto-retrying a photo that can never be accepted until a host reopens the gallery. A
|
||||||
|
// guest who lost the race with `release_gallery` must get `gallery_released` so the queue
|
||||||
|
// parks it instead.
|
||||||
|
if released_at.is_some() {
|
||||||
|
return Err(AppError::GalleryReleased(
|
||||||
|
"Die Galerie ist abgeschlossen — es können keine neuen Fotos mehr hinzugefügt \
|
||||||
|
werden."
|
||||||
|
.into(),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
if locked_at.is_some() {
|
||||||
return Err(AppError::UploadsLocked("Uploads sind gesperrt.".into()));
|
return Err(AppError::UploadsLocked("Uploads sind gesperrt.".into()));
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -677,7 +705,47 @@ pub async fn upload(
|
|||||||
let h = Hashtag::upsert(&mut *tx, auth.event_id, tag).await?;
|
let h = Hashtag::upsert(&mut *tx, auth.event_id, tag).await?;
|
||||||
Hashtag::link_to_upload(&mut *tx, upload.id, h.id).await?;
|
Hashtag::link_to_upload(&mut *tx, upload.id, h.id).await?;
|
||||||
}
|
}
|
||||||
tx.commit().await?;
|
|
||||||
|
// Hand the bytes to the row BEFORE committing, not after.
|
||||||
|
//
|
||||||
|
// `tx.commit().await` is a suspension point, and a COMMIT already written to the
|
||||||
|
// socket is applied by Postgres whether or not this future lives to read the reply.
|
||||||
|
// Disarming afterwards left a real window: the guest walks out of range mid-commit,
|
||||||
|
// axum drops the future, Postgres commits the row anyway, and `Drop` deletes the file
|
||||||
|
// that freshly committed row points at. The result is invisible to every repair path
|
||||||
|
// — the row is live so the deleted-media sweep skips it, the file is gone so the
|
||||||
|
// orphan sweep skips it — and it is missing from the keepsake with nothing in the log
|
||||||
|
// naming it as loss.
|
||||||
|
//
|
||||||
|
// Disarming first cannot fix the cancellation (nothing in-process can), but it moves
|
||||||
|
// the failure to the recoverable side: if we are dropped mid-commit the bytes leak,
|
||||||
|
// and leaked bytes under a final name are exactly what the orphan sweeper reclaims.
|
||||||
|
// A committed row whose file we deleted is unrecoverable. Prefer the leak.
|
||||||
|
file_guard.disarm();
|
||||||
|
if let Err(e) = tx.commit().await {
|
||||||
|
// Deliberately do NOT re-arm the guard here.
|
||||||
|
//
|
||||||
|
// A `commit()` that returns `Err` is INDETERMINATE, not "definitely rolled back".
|
||||||
|
// sqlx writes `COMMIT` to the socket and awaits the reply; if the connection dies
|
||||||
|
// after Postgres flushed the WAL record but before that reply arrives (a db
|
||||||
|
// restart, a killed backend, a network blip), the row is durably committed and we
|
||||||
|
// are told it failed. Re-arming would then delete the file a live row points at —
|
||||||
|
// the exact unrecoverable case the comment above says to avoid, just reached
|
||||||
|
// through the error path instead of the cancellation path.
|
||||||
|
//
|
||||||
|
// It is worse than it sounds, because the client retries: the idempotency fast
|
||||||
|
// path finds the committed row, answers 200, and the phone purges the only other
|
||||||
|
// copy of the photo. So we prefer the leak in both directions. If the commit
|
||||||
|
// genuinely did not apply, `sweep_orphan_originals` reclaims the bytes on its next
|
||||||
|
// pass (it deletes files with no DB row, which is precisely this case).
|
||||||
|
tracing::error!(
|
||||||
|
error = ?e,
|
||||||
|
path = %absolute_path.display(),
|
||||||
|
"upload commit returned an error; leaving the file in place because the commit \
|
||||||
|
may still have applied — the orphan sweeper reclaims it if it did not"
|
||||||
|
);
|
||||||
|
return Err(e.into());
|
||||||
|
}
|
||||||
Ok(upload)
|
Ok(upload)
|
||||||
}
|
}
|
||||||
.await;
|
.await;
|
||||||
@@ -687,6 +755,9 @@ pub async fn upload(
|
|||||||
// and reclaims the bytes on the way out. That covers the concurrent-duplicate loser below
|
// and reclaims the bytes on the way out. That covers the concurrent-duplicate loser below
|
||||||
// as well as the plain error case, and unlike the explicit `remove_file` calls it replaces,
|
// as well as the plain error case, and unlike the explicit `remove_file` calls it replaces,
|
||||||
// it also covers axum dropping this future instead of returning.
|
// it also covers axum dropping this future instead of returning.
|
||||||
|
//
|
||||||
|
// The successful-commit case disarmed the guard inside the block, immediately before
|
||||||
|
// `tx.commit()` — see the comment there for why it cannot be done out here.
|
||||||
let upload = match tx_result {
|
let upload = match tx_result {
|
||||||
Ok(u) => u,
|
Ok(u) => u,
|
||||||
// The concurrent duplicate resolved inside the transaction. The winner's row is committed;
|
// The concurrent duplicate resolved inside the transaction. The winner's row is committed;
|
||||||
@@ -713,8 +784,6 @@ pub async fn upload(
|
|||||||
}
|
}
|
||||||
Err(e) => return Err(e),
|
Err(e) => return Err(e),
|
||||||
};
|
};
|
||||||
// The committed row now references these bytes — hand ownership over.
|
|
||||||
file_guard.disarm();
|
|
||||||
|
|
||||||
// Spawn compression task
|
// Spawn compression task
|
||||||
state
|
state
|
||||||
@@ -771,7 +840,8 @@ pub async fn edit_upload(
|
|||||||
|
|
||||||
// This endpoint had no rate limit of any kind, while every other mutating route has one.
|
// This endpoint had no rate limit of any kind, while every other mutating route has one.
|
||||||
let rate_limits_on = config::get_bool(&state.config_cache, "rate_limits_enabled", true).await;
|
let rate_limits_on = config::get_bool(&state.config_cache, "rate_limits_enabled", true).await;
|
||||||
let edit_rate_on = config::get_bool(&state.config_cache, "upload_edit_rate_enabled", true).await;
|
let edit_rate_on =
|
||||||
|
config::get_bool(&state.config_cache, "upload_edit_rate_enabled", true).await;
|
||||||
if rate_limits_on && edit_rate_on {
|
if rate_limits_on && edit_rate_on {
|
||||||
let edit_rate =
|
let edit_rate =
|
||||||
config::get_i64(&state.config_cache, "upload_edit_rate_per_min", 30).await as usize;
|
config::get_i64(&state.config_cache, "upload_edit_rate_per_min", 30).await as usize;
|
||||||
@@ -1141,7 +1211,8 @@ fn quota_limit_bytes(free_disk: i64, tolerance: f64, active_uploaders: i64, expe
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// Computes the per-user storage quota using
|
/// Computes the per-user storage quota using
|
||||||
/// `floor((free_disk * tolerance) / max(active_uploaders, 1))`. Returns `limit_bytes =
|
/// `max(floor((free_disk * tolerance) / max(active_uploaders, estimated_guest_count, 1)), 500 MiB)`
|
||||||
|
/// — see [`quota_limit_bytes`] for the floor's exact conditions. Returns `limit_bytes =
|
||||||
/// None` whenever the storage quota is currently disabled — callers should skip the
|
/// None` whenever the storage quota is currently disabled — callers should skip the
|
||||||
/// check (upload handler) or hide the UI (quota endpoint).
|
/// check (upload handler) or hide the UI (quota endpoint).
|
||||||
pub async fn compute_storage_quota(state: &AppState) -> QuotaEstimate {
|
pub async fn compute_storage_quota(state: &AppState) -> QuotaEstimate {
|
||||||
@@ -1150,11 +1221,20 @@ pub async fn compute_storage_quota(state: &AppState) -> QuotaEstimate {
|
|||||||
config::get_bool(&state.config_cache, "storage_quota_enabled", true).await;
|
config::get_bool(&state.config_cache, "storage_quota_enabled", true).await;
|
||||||
let tolerance = config::get_f64(&state.config_cache, "quota_tolerance", 0.75).await;
|
let tolerance = config::get_f64(&state.config_cache, "quota_tolerance", 0.75).await;
|
||||||
|
|
||||||
let (active_count,): (i64,) =
|
// Scoped to THIS event (H12). Without the filter, reusing the install for a second event
|
||||||
sqlx::query_as("SELECT COUNT(DISTINCT user_id) FROM upload WHERE deleted_at IS NULL")
|
// carried the first one's uploaders forward permanently: event one's 30 photographers stayed
|
||||||
.fetch_one(&state.pool)
|
// in event two's quota divisor, silently shrinking every new guest's ceiling for a party they
|
||||||
.await
|
// had nothing to do with. There is no reset path anywhere in the code or the runbook, so the
|
||||||
.unwrap_or((0,));
|
// only fix would have been hand-written SQL.
|
||||||
|
let (active_count,): (i64,) = sqlx::query_as(
|
||||||
|
"SELECT COUNT(DISTINCT up.user_id) FROM upload up
|
||||||
|
JOIN event e ON e.id = up.event_id
|
||||||
|
WHERE up.deleted_at IS NULL AND e.slug = $1",
|
||||||
|
)
|
||||||
|
.bind(&state.config.event_slug)
|
||||||
|
.fetch_one(&state.pool)
|
||||||
|
.await
|
||||||
|
.unwrap_or((0,));
|
||||||
let active = active_count.max(1);
|
let active = active_count.max(1);
|
||||||
// The operator's expected headcount, used as a FLOOR on the divisor so the ceiling doesn't
|
// The operator's expected headcount, used as a FLOOR on the divisor so the ceiling doesn't
|
||||||
// slide down as guests arrive — see `quota_limit_bytes`. Admin-editable at runtime.
|
// slide down as guests arrive — see `quota_limit_bytes`. Admin-editable at runtime.
|
||||||
@@ -1196,7 +1276,7 @@ pub async fn compute_storage_quota(state: &AppState) -> QuotaEstimate {
|
|||||||
|
|
||||||
/// Outcome of parsing a `Range` request header against a known file length.
|
/// Outcome of parsing a `Range` request header against a known file length.
|
||||||
#[derive(Debug, PartialEq, Eq)]
|
#[derive(Debug, PartialEq, Eq)]
|
||||||
enum RangeSpec {
|
pub(crate) enum RangeSpec {
|
||||||
/// No `Range` header, or one we deliberately don't honour (multi-range, non-`bytes`
|
/// No `Range` header, or one we deliberately don't honour (multi-range, non-`bytes`
|
||||||
/// unit, malformed). RFC 9110 lets a server ignore a Range it can't process and reply
|
/// unit, malformed). RFC 9110 lets a server ignore a Range it can't process and reply
|
||||||
/// 200 with the full body, which is what every one of these cases does.
|
/// 200 with the full body, which is what every one of these cases does.
|
||||||
@@ -1213,7 +1293,7 @@ enum RangeSpec {
|
|||||||
/// Deliberately supports only the three forms a media element actually sends —
|
/// Deliberately supports only the three forms a media element actually sends —
|
||||||
/// `bytes=N-`, `bytes=N-M`, `bytes=-S` (suffix) — and treats everything else as `Full`.
|
/// `bytes=N-`, `bytes=N-M`, `bytes=-S` (suffix) — and treats everything else as `Full`.
|
||||||
/// Multi-range responses need `multipart/byteranges`, which no `<video>` requires.
|
/// Multi-range responses need `multipart/byteranges`, which no `<video>` requires.
|
||||||
fn parse_range(header: Option<&str>, len: u64) -> RangeSpec {
|
pub(crate) fn parse_range(header: Option<&str>, len: u64) -> RangeSpec {
|
||||||
let Some(raw) = header else {
|
let Some(raw) = header else {
|
||||||
return RangeSpec::Full;
|
return RangeSpec::Full;
|
||||||
};
|
};
|
||||||
@@ -1358,6 +1438,29 @@ async fn stream_media_file(
|
|||||||
/// soft-deleted and ban-hidden uploads (via `find_visible_media`) so moderation actually
|
/// soft-deleted and ban-hidden uploads (via `find_visible_media`) so moderation actually
|
||||||
/// removes access to content. Preview and thumbnail variants are gated the same way (see
|
/// removes access to content. Preview and thumbnail variants are gated the same way (see
|
||||||
/// [`get_preview`] / [`get_thumbnail`]).
|
/// [`get_preview`] / [`get_thumbnail`]).
|
||||||
|
/// NO per-IP rate limit on this route, deliberately — a 600/min ceiling was added here and had to
|
||||||
|
/// come back out.
|
||||||
|
///
|
||||||
|
/// The reasoning that put it in was that `/original` serves "100 guests occasionally tapping
|
||||||
|
/// 'Original anzeigen'", so a venue-wide 10/s could only ever catch a scraper. That is not what
|
||||||
|
/// this route is. `pickMediaUrl` (frontend/src/lib/data-mode-store.ts) resolves to
|
||||||
|
/// `preview_url ?? thumbnail_url ?? /original`, and a freshly committed upload has BOTH derivatives
|
||||||
|
/// null until the compression worker reaches it — at `COMPRESSION_WORKER_CONCURRENCY=2` that is
|
||||||
|
/// minutes during a post-ceremony burst. So `/original` IS the feed's hot path for exactly the
|
||||||
|
/// newest photos, in a newest-first grid, at the busiest moment; `VirtualFeed.svelte` says as much
|
||||||
|
/// where it explains its broken-tile retry.
|
||||||
|
///
|
||||||
|
/// With every guest behind one NAT address the bucket is venue-wide: ~6 new photos fanned out by
|
||||||
|
/// `upload-new` to ~100 open feeds exhausts 600 on its own, and then every original fetch from
|
||||||
|
/// anyone at the party 429s for the rest of the window. The tiles' own 4-second retry uses a fresh
|
||||||
|
/// `?r=` nonce, so the clients then hold the bucket saturated themselves. The whole venue watches
|
||||||
|
/// the newest photos render as broken tiles, and the projector starts skipping slides.
|
||||||
|
///
|
||||||
|
/// A per-IP bucket cannot separate "one scraper" from "the entire party" when they share an
|
||||||
|
/// address, and these four media routes are unauthenticated by design (an `<img>` cannot send a
|
||||||
|
/// bearer token), so there is no per-user key to move to. Bandwidth abuse belongs at the proxy,
|
||||||
|
/// where per-connection limits still work; the certain harm here outweighed the speculative
|
||||||
|
/// protection.
|
||||||
pub async fn get_original(
|
pub async fn get_original(
|
||||||
State(state): State<AppState>,
|
State(state): State<AppState>,
|
||||||
headers: axum::http::HeaderMap,
|
headers: axum::http::HeaderMap,
|
||||||
@@ -1652,7 +1755,8 @@ mod tests {
|
|||||||
while media < USABLE {
|
while media < USABLE {
|
||||||
media += step;
|
media += step;
|
||||||
let free = USABLE - media;
|
let free = USABLE - media;
|
||||||
if free >= crate::services::export::required_free_bytes(media as u64, 2) as i64 + reserve
|
if free
|
||||||
|
>= crate::services::export::required_free_bytes(media as u64, 2) as i64 + reserve
|
||||||
{
|
{
|
||||||
ceiling = media;
|
ceiling = media;
|
||||||
}
|
}
|
||||||
@@ -1672,7 +1776,8 @@ mod tests {
|
|||||||
let over = ceiling + step;
|
let over = ceiling + step;
|
||||||
let free_over = USABLE - over;
|
let free_over = USABLE - over;
|
||||||
assert!(
|
assert!(
|
||||||
free_over < crate::services::export::required_free_bytes(over as u64, 2) as i64 + reserve,
|
free_over
|
||||||
|
< crate::services::export::required_free_bytes(over as u64, 2) as i64 + reserve,
|
||||||
"the gate should already be closed one step past the ceiling"
|
"the gate should already be closed one step past the ceiling"
|
||||||
);
|
);
|
||||||
|
|
||||||
@@ -1719,8 +1824,8 @@ mod tests {
|
|||||||
);
|
);
|
||||||
|
|
||||||
// Global: the keepsake needs both halves plus the reserve, and they no longer fit.
|
// Global: the keepsake needs both halves plus the reserve, and they no longer fit.
|
||||||
let required =
|
let required = crate::services::export::required_free_bytes(media as u64, 2) as i64
|
||||||
crate::services::export::required_free_bytes(media as u64, 2) as i64 + DISK_RESERVE_BYTES;
|
+ DISK_RESERVE_BYTES;
|
||||||
assert!(
|
assert!(
|
||||||
free < required,
|
free < required,
|
||||||
"the global gate must already be closed at {media} bytes of media: free {free} \
|
"the global gate must already be closed at {media} bytes of media: free {free} \
|
||||||
@@ -1771,7 +1876,10 @@ mod tests {
|
|||||||
fn dropping_an_armed_guard_reclaims_the_file() {
|
fn dropping_an_armed_guard_reclaims_the_file() {
|
||||||
let p = scratch("armed.tmp");
|
let p = scratch("armed.tmp");
|
||||||
drop(TempFileGuard::new(p.clone()));
|
drop(TempFileGuard::new(p.clone()));
|
||||||
assert!(!p.exists(), "an abandoned upload must not survive the request");
|
assert!(
|
||||||
|
!p.exists(),
|
||||||
|
"an abandoned upload must not survive the request"
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
@@ -1780,7 +1888,10 @@ mod tests {
|
|||||||
let mut g = TempFileGuard::new(p.clone());
|
let mut g = TempFileGuard::new(p.clone());
|
||||||
g.disarm();
|
g.disarm();
|
||||||
drop(g);
|
drop(g);
|
||||||
assert!(p.exists(), "a committed upload's bytes must never be deleted");
|
assert!(
|
||||||
|
p.exists(),
|
||||||
|
"a committed upload's bytes must never be deleted"
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
@@ -1792,7 +1903,10 @@ mod tests {
|
|||||||
std::fs::remove_file(&old).unwrap();
|
std::fs::remove_file(&old).unwrap();
|
||||||
g.retarget(new.clone());
|
g.retarget(new.clone());
|
||||||
drop(g);
|
drop(g);
|
||||||
assert!(!new.exists(), "the final-named original is orphaned too until the row commits");
|
assert!(
|
||||||
|
!new.exists(),
|
||||||
|
"the final-named original is orphaned too until the row commits"
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
@@ -1807,7 +1921,7 @@ mod tests {
|
|||||||
/// commit transaction — which holds a FOR SHARE lock on the event row, so one request could
|
/// commit transaction — which holds a FOR SHARE lock on the event row, so one request could
|
||||||
/// stall every other upload behind tens of thousands of round trips.
|
/// stall every other upload behind tens of thousands of round trips.
|
||||||
mod hashtag_caps {
|
mod hashtag_caps {
|
||||||
use super::super::{MAX_HASHTAGS_PER_UPLOAD, MAX_HASHTAG_LENGTH, normalize_tags};
|
use super::super::{MAX_HASHTAG_LENGTH, MAX_HASHTAGS_PER_UPLOAD, normalize_tags};
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn a_huge_csv_is_capped_not_upserted_in_full() {
|
fn a_huge_csv_is_capped_not_upserted_in_full() {
|
||||||
|
|||||||
@@ -97,7 +97,8 @@ impl CompressionWorker {
|
|||||||
let mut attempt = 1u32;
|
let mut attempt = 1u32;
|
||||||
let outcome = loop {
|
let outcome = loop {
|
||||||
match worker
|
match worker
|
||||||
.do_process(upload_id, &original_path, &mime_type)
|
// Charge the lifetime budget once per episode, on the first attempt only.
|
||||||
|
.do_process(upload_id, &original_path, &mime_type, attempt == 1)
|
||||||
.await
|
.await
|
||||||
{
|
{
|
||||||
Ok(v) => break Ok(v),
|
Ok(v) => break Ok(v),
|
||||||
@@ -205,11 +206,25 @@ impl CompressionWorker {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// `charge_lifetime_attempt` is true only for the FIRST `do_process` of a given
|
||||||
|
/// `process()` call, so the two budgets stay independent.
|
||||||
|
///
|
||||||
|
/// They were not. `MAX_PROCESS_ATTEMPTS` (in-request retries, 3) and
|
||||||
|
/// `MAX_DERIVATIVE_ATTEMPTS` (lifetime, 3) are equal, and every retry re-entered here and
|
||||||
|
/// charged the lifetime counter — so one request's three retries, six seconds apart,
|
||||||
|
/// exhausted the entire lifetime budget. A ten-second pool blip during the arrival burst
|
||||||
|
/// therefore stranded every photo whose worker was inside that window with no preview and no
|
||||||
|
/// display derivative, permanently, recoverable by nothing: the boot backfill re-selects them
|
||||||
|
/// and immediately gives up on the same exhausted counter.
|
||||||
|
///
|
||||||
|
/// The two exist to bound different things — "this request is flapping" versus "this INPUT is
|
||||||
|
/// poison" — and only the second should survive across requests.
|
||||||
async fn do_process(
|
async fn do_process(
|
||||||
&self,
|
&self,
|
||||||
upload_id: Uuid,
|
upload_id: Uuid,
|
||||||
original_path: &str,
|
original_path: &str,
|
||||||
mime_type: &str,
|
mime_type: &str,
|
||||||
|
charge_lifetime_attempt: bool,
|
||||||
) -> Result<()> {
|
) -> Result<()> {
|
||||||
Upload::set_compression_status(&self.pool, upload_id, "processing").await?;
|
Upload::set_compression_status(&self.pool, upload_id, "processing").await?;
|
||||||
|
|
||||||
@@ -218,8 +233,15 @@ impl CompressionWorker {
|
|||||||
if mime_type.starts_with("image/") {
|
if mime_type.starts_with("image/") {
|
||||||
// Count the attempt BEFORE doing the work — see `begin_derivative_attempt`. If this
|
// Count the attempt BEFORE doing the work — see `begin_derivative_attempt`. If this
|
||||||
// input is the one that kills the container, this write is the only record that
|
// input is the one that kills the container, this write is the only record that
|
||||||
// survives, and it is what stops the boot backfill replaying it forever.
|
// survives, and it is what stops the boot backfill replaying it forever. Charging on
|
||||||
match Upload::begin_derivative_attempt(&self.pool, upload_id).await? {
|
// the first attempt preserves that: a container-killing input never reaches a second.
|
||||||
|
let charged = if charge_lifetime_attempt {
|
||||||
|
Upload::begin_derivative_attempt(&self.pool, upload_id).await?
|
||||||
|
} else {
|
||||||
|
// Already charged for this episode. Re-read the row only to notice it vanished.
|
||||||
|
Upload::derivative_attempts(&self.pool, upload_id).await?
|
||||||
|
};
|
||||||
|
match charged {
|
||||||
Some(attempts) if attempts > Self::MAX_DERIVATIVE_ATTEMPTS => {
|
Some(attempts) if attempts > Self::MAX_DERIVATIVE_ATTEMPTS => {
|
||||||
anyhow::bail!(
|
anyhow::bail!(
|
||||||
"derivative generation gave up after {} attempt(s)",
|
"derivative generation gave up after {} attempt(s)",
|
||||||
@@ -304,7 +326,6 @@ impl CompressionWorker {
|
|||||||
/// saving rather than risk the OOM kill.
|
/// saving rather than risk the OOM kill.
|
||||||
const OXIPNG_MAX_PIXELS: u64 = 8_000_000;
|
const OXIPNG_MAX_PIXELS: u64 = 8_000_000;
|
||||||
|
|
||||||
|
|
||||||
/// Wall-clock ceiling for one oxipng run.
|
/// Wall-clock ceiling for one oxipng run.
|
||||||
///
|
///
|
||||||
/// Bounds TIME, NOT MEMORY — oxipng checks the deadline between trials, so a single trial
|
/// Bounds TIME, NOT MEMORY — oxipng checks the deadline between trials, so a single trial
|
||||||
@@ -336,8 +357,10 @@ impl CompressionWorker {
|
|||||||
// the upload handler already does via `exceeds_decode_budget`) and, if this job is a
|
// the upload handler already does via `exceeds_decode_budget`) and, if this job is a
|
||||||
// giant, take the exclusive permit so it cannot overlap another giant. Held for the
|
// giant, take the exclusive permit so it cannot overlap another giant. Held for the
|
||||||
// whole blocking section, released on drop including on error.
|
// whole blocking section, released on drop including on error.
|
||||||
let estimate =
|
let estimate = crate::services::imaging::estimated_processing_peak_bytes(
|
||||||
crate::services::imaging::estimated_processing_peak_bytes(&original, Self::DISPLAY_MAX_EDGE);
|
&original,
|
||||||
|
Self::DISPLAY_MAX_EDGE,
|
||||||
|
);
|
||||||
let _heavy_permit = match estimate {
|
let _heavy_permit = match estimate {
|
||||||
Some(bytes) if bytes > crate::services::imaging::HEAVY_IMAGE_BYTES => {
|
Some(bytes) if bytes > crate::services::imaging::HEAVY_IMAGE_BYTES => {
|
||||||
tracing::debug!(
|
tracing::debug!(
|
||||||
@@ -345,14 +368,24 @@ impl CompressionWorker {
|
|||||||
estimated_mib = bytes / (1024 * 1024),
|
estimated_mib = bytes / (1024 * 1024),
|
||||||
"waiting for the heavy-image permit"
|
"waiting for the heavy-image permit"
|
||||||
);
|
);
|
||||||
Some(crate::services::imaging::HEAVY_IMAGE_PERMITS.acquire().await)
|
Some(
|
||||||
|
crate::services::imaging::HEAVY_IMAGE_PERMITS
|
||||||
|
.acquire()
|
||||||
|
.await,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
_ => None,
|
_ => None,
|
||||||
};
|
};
|
||||||
|
|
||||||
// Run blocking image operations in a spawn_blocking task
|
// Run blocking image operations in a spawn_blocking task
|
||||||
tokio::task::spawn_blocking(move || {
|
tokio::task::spawn_blocking(move || {
|
||||||
write_image_derivatives(upload_id, &original, &mime_owned, &preview_path, &display_path)
|
write_image_derivatives(
|
||||||
|
upload_id,
|
||||||
|
&original,
|
||||||
|
&mime_owned,
|
||||||
|
&preview_path,
|
||||||
|
&display_path,
|
||||||
|
)
|
||||||
})
|
})
|
||||||
.await??;
|
.await??;
|
||||||
|
|
||||||
|
|||||||
@@ -55,15 +55,23 @@ impl MediaTotalCache {
|
|||||||
/// quota path and the export preflight: a database blip must not turn into "every upload
|
/// quota path and the export preflight: a database blip must not turn into "every upload
|
||||||
/// refused". The disk-space half of the gate still applies, so a failure here degrades the
|
/// refused". The disk-space half of the gate still applies, so a failure here degrades the
|
||||||
/// check to the old flat-reserve behaviour rather than disabling it.
|
/// check to the old flat-reserve behaviour rather than disabling it.
|
||||||
pub async fn get(&self, pool: &PgPool) -> i64 {
|
pub async fn get(&self, pool: &PgPool, event_slug: &str) -> i64 {
|
||||||
if let Some((bytes, at)) = *self.inner.read().unwrap()
|
if let Some((bytes, at)) = *self.inner.read().unwrap()
|
||||||
&& at.elapsed() < TTL
|
&& at.elapsed() < TTL
|
||||||
{
|
{
|
||||||
return bytes;
|
return bytes;
|
||||||
}
|
}
|
||||||
|
// Scoped to THIS event (H12). The unscoped `SUM(total_upload_bytes) FROM "user"` summed
|
||||||
|
// every user row in the table, so reusing the install for a second event carried the first
|
||||||
|
// one's bytes into the second one's keepsake-headroom gate — closing uploads early with a
|
||||||
|
// message about "the event's storage" being full, counting media that belongs to a party
|
||||||
|
// that already happened (and whose files are never reclaimed either).
|
||||||
let queried = sqlx::query_scalar::<_, Option<i64>>(
|
let queried = sqlx::query_scalar::<_, Option<i64>>(
|
||||||
"SELECT SUM(total_upload_bytes)::bigint FROM \"user\"",
|
"SELECT SUM(u.total_upload_bytes)::bigint FROM \"user\" u
|
||||||
|
JOIN event e ON e.id = u.event_id
|
||||||
|
WHERE e.slug = $1",
|
||||||
)
|
)
|
||||||
|
.bind(event_slug)
|
||||||
.fetch_one(pool)
|
.fetch_one(pool)
|
||||||
.await;
|
.await;
|
||||||
|
|
||||||
|
|||||||
@@ -82,12 +82,7 @@ impl UploadAdmission {
|
|||||||
pub async fn reserve(&self, cap_bytes: usize) -> Option<OwnedSemaphorePermit> {
|
pub async fn reserve(&self, cap_bytes: usize) -> Option<OwnedSemaphorePermit> {
|
||||||
let mib = cap_bytes.div_ceil(1024 * 1024).max(1);
|
let mib = cap_bytes.div_ceil(1024 * 1024).max(1);
|
||||||
let want = u32::try_from(mib).unwrap_or(BUDGET_MIB).min(BUDGET_MIB);
|
let want = u32::try_from(mib).unwrap_or(BUDGET_MIB).min(BUDGET_MIB);
|
||||||
match tokio::time::timeout(
|
match tokio::time::timeout(WAIT, self.permits.clone().acquire_many_owned(want)).await {
|
||||||
WAIT,
|
|
||||||
self.permits.clone().acquire_many_owned(want),
|
|
||||||
)
|
|
||||||
.await
|
|
||||||
{
|
|
||||||
Ok(Ok(permit)) => Some(permit),
|
Ok(Ok(permit)) => Some(permit),
|
||||||
// The semaphore is never closed, so `Err` here is unreachable in practice; treat it
|
// The semaphore is never closed, so `Err` here is unreachable in practice; treat it
|
||||||
// the same as a timeout rather than panicking on the upload path.
|
// the same as a timeout rather than panicking on the upload path.
|
||||||
@@ -124,12 +119,12 @@ mod tests {
|
|||||||
|
|
||||||
// Nothing left: a second reservation must not be granted. Raced against a short timeout so
|
// Nothing left: a second reservation must not be granted. Raced against a short timeout so
|
||||||
// the test does not sit for the full WAIT.
|
// the test does not sit for the full WAIT.
|
||||||
let blocked = tokio::time::timeout(
|
let blocked =
|
||||||
Duration::from_millis(150),
|
tokio::time::timeout(Duration::from_millis(150), admission.reserve(1024 * 1024)).await;
|
||||||
admission.reserve(1024 * 1024),
|
assert!(
|
||||||
)
|
blocked.is_err(),
|
||||||
.await;
|
"budget exhausted, yet a reservation was granted"
|
||||||
assert!(blocked.is_err(), "budget exhausted, yet a reservation was granted");
|
);
|
||||||
|
|
||||||
// ...and releasing the permit makes room again, so the budget is not a one-way latch.
|
// ...and releasing the permit makes room again, so the budget is not a one-way latch.
|
||||||
drop(whole);
|
drop(whole);
|
||||||
|
|||||||
Reference in New Issue
Block a user