Files
EventSnap/backend/tests/upload_idempotency.rs
Fabian Hamm (Privat) 1d0df3ebf6 feat(upload): make uploads idempotent so a lost response cannot duplicate a photo
The ordinary mobile failure, not an exotic one: the server receives the body,
validates it, commits the row — and the response is lost on the way back because the
guest walked out of range or the AP dropped the connection. The client sees a network
error with the blob still in hand and re-sends it, both when the guest taps "Erneut"
and automatically when the queue requeues on reconnect. Every attempt minted a fresh
`Uuid::new_v4()` server-side, so the same photo landed in the gallery two or three
times and was charged against the guest's storage quota each time.

The client already has a stable per-queue-item UUID, so it costs nothing to send.
Migration 022 adds `client_upload_id` with a partial unique index — partial so the
NULLs of every pre-022 upload, and of any caller that doesn't send one, keep working
untouched.

Two paths, because there are two races:

- Sequential retry: a lookup before the transaction finds the stored row, deletes the
  re-sent bytes and replays the original response as 200. The body has necessarily
  already been streamed, since the key arrives as a multipart field — re-sending is the
  client's cost and is already paid by the time we see it. What must be prevented is a
  second ROW.
- Concurrent retry: two attempts in flight at once. `ON CONFLICT DO NOTHING` returns no
  row to the loser, which abandons its transaction (quota increment included) and
  replays the winner. Letting the unique index raise instead would only surface after
  the transaction had aborted, as an opaque error the caller would have to string-match.

The replay reads live state rather than assuming a fresh row: a reconnect can be
minutes later, by which time the derivatives may exist and the photo may have been
liked. Every read there fails soft — the upload is already safely stored, so a sparser
response is fine and failing the request is not.

Verified live: the same photo sent three times returns 201, 200, 200 with one id, one
row, and the quota charged exactly once.

Also in this file: the two image-header probes at admission now run on `spawn_blocking`.
Both open the file and run the codec's header parse synchronously, and `#[tokio::main]`
gives two worker threads on a 2-vCPU box — so every upload stalled half the runtime's
request-serving capacity. Everything else that blocks here (image encode, bcrypt) was
already offloaded; this was the one that wasn't.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 18:35:05 +02:00

176 lines
6.5 KiB
Rust

//! DB-backed tests for the upload idempotency key (migration 022).
//!
//! The guarantee under test is the one thing standing between a lost response and a duplicated
//! wedding photo: a retry of an upload that already committed must NOT create a second row, and
//! must not charge the guest's storage quota twice. The whole mechanism is SQL — a partial unique
//! index plus `ON CONFLICT DO NOTHING` — so it is tested against a real database with the real
//! migrations applied, using the same statements `src/` runs.
mod common;
use common::*;
use sqlx::PgPool;
use uuid::Uuid;
/// SRC: `models/upload.rs::Upload::create` — the insert, verbatim.
///
/// Returns the new row's id, or `None` when the key was already stored. The handler treats
/// `None` as "a concurrent retry won" and replays the stored row instead of committing.
async fn create_upload(
pool: &PgPool,
event_id: Uuid,
user_id: Uuid,
original_path: &str,
client_upload_id: Option<Uuid>,
) -> Option<Uuid> {
let row: Option<(Uuid,)> = sqlx::query_as(
"INSERT INTO upload (event_id, user_id, original_path, mime_type, original_size_bytes, caption, client_upload_id)
VALUES ($1, $2, $3, $4, $5, $6, $7)
ON CONFLICT (client_upload_id) WHERE client_upload_id IS NOT NULL DO NOTHING
RETURNING id",
)
.bind(event_id)
.bind(user_id)
.bind(original_path)
.bind("image/jpeg")
.bind(1_000i64)
.bind(Option::<String>::None)
.bind(client_upload_id)
.fetch_optional(pool)
.await
.expect("create_upload");
row.map(|(id,)| id)
}
/// SRC: `models/upload.rs::Upload::find_by_client_upload_id` — the lookup, verbatim.
async fn find_by_key(pool: &PgPool, user_id: Uuid, client_upload_id: Uuid) -> Option<Uuid> {
let row: Option<(Uuid,)> = sqlx::query_as(
"SELECT id FROM upload
WHERE client_upload_id = $1 AND user_id = $2 AND deleted_at IS NULL",
)
.bind(client_upload_id)
.bind(user_id)
.fetch_optional(pool)
.await
.expect("find_by_key");
row.map(|(id,)| id)
}
async fn upload_count(pool: &PgPool) -> i64 {
sqlx::query_scalar("SELECT COUNT(*) FROM upload")
.fetch_one(pool)
.await
.expect("upload_count")
}
/// The core guarantee. A phone that loses the response and re-sends the same photo gets the
/// original row back, not a second copy in the gallery and a second charge against its quota.
#[sqlx::test]
async fn the_same_key_can_only_ever_store_one_upload(pool: PgPool) {
let event_id = seed_event(&pool, "wedding").await;
let user_id = seed_user(&pool, event_id, "Wackelige Wanda").await;
let key = Uuid::new_v4();
let first = create_upload(&pool, event_id, user_id, "originals/a.jpg", Some(key)).await;
assert!(first.is_some(), "the first attempt must store the upload");
// The retry: same key, and (as after a real re-send) a different file on disk.
let second = create_upload(&pool, event_id, user_id, "originals/b.jpg", Some(key)).await;
assert!(
second.is_none(),
"a retry of a committed upload must not insert a second row"
);
assert_eq!(upload_count(&pool).await, 1);
// And the handler can find the winner to replay it.
assert_eq!(find_by_key(&pool, user_id, key).await, first);
}
/// The index must not over-reach. Two genuinely different photos carry different keys and must
/// both land — this is the ordinary case, and breaking it would silently drop uploads.
#[sqlx::test]
async fn different_keys_are_different_uploads(pool: PgPool) {
let event_id = seed_event(&pool, "wedding").await;
let user_id = seed_user(&pool, event_id, "Fleißige Frieda").await;
for _ in 0..5 {
assert!(
create_upload(
&pool,
event_id,
user_id,
"originals/x.jpg",
Some(Uuid::new_v4())
)
.await
.is_some()
);
}
assert_eq!(upload_count(&pool).await, 5);
}
/// The index is PARTIAL, and this is why. Every upload that predates migration 022, and any
/// client that doesn't send a key, carries NULL — if those collided, the first such upload would
/// block every subsequent one and the whole event would fail after one photo.
#[sqlx::test]
async fn uploads_without_a_key_never_collide(pool: PgPool) {
let event_id = seed_event(&pool, "wedding").await;
let user_id = seed_user(&pool, event_id, "Alte Anna").await;
for _ in 0..5 {
assert!(
create_upload(&pool, event_id, user_id, "originals/legacy.jpg", None)
.await
.is_some(),
"a NULL key must never be treated as a duplicate"
);
}
assert_eq!(upload_count(&pool).await, 5);
}
/// The lookup is scoped to the owner. The key alone is unique, so this can only matter if a key
/// ever repeated across users — but a replay that handed one guest another guest's upload row
/// would be a data leak, so the scope is asserted rather than assumed.
#[sqlx::test]
async fn the_replay_lookup_never_crosses_users(pool: PgPool) {
let event_id = seed_event(&pool, "wedding").await;
let owner = seed_user(&pool, event_id, "Besitzerin Bea").await;
let other = seed_user(&pool, event_id, "Fremder Franz").await;
let key = Uuid::new_v4();
let id = create_upload(&pool, event_id, owner, "originals/a.jpg", Some(key)).await;
assert_eq!(find_by_key(&pool, owner, key).await, id);
assert_eq!(
find_by_key(&pool, other, key).await,
None,
"another guest's retry must not resolve to this upload"
);
}
/// A deleted photo must not be resurrected by a stale queue item. If the guest uploaded, deleted,
/// and their queue then retried the original request, replaying the deleted row would put the
/// photo they removed back in the gallery — so the lookup excludes soft-deleted rows and the
/// retry becomes a fresh upload instead.
#[sqlx::test]
async fn a_deleted_upload_is_not_replayed(pool: PgPool) {
let event_id = seed_event(&pool, "wedding").await;
let user_id = seed_user(&pool, event_id, "Reumütige Rita").await;
let key = Uuid::new_v4();
let id = create_upload(&pool, event_id, user_id, "originals/a.jpg", Some(key))
.await
.expect("first insert");
sqlx::query("UPDATE upload SET deleted_at = NOW() WHERE id = $1")
.bind(id)
.execute(&pool)
.await
.expect("soft delete");
assert_eq!(
find_by_key(&pool, user_id, key).await,
None,
"a soft-deleted upload must not be replayed"
);
}