diff --git a/PROJECT.md b/PROJECT.md index 7095f83..034a906 100644 --- a/PROJECT.md +++ b/PROJECT.md @@ -1133,16 +1133,19 @@ eventsnap/ ### Backup Strategy -```bash -# Daily (e.g. as a separate Compose service or cron on the VPS) -pg_dump $DATABASE_URL | gzip > /media/backups/db_$(date +%Y-%m-%d).sql.gz +Three artefacts in three places: the database, the `media_data` volume +(originals + derivatives), and the **separate** `exports_data` volume. See +[README.md](README.md#backup) for the exact commands. -# Weekly: rsync /media volume to Hetzner Storage Box -rsync -az /opt/eventsnap/media/ \ - user@u123456.your-storagebox.de:backup/eventsnap/ -``` +Everything runs through `docker compose` / `docker run`, because `DATABASE_URL` +and the `/media` and `/exports` paths only exist inside the compose network — +they are not host paths, and `DATABASE_URL` is never exported into an operator's +shell. -The `/media` volume contains originals, previews, thumbnails, generated exports, and DB backups — a single volume to back up. +Export archives are deliberately outside `MEDIA_PATH` (`EXPORT_PATH=/exports`): a +keepsake contains every photo in the event, and keeping it off the media tree is +what stops it being reachable except through the ticket-gated handler. A backup +of the media volume alone silently loses every generated keepsake. --- diff --git a/README.md b/README.md index f4f1751..73d880c 100644 --- a/README.md +++ b/README.md @@ -162,23 +162,58 @@ See [.env.example](.env.example) for the full list with descriptions and default └────────┘ ``` -- `/api/*` and `/media/*` → Rust backend +- `/api/*` → Rust backend - Everything else → SvelteKit frontend (`adapter-node`) -- Named volumes: `postgres_data`, `media_data`, `caddy_data` +- Named volumes: `postgres_data`, `media_data`, `exports_data`, `caddy_data` + +Media is **not** served as static files. Every image goes through a +visibility-checked alias (`/api/v1/upload/{id}/{preview,display,thumbnail,original}`) +so a host takedown or a ban actually revokes access to the bytes. --- ## Backup -```bash -# Database snapshot -pg_dump $DATABASE_URL | gzip > /media/backups/db_$(date +%Y-%m-%d).sql.gz +There are **three** things to back up, and they live in three different places. +`DATABASE_URL` and the container paths (`/media`, `/exports`) are meaningful only +*inside* the compose network — they are not host paths, and `DATABASE_URL` is +never exported into an operator's shell — so every command below runs through +`docker compose` from the repo directory. -# Weekly offsite sync (Hetzner Storage Box or similar) -rsync -az /opt/eventsnap/media/ user@storagebox.example.com:backup/eventsnap/ +```bash +# 1. Database snapshot. Runs pg_dump inside the db container (the app image has no +# postgres client), reading credentials from the compose environment. +mkdir -p ./backups +docker compose exec -T db \ + sh -c 'pg_dump -U "$POSTGRES_USER" "$POSTGRES_DB"' \ + | gzip > ./backups/db_$(date +%Y-%m-%d).sql.gz + +# 2. Uploaded media (originals + derivatives) out of the named volume. +# NOTE the mountpoint is /src, not /media: if the volume is ever empty, Docker +# pre-populates a fresh mount from the image's own directory, and alpine ships a +# /media containing cdrom/floppy/usb. Mounting somewhere the image has nothing +# avoids silently tarring (and polluting the volume with) those. +docker run --rm \ + -v eventsnap_media_data:/src:ro -v "$PWD/backups":/backup \ + alpine tar czf /backup/media_$(date +%Y-%m-%d).tar.gz -C /src . + +# 3. Export archives — a SEPARATE volume (see the security note below). +docker run --rm \ + -v eventsnap_exports_data:/src:ro -v "$PWD/backups":/backup \ + alpine tar czf /backup/exports_$(date +%Y-%m-%d).tar.gz -C /src . + +# Weekly offsite sync of the three artefacts above. +rsync -az ./backups/ user@storagebox.example.com:backup/eventsnap/ ``` -The `/media` volume holds originals, previews, thumbnails, exports, and DB backups — a single path to back up. +Volume names are prefixed with the compose project name — `eventsnap_` if you run +from a directory called `eventsnap`. Confirm yours with `docker volume ls`. + +> **Exports are deliberately NOT under `/media`.** They live on their own +> `exports_data` volume (`EXPORT_PATH=/exports`) because a keepsake archive +> contains every photo in the event; keeping it outside the media tree is what +> stops it being reachable except through the ticket-gated download handler. +> Backing up only the media volume therefore loses every generated keepsake. --- diff --git a/e2e/docker-compose.test.yml b/e2e/docker-compose.test.yml index adf77d5..15a0440 100644 --- a/e2e/docker-compose.test.yml +++ b/e2e/docker-compose.test.yml @@ -42,11 +42,20 @@ services: EVENT_NAME: E2E Test Event APP_PORT: '3000' MEDIA_PATH: /media + # Exports MUST live outside MEDIA_PATH — see the note on the volume below and + # config.rs::validate. Omitting this left exports on the container's writable + # layer at the /exports default, so the test stack diverged from the prod layout + # it claims to mirror, and export-leak/export-video wrote real archives into + # ephemeral storage. + EXPORT_PATH: /exports SESSION_EXPIRY_DAYS: '30' EVENTSNAP_TEST_MODE: '1' # ENABLES /admin/__truncate — never set in prod RUST_LOG: eventsnap_backend=info,tower_http=warn volumes: - media_data:/media + # Separate volume, exactly as in production: a keepsake archive contains every + # photo in the event, so it is kept off the media tree. + - exports_data:/exports expose: - '3000' @@ -75,3 +84,4 @@ services: volumes: media_data: + exports_data: diff --git a/e2e/specs/02-upload/quota.spec.ts b/e2e/specs/02-upload/quota.spec.ts index f054d7b..34126eb 100644 --- a/e2e/specs/02-upload/quota.spec.ts +++ b/e2e/specs/02-upload/quota.spec.ts @@ -56,14 +56,21 @@ function upload(jwt: string, name: string) { /** * Pick a `quota_tolerance` that makes the per-user ceiling land on `targetBytes`. * limit = floor(free_disk * tolerance / max(active, 1)) ⇒ tolerance = target * active / free. + * + * `staffJwt` reads the calibration inputs, `jwt` is the guest the limit is being aimed at. + * They must be different tokens: `free_disk_bytes` and `active_uploaders` are raw server + * telemetry and `/me/quota` zeroes both for non-staff (handlers/me.rs — "must never reach a + * guest"). Calibrating off the guest's own response divides by zero and yields a NaN + * tolerance, which is what silently broke this whole describe block. */ async function setLimitTo( api: any, adminToken: string, + staffJwt: string, jwt: string, targetBytes: number ): Promise { - const q = await quotaOf(jwt); + const q = await quotaOf(staffJwt); expect( q.free_disk_bytes, 'the disk must be readable, else quota fails OPEN and proves nothing' @@ -72,6 +79,7 @@ async function setLimitTo( const tolerance = (targetBytes * active) / (q.free_disk_bytes as number); await api.patchConfig(adminToken, { quota_tolerance: tolerance.toExponential(12) }); + // Read back through the GUEST, whose ceiling is the one under test. const after = await quotaOf(jwt); expect(after.enabled).toBe(true); return after.limit_bytes as number; @@ -89,10 +97,11 @@ test.describe('Upload — storage quota enforcement', () => { api, adminToken, guest, + host, }) => { const g = await guest('QuotaOver'); // Ceiling below one file: the very first upload must be refused. - const limit = await setLimitTo(api, adminToken, g.jwt, Math.floor(SIZE / 2)); + const limit = await setLimitTo(api, adminToken, host.jwt, g.jwt, Math.floor(SIZE / 2)); expect(limit).toBeLessThan(SIZE); const res = await upload(g.jwt, 'too-big.jpg'); @@ -111,9 +120,10 @@ test.describe('Upload — storage quota enforcement', () => { api, adminToken, guest, + host, }) => { const g = await guest('QuotaUnder'); - await setLimitTo(api, adminToken, g.jwt, SIZE * 4); + await setLimitTo(api, adminToken, host.jwt, g.jwt, SIZE * 4); expect((await upload(g.jwt, 'fine.jpg')).status).toBe(201); expect((await quotaOf(g.jwt)).used_bytes).toBe(SIZE); @@ -123,11 +133,12 @@ test.describe('Upload — storage quota enforcement', () => { api, adminToken, guest, + host, }) => { const g = await guest('QuotaRacer'); // Room for exactly ONE file. - const limit = await setLimitTo(api, adminToken, g.jwt, Math.floor(SIZE * 1.5)); + const limit = await setLimitTo(api, adminToken, host.jwt, g.jwt, Math.floor(SIZE * 1.5)); expect(limit).toBeGreaterThanOrEqual(SIZE); expect(limit).toBeLessThan(SIZE * 2); @@ -170,10 +181,11 @@ test.describe('Upload — storage quota enforcement', () => { api, adminToken, guest, + host, }) => { // Zero test hits before this — and it is the source of the "X von Y MB genutzt" widget. const g = await guest('QuotaWidget'); - await setLimitTo(api, adminToken, g.jwt, SIZE * 10); + await setLimitTo(api, adminToken, host.jwt, g.jwt, SIZE * 10); const before = await quotaOf(g.jwt); expect(before.enabled).toBe(true);