fabi/EventSnap
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: add a restore procedure, fix the backup cadence, and correct quota_tolerance

Browse Source 
Four things, all found by the same question: what does an operator standing at the
venue actually need?

A RESTORE PROCEDURE. There was none anywhere, and a backup you have never restored
isn't a backup. Two hazards worth writing down: media must be extracted preserving
ownership (the app runs as uid 100 / gid 101, and a root-owned restore makes every
upload fail with EACCES surfacing as a generic 500), and the app must be STOPPED
first, because migrations run on boot and a live pool will fight the restore.

Both the backup and the restore commands were run against the real stack before being
written down, which caught two that would have failed:

  - The plain `pg_dump` did not restore: `psql` aborted on `ERROR: schema
    "_sqlx_test" already exists`. pg_dump emits no DROPs without --clean --if-exists,
    so the documented dump could only ever be restored into an empty database. Fixed
    at the source (the dump is now self-cleaning) and verified end to end: 16 tables
    back, exit 0.
  - `--same-owner` does not exist in BusyBox tar, which is what `alpine` ships, so
    the extract aborted before unpacking anything. `--numeric-owner` plus the
    explicit chown, verified to land 100:101.

BACKUP CADENCE. "Weekly offsite" is the wrong shape when every irreplaceable byte is
created in one eight-hour window and nobody can retake a wedding. The backup that
matters runs that night, and again after the release so the keepsake is captured.
Also: take the DB dump and the media tarball back to back, or you get rows pointing
at files the dump doesn't know about.

quota_tolerance WAS DOCUMENTED AS SOMETHING IT ISN'T. .env.example called it "fraction
of disk that triggers the low-storage warning". It is the multiplier in
`floor(free_disk * tolerance / active_uploaders)` -- so an operator who wants "warn me
later" and sets 0.95 is actually authorising guests to fill 95% of the disk, moving
the fixed point from 43% to ~49% and eating the export headroom. The admin UI labelled
it "Toleranz (0-1)" with no explanation at all, which invites exactly that reading;
it is now "Speicher-Anteil für Gäste" with the formula in the hint. Wrong docs on a
tuning knob are worse than no docs.

SIZING. New section with the arithmetic: three volumes on one filesystem, the quota
fixed point at tolerance/(1+tolerance), and the fact the 80 GB baseline does not cover
the keepsake -- both archives are built concurrently and each is roughly a second copy
of every original. Provision ~3x expected media, or give exports its own volume.

Also ticks the low-disk alert off the roadmap, since it now exists.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
fabi
2026-07-29 19:51:48 +02:00
parent e6e8a52d87
commit a4a4e46c53
3 changed files with 154 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

125
README.md
View File

@@ -34,7 +34,6 @@ A guest scans the QR code on their way in, types their name, and is immediately
### Planned (v1.x)
- Individual file download button
- Low-disk alert (< 10 GB free)
- Event banner / cover image
- Chunked resumable upload for large videos
- Host-curated story highlights
@@ -231,6 +230,45 @@ so a host takedown or a ban actually revokes access to the bytes.
---
## Sizing the disk
`postgres_data`, `media_data` and `exports_data` are all Docker named volumes under
`/var/lib/docker/volumes`, so **they share one filesystem**. Filling it does not
degrade one subsystem — Postgres stops being able to write and the whole event goes
down.
Uploads are self-limiting. `per_user_limit = free_disk × quota_tolerance ÷
active_uploaders` is recomputed against live free space on every upload, so guests
converge on a fixed point at `tolerance / (1 + tolerance)` of the free space you
started with — **43%** at the default 0.75. On an 80 GB box with ~70 GB free after
the OS and images, media settles at ~30 GB and stops.
**The keepsake is what the 80 GB baseline does not cover.** `Gallery.zip` and
`Memories.zip` are built concurrently and each is roughly a second copy of every
original: both write their media `Compression::Stored`, and `Memories.zip` streams the
untouched original for every video and for every image at or under 5 MB. So a release
wants room for **two more copies of the gallery** on top of the gallery itself.
| Stage | Used | Free (80 GB box) |
|---|---|---|
| Fresh box (OS + images) | ~10 GB | ~70 GB |
| Guests reach the quota fixed point | ~40 GB | ~40 GB |
| Host releases → both archives | ~100 GB | **ENOSPC** |
Two ways to size for it:
- **Provision ~3× your expected media** on one volume (media + two archives), or
- **give `exports_data` its own volume** so a full export cannot reach Postgres, and
size that one at ~2× expected media.
This is no longer silent. The export refuses up front with the two numbers rather than
hitting ENOSPC halfway through a multi-GB write, a rebuild reclaims the superseded
generation before it starts (so peak is one generation, not two), and the host
dashboard warns as soon as the keepsake would not fit — which is the only point at
which anyone can still do something about it.
---
## Backup
There are **three** things to back up, and they live in three different places.
@@ -242,9 +280,12 @@ never exported into an operator's shell — so every command below runs through
```bash
# 1. Database snapshot. Runs pg_dump inside the db container (the app image has no
# postgres client), reading credentials from the compose environment.
# --clean --if-exists makes the dump SELF-CLEANING: without it the restore below
# aborts on the first "already exists" against a database that has ever booted,
# which is every database you would actually want to restore over.
mkdir -p ./backups
docker compose exec -T db \
sh -c 'pg_dump -U "$POSTGRES_USER" "$POSTGRES_DB"' \
sh -c 'pg_dump --clean --if-exists -U "$POSTGRES_USER" "$POSTGRES_DB"' \
| gzip > ./backups/db_$(date +%Y-%m-%d).sql.gz
# 2. Uploaded media (originals + derivatives) out of the named volume.
@@ -261,7 +302,7 @@ docker run --rm \
-v eventsnap_exports_data:/src:ro -v "$PWD/backups":/backup \
alpine tar czf /backup/exports_$(date +%Y-%m-%d).tar.gz -C /src .
# Weekly offsite sync of the three artefacts above.
# Offsite sync of the three artefacts above.
rsync -az ./backups/ user@storagebox.example.com:backup/eventsnap/
```
@@ -274,6 +315,82 @@ from a directory called `eventsnap`. Confirm yours with `docker volume ls`.
> stops it being reachable except through the ticket-gated download handler.
> Backing up only the media volume therefore loses every generated keepsake.
### When to run it
**A nightly cron is the wrong shape for this app.** Every irreplaceable byte is
created inside one eight-hour window, and nobody can retake a wedding. Run the three
commands above:
1. **The night of the event**, once uploads have stopped. This is the backup that
matters; everything else is a formality.
2. **After the host releases the gallery**, so the generated keepsake is captured too.
3. Weekly thereafter, until the event is archived and torn down.
Take the DB dump and the media tarball **back to back**, without uploads in flight
between them. Upload rows reference files by path — a database from 22:00 and a media
volume from 23:00 gives you rows pointing at files the dump doesn't know about, and
rows whose files aren't in the tarball. Locking uploads from the host dashboard first
(**Uploads sperren**) makes the pair genuinely consistent.
---
## Restore
An untested backup is not a backup. Run this once against a scratch host **before**
the event — it is roughly ten minutes, and it is the only way to find out that your
tarball is empty or your dump is truncated while that is still a small problem.
```bash
# 0. Stop the app FIRST. Migrations run on boot and a live pool will fight the
# restore — a booting app against a half-restored schema can leave the migration
# table and the schema disagreeing, which is its own recovery problem.
# Leave `db` running: the dump is restored through it.
docker compose stop app caddy
# 1. Database. The dump carries its own DROPs (step 1 of Backup), so this replaces
# rather than collides. A dump taken WITHOUT --clean --if-exists will abort here
# on the first "already exists" — restore that one into a fresh empty database
# instead.
gunzip -c ./backups/db_2026-07-29.sql.gz \
| docker compose exec -T db \
sh -c 'psql -U "$POSTGRES_USER" -d "$POSTGRES_DB" --set ON_ERROR_STOP=1'
# 2. Media. NOTE the `--numeric-owner` and the chown: the app runs as a
# NON-ROOT user (uid 100, gid 101 — `addgroup -S app && adduser -S app`), and a
# restore that lands root-owned files makes every upload fail with EACCES deep in
# the write path, surfacing to the guest as a generic 500 with nothing in the UI
# to suggest permissions. The explicit chown is what guarantees it — BusyBox tar
# (which is what `alpine` ships) has no --same-owner, and restores ownership only
# because it runs as root here.
docker run --rm \
-v eventsnap_media_data:/dst -v "$PWD/backups":/backup:ro \
alpine sh -c 'tar xzf /backup/media_2026-07-29.tar.gz -C /dst \
--numeric-owner && chown -R 100:101 /dst'
# 3. Exports. Same volume-name caveat, same ownership rules.
docker run --rm \
-v eventsnap_exports_data:/dst -v "$PWD/backups":/backup:ro \
alpine sh -c 'tar xzf /backup/exports_2026-07-29.tar.gz -C /dst \
--numeric-owner && chown -R 100:101 /dst'
# 4. Back up. Migrations run, then export recovery re-arms any keepsake whose file
# didn't come back with the volume.
docker compose up -d app caddy
docker compose logs -f app # watch for "migrations applied"
# 5. Verify — all three, not just the first.
curl -fsS https://DOMAIN/health && echo # → ok
# … then sign in as host and confirm the feed renders images (proves the media
# volume restored AND is readable by uid 100), and that the keepsake downloads.
```
If the media volume restored but images 404 while the feed lists them, the paths are
there and the bytes aren't — check `docker compose exec app ls -ln /media/originals`
and confirm both the files and the `100:101` ownership.
The restore is deliberately **not** automated. It is rare, destructive, and the one
operation where a script that half-works is worse than a checklist someone reads.
---
## Running the backend test suite
@@ -348,7 +465,7 @@ Open:
- [ ] SSE delta-fetch on foreground reconnect (scaffolded in [sse.ts](frontend/src/lib/sse.ts), not wired)
- [ ] Live diashow / slideshow mode — see [docs/CONCEPT_DIASHOW.md](docs/CONCEPT_DIASHOW.md)
- [ ] Individual file download button per post
- [ ] Low-disk alert (< 10 GB free)
- [x] Low-disk alert — host dashboard warns below 10 GB free, or whenever the keepsake would not fit
- [ ] Event banner / cover image
- [ ] Chunked resumable upload for files > 100 MB
- [ ] Shared Tailwind config between main app and export-viewer