EventSnap/e2e/loadtest/README.md

EventSnap load / stress test

Simulates a real event: ~100 guests joining and uploading ~1000 images in bursts (10–20 at a time) spread across a compressed time window, a pool of viewers holding live SSE connections, and one real browser on /diashow acting as the showcase display.

Goal (this harness is tuned for it)

Validate the shipping config. We run at the real production defaults — compression concurrency (COMPRESSION_WORKER_CONCURRENCY, default 2), DB pool (default 10), quotas on — and answer: does the app survive the event, and how far behind real-time does the diashow fall?

The headline metric is pipeline latency: time from an upload succeeding to its preview being ready (upload-processed SSE event) — i.e. how long until the photo appears on the diashow. A backlog that builds is fine; a backlog that never drains is a fail for a live event.

Methodology: what we change vs. shipping

We only disable rate limits. They're per-IP / per-user anti-abuse guards; a synthetic test from one IP trips them in a way real guests (distinct IPs, phones) never would — leaving them on would measure the limiter, not the pipeline. Everything else (compression concurrency, DB pool, quotas) stays at the real default so the result is honest.

Standalone finding to remember: the shipping upload_rate_per_hour default is 10. A real guest uploading a burst of 10–20 photos would be throttled by the shipping config too. That's a genuine event-day issue worth surfacing separately from this pipeline test.

Prereqs

• The isolated test stack up: cd e2e && npm run stack:up (Caddy on :3101, EVENTSNAP_TEST_MODE=1, /admin/__truncate live).
• Node 24+ (global fetch/FormData/Blob), Python 3 + Pillow, Docker CLI access (used for docker stats + docker exec psql ground-truth sampling).
• @playwright/test (already an e2e dep) for the diashow watcher.

1. Generate the image pool (once)

Realistic phone-sized JPEGs (~2–4 MB, 12 MP, high entropy). The driver reuses this pool at random across all 1000 uploads — real load is byte size + decode cost, not file uniqueness.

python3 e2e/loadtest/gen-images.py 40      # → /tmp/eventsnap-loadtest/photos

~40 images ≈ 120 MB pool; projects to ~3–4 GB of originals for 1000 uploads (previews/thumbnails add more). The generator prints the projection; check disk.

2. Smoke run first (~1 min)

Proves the wiring — join, upload, SSE correlation, drain, metrics — before the real thing:

LT_GUESTS=5 LT_IMAGES=50 LT_WINDOW_SEC=60 node e2e/loadtest/driver.mjs

3. Full run (~15 min + drain)

Two terminals. Start the showcase display first, then the driver:

# terminal A — the showcase device
node e2e/loadtest/diashow-watch.mjs

# terminal B — 100 guests / 1000 images / 15-min window (defaults)
node e2e/loadtest/driver.mjs

The driver truncates event data first (LT_TRUNCATE=0 to keep), disables rate limits, joins guests, opens SSE, runs the burst schedule, then waits for the compression backlog to drain before reporting.

Output

• Console: live progress every 10 s, then a RESULTS block with pass/fail flags.
• e2e/loadtest/results/run-<timestamp>.json: full metrics — upload latency percentiles, pipeline latency percentiles, drain time, per-status counts, SSE reconnect/resync counts, and a docker stats + DB-connection time series.
• e2e/loadtest/results/diashow/: periodic screenshots of the live display.

What the flags mean

Flag	Meaning
✗ 5xx	server errored under load — hard fail
✗ 507	quota rejected uploads — disk/quota misconfig for the event size
✗ backlog did not drain	compression can't keep up even after uploads stop — diashow never catches up
⚠ pipeline p95 > 60s	photos take >1 min to appear on the diashow at peak
⚠ SSE resyncs	live consumers lagged the broadcast channel

Knobs

All via env (see header of driver.mjs): LT_GUESTS, LT_IMAGES, LT_WINDOW_SEC, LT_BURST_MIN/MAX, LT_BURST_CONC, LT_VIEWERS, LT_TRUNCATE, LT_DRAIN_TIMEOUT_SEC, LT_KEEP_RATELIMITS, LT_BASE, LT_APP_CONTAINER, LT_DB_CONTAINER.

To later answer "what config should I deploy?", re-run with a rebuilt stack that sets COMPRESSION_WORKER_CONCURRENCY higher (boot-time env var in docker-compose.test.yml) and compare the pipeline-latency / drain numbers.

Teardown

cd e2e && npm run stack:down     # wipes volumes (media + db)